Getting Data from a Remote Server to the Command Line in C (Libcurl)

LibCurl is the free C library that underpins the curl command. This tool can be used for transferring data from a remote server to the command line, as shown in this note. Curl accepts the --libcurl option (appended after the url) and a file name for the self-generated corresponding C program.

For example, for the following network request:

   $ curl -sf  https://tessarinseve.pythonanywhere.com/staticweb/empty.csv --libcurl -

the generated C code is printed to the standard output, as shown in the video below.

The code can be expended to support arbitrary URLs as follows:

    /********* Sample code generated by the curl command line tool **********
     * All curl_easy_setopt() options are documented at:
     * https://curl.se/libcurl/c/curl_easy_setopt.html
     ************************************************************************/
    /* filename: get_csv_data.c */
    #include <curl/curl.h>
    #include <stdio.h>
    #include <stdlib.h>
    #include <unistd.h>
    #include <curl/curl.h>
    #include <string.h>
    #include <sys/types.h>

    int main(int argc, char *argv[])
    {
      CURLcode ret;
      CURL *hnd;

      if(argc < 2) {
        printf("Usage: %s <URL>\n", argv[0]);
        return 1;
      }

      hnd = curl_easy_init();
      curl_easy_setopt(hnd, CURLOPT_BUFFERSIZE, 102400L);
      curl_easy_setopt(hnd, CURLOPT_URL, argv[1]);
      curl_easy_setopt(hnd, CURLOPT_NOPROGRESS, 1L);
      curl_easy_setopt(hnd, CURLOPT_FAILONERROR, 1L);
      curl_easy_setopt(hnd, CURLOPT_USERAGENT, "curl/7.77.0");
      curl_easy_setopt(hnd, CURLOPT_MAXREDIRS, 50L);
      curl_easy_setopt(hnd, CURLOPT_HTTP_VERSION, (long)CURL_HTTP_VERSION_2TLS);
      curl_easy_setopt(hnd, CURLOPT_SSH_KNOWNHOSTS, "C:\\Users\\Seve/.ssh/known_hosts");
      curl_easy_setopt(hnd, CURLOPT_FTP_SKIP_PASV_IP, 1L);
      curl_easy_setopt(hnd, CURLOPT_TCP_KEEPALIVE, 1L);

      ret = curl_easy_perform(hnd);

      curl_easy_cleanup(hnd);
      hnd = NULL;

      return (int)ret;
    }
    /**** End of sample code ****/

Finally, it can be compiled with the GCC compiler (Mingw-w64) by linking the libcurl library:

   gcc -o get_csv_data.exe get_csv_data.c -lcurl

In the above video, this program fetches a remote CSV file and then streams it to the terminal standard output.