Using HTTP in Modern C++

In this lesson, and the next few lessons, we’ll focus on letting our programs communicate with other machines, over the internet or other networks. This unlocks many new possibilities for us. For example:

• Programs that load up-to-date data from the internet
• Multiple-user experiences, such as multiplayer games
• Live services, where we can do things like store data and save states to "the cloud" (ie, online servers we control). This makes them accessible from other machines, allowing users to collaborate

There are many different ways we can use to accomplish this - the options we have are often referred to as networking protocols. Complex applications often use multiple protocols, depending on their use case. Useful protocols broadly fall into two categories:

• TCP-based protocols, which are reliable but slow
• UDP-based protocols, which are fast but unreliable

HTTP (hypertext transfer protocol) is a common choice within the TCP category. Later in this chapter, we’ll look at protocols in the second category, which we rely on when making fast-paced, real-time applications such as multiplayer games.

We’re likely most familiar with HTTP, and its secure, encrypted extension HTTPS, in the context of browsing the web. It’s how our browsers request and receive web pages, images, and other data. We’ve likely seen the https:// prefix to URLs in our address bar.

But HTTP has become ubiquitous in other contexts, too. Countless organizations are offering their data and products via HTTP, often for free. Additionally, if we ever need to build our own HTTP services to support our products, there are hundreds of tools that make that quick and easy.

Using HTTP in C++ with cpr

Many C++ libraries allow us to make use of HTTP within our programs. In this lesson, we’ll use cpr

CPR can be installed through our package manager. In a previous lesson, we walked through the installation of vcpkg:

Installing vcpkg on Windows

An introduction to C++ package managers, and a step-by-step guide to installing vcpkg on Windows and Visual Studio.

If we were using vcpkg as our package manager, we would use it to install cpr from our terminal in the same way we install any other vcpkg package:

1.\vcpkg install cpr

Regardless of the installation process we use, we should ensure cpr is set up correctly in our project before proceeding.

We can do that by ensuring the preprocessor can find the header files, and then creating an object with a type like cpr::Url to ensure the linker can find the library:

1#include <cpr/cpr.h>
2
3int main() {
4  cpr::Url Test;
5}

If this compiles, we should be all set!

HTTP APIs

There are thousands of public HTTP APIs we can use. A list of free options is available here: https://github.com/public-apis/public-apis

For our examples in this lesson, we’ll use the Dungeons and Dragons API, available here: http://www.dnd5eapi.co/

Feel free to experiment with any but, to follow along with the initial examples, use one that doesn’t require authentication. We’ll cover authentication a little later in this lesson.

The vast majority of these APIs send and receive data in JSON format. So, we’ll be relying on the nlohmann::json library to help us with this. We have a dedicated lesson on JSON, and this library, available here:

Using JSON in Modern C++

A practical guide to working with the JSON data format in C++ using the popular nlohmann::json library.

Sending an HTTP Request

The common terminology for HTTP traffic is request and response. A computer on the network sends a request to a network location, and a computer at that location sends a response back. The requestor is sometimes called the client, whilst the responder is the server. In this lesson, we focus on being the client - ie, sending requests that some other computer is responding to.

To make an HTTP request using cpr, we need the API URL to which we’ll be sending the request. URLs are stored in the cpr::Url type, which we can create by passing a simple string to the constructor:

1#include <cpr/cpr.h>
2#include <iostream>
3
4int main() {
5  cpr::Url URL{
6    "http://www.dnd5eapi.co/api/monsters/"
7    "giant-spider"
8  };
9}

1std::string Greeting{"Hell"    "o"};

1cpr::Url SomeUrl{
2  "http://www.some-website.com/"
3  "some-directory/within-the-site"
4};

To use this URL to make an HTTP request, we can pass it to the cpr::Get function. This returns a cpr::Response object, containing the server’s response:

1#include <cpr/cpr.h>
2#include <iostream>
3
4int main(){
5  cpr::Url URL{
6    "http://www.dnd5eapi.co/api/"
7    "monsters/giant-spider"};
8
9  cpr::Response Response{cpr::Get(URL)}; 
10
11  std::cout << Response.text;
12}

A cpr::Response includes a text field, containing the body of the response. The previous code outputs a large blob of JSON, with 30+ fields covering abilities, stats, equipment, resistances, and more.

We’ve formatted and shown a small sample of the response below:

1{
2  "name": "Giant Spider",
3  "desc": "To snare its prey, a giant spider...",
4  "type": "beast",
5  "hit_points": 26,
6  "xp": 200,
7  "special_abilities": [
8    ...
9  ],
10  ...
11}

To know what responses we can expect from an API, we can check the documentation for the specific service we’re using. Monster records of the D&D5 API include fields like name and desc.

We can use our JSON library to parse the data and convert these fields to regular C++ types which we can work with normally. Below, we grab the name and description of the monster as std::string objects:

1#include <cpr/cpr.h>
2#include <iostream>
3#include <nlohmann/json.hpp>
4
5using json = nlohmann::json;
6
7int main() {
8  cpr::Url URL{
9    "http://www.dnd5eapi.co/api/"
10    "monsters/giant-spider"
11  };
12
13  cpr::Response Res{cpr::Get(URL)};
14
15  json Doc{json::parse(Res.text)};
16
17  std::cout
18      << Doc.at("name").get<std::string>()
19      << '\n'
20      << Doc.at("desc").get<std::string>();
21}

1Giant Spider
2To snare its prey, a giant spider spins
3elaborate webs or shoots sticky strands of
4webbing from its abdomen. Giant spiders are 
5most commonly found underground, making their
6lairs on ceilings or in dark, web-filled
7crevices. Such lairs are often festooned 
8with web cocoons holding past victims.

Response Headers

An HTTP response has two different components:

• A body, which contains the main content of the response. This is available through the text field of a cpr::Response, as we saw above.
• A collection of headers, which provide supplemental metadata. This is available through the header field of a cpr::Response

Headers are a collection of key-value pairs, containing information like the date the response was generated, how long it is, the type of data that is in the body, and more.

cpr implements each header as a std::pair where the key is first and the value is second.

Using std::pair

Master the use of std::pair with this comprehensive guide, encompassing everything from simple pair manipulation to template-based applications

These pairs are then gathered together into an associative container with an API similar to a std::map. This container is accessible through the header variable on a cpr::Response object.

We can access individual headers using the [] operator, or iterate through the headers using techniques such as a range-based for loop:

1#include <cpr/cpr.h>
2#include <iostream>
3
4int main() {
5  cpr::Url URL{
6      "http://www.dnd5eapi.co/api/monsters/"
7      "giant-spider"};
8
9  cpr::Response Response = cpr::Get(URL);
10
11  auto t = Response.header["date"];
12
13  std::cout << "The date header is "
14            << Response.header["date"];
15
16  std::cout << "\n\nAll Headers:\n";
17
18  for (auto& Header : Response.header) {
19    std::cout << Header.first << ": "
20              << Header.second << '\n';
21  }
22}

1The date header is Tue, 27 Jun 2023 20:11:10 GMT
2
3All Headers:
4Access-Control-Allow-Origin: *
5Connection: keep-alive
6Content-Length: 2558
7Content-Type: application/json; charset=utf-8
8Date: Tue, 27 Jun 2023 20:11:10 GMT
9Etag: W/"9fe-cSLNtEpeu+jMBqWZ25Erl2gQZz8"
10Server: Cowboy
11Via: 1.1 vegur
12X-Powered-By: Express
13X-Ratelimit-Limit: 10000
14X-Ratelimit-Remaining: 9999
15X-Ratelimit-Reset: 1687896672

The HTTP specification includes some standard header names, like Date and Content-Type which servers can choose to include. Servers can additionally include their own, custom headers, that might be useful to consumers of their specific service.

By convention, these custom headers start with X-. In the above example, we see the API is returning some rate-limiting information as custom headers.

These headers are telling us we’re limited to 10,000 requests, we have 9,999 remaining, and our allocation will be reset at 1687896672, which is a date and time represented as a Unix timestamp

HTTP Status Codes

So far, we’ve been assuming our HTTP request works as expected. That is not always the case. Networking is never 100% reliable.

HTTP responses include a status code, which summarises the result of the request.

The numeric status code is available through the status_code property of cpr::Response objects. A string is available as status_line which provides some additional info:

1#include <cpr/cpr.h>
2#include <iostream>
3
4int main() {
5  cpr::Url URL1{
6      "http://www.dnd5eapi.co/api/monsters/"
7      "giant-spider"};
8
9  cpr::Response Response1{cpr::Get(URL1)};
10
11  std::cout << "Response1 Status Code: "
12            << Response1.status_code;
13  std::cout << "\nResponse1 Status Line: "
14            << Response1.status_line;
15
16  cpr::Url URL2{
17      "http://www.dnd5eapi.co/api/monsters/"
18      "does-not-exist"};
19
20  cpr::Response Response2{cpr::Get(URL2)};
21
22  std::cout << "\n\nResponse2 Status Code: "
23            << Response2.status_code;
24  std::cout << "\nResponse2 Status Line: "
25            << Response2.status_line;
26}

1Response1 Status Code: 200
2Response1 Status Line: HTTP/1.1 200 OK
3
4Response2 Status Code: 404
5Response2 Status Line: HTTP/1.1 404 Not Found

A list of all possible status codes is available on this Wikipedia page. We’ve summarised the most common response codes below, and how they apply to our use cases

200-299 range: success

Successful responses trigger 2xx status codes. The most common is 200

300-399 range: redirection

When service providers move their resources to a different URL, it’s common to leave a breadcrumb at the old location, to tell consumers it has moved. When this happens, the status code should be in the 3xx range.

The new location is provided as a header within the HTTP response. The header will have the key of location

By default, when cpr encounters a redirection response, it will automatically "follow" the redirect. That is, it will make a new request to the URL specified in the location header.

So, unless we change that behavior, we won’t see a 3xx response from cpr. The official docs show how we can modify the redirect behavior.

400-499 range: client errors

Errors in the 4xx range often indicate there are problems with the request we made. These are commonly called "client errors", as they’re things that we as the client can fix. The common status codes we’ll encounter in this category include:

• 401 - the action we’re trying to perform requires us to authenticate ourselves. We cover authentication a little later in this section
• 403 - we do not have permission to do what we’re trying to do
• 404 - the URL we’re making a request to cannot be found
• 422 - the contents of our HTTP request are not valid. This is most common when we’re trying to create or edit a record on the server, which we’ll cover later in this section
• 429 - we’ve made too many requests to the API. APIs generally limit the number of requests users can make. This can be part of the API’s business model - eg, the service is free for light use, but heavy users may need to pay

500-599 range: server errors

Errors in the 5xx range generally indicate something has gone wrong on the service provider's side. The API may be down or have some defect that our request is triggering.

Request Parameters

Aside from the URL, our HTTP requests will often need to include additional data. The providers of the API will have documented what data we need to provide, in what situation, and in what way.

There are three ways we can provide additional data as part of an HTTP request:

• URL parameters
• The request headers
• The request body

We’ll cover all of them here, starting with URL parameters.

URL parameters are generally the most visible of these, as they show up in the URL. As such, we may have already noticed them in our browser’s address bar as we navigate the internet.

For example, a Google search for C++ generates the following URL:

1https://www.google.com/search?q=C++

This URL has a parameter called q, with a value of C++

The part of the URL that contains the parameters is sometimes called the query string. It begins with a ? and is then followed by a collection of key-value pairs. Keys and values are separated by =, whilst parameters are separated by &.

If we wanted to provide the following three parameters:

1cat: meow
2dog: bark
3cow: moo

Our query string would look like this:

1?cat=meow&dog=bark&cow=moo

We could attempt to build these strings and add them to our URL. However, most HTTP libraries provide a more convenient abstraction. In cpr, we have the cpr::Parameters type that stores a collection of key-value pairs. We can create them like this:

1cpr::Parameters Params{{"cat", "meow"},
2                       {"dog", "bark"}};

Once created, we can just pass our parameters into our Get calls as an argument:

1#include <cpr/cpr.h>
2#include <iostream>
3
4int main() {
5  cpr::Url URL{"http://www.example.com"};
6  cpr::Parameters Params{{"cat", "meow"},   
7                         {"dog", "bark"}};  
8
9  cpr::Response Response{cpr::Get(URL, Params)};  
10
11  // The url property contains the URL
12  // that was used for the request
13  std::cout << "URL: " << Response.url;
14}

1URL: http://www.example.com/?cat=meow&dog=bark

1#include <cpr/cpr.h>
2#include <iostream>
3
4int main() {
5  cpr::Url URL{"http://www.example.com"};
6  cpr::Parameters Params{{"cat", "meow"}};
7
8  cpr::Response Res1{cpr::Get(URL, Params)};
9  cpr::Response Res2{cpr::Get(Params, URL)};
10
11  if (Res1.url == Res2.url) {
12    std::cout << "Equivalent!";
13  }
14}

1Equivalent!

Request Headers

Just like responses can contain headers, so too can our requests. What headers (if any) we need will depend on the API we’re using. For example, some APIs require us to pass authentication information, or API keys in the headers of our requests.

In cpr, we can create a collection of headers using the cpr::Header type, and pass them into our cpr::Get request as an argument.

We can use a simple utility API at http://httpbin.org/headers that will echo back the HTTP headers we used, as a JSON response.

In our code, it looks like this:

1#include <cpr/cpr.h>
2#include <iostream>
3
4int main() {
5  cpr::Url URL{
6      "http://www.httpbin.org/headers"};
7  cpr::Header Headers{{"cat", "meow"},
8                      {"dog", "bark"}};
9
10  cpr::Response Response{
11      cpr::Get(URL, Headers)};
12
13  std::cout << Response.text;
14}

We can see our headers were added to the request. Our machine, and intermediate servers our request passes through, may also add some additional headers:

1{
2  "headers": {
3    "Accept": "*/*",
4    "Cat": "meow",
5    "Dog": "bark",
6    "Host": "www.httpbin.org",
7    "User-Agent": "curl/8.1.2-DEV",
8    "X-Amzn-Trace-Id": "redacted"
9  }
10}

HTTP Methods and Request Bodies

So far, our HTTP requests have been attempting to retrieve data from the server. These are called GET requests, which we’ve been calling with cpr::Get. "GET" is an example of an HTTP method, and we have many more we can use.

As always, it’s up to the service provider to decide which HTTP methods they support on each URL, as well as how those methods work, and who can use them. But, in general, the following conventions tend to be followed:

• GET (cpr::Get) requests are for retrieving records
• POST (cpr::Post) requests are for creating records
• DELETE (cpr::Delete) requests are for deleting records
• PATCH (cpr::Patch) requests are for editing properties within records
• PUT (cpr::Put) requests are for replacing entire records

With many of these methods, it’s generally anticipated that we provide some data in the body of our request. For example, if we want to create a record, we need to provide the fields for that record.

With cpr, we can create a body using the cpr::Body type, and include it as an argument when making our request with a function like cpr::Post:

1#include <cpr/cpr.h>
2#include <iostream>
3
4int main() {
5  cpr::Url URL{"https://www.example.com/post"};
6  cpr::Body Body{"Hello World"};
7  cpr::Response Response{cpr::Post(URL, Body)};
8}

We can send any type of data over HTTP, such as JSON, images, or videos. Therefore, when providing a body, it’s also generally recommended to provide a Content-Type header, describing what the content in the body represents.

Content types have a category, like "image" and a subtype, like "jpeg". The type and subtype are separated by a /. Some examples include:

• image/jpeg
• text/plain
• video/mp4
• application/json

When communicating with JSON APIs, the content type is generally going to be application/json. The https://www.httpbin.org/post API gives us a quick way to check the contents of our request body

1#include <cpr/cpr.h>
2#include <iostream>
3#include <nlohmann/json.hpp>
4
5using json = nlohmann::json;
6
7int main() {
8  using namespace nlohmann::literals;
9
10  json Doc{R"({
11    "name": "Bob",
12    "age": 30
13  })"_json};
14
15  cpr::Url URL{"https://www.httpbin.org/post"};
16  cpr::Header Headers{
17      {"Content-Type", "application/json"}};
18  cpr::Body Body{Doc.dump()};
19
20  cpr::Response Res =
21      cpr::Post(URL, Headers, Body);
22
23  std::cout << Res.text;
24}

1{
2  "args": {},
3  "data": "{\"name\":\"Bob\",\"age\":30}",
4  "files": {},
5  "form": {},
6  "headers": {
7    "Accept": "*/*",
8    "Content-Length": "52",
9    "Content-Type": "application/json",
10    "Host": "www.httpbin.org",
11    "User-Agent": "curl/8.1.2-DEV",
12    "X-Amzn-Trace-Id": "redacted"
13  },
14  "json": {
15    "name": "Bob",
16    "password": "secret"
17  },
18  "origin": "redacted",
19  "url": "<https://www.httpbin.org/post>"
20}

Authentication

Often, when we’re making HTTP calls, we need to provide authentication details. With APIs, the service will often provide us with a secret API key, which we need to attach to all of our HTTP requests.

This is typically done as a request header or request parameter, depending on the API. We covered those earlier in the lesson so, if this is how the API you’re using chooses to authenticate, we should be all set.

However, HTTP also has its own, native form of authentication, managed through a special Authentication header.

There are three common ways of doing HTTP authentication:

• Basic Authentication
• Digest Authentication
• Bearer Tokens

The method the server requires us to use will determine the value we pass in that authentication header.

Fortunately, cpr can help us manage the complexity here. We just need to pass an appropriate cpr::Authentication object along as an argument.

Basic Authentication

Basic HTTP authentication uses a simple username and password combination, which is placed into the Authentication header

Below, we use the http://www.httpbin.org/basic-auth/ API to simulate a URL that requires basic authentication.

We include the username and password we want to use in the URL - for example, opening http://www.httpbin.org/basic-auth/bob/secret in a web browser will prompt us for a username and password. The correct username will be bob, and the correct password will be secret.

When accessing the URL programmatically, we need to construct an appropriate Authentication header that contains the username and password, which cpr can construct for us:

1#include <cpr/cpr.h>
2#include <iostream>
3
4int main() {
5  cpr::Url URL{
6      "http://www.httpbin.org/basic-auth/"
7      "bob/secret"};
8
9  cpr::Authentication Auth{
10      "bob", "secret", cpr::AuthMode::BASIC};
11
12  cpr::Response Response{cpr::Get(URL, Auth)};
13
14  std::cout << Response.text;
15}

1{
2  "authenticated": true,
3  "user": "bob"
4}

Digest Authentication

Digest authentication is similar to basic authentication in the sense that it is based on a username and password. However, the credentials are inserted into the Authentication header in an encrypted form.

Ensuring the credentials are encrypted in a way the server can decrypt is quite a complex process, involving additional preliminary communication between our client and the server before we even send our main request.

However, cpr manages all that complexity for us. We just need to create a cpr::Authentication object containing our username, password, and the cpr::AuthMode::DIGEST flag.

Similar to the previous example, httpbin.org provides a service where we can test our digest authentication. http://www.httpbin.org/digest-auth/auth/bob/secret will ask us to provide a username of bob, and a password of secret.

It will use the digest authentication process to receive and validate our credentials.

1#include <cpr/cpr.h>
2#include <iostream>
3
4int main() {
5  cpr::Url URL{
6      "http://www.httpbin.org/digest-auth/auth/"
7      "bob/secret"};
8
9  cpr::Authentication Auth{
10      "bob", "secret", cpr::AuthMode::DIGEST};
11
12  cpr::Response Response{cpr::Get(URL, Auth)};
13
14  std::cout << Response.text;
15}

1{
2  "authenticated": true,
3  "user": "bob"
4}

Bearer Token

The final form of authentication commonly used involves a token rather than a username and password. Services that use this process will provide us with a token - a long string that we keep secret.

We include the token in the header of our requests and, once the server checks the header and sees our token, it will authenticate the request as having been sent by us, the "bearer" of the token.

We pass the token by setting the Authentication header to the value of Bearer followed by a space, and then the token:

1cpr::Header Headers{{"Authentication",
2                     "Bearer my-secret-token"}};

Alternatively, we can ensure this header is included in the correct format by creating a cpr::Bearer object with our token and passing it as an argument to our function call.

The http://www.httpbin.org/bearer API lets us make sure we’re passing the token correctly:

1#include <cpr/cpr.h>
2#include <iostream>
3
4int main() {
5  cpr::Url URL{"http://www.httpbin.org/bearer"};
6  cpr::Bearer Token{"my-secret-token"};
7  cpr::Response Response{cpr::Get(URL, Token)};
8  std::cout << Response.text;
9}

1{
2  "authenticated": true,
3  "token": "my-secret-token"
4}

Asynchronous Requests and Callbacks

The previous code snippets all create what is known as synchronous requests. These have the benefit of being quite simple to use and understand. In essence, it allows us to treat our HTTP communication as a regular function call. We make the request and, on the next line, the response is available to us.

However, there is a problem here - HTTP requests, and network traffic in general, take a relatively long time to complete.

With a synchronous request, our thread is effectively frozen until that process is complete. For this reason, synchronous calls are sometimes also referred to as blocking calls.

A typical HTTP call can take in the region of 50-500 milliseconds to complete. In real-time applications, we can’t be blocked for that long - we need to be responding to user input almost immediately, and updating the screen every 5-30 milliseconds.

For this reason, we have asynchronous requests. Here, we send the request, but we do not wait for it to complete - we keep running the application as normal.

Later, when the request is complete, a function is called to let us react accordingly. This type of function is often referred to as a callback.

In cpr, asynchronous requests that use a callback have names like cpr::GetCallback, cpr::PostCallback, and so on.

Their first argument is a callback function, that will receive the cpr::Response as a parameter. We can choose to receive it as a copy, or as a const reference.

The remaining arguments match the synchronous versions of the respective functions, allowing us to pass URLs, headers, etc in the normal way:

1#include <cpr/cpr.h>
2#include <iostream>
3
4int main() {
5  cpr::Url URL{
6      "http://www.dnd5eapi.co/api/monsters/"
7      "giant-spider"};
8
9  auto callback{[](const cpr::Response& Res) {
10    std::cout << "Request Complete: "
11              << Res.status_code;
12  }};
13
14  std::cout << "Starting Request\n";
15  cpr::GetCallback(callback, URL);
16
17  // This is not blocked
18  std::cout << "Loading...\n";
19}

1Starting Request
2Loading...

This simple program compiles and runs successfully, but it ends before our request is completed.

We can expand it to something slightly more elaborate. Below, we’re using a while loop to keep our application running until something tells it to stop.

This is a common pattern, often called the application loop.

In the following example, our program continues until our HTTP request has been completed. This calls our callback lambda, which sets run to false, thereby causing our while loop to stop and our program to end:

1#include <cpr/cpr.h>
2#include <nlohmann/json.hpp>
3#include <iostream>
4using json = nlohmann::json;
5
6int main() {
7  bool run{true};
8  cpr::Url URL{
9      "http://www.dnd5eapi.co/api/monsters/"
10      "giant-spider"};
11
12  auto callback{
13      [&run](const cpr::Response& Res) {
14        json Doc{json::parse(Res.text)};
15
16        std::cout
17            << Doc.at("name").get<std::string>()
18            << " is ready after " << Res.elapsed
19            << " seconds\n";
20
21        run = false;
22      }};
23
24  std::cout << "Loading...\n";
25  cpr::GetCallback(callback, URL);
26  std::cout << "We're not blocked!\n";
27
28  while (run) {
29    // App loop
30  }
31  std::cout << "Bye!";
32}

1Starting Request
2Loading...
3Giant Spider is ready after 0.270608 seconds
4Bye!

Asynchronous Race Conditions

A common source of confusion and bugs occurs when dealing with asynchronous code. Inherently, we don’t know how long an asynchronous process will take to complete, and the duration can be different each time. For example, we do not know what the output of the following program will be:

1#include <cpr/cpr.h>
2#include <iostream>
3
4int main() {
5  bool run{true};
6  cpr::Url URL{"http://www.example.com/"};
7
8  cpr::GetCallback(
9      [&run](const cpr::Response& Res) {
10        std::cout << "a";
11        run = false;
12      },
13      URL);
14
15  cpr::GetCallback(
16      [&run](const cpr::Response& Res) {
17        std::cout << "b";
18        run = false;
19      },
20      URL);
21
22  while (run) {
23    // app loop
24  }
25}

It might log a, b, ab, or ba. It depends on which request finishes first, and how much slower the other one is.

This is an example of a race condition, and they happen any time we have asynchronous processes running concurrently, and our code is making assumptions about which one will finish first. If our assumption is wrong, our program will not behave as expected.

These bugs can be particularly insidious, as they can be very hard to detect. For example, if the process we’re assuming will win the race does indeed win 99% of the time, we may never detect we have a race condition. But, once our program is deployed at scale, the 1% edge case will happen regularly.

We cover concurrency and parallel programming in more detail in a dedicated chapter later in the course.

Sessions

cpr provides an alternative way to manage our HTTP requests, in the form of session objects. Unlike the functions we’ve been using before, sessions are stateful. We can provide them with values that are maintained, even after we send a request, and receive a response.

This is particularly useful when we want to send multiple requests with the same, or similar, settings. We can set our session up with URLs, parameters, headers, callbacks, and interceptors (covered next), and then re-use them to make multiple calls through the life of our application.

We update the state of our sessions using functions like SetUrl() and SetParameters(). When we’re ready to make a request, we use functions like Get(), Post(), and Patch() depending on the HTTP method we want to use,

1#include <cpr/cpr.h>
2#include <iostream>
3
4int main() {
5  cpr::Session Session;
6  Session.SetUrl("http://www.test.com");
7
8  Session.SetParameters(
9      {{"first", "one"}, {"second", "two"}});
10
11  cpr::Response Res1{Session.Get()};
12  std::cout << "Completed request: \n"
13            << Res1.url;
14
15  Session.SetParameters({{"third", "three"}});
16  cpr::Response Res2{Session.Get()};
17  std::cout << "\nCompleted request: \n"
18            << Res2.url;
19}

1Completed request:
2http://www.test.com/?first=one&second=two
3Completed request:
4http://www.test.com/?third=three

Sessions can also be used with asynchronous requests, using methods like GetCallback() and PostCallback(). However, when doing this, we need to manage the session through a shared pointer. This allows cpr to maintain its lifecycle correctly.

1#include <cpr/cpr.h>
2#include <iostream>
3#include <memory>
4
5int main() {
6  bool run{true};
7  auto Session{
8      std::make_shared<cpr::Session>()};
9
10  Session->SetUrl("http://www.example.com");
11
12  Session->GetCallback([&run](cpr::Response R) {
13    std::cout << "Request Complete!";
14    run = false;
15  });
16
17  std::cout << "Loading...\n";
18
19  while (run) {
20    // App loop
21  }
22}

1Loading...
2Request Complete!

Interceptors (Middleware)

When working with a session, we can provide an interceptor to that session. Interceptors are functions that allow us to run code before our request is sent over the network, or before our response is returned to the caller.

This design pattern is sometimes also referred to as middleware.

Interceptors allow us to modify or monitor the traffic flowing through the session.

A minimal example of an interceptor looks like this:

1#include <cpr/cpr.h>
2using cpr::Session, cpr::Response;
3
4class MyInterceptor : public cpr::Interceptor {
5 public:
6  Response intercept(Session& s) override {
7    Response Res{proceed(s)};
8    return Res;
9  }
10};

To summarise the key points:

• Interceptors inherit from the cpr::Interceptor class
• They override the intercept method.
• The intercept method receives a reference to the cpr::Session and returns the cpr::Response to the caller.
• We can get the cpr::Response by calling the inherited proceed method and passing in the session.

This interceptor doesn’t do anything, but it outlines the basic framework. We can now add code to this scaffolding as needed. For example, we can monitor or modify the session or the cpr::Response objects.

Things we want to happen before the request is sent need to appear before the call to proceed(), and things we want to happen after the response is received should come after it.

Finally, we attach an interceptor to a session by passing it a shared pointer to the AddInterceptor method.

Below, we’ve added an interceptor that simply logs some information about the traffic flowing through the session to which it is attached:

1#include <cpr/cpr.h>
2#include <iostream>
3#include <memory>
4using cpr::Session, cpr::Response;
5
6class MyInterceptor : public cpr::Interceptor {
7 public:
8  Response intercept(Session& Ses) override {
9    std::cout << "Request to "
10              << Ses.GetFullRequestUrl()
11              << " started";
12
13    Response Res{proceed(Ses)};
14
15    std::cout << "\nResponse received after "
16              << Res.elapsed << " seconds";
17
18    return Res;
19  }
20};
21
22int main() {
23  Session Ses{};
24  Ses.AddInterceptor(
25      std::make_shared<MyInterceptor>());
26  Ses.SetUrl("http://www.example.com");
27  Ses.Get();
28}

1Request to http://www.example.com started
2Response received after 0.196316 seconds

We can trigger multiple HTTP requests from a single call to intercept. This can be done by calling proceed multiple times or using any other way of creating HTTP requests we’ve seen.

This is useful for scenarios where we want our interceptor to retry failed requests, or we need to make preliminary or follow-up requests.

In the following example, we create an interceptor that will retry requests if the server responds with an error. We’re sending requests to http://www.httpbin.org/status/504 - an HTTP utility API that will respond with the status code we specify.

In this case, we’re asking it to return a 504 error, which is the response we’d get if the server is taking too long to respond.

We’ve added middleware that retries failing requests up to three times. This is a realistic use case - networking can be inherently unstable and requests can fail at random. When this happens, we often want to try again rather than immediately give up:

1#include <cpr/cpr.h>
2#include <iostream>
3#include <memory>
4
5using cpr::Session, cpr::Response;
6class MyInterceptor : public cpr::Interceptor {
7 public:
8  Response intercept(Session& Ses) override {
9    std::cout << "Request to "
10              << Ses.GetFullRequestUrl();
11
12    Response Res;
13    int RemainingTries{3};
14    while (RemainingTries > 0) {
15      Res = proceed(Ses);
16      if (Res.status_code >= 500) {
17        std::cout << "\nRequest failed: "
18                  << Res.status_line;
19        --RemainingTries;
20      } else {
21        return Res;
22      }
23    }
24
25    std::cout << "\nNo retries left - "
26                 "mission failed";
27
28    return Res;
29  }
30};
31
32int main() {
33  Session Ses{};
34  Ses.AddInterceptor(
35      std::make_shared<MyInterceptor>());
36  Ses.SetUrl(
37      "http://www.httpbin.org/status/504");
38  Ses.Get();
39}

1Request to http://www.httpbin.org/status/504
2Request failed: HTTP/1.1 504 GATEWAY TIMEOUT
3Request failed: HTTP/1.1 504 GATEWAY TIMEOUT
4Request failed: HTTP/1.1 504 GATEWAY TIMEOUT
5No retries left - mission failed

We can also have multiple interceptors attached to a single session. In the following scenario, the "B" interceptor is added to our session after the "A" interceptor:

1#include <cpr/cpr.h>
2#include <iostream>
3#include <memory>
4using cpr::Session, cpr::Response;
5
6class AInterceptor : public cpr::Interceptor {
7 public:
8  Response intercept(Session& Ses) override {
9    std::cout << "A Starting\n";
10    Response Res{proceed(Ses)};
11    std::cout << "A Done\n";
12    return Res;
13  }
14};
15
16class BInterceptor : public cpr::Interceptor {
17 public:
18  Response intercept(Session& Ses) override {
19    std::cout << "B Starting\n";
20    Response Res{proceed(Ses)};
21    std::cout << "B Done\n";
22    return Res;
23  }
24};
25
26int main() {
27  Session Ses{};
28  Ses.AddInterceptor(
29      std::make_shared<AInterceptor>());
30  Ses.AddInterceptor(
31      std::make_shared<BInterceptor>());
32  Ses.SetUrl(
33      "http://www.httpbin.org/status/200");
34  Ses.Get();
35}

1A Starting
2B Starting
3B Done
4A Done

We can see later interceptors being somewhat "nested" inside the ones that came before.

Specifically, the proceed call from A is calling the next interceptor, B. There are no interceptors after B, so B’s proceed call is making the HTTP request. When B receives the response it returns it to A to pick up where it left off.

This is perhaps not surprising, as we’re just dealing with a function call stack here. The only difference is that we don’t need to determine what the next function is - we just always call proceed(). Behind the scenes, cpr is managing the process for us, deciding what the next interceptor should be or, if there are no more interceptors, proceeding to start the HTTP request.

Summary: Using HTTP in C++

In this lesson, we explored various aspects of using HTTP in C++, to enable communication between programs and external resources over the Internet. Key topics covered include:

• Networking Protocols: Introduction to networking protocols, with a focus on HTTP (Hypertext Transfer Protocol) and its significance in web-based communications.
• Using the cpr Library: Installing the cpr library, and using it to make HTTP requests and responses easier to create and manage
• HTTP APIs: How to interact with public HTTP APIs, with an example using the Dungeons and Dragons API.
• HTTP Request and Response: Detailed discussion on the structure and processing of HTTP requests and responses, including URL handling and parsing JSON responses.
• HTTP Methods and Status Codes: Overview of different HTTP methods like GET, POST, DELETE, etc., and understanding HTTP status codes and their implications.
• Handling Headers and Parameters: Techniques for managing headers in HTTP requests and responses, and incorporating parameters (query strings) into HTTP requests.
• Authentication Methods: Understanding different authentication mechanisms in HTTP, including Basic, Digest, and Bearer Token authentication, and how to use them with cpr.
• Asynchronous Requests: Introduction to asynchronous HTTP requests in C++, explaining the concept and implementation of non-blocking calls with callbacks.
• Handling Sessions and Interceptors: Utilizing sessions for stateful HTTP communication and interceptors (middleware) for modifying or monitoring HTTP traffic.

This lesson provides a foundational understanding of HTTP communication in C++ and equips learners with the skills to integrate network capabilities into their C++ applications.

Next Lesson

Binary Serialization using Cereal

A detailed and practical tutorial for binary serialization in modern C++ using the cereal library.

ContinueView Course