FlowingDev

HTTP Messages, Deconstructed: Building the Web's Raw Packets

Learn the anatomy of raw HTTP messages, from the start-line and headers to complex multipart bodies, to understand how web clients and servers communicate.

Try the tool: HTTP Message Builder

In one sentence

An HTTP message is a formatted block of plain text that web browsers and servers exchange, acting as a combination of a shipping label, an instruction manual, and the package contents for every single web interaction.

The problem it solves

In the primordial ooze of the early 1990s web, things were simple. A browser needed a way to ask a server, "Hey, can I have that science.html file?" and the server needed a way to reply, "Sure, here it is," or "Sorry, couldn't find it." This conversation needed rules—a protocol. That protocol became HTTP, the Hypertext Transfer Protocol.

The "problem" it solved was creating a universal, unambiguous language for the web. Without a standard format, one server might expect the request on a single line, while another might need a haiku. It would be chaos. The initial HTTP/0.9 was dead simple: GET /the-page-i-want.html. The server would just dump the HTML back.

But the web didn't stay simple. We needed to send data to the server to fill out forms. We needed to handle different content types like images and, later, JSON. We needed security, caching, and a way for browsers to describe themselves. The simple one-line request evolved into a structured, multi-part "message" with a start-line, a block of metadata (headers), and an optional body for the actual payload. Crafting these messages by hand became the fundamental skill for anyone working directly with web infrastructure, APIs, or security, solving the problem of how to conduct increasingly complex business over the web's simple request-response dialogue.

How it works under the hood

At its core, an HTTP message is just text. You could literally type one into a terminal and pipe it to a server if you were so inclined. This text is divided into three parts: a start-line, a block of headers, and an optional body, all separated by specific line breaks (\r\n, or CRLF for "Carriage Return, Line Feed").

There are two types of messages: requests (client to server) and responses (server to client). They look almost identical but have a different first line.

Anatomy of a Request Message

This is your browser asking for something.

GET /documentation/guides/http-builder HTTP/1.1
Host: flowing.dev
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:109.0) Gecko/20100101 Firefox/117.0
Accept: text/html,*/*
Accept-Language: en-US,en;q=0.5
Connection: keep-alive

<-- The body would go here, but GET requests usually don't have one -->

  1. The Start-Line: GET /documentation/guides/http-builder HTTP/1.1

    • GET: The HTTP method (or verb). It's what you want to do. GET retrieves data, POST submits new data, PUT updates existing data, DELETE removes data.
    • /documentation/...: The resource path. Combined with the Host header, this forms the full URL.
    • HTTP/1.1: The protocol version.

  2. The Headers: A list of key-value pairs that provide crucial metadata about the request.

    • Host: flowing.dev: Who is this request for? This header is mandatory in HTTP/1.1.
    • User-Agent: Mozilla/5.0...: Who is sending this request? The browser identifies itself.
    • Accept: text/html,*/*: What kind of response format can I understand? Here, the browser prefers HTML but will accept anything.

  3. The Blank Line: After the last header, a single blank line (\r\n) signals "headers are done, the body is next." This is non-negotiable. Omitting it will break everything.

  4. The Body: The actual data payload. For a GET request, it's usually empty. For a POST or PUT, this is where your form data or JSON payload lives.

Anatomy of a Response Message

This is the server replying to the request.

HTTP/1.1 200 OK
Content-Type: text/html; charset=utf-8
Content-Length: 15328
Server: Vercel
Date: Mon, 25 Sep 2023 10:30:00 GMT
Cache-Control: public, max-age=0, must-revalidate

<!DOCTYPE html>
<html>
  <head>...</head>
  <body>...</body>
</html>

  1. The Status-Line: HTTP/1.1 200 OK

    • HTTP/1.1: The protocol version, same as the request.
    • 200: The Status Code. A three-digit number summarizing the result. 2xx means success, 3xx means redirection, 4xx means you (the client) messed up, and 5xx means I (the server) messed up.
    • OK: The Reason Phrase. A human-readable summary of the status code.

  2. The Headers: Metadata about the response.

    • Content-Type: text/html: "The body I'm sending you is HTML." This is critical for the browser to know how to render the payload.
    • Content-Length: 15328: "The body is exactly 15,328 bytes long."
    • Set-Cookie: ...: How servers tell browsers to store cookies.
    • Cache-Control: ...: Instructions for how the browser or intermediate proxies should cache this response.

  3. The Body: The resource the client asked for—HTML, CSS, a JSON object, image data, etc.

Body Bonanza: Encoding the Payload

When a request has a body, it needs a Content-Type header to explain its format. The three most common are:

  • application/x-www-form-urlencoded: The default for old-school HTML forms. It's just a query string in the body.

    name=Grace+Hopper&title=Rear+Admiral

  • application/json: The king of modern APIs. The body is a JSON string.

    {
  "name": "Grace Hopper",
  "title": "Rear Admiral"
}

  • multipart/form-data: The format for submitting forms that include file uploads. It's like a message within a message. The body is broken into parts, each separated by a "boundary" string. Each part can have its own mini-headers (like Content-Disposition and Content-Type) and its own content.

    POST /profiles/edit HTTP/1.1
Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW

----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="username"

ada_lovelace
----WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="avatar"; filename="portrait.jpg"
Content-Type: image/jpeg

<...raw binary data of the image goes here...>
----WebKitFormBoundary7MA4YWxkTrZu0gW--

Real-world stories

The Case of the Missing Content-Type

A developer was building their first REST API. The endpoint was supposed to accept a JSON payload to create a new user. They wrote the server code and tested it with a command-line tool, sending a perfectly valid JSON object. But the server kept responding with 400 Bad Request. They spent two hours staring at their JSON, convinced they'd missed a comma. In desperation, they asked a senior dev for help. The senior dev took one look at the request and asked, "Where's your Content-Type header?" The developer had sent the JSON data, but they never told the server it was JSON. The server framework, expecting the default x-www-form-urlencoded, tried to parse the JSON as a query string, failed miserably, and rejected the request.

Lesson: The body of a message is meaningless without the Content-Type header to give it context. You have to label your package correctly.

The Multipart Mix-up

A team was creating a "settings" page where a user could change their name and optionally upload a new profile picture. The junior frontend dev implemented this with two separate API calls: a PUT request with the user's name in a JSON body, and then, if a picture was selected, a POST request with the image data. It worked, but it was clunky and created race conditions. What if the name change succeeded but the image upload failed? The user would be left in an inconsistent state. A backend engineer saw the network traffic and pulled them aside. "This is a perfect use case for multipart/form-data," she explained. They refactored the code to build a single POST request with two parts: one for the name field and one for the image file. It simplified the code and made the entire update an atomic operation.

Lesson: multipart is not just for files. It's for sending a mixed bag of data—text fields, files, different content types—in a single, reliable request.

The Ghost in the Cache

An e-commerce site was running a flash sale, but users complained that they were seeing stale prices. The ops team was stumped; their server-side caching was configured correctly. A web performance expert was called in. Instead of using browser dev tools, she used a tool to inspect the raw HTTP response for a product page. She found the culprit instantly. A misconfigured load balancer in front of the web servers was injecting its own Cache-Control: public, max-age=3600 header, overriding the server's intended Cache-Control: no-cache header. This rogue header was telling browsers and CDNs to cache the prices for an hour, no matter what the application server said.

Lesson: The raw HTTP message is the ultimate source of truth. High-level tools can sometimes hide or misinterpret details that are plain as day in the text itself.

Common mistakes and traps

  • Forgetting the blank line. An HTTP message must have a CRLF (\r\n) between the headers and the body. If it's missing, parsers will think your body is just another malformed header and the request will fail.
  • Mismatched Content-Length. If you declare a Content-Length header, its value must be the exact byte size of the body. If it's too small, your data will be truncated. If it's too large, the server will wait forever for bytes that never arrive.
  • Wrong Content-Type. Sending a JSON body but labeling it as text/plain is a recipe for a 4xx error. The header and body must agree.
  • CRLF vs. LF. The official spec demands \r\n for line breaks. Most modern servers are tolerant and will accept a simple \n (Line Feed). However, relying on this can cause your request to fail with older, stricter servers, proxies, or firewalls.
  • Encoding special characters. Forgetting to URL-encode data in a query string or x-www-form-urlencoded body is a classic bug. A space must become %20, an & must become %26, and so on, or you risk corrupting your data.

Why it belongs on your radar

Most of the time, your browser, framework, or library (like axios or requests) handles the messy details of building HTTP messages for you. But you should know how to do it manually when:

  • You're deep in a debugging session. When an API call isn't working and the error message is vague, inspecting or re-creating the raw HTTP message is the final arbiter. It lets you see exactly what's going over the wire, free from any abstraction.
  • You're building or testing an API. Understanding the message structure is fundamental to designing good API endpoints and writing effective integration tests. Security testers spend their days crafting malformed messages to find vulnerabilities.
  • You're scraping a website. To successfully mimic a real browser and bypass anti-bot measures, you often need to construct a request with a very specific combination of headers (User-Agent, Referer, Accept-*, etc.).
  • You're working with webhooks. When your application receives a webhook from a service like Stripe or GitHub, you're on the receiving end of a raw HTTP request. You'll need to parse its headers (e.g., for security signatures) and its body to act on the event.

Knowing how to assemble an HTTP message from scratch is like a mechanic knowing how an internal combustion engine works. You don't do it every day, but when something goes wrong, that fundamental knowledge is priceless.

Go deeper

Theory done. Time to get your hands dirty — 100% in your browser.

Try the tool: HTTP Message Builder