Skip to main content
How to spy on your browser's HTTP requests and responses

This post is from the CollabNet VersionOne blog and has not been updated since the original publish date.

Last Updated Feb 07, 2013 — Enterprise Agile Planning expert

How to spy on your browser's HTTP requests and responses

Enterprise Agile Planning

In this article, you will

  • Learn how to spy on your Web browser's HTTP requests and responses
  • Start to understand why your browser sends HTTP request headers and why web servers send HTTP response headers

What you'll need

  • Google Chrome for its developer tools
  • Alternatively, you can use other browsers if you can figure out how to use their own development tools

HTTP is no secret at all and it's super easy to spy on your browser

Does it ever seem like cryptic magic what your browser and remote servers do together? Lest you think that HTTP requests and responses are like some secretive handshake that only browser makers and website operators know and can see: they're not. Assuming you are using Google Chrome, do this:

  1. Open a new window or tab
  2. Hit the F12 key to pull up the Google Chrome developer tools, and click on the Network tab
  3. Load Google in the new window. You can study the guts of the HTTP request used to GET the Google.com home page.

Notice the request method and request headers section

As you can see, the request method is GET, and below that is a big bag of request headers. Let's highlight just the major headers for now. HTTP request headers will come into play when we start to study how to send data to the Digital.ai Agility, formerly VersionOne, REST API because we'll need to specify a few "custom headers."

HTTP headers provide more information to help the server respond properly

Your browser sends HTTP headers as part of the HTTP request to give the server additional information it needs to know in order to process the request. The most relevant ones, practically speaking, are the accept header and the cookie one. The accept header tells the server what kind of formats the browser wants to get the response in, and the cookie is actually a unique value that identifies me that my browser previously got from Google. My browser sends the cookie back to Google on subsequent requests so that Google can correlate separate requests with me.

Now, scroll the pane down to see the response headers section

Similar, the response has headers. The content/type header is the most relevant here, and you'll notice that text/html corresponds to one of the values the initiating HTTP request contained its accept header. This will be important later when we use JavaScript to create requests for the Digital.ai Agility REST API.

Click the response tab

Do you remember in the last article the web's inventor Tim Berners/Lee's statement about the HTTP GET method when he first designed it? He said:

The response from the server was always an HTML page.

Even though the top of the response says <!doctype html>, most of the response contains JavaScript code, and really bizarre looking JavaScript code at that. This is called minified JavaScript. Developers minify script to make the size of a response smaller. But, it certainly makes it harder to understand! 

Now, let's examine an HTTP POST request

Assuming you still have another browser or tab open and the developer tools handy:

  1. Navigate to http://www.pastebin.com
  2. Enter something into the new paste form, scroll down and hit submit

POST request and response highlights

Here are few highlights from the request and response that you can see in your developer tools window:

Request headers

Request URL:http://pastebin.com/post.php
Request Method:POST

Request payload

The values contained in the payload correspond to HTML form field values. The source code for the HTML form element looks like this:

<textarea name="paste_code"></textarea>

If, like me, you typed Digital.ai Agility is my favorite agile management tool! into the new paste field (named paste_code in the HTML), then your payload should look similar to this:

//////WebKitFormBoundaryM0FtBA3eFtIeoTma
Content/Disposition: form/data; name="post_key"

KgvURkjA
//////WebKitFormBoundaryM0FtBA3eFtIeoTma
Content/Disposition: form/data; name="submit_hidden"

submit_hidden
//////WebKitFormBoundaryM0FtBA3eFtIeoTma
Content/Disposition: form/data; name="paste_code"

Digital.ai Agility is my favorite agile management tool!

Response headers

Again, like we saw for the GET response from Google.com, the content/type header is text/xml, but the new location header specifies where the browser should automatically navigate to to GET the response document, since if you look at the response tab you'll see that it is empty. This is a good design pattern called the Post/Redirect/Get pattern, and we'll learn more about it in later articles.

Content/Type:text/html
location:/Kxa3YUQL

More from the Blog

View more
Apr 08, 2021

Making IT services more agile

Enterprise Agile Planning
The agile revolution completely transformed how we create digital prod ...
Read More
Feb 14, 2021

Reflecting on the 20th anniversary of the Agile Manifesto

Enterprise Agile Planning
Over the past 20 years, it’s been amazing to watch an idea from ...
Read More
Feb 08, 2021

How does agile apply to an entire organization?

Enterprise Agile Planning
Before we dive into the main subject of this blog post, it is importan ...
Read More
Feb 03, 2021

It took a pandemic to realize why digital transformation actually matters

Enterprise Agile Planning
Before anyone had ever heard of COVID-19, businesses across the globe ...
Read More
Contact Us