Host header required?
yes! many webservers serve multiple websites, and the
Host header is
how the server can tell which domain/subdomain you're requesting.
User-Agent header required?
the only required header is the
always set, though, and servers sometimes use it to decide which version of the site to serve you.
your browser saves any cookies the server set and will send them back to the server on every request it makes.
when someone clicks a link, by default browsers will set the
Referer header to the address of the site they were on
when they clicked the link.
Referer is misspelled :) )
Accept-Encoding: gzip requests a compressed response. This
saves on bandwidth, so browsers will basically always set this header.
requests a response in Spanish. As with
there's no guarantee that you'll actually get a response in Spanish,
but some websites will respect this header!
header lets you send a username & password. They're base64
encoded but not encrypted, so it's important to use TLS.
Authorization: Basic OTA0MmYyNzExM...
some APIs instead need you to put the API key somewhere else, like in the request body
wget has a
--continue flag that lets you
continue a download you interrupted. Which header does it use to do
Range lets you request a specific part of a document, like "everything after 20392383 bytes" for large download.
If you don't set
Content-Type, often the server won't know how to interpret your request.