- 1.What is Hypertext Transfer Protocol (HTTP) anyway?
- 2.Ultimate Guide Understanding HTTP Status Codes - 300 Series
- 3.Ultimate Guide to Understanding HTTP Status Codes – 400 Series
- 4.Ultimate Guide to Understanding HTTP Status Codes 500 - Internal Service Error
Ultimate Guide to Understanding HTTP Status Codes – 400 Series
400 series request codes deal with malformed and unserviceable requests from a user-agent. Whenever a 400 series is encountered the problem is usually on the client side.
Issues with a user-agent client can range from requesting resources that are no longer available (by mis-typing a URI) or even the server reacting to requests fitting a security-attack signature.
The most common is the 404 – document not found. 404 Document Not Found is returned when a resource is requested the web-server cannot find:
Let’s go over the most common and interesting 404 series response codes in a little more detail.
400 – Bad Request
400 Bad Request is (usually) the result of a user-agent sending something in the request a web-server could not process. Let’s generate a 400 Bad Request to understand what can cause this error.
We will go old-school on Linux and use a tool called netcat. Netcat can be used for sending raw data over sockets. After all, HTTP requests are just that – raw text.
First, let’s send a legitimate HEAD Request to google:
Let’s force a 400 – Bad Request:
We purposfully sent a request the server could not understand – let’s look at the request method used:
mint ~ # nc www.google.com 80 BAD / HTTP/1,1 Host: www.google.com
The web-server could not process the method: BAD
It’s unlikely a modern web browser would send malformed requests like depicted above. However, if you’re using a custom developed HTTP application like a web-scraper, it’s time to debug.
Following the procedures below will resolve most all 400 Bad Request Errors.
1. Check The URL
If clicking on a link it is quite possible the web-author made a typo. Examine the entire URL paying particular attention to the path, document title, and query-string portions.
2. Clear all Cookies and Cache
Clearing the browser cache will solve 80% of 400 Bad Request Codes. Following these steps is a good universal fix for a lot of 400 Status Codes, too.
- clear all cookies and browsing history
- re-try the request
Let’s see how this is done with Chrome:
- Go into Settings
- Go to Advanced
- Click “Clear Browsing Data”
- Select the following options
Clearing all cookies and browsing history will assure anything cached and/or corrupted (like cookies) are gone. More than likely this will fix your issues with most 400 Bad Request Codes
3. File Upload Size
The problem could very well be a back-end script itself, uploading too large of a file or a file with an incorrect media type.
- If the 400 bad request presents itself while uploading a file, try a smaller file.
- Make sure the file-type is acceptable. 2.
- If uploading an image try both jpeg and png formats.
- If an uploading a document usually text files .txt and Microsoft Word files are permitted.
4. Malformed Headers
Malformed headers would be a pretty low-level issue and could very well leave the web-application itself at fault.
Using the HTTPFox plugin with Firefox we can watch the entire HTTP transaction for anything looking out of the ordinary.
5. Update Your Web Browser and Disable Plugins
Like anything HTTP goes through versions and revisions. If your browser is out of date it could very well be sending antiquated headers and requests.
Also, try removing or disabling all plug-ins and extensions. Trying the same request in another web browser can assist in isolating issues as well.
6. Check Your Internet Connection
A bad Internet Connection can cause data to be corrupted at lower layers, thus sending malformed data to the web server.
Using a good Internet Speed Test utility can help troubleshoot any issues with an Internet connection. Try running the utility 5 or 6 times for comparison.
7. Try Logging Out
If using a web application with user-based authentication and sessions try logging out and then clearing the browser cache.
401 – Unauthorized
401 Unauthorized is returned when a web server requests credentials for a secured document.
403 – Forbidden
403 Forbidden is usually one of four things:
- directory listings are not permitted
- currently authenticated user does not have sufficient permissions
- the web-server process has inadequate permissions
- requesting address has been banned
Directory Listing Not Permitted
IIS Showing Directory Permissions are not granted to unauthenticated users. The simplest method is to put an index file in the directory. Otherwise, directory listings to unauthenticated users will need to be explicitly set.
Authenticated User Does Not Have Permissions
The best method in this situation is making a group for all users who need access to a resource. Give this group the necessary permissions at the filesystem level. Then add permitted users to the group.
Web Server Process Does Not Have Sufficient Permissions
We are unable to access the test.php page in the webroot in /var/www/html. So first we need to:
check permissions to the webroot:
assign the www-data group to the webroot and give read + execute permissions:
mint# ls -ld /var/www/html/ drwx------ 3 root root 4096 Nov 24 03:47 /var/www/html/
the /usr/var/html/ directory is owned by the root user and root group with no access to any other user.
To fix a permissions issue like this, we need to modify permissions to the directory:
mint# chgrp -R www-data /var/www/html mint# chmod 750 /var/www/html mint# chmod g+rx /var/www/html/test.php mint html # chmod g-w /var/www/html/test.php mint# ls -ld /var/www/html/ drwxrw---- 3 root www-data 4096 Nov 24 03:47 /var/www/html/ mint# ls -l /var/www/html/test.php -rwxr-x--- 1 root www-data 91 Nov 16 06:52 /var/www/html/test.php
A little theory on how the problem was fixed:
- change the owning group of the webroot to www-data
- give proper permissions to the web-root for the www-data group. directory should be read and execute
- give read and execute permissions to test.php for the www-data group
Permissions, as mentioned above, are for a production environment. But can be somewhat restrictive for development stages.
Since we took away write permissions to the directory and to test.php only root can add and edit files. This is good in production – we are 100% sure the web-process cannot write to and create files if a script is compromised.
However, this is very impractical for development. In a staging environment, it would be best to put developers in the www-data group. Then give them write permissions to the web-root and all its files. This is a good example of why modern DevOps keeps staging and production segmented.
Another common scenario for a 403 Forbidden Status may be an IP Address or IP Range being banned in a .htaccess file by the server Administrator.
404 – Not Found
404 Not Found means the web-server could not find a requested resource. When encountering a 404 error a few things can be done:
check spelling of host, domain, path, and document
check case of file and path. Linux and Unix web-servers (unless configured differently) usually distinquish case, with tradition of using all lowercase for each.
use Google to find an alternative URL to the resource
try getting a cached page from google
Using Advances Google Operators
Many times I have been able to locate a document with expired links (resulting in a 404) using a Google query similar to the one above. Listing of Advanced Google Operators
Try Getting the Same Resource From Google’s Cache
Using advanced Google Operators and Google’s Cache you may be able to find the resource even if it has been taken down.
405 – Method Not allowed
A 405 Method Not Allowed is presented when attempting a request method that is unpermitted by the web-server.
406 – Not Acceptable
406 Not Acceptable will be given when the user-agent and client cannot establish acceptable content/encoding types or character-sets.
When resolving a 406 Status Code using a tool like HTTPFox is necessary.
Header values sent by the client to look over include:
A modern web-browser will never experience this issue. If it does, try upgrading your web-browser to later version.
407 – Proxy Authentication Required
407 Proxy Authentication Required will usually mean the user-agent needs to provide credentials to access a proxy server.
408 – Request Timeout
This status code will usually result from latency between your computer and the web-server.
To assist in resolving 408 Request Timeout Errors:
- try another website to make sure it is not the server infrastructure
- make sure no other downloads are running saturating your bandwidth
- reconnect to your WiFi
- reboot your router
- try an Internet Speed Test
If able to connect to another website fine, it is probably a remote network. If an Internet Speed Test is showing latency the problem could be your ISP or LAN.
Speedtest.net offers a great and free online app for testing Internet connection speeds. The app will check upload speed, download speed, and latency. Speedtest.net Speed Testing App Online
429 – Too Many Requests
429 will usually mean rate limiting has been enabled on the server. Rate limiting will only allow a certain amount of requests in an allotted amount of time.
When recieving 429 – Too Many Requests the user-agent will be blocked for a time period, usually by IP address.
To bypass 429 and 403 restricted errors try using Tor Browser
451 – Unavailable for Legal Reasons
451 could be seen when a web-page violates copyrights, trademarks, intellectual property laws, or has been deemed a threat to national security.