Proxy Server Request & Response Headers
This article covers the following topics:
- Request headers that can be used to modify the default behavior of the ProxyMesh servers
- Response headers from the proxy server
- Response codes indicating possible errors or server issue
- The HTTPS CONNECT method used with unencrypted custom headers.
The ProxyMesh proxy servers accept a few custom headers that can be used to modify the default behavior. The proxy server will also return certain response codes when a request is not successful. All requests to the proxy server will return with an additional header,
X-ProxyMesh-IP, which contains the IP used to complete the request. Note that HTTPS requests require extra work to make use of the custom headers.
If you have any questions about these headers, or problems using them, please email ProxyMesh Support.
If no custom headers are given, the proxy server will choose an IP at random, and use a timeout of 20 seconds when waiting for a response from the remote site. You can use the headers described below to alter this behavior.
Use this header when you want to use a specific IP for a request. This IP should come from the
X-ProxyMesh-IP response header of a previous request. If the IP is no longer available, you will get a
502 error code with the
X-ProxyMesh-IP-Not-Found response header.
Use this header when you want to use a specific IP for a request, but you also want the request to succeed even if the IP is no longer available. This is like the
X-ProxyMesh-IP header, but when an IP is not available, a random IP is chosen instead. You can tell when this happens by comparing the
X-ProxyMesh-IP response header to the IP you sent with the
X-ProxyMesh-Prefer-IP header. If they are different, then the IP you tried to use is no longer available.
Use this header to exclude a specific IP from the available choices. The actual IP chosen will be random, as long as it is not the given IP. You can also send a comma list of IPs to exclude multiple IPs. You will get a
502 error code if no more IPs are available, after excluding the IPs sent in the header.
The open and world proxies have IPs in multiple countries. Use the
X-ProxyMesh-Country header to restrict the IP choices to a specific country. The value of the header should be a 2 character ISO country code, such as
RU. Check the status page for each proxy to see what country codes and how many IPs are currently available. You will get a
503 response code if you use a country code with no IPs.
Note: To connect to sites – especially in large numbers – in a specific geographical area, we recommend specifying proxies located near that area and, if possible, in the same domain. This can help speed responses to requests.
The proxy server has a default response timeout of 20 seconds. If the remote server does not respond in that time, you will get a
408 response code. If you need to wait longer for a complete response, use the
X-ProxyMesh-Timeout header to specify the number of seconds you want to wait.
These headers are inserted by the proxy server. They will be included with the other response headers from the remote site, unless it is an HTTPS request, in which case the headers are included with the Connection response.
This header is included with every response, and its value is the IP used for the request. If you'd like to use the same IP for a subsequent request, pass in this header unchanged.
When an IP specified by the
X-ProxyMesh-IP request header is no longer available, you will get a
502 response with a
X-ProxyMesh-IP-Not-Found header containing the IP.
A remote site can return any number of response codes, including the same ones listed below, but for different reasons. It is important to inspect the response body in order to differentiate between whether the response came from the proxy server or the remote site.
HTTPS CONNECT Method with Custom Headers
HTTPS requests with custom headers are difficult because the actual request headers are encrypted. The only point at which unencrypted data is sent to the proxy server is with the initial CONNECT method. This is where you must insert the custom headers:
CONNECT example.com:443 HTTP/1.1 X-ProxyMesh-IP: 123.456.789.000 *actual request headers*
Similarly, the proxy server cannot inject an extra header into the final response. Instead, the
X-ProxyMesh-IP response header is injected immediately after the Connection response, which looks like this:
HTTP/1.1 200 Connection established X-ProxyMesh-IP: 123.456.789.000 *final response headers & body*
For additional details, please see Proxy Server Requests over HTTPS.