Proxy Connection Problems

If you're having trouble connecting to a proxy server, please read the following questions & answers...

Are you behind a firewall?

Some hosting providers will block outgoing access to port 31280. Check with your hosting provider to find out if this is the case for you.

Also, if you've recently upgraded your ProxyMesh plan for access to more proxy servers, make sure the added servers are allowed through your firewall.

Are you using incorrect login credentials?

Please check to be sure you are using the correct login credentials and try logging in again.

Is your IP blocked?

ProxyMesh is not able to whitelist a blocked IP. Please contact your Internet Service Provider for a good reputation IP.

Have your authenticated IPs disappeared from your dashboard?

This can happen if your subscription was cancelled due to too many declined transactions, and was afterwards reactivated. If so, your authorized IPs get deleted automatically. After reactivation, you can add them back.

Are you getting Connection Refused from AWS/EC?

You probably need to explicitly enable outgoing access to port 31280 in your EC2 firewall/networking configuration.

Did you get a bunch of 402 and/or 407 response codes? Are you getting Connection Refused or Cannot Connect to Proxy ?

If you get too many 402 and/or 407 response codes from a proxy, your IP will be temporarily blocked for 4 to 6 hours, during which time you will get  Connection Refused errors. 

• A 402 error indicates that you do not have access to the proxy and you may not have authorized it. You will need to go to Edit Proxies in your account dashboard to authorize the proxy. Or you can change to a different proxy that has not sent the error messages, and try again immediately. For further details, click here.
• If your log messages show a "dead" proxy, please send a copy of the log to Support for research. If these messages are visible to Support as 402 errors, the staff may ask you a series of questions for troubleshooting purposes.
• Some users may send data requests from an IP address without first authorizing it to the proxy. This causes an automatic block from the proxy and, hence, from the Internet. The block should expire, allowing you to try again after authorizing your IP to the proxy.
• If you are trying to connect to a specific proxy but are sent to a different proxy, this can mean that something is not configured correctly at your end. Please check your configurations. If you are using middleware, consider using the Scrapy default proxy configuration.

• A 407 error indicates an authentication problem. The easiest way to fix this is to Add your IP to ProxyMesh.
• A 407 error can also indicate an incorrect password. Please check your password for accuracy or consider authenticating your IP to a ProxyMesh proxy instead of using the username:password method. If you've already added your IP, then ensure you have removed the username & password authentication.

Have you gotten a lot of 407 responses when using one proxy specifically?

A large number of 407 request errors from one particular proxy server, while requests through other proxy servers are working correctly, may indicate a configuration issue. We suggest you check on the following points:

• Do you have 2 or more different scripts, and could one of them be misconfigured?
• Have you added your IP address for authentication? If so, please make sure you are not passing in any username:password authentication headers, which are the most likely cause of the 407 responses. Use only IP authentication.
• Please note that after you add an IP to your account, it should be automatically authorized at the servers within a few minutes. Please wait until this is done before you make requests from that IP to avoid a temporary block.

The easiest way to fix this issue is to Add your IP to ProxyMesh. Then our own Python-based proxy server forwards your requests through a set of rotating Squid proxies, which remove any request headers that might compromise your privacy and anonymity. Your actual requests are not changed in the process.

More about the 407 Response

• Be aware that authorizing an IP's access to the proxy during a 4-hour ban does not immediately lift the ban. You still need to wait until the 4 hours are up. Also note that ProxyMesh cannot manually remove this temporary block as the block is controlled by the hosting service or a remote site.
• ProxyMesh doesn't whitelist customers' IPs; but in effect you are whitelisting when you add an authorized IP. As long as your IPs are authorized beforehand, you should have no problem getting access to the proxies.

Are you getting 408 timeouts?

If a significant portion of your requests are timing out (the 408 response code), there's a few reasons this could happen.

1. The network connections between you and the proxy and/or between the proxy and the remote site are unreliable. Try some different proxies and see if that fixes the problem.
2. The pages you're requesting take a long time. The proxy server normally waits for 20 seconds, but the more popular crawling targets have a high volume of incoming traffic, which could mean a longer wait. However, you can increase that time with the X-ProxyMesh-Timeout custom header.
3. The proxy IPs have been blocked by the remote site. If you think this is the case, then you'll want to switch proxies, and ideally use multiple proxies to distribute your requests.

However, if only a small percentage of your requests are getting 408 response code, then the best thing to do is to retry your requests up to 3 times, ideally with a short delay of at least 1 second between retries.

Are you getting errors with the message body "ERR_PROXY_CONNECTION_FAILED"?

Here are some possible causes and solutions:

• It could be that on your end of the process the 408 timeout response code gets translated to either "net::ERR_PROXY_CONNECTION_FAILED" or "The request was aborted: Could not create SSL/TLS secure channel." We recommend you review "Request Retry Strategy" for ideas on retrying failed requests that return these message bodies. Having a good retry loop may eliminate those errors.
• Bear in mind that when you connect to an HTTPS site through the proxy server, all headers are encrypted. As a result, when you pass an Authorization header in, the proxy cannot read it, and it is passed through to the target site. If the target site is looking for that header, it will block requests that include it. For details on HTTPS requests, please review "Proxy Server Requests over HTTPS."
• It is also possible that the remote site blocks the proxy IPs. Then you need to wait for the IPs to rotate again. If you suspect this is the case, we recommend trying the World Proxy, which has many more IPs available.
• One other possibility is that the website with the issue might be upgrading its Transport Layer Security (TLS) requirement. If you are running queries on a Windows 10 computer, you may need additional configuration to negotiate the upgraded TLS protocol correctly.

For further information on the newest TLS protocol version, you may wish to follow this link.

Are You Attempting a Persistent Connection?

Although HTTP, the protocol supported by ProxyMesh, can use both persistent and nonpersistent connections, the proxy service is not designed for persistent connections. That is, a proxy connection normally closes after transmission of just one request and one response. The proxy service may detect and disable a keep-alive header included in a request.

Are You Blocked Due to Geolocation?

Some remote sites use geolocation to detect the location of IP addresses from a specific server. Geolocation may report some IP addresses as being in a different location from the one you intend.

If your requests are blocked for this reason, click here for steps to resolve the issue.

Are You Unable to Re-Authenticate an IP?

On occasion a proxy may start refusing connection to an IP address that was previously authenticated and successfully sending requests. The user is unable to re-authenticate the IP from the dashboard. If you have this issue, you may be able to troubleshoot it using the checklist below. Follow the links in these items for more detail.

• Were you sending your requests to a proxy you’ve selected on the dashboard?
• Was your IP address already authenticated to the proxy?

If not, go to Edit Proxies in your account dashboard to authorize the proxy.

• Are you trying to use one proxy, or multiple proxies? If multiple, is the problem occurring on all of them?

Be prepared to share this information if you need assistance from the Support staff.

• Did this issue come up after several 402, 407 , or 408 response codes? Were you getting Connection Refused or Cannot Connect to Proxy ?

If you get too many 402 and/or 407 or 408 response codes from a proxy, your IP will be temporarily blocked for 4 to 6 hours.  You’ll just need to wait out the ban; then you should be able to re-authenticate and successfully send requests.

• Is your IP address shared with someone who may or may not use the proxy, perhaps someone in a different organization?

A shared office IP could be the reason you cannot add your IP to the authentication list. IP sharing between accounts is impossible because the proxy can't know which account is using the IP for any given request.

Your best solution is to run your processes on a separate cloud server instead of in the office, or wherever the IP sharing takes place. That way, you get a unique IP address.

• Were you running HTTPS requests using Selenium along with your script and browser?

With that combination of tools, authentication can be tricky. If you have username:password, the proxy will use that for authentication. Otherwise, IP authentication is used instead. 

For HTTPS with web browsers or Selenium, IP authentication is the most reliable and easiest method. That's because the username:password header is not always passed to the proxies correctly for HTTPS requests. Even many programming language clients don't do that. Python requests is one of the few that can do it correctly.

• Have you checked the Proxy Status Page?

When you're preparing to run requests, it’s a good idea to check the condition of the proxy servers you'll use.

• Are You Using the Open Proxy?

To keep the list of open proxies fresh, the IPs are checked every 15 minutes, and any proxies that fail these checks are removed. However, issues can arise between checking times.

One possible error is a Sign In prompt asking for your username and password even though your IP address is whitelisted. As this might be a password collection attempt, we do not recommend entering your username and password. Instead, we suggest you cancel the request and then retry it.

If you still need help...

You can contact Support to investigate the issue. Be prepared to let them know:

• What errors you've been getting.
• What IP address you've been trying to send from.
• Whether you've gotten the errors on a single proxy, or more than one.
• Whether you've tried unsuccessfully to re-authenticate.
• Whether you share the IP address with a non-proxy user.

Do the logs show the 402 errors you've been seeing?

Support may review usage logs to pinpoint the cause of a proxy connection problem. Sometimes, however, the logs show no rejection of a data request for your account even though you're receiving 402 error messages.

If this is the case, it's possible that the error is coming from the remote site rather than the proxy server or API.

The behavior of a remote site is generally beyond ProxyMesh’s control, but here are some things you can try:

• As a temporary ban may be in effect, wait for 3 to 4 hours and try re-sending the request.
• If you're still getting errors after that period, try authorizing and using other rotating proxies to send the request.
• Try sending requests to the site via the World Proxy.