Open Proxies API
Most scrapers know about the drawbacks of open proxies. Those proxies are said to be slow and unreliable, some of them totally unavailable, while some that work consistently may leave you vulnerable to identity theft or harmful alteration of your messages.
By some estimates, these issues mean that only 20 or so percent of open proxies can be trusted.
But the advantages can be significant too, such as using a large diversity of IP address. If you can eliminate or reduce the downside, you can enjoy the benefits, including convenience and online privacy. ProxyMesh’s Open Proxies API offers an efficient, reliable way to keep on top of open-proxy status.
ProxyMesh Open Proxies API
The ProxyMesh Open Proxies API can make it easy to do what’s required to use free open proxies safely and comfortably. You have only to download 1 list of open proxies from a single reliable API, while the API handles the rest:
- Finding dependable open proxy lists
- A means of downloading or scraping from those sources regularly
- Checking whether each proxy is up and running now
- Setting up a file on a remote server for testing
- Regularly testing whether each proxy can fetch that file in a reasonable amount of time, and does not modify the content
To build its list, ProxyMesh tests over 1,000 open proxies every 15 minutes. It checks whether proxies respond within 8 seconds, and then excludes all the proxies that either do not work or maliciously manipulate requests (e.g., by inserting malicious code in responses). Typically, some 200 to 400 of these open proxies pass all the tests.
ProxyMesh uses these same proxy IPs in its own ProxyMesh Open Proxy. The Open Proxies API provides access to that list of IPs meeting acceptance criteria. You can view it in in a simple proxy list format that can be used by nearly all scraping software tools.
Please keep in mind that, because the open-proxy environment changes so rapidly, there can be no guarantee that the proxies will still be functional at retrieval time. To help you stay up to date, ProxyMesh offers a choice of reporting intervals. Via the GET command at the API Endpoints, you can download open proxy lists updated every 15 minutes, every hour, or once a day at around 00:00 UTC.
It’s a good idea to do your own monitoring too, since the open proxy environment can change so rapidly. After all, when proxies fail, your requests start failing. You also need confirm that:
- Your requests succeed in a reasonable amount of time
- Your requests have not been modified in transit
- No malicious code has been inserted into the response
The best way to do this is to have a piece of content you control on a remote server, then check that requests through each proxy return that content unmodified, without timing out or taking too long. The Open Proxies API does this already.
Check out the subscription service for ProxyMesh’s Open Proxy API. Besides reasonably priced paid plans, there is a free plan so you can confirm how it easy it is to use before you sign up. You’ll find code samples for 15 different programming languages and tools. Open Proxies API can make your proxying simpler and safer!