PhantomBuster Tips for Users
PhantomBuster Tips for Users
ProxyMesh is great for running with many specialized scraping tools, like PhantomBuster. Implementing PhantomBuster with ProxyMesh can be an effective web scraping strategy, combining key strengths and use cases of both.
This article discusses certain known issues that can arise in implementing the two applications together. Drawing on reports from ProxyMesh customers, we use a Q&A format to suggest ways to help you through difficulties you may encounter online.
If you haven’t already done so, we encourage you to read PhantomBuster Strategies with ProxyMesh for PhantomBuster questions.
| Issue | Response |
| What are my username and password? | They’re the ones you used to sign up with ProxyMesh. |
| Where do I enter them? | Enter them in ProxyMesh; and when you integrate with PhantomBuster, you need to also enter them in your Phantom's configuration (Settings) screen. |
| PhantomBuster isn’t taking my login info. | Make sure you’re not running a VPN. That’s because VPNs are sometimes used for fraud activity, and the system is wary of them. Also make sure your ProxyMesh username and password do not contain any special characters such as "@" or "#", as this will cause the Phantom not to work. This means you should not use an email address as your username. |
| I can log in, but my data requests are not working. | Choose a proxy in a geographic location close to your own. If it is not close enough, social networks may become suspicious of the Phantom's login attempts. |
| I'm using ProxyMesh with PhantomBuster as recommended, but it doesn't seem to work. | Make sure you’ve filled in the proxy information you received from ProxyMesh – it should look like this example: fr.proxymesh.com:31280 – and your login details. You can include your username and password in a script. Again, make sure they don't contain any special characters such as "@" or "#." |
| The proxy is not working. | Test the proxy in your browser to check whether or not this is the case:
Once you're sure it's working with your browser, we'd recommend logging into your Facebook account with the proxy installed before using your Phantom. Don't forget to give the Phantom your updated session cookies! (If you're using Firefox, check your browser settings to make sure it auto-detects your system proxy settings.) Note: For more information, including Mac procedures with screenshots, please see Proxy Not Working. |
| I am getting errors while Support is seeing more successful connections through the proxy. | This condition occurs when the proxy is working but the remote site is somehow blocking. Try another proxy and reduce your crawl speed. |
| I am getting the error message: "❌ Can't connect to Facebook, the proxy used may not be working." While using phantombuster I have tried a number of different proxies. |
You will see a note triggered by the PhantomBuster workflow, advising that you refer to PhantomBuster Strategies with ProxyMesh for PhantomBuster questions. If you were getting 402 errors, that could mean that you have not authorized your connection to servers. Generally, you should authenticate your IP address to the proxy and make sure the proxy is authorized. |
| |
|
| Selected Error Messages from PhantomBuster | |
| Error: Expired Session Cookie | The log indicates a problem with the Phantom's configuration – session cookie missing, could not launch properly. Session cookies should expire about every 3 months, or on manual logout of a website. If yours expire too often and you're not logging out yourself, perhaps the social network is logging you out as your activity rate is too high. Try reducing activity, keeping in mind the rate limits for different websites. These limits apply to the total number of actions done on your account in a single day.
|
| Error: [Social Network] disconnected or timed out | If Facebook or Instagram disconnected or took too long for your Phantom to connect, perhaps the social network thinks your rate of activity is too high. Make sure your session cookies are connected and you're automating within the recommended rate limits. |
| Error: No more search engines available | URL Finder Phantoms use search engines to find the URLs. When too many requests are processed from PhantomBuster IP addresses, the Phantom can get blocked for a few minutes. The Phantom will then display the error message "No more search engines available, please retry later." In this case, you just need to wait about 15 minutes before relaunching the Phantom. For ideas on how to improve the results with these Phantoms, check out URL Finder Phantoms. |
PhantomBuster has articles to help you take the initial steps easily and avoid potential problems.
- Proxy Not Working: Detailed discussion of PhantomBuster settings and issues that could arise further into a proxied PhantomBuster session.
- How to set up your first Phantom: Tutorial and video on the basics of setting up a Phantom and using the tools available to you on PhantomBuster.
- Log in to a website: Describes how to log in and use your script.
You might also check out the PhantomBuster blogsite for ideas on strategy and use cases.
MacOS Mojave or firewalls operating on MacOS Mojave sometimes block requests over HTTP. For details, check out this Stackoverflow forum.
And see our blog article 5 Tips for Successfully Using a Proxy with Phantombuster