Using Selenium in the Proxy

Selenium, an open-source software, provides a portable automated-testing framework for HTTP requests. Selenium includes a playback tool for users to author functional tests without needing to know a test scripting language.

The WebDriver feature, included in Selenium as of version 2.0, provides the WebDriver API, a platform- and language-neutral wire protocol. 

This enables users to write instruction sets to run interchangeably on most web browsers now in use. Selenium can be deployed on Windows, Linux and macOS.

This article describes configuration of Selenium for use in several popular programming languages and major browsers. Though Selenium comes with a test domain-specific language, called Selenese, you can also create tests in most scripting languages commonly used for ProxyMesh requests.

You can use any URL/hostname you already have access to for Selenium.

For web browsers, you must use IP address authentication, then configure your network proxy settings. If you use Firefox or Chrome, the FoxyProxy plugin makes it very easy to configure your proxy settings.

Regarding web browsers, you'll find additional information in our article How to Change Web Browser Proxy Settings.

C#

Selenium in C#

You'll find download and installation instructions at The Selenium Browser Automation Project. For automated testing, Visual Studio contains WebDriver's C# language bindings. Many users select NUnit as their testing framework. You'll also need the ChromeDriver executable.

Browsers with Selenium in C#

The Most Complete Selenium WebDriver C# Cheat Sheet includes basic and advanced browser settings for Selenium in C#.

Java

Selenium in Java

Here's a link to installation instructions for Selenium with Java.

To use Selenium and Java with ProxyMesh, see these 2 links for examples:

• How to start a Selenium Browser with Proxy
• How to set proxy configuration

The following example is from
Launching chrome driver with proxy settings in Selenium WebDriver java

Browser Configuration for Selenium in Java

How to launch chrome driver with proxy server

import org.openqa.selenium.Proxy;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.chrome.ChromeOptions;
import org.openqa.selenium.remote.DesiredCapabilities;

public class ChromeAutomationExtension {
  public static void main(String[] args) {
    System.setProperty(“webdriver.chrome.driver”, “C:\\selenium\\chromedriver.exe”);
    Proxy proxy = newProxy(); 
    proxy.setHttpProxy(“PROXYHOST:PORT”); 
    proxy.setSslProxy(“PROXYHOST:PORT”);

    DesiredCapabilities capabilities = DesiredCapabilities.chrome(); 
    capabilities.setCapability(“proxy”, proxy);

    ChromeOptions options = newChromeOptions();
    options.addArguments(“start-maximized”); 
    capabilities.setCapability(ChromeOptions.CAPABILITY, options);

    WebDriver driver = newChromeDriver(capabilities); 
    driver.get(“http://www.example.com”); 
    driver.manage().window().maximize(); 
    driver.quit();
  }
}

Proxy Settings Summary

• proxy.setProxyType(proxyType) ==> Indicates the type of proxy configuration.
• values are: “pac”, “direct”, “autodetect”, “system”, or “manual”.
• proxy.setProxyAutoconfigUrl(proxyAutoconfigUrl) ==> Defines the URL for a proxy auto-config file if proxyType is equal to “pac”.
• values are: any valid URL
• proxy.setHttpProxy(“rdc-proxy.server.com:PORT”) ==> Defines the proxy host for HTTP traffic when the proxyType is “manual”.
• values are: A host and optional port for scheme “http”.
• proxy.setNoProxy(noProxy) ==> Lists the address for which the proxy should be bypassed when the proxyType is “manual”.
• values are: A List containing any number of Strings.
• A proxyType of “direct” indicates that the browser should not use a proxy at all.
• A proxyType of “system” indicates that the browser should use the various proxies configured for the underlying Operating System.
• A proxyType of “autodetect” indicates that the proxy to use should be detected in an implementation-specific way.

Example to set proxy for Firefox driver instance

packageTesting_Pack;import java.io.IOException;

import org.openqa.selenium.Proxy;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.remote.CapabilityType;
import org.openqa.selenium.remote.DesiredCapabilities;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;

public class ProxySettings {
  WebDriver driver;

  @BeforeTest
  public void setUpDriver() {
    //Set proxy IP and port. Here PROXYHOST Is proxy IP and PORT Is Port number.
    //You can change both values as per your requirement.
    String PROXY ="PROXYHOST:PORT";
    //Below given syntaxes will set browser proxy settings using DesiredCapabilities.
    Proxy proxy = newProxy(); 
    proxy.setHttpProxy(PROXY).;
    DesiredCapabilities cap = newDesiredCapabilities(); 
    cap.setCapability(CapabilityType.PROXY, proxy);
    //Use Capabilities when launch browser driver Instance. 
    driver = newFirefoxDriver(cap);
  }

  @Test
  public void start() throws IOException {
    System.out.println("Check your webdriver driver Instance's proxy setttings.");
  }
}

JavaScript

Selenium in JavaScript

This Selenium JavaScript takes you to a tutorial on installing and configuring Selenium in JavaScript.

You can use Selenium and JavaScript together, along with nodejs, to run tests across most popular browsers.

If you are using nodejs to develop your application, the Selenium Project provides JavaScript bindings on npm.

npm install selenium-webdriver
	

The blog Selenium + JavaScript Best Practices offers tips on combining them for best results, as well as for test-writing in Selenium.

Browser Configuration for Selenium in JavaScript

You can use Selenium and JavaScript together, along with nodejs, to run tests across most popular browsers.

The blog Selenium + JavaScript Best Practices offers tips on combining them for best results, as well as test writing in Selenium.

To use JavaScript with Selenium, you will need to download additional components to work with each of the major browsers. The drivers for Chrome, Firefox, and Microsoft's IE and Edge web browsers are all standalone executables that should be placed on your system path. Apple's safaridriver is shipped with Safari 10 for OS X El Capitan and macOS Sierra. You will need to enable Remote Automation in the Develop menu of Safari 10 before testing.

Perl

Selenium in Perl

Follow this GitHub link for information on installing and using Selenium WebDriver with Perl.

Metacpan, an archive for the Comprehensive Perl Archive Network, provides additional usage examples here.

Browser Configuration for Selenium in Perl

Firefox & Chrome

To configure the web browser to use with Perl, follow this link to GitHub for steps to install Firefox (older and newer versions) and Chrome.

my $firefox =Selenium::Firefox->new; 
$firefox->get('http://www.example.com');
my $chrome = Selenium::Chrome->new; 
$chrome->get('http://www.example.com');
my $ghost = Selenium::PhantomJS->new; 
$ghost->get('http://www.example.com');
	

PHP

Selenium in PHP

The PHP language bindings are designed to work in PHPUnit, a unit testing framework for the PHP language.

Bindings are provided by the following third parties:

• Chibimagic
• Lukasz Kolczynski
• Facebook

Please refer to their respective documentation sites, on the above links, for steps to install and get started.

Browser Configuration for Selenium in PHP

The BrowserStack page for Selenium with PHP provides technical detail, including downloads for language bindings and settings for several versions of major browsers.

Python

Selenium in Python

To add Selenium to your Python environment, you must install Pip, the package installer for Python. Pip also has a dependency on setuptools.

pip install selenium
	

Browser Configuration for Selenium in Python

Chrome

To configure the Python WebDriver for Selenium to use Chrome, see  Installing Selenium WebDriver Using Python and Chrome.

Also see answer 5 in how do i set proxy for chrome in python webdriver.

PROXY = "PROXYHOST:PORT" # IP:PORT or HOST:PORT
chrome_options = webdriver.ChromeOptions() 
chrome_options.add_argument('--proxy-server=%s' % PROXY) 
chrome = webdriver.Chrome(options=chrome_options) 
chrome.get("http://whatismyipaddress.com")
	

Firefox

profile = webdriver.FirefoxProfile()
profile.set_preference("network.proxy.type", 1)
profile.set_preference("network.proxy.http",'HOST') 
profile.set_preference("network.proxy.http_port", PORT) 
profile.set_preference("network.proxy.ssl", 'HOST') 
profile.set_preference("network.proxy.ssl_port", PORT) 
driver = webdriver.Firefox(firefox_profile=profile)
	

Selenium Chrome Proxy Authentication

To use the proxy with Python and the Selenium library with Chromedriver, you might use code that looks like this:

chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server=http://%s:%s' % (hostname, port))
driver = webdriver.Chrome(chrome_options=chrome_options)

This code should work for a proxy with IP authentication. However, the code may not work if you to log in with username:password .

If so, try this code after removing the IP authentication:

options = webdriver.ChromeOptions()
options.add_argument('--proxy-server=http://%s:%s@%s:%s' % (username, password, hostname, port))
driver = webdriver.Chrome('chromedriver.exe', chrome_options = options)

If you've added the username:password back into the proxy string in your script in place of IP authentication, but you still don’t connect with the proxy, then follow the link to this alternative code: https://botproxy.net/docs/how-to/setting-chromedriver-proxy-auth-with-selenium-using-python/.

R

Selenium in R

For automated testing, the R Studio provides a link to install the Selenium 2.0 WebDriver. The following link takes you to a general how-to: How to drive a Web browser with R (and RSelenium).

The "RSelenium" package, with a complete set of R language bindings for Selenium WebDriver 2.0, is available as a PDF.

Scraping with Selenium discusses a variety of ways R and Selenium can be used together with a proxy.

Browser Configuration for Selenium in R

For configuring your web browser to use R and RSelenium, you’ll find more information in How to drive a Web browser with R (and RSelenium).

For testing, the article referenced above, “How to drive a Web browser with R (and RSelenium)” includes detail on how to set up R Studio to open several different browsers in succession.

Ruby

Selenium in Ruby

The testautomationu.applitools site offers videos on:

• Selenium Webdriver with Ruby
• Getting Set Up with Webdriver
• Using the Webdriver API

... and advanced topics.

Browser Configuration for Selenium in Ruby

The BrowserStack page for Selenium with Ruby provides technical detail, including downloads for language bindings and settings for several versions of major browsers.

Custom Headers

Use the X-ProxyMesh-IP to specify a particular IP for the request. This IP should come from the X-ProxyMesh-IP response header of a previous request.

OR

Use the X-ProxyMesh-Prefer-IP header when you want to use a specific IP for a request, but you also want the request to succeed even if the IP is no longer available, in which case a random IP is chosen instead.