Selenium Wire for HTTPS Requests

Selenium Wire is a great way to proxy HTTPS requests. With few external dependencies, this library extends Selenium’s Python bindings to give you the ability to inspect requests made by the browser.

With Selenium Wire, you can create tests the same way as in Selenium. But Selenium Wire also provides an easy-to-use API that lets you control, intercept, and modify the request/response headers, status code, body content, and other project elements.

With Selenium Wire, you can capture both HTTP and HTTPS requests. Selenium Wire supplies its own certificates and supports Chrome, Firefox, and Remote Webdriver.

Installation and drivers

Install Selenium Wire using pip:

pip install selenium-wire

You won’t need to configure a browser; however, as with Selenium, be sure to download the ChromeDriver and GeckoDriver, which enable remote control of Chrome and Firefox.

OpenSSL

Selenium Wire requires OpenSSL for decrypting HTTPS requests. The exception among operating systems is Windows, which does not require this installation. Generally, with other systems such as Linux and MacOS, you’ll find OpenSSL pre-installed. If not, you can follow this link for steps to installation.

Webdriver and Chromedriver

Import webdriver from the seleniumwire package:

from seleniumwire import webdriver
# Create a new instance of the Chrome driver
driver = webdriver.Chrome()
# Go to the Google home page
driver.get('https://www.google.com')
# Access requests via the `requests` attribute
for request in driver.requests:
    if request.response:
       print(
           request.url,
           request.response.status_code,
           request.response.headers['Content-Type']
        )

Proxies

For a remote site behind a proxy server, you can set options in Selenium Wire via the option settings passed to the webdriver. Follow the link for sample options code as well as authentication methods.

Certificates

Selenium Wire decrypts HTTPS transmissions using its own root certificate. The browser isn’t prompted to trust the certificate, so it functions normally although a “Not Secure” message displays in the address bar. You can download the root certificate by following this link. Then import it to your browser settings under “Certificates” in the “Authorities” section.