Selenium Wire for HTTPS Requests
Selenium Wire is a great way to proxy HTTPS requests. With few external dependencies, this library extends Selenium’s Python bindings to give you the ability to inspect requests made by the browser.
With Selenium Wire, you can create tests the same way as in Selenium. But Selenium Wire also provides an easy-to-use API that lets you control, intercept, and modify the request/response headers, status code, body content, and other project elements.
With Selenium Wire, you can capture both HTTP and HTTPS requests. Selenium Wire supplies its own certificates and supports Chrome, Firefox, and Remote Webdriver.
Installation and drivers
Install Selenium Wire using pip:
pip install selenium-wire
You won’t need to configure a browser; however, as with Selenium, be sure to download the ChromeDriver and GeckoDriver, which enable remote control of Chrome and Firefox.
Selenium Wire requires OpenSSL for decrypting HTTPS requests. The exception among operating systems is Windows, which does not require this installation. Generally, with other systems such as Linux and MacOS, you’ll find OpenSSL pre-installed. If not, you can follow this link for steps to installation.
Webdriver and Chromedriver
webdriver from the seleniumwire package:
from seleniumwire import webdriver # Create a new instance of the Chrome driver driver = webdriver.Chrome() # Go to the Google home page driver.get('https://www.google.com') # Access requests via the `requests` attribute for request in driver.requests: if request.response: print( request.url, request.response.status_code, request.response.headers['Content-Type'] )
For a remote site behind a proxy server, you can set options in Selenium Wire via the option settings passed to the
webdriver. Follow the link for sample options code as well as authentication methods.
Selenium Wire decrypts HTTPS transmissions using its own root certificate. The browser isn’t prompted to trust the certificate, so it functions normally although a “Not Secure” message displays in the address bar. You can download the root certificate by following this link. Then import it to your browser settings under “Certificates” in the “Authorities” section.