Why is this needed?
We all know that Selenium is the most widely used open-source tool for automating testing suite for web applications across a range of platforms and browsers. It can achieve the operating of web controls and different web browsers. But sometimes we need to get the data returned by the API to the front-end for testing, logging and debugging which Selenium cannot do.
When talking about capture HTTP requests, most folk think of using Fiddler, Wireshark and Badboy which allow you to start-up a proxy server and intercept all requests and responses. But these tools lack of integrate methodology via API/CLI.
In this blog I will present you the ways to capture HTTP requests using Selenium.
Selenium Wire
As mentioned above to capture HTTP requests and network traffic you need to use an HTTP proxy and configure the browser to direct all traffic through the proxy.
Selenium Wire is a python library extends Selenium Webdriver bindings to give your tests access to the underlying requests made by the browser. It is a lightweight library designed for ease of use with minimal external dependencies.
With Selenium Wire, you write your tests in just the same way as you do with Selenium, but you get an additional user-friendly API for accessing things such as the request/response headers, status code and body content.
Runbook
Installation:
install using pip
Init web browser:
Accessing requests:
Selenium Wire captures all HTTP/HTTPS traffic made by the browser during a test. You can retrieve all requests with the driver.requests attribute. The requests are just a list and can be iterated and indexed:
Request Attributes:
Requests have the following attributes:
- method: the HTTP method type, e.g. GET or POST
- url: the request URL
- path: the request path
- querystring: the query string, e.g. foo=bar
- params: a dictionary of request parameters
- headers: a case-insensitive dictionary of request headers
- body: the request body as bytes
- response: the response associated with the request
Response Attributes:
The response can be retrieved from a request via the response attribute. Response has the following attributes:
- status_code: the status code of the response
- reason: the reason phrase, e.g. OK or Not Found
- headers: a case-insensitive dictionary of response headers
- body: the response body as bytes
Simple Example:
Console output:
Limitations & Additions
- Selenium Wire will currently work with tests that run on the same machine as the browser. A distributed setup using Selenium Grid is not yet supported.
- Sites that use NTLM authentication (Windows authentication) cannot currently be tested with Selenium Wire. NTLM authentication is not supported.
Find us to know more about Selenium Wire and web UI automation testing best practices.
