Establishment of a Docker-and-Chrome-based Selenium-grid for web-application-tests
For running Chrome based web-tests in parallel - in order to improve the overall elapsed times of the tests and enable running a smoke-tests-set per-commit - the task were to establish a Docker-based Chrome-Selenium-grid for the tests to use.
The Grid contains n nodes (where n is a number ranged 1-100 for reasons explained later on) with only 1 Chrome instance per node.
Docker compose implementation
Here’s the docker-compose.yml file of the Selenium-grid:
seleniumhub: image: selenium/hub restart: always ports: - 4444:4444 chrome: image: selenium/node-chrome-debug restart: always ports: - "5900-5999:5900" environment: - VNC_PASSWORD=secret - SCREEN_WIDTH=1920 - SCREEN_HEIGHT=1080 links: - seleniumhub:hub volumes: - /dev/shm:/dev/shm
The grid is based on 2 types of Docker containers:
- Selenium-grid hub - based on the Docker-hub image of selenium/hub, loaded into the default 4444 port. Only 1 instance of the hub is needed per Selenium grid.
Selenium-grid node - based on the Docker-hub image of selenium/node-chrome-debug. It is used in the debug version in order to enable VNC connections to each of the nodes. The configuration sets a different VNC port for each node limited to the 5900-5999 ports, but it can be extended if needed, and this is the reason for the 100 nodes limit. If no VNC connections is needed, it can be replaced with the selenium/node-chrome image (in such case, there will be no limitation for the number of nodes and the VNC_PASSWORD environment variable and the ports setting - of VNC port 5900 - will not be needed). The following environment variables are needed for the Selenium-grid node:
- VNC_PASSWORD - the password - set to be secret - needed when connecting the node using a VNC client.
- SCREEN_WIDTH and SCREEN_HEIGHT - for setting the node screen resolution to match the application requirements. By default the resolution is set to be a non-common resolution of 1360x1020 which were not OK with the tested application.
The docker-compose.yml is based on Selenium template, as described in: https://github.com/SeleniumHQ/docker-selenium/wiki/Getting-Started-with-Docker-Compose.
Note: I found that most of the changes for the tests to work properly with the grid should be done inside the code and not in the grid configuration, so no other special settings were used in the docker-compose.yml file, even that some were tested.
Here’s an example of a VNC connection to one of the Selenium-grid nodes:
For loading and reloading the grid properly, I’ve created a bash-shell script loadselenium-grid.sh (resided in the same folder as the docker-compose.yml file):
#!/bin/bash echo "" echo [stop and clear previous grid] docker rm -f $(docker ps -a -q) > /dev/null | true service docker restart CHROMESCALE=1 if [ "$1" != "" ]; then CHROMESCALE=$1 fi echo "" echo [load the grid] docker-compose up --scale chrome=$CHROMESCALE -d echo "" echo [list the containers] docker ps
The first part of the script remove all the current container on the server and restarts the docker service as well. This can be performed only on a server that is dedicated to the Selenium grid and should not be performed if the grid is hosted on a server with other containers.
docker rm -f $(docker ps -a -q) > /dev/null | true command removes all containers whether active or not. it can be replaced with looking for the grid containers only (e.g. all containers named with grid_ prefix). The last part of the command makes sure that it will not fail the script and garbage the output of it if no containers are running currently.
service docker restart command is needed for resetting the VNC port-range to start with port 5900. I currently didn’t find any solution to the issue that after each reload of the grid the allocated VNC ports starts from the preciding port of the latest allocated port (e.g. if on first load it allocates 10 ports for 10 notes, ranged 5900-5909, on next 10 nodes load it will alocated ports 5910-5919). In such case it will be hard to track the VNC ports to connect to so restart of the service reset the ports to starts the allocation from port 5900.
For loading the Selenium-grid using the above docker-compose configuration, the following server requirements are needed:
- Docker daemon and cli installed.
- docker-compose installed.
- Resources for up to 101 (1 hub and 100 nodes) containers loaded and running along with up to 100 test-classes running at the same time.
- ports 4444 and 5900-5999 available.
The above setup were tested only on Linux (Centos and Ubuntu) servers.
Default timeout of 60 seconds wait time for allocated browser
The default timeout for a test to wait for an allocated browser is 60 seconds. Since we allocated only one browser per node (since of an issue with 2 tests running on overlapping browser on the same machine/node), we might encounter a use case where there are more tests than nodes.
var timeout = TimeSpan.FromHours(1); var driver = new RemoteWebDriver(new Uri(node), new BrowserFactory().Create().ToCapabilities(), timeout);
We set the timeout to 1 hour assuming that it will take much less for all tests to be done.
The solution is in the code. Any timeout settings in the hub or node level did not provied a solution to the above issue.
NTLM authentication from a dockerized Selenium-grid node browser
When running a test on a browser that runs on a Selenium-grid node, and the node is dockerized and located on a Docker daemon which is outside of the NTLM-based domain, while the web-application is installed on an IIS-server and requires an NTLM authentication to client outside the domain - the authentication pops-up an login dialogbox which the Selenium in Chrome can’t handle.
We tried to set a NTLM-based (CNTLM) proxy and connect it to a HTTP-Proxy (SQUID), but the issue were not solved and it even made it worse (valid credentials entered manually in the login dialogbox were not accepted).
The solution were to run the tests using a URL to the web-application which contains the authentication information, in the following format:
The establishment of a Docker-and-Chrome-based Selenium-grid for web-application-tests is done using:
- docker-compose with minimum configurations
- bash-shell script for automation and safe load of the grid
- Parallel tests run with code-level-solutions to authentication and timeouts issues