In the previous article we went over a load test whose data could be used to choose a load-appropriate server. In the course of the testing, we would publish a stream on one WCS, and we would pick up that stream several times using a second WCS. The acquired results could be used as a basis for decisions on server operability.
Some would (justly) have concerns regarding the possible biases in such a test — after all, one of our servers was used to test another one of our servers. Could it be that we were using a specially optimized code that skewed the results in our favor?
It is true that the end user will not use a second server to watch streams. The end user will watch them in browser. This is why, seemingly the simplest and most logical way would involve manual testing: open a browser, open a tab with the player, specify the stream name and click "Play;" and then repeat 1000 times. All that's left is to find a guinea pig a volunteer tester and a PC that could handle 1000 tabs with video. Alternatively, and more realistically, the same test could involve a group of people and multiple PCs. Or we could just use scripts.
In this article we are going to discuss another testing method, one that uses a headless browser. We will compare the test results with the results from the stream capture-based testing.
You can't leave your head on
Нeadless-browser is something without a head. In the context of the frontend, it is an indispensable developer tool with which you can test code, check the quality and consistency of the layout, programmatically create scenarios of user interaction with the site, followed by fixing the results of these scenarios for use in tests.
Headless Chrome) is a full-featured browser without a graphical interface, which means it draws everything in memory. Headless Chrome is faster and uses less memory than a regular browser.
The last statement seems like a contradiction. But the reduction in memory usage is achieved by the absence of a graphical component. The headless browser has no real content rendering, which means that it does not need to render illustrations weighing several gigabytes, with which modern websites are often flooded. At the same time, the headless browser will download all the content from the web page, just like a regular browser.
Headless Chrome is software-driven using an API and can be installed on "pure" Linux. You just need to install the package, and the browser will work out of the box, just like its brother with a head – Google Chrome.
Load testing using Headless Chrome will be closest to reality because it simulates real users connecting to WCS using a browser.
So here we go.
WebRTC load test in a headless browser
Server under test
We will install on the server with the following characteristics
2x Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz (a total of 24 cores, 48 streams);
192GB RAM;
2x 10Gbps
a WCS and perform work to prepare for operation in production. Read more in the article and in documentation.
In the standard Two-way Streaming example, we published the stream from the virtual camera titled "stream1".
That's all. Now leave the server to wait for the load test.
Testing server
A server with Ubuntu 20.04 OS was used in testing
Characteristics:
2x Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz (a total of 24 cores, 48 streams);
192GB RAM;
2x 10Gbps
Setup and testing process:
1. Install Xvfb, Xorg for working with a virtual output device:
apt-get install xvfb -y
apt-get install x11-xkb-utils xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic xserver-xorg-core xserver-xorg-video-dummy alsa-base -y2. Configure the virtual monitor – edit the configuration file /usr/share/X11/xorg.conf.d/xorg.conf:
Section "Device"
Identifier "Configured Video Device"
Driver "dummy"
Option "ConstantDPI" "true"
VideoRam 192000
EndSection
Section "Monitor"
Identifier "Configured Monitor"
HorizSync 31.5-48.5
VertRefresh 50-70
Modeline "1600x1200" 22.04 1600 1632 1712 1744 1200 1229 1231 1261
Modeline "1600x900" 33.92 1600 1632 1760 1792 900 921 924 946
Modeline "1440x900" 30.66 1440 1472 1584 1616 900 921 924 946
ModeLine "1366x768" 72.00 1366 1414 1446 1494 768 771 777 803
Modeline "1280x1024" 31.50 1280 1312 1424 1456 1024 1048 1052 1076
Modeline "1280x800" 24.15 1280 1312 1400 1432 800 819 822 841
Modeline "1280x768" 23.11 1280 1312 1392 1424 768 786 789 807
Modeline "1360x768" 24.49 1360 1392 1480 1512 768 786 789 807
Modeline "1024x768" 18.71 1024 1056 1120 1152 768 786 789 807
Modeline "768x1024" 19.50 768 800 872 904 1024 1048 1052 1076
EndSection
Section "Screen"
Identifier "Default Screen"
Monitor "Configured Monitor"
Device "Configured Video Device"
DefaultDepth 24
SubSection "Display"
Depth 24
Modes "1600x1200" "1680x1050" "1600x900" "1400x1050" "1440x900" "1280x1024" "1366x768" "1280x800" "1024x768"
EndSubSection
EndSection3. Install Headless Chrome
wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add -
echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google-chrome.list
apt-get update
apt-get install google-chrome-stable4. Download and unpack the load testing scripts. An archive with load testing scripts is available for downloading here.
wget https://flashphoner.com/downloads/docs/2017/xload.tar.gz
tar -xvzf xload.tar.gz5. Go to the directory in which the archive was extracted
cd xloadThe directory contains:
player with Web-player scripts
start_xorg.sh - script for running Xorg
loadtest.sh - script for running tests
The following parameters can be passed to the loadtest.sh script:
required:
url - URL of the player page with the necessary parameters
or
urlsfile - the path to the file containing several such URLs, which will be traversed in order
optional:
stressrate - interval of adding a new subscriber in milliseconds;
ttl - subscriber lifetime in seconds;
maxsubscribers - maximum number of subscribers.
6. Run testing
./loadtest.sh -url http://Your.WCS.server:8081/client2/examples/demo/streaming/player/player.html?streamName=stream1&autoplay=true -maxsubscribers 100 -stressrate 500 -ttl 600The screenshot below shows the test result:
Unfortunately, because of the heavy load on the testing server it was not possible to connect to the test 1000 subscribers. Chrome, although headless, but took up all available CPU and RAM resources, with only 50 subscribers, as you can see in the CPU Load Averrage and memory consumption graphs. A long-lasting CPU Load Averrage value of more than 100 points indicates a high load on the processor. The RAM of the testing server also turned out to be occupied by more than 50%. And if the RAM resource was still available, the CPU load did not allow to reach the planned number of spectators.
In this test it was possible to connect a little more than 50 subscribers. In this case, there is no heavy load on the WCS server.
CPU Load Averrage value does not exceed 1 point. Pauses in the operation of Z Garbage Collector did not exceed 3.5 milliseconds. There were no degraded streams.
WebRTC load test "from WCS to WCS"
The testing methodology is discussed in detail in the article and in documentation
We will use two servers with the following specifications to test using stream capture:
2x Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz (a total of 24 cores, 48 streams);
192GB RAM;
2x 10Gbps
Let's launch load testing using the Console web app:
1. On the first server, open the Console app via HTTP http://your.WCS.server.name:9091/client2/examples/demo/streaming/console/console.html
2. Specify the domain name or IP address of the first server and click "Add node." This will be the server under test, which will be the source of streams. Then similarly connect the second server, which will simulate subscribers and capture streams.
3. For the first server, run the standard Two-way Streaming example and publish the stream from the web camera. The stream name can be anything.
4. In the Console app, select the second server, click 'Pull streams' button, and set the test parameters:
Choose node — choose the first server;
Local stream name — specify the name of the stream on the testing server in which the stream from the one being tested will be captured. (index corresponding to the number of the captured stream will be added to the stream name)
Remote stream name — specify the name of the stream published on the server under test;
Qty — specify the number of spectators (for our test — 1000).
5. Then press the "Pull" button to start the test:
The test result is on the screenshot below:
When testing using stream capturing, we managed to reach the planned number of viewers (1000 subscribers). The graphs show the increased load on the WCS server being tested.
The test using the second WCS server cannot be considered completely independent. We test our server using the second server of the same type. And, as it was already mentioned above, this variant is very rare in practice.
Despite the fact that we were not able to reach the estimated load of 1000 viewers when testing with Headless Chrome, this test clearly shows that with a small number of real subscribers there is no need to use powerful hardware to host WCS, and you can save some money when buying or renting a server.
Happy streaming!
