RealSprint AB
Döbelnsgatan 19
90330, Umeå Sweden
info@realsprint.com
VAT SE556946531201
This page contains useful guides and information on how to perform proper tests and evaluations of different live streaming functionalities. To properly assess the performance of a CDN, the main challenges in video delivery must be considered, namely sync, resilience, latency, and quality.
Network performance is usually non-perfect and introduces latency, packet loss, and jitter – regardless of whether the viewer is using a mobile connection or home wifi. The connection the viewer is using is often called the last mile and is outside of your control. Therefore, any solution must mitigate the conditions that could reasonably be expected as this is what your viewers will see, or at least a reasonable approximation.
While the examples below are utilizing the Vindral Demo, we strongly recommend carrying out a proper test for all vendors considered to compare stream performance under realistic network conditions. The Vindral demo is located at demo.vindral.com.
Evaluate all vendors identically.
Check so that the ingest workflow is not creating different latencies or additional transcoding steps for different vendors.
If you are seeing anything other than sub-second latency when evaluating Vindral, please reach out to us, and we will help you identify the culprit.
Step 1
Start by going to demo.vindral.com.
Step 2
Choose a video quality in the quality-control section, as well as buffer size. When choosing 4K quality, the 4K mark at the bottom right of the stream becomes visible when activated.
Step 3
Click any of the latency tests to see the measured latency. The visual test will inject your visitor ID (shown at the top right corner) into the stream while the audio test will inject a beep sound.
How it works:
The latency test injects your visitor ID into our encoder in Sweden, which then inserts a graphic element with the ID, into the stream. Vindral’s frame-accurate cue point system then catches the event when your ID is displayed in the stream to measure the latency. Note that it is not a web page overlay, but a part of the live video.
Step 4
As it is important to emulate realistic network conditions, make sure to use different buffer sizes, and watch how the latency adapts.
Step 1
Go to demo.vindral.com on at least two different devices.
Step 2
If on a smartphone, turn off the wifi, as that will force the signal to take another route compared to the computer.
Step 3
Make sure you have the same buffer settings on all devices and listen to the synchronization in audio playout.
Step 4
Compare the time on the in-stream clocks and the picture, for an accurate value on time difference.
Step 5
Make sure to test competing technologies in the same way. Most CDN-providers have a live demo which you can enter on your own. If not, they will share a test channel upon request. This test is important, as many live applications require a high-level of synchronization and compatibility.
This quick guide will focus on a particular method of testing stability (resilience and quality) by shaping the network to emulate realistic conditions.
Step 1 – Get a network shaper
A network shaper can be used to induce a reasonable amount of jitter, latency, and packet loss.
If you are using a PC with Windows, follow this link to download NetBalancer. We recommend using the paid version as it enables the testing described in this guide. Install NetBalancer in the location of your choice.
If you are using macOS, the free software XCode has an add-on called Network Link Conditioner, which can be used to induce delay and drops. However, adding jitter is not possible so using NetBalancer on a PC is strongly recommended instead as jitter is crucial to accurately emulate network congestions. Download XCode, and find the add-on in “Additional Tools for XCode” on the Apple Developers page.
Step 2 – Launch the shaper
Run the browser of your choice and visit the stream player that you wish to evaluate. With the browser running, launch NetBalancer. You will see a list of processes.
Step 3 – Identify the process
Identify the process running the stream. E.g., entering “chrome” into the search area will filter for the running instances of Google Chrome. If you are consuming the stream in one Chrome window, it will be obvious which process should be run through a shaped network.
Step 4 – Configure the shaper
Double-click on the identified process to bring up the priority window. Choose “Custom” in the download drop-down and specify whichever values you find interesting. From a 2020 report on cellular networks, Verizon and AT&T were reported to experience around 80 ms of delay, 10 ms of jitter, and up 1,5% drops.
There are of course home wifi networks and scenarios where the user is on the move, where any of these values will be higher.
In general, recommended configurations to induce congestion using a network shaper are:
2Mbps or higher stream.
RealSprint AB
Döbelnsgatan 19
90330, Umeå Sweden
info@realsprint.com
VAT SE556946531201
