Video is taking over. Until 2029, the compound annual growth rate of video streaming is close to 20%, as reported by Fortune Business Insight. Yet, the live experience of 2022 is not nearly as good as it should be, and it’s partially due to the inadequacy of established high-quality video codecs supported by the majority of devices.
Alliance for Open Media
In a joint effort to develop a royalty-free, alternative video codec, the Alliance for Open Media was formed, involving practically every influential tech company. Its founding members: Google, Intel, Microsoft, Meta, Amazon, Apple, Huawei, Arm, Cisco, Mozilla, Nvidia, Netflix, Tencent, and Samsung, are all heavy hitters, accompanied by many more promoting members.
The Alliance for Open Media set out with the goal to create an open and royalty-free codec for the future of live streaming content and video on demand. The collaboration around AV1 centers around four aspects: a royalty-free ecosystem, patent-review process and legal defense fund, cutting-edge technology, as well as a collaborative, open-source development. As such, the nature of this collaboration is distinctively different, steering away from the licensing aspect, previously inhibiting the broad-scale adoption of any codec. Today, most codecs have a vast array of patents, with the price for licensing vary depending on usage, volume, and revenue. AV1 is set out to break with this tradition.
Pushing H.264 off the top
AV1 is not only a matter of facilitating licensing – it also transforms performance at large. The compression of AV1 is around 60% more effective compared to H.264, which fundamentally improves the viewing experience and offers next-level scalability. Just like any other prevailing codec, AV1 uses lossy compression, and as a result, video quality degrades. The counterpart, lossless compression, retains the quality of the initial capture but without any significant bitrate reduction. So naturally, live streaming of uncompressed high-quality video is unrealistic simply because it yields a higher bitrate which consumes more bandwidth. Using lossy compression significantly reduces the necessary bitrate without the quality degradation being perceivable to the viewer, which opens the possibility to live streaming high-quality video. There are, however, complexities around the encoding time of AV1, which has been the major challenge since the codec’s initial release. Without fast encoding, AV1 becomes futile in any use case requiring fast delivery. NETINT Technologies’ Quadra VPU cards support real-time AV1 hardware encoding, but there’s a lack of software-based alternatives. And in relation to H.264, it’s mainly the question of encoding speed that put a halt to AV1 in the ultra-low latency space.
As of today, H.264 (AVC) is the most standardized codec for the web, widely applied in live streaming, offering a high level of flexibility as it effectively processes different qualities, bitrates, and container formats. The introduction of H.264 presented a significant advantage over previous codecs in terms of compression efficiency and speed, allowing for high video resolutions at reduced bandwidth and higher quality at lower bitrates. But even so, most live streaming is still being done at low quality, especially when considering the rising standards of today. A 4K frame is practically four times the size of a 1080p frame, which proves challenging for most codecs, H.264 included, as there is a lot more data per frame to process and deliver. Its successor, H.265 (HEVC), was introduced to tackle the issue of rising quality with more efficiency. H.265 reduces the required bandwidth and cuts down on bitrate. However, the increased efficiency demands far more processing power for both encoding and decoding, which limits its utilization. The adoption has also been slow due to more complex royalty costs, leading to poor browser support. As a result, H.264 has remained the go-to codec for web distribution. Now, video quality is more pressing than ever, and for live streaming, the expectation of sub-second glass-to-glass delivery adds to the challenge. And it’s in this context that AV1 truly comes into play.
A compression-efficiency improvement of 60% over H.264 opens for live streaming at 4K (8K, too, in fact), even at ultra-low latency. But more notably, it becomes achievable even in regions with low bandwidth, which is a massive advantage over competing codecs. A side-by-side live demo comparison between AV1 and H.264 at the same bitrate is found at comparison. Versus both H.265 and VP9, AV1 remains solid with around 30% more effective compression.
AV1 adoption
Just as with any preceding codec, slow adoption is anticipated, and AV1 is no exception. But the promising codec is constantly gaining new ground. OTT services like Netflix and Youtube have both adopted AV1 to future-proof their offerings. Besides, applying AV1 means steering away from the royalty costs associated with both H.264 and H.265. However, as encoding the entire catalog of Netflix would be rather extensive when the number of AV1 capable devices is still limited, Netflix mainly uses AV1 encoding for their most popular titles, and with convincing results. Improving the viewing experience is ultimately Netflix’s focus and utilizing AV1 made it easier for viewers to receive higher quality on lower bandwidth. Thus, the duration of 4K content being streamed increased without negatively impacting the rebuffer rate.
In terms of decoding support for AV1, dav1d was released as an open-source cross-platform AV1 video decoder, created in collaboration between the communities of VideoLAN and FFmpeg. The decoder runs on macOS, Android, iOS, Windows, and Linux and is also sponsored by the Alliance for open media. As for TVs, decoding relies on hardware accelerated solutions. Samsung is one of AOM’s founding members and initiated adding AV1 support to its devices early on. LG, as a promoting member of AOM, has since been following Samsung’s example along with Panasonic and Philips, among others.
The ecosystem for AV1 will continue to prosper as AV1 support is steadily growing. As reported by Streaming Media Magazine recently, Vindral utilized AV1 in a 4K 10-bit HDR live stream at ultra-low latency and synchronized playout, showcasing that the future of live streaming is approaching fast, with AV1 expected to push H.264 off the top.