Co-Authored By Dane Kottke
In their quest to provide the best viewing experience at reasonable costs, OTT video distribution companies have traditionally relied on adaptive bitrate (ABR) streaming. In ABR streaming, each source video is scaled to multiple frame resolutions and each frame resolution is encoded at one or two bitrates, with the collection of frame resolutions and bitrates termed an encoding ladder. The streaming client then determines, based on network conditions and device playback capabilities, the “best” frame resolution and bitrate at which to stream the video. The content provider needs to encode and store each source video at all the frame resolutions and bitrates specified by the encoding ladder, providing the “best” encoding on demand to the streaming client.
The Challenges of Adaptive Bitrate Streaming
A common problem with standard ABR encoding ladders is that they are too rigid, encoding all videos for each frame resolution at one or two bitrates, regardless of the content in the videos. For videos with visually complex content such as sports or action movies, the encoding bitrate specified by the encoding ladder might be too low, resulting in poor visual quality on playback. For simpler videos such as animation or talking heads, the bitrate from the encoding ladder might be too high, resulting in wasted bandwidth (because acceptable quality could have been achieved with fewer bits).
To address this rigidity in traditional ABR streaming, many encoding companies are now proposing implementations that adapt the ABR encoding ladder based on information about the video itself. Such implementations are termed content-adaptive encoding (CAE). CAE potentially benefits ABR streaming in three ways. For simpler videos, CAE may adjust the encoding ladder bitrates lower, enabling viewers to stream the same video at a higher frame resolution with the same amount of bandwidth, thereby improving quality of experience, or to stream the same video at the same frame resolution but with lower bandwidth consumption. For more complex videos, CAE could allocate bits more effectively during the encoding of the video, thereby improving the visual quality of the encoding while maintaining the same encoding bitrate. And by adapting the encoding ladder to each video, CAE will likely reduce the number of “rungs” (frame resolution/bitrate combinations) on the ladder, lowering storage and encoding costs for the content provider.
Internal vs External CAE
It is important to distinguish two different methods of applying CAE. First, CAE may be applied internally to the encoder by adjusting the encoding decisions within the encoder based on perceptual considerations (considerations of how the human visual system perceives the video data). Second, CAE may be applied externally to the encoder by adjusting encoding parameters (such as the encoding bitrate) based on characteristics of the video data.
The internal application of CAE represents an encoding improvement, while the external application of CAE represents an encoding workflow improvement. Both types of CAE applications can save bandwidth by using fewer bits on less-demanding content or improve quality by using more bits on complex content, but they achieve those goals differently. By analogy, one might compare the use of CAE to the challenge of improving a car’s driving experience. Internal application of CAE would be analogous to swapping out the engine for a more efficient engine. External application of CAE would be analogous to adaptive cruise control, which throttles the engine based on traffic, road, and weather conditions.
EuclidIQ CAE Solutions
EuclidIQ has developed both internal and external CAE technologies as part of its overall CAE solution. Our internal CAE solution is an encoding enhancement that uses perceptual quality optimization (PQO™) technology to determine on a block-by-block basis which blocks need less quantization (for higher encoding quality) or more quantization (for greater bitrate savings). The quantization adjustment is determined by a unified importance map that makes use of both spatial and temporal inputs to predict which blocks in a frame or group of frames is likely to be most noticeable to a human observer. PQO has been successfully applied to H.264 and HEVC but is adaptable to any other block-based encoding standard, including VP9, AV1, and MPEG-2.
EuclidIQ’s external CAE solution is an encoding workflow enhancement that uses signal-adaptive bitrate estimation (SABRE) technology to determine the lowest bitrate at which our PQO-enhanced encoder can achieve a target quality level. While many companies have proposed external CAE solutions, two major components separate our SABRE solution from others. First, our complexity metric is tied to a quality metric based on mean opinion score (MOS) data collected from our unique subjective testing methodology. While other methods try to approximate MOS (subjective quality ratings from human observers), our method is designed to achieve target MOS levels. Second, SABRE works in tandem with our PQO-enhanced encoder to maximize bitrate savings while achieving the desired MOS-based quality level. While other companies use variations of standard encoding that are not designed to optimize perceptual quality, our PQO technology improves perceptual quality internally to the encoder.
We combine both PQO and SABRE into a cloud-based, scalable CAE solution called Blackbox, and we’re excited to roll out the Blackbox solution in coming months. Powered by PQO (the better engine) Blackbox generates perceptually-optimized ABR bitstreams. Blackbox also uses SABRE (the adaptive cruise control) to determine the maximum bitrate savings that can preserve a desired, MOS-based quality level. With both PQO and SABRE implemented inside, Blackbox provides the best adaptive bitrate streaming “driving experience” for online content delivery operators.
To request a Blackbox demo, or to find out more about PQO, SABRE, or Blackbox, contact us at firstname.lastname@example.org.