Apple’s recent announcement at its Worldwide Developer Conference that it would support HEVC/H.265 in its next versions of iOS (iOS11) and macOS (High Sierra) resulted in a flurry of excited posts within the video compression community. Such a reaction was no surprise, as many companies had spent substantial amounts of time and resources developing HEVC encoding solutions, only to watch nervously as widespread adoption of HEVC was slowed by licensing and pricing uncertainties. Apple’s announcement ensures a foothold for HEVC in the all-important mobile device market. At EuclidIQ, where our business is innovative video compression solutions, we have been following HEVC and quietly performing our own testing with it for a while now, and now is as good a time as any to take stock of where we feel HEVC is and where we can help it to go. Here, then, are some of our observations to date about HEVC.
The claimed compression gains of HEVC over H.264 are real, but overstated.
From the time of its first widespread release in 2013, HEVC has been advertised as being able to provide the same encoding quality as H.264 at half the bitrate. This purported 50% compression gain of HEVC vs. H.264 has been stated so many times in so many different contexts that it is widely accepted as “truth” (as much as any statement can be considered “true” today). Yet a detailed (though not exhaustive) examination of six published compression performance comparison studies from 2012-2016 shows HEVC gains between 20% and 45% over H.264, with the average reported gain about 35%. As is well known, the amount of compression gain for one encoder versus another depends on and varies with several factors: (1) the type of content being encoded; (2) the encoding settings, including encoder bitrate; (3) the specific encoders in the comparison; and (4) the quality metric used in the comparison. In our own study comparing x265 versus x264 for a set of 20 HD (1080p) videos, with encoder settings including a combination of medium and slow presets, using bitrates near the video quality breakdown bitrate for each video, and using subjective mean opinion score as the quality metric (see our white paper for details), we found x265 to provide an average 39% gain relative to x264 over the 20 videos – in line with the findings of the other studies. We believe HEVC gains over H.264 are real and significant – just not 50%.
HEVC’s compression gains are content-dependent and context-dependent and come at the cost of increased computation time.
To understand why HEVC compression gains vary with content, it’s useful to review why HEVC should provide better compression than H.264. While HEVC also provides more sophisticated in-loop filtering and more intra-prediction modes than H.264, the most important source of HEVC’s compression gain over H.264 is its greater number and flexibility of inter-prediction modes. While H.264 is limited to 16×16 macroblocks and a limited number of subpartitions for inter-prediction, HEVC inter-prediction allows up to 64×64 coding units and more subpartitions accessible through a quadtree search. Why is this better? At its most basic level, video compression exploits temporal redundancy in the video data by “predicting” new regions of data to be encoded using similar regions in previously encoded frames – the better the prediction, the greater the compression. HEVC’s sheer number of possible inter-prediction modes, much greater than that of H.264, means that its search for a good prediction (motion estimate) is likely to wind up with a better answer than H.264. But HEVC’s inclusion of larger blocks in its inter-prediction scheme is especially helpful and produces larger compression gains when data is homogeneous, either spatially (with the presence of “flat” texture or color regions in the frame) or temporally (meaning that the video is “low-motion”). This also means that HEVC gains are likely to be higher for higher-resolution videos, because the greater pixel density of higher-resolution frames increases the spatial homogeneity of the video data. Indeed, multiple studies have shown that HEVC gains over H.264 are higher for 4K data than for 1080p data, and higher for 1080p data than for smaller frame resolutions. Conversely, HEVC is less likely to provide large gains over H.264 for videos that are either spatially or temporally complex, or for videos with smaller frame resolutions. HEVC’s greater number of prediction modes do come at the cost of greater computation time than H.264 because of the bigger search space. HEVC encoding will always be slower than H.264 encoding; it’s just a question of how much slower and whether the better compression of HEVC makes the tradeoff worth it. As with other encoders, running HEVC at “faster” presets or lower profiles to improve speed will reduce compression performance.
There is still room for significant improvement in HEVC encoding.
As noted above, there are certain types of content and encoding scenarios for which HEVC provides smaller gains relative to H.264 and its compression performance could be improved. In these scenarios, improving on compression performance requires a fundamental improvement in the compression approach. At EuclidIQ, we’ve been working for a while now on perceptual quality optimization (PQO™) as this new compression approach. Our version of PQO considers the way the human visual system views a video and modifies the encoder quantization accordingly on a block-by-block basis, redistributing the bits in a frame to encode the most important blocks in the frame with high quality while saving bits on the less important blocks in the frame. We have applied our PQO technology successfully to H.264 and have successfully integrated it with HEVC, with initial results showing improved compression performance over a reference, non-PQO HEVC implementation.
Another area of potential improvement for HEVC-based workflows is content-adaptive encoding (CAE).
CAE makes use of automatically-calculated quality measures to determine the lowest bitrate at which one can encode a given piece of content with a given encoder while maintaining a desired quality level. CAE is a hot area, and many companies have released their versions of it to help content providers with large video libraries to reduce their encoding footprint by identifying “simpler” content and encoding such content at lower bitrates. While CAE to date has been applied most often to H.264-based workflows, it can certainly be applied effectively to HEVC-based workflows to minimize the encoding footprint of HEVC encodings. We believe our version of CAE, called SABRE (signal-adaptive bitrate estimation), is superior to other solutions on the market because it is tied to subjective mean opinion score (MOS), complementing our PQO-based encoding. We will be writing much more about SABRE in the coming months.
To summarize, the HEVC train is leaving the station, and EuclidIQ will be on board with innovative encoding solutions for HEVC. To find out more about PQO and SABRE, contact us at email@example.com or visit our website.