Encoder enhancements

Frame threads are now decided based on number of threads specified in the --pools, as opposed to the number of hardware threads available. The mapping was also adjusted to improve quality of the encodes with minimal impact to performance.

CSV logging feature (enabled by --csv) is now part of the library; it was previously part of the x265 application. Applications that integrate libx265 can now extract frame level statistics for their encodes by exercising this option in the library.

Globals that track min and max CU sizes, number of slices, and other parameters have now been moved into instance-specific variables. Consequently, applications that invoke multiple instances of x265 library are no longer restricted to use the same settings for these parameter options across the multiple instances.

x265 can now generate a seprate library that exports the HDR10+ parsing API. Other libraries that wish to use this API may do so by linking against this library. Enable ENABLE_HDR10_PLUS in CMake options and build to generate this library.

The CSV log is now more elaborate with additional fields such as PU statistics, average-min-max luma and chroma values, etc. Refer to documentation of --csv for details of all fields.

x86inc.asm cleaned-up for improved instruction handling.

API changes

New API x265_encoder_ctu_info() introduced to specify suggested partition sizes for various CTUs in a frame. To be used in conjunction with --ctu-info to react to the specified partitions appropriately.

Rate-control statistics passed through the x265_picture object for an incoming frame are now used by the encoder.

Options to scale, reuse, and refine analysis for incoming analysis shared through the x265_analysis_data field in x265_picture for runs that use --analysis-reuse-mode load; use options --scale, --refine-mv, --refine-inter, and --refine-intra to explore.

Today Intel launched the next generation of Xeon processors, the Intel Xeon Scalable Processor Family (code-named “Purley”), based on the Skylake CPU architecture. The Intel Xeon Scalable Processor Family is a powerful new generation of 14nm chips which provide significant improvements over the previous generation of Xeon processors (Xeon E5 v4 and E7 v4, code named “Broadwell”), including many fundamental CPU architectural improvements, a much faster internal data transfer architecture (a mesh architecture with 2x the bandwidth instead of a ring architecture), AVX-512 vector processing, improved cache, and improved I/O architecture with six DDR4 memory channels and 48 PCIe lanes.

With x265 pushing the previous generation processors to the edge for memory bandwidth and threading, the benefits that these new Xeons provide for x265 users will be game changing. Our initial results with the latest build of x265 show a 67% average per-core gain for encoding using HEVC Main profile, and a 50% average gain with Main10 profile across different presets. In particularly, off-line encoding of 4K content is seeing tremendous benefits due to the higher memory bandwidth that the CPUs are able to utilize from cache and system memory. Intel’s Xeon Scalable Processor Family makes x265 and UHDkit the ideal option for a wider range of scenarios including both live and offline HEVC encoding, and they double the performance/cost you’ll get with our software-based encoding libraries. We’re also seeing significant performance improvements with x264 – roughly 40% higher performance per core on average.

As we enhance x265 to take advantage of the new technologies that these new processors bring to the light, including AVX-512, we expect that users of x265 will love the benefits that they see with these new Xeons. This even extends to the Core i9 (Skylake-X) consumer processor family, which are based on the same Purley architecture. Give them a spin, and let us know what you think!

Encoder enhancements

HDR10+ supported. Dynamic metadata may be either supplied as a bitstream via the userSEI field of x265_picture, or as a json jile that can be parsed by x265 and inserted into the bitstream; use --dhdr10-info to specify json file name, and --dhdr10-opt to enable optimization of inserting tone-map information only at IDR frames, or when the tone map information changes.

Enhanced HDR10 encoding with HDR-specific QP optimzations for chroma, and luma planes of WCG content enabled; use --hdr-opt to activate.

Ability to accept analysis information from other previous encodes (that may or may not be x265), and selectively reuse and refine analysis for encoding subsequent passes enabled with the --refine-level option.

Slow and veryslow presets receive a 20% speed boost at iso-quality by enabling the --limit-tuoption.

The bitrate target for x265 can now be dynamically reconfigured via the reconfigure API.

Haivision contributions to the x265 open-source initiative have pushed boundaries on quality and performance of live video streaming on Intel processors

MONTREAL, CANADA – APRIL 19, 2017 – At the 2017 NAB Show, Haivision will demonstrate a breakthrough in live 4Kp60 HEVC software-only performance video streaming, leveraging the unparalleled quality of x265 software encoding, while running at a performance level that was previously only possible with dedicated hardware. This demonstration will be presented by Haivision’s HaiGear Labs, the company’s technologies research group, at the Renaissance Hotel (suite Ren Deluxe – B) next to the Las Vegas Convention Center.

Through the use of commodity off-the-shelf processing capabilities, Haivision will showcase how x265 software encoding, running on readily available dual-socket servers from the Intel® Xeon® Processor E5-2600 v4 product family, addresses the growing demand for high-quality 4K video streaming. This development brings down the costs associated with encoding live 4K video and enables 4K video streaming on ubiquitous Intel cloud compute architectures.

The foundation for these video streaming innovations comes from the company’s four years of active involvement in the x265 open source project, a commercially backed initiative founded with the goal of producing the highest performance, most efficient HEVC/H.265 video encoder software implementation. Haivision is an original charter licensee of the x265 project and has made significant contributions to the x265 initiative through tight technology collaboration with MulticoreWare, the primary developer of the widely adopted open-source codec.

Haivision’s quality-to-performance in its live 4K HEVC demonstration leverages UHDKit, MulticoreWare’s extended encoding library built on top of the x265 HEVC encoder. By heavily investing in advancing the UHDKit for low-latency live encoding, Haivision has been able to push the boundaries on what has been possible in HEVC software encoding.

“Haivision has been an active contributor to x265 and UHDKit and has helped MulticoreWare push the envelope with regard to live encoding performance,” said Tom Vaughan, vice president, general manager for video, MulticoreWare. “Haivision’s numerous contributions are invaluable to every user of x265.”

“Haivision’s long-term association with MulticoreWare’s x265 project and our tuning of the UHDKit for high performance streaming on the Intel platform has enabled our customers to benefit from software-only or CPU/GPU balanced performance,” said Mahmoud Al-Daccak, chief technology officer, Haivision. “We will continue to pioneer and contribute to these development communities that rely on open-source initiatives to move the streaming video industry forward.”

As a pioneer in high performance streaming solutions, Haivision innovates in the areas of live hardware and software encoding/decoding, video stream transport and management. The company is dedicated to pushing the technology envelope, and fostering partnerships and collaboration within the industry to expand the ecosystems of performance video that its customers depend on. To learn more or book a demonstration, visit haivision.com/nab.

About Haivision
Haivision, a private company founded in 2004, provides media management and video streaming solutions that help the world’s leading organizations communicate, collaborate and educate. Haivision is recognized as one of the most influential companies in video by Streaming Media and one of the fastest growing companies by Deloitte’s Technology Fast 500. Haivision is headquartered in Montreal and Chicago, with regional offices located throughout the United States, Europe, Asia and South America. Learn more at haivision.com.

MulticoreWare, the developers of x265, will be at the National Association of Broadcasters convention in Las Vegas, Nevada, April 24-27th. You can find us in the South Hall, Upper, booth SU14002. We’ll be demonstrating the latest advances to x265, and our premium video encoding framework, UHDkit (which includes both x264 and x265, plus many extended capabilities). If you haven’t registered, you can get a free guest pass by using Guest Pass Code: LV6642. Contact us through our x265 Facebook page if you would like to schedule a meeting.

Version 2.3

Release date – 15th February, 2017.

Encoder enhancements

New SSIM-based RD-cost computation for improved visual quality, and efficiency; use --ssim-rdto exercise.

Multi-pass encoding can now share analysis information from prior passes (in addition to rate-control information) to improve performance and quality of subsequent passes; to your multi-pass command-lines that use the --pass option, add --multi-pass-opt-distortion to share distortion information, and --multi-pass-opt-analysis to share other analysis information.

Major initiative designed to rapidly accelerate widespread HEVC (H.265)/UHD adoption on mobile devices and personal computers

BOSTON, Nov. 22, 2016 /PRNewswire/ — HEVC Advance, an independent licensing administrator, today announced a major software policy initiative to rapidly accelerate widespread HEVC/UHD adoption in consumer mobile devices and personal computers. Under the software initiative, HEVC Advance will not seek a license or royalties on HEVC functionality implemented in application layer software downloaded to mobile devices or personal computers after the initial sale of the device, where the HEVC encoding or decoding is fully executed in software on a general purpose CPU. Examples of the types of software within the policy include browsers, media players and various software applications.

According to Peter Moller, CEO of HEVC Advance, “We are very pleased to offer this initiative. A critical goal of HEVC Advance is to encourage widespread adoption of HEVC/UHD technology in consumer devices. While HEVC technology implemented in specialized hardware circuitry provides the best and most efficient user experience, there are millions of existing mobile devices and personal computers that do not have HEVC hardware capability. Our initiative is tailored to enable software app and browser providers to include HEVC capability in their software products so that everyone can enjoy HEVC/UHD video today. I’d like to specifically thank Tom Vaughan at MulticoreWare for his guidance in our development of this initiative.”

Tom Vaughan, VP and GM of Video at MulticoreWare remarked: “We have invested heavily in the development of our HEVC codecs and associated libraries. From the start, HEVC Advance worked very hard to listen, understand and develop solutions to respond to the market’s concerns about HEVC Adoption. We are thankful for their efforts and believe this initiative will encourage and facilitate software developers, web service and content providers, mobile device manufacturers and others to take advantage of the tremendous competitive advantage that HEVC provides as quickly as possible.”

Subject to certain exceptions/conditions. Full details on the software policy initiative are available on the HEVC Advance website: http://hevcadvance.com

About the HEVC Advance Licensing ProgramIn addition to the above described software initiative, HEVC Advance offers incentives to encourage consumer device manufacturers to include HEVC functionality at initial sale. For example, HEVC Advance only seeks one device royalty for a consumer device, even if that device includes multiple HEVC decoders or encoders at the time of the initial sale (subject to limited exceptions). Therefore, any number of HEVC software products may be included in a consumer device at initial sale without incurring additional royalties, providing the applicable device royalty has been (or will be) paid by the consumer device manufacturer. In addition, even after initial sale, HEVC Advance Licensees can receive waivers for device royalties on HEVC software products if these software products are installed on a consumer device for which the applicable device royalty has been (or will be) paid.

For more details or questions, please contact licensing@hevcadvance.com. To request a license or for a detailed summary of the HEVC Advance licensing structure and incentive program, please visit the HEVC Advance website: http://hevcadvance.com

About HEVC AdvanceHEVC Advance is an independent licensing administrator company formed to lead the development, administration and management of an HEVC/H.265 patent pool for licensing essential patents. HEVC Advance provides a transparent and efficient licensing mechanism for HEVC patented technology. For more information about HEVC Advance, visit hevcadvance.com.

Clogged pipes Annual global Internet traffic is no longer measured in Megabytes, Gigabytes, Terabytes, or even Petabytes. This year, global IP traffic will be more than one Zettabyte (a trillion billion, or 1,000,000,000,000,000,000,000 bytes)[1]. By far the biggest driver of IP traffic growth is video. Today, more than 70% of all IP traffic is video. By 2020, video will consume an estimated 82% of 2.3 Zettabytes of IP traffic[2]. Video has become strategic priority for many companies, including leading social media and messaging services. In addition to the explosive growth in on-demand video, we are in the middle of a transition from high definition video to ultra high definition (UHD) video, with higher resolution, higher color accuracy, and higher dynamic range. UHD content is forecast to grow to 20.7% of global IP video traffic by 2020. Internet bandwidth is never as cheap or plentiful as we would want, so there is a serious need for more efficient video compression. Fortunately, a new standard is available which can deliver identical quality to consumers using half the bandwidth of previous video compression standards. Imagine a technology that could theoretically free up more than 30% of worldwide IP bandwidth, reducing congestion and allowing that bandwidth to be used to deliver a much higher quality of experience. This technology is ready, mature, and optimized, but it’s hardly being utilized.

Twice the Efficiency HEVC, also known as H.265, is a new video coding standard, ratified by the ITU and ISO in January 2013. Many leading technology companies and research groups contributed to the new standard, including Microsoft, Apple, and Samsung. These organizations contributed their researcher’s time, efforts, and intellectual property to create a significantly more powerful video compression standard, and the results are outstanding. A 1080P movie that required 6 Mbps to be delivered in high quality with AVC/H.264 (today’s most widely used video compression standard), typically only requires about 3 Mbps to be delivered in the same quality with HEVC/H.265. The improved efficiency of HEVC results in significantly higher quality at any fixed bandwidth. So, for anyone trying to watch a video over a congested or bandwidth limited Internet connection, HEVC is able to deliver much better picture quality than AVC.

Ready to Rock 3 ½ years after the standard was ratified, HEVC hardware and software implementations, full-featured, efficient and widely available. HEVC hardware decoders are built in to the latest smartphone SOCs, high-end PC graphics chipsets, and most high-end televisions. Hardware HEVC encoders are embedded in smartphones, PCs, cameras and broadcast encoders. The x265 HEVC encoder software is available under the GPL v2 open source license, and it’s been incorporated in FFMPEG, VLC, and dozens of open source and commercial applications. There are many other commercial HEVC encoder and decoder software implementations. HEVC is ready to rock and roll. Sadly, HEVC is not in widespread use. There isn’t a whole lot of HEVC encoded content, or HEVC enabled software applications reaching end-users. The exception to this rule is for 4K content streamed by Netflix, Amazon and other movie streaming services to the latest generation of 4K TVs. But although billions of devices are capable of supporting HEVC, in almost every application we continue to use AVC/H.264, which requires roughly twice the bit rate (and file size) to achieve the same quality as HEVC. Consumers and enterprises aren’t getting the benefits of this incredibly powerful new technology. There is no technical reason that HEVC isn’t being used for most video applications. The hold up is due to the cost and uncertainty associated with HEVC patent licensing.

Patent Licensing Impasse The HEVC video coding standard was developed by a team of experts jointly managed by two standards bodies; ITU and ISO. It benefits consumers when multiple companies collaborate to develop a new industry standard. Standards development involving many contributing organizations reduces the overall cost of developing a new technology, can produce a better overall result by combining valuable, proprietary innovations from many contributors, while providing compatibility across many different company’s products. As per the policies of the ITU and ISO, the companies involved in setting the HEVC standard must disclose any patents they have or may file on the techniques that they contributed to the standard, and they pledge to license their patents on Reasonable, and Non-Discriminatory (RAND) terms.

Thirteen years ago, the majority of companies that developed AVC agreed to license their patents through one organization, MPEG LA. Today, some of the companies that developed HEVC have agreed to make their patents available through MPEG LA, and some have formed a 2nd patent pool, called HEVC Advance. A number of important companies with HEVC patents have not yet joined one of the patent pools. This fragmentation, combined with total patent royalties that are potentially many times greater than video solution developers currently pay for AVC, has caused many potential HEVC adopters to hold off for now. We’ve all got too much invested, and HEVC is far too good to let the patent licensing situation delay adoption much longer.

Available Options HEVC is the best video compression standard available today, and it is likely to remain the best video compression standard available for the next several years. That isn’t to say that it doesn’t have any competition. Google’s VP9 is a very capable video codec. Google, Microsoft, Netflix, Amazon, Intel, Cisco and Mozilla have formed the Alliance for Open Media (AOM), a group dedicated to developing a next-generation video compression standard. Some may have thought that the AOM was organized simply to gain leverage over HEVC patent negotiations. The work that the AOM is doing makes it clear that it is a very serious project. The AOM represents the merger of three royalty -free video codec development efforts (Google’s VP10, Cisco’s Thor, and Xiph/Mozilla’s Daala formats). It has the backing of many industry leaders, including Adobe, ARM, AMD, NVIDIA and Vidyo. More leading companies will join in the coming months, and I expect the AOM to be successful in their goal of establishing a widely adopted, royalty free next-generation video standard. But it will take time to finalize this standard, and more time to develop and deploy implementations. There is a window of opportunity for HEVC to achieve the widespread, pervasive adoption we all want to enjoy, in order to ensure a long shelf life and a good return on the massive combined industry investment.

Moving Forward I manage the video software business for MulticoreWare, developers of x265; the world’s most widely adopted HEVC encoder. This has enabled me to build strong relationships with many of the leading adopters of HEVC, including movie studios and post production companies, semiconductor companies, broadcast and streaming video encoding system vendors, web video streaming services, web video processing services, device OEMs, and major cloud and device platform owners. I have had many discussions on the topic of HEVC adoption with key players, and I’ve found that all involved are intelligent, reasonable people, but their perspectives differ widely. It’s clear from the people I’ve spoken with that there is a big gap in the patent licensing discussions. It’s time for a détente. HEVC is poised for breakout success, but it won’t happen unless we see some significant improvements in the patent licensing situation. I’m optimistic that most organizations involved are recognizing this, and are ready to find ways to accelerate adoption.

Some believe the only way to solve the problem we see today is to move to royalty-free standards developed outside of international standards bodies. While that can work well, it’s not the only way a great standard can be developed. I believe that it’s perfectly reasonable for inventors to be given an incentive to contribute their valuable developments to a global standard, and to be compensated for their contributions. Every company recognizes that intellectual property; whether it is a movie, TV show, software, hardware or invention; takes a lot of time, money, and unique talent to develop. So it’s unreasonable to expect that the only way to develop new standards is to force contributors to donate their IP to a new standard royalty free. However, in standards setting organizations, contributors shouldn’t come to the party with the goal of earning a big windfall on their R&D investment. If that is your goal, you might be ‘persona non grata’ when it’s time for the next party. A reasonable ROI is fine, but patents for techniques contributed to a technical standard are RAND encumbered, and must be reasonably priced.

So, how can the patent licensing situation be resolved? There are several possibilities. The stalemate could continue, with many companies sticking with H.264 or VP9 until AV1 is available. In this scenario, most everyone loses. It could be resolved in court. Of course, a legal battle is the least attractive option for all concerned. The best option is for patent holders, including those on the sidelines, to come together and compromise and offer a licensing solution acceptable to the vast majority of potential licensees.

It’s a Web and Mobile World AVC/H.264 was finalized as a specification in March, 2003, roughly 10 years before HEVC/H.265. Ten years ago, at this stage in the life cycle of AVC/H.264, things were quite different. AVC/H.264 patent holders pooled their patents together, enabling licensing through a single organization, called MPEG LA. In terms of video distribution, was a “set-top box and DVD” world. The Blu-ray Disc format was competing with HD-DVD to become the next generation optical disc standard. The VC-1 video codec (developed by Microsoft and others) was competing with AVC, and both were supported by HD-DVD and Blu-ray Disc. VC-1 patents were offered under very competitive terms. The iPhone did not exist. YouTube was less 8 months old, offering 320 x 240 pixel videos. Netflix streaming, Amazon Video and Hulu were not available. Facebook was a closed social network for college students. Today, we live in a web and mobile world, where Internet video streaming is no longer a science experiment; it’s the primary method of accessing video content for billions of people. Support from the leading web browsers and mobile platforms is essential for any new video standard to succeed.

Key to the success of AVC/H.264 was adoption by all of the leading device OEMs and web browsers. Once H.264 was supported natively on every popular computing device, software developers could utilize it without an additional patent license or added cost. Today, popular web services don’t charge for the client software required to access the service. Instead, you just access the service through your browser, or download a free client app. HEVC will not achieve critical mass if streaming video services, social media services and video conferencing services can’t continue to provide free client apps. Web browsers will not support HEVC if there is an added royalty cost. Unless they are content to cede the consumer PC and mobile video compression market to royalty free codecs, HEVC patent holders need to recognize the reality of today’s technology ecosystem, and adjust their license terms to compete successfully. Treating software the same as hardware is a mistake. Hardware is the engine, but software and content are the fuel. Without the fuel, the engine won’t start, and it won’t run.

Proposal To accelerate HEVC adoption, I propose that HEVC patent licensors agree to the following principles;

All HEVC patent holders should make their patents available through a patent pool

Ideally, all patent holders should join one patent pool

Only one reasonable royalty should be paid per device

Software decoding on consumer devices must be royalty free

Software encoding on consumer devices must be royalty free

Content distribution must be royalty free

There must be a reasonable cap on total royalties owed for HEVC implementations

The HEVC standard was first ratified 3 ½ years ago. It’s time for all HEVC patent holders to make their patents available under reasonable terms, through a single patent pool. Holdouts need to fulfill their obligation to license their patents on reasonable and non-discriminatory terms. Ideally, all patent holders should join a single patent pool, to eliminate redundancy, and to insure more reasonable royalty rates. It much easier for licensees to track and maintain compliance if they don’t have to sign two or more patent license agreements.

I’m going to hold off suggesting the ideal compromise for patent royalty rates. This is something that needs to be worked out in discussions between licensors and licensees. Patent license revenues are a function of both royalty rates and adoption rates. At this point, patent licensors should be concerned with accelerating adoption.

Royalty free software decoding would immediately enable more than a billion legacy devices to support HEVC playback, massively accelerating adoption. This proposal would enable all web browsers, social media and video player apps to immediately add support for HEVC. Royalty free software encoding would mean that video chat, video conferencing and video sharing services wouldn’t hesitate to significantly improve the quality of their services. For clarification, this would not be a workaround for device manufacturers to offer HEVC support. Under my proposal, only software distributed to hardware devices by third party companies after the first sale of the device would be considered royalty free.

Native Hardware Support Of course, software decoding and encoding is just a good first step to unleash the ecosystem. Consumers will recognize that dedicated video decoding hardware reduces power consumption, drastically improves battery life, and ensures reliable playback. Hardware accelerated encoding is desirable on battery powered mobile devices for video recording and live video messaging or streaming applications. As HEVC adoption quickly spreads to every app and service that utilizes video, native HEVC support will be essential for any device to remain competitive. This is not a problem, as every leading semiconductor manufacturer already supports HEVC in their PC, mobile device and embedded graphics.

Is this proposal unrealistic? Not at all. HEVC patent holders understand that it is in their interest to see HEVC adoption accelerated quickly and significantly, and that it is critically important for HEVC to become common for web video distribution and mobile applications. Effectively, this proposal mirrors the current state of AVC patent licensing. AVC has long been supported natively on every device that matters, and app developers don’t have to worry about AVC royalties. The vast majority of software and web service developers currently do not, and will not, pay “device royalties” for client applications, or for video playback in a web browser. Once leading browsers, apps and services are using HEVC, consumer demand for a fast, reliable experience will ensure that every leading device OEM offers native HEVC hardware support. There are roughly 2.5 billion consumer compute-capable devices (PCs, tablets and mobile phones) sold each year[3], and hundreds of millions more consumer electronic devices (TVs, cameras, set-top boxes), it’s clear that the total addressable market for licensing HEVC is more than sufficient to provide a reasonable return on investment to HEVC patent holders. It should be clear to all that the benefits of adopting this proposal will far outweigh any perceived cost. I believe this proposal represents a win/win situation for licensors and licensees, and the biggest winners of all would be the end consumer worldwide. I look forward to the conversation that is sure to follow.