Video Processing Unit (VPU) – the NETINT’s Journey to a Superior Discovery

Discover how NETINT’s evolution from SSD innovator to VPU (video processing unit) category creator drives new levels of video encoding operational efficiency and performance. This article describes the journey and provides valuable insights and lessons for video encoding and processing engineers and practitioners. If you design, build, or operate content delivery networks, cloud gaming platforms, or video streaming services, this is for you.

A Journey from SSDs to Video Transcoding Mastery

The Original Vision: Revolutionizing SSDs

In 2015, NETINT’s co-founders, Alex Liu, and Tao Zhong, set out to design state-of-the-art SSDs with an integrated video encoding engine and achieved a significant milestone by creating the world’s first PCIe Gen-4 computational storage device. However, the dynamic SSD market and architectural system changes led the company to leverage the initial work to create a high-density ASIC-powered video encoding solution.

Although this early work in computational storage was not immediately successful in the market, it positioned NETINT at the forefront of an emerging trend in data processing. The insights gained from this endeavor have influenced the development of edge computing solutions and contributed to the ongoing convergence of storage and processing technologies in modern data centers.

Pivoting to Video Transcoding

NETINT discovered a promising niche in high-density video encoding and decoding. The team adapted the storage protocol NVMe transforming off-the-shelf storage servers into powerful transcoding servers.

The pivot to video transcoding was a strategic move that capitalized on the growing demand for efficient video processing solutions. By leveraging its expertise in storage technologies, NETINT created a unique value proposition that addressed the needs of content delivery networks, streaming platforms, and other high-volume video processing applications.

As a consequence of this pivot, NETINT is shipping solutions that dramatically reduce the total cost of ownership for video processing infrastructure. Clients have reported significant reductions in power consumption, space requirements, and operational complexity, while simultaneously increasing their video processing capabilities. This has enabled streaming services to scale more efficiently and contributed to the proliferation of high-quality video content across the internet.

From T408 to Quadra: NETINT's Game-Changing Journey

The Birth of T408: A New Era in Video Transcoding

In 2017, NETINT released their first encoding ASIC, the T408, capable of transcoding 4Kp60 video. This breakthrough found applications in CDNs and cloud gaming. The T408 represented a significant leap forward in video transcoding technology, offering capabilities that were previously difficult to achieve with traditional CPU-based solutions. Its success in CDNs and cloud gaming applications demonstrated the potential of dedicated hardware for video processing, setting the stage for further innovations.

For cloud gaming providers, the T408’s low-latency encoding capabilities helped reduce input lag, significantly improving the gaming experience for users. These advancements contributed to the rapid growth of 4K content availability and the expansion of cloud gaming services.

Continuous Innovation for Better Solutions

NETINT’s journey from T408 to Quadra showcases a commitment to innovation and market responsiveness. As demand for higher density and more powerful encoding solutions grew, NETINT developed the second generation Quadra chip, which quadrupled encoding capacity and added AV1 encoding, AI, and 2D graphics capabilities.

The development of the Quadra chip was driven by market demands for even higher encoding densities and support for newer codecs like AV1. By incorporating AI capabilities and 2D processing, NETINT expanded the potential applications of their technology, positioning themselves at the forefront of video processing innovation.

The introduction of the Quadra chip has had a transformative effect on the video processing landscape. Support for AV1 encoding accelerated the adoption of this more efficient codec, leading to reduced bandwidth requirements for streaming services and improved video quality for end-users. The AI capabilities have opened up new possibilities for real-time video enhancement and content analysis, enabling more sophisticated content recommendation systems and improved viewer experiences.

Demystifying ASICs: The Future of Efficient Video Processing

Understanding ASICs: Specialized for Efficiency

ASICs (Application-Specific Integrated Circuits) are designed for specific tasks, such as video processing, making them highly efficient compared to general-purpose CPUs, GPUs, or reprogrammable FPGAs. Optimized for stable, long-established codecs like H.264, HEVC and AV1, ASICs excel in power consumption and performance, ideal for the growing demand for high-density video encoding.

The specialization of ASICs allows for optimizations that are simply not possible with general-purpose processors. By focusing on specific video codecs and processing tasks, NETINT’s ASICs can achieve levels of efficiency and performance that are crucial for large-scale video processing operations.

The adoption of ASIC-based solutions for video processing has led to a significant reduction in energy consumption in data centers dedicated to video streaming. This has not only reduced operational costs for streaming providers but has also contributed to the overall sustainability of the rapidly growing video streaming industry.

Companies using NETINT’s ASIC solutions have reported substantial improvements in their carbon footprint metrics, aligning with global efforts to reduce the environmental impact of digital technologies. The compact size of ASICs allows for high-density solutions, enabling the processing of multiple video streams within a small physical footprint.

Some companies have reported being able to handle up to 10 times more concurrent streams in the same physical space after switching to NETINT ASIC-powered VPU solutions. This increased density has enabled the rapid scaling of video streaming services, supporting the growing demand for high-quality video content across various platforms and devices.

Unlocking Video Quality: How NETINT's ASICs Achieve Superior Encoding

Setting the Benchmark: Quality Targets for H.264 and H.265

NETINT aims to match the quality of x264 medium for HEVC and surpass x265 medium for HEVC. ASICs offer adjustable quality and performance settings, ensuring they meet and exceed industry standards. This flexibility allows NETINT’s hardware encoders to deliver superior video quality tailored to various applications.

The ability to match or exceed software encoding quality while maintaining hardware efficiency is a significant achievement. NETINT’s focus on quality ensures that their solutions are suitable for professional video applications where visual fidelity is paramount

Optimizing for Different Video Scenarios

NETINT’s ASICs are optimized for diverse scenarios, from cloud gaming, where latency minimization is crucial, to live streaming, where look-ahead features enhance quality. This adaptability ensures smooth, high-quality video processing across different applications.

The optimization for different video scenarios demonstrates NETINT’s understanding of the diverse requirements in the video processing industry. By providing solutions that can adapt to various use cases, they ensure their technology remains relevant across a wide range of applications.

Transforming Servers into High-Density Transcoding Powerhouses

Maximizing Server Capacity with NETINT Solutions

NETINT VPUs utilize U.2 and PCIe form factors, enabling integration of multiple transcoding chips into standard Arm and X86 servers. This represents a paradigm shift in video processing infrastructure, an approach that allows organizations to maximize their existing hardware investments while dramatically increasing their transcoding capabilities.

Revolutionizing Transcoding Efficiency

NETINT’s approach leverages existing server infrastructure to achieve unprecedented encoding density and performance. This method boosts efficiency and provides a scalable solution for growing video processing demands.

By enabling high-density transcoding within existing server frameworks, NETINT’s solutions offer a cost-effective path to scaling video processing capabilities. This efficiency is particularly valuable as the demand for video content continues to grow across various industries.

Overcoming Bottlenecks: How NETINT's Transcoders Maximize Performance

Optimized Transcoding with Hardware Frames

NETINT uses hardware frames to keep YUV frames on the device during transcoding, reducing CPU involvement and avoiding PCI bottlenecks. This method ensures smooth and efficient operations, optimizing performance.

The use of hardware frames represents a sophisticated approach to managing data flow in the transcoding process. By minimizing data transfer between the ASIC and the host system, NETINT’s solution reduces latency and improves overall system efficiency.

Handling Complex Scenarios with Software Decoding

In scenarios requiring software decoding or scaling, NETINT provides solutions to avoid CPU bottlenecks. Their expertise in performance tuning ensures efficient processing, even with high-density encoding.

The ability to handle complex scenarios that involve a mix of hardware and software processing demonstrates the flexibility of NETINT’s solutions. This hybrid approach allows for the handling of a wide range of input formats and processing requirements while maintaining high performance.

Harnessing the Power of Arm CPUs and FFmpeg 6.1

Empowering CPUs with NETINT Offloading

Arm CPUs, especially those from Ampere, are ideal for offloading intensive tasks to NETINT devices, freeing the CPU for other operations. This efficient division of labor maximizes system performance. The synergy between Arm CPUs and NETINT VPUs represents a forward-thinking approach to system design. By leveraging the strengths of both technologies, NETINT enables more efficient and powerful video processing solutions.

Maximizing Efficiency with FFmpeg 6.1

FFmpeg 6.1 enhances multithreading for decoding, improving performance for complex workflows. Combined with NETINT’s technology, this setup can handle diverse video workloads efficiently. The integration with FFmpeg 6.1 showcases NETINT’s commitment to working with industry-standard tools and frameworks. This compatibility ensures that their solutions can be easily integrated into existing workflows and systems.

Mastering Video Processing: NETINT's Versatile Software Solutions

Libxcoder SDK: The Foundation of Flexibility

NETINT’s Libxcoder SDK provides a basic interface for managing encoding and decoding processes, ideal for simple applications like cloud gaming. It supports various demos, showcasing its versatility.

The Libxcoder SDK offers developers a powerful tool for integrating NETINT’s hardware capabilities into their applications. Its flexibility and support for various use cases make it an attractive option for developers working on video processing projects.

FFmpeg Integration: Powering Advanced Applications

NETINT supports FFmpeg through libavcodec and libavfilter libraries, offering demos and patches for hardware-accelerated encoding and decoding. This integration ensures seamless incorporation into existing workflows.

The integration with FFmpeg extends the reach of NETINT’s technology, making it accessible to a wide range of developers and applications. This support for industry-standard tools enhances the adoption potential of NETINT’s solutions.

Gstreamer Support: Seamless Hardware Acceleration

For Gstreamer users, NETINT integrates the Libxcoder library, providing hardware-accelerated encoding, decoding, and filtering. This comprehensive solution leverages Gstreamer’s robust features with NETINT’s hardware acceleration.

Support for Gstreamer further expands the ecosystem of applications that can benefit from NETINT’s hardware acceleration. This integration allows developers to easily incorporate NETINT’s capabilities into complex media processing pipelines.

Efficient Load Management with NETINT

Smart Distribution with Automatic Load Balancing

NETINT’s load management system distributes jobs efficiently across multiple cards, maintaining smooth processing. The resource monitor provides real-time updates, ensuring optimal performance.

The automatic load balancing feature demonstrates NETINT’s focus on simplifying complex video processing operations. By intelligently distributing workloads, the system ensures maximum utilization of available resources without requiring manual intervention.

Flexibility for Custom Solutions

NETINT offers manual load management for those who prefer control over resource allocation. However, most customers find the automatic system meets their needs, ensuring efficient processing with minimal intervention.

The option for manual load management caters to users with specific requirements or unique use cases. This flexibility ensures that NETINT’s solutions can adapt to a wide range of operational needs and preferences.

Virtualization in Video Processing: NETINT's Advanced Quadra Solutions

Virtualization for Multi-Tenant Environments

NETINT’s Quadra cards support virtualization, allowing resource sharing among multiple users while maintaining isolation. This setup ensures secure and efficient resource allocation in cloud gaming and desktop applications.

The support for virtualization opens up new possibilities for multi-tenant environments, such as cloud gaming platforms or shared rendering farms. By enabling secure resource sharing, NETINT’s solutions contribute to more efficient and cost-effective infrastructure utilization.

Ensuring Quality of Service and Isolation

NETINT employs strategies to maintain performance and isolation, such as assigning QoS targets to namespaces. Memory management is handled in firmware, ensuring robust security.

The focus on quality of service and isolation in virtualized environments demonstrates NETINT’s commitment to security and performance. These features are crucial for applications where multiple users or processes share the same hardware resources.

Unlocking Virtualization: NETINT's Versatile Schema for Any Environment

Versatility Across Virtualization Platforms

NETINT’s virtualization schema works efficiently across various environments, from Docker to Hyper-V. The broad support for various virtualization platforms showcases the versatility of NETINT’s solutions. This flexibility allows organizations to integrate NETINT’s technology into their existing infrastructure without major changes to their virtualization strategy.

Broad Compatibility for Diverse Needs

NETINT’s solutions are rigorously tested for compatibility and performance, providing reliable support for high-performance applications and complex virtualized environments. The extensive testing and broad compatibility of NETINT’s solutions provide confidence to users implementing their technology in diverse and demanding environments. This reliability is crucial for mission-critical applications in video processing and delivery.

Enhancing Reliability: Five Years of NETINT's Transcoding Excellence

Tackling Decoder Challenges

NETINT faced early challenges with decoders, especially for UGC and satellite content. Through extensive testing and real-world deployments, the team has significantly hardened our decoders, ensuring reliability in various scenarios.

The journey of improving decoder reliability showcases NETINT’s commitment to continuous improvement. By addressing challenges head-on and learning from real-world deployments, we have enhanced the robustness of our solutions over time.

Evolving with Customer Needs

NETINT’s encoders support diverse applications, from AI and VR to live streaming. Working closely with customers, we have fine-tuned our products to meet unique requirements, ensuring robustness and adaptability. This responsiveness ensures that their solutions remain relevant and effective across a wide range of use cases.

Proven in the Field with 200 Billion Minutes Encoded

NETINT’s transcoders have demonstrated reliability over five years, and with more than 200 billion minutes of encoded video streamed to users. Continuous feedback loops allow for swift issue resolution. This experience ensures consistent performance in unpredictable conditions and the proven track record in real-world deployments builds confidence in the technology.

Lessons in technological innovation

NETINT’s journey from SSD pioneers to video transcoding leaders showcases the power of adaptability and innovation. The commitment to continuous improvement and customer collaboration has resulted in reliable, efficient solutions for diverse video processing needs.

Ready to enhance your video processing capabilities? Schedule a consultation with the NETINT team today and discover the difference expertise makes.

The story of NETINT’s evolution in the video processing industry offers valuable lessons in technological innovation, market responsiveness, and the importance of customer-driven development. As video continues to dominate digital communication and entertainment, solutions like those offered by NETINT will play a crucial role in shaping the future of content delivery and processing.