AI-Powered Video Analytics for Public Safety and Retail Security

By enabling real-time analysis of massive video data streams, AI can detect anomalies, identify potential threats, and trigger timely alerts. However, achieving widespread deployment and high-density coverage of video analytics necessitates efficient and cost-effective hardware solutions.

Traditional video decoding and analysis approaches can be computationally intensive and expensive, especially when dealing with high-resolution video streams and complex AI algorithms. To address this challenge, a promising solution lies in combining specialized hardware accelerators for video decoding and AI inference. Video Processing Units (VPUs) are specifically designed to efficiently decode video streams, while Graphics Processing Units (GPUs) excel in performing parallel computations required for AI inference tasks.

By leveraging the synergistic capabilities of VPUs and GPUs, organizations can build scalable and cost-effective video analytics solutions. This division of labor not only maximizes throughput and minimizes latency but also optimizes resource utilization and reduces overall costs. The combination of VPUs and GPUs can be particularly beneficial in scenarios where real-time analysis and high-density coverage are critical..

This mini-report examines the architectures that enable such systems, the core AI models used, and how high-volume video streams are handled cost-effectively in the cloud using VPU decoders. A high-density case study demonstrates how adopting efficient hardware and workflows can significantly reduce operational costs while expanding coverage.

Cloud-Centric Workflow Architecture

Video Ingestion

In high-density surveillance setups – such as a stadium or retail chain – dozens or hundreds of cameras feed compressed video streams (commonly H.264, H.265/HEVC, or newer AV1) into a cloud-based platform. Ingestion services typically leverage RTSP/RTP or HTTP streaming protocols, buffering data before sending it to downstream processing. Scalable microservices-based designs rely on load-balancing frameworks or message brokers to manage potentially thousands of concurrent streams.

Decoding and Pre-Processing

Once ingested, each compressed stream must be decoded into raw frames for analysis. This step is computationally heavy at scale: decoding 1080p or 4K streams in large quantities can saturate CPU resources. Offloading to specialized hardware – GPUs or dedicated video processing units (VPUs) – dramatically increases throughput. Pre-processing may include resizing, cropping, or frame-rate reduction. Reducing the frame rate to 10 or 15 fps can significantly cut compute load without sacrificing security insights.

AI Inference

The pre-processed frames are sent to AI inference engines running object detection, face recognition, behavior analytics, or sentiment analysis. These engines are typically containerized services that can scale horizontally. For real-time requirements, sub-second latency is ideal; operators often deploy inference on GPU servers or specialized accelerators to achieve this. Load balancing and batching algorithms ensure multiple streams can be analyzed simultaneously.

Alerts and Further Analysis

When AI models detect a relevant object or behavior – e.g., a weapon, a fight, or a suspected shoplifter – the system generates alerts, which can be integrated with security dashboards or incident management platforms. Cloud-centric solutions also allow centralized video storage, indexing, and advanced search (e.g., queries for a person wearing a specific color across all cameras). For public safety at large events, alerts might be routed to onsite security staff; for retail, a real-time notification can prompt store managers to address a potential theft.

This modular, cloud-based pipeline – comprising ingestion, decoding, inference, and alert management – provides scalability and easier updates. Organizations can add new AI features or cameras by deploying more container instances rather than overhauling the entire system. For older camera infrastructures, migrating to the cloud via standard protocols can extend camera lifespans and minimize capital expenses.

AI Models for Security and Behavior Detection

Object Detection

Object detection is the foundation of most AI analytics systems. Standard models (e.g., YOLO, Faster R-CNN) are trained to recognize specific objects – like firearms, knives, or suspicious packages – so that they can trigger immediate alerts. These models also enable face detection to identify persons of interest on watchlists. In retail, object detection helps detect theft (e.g., an item being concealed).

Behavior Analysis and Anomaly Detection

Beyond static objects, advanced systems detect suspicious actions: fights, loitering, people entering restricted zones, or an individual collapsing. This typically leverages temporal modeling (analyzing multiple frames to see movement patterns). Anomaly detection can learn “normal” activity patterns for a scene, then automatically flag unusual actions (e.g., large crowd gatherings, erratic movements). Such algorithms reduce false positives by adapting to typical day-to-day behaviors.

Sentiment and Emotion Detection

Sentiment analysis uses facial expression recognition to infer whether individuals are calm, angry, or distressed. This might help security teams anticipate aggression or panic in a stadium crowd. In retail settings, it can identify customer frustration at checkouts. While accuracy can vary with camera angles and environmental conditions, such context can provide early warning before incidents escalate.

Motion Tracking and Event Recognition

Many deployments include multi-object tracking, which follows individuals across camera views – crucial when responding to suspicious behavior. Other specialized models track events like falls, license plates, or perimeter breaches. Retailers often use people-counting to monitor foot traffic or identify hot spots in the store. Combining these outputs with object or sentiment detection provides a richer picture of security and operational insights.

High-Density Video Processing & Hardware Comparison

A significant challenge in cloud-based AI analytics is dealing with large numbers of concurrent video feeds. For every feed, decoding must happen in near-real time. Hardware choices have massive impacts on throughput, power usage, and costs.

CPU-Only Decoding

Using software decode on a CPU is flexible but scales poorly. CPU usage can skyrocket for modern codecs like AV1 or HEVC, and the system power draw is high. Although a large, multi-core server can handle dozens of streams, scaling to hundreds or thousands of feeds becomes cost-ineffective. CPU-only solutions often require many servers, each consuming substantial power, rack space, and cooling capacity.

GPU-Accelerated Decoding

GPUs incorporate hardware decoders, such as NVIDIA’s NVDEC, that handle multiple HD streams in parallel. An NVIDIA T4, for instance, can decode around 17 full-HD H.264 streams at 60 fps. This is far more than a CPU, but GPUs still draw considerable power (70 W or more under load) and may be less efficient if you only need decoding rather than combined decode + AI inference on the same card.

In practice, GPUs can perform decoding and model inference in a single pipeline, simplifying system design. Still, GPU solutions can become expensive to purchase and operate on a large scale if they also run at maximum load to decode hundreds of streams. One issue with GPUs is that the video IP block only takes up to 15% of the wafer, meaning the system will reach capacity before the GPU is saturated. Thus, video decoding is not uncommon as a bottleneck instead of inferencing capacity.

NETINT VPUs (ASICs)

Dedicated video processing units (VPUs) like NETINT devices address the scalability problem by offloading decoding/encoding to specialized ASIC hardware. For example, a single NETINT Quadra T1U consumes barely more than twenty watts while handling up to 48 simultaneous 1080p30 H.264, HEVC, or VP9 decodes. Ten NETINT VPUs in a 1U server can decode up to 480 HD channels, enabling high channel density at very low power consumption – substantially lower per stream than GPU-based decode.

By assigning video decode tasks to VPUs, GPUs can handle AI inference without saturating their internal decoders or tying up resources. This heterogeneous architecture – CPU for orchestration, VPU for decode, and GPU for model inference – can yield the best performance-per-watt for large deployments.

Power Efficiency and OPEX

Power consumption directly drives cooling costs and overall OPEX. CPU-bound decoding at scale can demand hundreds of watts per server, while GPUs can range from 70 W to 300 W per card when fully utilized. ASIC-based VPUs are often an order of magnitude more efficient per stream. For example, a Quadra T1U consumes slightly less than half a watt per HD stream. In data centers monitoring thousands of camera feeds, these savings substantially reduce the hardware footprint and ongoing electricity costs.

For cost-conscious organizations – whether a city government or a retail chain – these power and density advantages determine the feasibility of real-time analytics. By deploying VPUs, one can cut the number of servers needed and minimize capital (hardware) and operational (power, cooling) expenses. Although VPUs do not run the commonly used AI models, pairing them with general-purpose GPUs or Arm CPUs from Ampere results in a balanced, high-throughput system.

Case Study: Large Stadium Deployment

A major sports stadium serves as a prime example of high-density AI video analytics. With around 400 cameras covering entry gates, concourses, seating areas, and parking lots, the stadium needed real-time threat detection and crowd management. Initial estimates suggested requiring 40–80 commodity servers or expensive cloud GPU instances to handle all feeds. That hardware cost, plus ongoing power expenses, threatened the project’s economic viability.

Using an optimized platform that combined specialized hardware accelerators and AI software, the stadium achieved similar coverage with only 10 servers. Each server leveraged hardware-accelerated decoding (on GPUs or dedicated ASICs) and then used GPU-based AI inference to detect intrusions, fights, or suspicious packages. The resulting system processed around 80–100 camera feeds per server, dramatically cutting capital expenditures and utility bills. The scalable design also enabled the stadium to add new analytics features – like crowd sentiment monitoring or license plate recognition – without re-architecting the entire solution.

Ultimately, the pilot’s success prompted a full-scale rollout of AI security across the venue. The stadium avoided costly camera upgrades because the solution was plugged into existing camera infrastructure. Consolidating video processing in the data center also simplified ongoing maintenance and made training or updating the AI models easier. Over time, the system’s false alarm rate dropped as it learned stadium-specific behaviors, improving reliability for security teams.

Future-proofing security investments

AI-based video analytics revolutionizes public safety at large events and retail store security by providing real-time insights into objects, behaviors, and sentiment across camera feeds. A typical cloud-centric pipeline ingests and decodes compressed video, pre-processes frames, runs AI inference for threat or anomaly detection, and generates alerts and analytical data. Object detection, behavior analysis, and emotional state models can be combined for a more complete view of security.

However, achieving high-density coverage – potentially hundreds of HD streams – demands efficient decoding hardware. CPU-only solutions quickly become cost-prohibitive at scale due to high power draw and low throughput. GPUs perform better but can still consume considerable power, especially when handling both decoding and inference.

Dedicated video processing ASICs like NETINT VPUs significantly improve channel density and power efficiency, making large-scale cloud analytics economically viable. By combining VPUs for decoding with GPUs for inference, organizations minimize the number of servers needed and reduce operating expenses.

In the stadium case study, an integrated approach enabled a 75% reduction in hardware and a dramatic drop in energy usage without compromising on real-time alerts. These benefits apply equally to major retail chains looking to protect multiple stores or municipalities planning area-wide surveillance. Because a cloud-based microservices architecture scales seamlessly, adding cameras or new analytics capabilities becomes straightforward – key for future-proofing security investments.

The recommendation for executives and technical planners is clear: harness cloud-native AI analytics, employ specialized accelerators for video decoding, and reserve GPUs for AI model inference tasks. This architecture delivers a strong return on investment, ensures real-time responsiveness, and positions the organization for emerging analytics like advanced sentiment recognition or 3D object tracking. The trend is toward heterogeneous compute infrastructures that extract the best of each component – CPU, GPU, and VPU – to maximize throughput and minimize cost. By adopting these design principles, public venues and retailers can provide safer, more efficient environments while containing expenses over the long term. AI-powered video analytics has emerged as a transformative tool in enhancing public safety and security.

Schedule a meeting to learn how NETINT VPUs can enhance live streaming with energy-efficient, scalable solutions.

by Mark Donnigan