What's New in OpenCV 5

OpenCV has been the standard toolkit for computer vision for over two decades, but its last major release was in 2018. In the years since, the field moved heavily toward transformer architectures, large vision-language models, and inference patterns that the old engine struggled to handle. OpenCV 5 addresses that gap with a completely rewritten DNN module, native support for diffusion models and VLMs, and a cleaner core API. This article covers what changed, why it matters, and what the new capabilities look like in practice.

The new DNN engine

The most significant change in OpenCV 5 is the complete rewrite of its Deep Neural Network module. The old engine in OpenCV 4.x processed networks as a flat list of layers, walking through them sequentially. That approach worked for the relatively simple architectures of its era but broke down with modern models that use dynamic input shapes, branching, and attention mechanisms.

The new engine builds a typed operation graph from the full network before running anything. By understanding the entire model upfront, it can perform shape inference across dynamic inputs, pre-compute constant subgraphs (constant folding), and fuse common operation patterns like the MatMul → Softmax → MatMul sequence in attention layers into single optimized operations.

The practical result of this architectural change is a dramatic increase in ONNX operator coverage. ONNX is the standard interchange format for AI models: you train in PyTorch, export to ONNX, and run the exported model in whatever environment you need. In OpenCV 4.x, the DNN engine covered roughly 22% of ONNX operators, which meant most modern models hit unsupported operations immediately. OpenCV 5 covers over 80%.

That's the number that matters most in this release. It means the majority of current models, including transformer-based architectures, run natively in OpenCV without PyTorch, ONNX Runtime, or any other external inference dependency.

CPU performance

More compatibility is only useful if performance holds up. The OpenCV team benchmarked the new engine against Microsoft's ONNX Runtime, the standard dedicated inference backend, on CPU. The results across several real-world models:

• OWLv2: 36.6% faster than ONNX Runtime
• BiRefNet: 32.4% faster
• XFeat: 31.25% faster
• YOLOv8n: 11.5% faster

One important caveat: the new DNN engine is CPU-only at launch. GPU support via CUDA and OpenVINO is on the roadmap for later in the 5.x cycle. If you need GPU inference now, the library falls back to the classic engine, which retains that support.

Core and language changes

Beyond the DNN module, several long-standing friction points in the core API have been addressed.

cv::Mat previously required at least two dimensions, making scalars and 1D arrays awkward to work with. OpenCV 5 adds proper 0D and 1D support with broadcasting, which removes a lot of clumsy reshaping code from typical workflows.

Native FP16 and BF16 data types are now supported throughout the library. These are the standard precisions for modern AI inference, and having them built in avoids the costly conversions that were previously required.

The legacy C API has been officially deprecated, removing a significant amount of historical baggage. C++17 is the new minimum recommended standard. On the Python side, the library now has deeper NumPy integration and supports named keyword arguments for C++ algorithms.

Image colorization

The DNN examples that ship with OpenCV 5 illustrate the range of what the new engine can handle. Image colorization takes a grayscale input and uses a neural network to generate plausible color output:

python3 colorization.py --onnx_model_path /path/to/colorizer.onnx --input /path/to/image.jpg

Generation is near-instant on CPU. The quality depends on the model, but the notable point is that it runs with zero external dependencies.

Real-time object detection with YOLO

YOLOv8s running on video through the new engine:

python3 object_detection.py yolov8s --input /path/to/video.mp4

This processes video in real time on CPU, drawing bounding boxes around detected objects. The ONNX coverage improvement is what makes this possible without any additional setup: the model loads and runs without hitting unsupported operators.

Inpainting with a latent diffusion model

OpenCV 5 includes an example for diffusion-based inpainting, the same underlying approach used by Stable Diffusion. The workflow is to mask an area of an image, then have the model fill it in:

python3 ldm_inpainting.py --input image.jpg --samples 5

An interactive window opens for drawing the mask. After pressing space, the model runs its iterative denoising process. On CPU this takes noticeably longer than the simpler models, but the output is coherent: the masked region is filled with texture and lighting consistent with the surrounding image. Running a diffusion model entirely within OpenCV, without a separate inference stack, is new in version 5.

Image captioning with PaliGemma

The most forward-looking demo uses Google's PaliGemma vision-language model to generate text descriptions of images. This requires loading several model components:

python3 vlm_inference.py \
    --siglip ~/paligemma_onnx/siglip/onnx/vision_model.onnx \
    --embedding ~/paligemma_onnx/embedding/onnx/embedding.onnx \
    --gemma ~/paligemma_onnx/gemma/onnx/gemma2_3b.onnx \
    --tokenizer_path ~/paligemma_onnx/config.json \
    --input /path/to/image.jpg \
    --prompt $'cap en\n'

On a powerful CPU, this takes over a minute and returns a basic caption. The output quality and speed aren't practical for production on CPU, but the significance is architectural: OpenCV 5 now includes the building blocks for running LLMs natively, including tokenizers, attention layers, and KV caching. Vision and language pipelines can run within a single library without assembling a separate inference stack.

What this means in practice

OpenCV 5 is a meaningful shift in what the library is. The 4.x line was a strong toolkit for classical computer vision algorithms with limited and often frustrating deep learning support. Version 5 is a capable inference engine for modern models that also happens to include everything from the old library.

The practical implication is that for CPU-based deployment, OpenCV 5 is now a serious alternative to maintaining a separate inference dependency. You train in PyTorch, export to ONNX, and run in OpenCV with performance that often matches or beats dedicated inference backends. The GPU gap is a real limitation for now, but that's explicitly on the roadmap.

The library's reach makes this upgrade significant beyond the feature list. With over 7 million PyPI installs per week and an 89% share of the embedded vision market, the shift to a modern inference engine in OpenCV 5 has downstream effects across a substantial portion of deployed computer vision systems.

Got an article suggestion? Let us know

Next article

Apfel: How to Use macOS's Built-In LLM from the Terminal

Apfel is an open-source tool that exposes Apple's on-device language model through a CLI, chat mode, and an OpenAI-compatible local server. Here's how it works, what it can do, and where it falls short.

→

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.