NVIDIA's OmniVinci: 9B model open-source downloads break 10,000, triggering a new wave of AI multimodality

So far this year.Open SourceThe Great Model Battlefield is full of smoke.

People from all walks of life are fully engaged here, trying to seize the ecological position in the next era of AI. And a trend that can't be ignored is that Chinese big models, are strongly ruling the 'Hall of Fame' of open source base models.

From DeepSeek's amazing performance in code and mathematical reasoning to the full blossoming of the Qwen (Tongyi Thousand Questions) family in multimodal and general-purpose capabilities, they have long been an inescapable point of reference for AI practitioners around the globe by virtue of their superior performance and rapid iteration.

Just when everyone thought that this wave of open source basic modeling would be driven mainly by top Internet giants and star startups, a giant that was supposed to be selling water on the side also added fuel to the fire.

Yes, as the biggest beneficiary of the AI wave - theNVIDIA,(NVIDIA) - hasn't been slacking on self-researching big models.

Now comes an important piece of the puzzle for NVIDIA's big model matrix.

Without further padding, Old Yellow's latest trump card is officially on the table:The strongest 9B video and audio omnimodal large model OmniVinci, strong open source!

Link to paper:https://arxiv.org/abs/2510.15870

Code Link:https://github.com/NVlabs/OmniVinci

OmniVinci demonstrated crushing performance on several mainstream omnimodal, audio understanding, and video understanding lists:

If NVIDIA's previous open-source model was only a niche layout in a specific area, then the release of OmniVinci is a true 'all-out press'.

NVIDIA defines OmniVinci as "Omni-Modal" - a unified model that understands video, audio, images and text simultaneously.

It is only 9 billion (9B) parameters in size, yet it has demonstrated 'table-turning' levels of performance in a number of key multimodal benchmarks.

According to the paper released by NVIDIA, OmniVinci's core strengths are extremely overriding:

NVIDIA's entry sends a clear signal that the king of hardware, likewise, must hold the right to define the model.

Video + Audio Comprehension: 1+1>2

Did the addition of audio actually make the multimodal model stronger? The experiments give a clear answer: yes, and the improvement is very significant.

The team noted that sound introduces a whole new dimension of information to the visual task, benefiting the model in terms of video comprehension.

Specifically, there is a stepwise jump in model performance from relying solely on vision, to implicit multimodal learning in combination with audio, to the introduction of an all-modal data engine for explicit fusion.

Especially after adopting the explicit learning strategy, there are breakthroughs in a number of metrics, as shown in the table below, and the performance is almost 'skyrocketing'.

Not only SFT, but also the inclusion of audio modalities in the post-training phase can further enhance the effect of GRPO:

Full Modal Agent, Landing Scene Pulling Full

The omnimodal model with both video and audio breaks through the modal limitation of the traditional VLM to understand the video content more fully, and therefore has a wider range of application scenarios.

For example, summarizing the interview with Old Yellow:

It can also be able to be transcribed into text:

Or maybe voice commanding a robot to navigate:

It's a friend, not a foe, in the open source world.

In the past year.

DeepSeek has become synonymous with 'the strongest science student' by refreshing the upper limit of the open source list time and time again with its superb strength in code and mathematical reasoning.

Qwen, on the other hand, has constructed a huge matrix of models, ranging from the smallest 0.6B to the giant 1T large model, and is one of the "all-rounders" with the most complete ecology and the most balanced comprehensive capability.

OmniVinci's open source is more like a catfish. It has set up the research benchmark of SOTA with extreme efficiency and strong performance, stirred up the battlefield of open source big models, and urged the friends to come up with better models to help mankind towards AGI.

For NVIDIA, which "sells shovels", the more people use the open source model -> the more people buy GPUs, which is undoubtedly the biggest gainer of the open source model, and because of this, NVIDIA is a firm friend of the open source modeling team, not an opponent.

concluding remarks

Community Rave, Wave Acceleration, Together at AGI

The release of NVIDIA OmniVinci was like a boulder smashing into the already turbulent sea of open source, and has already garnered more than 10,000 downloads on huggingface.

Overseas tech bloggers are snapping up videos and articles to share relevant technology

It is a natural extension of NVIDIA's "hardware and software" ecosystem and a powerful "boost" to the entire AI open source ecosystem.

The open source landscape, as a result, is much clearer.

On one side, there is the Chinese open source force represented by DeepSeek and Qwen, who have built a thriving developer base with their extremely fast iteration speed and openness.

On the other side, it is NVIDIA, which holds the hegemony of arithmetic power in its hands, personally coming down to accelerate the whole process as an open-source friend with "technology benchmarking" and "ecological incubation".

The wave is already accelerating, and no one can stay out of it. For every AI practitioner, an era of stronger, faster, and more 'rolled' AI has just begun.

References:

https://arxiv.org/abs/2510.15870