Marengo 3.5: Precision & Control for Frontier Video Intelligence

Today, we’re launching Marengo 3.5, our latest video-native multimodal embedding model, redefining standards as the leading foundation model for real-world multimodal video search at production scale.

Marengo 3.5 represents video, audio, images, text, and documents in a shared semantic space. Developers can retrieve within one modality, across modalities, or with several inputs combined into one query.

The release improves retrieval quality over Marengo 3.0 and expands what teams can build around the model. Composable queries express more specific intent. Flexible embedding sizes will let teams balance retrieval quality with storage and index costs. Timestamped business context can stay attached to the moments it describes. An optional uncertainty signal helps applications decide when to trust a result, ask for clarification, or route a case for review. Long-form asynchronous ingestion removes the previous four-hour duration cap.

Marengo 3.5 also changes how model usage is priced. Each input modality is metered separately, so customers pay for the work their request actually uses. That gives teams more control today and creates a cleaner foundation for more configurable performance and cost options over time.

Marengo 3.5 benchmarks: The new frontier for video intelligence

Multimodal embedding benchmarks summary for Marengo 3.5, Marengo 3.0, Gemini Embeddings 2 (GE2), and Amazon Nova Multimodal Embeddings (MME). Scores are task-group means unless noted; higher is better. The default output dimension is 512 for Marengo 3.5 and Marengo 3.0, and 3,072 for GE2 and Nova MME. Bold indicates the highest reported score in each row. N/A means the model does not report a result.

A single benchmark rarely reflects what production retrieval systems need. They must find moments in video, connect text with images and audio, work across languages, and increasingly retrieve information from visual documents.

On five standard text-to-video retrieval datasets, Marengo 3.5 reaches a mean score of 70.57, a 6.29-point improvement over Marengo 3.0. It also improves across video classification, question-answer retrieval, moment retrieval, image retrieval, audio tasks, and multilingual text.

The comparison with other multimodal embedding models is strongest when performance and deployment footprint are considered together. Across an eight-group aggregate covering video, images, visual documents, multilingual text, and audio, Marengo 3.5 scores 67.38 at 512 dimensions. The largest reported 3,072-dimensional outputs from Gemini Embedding 2 and Amazon Nova Multimodal Embeddings score 62.35 and 50.34, respectively.

For the full evaluation set, scoring methodology, and production retrieval studies, check out Marengo 3.5: Built for the Structure of Video.

New capabilities

Multimodal search with fully composable queries

Text alone is often too vague. A reference image can identify the person, product, location, or style. An audio clip can identify a sound. A short video can describe motion that is awkward to put into words.

Marengo 3.5 lets developers combine text with named video, audio, or image inputs and receive one composed query embedding. A sports application could combine a player image, a clip of a particular move, and a text description of the game situation. A media archive could combine a reference still with a written description of the scene it needs.

This reduces the amount of custom fusion logic teams have to build around the model. More of the user’s intent reaches retrieval as one query, which can improve precision when any single modality would be incomplete.

Multimodal composable query example. Composable queries combine text with named image, video, and audio references into one query vector, so the full multimodal intent reaches retrieval together.

Configurable embedding dimensions with Matryoshka Representation Learning

Marengo 3.5 uses Matryoshka Representation Learning so it can produce useful embeddings at multiple sizes. The model is designed for 128-, 256-, and 512-dimensional outputs. Smaller vectors reduce storage, memory, bandwidth, latency, and vector-index costs; the full 512-dimensional output preserves more retrieval quality when the additional footprint is justified.

On the eight-group aggregate in our technical evaluation, 256 dimensions retained 98.3% of the 512-dimensional score, while 128 retained 94.6%. This gives teams a practical way to choose between unit economics and retrieval quality based on the workload.

Performance across embedding dimensions for Marengo 3.5, GE2, and Nova MME. Marengo 3.5 retains 98.3% of its 512-dimensional score at 256 dimensions and 94.6% at 128. At 128 dimensions, it scores 63.75, exceeding the largest reported 3,072-dimensional outputs from GE2 (62.35) and Nova MME (50.34).

Put business context on the timeline with time-based metadata (TBM) fusion

Some of the most useful information about a video is not visible or audible. A score, player name, product ID, scene number, case reference, or broadcast rundown may live in another system.

Time-based metadata lets developers provide timestamped text with video or audio. Marengo 3.5 aligns that context to the relevant media segments and includes it in the fused representation. The underlying visual and audio embeddings remain available as separate representations.

For a sports archive, the footage can stay connected to play-by-play data. For news and entertainment, a scene can stay connected to a rundown, script, or rights record. For enterprise video, an event can stay connected to operational data that would otherwise be lost during retrieval.

Time-based metadata fusion. Time-based metadata adds provided context to the moments it describes. Marengo consumes that context; it does not generate the metadata.

Single embedding model for all your data with added visual document support

For customers, video usually isn’t their only data. Many business records are visual documents: reports with charts, scanned pages, presentation-like layouts, forms, and manuals where meaning depends on both text and page structure.

Marengo 3.5 can create page-level visual embeddings for PDFs through asynchronous ingestion. Those pages can be retrieved in the same semantic space as video, audio, images, and text queries.

At launch, document support begins with visual PDF pages. Text-document ingestion, more diverse document types, additional document controls, and more advanced document features are coming soon.

Make informed business decisions with embedding uncertainty

Similarity scores answer one question: how close are two embeddings? They do not always show whether a query is broad, whether a match is weakly supported, or whether two equally similar results carry different levels of ambiguity.

Marengo 3.5 can return an optional learned uncertainty signal alongside an embedding. Applications can use it to ask for a more specific query, rerank candidates, abstain from an automated decision, or send difficult cases to a more intensive workflow.

The signal is not a universal probability of correctness. Thresholds should be calibrated on representative, labeled traffic for the application. Used that way, uncertainty gives retrieval systems another input for deciding what should happen after similarity search.

No duration limit

Marengo 3.0 supported media up to four hours. Marengo 3.5 removes that duration cap for asynchronous video and audio ingestion. Existing file, upload, rate, and task-processing limits still apply.

For teams working with broadcasts, hearings, surveillance archives, lectures, and other long recordings, this removes a common preprocessing step. Content no longer has to be divided simply to satisfy a model-duration limit, which reduces orchestration work and keeps asset-level workflows cleaner.

Pricing

Marengo 3.0 used a duration-based price for video. It was simple, but it forced very different workloads toward the same middle.

Some customers need the lowest possible unit cost across large archives and are willing to trade granularity or performance for better economics. Others are working on high-value retrieval tasks where maximum quality and fine temporal detail justify a higher cost. A single duration price does not give either group much control.

Marengo 3.5 moves Embed API pricing to modality-specific billable tokens. Video, audio, image, document, and text each have their own usage meter and per-million-token rate. The API reports how much of each modality a request processed. The charge for each modality is calculated separately, then the currency charges are added together.

Per billable modality pricing structure. Each modality is priced independently. API-reported usage is the billing record. Raw modality tokens are not interchangeable.

Pricing table demonstrating billable modality tokens at launch. Check the pricing page for up-to-date pricing.

How common requests are billed

• Video and audio are separate. A video request that processes both the visual stream and its audio track reports both video and audio usage. A visual-only request does not incur an audio charge.

• Time-based metadata uses the text meter. The video or audio is priced on its own meter, and the associated timestamped text is priced at the text rate. Returning a fused embedding does not add another input meter.

• Visual and text documents follow different paths. Visual PDF pages use the document meter. Text-documents like .txt or .md files use the text meter.

• Composable queries add the charges for the inputs used. A query containing text, an image, and an audio clip reports usage for those three modalities. Each charge is calculated at its own rate and the currency amounts are summed.

The result is straightforward: pay for only what you use. A customer processing visuals without audio does not pay an audio rate. A query that does not use an image does not pay an image rate. Fusion combines representations, but it does not create a separate fusion surcharge.

Source-unit conversions such as hours of video or pages of a PDF remain useful for planning. Actual invoices use the modality-specific usage reported by the API.

A foundation for more customer control

The pricing change matters beyond the rate card. It separates the cost drivers that duration-based pricing blended together.

As more configurable options become available, customers will be able to choose an operating point that fits the workload. Cost-sensitive applications can favor options that improve unit economics when the quality trade-off is acceptable. Performance-sensitive applications can spend more deliberately where quality, resolution, or granularity matters most.

Get started with Marengo 3.5

Marengo 3.5 is available through the TwelveLabs Embed API. Use the synchronous Embed endpoint to create query embeddings, including composed multimodal queries. Use asynchronous Embed tasks to ingest video, audio, images, and visual PDFs for retrieval.

Read the technical launch blog for the benchmark and model deep dive, and get started with Marengo 3.5 today.