In the previous update, we mentioned Automatic Reindexing, Tensor Optimizations, Query Profile Variant Initialization Speedup, Explainlevel Query Parameter and PR System Testing. Subscribe to the mailing list to get these updates delivered to your inbox.

This month, we’re excited to share the following updates:

New features in document/v1/

The /document/v1/ API is the easiest way to interact with documents. Since Vespa 7.354, this API lets users easily update or remove a selection of the documents, rather than just single documents at a time. It also lets users copy documents directly between clusters. These new features are efficient and useful for production use-cases; and also increase the expressiveness of the API, which is great for playing around with- and learning Vespa.

weakAnd.replace

Queries with many OR-terms can recall a large set of the corpus for first-phase ranking, hence increasing query latency. In many cases, using WeakAnd (WAND) can improve query performance by skipping the most irrelevant hits. Since Vespa 7.356, you can use weakAnd.replace to auto-convert from OR to WeakAnd to cut query latency. Thanks to Kyle Rowan for submitting this in #16411!

Improved feed-block at full node

Vespa has protection against corrupting indices when exhausting disk or memory: Content nodes block writes at a given threshold. Recovering from a blocked-write situation is now made easier with resource-limits - this blocks external writes at a lower threshold than internal redistribution, so the content nodes retain capacity to rebalance data.

Reduced memory at stop/restart

Index and attribute structures are flushed when Vespa is stopped. Since Vespa 7.350, the flushing is staggered based on the size of the in-memory structures to minimize temporary memory use. This allows higher memory utilization and hence lower cost, particularly for applications with multiple large in-memory structures.

About Vespa: Largely developed by Yahoo engineers, Vespa is an open source big data processing and serving engine. It’s in use by many products, such as Yahoo News, Yahoo Sports, Yahoo Finance, and the Verizon Media Ad Platform. Thanks to feedback and contributions from the community, Vespa continues to grow.