In the previous update, we mentioned new Container Thread Pools and Feed Throughput improvements.

Subscribe to the mailing list to get these updates delivered to your inbox.

This month, we’re excited to share the following updates:

Tensor Performance Improvements

Vespa 7.319.17 and onwards includes new optimizations to tensors with sparse dimensions. We have implemented new memory structures to represent sparse and mixed tensors and a new pipeline for evaluating tensor operations. This has enabled applications to deploy new advanced ranking models using mixed tensors in production. An example is a use case where end-to-end average latency went from 135ms to 13ms; a 10x speedup. When measuring the latency of only mixed tensor operations, the speedup is 150x. Latency improvement for basic sparse tensor operations is around 40%, while more advanced sparse tensor operations have a speedup of up to 50x.

Vespa Container Apache ZooKeeper Integration

Vespa allows you to add custom Java components for query and document processing. If this code needs a shared lock across servers in a cluster, you can now configure a container cluster to run an embedded ZooKeeper cluster and access it through an injected component. Read more

Pyvespa

pyvespa is a python library created to enable faster prototyping and facilitate Machine Learning experiments for Vespa applications. The library is under active development and ready for trial usage. Please give it a try and help the Vespa team improve it through feedback and contributions. Read more

ONNX Runtime

To increase Vespa’s capacity for evaluating large models, both in performance and model types supported, Vespa has integrated ONNX Runtime. This makes it easier to use both Vespa and ONNX, as there is no conversion. See the blog post for details.

About Vespa: Largely developed by Yahoo engineers, Vespa is an open source big data processing and serving engine. It’s in use by many products, such as Yahoo News, Yahoo Sports, Yahoo Finance, and the Verizon Media Ad Platform. Thanks to feedback and contributions from the community, Vespa continues to grow.