Photo by ThisisEngineering RAEng on Unsplash

In the August updates, we mentioned NLP with Transformers on Vespa, Grafana How-to, Improved GEO Search and Query Profile Variants.

This month, we have several exciting updates to share:

ONNX-Runtime

We have completed integration with ONNX-Runtime in Vespa’s ranking framework, which vastly increases the capabilities of evaluating large deep-learning models in Vespa both in terms of model types we support and evaluation performance. New capabilities within hardware acceleration and model optimizations - such as quantization - allows for efficient evaluation of large NLP models like BERT and other Transformer models during ranking. To demonstrate this, we have created an end-to-end question/answering system all within Vespa, using approximate nearestneighbors and large BERT models to reach state-of-the-art on the Natural Questions benchmark. Read more.

Hamming Distance

The approximate nearest neighbor ranking feature now also supports the hamming distance metric.

Conditional Update Performance Improvements

Conditional writes are used for test-and-set operations when updating the document corpus. As long as the fields in the condition are attributes (i.e. in memory), the write throughput is now the same as without a condition, up to 3x better than before the optimization.

Compressed Transaction Log with Synced Ack

Vespa uses a transaction log for write performance. The transaction log is now synced to disk before the write ack is returned. The transaction log is now also compressed in order to reduce IO, and can improve update throughput by 10X if writing to attributes only.

In the News

Learn from the OkCupid Engineering Blog about how OkCupid uses Vespa to launch new features, ML models in query serving, simplify operations and cut deployment drastically: tech.okcupid.com/vespa-vs-elasticsearch-for-matching-millions-of-people

About Vespa: Largely developed by Yahoo engineers, Vespa is an open source big data processing and serving engine. It’s in use by many products, such as Yahoo News, Yahoo Sports, Yahoo Finance, and the Verizon Media Ad Platform. Thanks to feedback and contributions from the community, Vespa continues to grow.

We welcome your contributions and feedback (tweet or email) about any of these new features or future improvements you’d like to request.

Subscribe to the mailing list for more frequent updates!