In a recent benchmark, Vespa outperformed Elasticsearch in query efficiency, delivering:
-12.9x higher throughput for vector searches,
-8.5x for hybrid queries, and
-6.5x for lexical searches.

It also offers 4x greater efficiency in handling in-place updates, making it a robust choice for modern, scalable search solutions.

Search Modernization: To Replace or Not to Replace?

Elasticsearch is a cornerstone of the eCommerce market, valued for its powerful search and analytics capabilities and supported by a strong open-source community. However, as data and traffic scale, many users face challenges with complex configurations and high management costs. For fast-growing retailers like Vinted, these issues made sustaining the advanced search functionality needed to support their revenue growth difficult. Read Vinted’s Engineering Blog.

ECommerce search engines are a “sticky” technology. Replacing them is rarely straightforward, requiring careful technical, operational, and business evaluation. Complete visibility into the costs of the current system and the benefits of a potential replacement is critical before considering a replacement. Warning indicators of a search engine doomed for the scrap yard include:

1. Scalability and Performance Issues: Struggles with growing data volumes or complex queries.
2. Run-time Efficiency: Real-time updates or indexing disrupt searches.
3. Cost Optimization: High licensing or operational inefficiencies strain budgets.
4. Feature Limitations: It lacks advanced capabilities, such as integrating lexical and vector search with structured data filtering for complex, relevant search experiences.

These issues will have measurable business impacts, allowing the cost of poor quality (COPQ) to be calculated.

Bench Testing Alternatives

Bench tests support B2B enterprise software purchase decisions by providing a data-driven evaluation of performance, scalability, and compatibility under well-documented real-world conditions. They validate vendor claims, uncover bottlenecks, and estimate the total cost of ownership, ensuring the chosen software meets operational needs and reduces implementation risks. Inspired by Vinted’s modernization of Elasticsearch to Vespa and to simplify life for other organizations considering this path, Vespa conducted a comprehensive bench test comparison of the two products. While not independent, the test was thorough and fully documented in this 80-page report, with this concise management summary also available here for quick insights.

Vespa demonstrated a significant performance advantage over Elasticsearch, delivering order-of-magnitude improvements in query efficiency for vector searches and substantial cost savings through superior CPU efficiency. The bench test highlights Vespa’s substantial query performance:

Vespa’s Performance in Benchmarks:

• Hybrid Queries: 8.5x higher throughput per CPU core.
• Vector Searches: 12.9x higher throughput per CPU core.
• Lexical Searches: 6.5x better throughput per CPU core.
• Updates: 4x more efficient for in-place updates, maintaining efficiency for ongoing updates.

Updates: Vespa is 4x more efficient for in-place updates. While Elasticsearch performs well during initial writes (e.g., bootstrapping from 0 to 1M records), Vespa maintains superior efficiency for ongoing updates. Furthermore, Vespa proved greater CPU efficiency – directly reducing infrastructure costs, where the efficiency improvements yield up to a 5x reduction in infrastructure costs.

These results are not limited to eCommerce but apply broadly to other use cases. Vespa is particularly well-suited for applications with high query rates or frequent updates to data or signals. For example, personalized recommendations in media streaming platforms or behavior-based ad placement in ad-tech platforms.

Vespa in the Real World at Vinted

Vespa is a more scalable, efficient, and flexible solution for Vinted’s growing search demands, surpassing Elasticsearch in performance and capabilities. Vespa delivered substantial cost savings, operational simplicity, and improved search relevance, positioning it as the ideal platform for future scalability and innovation.

Here is a summary of how Vespa stacked up in the real world according to Vinted, all contributing to Vinted’s rapid growth and superior customer engagement.

Scalability and Performance

Vespa’s architecture provides superior scalability and performance compared to Elasticsearch. Vespa eliminates the need for complex shard and replica configurations by supporting horizontal and vertical scaling over fine-grained virtual content buckets. It delivers faster search experiences, reducing query latency by 2.5x and indexing latency by 3x. Items are indexed and searchable in just 5 seconds, a drastic improvement from Elasticsearch’s 300-second refresh interval. Furthermore, Vespa ensures even load distribution across nodes, avoiding bottlenecks and maintaining consistent performance as query volumes grow.

Data Handling and Indexing

Vespa excels in handling real-time data updates and reindexing processes. Unlike Elasticsearch, Vespa can continuously index new data without downtime or operational complexity, such as rebalancing shards or switching aliases. It supports real-time updates with a daily throughput of 10,300 RPS, ensuring that data is consistently up to date. The system’s ability to handle 1 billion searchable items efficiently demonstrates its capacity for managing large-scale indexing workloads.

Querying and Search Features

Vespa integrates advanced querying capabilities, combining lexical search, vector search, and structured data filtering within a single query framework. This flexibility enables more complex and relevant search experiences tailored to diverse use cases. Vinted achieved a threefold increase in ranking depth, boosting search result relevance and delivering significant business impact.

Cost and Operational Efficiency

Vespa significantly streamlined Vinted’s infrastructure by reducing server requirements from six Elasticsearch clusters to a single Vespa cluster with fewer, more powerful nodes. This consolidation lowered operational complexity and costs. Vespa’s efficient real-time indexing and low latency reduced the time and resources required to manage search infrastructure.

Summary

Vespa delivers faster, more scalable, and cost-efficient search capabilities both in the lab and in the real world. This enhances user experiences, reduces infrastructure costs, and enables businesses to operate more efficiently, positioning them for sustained growth and innovation. The fully transparent bench tests are available to help you modernize your eCommerce search engine.