Optimizing a Large-Scale Similarity Search System
A team is developing a feature to find the most similar items from a catalog of millions of entries. The entire catalog is converted into a key-value datastore, which is built only once from a fixed, unchanging set of training data. During live operation, the system needs to respond to user queries in milliseconds, but the team finds that searching the datastore for the nearest neighbors for each query is too slow, taking several seconds. Evaluate the team's current implementation strategy. What is the fundamental reason for the poor performance, and what change should they implement to meet the speed requirements, given the nature of their datastore?
0
1
Tags
Ch.2 Generative Models - Foundations of Large Language Models
Foundations of Large Language Models
Foundations of Large Language Models Course
Computing Sciences
Evaluation in Bloom's Taxonomy
Cognitive Psychology
Psychology
Social Science
Empirical Science
Science
Related
Optimizing a Large-Scale Similarity Search System
A development team is building a system that retrieves the most similar items from a very large, fixed collection of data vectors. To ensure fast retrieval times, they decide to pre-process the collection by creating a search index offline. Which characteristic of their setup is most critical for making this pre-indexing strategy a viable and efficient solution?
Rationale for Offline Indexing