LiteIndex
Embedded, thread and process safe, disk backed, easy to use, query-able, fast Index (Cache) implementations
pip install --upgrade liteindex[all]
DefinedIndex
- Documentation | Detailed example | Benchmarks
- fixed schema index i.e: schema has to be defined before hand
number, boolean, datetime, string, compressed_string, blob, json, normalized_embedding, and other types are supported and can be queried upon- can store any python objects with varying levels of query-ability
Query language is subset of mongodb's. All queries are documented- seamless, very fast nearest neighbor search with filtering. can order by, filter by as usual with embeddings
- works across threads, processes seamlessly
- handy features like search and update in single query, batch operation support for update, search, del, pop etc ..
- in-place math operations support wherever applicable i.e: += etc can be done in single query optimally
- compression is supported natively and optional custom compression dictionaries can be built automatically
KVIndex
- Documentation | Detailed example | Benchmarks
- simple key value store, can store any python objects, can be queried for equality, sorting, max, min etc wherever applicable
- in-place math operations support wherever applicable i.e: += etc can be done in single query optimally
- has a python dict like interface
- batch operation support for update, search, del, pop etc ..
- ultra fast, works across threads, processes seamlessly
- Eviction policies supported:
LRU, LFU, any and age based invalidation and size, count based eviction
function_cache
- Documentation | Detailed example | Benchmarks
- based on KVIndex, can be used to cache function calls of any type
- batch inference caching friendly
