$npx -y skills add skilld-dev/skilld --skill sqlite-vec-skilldALWAYS use when writing code importing \"sqlite-vec\". Consult for debugging, best practices, or modifying sqlite-vec, sqlite vec.
| 1 | # asg017/sqlite-vec `sqlite-vec` |
| 2 | |
| 3 | **Version:** 0.1.7 |
| 4 | **Tags:** latest: 0.1.7, alpha: 0.1.7-alpha.13 |
| 5 | |
| 6 | **References:** [package.json](./.skilld/pkg/package.json) — exports, entry points • [README](./.skilld/pkg/README.md) — setup, basic usage • [Docs](./.skilld/docs/_INDEX.md) — API reference, guides • [GitHub Issues](./.skilld/issues/_INDEX.md) — bugs, workarounds, edge cases • [Releases](./.skilld/releases/_INDEX.md) — changelog, breaking changes, new APIs |
| 7 | |
| 8 | ## Search |
| 9 | |
| 10 | Use `skilld search` instead of grepping `.skilld/` directories — hybrid semantic + keyword search across all indexed docs, issues, and releases. If `skilld` is unavailable, use `npx -y skilld search`. |
| 11 | |
| 12 | ```bash |
| 13 | skilld search "query" -p sqlite-vec |
| 14 | skilld search "issues:error handling" -p sqlite-vec |
| 15 | skilld search "releases:deprecated" -p sqlite-vec |
| 16 | ``` |
| 17 | |
| 18 | Filters: `docs:`, `issues:`, `releases:` prefix narrows by source type. |
| 19 | |
| 20 | <!-- skilld:api-changes --> |
| 21 | ## API Changes |
| 22 | |
| 23 | This section documents version-specific API changes — prioritize recent major/minor releases. |
| 24 | |
| 25 | - BREAKING: DELETE operations now properly clear vector data and free space — v0.1.7 changed behavior from only setting validity bits. Code using DELETE statements may see different storage behavior [source](./.skilld/releases/v0.1.7.md:L16) |
| 26 | |
| 27 | - NEW: Distance column constraints in KNN queries — v0.1.7 adds support for `>`, `>=`, `<`, `<=` constraints on the distance column, enabling pagination-like patterns without requiring large k values [source](./.skilld/releases/v0.1.7.md:L17) |
| 28 | |
| 29 | - NEW: Metadata columns in vec0 virtual tables — v0.1.6 added ability to declare metadata columns that can be filtered in WHERE clauses of KNN queries alongside vector matching [source](./.skilld/releases/v0.1.6.md:L13-27) |
| 30 | |
| 31 | - NEW: Partition keys for internal index sharding — v0.1.6 added `partition key` syntax to internally shard vector indexes by column values [source](./.skilld/releases/v0.1.6.md:L23-24) |
| 32 | |
| 33 | - NEW: Auxiliary columns with `+` prefix — v0.1.6 added support for auxiliary columns (prefix with `+`) that are unindexed but available for fast lookups in KNN query results [source](./.skilld/releases/v0.1.6.md:L31-33) |
| 34 | |
| 35 | - BREAKING: `vec_npy_each` table function removed from default entrypoint — v0.1.3 moved this experimental function out due to CVE-2024-46488 security mitigation; affected code using untrusted SQL or the rare `vec_npy_each` function [source](./.skilld/releases/v0.1.3.md:L9) |
| 36 | |
| 37 | **Also changed:** Static linking support for SQLite 3.31.1+ · `serialize_float32()` / `serialize_int8()` Python functions added |
| 38 | <!-- /skilld:api-changes --> |
| 39 | |
| 40 | <!-- skilld:best-practices --> |
| 41 | ## Best Practices |
| 42 | |
| 43 | - **Use two-column re-scoring pattern for binary quantization** — store both quantized and full-precision vectors; query coarse index with quantized vectors, then re-score top candidates with full precision to recover quality lost from extreme dimensionality reduction [source](./.skilld/docs/binary-quant.md#re-scoring) |
| 44 | |
| 45 | - **Combine `vec_slice()` with `vec_normalize()` for Matryoshka embeddings** — truncating dimensions requires subsequent normalization to maintain embedding quality and semantic meaning [source](./.skilld/docs/matryoshka.md#matryoshka-embeddings-with-sqlite-vec) |
| 46 | |
| 47 | - **Prefer scalar quantization over binary quantization for moderate storage savings** — trade off storage efficiency against quality loss; `vec_quantize_float16` (2 bytes per value) and `vec_quantize_int8` (1 byte per value) offer better quality retention than binary quantization for many use cases [source](./.skilld/docs/scalar-quant.md#L1:26) |
| 48 | |
| 49 | - **Use partition keys to shard large vector datasets** — declare a `partition key` column in `CREATE VIRTUAL TABLE` to internally shard the vector index on that column, improving query performance by reducing search scope [source](./.skilld/releases/v0.1.6.md#L23:24) |
| 50 | |
| 51 | - **Combine metadata columns (indexed) with auxiliary columns (unindexed) for efficient filtering** — use regular metadata columns for dimensions you filter on in KNN WHERE clauses; prefix columns with `+` to store related data without indexing overhead [source](./.skilld/releases/v0.1.6.md#L26:33) |
| 52 | |
| 53 | - **Use distance constraints instead of oversampling for pagination** — as of v0.1.7, apply `distance > threshold` or `distance < threshold` constraints in WHERE clauses to paginate through KNN results without fetching excess candidates [source](./.skilld/releases/v0.1.7.md#L17) |
| 54 | |
| 55 | - **Monitor the k value limit when performing large KNN queries** — the default maximum k is 4096 (configurable) to prevent memory exhaustion; be aware that kNN results are materialized in memory and internally use O(n²) complexity on k [source](./.skilld/issues/issue-157.md#L22:33) |
| 56 | |
| 57 | - **Rely on v |