How do metadata filtering and HNSW index parameters affect query latency in multi-tenant RAG systems? (Part 2 Focus)

Metadata filtering adds a constant O(F) overhead, while HNSW parameters scale the ANN search cost logarithmically.

1. Measure filter selectivity – run a dry‑run query with filter={"tenant_id": "t123"} and record the hit count. If selectivity < 5 % the filter is cheap; > 20 % it dominates latency.
2. Set HNSW efSearch – start at 40, increase until latency budget (e.g., 20 ms) is hit. Use the formula latency ≈ base + k·log(efSearch) where k≈0.3 ms for a 1 M‑vector index.
3. Adjust M (max degree) – higher M (e.g., 48) improves recall but adds ~10 % latency per 16 increase; keep M ≤ 32 for multi‑tenant shards.
4. Choose index scope – a shared index with tenant filter avoids duplication but incurs filter cost; a per‑tenant index eliminates filter latency at the expense of ~30 % more RAM.
5. Benchmark per tenant – run milvus CLI:

milvus benchmark search --collection rag_vectors --filter tenant_id=t123 --efSearch 80

Record 95th‑percentile latency.

Quick comparison

| efSearch | Avg latency (ms) | Recall@10 |
|----------|------------------|-----------|
| 40 | 12 | 0.92 |
| 80 | 18 | 0.96 |
| 120 | 27 | 0.98 |

Python example (Qdrant)

client.search(
    collection_name="rag_vectors",
    query_vector=vec,
    limit=10,
    filter={"must":[{"key":"tenant_id","match":{"value":"t123"}}]},
    hnsw_ef=80,
)

Monitor with Prometheus vector_search_latency_seconds_bucket and adjust efSearch per tenant when the 99th‑percentile exceeds the SLA.