Search Debug
A powerful troubleshooting tool that visualizes the entire search pipeline — see exactly how your query flows through vector search, BM25, and fusion scoring without making an LLM call.
analyst manager superadmin
Overview
The Search Debug page runs the exact same search algorithm as production queries but skips the LLM call. This lets you inspect every stage of the pipeline to understand why certain results rank higher, why some are filtered out, and how to improve search quality.
- A user reports that a query isn't returning expected results
- You want to verify a newly uploaded document is searchable
- You're tuning relevance threshold, top-k, or search method settings
- You want to understand how pricing/spec keyword detection works
Debug Controls
| Control | Description |
|---|---|
| Query input | Type the search query to debug |
| Top-K | Number input — override the top-k value for this debug session |
| Sim Threshold | Slider + number input — visually re-filter results at a different threshold (simulation only, doesn't affect stored settings) |
| Sim Method | Radio buttons — simulate using a different search method (Hybrid / Vector / BM25) |
The 9 Debug Sections
Debug results are displayed in 9 collapsible sections, each showing a different stage of the pipeline:
1. Configuration Banner
Shows the current search settings used for this debug run:
- Top-K value
- Vector weight / BM25 weight
- Chunk size / overlap
- Search method (Hybrid / Vector / BM25)
- Relevance threshold
2. Query Detection
Shows whether pricing keywords or spec keywords were detected in the query. Displays the matched keywords with highlights.
3. Summary Cards
Seven metric cards at a glance:
| Card | Meaning |
|---|---|
| To LLM | Number of results that would be sent to the LLM |
| Passed | Results that passed the relevance threshold |
| Hybrid | Total hybrid fusion results (before filtering) |
| Vector | Raw vector search result count |
| BM25 | Raw BM25 search result count |
| Pricing | Pricing database matches (if pricing keywords detected) |
| Specs | Product spec matches (if spec keywords detected) |
4. LLM Context Sources
The final top-K ranked list of results that would be sent to the LLM as numbered sources. This is the most important section — it shows exactly what the AI would "see".
5. Threshold Filter
Shows every result with its score and whether it passed or failed the relevance threshold. Useful for understanding why certain results were excluded.
6. Hybrid Fusion
The combined scoring table showing how vector and BM25 scores were merged:
final_score = (vector_weight × vector_score) + (bm25_weight × bm25_score)Each row shows the individual vector score, BM25 score, and final combined score.
7. Vector Search (Raw)
Raw vector similarity scores from ChromaDB. Shows the embedding query that was sent and cosine similarity for each result.
8. BM25 Search (Raw)
Raw BM25 ranks from SQLite FTS5. Shows the tokenized query and BM25 rank for each result.
9. Pricing & Specs Results
If pricing or spec keywords were detected, this section shows the matched records from those databases.
Score Visualization
Scores are displayed as colored progress bars with the numeric value:
| Color | Score Range | Interpretation |
|---|---|---|
| Green bar | ≥ 0.5 | Strong match — highly relevant |
| Yellow bar | ≥ 0.3 | Moderate match — likely relevant |
| Red bar | < 0.3 | Weak match — may be filtered out |
Troubleshooting Tips
- 0 results — Check if documents are uploaded and VectorDB is synced. Verify the query doesn't contain only stop words.
- Low scores — Consider re-indexing documents with smaller chunk sizes for better precision, or adjust the relevance threshold down.
- Pricing not detected — Verify the query contains at least one pricing keyword (harga, price, berapa, etc.).
- Wrong results ranking high — Check FTS weights in the Schema Editor. Consider adjusting vector/BM25 weight balance.