Settings
Configure your LLM provider, search method, tuning parameters, and chunk settings to optimize search quality and AI response behavior.
analyst manager superadmin
Overview
Settings are persisted in a dedicated SQLite database (settings.db).
Environment variables serve as initial defaults only — once you change a setting via the UI,
the stored value takes priority.
All Configurable Settings
| Setting | Type | Default | Range | Description |
|---|---|---|---|---|
| Model Provider | Select | bedrock | bedrock / gemini | Which LLM provider to use for generating AI responses |
| Search Method | Radio | hybrid | hybrid / vector / bm25 | Which search algorithm to use (see How Search Works) |
| Temperature | Slider | 0.7 | 0.0 – 1.0 | Controls LLM creativity. Lower = more factual, Higher = more creative |
| Relevance Threshold | Slider | 0.25 | 0.0 – 1.0 | Minimum score for search results to be included. Higher = stricter filtering |
| Top K | Number | 5 | 1 – 50 | Maximum number of search results sent to the LLM as context |
| Chunk Size | Slider | 500 | 100 – 10,000 | Number of characters per document chunk (affects new uploads only) |
| Chunk Overlap | Slider | 50 | 0 – 5,000 | Characters shared between consecutive chunks |
| Use LLM | Toggle | On | On / Off | When off, queries return raw search results without AI-generated answers |
| Streaming | Toggle | On | On / Off | When on, responses are streamed word-by-word via SSE |
LLM Provider
BABEH supports two LLM providers. You can switch between them at any time.
AWS Bedrock
| Property | Value |
|---|---|
| Default model | amazon.nova-lite-v1:0 |
| Also supports | Anthropic Claude models via Bedrock |
| Features | Sync + streaming responses |
| Auth | AWS credentials (Access Key, Secret Key, Region) |
Google Gemini
| Property | Value |
|---|---|
| Default model | gemini-2.0-flash-exp |
| Also supports | Any Gemini model (configurable via env var) |
| Features | Sync + streaming responses |
| Auth | Gemini API key |
LLM provider credentials (AWS keys, Gemini API key) are configured as environment variables on the server — they cannot be changed from the Settings UI. Contact your system administrator to update credentials.
Temperature Guide
Temperature controls the randomness/creativity of the AI's responses:
| Value | Behavior | Best For |
|---|---|---|
| 0.0 – 0.3 | Very factual, deterministic | Pricing queries, spec lookups, factual Q&A |
| 0.3 – 0.7 | Balanced (default: 0.7) | General queries, explanations |
| 0.7 – 1.0 | More creative, varied | Content generation, brainstorming |
Tuning Tips
- Low relevance threshold (0.1–0.2) — More results, but may include less relevant ones. Good for broad queries.
- High relevance threshold (0.4–0.6) — Fewer, more precise results. Good when you need accuracy.
- Higher Top-K (10–20) — Gives the LLM more context, useful for comparison queries.
- Smaller chunks (200–400) — Better for precise, specific questions.
- Larger chunks (800–1500) — Better for context-heavy, explanatory questions.
Save & Reset
- Save — Persists all current settings to the database. Changes take effect immediately for new queries.
- Reset — Reverts all settings to their default values.
Current settings can be read via the public API at GET /api/settings (requires API key). The widget and other integrations use this to auto-configure themselves.