View Markdown# Laxmi server: setup and operations guide
Everything needed to run the server and manage its models, start to end, with commands.
Last updated: 2026-09-21. Also on the running server at `/guide/setup`.
| Part | What |
|---|---|
| [1](#1-overview) | Overview: what runs where |
| [2](#2-prerequisites) | Prerequisites |
| [3](#3-first-time-setup) | First-time setup (venv, packages, `.env`, models) |
| [4](#4-start-the-server) | Start the server (`./start.sh`) |
| [5](#5-api-overview) | API overview, the browser API console, test commands |
| [6](#6-speech-transcript-asr-models) | Speech → transcript (ASR) models: engines, export, verify, serve, download |
| [7](#7-transcript-json-nlu-model) | Transcript → JSON (NLU) model: how it's trained, published, switched |
| [8](#8-languages) | Languages: adding one (e.g. Marathi) |
| [9](#9-tests) | Tests |
| [10](#10-maintenance) | Maintenance: database, nightly jobs, queries |
| [11](#11-troubleshooting) | Troubleshooting |
| [12](#12-file-map) | File map |
---
## 1. Overview
```
┌──────────────────────────── server/ (FastAPI, port 8082) ─────────────────────────────┐
audio ──► ASR ──► transcript ──► NLU ──► expense JSON ──► user saves ──► training labels ──► nlu_model/
ASR /v2/asr/transcribe, /v2/voice/parse engine "onnx": one FP16 ONNX pack per language (gu, hi)
engine "nemo": the multilingual .nemo (22 languages, e.g. Marathi)
NLU /v2/nlu/parse, /v2/voice/parse one laxmi_nlu.onnx file (gu, hi, en) + rules check
+ Gemini fallback (limited) + rules
data /v2/nlu/results what the user saved (online + offline) → nlu_events
files /v2/models the same model files, downloaded by the apps for offline use
langs /v2/languages supported languages + user requests for new ones
└──────────────────────────────────────────────────────────────────────────────────────┘
nlu_model/ (repo root) builds laxmi_nlu.onnx from server/data/laxmi.db + generated sentences:
label → train (Google Colab GPU or on the PC) → publish into server/models/nlu/<version>/
server-old/ the retired old server (reference only; nothing reads it)
```
**Rules:**
- **The server and the apps run the same model files.** The apps download them through `GET /v2/models` and run them
offline with ONNX Runtime.
- **The server is self-contained.** It has its own code, models (`models/`), Python environment (`.venv/`) and
database (`data/laxmi.db`).
- **Model files, data and `.env` are not in git.** `server/models/`, `server/data/`, `server/.venv/` and
`server/.env` are gitignored.
- **Every error has one shape:** `{"code", "message"}`. Every app endpoint needs the app key (`X-API-Key`), and
the device is always identified by the `X-Device-Id` header.
---
## 2. Prerequisites
| Need | Why | Check |
|---|---|---|
| macOS (or Linux) with Python 3.9+ | the server virtualenv (`server/.venv`) | `python3 --version` |
| `ffmpeg` (optional) | decode m4a / aac / mp3 uploads that libsndfile cannot read | `ffmpeg -version` |
| ~5 GB free disk | models (ONNX packs 0.5 GB, NLU versions 0.2 GB, `.nemo` files 3.5 GB) | `df -h .` |
| ~3 GB RAM | everything loaded (ONNX + NLU + `.nemo`); ~1 GB without the `.nemo` engine | |
| Gemini API key (optional) | the NLU fallback | `LAXMI_GEMINI_API_KEY` in `server/.env` |
| Google account with Colab (recommended) | train the NLU model on a GPU | <https://colab.research.google.com> |
Model files in `server/models/` (see 3.3 for how to get them):
| File | Used by |
|---|---|
| `asr/gu/`, `asr/hi/` (6 files each) | ASR engine `onnx`; also downloaded by the apps |
| `nemo/indicconformer_stt_multi_hybrid_rnnt_600m.nemo` (2.5 GB) | ASR engine `nemo` (22 languages) |
| `nemo/indicconformer_stt_gu_hybrid_rnnt_large.nemo`, `..._hi_...` | the source of the ONNX packs (export only, part 6) |
| `nlu/<version>/` + `nlu/latest.json` | the NLU model; also downloaded by the apps |
---
## 3. First-time setup
### 3.1 Virtualenv and packages
`start.sh` creates the venv on the first run if it is missing. To do it by hand:
```bash
cd server
python3 -m venv .venv
.venv/bin/pip install --upgrade pip
.venv/bin/pip install -r requirements.txt # API + ONNX engines (enough to serve gu / hi + NLU)
```
| Also needed for | Install |
|---|---|
| the `.nemo` engine and ASR export / verify (part 6) | torch + the AI4Bharat NeMo fork. The exact tested set is in `requirements-lock.txt`: `.venv/bin/pip install -r requirements-lock.txt` |
| training the NLU model on this PC (part 7) | `.venv/bin/pip install -r ../nlu_model/requirements.txt` |
On this Mac, `server/.venv` already has all of them.
### 3.2 `.env`
```bash
cd server
cp .env.example .env # first time only, then edit
```
The server reads `server/.env` **once at startup**; after editing it, restart. Every setting is optional, and
relative paths are relative to `server/`.
| Variable | Default | Meaning |
|---|---|---|
| `LAXMI_SUPPORTED_LANGUAGES` | `gu,hi` | languages offered in the app (`GET /v2/languages` → `supported`) |
| `LAXMI_ASR_ONNX_DIR` | `models/asr` | ONNX speech packs, one folder per language |
| `LAXMI_ASR_ONNX_LANGUAGES` | `gu,hi` | which packs are enabled |
| `LAXMI_ASR_NUM_THREADS` | `2` | CPU threads per ONNX session |
| `LAXMI_ASR_NEMO_PATH` | unset (off) | the multilingual `.nemo`; e.g. `models/nemo/indicconformer_stt_multi_hybrid_rnnt_600m.nemo` |
| `LAXMI_ASR_NEMO_LANGUAGES` | the 22 languages | languages the `.nemo` engine accepts |
| `LAXMI_ASR_NEMO_DIR` | `models/nemo` | single-language `.nemo` sources for the export / verify scripts |
| `LAXMI_ASR_STUB` | `false` | `true` = a fixed transcript, no models (tests / demo machines) |
| `LAXMI_NLU_REGISTRY_DIR` | `models/nlu` | NLU model versions |
| `LAXMI_NLU_MODEL_VERSION` | unset | **pin** the served NLU version (rollback); unset = `models/nlu/latest.json` |
| `LAXMI_NLU_LANGUAGES` | `gu,hi,en` | languages sent to the NLU model; others go to the fallback |
| `LAXMI_NLU_NUM_THREADS` | `2` | CPU threads for the NLU model |
| `LAXMI_NLU_CACHE_SIZE` | `2000` | answers kept in memory (the same sentence on the same day) |
| `LAXMI_NLU_REQUIRE_AMOUNT_EVIDENCE` | `true` | ignore a model amount when the transcript has no digits and the rules find no number word (returns amount 0 for review) |
| `LAXMI_GEMINI_API_KEY` | unset | Gemini fallback; unset = off (keep it secret) |
| `LAXMI_GEMINI_MODEL` | `gemini-3.1-flash-lite` | Gemini model name |
| `LAXMI_GEMINI_ENABLED` | `true` | allow the Gemini fallback |
| `LAXMI_GEMINI_DAILY_LIMIT_PER_DEVICE` | `10` | Gemini calls per device per day (UTC) |
| `LAXMI_GEMINI_MONTHLY_LIMIT` | `3000` | Gemini calls per calendar month; `0` = off |
| `LAXMI_GEMINI_TIMEOUT_SECONDS` | `6` | Gemini request timeout |
| `LAXMI_APP_API_KEYS` | unset (**app endpoints closed**) | **required.** The app key(s) the apps send as `X-API-Key`. Comma-separated for rotation: `new,old` (see 10.5). Unset = every app endpoint answers `503 api_key_not_configured` |
| `LAXMI_ADMIN_API_KEY` | unset | key for `/v2/admin/*` (`X-Admin-Key`); unset = admin endpoints off |
| `LAXMI_PUBLIC_BASE_URL` | unset | in production behind a proxy: the base of the download links, e.g. `https://api.example.com` |
| `LAXMI_DATA_DIR` | `data` | database, lexicons, audio |
| `LAXMI_AUDIO_RETENTION_DAYS` | `90` | how long opted-in audio is kept |
| `LAXMI_MAX_AUDIO_BYTES` | `10485760` | upload limit (10 MB) |
### 3.3 Models
`models/` is not in git. On a new machine, copy the whole `server/models/` folder from a machine that has it, with
the layout in part 2. Or build the pieces:
| Piece | How |
|---|---|
| ONNX speech packs | export them from the single-language `.nemo` files (6.3) |
| `.nemo` files | AI4Bharat IndicConformer checkpoints; copy them into `models/nemo/` |
| NLU model | train and publish it (part 7), or copy `models/nlu/` |
Check what the server sees: `GET /v2/health` (4).
---
## 4. Start the server
```bash
cd server
./start.sh # port 8082
./start.sh 9000 # another port
```
What `start.sh` does:
1. creates `.venv` if it is missing
2. prints the local and LAN URLs
3. runs `uvicorn app.main:app --host 0.0.0.0 --port <port>`
| Open | URL |
|---|---|
| Home page (status + links to all docs) | `http://127.0.0.1:8082/` |
| Interactive API docs (Swagger) | `http://127.0.0.1:8082/docs` (ReDoc: `/redoc`) |
| API reference | `http://127.0.0.1:8082/guide/api` |
| Mobile guide (for the iOS / Android team) | `http://127.0.0.1:8082/guide/mobile` |
| NLU training guide | `http://127.0.0.1:8082/guide/training` |
| This setup guide | `http://127.0.0.1:8082/guide/setup` |
| Changes from the old server | `http://127.0.0.1:8082/guide/changes` |
| **Admin: API console**: call every API from the browser, step by step | `http://127.0.0.1:8082/admin/console` |
| **Admin: review user edits** (admin key) | `http://127.0.0.1:8082/admin/reviews` |
| **Admin: models**: NLU switch / roll back, speech packs + .nemo details (admin key) | `http://127.0.0.1:8082/admin/models` |
| **Admin: word lists**: every training word, pattern and rule keyword (admin key) | `http://127.0.0.1:8082/admin/lexicon` |
| Health | `http://127.0.0.1:8082/v2/health` |
| From a phone on the same Wi-Fi | the `Network:` URL printed by `start.sh` (allow Python in the macOS firewall if asked) |
**Notes:**
- There is no auto-reload. After changing code or `.env`, stop the server (Ctrl+C) and run `./start.sh` again.
(`models/nlu/latest.json` is the exception: it is re-read on every request.)
- The first `/v2/nlu/parse` call loads the NLU model (~3 s). Later calls take ~90 ms on this Mac.
- The first `.nemo` request loads that model (~45 s). Later calls take ~1-2 s.
The old server can still be started for reference: `cd server-old && sh start.sh` (port 8081).
---
## 5. API overview
### 5.1 Endpoints
| Endpoint | Does |
|---|---|
| `GET /v2/health` | what is installed (ONNX languages, `.nemo`, NLU version, Gemini); loads no model |
| `POST /v2/asr/transcribe` | audio + `language` → transcript (`engine` auto / onnx / nemo, `romanize`) |
| `POST /v2/voice/parse` | audio + `language` → transcript → NLU → expense (+ `asr` block) |
| `POST /v2/nlu/parse` | transcript → NLU model → rules check → Gemini (limited) → rules → expense; stored in `nlu_events` |
| `POST /v2/nlu/results` | what the user saved: online (`event_id`) and offline entries, batched, safe to retry |
| `GET /v2/models` | downloadable models (ASR packs + NLU) with links, sha256 and `update_available` |
| `GET/HEAD /v2/models/{id}/files/{name}` | download one model file (Range supported) |
| `GET /v2/languages` | every language: supported, offline pack, server speech model, requested by this device |
| `POST /v2/languages/requests` | a user asks for a new language (one vote per device) |
| `GET /v2/admin/language-requests` | demand per language (`X-Admin-Key`) |
| `GET /v2/admin/training-data` | NLU events + what users saved (`X-Admin-Key`) |
| `GET/POST /v2/admin/reviews…` | the review queue of user edits: list, detail, approve / reject / skip, bulk (`X-Admin-Key`) |
| `GET/POST /v2/admin/nlu-models…` | NLU versions: list, check, switch (activate), roll back, compare on a phrase (`X-Admin-Key`) |
| `GET /v2/admin/asr-models`, `POST …/{lang}/check` | speech models: ONNX packs, the .nemo model, engine per language, export sources; pack file check (`X-Admin-Key`) |
| `GET /v2/admin/lexicon`, `POST …/lexicon/try` | the word lists per category (items, English loanwords, shops, rule keywords) + sentence patterns; what the rules make of a sentence (`X-Admin-Key`) |
| `DELETE /v2/user-data` | delete this device's audio, events, labels, edit history, reviews and language requests |
Every request, response and error code: [API reference](/guide/api). The endpoint map from the old server:
[changes](/guide/changes).
### 5.2 Documentation pages (not in Swagger)
| Route | Shows |
|---|---|
| `GET /` | home: live status, links, the endpoint list |
| `GET /guide/{api,mobile,training,setup,changes,readme}` | `server/docs/*.md`, this file and `README.md` as web pages |
| `GET /guide/<page>.md` | the same page as raw Markdown |
| `GET /docs`, `/redoc`, `/openapi.json` | FastAPI Swagger UI, ReDoc, OpenAPI JSON |
| `GET /admin/console` | the API console: guided flows + an explorer for every endpoint (asks for the keys; holds no data itself) |
| `GET /admin/reviews` (`/admin`) | the admin page for reviewing user edits (asks for the admin key; holds no data itself) |
| `GET /admin/models` | admin page, two tabs: **NLU model** (check, switch, roll back, compare) and **Speech models** (ONNX packs, .nemo, engine per language; read only) |
| `GET /admin/lexicon` | admin page: the words and sentence patterns the model is trained on, and the keywords the rules match; search, and a rules test box (read only) |
Edit the Markdown file and reload the page; no restart is needed.
### 5.3 API console (browser)
**`/admin/console`** calls every API from the browser, so you can see exactly how each one works and how the calls
chain together. Enter the app key, the admin key (for admin calls) and a test device id on the left. They are kept
only in that browser tab.
| Part | What it does |
|---|---|
| **Flows** (step by step) | the real call sequences of the apps, with each step's input and output handed to the next: **voice → expense → save → admin review** · typed text → expense → save · **offline entry → sync → retry** (duplicates) · app start: model update check → HEAD → resumable download (Range) · languages: picker → request → admin demand · speech only: compare the onnx / nemo / auto engines · security and errors (no key, wrong key, invalid body, …) · delete my test data |
| **Endpoints** | every endpoint, built from `/openapi.json` (always current): path / query fields, a JSON body with a working example, audio upload fields, an optional Range header |
| Every call | shows **Response** (status, time, headers, body), **Request** (method, URL, headers, body) and **curl** (ready to copy; keys shown as `$LAXMI_APP_KEY` unless "Show keys in curl" is ticked) |
| **Call log** | every call of the session, newest first |
| **Audio** | record in the browser (needs `localhost` or https) or upload a file. Recordings, and formats the server can't read without `ffmpeg` (m4a, webm, aac), are converted to 16 kHz WAV in the browser before upload |
The console writes real data under the test device id. Clean up with its **Delete my test data** flow.
### 5.4 Test commands
These were run against this server on 2026-09-21; the responses below are real. The same calls, with the request
and response shown step by step, are in the API console (5.3).
```bash
BASE=http://127.0.0.1:8082
KEY="X-API-Key: $(grep ^LAXMI_APP_API_KEYS= .env | cut -d= -f2- | cut -d, -f1)" # the app key (first one)
DEV="X-Device-Id: test-device"
# health: what is installed
curl -s $BASE/v2/health
# ASR only (romanize = also in Latin script)
curl -s -X POST $BASE/v2/asr/transcribe -H "$KEY" -F 'audio=@clip.mp3' -F 'language=gu' -F 'romanize=true'
# {"transcript":"કાલે ત્રણસો ની શાકભાજી ખરીદી","romanized":"kale tranaso ni sakabhaji kharidi","language":"gu",
# "engine":"onnx","model_version":"onnx-gu-fp16","confidence":0.98,"asr_event_id":"…"}
# voice → expense
curl -s -X POST $BASE/v2/voice/parse -H "$KEY" -H "$DEV" -F 'audio=@clip_hi.mp3' -F 'language=hi' -F 'reference_date=2026-09-15'
# {"amount":300.0,"category":"Fuel","note":"पेट्रोल","spent_at":"2026-09-14","language":"hi","source":"model",
# "transcript":"कल पेट्रोल पे तीन सौ रुपये खर्च किए", …, "asr":{"engine":"onnx", …}}
# text → expense
curl -s -X POST $BASE/v2/nlu/parse -H "$KEY" -H "$DEV" -H 'Content-Type: application/json' \
-d '{"transcript":"ત્રણ દિવસ પહેલા ચારસો ના કેળા લીધા","reference_date":"2026-09-15"}'
# {"amount":400.0,"category":"Groceries","merchant":null,"note":"કેળા","spent_at":"2026-09-12","language":"gu",
# "event_id":"527bd008…","transcript":"…","currency":"INR","confidence":1.0,"needs_review":false,
# "source":"model","model_version":"1.2.0","fallback_reason":null}
# the user saved it (use the event_id from parse)
curl -s -X POST $BASE/v2/nlu/results -H "$KEY" -H "$DEV" -H 'Content-Type: application/json' -d '{"entries":[{
"id": "test-result-0001", "event_id": "527bd008…",
"final": {"amount": 450, "category": "Groceries", "merchant": null, "note": "કેળા",
"spent_at": "2026-09-12", "language": "gu"},
"edited_fields": ["amount"], "confirm_time_ms": 2100, "edit_reason": "user_changed"}]}'
# {"saved":["test-result-0001"],"duplicates":[],"rejected":[]}
# what the apps check on start
curl -s -H "$KEY" "$BASE/v2/models?installed=nlu@1.1.0" # → nlu 1.2.0, update_available: true
# a language request
curl -s -X POST $BASE/v2/languages/requests -H "$KEY" -H "$DEV" -H 'Content-Type: application/json' -d '{"language":"Marathi"}'
# {"language":"mr","name":"Marathi","native_name":"मराठी","supported":false,"already_requested":false,
# "device_request_count":1,"total_devices":1}
# admin (key from .env)
curl -s $BASE/v2/admin/language-requests -H "X-Admin-Key: $(grep ^LAXMI_ADMIN_API_KEY= .env | cut -d= -f2-)"
# clean up the test device
curl -s -X DELETE $BASE/v2/user-data -H "$KEY" -H "$DEV"
# without the key: rejected
curl -s $BASE/v2/languages
# {"code":"missing_api_key","message":"Send the X-API-Key header."}
```
**`source` values:** `model` (the NLU model), `gemini` (fallback), `rules` (the deterministic parser).
**`fallback_reason`** explains why the model result alone was not used (e.g. `low_confidence; gemini_disabled`):
| `fallback_reason` part | Meaning |
|---|---|
| `unsupported_language` | the language is not in `LAXMI_NLU_LANGUAGES` (e.g. Marathi): the model is skipped |
| `rules_disagree_amount` / `rules_disagree_date` | the rules read a different amount / date from the sentence |
| `amount_not_in_transcript` | the model gave an amount but the sentence has none |
| `low_confidence` | the model is unsure |
| `invented_note` / `invented_merchant` | a value not in the sentence was removed; the result is returned for review, **Gemini is not called** |
| `amount_not_said` | the model returned amount 0 and the sentence has no number: returned for review (the app asks for the amount), **Gemini is not called** |
| `amount_missing` | the model returned amount 0 but the sentence has a number: Gemini is asked |
| `model_unavailable` / `invalid_model_output` | no NLU model served / its output was unreadable |
| `cache_hit` | (stored events only) the same sentence was answered before |
| `gemini_skipped` | Gemini not called because it cannot help (no amount said) |
| `gemini_failed` | Gemini did not answer (HTTP error, timeout, unreadable reply) |
| `gemini_no_amount` | Gemini answered but found no amount either |
| `gemini_invalid` | Gemini answered with an invalid value (e.g. an unknown category) |
| `gemini_disabled` / `gemini_*_limit_reached` / `gemini_circuit_open` | Gemini blocked by settings, budget or repeated failures |
---
## 6. Speech → transcript (ASR) models
### 6.1 Engines
| Engine | Model | Languages | Speed | Used when |
|---|---|---|---|---|
| `onnx` | one FP16 ONNX pack per language in `models/asr/<lang>/`: the same files the apps download | gu, hi | ~1 s | `engine=auto` and the language has a pack, or `engine=onnx` |
| `nemo` | the multilingual IndicConformer 600M `.nemo` (`LAXMI_ASR_NEMO_PATH`); needs torch + AI4Bharat NeMo | 22 Indic languages (Marathi, Tamil, …) | ~45 s first load, then ~1-2 s | `engine=auto` and no pack for the language, or `engine=nemo` |
| `stub` | a fixed transcript (`LAXMI_ASR_STUB=true`) | all | instant | tests / demo machines |
`language` is always required: the multilingual model cannot detect the language itself. For
`/v2/voice/parse`, the expense's `language` is the spoken one. (Marathi and Hindi share a script, so the transcript
alone can't tell them apart.)
**See it all on one page:** `/admin/models` → **Speech models** tab (admin key). It shows each ONNX pack (version,
export date, size, files, source `.nemo`, loaded or not) with a **Check files** button (size + sha256 against
`manifest.json`), the multilingual `.nemo` (path, size, toolkit installed, loaded, its 22 languages), which engine
each language uses, what works offline in the apps, and the single-language `.nemo` export sources. It is read
only: change speech models in `server/.env` and restart. It also warns about a pack that is enabled but
incomplete, or installed but not enabled.
### 6.2 How a pack is decoded (the server and the apps)
`app/engines/asr/onnx_pack.py` (`OnnxPack`) is the reference implementation:
1. 16 kHz mono float32 → `preprocessor.onnx` → `encoder.onnx`
2. frame-by-frame greedy RNNT search with `decoder.onnx` + `joiner.onnx`
3. the `tokens.txt` pieces joined
The on-device steps are in the [mobile guide](/guide/mobile#6-speech-model-on-the-device-asr-packs). Don't use
sherpa-onnx's own feature extraction with these packs: it drops words.
### 6.3 Packs: export → verify → serve → download
Run these from `server/`. The venv has the AI4Bharat NeMo fork.
**Step 1: export** (after every ASR model change). The sources are in `models/nemo/`:
```bash
cd server
.venv/bin/python scripts/export_asr_packs.py --langs gu,hi
```
It writes `models/asr/<lang>/`: `preprocessor.onnx encoder.onnx decoder.onnx joiner.onnx tokens.txt manifest.json`
(~261 MB per language, FP16 weights). The script checks itself against NeMo and stops on a mismatch. Options:
`--nemo-dir` (default `models/nemo`), `--out` (default `models/asr`).
**Step 2: verify** on real recordings. It must print `IDENTICAL`:
```bash
.venv/bin/python scripts/verify_asr_packs.py --lang gu clips_gu/*.mp3
.venv/bin/python scripts/verify_asr_packs.py --lang hi clips_hi/*.mp3
```
Last verified on 2026-09-22:
- **gu:** 3 of 3 clips IDENTICAL.
- **hi:** 3 of 4. In `audio_5.mp3` the pack writes **रुपये** where the `.nemo` writes **रुपए** (two spellings of
"rupees"; the amount and meaning are the same). The old server's script gives the identical result, so this is
the pack's FP16 rounding, not the port.
**Step 3: serve.** Restart the server (`./start.sh`). `GET /v2/health` lists the packs found.
**Step 4: apps download.**
```bash
curl -s -H "X-API-Key: <app key>" "http://127.0.0.1:8082/v2/models?type=asr&lang=gu" | python3 -m json.tool
curl -O -H "X-API-Key: <app key>" http://127.0.0.1:8082/v2/models/asr-gu/files/encoder.onnx
```
`version` (the export time) changes after each export, so the apps download again and check each file's `sha256`.
---
## 7. Transcript → JSON (NLU) model
Everything about the NLU model is in the **[NLU training guide](/guide/training)** (`docs/NLU_TRAINING.md`):
how the model works, training data, the golden set, labelling and the label rules, training on **Google Colab** or
**on the PC**, publishing, switching / rollback, and troubleshooting.
In short:
```bash
cd server && .venv/bin/python scripts/label_nlu_events.py # 1. label what users saved
cd nlu_model && python3 -m laxmi_nlu.colab_pack # 2a. Colab: zip → notebook → result zip
../server/.venv/bin/python -m laxmi_nlu.publish --result <result.zip> # publish
../server/.venv/bin/python -m laxmi_nlu.retrain --force # 2b. or all on the PC (publishes itself)
../server/.venv/bin/python -c "from laxmi_nlu import registry; registry.promote('<version>')" # 3. serve it
```
- **What `nlu_model/` uses:** it reads only this server's `data/laxmi.db` and publishes to `models/nlu/` (its
defaults).
- **Switching versions:** easiest from the admin page **`/admin/models`** (check → switch → roll back, with a history).
`models/nlu/latest.json` is re-read on every request, so a switch needs no restart; a `.env` pin overrides it.
To pin a version instead, set `LAXMI_NLU_MODEL_VERSION` and restart.
- **What the apps see:** only released versions (the served one and older ones). A newly published version stays
hidden until it is promoted or pinned.
### 7.1 Old server data (imported)
The old server's data was imported on 2026-09-21 with `scripts/import_from_server_old.py`: 35 NLU events,
36 v1 logs, 15 voice logs (+1 audio file). The script is safe to run again; it skips what is already there.
Checked afterwards:
- the dataset from the new database equals the one from the old database (byte-identical `train` / `val` / `hard`)
- the new labelling job gives the same labels as the old one on the same events (4 of 4, every column but the time)
- the Colab zip data equals the old setup's, apart from the label timestamp and `device_id`, which the zip removes
- both training flows (Colab, simulated locally, and PC `retrain`) ran end to end into a scratch model folder, and
promoting the result made the server serve it with no restart
---
### 7.2 User edits need approval
Since 2026-09-22 a result the **user edited** is never used for training automatically. Every change is recorded
(old → new) and the result waits on the admin page **`/admin/reviews`** until an admin approves it, possibly with
corrected values. Unedited results are labelled automatically as before. Details:
[NLU training guide, 4a](/guide/training#4a-step-2-review-what-users-edited-admin-page).
When this started, the 2 edited results already in the database moved from training to the review queue:
- one where the user saved Swagger test values (`"string"`)
- one where the user changed 300 → 500 while the transcript says ત્રણ સો (300)
The database before that is kept as `data/laxmi-before-review-gate-20260922.db`.
## 8. Languages
### 8.1 How languages work
| Where | Setting / source |
|---|---|
| Offered in the app | `LAXMI_SUPPORTED_LANGUAGES` → `GET /v2/languages` `supported` |
| Offline speech pack | `models/asr/<lang>/` + `LAXMI_ASR_ONNX_LANGUAGES` → `offline_asr` |
| Server speech | a pack, or the `.nemo` engine → `server_asr` |
| Expense extraction | `LAXMI_NLU_LANGUAGES` (the NLU model); other languages → Gemini → rules |
| Requests from users | `POST /v2/languages/requests` (one vote per device); demand: `GET /v2/admin/language-requests` |
### 8.2 Add a new language (e.g. Marathi)
1. **Look at demand:** `GET /v2/admin/language-requests`.
2. **Speech, server only:** this already works through the `.nemo` engine. `POST /v2/asr/transcribe` with
`language=mr` is served today.
3. **Speech, offline pack:**
1. Put `indicconformer_stt_mr_hybrid_rnnt_large.nemo` in `models/nemo/`.
2. Run `export_asr_packs.py --langs mr` and verify the result (6.3).
3. Add `mr` to `LAXMI_ASR_ONNX_LANGUAGES`.
4. **Expense extraction:** until the NLU model is retrained with Marathi, Marathi sentences go to Gemini (then to the
rules). Add the language to `nlu_model/` and retrain (training guide, "Adding a language"). Then publish, and add
`mr` to `LAXMI_NLU_LANGUAGES`.
5. **Offer it:** add `mr` to `LAXMI_SUPPORTED_LANGUAGES` and restart. Language requests for `mr` then answer
`supported: true`.
---
## 9. Tests
```bash
cd server
.venv/bin/python -m pytest -q # 49 tests
cd ../nlu_model
../server/.venv/bin/python -m pytest -q # 40 tests: the training pipeline
```
- **Offline:** the server tests never read `.env`. They use a stub speech engine, a fake NLU model, a fake Gemini
and a temporary database, so they need no models, make no network calls and don't touch `data/laxmi.db`.
- **Labelling:** `tests/test_labelling.py` holds the label rules (ported from the old server, with the same cases).
- **Real models:** checked by hand, with the API console (5.3), the commands in 5.4 and the verify script in 6.3.
---
## 10. Maintenance
### 10.1 Nightly jobs
| Task | Command |
|---|---|
| Training labels from what users saved | `.venv/bin/python scripts/label_nlu_events.py` (`--status`: what is ready for a retrain) |
| Delete opted-in audio past its retention date | `.venv/bin/python scripts/purge_expired_audio.py` |
With cron (`crontab -e`):
```
15 2 * * * cd /Users/imac/Documents/ios-project/laxmi-expense-manager/server && .venv/bin/python scripts/label_nlu_events.py >> data/labelling.log 2>&1
30 2 * * * cd /Users/imac/Documents/ios-project/laxmi-expense-manager/server && .venv/bin/python scripts/purge_expired_audio.py >> data/purge.log 2>&1
```
### 10.2 Database
SQLite at `data/laxmi.db` (the path `nlu_model/` reads). The tables are created on first use.
| Table | One row per | Written by |
|---|---|---|
| `asr_events` | transcription (engine, model, text, timing; audio path only if opted in) | `/v2/asr/transcribe`, `/v2/voice/parse` |
| `nlu_events` | server parse or offline entry: model / Gemini output, the answer returned, what the user saved | `/v2/nlu/parse`, `/v2/voice/parse`, `/v2/nlu/results` |
| `nlu_labels` | training label (gold / silver / skip) | `scripts/label_nlu_events.py` (unedited results), admin decisions (edited results) |
| `nlu_edits` | changed field: old value → new value, by the user or an admin | `/v2/nlu/results`, admin decisions |
| `nlu_reviews` | user-edited result: `pending` / `approved` / `rejected` / `skipped`, the rules' suggestion, the approved values | `/v2/nlu/results`, the labelling job, `/admin/reviews` |
| `language_requests` | (device, language), with `request_count` | `/v2/languages/requests` |
| `nlu_model_switches` | every NLU version switch / rollback: from → to, who, why, the checks it passed | `/admin/models` |
| `nlu_training_logs` | the old server's v1 parse logs | imported once; never written here, read by `nlu_model/` |
### 10.3 Useful commands
| Task | Command |
|---|---|
| Latest NLU events | `sqlite3 data/laxmi.db "select created_at, source, fallback_reason, transcript from nlu_events order by created_at desc limit 20"` |
| Gemini calls this month | `sqlite3 data/laxmi.db "select count(*) from nlu_events where gemini_called=1 and created_at >= strftime('%Y-%m-01')"` |
| Saved results (training signal) | `sqlite3 data/laxmi.db "select channel, count(*), sum(final_json is not null) from nlu_events group by 1"` |
| Labels | `sqlite3 data/laxmi.db "select label_quality, count(*) from nlu_labels group by 1"` |
| Review queue | `sqlite3 data/laxmi.db "select status, count(*) from nlu_reviews group by 1"` (or the page `/admin/reviews`) |
| Language demand | `GET /v2/admin/language-requests` (with `X-Admin-Key`) |
| Training data export | `GET /v2/admin/training-data` (with `X-Admin-Key`) |
| Delete a device's data | `DELETE /v2/user-data` with `X-API-Key` + `X-Device-Id: <id>` |
| Back up the database | `sqlite3 data/laxmi.db ".backup data/laxmi-backup.db"` |
### 10.4 Operations notes
- **Memory:** about 2.8 GB with everything loaded (the gu ONNX pack, the NLU model and the `.nemo`; measured
2026-09-21). Most of that is the `.nemo`, which loads on its first use. Leave `LAXMI_ASR_NEMO_PATH` unset on small
machines.
- **Gemini cost:** capped by the two limits. `fallback_reason` in `nlu_events` shows why each call happened.
- **Rate limiting:** there is none yet. Changing `X-Device-Id` gets around the per-device Gemini limit; the monthly
limit still caps cost.
### 10.5 The app key (`X-API-Key`)
Every app endpoint (everything under `/v2/` except `/v2/health` and `/v2/admin/*`) needs `X-API-Key` to match one
of the keys in `LAXMI_APP_API_KEYS`. It is checked **before** anything else runs: no audio decoding, model run or
database write happens for a caller without the key. Swagger (`/docs`) has an **Authorize** button for it.
| Task | How |
|---|---|
| Create a key | `python3 -c "import secrets;print(secrets.token_urlsafe(32))"` → put it in `server/.env` as `LAXMI_APP_API_KEYS=<key>`, restart |
| Share it with the app teams | through a password manager or a secret channel, never in chat, email, git or the docs |
| Rotate (e.g. after a leak) | 1. `LAXMI_APP_API_KEYS=<new>,<old>` and restart: both work. 2. Ship the app with the new key. 3. When old app versions are gone, `LAXMI_APP_API_KEYS=<new>` and restart |
| Check it | `curl -s http://127.0.0.1:8082/v2/languages` → `401 missing_api_key`; with `-H "X-API-Key: <key>"` → 200 |
**What it protects against:** other apps, scripts, and people who found the URL. **What it doesn't:** a key built
into an app can be extracted by someone who decompiles the app. When that matters, add on top:
- rate limits per device / IP (e.g. in the reverse proxy)
- HTTPS only (so the key is never sent in clear text)
- app attestation (Apple App Attest / Google Play Integrity)
- or per-user tokens after a login
- **Logs:** uvicorn's stdout.
---
## 11. Troubleshooting
| Problem | Cause / fix |
|---|---|
| `/v2/health` → `nlu_model_version: null`, or `fallback_reason: model_unavailable` | no NLU version is served: check `ls models/nlu` and `models/nlu/latest.json`, or pin `LAXMI_NLU_MODEL_VERSION` and restart |
| `/v2/health` → `asr_onnx_languages: []` | no complete pack in `models/asr/<lang>/` (all 6 files + `manifest.json`); export it (6.3) |
| `/v2/health` → `asr_nemo_enabled: false` | `LAXMI_ASR_NEMO_PATH` is unset or the file is missing |
| `400 unsupported_language` on speech | no pack and no `.nemo` for that language (or the forced `engine` has none) |
| `503 model_unavailable` on speech | the model could not load (e.g. torch / NeMo not installed for the `.nemo` engine) |
| `400 bad_audio` | the upload is not audio, or is m4a / aac without `ffmpeg` installed |
| `401 missing_api_key` / `invalid_api_key` | send `X-API-Key` with a key from `LAXMI_APP_API_KEYS` (after a rotation: the app has the old key) |
| `503 api_key_not_configured` | set `LAXMI_APP_API_KEYS` in `.env` and restart |
| `400 missing_device_id` | send the `X-Device-Id` header |
| `503 admin_disabled` / `401 unauthorized` | set `LAXMI_ADMIN_API_KEY` / send the right `X-Admin-Key` |
| changed `.env` or code but nothing changed | restart: `./start.sh` (there is no auto-reload) |
| `fallback_reason` ends with `gemini_disabled` | no `LAXMI_GEMINI_API_KEY`, `LAXMI_GEMINI_ENABLED=false`, or monthly limit `0` |
| `gemini_failed` often | Gemini is slow or down: `LAXMI_GEMINI_TIMEOUT_SECONDS=10`; the circuit breaker pauses Gemini after 3 failures |
| `gemini_daily_device_limit_reached` / `gemini_monthly_budget_reached` | the limits in `.env` were reached (by design) |
| Marathi (or another language) text treated as Hindi | send `language` in `/v2/nlu/parse`; voice always knows it |
| training: `train_by_source` has no `v2_label` | run `scripts/label_nlu_events.py` first ([training guide](/guide/training)) |
| `ModuleNotFoundError: pydantic` running `colab_pack` | the rule files in `app/engines/rules/` must stay standard-library only |
| `database is locked` | another process holds a long write: the database runs in WAL mode; stop scripts that keep it open |
| Port already in use | `./start.sh 9000`, or stop the other server (`lsof -iTCP:8082 -sTCP:LISTEN`) |
---
## 12. File map
```
server/
├── SETUP_GUIDE.md ← this file (also at /guide/setup)
├── README.md overview (/guide/readme)
├── docs/
│ ├── API_REFERENCE.md every endpoint (/guide/api)
│ ├── MOBILE_GUIDE.md iOS / Android integration (/guide/mobile)
│ ├── NLU_TRAINING.md NLU training: Colab + PC (/guide/training)
│ └── MIGRATION.md changes from the old server (/guide/changes)
├── start.sh start the server (./start.sh [port])
├── .env.example → .env settings (read at startup)
├── requirements.txt minimum packages; requirements-lock.txt = the exact tested set
├── app/
│ ├── main.py the FastAPI app: routers + error handlers
│ ├── config.py all LAXMI_* settings
│ ├── db.py SQLite connection + tables
│ ├── schemas.py request / response models
│ ├── errors.py deps.py error shape; X-API-Key / X-Device-Id / X-Admin-Key headers
│ ├── routers/ docs health voice nlu models languages admin user_data
│ ├── services/ nlu_store asr_store languages model_catalog labelling reviews (edit history + review queue)
│ ├── web/admin_reviews.html the admin review page (/admin/reviews)
│ ├── web/admin_console.html the API console (/admin/console)
│ ├── web/admin_models.html models page: NLU switch / roll back + speech models (/admin/models)
│ ├── web/admin_lexicon.html the training word lists + rule keywords (/admin/lexicon)
│ └── engines/
│ ├── audio.py decode + resample to 16 kHz
│ ├── asr/ onnx_pack (per-language packs), nemo_multi (.nemo), stub
│ ├── nlu/ model (laxmi_nlu.onnx), pipeline, verifier, matching, gemini
│ └── rules/ numbers_indic dates categories (standard library only: nlu_model loads them)
├── scripts/
│ ├── label_nlu_events.py what users saved → nlu_labels (+ --status)
│ ├── export_asr_packs.py .nemo → ONNX speech packs
│ ├── verify_asr_packs.py pack vs .nemo on real audio (must be IDENTICAL)
│ ├── purge_expired_audio.py retention clean-up
│ └── import_from_server_old.py one-off: the old server's data → data/laxmi.db (done 2026-09-21)
├── models/ (gitignored) asr/{gu,hi}/ nlu/<version>/ + latest.json nemo/*.nemo
├── data/ (gitignored) laxmi.db, audio/, indic_*.json lexicons
├── .venv/ (gitignored) Python environment (includes torch + NeMo)
└── tests/ pytest (part 9)
nlu_model/ (repo root) NLU model pipeline: README.md, colab/ (notebook)
server-old/ (repo root) the retired old server (reference only)
contracts/ (repo root) openapi.yaml (old server's contract), nlu-model-output.schema.json
```