Engineering log

A voice that answers a real phone line has no forgiving failure modes.

Mzia answers calls for small businesses in Georgian, Armenian, Russian and English. This page is the working log behind it: what broke, why, what changed in the code, and what we measured instead of assumed. It is written for people who intend to check.

There are no uptime percentages and no compliance badges on this page. We are pre-revenue with modest traffic, so a percentage here would be decoration, and a badge would be for a certification we do not hold. Everything below is either linked to a live endpoint you can open yourself, or dated and named specifically enough to ask us about.

Live configuration reading…

Phone line

Answers the PSTN number. Never shares a process with anything public.

open the raw endpoint ↗

Browser demo

Public, unauthenticated, spends money on every connection.

open the raw endpoint ↗
The log

Eight things that broke, and what each one changed

Two of these never reached a caller because a probe caught them first. The rest did reach production, and each one is now a test that fails if the fix is removed.

29 JUL 2026 The browser demo answered 400 to every visitor — with 351 tests passing Outage
Symptom
Every attempt to start a browser demo failed at the WebSocket handshake. The phone line was unaffected.
Cause
Two WebSocketServer instances were constructed on one HTTP server, one per path. The ws library attaches an independent upgrade listener per instance, so both fired for every connection — and the one registered first aborted any path it did not own, before the second was ever consulted.
Why tests missed it
Every existing test asserted over fetch, and fetch never performs a WebSocket upgrade. The suite was green because it had never exercised the thing that was broken.
Fix
One upgrade router that dispatches by pathname, with both servers in noServer mode. Plus tests that open real sockets to both paths — verified by running them against the broken code first and watching all six fail.

The general lesson was cheap to state and expensive to learn: “the suite passes” and “a stranger can use it” are different claims, and only one of them was ever being made.

29 JUL 2026 The greeting played three times, over the top of itself Quality
Symptom
On a laptop with speakers on, the opening line came back stuttered and layered — the same words restarting across their own tail.
Cause 1
Speaker output re-entered the microphone. The model's voice-activity detection heard speech, concluded the caller had interrupted, and restarted the greeting. Browser echo cancellation does not prevent this: it cleans what we send, but the residue is still speech-shaped.
Cause 2
The browser scheduled audio ahead of real time and never kept a reference to it, so an interruption could not actually stop what was already queued. The restarted greeting therefore played on top of the original rather than instead of it.
Fix
A barge-in shield on the demo path — which the phone path had held since an earlier incident, and which the demo had simply never been given — plus explicit teardown of every queued audio buffer on interruption.

Two independent defects producing one symptom. Fixing either alone would have left it audibly broken, which is why the second was found only after the first stopped explaining everything.

28 JUL 2026 Mzia answered an Armenian caller in Georgian Behaviour
Symptom
A live call drifted across roughly twenty turns. The caller's speech was transcribed as Spanish, German and Hindi in turn, and the call ended in Georgian — a language nobody present was speaking.
Cause
Ours, not the model's. Our own system prompt instructed her to switch language the moment the caller appeared to speak another one. On a narrow-band phone line, transcription is unreliable about which language was spoken, so every misreading became a language switch. The model did exactly what it was told.
Fix
A closed language list; an explicit rule that an implausible language is a bad line rather than a fact; never inferring language from a name or an accent; switching only on two consecutive sensible sentences; and nonsense never changing the language at all.
Guard
Each of those rules has an assertion in the prompt linter, so deleting one fails the build. The assertions were verified by breaking the prompt on purpose.
28 JUL 2026 A parameter the SDK declares, the API rejects — it would have failed every call Caught pre-deploy
What
The vendor's own type definitions declare a transcription parameter that looked like the correct root-cause fix for the language incident above. The live API refuses it and the connection fails.
Consequence avoided
Every call would have failed to connect — a total outage of the line, three days before a competition semifinal.
How it was caught
By running it against the real API before deploying, not by review. The type was correct-looking and wrong.
Fix
The feature ships disabled, with the intended values kept in the code so re-enabling is a one-line change if the API ever accepts it, and a test that pins the default with the reason attached.

This is the third time in one week an authoritative-looking source was wrong in a way that would have reached production. It is the reason the working rule is now: types and documentation are hypotheses; the API is the fact.

27 JUL 2026 Every database write would have failed — silently Caught pre-deploy
What
Our Postgres driver uses prepared statements by default. The connection pooler we run through does not support them.
Consequence avoided
Every write would have failed into an error reporter that was switched off — a service reporting itself perfectly healthy while storing nothing at all. It would have been discovered days later, by noticing an empty table.
Fix
Prepared statements disabled for pooled connections, and a migration suite that runs against a real Postgres rather than a mock. A heartbeat write proves the path end to end on every deploy.
27 JUL 2026 A stale pointer would have made an information line answer as a dental clinic Data
What
A seeded pointer in the database aimed the active persona at a demo record for a Georgian dental practice. The relay reads that pointer on every call and swaps the deployed prompt for it.
Consequence avoided
The first consequence of switching call capture on would have been an information line answering attendees as a dentist, in the wrong language.
Fix
A migration that corrects the pointer, plus a per-tenant flag that pins a line to its deployed prompt so a database row cannot silently replace it.
28 JUL 2026 The silence watchdog interrupted her mid-sentence Behaviour
Symptom
During a long answer, she stopped and asked whether anyone was still there.
Cause
The silence clock only advanced when a turn completed. While she was speaking it stood still, so a long answer looked identical to a dead line.
Fix
The clock now follows her audio, and the watchdog stands down entirely while she is speaking. The regression test pins the bug's own arithmetic — a frozen clock reports a check-in at twenty seconds, a following clock reports none — and the opposite case is pinned too, so the fix cannot over-correct into never checking in.
28 JUL 2026 Background noise chopped her mid-word Quality
Symptom
Outdoors, wind and nearby conversation cut her off mid-sentence, and on the first turn restarted her whole introduction.
Cause
The model's default requires only 20 milliseconds of sound before treating it as speech. That is a reasonable default in a quiet room and useless anywhere else.
Fix
Two layers, deliberately not interchangeable: raised detection thresholds so speech must persist before it counts, and a client-side shield that refuses to act on an interruption arriving in the first moments of a turn — longer for the greeting, when the caller has said nothing yet. An interruption while she is silent is always honoured, because that is how someone takes the floor.

Stated honestly: our test signal is ambient room tone, and it proves the thresholds reject ambience. It does not prove they reject a neighbouring conversation, which is real speech. That case is what the second layer is for.

Architecture

A public endpoint cannot take the phone line off the air

The browser demo is unauthenticated, reachable by anyone, and spends money on every connection. The phone line is what a paying business depends on. They run as two isolated services that share no process, no memory and no failure.

Browser demo Anyone on the internet. Rate-limited, capped, budgeted. public
Phone line The number a business publishes. Unaffected by anything on the left. production

Deployed separately, on purpose

A crash, a traffic spike or a runaway cost on the demo cannot restart, slow or silence the line a business answers its customers on.

The server owns the clock

Session length is enforced server-side, never by the browser. A tab that is paused, throttled or lying still gets cut off at the same limit.

Risky changes ship behind a flag

Each defaults to the conservative setting, and every one is visible on the health endpoint above — so a misconfiguration is a page load rather than a phone call.

Cost control

A crash loop must not become a billing loop

Every second of the demo costs real money to a third party. The monthly ceiling is therefore stored as a row in the database, not a constant in the code.

A restart cannot reset the month

Spend is computed by summing the sessions that actually happened. A redeploy, a crash loop or a second instance all read the same number, because the number is the data.

Refused before, never after

A session that could breach the ceiling never starts. Checking afterwards means the ceiling is always exceeded by exactly one session.

Every refusal is a row

A month of silent refusals looks identical to a month of no interest. Recording them is the only way to discover the limit was set too low.

Raised without a deploy

Because it is data, the ceiling can be lifted in seconds when a campaign converts better than expected — deliberately, rather than discovered on a statement.

Threat model

Text we did not write is never allowed to give instructions

Anything placed into a language model's context is read the same way as our own instructions. A business description typed into a public form, or a page fetched from the web, is indistinguishable from us speaking — unless the distinction is made explicit.

Fenced as data

Untrusted text is wrapped in markers that tell the model, in the same breath, that the contents are data and can never change its role, its rules or its language. Fetched text cannot close the fence and escape.

Scrubbed and bounded

Instruction-shaped patterns are removed before framing, and length is capped so a payload cannot bury the real instructions under noise.

Constrained at the boundary

Values that reach the prompt outside the fence — a sector, a language — are matched against a fixed list, never echoed. An unrecognised value becomes a safe default rather than the stranger's own string.

Nothing outbound is caller-steered

The one external lookup we perform takes no caller input at all: its parameters are fixed. A caller cannot steer a request, smuggle text into one, or make us fetch a URL of their choosing. A test asserts none of their words appear in it.

Data handling

Deletion is enforced by code, not promised in a policy

Audio is never stored

No recording of any call is written to disk, at any point, in any mode. What a caller said exists only for the duration of the call.

Metadata by default

The default mode records that a call happened and how it went — never a word of what anybody said. Quality signals are computed in memory during the call; only the conclusion survives it.

Addresses are hashed, never kept

An IP is only ever needed as an equality check for rate limiting, so it is salted and hashed on the way in and the raw value is never written.

Ninety days, then gone

Retention is enforced by a purge that runs on a timer inside the service, in the EU. Not a calendar reminder, and not a sentence in a policy.

Method

Measured, not assumed

A public bug report claimed the model we run is unstable and recommended an alternative. Rather than switch on the strength of it, we ran both against our own audio, three rounds each, with identical prompts and configuration.

Model comparison on our own audio
Measured on our audioModel we runSuggested alternative
Time to first audio2,213 ms4,759 ms
Turn events per minute0.71.0
Self-interruptions on room tone00
Armenian script accuracy71%72%

The reported instability did not reproduce on our configuration, and the suggested replacement was 2.2× slower to speak — on a phone call, the difference between a receptionist and a lag. We stayed where we were, and the probe is still in the repository so the comparison can be re-run in front of anyone who asks.

9.1sFrom landing on the site to hearing her speak, on a laptop — two clicks
1Network request across nine representative questions; the rest are answered without leaving the process
$0Measured cost of call transcription — the pre-measurement estimate was wrong, and the measurement stands
Regression

Every incident above became a test

390Relay — call handling, cost guards, prompt rules, persona isolation
52Database migrations, run against real Postgres
74Dashboard
516Total, offline, no external calls

The count matters less than what the tests are attached to. They cover the WebSocket upgrade for both paths with real sockets; the budget ceiling and every refusal reason; language and anti-repetition rules asserted against the live prompt, so deleting a rule earned by an incident fails the build; prompt-injection fencing; multi-tenant persona isolation; and the migration order as it will actually run in production.

Two tests are excluded by default because they call a paid API. Regression tests are verified against the broken code before the fix lands — a test that has never failed has not been shown to test anything.

Honesty

What is not built

This is the section we would want to read first. Everything here is a real gap, stated plainly, because finding it later is worse than being told now.

No self-serve signup or billingA business is onboarded by hand today. There is no account system and no payment flow.
Phone numbers are provisioned manuallyEach number requires identity paperwork per line. It is not automated and will not be soon.
No error tracking wired upFailures are found by reading logs. A reporter is integrated but switched off, which is the wrong way round.
Owner alerts are stored, not sentWhen a caller asks for a human, it is recorded — but nobody is notified yet. A paying pilot needs this before anything else on this list.
The customer dashboard is built and untested in the wildIt exists and it passes its tests. It is not deployed, and an internal view is what is actually used.
One metric under-reports itselfArmenian is sometimes transcribed in Latin characters, which makes those calls score as English and undercounts the number of languages handled. Cosmetic to a caller, wrong on a slide.