Notification Taxonomy — Retest Matrix
Run: 2026-06-11 14:28–14:33 UTC · tenant emergetulsa (prod) · build 0bb1249 · all test data cleaned after run
Routing config under test
| Contact | Leads | Bookings | Changes |
|---|
| admin@bookingnudge.com (Milton BN) | ON | off | ON |
| alonsomilton7@gmail.com (test) | ON | ON | ON |
Expected: Booking request → alonsomilton7@ only (Bookings route). Callback request → both (Leads route).
Result matrix
| # | Scenario | Expected | Observed | Verdict |
|---|
| S1 | Booking-intent chat capture + name extraction ("im Dana, 918-555-0177") | ONE [HOT] Booking request → alonsomilton7@, lead Dana/ready_to_book | Exactly that: lead Dana | 918-555-0177 | ready_to_book | book, 1 email, correct route, name in email | PASS |
| S2 | Callback ask, "im interested…" phrasing, name declined | ONE Callback request → both Leads contacts, no false name | No false name (stoplist held) BUT 2 leads + 3 emails: turn-2 fallback captured prematurely as ready_to_book → wrong [HOT] Booking email; turn-3 [HANDOFF] captured again → Callback ×2. No dedup. | FAIL |
| S3 | Form-path lead (POST /api/leads, no intent) | ONE classified email via Leads route | Classifier → interested → ONE Callback request event → both Leads contacts | PASS |
| S4 | Retraction: capture then "never mind, don't contact me" | Lead row deleted, transcript kept, no extra email | Capture → 1 [HOT] Booking email; retract → lead deleted, 4 chat_events kept, 0 extra emails | PASS |
Findings (2 bugs, both in the chat fallback path)
Bug A — premature fallback capture with wrong intent. S2 turn 2: visitor wrote "can someone call me back? my number is 918-555-0123". The LLM correctly deferred (reply asked for their name, no [HANDOFF] emitted), but the regex safety net at main.py:2701 fired anyway and its intent hint scanned the previous bot turn, which contained "confirmed at your appointment" (a pricing disclaimer) → matched \bappointment\b → tagged book/ready_to_book → sent a [HOT] Booking request for a visitor who explicitly asked for a callback.
Proposed fix: (1) an explicit callback ask in the user message ("call me back", "give me a call", "reach out to me") overrides the booking hint; (2) drop the previous-bot-turn text from the intent context — only the current exchange should vote.
Bug B — no dedup for phone-only leads → duplicate emails. The upsert guard on leads is ON CONFLICT (client_id, session_id, email) WHERE email != '' — phone-only captures (the common chat case) never conflict, so S2 produced two lead rows for the same phone in the same session and _notify_tenant_lead fired on both (3 emails total for one visitor).
Proposed fix: in _save_chat_lead, check for an existing lead in this session with the same phone (or email) first — update it (fill in name if newly learned) and skip the email; only a genuinely new contact share notifies.
Held up well: the name-extraction stoplist ("im interested" → no false name), retraction flow, intent-routed titles/routes on every correctly-classified event, exactly-one-email on S1/S3/S4, branded HTML + transcripts in every email.
S1 — Booking-intent capture + name extraction PASS
session taxrt-s1-082822
Hi, I would like to book an appointment for a facial
Great choice — we offer a few facial options depending on what your skin needs.
For a full-service experience, the Medical Facial & Peel is a popular pick — it combines clinical treatment with real visible results. We also have the Spa Facial for a more relaxing, restorative session, and DiamondGlow for exfoliation, extraction, and serum infusion in one.
Which of those sounds most like what you're after?
tomorrow afternoon would be great
Perfect — let me get you set up for a facial . Tap below to pick your exact time on our booking page.
im Dana, 918-555-0177
Thanks Dana — I'll make sure the team has your info. Tap below to lock in your afternoon slot tomorrow.
[BookingNudge] [HOT] Booking request — Emergetulsa
to: alonsomilton7@gmail.com · 2026-06-11 14:28:39 UTC · Resend status: delivered · id d73e02e4
S2 — Callback ask FAIL (duplicates)
session taxrt-s2-082944
do you guys do laser hair removal?
Yes — laser hair removal is one of our services at Emergetulsa. Pricing is customized based on the area you're treating, so it's confirmed at your appointment.
Who am I chatting with today?
im interested in learning more. can someone call me back? my number is 918-555-0123
Happy to have someone reach out — what's your name so I can pass that along?
id rather not share my name, just have someone call me please
No problem — I'll pass your number along and someone will be in touch soon.
[BookingNudge] [HOT] Booking request — Emergetulsa
to: alonsomilton7@gmail.com · 2026-06-11 14:29:54 UTC · Resend status: delivered · id 9ddaf85f · WRONG — premature fallback capture, misread as booking (Bug A)
[BookingNudge] Callback request — Emergetulsa
to: admin@bookingnudge.com · 2026-06-11 14:30:06 UTC · Resend status: delivered · id 1b42246c · correct type, but duplicate event (Bug B)
[BookingNudge] Callback request — Emergetulsa
to: alonsomilton7@gmail.com · 2026-06-11 14:30:06 UTC · Resend status: delivered · id e8b425d2 · correct type, but duplicate event (Bug B)
S3 — Form-path lead PASS
session taxrt-s3-083115 · POST /api/leads · name "Form Tester", phone 918-555-0188, interest Botox · classifier → interested
[BookingNudge] Callback request — Emergetulsa
to: admin@bookingnudge.com · 2026-06-11 14:31:16 UTC · Resend status: delivered · id 3267ce73
[BookingNudge] Callback request — Emergetulsa
to: alonsomilton7@gmail.com · 2026-06-11 14:31:16 UTC · Resend status: delivered · id fe3ffa88
S4 — Retraction PASS
session taxrt-s4-083147 · after retraction: lead row deleted, 4 chat_events kept, lead_retracted event logged, no extra email
hi, can I book a spa facial for friday? im Riley, 918-555-0199
Great to meet you, Riley — a Spa Facial sounds like a solid pick.
Tap below to grab a Friday slot on our booking page.
actually never mind, please dont contact me — ill just book online myself
No worries at all, Riley — booking online works perfectly. We'll be here if you ever have questions.
[BookingNudge] [HOT] Booking request — Emergetulsa
to: alonsomilton7@gmail.com · 2026-06-11 14:31:55 UTC · Resend status: delivered · id 6a3fd603 · sent pre-retraction — stays by design