Day 42 drills

Written on paper first, with numbers. The production AI chat system, broken into its decisions.

D1 (cost estimate: questions/sec, tokens/day, monthly bill, which tokens dominate):

D2 (retrieval quality: why retrieval not the model decides groundedness, a chunking mistake, why hybrid retrieval):

D3 (serving: two reasons to stream, why continuous batching, when to route to a smaller model):

D4 (cost control: semantic cache vs exact-match, token-based rate limiting, prompt caching):

D5 (reliability + security: provider outage fallback, where permission filtering happens and why not in the prompt):