Retrieval is the first model
Before tuning prompts, check whether the right passages are being retrieved for real questions. If the evidence is missing, no prompt will fix the answer.
Respect permissions at retrieval time
Filtering after generation is too late. Carry source permissions into the index so restricted content is never retrieved for the wrong person.
Build an evaluation set early
Collect real questions, expected sources, and cases that should be refused. Run the set on every change to chunking, models or prompts.
Operate it like any production service
Version prompts and models, monitor quality, latency and cost, and route user feedback into the evaluation set. Reliability comes from the loop, not the launch.