We tested AI drafts for policy FAQs. Editing time told us more than output.

Context

We are two HR people supporting clinics and outreach teams. The same questions about leave, travel reimbursement and scheduling arrive every week, and our written FAQs were out of date. We wanted to know whether an AI writing tool could help us refresh them faster.

Hypothesis

If we gave the tool our already-published policy documents, it could produce FAQ drafts that would take less time to finalize than writing from scratch. We agreed up front that no employee names, cases, medical information or anything not already public to staff would go into the tool.

What we did

• Picked twelve frequently asked questions across three policies.
• For six, we wrote answers from scratch. For six, we started from an AI draft.
• Timed ourselves on each: from blank page (or draft) to “ready for a manager to read.”
• Asked a skeptical clinic manager to review all twelve without knowing which were which.

What we observed

The AI drafts looked finished on arrival. That was the trap. They used a tone our staff would find stiff, occasionally described practices from other jurisdictions, and twice stated something our policy did not say. Editing those drafts took roughly as long as writing from scratch for the complex questions, and noticeably less for the simple, structured ones.

The drafts were fastest where we needed help least, and slowest where judgment mattered most.

The clinic manager correctly identified four of the six AI-started answers — mostly, she said, because they “sounded like nobody in particular.” She preferred the edited versions of the simple ones.

Limits

Twelve questions and two writers is a tiny sample. We knew which answers were AI-started while editing, which may have changed how hard we looked. The tool, our prompts and our policies are specific to us; your results could differ in either direction.

What we will try next

We will use drafts only for structure on simple, procedural questions, and keep writing the answers that involve discretion ourselves. We also wrote down what we decided not to automate — anything touching individual circumstances — so the decision does not quietly erode when we are busy.