We tested AI drafts for policy FAQs. Editing time told us more than output.
A six-week trial with non-sensitive inputs, a stopwatch, and one skeptical colleague.
We stopped counting drafts and started timing edits.
Illustrative example. This story and its author were written by the hr.cafe editors to demonstrate the What We Tried format. It does not describe a real person or event. Real, contributor-approved stories will replace it.
Key takeaways
- Only published policy text went into the tool: no employee data, cases or personal information.
- Drafts looked finished but needed substantial editing for tone, local practice and accuracy.
- Timing the edit, not counting the drafts, showed where the tool helped (structure) and where it did not (judgment).
- Role
- HR Generalist
- Organization
- Nonprofit, ~250 people
- Region
- United States
- Timeframe
- Six weeks
Context
We are two HR people supporting clinics and outreach teams. The same questions about leave, travel reimbursement and scheduling arrive every week, and our written FAQs were out of date. We wanted to know whether an AI writing tool could help us refresh them faster.
Hypothesis
If we gave the tool our already-published policy documents, it could produce FAQ drafts that would take less time to finalize than writing from scratch. We agreed up front that no employee names, cases, medical information or anything not already public to staff would go into the tool.
What we did
- Picked twelve frequently asked questions across three policies.
- For six, we wrote answers from scratch. For six, we started from an AI draft.
- Timed ourselves on each: from blank page (or draft) to “ready for a manager to read.”
- Asked a skeptical clinic manager to review all twelve without knowing which were which.
What we observed
The AI drafts looked finished on arrival. That was the trap. They used a tone our staff would find stiff, occasionally described practices from other jurisdictions, and twice stated something our policy did not say. Editing those drafts took roughly as long as writing from scratch for the complex questions, and noticeably less for the simple, structured ones.
The drafts were fastest where we needed help least, and slowest where judgment mattered most.
The clinic manager correctly identified four of the six AI-started answers, mostly, she said, because they “sounded like nobody in particular.” She preferred the edited versions of the simple ones.
Limits
Twelve questions and two writers is a tiny sample. We knew which answers were AI-started while editing, which may have changed how hard we looked. The tool, our prompts and our policies are specific to us; your results could differ in either direction.
What we will try next
We will use drafts only for structure on simple, procedural questions, and keep writing the answers that involve discretion ourselves. We also wrote down what we decided not to automate (anything touching individual circumstances) so the decision does not quietly erode when we are busy.
Questions for peers
Daniel Brooks would like to hear how this plays out in your context.
- What did you decide not to automate, and why?
- How do you measure whether an AI workflow saves time rather than moves it?
- What inputs do you consider safe to use in a trial like this?
Respond to Daniel
Responses are read by a moderator before anything is shown publicly, and only with your permission. There are no public responses on this story yet.