- Every AI use catalogued and classified, with hiring uses treated as high-impact
- Candidates told when AI is used and how to ask for human review
- No rejection without a person's decision, recorded in the workbench
- Matching and summaries evaluated before each release, including across groups
- A traceable record of inputs, outputs and decisions for each application
Anonymised composite. Client names, identifying details and confidential figures are withheld; public facts are cited in the references.
The situation
A South Korean recruitment platform served employers and job seekers. It used machine learning to match candidates to roles and, more recently, a language model to summarise applications and draft screening notes for recruiters. Employers liked the speed. Nobody had written down which steps used AI, how they were tested, or what a candidate could do if they disagreed.
Why now. South Korea's AI Basic Act took effect on 22 January 202612. Commentary describes core transparency duties for generative and high-impact AI, with high-impact uses, including areas such as hiring, bringing duties around risk management and human oversight23. Observers note the Act tries to balance innovation with social risk, and that its details are being filled in through implementing rules4. With enforcement maturing and enterprise customers asking for assurances, the platform chose to build for the high-impact obligations now.
What had to be true
- Every AI use known and classified, with hiring uses treated as high-impact.
- Candidates informed when AI is involved, with a route to human review.
- A person decides: AI can rank and summarise, but no rejection without a recruiter's decision.
- Models tested before release, including for differences across groups of candidates.
- Evidence on demand for any application: inputs, outputs, model version and decision.
Architecture
The data flow, end to end
- ApplyCandidate applies and sees a notice explaining where AI is used
- MatchA versioned model ranks fit against the role's stated criteria
- SummariseAn LLM drafts a summary that cites the CV for every claim
- ReviewA recruiter sees AI output beside the source and decides
- RecordDecision, reviewer and model versions logged with the trace
- RespondCandidate can request human review or an explanation
The hard part: evidence without slowing recruiters down
Oversight fails when it becomes paperwork. The platform made evidence a by-product of normal work: every AI call was traced with OpenTelemetry's GenAI conventions, capturing the input, the output, the tools used and the model, so any application could be reconstructed later5. The reviewer workbench put the AI summary next to the original CV with citations, so checking it took seconds rather than minutes.
In high-impact AI, the log is part of the product. If you cannot reconstruct a decision, you cannot defend it, improve it or explain it to the person it affected.
Personal data in traces was minimised and access-controlled; summaries were stored with the application, not in a separate analytics copy. Evaluation followed the approach in evaluation sets that predict production, with checks for differences across candidate groups added to the release gate. See also LLM observability with OpenTelemetry.
How it ran
- Scope and risk classes
- Notices and review rights
- Employer contract terms
- AI inventory and registry
- Evaluation gate and tracing
- Reviewer workbench
- Workbench design input
- Decision guidelines
- Handling review requests
- Every AI use
- Risk classification
- Owners
- GenAI tracing
- Model registry
- Decision log
- Reviewer workbench
- No automated rejections
- Candidate notices
- Evaluation gate
- Review requests
- Employer reporting
Key decisions and trade-offs
| Decision | Alternative | Why |
|---|---|---|
| Treat all hiring AI as high-impact | Argue each use out of scope | Simpler, defensible, and what employers expected |
| No automated rejections | Auto-reject below a score | Keeps a person accountable for adverse decisions |
| Summaries must cite the CV | Free-text summaries | Reviewers can verify every claim quickly |
| Tracing as the evidence store | Separate compliance logging | One record for debugging, evaluation and accountability |
| Group-level checks in the release gate | Periodic audits only | Problems found before candidates are affected |
What I'd do again, and differently
Again: build the inventory and tracing first; everything else depended on them. Differently: design the candidate-facing review process earlier. Explaining an AI-assisted outcome in plain language was harder than producing the technical evidence. For broader governance context, see an AI governance programme across jurisdictions and ISO 42001 vs NIST AI RMF.
If you deploy AI in hiring, lending or other high-impact decisions in Asia-Pacific or elsewhere, my AI architecture work and regional pages explain how I approach it, or analyse your challenge and bring the result to a call.
- 1South Korea AI Basic ActInternational Trade Administration
- 2South Korea's AI Basic Act: overview and key takeawaysCooley · 27 Jan 2026
- 3AI Watch: global regulatory tracker, South KoreaWhite & Case
- 4
- 5Inside the LLM Call: GenAI Observability with OpenTelemetryOpenTelemetry blog (James Newton-King) · 14 May 2026