Making a decades-old media archive findable with model enrichment and a controlled vocabulary

The situation

A media organisation held decades of video, audio and photography. The material was valuable for new programmes, licensing and research, but most of it was hard to find. Cataloguing conventions had changed several times, many items carried only a title and a broadcast date, and the same subject was described in a dozen ways. Search worked if you already knew the exact words a cataloguer had used years earlier, and otherwise depended on a few long-serving people who remembered where things were.

The organisation wanted to use AI to describe the backlog. The concern, rightly, was that a model left to itself would produce confident, inconsistent tags at a scale nobody could review.

What had to be true

• One controlled vocabulary, so a subject, place or person is described the same way across the archive.
• Machine suggestions, human authority: models propose, people decide where confidence is low or the subject is sensitive.
• Provenance on every field, so anyone can see whether a label came from a cataloguer, a model or a reviewer.
• Search by meaning and by exact term, for editors who know what they want and for those who are exploring.
• A taxonomy the archive team owns, not one hidden inside prompts or code.

Architecture

The vocabulary was modelled as a SKOS concept scheme, which gives each concept a preferred label, alternative labels and broader or narrower relationships¹. Descriptive fields such as creator, date, subject and rights were aligned with DCMI Metadata Terms so that records stayed portable between systems².

The data flow, end to end

For semantic search, transcripts and summaries were embedded with a sentence-embedding model, which produces vectors that can be compared with cosine similarity³. Nearest-neighbour lookup used an HNSW index, a graph-based approximate search method that scales well to large collections⁴. Model choice and storage trade-offs are covered in choosing embedding models and pgvector or a vector database.

The hard part: a vocabulary people agree on

Running models over the archive was the easy part. Agreeing what the labels should be took far longer. The approach that worked:

• Seed the concept scheme from the terms cataloguers already used most, then merge duplicates as alternative labels rather than deleting them, so old records still match.
• Use SKOS mapping properties to link local concepts to external vocabularies where they existed, instead of copying those vocabularies wholesale¹.
• Never let the pipeline create a concept. A model suggestion with no match becomes a proposal for the taxonomy owner, with example items attached.
• Treat people and sensitive topics as always-reviewed fields, whatever the model's confidence.

The taxonomy design choices are discussed further in SKOS vs OWL for enterprise taxonomies.

How the engagement ran

Thresholds were set against a hand-labelled evaluation set drawn from the pilot collection, not against the model's own sense of certainty. The method is in LLM evaluation sets that predict production.

Key decisions and trade-offs

What I'd do again, and differently

Again: put the vocabulary at the centre and make provenance a field, not a log file. Differently: involve the cataloguers in designing the review screen from the first week. The speed and quality of review decided the throughput of the whole system, far more than the model did.

If you're planning something similar, the ontology and knowledge graph page describes how I run these engagements, or analyse your challenge and bring the result to a call.