Ontology vs knowledge graph vs semantic layer: what's the difference, and which do you need?

The short version

• An ontology is the model of meaning: what kinds of things exist in your business and how they relate. "A Customer places an Order. An Order is governed by a Contract."
• A knowledge graph is the data organised by that model: this specific customer, these orders, that contract, connected.
• A semantic layer is how that meaning is served to the people and tools that use it: consistent metrics, definitions and access rules for BI, APIs and AI agents.

One way to remember it: the ontology is the grammar, the knowledge graph is the sentences, and the semantic layer is the dictionary everyone looks things up in.

Ontology: the model of meaning

An ontology names the concepts in a domain, their properties and the relationships between them, along with rules. It sounds academic, but the problem it solves is very ordinary. In most organisations, "customer" means something different in sales, billing and support. "Active" means three different things depending on which dashboard you open. Reports never reconcile, and nobody can say which number is right.

A good ontology settles those questions once, in writing, in a form both people and machines can read. Formally it is often written in OWL or RDF Schema, but the format matters less than the discipline:

• Concepts have one definition, agreed with the business owner.
• Relationships are explicit: a Contract covers one or more Products; an Order is placed by exactly one Customer.
• It is versioned and reviewed like code, because the business changes.

The ontology is small. A useful first version for a mid-sized business often fits on one whiteboard.

Knowledge graph: the connected facts

A knowledge graph is your actual data, organised according to the ontology, as nodes and edges. Customer C-1042 placed Order O-88, which is governed by Contract K-7, which expires next March and was signed by a named account owner.

Building one is mostly a data engineering problem:

• Mapping source systems (CRM, ERP, documents, tickets) onto ontology concepts.
• Entity resolution: working out that "ACME Ltd", "Acme Limited" and customer ID 1042 are the same organisation. This is where most of the effort goes, and where most of the value is.
• Storage: a triple store if you use RDF and SPARQL, or a property graph such as Neo4j with Cypher. I choose based on the workload and the team, not on ideology.

The graph earns its keep on questions that cross systems: "Which customers on contracts expiring this quarter have open high-priority support tickets?" That query is painful across four databases and natural in a graph.

Semantic layer: meaning, served

A semantic layer is the interface that exposes consistent definitions to consumers. In analytics, that means metrics like revenue, churn and active users defined once and used by every dashboard. In applications, it means APIs that return business concepts rather than raw tables. For AI, it means agents query "contracts expiring this quarter" through a governed interface rather than writing their own SQL against production.

A semantic layer can sit on a warehouse without any graph at all, and many good ones do. It becomes more powerful when it is driven by the ontology, because then the definitions in your dashboards, APIs and AI tools come from the same source.

A worked example: Customer, Order, Contract

Here is a deliberately small slice of an ontology, written informally:

Customer   --places-->       Order
Order      --governedBy-->   Contract
Contract   --covers-->       Product
Contract   --ownedBy-->      AccountManager
Customer   --hasStatus-->    {Prospect, Active, Lapsed}

Rule: a Customer is Active if they hold at least one
      Contract whose end date is in the future.

Notice the rule. Before this existed, sales counted a customer as active if they had a meeting in the last 90 days, and finance counted anyone who had paid in the last year. Neither was wrong. They were answering different questions with the same word. The ontology forces the organisation to name both concepts separately, perhaps "Engaged prospect" and "Active customer", and the arguments about whose number is right largely stop.

The knowledge graph then holds the real customers, orders and contracts. The semantic layer exposes "Active customers" as a metric used by the board pack, the CRM and the AI assistant alike.

Why this matters for LLMs and RAG

Most retrieval-augmented generation starts with vector search: split documents into chunks, embed them, and retrieve the most similar chunks for each question. That works well for "what does our travel policy say about hotels?" It works badly for questions that depend on relationships, exact facts or permissions.

Ask a plain RAG system "which of my accounts renew this quarter?" and it will retrieve paragraphs that sound similar, then the model will guess. Ground the same question in a knowledge graph, an approach often called GraphRAG, and the system can:

• Resolve "my accounts" to the current user's AccountManager node.
• Traverse to Contracts with end dates in the quarter, as a precise query rather than a similarity match.
• Pull supporting documents only for those contracts, and cite them.
• Apply permissions from the graph, so the assistant never shows a contract the user cannot see.

In my experience this is the single biggest lever for moving an LLM assistant from impressive to trustworthy. I cover the rest of that journey in From RAG demo to production: a checklist.

Which do you need first?

In healthcare the standards do much of the ontology work for you. SNOMED CT, LOINC and RxNorm, and the OMOP common data model, are effectively shared ontologies. I wrote about that in FHIR to OMOP: a practical architecture.

Common mistakes

1. Modelling the whole world. Ontology projects die when they try to define everything before delivering anything. Model the concepts one valuable use case needs, ship it, then extend.
2. Letting engineers define business terms alone. The business owner must agree the definition of "active customer". Otherwise you have encoded one team's opinion.
3. Treating it as a one-off. The ontology needs an owner, versioning and a change process, or it drifts back into chaos within a year.
4. Skipping entity resolution. A graph full of duplicate customers is worse than no graph, because it looks authoritative.
5. Choosing the database first. The store should follow the model and the workload, not the other way round.

How I run an ontology project

The method matters more than the tooling. This is the sequence I follow, and it keeps projects small enough to deliver value within weeks rather than quarters.

1. Pick one use case with a visible payoff. A board metric that never reconciles, or an AI assistant that keeps getting a specific type of question wrong.
2. Run short domain workshops. Two or three sessions with the people who own the concepts. I bring a draft; they correct it. Sticky notes beat software at this stage.
3. Write it down formally. Concepts, definitions, relationships and rules, in a format that can be versioned. For teams new to this, a well-structured YAML or JSON definition reviewed in pull requests is a fine start before moving to OWL.
4. Map two sources, not ten. Prove the mapping and entity resolution on the systems that matter most for the use case.
5. Serve it. Expose the result through the semantic layer, an API or an agent tool, so real users touch it.
6. Assign an owner. Someone in the business, not only in IT, who approves changes to definitions.

After the first use case, each new one gets cheaper, because most concepts already exist. That compounding effect is the real business case for an ontology.

Tooling choices, briefly

People want a product recommendation. Here is my general guidance instead:

• For the model itself, OWL and RDF give you formal semantics, reasoning and standards alignment. Even if you serve data from something else, keeping the ontology in a formal language pays off when you exchange data with partners or regulators.
• For the graph, a property graph such as Neo4j is often the fastest route for application teams, with Cypher being approachable. Triple stores with SPARQL suit linked data and standards-heavy domains.
• For entity resolution, start with deterministic rules you can explain, then add probabilistic matching for the hard cases, with human review of uncertain matches.
• For AI, expose the graph to agents as tools with clear schemas, for example over MCP, rather than letting a model generate arbitrary queries against it.

Whatever you pick, keep the ontology independent of the store. Databases get replaced; meaning should outlive them.

How I can help

I design ontologies, knowledge graphs and semantic layers that business teams, analytics and AI agents can all rely on, and I use them to ground production AI systems. Start small, prove value on one use case, then grow.

If you are not sure which of these you need, the challenge analyser will suggest a starting point in two minutes, or see the ontology case study in my work.