Ask a well-built RAG system "what does our refund policy say about digital goods?" and it will find the right paragraph. Ask it "what are the recurring complaints across last year's support tickets?" and it struggles. No single chunk contains the answer, so retrieving the top ten similar chunks doesn't help.
That second kind of question is what GraphRAG was designed for. I use it selectively, because it is powerful and it is not cheap.
How GraphRAG works
The approach comes from a 2024 Microsoft Research paper. It frames whole-corpus questions as query-focused summarisation rather than retrieval1. The method has two stages at indexing time and two at query time12:
- Extract a graph. An LLM reads the documents and extracts entities and relationships into a knowledge graph.
- Summarise communities. Closely related entities are grouped into communities, and the LLM writes a summary for each one in advance.
- Answer in parts. At query time, each relevant community summary produces a partial answer.
- Combine. The partial answers are summarised into a final response.
On global questions over corpora of around a million tokens, the authors reported substantially more comprehensive and diverse answers than a conventional RAG baseline1. Microsoft released an open-source implementation3.
Side by side
| Vector RAG | GraphRAG | |
|---|---|---|
| Best at | Specific questions answered by a passage | Themes, patterns and questions across the whole corpus |
| Indexing cost | Low: chunk and embed | High: many LLM calls to extract and summarise |
| Freshness | Easy to update incrementally | Harder; summaries go stale as content changes |
| Explainability | Cites passages | Cites entities, relationships and communities |
| Failure mode | Misses answers spread across documents | Extraction errors propagate into summaries |
The indexing cost is the main reason teams hesitate, and lighter variants have since appeared that aim to reduce it4. Check the current state of the library before budgeting.
An LLM-extracted graph is not an ontology
This is the most important caveat. GraphRAG's graph is extracted automatically from text. It is useful, but it is not governed: entity types vary, duplicates appear, relationships are whatever the model inferred. For exploration that is fine. For an enterprise system of record, it isn't.
The strongest setups I design combine both: a governed ontology and knowledge graph for the entities that matter, with resolved identities, and LLM extraction to enrich it from documents.
That means defining the core types and relationships first, as I describe in ontology vs knowledge graph vs semantic layer, resolving duplicates with the techniques in entity resolution for knowledge graphs, and only then letting an LLM attach document evidence to those entities.
A decision rule
- Mostly specific questions such as policies, procedures and product facts: use vector or hybrid search. Don't build a graph yet.
- Frequent "across everything" questions such as themes in research, risks across contracts or patterns in incidents: GraphRAG is worth piloting on a bounded corpus.
- Questions about known entities and their relationships, like which suppliers serve which plants or which patients are on which drugs: build a governed knowledge graph and query it directly, using the LLM to translate questions rather than to discover structure.
Whatever you choose, decide with an evaluation set that includes global questions, not just specific ones. Otherwise vector RAG will look better than it is.
How I can help
I design retrieval architectures that combine vector search, knowledge graphs and governed ontologies, and help teams decide which they need. See ontology and knowledge graphs, or analyse your challenge and bring the result to a call.
- 1From Local to Global: A Graph RAG Approach to Query-Focused SummarizationEdge et al., Microsoft Research (arXiv) · 2024
- 2
- 3GraphRAGMicrosoft, GitHub
- 4