Technical due diligence for AI companies: the checklist I use for investors and acquirers

1. Does the AI actually work, and how do they know?

• Is there a maintained evaluation set? Real examples, labelled with expected outcomes, run on every model or prompt change. If quality is judged by demos and anecdotes, that's the single biggest red flag. I describe what good looks like in evaluation sets that predict production.
• Is reliability measured, not just average accuracy? For agents, a single success rate hides inconsistency. Research on τ-bench introduced pass^k, the chance of succeeding on every one of k attempts, and found that agents were notably less consistent than their single-run success rates suggested⁵. See evaluating AI agents.
• Is there production monitoring? Feedback, escalations, sampled reviews and drift checks, tied to the evaluation set.
• Can they show improvement over time, with numbers from the same evaluation set across releases?

2. What is actually defensible?

• Which parts are theirs? Separate the third-party model, open-source components and the company's own work: data pipelines, evaluation, domain logic, workflow integration, proprietary data.
• What happens when the underlying model improves? If a better general model would make the product unnecessary, the moat is thin. If better models make the product better, that's healthier.
• Is there a data advantage? Proprietary data, feedback loops from usage, or curated domain knowledge such as a governed ontology or knowledge graph are harder to copy than prompts.
• How portable are they across model providers? A gateway and evaluation set that let them switch models are both a risk control and a negotiating position. See build vs buy for AI.

3. Do they have the right to their data?

• Provenance. Where did training, fine-tuning and retrieval data come from, under what licence or agreement?
• Customer data use. Do contracts allow customer data to be used for training or evaluation? Is that used in practice, and is it consistent with the contract?
• Personal and sensitive data. How is personal, health or financial data handled in prompts, logs, embeddings and evaluation sets? Residency matters for some markets, as in UAE health data and Saudi PDPL.
• Deletion. Can they actually delete a customer's data from every store, including vector indexes and caches?

4. Is the AI secure and contained?

• Known risk categories. Has the team assessed the product against a recognised list such as the OWASP Top 10 for LLM Applications¹?
• Prompt injection. The UK NCSC has argued that prompt injection may never be fully mitigated the way SQL injection was⁶, so the question is containment: what can a compromised agent actually do? See prompt injection defence.
• Tool and data permissions. Are agent tools scoped per user, enforced outside the model, and logged? See securing MCP servers.
• Tenant isolation. Can retrieval ever return another customer's data? Ask for the test that proves it can't.

5. Is the engineering organisation healthy?

• Delivery metrics. Deployment frequency, lead time, change failure rate and recovery time, with trends. DORA's 2025 research found that AI tends to amplify an organisation's existing strengths and weaknesses⁴, so a team leaning heavily on AI coding tools without good foundations may be accumulating risk; see DORA metrics in the AI era.
• Key-person risk. Is the AI capability concentrated in one or two people? What's documented?
• Decision records. Are major architecture choices written down, as in architecture decision records?

6. Do the unit economics hold?

• Cost per outcome. Model, infrastructure and human review cost per resolved ticket, document or transaction, not just per token.
• Sensitivity. What happens to margins if usage grows ten times, or a provider changes pricing?
• Levers in use. Caching, routing and right-sized models, as in prompt caching and model routing. Unused levers are upside; no visibility at all is a risk.

7. What is the regulatory exposure?

• Classification. Could any use case fall into a regulated or high-risk category under the EU AI Act³, or sector rules in health, finance or employment?
• Governance evidence. An AI system register, risk assessments, incident handling and training. Frameworks such as the NIST AI RMF² are a reasonable yardstick; see ISO 42001 or NIST AI RMF.
• Customer commitments. What have they promised enterprise customers about AI, and can they evidence it?

Reading the results

No early-stage company will score well on everything, and that's not the point. Diligence should separate fixable gaps, such as missing monitoring or undocumented decisions, from structural problems, such as no data rights, no evaluation discipline or a product that a better general model would replace. The first belong in a post-investment plan with owners and dates. The second belong in the valuation conversation.

How I can help

I carry out technical due diligence on AI and software companies for investors and acquirers in the US, Europe and the Gulf, and help portfolio companies close the gaps afterwards as a fractional CTO. Or analyse your challenge and bring the result to a call.