What 'Evidence-Grounded' Actually Means: A Field Guide for Buyers Tired of LLM Wrapper Demos

Every AI vendor that shipped a product in 2025 now claims their answers are "cited," "sourced," or "evidence-grounded." The marketing pages are full of footnote icons. The demos show provenance popovers. The buyer leaves the call feeling reassured.

In our experience, many of those claims don't survive a 20-minute technical review · because "evidence-grounded" isn't a binary, it's a spectrum, and most vendors are clustered near the soft end.

Many "evidence-grounded" marketing claims fail a 20-minute technical review. This is the field guide for telling the difference.

Section 1: The four grounding levels

Procurement teams need a vocabulary to judge vendor claims. Here's the one we use internally at PYRAMYD when evaluating partners and the one we recommend buyers use when evaluating us:

Level 0: Model-grounded (no retrieval)

The LLM answers from its training data. Citations, if any, are fabricated — the model invents plausible-sounding source URLs that may or may not exist. On the TruthfulQA benchmark, the best model tested (GPT-3) was truthful on only 58% of questions, against 94% for people.[1] Procurement red flag: any demo where the citation panel shows URLs but the vendor can't produce a request log showing those URLs were actually fetched.

Level 1: Document-grounded (vector RAG)

The system retrieves passages from a document corpus by cosine similarity, concatenates them into the prompt, and asks the LLM to answer with citations to the retrieved passages. This is the substrate of most "chat with your docs" products. The main failure mode is "lost in the middle"[2] · models use information at the start and end of their context far better than information in the middle, so answers get worse when the supporting evidence sits in the middle of the retrieved text.

Level 2: Node-grounded (typed-document RAG)

Documents are pre-processed into typed nodes — "Salesforce" tagged as a Vendor node, "CRM" tagged as a Category node, etc. Retrieval can filter by node type, not just similarity. The citation popover shows the node, not just the passage. Hallucination drops further because the LLM is constrained to talk about nodes it can name and link.

Level 3: Graph-grounded (Graph RAG)

The retrieval traverses typed edges between nodes — "What products did Salesforce ship last quarter" resolves via a graph walk from the Salesforce vendor node to its Product children to their Release events. Citations carry the full path. On questions that span a whole corpus, Microsoft's GraphRAG benchmark found graph-built answers more comprehensive than a conventional RAG baseline in 72-83% of head-to-head comparisons.[6] This is the bar PYRAMYD is built for.

Section 2: What a real citation looks like

A citation is real if · and only if · all four of the following hold:

• 1. The source URL was actually fetched. The request shows up in the vendor's retrieval logs, with a timestamp that precedes the LLM call. If the vendor can't show you the log, the URL was generated, not retrieved.
• 2. The cited passage supports the claim. Gao et al.'s 2023 citation-quality benchmark[3] distinguishes citation recall (each claim is fully supported by the passages it cites) from citation precision (no cited passage is irrelevant). Even the best systems it tested lacked complete citation support about half the time on one dataset. Always ask for both measures.
• 3. The retrieval timestamp is recent. A citation to a page that was fetched 18 months ago and the content has since changed is worse than no citation — it's a misleading one. Production-grade systems refresh on a defined cadence and surface the retrieval timestamp.
• 4. The provenance chain is auditable. NIST's AI Risk Management Framework[7] lists "accountable and transparent" and "explainable and interpretable" among its seven characteristics of trustworthy AI. If the audit log can't show which model produced the answer, with which prompt, against which retrieved set, at which time, the system is not auditable.

Section 3: The 7 disclosure controls procurement should require

Beyond the citation mechanics, there are seven structural disclosures any AI procurement contract should include. Several support the transparency and accountability goals in NIST's AI Risk Management Framework[7], and the first helps with the EU AI Act's Article 50 duty to tell people when they are dealing with AI[4]:

• 1. Model card. Which foundation models are in the inference path, at which versions, in which regions. Contract should include 30-day notice on changes.
• 2. Retrieval source manifest. Every data source the system can pull from, with refresh cadence and last-updated timestamp. Updated quarterly minimum.
• 3. Citation precision SLA. Vendor commits to a measured precision rate (e.g., >90%) on a defined benchmark — with quarterly third-party audit rights.
• 4. Hallucination disclosure. Vendor's self-reported hallucination rate, plus the test methodology. If they don't publish one, assume it's in the Level 0-1 range.
• 5. Audit log retention. Per-answer logs (prompt, retrieval set, model, citation set, user) retained for the duration of the contract plus 7 years for regulated buyers.
• 6. Exportable provenance. Your tenant data · including all citations, retrieval logs, and the substrate graph — is exportable in machine-readable format at any time. No lock-in on the audit trail.
• 7. Bias and fairness statement. Disclosure of any known systematic bias in the training data, retrieval corpus, or ranking algorithms. NIST's AI RMF names managing harmful bias as one of its seven characteristics of trustworthy AI; we recommend it as table-stakes for all buyers.

Section 4: Why the substrate matters more than the LLM

A common procurement mistake: buyers judge vendors on the LLM they use ("they ship on the newest frontier model, must be smart") instead of the substrate the LLM retrieves from. The substrate is the bottleneck.

In our experience, the retrieval layer limits answer quality more than the choice of model, so your AI procurement budget is better spent on substrate quality than on model premium.

Gartner predicts that over 40% of agentic AI projects will be cancelled by the end of 2027, citing escalating costs, unclear business value, and inadequate risk controls.[5] Our view: weak grounding feeds all three, because answers nobody can audit are hard to value and hard to control. The cancellations won't be because the LLMs got worse — they'll be because the grounding underneath was Level 0 or Level 1 when the regulator, the auditor, or the CFO asked Level 3 questions. The field guide above is how you avoid that outcome.