A legal AI system can explain a rule convincingly and still identify the wrong provision. Building a dependable compliance workflow requires accurate source identification, attention to legal context, and a way to preserve both as the law changes.
When I first began using large language models for legal research, I noticed a recurring problem: a model could explain the substance of a legal rule reasonably well, then attribute that rule to the wrong article.
Ask whether a regulation imposed a particular requirement, and the answer might be broadly correct. Ask where that requirement appeared, and the reference could lead to a provision dealing with something else.
For a lawyer checking an answer, this creates additional work. When the answer informs an automated compliance workflow, the consequences can extend into the design of the process itself. An incorrect or incomplete legal basis can become an inappropriate condition for onboarding a customer, updating a file or escalating a case.
My work on AML workflow automation has made this distinction particularly important. Every material legal conclusion needs a traceable basis, with its conditions, dependencies and temporal limits preserved.
The forthcoming application of the EU Anti-Money Laundering Regulation provides a useful example, and a practical starting point for thinking about how to engineer more reliable legal AI.
Why does legal AI get citations wrong?
Explaining a legal concept and identifying its exact source (legal basis) are two completely different tasks.
When an LLM generates an article number without resolving it against a source, that reference is a prediction. Familiarity with the surrounding legal language does not establish that the identifier is correct.
The May 2026 LegalCiteBench preprint illustrates this distinction. It evaluates citation recovery and related tasks using US judicial opinions and finds substantial difficulty with exact citation recovery. Its experiments are closed-book: they test models without external retrieval. The findings therefore concern a particular source-identification problem, rather than measuring the reliability of legal RAG products generally.
Retrieval-augmented generation, or RAG, helps by supplying external material to the model. But several failures remain possible:
- The system retrieves the wrong provision.
- It retrieves the right provision but attaches the wrong reference.
- It selects the wrong version of an otherwise relevant provision.
- It cites a genuine, relevant provision while overlooking a condition elsewhere.
- It draws a conclusion that the retrieved material does not support.
An August 2026 paper available on arXiv, How Much Do Legal RAG Systems Still Hallucinate?, examines eight RAG configurations across GDPR and French-language national civil-law material. Unsupported answers remain a problem in the systems tested, including responses to questions containing false assumptions. These are findings about specified configurations and datasets, not a universal error rate for legal AI.
A system can retrieve the right provision and still attach the wrong citation. It can also cite an existing article that does not support its conclusion. These errors require separate checks.
An AMLR example: can a low-risk customer be updated every five years?
Consider a financial institution designing its future customer identification workflow under Regulation (EU) 2024/1624 (the AMLR).
The question sounds straightforward:
Can we put low-risk customers on a five-year identification-update cycle?
First, the date matters. The AMLR generally applies from 10 July 2027, under Article 90. This example concerns implementation planning for that framework.
The relevant provisions perform different functions:
| Provision | What it contributes |
| Article 26(2), particularly point (b) | Updates follow a risk-based interval. Five years is the maximum for customers outside the higher-risk category covered by point (a). |
| Article 26(3) | Specified circumstances require review and, where relevant, an update. |
| Article 33(1)(b), (4) and (5) | Reduced identification-update frequency depends on qualifying for simplified due diligence, continued eligibility and the absence of circumstances excluding it. |
These provisions need to be read together. The five-year ceiling also extends beyond low risk customers. It is not itself a test for simplified due diligence.
The engineering implication is a workflow combining an appropriate schedule, event detection and eligibility reassessment. A single timer would leave material parts of that analysis unrepresented.
The “Article 23” citation trap
Both the AMLR and AMLA’s draft CDD RTS contain an Article 23, but the two provisions address different matters.
Article 23 of the AMLR concerns the timing of identity verification. Article 23 of AMLA’s draft CDD RTS, published for consultation on 9 February 2026, concerns identification-data updates in low-risk situations. An answer can therefore reproduce a genuine article number while attributing it to the wrong instrument.
The draft provision would require monitoring for changes, update-triggering events and activity inconsistent with the low-risk assessment. It also refers back to the AMLR’s update requirement. Its meaning depends on those connections.
Its status matters just as much. At the review date, AMLA’s regulatory register listed the CDD RTS as a draft with the consultation closed. The consultation text analysed here is a proposal, which will later “evolve” into a Commission Delegated Regulation.
The document nevertheless uses legislative formatting and mandatory language. A system that classifies legal authority from wording alone could mistake a proposed obligation for an applicable one.
A useful citation therefore needs the instrument, precise provision, version and status.
Why semantic relevance is insufficient
Searching legislation requires several kinds of matching.
| Retrieval method | Useful for | Limitation to address |
| Lexical search | Exact references, defined terms and distinctive statutory wording | The user may describe the issue using different language. |
| Semantic search | Connecting a practical question with conceptually related provisions | Similar subject matter does not establish applicability or completeness. |
| Explicit dependency links | Following definitions, conditions, exceptions and related instruments | The links can themselves be incomplete, incorrect or outdated. |
The 2025 Reasoning-Focused Legal Retrieval Benchmark shows how retrieval performance depends on the task and model. Structured query expansion improved several evaluated retrievers, while improvements were not uniform across models. That supports testing retrieval choices against the actual legal task rather than assuming one method will always dominate.
For the AMLR example, a passage discussing low-risk identification updates may be highly relevant. The system still needs to find the provisions governing review triggers and continued eligibility.
I would therefore combine lexical and semantic retrieval with source metadata and reviewed dependency links. Where the first retrieval leaves a material question unresolved, the system should retrieve further material or identify the gap.
This remains a form of RAG. Structured representations and dependency graphs can be components of a retrieval system.
What a structured legal model should preserve
In AML automation, I find it useful to start with a bounded decision: what, precisely, must the workflow determine?
That question determines which legal distinctions the representation needs to preserve.
Several concepts are often grouped together too loosely:
- XML or JSON provides a representation format.
- A taxonomy organises concepts and categories.
- A dependency graph records relationships between provisions or propositions.
- Executable rules specify how a system should act on particular inputs.
Writing legislation in XML does not, by itself, accomplish the other three tasks. Structured legal documents also have a substantial history: Akoma Ntoso, for example, is an established standard for representing legal and legislative documents.
For a bounded AML task, a useful starting point is a lawyer-reviewed map between the decision and its sources. A schematic XML representation might look like this:
<task id="customer-identification-update" purpose="AMLR-implementation-planning">
<legal-basis instrument="EU-2024-1624">
<provision ref="26(2)(b)" role="interval-ceiling"/>
<provision ref="26(3)" role="event-review"/>
<provision ref="33(1)(b)" role="reduced-frequency"/>
<provision ref="33(4)" role="continued-eligibility"/>
<provision ref="33(5)" role="exclusions"/>
</legal-basis>
<development document="AMLA-CDD-RTS-2026-02-09" provision="23" status="draft"/>
</task>This custom XML structure maps the update decision to relevant provisions and records the RTS as a draft. The accompanying legal model would capture the conditions, exceptions and assessments needed to apply those provisions to a particular customer relationship.
In an operational system, each reference should resolve to the preserved source text and its metadata, including the exact paragraph or point, language and version.
The accompanying legal model needs to retain distinctions that a summary can easily erase:
- whether an action is mandatory, permitted or prohibited;
- whether conditions are cumulative or alternatives;
- what activates an exception or ends an entitlement;
- which facts require assessment;
- where professional judgment remains necessary.
It should also distinguish the source text, the lawyer’s interpretation and the institution’s policy choice. Those layers may all influence the workflow, but they have different origins and review requirements.
A simplified representation is useful when it reduces the material needed for a defined task while retaining a route back to the complete sources. If it removes an exception or converts a discretionary assessment into an automatic permission, it has changed the legal meaning.
Automatically generated annotations need scrutiny for the same reason. Moving a proposition into a structured record does not establish its accuracy.
What citation verification should actually check
A reference-audit layer should work at the level of individual legal claims.
I would design it around four questions.
1. Does the reference resolve to the intended source?
Check the instrument, provision and version against the source repository. Generate citation labels from verified metadata wherever possible.
This can reduce invented or mismatched identifiers. It cannot determine whether the selected provision is the right one for the question.
2. Does the source contain the passage attributed to it?
Compare quotations with the preserved text. Check that extraction, translation or summarisation has not removed a qualification or attached text to the wrong heading.
Where meaning depends on the opening words of a paragraph or neighbouring provisions, preserve that context.
3. Does the passage support the proposition being made?
A source can discuss the correct subject without establishing the claimed permission, prohibition or obligation.
The assessment needs to consider the proposition’s scope, the facts assumed and the legal role of the source. A draft standard and an adopted rule must remain distinguishable throughout the process.
4. What else is needed to justify the conclusion?
Review the relevant definitions, conditions, exceptions and linked instruments. Record unresolved questions and the limits of the source coverage.
Completeness is difficult to establish. A system should explain the scope of its check rather than convert an incomplete search into an unconditional legal conclusion.
The July 2026 paper LegalCiteTrust, available on arXiv, distinguishes between citation existence, fidelity and applicability in Chinese legal research reports. Its jurisdiction and tasks differ from EU AML compliance, but the separation is useful when designing verification controls.
An LLM can assist with these checks, including identifying possible inconsistencies and missing qualifications. Its assessment still needs an evidential basis. Asking another model whether an answer “looks correct” is a weaker control than comparing specific claims with the underlying sources.
Likewise, a failed lookup should be recorded as a failed lookup. It may reflect a missing document, an identifier problem or incomplete database coverage. It does not automatically prove that the citation was fabricated.
The audit record should expose sources, assumptions, checks and decisions. A fluent explanation of an answer is not sufficient evidence of how reliably it was produced.
Version control belongs inside the legal analysis
A legal system needs to distinguish at least two timelines:
- When the provision applies to the circumstances being assessed.
- When the system obtained and reviewed the relevant material.
Those dates answer different questions.
A source retrieved today may contain an earlier rule. An adopted instrument may contain provisions whose application lies in the future. A historical question may require a superseded version.
The August 2026 paper Temporal Misgrounding in Legal RAG, available on arXiv, examines this problem using French tax legislation. Version-conditioned retrieval substantially improved performance on its deliberately selected, difficult temporal questions. The study supports treating dates and versions as retrieval inputs, while its results should not be generalised into an accuracy claim for unrelated legal systems.
For EU financial-services workflows, I would make jurisdiction, entity type, activity and relevant date explicit inputs. The source register should also distinguish EU instruments, national measures, court decisions, supervisory guidance and other interpretative material.
Each can have a different role in the analysis. Flattening them into interchangeable passages removes information the decision may depend on.
When a draft RTS is replaced, or an amendment changes a relevant provision, the system should identify which mapped decisions require review. Historical source versions and the records of earlier decisions should remain available.
This is also where a maintained dependency map becomes useful: it can help direct legal review towards affected workflows.
Start with a decision that matters
For an institution or regtech team, I would start with one operationally meaningful decision and ask:
- What facts must be established before the system can answer?
- Which sources support each part of the decision?
- Which conditions require continuing monitoring?
- What should happen when information, authority or interpretation remains unresolved?
- Who reviews the mapping when the law or the institution’s policy changes?
The resulting design may use retrieval, structured representations, conventional rules and LLM-assisted analysis in different proportions.
Fine-tuning may be worth considering for a bounded classification or extraction task. I would still preserve external source verification, version control and review of the legal mapping.
The architecture described here is a design proposal informed by the AMLR example and the research discussed. Its value would need to be established in the workflow where it is used.
For AML automation, a useful first deliverable is a decision whose legal basis can be inspected: the relevant sources are identifiable, the qualifications are visible, and unresolved issues reach someone authorised to decide them.
I advise on EU financial services regulation and work on legal engineering and compliance automation. To discuss a regulatory question, a compliance workflow or a legal AI project, get in touch.
Leave a Reply