The danger of artificial intelligence in legal work is not always that it produces an obviously incorrect answer. Sometimes, the greater risk is that it produces an answer that looks entirely credible. A judgment carries a familiar citation, the legal reasoning appears coherent, and the conclusion fits the question being asked. Yet the judgment may not exist, or the cited authority may say something altogether different.
Recent decisions of Indian courts have demonstrated that this is no longer a hypothetical concern. AI-generated or otherwise fabricated legal authorities have found their way into judicial and quasi-judicial proceedings, raising questions about verification, professional accountability and the limits of automated legal research.
For in-house legal departments, these developments deserve attention beyond the immediate issue of courtroom citations. Enterprises are increasingly experimenting with AI for contract review, legal research, dispute analysis and document management. The challenge is not simply whether these tools can improve productivity, but whether organisations have adequate safeguards to identify errors before they influence legal decisions.
When Fabricated Authorities Enter Judicial Proceedings
In July 2026, the Supreme Court addressed this problem in Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd.. The case concerned decisions of the National Company Law Tribunal and the National Company Law Appellate Tribunal that relied on fabricated or incorrectly attributed judicial precedents.
The Supreme Court set aside both decisions, holding that reliance on non-existent or hallucinated authorities compromised the integrity of adjudication. It emphasised the need for verification and directed the Bar Council of India to examine measures addressing the submission of fabricated precedents.
The Court also clarified an important distinction: legitimate AI assistance was not being prohibited. The concern was the presentation or reliance on fabricated material as genuine judicial authority.
The issue resurfaced in September 2026 in Vijay Ghanshyam Gadiya v. Union of India. The Supreme Court examined a customs adjudication order imposing a penalty of approximately ₹425 crore and found that several cited authorities were non-existent, incorrectly cited or did not support the propositions attributed to them.
The Court set aside the challenged orders and directed fresh adjudication. The significance of the decision extended beyond the immediate dispute. It reinforced that AI may assist legal decision-making, but cannot replace the responsibility to examine and verify the material on which a decision rests.
These decisions raise a practical question for corporate legal departments: if fabricated authorities can escape scrutiny within formal adjudicatory proceedings, what safeguards should enterprises establish before relying on AI-generated legal analysis?
Verification Must Extend Beyond Whether a Case Exists
A common response to fabricated citations is to require lawyers to check whether the cited judgment exists. That is necessary, but insufficient.
An AI system may identify a genuine case while attributing to it a proposition that the court never decided. It may combine reasoning from different judgments, omit qualifications or present observations as binding legal principles. These errors are harder to detect because the underlying authority is real.
In-house legal teams should therefore distinguish between citation verification and substantive verification.
The first establishes whether the judgment, statutory provision or regulatory instrument exists. The second examines whether the cited material actually supports the proposition for which it is being used.
A proper verification process should consider the relevant paragraph, factual context, jurisdiction, subsequent judicial treatment and applicability to the legal question under examination.
This becomes particularly important when AI-generated research informs litigation strategy, contractual interpretation or advice to senior management. An incorrect legal proposition may not immediately reach a courtroom, but it can still influence consequential commercial decisions.
Audit the Workflow Before Auditing the Tool
Legal departments often assess AI products through demonstrations of speed, drafting quality or document-processing capability. These indicators are useful, but they reveal relatively little about how the technology behaves within a complex legal workflow.
Before approving a rollout, an enterprise should identify precisely where AI-generated material will enter its legal processes.
Will the system summarise documents for internal reference? Will it recommend contractual language? Will it identify legal precedents? Could its outputs influence litigation assessments, settlement recommendations or regulatory responses?
Each use case carries different consequences if the information is inaccurate.
A document classification error may require correction. An invented precedent incorporated into a legal opinion may affect the advice given to management. An inaccurate interpretation of a contractual indemnity could influence the allocation of significant financial risk.
The appropriate level of scrutiny should therefore depend on the potential impact of the output rather than the apparent sophistication of the technology.
This risk-based approach also helps organisations avoid two extremes: allowing AI-generated content to pass without meaningful review, or imposing identical verification requirements on every low-risk administrative task.
Test the System Against Difficult Legal Questions
AI evaluation should not be limited to examples in which the correct answer is readily available.
A more meaningful assessment would test whether a system can handle conflicting authorities, similar case names, outdated precedents and questions for which no clear judicial answer exists.
Legal departments could create a controlled evaluation set using independently verified legal materials. The tests should examine whether the system correctly identifies authorities, accurately represents their reasoning and acknowledges uncertainty when reliable information is unavailable.
Equally important is how the system behaves when it cannot find an answer.
A tool that clearly states that no verified authority has been identified may be safer for legal research than one that produces a plausible citation to satisfy the user’s request.
Enterprises should also repeat these evaluations when underlying models, retrieval systems or data sources change. A satisfactory assessment at procurement does not guarantee that future versions will perform identically.
External Counsel Must Be Part of the Governance Framework
For many enterprises, legal work is distributed between internal teams and external law firms. AI governance that covers only employees therefore leaves an important gap.
In-house counsel should consider establishing clear expectations for the use of generative AI in externally commissioned legal research, opinions and litigation documents.
This need not involve prohibiting AI assistance. Instead, engagement protocols can require independent verification of cited authorities, appropriate protection of confidential information and clear responsibility for the final work product.
Where an external legal opinion materially influences a business decision, the enterprise should be able to establish that its supporting authorities have been verified, irrespective of whether AI was used during preparation.
The objective is not to transfer responsibility from external counsel to the client or vice versa. It is to prevent uncertainty about who is accountable for the accuracy of legal material supplied to the organisation.
Preserve Human Judgment Without Making Review a Formality
The requirement for human oversight is frequently included in AI governance policies. Yet the phrase has limited value unless organisations define what meaningful review involves.
A lawyer approving an AI-generated summary without examining the underlying documents may technically remain involved in the process, but that involvement offers little protection against substantive errors.
Meaningful oversight requires access to source material, adequate time for verification and clarity about which conclusions require independent professional judgment.
It also requires legal teams to recognise the difference between reviewing language and reviewing legal reasoning. An AI-generated document may be grammatically sound and internally consistent while resting on an incorrect interpretation of the law.
For higher-risk applications, organisations should document who reviewed the output, what material was checked and how significant discrepancies were resolved.
Such records are not a substitute for professional judgment. They help establish that judgment was actually exercised.
The Next AI Rollout Should Begin With an Accountability Question
The recent judicial decisions do not establish that AI has no place in legal practice. They demonstrate the consequences of treating generated material as authoritative without adequate verification.
For corporate legal departments, the lesson is broader than the problem of fabricated citations. AI systems can be useful for organising information, accelerating preliminary analysis and supporting legal professionals. But the reliability of the final work product depends on the controls surrounding their use.
Before the next rollout, legal leaders should be able to answer three questions: Can the organisation verify the sources behind an AI-generated conclusion? Can it identify where that conclusion enters a consequential legal decision? And can it establish who is responsible for reviewing and approving its use?
If those questions remain unanswered, faster legal research may simply mean that errors travel through the organisation more quickly.
The real measure of responsible legal AI adoption is not whether a system can produce a convincing answer. It is whether the legal function can establish that the answer is supported, appropriate for its intended purpose and subject to accountable human judgment.
