Hiring in life sciences? Share your open positions with our professional community. Read more Close

Advertisement

Fact construction as a missing layer in legal AI: a practice-grounded framework for evaluating multi-agent legal consultation.

Created on 04 Nov 2026

Authors

Hsuan-Lei Shao, Ying-Chu Yu, Sieh-Chuen Huang

Published in

Frontiers in artificial intelligence. Volume 9. Pages 1896166. Epub Oct 20, 2026.

Abstract

Legal AI research often evaluates large language models (LLMs) on legal reasoning tasks under the assumption that relevant facts are already structured and available. This assumption overlooks a prior and practically consequential stage of legal work: fact construction, namely the transformation of incomplete, fragmented, and interactionally disclosed information into coherent factual narratives. This study conceptualizes fact construction as a distinct evaluation layer in legal AI and introduces a practice-grounded framework for assessing it in AI-assisted legal consultation. Rather than proposing a legal advice system or doctrinal rule-matching architecture, we use a role-separated multi-agent workflow consisting of a fact-constrained Litigant Twin, a structured-questioning LegalAider, a NarrativeAgent for factual synthesis, and an Evaluator for factual-alignment assessment. Using 100 de-identified legal clinic consultation records, we evaluate multiple LLMs under rubric-based assessment, comparing generated factual narratives against original human-authored consultation records in terms of factual accuracy, completeness, and consistency. The results show that LLM-based agents can generate usable factual narratives under constrained conditions, particularly when cases are internally consistent, limited to a single dispute, and supported by clear actor-role relationships. However, cross-model evaluator agreement remains moderate, and outlier analysis reveals recurring structural sources of failure, including anonymization-induced referential ambiguity, third-party reporting, multi-episode case complexity, and numerically dense facts. These findings suggest that reliability in AI-assisted legal consultation depends not only on model capability, but also on interaction design, data representation, institutional record-keeping, and human oversight. By treating fact construction as a distinct analytical layer, this article contributes a practice-grounded framework for evaluating responsible LLM integration in legal settings and highlights the need for factual grounding, contestability, and human-supervised review before downstream legal analysis begins.

PMID:
42978730
Bibliographic data and abstract were imported from PubMed on 04 Nov 2026.

Read full publication at:
Please sign in to see all details.

Advertisement

Stats

  • Community rating n/a 0 votes
  • Reviewers' rating n/a 0 votes
  • Your rating

1-terrible, 9-excellent. How would you rate this publication? Sign in in to submit your rating.

  • Recommendations n/a n/a positive of 0 vote(s)
  • Views 32
  • Comments 0

Recommended by

  • No recommendations yet.

Post a comment

You need to be signed in to post comments. You can sign in here.

Comments

There are no comments yet.

Advertisement