LLM Lease Auditing — Aryan Iyappan
Benchmarking LLMs for lease audit accuracy
Velocius AI Labs · Jul – Aug 2024
Benchmarking LLMs for lease audit accuracy
Analyzed multiple state-of-the-art LLMs to assess their effectiveness in automated lease auditing — including GPT-4o, Claude 3.5 Sonnet v1, Meta Llama 3.1 70B Instruct, and Meta Llama 3.1 405B Instruct. Findings informed fine-tuning and optimization of selected models, enhancing accuracy and efficiency, while identifying alternative models as backup solutions for robustness and flexibility.
Multi-Model Benchmarking
Evaluated GPT-4o, Claude 3.5 Sonnet v1, Llama 3.1 70B, and Llama 3.1 405B for automated lease auditing accuracy. Each model tested across real lease documents measuring extraction precision, reasoning quality, and hallucination rates.
Tags: GPT-4O, CLAUDE, LLAMA
Optimization & Redundancy
Findings informed fine-tuning of selected models and identified backup solutions for robustness in production workflows. Not just picking a winner — building a fault-tolerant model ensemble.
Tags: FINE-TUNING, REDUNDANCY