LLM Lease Auditing — Aryan Iyappan

Benchmarking LLMs for lease audit accuracy

Velocius AI Labs · Jul – Aug 2024

Benchmarking LLMs for lease audit accuracy

Analyzed multiple state-of-the-art LLMs to assess their effectiveness in automated lease auditing — including GPT-4o, Claude 3.5 Sonnet v1, Meta Llama 3.1 70B Instruct, and Meta Llama 3.1 405B Instruct. Findings informed fine-tuning and optimization of selected models, enhancing accuracy and efficiency, while identifying alternative models as backup solutions for robustness and flexibility.

Multi-Model Benchmarking

Evaluated GPT-4o, Claude 3.5 Sonnet v1, Llama 3.1 70B, and Llama 3.1 405B for automated lease auditing accuracy. Each model tested across real lease documents measuring extraction precision, reasoning quality, and hallucination rates.

Tags: GPT-4O, CLAUDE, LLAMA

Optimization & Redundancy

Findings informed fine-tuning of selected models and identified backup solutions for robustness in production workflows. Not just picking a winner — building a fault-tolerant model ensemble.

Tags: FINE-TUNING, REDUNDANCY