Divinci AI Blog

Insights, updates, and thought leadership on artificial intelligence, RAG systems, and enterprise AI management.

Product

How to Diagnose Custom LLM QA Failures in 7 Steps

By Mike Mooring • May 31, 2026

Most 'QA failures' aren't model failures — they're eval gaps, judge mis-calibration, or training-serving skew. A 7-step diagnostic that proves it.

Read Article

Research notes from the Divinci AI team

The Divinci AI blog publishes original research, engineering deep-dives, and field notes from our work building production retrieval-augmented generation systems for regulated industries. We write about the vector indexes — we call them vIndexes — that power our RAG architecture, the routing logic that selects the right one for each question, the evaluation harness we use to benchmark them against frontier LLMs, and the hard-won lessons of deploying these systems where correctness, citations, and audit trails actually matter.

Our coverage spans three threads. First, technical work: how we build, score, and route across our open library of vIndexes published on Hugging Face; how we calibrate AI judges against human ground truth; what we learned shipping speculative decoding and FP8 quantization to production. Second, applied research: comparing retrieval strategies head-to-head in the RAG Arena, measuring grounding rates, and the economics of inference on commodity hardware. Third, perspective: where the field is going, what regulators are asking for next, and how teams should think about AI systems as the boundary between training and inference dissolves.

Most posts target an audience already fluent in AI engineering — engineers, researchers, and technical buyers — though we try to keep the framing accessible to anyone responsible for AI decisions at their organization. New posts publish irregularly when we have something genuinely worth saying. You can also find our open models, datasets, and vIndexes on our Hugging Face profile, and the source for our research tooling on GitHub.

Divinci AI Blog

How to Diagnose Custom LLM QA Failures in 7 Steps

Recent Articles

Deleting Paris from a Language Model

Divinci AI Joins Cloudflare Workers Launchpad Cohort #6

Universal Basic Income by 2035: A Feasible Path Forward

Automated LLM CI/CD Pipelines With Instant Rollback

Validating and Releasing Custom LMs in Regulated Fields

The 12 QA + Release Capabilities Every Custom-LLM Platform Ships

10 CI/CD Release Failures in Custom LMs — Which Stage Catches Each

CI Testing for Custom Language Models in 2026

Automated Regression Testing for Custom LLMs in 2026

How to Build an LLM CI/CD Pipeline With Divinci AI

Speculative Decoding for Free: Pairing DFlash with our DFO-Tuned Gemma 4 31B

The Two Models That Never Met. Both Measured at the Same Depth.

Calibrating the Judge: The Grader get Graded

When the Circuit Dissolves

Inside the RAG Arena: When the Judges Don't Agree

The Architecture Every Language Model Converges To

Light Logic: The Dawn of Photonic Ternary Computing for Universal Basic Compute

The Future of RAG Systems: Beyond Simple Document Retrieval

Optimizing Vector Embeddings for Better Search Results

Divinci AI Blog

How to Diagnose Custom LLM QA Failures in 7 Steps

Recent Articles

Deleting Paris from a Language Model

Divinci AI Joins Cloudflare Workers Launchpad Cohort #6

Universal Basic Income by 2035: A Feasible Path Forward

Automated LLM CI/CD Pipelines With Instant Rollback

Validating and Releasing Custom LMs in Regulated Fields

The 12 QA + Release Capabilities Every Custom-LLM Platform Ships

10 CI/CD Release Failures in Custom LMs — Which Stage Catches Each

CI Testing for Custom Language Models in 2026

Automated Regression Testing for Custom LLMs in 2026

How to Build an LLM CI/CD Pipeline With Divinci AI

Speculative Decoding for Free: Pairing DFlash with our DFO-Tuned Gemma 4 31B

The Two Models That Never Met. Both Measured at the Same Depth.

Calibrating the Judge: The Grader get Graded

When the Circuit Dissolves

Inside the RAG Arena: When the Judges Don't Agree

The Architecture Every Language Model Converges To

Light Logic: The Dawn of Photonic Ternary Computing for Universal Basic Compute

The Future of RAG Systems: Beyond Simple Document Retrieval

Optimizing Vector Embeddings for Better Search Results

Share with Friends