Use Case #28: Quality Score Redesign

The content quality score was supposed to help writers improve. Instead, they ignored it.

"What does 73% even mean?"

Fair question.

The Problem

The score was a black box. Multiple factors weighted somehow. No clear path to improvement. Writers either ignored it or gamed specific factors without improving actual quality.

The Research

Claude analyzed 6 months of content data: scores, human editor ratings, performance metrics. Found correlations. Or rather, found the lack of them.

The Findings

The existing score correlated weakly with actual quality. Human editors gave some 60-score articles A+ ratings. Some 95-score articles were mediocre.

The score was measuring the wrong things: keyword density too heavily weighted, readability formulas that punished complex topics, length requirements that encouraged padding.

The Redesign

New scoring system designed around what actually predicted quality:

Structure (30%) - Clear hierarchy, logical flow, proper sections. Measurable through heading analysis.

Completeness (30%) - Covers the topic adequately. Measured against topic models for that content type.

Readability (20%) - Appropriate for target audience. Not one-size-fits-all formula.

Engagement Signals (20%) - Based on actual reader behavior from historical data.

The UI

Old: Single percentage. "Your score: 73%"

New: Four-dimension breakdown. Clear explanations. Specific improvement suggestions. "Add a section on pricing to improve completeness."

The Result

Writers started using the score. Content quality improved. Human editor ratings increased 15% over 2 months.

---

Quality score redesign for ContentEngine, December 2025.