Controlled Duo Quality Protocol

Current status

Development verified — first provider run rejected. The protocol was first validated without calling OpenAI, then run once with explicit budget approval. The live response passed six of seven automated checks but selected the deliberately weaker comparison candidate. The owner rejected the compromise.

The question is narrow: can the production ranking prompt choose one plausible bridge between two contrasting profiles and explain the compromise without violating their shared constraints?

Fixed inputs

Bounded candidate set

The live protocol does not run retrieval, embeddings, TMDB discovery, database queries, poster lookup, or description generation. It passes four fixed movies directly to the same getRecommendation ranking function used by the product. That isolates compromise quality from candidate-retrieval quality and limits a live run to one provider call.

Every candidate already satisfies the shared hard constraints. The comparison movie remains in the set so the test can distinguish a valid candidate from a strong compromise.

Acceptance contract

Automated checks require the response to:

• match the production ranking response schema;
• select a title from the fixed candidate set;
• select one of the three pre-declared bridge candidates;
• name both Alex and Sam;
• represent at least one taste signal from each profile;
• explain the choice in at least 120 characters.

Passing those checks is not the same as passing a quality review. The owner must still answer:

1. Does the film feel like a fair bridge rather than a win for one person?
2. Does the explanation connect the film to both profiles rather than repeat prompt keywords?
3. Is the recommendation credible for the stated night?

Recorded live result

The first controlled live run was recorded on 2026-07-16 and made exactly one provider call. The complete sanitized report is stored in results/controlled-duo-live-2026-07-16.json.

The result stayed inside the fixed set, respected the shared constraints, named Alex and Sam, represented terms from both profiles, and provided a detailed explanation. It still failed the central compromise test. The explanation made the weaker overlap sound resolved instead of demonstrating that the selected film was a strong bridge.

This is useful negative evidence, not a recommendation-quality success. The next live run should wait until bridge fit is represented explicitly in the ranking policy.

Reproduce without provider spend

npm run eval:recommendations:duo

This validates the people records, candidate uniqueness, hard constraints, and bridge coverage. It writes apps/web/test-results/recommendation-evals/controlled-duo-report.json with providerCallCount: 0, status: protocol-ready, and reviewStatus: not-run.

Optional live run

Only run this after explicit API-budget approval and with OPENAI_API_KEY configured:

npm run eval:recommendations:duo -- --live

Live mode makes one ranking call. If automated checks pass, the report uses status: awaiting-owner-review and reviewStatus: pending-owner-review. It does not emit a final recommendation-quality pass.

Evidence boundary

Proven now: the protocol is executable, bounded, credit-free by default, uses valid production-shaped inputs, and can separate machine-checkable constraints from subjective review. One controlled provider response also showed that fluent reasoning can overstate a weak taste overlap.

Not yet proven: that the configured provider selects convincing bridges consistently, that the ranking policy prevents weak compromises, or that production users receive better recommendations.