Weighted Video Metadata Gating

Problem

The first real-video eval pass showed that Summry's metadata prior could be too literal. The classifier searched title, description, channel, YouTube categories, and tags as one combined text field. That meant a weak signal in tags could overpower the actual intent of the video.

One example was a Vim motion tutorial being classified as an interview or podcast because the metadata included an interview-related tag. The video was still supported, but the wrong category would send it toward the wrong summarization strategy.

Context

Summry's v1 gating is intentionally deterministic. The goal is not to make a perfect classifier yet; it is to make the early product boundary predictable enough that users get honest outcomes:

• supported knowledge-dense videos continue
• caution videos continue with caveats
• unsupported videos stop before expensive processing

Metadata is only the first prior. It should help choose a likely category, but it should not become a keyword trap.

Implementation

The metadata classifier now scores fields separately:

• title carries the strongest weight
• YouTube category carries medium weight
• description carries medium-low weight
• tags and channel name carry low weight

The implementation also adds a minimum category score before assigning a metadata category. Low-weight evidence alone can no longer confidently pick a strategy.

Tests now cover the failure shape from the eval pass:

• noisy tags should not turn a Vim tutorial into an interview
• an "explained" title should beat weaker course or lesson tags
• explicit podcast/interview titles should still classify as interview/podcast

Tradeoffs

This makes the heuristic more conservative. Some videos may fall back to a broader category such as explainer when metadata is weak or ambiguous. That is acceptable for v1 because a broad supported prior is less harmful than a confident but wrong strategy.

The tradeoff is a slightly more complex scoring function. Instead of one text blob and a keyword count, the classifier now carries field-level weighting and evidence labels.

Lessons Learned

Real evals are valuable because they reveal the difference between keyword presence and product meaning. "Interview" in a tag is not the same signal as "interview" in a title. The classifier needs to reflect that distinction.

This step also reinforced the right tuning order: fix category routing before tuning prompts. Prompt quality is hard to judge when the video is being sent through the wrong strategy.