Methodology

1. Deterministic selection & Eligibility Gate

A product candidate is discovered daily from structured web platforms rotating by day of the week:

• Monday: Hacker News Top
• Tuesday: GitHub Breakout (Trending)
• Wednesday: Show HN
• Thursday: Hugging Face Spaces (Trending)
• Friday: GitHub Developer Tools
• Saturday: GitHub Open Source Software
• Sunday: Cross-source compilation

Rather than simply choosing the most popular item, the system automatically filters candidates in order of popularity against the strict **Open Source Eligibility Gate (Selection Policy v2)**. Only the first candidate that satisfies all eligibility criteria is selected for evaluation.

• Public Source: A canonical, public source repository must exist (no README-only empty repos).
• Open-source License: Must have an explicit license normalizable to an approved SPDX identifier (e.g. MIT, Apache-2.0).
• Clear Purpose: Stated purpose and target users/use cases.
• Runnable or Reproducible: Presence of public demo, release, package registry, container image, or clear installation instructions.
• Freshness: Meaningful commits or releases in the past 18 months.

Popularity metrics (such as stars or points) are used solely for deterministic candidate discovery and selection; they do not affect the Jury Score.

2. Evidence collection

Evidence is gathered primarily from the official product source (repository, landing page, API metadata) and the source discussion thread. Third-party reviews are strictly excluded.

3. Evidence classification

The AI distinguishes between the following types of information:

• Confirmed in supplied source (source_confirmed): directly observable in the repository, docs, or API metadata
• Creator claim (creator_claim): stated by the product creator but not independently verified
• Jury inference (inference): a reasonable conclusion drawn by the AI
• Unknown (unknown): information not available in the supplied evidence
• Directly observed during review (runtime_observed): directly executed or observed during this review (when applicable)

Each criterion receives a confidence rating (high, medium, low, or not assessable).

4. Five-person evaluation

To balance professional biases, the evaluation coordinates five simulated professional perspectives (personas) evaluating the product simultaneously. JuryPress utilizes structured output engines to generate all five reviews simultaneously in a single, atomic API call. This ensures that all simulated perspectives evaluate the exact same evidence container under identical temperature and token budget parameters.

JuryPress uses the five personas and current rubric defined by Judgie-AI.

JuryPress runs its own automated publishing pipeline; it does not claim that Judgie-AI independently reviewed or endorsed each article.

5. Code-side score calculation & Not Assessable

The AI provides raw scores (0–5) for each criterion. Weighted scores and the Jury Score are recalculated server-side using fixed rubric weights. The AI cannot influence the final score beyond the raw criterion scores.

If a criterion lacks sufficient evidence for a complete assessment, it is rated "not assessable" and receives a null score. If any criterion is unassessable, the product's overall Jury Score is set to null, and the review is marked as **Unranked** due to insufficient evidence.

6. Automatic publication

The article is built as a static page and deployed without editorial review.

• Model: Current production model: gemini-3.5-flash
• Season Version: Season 2 — Open-Source Products
• Human Review: None.

7. Failure handling

If the AI generation fails validation (schema mismatch, prohibited phrases, missing evidence IDs, or homogenized verdicts), the run retries up to 3 times. If all retries fail, the run is logged and no article is published.