B
Offline
Step 1 of 10

Score the Session

8of 10~10 min
  • The 7-dimension evaluation rubric
  • How to identify session quality issues
  • Using scores to guide iteration
What you'll do
  • Apply the scoring rubric
  • Identify weak dimensions
  • Decide whether to iterate

Every session should be scored to understand its quality and identify areas for improvement. The 7-dimension rubric covers the key aspects of discriminative research.

The 7-Dimension Scorecard

Score each dimension from 1 (poor) to 5 (excellent):

1

Hypothesis Quality

"Are hypotheses mechanistic and testable?"

1-2:Vague descriptions, no mechanisms
4-5:Clear mechanisms, specific predictions
2

Third Alternative

"Is there a genuine third alternative?"

1-2:Missing or token alternative
4-5:Genuinely challenges the framing
3

Test Discriminativeness

"Do tests separate hypotheses?"

1-2:Tests only confirm favorites
4-5:Tests can cleanly eliminate hypotheses
4

Potency Checks

"Are potency checks present?"

1-2:No controls for confounds
4-5:Every test has potency check
5

Assumption Clarity

"Are load-bearing assumptions explicit?"

1-2:Hidden assumptions everywhere
4-5:All assumptions listed with tests
6

Adversarial Rigor

"Was the framing challenged?"

1-2:Surface-level critiques only
4-5:Fundamental framing challenged
7

Actionability

"Can you execute on this artifact?"

1-2:Unclear next steps
4-5:Clear experiment plan

1Run the Scoring Tool

The brenner CLI has an interactive scoring mode:

Terminal
# Run interactive scoring
brenner session score --thread-id "$SESSION_ID"
# Or score with artifact file
brenner score sessions/$SESSION_ID/artifact.json
# View score report
cat sessions/$SESSION_ID/score.json

Interpreting Scores

Average 4.0+: Excellent

Ready to publish or execute. Consider archiving to your research program.

Average 3.0-3.9: Good

Solid foundation. One more iteration could address weak spots.

Average 2.0-2.9: Needs Work

Significant gaps. Focus on lowest-scoring dimensions in next round.

Average <2.0: Restart

Fundamental issues. Consider reframing the research question or kickoff.

Pro Tip
Low scores aren't failures — they're diagnostic. A session that scores 2.5 on "Third Alternative" tells you exactly what to focus on next: ask the agents to challenge the framing more aggressively.

Scored? Based on your scores, decide whether to iterate (another round) or publish the artifact in the next step.