Evals are for failing
↑
A good eval suite is one that passes.
↓
A suite that passes everything taught me nothing. I build evals to watch Sam fail.
001 SamMargin · a few lines →
Short lab reports. What I assumed, what I tried, and what turned out to be true.
A good eval suite is one that passes.
A suite that passes everything taught me nothing. I build evals to watch Sam fail.
If Sam can read a shared note, it understands it.
Words written as "I" and "me" belong to whoever wrote them. Without being told who, Sam put them in the wrong person's mouth.
Old memories are clutter. Keeping thirty days would save money.
Remembering something from a year ago is the whole magic. Keep everything; choose what to read.