Vibe coding with Claude Code course Β· Module 15: Advanced Claude Code

An exam without supervision: /goal, the Stop hook and /verify

11 min read
In this lesson8

In the morning night-report.json has in the result field: "Done. All routes use lib/apiErrors.ts, the tests pass." Next to it waits the branch worktree-api-errors with a dozen or so changed files, and you did not see a single turn of that work. From lesson 10_4 you know what a report without proof is worth. In a session you are sitting at you catch a mistake within a minute. After a night you have only what the run wrote down itself. That is why you build the exam into the run before it sails, and in the morning you compare the report with the map of changes.

What you already know

In lesson 10_4 you learned the levels of rigour: a criterion in the request, /goal with a separate model that reads only the conversation, a Stop hook with code 2 and /verify, and in lesson 2_9 /code-review in a fresh subagent. From lesson 14_1 you know the fields of the claude -p report, from lesson 14_2 the five questions before you leave an agent, from lesson 14_6 cloud sessions and Remote Control, and from lesson 14_7 the night watch: a commit, a worktree, the sandbox and --max-turns. I do not repeat here what these tools are. I show how to set them up for a long voyage that you do not watch.

A goal in a run without you

/goal works in -p mode too: typed at the start of the request, it drives the loop to the end in a single call. It also works in the desktop app and through Remote Control. Setting a goal starts the work at once, and the condition is at the same time the instruction, so it must say what to do and how to tell it is finished. The documentation warns that with plain text output a long goal looks hung, which is why the night start from lesson 14_7 switches the format to a stream. You will launch it only after the hook from the next section.

Request for Claude Code (the start of the watch):

claude -p "/goal All routes in app/api return errors through lib/apiErrors.ts and have tests: grep -rn 'error:' app/api --include=route.ts prints nothing, and in the output of find app/api -name 'route*.ts' every route.ts has a route.test.ts next to it; npm run test -- --run ends with no failures, and npx tsc --noEmit with no errors; git diff --name-only --diff-filter=MD main -- '*.test.ts' prints nothing; or stop after 25 turns" \
  --worktree api-errors --permission-mode auto --settings scripts/night-watch.json \
  --max-turns 60 --output-format stream-json --verbose > night-stream.jsonl

The evaluator runs nothing, so every clause of the condition names a command whose output Claude must show in the conversation. An empty grep output proves that no hand-written error is left in the routes, the find list shows a pair of files for every route, the tests and tsc give pass or fail, and the last git diff guards the ban on changing tests. The condition may be up to 4000 characters, so you can fit in everything you want to see in the morning.

That last clause looks the way it does because of our trial. We set a goal with the condition git diff --stat -- '*.test.ts' prints nothing. Claude reported the condition met and added by itself that lib/plural.test.ts is untracked. git diff does not see untracked files, so a changed test outside a commit would have gone unnoticed. --diff-filter=MD main compares with the main branch and prints only files that were modified (M) and deleted (D), including those Claude has already committed. New tests pass and the old ones are protected, provided they were in a commit before the start.

You have a double limit. The clause "or stop after 25 turns" is judged by the evaluator from the conversation, so it is soft. --max-turns 60 is a hard brake of the program, counted in agent steps: for us one goal cycle with four tool calls gave num_turns equal to 5, so you set the limit with room to spare. The loop ends by itself in three cases: when the evaluator judges the condition met, when it judges it impossible (then it writes the reason), and when Claude answers for several turns in a row without using a tool. An error that only you can fix, for example a failed login or exhausted credit on the API account, clears the goal and leaves a warning that starts with Goal cleared after an unrecoverable error.

A Stop hook for a long watch

From lesson 10_4 you know that after eight blocks in a row Claude Code ends the turn anyway. A detail that matters at night: the counter resets every time Claude calls a tool. So the block limit will not interrupt a loop in which the agent keeps trying something: it protects only against pointless pushing. The brake of a long voyage is --max-turns. After the limit is exceeded Claude Code ends the turn with a warning, and a hook can tell that it has already sent Claude back to work once by the stop_hook_active field in its input.

The ban on changing tests is a rule without exceptions, so it gets a hook, but only for the night: in the daytime you change tests on purpose and you do not want a hook to send you back.

Request for Claude Code:

Write .claude/hooks/tests-untouched.sh: a Stop hook that ends with code 2
and a message on stderr when git diff against main shows a changed or deleted
*.test.ts or *.test.tsx file. Let new test files through. Add it to
scripts/night-watch.json next to the sandbox, not to .claude/settings.json.
Done when the script given JSON with a cwd field returns 2 for a changed test and 0 for a new one.

The request gives the event, the exit code, the place of the entry and a check, and the ban on writing to .claude/settings.json protects your daytime sessions.

What Claude wrote: the hook

Open .claude/hooks/tests-untouched.sh in Cursor:

#!/bin/bash
# tests-untouched.sh: a Stop hook that sends Claude back when it changed or deleted a test that exists on main

cd "$(jq -r '.cwd')"
CHANGED=$(git diff --name-only --diff-filter=MD main -- ':(top)*.test.ts' ':(top)*.test.tsx')

if [ -n "$CHANGED" ]; then
  echo "Tests from main changed or deleted: $CHANGED. Restore them and fix the code, not the test." >&2
  exit 2
fi

exit 0

The first line moves to the session directory from the cwd field, which the hook gets as input, because in a worktree Claude's deck is the worktree directory. Next comes the same command as in the goal condition, only now a program checks it, not a model. :(top) makes paths count from the root of the repository. Without it, a hook run from a subdirectory did not see a changed test in lib/ for us and let everything through. The message goes to stderr, so Claude gets it as the reason it cannot finish.

In scripts/night-watch.json, next to the sandbox key from lesson 14_7, after a comma, came an entry in the shape you know from lesson 14_5:

  "hooks": {
    "Stop": [
      {
        "hooks": [
          { "type": "command", "command": "bash", "args": ["${CLAUDE_PROJECT_DIR}/.claude/hooks/tests-untouched.sh"] }
        ]
      }
    ]
  }

The script, like the rules from lesson 14_7, must be in a commit before you sail. Test it as in lesson 14_5 by giving the JSON by hand: add one line to lib/pins.test.ts and run jq -n --arg cwd "$PWD" '{cwd: $cwd}' | bash .claude/hooks/tests-untouched.sh; echo $?. For us a changed test gave the message and 2, a new test file 0, and a change committed on the branch 2 again. Then restore the test.

The morning briefing: report versus diff

Start with the result of the run. The stream has many lines, and the summary is in the event of type result:

jq -c 'select(.type == "result") | {subtype, is_error, num_turns, denials: [.permission_denials[].tool_input]}' night-stream.jsonl

subtype and is_error tell you whether the run ended normally or at the limit, and denials show what the agent was not allowed to do. That output proves nothing yet, because it is still the words of the run itself. The proof is the output of the commands from the condition: open night-stream.jsonl in Cursor and search for Test Files. The last occurrence should show green tests.

Then the map of changes. In Cursor's terminal git diff --stat main...worktree-api-errors lists every file changed on the branch, and in Source Control you open them one by one. You read in both directions: every sentence of the report has its file in the diff, and every file in the diff has its reason in the report. A file outside app/api and lib/, for example package.json or something in .claude/, is a question before you merge anything.

Finally, fresh eyes. Open a session in the worktree directory (cd .claude/worktrees/api-errors, then claude) and type /code-review: with no target it reviews the branch commits together with uncommitted changes, in a subagent that has not seen the night's conversation. You order fixes in the same session. Only after you merge the branch into main in the main directory, where you have .env and the database from npx prisma dev, run /verify: Claude will build the app, run it and look at the result. You remove the worktree afterwards with git worktree remove.

The five questions as a briefing list

You ask the questions from lesson 14_2 twice: in the evening before the start and in the morning at the briefing.

QuestionIn this runIn the morning you check
Does it know what the end looks like?the /goal conditionwhether the stream has the output of every command from the condition
Can it check itself?tests, tsc, the Stop hookthe last Test Files in the stream
What may it touch?the worktree, the sandbox, deny rulesdenials and files outside the task in the diff
What will stop it?--max-turns 60 and the 25-turn clausesubtype and num_turns
Who approves the result?you, after /code-reviewyou merge it yourself, never the run

Captain's review

  1. The condition is visible in the conversation. Every clause of the goal names a command. A clause like "the code is readable" the evaluator will judge by eye.
  2. Tests untouched. git diff --name-only --diff-filter=MD main...worktree-api-errors -- '*.test.ts' prints nothing.
  3. Proof, not a sentence. In result or in the stream there is a command and its output, not just "all tests pass".
  4. Denials read. Every entry in denials is either a limit you set or the reason part of the work was not done.

A typical agent mistake: when a test will not pass, Claude weakens it or deletes it, and in the report writes about "tidying up the tests". You will recognise it by the letter M or D next to a .test.ts file in the branch diff. The hook will catch a change and a deletion, but a new, weak test, for example with only toBeTruthy, it will let through. Only you and /code-review will see that one.

Next step

The briefing has the same steps every morning, so close it in a script, like the review in lesson 14_1:

Save the morning briefing as scripts/briefing.sh: the result of the run from night-stream.jsonl
through jq, git diff --stat main...worktree-api-errors and a list of changed or
deleted tests. The script only reads: it merges nothing and deletes nothing.

Then a commit, for example "Night watch: a hook guarding the tests and a briefing", and a trial voyage with --max-turns 5 before you launch the full one. In the editor next to the lesson you will type a goal condition, and the checker will point to the clauses the evaluator will not see in the conversation.

Remember: a good captain does not believe reports, only entries in the ship's log, and the log of the night watch is the output of a command and the diff of the branch.

Code for this lesson: App.tsx
// /goal condition checker: which clauses the evaluator will see in the conversation and which it will not
import { useState } from 'react';

export type Level = 'ok' | 'warn' | 'error';

export interface Finding {
  level: Level;
  text: string;
}

export const MAX_LENGTH = 4000;

export const COMMANDS = ['npm ', 'npx ', 'git ', 'grep ', 'find ', 'ls ', 'jq ', 'wc ', 'node '];
export const CHECKS = ['npm test', 'npm run test', 'tsc', 'vitest', 'npm run build', 'npm run lint'];
export const VAGUE = ['readab', 'clean', 'consisten', 'sensible', 'tidy', 'well', 'correct', 'broke', 'works'];

export const SAMPLES: Record<string, string> = {
  'A report on my word of honour': '/goal the route code is readable and consistent; all tests pass; nothing broke',
  'The condition from lesson 10_4': '/goal npm run test -- --run ends with no failures, npm run lint and npx tsc --noEmit pass, and git diff --stat shows no changes in .test.ts or .test.tsx files; or stop after 15 turns',
  'Night watch': "/goal All routes in app/api return errors through lib/apiErrors.ts and have tests: grep -rn 'error:' app/api --include=route.ts prints nothing, and in the output of find app/api -name 'route*.ts' every route.ts has a route.test.ts next to it; npm run test -- --run ends with no failures, and npx tsc --noEmit with no errors; git diff --name-only --diff-filter=MD main -- '*.test.ts' prints nothing; or stop after 25 turns",
};

export function stripGoal(text: string): string {
  const t = text.trim();
  return t.startsWith('/goal') ? t.slice('/goal'.length).trim() : t;
}

export function isTurnClause(clause: string): boolean {
  return /after \d+ turn/.test(clause);
}

export function hasCommand(clause: string): boolean {
  const c = clause + ' ';
  return COMMANDS.some((k) => c.includes(k));
}

export function clauses(condition: string): string[] {
  return condition.split(';').map((c) => c.trim()).filter((c) => c.length > 0);
}

export function check(text: string): Finding[] {
  const condition = stripGoal(text);
  if (condition === '') return [{ level: 'error', text: 'Empty condition: /goal without a condition only shows the state of the goal.' }];
  const out: Finding[] = [];
  if (condition.length > MAX_LENGTH) {
    out.push({ level: 'error', text: 'The condition has ' + condition.length + ' characters, and /goal accepts at most ' + MAX_LENGTH + '.' });
  }
  for (const c of clauses(condition)) {
    if (isTurnClause(c)) continue;
    const short = c.length > 60 ? c.slice(0, 60) + '...' : c;
    if (hasCommand(c)) {
      out.push({ level: 'ok', text: '"' + short + '": names a command whose output Claude will show in the conversation.' });
    } else if (VAGUE.some((v) => c.toLowerCase().includes(v))) {
      out.push({ level: 'error', text: '"' + short + '": a judgement by eye. The evaluator runs no commands and reads no files, so there is nothing for it to check.' });
    } else {
      out.push({ level: 'warn', text: '"' + short + '": no command. Add what Claude is to prove it with.' });
    }
  }
  const lower = condition.toLowerCase();
  if (!CHECKS.some((k) => lower.includes(k))) {
    out.push({ level: 'warn', text: 'No check that passes or fails: tests, tsc, lint or a build.' });
  }
  if (!lower.includes('.test.')) {
    out.push({ level: 'warn', text: 'Nothing guards the tests. Add git diff --name-only --diff-filter=MD main -- *.test.ts or a Stop hook.' });
  } else if (lower.includes('git diff') && !lower.includes('--diff-filter=md main')) {
    out.push({ level: 'warn', text: 'This git diff does not see untracked files or changes Claude has already committed. Compare with main: git diff --name-only --diff-filter=MD main.' });
  }
  if (!clauses(condition).some(isTurnClause)) {
    out.push({ level: 'warn', text: 'No clause like "or stop after 25 turns". The hard limit still comes only from --max-turns.' });
  }
  return out.filter((f, i) => out.findIndex((g) => g.text === f.text) === i);
}

const LEVEL_CLASS: Record<Level, string> = {
  ok: 'text-emerald-400',
  warn: 'text-amber-400',
  error: 'text-rose-400',
};

export default function App() {
  const [text, setText] = useState(SAMPLES['A report on my word of honour']);
  const findings = check(text);
  const problems = findings.filter((f) => f.level !== 'ok').length;

  return (
    <div className="min-h-screen space-y-4 bg-slate-950 p-4 text-slate-100">
      <h1 className="text-xl font-bold">Will the evaluator see this?</h1>
      <div className="flex flex-wrap gap-2 text-sm">
        {Object.keys(SAMPLES).map((name) => (
          <button key={name} type="button" className="rounded bg-slate-800 px-2 py-1 hover:bg-slate-700" onClick={() => setText(SAMPLES[name])}>
            {name}
          </button>
        ))}
      </div>
      <label className="block text-sm">
        Goal condition
        <textarea className="mt-1 h-32 w-full rounded bg-slate-900 p-2 font-mono text-xs" value={text} onChange={(e) => setText(e.target.value)} />
      </label>
      <p className="text-xs text-slate-400">{stripGoal(text).length} / {MAX_LENGTH} characters</p>
      <p role="status" className="text-sm font-semibold">
        {problems === 0 ? 'Every clause can be confirmed by the output of a command.' : 'To fix: ' + problems}
      </p>
      <ul className="space-y-1 text-sm">
        {findings.map((f) => (
          <li key={f.text} className={LEVEL_CLASS[f.level]}>{f.level}: {f.text}</li>
        ))}
      </ul>
    </div>
  );
}

Spotted a mistake in this lesson?

Check yourself

Answer the questions from this lesson. Pick an answer to see right away whether it is correct.

  1. 1. Which /goal condition will the evaluator confirm in a night run without guessing?

  2. 2. Why can the condition "git diff --stat -- '*.test.ts' prints nothing" let a changed test through?

These are 2 of 4 questions for this lesson. Solve the rest in the game.

Hands-on tasks in the game

  • Vertical ordering

    Arrange the steps of the morning briefing after a night run in a worktree:

  • Horizontal ordering

    Arrange a command that gives the hook JSON with the session directory and prints its exit code:

  1. QuestionQuiz: A condition the evaluator can see
  2. QuestionQuiz: What git diff does not see
  3. Put in orderSorting: The morning briefing after a night run
  4. Put in orderSorting: A manual trial of the Stop hook
  5. QuestionQuiz: What stops a Stop hook
  6. QuestionQuiz: Five questions at the morning briefing

Useful articles