What went wrong, and what did we change?

The faults we found this week, and the rules we changed.

A lot, and that is the useful part of the week.

  • Agents waiting on each other. An audit of Solve's fault log found 59 entries: work with no owner, replies that never reached the asker, wake-ups that failed, and health checks that measured activity instead of progress. We fixed the wake path and the readers, and changed the rules: agents now resume work that is already authorized instead of waiting for a fresh go.
  • Checks that passed when they shouldn't have. An audit of 29 jobs found two checks that reported a pass while hiding a failure. The runner now records the raw exit status of one check command.
  • Answers bound to the wrong question. An approval could attach to the wrong card, or survive a retry it should not have. Answers are now bound to exactly what was offered, and answered cards are frozen.
  • Too much in a job's checkout. Broad staging let excluded files reach a job's working copy. Jobs are now staged file by file, from the permitted copy only.
  • Saying a thing landed before checking. Twice, a result was reported before anyone read what the code actually did. The rule now: read the implemented requirement before saying it landed.
  • Signer's next release. Review of the next Signer changes found more defects. We keep a clear line between the release that is running and the next one under review.

One chapter at a time. Use the links to keep reading, or swipe on touch screens.