My favorite part of shipping a product is crunch time. It’s the month or so before a major release when you’ve acknowledged that you can no longer delay the inevitable — you must ship this product.

What’s gratifying about crunch time is the velocity with which the product team moves. Decision-making occurs in the hallway. Meetings which used to be an hour shrink to twelve minutes so folks can get “get back to doing real work”. Whiteboards are written on… food is delivered at @ 10pm. Families are ignored.

Invariably one of the biggest decisions that must be is, “whether or not we can ship with this bug”. Hundreds of hours will be spent on bug triage and a ton of time will be wasted by folks defending their favorite bug. To help simplify this, I offer the criteria I use for defining a “bad bug”. A tip of the hat goes to Pac-man who originally mentored this into me.

1. Is it heinous?

First, you must acknowledge that there are bugs when you ship. That is a fact. Can this bug be one of them? Would we spin a quick re-release if this bug were discovered a week after ship? If YES is the answer to all of these questions, you move to…

2. Do we understand the bug?

Why is this bug occurring? What changed in the product to introduce it? If you get past #2, YOU’RE NOT EVEN HALF DONE YET… Deciding to fix a bug and actually producing a fix are two drastically different events. Everything is supposition until the engineer comes back with a fix in hand. This leads us to…

3. Do we understand the fix?

For #3, you’re really after secondary and tertiary effects. Ask yourself, do you really understand all the ramifications of the fix? Really really? Really like my next four raises are dependent on it?

The types of questions you ask for #1 through #3 will vary depending on the type of project you’re developing, but of the main questions, however, you define them, should be asked for each bug late in the game because any change to the product is risky.

I love it when an engineer says, “It’s just a string change!”. Sure, that’s a pretty low-risk change, but getting the product out the door is a hell of a lot more than just making sure it compiles on engineering machine. Did the change get correctly submitted to source control? Are we sure this is the only change submitted? Does the new version build for the build engineer? Wait, is it localized? I could go on and on with examples of places that folks can fuck up the product with a simple string change.

The goal for any product whether it’s an operating system or a consumer application is to get to a steady-state where folks stop messing with the bits. It’s only then that your qualification organization can test against a known quantity and every single change to the product increases the risk that you’re nowhere near ready to ship.

FYI, for comparative purposes, the catholic catechistic criteria for a Mortal Sin is:

1. Grievous offense

2. Sufficient reflection

3. Full consent of the will