Management Insist on understanding

The Process Myth

On the list of ways to generate a guaranteed negative knee-jerk reaction from an engineer, I offer a single word: process.

Folks, in order to make sure that we hit our ship date, we have a new bug triage… process.

You’ve heard the groans and you’ve seen the rolling eyeballs and made the fair assumption that engineers are genetically predisposed to hate process. It’s an incorrect assumption that doesn’t add up. Engineers are creatures who appreciate structure, order and predictability, and the goal of a healthy process is to define structure so order is  maintained and predictability is increased. The job of a software engineer is writing code, which is codified process.

So, what gives? Why the groaning?

Engineers don’t hate process. They hate process that can’t defend itself.

Don’t Answer the Question

At Apple, there is a creature called an Engineering Program Manager (“EPM”). Their job is process enforcement. They are the folks who sat in meetings like bug reviews and made sure that every part of the process was being followed. As a person who prefers to spend mental cycles on the people and product rather than the process, I appreciated the role of the EPM.

A good EPM’s job is to keep the trains on time by all reasonable means. However, my experience with program managers over the past two decades is that 70% of them are crap because while they are capable of keeping the trains running on time, they don’t know why they’re doing what they’re doing. When someone on the team asked them to explain the reasoning behind the process, they’d say something to the effect of, “Well, this is how we’ve always done it…”

If you want to piss me off, if you want me to hugely discount your value, do this: when I ask you a clarifying question that affects how I will spend my time, my most valuable asset, don’t answer the question. This non-answer is the root cause of an engineer’s hatred of process. A tool that should help bring order to the universe is a blunt instrument that incites rage in the hands of the ignorant.

Healthy Process is Awesome

It pains me to type that heading because of the 70% out there who are giving process a bad reputation with their blind enforcement. But if we explore where process might come from, you’ll understand three things: the circumstances that lead to the necessity of process, how it could be awesome, and most importantly, your role in maintaining the awesome.

With a small team, mostly you don’t need process because everyone knows everything and everyone. You don’t have to document how things occur because folks know how to get it done, and if they don’t they know exactly the right person to ask. If something looks broken, you don’t hesitate to stand up and say, “That’s broken. Let’s fix it.” You do this because, as a small team, you feel equally responsible for the company because everyone is doing everything.

Hidden among all this work are essential parts of your company that everyone knows, but no one sees: your values and your culture. If you’re a small team, you likely don’t have a mission statement, you have the daily impossible amount of work you must do to survive and the way you do that work is an embodiment of your culture and your values.

Now, if you stopped someone in the hallway of this hypothetical company and asked them to explain the values, they’d look at you like a crazy person and give you exactly the same damning answer as the program manager above: “Well, this is how we’ve always done it.” Double standard? No. The difference here is that if you could actually get the attention of the hallway person, if you pressed them, they’d be able to explain themselves. When you asked them, “Why must we debate every decision?” they’d say, “We encourage debate because we want to make the most informed possible decision.”

Then, at some magical Dunbar number, you pass two interrelated inflection points. First, the number of new hires arriving exceeds your population’s ability to organically infect culture and values. Second, because of the vast swath of preexisting people, the arriving individual erroneously believes that they as a single person can no longer influence the cultural course of the company. The team is fractured into two different groups that want exactly the same thing:

#1 The Old Guard. These are the folks who have been there for what seems like forever. They understand the culture and the values because they’ve been living and breathing them. They have a well-defined internal map of the different parts of the company that consist of the rest of the Old Guard. Whether they like it or not, they are the exemplars of what the company values.

#2 The New Guard. These folks have arrived in the last year and while they understand that there is culture and there are values out there, they spend a lot of time confused about these topics because no one has taken the time to sit them down and explain them and the folks who are qualified to do so are busy keeping the ship pointed in the right direction. This situation is exacerbated by the fact they don’t have an internal map of the company in their head and they don’t know who to ask what, so once their honeymoon period is over, they get angry because they don’t know why they’re doing what they’re doing.

Problem is, the Old Guard can’t conceive of a universe where everyone doesn’t know everything, and they have difficulty explaining what they find obvious. The Old Guard begins to hear the New Guard’s crankiness, but their suggestion is, “Duh, fix it. It’s your company. That’s what I did.” This useless platitude only enrages the New Guard because while they desperately want to fix it – they don’t know how – and having the Old Guard with their informed confidence and flippancy imply it’s simple is maddening.

Eventually, meetings are convened, whiteboards are filled with suggestions, and while different companies give the end result different names, it’s the same outcome: someone volunteers to document the means by which we get stuff done. They document the process.

When you think of process, I want you to think of this moment because it could be a noble moment. Process is being created not as means of control; it’s being built as documentation of culture and values. It’s likely you can’t imagine this moment because you’ve been clubbed into submission understanding process as the dry documentation of how rather than the essential explanation of why.

The Dry Documentation of How

Here’s some really boring process for you. It’s an internal transfer process. Leads refer to it when someone wants to move from one group to the next. Chances are, you may even be aware this thing exists. Lucky bastard. Here’s the breakdown:

  • Employees must have been in their current job for one year before applying for a new job.
  • Employees must have a performance rating of solid or higher in order to apply for a transfer.
  • An employee may have one conversation with a new job’s hiring manager before discussing the internal transfer with their current hiring manager.
  • And it goes on, but you get the idea.

Who wrote this? HR prescriptive bullshit, right?  Yeah, it probably was someone in HR that wrote this years before you arrived, but they were trying to help. When it was 42 of us, how did this internal transfer happen? Well, Frank wanted to try out design, so he talked with the design lead, Luke, who then talked with Frank’s lead, Alex, over a beer and it was done in a week.

This informal conversational process doesn’t work at 420 people for a lot of reasons: Frank doesn’t know if there are opportunities in design because he doesn’t know Luke. If he does figure out that there is a gig and has a chat, Larry doesn’t even think to talk with Alex because they don’t know each other. This leads to all sorts of misunderstandings and crankiness about who knows what, which leads to trust issues, crap communication, and politics that could have been all avoided if we simply agreed to document how our company felt about internal transfers.

I want you to look at this boring process from the perspective of someone who cares about preserving culture. What values are they attempting to capture? Look again.

  • Employees must be in their current job for one year before applying for a new job. We meet our commitments to our teams.
  • Employees must have a performance rating of solid or higher in order to apply for a transfer. If someone is failing at their job, we work to improve them rather than shuffling the problem elsewhere in the company. We fix problems, we don’t ignore them.
  • An employee may have one conversation with a new job’s hiring manager before discussing the internal transfer with their current hiring manager. We understand that situations change. We want people to grow, but we are adamantly transparent in our communications because we know that poor communications results in painful misunderstandings.

The unfortunate fact is that when an internal transfer policy does need to be defined, it often falls to an HR person who is good at defining process, but is shitty at explaining the culture. This means that as they diligently and capably do their job, they’re also merrily eroding your communicated culture and values. Process should be written by those who are not only intimately experiencing the pain of a lack of process, but who are also experts in the culture.

Imagine all process as a means of capturing and documenting culture and values. Unfortunately, in a larger company, it doesn’t work that way. Even if qualified cultural bellwethers took the time to document their pain and to write a process, these folks eventually leave. When they leave so does their cultural context, and the root pain that defined the process leaves with them. The company forgets the stories of how we ended up with all these bulleted lists, and when someone asks why, no one knows the story.

Defend Itself

An engineer instinctively asks why. When someone or something doesn’t make sense to them, they raise their hand and say, “This feels inefficient. Explain this to me.” Now, they don’t usually ask that way. They usually ask in a snarky or rude fashion that gives the process enforcers rage, but snarkiness aside, the engineer is attempting to discover the truth behind the bulleted list.

Anyone who interacts with process has a choice. You can either blindly follow the bulleted lists or you can ask why. They’re going to ignore you the first time you ask, the second time, too. The seventh time you will be labeled a troublemaker and you will run the risk of being uninvited to meetings, but I say keep asking why. Ask in a way that illuminates and doesn’t accuse. Listen hard when they attempt to explain and bumble it a bit because maybe they only know a bit of the origin story.

It’s a myth, but healthy process is awesome if it not only documents what we care about, but is willing to defend itself. It is required to stand up to scrutiny and when a process fails to do so, it must change.

Insist on understanding because a healthy process that can’t defend itself is a sign that you’ve forgotten what you believe.

Leave a Reply

Your email address will not be published. Required fields are marked *

19 Responses

  1. alexr 11 years ago

    In the days at Apple before Avie shut down the library, you could go in there and watch a video called “Keepers of the Culture.” In this video, early HR staff sat in a circle of bean bags and discussed their responsibility to the employees and the company to not forget their roots. (I always assumed the bong was always just out of the shot.)

    The failing here with HR is that they have too-frequent staff turnovers and thus Apple was homogenized with all over local engineering firms. Ask somebody to explain why the company bell curves everybody in a team. Why is that not named the same thing, “stack ranking,” that Microsoft calls it? You’ll find nobody who even remembers the “Burnt Sienna” HR initiative of the late ’90s, let alone an older process.

    It didn’t help that the NeXT executive team undervalued existing Apple staff and actively sought to displace them, but even for old guard that did remember all this stuff, HR isn’t accustomed to taking input from non-HR rank and file, so this stuff was destined to get lost anyway.

  2. At yammer, we’ve found that culture and values are VERY important, and that “process” typically gets in the way. Instead of a hard process, we push for efficient, transparent, and immediate communication. The ability to speak to those who started the culture and being a part of building the culture and values enables our team to keep itself informed about WHY we do things, leaving the how up to the individual. Communication trumps process every time.

  3. Menachem Began 11 years ago

    Some process documents run on for 15 pages, calling up other processes along the way. The end result is often mind bogglingly time consuming. One of my coworkers prepared a bottom up schedule and included all processes – two years. Two years to do ANY project.

    Two years.


  4. Benno 11 years ago

    There is of course a third option; work out the ‘why’ for yourself.

    This isn’t necessarily a good option to follow, but the EPMs of the world should realise that in the absence of them providing the ‘why’, some people will be making up their own!

  5. Jason M Baker 11 years ago

    It’s worth pointing out that research has shown that groups that value dissent generally have better outcomes than those that don’t. I personally believe that groups are defined by how well they encourage questioning the status quo.

    Thus, I have to ask: if you get excluded for questioning “the way it is” too much, do you really want to be included in this group anyway?

  6. @Jeffrey Erickson: I’m not sure you’re arguing against anything in the post, though — he’s not arguing for hard process, just for finding ways to pass on culture even when your core group is no longer easily accessible to everyone joining the company.

    Yammer has only even existed for 4-5 years, right? I suspect this will become more of an issue for you in the next few years.

    “Efficient, transparent, and immediate communication” is at the core of your culture and “process” for regular interactions. But what will happen when none of the people interacting with each other are part of the original team, or have easy access to them?

    AFAIK the only way to avoid the problem is to grow at glacial pace, or to simply stay small.

  7. Keith 11 years ago

    Great article!

    I would make 2 comments from my first reading:

    1/ *Software* Engineers (geeks/nerds/etc) hate process for a variety of reasons beyond what you have described. All too often there is an innate laziness(*) in some (“wannabe”) SWEngs causing them to hate process because they feel it leads to more work. For example, a development process may call for all code to be accompanied by a design (this may be in the form of javadoc or doxygen or a separate document even), but Joe just wants to cut the code and move on because writing a design is more work.

    (* I actually believe that “laziness” can be a strong asset for a good software engineer – it encourages innovations that minimise repetitive work, knowledge sharing that enables load sharing across a team, and helps maintain/build a focus on continuously improving efficiency. Unfortunately this take on laziness is not as common.)

    2/ There are more than 2 choices with process, I think there are at least 3: follow, question/improve, and ignore. In the case of Joe above, ignore is the default choice because questioning a process often leads to more work, so it is easier simply to ignore the process.

    Unfortunately the net result of 1 and 2 above is negative. It demonstrates Joe’s lack of understanding of the cultural context and/or poor fit with the culture of the team, resulting in either change or leave outcomes for Joe.

    I really enjoyed reading this article. Good food for thought.

  8. Great article (as usual) Rands.

    I agree with the ‘why’ question that engineers have. If only all process documents have this explicitly mentioned, I think engineers would have lesser of the snarkiness that entails with processes.

  9. Grant 11 years ago

    I find it amusing that programmers are calling themselves engineers.

  10. This is a really good article.

    The process is not so much to document the culture of the company, more to normalise everything, so that with a rapid turnover in staff, there is no threat to the quality of the product.

    I understand the ‘assurance’ reasons for having a process, but do not kid yourself that it is for the benefit of the workers. Management put a process in place to drive down wages, arguing that if the process is robust enough, then they do not need to employ highly-skilled or educated workers…

    As with everything, life is a balance and a compromise between what is right, and what is profitable.

  11. A nice defence against adding meaningless process is to insist that the process change process be documented first. Seems reasonable after all, eh? Then ask what process was followed to document the process change process. If the answer is “none” (and what else could it be?), then reserve the right to use the same process where necessary in writing software.

  12. “Management put a process in place to drive down wages, arguing that if the process is robust enough, then they do not need to employ highly-skilled or educated workers…”

    Can’t say that I agree with that take, at least not in the company that I’m at. We’re relatively light on process, but the ones that are there were put in place to ensure that smart people don’t waste their time chasing “Who’s got the login for…?” and “Can someone explain how to…?” style questions.

    Process at its best is like a project’s coding standards doc. It gets the mind-numbingly common, time-consuming debates out of the way so that people can focus on the genuinely tough problems. Like coding standards, though, process can get out of control and take a life of its own, burying people in busy-work.

  13. Great article and timely for us after our 2012 review highlighted we’ve reached the need for greater derail in our process documentation to reduce the stress of ‘new guard’.

    As the founder but also now a remote working member of the team, using process to define culture is vital for me. Not being there to explain and justify means that often individual interpretation causes confusion and additional stress. Our process documentation has been delegated within the team and I wouldn’t change that, but I now see that myself and long standing staff need to add the ‘why’ so future recruits can appreciate where the processes stemmed from.

  14. Joseph 11 years ago

    In the spirit of asking questions, what is the source for “70% out there giving process a bad reputation”?

  15. Sounds like the military. Busted process, and the only people who stick around are the ones who aren’t smart enough to fix the process.

  16. I’ve always found the preservation and understanding of company culture a fascinating challenge. It’s an obvious prophylactic against Chesterton’s Fence problems, but nobody wants to believe that they need to buy even the proverbial ounce of prevention – in time, employee hours, or any other currency.

    So naturally I’m fascinated by the notion that there are people whose job is to document and compel cultural norms. How do these people come to these positions? Is it a branch of HR, or management, or…? What are their qualifications? How do I get in on this?

  17. IBM nailed it. When I joined IBM, I experienced what it means to become an IBMer. The culture becomes part of you. It becomes part of the way you think. It is a huge corporation with thousands of employees but somehow the culture still permeates everywhere and everybody knows how to get stuff done. The process is documented somewhere but you hardly have to look at it. People already follow the process naturally. It is second nature.

    Unfortunately, when I worked there many years ago, I was only an intern. It was my first job and I simply assumed ever other place was like this as well. I have never found another place like this.

  18. The process of finding an answer to every question must be frustrating for engineers especially if they cannot come up with the right way to actually reach the right conclusion. They certainly have my admiration for always trying though.

  19. Andre 11 years ago

    Great article (as usual) and I really wish establishing of processes worked the way you describe. I don’t know about the US of course, but what I found in Europe is that companies in general and especially R&D departments do not care much about process as long as “it works”.

    Then for some reason the results get worse, the products don’t sell anymore and management calls external consultants, who define processes for the company. Which miraculously turn out to only help the consultant getting rich of course, but don’t help the company in any way because they often enough directly contradict culture.