“Stack Ranking” Observations from My Time at Microsoft

“I hear you’re a free man,” the Microsoft manager said to me, clasping a hand firmly on my shoulder, in that “I’ve got your back, bro” sort of way, and not without a gleam of envy in his eyes.

The manager and I never directly worked together, but we both were on the Developer and Platform Evangelism team. To borrow a phrase from the U.S. Army, our job was to win over programmers’ and IT professionals’ “hearts and minds” and convince them of the benefits of using Microsoft tools and technologies. It was February 2011, and we were at a cocktail party at an internal conference in downtown Seattle. He’d heard about my separation.

“That’s right,” I said, taking another swig of some local microbrew and probably wincing a little. It had been a rough couple of months; she’d moved out around Christmas, and I landed in the hospital with a near-fatal case of the flu in January.

“There’s an upside,” he said, “there’s not a married man out there who hasn’t imagined what they’d do if all of a sudden they were single again, knowing what they know now.” He wasn’t the first married guy to remind of this fact, and like all of them, he was right: I was living the married guy’s combination fantasy and nightmare.

“So what are you going to do now?” he asked.

“I have been asking myself that same question for the past few weeks,” I replied. Weeks. The time since the ex-wife and I were under the same roof could still be measured in mere weeks. “I’ve decided the best thing for me to do is to emulate Tony Stark,” I said half-jokingly.

“Tony Stark…”, he said, without even the slightest glimmer of recognition on his face. “Who’s that?”

Like religious evangelism, technical evangelism requires an understanding of the culture in which you work. I would argue that this understanding extends to knowing its fictional icons. You’d probably (and quite rightly) wonder about someone who works in advertising who hasn’t heard of Don Draper, or a NASA employee who doesn’t know who you’re talking about when you mention Captain Kirk. At the time this conversation took place, there had been two wildly successful Iron Man movies, and even Forbes had done a piece on Tony Stark in their list of top fictitious rich guys.

Continuing my chat with the manager, who was a pretty nice guy and showed genuine concern about my major life change, I discovered that he’d be completely lost if you sat him in front of Visual Studio and asked him to write a “Hello World” application for the desktop, the web, or mobile, and system setup beyond pulling a laptop out of a box, plugging in an external monitor and connecting to wifi often confounded him. And yet, he was an overseer of Microsoft’s outreach to developers and IT pros, often called to speak in front of them and make hand-waving presentations of software and tech that he neither used nor knew how to fit together.

Further conversation led me to discover that the skill that had earned him his lofty level 63 position (like World of Warcraft, Microsoft has levels; also like World of Warcraft, the real fun begins at level 60) was that he could make the scorecard green.

At Microsoft, you’re only as good as your metrics. “Metrics” often translated into money. As the Stratechery article If Steve Ballmer Ran Apple puts it:

Employees were incentivized by dollars and cents in the form of bonuses and stock grants. Bonuses and stock grants were tied to a stack ranking system, that devolved your performance to a number. What was measurable mattered, particularly if it was measured in money.

The result is inevitable: Microsoft is a company filled with people motivated by measurables like salary, bonus, and job level. Anyone who isn’t would necessarily leave. Unsurprisingly, said people make choices based on measurables, whether those be consumer preferences, focus group answers, or telemetrics. The human mind is flexible, but only to a point.

…

Yet, if Apple’s success has proven anything, it’s that measurables aren’t the half of it. Things like design can’t be measured, nor can user experience. How do you price delight, or discount annoyance? How much is an Apple genius worth?

In the consumer market, it’s the immeasurables that matter. It’s the ability to surprise and delight, and create evangelists. It’s about creating something that developers demand access to, and that consumers implicity trust. The consumer market is about everything you can’t measure, everything Microsoft’s legion of mini-Ballmer’s can’t see and will never appreciate.

Once a year, you sit with your manager and come to an agreement on a set of commitments — a set of “deliverables” — for which either you individually, or as part of a team, are responsible for. As a developer evangelist, I had individual commitments that included doing x presentations in front of y people, discovering and publicizing a specified number of “success stories” where Microsoft development tools and platforms played a part, and hitting a target number of Azure sign-ups that could be directly attributed to me.

The aggregate commitments applied to both our own local teams as well as the developer evangelism group as a whole, and were things ranging from tech and tool adoption rates to “sat”, short for satisfaction, which as far as Microsoft was concerned, was an actual quantifiable property, derived from surveys and distilled into a single, mysterious, context-free number that seemed to bounce between 130 and 150 during my tenure.

There’s a mid-term review where you and your manager review your commitments to see how well you’re doing, and where you need to redouble your efforts. Near the conclusion of the fiscal year, which ends on June 30th at Microsoft, you write up how you well you think you met your commitments, your manager does the same, and after which the two of you have a long one-on-one meeting and compare notes. It’s at this meeting when you’ll get a very clear idea of which bucket you’re going to be put in: the 70, the 20, or the 10.

It’s often referred to in the tech press as “stack ranking”, but within the walls of Microsoft Canada, I’ve only heard it referred to as “70-20-10”. Those numbers refer to the three categories into which every manager at Microsoft had to place every one of his or her direct reports. These categories are:

• “Achieve”, into which 70% of your direct reports would go. The official description of the sort of person your supposed to put into this bucket is “Demonstrates potential at minimum to broaden in one’s role or to advance one career stage or level as a leader – either as a People Manager and/or individual contributor. Past performance suggests capability of delivering consistent and significant contributions over long-term. Competencies typically are at expected levels.”
• “Exceed”, where you’d place the top 20%. If you did particularly well that year, you’d be put into this category. The description reads: “Demonstrates potential to advance faster than average as a leader – either as a People Manager and/or individual contributor – preferably multiple levels or two career stages. Past performance suggests capability of delivering exceptional results over long-term. Competencies typically are at or above expected levels.”
• “Underachieve”, or the bottom 10%. This category comes with a set of consequences and often a lot of self-loathing and voluntary departure from the company. “Demonstrates limited potential to advance.”

Once a year, during our team offsite, the managers would take the time to explain this system, all the while taking great pains to explain why it was a good thing, in that sort of “toxic sludge is good for you” way. There are all sorts of problems with this sort of system, two of which were pointed out quite astutely by Bruce Lynn in the 70-20-10 blog:

My concern about the application of these ‘distributions’ is two fold. First, they are statistical constructs and most executives lack [the] numeracy to properly interpret and work with them. For example, while these distributions might generally apply over large numbers, no reasonable statistician would expect them to hold identically for smaller populations. In companies, sometimes these distributions are imposed on teams of just a handful of people. This imposition violates all sorts of statistical principles.

Also, one significant variable that such simplistic systems omit is the outcome of the group. A group that has achieved great things is likely to have different distribution of these categories than one that hasn’t. A world champion team might have more than ‘20%’ of its team as ‘All Star’ talent. In fact, the very fact of the skew to high performance could explain the overall team success. In another scenario, none of the team might be standouts (ie. no one at the 20% high achiever level), but the team work exceptional leading to a whole greater than the sum of the parts.

But more important than the clumsy application of such distributions is their application at all. These breakdowns are ‘descriptive’ stats, but misguided managers use them as ‘prescriptive’ rules. Misguided executives introduce policies which force managers to classify their people into such categories. The execs like the numbers because they are a nice, repeatable, McDonalds-like process which is easy to implement and understand on a superficial level. Unfortunately, it is often misapplied in specific instances. It is sort of like expecting a day with a ‘50% chance of rain’ to rain for half the day and be dry for half the day.

The other problem with 70-20-10 is that it can pit teammates against each other (luckily, this was not the case with my group). I’ve seen it create situations like the old joke about the two campers who encounter a hungry bear in the woods. The first camper says “I’m glad I wore my running shoes!”, to which the second replies “You can’t outrun a bear!” The first camper then says “I don’t have to outrun the bear. I just have to outrun you.”

If you managed to outrun the bottom 10, you could do very well. When I worked there, annual bonuses at Microsoft ranged from 6 to 60% of your base salary, and salaries at Microsoft are pretty good (in fact, people who leave Microsoft to work elsewhere often experience “reverse sticker shock” when they discover that most places outside The Empire don’t pay as well). My last bonus put me at a point where I could’ve bought a brand new sedan with cash.

When you combine bonus in the tens of thousands of dollars and tech culture, where Asperger’s syndrome is sometimes considered to be a badge of honour, you get antisocial, every-man-for-himself behaviour. In his Vanity Fair piece on Microsoft last year, Kurt Eichenwald wrote:

At the center of the cultural problems was a management system called “stack ranking.” Every current and former Microsoft employee I interviewed—every one—cited stack ranking as the most destructive process inside of Microsoft, something that drove out untold numbers of employees. The system—also referred to as “the performance model,” “the bell curve,” or just “the employee review”—has, with certain variations over the years, worked like this: every unit was forced to declare a certain percentage of employees as top performers, then good performers, then average, then below average, then poor. …