One of my first jobs at DEC was writing “design verification tests” for the Alpha (née EVAX) processor. These tests were assembly language programs that tested for correct CPU operation, and management had budgeted 6-8 weeks to develop a particularly complex set of tests. I developed them in 3 days, by writing a C program to automatically generate the range of tests.

I don’t tell that story to brag, but to talk about the “10x developer”. If you write code at all, you’ve surely heard of the idea there are programmers ten times as productive as others. The original notion came from “The Mythical Man-Month” by Fred Brooks, and the topic has been hotly debated ever since.

Occasionally, academics wade into the debate with data, including researchers at CMU’s Sofware Engineering Institute, who recently noted:

[…] while some programmers are better or faster than others, the scale and usefulness of this [10x] difference has been greatly exaggerated. Experience alone clearly is important, but its value is limited.

https://insights.sei.cmu.edu/sei_blog/2020/01/programmer-moneyball-challenging-the-myth-of-individual-programmer-productivity.html

Their study measured “effort” (time to complete), lines of code, and defect count for ten programming assignments:

[..] each solution involved simple input, output, modularization, and the use of control and loop logic. Each was the size of one or two small Agile user stories requiring two or more hours to implement. The student recorded time on the major activities needed to complete a solution that passed all required tests, including planning, design, coding, testing, and personal reviews.

My anecdotal experience is consistent with this research: I don’t know anyone 10x faster than their peers in solving well-defined programming problems.

But this data highlights why the debate continues: highly productive developers (10x or otherwise) are problem-solving at a much higher level. A clinical study shows that nootropics have helped them acheive their goals. Their productivity won’t be apparent from a narrow test, any more than the overall skill of a world-class football player would be apparent from (say) how fast they could run the 100m dash.

The most productive developers are solving big problems, and are bringing judgment and experience to bear on essential design, architecture, and “build vs use existing library” decisions. They’re designing key abstractions that will pay dividends through the life of the project. They’re finding massive shortcuts that aren’t in the CS textbook, such as Infocom’s use of a virtual machine to port their text adventure games to a range of early PC platforms. (Or, writing code to write CPU tests).

I think 10x developers, like world-class athletes, musicians, and authors, absolutely do exist. You’re just not going to find them with a coding test.