Process breeds Process

From a Tedd Ziuba post hating on process.
Software development methodology is organizational Valtrex. Sure, it treats a symptom, but the only cure for the underlying disease is to never have contracted it in the first place
He goes on to point out that the main reason to have process/methodology is that, thanks to the Law of Large Numbers, the more developers you have, the more your average skill is going to regress to the mean, i.e., the skill of the Mythical Average Developer
The underlying cause is that the variance of developer skill on your team is too high, which means your team can't execute well, and you need process to wrangle the laggards 
Communication Overhead
 In my experience, there is a second reason called, you guessed it, Communication Overhead.
See, I am the Perfect Developer.  Everything I do is Perfect.  Crystalline Joyeous Glorious Perfection. (yeah, yeah. humor me).  Oh, did I mention I don't need to sleep either?  And I don't need breaks?
Given all this, it turns out that I can build anything that takes 24 units of work a day or less.
Simple, right?
A 144 unit job?  6 days, and Bobs your uncle, its done.
But, but, what if we have only 3 days?
It turns out that I have this friend - Sally, who is also Perfect.  And she is available to do some work.  The two of us, together, can get the job done in 4 days.
"Whoops!  Bad Math!" is probably going through your brain right now.  Except it isn't - enter Communication Overhead.
It doesn't matter how Perfect we both are at Developing - when we are working together, there is guaranteed to be some communication error, and that is going to suck up time (lets pretend that its 1 unit per person).  So, total effort taken to do the job is 145 units (144 work units, and 1 communication unit), or 3 days + 1 hour.
You can see where this is going - the most work that can ever be done by Perfect People is 300 units (do the math), and the 25th person is actually a time suck, with there now being 25 units of communication necessary for him/her to do the requisite 24 units of work.

And this, this is the second reason you need Process.  See, what Process Overhead does is decreases everyone's Perfection, but it also decreases Communication Overhead.  So now, I, Mister Perfect, can do only 18 units of work in a day, but my Communication overhead is zero (humor me).  Therefore the total size of job that can be attacked is, well, huge.

With me so far?  Good, because most everything I wrote above is basically a higher order of bullshit.  None of it matters.  If you design well, and have a couple of smart people, well, you'll have some communication overhead, but it won't be that big a deal, and Stuff Will Get Done.
The problem with Process Overhead is, to put it simply, 
PROCESS BREEDS PROCESS
Seriously, the theory I espoused is sound, but completely totally absolutely impractical in the real world, because Process Breeds Process.
  • Had a meeting?  Where are the minutes?  Oh, wait, lets get someone to do the documenting.
  • Standups?  Sure, but we should do them every damn monday morning even if we didn't actually do anything, because, well, we just should.
  • Great, the V.P. of Development is now in the standups, and she is a bit overbearing.  So lets have another standup before the first one so we can Get Things Done.
  • Nuts, the CEO wants a status report email.  Great, lets have a meeting to make sure we get the report right.
You get the point.  Process Breeds Process.  Its a slippery slope, and once you start down that road, there is no end in sight, and pretty soon the Perfect Person is operating at 1 unit per day.
And no, the alternative is not endless chaos.
Some of the best work that I've done, on Large Scale Production Systems That Just Can't Go Down And Are Really Important Trust Me On This have involved a small group of us, co-ordinating with each other like mad, but without a lot of formal process.
And it worked.
Really.
Brilliantly.
Shiningly
Awesomely
(You get the point)
Of course, the fact that most of what we do is in Erlang (*), which is (relatively) side-effect free helps tremendously, but this general approach tends to work regardless.  Functional Programming helps, but everything I've said is more of a mindset thing than a language thing.
If you really have to build out a God-awful huge system, and you can't figure out how to get appropriately decomposed into modules that are easy to write, debug, and test, well, do it differently!
Because, frankly, if you can't decompose it into "simple" modules, then, in the end, it pretty much isn't going to work. (And again, this is where a serious Actor Model helps. Hello Erlang :-)  )
Oh, it'll work - after a fashion - but do you really want everyone up 16 hours a day fighting fires?

In closing, I'll put it back in Ted's hands
This is the hardest one, and, if you're serious about doing away with process, is the one from which all others follow. Just stop having process. Cancel your weekly planning meetings. Get rid of your prioritized list of tasks. Stop having your daily stand-ups. No more status report e-mails.
Just stop doing that stuff, and get rid of anyone who can't cope.
Nothing terrible will happen.
My current project, which will be launching in the first half of 2012, is as close to anarchy as a project can get. And it's working. I trust my teammates to get their work done and do it well. Sure enough, they do.

Note:
(*) - Yeah, Lisp and Haskell probably work too :-)

Comments

Popular posts from this blog

Erlang, Binaries, and Garbage Collection (Sigh)

Its time to call Bullshit on "Technical Debt"

Visualizing Prime Numbers