SW Methodology - Shaolin Style

One advantage of blogging is that it enables you to articulate your vague inchoate thoughts more coherently and more importantly, it enables you to develop your thinking. After ranting about the inappropriate use of TDD, I've been able to develop my insights further. Expressing ideas helps to form them as Paul Graham says.

The most fruitful way to think of any pre defined software methodology seems to be that is an encoding of certain practices intended to teach you certain lessons.

In Karate (or any martial art) there are sequences of fixed movements called kata. In music, there are fixed forms called "etudes" (in western music. A rough analogue would be the varna in (South)Indian classical music). Conceptualising a methodology as an "encoding" of what worked for certain people in certain situations is an alternative to "methodology as religion"

The practitioner of a kata (or etude) seeks to perfect each component of the larger structure (a punch in a kata, a particular phrase in an etude) and since absolute perfection is elusive, the practice may go on for years or a life time.

If one thinks of the 12 practices of Extreme Programming (say) as forming a "fixed form" that teaches certain principles of software development (vs as some kind of sanctified system of "rules to follow or you'll burn in hellfire") then it becomes easy to both strive for perfection of each component (say unit tests) while practising, and modifying or discarding the same component during performance as need demands.

Combat is different from kata practice in a dojo. When you are in a dark alley facing someone who tries to gut you with a knife, the important things is to win that fight, not conform to the structure of the kata. When you are practising the kata, the idea is not be creative and do "free flow" but to focus on mastering the form exactly as expressed in the kata. The immediate goal of kata is "perfection of the form" (and the lessons learned therefrom) not "effectiveness in attaining the ultimate purpose".

Guitar Etudes have odd fingerings and chord sequences which will never make it into a lead solo. But when you practice the etude you do what the notation says , not what might best express your musical intent or satisfy an audience. While practising, a gutar maestro will be able to perform an etude perfectly as written in the notation, awkward fingerings and all, but when he is on stage, improvising a solo, he really doesn't think of "exactness of fingering". He focusses on musical effect and allows his fingers to go where they will.

Once you accept that a set of rules telling you how to develop software is just a "fixed form", it is easy to discern the two errors that software methodology practioners (not to mention the "evangelists" who have found the "One True Path") consistently fall into.

The first error is to apply the criteria of practising the form to the performance. iow, to insist that you should fight only using the movements of the kata you learned. "Ohhh you are not writing a test, you are not Agile... Code without tests is 'legacy code' Shame on you ".

The second, and more subtle, error is to apply the criteria of performance to the practice of the form, ie you treat the kata as a combat situation. "Aargh writing tests for javascript is impossible so I won't write any tests because obviously this unit test stuff is impractical" or " Writing tests means I write more code so I will be slower. So I will never write any tests".

Of late there have been some attempts to use the underlying principles of "kata" to improve software development skills. This is the "practise the form perfectly in a non combat situation" approach.

What has been missing is the more important second half -- "combat is not kata". In a hypothetical combat situation, in that dark alley with someone lunging at you with a knife, it isn't very important that you take up the perfect "horse stance" and then do a perfect roundhouse kick. If you end up in hospital, leaking blood from half a dozen stab wounds it doesn't matter if you "took the stance perfectly". As Bruce Lee once said "On the street, no on knows you are a black belt".

In the alley, facing a knife thrust, using a garbage can lid to block the knife and knock the attacker out cold will work fine, nevermind that in Shotokan Karate there are no "Garbage Can Lid Kata".Of course if you've practiced your forms (and other practices like kumite) well for many years, you won't be sweatily flailing around, panting for air and driven by adrenaline, like an untrained person will. You will be in control, using the minimal amount of movement and force and stay balanced all the time , and be aware of things like distance, force, balance, vector of attacks and so on.

You can even learn something from the combat situation and incorporate it into your practice ("hmm fighting in the dark in a cluttered alley is quite different from sparring in well lighted uncluttered dojos so how do I learn to fight better in such conditions?"). Who knows, you may be able to distill your experiences to create a "dark room kata" for others to learn from. And the practitoiners of this "dark room kata" wil in turn diverge from its fixed patterns in their own combat situations. And that is exactly as it should be.

One problem with software development is that "practise" and "performance" are mixed up thoroughly. Both generally happen within the context of a project someone else is paying for. A software developer learns "on the job". It is often impossible to change this state of affairs, but it is helpful to distinguish the "practice" and "performance" aspects , the kata and combat, the etude and the concert.

When you are learning extreme programing, do exactly what Kent Beck teaches in the "white book". When you are actually developing software it doesn't matter if you aren't "exact". If you have difficulties in applying a particular practice, make note and use what works. Later you can think of whether the problem is one of insufficient practice of the technique (in which case you practise more) or whether the technique itself is a misfit for your circumstances (in which case you modify the technique or discard it entirely). There is no intrinsic merit in force fitting the combat situation into the framework of your "school" of fighting. You'll just get stabbed.

There is no "perfect kata" which will reward rigid adherence with universally victorious combat ability. There is no "perfect methodology" which will guarantee success if it is followed "perfectly". Extreme Programming (to use one particular methodology) is not magic. Is is just the encoding of what Kent Beck (and others) learned by trying to get better in the projects they worked on. By all means learn from Kent. Don't worship him (or XP). Ignore the high priced consultants. Your situation and requirements are likely different from what Kent faced. Nobody cares what methodology you follow as long as you write code that delivers value. Arguments about "Shotokan Karate is better than Shaolin Kung Fu" are on the same level as "XP is better than RUP (or waterfall for that matter)".

Combat is not practice. A dojo is not an alley. And vice versa.