SourceDiving into Nethack

Manoj's recent post woke sleeping demons and they compelled me to fiddle with nethack.After winning the game(ascending in nethackspeak) a year ago, I forced myself to leave the best game in the world aside and get on with life.

This time though I wanted to hack the game a bit.So I downloaded the source version and started fiddling with it. The idea was to create a couple of new characters. For 20 year old code, the nethack source code is certainly well laid out. After years of OO coding it takes a while for the brain to adjust to C 's intricacies. The codebase has a fairly simple structure and once you wrap your head around all the #ifdefs, hacking becomes easier(though never really easy).

I've added a new dungeon(called the TWI level :-P) peopled by some new monsters. Manoj the VC throws gold around in abandon. Owen and Fiona, the Canadian Tourist Monsters attack you with blinding cameras and throw gobi manchurian at you, Rajesh the bookseller hits you with new books and steals gold from you.Naresh the Organizer randomly teleports you into several ASCI conference dungeons. And if this doesn't strike you as particularly funny, don't worry. These are all inside jokes that makes sense in my world.

Now I'm trying to add a new player class -the Ninja - combining about 70% of the Samurai's fighting skills with about 80% stealth of the Rogue with a few Monk spells. The "logic distributed all over the place" nature of C makes this difficult.If the code base was of a well factored OO design or the authors had used a language like Lisp, these changes would be trivial.(If anyone has a year or so of time to port nethack, let me know).

Having said that though , I wish "real world" software was as much fun to modify.

You will have to excuse me now. My Wizard character just ran out of spell points in the Gnomish Mines, with a horde of nasty gnomes and Giant Ants descending on him.Ravi the Wizard broke his Wand Of Light but managed to inscribe "Elbereth" on the floor and the power of that word is keeping the monsters away as the mana slowly builds up. However there is a Werewolf who just entered the Dungeon and morphed into human form Nethack Lore says such humanoids are not impressed with this protection spell.

Deep Trouble.

To add to all this there's a nymph waiting to seduce me and steal my Magic Cloak and a Floating Eye watching impassively....

If Linux Had MBA s

I don't like blog posts that just point to another page on the web and don't say anything original.

But just this once , I'll make an exception.

I've always liked the Mini Microsoft blog and the recent coverage by BusinessWeek (and Slashdot) has given it a lot of exposure. If you ever wondered what it was like to work for the Borg, read it first. Be sure to read the comments, which are mostly posted by people inside Microsoft. The comments are way more fun than the articles, which are themselves very good.

If you want a feel for what the blog is like, look here.

Snippets

"Oldskool MSFT managers wouldn't be afraid to say "no" to upper management or tell Bill or Steve to fuck themselves. Seriously. They better have a point and ammo to back themselves up, but they wouldn't be afraid to say it. The new managers don't like waves. You are punished if you criticize. That's why we end up on blogs instead of yelling "Bullshit" internally like we used to. Does Steve throw chairs? Heck yeah, so does Bill, most architects, etc. We used to have holes punched in walls around NT. But I would rather have that than the passionless yes-men corporate Dilberesque managers we have now. And there is no accountability at MSFT anymore because of this. No manager dares to point a finger because they are afraid of a finger being pointed at them."

More fun .. "For the people bitching about test, this tester says you can f'ing bite me. If you guys could ship code that worked, heck, code that even COMPILED PROPERLY without a buddy test and a test pass, Microsoft wouldn't be employing us. The growth of QA is specifically because developers cannot be relied on to write functional code."

Someone complained that "When I started, my first manager gave me a project by pointing me to the PM who essentially sold it to me as 'this is why we need it', and I started on the library and delivered a fully functioning project two months later... Fast forward to the present: I won't know what I'll be working on until a PM delivers a fifty page spec with pixel measurements and colors...Then I get to start on a document describing all technical aspects of the implementation in detail, together with a daily schedule of what I will do when. All of this will be debated in many more meetings. Then I start writing code."

A PM responded with

" ...An increasing number of PM roles are filled with top CS students from top schools that are looking to have a broader impact on the product, enjoy design, and can work with people (this was how they sold me the PM role). The result is more control being placed into their hands, often times turning the dev into a blackbox that inputs specs/PM direction and outputs coded features. ..."

to be floored by

"..What a load of bull. The PM positions have traditionally been filled with techies that couldn't handle code. They are a good dev's worst nightmare: someone who knows just enough about software to throw out the words, but a complete lack of depth to understand what they are saying. Combine that with the fact that their job title includes the word "manager" and the vaguest job description I've ever seen in my 20+ years of software development and you have a walking, talking, time-wasting disaster. Feature direction and design? Yeah, it's called today's whim.

"...the PM's I work with have backgrounds in CS, and would proudly call themsleves engineers.."

I'm sure they would...fortunately, no one would believe them. "

Hilarious!!! Of course there are more insightful posts too .

"..Regarding all of the recent comments about PMs, my opinion is that there are some great PMs that really help improve the product. The problem is that I think it is *MUCH* easier for a talentless PM to escape notice than a talentless dev. If a PM sucks, dev and test can ignore him and the feature still gets written. If a dev sucks, he can't hide the fact that the feature is late, always crashes, etc. .. "

More ...

"..Testing as a concept is fine. Testing as the accumulation of the political and inept employees is not. ..."

The part I liked best (and where the title of this post came from) was

So, when you think about it, MSFT used to be like the open source community. A bunch of engineers writing a ton of code. And, the engineers were motivated by the same thing. It wasn't about cash, it was about making something cooler than the competitors, just like the Linux guys want to make something cooler than Windows.

That was the root of "crushing the competitors" just like the Linux guys want to crush Windows.

So you open source guys should at least appreciate that. Imagine going from the environment you are in now to having a boss sitting there with charts and graphs, with people who don't code telling you that you need to go to idiotic meetings, etc.

We used to be like you except working on closed-source. In fact, I used to be big on FreeBSD and had conversed a few times with Linus when we were both in school and Linux was still a hacked Minix.

All things considered, engineers are engineers and want the same things whether they work on Linux or Windows. They want to write screaming code that is better than anyone elses.

But now, we've been Dilbertized. We've been internally infested by professional managers who don't know how to engineer.

If Linux had MBA's with no coding experience telling developers what to do, what do you think would happen?

Well, that's what's happening at MSFT. Put yourselves in our shoes for a minute, and I'm sure you'll be able to understand the disgust, frustration, and outrage. "

Bwaaaa ha ha ha ha ha!!! That's the funniest thing I've read in years ! And remeber all this is from just one post in the mini msft blog! Read ! Weep!

Manoj Evokes The Nethack Magic

My friend Manoj Govindan started blogging after some ..err.. encouragement from Yours Truly.

His first post is about how he got addicted to NetHack , a game Yours Truly pushed on him.Very nice post too, hopefully we'll see more posts on piano playing, wargaming and other hobbies of his.

Meditations on Mastery: Ethics and Transformation

I just finished reading George Leonard's book, Mastery . I've heard a lot of praise for this book and it is well justified. This is one of those rare books that seem very simple and obvious but then it grows on you and you find yourself spinning your own lessons out of it.

Most people who comment on the book emphasize that it teaches you to "love the plateau", in other words to keep on the path of practice even in the absence of any visible progress. So it does, but others have spoken on this so I won't. Instead , I will focus on something I learned from it.

In Chapter 11 (Tools For Mastery ), Leonard says "Acknowledge,but don't indulge your Dark Side" . He goes on to say ...

"...(We have a lot of)energy locked up in the submerged part of our personality, in what Carl Jung calls "the Shadow" ... (Poet and StoryTeller Robert Bly says),

'A young child can be visualized as a lively ball of Energy that radiates in all directions. But the child's parents don't like certain parts of this ball. In order to keep the parents's love, the child puts the parts of him they don't like in an invisible bag he draws around him...Then our teachers have their say 'Good Children don't get angry over such little things', So we take our anger and put it in the bag.'

By age twenty, he (Bly) says , only a thin slice of our original energy is left".

This is a metaphor that makes sense to me. The image of people who glimmer with pale flickers of uncertain light, while dragging along a huge invisible bag that thrums with "forbidden" energy but is firmly locked, is fascinating because throughout my life I have been fascinated with the notion of why(with exceptions) good people are so ineffective.

The people who get ahead and get things done are very worldly wise and know how to manipulate the systems around them and thus "play the game". But at the end of a lot of this game playing they find that their efforts have been meaningless and the rewards of what they gain are not as satisfying as they hoped. Thus politicans are very effective in the world but very very few of them are nice people, and few of them live meaningful lives or die content.

On the other hand are those who have a lot of idealism and caring but are stunningly ineffective in life. These folks have all the right ideas of what the world should be like but seem unable to move the world from here to there and are dismissed (rightly so) by the "players of the game" as insignificant or "idealistic". .

Thus on the one hand we have people who have an excellent sense of direction but no power, and on the other, people who have (or know how to get) power but are not clear on what to do with the power and end up either preserving the status quo, or worse changing it to a one of increased Tyranny and Evil.

The myths of our age show these archetypes very clearly. Thus in "The Lord Of the Rings", (I speak of the book version, the movies are a pale shadow ), the Dark Lord, Sauron, is impossible to defeat at the height of his power. Even with the loss of his Ring Of Power, he is frighteningly effective in drawing to himself massive armies (dwarfing anything the "good guys" can manage) and even his subordinates (The Nazgul) are almost unstoppable. What makes him a "dark" figure is that if he were successful, he would impose an Age of Tyranny and Terror on Middle Earth. In other words, if he were to be successful, all that terrifying force of will and ambition and cleverness would be applied in the wrong direction.

On the other side we have a motley band of "good guys", each of whom is no match for The Dark Lord individually,but collectively manage to defeat him. The Ring Bearer, Frodo has almost no heroic qualities except that his heart is in the right place. Frodo is the epitome of the "idealist", a guy who wants peace and calm and radiates "Good Will to all", but will essentially, if left alone, while his days away with his tobacco in a corner of the idyllic Shire.The other "Good Guys" all epitomize the perfection of one major "good" quality (Gandalf has Wisdom, Aragorn has the Leadership), all of whom have to work in concert to defeat the Dark Lord, who combines all "dark" qualities in one person.

Another Modern Myth , Star Wars , offers its own parallels.There are these "good guys" , the Jedi who shun the "Dark Side", and work for Peace and Goodness. And the Sith who are focussed on dominating the known world and stamping it in their image.What is often missed is that the Sith are tremendously more effective, with two Sith (there are always only two) matching and often defeating hundreds, if not thousands of Jedi. And the Hero is a kind of clueless Everyman, and really does not evolve very much through the story except to somehow mysteriously become a Jedi Master in a very short time.

Anyone with some common sense would know that becoming a Jedi Master would take decades of very hard work. This bending of the rules, where the "Chosen One" doesn't have to work very hard to achieve superhuman levels of mastery (Neo of the Matrix is another example) has its own dynamic in modern story telling but that is a subject for another post.

So here is the formula for creating a (modern) myth. First create a Bad(make him really really bad) Guy who has immense Power and is frighteningly effective. Bring him into conflict with an "Everyman Joe" kind of figure (so your readers/viewrs can empathise with him) who is nonetheless somehow the "Chosen One". Sorround this Good Guy with pseudo Masters who are all old and past their prime and speak in riddles. Throw in a "True Love" Aspect if you want to get the women to read the book/watch the movie. (These days it helps if the main female love interest is very feminist in her views and "just as good as the men". Thus the Leia as Jedi (bwaa ha ha ha) theme or Arwyn the Elven Princess facing off against the Nazgul (thankfully not in the book, only the movie).

Back to the main point. Leonard's analogy of the 'person with the bag(containing the Shadow Self)' is very apt and while his subsequent advice to harness the Dark Side(thus when you feel Anger rising, he advises you to not give into the Anger, but to harness its energy to do something useful) is easier said than done, it points the way to a different-from-expected but truer kind of Mastery.

Thus in my personal cosmology(feel free to whack away at this) a Master would not be some wimpy hero (like Frodo or Luke Skywalker), but a genuinely powerful being who could go toe to toe with the most powerful "bad guys" without any Deus Ex Machina popping in to conveniently save him just when the villain is about to make mincemeat of him. And one step to achieving this kind of power would be to consciously "empty the bag" and be Master of both Light and Shadow and be both worldly wise and idealistic.

Even in History(and not myth) the "good guys" are those who combine tremendous pragmatism with tremendous idealism to defeat the "bad guys" who are very very talented or powerful but pull in the wrong direction. (e.g. Roosevelt Vs Hitler, Abraham Lincoln vs the Confederates,Gandhi vs the British Colonizers).

Thus to go back to George Leonard's(or rather Robert Fly's) analogy, a true master will be neither a glowing child, nor a weary everyman figure with a huge bag of Darkness, who mysteriously becomes a master because he is "chosen", but an unencumbered figure with a consistent, calm sheen of power,who walks the line dividing Light and Dark,able to draw on either, and to transmute all the energies he harnesses or encounters into various combinations of light and shadow as appropriate to the environment, but chooses steadily move towards an idealistic end.

Thus , in my opinion, Leonard's advice would become,"Don't indulge your Dark Side. Acknowledge it.Then harness it".

Sounds tough? oh well, No one said mastery is easy !

Darth Ravi, Jedi Rajesh, And The Craftsmanship Manuscript

Dave Hoover blogs about getting feedback on his book. Rajesh and I have been reviewing Dave and Ade's forthcoming book.

At one point I said,

" ...Rajesh probably is more suited to chime in here than me. The standing joke amongst us is that Rajesh is the Jedi - basically a good guy focussed on people and I am the Sith - focussed on pragmatism and effectiveness ...We Sith have to be with two of us against an army of wimpy Jedi

And then, after seeing some of my "tear apart the material and point at every hole in the structure and logic " style reviews and Rajesh's more "softly, softly" approach, Dave started calling me "Darth Ravi" (and Rajesh, "Jedi Rajesh").

I like it! Now to get one of those shiny Vader masks to breathe through ...

The Meanings Of Mastery

Recently I was startled when I heard someone say "Linus Torvalds is not a "master programmer" because he doesn't 'mentor' people ".

My first reaction was "bwaaa ha ha ha ha ha !!! yeah right ". To me whether someone is master at an activity has nothing to do with any behaviour on his part but is a function of the degree of skill he has.

While commenting on my last blog entry someone else picked up on the same theme. The reasons given didn't stand up to scrutiny since they were based on an arbitrary religion (Hinduism) and what a particular view into that religious system claimed was "true" and "divine", but the phenomenon was interesting.

Two people making similair claims about something that was to me, "obviously" nonsense. My friend Manoj chimed in with a sharp mail that claimed "master " is used in contexts which have nothing to do with either skill or students (e.g. Post Master ; Thanks Manoj, that was good feedback)

Time for a closer look.

After looking up various meanings and usages of the word (a good idea Sriram), I find that the word expresses three distinct concepts, applicable to three different situations.

The first is "An expert in a topic" as in "He is a Master of 3 languages". In this case the "of" in "master of" is followed by a skill set, a non living thing. I will call this "variant 1" or "skill based" mastery. Here mastery arises from being able to do things in a certain "field of operations" that none else (or very few other people ) can .It is in this sense that Linus Torvalds is a master programmer.His mastery is over programming and only very indirectly over people .With or without people he is still a master of programming.

The second has to do with power relationships (however established). In other words "master" means " One that has control over another or others " as in "the dog ran towards its master". This is the sense in which "Post Master" (a person who controls other workers and processes ina a post office) or "master of a ship" (a person who dominates and has authority over other people on a ship) is used. I will call his "variant 2" or "people based" mastery.

In other words, this variant of "mastery" is present only in a context that includes dominating other people. This domination could be granted involuntarily by the dominated person (as in a master - slave relationship) or voluntarily (as in a master disciple relationship ). The fact remains that this mastery involves the exercise of power whether benevolently or not.

There is a variant of this meaning which comes from the Guild system in Medieval Europe. The twist this adds is that the master has to be "licensed" by some process, usually involving the approval of other similair "masters" to teach. Anyone who is not so licensed by some form of "guild" is not a master. I shall call this variant 2A or "guild mastery".

Thus, while Jesus Christ was a "Master" by the variant 2 meaning, (people accepted him as master and he had disciples) and anybody who became his disciple voluntarily granted him a dominance in certain areas of life, by the guild variation, Jesus would not be a master unless approved of by other "masters". Given that only John the Baptist "approved of" or "licensed" Jesus (and John himself does not seem to have any kind of "guild" approval) we will have to infer that a self proclaimed "guild of religious teachers" would have rejected his "mastery ".

And if you look carefully you will see that the positive versions of variant 2 would also depend on variant 1. In other words, Jesus was looked up to as a "Master" (by his disciples), not because some guild or bunch of people said he was a master because he could do things (clarify deep moral dilemmas , answer questions, work miracles) that no one else in his time and place could.

So now the question becomes, which of these models is appropriate to software development? To a certain extent this is a subjective question. However, that being said, There are people who want variant2 (a master and his disciples) or 2A (a "guild" which "sanctions" or "confers" mastery).

In today's world which is essentially egalitarian, networked, democratic and open, I stand with those who believe that mastery in programming (or anything else) is of the first variant, based essentially on the degree of skill. If you can develop programs in a demonstrably superior fashion, then you are a master. If you don't have a single disciple you are still a master. If you don't have a "guild" sanctioning you ,you remain a master. In other words, a Master first becomes a master and then may choose to teach in whatever manner he thinks appropriate (Sriram's idea. Thanks Sriram!). He is not forced to teach to claim masterhood or be "conferred" masterhood by onlookers on the sidelines. Thus Linus may or may not be choosing to teach by letting his code be open source. It doesn't matter. His mastery is self evident from the skill with which he writes code.

Martin Fowler is a master technical writer and speaker because he can speak better and write better than (most) other people can. Not because he is part of the Agile Alliance or Chief Scientist of Thoughtworks or any other "title" based attribute. These other memberships and positions flow naturally to someone who is a master by the "skill based" meaning of the word.

The main problem I have with the Guild meaning is that its essential purpose seems to be to satisfy some kind of psychological craving for certainty in an uncertain world, in an attempt to assuage an inner shakiness by formulating fixed levels and labels. You don't need a three fold, "peer sanctioned" guild like stratification to recognize someone better than you at programming and learn from him/her. Any programmer worth his salt does so all the time. And so I reject McBreen's book as half baked.

And anyway, whatever model you want to follow, invoking arbitrary religious principles or nostalgia for a romanticized view of the past to justify that choice is sloppy thinking. The solutions for the problems of this century are to be found here and now, not in the past.

So if you want to be a master programmer, focus on how to improve your code skills . Don't bother with whether some arbitrary group of people like those found on say Advogato "rate" you. If you must have these "strokes" from other people to validate your self-worth, be aware that the ratings as "Master" too will automatically happen when you have attained the appropriate levels of skill. And by then you won't care what other people say your "level" is.

As Eric Raymond says in his loginataka, (a parody of a guru-shishya dialogue),

"Rather must you Become, and Become, and Become, until Hackers respect thy Power, and other Wizards hail thee as a Brother or Sister in Wisdom, and you wake up and realize that the Mantle hath lain unknown upon thy Shoulders since you knew not when."

[Thank you Manoj, Rajesh and Sriram, for helping me clarify my thinking. Prakash too, though I absolutely reject your "guru/hinduism/indian culture/the Bible is divine" argument]

A Nostalgia for Guilds And Other Dangerous Ideas

After some thought provoking conversations over email with Dave , Ade and Rajesh recently, I read Pete Mc Breen's book on Software Craftsmanship .

The book's fundamental thesis is interesting. It says that the "software engineering" way of creating software is broken (I agree) and it posits a "craftsmanship" approach to software as the solution .(I disagree).

Let me clarify that statement.

I don't disagree with the idea that viewing software development as a craft rather than some kind of factory process will yield insights into how to progress along that craft.

The basic problem with the book is that it proposes what is essentially a medieval socioeconomic structure as a solution to a 21st century problem ,and selectively focusses on the virtues of the old "guild" system where practitioners of a craft were classified into 3 categories - apprentices, journeymen and masters .

Now let us examine how Mr McBreen proposes to solve the "software engineering problem" .

he says, (all emphases mine)

"Rather than attempt to industrialize and deskill software development, the software craftsmanship approach looks back into history to see how the problems of expensive labor and arcane skills were handled. " ( Chapter 5 )

While the book doesn't go into too much detail on which nation's history of what period is being "looked back at" , it is fairly obvious from the sorrounding context and talk of "Apprentice, Journeyman and Master" that he refers to the guild system that flourished in Medieval Europe.

So Mr Mc Breen, on the one hand says that a methodology originating in the 1960s and continously evolving since then (software engineering) doesn't work for software development in the 21st century so we should go back to the 14th , for solutions ! Interesting !

There are two ways to use labels like "apprentice", "journeyman " and "master".

One is that of a rough guide to where on a skill spectrum a practitioner of a craft falls.In any time period, in any craft or art, there will always be a fairly consistent ranking of people in terms of skill and it sometimes helps to mentally divide that spectrum into n categories and name those categories whatever you want.Yes, if n ==3 you can name the categories , "apprentice", "journeyman" and "master" .This is only a way of seeing the progress along a craft .Those labels have no real validity .

The second way is to use such terms as rigid unchanging classifications of people(however derived) and attaching separate privileges to these labels .

Thus you couldn't run a craft based business in medieval Europe unless you were a member of the appropriate guild and presumably of an appropriate "rank" to do so . Today anyone can start a business and thrive or crash based on the economic logic of the business. Which system would you prefer to live under ?

the fact that Pete Mc Breen supports the second ,"rigid label", way is evident from a paragraph that states

"A master craftsman may learn a new technology from an apprentice, but it does not mean that she is no longer a master. If anything, it reinforces her mastery because it illustrates a willingness to learn from everyone."

Hmm thus, irrepective of whether a so called "master" is demonstrably expert *in the technology being worked on * or not, he is always a master .And the poor fool who does, in this case, know enough to "teach the master" is still an apprentice . The labels are rigid .Once someone becomes a "master", even a situation where the practitioner fails totally does not remove "masterhood" .Nice!

You might think it would make sense to say that in this context where the new technology is relevant , the "master" and "apprentice" labels are reversed .Thus if an acknowledged superior programmer, say Linus Torvalds and I are pairing on creating a j2ee leasing system ,(I presume Linus will kill himself before he has to do this , this is all hypothetical, bear with me ), then, in the context of j2ee, logic would seem to indicate that *I* am the "master" (or "journeyman " or "Lord and Master" or "King Skeletor" .. use whatever labels you think cool ) and he is the "apprentice" (or "teeny weeny toddler" or whatever , as long as the label is "lesser" than mine ) .But no! Once a master always a master! it doesn't matter if you don't have a clue.

The more significant question of course is "does this kind of labelling make any sense" ? hmmmmm....

Other paragraphs in the book identify mastery with "peer recognition" . There is a very subtle fallacy here .In other words,as per the book , Bruce Lee was not a master martial artist because he could fight better than anyone, but because he is *seen* to be a good martial artist by his peers . This confuses cause and effect .

To see the fallacy embedded here, you can drop each condition in turn. Suppose Bruce Lee never became famous but retained as much of his fighting skill as he did . Is his "mastery" now non existent because he was not recognized ?

Now let us try it the other way . If Bruce Lee was much much more famous amongst "peer" (of course it is not defined how these "peers" would be selected. maybe they are people with the same label from the guild? ) martial artists but could never demonstrate his skill in combat (an analogous situation would be a large number of book authors having no really significant software to back up their supposed "expertise") would he still truly be a "master" ?

This labelling based on irrelevant attributes is the least part of the nonsense embedded in the book . More tragic is the notion of a whole set of practices to solve the "software crisis" built on this "labelling". Some of it is just common sense. Most of it is just misguided, intellectually unsound rubbish.

Thus two separate concepts of "apprentice, journey man and master" --one, a rough measure of skill , and two, artificial "labels" used to recreate an outmoded economic system-- are conflated. Now let us look into the guild system from which this terrible classification scheme is borrowed .

From the wikipedia entry for "Guilds" ,

" The guild was made up by experienced and confirmed experts in their field of handicraft. They were called master craftsmen. Before a new employee could rise to the level of mastery, he had to go through a schooling period during which he was first called an apprentice. After this period he could rise to the level of journeyman." This is almost a word for word description of the system McBreen wants us to replace present day practice with . Let us read further (the wikipedia article, not the book ) . " Apprentices would typically not learn more than the most basic techniques until they were trusted by their peers to keep the guild's or company's secrets. ".

Duh ? secrets ? "Journeymen were generally paid by the day and were thus day laborers. After being employed by a master for several years, and after producing a qualifying piece of work, the apprentice attained the rank of journeyman and was given a letter which entitled him to travel to other towns and countries to learn the art from other masters."

duh right . Does anyone else feel an increasing disconnect between the socio economic situation in the 14th century vs the present day ?

Continuing to quote "After long periods of apprenticeship to a master" ( an apprentice had to pay for this privilege !What do you think of having to pay a "master" or "journeyman" to learn programming? ) "and after producing a qualifying piece of work, the apprentice attained the rank of journeyman and was given a letter which entitled him to travel to other towns and countries to learn the art from other masters."

entitle him to travel ? a qualifying piece of work ? duh?

Now let us give Mr McBreen's ideas the best possible interpretation and decide that your "level" in software needs to be demonstrated, say by showing samples of your code , maybe your contributions to open source, before you get a job. Will this miraculously improve the success rates of projects ? Not at all, because the problem is that the need for programmers is well above what can be met by the "truly skilled" minority . If less than say ,250 people worldwide are truly "masters" and say another 3,000 or so of "journeymen" status in, say, Tennis (going by ratings and being generous) why should there be a greater number in software development ?

The difference is of course most people who play tennis do so because they at least minimally enjoy the game! To think that adopting some half baked scheme based on an ill defined outdated idea will raise these numbers(which is about the only way to "solve" the software engineering "crisis") is , at best , self deception and at worst .. hmmm ...let us not go there .

In other words the "software problem" is economic, not technical in nature . Unless a compelling economic argument can be demonstrated for this switch to an out moded way of "improving" programmers , the only readers of this book are likely to be the mediocre programmers who fantasize about achieving "master" status and earn 250 k $/year (as Mc Breen reccomends) .

Note to Mr McBreen . People who are that good are already earning 250k or more!! so what was the problem again ?

Pete has written another half baked book on "Questioning Extreme Programming ".Far be it from me to say that XP needs no challenging , but this book does such a shoddy job, setting up and destroying a sequence of strawman arguments that wouldn't stand up in a high school debate,that you'd be surprised publishers would let this through their editorial board .

Anyway if you want some good ideas on how to become a master programmer , go read this article . It has more wisdom in it than Mc Breen's Book of Mumbo Jumbo.

So what is my take on this "apprentice,jouneyman,master" trichotomy ? Well I believe that in these days of universally available internet access and open source software any half way decent programmer is aware of where on the skill scale he (and any of his fellow programmers) falls , in any given context . Taking these "rating" labels seriously (for eg: on a site like Advogato,where you can see comments like " hey i became a journeyman while i wasn't looking !! wow ! " ) is, at best, like wearing your karate belt over your street clothes, to advertize your fighting skills .

As Bruce Lee said ,"On the street, no one asks you your belt before they hit you " .

Translated to the software development domain, that means

"shut up and write code" .

Mack on Management

My last post drew a very articulate comment from Chandrakant .

Chandrakant brought up the idea of being an XP Coach as being the right way to do management .

This is a very intriguing idea . But in my experience the people who were excellent xp coaches formed a distinct set with little overlap with the set of really good managers.

I think Chandrakant is on to a good idea here .It is just that I was unable to articulate where an XP Coach crosses the line and becmes a manager . So i did what i usually do when I'm stuck. Ask someone who knows .

Mack C Adams,who I had the pleasure of working with in TW and is a really good manager, comments on this whole "how to judge a manager" meme . (liberally paraphrased from a yahoo messenger conversation, with Mack's permission )

" Me: ahh just the person I was thinking of .. you got a couple of minutes to spare ?

Mack : shoot - just got back from an hour of military fitness training...so, if things go quiet, then I've passed out in my chair! Me: --- explains what's been happening -- asks mack to look at the comments etc ...

Mack: hmm...both comments (Chandra's comment and your reply) on your post are quite good

Mack: for me, I would definitely distinguish between XP Coach and Project Manager ("manager" is a very vague term). XP Coach is akin to an Iteration Manager - in my view, the best ones are technically competent process tweakers

they spend half their time coding, and half their time tracking though, I suppose a coach doesn't have to be the IM

me: and what does a manager really do ? I have a hard time pinning this down

Mack: Conversely, the whole "manager" thing is tricky - I think you should try to get away from "manager" as a role, and see it more as an activity along with facilitation; planning; communication; organization

For me, as someone who's not technical, I get scared when I'm too close to heavy work, as I'm flying blind - I need to depend on trusted technical people not to screw me over, and so I build an intrinsic desire in the team to succeed - I work on the team first - feeding it, protecting it, directing it, and letting it do its thing

Mack :Once I got myself intro trouble since I'm actually not very good at meticulous planning (Grinding out a project like a good tech pm should do [Ravi's comment :- Don't you believe this. Mack si the most organized person i have ever seen ] ). I'm trying to use the word "orchestration" to describe my skillset - broad base of knowledge, big picture, looking/anticipating for opportunities and enabling (sorry about that word) the team to succeed .I think great managers create high performance teams; good managers try to do the work.

Me : so i guess one way of judging good managers from bad is to look at the sucess or otherwise of their teams ?by success i mean not only project out comes but also things like how happy they are ??

Mack: : ultimately, that's the only baromter.

it should be holistic - a happy team that doesn't deliver is going to get fired; and a delivery team that isn't happy is going to quit "

ok so there you have it from a guy who actually walks the talk . The success criteria for a manager is the same as the success criteria for a team including the happiness of the team in their work.

Now when you are hanging in there, grimly holding on for your green card to come through ,"quitting" isn't really an option i guess, but that arises from an exported desi's chosen response to a bureaucratic rule .

Given that 99 % of IT projects are terrible for the client , the team , or both, I think it is self evident that there aren't that many good managers around .

Also I wonder how much sense this definition of management makes in the "outsourcing == terrible projects" scenario ? A manager is bound to fail if the happiness of the team is a factor . Hmmm . more questions to ask Mack ..

Mack went on to say

" man, I need to get a blog! really have no excuses, as I do have much to say...so lazy...London makes you weak with its vices! "

yeah Mack you should tone down the partying and start a blog !

How shall I judge a suit - Part 2

In a comment to another post on this blog, Sundaresh asked,

"Ravi......i give you a problem statement and a laptop and ask you to write a piece of code and looking at that, i gauge your worth. Thats an excellent/good/bad programmer. What is a good manager ?"

And that is a good question. I have not been able to articulate a satisfactory answer for this and that is scary .

There's no way to judge a middle manager and they are all over the place ! Anyway this was my reply (slightly modified to include a few post-reply thoughts ).

" Sundaresh, That is a very good question.

There is a lot more to judging programming skill than handing a laptop + a problem statement to someone .

But I digress .

The gist of your question of your question seems to be "is there an objective, externally verifiable way for judging a manager's ability ? "

Except in the case of frontline managers like CEO's and marketing/sales managers or operational managers of a factory floor, who can be judged by numbers attached to their performance (market share, eps, operational efficiency ) , the answer is no, there is no externally verifiable measure . In other words there is no "laptop" you can give someone and ask them to display their skills.

This is precisely why a lot of people can get away with what , in Thoughtworks India jargon was called "PM giri"(a word approximating "PM ing " for non hindi speaking readers of this blog) as in "PM giri is no big deal".

Having said that ,there are a few heuristics(by definition a heuristic is not infallible) that I look for when I judge managers.

1.Is he/she extremely good at any activity beyond "managing" ? Of the truly excellent managers I have met, all of them without fail were skilled in something besides "managing people " (as in the movie "Office Space" :-) )

One is a decorated Combat Helicopter (Apache LongBow) Pilot. Another is a world class writer and athlete. One is changing the model of how charitable organisations work .Another is an excellent musician . Somehow the habit of excellence and the consequent ability to recognize it in other fields seems to make a difference.

Some examples from the outside world. Larry Ellison is probably an excellent manger .He is a world class sailor.

Krishna Bharat (India head, Google) is probably an excellent manager . He wrote Google News.That gives you a lot of confidence in his understanding of programmers and programming.

Now this doesn't always apply. Linus Torvalds is good at coding but by his own account he tried "managing" a project and was a big failure.But then he "manages" the kernel development effort very well. hmmm...

This is why this is just a rule of thumb.

This pattern is surprisingly persistent though .Even managers who are not yet at the height of their powers but are climbing the ladder show this trait . One of the managers i most respect was in IIT grad(says something about his intellect ) ,and ex techie who was put in charge of the India Office of his company. He used to say with genuine regret " I am losing my technical skills" . Yet another was fanatic about physical fitness and superbly organized. His projects would run like clockwork and he would sit quietly in a corner filling up crosswords !

2.They are highly spoken of by the people they manage .

Invariably the best managers are those whose subordinates worship the ground they walk on . This is not about popularity but about respect .

If a programmer rolls his eyes and looks heavenward when speaking about his manager, that is a strong warning sign . The age of the "command" paradigm in management is over. Nowadays , to effectively manage, People have to willingly give you their best . Again using my Thoughtworks experience there were managers who were so effective that the team thought of them as "one of their own" and others about whom people would make snide comments when they were not around .I am sure this is true worldwide and not just in Thoughtworks.

3. In the software industry , most (but not all) of the best managers are ex hackers or people who like technology .The top grade managers who are not techincal genuinely respect the wisdom of the technical people

4. Good managers *always* protect their team from the politicking that goes on .If a manager being added to the team *decreases* the politics , that is a good sign.

5. Not a single good manager in the software industry I know likes ISO/CMM style processes.They may quietly live with the process and work around it when necessary but if someone likes and enjoys these heavyweight processes I would be very wary of hiring him .

I could think of a few more characteristics but you get the idea . There is no sure fire way of determining the worth of a non frontline manager but in any industry or company there are thngs to look for .

I am interested in your heuristics for judging managers .What do you think ? what are the characteristics of the best managers you know?"

Sundaresh replied

"That was an excellent analysis. In my experience, i have worked with 11 project managers, out of which all but one were perfect charlatans or incompetent. Few things i have noticed in the one person whom i would rate as "very good" were,

1. As you said his subordinates "worshipped the ground" he walked
2. If anything was slipping out of our hands we could always call him and he wud run to us in a jiffy
3. The place he hated the most was his cabin. He could be seen walking around, sitting with developers and in some cases even testing the application
4. He made the strict hierarchy look very fluid.
5. I worked with him for 3 months ( he came in to replace a PM when the project was in crisis and left after 3 months when the project was delivered successfully.)
6. We were never in the firing line. He was the shield
7. He even used to participate in the technical meetings and showed a keen interest there in.
8. One thing that is common to the worse managers i have seen is that they remain cloistered in their cabins and always try to impose the feeling that he is the boss.

Sundaresh identifies a "pattern" (sorry Dave, :-D ) I missed. I'll call i "Cabin fever" .One of the most impressive things about Thoughtworks was that when Roy( the CEO) visisted Bangalore , he would avoid all the "cabins" and just find the first empty chair and plug in his laptop and go to work .A very refreshing change from the Indian habit of alloting cabins, "corner cubes" etc to "senior " people.I didn't even know he was the CEO the first time he came in . He started questioning me about how things were going and what I as new recruit thought of the company ,and i wondered "Who is this crazy old man ? " .Later , when he climbed on chair to address the staff meeting I asked someone who he was and was told he was the CEO.

For all the "Indianization" of the Bangalore branch, if you haven't worked in Thoughtworks, you should ! If anyone reads this blog and applies to TWI , please tell the HR folks i'll be around to collect my recruitment bonus !

I once joined a company (which shall remain nameless) where the first day I reported to work , the receptionist and a few other people stood up , snapped to attention and said "Good Morning , Sir" in unison (like kids in kindergarten greeting their teacher! ). I was shell shocked and asked them what was happening and they said they were under instructios to treat me as a "senior" person . Later I was taken aside by the owner/ceo who informed me that we have to make the whole office aware of seniority for "increased discipline". I quit the same day (though i didn't have a job in hand ). Thankfully I had a friend in Aztec who forwarded my cv to their HR and i got a job there the very next day .-- wiping sweat from my forehead-- Thinking of this event still gives me goose bumps.

To finish, I will ask the question i asked Sundaresh

I am interested in your heuristics for judging managers .What do you think ? what are the characteristics of the best managers you know?

How to Hire Me

Of late, there have been a few inquiries about how to hire me coming in on my email.

After answering in almost exactly the same words five times , I thought it best to just write a blog entry till I get around to putting up a "professional" web page.

First let me tell you why I may NOT be the right person to hire .

I am NOT the right person to hire(heh! i bet no consultant tells you that up front ) if (each of the following is derived from one or more offers i turned down)

1. you are looking for massive numbers of people to "offshore" work to .I work alone or with a small number of people.I do NOT (repeat NOT) run a body shop! Don't ask me to "get me 50 people in 2 months"
2. You want an existing enterprise application maintained/supported
3. You want some one with deep knowledge of Microsoft Technologies.I know next to nothing about Microsoft Tech
4. You want experts in Chip Design/ Hardware Design/Electronics.
5. You want rock bottom rates because the work is done out of Bangalore(I met a few people who wanted me to work for them free of cost! When you are a consultant you meet a lot of crazy folks )
6. you are a body shop looking for one more "body" to export
7. you want me to be a Project Manager AND have ISO/CMM processes in place
8. you have a project in which all the creative work is done in the USA and folks in India "extend" it

I may be the right person to talk to if one or more of the following apply

1. You are trying to run a startup product company and are looking to form a small team of sharp people
2. you have very tough requirements and need people with deep knowledge of algorithms,concurrency, language processors or applied artificial intelligence
3. you want people to work on maintaining /extending gpl ed or other open source software
4. you need people comfortable with mathematics and programming
5. you are looking to build a web based non enterprise product or service .
6. You want some time limited mentoring in Agile Processes . (the "time limited" part is very important if you are an "enterpise shop" . I am not available to write Banking Software !)
7. You work in "non standard" languages like Python or Ruby (or these days, C ! ) or even more exotic languages like smalltalk or Lisp or Erlang or Forth.
8. you want to create top quality software.

I am fully loaded till about the middle of November 2005 and cannot take on any work till then.

If you are still interested in hiring me please write to me at magicindian AT gmail DOT com and we'll take it from there.

Those Crazy Guys at Google

From John Battelle's blog, an excerpt from his upcoming book about google

"Clearly, Google was metastatizing—everywhere there was opportunity, it seemed the company was expanding. Google soon had more than one hundred engineers in the company, but no focused approach to managing how their time was spent. Unsure of the best way to handle such growth, the triumvirate set up a traditional management structure based on hierarchy—teams of engineers reporting to more than a dozen engineering managers, who in turn reported to Brin and Page. But the approach began to feel top-heavy and bureaucratic—it was slowing down innovation. In September 2001, Brin and Page gathered all the engineering managers together at a companywide meeting—then informed them they were out of a job. Most got jobs in other places in the company, but the founders had made a declaration—not only were they in charge, but things would be done differently at Google."

They dismantled a hierarchy of managers because they felt they were gtting too bureaucratic?

And the owners of the company actually cared about the increasing bureaucracy and took drastic steps to curtail it ? they are beyond redemption!

Are they crazy ? They should learn from some companies I know, who have Regional Committees, Global committees, Operating committes,"chairs" for all sorts of stuff,layer on layer of redundant managers , and focus on everything but code and clients !

Tch Tch Google you might have the best engineers on the planet and make truckloads of money but you have lots to learn about management !

What can you expect of a company where the founders think coding is cool ? I tell you they are beyond all hope!