If Linux Had MBA s

I don't like blog posts that just point to another page on the web and don't say anything original.

But just this once , I'll make an exception.

I've always liked the Mini Microsoft blog and the recent coverage by BusinessWeek (and Slashdot) has given it a lot of exposure. If you ever wondered what it was like to work for the Borg, read it first. Be sure to read the comments, which are mostly posted by people inside Microsoft. The comments are way more fun than the articles, which are themselves very good.

If you want a feel for what the blog is like, look here.

Snippets

"Oldskool MSFT managers wouldn't be afraid to say "no" to upper management or tell Bill or Steve to fuck themselves. Seriously. They better have a point and ammo to back themselves up, but they wouldn't be afraid to say it. The new managers don't like waves. You are punished if you criticize. That's why we end up on blogs instead of yelling "Bullshit" internally like we used to. Does Steve throw chairs? Heck yeah, so does Bill, most architects, etc. We used to have holes punched in walls around NT. But I would rather have that than the passionless yes-men corporate Dilberesque managers we have now. And there is no accountability at MSFT anymore because of this. No manager dares to point a finger because they are afraid of a finger being pointed at them."

More fun .. "For the people bitching about test, this tester says you can f'ing bite me. If you guys could ship code that worked, heck, code that even COMPILED PROPERLY without a buddy test and a test pass, Microsoft wouldn't be employing us. The growth of QA is specifically because developers cannot be relied on to write functional code."

Someone complained that "When I started, my first manager gave me a project by pointing me to the PM who essentially sold it to me as 'this is why we need it', and I started on the library and delivered a fully functioning project two months later... Fast forward to the present: I won't know what I'll be working on until a PM delivers a fifty page spec with pixel measurements and colors...Then I get to start on a document describing all technical aspects of the implementation in detail, together with a daily schedule of what I will do when. All of this will be debated in many more meetings. Then I start writing code."

A PM responded with

" ...An increasing number of PM roles are filled with top CS students from top schools that are looking to have a broader impact on the product, enjoy design, and can work with people (this was how they sold me the PM role). The result is more control being placed into their hands, often times turning the dev into a blackbox that inputs specs/PM direction and outputs coded features. ..."

to be floored by

"..What a load of bull. The PM positions have traditionally been filled with techies that couldn't handle code. They are a good dev's worst nightmare: someone who knows just enough about software to throw out the words, but a complete lack of depth to understand what they are saying. Combine that with the fact that their job title includes the word "manager" and the vaguest job description I've ever seen in my 20+ years of software development and you have a walking, talking, time-wasting disaster. Feature direction and design? Yeah, it's called today's whim.

"...the PM's I work with have backgrounds in CS, and would proudly call themsleves engineers.."

I'm sure they would...fortunately, no one would believe them. "

Hilarious!!! Of course there are more insightful posts too .

"..Regarding all of the recent comments about PMs, my opinion is that there are some great PMs that really help improve the product. The problem is that I think it is *MUCH* easier for a talentless PM to escape notice than a talentless dev. If a PM sucks, dev and test can ignore him and the feature still gets written. If a dev sucks, he can't hide the fact that the feature is late, always crashes, etc. .. "

More ...

"..Testing as a concept is fine. Testing as the accumulation of the political and inept employees is not. ..."

The part I liked best (and where the title of this post came from) was

So, when you think about it, MSFT used to be like the open source community. A bunch of engineers writing a ton of code. And, the engineers were motivated by the same thing. It wasn't about cash, it was about making something cooler than the competitors, just like the Linux guys want to make something cooler than Windows.

That was the root of "crushing the competitors" just like the Linux guys want to crush Windows.

So you open source guys should at least appreciate that. Imagine going from the environment you are in now to having a boss sitting there with charts and graphs, with people who don't code telling you that you need to go to idiotic meetings, etc.

We used to be like you except working on closed-source. In fact, I used to be big on FreeBSD and had conversed a few times with Linus when we were both in school and Linux was still a hacked Minix.

All things considered, engineers are engineers and want the same things whether they work on Linux or Windows. They want to write screaming code that is better than anyone elses.

But now, we've been Dilbertized. We've been internally infested by professional managers who don't know how to engineer.

If Linux had MBA's with no coding experience telling developers what to do, what do you think would happen?

Well, that's what's happening at MSFT. Put yourselves in our shoes for a minute, and I'm sure you'll be able to understand the disgust, frustration, and outrage. "

Bwaaaa ha ha ha ha ha!!! That's the funniest thing I've read in years ! And remeber all this is from just one post in the mini msft blog! Read ! Weep!

Manoj Evokes The Nethack Magic

My friend Manoj Govindan started blogging after some ..err.. encouragement from Yours Truly.

His first post is about how he got addicted to NetHack , a game Yours Truly pushed on him.Very nice post too, hopefully we'll see more posts on piano playing, wargaming and other hobbies of his.

Meditations on Mastery: Ethics and Transformation

I just finished reading George Leonard's book, Mastery . I've heard a lot of praise for this book and it is well justified. This is one of those rare books that seem very simple and obvious but then it grows on you and you find yourself spinning your own lessons out of it.

Most people who comment on the book emphasize that it teaches you to "love the plateau", in other words to keep on the path of practice even in the absence of any visible progress. So it does, but others have spoken on this so I won't. Instead , I will focus on something I learned from it.

In Chapter 11 (Tools For Mastery ), Leonard says "Acknowledge,but don't indulge your Dark Side" . He goes on to say ...

"...(We have a lot of)energy locked up in the submerged part of our personality, in what Carl Jung calls "the Shadow" ... (Poet and StoryTeller Robert Bly says),

'A young child can be visualized as a lively ball of Energy that radiates in all directions. But the child's parents don't like certain parts of this ball. In order to keep the parents's love, the child puts the parts of him they don't like in an invisible bag he draws around him...Then our teachers have their say 'Good Children don't get angry over such little things', So we take our anger and put it in the bag.'

By age twenty, he (Bly) says , only a thin slice of our original energy is left".

This is a metaphor that makes sense to me. The image of people who glimmer with pale flickers of uncertain light, while dragging along a huge invisible bag that thrums with "forbidden" energy but is firmly locked, is fascinating because throughout my life I have been fascinated with the notion of why(with exceptions) good people are so ineffective.

The people who get ahead and get things done are very worldly wise and know how to manipulate the systems around them and thus "play the game". But at the end of a lot of this game playing they find that their efforts have been meaningless and the rewards of what they gain are not as satisfying as they hoped. Thus politicans are very effective in the world but very very few of them are nice people, and few of them live meaningful lives or die content.

On the other hand are those who have a lot of idealism and caring but are stunningly ineffective in life. These folks have all the right ideas of what the world should be like but seem unable to move the world from here to there and are dismissed (rightly so) by the "players of the game" as insignificant or "idealistic". .

Thus on the one hand we have people who have an excellent sense of direction but no power, and on the other, people who have (or know how to get) power but are not clear on what to do with the power and end up either preserving the status quo, or worse changing it to a one of increased Tyranny and Evil.

The myths of our age show these archetypes very clearly. Thus in "The Lord Of the Rings", (I speak of the book version, the movies are a pale shadow ), the Dark Lord, Sauron, is impossible to defeat at the height of his power. Even with the loss of his Ring Of Power, he is frighteningly effective in drawing to himself massive armies (dwarfing anything the "good guys" can manage) and even his subordinates (The Nazgul) are almost unstoppable. What makes him a "dark" figure is that if he were successful, he would impose an Age of Tyranny and Terror on Middle Earth. In other words, if he were to be successful, all that terrifying force of will and ambition and cleverness would be applied in the wrong direction.

On the other side we have a motley band of "good guys", each of whom is no match for The Dark Lord individually,but collectively manage to defeat him. The Ring Bearer, Frodo has almost no heroic qualities except that his heart is in the right place. Frodo is the epitome of the "idealist", a guy who wants peace and calm and radiates "Good Will to all", but will essentially, if left alone, while his days away with his tobacco in a corner of the idyllic Shire.The other "Good Guys" all epitomize the perfection of one major "good" quality (Gandalf has Wisdom, Aragorn has the Leadership), all of whom have to work in concert to defeat the Dark Lord, who combines all "dark" qualities in one person.

Another Modern Myth , Star Wars , offers its own parallels.There are these "good guys" , the Jedi who shun the "Dark Side", and work for Peace and Goodness. And the Sith who are focussed on dominating the known world and stamping it in their image.What is often missed is that the Sith are tremendously more effective, with two Sith (there are always only two) matching and often defeating hundreds, if not thousands of Jedi. And the Hero is a kind of clueless Everyman, and really does not evolve very much through the story except to somehow mysteriously become a Jedi Master in a very short time.

Anyone with some common sense would know that becoming a Jedi Master would take decades of very hard work. This bending of the rules, where the "Chosen One" doesn't have to work very hard to achieve superhuman levels of mastery (Neo of the Matrix is another example) has its own dynamic in modern story telling but that is a subject for another post.

So here is the formula for creating a (modern) myth. First create a Bad(make him really really bad) Guy who has immense Power and is frighteningly effective. Bring him into conflict with an "Everyman Joe" kind of figure (so your readers/viewrs can empathise with him) who is nonetheless somehow the "Chosen One". Sorround this Good Guy with pseudo Masters who are all old and past their prime and speak in riddles. Throw in a "True Love" Aspect if you want to get the women to read the book/watch the movie. (These days it helps if the main female love interest is very feminist in her views and "just as good as the men". Thus the Leia as Jedi (bwaa ha ha ha) theme or Arwyn the Elven Princess facing off against the Nazgul (thankfully not in the book, only the movie).

Back to the main point. Leonard's analogy of the 'person with the bag(containing the Shadow Self)' is very apt and while his subsequent advice to harness the Dark Side(thus when you feel Anger rising, he advises you to not give into the Anger, but to harness its energy to do something useful) is easier said than done, it points the way to a different-from-expected but truer kind of Mastery.

Thus in my personal cosmology(feel free to whack away at this) a Master would not be some wimpy hero (like Frodo or Luke Skywalker), but a genuinely powerful being who could go toe to toe with the most powerful "bad guys" without any Deus Ex Machina popping in to conveniently save him just when the villain is about to make mincemeat of him. And one step to achieving this kind of power would be to consciously "empty the bag" and be Master of both Light and Shadow and be both worldly wise and idealistic.

Even in History(and not myth) the "good guys" are those who combine tremendous pragmatism with tremendous idealism to defeat the "bad guys" who are very very talented or powerful but pull in the wrong direction. (e.g. Roosevelt Vs Hitler, Abraham Lincoln vs the Confederates,Gandhi vs the British Colonizers).

Thus to go back to George Leonard's(or rather Robert Fly's) analogy, a true master will be neither a glowing child, nor a weary everyman figure with a huge bag of Darkness, who mysteriously becomes a master because he is "chosen", but an unencumbered figure with a consistent, calm sheen of power,who walks the line dividing Light and Dark,able to draw on either, and to transmute all the energies he harnesses or encounters into various combinations of light and shadow as appropriate to the environment, but chooses steadily move towards an idealistic end.

Thus , in my opinion, Leonard's advice would become,"Don't indulge your Dark Side. Acknowledge it.Then harness it".

Sounds tough? oh well, No one said mastery is easy !

Darth Ravi, Jedi Rajesh, And The Craftsmanship Manuscript

Dave Hoover blogs about getting feedback on his book. Rajesh and I have been reviewing Dave and Ade's forthcoming book.

At one point I said,

" ...Rajesh probably is more suited to chime in here than me. The standing joke amongst us is that Rajesh is the Jedi - basically a good guy focussed on people and I am the Sith - focussed on pragmatism and effectiveness ...We Sith have to be with two of us against an army of wimpy Jedi

And then, after seeing some of my "tear apart the material and point at every hole in the structure and logic " style reviews and Rajesh's more "softly, softly" approach, Dave started calling me "Darth Ravi" (and Rajesh, "Jedi Rajesh").

I like it! Now to get one of those shiny Vader masks to breathe through ...

The Meanings Of Mastery

Recently I was startled when I heard someone say "Linus Torvalds is not a "master programmer" because he doesn't 'mentor' people ".

My first reaction was "bwaaa ha ha ha ha ha !!! yeah right ". To me whether someone is master at an activity has nothing to do with any behaviour on his part but is a function of the degree of skill he has.

While commenting on my last blog entry someone else picked up on the same theme. The reasons given didn't stand up to scrutiny since they were based on an arbitrary religion (Hinduism) and what a particular view into that religious system claimed was "true" and "divine", but the phenomenon was interesting.

Two people making similair claims about something that was to me, "obviously" nonsense. My friend Manoj chimed in with a sharp mail that claimed "master " is used in contexts which have nothing to do with either skill or students (e.g. Post Master ; Thanks Manoj, that was good feedback)

Time for a closer look.

After looking up various meanings and usages of the word (a good idea Sriram), I find that the word expresses three distinct concepts, applicable to three different situations.

The first is "An expert in a topic" as in "He is a Master of 3 languages". In this case the "of" in "master of" is followed by a skill set, a non living thing. I will call this "variant 1" or "skill based" mastery. Here mastery arises from being able to do things in a certain "field of operations" that none else (or very few other people ) can .It is in this sense that Linus Torvalds is a master programmer.His mastery is over programming and only very indirectly over people .With or without people he is still a master of programming.

The second has to do with power relationships (however established). In other words "master" means " One that has control over another or others " as in "the dog ran towards its master". This is the sense in which "Post Master" (a person who controls other workers and processes ina a post office) or "master of a ship" (a person who dominates and has authority over other people on a ship) is used. I will call his "variant 2" or "people based" mastery.

In other words, this variant of "mastery" is present only in a context that includes dominating other people. This domination could be granted involuntarily by the dominated person (as in a master - slave relationship) or voluntarily (as in a master disciple relationship ). The fact remains that this mastery involves the exercise of power whether benevolently or not.

There is a variant of this meaning which comes from the Guild system in Medieval Europe. The twist this adds is that the master has to be "licensed" by some process, usually involving the approval of other similair "masters" to teach. Anyone who is not so licensed by some form of "guild" is not a master. I shall call this variant 2A or "guild mastery".

Thus, while Jesus Christ was a "Master" by the variant 2 meaning, (people accepted him as master and he had disciples) and anybody who became his disciple voluntarily granted him a dominance in certain areas of life, by the guild variation, Jesus would not be a master unless approved of by other "masters". Given that only John the Baptist "approved of" or "licensed" Jesus (and John himself does not seem to have any kind of "guild" approval) we will have to infer that a self proclaimed "guild of religious teachers" would have rejected his "mastery ".

And if you look carefully you will see that the positive versions of variant 2 would also depend on variant 1. In other words, Jesus was looked up to as a "Master" (by his disciples), not because some guild or bunch of people said he was a master because he could do things (clarify deep moral dilemmas , answer questions, work miracles) that no one else in his time and place could.

So now the question becomes, which of these models is appropriate to software development? To a certain extent this is a subjective question. However, that being said, There are people who want variant2 (a master and his disciples) or 2A (a "guild" which "sanctions" or "confers" mastery).

In today's world which is essentially egalitarian, networked, democratic and open, I stand with those who believe that mastery in programming (or anything else) is of the first variant, based essentially on the degree of skill. If you can develop programs in a demonstrably superior fashion, then you are a master. If you don't have a single disciple you are still a master. If you don't have a "guild" sanctioning you ,you remain a master. In other words, a Master first becomes a master and then may choose to teach in whatever manner he thinks appropriate (Sriram's idea. Thanks Sriram!). He is not forced to teach to claim masterhood or be "conferred" masterhood by onlookers on the sidelines. Thus Linus may or may not be choosing to teach by letting his code be open source. It doesn't matter. His mastery is self evident from the skill with which he writes code.

Martin Fowler is a master technical writer and speaker because he can speak better and write better than (most) other people can. Not because he is part of the Agile Alliance or Chief Scientist of Thoughtworks or any other "title" based attribute. These other memberships and positions flow naturally to someone who is a master by the "skill based" meaning of the word.

The main problem I have with the Guild meaning is that its essential purpose seems to be to satisfy some kind of psychological craving for certainty in an uncertain world, in an attempt to assuage an inner shakiness by formulating fixed levels and labels. You don't need a three fold, "peer sanctioned" guild like stratification to recognize someone better than you at programming and learn from him/her. Any programmer worth his salt does so all the time. And so I reject McBreen's book as half baked.

And anyway, whatever model you want to follow, invoking arbitrary religious principles or nostalgia for a romanticized view of the past to justify that choice is sloppy thinking. The solutions for the problems of this century are to be found here and now, not in the past.

So if you want to be a master programmer, focus on how to improve your code skills . Don't bother with whether some arbitrary group of people like those found on say Advogato "rate" you. If you must have these "strokes" from other people to validate your self-worth, be aware that the ratings as "Master" too will automatically happen when you have attained the appropriate levels of skill. And by then you won't care what other people say your "level" is.

As Eric Raymond says in his loginataka, (a parody of a guru-shishya dialogue),

"Rather must you Become, and Become, and Become, until Hackers respect thy Power, and other Wizards hail thee as a Brother or Sister in Wisdom, and you wake up and realize that the Mantle hath lain unknown upon thy Shoulders since you knew not when."

[Thank you Manoj, Rajesh and Sriram, for helping me clarify my thinking. Prakash too, though I absolutely reject your "guru/hinduism/indian culture/the Bible is divine" argument]

A Nostalgia for Guilds And Other Dangerous Ideas

After some thought provoking conversations over email with Dave , Ade and Rajesh recently, I read Pete Mc Breen's book on Software Craftsmanship .

The book's fundamental thesis is interesting. It says that the "software engineering" way of creating software is broken (I agree) and it posits a "craftsmanship" approach to software as the solution .(I disagree).

Let me clarify that statement.

I don't disagree with the idea that viewing software development as a craft rather than some kind of factory process will yield insights into how to progress along that craft.

The basic problem with the book is that it proposes what is essentially a medieval socioeconomic structure as a solution to a 21st century problem ,and selectively focusses on the virtues of the old "guild" system where practitioners of a craft were classified into 3 categories - apprentices, journeymen and masters .

Now let us examine how Mr McBreen proposes to solve the "software engineering problem" .

he says, (all emphases mine)

"Rather than attempt to industrialize and deskill software development, the software craftsmanship approach looks back into history to see how the problems of expensive labor and arcane skills were handled. " ( Chapter 5 )

While the book doesn't go into too much detail on which nation's history of what period is being "looked back at" , it is fairly obvious from the sorrounding context and talk of "Apprentice, Journeyman and Master" that he refers to the guild system that flourished in Medieval Europe.

So Mr Mc Breen, on the one hand says that a methodology originating in the 1960s and continously evolving since then (software engineering) doesn't work for software development in the 21st century so we should go back to the 14th , for solutions ! Interesting !

There are two ways to use labels like "apprentice", "journeyman " and "master".

One is that of a rough guide to where on a skill spectrum a practitioner of a craft falls.In any time period, in any craft or art, there will always be a fairly consistent ranking of people in terms of skill and it sometimes helps to mentally divide that spectrum into n categories and name those categories whatever you want.Yes, if n ==3 you can name the categories , "apprentice", "journeyman" and "master" .This is only a way of seeing the progress along a craft .Those labels have no real validity .

The second way is to use such terms as rigid unchanging classifications of people(however derived) and attaching separate privileges to these labels .

Thus you couldn't run a craft based business in medieval Europe unless you were a member of the appropriate guild and presumably of an appropriate "rank" to do so . Today anyone can start a business and thrive or crash based on the economic logic of the business. Which system would you prefer to live under ?

the fact that Pete Mc Breen supports the second ,"rigid label", way is evident from a paragraph that states

"A master craftsman may learn a new technology from an apprentice, but it does not mean that she is no longer a master. If anything, it reinforces her mastery because it illustrates a willingness to learn from everyone."

Hmm thus, irrepective of whether a so called "master" is demonstrably expert *in the technology being worked on * or not, he is always a master .And the poor fool who does, in this case, know enough to "teach the master" is still an apprentice . The labels are rigid .Once someone becomes a "master", even a situation where the practitioner fails totally does not remove "masterhood" .Nice!

You might think it would make sense to say that in this context where the new technology is relevant , the "master" and "apprentice" labels are reversed .Thus if an acknowledged superior programmer, say Linus Torvalds and I are pairing on creating a j2ee leasing system ,(I presume Linus will kill himself before he has to do this , this is all hypothetical, bear with me ), then, in the context of j2ee, logic would seem to indicate that *I* am the "master" (or "journeyman " or "Lord and Master" or "King Skeletor" .. use whatever labels you think cool ) and he is the "apprentice" (or "teeny weeny toddler" or whatever , as long as the label is "lesser" than mine ) .But no! Once a master always a master! it doesn't matter if you don't have a clue.

The more significant question of course is "does this kind of labelling make any sense" ? hmmmmm....

Other paragraphs in the book identify mastery with "peer recognition" . There is a very subtle fallacy here .In other words,as per the book , Bruce Lee was not a master martial artist because he could fight better than anyone, but because he is *seen* to be a good martial artist by his peers . This confuses cause and effect .

To see the fallacy embedded here, you can drop each condition in turn. Suppose Bruce Lee never became famous but retained as much of his fighting skill as he did . Is his "mastery" now non existent because he was not recognized ?

Now let us try it the other way . If Bruce Lee was much much more famous amongst "peer" (of course it is not defined how these "peers" would be selected. maybe they are people with the same label from the guild? ) martial artists but could never demonstrate his skill in combat (an analogous situation would be a large number of book authors having no really significant software to back up their supposed "expertise") would he still truly be a "master" ?

This labelling based on irrelevant attributes is the least part of the nonsense embedded in the book . More tragic is the notion of a whole set of practices to solve the "software crisis" built on this "labelling". Some of it is just common sense. Most of it is just misguided, intellectually unsound rubbish.

Thus two separate concepts of "apprentice, journey man and master" --one, a rough measure of skill , and two, artificial "labels" used to recreate an outmoded economic system-- are conflated. Now let us look into the guild system from which this terrible classification scheme is borrowed .

From the wikipedia entry for "Guilds" ,

" The guild was made up by experienced and confirmed experts in their field of handicraft. They were called master craftsmen. Before a new employee could rise to the level of mastery, he had to go through a schooling period during which he was first called an apprentice. After this period he could rise to the level of journeyman." This is almost a word for word description of the system McBreen wants us to replace present day practice with . Let us read further (the wikipedia article, not the book ) . " Apprentices would typically not learn more than the most basic techniques until they were trusted by their peers to keep the guild's or company's secrets. ".

Duh ? secrets ? "Journeymen were generally paid by the day and were thus day laborers. After being employed by a master for several years, and after producing a qualifying piece of work, the apprentice attained the rank of journeyman and was given a letter which entitled him to travel to other towns and countries to learn the art from other masters."

duh right . Does anyone else feel an increasing disconnect between the socio economic situation in the 14th century vs the present day ?

Continuing to quote "After long periods of apprenticeship to a master" ( an apprentice had to pay for this privilege !What do you think of having to pay a "master" or "journeyman" to learn programming? ) "and after producing a qualifying piece of work, the apprentice attained the rank of journeyman and was given a letter which entitled him to travel to other towns and countries to learn the art from other masters."

entitle him to travel ? a qualifying piece of work ? duh?

Now let us give Mr McBreen's ideas the best possible interpretation and decide that your "level" in software needs to be demonstrated, say by showing samples of your code , maybe your contributions to open source, before you get a job. Will this miraculously improve the success rates of projects ? Not at all, because the problem is that the need for programmers is well above what can be met by the "truly skilled" minority . If less than say ,250 people worldwide are truly "masters" and say another 3,000 or so of "journeymen" status in, say, Tennis (going by ratings and being generous) why should there be a greater number in software development ?

The difference is of course most people who play tennis do so because they at least minimally enjoy the game! To think that adopting some half baked scheme based on an ill defined outdated idea will raise these numbers(which is about the only way to "solve" the software engineering "crisis") is , at best , self deception and at worst .. hmmm ...let us not go there .

In other words the "software problem" is economic, not technical in nature . Unless a compelling economic argument can be demonstrated for this switch to an out moded way of "improving" programmers , the only readers of this book are likely to be the mediocre programmers who fantasize about achieving "master" status and earn 250 k $/year (as Mc Breen reccomends) .

Note to Mr McBreen . People who are that good are already earning 250k or more!! so what was the problem again ?

Pete has written another half baked book on "Questioning Extreme Programming ".Far be it from me to say that XP needs no challenging , but this book does such a shoddy job, setting up and destroying a sequence of strawman arguments that wouldn't stand up in a high school debate,that you'd be surprised publishers would let this through their editorial board .

Anyway if you want some good ideas on how to become a master programmer , go read this article . It has more wisdom in it than Mc Breen's Book of Mumbo Jumbo.

So what is my take on this "apprentice,jouneyman,master" trichotomy ? Well I believe that in these days of universally available internet access and open source software any half way decent programmer is aware of where on the skill scale he (and any of his fellow programmers) falls , in any given context . Taking these "rating" labels seriously (for eg: on a site like Advogato,where you can see comments like " hey i became a journeyman while i wasn't looking !! wow ! " ) is, at best, like wearing your karate belt over your street clothes, to advertize your fighting skills .

As Bruce Lee said ,"On the street, no one asks you your belt before they hit you " .

Translated to the software development domain, that means

"shut up and write code" .

Mack on Management

My last post drew a very articulate comment from Chandrakant .

Chandrakant brought up the idea of being an XP Coach as being the right way to do management .

This is a very intriguing idea . But in my experience the people who were excellent xp coaches formed a distinct set with little overlap with the set of really good managers.

I think Chandrakant is on to a good idea here .It is just that I was unable to articulate where an XP Coach crosses the line and becmes a manager . So i did what i usually do when I'm stuck. Ask someone who knows .

Mack C Adams,who I had the pleasure of working with in TW and is a really good manager, comments on this whole "how to judge a manager" meme . (liberally paraphrased from a yahoo messenger conversation, with Mack's permission )

" Me: ahh just the person I was thinking of .. you got a couple of minutes to spare ?

Mack : shoot - just got back from an hour of military fitness training...so, if things go quiet, then I've passed out in my chair! Me: --- explains what's been happening -- asks mack to look at the comments etc ...

Mack: hmm...both comments (Chandra's comment and your reply) on your post are quite good

Mack: for me, I would definitely distinguish between XP Coach and Project Manager ("manager" is a very vague term). XP Coach is akin to an Iteration Manager - in my view, the best ones are technically competent process tweakers

they spend half their time coding, and half their time tracking though, I suppose a coach doesn't have to be the IM

me: and what does a manager really do ? I have a hard time pinning this down

Mack: Conversely, the whole "manager" thing is tricky - I think you should try to get away from "manager" as a role, and see it more as an activity along with facilitation; planning; communication; organization

For me, as someone who's not technical, I get scared when I'm too close to heavy work, as I'm flying blind - I need to depend on trusted technical people not to screw me over, and so I build an intrinsic desire in the team to succeed - I work on the team first - feeding it, protecting it, directing it, and letting it do its thing

Mack :Once I got myself intro trouble since I'm actually not very good at meticulous planning (Grinding out a project like a good tech pm should do [Ravi's comment :- Don't you believe this. Mack si the most organized person i have ever seen ] ). I'm trying to use the word "orchestration" to describe my skillset - broad base of knowledge, big picture, looking/anticipating for opportunities and enabling (sorry about that word) the team to succeed .I think great managers create high performance teams; good managers try to do the work.

Me : so i guess one way of judging good managers from bad is to look at the sucess or otherwise of their teams ?by success i mean not only project out comes but also things like how happy they are ??

Mack: : ultimately, that's the only baromter.

it should be holistic - a happy team that doesn't deliver is going to get fired; and a delivery team that isn't happy is going to quit "

ok so there you have it from a guy who actually walks the talk . The success criteria for a manager is the same as the success criteria for a team including the happiness of the team in their work.

Now when you are hanging in there, grimly holding on for your green card to come through ,"quitting" isn't really an option i guess, but that arises from an exported desi's chosen response to a bureaucratic rule .

Given that 99 % of IT projects are terrible for the client , the team , or both, I think it is self evident that there aren't that many good managers around .

Also I wonder how much sense this definition of management makes in the "outsourcing == terrible projects" scenario ? A manager is bound to fail if the happiness of the team is a factor . Hmmm . more questions to ask Mack ..

Mack went on to say

" man, I need to get a blog! really have no excuses, as I do have much to say...so lazy...London makes you weak with its vices! "

yeah Mack you should tone down the partying and start a blog !

How shall I judge a suit - Part 2

In a comment to another post on this blog, Sundaresh asked,

"Ravi......i give you a problem statement and a laptop and ask you to write a piece of code and looking at that, i gauge your worth. Thats an excellent/good/bad programmer. What is a good manager ?"

And that is a good question. I have not been able to articulate a satisfactory answer for this and that is scary .

There's no way to judge a middle manager and they are all over the place ! Anyway this was my reply (slightly modified to include a few post-reply thoughts ).

" Sundaresh, That is a very good question.

There is a lot more to judging programming skill than handing a laptop + a problem statement to someone .

But I digress .

The gist of your question of your question seems to be "is there an objective, externally verifiable way for judging a manager's ability ? "

Except in the case of frontline managers like CEO's and marketing/sales managers or operational managers of a factory floor, who can be judged by numbers attached to their performance (market share, eps, operational efficiency ) , the answer is no, there is no externally verifiable measure . In other words there is no "laptop" you can give someone and ask them to display their skills.

This is precisely why a lot of people can get away with what , in Thoughtworks India jargon was called "PM giri"(a word approximating "PM ing " for non hindi speaking readers of this blog) as in "PM giri is no big deal".

Having said that ,there are a few heuristics(by definition a heuristic is not infallible) that I look for when I judge managers.

1.Is he/she extremely good at any activity beyond "managing" ? Of the truly excellent managers I have met, all of them without fail were skilled in something besides "managing people " (as in the movie "Office Space" :-) )

One is a decorated Combat Helicopter (Apache LongBow) Pilot. Another is a world class writer and athlete. One is changing the model of how charitable organisations work .Another is an excellent musician . Somehow the habit of excellence and the consequent ability to recognize it in other fields seems to make a difference.

Some examples from the outside world. Larry Ellison is probably an excellent manger .He is a world class sailor.

Krishna Bharat (India head, Google) is probably an excellent manager . He wrote Google News.That gives you a lot of confidence in his understanding of programmers and programming.

Now this doesn't always apply. Linus Torvalds is good at coding but by his own account he tried "managing" a project and was a big failure.But then he "manages" the kernel development effort very well. hmmm...

This is why this is just a rule of thumb.

This pattern is surprisingly persistent though .Even managers who are not yet at the height of their powers but are climbing the ladder show this trait . One of the managers i most respect was in IIT grad(says something about his intellect ) ,and ex techie who was put in charge of the India Office of his company. He used to say with genuine regret " I am losing my technical skills" . Yet another was fanatic about physical fitness and superbly organized. His projects would run like clockwork and he would sit quietly in a corner filling up crosswords !

2.They are highly spoken of by the people they manage .

Invariably the best managers are those whose subordinates worship the ground they walk on . This is not about popularity but about respect .

If a programmer rolls his eyes and looks heavenward when speaking about his manager, that is a strong warning sign . The age of the "command" paradigm in management is over. Nowadays , to effectively manage, People have to willingly give you their best . Again using my Thoughtworks experience there were managers who were so effective that the team thought of them as "one of their own" and others about whom people would make snide comments when they were not around .I am sure this is true worldwide and not just in Thoughtworks.

3. In the software industry , most (but not all) of the best managers are ex hackers or people who like technology .The top grade managers who are not techincal genuinely respect the wisdom of the technical people

4. Good managers *always* protect their team from the politicking that goes on .If a manager being added to the team *decreases* the politics , that is a good sign.

5. Not a single good manager in the software industry I know likes ISO/CMM style processes.They may quietly live with the process and work around it when necessary but if someone likes and enjoys these heavyweight processes I would be very wary of hiring him .

I could think of a few more characteristics but you get the idea . There is no sure fire way of determining the worth of a non frontline manager but in any industry or company there are thngs to look for .

I am interested in your heuristics for judging managers .What do you think ? what are the characteristics of the best managers you know?"

Sundaresh replied

"That was an excellent analysis. In my experience, i have worked with 11 project managers, out of which all but one were perfect charlatans or incompetent. Few things i have noticed in the one person whom i would rate as "very good" were,

1. As you said his subordinates "worshipped the ground" he walked
2. If anything was slipping out of our hands we could always call him and he wud run to us in a jiffy
3. The place he hated the most was his cabin. He could be seen walking around, sitting with developers and in some cases even testing the application
4. He made the strict hierarchy look very fluid.
5. I worked with him for 3 months ( he came in to replace a PM when the project was in crisis and left after 3 months when the project was delivered successfully.)
6. We were never in the firing line. He was the shield
7. He even used to participate in the technical meetings and showed a keen interest there in.
8. One thing that is common to the worse managers i have seen is that they remain cloistered in their cabins and always try to impose the feeling that he is the boss.

Sundaresh identifies a "pattern" (sorry Dave, :-D ) I missed. I'll call i "Cabin fever" .One of the most impressive things about Thoughtworks was that when Roy( the CEO) visisted Bangalore , he would avoid all the "cabins" and just find the first empty chair and plug in his laptop and go to work .A very refreshing change from the Indian habit of alloting cabins, "corner cubes" etc to "senior " people.I didn't even know he was the CEO the first time he came in . He started questioning me about how things were going and what I as new recruit thought of the company ,and i wondered "Who is this crazy old man ? " .Later , when he climbed on chair to address the staff meeting I asked someone who he was and was told he was the CEO.

For all the "Indianization" of the Bangalore branch, if you haven't worked in Thoughtworks, you should ! If anyone reads this blog and applies to TWI , please tell the HR folks i'll be around to collect my recruitment bonus !

I once joined a company (which shall remain nameless) where the first day I reported to work , the receptionist and a few other people stood up , snapped to attention and said "Good Morning , Sir" in unison (like kids in kindergarten greeting their teacher! ). I was shell shocked and asked them what was happening and they said they were under instructios to treat me as a "senior" person . Later I was taken aside by the owner/ceo who informed me that we have to make the whole office aware of seniority for "increased discipline". I quit the same day (though i didn't have a job in hand ). Thankfully I had a friend in Aztec who forwarded my cv to their HR and i got a job there the very next day .-- wiping sweat from my forehead-- Thinking of this event still gives me goose bumps.

To finish, I will ask the question i asked Sundaresh

I am interested in your heuristics for judging managers .What do you think ? what are the characteristics of the best managers you know?

How to Hire Me

Of late, there have been a few inquiries about how to hire me coming in on my email.

After answering in almost exactly the same words five times , I thought it best to just write a blog entry till I get around to putting up a "professional" web page.

First let me tell you why I may NOT be the right person to hire .

I am NOT the right person to hire(heh! i bet no consultant tells you that up front ) if (each of the following is derived from one or more offers i turned down)

1. you are looking for massive numbers of people to "offshore" work to .I work alone or with a small number of people.I do NOT (repeat NOT) run a body shop! Don't ask me to "get me 50 people in 2 months"
2. You want an existing enterprise application maintained/supported
3. You want some one with deep knowledge of Microsoft Technologies.I know next to nothing about Microsoft Tech
4. You want experts in Chip Design/ Hardware Design/Electronics.
5. You want rock bottom rates because the work is done out of Bangalore(I met a few people who wanted me to work for them free of cost! When you are a consultant you meet a lot of crazy folks )
6. you are a body shop looking for one more "body" to export
7. you want me to be a Project Manager AND have ISO/CMM processes in place
8. you have a project in which all the creative work is done in the USA and folks in India "extend" it

I may be the right person to talk to if one or more of the following apply

1. You are trying to run a startup product company and are looking to form a small team of sharp people
2. you have very tough requirements and need people with deep knowledge of algorithms,concurrency, language processors or applied artificial intelligence
3. you want people to work on maintaining /extending gpl ed or other open source software
4. you need people comfortable with mathematics and programming
5. you are looking to build a web based non enterprise product or service .
6. You want some time limited mentoring in Agile Processes . (the "time limited" part is very important if you are an "enterpise shop" . I am not available to write Banking Software !)
7. You work in "non standard" languages like Python or Ruby (or these days, C ! ) or even more exotic languages like smalltalk or Lisp or Erlang or Forth.
8. you want to create top quality software.

I am fully loaded till about the middle of November 2005 and cannot take on any work till then.

If you are still interested in hiring me please write to me at magicindian AT gmail DOT com and we'll take it from there.

Those Crazy Guys at Google

From John Battelle's blog, an excerpt from his upcoming book about google

"Clearly, Google was metastatizing—everywhere there was opportunity, it seemed the company was expanding. Google soon had more than one hundred engineers in the company, but no focused approach to managing how their time was spent. Unsure of the best way to handle such growth, the triumvirate set up a traditional management structure based on hierarchy—teams of engineers reporting to more than a dozen engineering managers, who in turn reported to Brin and Page. But the approach began to feel top-heavy and bureaucratic—it was slowing down innovation. In September 2001, Brin and Page gathered all the engineering managers together at a companywide meeting—then informed them they were out of a job. Most got jobs in other places in the company, but the founders had made a declaration—not only were they in charge, but things would be done differently at Google."

They dismantled a hierarchy of managers because they felt they were gtting too bureaucratic?

And the owners of the company actually cared about the increasing bureaucracy and took drastic steps to curtail it ? they are beyond redemption!

Are they crazy ? They should learn from some companies I know, who have Regional Committees, Global committees, Operating committes,"chairs" for all sorts of stuff,layer on layer of redundant managers , and focus on everything but code and clients !

Tch Tch Google you might have the best engineers on the planet and make truckloads of money but you have lots to learn about management !

What can you expect of a company where the founders think coding is cool ? I tell you they are beyond all hope!

An Apprenticeship Pattern

Dave Hoover maintains a list of what he calls Apprenticeship Patterns, which are essentially heuristics that help you navigate the transition from being an "apprentice" programmer to a "journeyman" (The terms "apprentice", "journeyman" and "master" come from the terminology of medieval guilds which used these terms to delineate increasing levels of competence at a craft) .

While I am not so sure if the "patterns" format really fits , Dave has a very impressive list of excellent advice . Please go and read them if you haven't (and buy the book when it comes out) .

Recently a friend asked me how to become "as good as you" ( bwaa ha ha haa ) in programming and I drew out a couple of "patterns " from what i told her .

Here is the first

Book Chain

To gain mastery in any given field the best way is often to read and work through a series of "classic" books in sequence .

Since the books we are speaking about are books about programming there will be plenty of code to write as well.

The problem is that there are often many many books about any particular subject and it is not easy for an "apprentice" to determine what to read and in what order . A person who has already walked the path is often able to guide an "apprentice" in constructing the book chain .Working through such a reccomended chain of books will save you years of flailing around.

Here are a couple of examples of book chains .

Topic : Compilers

First work through Language Processors in Java (by David Watt ) .

This book gives you a "vertical slice" through constructing an interpreter for a (fairly limited or "toy" )language .

Then work through Essentials of Programming Languages (Wand and Haynes) .This book gives you deep insights into how a langauge works "from the inside".

Follow this up with Van Roy and Haridi's "Concepts, Techniques and Models of Computer Programming " which dissects the various programming paradigms and shows how each paradigm is best suited to a particular set of problems and more importantly what the limitations are . Again Paradigms are best understood by getting an insight into how they work internally rather than as random rules like for example ,"use polymorphism in preference to inheritance" .

Next , read a book (and look at the corresponding source code ) that explains how a production quality compiler/interpreter works .

Three books that fit the bill are Lisp in Small Pieces, A Retargetable C Compiler - Design and Implementation and Compiling with continuations

Now you are all set to look into a real world compiler/interpretr/vm (say the Ruby Interpreter or the JVM ) .And once you get through that , you are a "master" (or very close to mastery as makes no difference) .

And now for a slightly less technical example

2. Extreme Programming

First read Kent Becks "Extreme Programming" to get an overall idea . Then read (and work through ) Test Driven Design by the same author. Learn to use a unit testing farmework like JUnit

Then read and work through Martin Fowler's "Refactoring " and Josh Kerievsky's "Refactoring to Patterns" , and optionally Ron Jeffries's "Adventures in C# " . Learn how to operate a Continous Integration Framework (like Cruise Control) and how it works, and then understand how Dependency Inversion works and learn to use a Mock Objects framework like JMock. .

Follow this up be learning how to break up required functionality into stories . To learn more about the XP process read Jim Highsmiths book on Agile Project management , and optionally the Scrum book .

And there you are .If you have learned (and more importantly applied ) all these concepts , you are an XP "master" .

The best way to get a "book chain" for a topic you intend to specialize in is to ask someone who is already expert in the field to give you a sequence of books to study.

A poor second option is to find the "best book to begin with " (for Artificial Intelligence , this is Norvig and Russell's "Artificial Intelligence, a Modern Approach", for example ) and pick and choose from the bibliography.

Light In Dark Corners

Updated to include an opinion on Google India

Joel Spolsky has written an excellent article where he explains why it makes no sense to hire mediocre programmers for Product Development.His thesis is that mediocre programmers just cannot hit the "high notes" that top grade programmers can (read the article) and since the cost of reproduction of software is essentially zero , software products live in a winner take all environment .

He also points out that this does not apply to non product software development . I quote ..

"Sadly, this doesn't really apply in non-product software development. Internal, in-house software is rarely important enough to justify hiring rock stars. Nobody hires Dolly Parton to sing at weddings. That's why the most satisfying careers, if you're a software developer, are at actual software companies, not doing IT for some bank. "

The programmer who is in the top 1% and prefers to create enterprise software vs product development (almost ) doesn't exist .

But beyond that there is one more factor that ensures that companies which create mostly enterprise software accumulate mediocre(and worse) programmers over time . And this is the "per hour" billing model, which immediately puts the goals of the consulting organization and the client organization at loggerheads .

The client organization gains by reducing the total number of developer hours (and the dollar per hour rate of the individual developer) . The consulting organization gains by increasing the number of developers and the dollar rate(which is mostly done by hiring people with greater "number of years of experience , there being no objective way to measure "goodness of value delivered" ) . Thus sheer economic logic dictates that client and consultant pull in opposite directions. Most of the dysfunctionality of enterprise software development arises from this fundamental anomaly .For example the relentless pressure most consulting oranizations face to add "resources" comes from this "increase the number of people on a project " (rather than "deliver more value " ) mentality. And this pressure eventually translates into terrible people getting entrenched in even the finest organizations . Then of course you need more "committees" to manage the mess that results, and new games come into existence , all resulting in the more talented people fleeing or giving up.

This downward pressure forces the client to outsource the work ("You say it takes 20 developers , 3 managers and 5 QA guys? ok but i want 15 developers,2 managers and the QA folks in India at India rates" ) .

Product development work (not maintenance!) is almost never outsourced.Product development companies do the initial development of projects wherever the really good programmers are (and pay them really well . The money involved would give most "enterprise" developers heart attacks) and then outsource the bug fixes, operations etc .(the spiels of "India/China/--insert cheap offshoring destination here-- has top grade talent and so we started an office in Bangalore/Shanghai/wherever" are trotted out to make sure the offshore developers feel happy and often have no basis but the decreased "dollar per hour" rate for mindless repetitive work) .

So the best programmers in developed countries have nothing to fear . The average devloper who can churn out endless jsp/xml/ejb etc needs to be terribly afraid. And rightly so .

The one possible point of satisfaction in enterprise software comes from really understanding a deep clientside problem and writing good code to fix this. Alas, even this is denied to offshore programmers who have to play many games (see Eric Berne's "Games People Play") and fight amongst themselves like dogs to get one of the coveted "onshore" assignments and get anywhere near the real clients(where they are still paid less than the local developers) .The only strategy(in the Game Theory sense) that has a positive payoff in the offshore developer's world is to join the ranks of management as soon as possible and leave "lowly" work (and most of the work is terrible) behind . Then you can enjoy a "money for nothing" existence (at the cost of having a "meetings, committees and paperwork" life and losing your technical skills, if any ).

There are some efforts going on to change the basis of payment from the insidious "dollar per hour" concept .But so far nothing has been very succesful because changing the game would mean that both clients and consultants would have to actually think of what they are doing and calculate the ROI of every feature they add(and God forbid that such heresies come to pass :-) ) .

Meanwhile developers , if they are hell bent on developing enterprise software, should probably join one of the better companies around and pray (very, very hard) you get a greenfield , challenging project.

But the question remains.If you want to work with the best people and have a satisfying life working with technology why flirt with Enterprise Software at all? Do you really (really really) get your kicks writing Banking Software ?

And even if you do, it may make sense to constantly re evaluate your commitment .Here is a simple way to do this .

When you get up in the morning, knowing you have to go to work , do you smile or grimace? And when you finish the average working day ,are you happy, having learnt something new and created good features ?Or has the daily grind just chewed up another 8 (more likely 12, if you are in Bangalore) hours of your life leaving you drained and exhausted ?

Update : Apparently Google does things differently (Why am I not surprised? ). I say "apparently" because I have no confirmation yet.I was told that Google's India Office is really about hiring only top notch talent and not outsourcing for cheap labour.Apparently anyone who makes it through their (very tough obviously) interview processes can select which office they want to work for .In other words at the end of your interview process you can choose to work out of,say California or Zurich . If true , this is absolutely incredible and needs wider publicity in Bangalore!

Anyway this blog entry was mostly about the yuckiness of most outsourced enterprise projects and not about product companies,although it remains true that most product development is done abroad . Google is about as far from "yucky" as you can get .

Help me Find a book please ( Kramik Pustak Malika by Prof. Vishnu Narayan Bhatkhande )

I looked for this book both online and offline,but couldn't find a copy.Basically the book is "a six volume work ..of major archival significance. It is basically a collection of notated compositions, grouped according to the Ragas in which they are composed. It contains 1849 compositions in 189 Ragas. To this day there is no compendium of traditional compositions which comes close to it for variety and accuracy. " according to this site .

As far as i can make out this book is quite easy to find in North India .Jaipur University even prescribes at as a textbook in their Music Degree courses . Alas, I am in the distant South(Bangalore) where the dominant musical system is the Carnatic system .If anyone can get a copy for me ,(or even have a copy they wish to sell), I will be glad to pay the cost and postage charges. Please contact me at magicindian AT yahoo DOT com and we can work out the arrangements.

Let us see if the blogosphere helps where traditional bookshops have fallen down .

So What Should I Write On My Blog?

Yesterday one of my friends called me with a simple question "What do i put on my (to be started) blog" ?

It is sometimes possible for one to be so used to technology that it is hard to conceive that a large number of people just dont' "get" it . As we shall see blogs mean vastly different things to different people and it is difficult to generalize across all these view points.A lot of people don't blog because they are confused by these myriad perspectives and hence are not sure what exactly to write on a blog . And that is a great loss .

Anyhow my friend was waiting for my answer on the other end of a long distance call so I had to organize my thoughts to reply and this is the result.Maybe it will help someone "get" blogging.

1. The simplest use of a blog is as a "web log" , in other words, a record (on the web) of things you do/encounter/experience.I believe this is how blogs were initially used .Entries like "I went out for dinner with X today and later popped in to see the X-Men movie. The movie was fun" or "Guitar class was tough . I just can't get the finger shift for CMajor-Bminor" belong to this category.

   This type of blog combines the record keeping benefits of an activity log with the publishing capability of the Internet. Thus the blogger's community of friends can keep track of what is happening in his life.

   An interesting variant of this is where people blog about the arrival of a baby , detailing information about doctor visits and ultra sound scans, all the way to photos of the various stages of the baby's growth("hey my baby looked at me today. wow! here's a photo".)

   Yet another variant is a project blog with entries detailing the progress (or otherwise) of a project and is often used in the open source community .

2. The second common way to use a blog is as commentary . The mode of posting used here is that of a critic or reviewer , passing judgement on things that interest him .Essentially each post is a review of a movie or book or piece of software, or a court judgement , or anything else that is of interest to the blogger .My friend ,for instance, asked if a chapter by chapter review of the Sanskrit to Malayalam translation of the Ramayana would make a good blog entry . The answer is "yes" .
3. The third way is to use a blog as a forum for thinking about ideas or distilling experience .This is essentially the above two forms of blogging plus the recording of extrapolatory thinking.In other words, various things that strike the blogger's interest are logged and then there is an additional facet of thinking about them .It is taken for granted that such thinking is incomplete .

   For example I learn about the Theory of Constraints and the three main metrics it uses (Throughput, Inventory, and Operational Expense) .I might write a blog entry just applying the idea to a software consulting company and trying to extrapolate a bit ("ok , so applying TOC to software, a programmer on the beach is Operating Expense. So are all the non billable managers, recruiting staff etc. Code written but not paid for is Inventory .Or is it ?If the customer doesn't pay can this code be re used or sold? If not then is that too Operational Expense?" etc.Note the theme of "thinking out loud".

   The blogs of most software developers have this quality.A commenting facility and a strong community of readers adds an extra dimension to such blogging if they help you to flush out inconsistencies in your thinking and/or add new ideas .

4. Another way to use a blog is as a forum for writing/thinking. The blogger sees himself as an author and a blog posting is treated as a literary form just like a magazine article or an essay .In this world view a blog entry is for very short pieces of writing which are not long enough to make it ento an essay or article. Martin Fowler's blog is an excellent example .

   The main difference from the "exploratory" mode above is that the final piece of writing is much more polished and definite for all its (comparitively) small size. Any thinking is done outside the blog entry itself.Comments may or may not be present (just as there are essays and articles without commenting in books and magazines). This type of blog is used by professional writers or thinkers (management consultants etc) where the blog is additional evidence of the writer's expertise and thus is more "professional" than the "exploratory" or "logging of personal experience" type blogs.

5. The fifth way of using a blog is to explicitly focus on building a community and sharing of thoughts within an interest group , something like a distributed news group . Thus you have "java blogs" or "home schooling blogs" . The additional concept here is that the size and quality of the community built up around a blog , with the number and quality of comments, trackbacks and pingbacks is a key metric of how successful a blog is .
   In this school of thought (which i don't subscribe to , by the way) a blog is primarily a community building tool, which enables you to reach out and connect to people who share your interests but are distributed across the world. A variant of this is to use a blog to evangelize a particular company or product.Taking this even further, there is the notion of competing with other bloggers for "ranking" in the blogosphere ( a fancy term for "all the world's blogs" ) .
   There are people who are professional bloggers and make huge amounts of money from their blogs , either through running ads on their blog sites, getting book contracts, or being sponsored by companies.
6. There is an associated concept of "corporate blogging" , that treats blogs as a new medium of communication between the various stakeholders of a company (employees, shareholders etc) . when you hear of someone being fired for "inappropriate blogging", or a company exposing a corporate blog for potential cutomers to know how cool it is, this is the facet at work. There is also a whole ecosystem of visionaries, consultants and snake oil salesmen(not all mutually exclusive) who try to sell blogging tools (often just jazzed up versions of freely available blogging tools priced fantastically to make them "enterprise" worthy ) to bewildered managers as the next silver bullet that will solve all corporate ills.

In practice all these modes intermingle and there are various ratios of all these approaches simultaneously present in most blogs.There is no "right way". Use whatever works for you .

I , for one, tend to see my blog as a place for self expression and improving my writing skills and am not too bothered about community building or "reaching out". My friend Rajesh is more concerned about communities and sharing of experience .
I guess it just shows I am more of an "anti social element" than he is :-).

But then this is not static either. I created this new blog explicitly because many people asked for commenting facilities .So maybe in a year or so I might focus on "community".Right now , for me , a blog is a combination of log of events(minor focus), an arena for exploring thoughts(major focus) and practicing writing(major focus) .

Blogging with Blogger

The last post turned out be quite popular and more than 20 people requested me to add a "comment facility " . The choice was between (1)run a mysql based blogging app on my website or (2) go with a hosted version .

I first looked at WordPress .I even installed and ran it .While it is a very impressive in terms of fucntionality , I really hate PHP and the thought of mucking around with PHP to make minor tweaks to the site is disgusting. I had to dig around in the template files('sidebar.php' for those who want to know) and change the feed indicator to remove an extra "feed:" string that gave a result of "not a valid protocol" when i tried to get the feed from bloglines. From what little I understand of this issue it seems that WP is on the "cutting edge" but the aggregators haven't caught up yet , or some such nonsense .
And of course once I tweak the template I am locked into it because the other templates still have this "feed:" rubbish hard coded in .

I next considered Typo . The biggest advantage Typo has is that its is written in Ruby, and built on top of Rails which add all kinds of goodness . If anyone is not convinced of the superiority of Ruby Vs Php I suggest they take a look at the source code of Typo and then the corresponding PHP code in WordPress. The only reason i didn't put my blog on Typo is that I have a sense it is evolving *very* fast and i would like it to reach a point of stability .
That leaves other yucky systems written in perl or hosted solutions where you don't have to bother too much about what is under the hood . Since I hate perl almost (but not quite) as much as i dislike PHP , I decided to look for a fairly minimal free hosted solution .Blogger meets all my needs . I am not sure it supports trackbacks and pingbacks but I am ambivalent about this whole "blogosphere / social text" meme anyway .All I need for now are permalinks and commenting (and i may yet turn the latter off ) and blogger has both .

The one thing I would like to have had are categories . But on second thought, I haven't subscribed to any category speciic urls . I may just spawn a new blog if i need one for a separate category .And knowing the quality of the people at google, I am not too concerned about the software crashing etc.

Anyway that's the story . I moved most (but not all) of my old blog entries here,tweaked the template a bit to make permalinks explicit and everything looks good .I still need to figure out how to put up an rss link . Bloger seems to use Atom .

But so far so good .

let us see how it goes .

Why Hackers Should Read History

It all started very innocently . A manager at one of the many software companies in Bangalore announces a scheme to "tax" Indians travelling abroad on the company's business .

The basic (at best naive, at worst foolish) idea was that Indians who go to the USA would "contribute" a percentage of their (dollar) earnings into a "common pool" and this "tax" would be distributed amongst all the poor unfortunate Indians who couldn't travel to the Land Of The Golden Mountain.

The net buzzed with mails about "American arrogance" (the Manager is American) and "whoever heard of a communist company? "(the CEO of the company is supposed to have socialist leanings).

On a parallel thread , a couple of naive Canadians who came to India for a short visit were interviewed by a Canadian reporter about their Indian experience and the result was a fairly funny (to me anyway) article in the Canadian Press about how their luggage almost got stolen and a few comments about the first-world-and-third-world-existing-side-by-side aspect of Indian life (at least in Bangalore) .Mixed in with all the naivety were a couple of nasty comments about how Canada (unlike India) was "a place where you were not measured by your class stature" (perhaps they meant "caste" instead of "class" ?)

In yet another encounter , a couple of scientists at the Indian Defence Department were very scathing about "barbaric" Americans and their "decadent" life styles and I found myself in the position of having to point out the many *good* things about America and Americans and the things that need improvement in India .

The funny thing was that,unlike me, the speakers themselves had never actually visited America .But they were certain that Indian "civilization" was way ahead of a rich but decaying America.

I generally don't blog too much about politics or other "sensitive" areas but i figure it is time to take a stand.

Here are the principles I follow when dealing with people .

1. The intrinsic worth of a person has nothing to do with race or nationality . It is all about how you think and what you can do and how well or badly you do it.
2. Some Americans are arrogant . This is a problem of belonging to the dominant culture at a given point of time . The Romans had this arrogance. So did the Persians , the Assyrians , the Mongols, and later ,the Spanish, the Venetians and the British . And if you go back to a time when the Indians had a flourishing civilization when Europeans lived like animals , I dare say some of those Indians were arrogant too.
   All this has nothing to do with the world as it is today . In the end if you are good at something other people need , you will be given due respect .
   And if you still don't like the way the world works you have the freedom to try to change it . The only guaranteed freedom is the freedom to try.
   The thing to do when you face arrogance based on the latitude and longitude of your birth (or skin color) directed at you is to ignore it (or smash it flat depending on how combative you feel) .
   Which brings me to
3. A large number of Indians are subservient to those in Authority , especially if that Authority Figure has a white skin . This comes of a combination of lack of self confidence , an excess of greed (to grab dollars, green cards , whatever ) and suffering from a simultaneous lack of guts and liquidity of the spine .
   When someone dreams up a totally foolish idea (a great defect in many people who end up as managers , not because they are any good at management,but because they are useless for anything else) the thing to do is to point out the absurdity in a rational manner ,irrespective of whether the proposer came from Madras, Manhattan or Mars.
   And pour scorn on it if you think it is appropriate to do so (though there are often more subtle ways ofgetting what you want).Often just a polite refusal to play along will do the trick . It also helps if you have the courage of your convictions and a feeling of self worth independent of what those "higher up the ladder" think of you.
4. All cultures have good points and defects . The "Superior Man" (as the I Ching would say ) recognizes and promotes the virtues of all cultures and opposes and suppresses their vices both within himself and his society .
5. All cultures have great men and despicable ones .
   Those who think Americans are "decadent" should learn about(confining ourselves to Politics) Abraham Lincoln (there is a lot more to his greatness than "freeing the slaves" which is about all most Indians know of this incredible man. Read Bruce Catton's three  part  history of the American Civil Warto understand Lincoln in his context ) , Roosevelt, and Jefferson.
   To see how fools constantly try to ruin the powerful nation these great men built ,read up on ( for eg, again confining ourselves to the political sphere),Clement Vallandigham,Gerald Nye,Edgar J Hoover and the inimitable Dubya.
   Dilbert's PHB lives on in history and politics just as much as in cubicle land .
   Westerners who pride themselves on their scientific prowess should meditate on the fact that even multiplication was hard before zero (and thus the decimal system) was invented in "primitive" India . And that the theory of Evolution is still anathema in major chunks of the "scientifically advanced" USA .
   Those who take pride in their dominant economic status should look to European Kosovo to see how easily civilization crumbles when Time's judgement rolls around. Some believed till the end , "That can't happen here . we are Europeans".
   If you look a bit deeper into the shakiness of the American Trade Deficit and potential Dollar crash ,you'll see how feeble such assumptions are and how fast the shoe could be on the other foot.
6. Indians who believe in America's "decadence" should try to actually get to know some of the History of India rather than prattle on about "five thousand years of civilization".
   The "Golden Age" preached by Hindu Fundamentalists is largely a myth(barely half a century old) created to counter feelings of inferiority vis a vis the West. Historically the "morality" and social structures of the periods in which India came close to being a dominating culture have significant similairities with the mores of the West . A good book to start with is The Gem In the Lotus,followed by The Emperors of the Peacock Throne .
   To figure out how to "defeat" the west , read Victor Hanson's(Hanson is a slightly over the top ,but nontheless brilliant ,writer),How The West Has Won (don't worry about the alternate title.It is the same book). Indian editions of most of these books are available , a situation in which the "cheap Indian" tag actually works in your favor!
7. To shift the context a bit, There are Indians ,some of them in the highest strata of society,who judge other Indians on the basis of caste, especially when it comes to things like marriage . There are Americans ,some of them in the highest strata of society, who judge other Americans on the basis of race , especially when it comes to things like marriage . Caste Based Discrimination is illegal in India except for government sanctioned Reservations . Race Based Discrimination is illegal in America except for government sanctioned Affirmative Action . All of which shows that both societies are not perfect and contain substantial chunks of deluded people and a comparitively fewer number of intelligent ones .
8. and finally understand that one of the great powers that moves the world is Stupidity . "Never ascribe to malice that which can be adequately explained by stupidity." .So said Napoleon , a European who shook his continent and a good chunk of the world, a man who while symbolising the quintessential westerner , was secretly most comfortable with the mores of the Orient .

Stupidity And Brilliance have nothing to do with culture. And yes , the "company enforced travel tax" is a tremendously stupid idea which tries to fix a deep and complex problem with spit and duct tape. But it could, in theory, be used a creative provocation to evolve a better one . Pretty unlikely . But who knows? If everthing fails , this will provide some consolation to the oppressed masses . Arise ! awake ! You have nothing to lose but your money and self respect!

Goodbye to Java

Dear Java,

These days I program mostly in Lisp and Ruby (with a smidgen of Erlang and a dollop of C ). Coming back to you from such incredibly powerful languages is painful at the best of times , but after an attempt to invoke your new generics feature to do something conceptually simple resulted in almost unreadable code , I decided to throw in the towel . (I won't go into the messy details here . A google search for 'java generics suck ' should turn up enough arguments (and counter arguments ) . Tim Bray , Bruce Eckel and others do a good job of debunking the "new java" . ) .I'll still be in touch with you to complete the AIMA code (the forthcoming release is in Java 5 btw ) but I'll be damned if I ever use you for any other project .

I stayed well away from your newest competitior C# so that's quite all right too .

And so dear Java , it's been nice knowing you . While you were never a stunning beauty like Lisp or smalltalk or C, your massive libraries endowed you with a certain charm for a while . But alas today you are fat and shapeless and you desperately need a deodorant .

Not to worry though . The millions of lines of atrocious "enterprise" code that belong to you will ensure that you won't lack for suitors for the next 10 years . And the fast depleting billions in Father Sun's coffers will prop up your bulk for a while .So be of good cheer. Even the undead Cobol manages to drag herself through the years .So you have nothing to fear.You'll probably outlast me.

Meanwhile this is one programmer who is now rediscovering the sheer joy of programming writing some meaningful code that actually exercises some grey cells .

Hasta La Vista, Baby and Take Care .

Regds,

Ravi

Levelling up in Math land

(Edited on Feb 4 2011), This entry is correct but is now obsolete.I rewrote this with more detail here. Please use that instead.

Concrete Mathematics is a fine book. But in the very first there is a bit of handwaving that goes on to eventually get seemingly magical and the novice who is using the book for self study (iow, someone like me ) can get lost very easily .

A good example is the "solving a rcurrence by the repertoire method " explanation on pages 12 - 15 ). A margin note by a student (an excellent idea to include these) warns us "Beware . authors expect us to figure out the idea of the repertoire method by seat-of-the-pant examples instead of giving us a top down presentation " .

Unfortunately that doesn't help a reader to figure out what is going on and how to use the "repertoire method " to solve a recurrence . I've been banging my head against this for a few hours and i think i found out how the repertoire method works . here it is .

I will use the same recurrence the authors use .

Recurrence R =

f(1) = alpha
f(2n) = 2 * f(n) + beta for all n greater than or equal to 1
f(2n + 1 ) = 2 * f(n) + gamma for all n greater than or equal to 1
to solve a recurrence using the repertoire method ,

step 1
make sure that the recurence is expressible as a sum of parameters multplied by functions of n

the above recurrence can be transforemd to f(n) = A(n)*alpha + B(n)*beta +C(n)* gamma (this is done on page 13 of the book so i don't repeat it here)

where A, B and C are functions on n and alpha beta and gamma are constants

step 2
p times , where p is the number of parameters (in the above case, alpha, beta and gamma so p = 3 ) *assume* that f(n) = a simple function of n and solve using the original recurrence to get an equation . eg: in the above recurrence there are three parameters , alpha , beta and gamma

so say
f(n) = 2^m, --> Eq(1) (EDITED on Feb 4, 2011 - This is actually proved by induction in the book so this guess for the value of f(n) , but this guess does work in providing the value of A(n) better than proving it by induction)
f(n) = 1 -->Eq(2) and
f(n) = n -->Eq(3) be your assumed functions .

solving R with Eq(1) gives A(n)= 2^m --> Eq(4)
Solving R with Eq(2) gives A(n) - B(n) -C(n) = 1 -->Eq(5)
and solving R with Eq(3) gives A(n) + C(n) = n --> Eq(6)

step 3
solving these 3 equations (eqs 4,5 and 6) gives us the values of the coefficients .

A(n) = 2^m
B(n) = 2^m -1 - l (the last is the letter "ell" )
and C(n) = l (the letter ell)
thus the closed form of the recurrence is

A(n) alpha + B(n) beta + C(n) gamma = 2^m * alpha + (2^m - 1 -l) * beta + l*gamma

now the question is how can we be sure that we guess the "right" functions in step 2 ? how do we know that f(n) = 1 is a "good " guess ?

the answer is that if you guess "wrong" you won't be able to solve for that equation and will end up with an impossible equality .

and that is how the repertoire method works!

happy repertoiring

None So Blind... The Agile India Conference Experience Report Part 2

Thanks to problems setting up the laptop I missed Brett's presentation. I listened to Mridul's talk on "Kernel Programming + Agile" and then it was my turn . We grabbed some lunch and then went to see if we could attend some seminars . Unfortunately all of the presentations in the afternoon were boring as hell . Most were about "combining waterfall and agile" Apparently there was a talk earlier in the morning about a "wagile" methodology . If you, the reader, just fell off your chair or threw up , you can imagine my reaction.

I think God is having His revenge on me for generally being disrespectful and specifically for using a Dilbert Cartoon as a presentation aide. when i looked at all those PHB types drone on and on about "waterfall and agile are complementary " , i felt i was inside a Dilbert strip . I think a lot of clueless managers in Bangalore have latched on to the "latest buzzword" and are busy "optimizing" (bwaaa ha ha !! this is a private joke . i actually met a (totally dilbertian) manager who said "my main skill is optimizing" if you mention this to Abey or me, we'll fall around in hysterics ) and "combining" it with whatever outmoded nonsense they know about "managing software teams" .

It is very frustrating to see how much traction this stupid idea is getting in Bangalore amongst people who don't want to change how they or their companies function but want to be "agile" so they can fool their clients into giving them more money.

I have said so before and i will say it again .

The primary facilitating factor for "going agile" is a culture that respects no hierarchy but that of competence and maintains an atmosphere of total openess and trust.

now how many companies in Bangalore work like this ? 3 ? 5 ? you get the idea .

In the evening the organizers asked me to be on a panel on "distributed agile " . There was this person on the panel (who shall remain nameless) who claimed that his "whole life is agile" (!!! ) and then went on to proclaim a lot of "waterfall " ideas (eg:" 'novices ' should have their yahoo messengers monitored by a 'senior' developer to make sure they don't say anything "dangerous" to a client" ).I have never heard such nonsense in my life . I really lost my cool and did what i could to rip his arguments apart.

The audience loved it :-D .

An artist sketching the panel drew me in a furious, scowling figure. If i can get a copy of the drawing from Manoj Bharadwaj i will certainly put it up here .

Because one of my friends was leaving Bangalore , i missed the second day of the conference .

too bad, because i was looking forward to hear Fred George, Dakshinamurthy Karra ,Simon Harris(author of Simian), Henry Jacob and Luke Barret speak .The last two seem to be focussed on Interaction Design in an agile fashion. Alas it was not to be .

All in all it is wonderful that such conferences are happening in India . If the Agile India Society can just keep going we may see many more such events .On the other hand it may just wither and die.

we'll see .

Slide, Slide on the wall... The Agile India Conference Experience Report Part 1

I attended (and spoke at) the Agile India Conference. My speech about applying XP to a large AI classifier system I am working on was well received . I used slides only to represent visualisation of complex data (and one for a dilbert cartoon).

I think most people use slides because

1. they need to structure their thoughts as they create their speech,
2. they can "lean on" the slides,use them as a psychological crutch while on stage,to take away some of the stress associated with speaking
3. that is the only way they know how to make a presentation. That is how they learned to do it

Though i was a fairly good speaker and debater in college ,soon after i got into the software "industry" (note the quotes) i fell into the habit of creating and using slides for presentations. Recently i read this , this and this .

I also noticed that the very best speakers , even in the software field, (example Martin Fowler ) do not use slides .

ok so here was a chance to face an audience without any shields again after many years . The surprising thing was how powerful the hold of PowerPoint was. Though,years ago, i had spoken to large audiences before, without slides,or even notes, now it felt very strange to do so without PowerPoint guarding my back !! this was scary !

anyway , an intial awkwardness aside, the speech went off fine . The best compliment was given by a student volunteer (the conference was held at the PESIT Engineering College) who declared that he was able to follow the talk easily, though it was about a highly technical topic.

Vivek Singh , whom i respect tremendously, even while disagreeing with him about most things , was there and he didn't throw his shoes at me . .

So i guess it all turned out fine .Thanks to the talk, I met a lot of interesting people , including a person who writes browsers for mobile phones .We had a very interesting conversation.

And i will never use PowerPoint again . What i need is something that would allow me to outline the structure of a speech as i am speaking, annotate it with audience questions(as recorded audio ? ), connecting links etc .A cross between an outliner,a multi media blogging tool and a brain map .. hmmm time to fire up squeak.

More about the conference in the next entry .

Mathematics An Epiphany

Today morning , I was at the Centre of Artificial Intelligence Research staring rather stupidly at a page covered with strange squiggles and cryptic sigils . I had gone there to "gather requirements" and the person in charge dropped a couple of papers from the IEEE transactions on Machine Intelligence on the table and proceeded to talk in a language that brought back faint memories of drowsy classrooms from oh so long ago .

Sample sentence = "To solve the problem defined in expression (1), a single unconstrained problem equivalent to expression(1) is obtained by defining the augmented Lagrangian function denoted by phi ".

doh...

After ten minutes or so of this, i put up my hand and said "what does that mean in English? i didn't understand anything". My friend Mridul caught on and both of us asked increasingly dumb questions till the scientist gave up the struggle and patiently explained the math to us term by term translating into English and into a 2 dimensional diagram . I was very skeptical that i would ever understand anything but ever so slowly the squiggles resolved into a stunning set of ideas laying out a new way to do machine learning. After few hours, we were nodding along, even chipping in at various points with ideas that made sense . I had gone from skepticism to bemusement to fascination to awe.

It is hard to state this bluntly but i came out of that meeting convinced that a programmer who doesn't know math is somehow crippled(so is a mathematician who can't program, but that is another story). The power of those equations ,drawing down a whole set of tremendously complex abstractions into clean lines of force that thrummed with power , and seized even our bumbling thoughts and flung them unceremoniously into a world of bewildering possiblities, seemed to have affected Mridul too.

The first thing he said after the meeting was "Damn ! we should learn math.If we could wield it the way these scientists do, we could create so many fantastic programs ".

We had a long discussion over lunch on what we had perceived and distilled it down to these basic points.

1. the combination of deep mathematical skill and expert level programming ability would be the ultimate "level up"
2. the first step in mastering math is be to learn to read the notation. just like learning the syntax of a new programming language
3. the second is to grasp the reality expressed by the notation at a gut level , like understanding the paradigm and patterns lying underneath a programming language , like ,say ,beginning to grok "oo"
4. the third is to use that understanding to create new possibilities
5. the fourth is to use a programming language to embody and refine those possibilities , thus creating programs that do what has never been done before.

Strange .In one day, I have gone from hating mathematics to bedazzled wonder and a craving to learn .

Life is so wonderful .

RunAwayScreaming

I came across Run Away Screaming

so.... having nothing better to do .. i wrote

When they sell you as cheap Indians
Then tell you "you are   cool"
And Power is for Caucasians
And the PM is a bigger fool
You want to RunAwayScreaming....

When they tell you "we do XP"
And tell you "bill more hours"
And ship code that is buggy
To folks in Ivory Towers
You want to RunAwayScreaming....

When you ask for a laptop
And they form a Committee
And talk and argue till they drop
About if you are worthy
you want to RunAwayScreaming....

when for every new situation
someone makes complex laws
and a PHB fascination
becomes the latest  Cause
you want to RunAwayScreaming....

When body count is the Holy Grail
And "suck up" the success path
More Process leads to  "How to scale"
and Logic invokes wrath
you want to RunAwayScreaming....

yes it sucks!! i know .

but this is my blog and i can write what I want ! so there!