Ravi Mohan's Blog
Tuesday, September 18, 2007
The Lame and the Blind - Proofs And Code in research software
There is this old story about a blind man teaming up with a legless man. The blind man carries the legless man on his shoulders and provides the locomotion while the sighted cripple tells him where to head. Similar divisions of labor exist in enterprise software (think "developer" and "analyst") but this two fold division of labour is exacerbated when one focuses on research software. Here the scientists are the "sighted folks" who know where to go but can't move and the developers are (ideally, as we'll see later) the folks who know how to move the project forward by writing code, but don't have the requisite mathematical/scientific skills to navigate and choose one direction over another. Earlier today, I was talking to a scientist trying to move a multi million dollar machine learning project forward, who was feeling hamstrung because it was impossible to find a few people who could follow advanced calculus and also write good C code. Unlike in the folktale, a real life arrangement consisting of a blind man and a cripple would be very inefficient, with the occasional fall, not to mention the constant stubbing of toes and odd outburst of swearing. But equally, nothing in real life mandates this kind of "pairing". Nothing prevents a scientist from learning how to code (except a feeling that coding is a relatively "blue collar" activity) and nothing prevents a developer from learning the requisite maths and science (except that most developers, especially the enterprise/agile/dsl types either don't have the caliber to learn hard math or think erroneously that these skills are beyond them or just don't want to work that hard). The few folks who do master both proofs and code, can and do make a tremendous difference. The very best people in every research area that incorporates software and maths are very capable scientists/mathematicians and very capable programmers. Peter Norvig is an example. As is Donald Knuth. Or Manolis Kellis. Or Yann leCun. If you are running a research software project you'd want to staff it with high level multi-class (think dungeons and dragons) scientist-mathematician-developers. Unlike in D and D specializing in one area does not need to involve trading off the other. After all the lame man/blind man arrangement is not the best possible one. The best (and normal) situation is having two (or n) people who can both walk and see.