Ravi Mohan's Blog

Saturday, November 03, 2007

To "learn AI"

first learn basic mathematics.

I keep getting asked this question and I keep saying the same thing - to three people in the last week, for e.g, two of whom were working through (or planning to work through) AIMA - so I thought I'd put this down here (and point anyone who asks the same question to this entry in the future).

Learning AI (or any deep comp.sci for that matter) is not like learning J2EE or ruby "dsl"s or whatever the fad du jour in the enterprise software world is - read a few chapters of the latest bestselling "pragmatic" book, write some crappy web site and hey presto you are the expert.

"Real" comp sci doesn't quite work like that.

To understand a standard 3 layer feed forward neural network, for example, you need to have a solid grip on

  • vector spaces
  • basis vectors and change of bases
  • eigen vectors and eigen values
  • Basic matrix operations like inversion
  • multi dimensional performance surfaces
  • Quadratic functions and finding their global maxima within Newton's method and Conjugate gradients
  • performance measures and steepest descent
  • partial differentiation of vector values
  • numerical stability of algorithms

Without that background you will be able to use conceive of "class NeuralNetwork" that "has an" instance of "class Node" etc, but you will *not*,(repeat NOT) be able to do anything useful with real world data (writing a XOR classifier doesn't count!).

And, a feed forward neural network is only one type of pattern recognizer (or function approximator). There are many more, each with its own trade offs , and you have to know the math to make the trade offs.

That being said, the best book to start with is AIMA as long as you (1) learn the required math parallely (2) do all the exercises at the end of every chapter.(I can't emphasize this enough). Reading through != "working through".

If one is willing to work hard, there are very few fields as fascinating as the various branches of AI.

Once you start down the rabbit hole, however it may be very hard to continue writing all that heavy lifting enterprise software without hearing the giant sucking sound of your life going down the drain as you write yet another jsp page for the latest leasing system. ;-)

Be warned! :-D.

Anyway the really hard part of "learning AI" is not getting the books or working through them systematically (which is hard enough), but that is for another blog entry.

Post Script - Some book recommendations for beginners.

  1. Linear Algebra by Gilbert Strang
  2. Calculus by Gilbert Strang
  3. Artifcial Intelligence A Modern Approach By Russell and Norvig
  4. Machine Learning By Tom Mitchell
  5. Neural Networks for Pattern Recognition by Chris Bishop
  6. Paradigms of Artificial Intelligence Programming by Peter Norvig (ok, this one is not quite an cutting edge AI book, but imo this is the best book on programming ever written and if you call yourself a programmer you should have it on your book shelf)

6 comments:

Anonymous said...

Nice book list! I just placed an order for some of those books.

I am a physicist (and no AI expert) but I feel a basic knowledge of machine learning algorithms may help me to make sense of the reams of data generated by my experiments.

I've sent you an email with some questions (I hope that's all right) on applying ML algorithms to physics data.

Ravi said...

"I've sent you an email with some questions (I hope that's all right) "

That's **quite** all right, Joe!

I've answered your questions and the return email should be in your mailbox by now.

Anonymous said...

Ravi, I couldn't help but notice your strong contempt towards enterprise developers. I think if one looks close enough, there is enough excitement and challenges even in "enterprise" world. May be you have been seeing too many J2EE developers who just edit some XML files or write some "crappy" remote method implementations.
<<
Learning AI (or any deep comp.sci for that matter) is not like learning J2EE or ruby "dsl"s or whatever the fad du jour in the enterprise software world is
>>
This is exactly the thought/cliché that has created so many wasteful software developers - and that is why there are so many crappy applications.
I think only a few people give enough attention really *learning* how to program *good* enterprise software. Learning any subject or skill is equally serious compared to the other. (I am not talking about just *learning* J2EE. But learning J2EE or any X technology to implement the *best* solution.)
- not to deny the fact that it is essential to have "comp.sci" basics before even thinking of entering the world of programming.

Ravi said...

@ranjith,

You got it wrong. I am not contemptuous of enterprise programmers. I was one myself now too long ago and I know some wonderful people who write that kind of code.

However I **am** (these days, relatively) contemptuous of most enterprise *programming*,

This is subtly different. Or think of it as an opportunity cost thing.

"May be you have been seeing too many J2EE developers who just edit some XML files or write some "crappy" remote method implementations."

You forget. I used to work for Thoughtworks. TW is on the cutting edge of enterprise software and thoughtworkers do a lot more than "edit xml files". They work on some of the most complex enterprise software in the world.

So unless you yourself are writing some kind of uber enterprise software that is way better than what Thoughtworkers have seen (if so please let us know) , i suspect you are responding to your reactions to what I wrote rather than directly to what I wrote.
The point I make is more subtle. I've seen (and been part of) the cutting edge of enterprise software. And it isn't enough.

No need for you to feel defensive about my ravings. If you write enterprise software and derive intellectual satisfaction from it, then good for you. Just tell us what this cool enterprise software you work on is, the likes of which we peasants haven't come across yet :-)

"Learning any subject or skill is equally serious compared to the other. (I am not talking about just *learning* J2EE. But learning J2EE or any X technology to implement the *best* solution.)"

This is politically correct, but simply not true. There *are* skills that are tougher to learn than others. Being a karate sensei is much harder than being the local hopskotch champion.

And just imo J2EE is by and large a steaming pile of s***.

bodhi said...

Hi Ravi, Thanks for your blog :)
I like your book list. btw, any idea where can I get the Norvig book on AI (in bangalore)?

Ravi said...

"where can I get the Norvig book on AI (in bangalore)?"

every (technical) bookstore will have a copy.