The first time I went into it was to write a driver for a device in my undergrad. After that I've changed a driver here or there (never anything worth merging), and needed the documentation of the sound systems.

It's not an easy thing, by any means. Just locating where you have to touch on the source tree is a problem that will lead you to plenty of books or sites. But don't try reading those before you have a problem to solve, you will lose time and drown in information.

(By the way, I am assuming you know how syscalls work. If you don't, go study that before you start anything.)