I first read Surely You're Joking, Mr Feynman in my mid 20's. That got me interested in science in a rather general way. Six Easy Pieces followed and listening to those lectures and reading the book got me completely hooked on physics.
My mathematics was in a terrible state - but I wanted to read the more difficult lectures. So I bought my copy of The Feynman Lectures on Physics with a view to reading them as I learned the mathematics. First I had to teach myself basic algebra, calculus, vectors and complex numbers... after about ten years of reading and learning and horsing around with physics as a fun passtime, I entered university and enrolled in a double major in mathematics and physics.
I have found that my background - which mostly consisted of Feynman's teachings - was strong enough to get me through nearly all of my undergraduate course without too much extra effort. There is absolutely no better preparation for a physics/maths degree than learning from the Feynman Lectures.
I'm, now 37 years old and in my final year of undergraduate physics and maths and am preparing to do honours. There are two people I feel are to blame for this: the kind professor who encouraged me to enrol at university and of course, Richard Feynman, who always opens your eyes, fuels your imagination, he lays out the vast ocean of discovery before you and makes it all seem so accessible.