Since OS X is unix based, we get the benefit of all of the open sourced unix tools out there, but the downside is that it can be very difficult to find exactly what you're looking for if you're new to unix software.
I don't know if there's an all-in-one package sollution, but if you break down the steps, you're obviously going to need DVD ripping software (such as <a href="http://www.mactheripper.org/" target="_blank">MacTheRipper</a>), and then software to convert the ripped content to a PMP compatible format. Do you have <a href="http://homepage.mac.com/major4/" target="_blank">FFMPEGX</a> yet? It's pretty helpful in providing a variety of conversion presets, but you can do everything it does (that I'm aware of), and more using the CLI tool "mencoder" (which, coincidentally, FFMPEGX requires to run anyway).
So far I've never gotten the urge to convert DVD's for use with the PSP, so I haven't tried this out. I've used each of the tools above on different occassions, but for some odd reason, I always need them to do something that they're not quite capable of (still looking for a way to playback foreign DVD's on a US iBook G4 without having to worry about region restrictions), but my hunch is that they should fit the bill for your purposes. Even if they don't, it's a common enough problem that I'd bet you could find answers in a forum dealing with video conversion (such as the one for MacTheRipper).
If you come up with a solid sollution, you should post your results here so that they'll be readily available to the members of this board (!_!)