I've installed Linux on my PS3, Gentoo to be exact. I haven't messed with it too much lately since I haven't had the time but I do have some usability issues with how it integrates.
I think Sony needs to fix how they allow switching between operating systems. Currently if you run Linux and want to boot into the Game OS you have to run a command which shuts down Linux and boots the GameOS. Well that's all fine till you want to reboot into Linux. In order to reboot into Linux you have to go though the XMB settings and find the correct option to reselect the Other OS as the default os otherwise it'll book back into the GameOS on the next reboot. The Default OS setting needs to be preserved for the PS3 to be more usable.. Linux could handle a simple boot loader menu to select between the two different OS's at startup to allow regular switching between the two.