I was going to give up after having all of the same problems everyone mentions above (i.e., analog stick pulling one direction after I replaced a broken stick.). After almost giving up and tossing it, there were two things that helped me out: rotating the stick when booting, and manipulating the pressure on the psp when screwing it in.
For the rotation, I found a post somewhere else where someone said specifically to rotate the stick once counter-clockwise, then once clockwise when it was booting, rather than just moving it any direction. This seemed to reset the stick, assuming I had the right amount of pressure on the faceplate.
The issue I had which joekid mentions is that no matter what I did, I couldn't screw the face in hard enough to keep the pressure I needed on the psp (I had to hold it really tight). So what I did, which may sound stupid, was to put some additional spacers (kind of like a washer) along the screws on the analog stick so that the screws weren't flush against the faceplate.
This created a thicker faceplate, which creates more pressure against the unit. A bit ghetto, but it seems to work. What I used to create the washer were foil candy wrappers (you could use tin foil probably), but in a very tight package that was big enough to cover the area around the screw, but not big enough to interfere with the other buttons/motherboard. I created a hole in the foil, and put the screw through that.
Was working great for a few weeks, but then the cheap stick I bought off ebay busted on me, so now I have to get a new stick, but should be able to slap in and go I hope...
