Shad_ow!: Don't revive old threads, please use the search feature.

gpSP is the best GBA emu for PSP, it is linked to on the menu down the left hand side of the front page.

You cannot run any homebrew unless you first downgrade to firmware 1.5 (you may then update to one of the custom firmwares if you wish).

The 3.50 downgrader can be found here.
Just follow the instructions in the readme file included in the download.