You need either Sony's firmware 1.5 or below, one of Dark_Alex's custom firmwares, or the eLoader and a compatible Sony firmware (meaning 2.8 or below) to use this, or any other homebrew that I'm familiar with. If you're unsure of exactly what firmware you're running, look in Settings -> System Settings -> System Information -> System Software on your PSP. If it says "OE" after the firmware version number, you're good. If not, you're probably going to have some trouble.
If you're running 3.30 OE (or 1.5, or any OE firmware), you need to create the folder ms0:/PSP/GAME150, and then copy the FRONTIER and FRONTIER% folders to that folder. I would guess that the other two rars need to be unpacked into the FRONTIER folder, but I haven't tried this particular program out yet, so I'm not 100% sure. That's the way it usually works though.
If you're running 3.30 (not OE), then this program will not run, and I don't think you'll be able to run any homebrew at all on your PSP at this point in time. To my knowledge there are no downgraders for 3.30 yet either, so for now you're stuck. Don't upgrade to any new official firmwares, and eventually some enterprising coder will come out with a downgrader that you can use.
If by some chance you were completely mistaken about your firmware version and you have the official Sony firmware 2.00-3.03, you have the option of either using eLoader or downgrading to firmware 1.5. Exactly what you need to do for either of those options depends on what version firmware you're starting from. Bear in mind, however: eLoader sometimes has compatibility issues that will prevent some homebrew from running, and with downgrading (and updating) firmware, there's always a risk of bricking your PSP. Do your research, make sure you're reading all of the version numbers and folder names right, follow the steps carefully, and you should be fine.
Here are some links that'll help:
Noobz
Dark_Alex
TA-082-Motherboard-Tutorial
DCEmu official guide - how to install 3.02 OE-A, then to OE-B, to run PSOne Games
And two last precautions. First, you cannot update from firmware 2.x/3.x directly to an Open Edition or Special Edition firmware. You MUST downgrade to Sony's 1.5 firmware first and THEN upgrade, or you will brick your PSP. Second, if you have a PSP with a TA-082 or higher, do not downgrade below the official firmware 2.5. If you have one of these motherboards, your only option at this time is eLoader or possibly HEN (which I've never used, so I can't really elaborate on it).
Good luck.