How does a Homebrew Enabler work?

via http://wololo.net/wagic/2012/05/28/h...-enabler-work/

I believe everybody who visits this blog at least knows what a homebrew is, and knows that the only way to launch certain homebrews on his PSP is using a Homebrew Enabler (HEN). So, what exactly is a HEN? Most of you will probably answer “a homebrew that allows me to launch other homebrews”, correct, but how does it work exactly? It may look like black magic, but it’s not, in fact, most of the time spent coding a HEN is updating patches since big part of the mechanism is pretty much always the same. To understand better how a homebrew enabler works, we have to split it in three parts: the exploit, the payload, and systemctrl.



– Part 0 – Modules? –

To understand this, you first need to understand what a module is. A module is basically an executable. The PSP system is made by a lot of modules, each one of them accomplishes different tasks. A module can run in three different levels: user, vsh and kernel.

User level is the level of homebrews, it can’t do nothing special, just normal tasks. Vsh level (also known as updater mode) is the level of the VSH (you don’t say?), it is like user level, but has some “extra” permissions, it can load modules and reassign flashes for writing. Kernel level is the highest, it has no limits, a module running in kernel mode can do anything.

With HBL we can load homebrews that run only in user level, while a HEN allows you to run executables that require any permission level. All official modules are signed, you may know we can “sign” usermode homebrews (it’s a dirty trick, but works great, props to the guys who figured this out!), especially kernel modules, the system checks signatures of all executables, if one isn’t valid, the system will refuse to execute it. That’s exactly what a HEN does: modifies the system to make our executables look “valid” even if they’re not signed.

– Part 1 – The Exploit –

Let’s say you’re a hacker, you have found a kernel exploit (I assume you already know what a kernel exploit is, in case you don’t, I will give just a brief explanation: it’s a vulnerability in the PSP kernel that allows you to execute code with kernel permissions a.k.a. you can do whatever you want in the system) and you can run your own code (a homebrew), this is all you need to start working on a Homebrew Enabler.

The first thing a HEN does, is acquiring kernel permissions with the kernel exploit. Once you have kernel permissions you can write to kernel memory (the part of protected RAM where the “important stuff” is).

What exactly do we want to write to kernel memory and why? We want to write a few “patches” to loadexec. LoadExec a the part of the system responsible for many important tasks, like launching executables, it is also responsible of exiting an application. The process of exiting and executing an application is pretty simple:
1.LoadExec is told to execute/exit from an application
2.LoadExec loads in memory a raw binary executable called “reboot.bin” and jumps to it, leaving all the control to reboot (if it’s told to launch an application, it loads it in memory first)
3.reboot.bin parses a file called pspbtcnf, which contains a list of modules that need to be loaded
4.All the modules in the list are loaded, reboot gives control back to the system

– Part 2 – The Payload –

Why is this so important to us? Well, to keep our HEN alive in memory, we have to hack this simple process. After patching LoadExec in memory, this is what the process looks like:
1.LoadExec is told to execute/exit from an application
2.LoadExec loads in memory TWO raw binaries “reboot.bin” and “rebootex” (which is injected by us) and jumps to rebootex
3.Rebootex injects a module called “systemctrl” (the core of our HEN) in pspbtcnf, and patches reboot to allow loading of our unsigned module, then it jumps to reboot
4.reboot.bin parses pspbtcnf and loads all the modules, including our systemctrl, then gives control back to the system.

What is rebootex? Rebootex is our payload, nothing more than a raw binary that needs to accomplish two easy tasks: inject an entry in pspbtcnf that points to our systemctrl and patch the system in order to load systemctrl, even if it’s not signed.

– Part 3 – Systemctrl –

Systemctrl, finally, is the core of a Homebrew Enabler, it’s a module always running in kernel mode (as long as the HEN is loaded, of course) that does a lot of different things: takes care of the vshmenu, allows you to load plugins and many others, but most important: it patches the system.

More patches? Yes, more patches, systemctrl patches loadexec again to load rebootex (otherwise the HEN would be lost every time you launch an application), and other parts of the system to allow you to run unsigned code (homebrews and plugins) that otherwise wouldn’t be loaded, as I explained earlier.

Conclusion

Of course, this is just a general explanation of how a Homebrew Enabler works; this does not apply to every HEN, some developers may design it differently, but these are the core concepts of every enabler. I tried to keep this explanation as simple as possible for normal users, if you’re a dev, you may find some parts “too easy”, I know, but again, you’re not a normal user, aren’t you? — Freddy