PSJailbreak Payload Reverse Engineering!

news via http://psx-scene.com/forums/showthread.php?t=66115

The last part of the puzzle has finally been made public in "assembly" format now, for the original PSJailbreak design.

The first part was done already, and that produced the USB hub, and plugging in and out, and the loading of the "payload". -- And the making of PSGroove. -- But now finally, the "payload" has been ripped apart, and make public in full assembly language, instead of just a bunch of HEXadecimal.


Quote:
1. It gets control at Exploit_Entry, which copies the rest of the payload to the fixed address 0x8000000000700000.

2. Exploit_Main copies a resident part of the payload to another location, creates virtual USB device driver called "MOD" with 3 functions, hooks some VSH functions via TOC entry and does some permanent in-ram patching. when the work is done it zeroes itself out.

3. The resident part has basically 3 purposes: (a)It manages virtual USB device, (b)It does some on-the-fly patching and (c)It hooks all the game disk file accesses from the VSH.

3a. The virtual USB device is needed to make sure the original PS3JB device in plugged in. Once the correct device is plugged (the one with the AAAAC0DE) device driver initializes the variable to 1 (see kmod_func1 - probably "identify device", and kmod_func2 - "initialize device"). If one pulls the device out, the function kmod_func3_call_panic "term device" is called which causes a kernel panic.

3b. The on-the-fly patching part of the code is probably called on virtual memory page remapping and does additional patching in-place. It identifies if the pages requires patching byte calculating it's "hash" and comparing to the table entries. One of the patches enables developer menu/settings called "category_game_tool2.xml#root" which probably enables support of the pkgs and other dev stuff.

3c. The hooks from the VSH are intended to redirect all on-BDVD file requests (or probably just "open") from VSH to the HDD saved backup. The launcher saves the base directory of the game started and after that all the file names are prepended with it. that's how the backup feature works. The LV1 still needs BDVD auth to launch the game, so the original disc in BDVD is still required.

4. Adds a Syscall (Syscall 36) which will be called by Backup Loader to activate the virtual bluray drive with the correct backed-up disk.

5. Patches the return value from Hypercall 99 so that we can launch unsigned apps.

News Source: http://twitter.com/Mathieulh