Here is a draft of the separate kernel module:
git clone http://mandos.homelinux.org/~glaurung/git/ps3rsx.git
I've updated the Xorg driver accordingly:
git clone http://mandos.homelinux.org/~glaurun...-video-ps3.git

It is not yet finished, and mostly untested, but it shows my plan for the interface with userspace. Basically we have two options to access the FIFO:
- exclusive: we disable framebuffer redraw, allocate a context, then use the FIFO workaround to start running the context's FIFO and use it until the application is over. No other application can use the GPU, and any blit from either the fb driver or another context would break the application in exclusive mode. That's how we've been using the RSX so far.
- shared: we allocate a context, then run the FIFO for some time using a fake blit (size 0x0). When commands are finished processing, a new fake blit is needed to kick the FIFO again and process more commands. In the meantime, other contexts (including FB driver) can use the GPU. Note that contexts are not preempted, so if we put FIFO in a loop, other contexts are not able to execute commands (and we get error -24 for FB blit). So this is cooperative sharing.

I want to be able to support both modes. The former is easier to use and more efficient (useful for e.g. 3D demo or game), while the latter allows mixing multiple users of the GPU (e.g. Desktop usage, with one context for Xorg, one for GLX, and one for VRAM used as swap). The current module provides those two modes, with one ioctl() to enter exclusive mode, and one ioctl for one-shot kicking of the fifo (shared mode). The Xorg driver still uses exclusive mode ATM; I've code for shared mode but it needs testing and benchmarking. My short term goal is usable Desktop usage (X is improved, now we need more RAM).

On the long term, my plan is to add RAMIN/VRAM memory management to the module, have libps3rsx provide the user space API for FIFO control and object instanciation. Also, Xorg driver should use libps3rsx to avoid code duplication. Maybe when can then move slowly from ps3rsx.ko/librsx to drm/libdrm API (but keep exclusive mode option in parallel). Sound good?

http://forums.ps2dev.org/viewtopic.p...=asc&start=211