Future of GPU shaders in emulators

News via http://blackbird.usask.ca/wordpress/?p=124

For many years, emulator users have been able to take advantage of improved (pixel purists might disagree) image quality using well known filters such as HQ2x, SaI2x, etc. These run on CPU and incur a noticeable performance loss. On a PC, this might not be such a big deal with our new shiny CPUs that are just waiting to waste clock cycles, however, on our PlayStation 3, using very valuable clock cycles on filtering graphics on the PPU is simply not acceptable, unless you’re comfortable playing old-school games at 20 fps.



Plain nearest-neighbor




HQ2x, a classic CPU-based filter

Luckily, we have a lovely device called the GPU, which specializes in crunching through calculations in a highly parallel fashion. Graphics processing can be parallelized extremely well, and thus explains the design of modern GPUs, tons of weaker cores.

Up till recent years, graphics cards were locked into one specific function. Programming legacy OpenGL back in the day was essentially turning on a zillion different knobs of a big black box (the GPU). You wanted lighting, you enabled lighting, set some parameters here and there, and done. Easy to get started, but horribly inflexible once you want to do complex stuff. This is now called the fixed-function pipeline.

Eventually, developers understood that we needed a way to treat the GPU more like a conventional computer. Rather than turning on a tons of different knobs, you implemented the things you actually used in a simplified programming language. Full control and flexibility at the cost of a slightly steeper learning curve. These languages are what we call shader languages. The three most popular ones today are:

•GLSL: Native OpenGL shading language.
•Cg: nVidias shading language that works with either OpenGL or Direct3D. Syntax resembles HLSL.
•HLSL: Microsoft’s shading language for use with Direct3D.
Before, it was practically impossible to implement a filter like HQ2x on the GPU using fixed-function pipeline. The hardware was far too inflexible to allow this. However, with our new programmable hardware, it suddenly started to sound feasible to program filters on the GPU.

Pete’s OpenGL2 plugin
Some emulator authors started to see opportunities to use these new possibilities. One well known example of this is Pete’s awesome OpenGL2 plugin for the plugin-based PlayStation 1 emulators. Obviously there are far too many emulators out there, so I can’t make an exhaustive list or something like that. At the very least, shader support certainly isn’t very common, even today, but I believe this will soon become a “must have”-feature.





Example of some interesting shaders running in Petes OGL2 plugin

Shaders instead of CPU filters by necessity
On the PlayStation 3, Sony provides developers with a Cg shader environment, and this was basically an invitation for us to use it in our emulator ports. The RSX found on the PlayStation 3 is reasonably powerful despite its age, while the PPU unit is quite weak relatively speaking (runs conventional code very badly), so it makes perfect sense to offload all graphics processing to the RSX, where it belongs.

One problem with shaders is that they typically need to be written specifically for each emulator, even if they were written in the same language. As with normal code, where you have APIs you can communicate with, the rendering code needs to communicate information with the shader program running on the GPU. You need to tell it things like: “Where is the texture located? How big is my texture? What are the coordinates?”, and things like that. Usually, every emulator out there tended to implement their own interfaces, so shaders still needed to be ported between every emulator that wanted to use them.

With some recent developments, I believe we’re one step closer to achieve some kind of specification that other emulator authors might desire to follow.

I believe SNES9x-PS3 (and its brother ports) was the first emulator port on a console to utilize shaders, at least as a major feature. Its shader library is becoming reasonably sized. When the shader implementation was being written, it was actually first implemented in SSNES. This meant that shaders could easily be tested on the PC before being transferred over the PS3, achieving the same result, greatly helping out with debugging. Shaders are essentially simple .cg files, which are compiled at run-time and transferred directly to the GPU. No need to compile C/C++ filters straight into the emulator itself which used to be quite typical, which would be a horrible situation indeed.

Essentially, doing as much on the GPU as possible was a design choice from the start. Much, much better performance and arguably easier to program than optimizing C/assembly code. It’s a win-win situation.

Later on, I implemented a multi-pass specification for the .cg shaders, using several .cg shaders and combining them into one effect. This is known on the PS3 ports as the FBO-mode, which allows two different shaders to be used in a chain in a mix-and-match fashion. This feature is encapsulated in .cgp files, essentially a config file similar to the .presets found on the PS3 ports.

Game Aware Shaders
We also played around with the idea of letting shaders peek into game state, in order to perform dynamic effects, dubbed “Game Aware shaders”. We extended this idea a bit to allow for a “game manual” shader of sorts, a proof-of-concept. I believe lots of ideas could spring out of this at least. Almost all the processing happens on the GPU.

Seiken Densetsu 3 – Game Manual Shader

Specification
The shader interface was documented so other emulators could implement it, and recently, OV2, the current developer of mainline SNES9x/Win32, also implemented the same specification. This means that writing a .cg shader against these specs will allow it to run unmodified on quite different emulators. The single-pass implementation is usable in the recent 1.53 release, and the multi-pass implementation will most likely be available in 1.54. If you’re willing to compile yourself (or ask someone to), you can grab it from Git.



Border shader/cgwg-CRT-flat running in SNES9x/Win32 (click to zoom)


I hope that more emulator authors choose to adopt this spec in the future. Ideally, I think a shader should be expected to work on a broad range of emulators out there. By now, even handhelds being able to utilize GLSL shaders in OpenGL ES. I believe shaders will completely replace CPU filters eventually.