As I
mentioned on Sunday, my main focus for R12 is to fix whatever's causing the dynarec to hang with SSB, but to start with I've spent a little time looking into fixing some of the graphical issues plaguing Super Smash Bros.
In the end I fixed four or five separate problems which were all causing quite significant glitches.
The first bug was that I was assuming that the fixed-point texture coordinates specified in the TexRect (i.e. 2d textured rectangle) commands were unsigned. It turns out that they're actually signed - SSB just happens to be one of the few roms that ever specifies negative values.
The next issue was that I was ignoring the RDP tile* mask parameters, and assuming that the tile width and height always defined the extents for both wrapping and clamping. As it turns out, the mask values (if set) define how many bits from the texture coordinate to accept - all the other bits from the texture coordinate are ignored. This means that if a tile defines a mask of 5, the texture will wrap every 2^5, or 32 texels. What's quite interesting about the N64 is that it lets you define different limit for clamping than for wrapping. For instance, you can wrap every 32 texels, but clamp the texture to a coordinate of 100 (i.e. after the texture has repeated 3 and a bit times.) It turns out this is exactly what SSB was doing.
A related issue was that as the PSP only supports textures with power-of-2 dimensions, I have to pad out non-power-of-2 textures from the n64 with empty space. As I was assuming that the n64 always clamped at power of 2 boundaries, it meant that the PSP occasionally displayed texels from bits of the texture that I hadn't initialised. To get around this I just manually repeat the last column/row of the texture when converting it on the PSP.
Another issue I fixed was relating to how I handled 'swizzled'** textures that were not an even number of texels. In the second screenshot on the left below, you can see a fringe of corrupt pixels down the right-hand side of each character's icon. This was due to me not correctly handling all of the texels the swizzled rows, but it only happened when the rows were not a multiple of 8 bytes (in the example below, the textures were 30x32 texels at 16bpp). Again, this is something SSB does that I've not seen in other roms.
I fixed a bug relating to how I was recolouring textures. In order to support some of the N64's more complex combiner modes on the PSP, I need to replace the RGB channels of a texture while maintaining the existing alpha channel. As it turns out, I wasn't handling this correctly for textures that weren't a power of 2. This is what is messing up the black background behind the 'Super Smash Bros' text on the first screenshot. Whoops.
Wow! That's quite a few bugs! Here's a few screenshots, with SSB running in R11 on the left, and the R12 work-in-progress on the right:
What's quite nice is that even though I've been fixing these with SSB in mind, there are likely to be lots of other roms which I've not tested that are relying on the same behaviour.
Graphically SSB is looking pretty slick, so now it's on to chasing down that dynarec bug :)
-StrmnNrmn
* The RDP has 8 tile attribute descriptors, which are used to tell the RDP how to interpret the data loaded into it's 4KiB of texture memory. They define properties such as the tile's format, width, height and whether to clamp or wrap and so on.
** On the N64, all odd rows in a texture have pairs of texels alternated (or more precisely pairs of 4-bytes of data.) The reason for this is that the N64's texture memory is composed of two separate banks of 2KiB, and organising texels like this allows for multiple texels to be accessed simultaneously when sampling from the texture. I use the term 'swizzled' loosely as it's quite a different process than swizzled textures on modern consoles (such as the
PSP and the
Xbox (see page 28)), even if it is performed for similar reasons.
