wraggster
May 18th, 2008, 10:17
Marcan has posted a new video and article about the Homebrew Channel:
The Homebrew Channel was missing its banner in the video that Bushing posted some time ago. As making banners is quite complicated and can easily result in bricking, that part of the channel wasn’t quite ready yet.
Who am I kidding. I had the tools done (as evidenced by the working icon), I was just too laz^H^H^Hbusy to get the big banner coded up in time. So here I present what will most likely be the final banner in the completed channel.
http://www.youtube.com/v/CGeMP1zsW2I
The entire banner, down to the last byte, was developed using homebrew tools - no leaked Nintendo SDK utilities were employed, not even to reverse engineer them. Everything was done by staring at many other banners (or rather, at the files that compose them) and reverse engineering the System Menu code that plays them back.
A few technical details on how this works:
The entire channel banner, which is stored as a “channel content” inside the channel or wad (in particular, content index 0), is composed of a small header detailing the channel name in all languages (this is what you see when you hover over the channel and what ends up in the play history) and some other metadata about the banner - this is called the IMET. Following it is a U8 (.arc) archive, containing three other files: /meta/banner.bin, /meta/icon.bin, and /meta/sound.bin.
sound.bin is the file that contains the sound snippet that plays when you select the channel. It is composed of a BNS audio file, which is a header followed by GC-DSP ADPCM audio data. It supports loops with intros, as seen in our banner - you can play back the file and loop at any predetermined point, so you can have an “intro” and then a part that loops forever. I had to write an encoder for this format, as I couldn’t find any GC-DSP ADPCM encoders. The BNS file is prepended with an IMD5 header, which simply contains its MD5 sum, to create the sound.bin file.
icon.bin and banner.bin are identical in format. icon.bin contains the layout, animation, and bitmap data for the channel thumbnail icon (what you see in the System Menu overview, and what you see in the Channel Management screen), while banner.bin contains the same data for the large full-screen banner. The files begin with an IMD5 header, which wraps an LZ77 compressed block of data. LZ77 compression is the same as used in the GBA and Nintendo DS BIOS routines. The compressed data is nothing but another U8 archive, with the following structure:
arc/
blyt/
icon.brlyt or banner.brlyt
anim/
icon.brlan or banner.brlan
- or -
icon_Start.brlan or banner_Start.brlan
icon_Loop.brlan or banner_Loop.brlan
timg/
several .tpl files
The .brlyt file is interesting. Its structure is similar to the IFF format, and it consists of several sections that define, much like a 2D layout program but with influences from the 3D world, what the static banner looks like. First it enumerates the textures (.tpl files) that are in use. These .tpl files are in a standard GC format, and include several standard color formats, some of which include an alpha channel. Then, it assigns these textures to materials, which also include things such as several types of coloring (multiplied with the colors of the texture), texture coordinates and wrapping, etc. It also supports multitexturing and things such as alpha masks and other effects, most of which we haven’t figured out yet. Each material has an ASCII name. After the materials section, the bulk of the file consists of an object tree: there are panes (the outermost of which is a root pane), and pictures. Panes define a set of coordinates and contain other objects - they allow the grouping of several objects, which can be moved as a whole. Pictures are actually panes too (they’d be considered a subclass of them in an object oriented world), so they can contain other pictures and panes, but they also reference a material and a set of coordinates with which to draw with it. The coordinates used include the obvious X, Y, Width, and Height, but also rotation, magnification in the X and Y dimensions, and probably shearing and others too. Pictures also define an alpha value for the entire picture. There are also other object types, such as text rendered using a font, but we don’t know much about those. Pictures and Panes also have ASCII names. Finally, the brlyt contains a grouping section, which is used to conditionally show or hide pictures and panes depending on the current language, to enable language-dependent banners (for example, channel names).
Finally, the .brlan file is also an IFF derivative and contains the animations (and a header with the total length in frames). The format consists of, essentially, a set of nested lists. The brlan file lists the objects that are to be animated (by name). Each object has a list of “animation categories”, which includes things like coordinates, parameters, etc. Each category contains a list of actual animated properties, and each property contains a list of animation triplets (keyframes). For example, you can have a bubble object with a coordinate category, an X property, and then a set of keyframe triplets. Each category is given a FOURCC identifier, and each property is a numerical ID. For coordinates, for example, the ID is just the index into the coordinate array (X is 0, Y is 1, etc), while for “parameters” (our term) there’s an ID for the alpha value. We still don’t know about other animatable properties, but I think material properties can also be animated (which would affect all pictures using that material). The “triplets” are tuples of three 32-bit floats: the first is the (potentially non-integer) frame number, the second is the value at that frame, and the last number is related to the slope at that keyframe, and affects the mathematical formula used for interpolation (which seems to be custom, not a standard bézier curve or spline). For each frame, the triplet before it and the triplet after it, together with the frame number, are used in an interpolation formula. If the frame is before the first triplet, or after the last, it just keeps the value defined at that triplet. Loops work by playing the _Start file first, and then the _Loop file repeatedly. Notably, triplets can contain negative times (before the start of the animation) and times after the last frame, and the interpolation still works (you’ll just never see part of it). I exploit this feature to simplify the building of our banner: I actually build the entire animation, start and loop, as one continuous structure, with bits extending before the start of the start and beyond the end of the loop. For example, the ones extending beyond the end of the loop are duplicated at the start of the loop, so the transition is seamless, and the ones extending to negative times are there just for mathematical simplicity. Our _Start and _Loop files actually both contain the entire animation (this is laziness on my part; I should remove the useless parts), but the frame numbers are offset to make it play back the relevant part.
We build our banner using a bunch of small utilities to, say, compress with LZ77 or pack a U8 archive, tied together using Makefiles. However, building brlyt and brlan files is particularly challenging. Initially, Daeken and I wrote a banner “simulator” that attempts to play back existing channel banners, which was a great aid during reverse engineering (we never intend for this to be a fully-compatible renderer, just good enough to help us understand the format). The Python program reads in the brlyt and brlan into a bunch of objects. What I did was rework these objects to be instantiable on their own, and then add methods to dump the data back out to brlyt and brlan. This way, I could use the Nintendo banners, read them in, dump them out, and verify that the resulting files are identical. To build our actual channel banner, I have a Python script that manually instantiates all of these objects with the right properties, parameters, and coordinates, and then calls the “dump out” method. I also use a bit of Python code to generate elements of the banner programmatically - for example, the large amount of bubbles is generated by a set of Python classes that create them randomly and then assign them to Pictures, reusing Pictures of the same types of bubbles throughout the animation.
Overall, this took quite a bit of time, but the result is quite satisfying and I feel it was all worth it. The tools used will be released after the Homebrew Channel is finally out.
http://hackmii.com/2008/05/the-elusive-banner/
The Homebrew Channel was missing its banner in the video that Bushing posted some time ago. As making banners is quite complicated and can easily result in bricking, that part of the channel wasn’t quite ready yet.
Who am I kidding. I had the tools done (as evidenced by the working icon), I was just too laz^H^H^Hbusy to get the big banner coded up in time. So here I present what will most likely be the final banner in the completed channel.
http://www.youtube.com/v/CGeMP1zsW2I
The entire banner, down to the last byte, was developed using homebrew tools - no leaked Nintendo SDK utilities were employed, not even to reverse engineer them. Everything was done by staring at many other banners (or rather, at the files that compose them) and reverse engineering the System Menu code that plays them back.
A few technical details on how this works:
The entire channel banner, which is stored as a “channel content” inside the channel or wad (in particular, content index 0), is composed of a small header detailing the channel name in all languages (this is what you see when you hover over the channel and what ends up in the play history) and some other metadata about the banner - this is called the IMET. Following it is a U8 (.arc) archive, containing three other files: /meta/banner.bin, /meta/icon.bin, and /meta/sound.bin.
sound.bin is the file that contains the sound snippet that plays when you select the channel. It is composed of a BNS audio file, which is a header followed by GC-DSP ADPCM audio data. It supports loops with intros, as seen in our banner - you can play back the file and loop at any predetermined point, so you can have an “intro” and then a part that loops forever. I had to write an encoder for this format, as I couldn’t find any GC-DSP ADPCM encoders. The BNS file is prepended with an IMD5 header, which simply contains its MD5 sum, to create the sound.bin file.
icon.bin and banner.bin are identical in format. icon.bin contains the layout, animation, and bitmap data for the channel thumbnail icon (what you see in the System Menu overview, and what you see in the Channel Management screen), while banner.bin contains the same data for the large full-screen banner. The files begin with an IMD5 header, which wraps an LZ77 compressed block of data. LZ77 compression is the same as used in the GBA and Nintendo DS BIOS routines. The compressed data is nothing but another U8 archive, with the following structure:
arc/
blyt/
icon.brlyt or banner.brlyt
anim/
icon.brlan or banner.brlan
- or -
icon_Start.brlan or banner_Start.brlan
icon_Loop.brlan or banner_Loop.brlan
timg/
several .tpl files
The .brlyt file is interesting. Its structure is similar to the IFF format, and it consists of several sections that define, much like a 2D layout program but with influences from the 3D world, what the static banner looks like. First it enumerates the textures (.tpl files) that are in use. These .tpl files are in a standard GC format, and include several standard color formats, some of which include an alpha channel. Then, it assigns these textures to materials, which also include things such as several types of coloring (multiplied with the colors of the texture), texture coordinates and wrapping, etc. It also supports multitexturing and things such as alpha masks and other effects, most of which we haven’t figured out yet. Each material has an ASCII name. After the materials section, the bulk of the file consists of an object tree: there are panes (the outermost of which is a root pane), and pictures. Panes define a set of coordinates and contain other objects - they allow the grouping of several objects, which can be moved as a whole. Pictures are actually panes too (they’d be considered a subclass of them in an object oriented world), so they can contain other pictures and panes, but they also reference a material and a set of coordinates with which to draw with it. The coordinates used include the obvious X, Y, Width, and Height, but also rotation, magnification in the X and Y dimensions, and probably shearing and others too. Pictures also define an alpha value for the entire picture. There are also other object types, such as text rendered using a font, but we don’t know much about those. Pictures and Panes also have ASCII names. Finally, the brlyt contains a grouping section, which is used to conditionally show or hide pictures and panes depending on the current language, to enable language-dependent banners (for example, channel names).
Finally, the .brlan file is also an IFF derivative and contains the animations (and a header with the total length in frames). The format consists of, essentially, a set of nested lists. The brlan file lists the objects that are to be animated (by name). Each object has a list of “animation categories”, which includes things like coordinates, parameters, etc. Each category contains a list of actual animated properties, and each property contains a list of animation triplets (keyframes). For example, you can have a bubble object with a coordinate category, an X property, and then a set of keyframe triplets. Each category is given a FOURCC identifier, and each property is a numerical ID. For coordinates, for example, the ID is just the index into the coordinate array (X is 0, Y is 1, etc), while for “parameters” (our term) there’s an ID for the alpha value. We still don’t know about other animatable properties, but I think material properties can also be animated (which would affect all pictures using that material). The “triplets” are tuples of three 32-bit floats: the first is the (potentially non-integer) frame number, the second is the value at that frame, and the last number is related to the slope at that keyframe, and affects the mathematical formula used for interpolation (which seems to be custom, not a standard bézier curve or spline). For each frame, the triplet before it and the triplet after it, together with the frame number, are used in an interpolation formula. If the frame is before the first triplet, or after the last, it just keeps the value defined at that triplet. Loops work by playing the _Start file first, and then the _Loop file repeatedly. Notably, triplets can contain negative times (before the start of the animation) and times after the last frame, and the interpolation still works (you’ll just never see part of it). I exploit this feature to simplify the building of our banner: I actually build the entire animation, start and loop, as one continuous structure, with bits extending before the start of the start and beyond the end of the loop. For example, the ones extending beyond the end of the loop are duplicated at the start of the loop, so the transition is seamless, and the ones extending to negative times are there just for mathematical simplicity. Our _Start and _Loop files actually both contain the entire animation (this is laziness on my part; I should remove the useless parts), but the frame numbers are offset to make it play back the relevant part.
We build our banner using a bunch of small utilities to, say, compress with LZ77 or pack a U8 archive, tied together using Makefiles. However, building brlyt and brlan files is particularly challenging. Initially, Daeken and I wrote a banner “simulator” that attempts to play back existing channel banners, which was a great aid during reverse engineering (we never intend for this to be a fully-compatible renderer, just good enough to help us understand the format). The Python program reads in the brlyt and brlan into a bunch of objects. What I did was rework these objects to be instantiable on their own, and then add methods to dump the data back out to brlyt and brlan. This way, I could use the Nintendo banners, read them in, dump them out, and verify that the resulting files are identical. To build our actual channel banner, I have a Python script that manually instantiates all of these objects with the right properties, parameters, and coordinates, and then calls the “dump out” method. I also use a bit of Python code to generate elements of the banner programmatically - for example, the large amount of bubbles is generated by a set of Python classes that create them randomly and then assign them to Pictures, reusing Pictures of the same types of bubbles throughout the animation.
Overall, this took quite a bit of time, but the result is quite satisfying and I feel it was all worth it. The tools used will be released after the Homebrew Channel is finally out.
http://hackmii.com/2008/05/the-elusive-banner/