DS2x86 version 0.07 released!

Pate has released a new version of his Dos Emulator for the DS Console:

Sorry for the one day delay to my normal Sunday updates, but as this was a long Easter weekend I wanted to use the one extra day to work on DS2x86 and get as many fixes implemented as possible.
DS2x86 0.07 release info


Okay, this new version has the following fixes:

• There is now (finally) a blinking cursor in the text modes, most importantly in the 4DOS command prompt.
• HIMEM.SYS emulation added. This allows Chaos Engine to run, and will enable support for Windows 3.00a in Standard Mode in the future.
• Fixed a serious total crash problem in 32-bit string opcodes. This happened in games like Abuse, Ascendancy and Carmageddon. Those games still don't work properly, but they don't hang the system completely any more so I have a chance of debugging and improving them further.
• Fixed a stack alignment problem that could cause "Exception 4" BSOD errors. This fixes at least Micro Machines.
• Improved the BSOD reporting. Now the printed data includes a stack trace, so I can better determine the location of the problem.
• Improved horizontal blank reporting, this might solve hanging problems in some games, like Pinball Fantasy.
• Improved Mode-X graphics blitting code to handle wrap-around properly.
• Changed the screen refresh interrupt interval from 60Hz to 59Hz. This might help with the intermittent "stuck key" problem in some games.
• Direct file read to EGA memory is now supported (for example in Heimdall).
• Implemented a number of previously missing opcodes.

DS2x86 0.07 background information

After the previous blog post I worked on Jazz Jackrabbit for a while, but could not get any notable progress with it. It still needs a lot of debugging to determine the problem, and so I decided to move on to some other issues that might be faster to solve.
First, I decided to add HIMEM.SYS emulation, as I could simply port the code from the most recent DSx86 version. This in itself didn't take long, although I ran into a very strange problem where the whole DS2x86 hung immediately when launching any game. It loaded 4DOS.COM fine, and I could give internal commands like "dir" or "memory", but immediately after launching any game it hung totally. I then began commenting out the new HIMEM.SYS stuff, until the only things remaining were a couple of static integer variables in the new C++ module! Commenting these variables out got rid of the problem, but when I added these static variables back, the problem came back. One of the variables had an initial value based on the addresses of two other variables, which seemed to cause the problem. I switched to using a #define, and the problem was solved, but I still don't quite understand how a static variable initialization can break the whole software.
After the HIMEM.SYS emulation started working, Chaos Engine began to ran otherwise fine, except that the display had a moving line of black pixels, and scrolling the zoomed screen did not work properly. The game wraps the display around the A000 segment while scrolling (similarly to what Commander Keen 4 does in EGA mode), and this feature was not yet supported in my Mode-X screen blitting code. I took some time to make this work correctly in both Zoom and Scale screen modes, and now this type of wrap-around should cause no more visual glitches. This is what Chaos Engine now looks like, with a wrap around in the middle of the screen (but not showing :-).

After the HIMEM.SYS emulation in place, I wanted to see whether Windows 3.00a would run in Standard Mode. I had several problems with it, all due to various bugs in my protected mode implementation which I had to fix. There is still a problem with the Global Descriptor Table handling, as Windows uses a segment selector 0x0101 when returning from protected mode to real mode, but in DS2x86 the GDT entry points to a data segment, while in DSx86 and DOSBox it points correctly to a code segment. So, no Windows 3.00a in Standard Mode yet in this version, but I'll try to fix this problem in the near future.
Next, I was looking whether I had Hexen on my SD card. I didn't find it, but as there was Heimdall, I decided to test how it runs. It started fine, but when talking to a character I got a BSOD reporting an invalid store address of 0x600CE030. In this same situation DSx86 shows garbage on the screen, so I thought that this might be a chance to find out what exactly is wrong in my emulation code. I checked the address reported by the BSOD, but it pointed to inside the C library memcpy() function. Obviously, memcpy is used all around the code, so that was not of much help. I needed to find out what code it was that called the memcpy, so I implemented a simple stack traversal routine into my exception handler code. It attempts to find return addresses pointing to my own DS2x86 code by traversing the stack above the exception handler stack frame. When I ran Heimdall again, I got a stack trace that (after converting the hex addresses to function names using the dump file) displayed the following call hierarchy: run() -> INTHandler() -> int21h() -> DOSRead() -> fat_fread() -> memcpy().
So, the problem was that the DOS int21 handler used an invalid memory address to store the data read from disk. However, since the address is calculated from the DS:DX registers, which can only point to the real mode memory within the first megabyte of DOS RAM, it should not be possible for the address to point outside the emulated DOS RAM area. Except, that I have mapped the graphics memory segment A000 differently, in order to trap accesses to graphics memory. It had not occurred to me that a game would read data from disk DIRECTLY to graphics memory, so I hadn't trapped the graphics memory address inside the DOS file routines. And, as it turned out, Heimdall loads data directly to EGA VRAM from disk, so trapping the address in the DOSRead routine fixed this problem. This is obviously also the reason why Heimdall in DSx86 displays garbage on the screen, it just does not crash like DS2x86 did. I'll fix this problem into DSx86 also for the next DSx86 version.
At this point it was already Saturday evening, so I decided to work on the text mode cursor emulation on Sunday. I had tried to implement the cursor emulation earlier, but at that point it made the "stuck key" problem much worse (most likely because the text mode screen then needed to update much faster. I use a dirty buffer approach with the text mode handling, so that I don't need to convert those fonts to graphics where the character has not changed). But now with the 60Hz -> 59Hz screen refresh rate change, I thought I could try to add the cursor emulation again. It took me most of the Sunday to implement, as I had an unexpected bug in the dirty buffer handling. It had caused no symptoms until I implemented the cursor, so it took me a while to look into the correct routine for the bug. In any case, by Sunday afternoon I got the cursor emulation working.
On Sunday evening I then continued debugging Ascendancy, which hung DS2x86 completely (its is just one of many DOS4GW games that seem to exhibit similar behaviour). I had already a couple of days earlier found the high-level call that causes the hang, so now I just burrowed through to the lower level calls. Finally I found out that it was a simple and common REP MOVSW opcode that caused the hang. This was a bit strange, as this opcode is called thousands of times in all games, why does it hang in this case? The addresses and counts looked fine, it tried to move 512 bytes from high memory to the beginning of a segment in low memory. I had to go through my helper macros one by one until I finally found the problem. In the code that tests whether the output offset wraps around the segment, the 32-bit routine (where the segment size can be a full 4GB), a temporary variable caused an overflow when the input offset was zero. It is rare for the 32-bit input offset to be zero, as usually the protected mode programs use flat segments starting at the beginning of the RAM. Since the beginning of the DOS RAM contains real mode interrupt vectors, the protected mode offsets normally do not start from zero. In this case the segment did not start at the beginning of the RAM, so a zero offset was OK. After fixing the helper macro I got rid of this hang, so in the future I can debug these games properly.
So, that's about the amount of work I managed to do for this version. I also added many missing opcodes from the debug logs you have sent, thanks again for sending me those! I think pretty much all the opcode errors in the logs have now been implemented, except for some obscure problems that are most likely caused by the program executing data. These I will continue working on.

Download Here

http://dsx86.patrickaalto.com/DSblog.html