Release DraStic Nintendo DS emulator


Thank you for the explanation of clipping, I didn't know a new, fitting polygon is created, when the old polygon is partly inside and outside the frustum. I thought it would somehow work, now I know how it works^^.
 
The annoying thing with the clipping on DS is it's not even totally accurate thanks to all the precision problems with the rasterizer and probably the clipper itself. When stuff gets clipped you can see it distort. Like if you move a triangle down the screen, when it crosses the bottom you'll see its edges shift. Nintendo even documented that there are precision errors for clipping that can somehow cause the colors to wrap around.. how this happens I have no idea..

I always thought DS should have used a guard-band with 16-bit indexes or so. Spend a little more storage on vertexes but then it wouldn't need nearly as many because the risk of clipping would be lower. But they probably didn't because it'd make some rasterizer operations more expensive, and since it's a pure scanline renderer it re-does these operations every line.
 
And my last question today, to me sane: your renderer is integer only but your geometry engine is using floats right? Or do you clip with with integers, do the "isInsideFrustum" check with integers and all that stuff? If you do this, aren't there a lot of accuracy problems? The nintendo ds uses float (probably not doubles) for this, right?
 
Might just be me, but mario kart ds selection screens seem to be running not faster but more fluid. 

i'll dig around and see if I can find the fps overlay option and report back

Thanks again Exophase
 
rohezal said:
And my last question today, to me sane: your renderer is integer only but your geometry engine is using floats right? Or do you clip with with integers, do the "isInsideFrustum" check with integers and all that stuff? If you do this, aren't there a lot of accuracy problems? The nintendo ds uses float (probably not doubles) for this, right?
Click to expand...

Other way around. Current geometry engine is fully integer except for the edge clip interpolation function which uses floats for the clip ratio. Rasterizer uses floats.

Nintendo DS is totally integer. It uses different precisions depending on the operation and a lot of them aren't fully known to me, as well as order of operations and accuracy of all divisions. I've RE'd some but not all of it. The rasterizer is really difficult to reverse engineer and the more complex parts of the geometry engine aren't that better.

The world coordinates are 32-bit fixed point (1.19.12 format), although Nintendo says that only the bottom 24-bits are reliable, meaning that there's probably some overflow problems for some calculations and I ran into some of these but don't know exactly what they are. It's not totally consistent for everything. There's some other stuff that's just impossible to explain, like transformed normals have a 1.0.12 format when you look at the result of the VEC_TEST function and I've confirmed that all the bits are good. But I've also confirmed that when it transforms normals for lighting the result is really in 1.6.9 (or maybe 1.5.9, can't totally remember) format, if you calculate it with more fractional bits you definitely get wrong results in scenes. So it's like they use two different pieces of hardware for the same function...
 
Last edited by a moderator:
Oh god, you must have spended ages to figure this stuff out (with a lot try and error i guess). Thank you again for creating such a great emulator. I would have gone crazy to find out all this formats and then see "oh it is a different format sometimes, looks like 2 different chips are used for the same stuff". You are unbelievable. 
 
Really cool to play DS on Pandora.  Once the 3D engine is optimized I think this thing is gonna go through the roof.
 
Thanks awfully for the update - the emulator is now a lot more stable for me, and I've put in a couple hours of picross and almost the same of Fire Emblem without a crash, which is unheard of. I have managed to force a crash now though, by pausing the game (by opening the emulator menu) then later, forgetting I'd done that, putting the system into low power mode. On bringing the system back up the menu was still up and operational, but on returning to the game it froze with a blank screen. I was able to use the Pandora button to get back to an operational system, though it was overlaid by the two DS screens, both blank.

While I was hunting for the process ID to kill it, my instance of top got nixed:


Abortedbc detected *** top: malloc(): memory corruption: 0x00027c40 ***


 However I've not reboted since then and the system looks pretty stable still. The pndrun.out didn't seem to contiain anything suspicious, to my untrained eye:

=======================================================================================
PND : /media/pandora_data/pandora/menu/drastic.pnd
PND_FSTYPE : Squashfs
APPDATADIR : /media/pandora_data/pandora/appdata/DraStic
APPDD_FSTYPE : ext2PND_CPUSPEED : EXENAME : drastic.sh
ARGUMENTS : =======================================================================================
[ START ]--- Mount the PND ----------
Mounting : mount -t squashfs -o ro "/dev/loop2" "/mnt/utmp/DraStic"
Mounting the Union FS : mount -t aufs -o exec,dirs="/media/pandora_data/pandora/appdata/DraStic=rw+nolwh":"/mnt/pnd/DraStic=rr" none "/mnt/utmp/DraStic"
[sUCCESS]--- Mount the PND ----------
[ START ]--- Starting the application (drastic.sh ) ----------
Starting DraStic (version r1.4p)
Checking directory /mnt/utmp/DraStic/backup: It's there.
Checking directory /mnt/utmp/DraStic/savestates: It's there.
Checking directory /mnt/utmp/DraStic/config: It's there.
Checking directory /mnt/utmp/DraStic/profiles: It's there.
Checking directory /mnt/utmp/DraStic/system: It's there.
Loading directory config file /mnt/utmp/DraStic/config/drastic.cf2
Loading config file /mnt/utmp/DraStic/config/drastic.cfg
got keypad at device keypadgot
gpio-keys at device gpio-keysUsing
touchscreen at /dev/input/touchscreen0.
Got memory mapped buffer at 0x40352000 (drastic_mapped_memory.dat)
Direct mapping main RAM to 2000000
Direct mapping main RAM to 2400000
Direct mapping main RAM to 2800000
Direct mapping main RAM to 2c00000
Initializing ARM9 bus-level memory map.
Initializing ARM7 bus-level memory map.
Initializing CP15.Initializing DMA.
ALSA output: requested rate of 44100, got 44100.
ALSA output: requested buffer size of 8192, got 8192.
ALSA output: requested period size of 2048, got 2048.
ALSA output: Using synchronous engine.
1524.326172 cycles per output sample (0.000656 samples per cycle)
Loaded 6406 games in database file /mnt/utmp/DraStic/game_database.xml.
Processing command line options.
Loaded 20 files in file info cache.
Found file info cache older than game database file. Not using.
Directory load took 0.000824 seconds.
[sUCCESS]--- Starting the application (drastic.sh ) ----------
[ START ]--- Restoring the frame buffer status ----------
[sUCCESS]--- Restoring the frame buffer status ----------
[ START ]--- uMount the PND ----------
[ START ]--- Waiting the Union to be available ----------
[sUCCESS]--- Waiting the Union to be available ----------
auplink:plink.c:223: AUFS_CTL_PLINK_MAINT: Inappropriate ioctl for device
rmdir: failed to remove `/mnt/utmp/DraStic': Device or resource busy
[ START ]--- Waiting the PND mount dir to be free ----------
[sUCCESS]--- Waiting the PND mount dir to be free ----------
cleanup done
[sUCCESS]--- uMount the PND ----------
=======================================================================================Return code is : 1

Similarly, last_kmsg just contained...


wtmp begins Wed Mar 20 09:24

...like last time. Hope that helps. As I say, the system now seems a lot more stable if I avoid sending it into low power mode. Thanks for the great update.
 
Last edited by a moderator:
I'll have to look into this low power issue. It should be unrelated to the other big freeze problem though.
 
That pndrun.out file looks like you just started DraStic and quit it without loading a ROM.  Keep in mind that the pndrun_DraStic.out file is only useful if you haven't restarted DraStic (or rebooted for that matter) since experiencing the issue you are referring to.

- Neelix
 
I hope I am not to annoying, just learned a bit about matrix geometrie transformations today in university.

All vertices of one model are transformed with the one same matrix into the projection plane of the camera. So in best case (one model) all vertice must be multiplied with the same matrix. Can't it be done with the dsp? Some code like this?

init_dsp();

matrix  = calculate_transform_matrix();

dsp_async_transform_vertices_into_projection_plane(matrix,0,2048);

async_emulate_sound();

async_emulate_gamelogic_for_next_frame();

async_clip_2048_vertices();

dsp_async_transform_vertices_into_projection_plane(matrix,2048,4096);

async_clip_2048_vertices();

dsp_async_transform_vertices_into_projection_plane(matrix,4096,6144);

async_clip_2048_vertices();
Click to expand...
I know that there are far more then 1 model, but I am not sure, how many? And how big they are. The big mushroom Mario from New Super Mario Brothers will fill 1/8 of the screen and probably consist out of 1000 vertices+. Maybe it is possible to decide per model or gl_draw() call (look how big the VBO is, if it is bigger then 200 vertices use the dsp, else use the cpu)?

This way some time could be saved for rendering.
 
Last edited by a moderator:
@Exophase how do you check your results when RE'ing the DS hardware? In my head you'd be writing small test programs for the DS that render a triangle that intersects the frustum, run it, then dump the VRAM from a DS. From there you could work out what you feel the interpolation/blending results should be per pixel, compare that to the VRAM dump and see if you have made correct guesses assumption. Is it something along these lines? (Is it even possible to dump out the VRAM from a DS?) I can only imagine how time consuming and complex it must be to RE so much stuff, and I am interested to learn more about how you go about it.
 
@Neelix Yes, it looks that way to me too. I can confirm that I checked it just after I killed the Drastic process though, and had not restarted it.


@Exophase Okay, I'll hold fire on any low power related bug reports for the time being. I've not had any other sort of crash since updating though, which is a big improvement for me :)
 
Last edited by a moderator:
rohezal said:
I hope I am not to annoying, just learned a bit about matrix geometrie transformations today in university.

All vertices of one model are transformed with the one same matrix into the projection plane of the camera. So in best case (one model) all vertice must be multiplied with the same matrix. Can't it be done with the dsp? Some code like this?

init_dsp();

matrix  = calculate_transform_matrix();

dsp_async_transform_vertices_into_projection_plane(matrix,0,2048);

async_emulate_sound();

async_emulate_gamelogic_for_next_frame();

async_clip_2048_vertices();

dsp_async_transform_vertices_into_projection_plane(matrix,2048,4096);

async_clip_2048_vertices();

dsp_async_transform_vertices_into_projection_plane(matrix,4096,6144);

async_clip_2048_vertices();
Click to expand...
I know that there are far more then 1 model, but I am not sure, how many? And how big they are. The big mushroom Mario from New Super Mario Brothers will fill 1/8 of the screen and probably consist out of 1000 vertices+. Maybe it is possible to decide per model or gl_draw() call (look how big the VBO is, if it is bigger then 200 vertices use the dsp, else use the cpu)?

This way some time could be saved for rendering.
Click to expand...
Refer back to my list of things that the geometry engine does. It's way more than just vertex transformation and clipping. And I'm not totally sure but I don't think the DSP is really hugely better than NEON for this. Maybe a bit because it can do 16x32-bit multiplications faster, but you have to take into consideration the differences in clock speed.

I also think you're confused about things here, the geometry engine doesn't take a list of (up to) 2048 polygons + 6144 vertexes. That's what it generates and gives to the rasterizer to render. It can accept as many vertexes/polygons as you submit to it in commands. It doesn't care if 90% of them get culled. And it's fast enough that it can support a huge number like this. NSMB's world map submits something like 2700 polygons even though only somewhere around 700 end up visible.

There are no VBOs or any other high level structures here. Just a stream of commands. They could be coming from DMA but don't have to be, NSMB for example (again) generates all of the commands straight from the CPU. All you see is commands to manipulate the active matrixes and commands to submit vertexes (and colors, material properties, lights, etc). You can literally modify the matrix in the middle of a polygon submission and it works fine, in fact this technique can be useful for vertex skinning. The game can also read back status from the geometry engine as things go along, like how many vertexes or polygons have been submitted. So you need to be able to get back this data quickly.

If I'd try offloading anything it'd be the rasterizer, because for the most part that really is treated like a black box that queues up an entire frame's worth of data and spits out stuff to the screen (and/or back to VRAM). But trying to use the DSP for something that'd be a huge amount of work is just not worth it to me. Even if it ends up faster, which is a huge gamble, it's something that would only benefit Pandora and probably not even Pandora 2.

Steven Craft said:
@Exophase how do you check your results when RE'ing the DS hardware? In my head you'd be writing small test programs for the DS that render a triangle that intersects the frustum, run it, then dump the VRAM from a DS. From there you could work out what you feel the interpolation/blending results should be per pixel, compare that to the VRAM dump and see if you have made correct guesses assumption. Is it something along these lines? (Is it even possible to dump out the VRAM from a DS?) I can only imagine how time consuming and complex it must be to RE so much stuff, and I am interested to learn more about how you go about it.
Click to expand...
The DS actually normally renders 3D directly to the screen instead of VRAM. But there's a part called the capture unit that can put it to VRAM too. You only get 16-bits, meaning you miss 3-bits of color data (natively 18-bit RGB), most of the alpha channel (normally 5 bits) and nothing like depth, stencil, or polygon ID info that the DS generates per-pixel. The other problem is that this takes up one of four VRAM banks that are normally available for texturing, so you can no longer texture as much, which could be an issue if you're trying to capture some game scene.

When I started working on the PS1 GPU renderer I asked notaz if he could capture some stream of commands from games over one frame of execution. Then later I got him to recreate these on a real PS1, dump the VRAM, and transfer it to a file. So I was able to emulate a frame using the dumped command list and compare it to the VRAM dump from the PS1 and see if any pixels are wrong. But there's like a billion variables in emulating a scene so it's really hard to try to get it all right that way. It was hard for PS1 and DS is much more complicated and implemented in a much less intuitive way than PS1 was. I don't know if I'll ever really be able to get pixel perfect scenes in it..

My basic approach thus far has been this - have a test program that lets you input a bunch of things that it passes through to the hardware. Then have the test program try to emulate that thing itself and compare against the hardware. Play around with different things manually and try to see if I can find a pattern for where it doesn't match. If it's advanced to the point where I feel like it could be right try running a test program which generates a bunch of test cases automatically and checks that they all pass.

I've varied somewhat on what I've tried from here.. first off I wrote results to a file when they didn't match. Then loaded these up on a program running on PC, and tried changing the algorithm to match the hardware results. On the PC I can test the whole dataset much much faster, as well as try different things. Another thing I've tried is to make the test program stop on an error then incrementally see if I could get past them one at a time. I guess both approaches could have their pros and cons. Sometimes I've even tried making the test program iteratively try different things in order to see what values can fix it. It gets pretty complicated fast.

Your best chances in RE will be if you can distill what the hardware is doing down to the simplest cases possible. For example.. I did a bunch of geometry engine RE by capturing a scene and rendering it on the DS, with options to try to bypass parts of the geometry engine with emulation. The important thing here is I still got the hardware to do part of the geometry engine work and all the rasterization work, reducing the number of variables that I have to simultaneously get right. Another example: I've emulated triangles using a mixed software/hardware approach - I'd split it into a bunch of horizontal lines. This way the edge scanning and vertical interpolation happens in software, but the horizontal interpolation and per-pixel stuff happens in hardware. It also confirmed that the engine didn't interpolate line edges with higher precision than what I can submit to (16-bits in this case since I was using texture coordinates). If it did that would have broken the whole approach.
 
Last edited by a moderator:
Levi said:
@Neelix Yes, it looks that way to me too. I can confirm that I checked it just after I killed the Drastic process though, and had not restarted it.
Click to expand...
Sounds like you ran 2 instances of it. DraStic uses op_runfbapp which attempts minimizing on pandora button press.
 
Last edited by a moderator:
so ive been playing a couple of games for hours, saving and loading numerous savestates, trying to catch a freeze... no luck, drastic performs flawlessly now :wub:

what did i change?

i disconnected the battery for like 10 min.

i switched to one-screen-display (been using simultaneous display in most cases before). maybe that makes a difference?
 
yessir!

drastic r1.4p on a 3.2.39 #872 kernel

but:

after upgrading to the current version i had a freeze as usual.

i was reading about notaz' tweaked kernel and the last_kmesg log. so i fired up the update pandora os tool as described.

someone mentioned to disconnect the battery for 5 min in that thread.

maybe its just a coincidence, but since i did that, drastic didnt fail once.
 
You must log in or register to reply here.

Similar threads

E
Future DraStic Android port: help needed
40 41 42
Replies
825
Views
202K
Nintendo
Nintendo
notaz
  • Sticky
SuperZaxxon incremental update changelog
2 3
Replies
57
Views
31K
notaz
notaz
Nintendo
Nintendo 3Ds A.k.a. Nintendo Ds2
2
Replies
36
Views
5K
kingoddball
kingoddball
sebt3
The Big List Of Official Dev Team Updates
Replies
15
Views
3K
sebt3
sebt3
E
DraStic Nintendo DS emulator teaser video
13 14 15
Replies
295
Views
74K
Kumaki
Kumaki
Back
Top