GP2X Planning My Software 3d Renderer

[Dimension]

Hi everyone! I recieved my gp2x two days ago and I absolutely love the thing it does everything I could ever want and if I didnt have 1 dead pixel on the screen it would have been perfect,lol now that I have it I want to get right on coding for the various games( mostly 3d ) to this wonderful handheld. As the system has major bottlenecks I will be designing the renderer to render as efficiently as possible while hopefully being able to generate dynamic worlds such as some of my games require. I would like to outline some ideas I have and would like to know what you guys think and would like to here your suggestions. again these are just ideas for optimization:

-eliminate the use of the y axis for games that don't require it(ex: doom) less math as I will use 3x3 matrices.

- disable looking up and down for the same reasons as doom

- Utilize blitter for faster fill rates as this takes the most processing power.

- octree and frustrum culling

- polygon partitioning which is basically a simple subdivision of the floor polygons which only fills the section of the polygon that needs to be filled(experimental in another renderer of mine)

- vertice duplication and 2d geometric transformation which is another experimental system of mine which basically transforms all coordinates on a 2d plane and then uses a set of instructions to project the coordinates in 3d by vertex duplication. Ill post a little more on this later.

- utilize second core for tasks such as rasterization and Transform and lighting.

This is all of the stuff that I can think of for now. As I am planning on writing this from the ground up planning is key. So if anyone has any ideas feel free to share.

 

[dpoarch]

Great nice to have new games. I dont have any suggestions just that I look foward to whatever you release.

 

[jlebrech]

Try pre rotating and scaling the bitmaps at the preloader stage, or even have them on the disc (faster load the second time round if theres enough space on disc).

 

[jix]

When doing perspective texture mapping you could only do it right every like 8 pixels and do linear interpolation of VU coordinates in between. (And of course detect horizontal vertical surfaces which do only need perspective texture mapping in one direction). But there might be some clever algorithms for doing perspective texture mapping that are fast enough... (i read something about interpolating 1/z and z/u and z/v or something like that linear some time ago but i don't remember)

 

[rixed]

-eliminate the use of the y axis for games that don't require it(ex: doom) less math as I will use 3x3 matrices.
- disable looking up and down for the same reasons as doom

Come on, the GP2X is not that slow !

 

[Nmn]

Actually, the gp2x is very fast. Just take a look at quake with a little touch of over clocking!

For speed:

- Over clocking, but remember to allow the user to choose (Since theres a 275 Mhz version which can over clock higher and the original version usually freezes when over clocking to 275 mhz.)

- Only use floats where absolutely required (Floats are quite slow on the gp2x. Software floating point will be much faster than hardware floating point but still avoid it. Avoid it by making things that would use float but goto a value that integers could reach 100x the max value, whereas you could have an int and multiply by 100 and divide by 100 when needed.)

- Simplify expressions (Never use math that could be pre-calculated first. You can have tables for trigonometry or if there are any things that are calculated every frame and don't need to be you could directly solve them if possible or have a variable that contains the answer for fast access.)

- MMU Hack and the other set of hacks available for memory can help you with caching things like fonts and such very fast. The MMU hack can also speed up frame buffer access apparently.

- Assembly code, if possible. ARM Assembly is something i don't know much of, but i do know that in the proper hands, assembly can speed anything up. Since texture and lighting can be quite slow, you could do it all in pure assembly if you knew it and interface it in your C code.

- Correct use of both cpus. Well managed use of both processors will definitely speed things up. Perhaps if you needed sound or you wanted to separate two major rendering procedures, this could help.

- Fast collision. Quake is a great example of fast collision and its methods are quite well. Maybe you could explore quakes source code a bit since theres so much about it thats good. Of course, you could just write a BSP-based collision system since thats all quake used.

- Fast lighting. Dynamic lights are always nice looking. Light maps that are pre-applied to textures inside of the 3D format will work but dynamic lights are possible as the commercial game payback shows. Payback is closed source, of course.

- Direct frame buffer access. I've heard, for some reason, that the MMU hack will make frame buffer access faster. While i rarely access the frame buffer directly, it seems possible since the frame buffer is in the upper 32mb, i think. I don't know. To me its a rather gray area. i think SDL for the gp2x directly accesses the frame buffer, but by using SDL your probably getting some processor overhead since SDL does a lot more than graphics.

Of course, I'm still a n00b of optimizations and the MMU hack. These are probably rather basic and I don't know if everything I said was completely true.

 

[BradN]

You want to be careful with how you use precalculated tables - if they get too large, they won't fit in the cache anymore. You should try to design the engine so any tables it uses for a given loop fit in the data cache. Don't be afraid to design the engine with different modular parts that can be replaced to see if something else performs better.

 

[j_villa1983]

Why overclock? A well programmed application on a console shouldn't need overclocking! It seems that people are using that as an answer rather than optimizing their code.

-JV

 

[BradN]

Even if you assume well optimized code, there's always that extra 30% or so you can squeeze out of overclocking. I agree it shouldn't be depended on (~240MHz is probably reasonable for this device), but it never hurts to consider the option, especially if there are different rendering qualities available.

 

[Dimension]

Hi guys thanks for your comments!

Actually, the gp2x is very fast. Just take a look at quake with a little touch of over clocking!

Well lets remember that quake uses bsp which is fast but is not suitable for my needs as I like to have some dynamic scene generation. Without bsp I believe it would be a little bit lagging. Of course the 2x is relatively fast but to run a scene with a large polygon count in software mode with some decent speed some optimizations will have to be made.

Come on, the GP2X is not that slow !

definetly not that was just an idea. But were not talking bsp here so it might be necessary with a large scene but again I don't know.


Again thanks guys1 I will get coding and see what I can come up with! Thanks!

 

[Epicenter]

- Over clocking, but remember to allow the user to choose (Since theres a 275 Mhz version which can over clock higher and the original version usually freezes when over clocking to 275 mhz.)

Note that many GP2X units cannot exceed 266, or even 240 MHz. A few crash beyond 230!

- Only use floats where absolutely required (Floats are quite slow on the gp2x. Software floating point will be much faster than hardware floating point but still avoid it.

There is no hardware floating point support on the GP2X. If it did have hardware support, it would be much faster than a software FP implementation running on an integer-based core.

- MMU Hack and the other set of hacks available for memory can help you with caching things like fonts and such very fast. The MMU hack can also speed up frame buffer access apparently.

The 'memory hacks' are really just settings to tighten RAM timings so less bus clocks trigger a read/write operation on the memory, which can provide some small speed improvements. I don't know where you'd get the impression this had an impact on caching of fonts, of all things. The MMU hack will accelerate access of the upper 32MB of memory by the ARM920T, which includes the framebuffer (hence faster drawing ability).

- Direct frame buffer access. I've heard, for some reason, that the MMU hack will make frame buffer access faster. While i rarely access the frame buffer directly, it seems possible since the frame buffer is in the upper 32mb, i think. I don't know. To me its a rather gray area. i think SDL for the gp2x directly accesses the frame buffer, but by using SDL your probably getting some processor overhead since SDL does a lot more than graphics.

Since the framebuffer is in the upper 32MB access to it by the 920T is accelerated when using the MMU hack, it will have no impact on access speed by the 940T. This acceleration will happen whether you are accessing the framebuffer directly or through SDL. Overhead through SDL should be pretty low (look at all the fuss about Linux 'stealing cycles' when it really uses nearly no CPU time), although for maximum speed ARM9 ASM is ideal for any operation that will be happening frequently or is unusually intensive.

 

[Nmn]

Yeah, i know. I'm a n00b. I still have no idea how or what memory hacks actually do, but now i kind of do.

As for Floating point, thats what i meant. Software floating points are faster because hardware floating points are emulated through the kernel.

And as for BSP, I'm sure its not impossible to whip up a rather fast dynamic collision system in its place. OK, so its not as fast. So? I think people are worried about a 3D renderer that actually has 3 dimensions more than speed. For huge worlds: say yes to culling, and maybe quick fog and clipping code can make the game run faster if theres a lot of open areas.

Good luck on this 3D renderer.

 

[rixed]

There is no hardware floating point support on the GP2X. If it did have hardware support, it would be much faster than a software FP implementation running on an integer-based core.

Certainly.

Anyway, one can almost always replace floats with integers (in games). And these, even when a processor actually have hardware floats, are often faster than hardware floats.

So the rule should be stated as : don't use floating points (soft nor hard) if you don't need them. Use fixed point instead.

 

[jix]

- Assembly code, if possible. ARM Assembly is something i don't know much of, but i do know that in the proper hands, assembly can speed anything up. Since texture and lighting can be quite slow, you could do it all in pure assembly if you knew it and interface it in your C code.

You should like never code something in assembler for speed reasons before you did some profiling. Compilers arn't that stupid. Most of the time they generate fast code and they know a lot of algorithms for common things like optimizing math expressions and especially relevant for the arm, fast division routines. So write everything in c first, do profiling, rewrite the critical ones in assembler (but always keep the c version up to date) and really check whether it speed things up or not. Doing the whole texture and lightning part in pure assembly doesn't sound like a sane idea because it makes it very difficult to maintain that part and to add new features.

 

[Dimension]

Got to agree with Jix here. Its how I wrote a simple rendrer for my pocket pc. First in C then I attempted to rewrite the lighting and backface culling code but it never worked out. Anyways I have a development environment working now and im still trying to decide what to write the renderer in. I was looking into the minimal library which can access the frame buffer directly and the major difference the 940T. But unfortunately its not cross platform and does no use the blitter on the other hand SDL uses the blitter and is cross platform but as far as I know doesent use the second processor. WHich one would you guys recommend? Anyways Thanks again guys! Ill try to make it as true a 3d engine I can without suffering in performance.

 

[Nmn]

Personally, i recommend direct access to the frame buffer using a minimal library. If you run linux, i'm sure you could somehow wire it all up... but beside that, you could make two rendering engines, one in sdl and one in a minimal library. Perhaps someone could screw with qemu-arm emulator to make it work with all of the gp2x files? Only problem is it would have 1 processor.

I've never accessed the framebuffer directly before, but my programs running slow. So along with the new support, i'm going to try some FB work and hopefully i can get it optimized.

BTW, I just recently heard of profiling, and it certainly kicks alot of ass

 

[Dimension]

i agree framebuffer access is probaly the best way to go. just a quick question about the minimal library. Im going to be using it with devcpp as for compiling it do I have to add anything different to the linker or just keep the standard sdl stuff? thanks!

 

[Lint]

writing everything in assembler would help some software acceleration, once I started to port some procedures from C to x86 assembly from a nes/master-system emulator called darcnes, but I kept the same C structure, just rewritten the calls, so anytime I could mix original C code and assembly-crafted versions as I wished, it turned out that almost everytime the compiler added some useless code to the C version, but the speed gain was something very discutible.... but x86 has only 4 almost-general-purpose register, wihle arm have 13 or 14, so I think arm's C compiler does not push every single parameter into the stack

 

[marmakoide]

Well, before focusing on assembler, MMu hack and the blitter, focus on the algorithm. When using a vector instead of a linked list (for random index insertion as instance), even the most optimised vector on an overclocked CPU is slow. 'O(1) versus O(n)' or thing like this before anything.

For a 3d renderer for GP2X, I would go first for a raycasting solution, Wolfenstein style. Then, when I reach good efficiency, I would replace the square grid world representation for something more flexible and nice like portals (Duke Nukem) or quad trees (Cube Engine). With raycasting, nice tricks like mip-mapping and skyboxes are very cheap to implement. I currently coding something like that for the GP2X...

 

[Blah]

But that wouldn't be 3D, now would it?