Which Function For Scaling/zooming?

[john4p]

I'll give it a try. If we'd just have <SDL_opengl.h> support OpenGL ES that'd make it easier though.

 

[john4p]

(double post)

 

[torpor]

It may not be so helpful for me to suggest this to you at this stage, since you're pretty committed to SDL it seems, but if you want some example code that works to get the GL ES 1.1 setup done quick and easy, check out the code in Pandora WakeBreaker: http://w1xer.at/pandora/

If you look at main.cpp you can see all the GL ES setup stuff, then just check out the Texture class for details on how to set up a quad and all that for texture displays .. WakeBreaker uses a couple textures (low-res) for the splash screen and so on - these could easily be abstracted to handle any texture size, etc. You'll need a .png/texture-file loader, however ..

 

[john4p]

It's not me who is committed to SDL - it's the application (UAE4All) which I try to add something to that is committed to SDL.
It has SDL everywhere...

I won't load a picture from a file but instead get 50 times per second an SDL_Surface* that I want to (hardware-)scale to use the full height of the Pandora's display (while keeping aspect ratio).

Thanks for the link - at least now I know how to initialize OpenGL ES.

 

[Exophase]

It may not be so helpful for me to suggest this to you at this stage, since you're pretty committed to SDL it seems, but if you want some example code that works to get the GL ES 1.1 setup done quick and easy, check out the code in Pandora WakeBreaker: http://w1xer.at/pandora/

If you look at main.cpp you can see all the GL ES setup stuff, then just check out the Texture class for details on how to set up a quad and all that for texture displays .. WakeBreaker uses a couple textures (low-res) for the splash screen and so on - these could easily be abstracted to handle any texture size, etc. You'll need a .png/texture-file loader, however ..

This thread is about scaling a soft generated framebuffer for an emulator. Something that is ideally updated at presumably 50 or 60Hz. It's a very different scenario from 2D games. The texture upload speed is a major bottleneck on this platform, even with PBOs. The only feasible option is TI's texture stream extension, and I'm not aware if that's supported on Pandora yet or not. I hear that even with that the performance is lacking from what you'd expect, although I haven't seen extensive testing on it.

From a utilization point of view, if you want 400x240 then the alleged LCD mode (which may or may not exist) is the lowest impact option because the scaling happens per-pixel and completely off the chip.

Then, the OMAP's DSS scaling is the next best option because the small framebuffer is read by the DSS and then output off the bus to the LCD at the large resolution.

Next is using the DSP to scale. This involves reading from memory at the low resolution, writing to memory at the high resolution, the DSS reading at the high resolution, and the DSS writing at the high resolution. But computationally speaking it would be close to entirely offloaded from the CPU and could be done with very low latency.

Next is using the SGX to scale. This has the same memory footprint as using the DSP, but you incur CPU overhead that the driver steals and latency overhead due to the long pipeline of the SGX's rendering.

Then there's doing it in software on the CPU (possibly with NEON code to save a few cycles). The memory footprint will be a little lighter if you can get your small framebuffer to stay in L2 cache. This is the most efficient way to stream out using the DSP - if your framebuffer isn't cached (and it shouldn't be) then you should be able to peak write bandwidth. Since the CPU will still be stalled by the memory subsystem on write outs you're going to be spending CPU time - for 320x240->640x480 at 500MHz expect to spend at least 1-2ms.

There is one more thing you can try that may or may not have any potential - scale in scanline bundles, with the result in L2 cache, and have a timer which at some point later evicts the L2 cache (hopefully by this point it'll already be off, if the L2 is configured as write-through, so it won't stall), and DMAs the result to a real framebuffer or the LCD directly. This could have a larger memory footprint but would lighten CPU time somewhat. It's probably not worth it though.

Fact is, for most things 1-2ms won't really hurt you. Just don't make your emulator always use an lq2x scaler (cough, gpfce...)

 

[Exophase]

ofbset -fb /dev/fb1 -pos 0 0 -size 800 480 -mem 307200 -en 1
fbset -fb /dev/fb1 -g 320 240 320 480 16
export SDL_VIDEODRIVER=fbcon
export SDL_FBDEV=/dev/fb1

op_runfbapp ./your_app_here

ofbset -fb /dev/fb1 -pos 0 0 -size 0 0 -mem 0 -en 0
[/code]

What it does:

• allocates OMAP DSS layer, asks video output to be 800x480 at position 0,0 (could set it to 640x480 at 80,0 instead to get centered 2x scaling of 320x240). 307200 bytes of video memory are allocated for 2 320x240 16bpp screens (for doublebuffering).
• sets video mode to 320x240@16bpp, virtual resolution 320x480 for doublebuffering.
• asks SDL to use the new layer through /dev/fb1
• runs your app and hides X (since SDL is now in framebuffer mode)
• cleans the video layer on exit

After this, you can act as if the screen was 320x240 in your code.
Hopefully in future we'll have SDL handling this transparently
*looks at paeryn's post above*

paeryn: I'm slowly documenting kernel stuff at http://pandorawiki.o...ernel_interface , could probably expose OMAP DMA for GP2X blitter like functionality.

Thanks for this information notaz. I agree that having SDL handle it transparently would be good, but if so there should be some kind of extension for maintaining aspect ratio. I don't think this is something apps should have to worry a lot about rigging themselves, especially if it might mean they're forced to blit the bars.

I have some questions:

- Does SDL's double buffering work with this?
- And if it does, will SDL wait for vsync using the proper method? If not (or no double buffering at all) then this is a good reason to not use SDL for video, for the time being.
- Is it safe to call ofbset and fbset while the program is running using system or some other method? Do they need root? (I have no idea if the PND run script is ran with root or whatever)
- What's the pixel format of the 16bpp framebuffer? Is this in any way configurable?
- Are other formats like paletted 8bpp options? Not that I personally care but someone might want them.
- Can you allocate the second scaler, say to fb2?
- And if you can, how exactly do the two interact? You said they're both overlaid on the screen. Does this mean that they're on top of /dev/fb0; for instance, could you write directly to /dev/fb0 to draw a border? If they overlap, is there any means for transparency or blending? How is priority determined?
- Can other apps steal the framebuffer from me? Could the system potentially? It'd be good to know that one day we could have a system menu which steals the framebuffer from us (perhaps pauses our program) and pops up on top of it. I want to know if this is a reason not to use it.
- Is the default refresh rate 60Hz? How can it be changed? Is 120Hz an option?

 

[Pickle]

just to mention SDL 1.3 does support the pandora opengles (although it wouldnt help with SDL 1.2, unless converted)

 

[DaveC]

What it does:

• allocates OMAP DSS layer, asks video output to be 800x480 at position 0,0 (could set it to 640x480 at 80,0 instead to get centered 2x scaling of 320x240). 307200 bytes of video memory are allocated for 2 320x240 16bpp screens (for doublebuffering).
• sets video mode to 320x240@16bpp, virtual resolution 320x480 for doublebuffering.
• asks SDL to use the new layer through /dev/fb1
• runs your app and hides X (since SDL is now in framebuffer mode)
• cleans the video layer on exit

After this, you can act as if the screen was 320x240 in your code.
Hopefully in future we'll have SDL handling this transparently
*looks at paeryn's post above*

paeryn: I'm slowly documenting kernel stuff at http://pandorawiki.org/Kernel_interface , could probably expose OMAP DMA for GP2X blitter like functionality.

Do you think this video output number could be put into an option menu for emus? This way the user could decide which method to use for the scaling rather than it being hardcoded in. So if a user wanted straight 2x in Picodrive he would set this number to 640 x 448 to get it. Fullscreeners would set to 800 x 480 etc.

For systems with multiple resolutions could these settings be put in a config such as "option1, option2, etc and have the emu use those for the various modes?

EX Picodrive (integer mode) these would be a setting in a menu or in a config file:

option 1: 320 x 224 = 640 x 448
option 2: 256 x 224 = 768 x 448
option 3: 320 x 448 = 640 x 448

Would that work or is that too hard to do?

 

[darkblu]

The only feasible option is TI's texture stream extension, and I'm not aware if that's supported on Pandora yet or not. I hear that even with that the performance is lacking from what you'd expect, although I haven't seen extensive testing on it.

Are you referring to GL_IMG_texture_stream2? It appears to be in the current drivers. But the only meaningful information I could find about it was this TI code: https://texinst1.gforgegroup.com/gf/project/gleslayer/scmsvn/?action=browse&path=%2Ftrunk%2FPackages%2FOMAP3_Graphics_SDK%2FGLESLAYER_SGXSINK_20%2Fsgxsink_api.cpp&revision=13&view=markup

 

[Exophase]

The only feasible option is TI's texture stream extension, and I'm not aware if that's supported on Pandora yet or not. I hear that even with that the performance is lacking from what you'd expect, although I haven't seen extensive testing on it.

Are you referring to GL_IMG_texture_stream2? It appears to be in the current drivers. But the only meaningful information I could find about it was this TI code: https://texinst1.gfo...=13&view=markup

Yeah that. Do you have a Pandora? If so, are you willing to write a test program using it and measuring performance?

Here's a reference to someone who used it: http://bloggingthemonkey.blogspot.com/2009/10/texture-streaming.html

 

[darkblu]

Yeah that. Do you have a Pandora? If so, are you willing to write a test program using it and measuring performance?

Here's a reference to someone who used it: http://bloggingthemonkey.blogspot.com/2009/10/texture-streaming.html

Unfortunately I'm yet as pandora-less as the best of them. But I've been using paeryn's services to run some code on the panda. Apropos, if you unspoiler the second spoiler in that post you might see interesting things about the current SGX driver edge.

Re that extension, I have the gut feeling it's YUV-centric, but I may be totally wrong about that.

 

[notaz]

- Does SDL's double buffering work with this?
- And if it does, will SDL wait for vsync using the proper method? If not (or no double buffering at all) then this is a good reason to not use SDL for video, for the time being.

No for current SDL (which runs on unmodified x11 driver now). AFAIK nobody worked on SDL for OMAPs so far..

- Is it safe to call ofbset and fbset while the program is running using system or some other method? Do they need root? (I have no idea if the PND run script is ran with root or whatever)

yes they can be called (zodttd does that), no need for root, pnd is always started with regular user rights. ofbset and fbset are just frontends for some driver ioctls, you are free to use those directly instead.

- What's the pixel format of the 16bpp framebuffer? Is this in any way configurable?

RGB565, not at the moment

- Are other formats like paletted 8bpp options? Not that I personally care but someone might want them.

Driver supports some, but I've never tried them.

- Can you allocate the second scaler, say to fb2?

Yes, but I'm thinking about reserving it for system functions like TV-out.

- And if you can, how exactly do the two interact? You said they're both overlaid on the screen. Does this mean that they're on top of /dev/fb0; for instance, could you write directly to /dev/fb0 to draw a border? If they overlap, is there any means for transparency or blending? How is priority determined?

priority is always fb0 < fb1 < fb2 (fb2 on the top, then 1 and 0). There is transparency color keying support.

- Can other apps steal the framebuffer from me? Could the system potentially? It'd be good to know that one day we could have a system menu which steals the framebuffer from us (perhaps pauses our program) and pops up on top of it. I want to know if this is a reason not to use it.

Yes, anyone can mmap any framebuffer and fight for it.

- Is the default refresh rate 60Hz? How can it be changed? Is 120Hz an option?

Currently 50 and 60 only, 120 can also be done but it's less efficient due to scanning system memory more (do you think it's worth it anyway?)

Do you think this video output number could be put into an option menu for emus? This way the user could decide which method to use for the scaling rather than it being hardcoded in. So if a user wanted straight 2x in Picodrive he would set this number to 640 x 448 to get it. Fullscreeners would set to 800 x 480 etc.

yes but it's still up to the coder to do it. And it does that 5-tap poly-phase interpolation that you hate so much.

 

[Exophase]

Unfortunately I'm yet as pandora-less as the best of them. But I've been using paeryn's services to run some code on the panda. Apropos, if you unspoiler the second spoiler in that post you might see interesting things about the current SGX driver edge.

Okay, I can help compile things for you too if you need it. It'd be best if we could talk on IRC or AIM or something. I just don't want to have to dig a lot myself because I really suck at OGL - I've been struggling big time to get double buffered PBOs working on my desktop. I had them working and moved them somewhere else and poof, white screen.

I'm really not very excited about the GL extensions available right now. No depth textures, multiple render targets, depth/stencil readback, and so on. The annoying thing is that TI actually claims you can read back depth buffer and stencil:

http://processors.wiki.ti.com/index.php/Render_to_Texture_with_OpenGL_ES

Re that extension, I have the gut feeling it's YUV-centric, but I may be totally wrong about that.

I don't see any reason why it should be tied to YUV - if it does as advertised then it should be allowing to replace textures directly and this should mean any texture format. The blog post does say "or RGB", using it for YUV conversion was just the useful application.

 

[fusion_power]

AFAIK nobody worked on SDL for OMAPs so far..

Sounds like SDL could be some kind of bottle neck if you want good performance. h34r:
Because many devs seems to use SDL to code stuff for the Pandora, it would be cool if SDL would be more optimized for the needs of the OMAP Chip.
I guess we not even bareley touched the possible Hardware Power of the Pandora yet. ^^

 

[darkblu]

Okay, I can help compile things for you too if you need it. It'd be best if we could talk on IRC or AIM or something. I just don't want to have to dig a lot myself because I really suck at OGL - I've been struggling big time to get double buffered PBOs working on my desktop. I had them working and moved them somewhere else and poof, white screen.

Thanks for the offer, I'll surely pester you with pointless code build requests. What IRC do pandoreans frequent, btw?

I'm really not very excited about the GL extensions available right now. No depth textures, multiple render targets, depth/stencil readback, and so on. The annoying thing is that TI actually claims you can read back depth buffer and stencil:

http://processors.wiki.ti.com/index.php/Render_to_Texture_with_OpenGL_ES

Well, the current extension set is not exactly earth-shattering, by I tend to follow the 'half-full' philosophy, so things like

GL_OES_vertex_half_float
GL_OES_texture_float
GL_OES_texture_half_float
GL_OES_mapbuffer
GL_OES_fragment_precision_high
GL_OES_standard_derivatives

warm my graphician's heart. I was particularly surprised to see mapbuffer on a device of this class. Then again, SGX being a tiler keeps resorces in UMA, so implementing that ext must not have been a big sweat.

Lack of depth texture is a compatibility issue more than anything - i.e. instead of the easy way of reading back the depth buffer, shadow code on the panda will have to output its own depth to (half-) float textures. Not a biggie, unforseen performance hurdles nonwithstanding.

I don't see any reason why it should be tied to YUV - if it does as advertised then it should be allowing to replace textures directly and this should mean any texture format. The blog post does say "or RGB", using it for YUV conversion was just the useful application.

Could be. I'm still looking at that TI code there. Do you happen to have any other links to resource on the subject?

 

[Exophase]

Thanks notaz! Very useful information. Hopefully Paeryn will look at SDL so we don't have to; of course I have no problem writing fbdev drivers but I'll probably keep using SDL for ALSA output, and for events if it reports things like a menu key. That is, unless the OSS emulation can be trusted. Maybe if it can I'll move to that in case Pandora ever does move to OSSv4 (okay, probably won't happen)

One other thing - is it possible to choose the taps of the polyphase interpolation or are those modeled dynamically based on the resampling factor?

 

[notaz]

That is, unless the OSS emulation can be trusted.

I still use it in PicoDrive..

One other thing - is it possible to choose the taps of the polyphase interpolation or are those modeled dynamically based on the resampling factor?

Haven't really looked into that but I know there are some registers for the filter. Currently the driver uses single static configuration bet we should be able to tune that in future.

 

[Exophase]

Thanks for the offer, I'll surely pester you with pointless code build requests. What IRC do pandoreans frequent, btw?

I don't really know yet, maybe #pandoradev on efnet and #openpandora on.. I don't know, somewhere

GL_OES_vertex_half_float
GL_OES_texture_float
GL_OES_texture_half_float
GL_OES_mapbuffer
GL_OES_fragment_precision_high
GL_OES_standard_derivatives

I don't know about mapbuffer or standard derivatives, but the formats seem really simple to support given hardware support for them.

Lack of depth texture is a compatibility issue more than anything - i.e. instead of the easy way of reading back the depth buffer, shadow code on the panda will have to output its own depth to (half-) float textures. Not a biggie, unforseen performance hurdles nonwithstanding.

Except right now float textures aren't allowed as render targets and there's no support for MRT. Lack of depth textures and lack of depth/stencil readback are two different issues in my mind, although obviously they're connected depth textures gives you a way to initialize depth from the CPU as well, which can be useful for some applications.

Could be. I'm still looking at that TI code there. Do you happen to have any other links to resource on the subject?

Sorry, not at the moment, I could try looking around but you probably have a better idea of who to ask ^^

I still use it in PicoDrive..

Okay, I might use it.. I dunno. I guess bringing in SDL isn't a huge deal when it's dynamically linked to begin with.

Haven't really looked into that but I know there are some registers for the filter. Currently the driver uses single static configuration bet we should be able to tune that in future.

I have to look at the exact algorithm, but a resampling filter should conceptually involve both interpolation and decimation: the interpolation filter should have a cutoff frequency of the source image's sampling rate in order to avoid spectral images, and the decimation filter should have a cutoff frequency of under one half the destination sample rate. For the effective filter you pick the one with the lower cutoff: for scale factors under 2 you would use the destination rate divided by two and for factors equal to or over 2 you'd use the source rate.

For a 320 to 640 or 240 to 480 the coefficients (modeled with matlab's fir1 function) look like this:

0.00011
0.23162
0.67244
0.23162
0.00011

You can see the first and fifth are so small as to almost be zero, and given the actual format of the coefficients might very well be. That, and the gain is way over 1, so I'm not sure this was really modeled that well >_>

For 320 to 800 the coefficients will look like this:

0.011299
0.244503
0.597403
0.244503
0.011299


What is it using right now?

 

[skeezix]

Exo - sdl does capture the keys nicely, just oddly ie. Pandora is left-alt or somesuch for example, I forget offhand. Easy enough to determine, or look in pnd-io-evdev in libpnd to get a mapping in evdev terms and you can do it up in sdl. I have sdl in a few apps so could paste a snippet if you need

Once sdl gets tweaked up, a lot of apps are going to benefit

sdl on fb and sdl on gles will be interesting. I did a couple lame quick hacks and even just free scaling was worth a couple fps but plain sdl actually performs not too badly when naively used

jeffPhone

 

[DaveC]

Do you think this video output number could be put into an option menu for emus? This way the user could decide which method to use for the scaling rather than it being hardcoded in. So if a user wanted straight 2x in Picodrive he would set this number to 640 x 448 to get it. Fullscreeners would set to 800 x 480 etc.

yes but it's still up to the coder to do it. And it does that 5-tap poly-phase interpolation that you hate so much.

Is there no way to avoid the blurryness of this then? No way to shut off the interpolation even if you are just doing a pixel double?

If that is the case then there should be an option not to use this or maybe a differeent method altogether should be used. What would be the advantage of using this if all you get in the end is something that reduces contrast and makes everything fuzzy. If someone wanted to use other filters (phosphor 2x etc) with this wouldn't that not work? Filtering a filter wouldn't look so hot.

Are there better options out there if you want the sharp graphics?

Also Mupen64 has a config file where you set the frame that the graphics are scaled to. If you set at 640 x 480 you get 2x (for bitmap, 3D is full res). It is kind of nice as user can set the scaling to whatever they like.