Beautiful landscapes are prettier than portraits

[Granitehead]

I wouldn't call that test image "hard core".

It's an image specifically designed to make compression very very hard. That's what makes it hardcore.Using the ImageMagick compare utility, most of the images are just kind of splattered with red, lines tracing objects and letters where there are hard colour borders mostly, and then the hardcore image is almost solid red meaning just about every single pixel was changed to compress it.

I was fully aware of the reason he called it hardcore, but I think it would be possible to create images which are much harder to compress, depending on how the algorithm works. I got the impression it never analyses the image as a whole because it has to compress it as it receives it (if it doesn't, it introduces too much latency, I'd think). So for an actual hardcore test, that should be taken into account. For one, the gradient should spread over all of the image. If the algorithm always only sees a small part at the time it's much harder to compress multiple steeper gradients.

 

[Exophase]

Also for emulation of classic consoles there should be no problem to bypass the rotation chip completely, use the screen losslessly in portrait mode and do the scaling/rotation manually on the second core.

Similarly, anything GPU-based should not be too hard to convert so that it renders everything to an uncompressed portrait screen.

What I think is undesirable for legacy PNDs (and for a lot of emulators there's not that much incentive to do a special Pyra version unless you make use of the extra buttons) is for the OMAP5 to scale up the display to 800x480, and the SSL chip to then scale and rotate that to 720x1280. That'd cause scaling to be applied twice, which will make the picture look more blurry than it would otherwise.

It would be better if, where the Pandora scaler is used, one of two things happens:

1) The output is scaled all the way from the surface resolution to the output resolution, so if the PND wants something like 400x240 that's what the input to the SSL chip becomes. The problem with this approach is that the display on Pandora is really fb0 + fb1, and even if you only write to fb1, unless it takes up the whole screen fb0 will still be visible. So for example, if you had 320x240 instead, and were maintaining correct aspect ratio, there'd be black bars on the right and left and those bars would be fb0. I think op_runfbapp actually sets fb0 to black (if you run without it you get fb1 overlaid on whatever you were already running instead), so maybe that can be detected as a special case (until the app opens fb0) and an enlarged virtual framebuffer with the non-visible parts blacked out could be used.

2) The fb driver is adjusted to output to 1280x720 and the chip is only used for rotation. You'd be hit with the compression, but that's minor in comparison, especially since it'd handle highly scaled stuff better than usual anyway.

But either one of these is tricky because the PNDs perform low level memory mapping with their buffers, so double scaling may be unavoidable. Maybe things can be different with an updated SDL at least. Not sure how many apps are accessing fbdev directly, outside of notaz's emulators and DraStic, which will both probably get native Pyra ports.

There's also a question of what the SSL chip's scaling is like, vs the OMAP5's scaling (which is probably the same sort of polyphase FIR filter as on the OMAP3, but I'd have to dig through the manual to confirm)

 

[EvilDragon]

I wouldn't call that test image "hard core".

It's an image specifically designed to make compression very very hard. That's what makes it hardcore.Using the ImageMagick compare utility, most of the images are just kind of splattered with red, lines tracing objects and letters where there are hard colour borders mostly, and then the hardcore image is almost solid red meaning just about every single pixel was changed to compress it.

I was fully aware of the reason he called it hardcore, but I think it would be possible to create images which are much harder to compress, depending on how the algorithm works. I got the impression it never analyses the image as a whole because it has to compress it as it receives it (if it doesn't, it introduces too much latency, I'd think). So for an actual hardcore test, that should be taken into account. For one, the gradient should spread over all of the image. If the algorithm always only sees a small part at the time it's much harder to compress multiple steeper gradients.

 This is the exact algorithm used in the chip.

It uses the full image and compresses it.

This doesn't add a latency - as mentioned, the OMAP5 can render 500fps with FullHD, so a lot more with 720p.

Writing the image into the framebuffer only takes a couple of ms.

What I think is undesirable for legacy PNDs (and for a lot of emulators there's not that much incentive to do a special Pyra version unless you make use of the extra buttons) is for the OMAP5 to scale up the display to 800x480, and the SSL chip to then scale and rotate that to 720x1280. That'd cause scaling to be applied twice, which will make the picture look more blurry than it would otherwise.

Well, our current idea is to use the Solomon chip similar to a computer monitor:

It offers some fixed resolutions and automatically scales and rotates these to 1280x720.

However, it's up to the dev / user to set these.

A developer could create a game for 800x480 and set the resolution to that.

It will then be scaled by the Solomon chip.

However, he can also set the resolution to 1280x720 and use the OMAP5 scaler to scale his 800x480 screen up.

It's up to him.

That's probably the most proper inclusion.

The OMAP5 has a scaler for each overlay, so you could use 3 overlays with different scalers and let the Solomon Systech render that screen incl. overlay as 1280x720.

Pretty useful if you run windowed apps, I think.

A user could set his X desktop to 800x480, run a windowed GameBoy emulation (scaled by OMAP5) and the Solomon chip would still scale the full desktop to 1280x720.

No idea if anyone would ever want to do that (before I run my desktop at 800x480, I'd run it at 1280x720 and change the font and icons sizes ), but this would probably come as close as possible to a normal Linux PC behaviour.

 

[notaz]

as mentioned, the OMAP5 can render 500fps with FullHD

Ugh, where did you get that number? I could only reach ~130fps (meaning you spend half of one CPU core just to update the screen), one of the reasons I don't want fullhd even if it was portrait and had no rotation problem.

 

[Exophase]

A user could set his X desktop to 800x480, run a windowed GameBoy emulation (scaled by OMAP5) and the Solomon chip would still scale the full desktop to 1280x720.

Yes, and it wouldn't look nearly as good as if the desktop were 1280x720 and the emulator was scaled to fit in the desktop. If we're talking about running stuff in windows it'd be using X11 and this could at least be abstracted, although it'd probably end up using the 2D accelerator for scaling instead of one of the overlays.

Do you know if the SSL chip can handle scaling an image that doesn't take up the whole screen (for aspect ratio purposes), with some fixed color displayed outside the image area? If it can do this then the normal op_runfbapp case where /dev/fb0 isn't opened could be emulated by making the SSL chip perform all of the scaling directly instead of the OMAP5.

Ugh, where did you get that number? I could only reach ~130fps (meaning you spend half of one CPU core just to update the screen), one of the reasons I don't want fullhd even if it was portrait and had no rotation problem.

This is for copying from cached memory to something fbdev allocated with write combining on and cache off? And you're using 2x32-bit DDR3 @ 533MHz? Sounds like yet another case of TI making it hard for the CPU to get anything but a small percentage of available RAM bandwidth.. :/

 

[EvilDragon]

as mentioned, the OMAP5 can render 500fps with FullHD

Ugh, where did you get that number? I could only reach ~130fps (meaning you spend half of one CPU core just to update the screen), one of the reasons I don't want fullhd even if it was portrait and had no rotation problem.

Ah, sorry, was that for some slower resolution...?

Thanks for the correction.

(Probably too much stuff flying around inside of my head )

Well, still, 130fps is fast enough

 

[Exophase]

Well you said earlier you could transfer data for a 1280x720 screen to the SSL chip at 500fps, but I think that isn't really right either. 4x1.5Gbps should allow a max rate of ~271fps for 1280x720, or about 3.7ms per transfer.

I don't expect there to be a lot of latency beyond this, since the compression and scaling having to be nearly realtime in principle since they must interface the input and output directly (respectively) since there's no RAM to buffer complete images. Performing rotation in addition to this is also trivial, since everything is probably loaded into a fast SRAM block buffer you just flip some address bits around. I doubt the chip is going to do separate framebuffer passes for any of its effects processing either since it'd mean decompressing and recompressing the framebuffer every time. Or at least I hope it doesn't do that.

So unless the did something really wrong, the processing overhead should add well under 1ms to the < 4ms it takes to transfer the image, but it'd still be good to confirm this somehow.

On a similar note, I'd be very curious to know what kind of latency can be attained from the touch interface. You could measure a combined touch + video + additional processing latency with a simple test and a decent video camera.

 

[WizardStan]

http://rebirthofxeen.com/files/Compressordiffs.zip

I used ImageMagick to create some image diffs, simple subtraction: black means there's no difference while brighter means more of a change. Some, like hardcore, and the intense pixel art, produce obvious patterns you can see while others, like slashdot, look solid black. Most have at least some spotches where you can see a difference, pick out borders and such.

I can also run the diffs through the contrast-stretch function so the differences will be brighter, but I'm not sure it'd be any more useful than the compares _wb_ already posted, they'd be the same images just a little more colourful.

 

[Rockthesmurf]

If it is of any help, I can provide a build of KAMI RETRO that asks for a 1920x1080 frame buffer and is packed with 1920x1080 assets (the assets origin is from 3D meshes with no textures, just lighting, so can be rendered out at any resolution).

 

[God Ginrai]

Ok, so I've had time to look through them finally. All of my findings here are without zooming.

Out of the 3 images I had asked EvilDragon to try from Iconoclasts, only the desert one was uncompressed, and therefore useful.

• Iconoclasts: There was degradation in the background, especially on the rocks.
• Knytt Underground: I noticed some problems in the vines.
• GTA: The only problem I noticed here was on the character's face.
• Pixel Art: There was a lot of problems in the upper right, near the Tardis and the Portal display.
• Pixel Art 2: Most obvious ones are the top of the tickets building and the toboggan run letters; there are also some problems with the Winterland letters, the windows on the chocolate factory, and some of the awnings.
• Hardcore Test: There was really obvious problems here, but that's understandable since this is supposed to be a worst-case scenario.
• Slashdot: I didn't notice anything in particular.
• Gradient: I didn't notice anything in particular.
• Skyrim: I didn't notice anything in particular.
• Cave Story: I didn't notice anything in particular.
• Cave Story Village: I didn't notice anything in particular.

As for games like GTA and Skyrim, I believe the reason that they tend to look better than the others are because due to their realistic art-styles, they have lots of colors that can be barely noticed. This is also why you tend to notice artifacts less in JPEGs of photos rather than JPEGs of sprite art or other things.
I don't know why I don't particularly see anything wrong with Cave Story. My guess is that the algorithm can take advantage of the tiling that Cave Story uses?

-God Ginrai

 

[levi]

Most of all those diffs convinced me that pixel differences won't matter to pixel art which is generally done in lower resolution than the diffs shown, or at least in 256 or fewer colour pixels, which will likely subsume such localised diffs. And don't particular matter to high-res 3d graphics, given how many stages of texture mapping and shading they've been though before the hit the screen anyway.

 

[Exophase]

I don't know why I don't particularly see anything wrong with Cave Story. My guess is that the algorithm can take advantage of the tiling that Cave Story uses?

More like lack of color diversity due to using small palettes for sprites like 16-bit era consoles.

The pixel art ones would normally have that benefit too but they have a lot of overdraw going on.

 

[God Ginrai]

Most of all those diffs convinced me that pixel differences won't matter to pixel art which is generally done in lower resolution than the diffs shown, or at least in 256 or fewer colour pixels, which will likely subsume such localised diffs..

 I feel exactly the opposite. There are a lot of HD 2D games coming out that could make their way to the Pyra. Pier Solar HD, for example. And who knows if Konjak might be able to bring any of the games he's coded to the platform. (Sure, the early ones are MMF Fusion, so that's a no, but he's been using some form of construct lately, and IIRC, that can be ported to Linux)

The pixel art ones would normally have that benefit too but they have a lot of overdraw going on.

Overdraw?

-God Ginrai

 

[levi]

Overdraw, as in things being drawn over the top. Problem that causing is that it can bring together two colour palettes together in irregular ways, and make the compressor track more colours at once.


That said, that Cave Story scene didn't seem to give it much trouble. The only obvious glitches are in repeating tiles in the background.

 

[FaeMinx]

The differences in all the tests are minor enough for me to not be concerned. Not that I'm a voice of authority, but I see no real issues with using this solution...

...but I'm still waiting to hear more on the potential double scaling issues...

 

[EvilDragon]

Well, and any games using OpenGL ES could deactivate the rotation and use GL to rotate uncompressed.

 

[WizardStan]

So basically the scaling/compression/rotation is only required for old apps and quick ports: one point of "optimization" would be to turn off the rotation and render everything to portrait manually, if it's required. I'm going to guess that for the vast majority of games and apps it won't be visible unless you're looking for it.

 

[_wb_]

So basically the scaling/compression/rotation is only required for old apps and quick ports: one point of "optimization" would be to turn off the rotation and render everything to portrait manually, if it's required. I'm going to guess that for the vast majority of games and apps it won't be visible unless you're looking for it.

In some ways this situation reminds me of how it currently is on the Pandora: by default, you get only 16 bit color so if you don't explicitly use something like notaz' SDL, you have a limited color depth with possibly visible artifacts (e.g. on gradients). For the vast majority of games and apps it does not matter much.

Only in this case the compression is probably less visible than that. Especially if you keep in mind that this is a 5" screen. If you use the hdmi video out to a big FullHD TV or beamer, the compression does not apply because no rotation is needed. So this is certainly something I can live with.

I am more worried now about the extra power consumption of this thing -- an additional 100mW is not insignificant. Of course, if the chip also has power saving features, then maybe the net effect could be neutral or even beneficial. I would be interested in some information about the power aspect.

 

[sebt3]

Well, and any games using OpenGL ES could deactivate the rotation and use GL to rotate uncompressed.

If doing that how X11 will react ?

Even when displaying in GLES, the events come throw X11. so I bet the coordinate might be rotated. But just be sure it doesnt get truncated...

 

[ptitSeb]

Well, and any games using OpenGL ES could deactivate the rotation and use GL to rotate uncompressed.

If doing that how X11 will react ?

Even when displaying in GLES, the events come throw X11. so I bet the coordinate might be rotated. But just be sure it doesnt get truncated...

Souldn't have any impact. As far as X11 is concerned, screen is Landscape, and so is the digitizer and mouse imput. Rotation is indepednat of X11 if I understood correctly.