Beautiful landscapes are prettier than portraits

[andys]

Notaz, have you tried things like

http://omappedia.org/wiki/Bootargs_for_enabling_display

? I know it's OMAP4 specific, but it seems to give an example of what the end goal is - assuming that omapfb is fine for what you want in terms of framebuffer. It seems to be the right thing. The alternative thing to look at is potentially any phones that support hdmi with the omap4 and see what they do. 

My big concern with a lower resolution screen is that it makes the Pyra less 'exciting' to 'normal' users. At the moment I have a Nexus 4, and the resolution is fine, but people are automatically going to compare, and I think a saving of maybe €50 is not going to be enough to cover it. I also think even if we think the 720p screen is generally fine, we should consider trying to get the TILER to work properly - without it we'll always have latency and the compression sounds worrying.

I'm pretty much going to buy it no matter what we end up with, but I think it might hurt it's marketability. The OMAP5 is a nice chip and has plenty of power on its own, but I think that if we cut the spec too much people will see it as underpowered.

 

[Letalis Sonus]

HD Ready just ensures that the screen has at least 720 (vertical) pixel lines, there are a lot of weird resolutions on those panels.

Fun fact: Due to overscan still being applied for HD material for no valid reason at all, even a "native" 720p (or FullHD) TV will never give you the 1:1 pixel playback you crave, unless you feed it from an external device and manually disable it in the device settings.

Additionally, the closed source video drivers love to automatically downsize everything when connected via HDMI to compensate that, so by default the desktop of a connected PC is being downscaled just to be upscaled again by the TV, and most people don't even notice it - awesome, right?

 

[notaz]

Notaz, have you tried things like

http://omappedia.org/wiki/Bootargs_for_enabling_display

? I know it's OMAP4 specific, but it seems to give an example of what the end goal is - assuming that omapfb is fine for what you want in terms of framebuffer.

omapfb is no longer used so all these options are irrelevant, and DMA-based rotation doesn't work on high resolutions.

 

[_jr_]

Thanks for the update. Not being able to rotate efficiently and transparently on the OMAP is slightly disappointing, and both the compression and the 1 frame latency of the Solomon solution seems suboptimal (and there will probably be cases where both are noticeable), but I can live with that. Actually I could also live with a 1026x600 display.

 

[SNESFAN]

Man, I sure called this one didn't I? Good thing it wasn't just me being a derp and I was fighting a mountain sized problem with my project on the shield which ran into the same problem. Hopefully I helped identify a problem before it became a huge problem later down the road.

So 720p display doesn't look bad at all on 5 inches. Plenty enough for a sharp desktop environment and not having to resize or move dialog windows around like is the current solution.

Funny you mention the rift as having the same issue and having to produce their own. Do you have contact with palmer? Would be interesting to see if he would be able to sell the displays he's undoubtedly going to be manufacturing or paying a company to produce. Or maybe they were able to overcome this problem with their product? It does sound as if heir target is a low latency, low cost 1080p landscape as well so I don't think it would hurt to inquire with him, especially if he had interest in the Pandora in its early days. Its maybe a lot harder now to do external business after being bought by Facebook, but the fact still remains both you and them are essentially looking for the same thing.

EDIT: damn forum ate my formatting

 

[EvilDragon]

Funny you mention the rift as having the same issue and having to produce their own. Do you have contact with palmer? Would be interesting to see if he would be able to sell the displays he's undoubtedly going to be manufacturing or paying a company to produce. Or maybe they were able to overcome this problem with their product? It does sound as if heir target is a low latency, low cost 1080p landscape as well so I don't think it would hurt to inquire with him, especially if he had interest in the Pandora in its early days. Its maybe a lot harder now to do external business after being bought by Facebook, but the fact still remains both you and them are essentially looking for the same thing.

 I'm in pretty good contact with him, writing every few days and exchanging our ideas

They are aiming for a display with a higher resolution than 1080p, but it's also bigger than 5", so it won't help us.

Thanks for the update. Not being able to rotate efficiently and transparently on the OMAP is slightly disappointing, and both the compression and the 1 frame latency of the Solomon solution seems suboptimal (and there will probably be cases where both are noticeable), but I can live with that. Actually I could also live with a 1026x600 display.

The 1 frame latency is a theoretical MAX. In reality, it will probably be a LOT lower, most probably only a few ms.

About compression:

I don't know exactly how their compression works, but when their algorithm is pretty good, it should not be visible.

Try the following:

Open as many windows as you like on your desktop (with a lot of text and small details).

Make a screenshot and save it as BMP (uncompressed) and as PNG (set to maximum compression).

Don't change anything else (i.e. bit depth, etc.) and compare the sizes.

The PNG is usually about 25 - 30% the size on a normal desktop.

And it compresses purely mathematical, so totally lossless.

If their algorithm is smart enough to do something like that, then you will have a lossless compression most of the time.

And even if it is a lossy compression - if it's smart enough to keep text, etc. sharp and clear, you won't see it's compressed.

I have no idea how their compression works, but we're gonna find out.

 

[SNESFAN]

Funny you mention the rift as having the same issue and having to produce their own. Do you have contact with palmer? Would be interesting to see if he would be able to sell the displays he's undoubtedly going to be manufacturing or paying a company to produce. Or maybe they were able to overcome this problem with their product? It does sound as if heir target is a low latency, low cost 1080p landscape as well so I don't think it would hurt to inquire with him, especially if he had interest in the Pandora in its early days. Its maybe a lot harder now to do external business after being bought by Facebook, but the fact still remains both you and them are essentially looking for the same thing.

 
I'm in pretty good contact with him, writing every few days and exchanging our ideas


They are aiming for a display with a higher resolution than 1080p, but it's also bigger than 5", so it won't help us.

I thought they were using a 5.6" display which I'd imagine could be shoehorned into the design of the pyra if required.

But anyway, sounds like you're already on top of it. Good deal, and good luck.

sidenote; nerdgasm on speculated increase in already awesome specs on the rift =D

 

[doragasu]

I know little about image compression, but PNG doesn't always achieve good compression rates (and sometimes it achieves compression rates much higher than that). Usually it compresses poorly photographs (you should use JPG for that) but achieves high rates when compressing things with plain colours (like diagrams, or a screenshot of this forum).

Also I think PNG is a relatively CPU intensive algorithm, I don't think the algorithm that chip applies is as good as PNG (as it has to compress the data on the fly, in real time).

I'm a bit worried about the compression, I hope it doesn't cause image artifacts or other kinds of problems (frame drops, delays, etc).

 

[bzar]

Ummm... PNG does (or at least supports) lossless data compression (as you stated). So it stands to reason no artifacts there. Try a lossy format like JPEG to see how visible lossy compression is.

I should read what you write before replying

 

[EvilDragon]

I know little about image compression, but PNG doesn't always achieve good compression rates (and sometimes it achieves compression rates much higher than that). Usually it compresses poorly photographs (you should use JPG for that) but achieves high rates when compressing things with plain colours (like diagrams, or a screenshot of this forum).

Exactly!

The more complex your picture is, the more you need a lossy compression (which would be JPG).

So with mostly static images (desktop, etc.) or images that have a lot of the same colors (most emulated consoles and games), lossless compression is easily possible, so no artifacts, etc.

Only very complex images (photographs with many many details, etc.) will need compression. But even then, a little lossless is barely visible.

And when gaming and you have fast moving images with many details, you won't see that as well.

I don't know how well their algorithm works, we'll see that, but if it's good, then you won't see that.

Also I think PNG is a relatively CPU intensive algorithm, I don't think the algorithm that chip applies is as good as PNG (as it has to compress the data on the fly, in real time).

There's a dedicated chip in there that does the compression, so that's not an issue. 

With dedicated chips, you can easily do stuff like that in realtime.

 

[_wb_]

There are lots of ways to do image compression. I do have some clue about it - I developed a completely new image compression format myself some time ago - a rather complicated and computationally expensive one, but in terms of compression it performed quite well (at least better than PNG for lossless and depending on the quality metric, better than JPEG, JP2 and WebP for lossy).

Let's not speculate about potential compression artifacts before we know more about what kind of compression it is. If the compression is needed because of hard memory constraints, then it needs to have a guaranteed compression rate and thus it has to be lossy; whether or not it is lossless on realistic (low-entropy) images depends on the compression method -- it could in theory do something like simple lossless RLE with a fallback to something else if the lossless compression does not suffice. It could be based on simple color quantization (just throwing out some bits from the perceptually least important channels, perhaps after converting to some other color space like YUV), or some JPEG-style DCT transformation which reduces accuracy in the high-frequency components, or maybe something else entirely.

It's also not necessarily true that compression artifacts are by definition a bad thing. E.g. overshoot artifacts can make static images look sharper and can help to reduce ghosting in moving images.

 

[Rockthesmurf]

This all sounds very familiar, the first iPhone had a portrait screen, if you had a landscape game you can request the OS give you a landscape frame buffer, which it did, but it completely killed performance, so instead you had to ask for a portrait display and then handle rotation yourself (so rotate graphics output, rotate touch screen input, rotate accelerometer axis, etc.) that was all fine, but you still couldn't completely win, if the system presented any dialogues (system volume overlay, pop up messages, etc.) it would display them in portrait, which would be 90 degrees flipped from how your game was running. Luckily this got resolved in a later iOS or iDevice version, and can now more or less be ignored.

On Android; you don't have to use OpenGL ES, you can render with Java using other canvas style APIs, I don't know whether these ultimately have a GL backend, or whether the application renders to a buffer, and the surface flinger then takes care of getting it onto the screen (taking into account rotation, and presumably using GL to do cheap rotation).

For the Pyra, is it possible to direct all draws/frame buffer writes to a piece of memory that is 1920x1080, and then on vsync upload this a GL texture, and render rotated by 90 degrees? I have no idea how Linux works and whether this sort of thing is possible. Furthermore, I don't know offhand what the cost would be of uploading the 1920x1080 image to a GL texture (I am guessing how ever long it takes to move 8MB of data around, possibly using DMA) and how that compares to any current solutions that are available.

 

[WizardStan]

Solomon Systech owns three patents for image compression which are all basically the same. Their algorithm claims it can be lossless or lossy, and fewer bits can be compressed by quantizing the colours. What probably happens is they attempt their lossless compression first, and if it doesn't get good results it has some kind of alternate algorithm for figuring out which bits to drop off to make compression better.

For some reason I am remember reading something about quantizing over a 4 pixel vector, but I can't remember where I read that or if it is actually anything to do with the SSL compression method.

 

[God Ginrai]

And even if it is a lossy compression - if it's smart enough to keep text, etc. sharp and clear, you won't see it's compressed.

For me, if the game is misrepresented, then this is a big negative for me.

I have no idea how their compression works, but we're gonna find out.

Good, I hope we find out it's lossless.

-God Ginrai

 

[Exophase]

It could be based on simple color quantization (just throwing out some bits from the perceptually least important channels, perhaps after converting to some other color space like YUV), or some JPEG-style DCT transformation which reduces accuracy in the high-frequency components, or maybe something else entirely.

The patents describe color quantization which works like this: http://en.wikipedia.org/wiki/Color_quantization If there are few enough unique colors in a block it'll inherently be lossless, there's no need to support some other algorithm.

It probably will work well for most of a simple GUI image, but I don't agree that with games they'll move so fast it won't matter, games can have parts of the screen that are relatively still too, and contain a lot of complexity.

 

[_jr_]

I think in terms of 'standard' Linux software architecture Wayland is solving (or at least trying to solve) these issues.

Can someone please explain to me how this Solomon chip could rotate the image in less than one frame? Isn't it sitting between the MIPI out of the SoC and the MIPI in of the display? Is the transfer protocol flexible enough that the rotator actually has data to send to the display before it received (more or less) the whole frame from the SoC?

 

[EvilDragon]

It probably will work well for most of a simple GUI image, but I don't agree that with games they'll move so fast it won't matter, games can have parts of the screen that are relatively still too, and contain a lot of complexity.

Well, it's a frame-based compression, not with moving images.

So as long as the single image would still work mostly lossless, you probably wouldn't even see that.

I'm not sure they're only doing color quantization, some vector or additional mathematics compression will probaby also be used together.

In combination it could mean:

See if you can compress it lossless to fit into the buffer.

If not, do color quantization and compress it mathematically so it fits into the buffer.

Chances are you won't even need to reduce a huge amount of colors to fit the buffer... in most cases, it could be that not a single color needs to be skipped.

 

[EvilDragon]

Can someone please explain to me how this Solomon chip could rotate the image in less than one frame? Isn't it sitting between the MIPI out of the SoC and the MIPI in of the display? Is the transfer protocol flexible enough that the rotator actually has data to send to the display before it received (more or less) the whole frame from the SoC?

That's easy: A display shows 60fps, whereas the framebuffer can do e.g. 500fps.

So writing into the framebuffer is way faster than the display needs to show a frame.

Each about 16ms, one frame is being drawn.

If the framebuffer can do 200fps (it actually can do more), it takes about 5ms until the image is ready for pickup in the buffer by the display.

Rotation might take additional 1 - 3ms, so the rotated image would be ready in about 6 - 8ms.

You don't really lose much here

 

[rohezal]

A fixed image size to fit into the buffer sounds like jpg or an other "2 base color per block" algorithm to me. Or an edge image at full resolution and a lower resolution color image.

 

[EvilDragon]

A fixed image size to fit into the buffer sounds like jpg or an other "2 base color per block" algorithm to me. Or an edge image at full resolution and a lower resolution color image.

JPG has different image sizes, as it mixes lossy and lossless compression as well.

You set the quality and you will get a JPG with different sizes based on the image content.