Hi,
as we don't know yet if our current display will ever be working with proper speed rotating it with TILER, I've searched for some solutions.
Solomon Systech offers a MIPI bridge chip that can do stuff like rotation, scaling, color corrections, power saving, etc. That would be rotating without using the main system, so without any loss.
I've contacted them (find the datasheet of the chip attached). The chip has a pin connection for the OMAP5 for tear-free rotation.
The downside is: That chip can only do rotation up to 720x1280, so our display would not work.
BOE (huge LCD manufacturer) are sitting in the same building (in Germany), and they already sent me a datasheet for a 5" 720x1280 display (has 35ms response time though).
I've checked Panelook and apparently they also have a 5" 720x1280 display with 25ms response time.
So that display together with the Solomon Systech IC should solve the rotation issue - in case we can't get it to work with the LG display.
The display also needs a touchscreen, but as it's very similar to the LG one in size and the touchscreen will not have a frame but will glued to the LCD, it should work fine with a minor modification.
Let me know what you think about that :)
Am 29.03.2014 um 21:23 schrieb Michael Mrozek:
Hi,
as we don't know yet if our current display will ever be working with proper speed rotating it with TILER, I've searched for some solutions.
Solomon Systech offers a MIPI bridge chip that can do stuff like rotation, scaling, color corrections, power saving, etc. That would be rotating without using the main system, so without any loss.
I've contacted them (find the datasheet of the chip attached). The chip has a pin connection for the OMAP5 for tear-free rotation.
The downside is: That chip can only do rotation up to 720x1280, so our display would not work.
BOE (huge LCD manufacturer) are sitting in the same building (in Germany), and they already sent me a datasheet for a 5" 720x1280 display (has 35ms response time though).
I've checked Panelook and apparently they also have a 5" 720x1280 display with 25ms response time.
So that display together with the Solomon Systech IC should solve the rotation issue - in case we can't get it to work with the LG display.
The display also needs a touchscreen, but as it's very similar to the LG one in size and the touchscreen will not have a frame but will glued to the LCD, it should work fine with a minor modification.
Let me know what you think about that :)
a) Cool chip! But we have to find out how to initialize/control a display *behind* this chip. I.e. how we can write a driver that tunnels the panel driver commands. AFAIK the MIPI panel drivers assume that there is no proxy in between and send directly to the OMAP5.
b) the chip (128 pin QFN in 16x16mm package) can easily be mounted on the display adapter board, i.e. we don't have to change the main board of CPU board.
Anyways we most likely need a modified adapter if we replace the panel with a different one if the LG panel goes out of production.
c) personally I would prefer 1080x1920 over 720x1280
My now retired notebook from 2006 already had 900x1440 (on a 15 inch screen) and the new replacement has 1600 x 2560 on 13 inch.
Using VNC on this machine to remotely access some 1200x1920 screen or watch photos is amazingly brilliant...
So in summary, it is possible to see the difference between standard and high resolution displays.
This raises the question is what is more important (if we can't get both):
ultra-fast rotation or ultra-high resolution.
Personally, I would clearly choose ultra-high resolution (because I am not a heavy use gamer and therefore speed is of lower concern).
But that is my personal opinion...
d) we should really find out (e.g. ask on e2e.ti.com) why the OMAP5 rotator is slower than expected. It may just be that something else that is not optimally configured (e.g. DRAM caching?) if it was designed for exactly this purpose.
BR, Nikolaus
Am Sun, 30 Mar 2014 18:41:35 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Hiya,
a) Cool chip! But we have to find out how to initialize/control a display *behind* this chip. I.e. how we can write a driver that tunnels the panel driver commands. AFAIK the MIPI panel drivers assume that there is no proxy in between and send directly to the OMAP5.
As far as I know the full stream goes always through that chip, so there's no direct connection between the Panel to the OMAP5 in this case, but from the Panel to the IC to the OMAP5.
Regarding the BOE display: I already found the datasheet of the driver IC as well, with all MIPI stuff included.
So in case we really switch the panel, we have all we need.
b) the chip (128 pin QFN in 16x16mm package) can easily be mounted on the display adapter board, i.e. we don't have to change the main board of CPU board.
Great :)
Anyways we most likely need a modified adapter if we replace the panel with a different one if the LG panel goes out of production.
That's what I hoped for, yes.
c) personally I would prefer 1080x1920 over 720x1280 My now retired notebook from 2006 already had 900x1440 (on a 15 inch screen) and the new replacement has 1600 x 2560 on 13 inch. Using VNC on this machine to remotely access some 1200x1920 screen or watch photos is amazingly brilliant...
True, but that's already double the size of our panel with only 1/2 higher resolution ;)
This raises the question is what is more important (if we can't get both):
ultra-fast rotation or ultra-high resolution.
Personally, I would clearly choose ultra-high resolution (because I am not a heavy use gamer and therefore speed is of lower concern).
I'm not hoping for ultra-fast rotation. But right now, it's ultra-slow. (notaz can give more details about it).
Additionally, rotation eats up CPU power as well right now (100%)
d) we should really find out (e.g. ask on e2e.ti.com) why the OMAP5 rotator is slower than expected. It may just be that something else that is not optimally configured (e.g. DRAM caching?) if it was designed for exactly this purpose.
There's already a thread with no solution yet: http://e2e.ti.com/support/omap/f/885/p/322408/1121877.aspx
Hi,
Am 30.03.2014 um 19:39 schrieb Michael Mrozek:
Am Sun, 30 Mar 2014 18:41:35 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Hiya,
a) Cool chip! But we have to find out how to initialize/control a display *behind* this chip. I.e. how we can write a driver that tunnels the panel driver commands. AFAIK the MIPI panel drivers assume that there is no proxy in between and send directly to the OMAP5.
As far as I know the full stream goes always through that chip, so there's no direct connection between the Panel to the OMAP5 in this case, but from the Panel to the IC to the OMAP5.
I mean the problem that the panel driver in Linux assumes that it can directly send DCS commands to the panel. Therefore they arrive at the interconnect chip which expects them in a special format to forward them to the panel or it will try to interpret them directly. So I am simply not sure if the software architecture of panel drivers is prepared for this situation. It can certainly be done, but may be a major task (for kernel drivers).
Regarding the BOE display: I already found the datasheet of the driver IC as well, with all MIPI stuff included.
So in case we really switch the panel, we have all we need.
b) the chip (128 pin QFN in 16x16mm package) can easily be mounted on the display adapter board, i.e. we don't have to change the main board of CPU board.
Great :)
Anyways we most likely need a modified adapter if we replace the panel with a different one if the LG panel goes out of production.
That's what I hoped for, yes.
That is what MIPI displays are good in :)
c) personally I would prefer 1080x1920 over 720x1280 My now retired notebook from 2006 already had 900x1440 (on a 15 inch screen) and the new replacement has 1600 x 2560 on 13 inch. Using VNC on this machine to remotely access some 1200x1920 screen or watch photos is amazingly brilliant...
True, but that's already double the size of our panel with only 1/2 higher resolution ;)
But I hold the display closer to my eyes so that I see the pixels better :)
This raises the question what is more important (if we can't get both):
ultra-fast rotation or ultra-high resolution.
Personally, I would clearly choose ultra-high resolution (because I am not a heavy use gamer and therefore speed is of lower concern).
I'm not hoping for ultra-fast rotation. But right now, it's ultra-slow. (notaz can give more details about it).
Additionally, rotation eats up CPU power as well right now (100%)
Hm. I thought the tiler is working completely independently of the CPU cores? So it should only be set up once by the CPU?
d) we should really find out (e.g. ask on e2e.ti.com) why the OMAP5 rotator is slower than expected. It may just be that something else that is not optimally configured (e.g. DRAM caching?) if it was designed for exactly this purpose.
There's already a thread with no solution yet: http://e2e.ti.com/support/omap/f/885/p/322408/1121877.aspx
Is this really related? To me it appears as if someone has connected a camera and does memcpy a lot of data. This may of course drive the CPU to 100%.
So I would not give up too fast. Saving a chip and reducing maximum resolution IMHO warrants investing some more time into potential software issues...
BR, Nikolaus
Am Sun, 30 Mar 2014 20:39:59 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
As far as I know the full stream goes always through that chip, so there's no direct connection between the Panel to the OMAP5 in this case, but from the Panel to the IC to the OMAP5.
I mean the problem that the panel driver in Linux assumes that it can directly send DCS commands to the panel. Therefore they arrive at the interconnect chip which expects them in a special format to forward them to the panel or it will try to interpret them directly. So I am simply not sure if the software architecture of panel drivers is prepared for this situation. It can certainly be done, but may be a major task (for kernel drivers).
Well, it would probably need a special driver, but I can ask whether that is available or not.
I'm not hoping for ultra-fast rotation. But right now, it's ultra-slow. (notaz can give more details about it). Additionally, rotation eats up CPU power as well right now (100%)
Hm. I thought the tiler is working completely independently of the CPU cores? So it should only be set up once by the CPU?
Well, notaz may correct me in case I'm not 100% correct here (not sure I remember it all correctly).
One issue is: To rotate with TILER, the framebuffer needs to be in a special format. The X driver supports that format, so everything should be fine here. However, that is not the case or SDL or programs using directly the framebuffer.
These need to be converted - which is slowing it down a bit, but hopefully not too much. (besides: you can't simply recompile programs anymore - you need to add that conversion into the code as well)
The main problem is programs direcly writing to the hardware. The Pandora is very fast with notaz' emulators or enhanced SDL version, because it writes directly into the display memory.
And that's what's so horribly slow right now - and where it's related to that forum entry.
So as far as I understood: No problem when using X, but a huge issue with most games and emulators (due to SDL / Framebuffer access)
There's already a thread with no solution yet: http://e2e.ti.com/support/omap/f/885/p/322408/1121877.aspx
Is this really related? To me it appears as if someone has connected a camera and does memcpy a lot of data. This may of course drive the CPU to 100%.
We need to use memcpy as well for all non-X stuff - which is a lot.
So I would not give up too fast. Saving a chip and reducing maximum resolution IMHO warrants investing some more time into potential software issues...
Well, that issue is probably something we cannot solve alone. It's a simple memcpy command. It could be one of the many OMAP5 registers that's not properly setup, and only TI knows what they do.
Right now, rotation is faster when doing in software than using memcpy and using the TILER, so it' not really usable.
If I understood notaz correctly, we basically lose a few hundred (!!) fps because of that.
If no software fix can be found, we need to find a way to rotate the display using some hardware.
It would certainly be more convenient as well - as a simple recompile of games won't work, you need to add the rotation code.
My hope was that we can setup TILER independent from the programs, so that it always rotates the screen regardless what program is running. But that doesn't work, as it needs a special format.
notaz can probably explain it more accurate and better than me.
Am 30.03.2014 um 21:17 schrieb Michael Mrozek:
Am Sun, 30 Mar 2014 20:39:59 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
As far as I know the full stream goes always through that chip, so there's no direct connection between the Panel to the OMAP5 in this case, but from the Panel to the IC to the OMAP5.
I mean the problem that the panel driver in Linux assumes that it can directly send DCS commands to the panel. Therefore they arrive at the interconnect chip which expects them in a special format to forward them to the panel or it will try to interpret them directly. So I am simply not sure if the software architecture of panel drivers is prepared for this situation. It can certainly be done, but may be a major task (for kernel drivers).
Well, it would probably need a special driver, but I can ask whether that is available or not.
I'm not hoping for ultra-fast rotation. But right now, it's ultra-slow. (notaz can give more details about it). Additionally, rotation eats up CPU power as well right now (100%)
Hm. I thought the tiler is working completely independently of the CPU cores? So it should only be set up once by the CPU?
Well, notaz may correct me in case I'm not 100% correct here (not sure I remember it all correctly).
One issue is: To rotate with TILER, the framebuffer needs to be in a special format. The X driver supports that format, so everything should be fine here. However, that is not the case or SDL or programs using directly the framebuffer.
These need to be converted - which is slowing it down a bit, but hopefully not too much. (besides: you can't simply recompile programs anymore - you need to add that conversion into the code as well)
The main problem is programs direcly writing to the hardware. The Pandora is very fast with notaz' emulators or enhanced SDL version, because it writes directly into the display memory.
Isn't the display memory the same as the frame buffer?
And that's what's so horribly slow right now - and where it's related to that forum entry.
So as far as I understood: No problem when using X, but a huge issue with most games and emulators (due to SDL / Framebuffer access)
There's already a thread with no solution yet: http://e2e.ti.com/support/omap/f/885/p/322408/1121877.aspx
Is this really related? To me it appears as if someone has connected a camera and does memcpy a lot of data. This may of course drive the CPU to 100%.
We need to use memcpy as well for all non-X stuff - which is a lot.
So I would not give up too fast. Saving a chip and reducing maximum resolution IMHO warrants investing some more time into potential software issues...
Well, that issue is probably something we cannot solve alone. It's a simple memcpy command. It could be one of the many OMAP5 registers that's not properly setup, and only TI knows what they do.
Right now, rotation is faster when doing in software than using memcpy and using the TILER, so it' not really usable.
Ah, I think I get the understanding. Because the tiler needs a different memory layout than the standard framebuffer
If I understood notaz correctly, we basically lose a few hundred (!!) fps because of that.
Well, the panel isn't faster than 60fps...
If no software fix can be found, we need to find a way to rotate the display using some hardware.
It would certainly be more convenient as well - as a simple recompile of games won't work, you need to add the rotation code.
My hope was that we can setup TILER independent from the programs, so that it always rotates the screen regardless what program is running.
Yes, that is what I also assume. It should simply take a memory region and instead of address = y * width + x it should read address = x * height + y
But that doesn't work, as it needs a special format.
I have scanned a little through the TRM. Well, quite complex and difficult to understand :)
Basically it appears as if the TILER can do everything automatically, but not arbitrarily. I.e. we can not easily use any existing linear framebuffer scheme and apply the TILER to it.
Rather the framebuffer must be set up in a special way so that the address translation etc. works with "page size". Then, the TILER can work on that. I.e. the TILER defines the memory layout.
The "software" solution is most likely
on every vsync (or other timer) loop over all pixels copy from non-rotated (user space visible) to rotatable (not in user space)
This of course needs 60fps copy operations with ~2 Mpix each (for the current LG panel), i.e. 124 Mpix/s. This is quite demanding for a CPU. Just 8 clocks per pixel...
But it should be a little better if the program expects a lower resolution.
Then, only the low-res image (the framebuffer shared memory visible to the program) needs to be rearranged by software (to the framebuffer that is TILER compatible). And all the rest (rotation and upscaling) can be done by the TILER and GFX pipeline.
So how many existing programs that can't be recompiled assume our new full resolution display? I would assume they expect a Pandora resolution of 480x800. And a frame rate of 30fps.
This would only be 11 Mpix/s to be copied (because rotation and upscaling is done by the hardware). This should not drive the CPU to 100%.
So the idea would be to have an optional "pandorafb" that is compatible to the existing direct memory addressing schemes and does a repeated copy to the new one, that always uses the TILER (with TILER compatible memory layout).
Then, software can choose which framebuffer to use and only if the pandorafb is used, this copying process is running (inside the kernel).
But of course this is just an idea from looking roughly over fhe TRM but I don't now all limitations which could break this approach.
notaz can probably explain it more accurate and better than me.
Yes, he is certainly the one who did digg deepest in this topic...
BR, Nikolaus
Am Sun, 30 Mar 2014 22:37:27 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Right now, rotation is faster when doing in software than using memcpy and using the TILER, so it' not really usable.
Ah, I think I get the understanding. Because the tiler needs a different memory layout than the standard framebuffer
Correct.
If I understood notaz correctly, we basically lose a few hundred (!!) fps because of that.
Well, the panel isn't faster than 60fps...
Yes, But a slowed down rendering means less CPU for the program itself and more battery usage.
But that doesn't work, as it needs a special format
I have scanned a little through the TRM. Well, quite complex and difficult to understand :) Basically it appears as if the TILER can do everything automatically, but not arbitrarily. I.e. we can not easily use any existing linear framebuffer scheme and apply the TILER to it.
Exactly. The format is a little bit different. It's not a huge differnce (basically same format with blanks in), but either the program needs to render in that format or convert the normal format before it can be used.
The "software" solution is most likely on every vsync (or other timer) loop over all pixels copy from non-rotated (user space visible) to rotatable (not in user space)
That copy is what's slow right now. Something is wrong with the TILER memory setup probably right now.
It's using the same RAM chip, so we know the RAM is working. So it's either probably not correctly setup or there's an OMAP5 hardware bug (hopefully not)
See the last post at the TI thread to see the impact of copying to TILER memory or normal memory..
This of course needs 60fps copy operations with ~2 Mpix each (for the current LG panel), i.e. 124 Mpix/s. This is quite demanding for a CPU. Just 8 clocks per pixel...
Yep.
But it should be a little better if the program expects a lower resolution.
Then, only the low-res image (the framebuffer shared memory visible to the program) needs to be rearranged by software (to the framebuffer that is TILER compatible). And all the rest (rotation and upscaling) can be done by the TILER and GFX pipeline.
Correct, that's what we hope.
So how many existing programs that can't be recompiled assume our new full resolution display? I would assume they expect a Pandora resolution of 480x800. And a frame rate of 30fps.
All existing SDL programs. Anything not using X. PSX emulation, for example, would surely look great when rendering in higher resolutions than 480x800.
And there are a lot more SDL apps that have not been ported to the Pandora yet due to CPU speed or resolution issues.
This would only be 11 Mpix/s to be copied (because rotation and upscaling is done by the hardware). This should not drive the CPU to 100%.
Right now, I think anything higher than 320x240 does that already, but notaz has tested it more accurately.
So the idea would be to have an optional "pandorafb" that is compatible to the existing direct memory addressing schemes and does a repeated copy to the new one, that always uses the TILER (with TILER compatible memory layout).
On the Pandora, optimized programs are writing directly into the display memory, not using a framebuffer (using notaz' tweaked SDL), afaik.
Am 30.03.2014 um 22:52 schrieb Michael Mrozek:
Am Sun, 30 Mar 2014 22:37:27 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Right now, rotation is faster when doing in software than using memcpy and using the TILER, so it' not really usable.
Ah, I think I get the understanding. Because the tiler needs a different memory layout than the standard framebuffer
Correct.
If I understood notaz correctly, we basically lose a few hundred (!!) fps because of that.
Well, the panel isn't faster than 60fps...
Yes, But a slowed down rendering means less CPU for the program itself and more battery usage.
But that doesn't work, as it needs a special format
I have scanned a little through the TRM. Well, quite complex and difficult to understand :) Basically it appears as if the TILER can do everything automatically, but not arbitrarily. I.e. we can not easily use any existing linear framebuffer scheme and apply the TILER to it.
Exactly. The format is a little bit different. It's not a huge differnce (basically same format with blanks in), but either the program needs to render in that format or convert the normal format before it can be used.
The "software" solution is most likely on every vsync (or other timer) loop over all pixels copy from non-rotated (user space visible) to rotatable (not in user space)
That copy is what's slow right now. Something is wrong with the TILER memory setup probably right now.
It's using the same RAM chip, so we know the RAM is working. So it's either probably not correctly setup or there's an OMAP5 hardware bug (hopefully not)
See the last post at the TI thread to see the impact of copying to TILER memory or normal memory..
This of course needs 60fps copy operations with ~2 Mpix each (for the current LG panel), i.e. 124 Mpix/s. This is quite demanding for a CPU. Just 8 clocks per pixel...
Yep.
But it should be a little better if the program expects a lower resolution.
Then, only the low-res image (the framebuffer shared memory visible to the program) needs to be rearranged by software (to the framebuffer that is TILER compatible). And all the rest (rotation and upscaling) can be done by the TILER and GFX pipeline.
Correct, that's what we hope.
So how many existing programs that can't be recompiled assume our new full resolution display? I would assume they expect a Pandora resolution of 480x800. And a frame rate of 30fps.
All existing SDL programs. Anything not using X. PSX emulation, for example, would surely look great when rendering in higher resolutions than 480x800.
And there are a lot more SDL apps that have not been ported to the Pandora yet due to CPU speed or resolution issues.
This would only be 11 Mpix/s to be copied (because rotation and upscaling is done by the hardware). This should not drive the CPU to 100%.
Right now, I think anything higher than 320x240 does that already, but notaz has tested it more accurately.
So the idea would be to have an optional "pandorafb" that is compatible to the existing direct memory addressing schemes and does a repeated copy to the new one, that always uses the TILER (with TILER compatible memory layout).
On the Pandora, optimized programs are writing directly into the display memory, not using a framebuffer (using notaz' tweaked SDL), afaik.
I think working might be important. The display memory *is* the frame buffer. What we call "framebuffer" is the linux kernel driver.
And afaik there are at least two methods to access the frame buffer memory. Either through the driver (ioctl) or by mapping the display memory to the user space of a program. Then a simple *adress = 0x55 will change a pixel.
All this frame buffer memory is read by the DISPC - automatically. And AFAIK it can be read through the TILER by the DISPC. So that it only needs to be in memory in a way that it is understood by the TILER.
BR, Nikolaus
On 03/30/2014 09:39 PM, Dr. H. Nikolaus Schaller wrote:
Hm. I thought the tiler is working completely independently of the CPU cores? So it should only be set up once by the CPU?
No, it's just bunch of logic between devices and DRAM (it's part of memory controller). You basically have part of physical address space reserved to it, where the CPU or other devices can write. When something writes there, TILER redirects that write to some location of DRAM as if it was rotated. So basically as you draw the image from CPU (or other device) to that location, rotated image is constructed somewhere in DRAM. Then the display controller (or other devices?) can fetch rotated image from DRAM and send it to the screen. It may sound good but there are problems:
1. The source image can't be linear, some padding is needed after every line so that TILER could work with it. This is a problem with some software, especially some currently on pandora, although it can be fixed or worked around it's still extra effort or performance loss. 2. Today I found out that TILER memory can't be read, reading causes data abort. Every device since GP2X could read it's framebuffer, so this sucks. Same consequences like above, could also be OMAP misconfiguration, who knows.. 3, Writing to that memory currently is real slow. As the resolution increases it becomes even slower, it's like from 60% performance hit on low resolutions to 10x slowdown for fullhd. It would look like TILER can't keep up and is halting the CPU, from OS point of view it's 100% CPU usage. 4. Writing larger blocks to tiler memory is causing DISPC FIFO underflows, which means dropped frames (really bad for gaming console). 5. Mysterious L3 error interrupts from OMAP during writes 6. Xorg crash when xf86-video-omap is used with rotation 7. Some googling shows that TILER is not working with some formats used for video
(1) and (2) can be worked around, although it's quite a bit of extra effort. For (2) shadow buffer is required with extra copying, which means wasted CPU and battery time. Unless it can be solved somehow so that read just works.. (6) should be fixable (7) needs to be investigated 2-5 look like OMAP's bugs or misconfiguration and are real difficult to debug/fix.
So I would not give up too fast. Saving a chip and reducing maximum resolution IMHO warrants investing some more time into potential software issues...
I'd much prefer the chip and lower res, all the above problems would just go away. That LG screen is clearly an overkill for me.
Grazvydas
On 03/30/2014 11:39 PM, Grazvydas Ignotas wrote:
- The source image can't be linear, some padding is needed after every
line so that TILER could work with it. This is a problem with some software, especially some currently on pandora, although it can be fixed or worked around it's still extra effort or performance loss. 2. Today I found out that TILER memory can't be read, reading causes data abort. Every device since GP2X could read it's framebuffer, so this sucks. Same consequences like above, could also be OMAP misconfiguration, who knows.. 3, Writing to that memory currently is real slow. As the resolution increases it becomes even slower, it's like from 60% performance hit on low resolutions to 10x slowdown for fullhd. It would look like TILER can't keep up and is halting the CPU, from OS point of view it's 100% CPU usage. 4. Writing larger blocks to tiler memory is causing DISPC FIFO underflows, which means dropped frames (really bad for gaming console). 5. Mysterious L3 error interrupts from OMAP during writes 6. Xorg crash when xf86-video-omap is used with rotation 7. Some googling shows that TILER is not working with some formats used for video
Oh and 8. Corruption seen when rotated image is scaled
.. probably layer is set up incorrectly in the driver, should be fixable.
Am 30.03.2014 um 22:39 schrieb Grazvydas Ignotas:
On 03/30/2014 09:39 PM, Dr. H. Nikolaus Schaller wrote:
Hm. I thought the tiler is working completely independently of the CPU cores? So it should only be set up once by the CPU?
No, it's just bunch of logic between devices and DRAM (it's part of memory controller). You basically have part of physical address space reserved to it, where the CPU or other devices can write. When something writes there, TILER redirects that write to some location of DRAM as if it was rotated. So basically as you draw the image from CPU (or other device) to that location, rotated image is constructed somewhere in DRAM. Then the display controller (or other devices?) can fetch rotated image from DRAM and send it to the screen.
I did understand it the other way round - that the DISPC can use the TILER to read rotated data from DRAM. So there is no need to write data differently for different rotations. I.e. it is written only once - and by reprogramming the TILER, the image can be rotated.
At least I could not find a restriction that the TILER can only be used for writing.
But this may be "wishful thinking" and the OPAM5 may have implemented it completely different...
It may sound good but there are problems:
- The source image can't be linear, some padding is needed after every
line so that TILER could work with it. This is a problem with some software, especially some currently on pandora, although it can be fixed or worked around it's still extra effort or performance loss. 2. Today I found out that TILER memory can't be read, reading causes data abort. Every device since GP2X could read it's framebuffer, so this sucks. Same consequences like above, could also be OMAP misconfiguration, who knows.. 3, Writing to that memory currently is real slow. As the resolution increases it becomes even slower, it's like from 60% performance hit on low resolutions to 10x slowdown for fullhd. It would look like TILER can't keep up and is halting the CPU, from OS point of view it's 100% CPU usage.
Well, it is DMA and wrestling with the CPU (L3 interconnect) for DRAM access.
- Writing larger blocks to tiler memory is causing DISPC FIFO
underflows, which means dropped frames (really bad for gaming console). 5. Mysterious L3 error interrupts from OMAP during writes 6. Xorg crash when xf86-video-omap is used with rotation 7. Some googling shows that TILER is not working with some formats used for video
(1) and (2) can be worked around, although it's quite a bit of extra effort. For (2) shadow buffer is required with extra copying, which means wasted CPU and battery time. Unless it can be solved somehow so that read just works.. (6) should be fixable (7) needs to be investigated 2-5 look like OMAP's bugs or misconfiguration and are real difficult to debug/fix.
So I would not give up too fast. Saving a chip and reducing maximum resolution IMHO warrants investing some more time into potential software issues...
I'd much prefer the chip and lower res, all the above problems would just go away. That LG screen is clearly an overkill for me.
Grazvydas _______________________________________________ Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
On 30/03/14 23:56, Dr. H. Nikolaus Schaller wrote:
Am 30.03.2014 um 22:39 schrieb Grazvydas Ignotas:
On 03/30/2014 09:39 PM, Dr. H. Nikolaus Schaller wrote:
Hm. I thought the tiler is working completely independently of the CPU cores? So it should only be set up once by the CPU?
No, it's just bunch of logic between devices and DRAM (it's part of memory controller). You basically have part of physical address space reserved to it, where the CPU or other devices can write. When something writes there, TILER redirects that write to some location of DRAM as if it was rotated. So basically as you draw the image from CPU (or other device) to that location, rotated image is constructed somewhere in DRAM. Then the display controller (or other devices?) can fetch rotated image from DRAM and send it to the screen.
I did understand it the other way round - that the DISPC can use the TILER to read rotated data from DRAM. So there is no need to write data differently for different rotations. I.e. it is written only once - and by reprogramming the TILER, the image can be rotated.
At least I could not find a restriction that the TILER can only be used for writing.
But this may be "wishful thinking" and the OPAM5 may have implemented it completely different...
I'm not 100% sure about TILER, but VRFB on omap3 could be used in both configurations (DISPC accessing rotated content, or the users accessing rotated content). I don't see why TILER could not be used the same way.
I think the reason that it's recommended to configure DISPC to use non-rotated content is that DISPC is reading the full buffer 60 times a second, all the time, whereas the users of the buffer may only write to the buffer when there's something to draw, and maybe just to part of the buffer.
And accessing rotated content is heavier on the memory bus than non-rotated, so it saves mem bandwidth to read non-rotated content with DISPC.
Tomi
Am 31.03.2014 um 15:46 schrieb Tomi Valkeinen:
On 30/03/14 23:56, Dr. H. Nikolaus Schaller wrote:
Am 30.03.2014 um 22:39 schrieb Grazvydas Ignotas:
On 03/30/2014 09:39 PM, Dr. H. Nikolaus Schaller wrote:
Hm. I thought the tiler is working completely independently of the CPU cores? So it should only be set up once by the CPU?
No, it's just bunch of logic between devices and DRAM (it's part of memory controller). You basically have part of physical address space reserved to it, where the CPU or other devices can write. When something writes there, TILER redirects that write to some location of DRAM as if it was rotated. So basically as you draw the image from CPU (or other device) to that location, rotated image is constructed somewhere in DRAM. Then the display controller (or other devices?) can fetch rotated image from DRAM and send it to the screen.
I did understand it the other way round - that the DISPC can use the TILER to read rotated data from DRAM. So there is no need to write data differently for different rotations. I.e. it is written only once - and by reprogramming the TILER, the image can be rotated.
At least I could not find a restriction that the TILER can only be used for writing.
But this may be "wishful thinking" and the OPAM5 may have implemented it completely different...
I'm not 100% sure about TILER, but VRFB on omap3 could be used in both configurations (DISPC accessing rotated content, or the users accessing rotated content). I don't see why TILER could not be used the same way.
I think the reason that it's recommended to configure DISPC to use non-rotated content is that DISPC is reading the full buffer 60 times a second, all the time, whereas the users of the buffer may only write to the buffer when there's something to draw, and maybe just to part of the buffer.
Ah, I see.
In this case if we want to hide the rotated display, we always have to rotate the full screen. I.e. rotation is not a property of some "window" but the whole panel...
And the problem Notaz is reporting is when he wants to write faster to the framebuffer (130 fps) in Full-HD.
So the approach to write rotated data to the frame buffer that is read unrotated by the DISPC may need to be reversed to get full speed in such cases. And to hide rotation from the application code.
And accessing rotated content is heavier on the memory bus than non-rotated, so it saves mem bandwidth to read non-rotated content with DISPC.
Is this true for the TILER? I understood that it reads blocks from memory and then simply scans them either vertically or horizontally. So that the number of bytes to be read per frame is the same. They are just read in a different address stride. And since the tiler works in blocks the memory controller can still use autoincrement etc. in the same way and speed as for a linear memory layout.
BR, Nikolaus
On Mon, Mar 31, 2014 at 5:08 PM, Dr. H. Nikolaus Schaller hns@goldelico.com wrote:
Am 31.03.2014 um 15:46 schrieb Tomi Valkeinen:
On 30/03/14 23:56, Dr. H. Nikolaus Schaller wrote:
Am 30.03.2014 um 22:39 schrieb Grazvydas Ignotas:
On 03/30/2014 09:39 PM, Dr. H. Nikolaus Schaller wrote:
Hm. I thought the tiler is working completely independently of the CPU cores? So it should only be set up once by the CPU?
No, it's just bunch of logic between devices and DRAM (it's part of memory controller). You basically have part of physical address space reserved to it, where the CPU or other devices can write. When something writes there, TILER redirects that write to some location of DRAM as if it was rotated. So basically as you draw the image from CPU (or other device) to that location, rotated image is constructed somewhere in DRAM. Then the display controller (or other devices?) can fetch rotated image from DRAM and send it to the screen.
I did understand it the other way round - that the DISPC can use the TILER to read rotated data from DRAM. So there is no need to write data differently for different rotations. I.e. it is written only once - and by reprogramming the TILER, the image can be rotated.
At least I could not find a restriction that the TILER can only be used for writing.
But this may be "wishful thinking" and the OPAM5 may have implemented it completely different...
I'm not 100% sure about TILER, but VRFB on omap3 could be used in both configurations (DISPC accessing rotated content, or the users accessing rotated content). I don't see why TILER could not be used the same way.
I think the reason that it's recommended to configure DISPC to use non-rotated content is that DISPC is reading the full buffer 60 times a second, all the time, whereas the users of the buffer may only write to the buffer when there's something to draw, and maybe just to part of the buffer.
Ah, I see.
In this case if we want to hide the rotated display, we always have to rotate the full screen. I.e. rotation is not a property of some "window" but the whole panel...
OMAP5 has a graphics plane and 3 scalable overlays, with TILER every one of them has to be rotated separately as all of them are fetched from different buffers. On Pyra we'll want to use at least 2 of them simultaneously, one for Xorg output and another for video overlay or a low resolution game that has to be scaled. So no it's not a propertly of the whole panel..
And the problem Notaz is reporting is when he wants to write faster to the framebuffer (130 fps) in Full-HD.
So the approach to write rotated data to the frame buffer that is read unrotated by the DISPC may need to be reversed to get full speed in such cases. And to hide rotation from the application code.
I'm not sure that'll help as TILER likely won't be able to supply data fast enough, read below..
And accessing rotated content is heavier on the memory bus than non-rotated, so it saves mem bandwidth to read non-rotated content with DISPC.
Is this true for the TILER? I understood that it reads blocks from memory and then simply scans them either vertically or horizontally. So that the number of bytes to be read per frame is the same. They are just read in a different address stride. And since the tiler works in blocks the memory controller can still use autoincrement etc. in the same way and speed as for a linear memory layout.
You do know that DRAM is rather inefficient for non-sequential accesses, right? For sequential access you program DRAM row/column and fire up the transfer, it autoincrements and no extra row/column programming is needed. When you have some large address stride, the memory controller has constantly reprogram the column (or was it row?) address for every access, and when you hit another page (which will happen relatively often with such large strides), it has to reprogram both column and row. This should slow it to some extent but not 10 almost times that I'm getting.. Still even it it worked perfectly, I have some doubts for it being able to pull off fullhd at 60fps..
Gražvydas
Am 31.03.2014 um 16:36 schrieb Grazvydas Ignotas:
On Mon, Mar 31, 2014 at 5:08 PM, Dr. H. Nikolaus Schaller hns@goldelico.com wrote:
Am 31.03.2014 um 15:46 schrieb Tomi Valkeinen:
On 30/03/14 23:56, Dr. H. Nikolaus Schaller wrote:
Am 30.03.2014 um 22:39 schrieb Grazvydas Ignotas:
On 03/30/2014 09:39 PM, Dr. H. Nikolaus Schaller wrote:
Hm. I thought the tiler is working completely independently of the CPU cores? So it should only be set up once by the CPU?
No, it's just bunch of logic between devices and DRAM (it's part of memory controller). You basically have part of physical address space reserved to it, where the CPU or other devices can write. When something writes there, TILER redirects that write to some location of DRAM as if it was rotated. So basically as you draw the image from CPU (or other device) to that location, rotated image is constructed somewhere in DRAM. Then the display controller (or other devices?) can fetch rotated image from DRAM and send it to the screen.
I did understand it the other way round - that the DISPC can use the TILER to read rotated data from DRAM. So there is no need to write data differently for different rotations. I.e. it is written only once - and by reprogramming the TILER, the image can be rotated.
At least I could not find a restriction that the TILER can only be used for writing.
But this may be "wishful thinking" and the OPAM5 may have implemented it completely different...
I'm not 100% sure about TILER, but VRFB on omap3 could be used in both configurations (DISPC accessing rotated content, or the users accessing rotated content). I don't see why TILER could not be used the same way.
I think the reason that it's recommended to configure DISPC to use non-rotated content is that DISPC is reading the full buffer 60 times a second, all the time, whereas the users of the buffer may only write to the buffer when there's something to draw, and maybe just to part of the buffer.
Ah, I see.
In this case if we want to hide the rotated display, we always have to rotate the full screen. I.e. rotation is not a property of some "window" but the whole panel...
OMAP5 has a graphics plane and 3 scalable overlays, with TILER every one of them has to be rotated separately as all of them are fetched from different buffers. On Pyra we'll want to use at least 2 of them simultaneously, one for Xorg output and another for video overlay or a low resolution game that has to be scaled. So no it's not a propertly of the whole panel..
That might be a different view of the word "property". I mean that the *panel* is rotated, and therefore the DISPC channel driving it must fetch rotated data. In full resolution of the panel - and not some parts rotated and others not.
The latter case would make it useful to rotate small portions of the frame buffer during write.
And the problem Notaz is reporting is when he wants to write faster to the framebuffer (130 fps) in Full-HD.
So the approach to write rotated data to the frame buffer that is read unrotated by the DISPC may need to be reversed to get full speed in such cases. And to hide rotation from the application code.
I'm not sure that'll help as TILER likely won't be able to supply data fast enough, read below..
And accessing rotated content is heavier on the memory bus than non-rotated, so it saves mem bandwidth to read non-rotated content with DISPC.
Is this true for the TILER? I understood that it reads blocks from memory and then simply scans them either vertically or horizontally. So that the number of bytes to be read per frame is the same. They are just read in a different address stride. And since the tiler works in blocks the memory controller can still use autoincrement etc. in the same way and speed as for a linear memory layout.
You do know that DRAM is rather inefficient for non-sequential accesses, right? For sequential access you program DRAM row/column and fire up the transfer, it autoincrements and no extra row/column programming is needed.
exactly.
When you have some large address stride, the memory controller has constantly reprogram the column (or was it row?) address for every access, and when you hit another page (which will happen relatively often with such large strides), it has to reprogram both column and row. This should slow it
I guess it should only slow down if the blocks are small. If the block is so large that you have to reprogram the DRAM address anyways and start a new autoincrement it should not make any difference.
This is how I understand the TILER: it can work as a cache between DRAM and the video consumer (DISPC + Panel) so that it efficiently fetches whole blocks from DRAM but reads them in a different sequence than DRAM is able to provide.
The TRM talks about 4KiB pages.
Section 16.2.3.6.3.1.1 Buffer Size
says something about the size requirements of some buffer and picture.
Table 16-19 promises 100% efficiency.
"When using 32-bit DDR2 or DDR3 SDRAMs, whenever all previous guidelines are fulfilled, there is no penalty for these initiators to access a tiled object in any orientation, except for accesses to an interlaced frame in any orientation where the memory bandwidth is twice the requested bandwidth. When using DDR2 or DDR3 SDRAMs, all accesses, except interlaced accesses, are equally efficient and offer the full possible memory bandwidth. Similarly, page mode does not introduce any performance hit."
This is why I think it should somehow be possible to use it without performance degradation. We just don't know how...
to some extent but not 10 almost times that I'm getting..
Factor 10 is indeed strange.
Did I get it correctly that you are using the TILER in your experiments to write rotated data to a linear framebuffer?
This might simply have lower priority than CPU instruction fetches or generally be slower if it needs a read-modify-write.
Still even it it worked perfectly, I have some doubts for it being able to pull off fullhd at 60fps..
Why that? If the standard DISPC can bypass the TILER and fetch in 60fps it should also be possible with using TILER.
Generally we need roughly 7 MByte/frame which means 414 MByte/s.
The DRAM bandwidth should be ca. 1 GT/s and each transfer is 4 Bytes. This would be 4 GByte/s i.e. the panel should just consume 10% of the DRAM bandwidth. And we even have two DRAM channels (1 GByte each).
Ok, this may make a difference - if source and sink are in the different DRAM channels, they don't block each other. If they are in the same reads and writes must be sequential.
But if I look through the list of registers, there are many locations where we can have suboptimal settings.
BR, Nikolaus
On Mon, Mar 31, 2014 at 6:12 PM, Dr. H. Nikolaus Schaller hns@goldelico.com wrote:
Am 31.03.2014 um 16:36 schrieb Grazvydas Ignotas:
You do know that DRAM is rather inefficient for non-sequential accesses, right? For sequential access you program DRAM row/column and fire up the transfer, it autoincrements and no extra row/column programming is needed.
exactly.
When you have some large address stride, the memory controller has constantly reprogram the column (or was it row?) address for every access, and when you hit another page (which will happen relatively often with such large strides), it has to reprogram both column and row. This should slow it
I guess it should only slow down if the blocks are small. If the block is so large that you have to reprogram the DRAM address anyways and start a new autoincrement it should not make any difference.
This is how I understand the TILER: it can work as a cache between DRAM and the video consumer (DISPC + Panel) so that it efficiently fetches whole blocks from DRAM but reads them in a different sequence than DRAM is able to provide.
The TRM talks about 4KiB pages.
Section 16.2.3.6.3.1.1 Buffer Size
says something about the size requirements of some buffer and picture.
Table 16-19 promises 100% efficiency.
"When using 32-bit DDR2 or DDR3 SDRAMs, whenever all previous guidelines are fulfilled, there is no penalty for these initiators to access a tiled object in any orientation, except for accesses to an interlaced frame in any orientation where the memory bandwidth is twice the requested bandwidth. When using DDR2 or DDR3 SDRAMs, all accesses, except interlaced accesses, are equally efficient and offer the full possible memory bandwidth. Similarly, page mode does not introduce any performance hit."
This is why I think it should somehow be possible to use it without performance degradation. We just don't know how...
to some extent but not 10 almost times that I'm getting..
Factor 10 is indeed strange.
Did I get it correctly that you are using the TILER in your experiments to write rotated data to a linear framebuffer?
I just keep writing some color patterns linearly, padded as the TILER requires. The data itself is not rotated and colors are fetched from a small 256 entry buffer that easily fits in ARM's L1 cache.
This might simply have lower priority than CPU instruction fetches or generally be slower if it needs a read-modify-write.
Instructions shouldn't really matter (except on first iteration) as the loop is small and fits in ARM's L1 cache. In case I miscalculated something it should still easily fit in L2 cache and not bother DRAM at all, as well as that tiny color buffer, so it's not r-m-w, on the bus it should look like some data coming from ARM "out of nowhere". I've reattached my test, see the draw() function.
Still even it it worked perfectly, I have some doubts for it being able to pull off fullhd at 60fps..
Why that? If the standard DISPC can bypass the TILER and fetch in 60fps it should also be possible with using TILER.
Ok from what you quoted above, it's supposed to handle it. Although I'm not sure what "except for accesses to an interlaced frame in any orientation where the memory bandwidth is twice the requested bandwidth" means. Why does interlaced matter suddenly? All that padding in TILER format causes the data to be arranged somewhat like interlaced in DRAM already..
Generally we need roughly 7 MByte/frame which means 414 MByte/s.
The DRAM bandwidth should be ca. 1 GT/s and each transfer is 4 Bytes. This would be 4 GByte/s i.e. the panel should just consume 10% of the DRAM bandwidth. And we even have two DRAM channels (1 GByte each).
Ok, this may make a difference - if source and sink are in the different DRAM channels, they don't block each other. If they are in the same reads and writes must be sequential.
That's not really possible to control from userspace application..
But if I look through the list of registers, there are many locations where we can have suboptimal settings.
Agreed, the amount of registers on OMAPs is crazy, anything could be wrong..
Gražvydas
On 31/03/14 17:36, Grazvydas Ignotas wrote:
So the approach to write rotated data to the frame buffer that is read unrotated by the DISPC may need to be reversed to get full speed in such cases. And to hide rotation from the application code.
I'm not sure that'll help as TILER likely won't be able to supply data fast enough, read below..
That doesn't help.
When using TILER in 2D mode (i.e. with rotation support), both the CPU and the DISPC needs to use the TILER buffer. One of them uses the 0 degree view, and the other one uses the 90/180/270 degree view. With VRFB, and I believe also with TILER), using 0 degree view is faster then the others, and using 180 degree view is faster than 90/270 degree views.
That's why the DISPC should use 0 degree view, as it's constantly accessing the buffer.
But all that is not relevant to the performance issue here. The performance issue here is that we need to use TILER 2D buffer. It doesn't matter what the rotation is. When using TILER 2D buffer, omapdrm will only use two pages to map the whole buffer, bringing the performance down.
And accessing rotated content is heavier on the memory bus than non-rotated, so it saves mem bandwidth to read non-rotated content with DISPC.
Is this true for the TILER? I understood that it reads blocks from memory and then simply scans them either vertically or horizontally. So that the number of bytes to be read per frame is the same. They are just read in a different address stride. And since the tiler works in blocks the memory controller can still use autoincrement etc. in the same way and speed as for a linear memory layout.
You do know that DRAM is rather inefficient for non-sequential accesses, right? For sequential access you program DRAM row/column and fire up the transfer, it autoincrements and no extra row/column programming is needed. When you have some large address stride, the memory controller has constantly reprogram the column (or was it row?) address for every access, and when you hit another page (which will happen relatively often with such large strides), it has to reprogram both column and row. This should slow it to some extent but not 10 almost times that I'm getting.. Still even it it worked perfectly, I have some doubts for it being able to pull off fullhd at 60fps..
What you say is partially true. If we didn't use TILER, and just used DISPC to fetch the pixel lines vertically (we can do that), the above would happen. It would give so bad memory performance that DISPC wouldn't be able to show anything.
With TILER, the above issue is reduced. If I'm not mistaken, TILER uses Z-order curve to reduce the performance hit from rotation.
Tomi
Am 30.03.2014 um 22:39 schrieb Grazvydas Ignotas:
I'd much prefer the chip and lower res, all the above problems would just go away. That LG screen is clearly an overkill for me.
Well, ED could offer two variants :)
- one with lower res and - one with higher res (with limitations in rotation of raw frame buffer access)
BR, Nikolaus
Am Sun, 30 Mar 2014 23:02:27 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Nope, we don't want a development hell! The devices should be the same!
Am 30.03.2014 um 22:39 schrieb Grazvydas Ignotas:
I'd much prefer the chip and lower res, all the above problems would just go away. That LG screen is clearly an overkill for me.
Well, ED could offer two variants :)
- one with lower res and
- one with higher res (with limitations in rotation of raw frame
buffer access)
BR, Nikolaus _______________________________________________ Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
Am Sun, 30 Mar 2014 23:39:42 +0300 hat Grazvydas Ignotas grazvydas.ignotas@openpandora.org geschrieben:
Hi,
So I would not give up too fast. Saving a chip and reducing maximum resolution IMHO warrants investing some more time into potential software issues...
I'd much prefer the chip and lower res, all the above problems would just go away. That LG screen is clearly an overkill for me.
Same here - if the FullHD screen only works with a lot of issues and use a lot more battery due to performance needed, then that's really bad.
The Pandora has an awesome battery life and it often feels faster than a dualcore Cortex A9 phone, because it's pretty optimized.
These optimizations as well as the battery life is something I want to see on the Pyra as well.
So if TILER can't be used in an efficient way, then we need to change our plans.
Also, is users install standard SDL programs using the Debian Repo, or run a different distribution, all SDL programs will run in portrait mode...
Am 30.03.2014 um 23:07 schrieb Michael Mrozek:
Am Sun, 30 Mar 2014 23:39:42 +0300 hat Grazvydas Ignotas grazvydas.ignotas@openpandora.org geschrieben:
Hi,
So I would not give up too fast. Saving a chip and reducing maximum resolution IMHO warrants investing some more time into potential software issues...
I'd much prefer the chip and lower res, all the above problems would just go away. That LG screen is clearly an overkill for me.
Same here - if the FullHD screen only works with a lot of issues and use a lot more battery due to performance needed, then that's really bad.
The Pandora has an awesome battery life and it often feels faster than a dualcore Cortex A9 phone, because it's pretty optimized.
These optimizations as well as the battery life is something I want to see on the Pyra as well.
So if TILER can't be used in an efficient way, then we need to change our plans.
Also, is users install standard SDL programs using the Debian Repo, or run a different distribution, all SDL programs will run in portrait mode...
Unless the TILER is set up in kernel and invisible to user space. I am not yet convinced that this is impossible.
Regarding SDL (I have zero experience with that). How do they access the frame buffer? Like Qt? I.e. query the /dev/fb for properties and then map to user space and write the bits directly?
BR, Nikolaus
On 03/31/2014 12:26 AM, Dr. H. Nikolaus Schaller wrote:
Am 30.03.2014 um 23:07 schrieb Michael Mrozek:
Am Sun, 30 Mar 2014 23:39:42 +0300 hat Grazvydas Ignotas grazvydas.ignotas@openpandora.org geschrieben:
Hi,
So I would not give up too fast. Saving a chip and reducing maximum resolution IMHO warrants investing some more time into potential software issues...
I'd much prefer the chip and lower res, all the above problems would just go away. That LG screen is clearly an overkill for me.
Same here - if the FullHD screen only works with a lot of issues and use a lot more battery due to performance needed, then that's really bad.
The Pandora has an awesome battery life and it often feels faster than a dualcore Cortex A9 phone, because it's pretty optimized.
These optimizations as well as the battery life is something I want to see on the Pyra as well.
So if TILER can't be used in an efficient way, then we need to change our plans.
Also, is users install standard SDL programs using the Debian Repo, or run a different distribution, all SDL programs will run in portrait mode...
Unless the TILER is set up in kernel and invisible to user space. I am not yet convinced that this is impossible.
For X apps it can be invisible since xf86-whatever driver does the buffer setup, but there is always extra copying involved for app to display something on screen through X, and that is slow.
For direct access it can't be invisible without extra overhead for hidden copies, because of the page format that TILER needs. However it's possible express something that matches TILER's page format in things like SDL and programs that don't assume linear buffer will work.
I'd say right now the main issues now are slowness and read failures; I could only reach 16fps for fullhd drawing while rotated, which is way below 60fps. Without rotation same code can do over 130fps..
Regarding SDL (I have zero experience with that). How do they access the frame buffer? Like Qt? I.e. query the /dev/fb for properties and then map to user space and write the bits directly?
It's more or less like that, yes, however you can't use the scalers through /dev/fb any more since omapfb driver was replaced with DRM based omapdrm. DRM also provides /dev/fb but the functionality is very limited, so DRM API has to be used, which means SDL will have to be rewritten/ported in either case.
Grazvydas
Am 30.03.2014 um 23:50 schrieb Grazvydas Ignotas:
On 03/31/2014 12:26 AM, Dr. H. Nikolaus Schaller wrote:
Am 30.03.2014 um 23:07 schrieb Michael Mrozek:
Am Sun, 30 Mar 2014 23:39:42 +0300 hat Grazvydas Ignotas grazvydas.ignotas@openpandora.org geschrieben:
Hi,
So I would not give up too fast. Saving a chip and reducing maximum resolution IMHO warrants investing some more time into potential software issues...
I'd much prefer the chip and lower res, all the above problems would just go away. That LG screen is clearly an overkill for me.
Same here - if the FullHD screen only works with a lot of issues and use a lot more battery due to performance needed, then that's really bad.
The Pandora has an awesome battery life and it often feels faster than a dualcore Cortex A9 phone, because it's pretty optimized.
These optimizations as well as the battery life is something I want to see on the Pyra as well.
So if TILER can't be used in an efficient way, then we need to change our plans.
Also, is users install standard SDL programs using the Debian Repo, or run a different distribution, all SDL programs will run in portrait mode...
Unless the TILER is set up in kernel and invisible to user space. I am not yet convinced that this is impossible.
For X apps it can be invisible since xf86-whatever driver does the buffer setup, but there is always extra copying involved for app to display something on screen through X, and that is slow.
For direct access it can't be invisible without extra overhead for hidden copies, because of the page format that TILER needs. However it's possible express something that matches TILER's page format in things like SDL and programs that don't assume linear buffer will work.
I'd say right now the main issues now are slowness and read failures; I could only reach 16fps for fullhd drawing while rotated, which is way below 60fps.
But we won't get full hd rotated with the extra chip either (because it can't handle that resolution).
Without rotation same code can do over 130fps..
Which the display can't even handle...
Regarding SDL (I have zero experience with that). How do they access the frame buffer? Like Qt? I.e. query the /dev/fb for properties and then map to user space and write the bits directly?
It's more or less like that, yes, however you can't use the scalers through /dev/fb any more since omapfb driver was replaced with DRM based omapdrm. DRM also provides /dev/fb but the functionality is very limited, so DRM API has to be used, which means SDL will have to be rewritten/ported in either case.
Looks as if we are in the middle of some revolution...
BR, Nikolaus
Hello,
On Mon, Mar 31, 2014 at 11:49 AM, Dr. H. Nikolaus Schaller hns@goldelico.com wrote:
Am 30.03.2014 um 23:50 schrieb Grazvydas Ignotas:
On 03/31/2014 12:26 AM, Dr. H. Nikolaus Schaller wrote:
Unless the TILER is set up in kernel and invisible to user space. I am not yet convinced that this is impossible.
For X apps it can be invisible since xf86-whatever driver does the buffer setup, but there is always extra copying involved for app to display something on screen through X, and that is slow.
For direct access it can't be invisible without extra overhead for hidden copies, because of the page format that TILER needs. However it's possible express something that matches TILER's page format in things like SDL and programs that don't assume linear buffer will work.
I'd say right now the main issues now are slowness and read failures; I could only reach 16fps for fullhd drawing while rotated, which is way below 60fps.
But we won't get full hd rotated with the extra chip either (because it can't handle that resolution).
Perhaps we should not use fullhd screen then?
Without rotation same code can do over 130fps..
Which the display can't even handle...
This just shows that updating fullhd screen @60fps normally takes around half of processing power that CortexA15 has, this is quite a lot already. If we did some processing (alpha blending?), the CPU likely wouldn't be able to keep up at all.
With rotation, it would need ~4x more of something (better TILER response, better memory controller latency, less errors?) than current hw+sw setup we have to be able to update the screen @60fps.
We need to be able to at least update the whole screen at it's refresh rate and have a bit of CPU time left. If this can't be achieved, the setup is unacceptable IMO. I'd say having to use half of CPU time for 60Hz (in unrotated mode) is already bad enough and really limits things you can do..
Gražvydas
On 29/03/14 22:23, Michael Mrozek wrote:
Hi,
as we don't know yet if our current display will ever be working with proper speed rotating it with TILER, I've searched for some solutions.
Solomon Systech offers a MIPI bridge chip that can do stuff like rotation, scaling, color corrections, power saving, etc. That would be rotating without using the main system, so without any loss.
I've contacted them (find the datasheet of the chip attached). The chip has a pin connection for the OMAP5 for tear-free rotation.
The downside is: That chip can only do rotation up to 720x1280, so our display would not work.
If you use chip like that, make sure the rotation actually works. I don't know if they have something magical there, but normally you need a double buffer fb in the chip to support rotation. One buffer is written by the SoC, while the other is sent to the LCD. If there's just a single buffer, you'll get diagonal tearing.
Tomi
Am 01.04.2014 um 15:11 schrieb Tomi Valkeinen:
On 29/03/14 22:23, Michael Mrozek wrote:
Hi,
as we don't know yet if our current display will ever be working with proper speed rotating it with TILER, I've searched for some solutions.
Solomon Systech offers a MIPI bridge chip that can do stuff like rotation, scaling, color corrections, power saving, etc. That would be rotating without using the main system, so without any loss.
I've contacted them (find the datasheet of the chip attached). The chip has a pin connection for the OMAP5 for tear-free rotation.
The downside is: That chip can only do rotation up to 720x1280, so our display would not work.
If you use chip like that, make sure the rotation actually works. I don't know if they have something magical there, but normally you need a double buffer fb in the chip to support rotation. One buffer is written by the SoC, while the other is sent to the LCD. If there's just a single buffer, you'll get diagonal tearing.
Thanks for the hint!
I have looked into the data sheet and it appears that even 720x1280 drives the chip to its limits:
• Half-WUXGA (0.5 x 1920 x 1200 pixels) frame buffer • Support rotation up to 1280x800 resolution with SSL Compression • Support rotation up to 960x600 resolution without image compression
So it has only enough on-chip RAM for a single buffer.
And SSL-Compression means:
"SSL segment-based (4 pixels per segment) data compression/decompression with zero software overhead"
So it appears that we won't get full resolution 720x1280 rotated with that chip with less problems than with OMAP5 TILER! The problems will just be somewhere else.
BR, Nikolaus
Am 01.04.2014 um 15:22 schrieb Dr. H. Nikolaus Schaller:
Am 01.04.2014 um 15:11 schrieb Tomi Valkeinen:
On 29/03/14 22:23, Michael Mrozek wrote:
Hi,
as we don't know yet if our current display will ever be working with proper speed rotating it with TILER, I've searched for some solutions.
Solomon Systech offers a MIPI bridge chip that can do stuff like rotation, scaling, color corrections, power saving, etc. That would be rotating without using the main system, so without any loss.
I've contacted them (find the datasheet of the chip attached). The chip has a pin connection for the OMAP5 for tear-free rotation.
The downside is: That chip can only do rotation up to 720x1280, so our display would not work.
If you use chip like that, make sure the rotation actually works. I don't know if they have something magical there, but normally you need a double buffer fb in the chip to support rotation. One buffer is written by the SoC, while the other is sent to the LCD. If there's just a single buffer, you'll get diagonal tearing.
Thanks for the hint!
I have looked into the data sheet and it appears that even 720x1280 drives the chip to its limits:
• Half-WUXGA (0.5 x 1920 x 1200 pixels) frame buffer • Support rotation up to 1280x800 resolution with SSL Compression • Support rotation up to 960x600 resolution without image compression
So it has only enough on-chip RAM for a single buffer.
And SSL-Compression means:
"SSL segment-based (4 pixels per segment) data compression/decompression with zero software overhead"
So it appears that we won't get full resolution 720x1280 rotated with that chip with less problems than with OMAP5 TILER! The problems will just be somewhere else.
And one more thing to consider:
110 mA @ 1.2V = 130 mW.
This means (assuming 90% efficiency) 40 mA @ 3.7V additional battery drain (in addition to the panel).
Well, this is difficult to compare to the additional energy consumption of the OMAP5 and TILER.
BR, Nikolaus
Am Tue, 1 Apr 2014 15:58:52 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Hi,
And one more thing to consider:
110 mA @ 1.2V = 130 mW.
This means (assuming 90% efficiency) 40 mA @ 3.7V additional battery drain (in addition to the panel).
Hmm, one of their main advertisements for the chip is that you safe power since it supports special automatic power saving features for MIPI display.
So it might need power, but it could be it saves even more somehow.
Something to check as well!
Am 01.04.2014 um 19:02 schrieb Michael Mrozek:
Am Tue, 1 Apr 2014 15:58:52 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Hi,
And one more thing to consider:
110 mA @ 1.2V = 130 mW.
This means (assuming 90% efficiency) 40 mA @ 3.7V additional battery drain (in addition to the panel).
Hmm, one of their main advertisements for the chip is that you safe power since it supports special automatic power saving features for MIPI display.
Hm, In video mode there is no power saving. Only if the panel itself has a built-in frame buffer. Then, the communication can be shut down if nothing changes.
But we are all the time discussion real time image sequences with extremely high frame rate. These are the most challenging for any system.
So there is nothing to save...
So it might need power, but it could be it saves even more somehow.
Well, it would allow to turn off the OMAP5 completely to display a constant image. But that is not what we want...
Something to check as well!
-- Mit freundlichen Grüßen,
Michael Mrozek
OpenPandora GmbH Geschäftsführer: Michael Mrozek
Schäffbräustr. 11 85049 Ingolstadt Deutschland Tel.: 0841 / 13 177 33 http://www.openpandora.de/ HRB 4879, Amtsgericht Ingolstadt
ICQ: 15691903 MSN: EvilDragon1717@hotmail.com eMail: info@openpandora.de _______________________________________________ Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
Am Tue, 1 Apr 2014 20:42:44 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Hi,
Hmm, one of their main advertisements for the chip is that you safe power since it supports special automatic power saving features for MIPI display.
Hm, In video mode there is no power saving. Only if the panel itself has a built-in frame buffer. Then, the communication can be shut down if nothing changes. But we are all the time discussion real time image sequences with extremely high frame rate. These are the most challenging for any system. So there is nothing to save...
You didn't check the datasheet? It's right on Page 12 (Feature list):
VTCM has the following features:
"• DCS command or video bypass together with data compression (SSL segment-based & Qualcomm line-based) to reduce the bandwidth requirement during data transmission."
More info: "The Video compression unit allows compression of the video stream before going into MIPI DSI TX module for MIPI link transmission. This will reduce the MIPI link speed for lower power consumptions. It supports SSL-segment based compression"
We need to use SSL compression for rotation anyways.
"• Built-in frame buffer to support video data self-refresh, so the refresh frequency from application processor can be reduced or stopped during static frames. "
Might be useful when using it to read PDFs, when not much happens.
"• Low power mode support via frame skipping"
Hmmm... interesting when using X as well. Skipping frames when using a browser could be useful.
"• Variable pixel clock generation to dynamically change video refresh rate for power saving"
No idea exactly what that does.
I have no ideas how easy / hard that is to use, but maybe some stuff works automatically :)
Am 02.04.2014 um 14:11 schrieb Michael Mrozek:
Am Tue, 1 Apr 2014 20:42:44 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Hi,
Hmm, one of their main advertisements for the chip is that you safe power since it supports special automatic power saving features for MIPI display.
Hm, In video mode there is no power saving. Only if the panel itself has a built-in frame buffer. Then, the communication can be shut down if nothing changes. But we are all the time discussion real time image sequences with extremely high frame rate. These are the most challenging for any system. So there is nothing to save...
You didn't check the datasheet?
Not in all details.
It's right on Page 12 (Feature list):
VTCM has the following features:
"• DCS command or video bypass together with data compression (SSL segment-based & Qualcomm line-based) to reduce the bandwidth requirement during data transmission."
More info: "The Video compression unit allows compression of the video stream before going into MIPI DSI TX module for MIPI link transmission. This will reduce the MIPI link speed for lower power consumptions. It supports SSL-segment based compression"
We need to use SSL compression for rotation anyways.
Two things come to my mind: 1. dioes the panel support such a SSL compressed video stream? 2. compression works for movies here the image quality can be reduced whithout the eye seeing the difference
"• Built-in frame buffer to support video data self-refresh, so the refresh frequency from application processor can be reduced or stopped during static frames. "
Might be useful when using it to read PDFs, when not much happens.
But there we also don't have problems with TILER :)
"• Low power mode support via frame skipping"
Hmmm... interesting when using X as well. Skipping frames when using a browser could be useful.
"• Variable pixel clock generation to dynamically change video refresh rate for power saving"
No idea exactly what that does.
They reduce refresh rate to save power. Useless if we want to play 60fps high resolution games.
I have no ideas how easy / hard that is to use, but maybe some stuff works automatically :)
Yes, by reducing image quality.
I think we start to compare apples with peaches.
We must keep in mind what we want to achieve:
highest frame rate at highest possible resolution without quality reduction
TILER appears to have problems with 60fps, Full-HD, no compression
If we reduce fps, resolution or compression, TILER is likely to be ok. The rotator chip can save power if we go below 60fps, Half-HD, and use compression...
BR, Nikolaus
Am Wed, 2 Apr 2014 14:23:56 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Hi,
We need to use SSL compression for rotation anyways.
Two things come to my mind:
- dioes the panel support such a SSL compressed video stream?
Yes, BOE and Solomon Systech are sitting in the same building and working together. The Solomon chip is already being used with that exact display.
- compression works for movies here the image quality can be reduced
whithout the eye seeing the difference
Compression works for anything, depending on algorithms being used. TIFF and TGA support lossless (LZMA) compression as well.
"• Built-in frame buffer to support video data self-refresh, so the refresh frequency from application processor can be reduced or stopped during static frames. " Might be useful when using it to read PDFs, when not much happens.
But there we also don't have problems with TILER :)
What does that have to do with the power saving features of the chip? Right now it seems TILER will both cause a huge impact on the CPU as well as make coding more complex as it needs a special format, whereas the Solomon Systech chip CAN do rotation and scaling without the need of CPU power or special code. The kernel would simply see it as landscape display.
So we probably need that chip, but additionally, it offers power saving features for other use cases as well.
"• Low power mode support via frame skipping" Hmmm... interesting when using X as well. Skipping frames when using a browser could be useful. "• Variable pixel clock generation to dynamically change video refresh rate for power saving" No idea exactly what that does.
They reduce refresh rate to save power. Useless if we want to play 60fps high resolution games.
Which is perfectly fine, as we don't alway play 60fps high resolution games. It's no PSP, it's a Mini Linux PC that also offers gaming.
Additionally, if the OMAP uses 50% CPU just for the rotation, it'll need more battery power than the Solomon chip.
I have no ideas how easy / hard that is to use, but maybe some stuff works automatically :)
Yes, by reducing image quality.
Which is something you don't even know yet, and especially not how bad it is.
We must keep in mind what we want to achieve:
highest frame rate at highest possible resolution without quality reduction
We need to find the solution with the least negative impacts.
If we use TILER, we've got the following features:
NEGATIVE: Special code is needed for SDL / Framebuffer stuff -> Not-ported games will run rotated -> Other Linux-Distributions need to be changed as well NEGATIVE: Rotation will probably ALWAYS impact the CPU, regardless how fast we can get it.
NEGATIVE: Right now, it's too slow to even rotate and scale 320x240 in fullspeed - we don't know how to do improve that right now.
POSITIVE: No additional chip / hardware needed
The 1080x1920 display has the issue that we need to work A LOT in the Linux environment to make it usable. Even simple things like GIMP need modifications as the icons are too small to be used. FireFox needs an Addon, otherwise the text will always be too small when you start it (doesn't support DPI).
These are just two apps I tried, there are probably a lot more.
If we use the Solomon Systech Chip, we've got the following issues:
POSITIVE: The display appears as landscape display for the Linux OS, no special code needed, works out of the box with all programs.
POSITIVE: The chip does scaling and rotating without tearing and without using additional CPU power
POSITIVE: LCD + Solomon Chip (4,45 EUR) has a way lower MOQ and is about half the price of the FullHD LCD
NEGATIVE: The chip might need a bit more power, though these might be possible to compensate with the power saving features
NEGATIVE: Picture might have some compression artifacts (but that needs to be tested).
We've got limited manpower, so the less we need to care about changing code of normal games and apps just to make them work with TILER, the better.
TILER appears to have problems with 60fps, Full-HD, no compression If we reduce fps, resolution or compression, TILER is likely to be ok.
Right now, it would not even be able to run a 320x240 MegaDrive emulation fullspeed.
That's something the GP2X from 2005 could do...
The rotator chip can save power if we go below 60fps, Half-HD, and use compression...
Don't forget that the conversion will probably ALWAYS need CPU power as well. So it could very well be that the OMAP5 doing the conversion and then use TILER will need more power than the Solomon Systech chip.
On 02/04/14 15:54, Michael Mrozek wrote:
We've got limited manpower, so the less we need to care about changing code of normal games and apps just to make them work with TILER, the better.
There are lots of moving parts here, but I agree with Michael. As I've pointed out, my foremost recommendation is to get a portrait LCD. Having a chip between the SoC and the LCD that does the rotation (presuming it works perfectly), is almost as good.
Even if TILER worked quickly with CPU, it would require extra hassle in SW to operate.
And to defend my employer's SoC a bit, I believe TILER is a fine piece of HW, and works perfectly in the use cases it's designed for. It wasn't meant to make portrait LCDs look like landscape LCDs in a transparent manner, updated with CPU, which is what's needed here.
Oh, and one question about the chip (I don't have time to properly read the spec =): does the MIPI RX side support only DSI command mode, or both command mode and video mode? If it supports DSI video mode, then there are no extra modifications needed to the SW to update the display (which I mentioned earlier).
Even in that case it may be worth looking at the DSI Command mode support, as that can save power. But with DSI video mode it'd be easy (easier...) to get it running.
Also, it needs to be verified that the chip can receive full resolution picture at 60Hz. Maybe it can, but Table 11-3: DC Characteristics lists as a test condition 1920x1080 Playback MIPIRX@30Hz (command mode).
Tomi
Am 02.04.2014 um 14:54 schrieb Michael Mrozek:
Am Wed, 2 Apr 2014 14:23:56 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Hi,
We need to use SSL compression for rotation anyways.
Two things come to my mind:
- dioes the panel support such a SSL compressed video stream?
Yes, BOE and Solomon Systech are sitting in the same building and working together. The Solomon chip is already being used with that exact display.
What is the application? A TV (Moviel) screen or a computer display?
For movies they might get a good and completely invisible compression. But for Windows, Text, Line graphics it might not work well.
- compression works for movies here the image quality can be reduced
whithout the eye seeing the difference
Compression works for anything, depending on algorithms being used. TIFF and TGA support lossless (LZMA) compression as well.
But any lossless compression may get a compression rate of 0%. The system must be able to handle this special case as well.
"• Built-in frame buffer to support video data self-refresh, so the refresh frequency from application processor can be reduced or stopped during static frames. " Might be useful when using it to read PDFs, when not much happens.
But there we also don't have problems with TILER :)
What does that have to do with the power saving features of the chip?
For a static image the TILER only needs to rotate once.
Right now it seems TILER will both cause a huge impact on the CPU as well as make coding more complex as it needs a special format, whereas the Solomon Systech chip CAN do rotation and scaling without the need of CPU power or special code. The kernel would simply see it as landscape display.
No. The kernel must program the chip so that it works as intended. So there is an additional driver in the kernel. Although it will likely be straightforward and not very complex.
So we probably need that chip, but additionally, it offers power saving features for other use cases as well.
"• Low power mode support via frame skipping" Hmmm... interesting when using X as well. Skipping frames when using a browser could be useful. "• Variable pixel clock generation to dynamically change video refresh rate for power saving" No idea exactly what that does.
They reduce refresh rate to save power. Useless if we want to play 60fps high resolution games.
Which is perfectly fine, as we don't alway play 60fps high resolution games. It's no PSP, it's a Mini Linux PC that also offers gaming.
Additionally, if the OMAP uses 50% CPU just for the rotation, it'll need more battery power than the Solomon chip.
I have no ideas how easy / hard that is to use, but maybe some stuff works automatically :)
Yes, by reducing image quality.
Which is something you don't even know yet, and especially not how bad it is.
We will see it as soon as we have built a prototype.
We must keep in mind what we want to achieve:
highest frame rate at highest possible resolution without quality reduction
We need to find the solution with the least negative impacts.
If we use TILER, we've got the following features:
NEGATIVE: Special code is needed for SDL / Framebuffer stuff -> Not-ported games will run rotated -> Other Linux-Distributions need to be changed as well NEGATIVE: Rotation will probably ALWAYS impact the CPU, regardless how fast we can get it.
NEGATIVE: Right now, it's too slow to even rotate and scale 320x240 in fullspeed - we don't know how to do improve that right now.
The longer I think about it it appears to me that we are simply not sure how to use the TILER efficiently (see below).
POSITIVE: No additional chip / hardware needed
The 1080x1920 display has the issue that we need to work A LOT in the Linux environment to make it usable. Even simple things like GIMP need modifications as the icons are too small to be used. FireFox needs an Addon, otherwise the text will always be too small when you start it (doesn't support DPI).
These are just two apps I tried, there are probably a lot more.
Maybe I have a completely different pipelines in mind from my studying of the TRM and I must admit that I still don't exactly know what Notaz's experiments do where we get the 320x240 slowness.
Let's try to describe in words what I assume how it should be:
1. application (e.g. wants to display 480x800 screen 50fps) -> 2. linear memory mapped frame buffer -> 3. highly optimized kernel driver copying (low resolution = less bandwidth!) to -> 4. TILER compatible double buffer (if needed) BUT NOT DOING ANY ROTATION -> 5. TILER to rotate (ONLY READS from DRAM into DISPC - so no WRITE slowness) -> 6. DISPC Scaler Unit -> 7. DISPC/DSI -> 8. MIPI-> 9. panel (1080 x 1900 x 60fps)
Other applications (that are aware of the TILER layout) can directly write to the stage 4. and even turn off the upscaling to get full-hd.
Unfortunately I am only able to write this down but not to program a proof of concept to check and compare performance to convince anyone - or find out why it does not work this way.
All this TILER things would have to be set up only once during boot and just the Scaler Unit must be adjusted during run-time to different framebuffer dimensions.
With this I would assume we also have:
POSITIVE: The display appears as landscape display for the Linux OS, no special code needed, works out of the box with all programs.
POSITIVE: The chip does scaling and rotating without tearing and without using additional CPU power (as long as the application can cope with TILER formatted frame buffers which is for X, Qt, Wayland (?) and perhaps SDL)
Only applications that can NOT use the TILER memory layout natively need the CPU supported copy process. I would expect that this is a minority and are appllcations not assuming Full-HD anyways.
Overlays may add more complexity (but is there an example on the Pandora where overlays are really used?).
If we use the Solomon Systech Chip, we've got the following issues:
Just for completeness how the pipeline would look like with Solomon chip:
1. application (e.g. 480x800 screen 50fps) -> 2. memory mapped frame buffer -> 6. DISPC Scaler Unit -> 7. DISPC/DSI -> 8. MIPI -> 9. Rotator Chip -> 10. MIPI -> 11. panel (720 x 1280 x 60fps)
POSITIVE: The display appears as landscape display for the Linux OS, no special code needed, works out of the box with all programs.
POSITIVE: The chip does scaling and rotating without tearing and without using additional CPU power
POSITIVE: LCD + Solomon Chip (4,45 EUR) has a way lower MOQ and is about half the price of the FullHD LCD
POSITIVE: no limitations with overlays
NEGATIVE: you get 44% of the pixels at 50% of the price
NEGATIVE: The chip might need a bit more power, though these might be possible to compensate with the power saving features
NEGATIVE: Picture might have some compression artifacts (but that needs to be tested).
We've got limited manpower, so the less we need to care about changing code of normal games and apps just to make them work with TILER, the better.
TILER appears to have problems with 60fps, Full-HD, no compression If we reduce fps, resolution or compression, TILER is likely to be ok.
Right now, it would not even be able to run a 320x240 MegaDrive emulation fullspeed.
That's something the GP2X from 2005 could do...
The rotator chip can save power if we go below 60fps, Half-HD, and use compression...
Don't forget that the conversion will probably ALWAYS need CPU power as well. So it could very well be that the OMAP5 doing the conversion and then use TILER will need more power than the Solomon Systech chip.
Yes, that could be. But would use the more power to get higher resolution. If we would achive the same resolution with more power it would be a bad solution...
I am just the "Advocatus for the highest resolution display" :)
BR, Nikolaus
Am Wed, 2 Apr 2014 15:51:09 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Hi,
Yes, BOE and Solomon Systech are sitting in the same building and working together. The Solomon chip is already being used with that exact display
What is the application? A TV (Moviel) screen or a computer display?
Many gaming smartphones in China.
For movies they might get a good and completely invisible compression. But for Windows, Text, Line graphics it might not work well.
Why? I've got codecs on my video editing machine that generally does lossless compression (mathematical) and only uses lossy algorithms in case the bandwitdh is not enough.
Almost any desktop picture can easily compressed lossless to about 20% of the size, as colors don't change that much (windows often have 90% the same color, etc.).
I've just made a snapshot of my desktop with about 4 different open windows full of text, a background image, etc.
Uncompressed (BMP) it's 4,1MB, with compression (LZW) TIFF (lossless) it's 1MB, with highest PNG compression (lossless as well) it's 726kB.
I cannot see a reason why a standard desktop screen should not fit into 50% with mathematical compression.
- compression works for movies here the image quality can be
reduced whithout the eye seeing the difference
Compression works for anything, depending on algorithms being used. TIFF and TGA support lossless (LZMA) compression as well.
But any lossless compression may get a compression rate of 0%. The system must be able to handle this special case as well.
Yes, in which case lossy compression will be added. That already works fine with various encoders I use on my video editing machine: I set a maximum bitrate and can do picture-per-picture compression (I-Frame only, so no moving estimation) and it's only doing lossy compression in case the bitrate is set too low.
That's nothing special these days.
Might be useful when using it to read PDFs, when not much happens.
But there we also don't have problems with TILER :)
What does that have to do with the power saving features of the chip?
For a static image the TILER only needs to rotate once.
Which is not an issue. The issue is with non-static images, which also need to work on our system.
Right now it seems TILER will both cause a huge impact on the CPU as well as make coding more complex as it needs a special format, whereas the Solomon Systech chip CAN do rotation and scaling without the need of CPU power or special code. The kernel would simply see it as landscape display.
No. The kernel must program the chip so that it works as intended. So there is an additional driver in the kernel. Although it will likely be straightforward and not very complex.
According to Solomon Systech (and as I can see from the examples I've sent earlier this morning), you send the MIPI commands to the Solomon Systech chip instead of to the display.
So instead of a display driver talking to the display you got a "display" driver talking to the Solomon chip, which then takes care of the rest.
Yes, by reducing image quality.
Which is something you don't even know yet, and especially not how bad it is.
We will see it as soon as we have built a prototype.
I've got all datasheets and will send them to you tonight. 5 displays will be ready in about 20 days, the Solomon Systech chips will probably arrive next week.
The longer I think about it it appears to me that we are simply not sure how to use the TILER efficiently (see below).
Well, according to Tomi it seems TILER was never intended to be used to rotate a full display.
Maybe I have a completely different pipelines in mind from my studying of the TRM and I must admit that I still don't exactly know what Notaz's experiments do where we get the 320x240 slowness.
I don't know exactly where the issue is as well, but as far as I have understood it's the copying into TILERs memory that slows things down a lot.
All this TILER things would have to be set up only once during boot and just the Scaler Unit must be adjusted during run-time to different framebuffer dimensions.
That would be perfect if it works like that, but somehow it seems it doesn't work like that.
If we use the Solomon Systech Chip, we've got the following issues:
Just for completeness how the pipeline would look like with Solomon chip:
- application (e.g. 480x800 screen 50fps) ->
- memory mapped frame buffer ->
- MIPI ->
- Rotator Chip ->
- MIPI ->
- panel (720 x 1280 x 60fps)
Removed the scaler, as the rotator chip can scale and rotate.
NEGATIVE: you get 44% of the pixels at 50% of the price
MOQ: LG: 5000 = 400,000 EUR MOQ: BOE: 1000 = 30,000 EUR
If you have 370,000 EUR you can lend me until we sold the first 2000 - 3000 Pyras, then we could see that as negative point, yes :)
Am 02.04.2014 um 16:15 schrieb Michael Mrozek:
Am Wed, 2 Apr 2014 15:51:09 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Hi,
Yes, BOE and Solomon Systech are sitting in the same building and working together. The Solomon chip is already being used with that exact display
What is the application? A TV (Moviel) screen or a computer display?
Many gaming smartphones in China.
For movies they might get a good and completely invisible compression. But for Windows, Text, Line graphics it might not work well.
Why? I've got codecs on my video editing machine that generally does lossless compression (mathematical) and only uses lossy algorithms in case the bandwitdh is not enough.
Almost any desktop picture can easily compressed lossless to about 20% of the size, as colors don't change that much (windows often have 90% the same color, etc.).
I've just made a snapshot of my desktop with about 4 different open windows full of text, a background image, etc.
Uncompressed (BMP) it's 4,1MB, with compression (LZW) TIFF (lossless) it's 1MB, with highest PNG compression (lossless as well) it's 726kB.
I cannot see a reason why a standard desktop screen should not fit into 50% with mathematical compression.
Because it depends on the content if it can be compressed. There are (rare) situations where you can't compress the image losless to 50%.
- compression works for movies here the image quality can be
reduced whithout the eye seeing the difference
Compression works for anything, depending on algorithms being used. TIFF and TGA support lossless (LZMA) compression as well.
But any lossless compression may get a compression rate of 0%. The system must be able to handle this special case as well.
Yes, in which case lossy compression will be added. That already works fine with various encoders I use on my video editing machine: I set a maximum bitrate and can do picture-per-picture compression (I-Frame only, so no moving estimation) and it's only doing lossy compression in case the bitrate is set too low.
That's nothing special these days.
But that is IMHO a false analogy. Because your video editing equipment can do that it does not mean that the chip does it.
Might be useful when using it to read PDFs, when not much happens.
But there we also don't have problems with TILER :)
What does that have to do with the power saving features of the chip?
For a static image the TILER only needs to rotate once.
Which is not an issue. The issue is with non-static images, which also need to work on our system.
Right now it seems TILER will both cause a huge impact on the CPU as well as make coding more complex as it needs a special format, whereas the Solomon Systech chip CAN do rotation and scaling without the need of CPU power or special code. The kernel would simply see it as landscape display.
No. The kernel must program the chip so that it works as intended. So there is an additional driver in the kernel. Although it will likely be straightforward and not very complex.
According to Solomon Systech (and as I can see from the examples I've sent earlier this morning), you send the MIPI commands to the Solomon Systech chip instead of to the display.
So instead of a display driver talking to the display you got a "display" driver talking to the Solomon chip, which then takes care of the rest.
Yes, exactly what I mean.
Yes, by reducing image quality.
Which is something you don't even know yet, and especially not how bad it is.
We will see it as soon as we have built a prototype.
I've got all datasheets and will send them to you tonight. 5 displays will be ready in about 20 days, the Solomon Systech chips will probably arrive next week.
Fine!
The longer I think about it it appears to me that we are simply not sure how to use the TILER efficiently (see below).
Well, according to Tomi it seems TILER was never intended to be used to rotate a full display.
Maybe I have a completely different pipelines in mind from my studying of the TRM and I must admit that I still don't exactly know what Notaz's experiments do where we get the 320x240 slowness.
I don't know exactly where the issue is as well, but as far as I have understood it's the copying into TILERs memory that slows things down a lot.
Yes, and that is where I understood (wrongly?) the TRM that the TILER can be used directly by the DISPC to read only from memory. I.e. it does not write anything because it directly goes to the panel. No write, no write problems.
But I may have misinterpreted the TRM completely:
10.2.1 DISPC Overview • Common: – Rotation 0, 90, 180, and 270 degrees using DMM-TILER • DMA (internal to the DISPC): – Support for accessing tiled structure through the TILER inside the dynamic memory management (DMM) – Support for accessing nontiled structure through the TILER or directly – Support for rotation, flip-flop, and mirroring though the TILER inside the DMM – Support for memory fragmentation though the TILER inside the DMM – Integrated shared buffers between DMA engine and pipelines
10.2.4.7 DISPC Rotation and Mirroring The DISPC provides flexible mechanisms for efficient implementation of rotation using the DISPC, its DMA engine, and the rotation engine of the TILER. The rotation is handled only through the TILER, which supplies the encoded pixels to the DISPC.
Table 10-27. DISPC Register Settings for Rotation Using TILER
So it appears that the TI designers have thought about efficient interaction between TILER and DISPC.
All this TILER things would have to be set up only once during boot and just the Scaler Unit must be adjusted during run-time to different framebuffer dimensions.
That would be perfect if it works like that, but somehow it seems it doesn't work like that.
If we use the Solomon Systech Chip, we've got the following issues:
Just for completeness how the pipeline would look like with Solomon chip:
- application (e.g. 480x800 screen 50fps) ->
- memory mapped frame buffer ->
- MIPI ->
- Rotator Chip ->
- MIPI ->
- panel (720 x 1280 x 60fps)
Removed the scaler, as the rotator chip can scale and rotate.
Yes, you are right.
What I don't exactly get matched with our discussions are the specs on their web page:
SSD2848 4-lane MIPI in, 4-lane MIPI out bridge chip Transmission rate support up to 4 x 1.5Gbps lanes. display up to 1920*1200 Mass Production Send Request SSD2858 4-lane MIPI in, 8-lane MIPI out bridge chip Rx support up to 4 x 1.5Gbps lanes, Tx support up to 8 x 1.0Gbps lanes, display up to 2560*1600
This would mean we can use it with the LG Full-HD panel?
NEGATIVE: you get 44% of the pixels at 50% of the price
MOQ: LG: 5000 = 400,000 EUR MOQ: BOE: 1000 = 30,000 EUR
If you have 370,000 EUR you can lend me until we sold the first 2000 - 3000 Pyras, then we could see that as negative point, yes :)
Killer argument :)
If you want a better display you would try to get the 400k€. If you believe in 5000 per year, it does not matter.
But don't mind. We will do the experiments with the rotator and see.
BR, Nikolaus
-- Mit freundlichen Grüßen,
Michael Mrozek
OpenPandora GmbH Geschäftsführer: Michael Mrozek
Schäffbräustr. 11 85049 Ingolstadt Deutschland Tel.: 0841 / 13 177 33 http://www.openpandora.de/ HRB 4879, Amtsgericht Ingolstadt
ICQ: 15691903 MSN: EvilDragon1717@hotmail.com eMail: info@openpandora.de _______________________________________________ Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
Am Wed, 2 Apr 2014 16:49:24 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Hi,
I've just made a snapshot of my desktop with about 4 different open windows full of text, a background image, etc. Uncompressed (BMP) it's 4,1MB, with compression (LZW) TIFF (lossless) it's 1MB, with highest PNG compression (lossless as well) it's 726kB. I cannot see a reason why a standard desktop screen should not fit into 50% with mathematical compression
Because it depends on the content if it can be compressed. There are (rare) situations where you can't compress the image losless to 50%.
Extremely rare. On a normal desktop PC, I cannot see that happen, seriously.
- compression works for movies here the image quality can be
reduced whithout the eye seeing the difference
Compression works for anything, depending on algorithms being used. TIFF and TGA support lossless (LZMA) compression as well.
But any lossless compression may get a compression rate of 0%. The system must be able to handle this special case as well.
Yes, in which case lossy compression will be added. That already works fine with various encoders I use on my video editing machine: I set a maximum bitrate and can do picture-per-picture compression (I-Frame only, so no moving estimation) and it's only doing lossy compression in case the bitrate is set too low. That's nothing special these days.
But that is IMHO a false analogy. Because your video editing equipment can do that it does not mean that the chip does it.
The chip has been specifically designed to do that. The main usage is NOT rotation, the main usage is powersaving for smartphones, with using that compression, etc.
With that being the top priority, I'm pretty sure it would work like that. We'll see, but it would make me wonder.
The kernel would simply see it as landscape display.
No. The kernel must program the chip so that it works as intended. So there is an additional driver in the kernel. Although it will likely be straightforward and not very complex.
According to Solomon Systech (and as I can see from the examples I've sent earlier this morning), you send the MIPI commands to the Solomon Systech chip instead of to the display. So instead of a display driver talking to the display you got a "display" driver talking to the Solomon chip, which then takes care of the rest.
Yes, exactly what I mean.
Which means there's no additional driver, but a different driver in the kernel. Which would be the same for any display we use ;)
I don't know exactly where the issue is as well, but as far as I have understood it's the copying into TILERs memory that slows things down a lot.
Yes, and that is where I understood (wrongly?) the TRM that the TILER can be used directly by the DISPC to read only from memory. I.e. it does not write anything because it directly goes to the panel. No write, no write problems.
But somehow you need to get the data into the TILERs memory so that it can read it.
Which is the issue, as far as I understood.
10.2.1 DISPC Overview • Common: – Rotation 0, 90, 180, and 270 degrees using DMM-TILER • DMA (internal to the DISPC): – Support for accessing tiled structure through the TILER inside the dynamic memory management (DMM) – Support for accessing nontiled structure through the TILER or directly – Support for rotation, flip-flop, and mirroring though the TILER inside the DMM – Support for memory fragmentation though the TILER inside the DMM – Integrated shared buffers between DMA engine and pipelines
10.2.4.7 DISPC Rotation and Mirroring The DISPC provides flexible mechanisms for efficient implementation of rotation using the DISPC, its DMA engine, and the rotation engine of the TILER. The rotation is handled only through the TILER, which supplies the encoded pixels to the DISPC.
Table 10-27. DISPC Register Settings for Rotation Using TILER
So it appears that the TI designers have thought about efficient interaction between TILER and DISPC.
Well, it seems once we an solve the issue to send the data correctly to TILERs memory, it should work.
I do wonder: What IS the use of TILER, if it wasn't meant to rotate a display from portrait to landscape?
What other cases are there where you need rotation?
What I don't exactly get matched with our discussions are the specs on their web page:
SSD2848 4-lane MIPI in, 4-lane MIPI out bridge chip Transmission rate support up to 4 x 1.5Gbps lanes. display up to 1920*1200 Mass Production Send Request SSD2858 4-lane MIPI in, 8-lane MIPI out bridge chip Rx support up to 4 x 1.5Gbps lanes, Tx support up to 8 x 1.0Gbps lanes, display up to 2560*1600
This would mean we can use it with the LG Full-HD panel?
Yes, but not for rotating, just for color correction, scaling, etc.
NEGATIVE: you get 44% of the pixels at 50% of the price
MOQ: LG: 5000 = 400,000 EUR MOQ: BOE: 1000 = 30,000 EUR If you have 370,000 EUR you can lend me until we sold the first 2000 - 3000 Pyras, then we could see that as negative point, yes :)
Killer argument :) If you want a better display you would try to get the 400k€.
If we can get the LG display with TILER working without huge performance or battery hit, the way we want it to work, then I am up for that challenge.
If you believe in 5000 per year, it does not matter.
It does. It does a lot.
If you are a company which has a huge amount of money, it's no problem - as you will get the money back after a while.
However, after the development and pre-production process, there will be approximately 100k EUR left for production.
If the production costs of one Pyra is approx. 300 EUR, then a batch of 1000 would costs about 300,000 EUR (probably a bit more, because of MOQ of cases, etc.).
As I'm working with GC, I know I have to pay 60% in advance, remaining 40% after delivery.
So I'd need to pay about 200,000 EUR in advance (not everything is coming from GC).
Which means: With 1000 orders, all I'd need would be a prepayment of 100 EUR from the community, which is most probably possible.
That would be enough to start the production.
I plan to earn about 200 EUR per Pyra, so with 1000 sold Pyras, I easily have enough money for the next batch and already have most of the production costs back.
With 2000 sold Pyras, all investors could be paid back.
If I have to buy 5000 LG displays at once and need to pay them in advance, I'd need about 600,000 EUR to get the first production run going.
If every customer paid the full price in advance, it would probably work - but I might still need more money. And I'd have a LOT less preorders if people had to pay the full price...
So either I'd need to get a loan or some more investors - both would mean less profit overall and therefore increase the possible price for the Pyra.
The issue is not that I think I wouldn't sell those 5000 units, the problem is that I'd need to pay those in advance which increases the costs for the first run A LOT.
And I doubt there's a bank with a 0% loan ;)
But don't mind. We will do the experiments with the rotator and see.
Well, I didn't order samples because I am 100% sure we'll switch to it! I did that because we can still continue working on the rest (CPU board, etc.), as we know we've got a backup solution in case we can't get the LG one to work properly.
BR, Nikolaus
-- Mit freundlichen Grüßen,
Michael Mrozek
OpenPandora GmbH Geschäftsführer: Michael Mrozek
Schäffbräustr. 11 85049 Ingolstadt Deutschland Tel.: 0841 / 13 177 33 http://www.openpandora.de/ HRB 4879, Amtsgericht Ingolstadt
ICQ: 15691903 MSN: EvilDragon1717@hotmail.com eMail: info@openpandora.de _______________________________________________ Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
On 02/04/14 18:14, Michael Mrozek wrote:
The kernel would simply see it as landscape display.
No. The kernel must program the chip so that it works as intended. So there is an additional driver in the kernel. Although it will likely be straightforward and not very complex.
According to Solomon Systech (and as I can see from the examples I've sent earlier this morning), you send the MIPI commands to the Solomon Systech chip instead of to the display. So instead of a display driver talking to the display you got a "display" driver talking to the Solomon chip, which then takes care of the rest.
Yes, exactly what I mean.
Which means there's no additional driver, but a different driver in the kernel. Which would be the same for any display we use ;)
One driver for one component. With the exception, say, of multiple very similar panels, where one driver could be used for all the panels.
Well, it seems once we an solve the issue to send the data correctly to TILERs memory, it should work.
Just to make sure it's clear, there's no memory in TILER. There's just normal RAM. TILER is a "wrapper" around an area in the RAM. Reads and writes to to the TILER, get re-mapped a bit, and read/written to RAM.
I do wonder: What IS the use of TILER, if it wasn't meant to rotate a display from portrait to landscape?
What other cases are there where you need rotation?
It is meant for rotation. I meant that it's not designed to, say, make a portrait LCD look like landscape to all the applications, even with CPU doing the drawing.
A normal use case for tiler would be with a smartphone with portrait LCD. UI is drawn with SGX, and it can handle the rotation so that it works right no matter if the user holds the phone in landscape or portrait orientation, no TILER needed.
When a video is shown, DSP is used to decode it to a video overlay, and this video overlay is managed with TILER. That way the DSP can decode a landscape video, which is then shown in portrait LCD correctly in landscape orientation.
Nothing quite prevents using TILER all the time, as in Pyra's case, except that the CPU access is clearly quite slow. And good CPU access performance to TILER wasn't probably on the list of requirements.
I've sent queries about it, so maybe we can find a perf boost for it. But I wouldn't bet on it.
Tomi
Am 02.04.2014 um 17:28 schrieb Tomi Valkeinen:
On 02/04/14 18:14, Michael Mrozek wrote:
The kernel would simply see it as landscape display.
No. The kernel must program the chip so that it works as intended. So there is an additional driver in the kernel. Although it will likely be straightforward and not very complex.
According to Solomon Systech (and as I can see from the examples I've sent earlier this morning), you send the MIPI commands to the Solomon Systech chip instead of to the display. So instead of a display driver talking to the display you got a "display" driver talking to the Solomon chip, which then takes care of the rest.
Yes, exactly what I mean.
Which means there's no additional driver, but a different driver in the kernel. Which would be the same for any display we use ;)
One driver for one component. With the exception, say, of multiple very similar panels, where one driver could be used for all the panels.
Well, it seems once we an solve the issue to send the data correctly to TILERs memory, it should work.
Just to make sure it's clear, there's no memory in TILER. There's just normal RAM. TILER is a "wrapper" around an area in the RAM. Reads and writes to to the TILER, get re-mapped a bit, and read/written to RAM.
What about the shared buffers between TILER and DISPC DMA mentioned in the TRM? Aren't they exactly for this situation to avoid RAM access?
I do wonder: What IS the use of TILER, if it wasn't meant to rotate a display from portrait to landscape?
What other cases are there where you need rotation?
It is meant for rotation. I meant that it's not designed to, say, make a portrait LCD look like landscape to all the applications, even with CPU doing the drawing.
A normal use case for tiler would be with a smartphone with portrait LCD. UI is drawn with SGX, and it can handle the rotation so that it works right no matter if the user holds the phone in landscape or portrait orientation, no TILER needed.
I think I don't get the the difference from our situation to always rotating a smart phone to landscape mode.
If that can be done, our smartphone panel (it is approx. the same as the Nexus 5) should be possible to get the same performance.
When a video is shown, DSP is used to decode it to a video overlay, and this video overlay is managed with TILER. That way the DSP can decode a landscape video, which is then shown in portrait LCD correctly in landscape orientation.
Nothing quite prevents using TILER all the time, as in Pyra's case, except that the CPU access is clearly quite slow. And good CPU access performance to TILER wasn't probably on the list of requirements.
I've sent queries about it, so maybe we can find a perf boost for it. But I wouldn't bet on it.
BR, Nikolaus
On 02/04/14 18:35, Dr. H. Nikolaus Schaller wrote:
Just to make sure it's clear, there's no memory in TILER. There's just normal RAM. TILER is a "wrapper" around an area in the RAM. Reads and writes to to the TILER, get re-mapped a bit, and read/written to RAM.
What about the shared buffers between TILER and DISPC DMA mentioned in the TRM? Aren't they exactly for this situation to avoid RAM access?
The text you pasted said "Integrated shared buffers between DMA engine and pipelines". Not TILER and DISPC DMA.
I do wonder: What IS the use of TILER, if it wasn't meant to rotate a display from portrait to landscape?
What other cases are there where you need rotation?
It is meant for rotation. I meant that it's not designed to, say, make a portrait LCD look like landscape to all the applications, even with CPU doing the drawing.
A normal use case for tiler would be with a smartphone with portrait LCD. UI is drawn with SGX, and it can handle the rotation so that it works right no matter if the user holds the phone in landscape or portrait orientation, no TILER needed.
I think I don't get the the difference from our situation to always rotating a smart phone to landscape mode.
A smart phone that I discussed is not always rotated. It's also (mostly) used in portrait mode. And even when it's used in landscape mode, only the required overlays are rotated with TILER.
If that can be done, our smartphone panel (it is approx. the same as the Nexus 5) should be possible to get the same performance.
The differences with Pyra and a smartphone I see:
- Pyra is always rotated, and all overlays need TILER rotation. Smartphone only needs TILER rotation in particular use cases.
- A smartphone has a single display system and framework, that can handle the needed rotation. On Pyra, "all" frameworks should be supported.
- Smartphone draws normally with SGX. Pyra draws normally (at least in some scenarios) with CPU.
But again, I think the main issue is the slowness of CPU access to TILER. If that issue wasn't there, maybe TILER could be used.
But even so, it is a hassle to use TILER in the way Pyra needs, compared to native landscape panel (or a chip that does the rotation). All frameworks must understand about the rotation and it must be done for all overlays, and the display still looks like portrait display.
Tomi
Am 02.04.2014 um 17:59 schrieb Tomi Valkeinen:
On 02/04/14 18:35, Dr. H. Nikolaus Schaller wrote:
Just to make sure it's clear, there's no memory in TILER. There's just normal RAM. TILER is a "wrapper" around an area in the RAM. Reads and writes to to the TILER, get re-mapped a bit, and read/written to RAM.
What about the shared buffers between TILER and DISPC DMA mentioned in the TRM? Aren't they exactly for this situation to avoid RAM access?
The text you pasted said "Integrated shared buffers between DMA engine and pipelines". Not TILER and DISPC DMA.
I do wonder: What IS the use of TILER, if it wasn't meant to rotate a display from portrait to landscape?
What other cases are there where you need rotation?
It is meant for rotation. I meant that it's not designed to, say, make a portrait LCD look like landscape to all the applications, even with CPU doing the drawing.
A normal use case for tiler would be with a smartphone with portrait LCD. UI is drawn with SGX, and it can handle the rotation so that it works right no matter if the user holds the phone in landscape or portrait orientation, no TILER needed.
I think I don't get the the difference from our situation to always rotating a smart phone to landscape mode.
A smart phone that I discussed is not always rotated. It's also (mostly) used in portrait mode. And even when it's used in landscape mode, only the required overlays are rotated with TILER.
Ok, I see - but in most cases it is used for playing HD videos in high frame rate and full screen mode.
So the performance requirements should not be much relaxed.
If that can be done, our smartphone panel (it is approx. the same as the Nexus 5) should be possible to get the same performance.
The differences with Pyra and a smartphone I see:
- Pyra is always rotated, and all overlays need TILER rotation.
Smartphone only needs TILER rotation in particular use cases.
It appears that only one overlay is usually used.
- A smartphone has a single display system and framework, that can
handle the needed rotation. On Pyra, "all" frameworks should be supported.
- Smartphone draws normally with SGX. Pyra draws normally (at least in
some scenarios) with CPU.
But again, I think the main issue is the slowness of CPU access to TILER. If that issue wasn't there, maybe TILER could be used.
But even so, it is a hassle to use TILER in the way Pyra needs, compared to native landscape panel (or a chip that does the rotation). All frameworks must understand about the rotation and it must be done for all overlays, and the display still looks like portrait display.
Unfortunatley there are no 5 inch landscape displays. It likely has to do something with the panel driver chips. In the LG Full-HD chip the Renesas chip is capable of driving 1080 columns and 1920 lines (or as many as needed). This is not enough for driving 1200 columns on a landscape panel with horizontal lines.
Other panel driver chips definitively exist (e.g. for the 4k screens or Apple retina displays) but they may be bigger and don't fit on the glass of a 5 inch panel.
So I would guess that the best we can realistically get is 600x1024 which would again be a reduction.
BR, Nikolaus
Am Wed, 2 Apr 2014 18:12:25 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Hi,
A smart phone that I discussed is not always rotated. It's also (mostly) used in portrait mode. And even when it's used in landscape mode, only the required overlays are rotated with TILER.
Ok, I see - but in most cases it is used for playing HD videos in high frame rate and full screen mode.
I'm pretty sure they do that with OpenGL ES.
If that can be done, our smartphone panel (it is approx. the same as the Nexus 5) should be possible to get the same performance.
The differences with Pyra and a smartphone I see:
- Pyra is always rotated, and all overlays need TILER rotation.
Smartphone only needs TILER rotation in particular use cases.
It appears that only one overlay is usually used.
No, the Pandora already uses up to 3 overlays (with DraStic).
Unfortunatley there are no 5 inch landscape displays. It likely has to do something with the panel driver chips. In the LG Full-HD chip the Renesas chip is capable of driving 1080 columns and 1920 lines (or as many as needed). This is not enough for driving 1200 columns on a landscape panel with horizontal lines.
AFAIR I found a Renesas SP driver IC which offers rotation and scaling as well.
So I would guess that the best we can realistically get is 600x1024 which would again be a reduction.
Yes, there is a 5" Sharp 1024x600 display, but that's LVDS (so LCD Cable issue once again) and who knows how long it will still be produced (it's from 2010).
The better solution to that is a 1280x720 portrait display that's being rotated and scaled with a dedicated hardware.
Hi,
attached is the datasheet for the BOE LCD as well as the driver IC.
According to the datasheet, the connector needed is: http://www.digikey.com/product-detail/en/FH26-27S-0.3SHW(05)/HFQ327TR-ND/735...
Let me know if you need anything else :)
Hi,
okay... I'm now getting more and more LCDs which I could buy. This one would be 20,75 EUR (which is an awesome price).
The specs are similar to the BOE one, typ response rate is 25ms. The connector is a bit weird, but it could be we can get this changed.
I've attached the datasheet for both the display and the driver IC.
Nikolaus: Can we build the next display board as multi-LCD PCB, so adding connectors for these two displays (BOE and Success)?
Then I'd order samples as well - and we could test and compare both together with the Solomon Chip and select the best one :)
Am 07.04.2014 um 18:36 schrieb Michael Mrozek:
Hi,
okay... I'm now getting more and more LCDs which I could buy. This one would be 20,75 EUR (which is an awesome price).
The specs are similar to the BOE one, typ response rate is 25ms. The connector is a bit weird, but it could be we can get this changed.
I've attached the datasheet for both the display and the driver IC.
Nikolaus: Can we build the next display board as multi-LCD PCB, so adding connectors for these two displays (BOE and Success)?
Hm. That is a little difficult. Each panel needs a slightly different power supply. MIPI should be the same for all.
But I will check if these two are reasonably similar.
Then I'd order samples as well - and we could test and compare both together with the Solomon Chip and select the best one :)
-- Mit freundlichen Grüßen,
Michael Mrozek
OpenPandora GmbH Geschäftsführer: Michael Mrozek
Schäffbräustr. 11 85049 Ingolstadt Deutschland Tel.: 0841 / 13 177 33 http://www.openpandora.de/ HRB 4879, Amtsgericht Ingolstadt
ICQ: 15691903 MSN: EvilDragon1717@hotmail.com eMail: info@openpandora.de <NT35590B.pdf><SPEC-S90451-DI050HD V02(A1.1).pdf>_______________________________________________ Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
Am 07.04.2014 um 19:19 schrieb Dr. H. Nikolaus Schaller:
Am 07.04.2014 um 18:36 schrieb Michael Mrozek:
Hi,
okay... I'm now getting more and more LCDs which I could buy. This one would be 20,75 EUR (which is an awesome price).
The specs are similar to the BOE one, typ response rate is 25ms. The connector is a bit weird, but it could be we can get this changed.
I've attached the datasheet for both the display and the driver IC.
Nikolaus: Can we build the next display board as multi-LCD PCB, so adding connectors for these two displays (BOE and Success)?
Hm. That is a little difficult. Each panel needs a slightly different power supply. MIPI should be the same for all.
But I will check if these two are reasonably similar.
Yes, we are lucky. They are almost the same and the backlight has the same spec (2x6x3.0V @ 20mA)..
Both are a little better that they just needs a 1.8V for I/O and ~3.3V for supply. No special +/-5V converter like the LG panel (where I still wonder why they did require us to have it externally where we know that the Renesas chip could do it autonomously).
Both even use the same Hirose connector series FH26 - but the BOE uses a 27 pin variant FH26-27S-03SHW and the Spectrum a 31 pin variant FH26-31S-0.3SHW(05).
So it look as as if I can add both connectors for testing purposes :)
Then I'd order samples as well - and we could test and compare both together with the Solomon Chip and select the best one :)
-- Mit freundlichen Grüßen,
Michael Mrozek
OpenPandora GmbH Geschäftsführer: Michael Mrozek
Schäffbräustr. 11 85049 Ingolstadt Deutschland Tel.: 0841 / 13 177 33 http://www.openpandora.de/ HRB 4879, Amtsgericht Ingolstadt
ICQ: 15691903 MSN: EvilDragon1717@hotmail.com eMail: info@openpandora.de <NT35590B.pdf><SPEC-S90451-DI050HD V02(A1.1).pdf>_______________________________________________ Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
Am Mon, 7 Apr 2014 19:31:54 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
That's great, I'll order samples of these as well then :)
Am 07.04.2014 um 19:19 schrieb Dr. H. Nikolaus Schaller:
Am 07.04.2014 um 18:36 schrieb Michael Mrozek:
Hi,
okay... I'm now getting more and more LCDs which I could buy. This one would be 20,75 EUR (which is an awesome price).
The specs are similar to the BOE one, typ response rate is 25ms. The connector is a bit weird, but it could be we can get this changed.
I've attached the datasheet for both the display and the driver IC.
Nikolaus: Can we build the next display board as multi-LCD PCB, so adding connectors for these two displays (BOE and Success)?
Hm. That is a little difficult. Each panel needs a slightly different power supply. MIPI should be the same for all.
But I will check if these two are reasonably similar.
Yes, we are lucky. They are almost the same and the backlight has the same spec (2x6x3.0V @ 20mA)..
Both are a little better that they just needs a 1.8V for I/O and ~3.3V for supply. No special +/-5V converter like the LG panel (where I still wonder why they did require us to have it externally where we know that the Renesas chip could do it autonomously).
Both even use the same Hirose connector series FH26 - but the BOE uses a 27 pin variant FH26-27S-03SHW and the Spectrum a 31 pin variant FH26-31S-0.3SHW(05).
So it look as as if I can add both connectors for testing purposes :)
Then I'd order samples as well - and we could test and compare both together with the Solomon Chip and select the best one :)
-- Mit freundlichen Grüßen,
Michael Mrozek
OpenPandora GmbH Geschäftsführer: Michael Mrozek
Schäffbräustr. 11 85049 Ingolstadt Deutschland Tel.: 0841 / 13 177 33 http://www.openpandora.de/ HRB 4879, Amtsgericht Ingolstadt
ICQ: 15691903 MSN: EvilDragon1717@hotmail.com eMail: info@openpandora.de <NT35590B.pdf><SPEC-S90451-DI050HD V02(A1.1).pdf>_______________________________________________ Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
Hi,
just a quick info:
I've received the first case design file last week. They're not yet finished (missing shoulderbuttons and the keymat is not finalized), but it's enough to do some first testings.
The lid is currently made for the LG LCD (which will probably change, as we all know).
It's enough to ask the hinge company in US whether they can offer us a hinge or not.
We'll also receive 2 full printed 3D case sets April 17th which can be used to test the sizes for the PCB (especially if the CPU board will fit) and for testing the nubs, etc.
Am 02.04.2014 um 17:14 schrieb Michael Mrozek:
Am Wed, 2 Apr 2014 16:49:24 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Hi,
I've just made a snapshot of my desktop with about 4 different open windows full of text, a background image, etc. Uncompressed (BMP) it's 4,1MB, with compression (LZW) TIFF (lossless) it's 1MB, with highest PNG compression (lossless as well) it's 726kB. I cannot see a reason why a standard desktop screen should not fit into 50% with mathematical compression
Because it depends on the content if it can be compressed. There are (rare) situations where you can't compress the image losless to 50%.
Extremely rare. On a normal desktop PC, I cannot see that happen, seriously.
- compression works for movies here the image quality can be
reduced whithout the eye seeing the difference
Compression works for anything, depending on algorithms being used. TIFF and TGA support lossless (LZMA) compression as well.
But any lossless compression may get a compression rate of 0%. The system must be able to handle this special case as well.
Yes, in which case lossy compression will be added. That already works fine with various encoders I use on my video editing machine: I set a maximum bitrate and can do picture-per-picture compression (I-Frame only, so no moving estimation) and it's only doing lossy compression in case the bitrate is set too low. That's nothing special these days.
But that is IMHO a false analogy. Because your video editing equipment can do that it does not mean that the chip does it.
The chip has been specifically designed to do that. The main usage is NOT rotation, the main usage is powersaving for smartphones, with using that compression, etc.
With that being the top priority, I'm pretty sure it would work like that. We'll see, but it would make me wonder.
The kernel would simply see it as landscape display.
No. The kernel must program the chip so that it works as intended. So there is an additional driver in the kernel. Although it will likely be straightforward and not very complex.
According to Solomon Systech (and as I can see from the examples I've sent earlier this morning), you send the MIPI commands to the Solomon Systech chip instead of to the display. So instead of a display driver talking to the display you got a "display" driver talking to the Solomon chip, which then takes care of the rest.
Yes, exactly what I mean.
Which means there's no additional driver, but a different driver in the kernel. Which would be the same for any display we use ;)
I don't know exactly where the issue is as well, but as far as I have understood it's the copying into TILERs memory that slows things down a lot.
Yes, and that is where I understood (wrongly?) the TRM that the TILER can be used directly by the DISPC to read only from memory. I.e. it does not write anything because it directly goes to the panel. No write, no write problems.
But somehow you need to get the data into the TILERs memory so that it can read it.
Which is the issue, as far as I understood.
That would be the application writing something to the frame buffer. That is unavoidable.
10.2.1 DISPC Overview • Common: – Rotation 0, 90, 180, and 270 degrees using DMM-TILER • DMA (internal to the DISPC): – Support for accessing tiled structure through the TILER inside the dynamic memory management (DMM) – Support for accessing nontiled structure through the TILER or directly – Support for rotation, flip-flop, and mirroring though the TILER inside the DMM – Support for memory fragmentation though the TILER inside the DMM – Integrated shared buffers between DMA engine and pipelines
10.2.4.7 DISPC Rotation and Mirroring The DISPC provides flexible mechanisms for efficient implementation of rotation using the DISPC, its DMA engine, and the rotation engine of the TILER. The rotation is handled only through the TILER, which supplies the encoded pixels to the DISPC.
Table 10-27. DISPC Register Settings for Rotation Using TILER
So it appears that the TI designers have thought about efficient interaction between TILER and DISPC.
Well, it seems once we an solve the issue to send the data correctly to TILERs memory, it should work.
I do wonder: What IS the use of TILER, if it wasn't meant to rotate a display from portrait to landscape?
What other cases are there where you need rotation?
Good question. Rotating some small windows in a full screen is almost useless. Unless you want to show a picture in picture camera that itself is rotated against the main display.
So it should be the *standard* use case to be able to rotate the full screen!
What I don't exactly get matched with our discussions are the specs on their web page:
SSD2848 4-lane MIPI in, 4-lane MIPI out bridge chip Transmission rate support up to 4 x 1.5Gbps lanes. display up to 1920*1200 Mass Production Send Request SSD2858 4-lane MIPI in, 8-lane MIPI out bridge chip Rx support up to 4 x 1.5Gbps lanes, Tx support up to 8 x 1.0Gbps lanes, display up to 2560*1600
This would mean we can use it with the LG Full-HD panel?
Yes, but not for rotating, just for color correction, scaling, etc.
Ah, I see.
NEGATIVE: you get 44% of the pixels at 50% of the price
MOQ: LG: 5000 = 400,000 EUR MOQ: BOE: 1000 = 30,000 EUR If you have 370,000 EUR you can lend me until we sold the first 2000 - 3000 Pyras, then we could see that as negative point, yes :)
Killer argument :) If you want a better display you would try to get the 400k€.
If we can get the LG display with TILER working without huge performance or battery hit, the way we want it to work, then I am up for that challenge.
If you believe in 5000 per year, it does not matter.
It does. It does a lot.
If you are a company which has a huge amount of money, it's no problem
- as you will get the money back after a while.
However, after the development and pre-production process, there will be approximately 100k EUR left for production.
If the production costs of one Pyra is approx. 300 EUR, then a batch of 1000 would costs about 300,000 EUR (probably a bit more, because of MOQ of cases, etc.).
As I'm working with GC, I know I have to pay 60% in advance, remaining 40% after delivery.
So I'd need to pay about 200,000 EUR in advance (not everything is coming from GC).
Which means: With 1000 orders, all I'd need would be a prepayment of 100 EUR from the community, which is most probably possible.
That would be enough to start the production.
I plan to earn about 200 EUR per Pyra, so with 1000 sold Pyras, I easily have enough money for the next batch and already have most of the production costs back.
With 2000 sold Pyras, all investors could be paid back.
If I have to buy 5000 LG displays at once and need to pay them in advance, I'd need about 600,000 EUR to get the first production run going.
If every customer paid the full price in advance, it would probably work - but I might still need more money. And I'd have a LOT less preorders if people had to pay the full price...
So either I'd need to get a loan or some more investors - both would mean less profit overall and therefore increase the possible price for the Pyra.
The issue is not that I think I wouldn't sell those 5000 units, the problem is that I'd need to pay those in advance which increases the costs for the first run A LOT.
And I doubt there's a bank with a 0% loan ;)
If you plan for 200 EUR earnings you can afford a >0% loan if it enables the plan. But that is of course your decision.
Or you could look for additional investors.
But don't mind. We will do the experiments with the rotator and see.
Well, I didn't order samples because I am 100% sure we'll switch to it! I did that because we can still continue working on the rest (CPU board, etc.), as we know we've got a backup solution in case we can't get the LG one to work properly.
BR, Nikolaus
-- Mit freundlichen Grüßen,
Michael Mrozek
OpenPandora GmbH Geschäftsführer: Michael Mrozek
Schäffbräustr. 11 85049 Ingolstadt Deutschland Tel.: 0841 / 13 177 33 http://www.openpandora.de/ HRB 4879, Amtsgericht Ingolstadt
ICQ: 15691903 MSN: EvilDragon1717@hotmail.com eMail: info@openpandora.de _______________________________________________ Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
-- Mit freundlichen Grüßen,
Michael Mrozek
OpenPandora GmbH Geschäftsführer: Michael Mrozek
Schäffbräustr. 11 85049 Ingolstadt Deutschland Tel.: 0841 / 13 177 33 http://www.openpandora.de/ HRB 4879, Amtsgericht Ingolstadt
ICQ: 15691903 MSN: EvilDragon1717@hotmail.com eMail: info@openpandora.de
On 02/04/14 17:15, Michael Mrozek wrote:
But any lossless compression may get a compression rate of 0%. The system must be able to handle this special case as well.
Yes, in which case lossy compression will be added. That already works fine with various encoders I use on my video editing machine: I set a maximum bitrate and can do picture-per-picture compression (I-Frame only, so no moving estimation) and it's only doing lossy compression in case the bitrate is set too low.
I could be wrong, but I don't think that's possible here. The chip receives a constant pixel stream, and it doesn't have extra memory. It can't suddenly decide that it needs to change the compression, and somehow re-compress the already received parts of the frame.
No. The kernel must program the chip so that it works as intended. So there is an additional driver in the kernel. Although it will likely be straightforward and not very complex.
According to Solomon Systech (and as I can see from the examples I've sent earlier this morning), you send the MIPI commands to the Solomon Systech chip instead of to the display.
So instead of a display driver talking to the display you got a "display" driver talking to the Solomon chip, which then takes care of the rest.
It will need a driver for both the chip and the panel. The panel driver will send commands to the panel, which go through the chip.
There's nothing magical here, the omapdss driver supports these kind of setups (although we don't have such boards at the moment, so bits and pieces could be missing).
If we use the Solomon Systech Chip, we've got the following issues:
Just for completeness how the pipeline would look like with Solomon chip:
- application (e.g. 480x800 screen 50fps) ->
- memory mapped frame buffer ->
- MIPI ->
- Rotator Chip ->
- MIPI ->
- panel (720 x 1280 x 60fps)
Removed the scaler, as the rotator chip can scale and rotate.
The chip doing scaling is actually quite interesting thing. Kind of. With that, we can make omapdrm support modesetting with the chip, so that you can use standard linux software to setup a resolution you want, the same way as on a real monitor, instead of having a single resolution, and requiring programming of the DISPC scaler.
Not that the DISPC scaler is difficult to set up, but it's still something someone needs to do if upscaling is needed.
Tomi
On 02/04/14 16:51, Dr. H. Nikolaus Schaller wrote:
Let's try to describe in words what I assume how it should be:
- application (e.g. wants to display 480x800 screen 50fps) ->
- linear memory mapped frame buffer ->
- highly optimized kernel driver copying (low resolution = less bandwidth!) to ->
- TILER compatible double buffer (if needed) BUT NOT DOING ANY ROTATION ->
- TILER to rotate (ONLY READS from DRAM into DISPC - so no WRITE slowness) ->
No no. There are two separate performance issues here:
1) The "intrinsic" performance hit with TILER: a) as it needs to handle the tiled buffers, and b) extra hit when it needs to access rotated buffer.
2) The performance hit when accessing TILER with CPU.
The 1) case is there always when using TILER 2D buffers (i.e. those that support rotation), no matter if it's SGX, DISPC or CPU that's accessing it.
The performance hit from 1) case should also be quite low.
The main issue we have here is 2). It's there, even if the CPU accesses the 0 degree view.
- DISPC Scaler Unit ->
- DISPC/DSI ->
- MIPI->
- panel (1080 x 1900 x 60fps)
Other applications (that are aware of the TILER layout) can directly write to the stage 4. and even turn off the upscaling to get full-hd.
I don't think that's feasible. I've never heard of such use, even though in theory it sounds possible. I don't think the data format used by TILER is public, and even if it was, I believe doing the conversion with CPU is unoptimal.
Unfortunately I am only able to write this down but not to program a proof of concept to check and compare performance to convince anyone - or find out why it does not work this way.
All this TILER things would have to be set up only once during boot and just the Scaler Unit must be adjusted during run-time to different framebuffer dimensions.
It's not that simple. Well, depends on what kind of SW is used. But, for example, with DRM, you need to allocate the frame buffer using functions that allocate the buffer from TILER, and you need to configure the rotation for that.
And that needs to be done for each overlay. And the display still looks like a portrait display to the software (the one that manages the displays).
That said, if the "display server" (X server, SDL, or such) supports TILER, then I guess the end applications could see the display as landscape. And X + the omap X driver does that at the moment, at least for basic use.
With this I would assume we also have:
POSITIVE: The display appears as landscape display for the Linux OS, no special code needed, works out of the box with all programs.
POSITIVE: The chip does scaling and rotating without tearing and without using additional CPU power (as long as the application can cope with TILER formatted frame buffers which is for X, Qt, Wayland (?) and perhaps SDL)
Only applications that can NOT use the TILER memory layout natively need the CPU supported copy process. I would expect that this is a minority and are appllcations not assuming Full-HD anyways.
As I said above, I would expect to see no applications that support TILER memory layout.
Overlays may add more complexity (but is there an example on the Pandora where overlays are really used?).
You always need an overlay for each display. You can't show anything without an overlay.
Tomi
On Wed, Apr 2, 2014 at 6:03 PM, Tomi Valkeinen tomi.valkeinen@ti.com wrote:
On 02/04/14 16:51, Dr. H. Nikolaus Schaller wrote:
Let's try to describe in words what I assume how it should be:
- application (e.g. wants to display 480x800 screen 50fps) ->
I guess you mean 800x480 at 60fps (i.e. portrait), otherwise we would not need rotation.
- linear memory mapped frame buffer ->
- highly optimized kernel driver copying (low resolution = less bandwidth!) to ->
Still a waste of ~184MB/s..
- TILER compatible double buffer (if needed) BUT NOT DOING ANY ROTATION ->
- TILER to rotate (ONLY READS from DRAM into DISPC - so no WRITE slowness) ->
No no. There are two separate performance issues here:
- The "intrinsic" performance hit with TILER: a) as it needs to handle
the tiled buffers, and b) extra hit when it needs to access rotated buffer.
- The performance hit when accessing TILER with CPU.
The 1) case is there always when using TILER 2D buffers (i.e. those that support rotation), no matter if it's SGX, DISPC or CPU that's accessing it.
The performance hit from 1) case should also be quite low.
The main issue we have here is 2). It's there, even if the CPU accesses the 0 degree view.
- DISPC Scaler Unit ->
- DISPC/DSI ->
- MIPI->
- panel (1080 x 1900 x 60fps)
Other applications (that are aware of the TILER layout) can directly write to the stage 4. and even turn off the upscaling to get full-hd.
I don't think that's feasible. I've never heard of such use, even though in theory it sounds possible. I don't think the data format used by TILER is public, and even if it was, I believe doing the conversion with CPU is unoptimal.
I guess Nikolaus meant using 0 deg. view for the CPU. If it can cope with write-combining, (maybe just rotated view has a problem with write combining?), perhaps this would work. However I think you said that DISPC can't fetch from rotated view due to extra penalty? Or is that just for fullhd?
In either case this can't be tested as we don't have a driver. Writing such driver would be a waste of time if it turns out that DISPC can't fetch from the rotated view anyway.
Unfortunately I am only able to write this down but not to program a proof of concept to check and compare performance to convince anyone - or find out why it does not work this way.
All this TILER things would have to be set up only once during boot and just the Scaler Unit must be adjusted during run-time to different framebuffer dimensions.
It's not that simple. Well, depends on what kind of SW is used. But, for example, with DRM, you need to allocate the frame buffer using functions that allocate the buffer from TILER, and you need to configure the rotation for that.
And that needs to be done for each overlay. And the display still looks like a portrait display to the software (the one that manages the displays).
That said, if the "display server" (X server, SDL, or such) supports TILER, then I guess the end applications could see the display as landscape. And X + the omap X driver does that at the moment, at least for basic use.
With this I would assume we also have:
POSITIVE: The display appears as landscape display for the Linux OS, no special code needed, works out of the box with all programs.
POSITIVE: The chip does scaling and rotating without tearing and without using additional CPU power (as long as the application can cope with TILER formatted frame buffers which is for X, Qt, Wayland (?) and perhaps SDL)
Only applications that can NOT use the TILER memory layout natively need the CPU supported copy process. I would expect that this is a minority and are appllcations not assuming Full-HD anyways.
As I said above, I would expect to see no applications that support TILER memory layout.
Overlays may add more complexity (but is there an example on the Pandora where overlays are really used?).
You always need an overlay for each display. You can't show anything without an overlay.
On pandora, we use one overlay for hardware scaling (used by SDL and multiple emulators) and another for TV-out. The Nintendo DS emulator uses all 3 available layers at once, graphics plane for menu or one of DS screens, one overlay for scaling one of DS screens, and another for sending one of DS's screens to TV. I would like to allow similar uses on the Pyra, and not encumber coders with problems like TILER 2D layouts and such.
Gražvydas
On 02/04/14 18:44, Grazvydas Ignotas wrote:
I guess Nikolaus meant using 0 deg. view for the CPU. If it can cope with write-combining, (maybe just rotated view has a problem with write combining?), perhaps this would work. However I think you said
No, even 0 degree view has the issue.
that DISPC can't fetch from rotated view due to extra penalty? Or is that just for fullhd?
No, I haven't quite said that. Or at least I haven't meant that. Anyone accessing TILER's rotated view will have slight penalty (afaik) compared to the 0 degree view. But that penalty should be small, and especially compared to the CPU perf issue, that penalty just doesn't matter.
Tomi
Am Tue, 1 Apr 2014 15:22:47 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Hi,
If you use chip like that, make sure the rotation actually works. I don't know if they have something magical there, but normally you need a double buffer fb in the chip to support rotation. One buffer is written by the SoC, while the other is sent to the LCD. If there's just a single buffer, you'll get diagonal tearing.
I have looked into the data sheet and it appears that even 720x1280 drives the chip to its limits: • Half-WUXGA (0.5 x 1920 x 1200 pixels) frame buffer • Support rotation up to 1280x800 resolution with SSL Compression • Support rotation up to 960x600 resolution without image compression
So it has only enough on-chip RAM for a single buffer.
Well, according to Solomon Systech, the display got a pin specifically for tear-free rotation when used with an OMAP5.
Surely that pin has some reason... it wouldn't really make sense adding such pin if the OMAP5 can do the same with its own hardware. Something we really need to check.
And SSL-Compression means:
"SSL segment-based (4 pixels per segment) data compression/decompression with zero software overhead"
So it appears that we won't get full resolution 720x1280 rotated with that chip with less problems than with OMAP5 TILER! The problems will just be somewhere else.
Is that, once again, a special framebuffer format or is that something that maybe MIPI supports and it only needs to be switched off?
Am 01.04.2014 um 19:00 schrieb Michael Mrozek:
Am Tue, 1 Apr 2014 15:22:47 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Hi,
If you use chip like that, make sure the rotation actually works. I don't know if they have something magical there, but normally you need a double buffer fb in the chip to support rotation. One buffer is written by the SoC, while the other is sent to the LCD. If there's just a single buffer, you'll get diagonal tearing.
I have looked into the data sheet and it appears that even 720x1280 drives the chip to its limits: • Half-WUXGA (0.5 x 1920 x 1200 pixels) frame buffer • Support rotation up to 1280x800 resolution with SSL Compression • Support rotation up to 960x600 resolution without image compression
So it has only enough on-chip RAM for a single buffer.
Well, according to Solomon Systech, the display got a pin specifically for tear-free rotation when used with an OMAP5.
Surely that pin has some reason... it wouldn't really make sense adding such pin if the OMAP5 can do the same with its own hardware. Something we really need to check.
And SSL-Compression means:
"SSL segment-based (4 pixels per segment) data compression/decompression with zero software overhead"
So it appears that we won't get full resolution 720x1280 rotated with that chip with less problems than with OMAP5 TILER! The problems will just be somewhere else.
Is that, once again, a special framebuffer format or is that something that maybe MIPI supports and it only needs to be switched off?
As far as I understand it is a simple 24bit*lines*pixels bit stream coming in landcape out from the OMAP5 DISPC formatted in 1280 wide MIPI containers. One for each line. So the input is more or less a digitalized TV image.
This is temporarily stored in the frame buffer on the chip.
It is then read out in 720 vertical containers and encoded again as MIPI so that a portrait mode panel starts to scan vertical lines from left to right.
The issue is that the frame buffer in the chip is apparently only ~1 Mpix.
Which is sufficient for a 720x1280 panel.
But it is written on the input side while it is read out. So depending on sync you will have areas with frame x and areas with frame x+1.
This is what Tomi means with the risk of tearing.
Solomon Systech might have added the TE signal - which is more or less a VSYNC impulse. And that should be used by the kernel driver to write in some synced way.
But I have no idea about the details and if using that makes another bottleneck.
-- Mit freundlichen Grüßen,
Michael Mrozek
OpenPandora GmbH Geschäftsführer: Michael Mrozek
Schäffbräustr. 11 85049 Ingolstadt Deutschland Tel.: 0841 / 13 177 33 http://www.openpandora.de/ HRB 4879, Amtsgericht Ingolstadt
ICQ: 15691903 MSN: EvilDragon1717@hotmail.com eMail: info@openpandora.de _______________________________________________ Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
On 01/04/14 20:00, Michael Mrozek wrote:
Am Tue, 1 Apr 2014 15:22:47 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Hi,
If you use chip like that, make sure the rotation actually works. I don't know if they have something magical there, but normally you need a double buffer fb in the chip to support rotation. One buffer is written by the SoC, while the other is sent to the LCD. If there's just a single buffer, you'll get diagonal tearing.
I have looked into the data sheet and it appears that even 720x1280 drives the chip to its limits: • Half-WUXGA (0.5 x 1920 x 1200 pixels) frame buffer • Support rotation up to 1280x800 resolution with SSL Compression • Support rotation up to 960x600 resolution without image compression
So it has only enough on-chip RAM for a single buffer.
Well, according to Solomon Systech, the display got a pin specifically for tear-free rotation when used with an OMAP5.
Surely that pin has some reason... it wouldn't really make sense adding such pin if the OMAP5 can do the same with its own hardware. Something we really need to check.
I haven't really read the spec, but the chip looks like a normal DSI Command mode buffer chip, with some fancier features. The TE (tearing elimination or tearing effect) pin is a normal required pin on DSI Command mode devices, so there's nothing OMAP5 or OMAP specific with that.
With conventional displays the SoC will be driving the panel, i.e. the SoC will send vsync and hsync signals to the panel, and the panel will comply.
With a buffer chip, the chip is driving the panel in the same way as SoC does with conventional panels. What this means is that (obviously) the SoC does _not_ drive the panel, which means that the SoC doesn't know when there vsync happens.
TE is basically a vsync signal (although it can be programmed to give the signal at other points also). When the SoC gets the TE signal, it knows that it can start the frame transfer to the chip. This way tearing can be avoided, but only when rotation is not used.
I also need to point out that at the moment there's no automatic update support for DSI command mode displays in the omapdss driver. Normally the point of using DSI command mode displays is that you do not want to update the display from the SoC all the time. Instead, you only send new frame (or even just part of the frame) when something has changed.
But automatic update is something that can be implemented in the driver.
And SSL-Compression means:
"SSL segment-based (4 pixels per segment) data compression/decompression with zero software overhead"
So it appears that we won't get full resolution 720x1280 rotated with that chip with less problems than with OMAP5 TILER! The problems will just be somewhere else.
Is that, once again, a special framebuffer format or is that something that maybe MIPI supports and it only needs to be switched off?
I have never heard of that, so my guess is that it's nothing MIPI related.
Tomi
Am Wed, 2 Apr 2014 09:06:51 +0300 hat Tomi Valkeinen tomi.valkeinen@ti.com geschrieben:
Hi,
I've received some command examples for the chip and attached them.
Well, according to Solomon Systech, the display got a pin specifically for tear-free rotation when used with an OMAP5. Surely that pin has some reason... it wouldn't really make sense adding such pin if the OMAP5 can do the same with its own hardware. Something we really need to check.
I haven't really read the spec, but the chip looks like a normal DSI Command mode buffer chip, with some fancier features. The TE (tearing elimination or tearing effect) pin is a normal required pin on DSI Command mode devices, so there's nothing OMAP5 or OMAP specific with that.
Thanks for cleating that up.
With conventional displays the SoC will be driving the panel, i.e. the SoC will send vsync and hsync signals to the panel, and the panel will comply. With a buffer chip, the chip is driving the panel in the same way as SoC does with conventional panels. What this means is that (obviously) the SoC does _not_ drive the panel, which means that the SoC doesn't know when there vsync happens. TE is basically a vsync signal (although it can be programmed to give the signal at other points also). When the SoC gets the TE signal, it knows that it can start the frame transfer to the chip. This way tearing can be avoided, but only when rotation is not used.
Okay... so with rotation, we'll still have tearing? Hm. I'll send that question to the manufacturer.
So we have no way avoiding the tearing here with rotation?
I also need to point out that at the moment there's no automatic update support for DSI command mode displays in the omapdss driver. Normally the point of using DSI command mode displays is that you do not want to update the display from the SoC all the time. Instead, you only send new frame (or even just part of the frame) when something has changed. But automatic update is something that can be implemented in the driver.
Okay, if that's not too complex, then it's not a huge worry for me.
On 02/04/14 12:09, Michael Mrozek wrote:
With conventional displays the SoC will be driving the panel, i.e. the SoC will send vsync and hsync signals to the panel, and the panel will comply. With a buffer chip, the chip is driving the panel in the same way as SoC does with conventional panels. What this means is that (obviously) the SoC does _not_ drive the panel, which means that the SoC doesn't know when there vsync happens. TE is basically a vsync signal (although it can be programmed to give the signal at other points also). When the SoC gets the TE signal, it knows that it can start the frame transfer to the chip. This way tearing can be avoided, but only when rotation is not used.
Okay... so with rotation, we'll still have tearing? Hm. I'll send that question to the manufacturer.
So we have no way avoiding the tearing here with rotation?
Well, I didn't study the specs, so perhaps they have some magic functionality there to support rotation. But afaik, the only way to have tear-free rotation with that kind of chip is that the chip has a double framebuffer, so that the other can be written to while the other is read from.
Well, there _is_ a way to avoid the tearing issue in such a setup. Nokia N9 uses it. It's basically about updating the panel's/chip's framebuffer in two parts, so that the reads and writes never collide. The mainline driver doesn't support that, as it's a bit hacky and not very easy to support properly. I would not go that way.
Tomi
Am 02.04.2014 um 11:57 schrieb Tomi Valkeinen:
On 02/04/14 12:09, Michael Mrozek wrote:
With conventional displays the SoC will be driving the panel, i.e. the SoC will send vsync and hsync signals to the panel, and the panel will comply. With a buffer chip, the chip is driving the panel in the same way as SoC does with conventional panels. What this means is that (obviously) the SoC does _not_ drive the panel, which means that the SoC doesn't know when there vsync happens. TE is basically a vsync signal (although it can be programmed to give the signal at other points also). When the SoC gets the TE signal, it knows that it can start the frame transfer to the chip. This way tearing can be avoided, but only when rotation is not used.
Okay... so with rotation, we'll still have tearing? Hm. I'll send that question to the manufacturer.
So we have no way avoiding the tearing here with rotation?
Well, I didn't study the specs, so perhaps they have some magic functionality there to support rotation. But afaik, the only way to have tear-free rotation with that kind of chip is that the chip has a double framebuffer, so that the other can be written to while the other is read from.
From my calculations of the chip it does not have enough memory for a double buffer - even for a 720x1200 panel.
Well, there _is_ a way to avoid the tearing issue in such a setup. Nokia N9 uses it. It's basically about updating the panel's/chip's framebuffer in two parts, so that the reads and writes never collide. The mainline driver doesn't support that, as it's a bit hacky and not very easy to support properly. I would not go that way.
Would such a scheme allow full frame rate for full screen updates? This is the main reason why we are not happy with the Full-HD panel and TILER?
BR, Nikolaus
Am Wed, 2 Apr 2014 13:48:29 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Okay, got some replies from Solomon Systech :)
Well, I didn't study the specs, so perhaps they have some magic functionality there to support rotation. But afaik, the only way to have tear-free rotation with that kind of chip is that the chip has a double framebuffer, so that the other can be written to while the other is read from.
From my calculations of the chip it does not have enough memory for a double buffer - even for a 720x1200 panel
Reply from Solomon:
"Your engineer is correct, even with TE pin, it requires double frame buffer in order to avoid the tearing.
Since SSL compression is a 50% compression, each 720p frame data is 720x1280x24x50% which is roughly 11Mbit and we will do exactly what your engineer mentioned, to separate the Frame X and X+1 into 2 separate locations as a double frame (D2858 has ~27Mbit), and write IN as landscape and read OUT as portrait to perform the rotation."
So it DOES tearing free rotation.
I've just asked him whether SSL compression is lossless, will let you know.
But I know from video editing that a FullHD stream can easily be compressed to half the size without visible artifacts, most of the time lossless (as it's very unlikely that every pixel has a different color).
On 02/04/14 15:05, Michael Mrozek wrote:
Am Wed, 2 Apr 2014 13:48:29 +0200 hat "Dr. H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Okay, got some replies from Solomon Systech :)
Well, I didn't study the specs, so perhaps they have some magic functionality there to support rotation. But afaik, the only way to have tear-free rotation with that kind of chip is that the chip has a double framebuffer, so that the other can be written to while the other is read from.
From my calculations of the chip it does not have enough memory for a double buffer - even for a 720x1200 panel
Reply from Solomon:
"Your engineer is correct, even with TE pin, it requires double frame buffer in order to avoid the tearing.
Since SSL compression is a 50% compression, each 720p frame data is 720x1280x24x50% which is roughly 11Mbit and we will do exactly what your engineer mentioned, to separate the Frame X and X+1 into 2 separate locations as a double frame (D2858 has ~27Mbit), and write IN as landscape and read OUT as portrait to perform the rotation."
So it DOES tearing free rotation.
Ah, that's interesting. Sounds like it could work, then. One thing to confirm is that the SSL compression can be used only inside the chip. I.e. OMAP can send uncompressed data to the chip, and the chip will send uncompressed data to the panel. The diagram on the SSD2858 spec does hint that way, though.
I've just asked him whether SSL compression is lossless, will let you know.
It'd also be nice to get any pointers about the compression. I couldn't find anything with quick googling.
But I know from video editing that a FullHD stream can easily be compressed to half the size without visible artifacts, most of the time lossless (as it's very unlikely that every pixel has a different color).
Unlikely or not, it must be clear that any image, even one with each pixel of different color, will fit into half of the framebuffer.
Btw, did you check the SSD2848 chip? I don't know the diff to SSD2858, but http://www.solomon-systech.com/en/product/mobile-system/mipi-master-bridge-c... says it just has less TX pins, and I guess 4 would be just fine for Pyra.
Tomi
Am Wed, 2 Apr 2014 16:03:46 +0300 hat Tomi Valkeinen tomi.valkeinen@ti.com geschrieben:
Hi,
Since SSL compression is a 50% compression, each 720p frame data is 720x1280x24x50% which is roughly 11Mbit and we will do exactly what your engineer mentioned, to separate the Frame X and X+1 into 2 separate locations as a double frame (D2858 has ~27Mbit), and write IN as landscape and read OUT as portrait to perform the rotation." So it DOES tearing free rotation.
Ah, that's interesting. Sounds like it could work, then. One thing to confirm is that the SSL compression can be used only inside the chip. I.e. OMAP can send uncompressed data to the chip, and the chip will send uncompressed data to the panel. The diagram on the SSD2858 spec does hint that way, though.
Yes, it does do that. It receives the uncompressed stream and then compresses it.
One of the main use cases is to compress a normal stream to SSL to save power.
But I know from video editing that a FullHD stream can easily be compressed to half the size without visible artifacts, most of the time lossless (as it's very unlikely that every pixel has a different color).
Unlikely or not, it must be clear that any image, even one with each pixel of different color, will fit into half of the framebuffer.
Yes, though it might do lossy compression in that case, which would be fine as it would be really really rare.
I think that's the compression:
http://www.google.com/patents/US20100104207?dq=inassignee:solomon+inassignee...
Btw, did you check the SSD2848 chip? I don't know the diff to SSD2858, but http://www.solomon-systech.com/en/product/mobile-system/mipi-master-bridge-c... says it just has less TX pins, and I guess 4 would be just fine for Pyra.
The SSD2858 supports rotation, the SSD2848 not ;)
On 02/04/14 14:48, Dr. H. Nikolaus Schaller wrote:
Well, there _is_ a way to avoid the tearing issue in such a setup. Nokia N9 uses it. It's basically about updating the panel's/chip's framebuffer in two parts, so that the reads and writes never collide. The mainline driver doesn't support that, as it's a bit hacky and not very easy to support properly. I would not go that way.
Would such a scheme allow full frame rate for full screen updates? This is the main reason why we are not happy with the Full-HD panel and TILER?
That scheme requires a DSI command mode panel or chip. So the current full-HD panel would not be usable with that scheme anyway. TILER is not used with that scheme at all.
But yes, it gives full frame rate with rotation. It's complex on the SW side, and, with most DSI command mode peripherals, needs quite precise timer interrupt which is used to get an irq at the middle of the frame.
Tomi