Not everything is as fast as planned

[ashcloud]

Just an idea: The SOC also has two ARM Cortex-M4 cores. Maybe those could be used to rotate the framebuffer. So, are they powerful enough to do so? At least they have some kind of SIMD instructions that may be useful for that. Maybe it would be more energy efficient than using the GPU for that. Maybe it wouldn't, then it's just a stupid idea.

 

[slaeshjag]

There's a freaking 2D accelerator in the thing and you suggest doing it in software on what are basically microcontrollers?

 

[Dead1nside]

I think Notaz's and Exophase's comments, and other senior figures in the community, are as always very valuable and important for this.


It's important that we don't kid ourselves in the amount of effort and motivation required to finish any of these software projects, and low-level integration work.


But I don't think anyone has suggested that this is a no-go for the buying decision for the Pyra: we clearly all want the Pyra to happen, and it's being carried out with a lot more prudence than Craig's ushering of the Pandora.


It's annoying that the rotator chip has to be used at all, but it's a necessary compromise. I'm more than happy to purchase in full my Pyra (after pre-preordering) and if a landscape LCD does become available later then I'm confident that EvilDragon as he has done in the past will help anyone who wants to, to transition to that display.


As a community we can have educated critique without fear of jeopardising sales. These questions actually help clarify the real situation, by bringing more experienced minds to bare on the problem.


And as someone already said, it's better that these potential issues are out in the open and are being discussed. We just don't want that to erode confidence and lead people to hold-off on their purchases, which I think EvilDragon is quite rightly concerned about.
 

 

[Not_a_Nick]

As a community we can have educated critique without fear of jeopardising sales. These questions actually help clarify the real situation, by bringing more experienced minds to bare on the problem.

Thank you for summing it up, that's basically it.

 

[ashcloud]

There's a freaking 2D accelerator in the thing and you suggest doing it in software on what are basically microcontrollers?

Well, it's a microcontroller clocked at a freaking 1.7 Ghz (correct me if this is wrong) and it's basically just moving data around. That's what microcontrollers are good for

 

[slaeshjag]

No. The M4's are afaik 200 MHz without any SIMD. They aren't useful for much of anything that doesn't require realtime response, as said a billion times already on these boards, they're not high performers, be it computation or just raw bandwidth.

 

[hns]

There is also something to consider: memory bandwidth and CPU background activity.


The panel has 60fps 720x1280 RGB888 which means approx. 55 Mpx/s with 24 bit each. This is eqivalent to 1,3 Gbit/s. or 165 MByte/s. This must go from DRAM (framebuffer) to the OMAP to the MIPI interface. And HDMI has at least the same memory bandwidth demand.


If there is software in between to rotate it needs a temporary memory. This memory is not on the OMAP SoC but also in DRAM. So it has to read the non-rotated frame buffer, store after some address calculation and perhaps even some bit shifts, and then, the MIPI driver has to read it again. And since the framebuffer is quite big (2,7 MByte) it won't fit in the data cache on the SoC.


So this would mean 3 times the bandwidth. I.e. 495 MByte/s shuffled around in the background. For displaying a perhaps static wallpaper...


A rough estimate is that this will demand quite some processing power, reducing processing power available for drawing the graphics.


The external rotator chip does the same, but with its built-in framebuffer RAM. So there is no speed penalty for the OMAP and no extra memory bandwidth demand for the OMAP display subsystem.

 

[Exophase]

A rough estimate is that this will demand quite some processing power, reducing processing power available for drawing the graphics.

If the 2D accelerator does this, and nothing but this, it won't take up pure CPU time. It will stall the CPU while it's waiting on memory accesses that queued accesses from 2D accelerator are blocking. I don't know exactly how the arbitration works, only that the OMAP5 DMM (dynamic memory manager) has prioritization options from different sources. The rotation wouldn't need to be very high priority. It will still have a hit in bandwidth heavy applications, but most CPU tasks won't be for much of their runtime. So I wouldn't expect the typical performance degradation to be more than a few percent at worst, at least for a high clocked application.


But this is all moot if the 2D accelerator doesn't get used. For the 3D accelerator the situation could be sort of similar but in practice uploading textures tends to suck up a bunch of extra CPU time for various reasons. GPU is a different story, it's more of a concern that the GPU will become bandwidth limited in heavy applications.


The other pertinent question is power consumption. Going to DDR3L is going to use a lot more power than going to SRAM, using the 2D accelerator will use some power. But going through the rotator chip means that you're trading off a read and write to external DRAM for a read and write to an external serial display interface. And you're trading the 2D accelerator's operations for the logic operations on the rotation chip which includes both rotation and compression/decompression. So who knows what the real power consumption story is like.

 

[mclien]

I confess, I have no technical idea to solve this problem (and not enough knowledge anyway), BUT I'm going to see bunnie from the novena project at 32c3, who has very much knowledge and expierence in makeing own open source hardware.


So If somebody can do a writeup/summary what the exact problems are, I can think of two approaches:


- bunie/xobs have some software knowledge to contribute


- have contacts and expierence in Shenzen directly and might find the desired display (xobs once stated in a talk that there is a store where litteraly any kind of LCD display can be found, but you have to be on spot for that, what they are frequently)


And I would be happy to make that contact


EDIT:


meanwhile: Go on as planned! And thanks for the amazing work! As long as I get several x-terms , I'm fine ;-)

 

[Grench]

Just to clarify...


The SoC can output MIPI and/or HDMI.


MIPI 5" screens are only available in portrait.  Portrait screen in landscape use requires rotator.


HDMI output to external does not need rotation as those are generally already landscape.


Does anyone make a 5" HDMI pannel?


If that means that what is on the pannel would only ever be able to be the same as the HDMI port output, it might not be so bad.


Switching from MIPI to HDMI on the lid pannel may not even be possible or wise.  Just a crazy thought.  Would require lots of changes.


Would this do it?


http://m.alibaba.com/product/60366796503/5-lcd-hdmi-720p-display-5.html?spm=a2706.7843299.1998817009.5.f7jniP


How to tell if the MIPI screens are landscape? 


http://m.alibaba.com/product/60075736421/RoHS-ISO-5-inch-lcd-monitor.html?spm=a2706.7843299.1998817009.5.34o3K0

 

[Grench]

So - ED's last tweet from yesterday - the first prototype is apparently booting.


So - video coming?

 

[levi]

Hopefully.  I was hoping his tweet might have a photo attached, but I guess HNS isn't as quick with a camera as ED is

 

[Yoyobuae]

What I'm asking myself though is... there seem to be others who have used command mode panels with Linux.


Here is an example panel driver (command mode) in use with the OMAP:


git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/drivers/video/omap2/displays/panel-taal.c?id=c16fa4f2ad19908a47c63d8fa436a1178438c7e7


How did they solve the issue with the non-updating signal? I can't imagine they recompiled all of the Linux apps to work with command mode.


What's the trick here?

Sorry if I'm insisting too much, but did you see my post here:


https://boards.openpandora.org/topic/5600351-not-everything-is-as-fast-as-planned/?page=2#comment-410207


Maybe that's not relevant for our case, but from the little I read it seems like it could work. Nikolaus mentioned that the SSD chip can generate the required TE signal. The PDF I linked in above post talks about using the TE signal generated from the LCD panel (SSD chip in our case) in command mode to sync the sending of data to Vsync. And it mentions how this feature is not part of the MIPI DSI standard, but available on OMAP.

 

[hns]

Sorry if I'm insisting too much, but did you see my post here:


https://boards.openpandora.org/topic/5600351-not-everything-is-as-fast-as-planned/?page=2#comment-410207


Maybe that's not relevant for our case, but from the little I read it seems like it could work. Nikolaus mentioned that the SSD chip can generate the required TE signal. The PDF I linked in above post talks about using the TE signal generated from the LCD panel (SSD chip in our case) in command mode to sync the sending of data to Vsync. And it mentions how this feature is not part of the MIPI DSI standard, but available on OMAP.

Yes, command mode could use the TE signal easily. But command mode needs some user space graphic library to tell the display to update (synchronized to TE) through some ioctl(). This is something we can not expect.


Therefore we want to run in video mode. But the OMAP ignores TE in video mode. So we need to develop a trick to combine.


Technical details how I think it can be made working are here:


http://projects.goldelico.com/p/gta04-kernel/issues/703/


The result could be just some 10-20 lines of code but the complexity is finding the correct hooks into the Linux source codes and test the code and fill the gaps the idea leaves.

 

[Yoyobuae]

Yes, command mode could use the TE signal easily. But command mode needs some user space graphic library to tell the display to update (synchronized to TE) through some ioctl(). This is something we can not expect.


Therefore we want to run in video mode. But the OMAP ignores TE in video mode. So we need to develop a trick to combine.


Technical details how I think it can be made working are here:


http://projects.goldelico.com/p/gta04-kernel/issues/703/


The result could be just some 10-20 lines of code but the complexity is finding the correct hooks into the Linux source codes and test the code and fill the gaps the idea leaves.

Hmm, I see. Couldn't the same idea be applied in command mode driver? The driver ED linked  seems to do just that:


r = request_irq(gpio_to_irq(gpio), taal_te_isr,
IRQF_TRIGGER_RISING,
"taal vsync", dssdev);


Then later in taal_te_isr function they do the update. Not sure but it seems like it should be possible have command mode driver update continually on it's own without needing to be triggered from userspace (maybe after a bit of driver hacking).

 

[EvilDragon]

So - ED's last tweet from yesterday - the first prototype is apparently booting.


So - video coming?

Not yet  Wouldn't be very spectacular yet - it would boot into the command line and that's it.


Once I get my own prototype unit and aTc gets the OS working on it, I'll make sure to make more videos

 

[rygD]

Not yet  Wouldn't be very spectacular yet - it would boot into the command line and that's it.

Sounds like exactly the kind of thing some of us enjoy seeing.


Looking forward to whatever you share.

 

[WizardStan]

Wouldn't be very spectacular yet - it would boot into the command line and that's it

I think you are underestimating what many of us would find spectacular

 

[Da Dude]

Yep, We need those things as well. 

 

[klapse]

Let ED decide what to make videos of.