Use of the (Co-)Processors

[lukey]

While reading the Datasheet to the OMAP5432 i found, that there are lots of (Co-)Processors:


1x DSP;


2x Cortex-M4 "image processing unit (IPU)";


1x "image signal processor" + "still image coprocessor(SIMCOP)";


(And of course the Powervr SGX544 and the two Cortex-A15s)


How difficult is it to make use of them? And how much Performance is in them?

 

[levi]

There's quite a lot of information on these boards about using the DSP on the Pandora; I suspect how you use it is very similar on the OMAP5, although AIUI they've removed a few opcodes so you can't port software directly from one to the other - but the mechanics of posting data to it and setting it going is likely to be the same IMO.

As for the M4 chips, I don't know.  I suspect they're running all the time in some null loop, and there must be some way of making them branch to some useful instructions from the main CPU.  And what memory they have access to is also something I don't know - presumably they must have some window onto either dedicated on-chip RAM or to external RAM via some sort of read scheduler/multiplexer.

 

[Exophase]

The DSP in OMAP5 is weaker than the one in OMAP3430/DM3730, at least at the same clock speed. Don't know what the clock is in OMAP5 but I doubt it's very high. Combine that with the fact that OMAP5 also has a lot more CPU power and I think the DSP will be much less useful than it is on Pandora, where it has only been pretty mildly beneficial for a small number of programs.

The M4s are probably even lower powered. I haven't figured out what the clock speeds are supposed to be there either, but I doubt it's more than a few hundred MHz at most.

I think the M4s and DSP have access to the whole system address space. So if you want to do something that benefits from low latency direct access to hardware without the OS getting in the way they could be useful. Like if a touch screen input filter is desirable it could be better off there instead of in the kernel or even worse in user space like tslib.

 

[Grench]

I doubt the M4s would be present in the silicone without some imagined use for them.  What did TI have in mind for them to do?

 

[Exophase]

I doubt the M4s would be present in the silicone without some imagined use for them.  What did TI have in mind for them to do?

I dunno, phone stuff? Since the OMAP SoCs were made for phones, hence why TI stopped making them. Managing media (especially audio) and calls without having the A15s powered. I remember with OMAP3 TI talked about how much you could stretch battery life playing mp3s on the DSP with the CPU unpowered. But some programming tasks, even low powered ones, are a poor fit for the DSP and make more sense on the M4s.

OMAP3 actually does have a similar embedded low power core, an ARM9, it's just not really documented. It's used for managing the multimedia blocks.

 

[Linux-SWAT]

GCC "recently" added some kind of mixed CPU optimizations, e.g. -mcpu=cortex-a15.cortex-a7 .

Dunno how things work with this.

 

[Exophase]

GCC "recently" added some kind of mixed CPU optimizations, e.g. -mcpu=cortex-a15.cortex-a7 .

Dunno how things work with this.

That just means code is tuned for both Cortex-A15 and Cortex-A7 simultaneously. Presumably ensuring that no optimizations are done for one that are particularly hazardous for the other. This is for big.LITTLE systems.

 

[SNESFAN]

forgive my ignorance in this, but I've heard of the M4 as being potentially unused feature because of it being a different architecture than ARMv7 of the cortex a9 and a15 etc.

My question is, couldn't you simply compile any generic binary as M4 as the target? using something like "-mcpu=cortex-m4" for your library or binary and simply make calls to it to assist in processing on execution of your application?

I understand things like emulators where proper timing is crucial, or things that has high levels of ASM, but for low intensive stuff, and generic applications, or to offload background processes, it seems like it would be a fairly simple optimization. Not the nightmare I've heard it described as. Is this not the case?

Agian, if this is a silly question, please forgive me. But I've used similar flags in compiling and things perform noticeably better with those flags than without.

 

[slaeshjag]

The M4's kinda run their own race, like the DSP. Code you wish to run on an M4 runs like a separate process on a different system that have access to the same memory as you. Communication between them would also be much like it is to talk to the Pandora's DSP. They are quite weak performance-wise, although relatively uninterrupted. Real-time tasks that don't require a lot of computational performance would probably run well on them. Assistive processing on demand from applications is not a good use of them, as it'd probably be faster for the CPU to just do it by itself than to prepare the data, send it over and keep an eye out for the completion message.

I could imagine isolated processors that sees little communication could run on them (for example, the sound system in SNES.) If it's actually worth pushing anything like that onto it, no idea? Unlikely.

 

[sebt3]

I could imagine isolated processors that sees little communication could run on them (for example, the sound system in SNES.)

I realy doubt the sound emulation from snes9x 1.53 could fit an M4 performance wise (and no i'm not refering what kind of pain it would be to even try )

 

[slaeshjag]

It was more on example of independence than performance, I guess

 

[_wb_]

Perhaps we could use an M4 to manage the rgb leds? E.g. to cycle the logo leds through a permanent "flickering flame" loop without keeping the main cpu awake.

 

[Exophase]

forgive my ignorance in this, but I've heard of the M4 as being potentially unused feature because of it being a different architecture than ARMv7 of the cortex a9 and a15 etc.

My question is, couldn't you simply compile any generic binary as M4 as the target? using something like "-mcpu=cortex-m4" for your library or binary and simply make calls to it to assist in processing on execution of your application?

I understand things like emulators where proper timing is crucial, or things that has high levels of ASM, but for low intensive stuff, and generic applications, or to offload background processes, it seems like it would be a fairly simple optimization. Not the nightmare I've heard it described as. Is this not the case?

Agian, if this is a silly question, please forgive me. But I've used similar flags in compiling and things perform noticeably better with those flags than without.

The OS can't just run any old thread on the Cortex-M4 even if you did use a subset of the instruction sets that was compatible with both. They're not cache coherent with each other (as far as I know) which is something software expects for inter-thread communication. Even if it could, this would be really tricky for an OS to handle because there are other architectural differences that go beyond instruction set. I don't know if the M4s have a full featured MMU, but I do know they handle interrupts differently so you'd need at least different interrupt handlers running on the M4s, and probably other parts of the OS would have to be replicated/specialized.

It'd be really hard to get a performance boost using the M4s unless you're doing something that would have a high OS overhead cost on the A15s. The A15s are going to be something like 20-30x the performance of the M4s (worse if the applications are floating point or SIMD heavy) . Factor in communication and threading overhead (cache flushes, communication interrupts, increased memory pressure, spinlocks, inherent computational redundancy in algorithm, massive asymmetry of computation) and there's a good chance you'll make performance worse instead of better. But if you did use it you'd definitely want to target the M4s explicitly over the OS trying to schedule stuff there. Trying to force a common instruction set would mean you get no FP or SIMD instructions on the A15 which will hurt performance for a lot of things, far more than the M4s could help performance.

Perhaps we could use an M4 to manage the rgb leds? E.g. to cycle the logo leds through a permanent "flickering flame" loop without keeping the main cpu awake.

I'm sure you could.

Any one have some guesstimates on what the power consumption of the LEDs will be like? I doubt you'd be able to run this for more than a few days at most.

 

[Grench]

OK, combining this with what is being discussed on the Android thread...

Could an Android VM be compiled for and run continuously in the background on the M4 with minimal or no impact on main system processes on the A15?

My thoughts are a stripped down Android VM (2D?) with a corporate email application running on the M4 and minimal RAM talking to a wifi bridge on the Debian host running on the A15.  I.e. it can sit there, in low power mode, on a low power CPU, passing messages every 5-15 minutes if/when it can make a connection over the bridge.

Is that possible?

 

[TrashyMG]

ould an Android VM be compiled for and run continuously in the background on the M4 with minimal or no impact on main system processes on the A15?

I don't think it's powerful enough for that type of work, this may give you some context of what these cores are intended for:

The ARM Cortex-M4 processor is a low-power processor that features low gate count, low interrupt latency, and low-cost debug. The Cortex-M4F is a processor with the same capability as the Cortex-M4 processor, and includes floating point arithmetic functionality. These processors are intended for applications requiring digital signal processing functionality. The ARM Cortex-M4 processor implements the ARMv7-M architecture profile.

 

[SNESFAN]

OK, combining this with what is being discussed on the Android thread...

Could an Android VM be compiled for and run continuously in the background on the M4 with minimal or no impact on main system processes on the A15?

My thoughts are a stripped down Android VM (2D?) with a corporate email application running on the M4 and minimal RAM talking to a wifi bridge on the Debian host running on the A15.  I.e. it can sit there, in low power mode, on a low power CPU, passing messages every 5-15 minutes if/when it can make a connection over the bridge.

Is that possible?

 

from my understanding of what the wise and powerful exo just said, it's an all or nothing situation, a binary that could potentially be able to swap back and forth like a big.LITTLE would need to have a more limited common set of instructions if it were even to work at all. So you wouldn't potentially *WANT* a version of qemu or android emulator that was compatible with m4 and a15, and that the m4 by it's self is not powerful enough to get satisfactory performance.

So you would essentially want to make it armv7 entirely as to not gimp your application.

The m4 would be a good specific target for entire applications that don't require much processing. I think the part none of us are clear on, is what would be a maximum "acceptable" use. I think more testing will come once these get into people's hands.

EDIT:

My thoughts on a good use might be a "listener" service for communication applications, ones that reach out to the net on a schduel and do SMS, messaging, email, etc and give notifications via led, vibration, or sound

EDIT2:

I don't think it's powerful enough for that type of work, this may give you some context of what these cores are intended for:

that link inside your quote gives me a better idea as to the intended uses

.. targeting the motor control, automotive, power management, embedded audio and industrial automation markets

it does sound as if it might go unused unless someone uses the pyra for a DAC or something

 

[Grench]

I think the part none of us are clear on, is what would be a maximum "acceptable" use. I think more testing will come once these get into people's hands.

Henceforth all activity using a Pyra will be referenced as 'testing'...

Pyra:  "Whiz bang whir zap bang wahwah."

Wife:  "What are you doing?"

Me:  "Testing.  Just testing."

Looking up the M4, it's main ability seems to revolve around it's capacity to collect data and act as a low energy use system controller.  Basically, read the rain gauge and turn on the sprinklers.  Rudimentary, "If then else do," program type stuff.  What to use that for?

How about a process to monitor the gyroscope that if it senses the unit falling more than 20cm, have the speaker volume over-ridden and a blood curdling scream issue from the speakers followed by a lament of anguish and misery the moment it stops moving?

 

[Hồng Thất Công]

... a blood curdling scream issue from the speakers followed by a lament of anguish and misery the moment it stops moving?

You can do this yourself.  No need for the M4 cores to do this. Hence save more battery

 

[WizardStan]

Could an Android VM be compiled for and run continuously in the background on the M4 with minimal or no impact on main system processes on the A15?

No. Well maybe. The M4 cores do not have floating point units, they are integer only as far as I'm aware. Android can be compiled soft float (it's libraries are softfp for exactly this reason) but on a core with only a few hundre Mhz at most that'll cost you a lot.
My thoughts for the M4s are to run something simple that can benefit from always being up without worrying about whether the main CPU gets bogged down, something like a system control overlay: even if there's fork bomb you can still hit some hardware interrupt to get the M4 to spring to life and give you some kind of access to recover. A SysRq, if you will.

I wanted to port my companies voice control software to it but due to situations beyond my control that's looking highly unlikely now.

 

[Exophase]

I said before in another thread about this (I think Grench was asking then), but running two OSes at the same time on different cores is highly problematic. They can't both have exclusive access to the underlying hardware. One would have to abstract hardware access for the other which defeats being able to use the M4s with the A15s off for long durations of time.

And you can't actually run a VM on the M4s, they don't have the architectural support for it. So it'd have to be more like paravirtualization. Face it, even if it could work no one would put in the massive amount of work needed.