Hi,

Am 17.07.2016 um 17:58 schrieb Michael Mrozek EvilDragon@openpandora.org:

Am Fri, 15 Jul 2016 03:41:44 +0200 hat Matthijs van Duin matthijsvanduin@gmail.com geschrieben:

Actually, that's pretty interesting as we have issues running some of the CPU-Boards in 1.5GHz under full load for an extended period of time.

Maybe they don't like this overclocking.

I know the EVMs don't have a problem with that, but we probably have a different revision of OMAP5 on our PCBs... so who knows.

Nikolaus, would it be possible to do a test on one of the CPU-Boards where you KNOW it fails with 1.5GHz?

Well, currently I don't know which boards fail at 1.5GHz with all latest patches. In fact all boards did fail before March 2016. And currently 3 work with 1 GHz without problems.

Re-enable 1.5GHz in your board file, boot it up and set the divider to 8 using: omapconf write 0x4A004320 0x06000001

Re-run your stress test for a while. If it still crashes, it's something else. If it works fine, then we know our hardware is fine :)

Will make such tests asap.

...

Section 4.3.2 of the omap5432 datasheet warns:

The programmable divider for the asynchronous bridge to Audio Back-end (ABE) must be set to:

• MPU_GCLK / 16 when MPU_GCLK clock is running at

OPP_HIGH_MPU.

• MPU_GCLK / 8 when MPU_GCLK clock is running at OPP_NOM_MPU.

The programmable divider for the asynchronous bridge to L3 must be set to:

• MPU_GCLK / 8 when MPU_GCLK clock is running at OPP_HIGH_MPU.
• MPU_GCLK / 4 when MPU_GCLK clock is running at OPP_NOM_MPU.

This however does not seem to be happening (tested on 4.7-rc7-letux):

root@omap5:~# cpufreq-set -r -g performance root@omap5:~# omapconf show prcm mpu | grep Rat | Source Clock Rate | 1500.000MHz | | MPU-ABE SS Bridge Ratio | 8 | | MPU-L3MAIN Interc. Bridge Ratio | 4 |

(These settings are in CM_MPU_MPU_CLKCTRL.)

Matthijs _______________________________________________ Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel

-- Mit freundlichen Grüßen,

Michael Mrozek

---

OpenPandora GmbH Geschäftsführer: Michael Mrozek

Schäffbräustr. 11 85049 Ingolstadt Deutschland Tel.: 0841 / 990 5548 http://www.openpandora.de/ HRB 4879, Amtsgericht Ingolstadt

---

eMail: mrozek@openpandora.org

On 08/15/2016 11:44 PM, Matthijs van Duin wrote:

It is worth mentioning that based on tests I've done (on an omap5-uevm), the clock speed of the async L3 bridge appears to be the bottleneck for traffic between the Cortex-A15 and the L3 interconnect. As a result, if the bridge clock divider is adjusted as mandated by the datasheet, L3-heavy workloads will actually perform noticably worse at OPP_HIGH (bridge @ 1500/8 = 187.5 MHz) than at OPP_NOM (bridge @ 1000/4 = 250 MHz).

This is an awkward situation.

It would be nice to know what the actual maximum clock speed permitted for the async bridge is, to have an alternative OPP_HIGH mode where the bridge divider is left at /4 and the cpu speed only raised as much as the async bridge will tolerate. I have however not found this specified in the datasheet.

Also, what are the risks involved in overclocking the async bridge as is done currently? Will it adversely affect device lifetime? Result in silent data corruption? Cause protocol errors resulting in deadlock?

That would be an invalid Soc Operation condition -> Looks like a missed configuration path.

It seems curious to me that this problem presumably affects every omap5 and dra75x/am572x (and possibly dra72x/am571x?) currently out there, yet no problems have been encountered (or at least been attributed to it), even though leaving the divider at /4 would seem to severely overclock it (at 375 MHz). I have myself not observed any problems while attempting to flood the interface (from/to TILER).

A rare deadlock may actually be an acceptable performance tradeoff for some applications, especially since the cortex-A15 itself already includes rare deadlocks, some with performance-affecting workaround (801819, https://patchwork.kernel.org/patch/6960921/), some with no workaround (799271).

On the other hand, random silent corruption or a serious reduction in device POH would be less likely to be considered acceptable by anyone.

Some clarification from TI would be very welcome here.

Hmm.. This does seem on the first look to be a miss in our configuration - i recollect in older evil-vendor-production kernel (circa 3.0/3.4 kernels) we had handled this, but much has changed since then.

Tero is on vacation atm, will have to see how he'd like to handle this.

On 18/10/16 16:32, Matthijs van Duin wrote:

*bump*

Sorry, I think this was buried somewhere in my mailbox.

On 16 August 2016 at 15:44, Nishanth Menon nm@ti.com wrote:

...

On 08/15/2016 11:44 PM, Matthijs van Duin wrote:

...

It is worth mentioning that based on tests I've done (on an omap5-uevm), the clock speed of the async L3 bridge appears to be the bottleneck for traffic between the Cortex-A15 and the L3 interconnect. As a result, if the bridge clock divider is adjusted as mandated by the datasheet, L3-heavy workloads will actually perform noticably worse at OPP_HIGH (bridge @ 1500/8 = 187.5 MHz) than at OPP_NOM (bridge @ 1000/4 = 250 MHz).

This is an awkward situation.

It would be nice to know what the actual maximum clock speed permitted for the async bridge is, to have an alternative OPP_HIGH mode where the bridge divider is left at /4 and the cpu speed only raised as much as the async bridge will tolerate. I have however not found this specified in the datasheet.

Also, what are the risks involved in overclocking the async bridge as is done currently? Will it adversely affect device lifetime? Result in silent data corruption? Cause protocol errors resulting in deadlock?

That would be an invalid Soc Operation condition

That doesn't actually answer though

...

...

It seems curious to me that this problem presumably affects every omap5 and dra75x/am572x (and possibly dra72x/am571x?) currently out there, yet no problems have been encountered (or at least been attributed to it), even though leaving the divider at /4 would seem to severely overclock it (at 375 MHz). I have myself not observed any problems while attempting to flood the interface (from/to TILER).

A rare deadlock may actually be an acceptable performance tradeoff for some applications, especially since the cortex-A15 itself already includes rare deadlocks, some with performance-affecting workaround (801819, https://patchwork.kernel.org/patch/6960921/), some with no workaround (799271).

On the other hand, random silent corruption or a serious reduction in device POH would be less likely to be considered acceptable by anyone.

Some clarification from TI would be very welcome here.

Hmm.. This does seem on the first look to be a miss in our configuration - i recollect in older evil-vendor-production kernel (circa 3.0/3.4 kernels) we had handled this, but much has changed since then.

Tero is on vacation atm, will have to see how he'd like to handle this.

Any news?

I guess there are at least two ways to fix this:

1) Setup the asynch bridge dividers within the DPLL code for the MPU DPLL. You probably need a new ti_clk_features flag that gets setup in SoC setup for OMAP5 and gets checked in the DPLL code, and apply the bridge dividers as needed. Slightly hackish yes.

2) Or, you could add completely new clock nodes for the bridges which are children of MPU DPLL. These would automatically adjust their internal divider based on the parent clock rate changes. This would have the added benefit of the bridge clock rates being visible as separate clock entries for debugging purposes. (Assuming dynamic clock rate changes like this work properly within CCF, this would need some testing.)

Based on some internal discussions, and the fact that the data manual for DRA7 doesn't mention this feature (even though it is listed in TRM) my current assumption is that it is not needed on that SoC.

-Tero

On 18/10/16 18:56, Matthijs van Duin wrote:

On 18 October 2016 at 17:10, Tero Kristo t-kristo@ti.com wrote:

...

Based on some internal discussions, and the fact that the data manual for DRA7 doesn't mention this feature (even though it is listed in TRM) my current assumption is that it is not needed on that SoC.

Do you have any insights w.r.t. the first part of my email? Is there any hope of finding out what the actual max speed is of the async bridge, and what the consequences might be of exceeding it?

Personally I don't have any idea. It may impact the lifetime of the SoC, thats my best guess. I think Nishanth tried to ask around a bit about this, but if that didn't result into any reasonable answers, the best approach would be to raise a question in the TI support forums.

Nishanth, any thoughts?

-Tero

As I said, currently that bridge is being severely overclocked on any omap5 running a mainline kernel at OPP_HIGH, yet no problems are apparent even if I flood the L3 bridge with traffic. Setting the divider to /8 on the other hand hurts MPU-L3 performance badly and L3-heavy use cases may want to avoid it, leaving the choice of either sacrificing cpu speed or just keeping things as it is now and hope for the best.

Matthijs

On 18 October 2016 at 23:20, Nishanth Menon nm@ti.com wrote:

I have'nt had luck via the internal channels I know of - yes, I do suggest e2e.ti.com as the best option. Though [1] seems to have gone read-only ; however, reading [2], best I can find is [3].

[1] http://e2e.ti.com/support/omap/f/885 [2] http://www.ti.com/product/OMAP5432/support#community [3] http://e2e.ti.com/support/dsp/omap_applications_processors

[3] is just a forum category page, you have to pick a forum to actually post anything, and the only subforum left where you can do so is the one for omap-L1xx which would obviously be completely inappropriate.

The closest family member of the omap5 that is actively supported is the dra7/am572x, and in fact the parts that are relevant here (MPU subsystem and clock speed, L3 clock speed) seem to be identical based on available info.

Matthijs