Hi all, here is a very surprising result which needs independent confirmation.
root@letux:~# cpufreq-info cpufrequtils 008: cpufreq-info (C) Dominik Brodowski 2004-2009 Report errors and bugs to cpufreq@vger.kernel.org, please. analyzing CPU 0: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 250 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:10.27%, 500 MHz:37.39%, 750 MHz:9.76%, 1000 MHz:24.02%, 1.25 GHz:6.92%, 1.50 GHz:11.64% (60) analyzing CPU 1: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 250 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:10.29%, 500 MHz:37.38%, 750 MHz:9.76%, 1000 MHz:24.01%, 1.25 GHz:6.92%, 1.50 GHz:11.63% (60) root@letux:~# ./high-load -n 100% load stress test for 2 cores Sat Jan 1 04:52:17 UTC 2000 51° 52° 49° 36° 4002mV 1000MHz Sat Jan 1 04:52:17 UTC 2000 51° 52° 49° 36° 3991mV 1500MHz Sat Jan 1 04:52:18 UTC 2000 52° 53° 51° 36° 3854mV 1500MHz Sat Jan 1 04:52:20 UTC 2000 64° 54° 53° 36° 3876mV 1500MHz Sat Jan 1 04:52:21 UTC 2000 76° 62° 61° 36° 3855mV 1500MHz Sat Jan 1 04:52:23 UTC 2000 81° 69° 66° 37° 3852mV 1500MHz Sat Jan 1 04:52:24 UTC 2000 85° 71° 71° 37° 3832mV 1500MHz Sat Jan 1 04:52:26 UTC 2000 90° 75° 74° 37° 3829mV 1500MHz Sat Jan 1 04:52:27 UTC 2000 90° 75° 77° 37° 3824mV 1500MHz Sat Jan 1 04:52:29 UTC 2000 92° 78° 77° 38° 3843mV 1500MHz Sat Jan 1 04:52:30 UTC 2000 95° 81° 79° 38° 3813mV 1500MHz Sat Jan 1 04:52:31 UTC 2000 97° 84° 82° 38° 3825mV 1500MHz Sat Jan 1 04:52:33 UTC 2000 101° 86° 83° 38° 4021mV 500MHz Sat Jan 1 04:52:35 UTC 2000 102° 87° 84° 37° 4023mV 500MHz Sat Jan 1 04:52:37 UTC 2000 82° 84° 79° 37° 3848mV 1500MHz Sat Jan 1 04:52:38 UTC 2000 77° 79° 73° 37° 3813mV 1500MHz Sat Jan 1 04:52:40 UTC 2000 90° 78° 75° 37° 3836mV 1500MHz Sat Jan 1 04:52:41 UTC 2000 97° 84° 80° 38° 3822mV 1500MHz Sat Jan 1 04:52:43 UTC 2000 101° 86° 84° 38° 3964mV 750MHz Sat Jan 1 04:52:44 UTC 2000 104° 89° 86° 38° 4019mV 250MHz Sat Jan 1 04:52:47 UTC 2000 85° 86° 81° 37° 3820mV 1500MHz Sat Jan 1 04:52:48 UTC 2000 78° 81° 75° 37° 3832mV 1500MHz Sat Jan 1 04:52:50 UTC 2000 93° 81° 78° 37° 3797mV 1500MHz Sat Jan 1 04:52:51 UTC 2000 100° 86° 83° 37° 4029mV 250MHz Sat Jan 1 04:52:53 UTC 2000 104° 89° 86° 37° 3986mV 750MHz Sat Jan 1 04:52:55 UTC 2000 83° 84° 79° 37° 3815mV 1500MHz Sat Jan 1 04:52:57 UTC 2000 78° 80° 75° 37° 3824mV 1500MHz Sat Jan 1 04:52:58 UTC 2000 94° 82° 78° 37° 3799mV 1500MHz Sat Jan 1 04:53:00 UTC 2000 100° 87° 84° 39° 4056mV 500MHz Sat Jan 1 04:53:01 UTC 2000 105° 90° 87° 39° 4004mV 500MHz Sat Jan 1 04:53:04 UTC 2000 87° 88° 83° 39° 3816mV 1500MHz Sat Jan 1 04:53:05 UTC 2000 81° 83° 77° 38° 3797mV 1500MHz Sat Jan 1 04:53:07 UTC 2000 95° 83° 79° 38° 3824mV 1500MHz Sat Jan 1 04:53:08 UTC 2000 103° 88° 85° 38° 4106mV 250MHz ^Ckill 2653 2654
root@letux:~# cpufreq-info cpufrequtils 008: cpufreq-info (C) Dominik Brodowski 2004-2009 Report errors and bugs to cpufreq@vger.kernel.org, please. analyzing CPU 0: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 500 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:24.54%, 500 MHz:21.39%, 750 MHz:5.83%, 1000 MHz:11.23%, 1.25 GHz:4.45%, 1.50 GHz:32.56% (142) analyzing CPU 1: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 500 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:24.54%, 500 MHz:21.39%, 750 MHz:5.83%, 1000 MHz:11.23%, 1.25 GHz:4.45%, 1.50 GHz:32.56% (142) root@letux:~#
What I have changed was to apply Matthijs' impedance calibration patch
http://git.goldelico.com/?p=gta04-uboot.git;a=commit;h=918c4c46104714a3cfede...
and enabling the 1.5GHz OPP in DT. Plus recompile u-boot and kernel and then boot.
I always said it could be a software bug but nobody believed that it could be one. And I never believed that changing 1 bit in U-Boot could be sufficient (I thought we need to patch the kernel).
A realistic explanation is that our DDR3 is a little more sensitive to noise by 1.5GHz operation than the EVM and hence we get the problems but don't see them on the EVM if it is more robust.
At least my prototype was now running for 5 minutes @ 200% load @ 1.5GHz (cycling between 1.5GHz and 250MHz and 80°C and 110°C with a period of ca. 10 seconds) until I got
thermal thermal_zone0: critical temperature reached(125 C),shutting down
We should evaluate this further!
BR and please cross-check, Nikolaus
And what do you think about this?
root@letux:~# echo $(cat /sys/firmware/devicetree/base/model) Pyra-Handheld-V5.1 root@letux:~# cpufreq-info cpufrequtils 008: cpufreq-info (C) Dominik Brodowski 2004-2009 Report errors and bugs to cpufreq@vger.kernel.org, please. analyzing CPU 0: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.70 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz, 1.70 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.70 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 500 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:22.26%, 500 MHz:30.90%, 750 MHz:15.35%, 1000 MHz:13.11%, 1.25 GHz:7.68%, 1.50 GHz:3.79%, 1.70 GHz:6.90% (96) analyzing CPU 1: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.70 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz, 1.70 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.70 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 500 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:22.26%, 500 MHz:30.91%, 750 MHz:15.34%, 1000 MHz:13.11%, 1.25 GHz:7.68%, 1.50 GHz:3.79%, 1.70 GHz:6.90% (96) root@letux:~# ./high-load -n 100% load stress test for 2 cores Sat Jan 1 06:07:22 UTC 2000 44° 45° 44° 29° 4036mV 1000MHz Sat Jan 1 06:07:22 UTC 2000 44° 45° 44° 29° 3992mV 1000MHz Sat Jan 1 06:07:23 UTC 2000 44° 45° 45° 29° 3870mV 1700MHz Sat Jan 1 06:07:25 UTC 2000 47° 45° 45° 29° 3859mV 1700MHz Sat Jan 1 06:07:26 UTC 2000 67° 52° 53° 29° 3862mV 1700MHz Sat Jan 1 06:07:27 UTC 2000 75° 58° 59° 31° 3857mV 1700MHz Sat Jan 1 06:07:29 UTC 2000 79° 65° 63° 31° 3827mV 1700MHz Sat Jan 1 06:07:30 UTC 2000 85° 68° 68° 31° 3843mV 1700MHz Sat Jan 1 06:07:32 UTC 2000 87° 72° 71° 31° 3840mV 1700MHz Sat Jan 1 06:07:33 UTC 2000 90° 74° 75° 32° 3830mV 1700MHz Sat Jan 1 06:07:35 UTC 2000 90° 74° 75° 32° 3818mV 1700MHz Sat Jan 1 06:07:36 UTC 2000 93° 79° 76° 32° 3808mV 1700MHz Sat Jan 1 06:07:38 UTC 2000 95° 81° 78° 32° 3799mV 1700MHz Sat Jan 1 06:07:39 UTC 2000 98° 83° 80° 33° 3811mV 1700MHz Sat Jan 1 06:07:41 UTC 2000 100° 84° 83° 33° 3974mV 1000MHz Sat Jan 1 06:07:42 UTC 2000 103° 86° 84° 33° 4034mV 250MHz Sat Jan 1 06:07:45 UTC 2000 81° 82° 77° 31° 3826mV 1700MHz Sat Jan 1 06:07:46 UTC 2000 73° 74° 72° 31° 3805mV 1700MHz ^Ckill 2659 2660
root@letux:~#
A realistic explanation is that our DDR3 is a little more sensitive to noise by 1.5GHz operation than the EVM and hence we get the problems but don't see them on the EVM if it is more robust.
I had ruled out the DDR3 during my tests in November because the EMIF runs at the same speed, independently of the ARM core clock.
Therefore I thought there can not be an influence or problem. But if higher ARM load makes more noise on the DDR3 wires as well...
BR, Nikolaus
On Sat, Feb 25, 2017 at 7:19 PM, H. Nikolaus Schaller hns@goldelico.com wrote:
And what do you think about this?
... Sat Jan 1 06:07:45 UTC 2000 81° 82° 77° 31° 3826mV 1700MHz Sat Jan 1 06:07:46 UTC 2000 73° 74° 72° 31° 3805mV 1700MHz ^Ckill 2659 2660
root@letux:~#
A realistic explanation is that our DDR3 is a little more sensitive to noise by 1.5GHz operation than the EVM and hence we get the problems but don't see them on the EVM if it is more robust.
I had ruled out the DDR3 during my tests in November because the EMIF runs at the same speed, independently of the ARM core clock.
Therefore I thought there can not be an influence or problem. But if higher ARM load makes more noise on the DDR3 wires as well...
Wow very surprising indeed. When you found that DDR3 was severely undervolted and fixed it, it also became more stable and could handle higher frequencies, so it hinted that this was a DDR problem, but of course it was hard to make a connection back then...
We are really lucky Matt has found this, imagine the disappointment after the final boards arrive and would still be unable to run at 1.5GHz (well maybe they'd work without the sw fix, but I suspect they won't considering how we have everything crammed into such a small space).
This development made me unpack my board again hoping it would boot with all the fixes, unfortunately it doesn't, must be dead beyond hope. Even the board version detection is unstable on it:
U-Boot SPL 2016.11-01304-g918c4c4 (Feb 26 2017 - 00:12:35) OMAP5432-GP ES2.0 Trying to boot from MMC1 Found LC15 V5.5 board_mmc_init called (vers = 0) ERROR : check_gpio: invalid GPIO -1 <sometimes hangs here>
U-Boot SPL 2016.11-01304-g918c4c4 (Feb 26 2017 - 00:12:35) OMAP5432-GP ES2.0 Trying to boot from MMC1 Found LC15 V5.0 board_mmc_init called (vers = 50) MLO hard: gpio7 = eMMC soft: gpio32 = 0 (mmc1=eMMC) ctrl: gpio33 = 1 (ctrl=soft) mkimage signature not found - ih_magic = ea0000be Jumping to U-Boot loaded - jumping to U-Boot... <always hangs here>
Gražvydas
Hi Notaz,
Am 25.02.2017 um 23:52 schrieb Grazvydas Ignotas notasas@gmail.com:
On Sat, Feb 25, 2017 at 7:19 PM, H. Nikolaus Schaller hns@goldelico.com wrote:
And what do you think about this?
... Sat Jan 1 06:07:45 UTC 2000 81° 82° 77° 31° 3826mV 1700MHz Sat Jan 1 06:07:46 UTC 2000 73° 74° 72° 31° 3805mV 1700MHz ^Ckill 2659 2660
root@letux:~#
A realistic explanation is that our DDR3 is a little more sensitive to noise by 1.5GHz operation than the EVM and hence we get the problems but don't see them on the EVM if it is more robust.
I had ruled out the DDR3 during my tests in November because the EMIF runs at the same speed, independently of the ARM core clock.
Therefore I thought there can not be an influence or problem. But if higher ARM load makes more noise on the DDR3 wires as well...
Wow very surprising indeed. When you found that DDR3 was severely undervolted and fixed it, it also became more stable and could handle higher frequencies, so it hinted that this was a DDR problem, but of course it was hard to make a connection back then...
Indeed. I was stopped from thinking out of the box because the EMIF DDR clock is constant and not increased with ARM clock or CPU load... Hence I denied the possibility of a connection.
We are really lucky Matt has found this, imagine the disappointment after the final boards arrive and would still be unable to run at 1.5GHz (well maybe they'd work without the sw fix, but I suspect they won't considering how we have everything crammed into such a small space).
Indeed!
This development made me unpack my board again hoping it would boot with all the fixes, unfortunately it doesn't, must be dead beyond hope. Even the board version detection is unstable on it:
U-Boot SPL 2016.11-01304-g918c4c4 (Feb 26 2017 - 00:12:35) OMAP5432-GP ES2.0 Trying to boot from MMC1 Found LC15 V5.5 board_mmc_init called (vers = 0) ERROR : check_gpio: invalid GPIO -1
Hm. I have seen this also once or twice on my boards. Version detect reports V5.5. I am not sure if the CPU is reading the version detect gpio too fast after switching the pinmux so that the tiny capacitors of the wires have not been (dis)charged. The GPIO -1 is a followup of having wrong V5.x detection.
I have just pushed a patch to add a tiny delay. Let's see if it makes this issue go away.
And, there might be wrong version encoding resistors installed (10k instead of 1k and the pull-ups/downs of OMAP5 are stronger than OMAP3).
Maybe the resistors can be repaired. But I also handed over 20 main+CPU boards to ED on Friday, so I am sure he has reserved one set for you.
Then you could send me your old one for repair (to make it a working collectible) later.
<sometimes hangs here>
U-Boot SPL 2016.11-01304-g918c4c4 (Feb 26 2017 - 00:12:35) OMAP5432-GP ES2.0 Trying to boot from MMC1 Found LC15 V5.0
^^^ that looks better
board_mmc_init called (vers = 50) MLO hard: gpio7 = eMMC soft: gpio32 = 0 (mmc1=eMMC) ctrl: gpio33 = 1 (ctrl=soft) mkimage signature not found - ih_magic = ea0000be Jumping to U-Boot loaded - jumping to U-Boot...
<always hangs here>
Gražvydas
BR, Nikolaus
On Sun, Feb 26, 2017 at 9:10 AM, H. Nikolaus Schaller hns@goldelico.com wrote:
Am 25.02.2017 um 23:52 schrieb Grazvydas Ignotas notasas@gmail.com:
This development made me unpack my board again hoping it would boot with all the fixes, unfortunately it doesn't, must be dead beyond hope. Even the board version detection is unstable on it:
U-Boot SPL 2016.11-01304-g918c4c4 (Feb 26 2017 - 00:12:35) OMAP5432-GP ES2.0 Trying to boot from MMC1 Found LC15 V5.5 board_mmc_init called (vers = 0) ERROR : check_gpio: invalid GPIO -1
Hm. I have seen this also once or twice on my boards. Version detect reports V5.5.
I can reproduce it reliably by doing this: - disconnect all power (battery, USB debug port), remove all boot media - wait for half a minute or so - connect battery, USB debug cable (no boot media at this point) - press and release power - insert boot SD
at this point it starts MLO and always reports V5.5. I've investigated it and it seems it's because DRAM init fails. There is a static variable "vers" and according to the map file it goes to DRAM, and reading that you'll always get 0 (but occasionally trash). gcc will cache the result of second pair of gpio_get_value() to a register so you get a partially valid result. I've attached patches 0001 and 0002 to deal with this situation.
I've tried to find out why DRAM init fails and found something with the help of patch 0003, where it prints: "SDRAM: identified size not same as expected size identified: 4 expected: 40000000" so something goes wrong during setup. There is a warm_reset() function that's used quite a bit by RAM init code (emif-common.c) and it returns 0x20 (EXTERNAL_WARM_RST) in above described scenario, so emif-common thinks it's a warm reset and probably skips some init (the code is confusing so I'm not sure). Might be worth printing warm_reset() result in lc15.c, might help to figure out the boot mysteries.
I'm not sure why I need patch 0004 (and apparently you don't), u-boot never loaded for me without it. Perhaps the "boot mode" that comes from ROM code is different on my broken board (u-boot SPL seems to use it to decide how to load main u-boot). You can read it from gd->arch.omap_boot_device , apparently arch/arm/include/asm/arch-omap5/spl.h has the meanings of that variable.
Gražvydas
Hi Notaz,
Am 26.02.2017 um 22:24 schrieb Grazvydas Ignotas notasas@gmail.com:
On Sun, Feb 26, 2017 at 9:10 AM, H. Nikolaus Schaller hns@goldelico.com wrote:
Am 25.02.2017 um 23:52 schrieb Grazvydas Ignotas notasas@gmail.com:
This development made me unpack my board again hoping it would boot with all the fixes, unfortunately it doesn't, must be dead beyond hope. Even the board version detection is unstable on it:
U-Boot SPL 2016.11-01304-g918c4c4 (Feb 26 2017 - 00:12:35) OMAP5432-GP ES2.0 Trying to boot from MMC1 Found LC15 V5.5 board_mmc_init called (vers = 0) ERROR : check_gpio: invalid GPIO -1
Hm. I have seen this also once or twice on my boards. Version detect reports V5.5.
I can reproduce it reliably by doing this:
- disconnect all power (battery, USB debug port), remove all boot media
- wait for half a minute or so
- connect battery, USB debug cable (no boot media at this point)
- press and release power
- insert boot SD
at this point it starts MLO and always reports V5.5.
Ok!
I already had observed that there is some "memory effect" which has something to do with how fast and in which sequence power is (re)applied and so far I thought that the Palmas backup (supercap) has something to do with it.
I've investigated it and it seems it's because DRAM init fails. There is a static variable "vers" and according to the map file it goes to DRAM, and reading that you'll always get 0 (but occasionally trash).
Ah, interesting! I always thought that MLO does not use DRAM at all because it runs from and within SRAM. But since it is now built as SPL from the U-Boot sources, the linker commands may do differently.
The problem might have become visible only after this version detection was moved to SPL as well. Before, we only had it in U-Boot where DRAM is already initialized by SPL/MLO.
gcc will cache the result of second pair of gpio_get_value() to a register so you get a partially valid result. I've attached patches 0001 and 0002 to deal with this situation.
Thanks! Have applied them and see no negative effects... Well, I haven't seen the positive one either, but let's observe.
I've tried to find out why DRAM init fails and found something with the help of patch 0003, where it prints: "SDRAM: identified size not same as expected size identified: 4 expected: 40000000" so something goes wrong during setup. There is a warm_reset() function that's used quite a bit by RAM init code (emif-common.c) and it returns 0x20 (EXTERNAL_WARM_RST) in above described scenario, so emif-common thinks it's a warm reset and probably skips some init (the code is confusing so I'm not sure). Might be worth printing warm_reset() result in lc15.c, might help to figure out the boot mysteries.
Yes, there must be something wrong with warm reset. If I "reboot" from Linux everything works well, up to Kernel starting... where nothing happens any more.
I'm not sure why I need patch 0004 (and apparently you don't), u-boot never loaded for me without it. Perhaps the "boot mode" that comes from ROM code is different on my broken board (u-boot SPL seems to use it to decide how to load main u-boot). You can read it from gd->arch.omap_boot_device , apparently arch/arm/include/asm/arch-omap5/spl.h has the meanings of that variable.
I have always seen the warning but it still did boot for me.
So I have applied all 4 patches and reverted the 2µs delay for reading gpios and it works equally good for me. Hopefully better for everyone else.
Are you now able to boot to Linux with these patches?
BR and thanks, Nikolaus
On 28 February 2017 at 09:18, H. Nikolaus Schaller hns@goldelico.com wrote:
I already had observed that there is some "memory effect" which has something to do with how fast and in which sequence power is (re)applied and so far I thought that the Palmas backup (supercap) has something to do with it.
Ah there's a supercap too? I hadn't even stopped to think about that one...
I've quite often seen strange memory effects, or "bistability" of some sort where it either boots fine (into u-boot at least) repeatedly, or utterly fails (often even without any message on console) repeatedly.
Ah, interesting! I always thought that MLO does not use DRAM at all
because it runs from and within SRAM.
I think it has some config options controlling how much is relocated to DRAM.
Yes, there must be something wrong with warm reset. If I "reboot" from Linux everything works well, up to Kernel starting... where nothing happens any more.
I've also noticed that the reset using shoulder button + power button rarely recovers the system from a wedged state (although it seems to work fine if the system is running fine).
Matthijs
Am Tue, 28 Feb 2017 20:14:45 +0100 hat Matthijs van Duin matthijsvanduin@gmail.com geschrieben:
Hi,
I already had observed that there is some "memory effect" which has something to do with how fast and in which sequence power is (re)applied and so far I thought that the Palmas backup (supercap) has something to do with it.
Ah there's a supercap too? I hadn't even stopped to think about that one.. I've quite often seen strange memory effects, or "bistability" of some sort where it either boots fine (into u-boot at least) repeatedly, or utterly fails (often even without any message on console) repeatedly.
Hm, that's very weird and we should try to find out what happens here. It doesn't really sound like a hardware instability issue if the hardware works fine once you managed to boot it up.
I wonder if there are some parameters set that freeze the system and that aren't resetted properly...?
Yes, there must be something wrong with warm reset. If I "reboot" from Linux everything works well, up to Kernel starting... where nothing happens any more.
I've also noticed that the reset using shoulder button + power button rarely recovers the system from a wedged state (although it seems to work fine if the system is running fine).
What does the Shoulder-Button + Power Button do? Does it send some reset signal to the OMAP / Palmas? Or does it have it's own reset circuit?
The reset should ALWAYS work, especially when the system crashes or freezes. If the OMAP / PALMAS freezes and ignores the reset signal, then that's not really useful :)
On 1 March 2017 at 04:36, Michael Mrozek EvilDragon@openpandora.org wrote:
Am Tue, 28 Feb 2017 20:14:45 +0100 hat Matthijs van Duin matthijsvanduin@gmail.com geschrieben:
Hi,
I already had observed that there is some "memory effect" which has something to do with how fast and in which sequence power is (re)applied and so far I thought that the Palmas backup (supercap) has something to do with it.
Ah there's a supercap too? I hadn't even stopped to think about that one.. I've quite often seen strange memory effects, or "bistability" of some sort where it either boots fine (into u-boot at least) repeatedly, or utterly fails (often even without any message on console) repeatedly.
Hm, that's very weird and we should try to find out what happens here.
I agree. I've been working on getting the early debug console for SPL working (u-boot's equivalent to earlycon) so we can get console output at any time. It's not working yet, but I'm fairly sure I know why so I'll give that another try.
Linux earlycon seems to work with: CONFIG_SERIAL_EARLYCON=y CONFIG_SERIAL_8250=y CONFIG_SERIAL_8250_CONSOLE=y CONFIG_SERIAL_8250_OMAP=y CONFIG_SERIAL_8250_OMAP_TTYO_FIXUP=y CONFIG_SERIAL_OF_PLATFORM=y # CONFIG_SERIAL_OMAP is not set CONFIG_DEBUG_LL=y CONFIG_DEBUG_OMAP2PLUS_UART=y CONFIG_DEBUG_OMAP4UART3=y CONFIG_EARLY_PRINTK=y
It doesn't really sound like a hardware instability issue if the
hardware works fine once you managed to boot it up.
No, I really have no idea yet what's going on there, or which stateful components are involved (since problems even seem to be somewhat persistent across power cycling by long-pressing the power button).
Note that I'm also however suffering from plain instability, I've never been able to run Linux for very long.
I wonder if there are some parameters set that freeze the system and
that aren't resetted properly...?
There are plenty of settings that aren't reset at all, but they also shouldn't have to be reset in the first place. They should simply be configured correctly.
What does the Shoulder-Button + Power Button do?
It asserts nRESWARM of omap5 and of palmas, and resets eMMC. This seems correct to me.
The reset should ALWAYS work, especially when the system crashes or freezes. If the OMAP / PALMAS freezes and ignores the reset signal, then that's not really useful :)
They definitely don't (and can't) ignore it. I don't really have an explanation yet why this degree of reset doesn't always resolve crashes, but as I said I'm even experiencing problems that seem to persist across power cycling.
Note that although warm reset doesn't reset *everything*, this is generally a good thing. For example a warm reset allows the system to determine what caused the reset, and in principle it should be possible to perform JTAG debugging across a warm reset. A cold reset doesn't allow either of these.
Matthijs
On Wed, Mar 1, 2017 at 10:41 AM, Matthijs van Duin matthijsvanduin@gmail.com wrote:
On 1 March 2017 at 04:36, Michael Mrozek EvilDragon@openpandora.org wrote:
Am Tue, 28 Feb 2017 20:14:45 +0100 hat Matthijs van Duin matthijsvanduin@gmail.com geschrieben:
I already had observed that there is some "memory effect" which has something to do with how fast and in which sequence power is (re)applied and so far I thought that the Palmas backup (supercap) has something to do with it.
Ah there's a supercap too? I hadn't even stopped to think about that one.. I've quite often seen strange memory effects, or "bistability" of some sort where it either boots fine (into u-boot at least) repeatedly, or utterly fails (often even without any message on console) repeatedly.
Hm, that's very weird and we should try to find out what happens here.
I agree. I've been working on getting the early debug console for SPL working (u-boot's equivalent to earlycon) so we can get console output at any time. It's not working yet, but I'm fairly sure I know why so I'll give that another try.
I've also tried to do that and the best I've managed is what I've sent with patch 0003 to get it just after setup_early_clocks(), so there's something needed in that function.
It doesn't really sound like a hardware instability issue if the hardware works fine once you managed to boot it up.
No, I really have no idea yet what's going on there, or which stateful components are involved (since problems even seem to be somewhat persistent across power cycling by long-pressing the power button).
Note that I'm also however suffering from plain instability, I've never been able to run Linux for very long.
Hmm that's not good, I thought only my broken board is suffering from that. I do always get the "U-Boot SPL" message though, where it hangs after that is random ranging from just after printing the "U-Boot SPL" line to somewhere in the middle of kernel booting.
Gražvydas
On 03/01/2017 12:30 PM, Grazvydas Ignotas wrote:
Note that I'm also however suffering from plain instability, I've never been able to run Linux for very long.
Hmm that's not good, I thought only my broken board is suffering from that. I do always get the "U-Boot SPL" message though, where it hangs after that is random ranging from just after printing the "U-Boot SPL" line to somewhere in the middle of kernel booting.
Mine seems to behave exactly the same as yours, it takes 10-20 minutes of warming up before it'll go anywhere near booting linux, the first half of which it doesn't get beyond the the U-Boot SPL line. Once it does actually go into u-boot proper for the first time, it does get easier to boot it.
I haven't tried your u-boot patches yet, maybe they'll make it go slightly smoother on mine too.
I did have it running linux for reasonable amounts of time (>1 hour of mate desktop use), but that mostly depends on what mood it's in. Most of the time it just feezes a few minutes/seconds after getting to the login prompt (if it gets there at all).
Am Wed, 1 Mar 2017 13:00:43 +0100 hat aTc atc@k-n-p.org geschrieben:
Hi,
well, I'm not surprised about notaz' and aTc's boards, as these are from the very first production run where the stencil layer wasn't really good and most of the CPU boards had been reworked... so who knows.
The weird thing is though that I had aTcs Pyra running throughout the full GamesCom without any crash or problem, and it didn't really need some warming up.
I was using an older kernel / u-boot from Nikolaus at that time, so who knows if the issue sits there... though it doesn't make much sense, as it works when it's warmed up.
zmatts PCB was not really tested, and I remember it crashed once when booting it up at GC as well, so maybe it's a bit flakey.
Oh... and that PCB has also been used for a crash test... I threw the assembled Pyra on the ground, so maybe it received some damage here ;)
I'm having 20 tested PCBs from Nikolaus here, so I'll try to assemble and ship them to you ASAP.
On 03/01/2017 12:30 PM, Grazvydas Ignotas wrote:
Note that I'm also however suffering from plain instability, I've never been able to run Linux for very long.
Hmm that's not good, I thought only my broken board is suffering from that. I do always get the "U-Boot SPL" message though, where it hangs after that is random ranging from just after printing the "U-Boot SPL" line to somewhere in the middle of kernel booting.
Mine seems to behave exactly the same as yours, it takes 10-20 minutes of warming up before it'll go anywhere near booting linux, the first half of which it doesn't get beyond the the U-Boot SPL line. Once it does actually go into u-boot proper for the first time, it does get easier to boot it.
I haven't tried your u-boot patches yet, maybe they'll make it go slightly smoother on mine too.
I did have it running linux for reasonable amounts of time (>1 hour of mate desktop use), but that mostly depends on what mood it's in. Most of the time it just feezes a few minutes/seconds after getting to the login prompt (if it gets there at all).
Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
On Wed, Mar 1, 2017 at 2:11 PM, Michael Mrozek EvilDragon@openpandora.org wrote:
Am Wed, 1 Mar 2017 13:00:43 +0100 hat aTc atc@k-n-p.org geschrieben:
The weird thing is though that I had aTcs Pyra running throughout the full GamesCom without any crash or problem, and it didn't really need some warming up.
Well mine was also fine just after it arrived, it would boot on the first try and run stable, but then after several days it went into it's current state. IIRC I haven't even changed any software back then as I had no time to mess with it, and there was no abuse on it, it just sat on the table.
Assuming Matt's was once fine too it's a bit worrying, I hope it's not something bad happening after the boards get used for a bit.
Gražvydas
Am Wed, 1 Mar 2017 23:14:21 +0200 hat Grazvydas Ignotas notasas@gmail.com geschrieben:
Hi,
The weird thing is though that I had aTcs Pyra running throughout the full GamesCom without any crash or problem, and it didn't really need some warming up.
Well mine was also fine just after it arrived, it would boot on the first try and run stable, but then after several days it went into it's current state. IIRC I haven't even changed any software back then as I had no time to mess with it, and there was no abuse on it, it just sat on the table.
Well, that wouldn't be weird for the first batch PCBs. The stencil layer wasn't optimized, so the CPU boards had too little solderpaste and therefore, bad solder joints. Who knows what happens with some heating...
Mine still works perfect since day one (I was lucky), and I had it with me on holidays as well as at the Gamescom and other events.
The reason we didn't ship many boards from the first batch was exactly for that reason: Most of them weren't really reliable. We managed to build four working ones for Gamescom, but that's it.
Assuming Matt's was once fine too it's a bit worrying, I hope it's not something bad happening after the boards get used for a bit.
Well, out of 20 PCBs Nikolaus tested from the current batch, 2 had stability issues as well. Matt's board never really has been tested before. We assembled it, made a quick hardware test for a few minutes, threw it on the ground (to see whether it survives that or not) and then shipped it to him.
What we don't know yes is why these 2 boards have stability issues. We can xray them at GC - but it might also be too much noise on the ground plane combined with different tolerances of both the PALMAS and the OMAP5.
On 1 March 2017 at 12:30, Grazvydas Ignotas notasas@gmail.com wrote:
I've also tried to do that and the best I've managed is what I've sent with patch 0003 to get it just after setup_early_clocks(), so there's something needed in that function.
Hah, I just saw your patch... nice.
See, I made the mistake of trying to go the official route since I had seen the CONFIG_DEBUG_UART option. I enabled that and set the appropriate config options below that, examined drivers/serial/ns16550.c and fixed the _debug_uart_init code which was obviously not going to work for the omap uart (failed to init mdr1), added board_debug_uart_init as instructed by the help text for CONFIG_DEBUG_UART_BOARD_INIT... and then discovered that instead of an earlycon I had an earlycrash since nothing ever actually invokes the debug console init code. I guess inserting a call to that was left as an exercise.
You just hacked in 8 lines and it works! :D
Matthijs
On Tue, Feb 28, 2017 at 10:18 AM, H. Nikolaus Schaller hns@goldelico.com wrote:
Are you now able to boot to Linux with these patches?
Sort of... if I "warm up" the board for half an hour or so by keeping it on with charger connected. While cold it hangs completely randomly, most often early in MLO, sometimes in the middle of printing serial messages.
Actually after a warmup it's working quite well today. It's interesting it can do 1.5GHz too now, that's even without SMPS3 fb trace fix. Maybe I should disable thermal throttling and run it for a while on NEON loop with hope to melt the bad solder point, although that may be bad for the battery (I can't get it working without a battery).
Gražvydas
Hi Notaz, thanks for noting the neon_loop2
I did have it running during my tests, but had simply overwritten the source code and forgot to git add it to the Letux kernel tree.
Hence it was missing in my link to the script.
I have fixed that now so that high-load -n2 runs the latest version.
It drives temperature up even faster, but no immediate crash:
root@letux:~# ./high-load -n2 100% load stress test for 2 cores running ./neon_loop2 Tue Feb 28 08:52:26 UTC 2017 49° 51° 49° 35° 4018mV 1000MHz Tue Feb 28 08:52:26 UTC 2017 49° 51° 49° 35° 3886mV 1500MHz Tue Feb 28 08:52:28 UTC 2017 50° 50° 49° 35° 3825mV 1700MHz Tue Feb 28 08:52:29 UTC 2017 63° 52° 51° 35° 3825mV 1700MHz Tue Feb 28 08:52:31 UTC 2017 78° 63° 62° 35° 3794mV 1700MHz Tue Feb 28 08:52:32 UTC 2017 86° 70° 69° 38° 3805mV 1700MHz Tue Feb 28 08:52:33 UTC 2017 92° 74° 74° 38° 3798mV 1700MHz Tue Feb 28 08:52:35 UTC 2017 96° 79° 79° 38° 3770mV 1700MHz Tue Feb 28 08:52:36 UTC 2017 96° 79° 79° 38° 3764mV 1700MHz Tue Feb 28 08:52:38 UTC 2017 101° 84° 82° 39° 4050mV 250MHz Tue Feb 28 08:52:40 UTC 2017 82° 83° 79° 39° 4012mV 750MHz Tue Feb 28 08:52:42 UTC 2017 75° 75° 73° 39° 3797mV 1700MHz Tue Feb 28 08:52:43 UTC 2017 75° 75° 74° 38° 3777mV 1700MHz Tue Feb 28 08:52:45 UTC 2017 91° 75° 74° 38° 3759mV 1700MHz Tue Feb 28 08:52:46 UTC 2017 99° 83° 81° 38° 3752mV 1700MHz Tue Feb 28 08:52:48 UTC 2017 104° 87° 85° 38° 4012mV 500MHz Tue Feb 28 08:52:49 UTC 2017 110° 92° 89° 37° 4030mV 500MHz Tue Feb 28 08:52:51 UTC 2017 87° 88° 84° 37° 3789mV 1700MHz Tue Feb 28 08:52:53 UTC 2017 79° 82° 77° 37° 3778mV 1700MHz ^Ckill 4831 4832
root@letux:~#
Well, I might have stopped early enough before it reaches the 125°C hard limit.
To me it looks as if the governor is too slow to react or it is re-enabling the highest clock frequency too early so that there is a lot of overshoot due to heat which has to travel first from the ARM/NEON core to the temperature measurement point. I.e. reducing clock to 500MHz does not cool down fast enough.
The good news is that we are only thermally limited now but no longer from switching noise.
There is still an area for optimization: it may be possible to reduce the OPP voltage at 1.7GHz to reduce heat generation and power consumption.
BTW: is it possible to define all OPPs in DT but stop the default on-demand governor to use higher clock rates until enabled somehow? So far it looks to me as if it really uses uses all OPPs defined in DT.
The problem is that even a simple apt-get update/upgrade or longer install might drive the CPU into 1.7GHz operation and potentially overheat.
In my ideal world, we would still define all OPPs up to 1.5/1.7 GHz in DT but set a limit of 1GHz unless someone overwrites the limit by some tool.
BR, Nikolaus
Hi,
On 2017-02-28 10:09, H. Nikolaus Schaller wrote:
The problem is that even a simple apt-get update/upgrade or longer install might drive the CPU into 1.7GHz operation and potentially overheat.
Speaking from my experiences with raspberry pi 3 I'd say it would be nice if there was a way to limit max. frequency as soon as the core is above XX°C, because the governor is just too slow, which results in constant switches between high frame rate and lags. It would be better to limit the CPU to e.g. 1.2GHz as soon as it hits max. temp so we'd have a constant frame rate, like this:
boot @ 1.7GHz, idle prompt @ 500 MHz, app stating @ 1.7GHz, thermal throttle @ 80°C -> 500MHz (lag), cool down to 50°C -> 1.2GHz (instead of 1.7GHz).
Also worth noting is that the latest Samsung galaxy S series devices have the same problem as the RPi3: thermal throttle = constant lag.
Best regards,
Andreas
On 26 February 2017 at 22:24, Grazvydas Ignotas notasas@gmail.com wrote:
I've tried to find out why DRAM init fails and found something with the help of patch 0003, where it prints: "SDRAM: identified size not same as expected size identified: 4 expected: 40000000" so something goes wrong during setup.
I also seem to be getting this nearly every time I'm failing to get into u-boot. I have no idea why this is merely printed as a warning instead of the panic() it should be. I have no idea how it is possible that Nikolaus is sometimes able to boot normally after this message, since it means it didn't even pass a rudimentary memory test.
My impression is that ddr3 leveling is failing. EMIF has a history of buggy hardware leveling, and incremental (as opposed to full) leveling is still marked as not-supported for omap5 and dra7. I'm also seeing lots of error flags set in the phy after leveling, though curiously that's also the case when it does boot, so maybe the flags are just garbage. I'll see if I can print more informative debug info about what's going on during EMIF initialization.
There is a warm_reset() function that's used quite a bit by RAM init code
(emif-common.c)
Yes, which is suspicious. It is not guaranteed that warm reset implies EMIF has been initialized before. The correct way to record that initialization has been done for some module is by clearing its "context lost" bit in PRM, which exist for exactly that purpose.
However, warm_reset() may still be the right thing to check if extra steps are required when emif is not necessarily in a clean state as it is just after cold reset. I've noticed that an example of such is being unintentionally skipped thanks to a typo: https://github.com/mvduin/u-boot/commit/5435989af0ad70e7c219 4eca79232a3a2b1fcc18
This might at least explain why warm reset rarely seemed to resolve anything? Let's see if that problem now improves.
I also have to say the whole EMIF initialization logic in that file really looks quite hacky and convoluted. Remarks like this one (in an older revision of the file) are also not exactly confidence-inspiring: https://github.com/mvduin/u-boot/blob/6c70935d7525/arch/arm/cpu/armv7/omap-c...
Matthijs
On 7 March 2017 at 04:24, Matthijs van Duin matthijsvanduin@gmail.com wrote:
My impression is that ddr3 leveling is failing. EMIF has a history of buggy hardware leveling, and incremental (as opposed to full) leveling is still marked as not-supported for omap5 and dra7. I'm also seeing lots of error flags set in the phy after leveling, though curiously that's also the case when it does boot, so maybe the flags are just garbage. I'll see if I can print more informative debug info about what's going on during EMIF initialization.
Well, this has been informative, although I'm not sure yet what conclusions to draw...
In case anyone else wants a vomit of ddr3 timings across their console at boot, I've pushed my verbose emif debugging patches to github: https://github.com/mvduin/u-boot
My repo is rather a mess of unrelated patches, but the three emif patches are at the top and can probably be cherry-picked cleanly.
Here's an apparently-successful power-up into u-boot:
do_sdram_init() 4c000000
omap5_ddr3_leveling(0x4c000000) cmd phy dlls: ok f6 ok f5 data phy dlls: ok f7 ok f4 ok fc ok f7 phy status 5-7: 0 99990924 924 read dqs ratios: 7f 40 7f 3c 41 43 3a 3b fifo we ratios: 69 7d 72 76 69 7d 73 74 write data ratios: 10a 6e 10a 7a 75 6d 6f 78 write dqs ratios: ea 4e ea 5a 55 4d 4f 58 fifo misalignment: yes yes yes yes fifo rst errors: 2 5 3 4 cmd phy dlls: ok f6 ok f5 data phy dlls: ok f8 ok f3 ok fc ok f7 phy status 5-7: 0 99990924 924 read dqs ratios: 7e 41 7e 3c 41 44 3a 3b fifo we ratios: 68 7c 71 75 68 7c 72 73 write data ratios: 103 6c 100 78 74 6b 6e 77 write dqs ratios: e3 4c e0 58 54 4b 4e 57 leveling errors: -w- --- -w- -w- fifo rst errors: 0 2 0 2 <<do_sdram_init() 4c000000
do_sdram_init() 4d000000
omap5_ddr3_leveling(0x4d000000) cmd phy dlls: ok f9 ok fb data phy dlls: ok f6 ok fb ok fc ok f3 phy status 5-7: 0 99990924 924 read dqs ratios: 3a 3a 37 46 3d 39 39 45 fifo we ratios: 70 79 7a 74 75 78 7f 72 write data ratios: 78 72 7a 78 77 70 7b 76 write dqs ratios: 58 52 5a 58 57 50 5b 56 fifo misalignment: yes yes yes yes fifo rst errors: 2 3 5 2 cmd phy dlls: ok f9 ok fc data phy dlls: ok f7 ok fc ok fb ok f3 phy status 5-7: 0 99990924 924 read dqs ratios: 3b 3b 36 46 3e 38 39 45 fifo we ratios: 6f 78 79 74 74 77 7e 71 write data ratios: 76 70 79 78 75 6f 79 76 write dqs ratios: 56 50 59 58 55 4f 59 56 leveling errors: -w- rw- --- -w- fifo misalignment: - - - yes fifo rst errors: 1 2 1 2 <<do_sdram_init() 4d000000 get_ram_size() successful
This again is "success":
do_sdram_init() 4c000000
omap5_ddr3_leveling(0x4c000000) cmd phy dlls: ok f7 ok f6 data phy dlls: ok f9 ok f4 ok fd ok f7 phy status 5-7: 0 99990924 924 read dqs ratios: 7f 41 7f 3b 40 44 3a 3b fifo we ratios: 6a 7d 74 77 6a 7e 74 76 write data ratios: 10a 6e 10a 79 75 6e 6f 78 write dqs ratios: ea 4e ea 59 55 4e 4f 58 fifo misalignment: yes yes yes yes fifo rst errors: 2 5 4 3 cmd phy dlls: ok f7 ok f6 data phy dlls: ok f9 ok f4 ok fc ok f7 phy status 5-7: 0 99990924 924 read dqs ratios: 7e 41 7e 3b 41 44 3a 3b fifo we ratios: 69 7c 73 76 69 7d 73 75 write data ratios: 103 6c 100 78 73 6b 6d 76 write dqs ratios: e3 4c e0 58 53 4b 4d 56 leveling errors: --- -w- --- --- fifo rst errors: 1 2 0 2 <<do_sdram_init() 4c000000
do_sdram_init() 4d000000
omap5_ddr3_leveling(0x4d000000) cmd phy dlls: ok f8 ok fc data phy dlls: ok f7 ok fb ok fb ok f3 phy status 5-7: 0 99990924 924 read dqs ratios: 3b 3c 37 46 3d 39 39 45 fifo we ratios: 70 79 7b 76 75 76 7f 71 write data ratios: 78 71 7a 78 76 70 7c 77 write dqs ratios: 58 51 5a 58 56 50 5c 57 fifo misalignment: yes yes yes yes fifo rst errors: 2 4 5 2 cmd phy dlls: ok f9 ok fc data phy dlls: ok f7 ok fb ok fc ok f3 phy status 5-7: 0 99990924 924 read dqs ratios: 3c 3c 37 46 3e 38 39 45 fifo we ratios: 6f 78 7a 75 74 75 7e 72 write data ratios: 76 6f 78 75 74 6f 79 75 write dqs ratios: 56 4f 58 55 54 4f 59 55 leveling errors: --- rw- --- --g fifo misalignment: - - - yes fifo rst errors: 1 1 1 2 <<do_sdram_init() 4d000000 get_ram_size() successful
Even ignoring the leveling errors (which could be bogus), the read fifo errors seem very serious to me. Nevertheless u-boot seemed to work fine, and I've even run a memory test (mtest $loadaddr $fdtaddr) which also seemed to confirm that there were no problems with memory.
I also have trouble believing the write leveling values for EMIF1 rank 0 bytes 0 and 2 could possibly be right considering how much larger they are than all others, especially since rank 0+1 bytes 0+1 are in a single package and ditto rank 0+1 bytes 2+3. It's possible though that there's enough margin that it can get away with those settings, but in that case it's quite an accomplishment that it also manages to pick non-working timings :P
Here for example is a failed boot:
do_sdram_init() 4c000000
omap5_ddr3_leveling(0x4c000000) cmd phy dlls: ok f6 ok f5 data phy dlls: ok f8 ok f3 ok fc ok f7 phy status 5-7: 0 99990924 924 read dqs ratios: 3e 44 3a 3b 7f 44 3b 3a fifo we ratios: 69 7c 73 75 1b 7c 74 76 write data ratios: 77 6f 70 79 76 6e 70 79 write dqs ratios: 57 4f 50 59 56 4e 50 59 fifo misalignment: yes yes yes yes fifo rst errors: 15 4 2 3 cmd phy dlls: ok f6 ok f5 data phy dlls: ok f7 ok f3 ok fb ok f7 phy status 5-7: 0 99990924 924 read dqs ratios: 3e 45 3a 3b 7e 45 3b 3a fifo we ratios: 68 7d 72 74 1a 7b 73 75 write data ratios: 75 6d 6d 78 74 6b 6d 76 write dqs ratios: 55 4d 4d 58 54 4b 4d 56 leveling errors: rw- --- --- --- fifo misalignment: yes yes - - fifo rst errors: 1 2 1 2 <<do_sdram_init() 4c000000
do_sdram_init() 4d000000
omap5_ddr3_leveling(0x4d000000) cmd phy dlls: ok f9 ok fc data phy dlls: ok f7 ok fc ok fc ok f4 phy status 5-7: 0 99990924 924 read dqs ratios: 3c 3a 38 46 3d 38 39 45 fifo we ratios: 71 77 7c 75 73 77 7e 73 write data ratios: 78 71 7c 78 78 70 7b 77 write dqs ratios: 58 51 5c 58 58 50 5b 57 fifo misalignment: yes yes yes yes fifo rst errors: 2 4 5 2 cmd phy dlls: ok f9 ok fb data phy dlls: ok f7 ok fb ok fb ok f3 phy status 5-7: 0 99990924 924 read dqs ratios: 3c 3b 38 46 3d 39 3a 45 fifo we ratios: 70 76 7b 74 72 76 7d 72 write data ratios: 75 71 7a 76 74 6d 79 75 write dqs ratios: 55 51 5a 56 54 4d 59 55 leveling errors: --- --- -w- --- fifo rst errors: 2 1 2 2 <<do_sdram_init() 4d000000 ### SDRAM: identified: 0 expected: 40000000 ###
This time no problems with write-leveling (other than the phy reporting errors with it) but it completely screwed up the read-leveling of EMIF1 rank 1 lane 0...
As final example, below is the u-boot log showing the "memory" effect of repeated similar failures across power cycling (my patches perform poweroff() when the memory test fails) until it finally succeeds. This time I was testing in 2GB configuration, hence the rank 1 values are nonsense and can be ignored.
This is also the first time I've seen leveling fail entirely, resulting in timeouts.
U-Boot SPL 2016.11-00249-gcb460d33abe8 (Mar 07 2017 - 09:18:21) OMAP5432-GP ES2.0
sdram_init()
in_sdram = 0
do_sdram_init() 4c000000
omap5_ddr3_leveling(0x4c000000) ddr3 write leveling timeout ddr3 read dqs gate training timeout cmd phy dlls: ok f6 ok f6 data phy dlls: ok f8 ok f4 ok fc ok f7 phy status 5-7: 0 99990820 924 read dqs ratios: 7f 40 7f 3b 0 0 0 0 fifo we ratios: 0 7d 0 74 700 700 700 700 write data ratios: 10a 6e 10a 79 36b 307 365 353 write dqs ratios: ea 4e ea 59 34b 2e7 345 333 fifo misalignment: - yes - yes fifo rst errors: 15 1 15 0 cmd phy dlls: ok f6 ok f6 data phy dlls: ok f8 ok f3 ok fb ok f7 phy status 5-7: 0 99990924 924 read dqs ratios: 7e 41 7e 3b 0 0 0 0 fifo we ratios: 7ff 7c 7ff 73 700 700 700 700 write data ratios: 106 6c 102 78 36b 307 365 353 write dqs ratios: e6 4c e2 58 34b 2e7 345 333 leveling errors: r-- --- r-- -w- fifo misalignment: yes - yes - fifo rst errors: 0 1 0 1 <<do_sdram_init() 4c000000
do_sdram_init() 4d000000
omap5_ddr3_leveling(0x4d000000) cmd phy dlls: ok f9 ok fc data phy dlls: ok f7 ok fc ok fc ok f3 phy status 5-7: 0 99990924 924 read dqs ratios: 3d 3b 38 46 0 0 0 0 fifo we ratios: 71 79 7b 75 700 700 700 700 write data ratios: 77 72 7b 78 1b3 3eb 287 d7 write dqs ratios: 57 52 5b 58 193 3cb 267 b7 fifo misalignment: yes yes yes yes fifo rst errors: 2 1 2 2 cmd phy dlls: ok f9 ok fc data phy dlls: ok f6 ok fc ok fc ok f3 phy status 5-7: 0 99990924 924 read dqs ratios: 3d 3c 38 46 0 0 0 0 fifo we ratios: 70 78 7a 74 700 700 700 700 write data ratios: 75 70 78 76 1b3 3eb 287 d7 write dqs ratios: 55 50 58 56 193 3cb 267 b7 fifo rst errors: 0 1 0 1 <<do_sdram_init() 4d000000 ### SDRAM: identified: 4 expected: 40000000 ###
U-Boot SPL 2016.11-00249-gcb460d33abe8 (Mar 07 2017 - 09:18:21) OMAP5432-GP ES2.0
sdram_init()
in_sdram = 0
do_sdram_init() 4c000000
omap5_ddr3_leveling(0x4c000000) ddr3 write leveling timeout ddr3 read dqs gate training timeout cmd phy dlls: ok f7 ok f6 data phy dlls: ok f8 ok f4 ok fc ok f8 phy status 5-7: c1 99990820 924 read dqs ratios: 7f 42 7f 3c 0 0 0 0 fifo we ratios: 0 7d 0 75 700 700 700 700 write data ratios: 10a 6e 10a 79 38b 307 365 12 write dqs ratios: ea 4e ea 59 36b 2e7 345 3f2 fifo misalignment: - yes - yes fifo rst errors: 15 2 15 1 cmd phy dlls: ok f6 ok f6 data phy dlls: ok f8 ok f4 ok fc ok f7 phy status 5-7: 0 99990924 924 read dqs ratios: 7e 42 7e 3d 0 0 0 0 fifo we ratios: 7ff 7e 7ff 75 700 700 700 700 write data ratios: 106 6c 102 79 38b 307 365 12 write dqs ratios: e6 4c e2 59 36b 2e7 345 3f2 leveling errors: r-- --g r-- -w- fifo misalignment: yes yes yes yes fifo rst errors: 0 1 0 0 <<do_sdram_init() 4c000000
do_sdram_init() 4d000000
omap5_ddr3_leveling(0x4d000000) cmd phy dlls: ok f9 ok fb data phy dlls: ok f7 ok fb ok fb ok f3 phy status 5-7: 0 99990924 924 read dqs ratios: 3b 3b 37 46 0 0 0 0 fifo we ratios: 6f 7b 7b 75 700 700 700 700 write data ratios: 78 71 7c 78 1b3 3eb 292 51 write dqs ratios: 58 51 5c 58 193 3cb 272 31 fifo misalignment: yes yes yes yes fifo rst errors: 1 2 2 2 cmd phy dlls: ok f9 ok fc data phy dlls: ok f7 ok fc ok fc ok f4 phy status 5-7: 0 99990924 924 read dqs ratios: 3b 3b 37 46 0 0 0 0 fifo we ratios: 6e 7a 7a 74 700 700 700 700 write data ratios: 75 70 79 77 1b3 3eb 292 51 write dqs ratios: 55 50 59 57 193 3cb 272 31 leveling errors: --- --- -w- --- fifo rst errors: 1 1 1 1 <<do_sdram_init() 4d000000 ### SDRAM: identified: 4 expected: 40000000 ###
U-Boot SPL 2016.11-00249-gcb460d33abe8 (Mar 07 2017 - 09:18:21) OMAP5432-GP ES2.0
sdram_init()
in_sdram = 0
do_sdram_init() 4c000000
emif_reset_phy(0x4c000000) omap5_ddr3_leveling(0x4c000000) cmd phy dlls: ok f6 ok f6 data phy dlls: ok f7 ok f3 ok fc ok f7 phy status 5-7: 0 99990924 924 read dqs ratios: 0 0 0 0 0 0 0 0 fifo we ratios: 700 700 700 700 700 700 700 700 write data ratios: 28e 1bb 2f 117 34b 2e7 345 332 write dqs ratios: 28e 1bb 2f 117 34b 2e7 345 332 cmd phy dlls: ok f7 ok f6 data phy dlls: ok f8 ok f3 ok fc ok f7 phy status 5-7: 0 99990924 924 read dqs ratios: 0 0 0 0 0 0 0 0 fifo we ratios: 700 700 700 700 700 700 700 700 write data ratios: 28e 1bb 2f 117 34b 2e7 345 332 write dqs ratios: 28e 1bb 2f 117 34b 2e7 345 332 <<do_sdram_init() 4c000000
do_sdram_init() 4d000000
emif_reset_phy(0x4d000000) omap5_ddr3_leveling(0x4d000000) cmd phy dlls: ok f9 ok fc data phy dlls: ok f6 ok fb ok fc ok f3 phy status 5-7: 0 99990924 924 read dqs ratios: 0 0 0 0 0 0 0 0 fifo we ratios: 700 700 700 700 700 700 700 700 write data ratios: e3 39b 276 2f 19b 3cb 262 39 write dqs ratios: e3 39b 276 2f 19b 3cb 262 39 cmd phy dlls: ok f9 ok fc data phy dlls: ok f7 ok fc ok fc ok f3 phy status 5-7: 0 99990924 924 read dqs ratios: 0 0 0 0 0 0 0 0 fifo we ratios: 700 700 700 700 700 700 700 700 write data ratios: e3 39b 276 2f 19b 3cb 262 39 write dqs ratios: e3 39b 276 2f 19b 3cb 262 39 <<do_sdram_init() 4d000000 ### SDRAM: identified: 4 expected: 40000000 ###
U-Boot SPL 2016.11-00249-gcb460d33abe8 (Mar 07 2017 - 09:18:21) OMAP5432-GP ES2.0
sdram_init()
in_sdram = 0
do_sdram_init() 4c000000
omap5_ddr3_leveling(0x4c000000) ddr3 write leveling timeout ddr3 read dqs gate training timeout cmd phy dlls: ok f7 ok f6 data phy dlls: ok f8 ok f4 ok fc ok f7 phy status 5-7: 0 99990820 924 read dqs ratios: 7f 41 7f 3c 0 0 0 0 fifo we ratios: 0 7d 0 74 700 700 700 700 write data ratios: 10a 6e 10a 79 26b 30f 365 12 write dqs ratios: ea 4e ea 59 24b 2ef 345 3f2 fifo misalignment: - yes - yes fifo rst errors: 15 2 15 1 cmd phy dlls: ok f6 ok f6 data phy dlls: ok f8 ok f4 ok fc ok f8 phy status 5-7: 0 99990924 924 read dqs ratios: 7e 41 7e 3d 0 0 0 0 fifo we ratios: 7ff 7c 7ff 73 700 700 700 700 write data ratios: 105 6d 103 78 26b 30f 365 12 write dqs ratios: e5 4d e3 58 24b 2ef 345 3f2 leveling errors: r-- -w- r-- -w- fifo misalignment: yes - yes - fifo rst errors: 0 1 0 1 <<do_sdram_init() 4c000000
do_sdram_init() 4d000000
omap5_ddr3_leveling(0x4d000000) cmd phy dlls: ok f9 ok fc data phy dlls: ok f7 ok fb ok fb ok f3 phy status 5-7: 0 99990924 924 read dqs ratios: 3b 3b 36 46 0 0 0 0 fifo we ratios: 70 7c 7c 77 700 700 700 700 write data ratios: 76 71 7b 77 1b3 2eb 292 51 write dqs ratios: 56 51 5b 57 193 2cb 272 31 fifo misalignment: yes yes yes yes fifo rst errors: 1 3 2 2 cmd phy dlls: ok f8 ok fc data phy dlls: ok f6 ok fb ok fb ok f3 phy status 5-7: 0 99990924 924 read dqs ratios: 3b 3b 37 46 0 0 0 0 fifo we ratios: 6f 7b 7b 76 700 700 700 700 write data ratios: 76 6f 78 77 1b3 2eb 292 51 write dqs ratios: 56 4f 58 57 193 2cb 272 31 leveling errors: -w- --- --- -w- fifo rst errors: 0 1 1 1 <<do_sdram_init() 4d000000 ### SDRAM: identified: 4 expected: 40000000 ###
U-Boot SPL 2016.11-00249-gcb460d33abe8 (Mar 07 2017 - 09:18:21) OMAP5432-GP ES2.0
sdram_init()
in_sdram = 0
do_sdram_init() 4c000000
omap5_ddr3_leveling(0x4c000000) ddr3 write leveling timeout ddr3 read dqs gate training timeout cmd phy dlls: ok f6 ok f6 data phy dlls: ok f9 ok f4 ok fc ok f7 phy status 5-7: c3 99990820 924 read dqs ratios: 7f 43 7f 3c 0 0 0 0 fifo we ratios: 0 7e 0 75 700 700 700 700 write data ratios: 10a 6e 10a 7a 26b 307 365 12 write dqs ratios: ea 4e ea 5a 24b 2e7 345 3f2 fifo misalignment: - yes - yes fifo rst errors: 15 3 15 1 cmd phy dlls: ok f6 ok f6 data phy dlls: ok f8 ok f4 ok fc ok f7 phy status 5-7: 0 99990924 924 read dqs ratios: 7e 43 7e 3c 0 0 0 0 fifo we ratios: 7ff 7d 7ff 74 700 700 700 700 write data ratios: 107 6d 103 77 26b 307 365 12 write dqs ratios: e7 4d e3 57 24b 2e7 345 3f2 leveling errors: r-- -w- r-- -w- fifo misalignment: yes - yes - fifo rst errors: 0 1 0 1 <<do_sdram_init() 4c000000
do_sdram_init() 4d000000
omap5_ddr3_leveling(0x4d000000) cmd phy dlls: ok f9 ok fc data phy dlls: ok f7 ok fb ok fb ok f4 phy status 5-7: 0 99990924 924 read dqs ratios: 3d 3b 38 46 0 0 0 0 fifo we ratios: 70 7b 7b 75 700 700 700 700 write data ratios: 78 72 7b 78 1a3 2eb 292 57 write dqs ratios: 58 52 5b 58 183 2cb 272 37 fifo misalignment: yes yes yes yes fifo rst errors: 1 1 2 2 cmd phy dlls: ok f9 ok fc data phy dlls: ok f7 ok fb ok fc ok f3 phy status 5-7: 0 99990924 924 read dqs ratios: 3d 3b 38 46 0 0 0 0 fifo we ratios: 6f 7a 7a 74 700 700 700 700 write data ratios: 76 70 79 76 1a3 2eb 292 57 write dqs ratios: 56 50 59 56 183 2cb 272 37 leveling errors: --- rw- rw- --- fifo rst errors: 0 1 0 0 <<do_sdram_init() 4d000000 ### SDRAM: identified: 4 expected: 40000000 ###
U-Boot SPL 2016.11-00249-gcb460d33abe8 (Mar 07 2017 - 09:18:21) OMAP5432-GP ES2.0
sdram_init()
in_sdram = 0
do_sdram_init() 4c000000
omap5_ddr3_leveling(0x4c000000) cmd phy dlls: ok f7 ok f6 data phy dlls: ok f9 ok f4 ok fc ok f8 phy status 5-7: 0 99990924 924 read dqs ratios: 3f 46 39 3b 0 0 0 0 fifo we ratios: 68 7c 74 74 700 700 700 700 write data ratios: 76 6e 6f 7a 28b 30f 365 12 write dqs ratios: 56 4e 4f 5a 26b 2ef 345 3f2 fifo misalignment: yes yes yes yes fifo rst errors: 1 1 1 2 cmd phy dlls: ok f7 ok f6 data phy dlls: ok f8 ok f4 ok fc ok f7 phy status 5-7: 0 99990924 924 read dqs ratios: 3f 46 39 3c 0 0 0 0 fifo we ratios: 67 7b 73 73 700 700 700 700 write data ratios: 74 6c 6d 78 28b 30f 365 12 write dqs ratios: 54 4c 4d 58 26b 2ef 345 3f2 fifo rst errors: 1 1 1 1 <<do_sdram_init() 4c000000
do_sdram_init() 4d000000
omap5_ddr3_leveling(0x4d000000) cmd phy dlls: ok f9 ok fc data phy dlls: ok f7 ok fb ok fc ok f3 phy status 5-7: 0 99990924 924 read dqs ratios: 3b 3b 37 46 0 0 0 0 fifo we ratios: 71 7c 7c 75 700 700 700 700 write data ratios: 78 71 7b 78 1bb 2eb 292 d1 write dqs ratios: 58 51 5b 58 19b 2cb 272 b1 fifo misalignment: yes yes yes yes fifo rst errors: 2 2 2 2 cmd phy dlls: ok f9 ok fc data phy dlls: ok f7 ok fc ok fc ok f3 phy status 5-7: 0 99990924 924 read dqs ratios: 3b 3b 37 46 0 0 0 0 fifo we ratios: 70 7b 7b 76 700 700 700 700 write data ratios: 75 6f 78 76 1bb 2eb 292 d1 write dqs ratios: 55 4f 58 56 19b 2cb 272 b1 leveling errors: --- --- --- -wg fifo misalignment: - - - yes fifo rst errors: 1 1 1 1 <<do_sdram_init() 4d000000 get_ram_size() successful <<sdram_init()
spl:board_init_r()
spl_init() Trying to boot from MMC1 Found LC15 V5.1 board_mmc_init for LC15 51 called gpio7 = eMMC gpio82 = 0 (mmc1=eMMC) gpio76 = 1 (ctrl=soft) spl: payload image: U-Boot 2016.11-00249-gcb460d33ab load addr: 0x807fffc0 size: 388720 Jumping to U-Boot loaded - jumping to U-Boot...
U-Boot 2016.11-00249-gcb460d33abe8 (Mar 07 2017 - 09:18:21 +0100)
CPU : ARM Cortex-A15 r2p2 SoC : OMAP5432-GP ES2.0 Model: Pyra-Handheld-V5.1 Board: Pyra with Letux Cortex 15 DRAM: >>sdram_init() 2 GiB Reset: peripherals tca642x@ 0x22 (24 pins): Bank 0 Configuration: iiiiiiii Polarity: 00000000 Input value: 00100000 Output value: 11111111 Bank 1 Configuration: iiiiiiii Polarity: 00000000 Input value: 01001000 Output value: 11111111 Bank 2 Configuration: iiiiiiii Polarity: 00000000 Input value: 00100010 Output value: 11111111 tca6424 init tca6424 initialized tca642x@ 0x22 (24 pins): Bank 0 Configuration: oooooooo Polarity: 00000000 Input value: 00000000 Output value: 00000000 Bank 1 Configuration: oiooiooo Polarity: 00000000 Input value: 01001000 Output value: 00000000 Bank 2 Configuration: ooooiiii Polarity: 00000000 Input value: 00000010 Output value: 00000000 MMC: OMAP SD/MMC: 0, OMAP SD/MMC: 1, OMAP SD/MMC: 2, OMAP SD/MMC: 3 *** Warning - bad CRC, using default environment
Found LC15 V5.1 Found Pyra MB V5.1 Device Tree: omap5-letux-cortex15-v5.1+pyra-v5.1.dtb SCSI: SATA link 0 timeout. AHCI 0001.0300 32 slots 1 ports 3 Gbps 0x1 impl SATA mode flags: 64bit ncq stag pm led clo only pmp pio slum part ccc apst scanning bus for devices... Found 0 device(s). Press SPACE to abort autoboot in 5 seconds =>
Matthijs
Hi, I have digged out an old mail describing OMAP5 temperature throttling experiments (using some 4.x kernel):
It looks as if throttling starts at 100°C and may go downwards by multiple OPPs (I had additional ones at 750, 500, 250 MHz).
Since TJ cools down quickly (<1 second) the clock rate goes up almost immediately to the highest OPP again.
I now think we do not need to do it differently for OMAP3, just different temperature.
So in total it may be simpler as we thought: * just add a 90° trip point (and 125°C "critcial" but no mapping) * map it to <&cpu THERMAL_NO_LIMIT THERMAL_NO_LIMIT> * if >90° it will go from OPP1G (state 3) -> OPP130 (state 2) * if this is not enough it will go OPP130 -> OPP100 (state 1) * if this is not enough it will go OPP100 -> OPP50 (state 0)
BR, Nikolaus
Anfang der weitergeleiteten Nachricht:
Von: "H. Nikolaus Schaller" hns@goldelico.com Betreff: 1.5GHz operation surprise Datum: 25. Februar 2017 um 17:28:48 MEZ An: kernel@pyra-handheld.com
Hi all, here is a very surprising result which needs independent confirmation.
root@letux:~# cpufreq-info cpufrequtils 008: cpufreq-info (C) Dominik Brodowski 2004-2009 Report errors and bugs to cpufreq@vger.kernel.org, please. analyzing CPU 0: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 250 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:10.27%, 500 MHz:37.39%, 750 MHz:9.76%, 1000 MHz:24.02%, 1.25 GHz:6.92%, 1.50 GHz:11.64% (60) analyzing CPU 1: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 250 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:10.29%, 500 MHz:37.38%, 750 MHz:9.76%, 1000 MHz:24.01%, 1.25 GHz:6.92%, 1.50 GHz:11.63% (60) root@letux:~# ./high-load -n 100% load stress test for 2 cores Sat Jan 1 04:52:17 UTC 2000 51° 52° 49° 36° 4002mV 1000MHz Sat Jan 1 04:52:17 UTC 2000 51° 52° 49° 36° 3991mV 1500MHz Sat Jan 1 04:52:18 UTC 2000 52° 53° 51° 36° 3854mV 1500MHz Sat Jan 1 04:52:20 UTC 2000 64° 54° 53° 36° 3876mV 1500MHz Sat Jan 1 04:52:21 UTC 2000 76° 62° 61° 36° 3855mV 1500MHz Sat Jan 1 04:52:23 UTC 2000 81° 69° 66° 37° 3852mV 1500MHz Sat Jan 1 04:52:24 UTC 2000 85° 71° 71° 37° 3832mV 1500MHz Sat Jan 1 04:52:26 UTC 2000 90° 75° 74° 37° 3829mV 1500MHz Sat Jan 1 04:52:27 UTC 2000 90° 75° 77° 37° 3824mV 1500MHz Sat Jan 1 04:52:29 UTC 2000 92° 78° 77° 38° 3843mV 1500MHz Sat Jan 1 04:52:30 UTC 2000 95° 81° 79° 38° 3813mV 1500MHz Sat Jan 1 04:52:31 UTC 2000 97° 84° 82° 38° 3825mV 1500MHz Sat Jan 1 04:52:33 UTC 2000 101° 86° 83° 38° 4021mV 500MHz Sat Jan 1 04:52:35 UTC 2000 102° 87° 84° 37° 4023mV 500MHz Sat Jan 1 04:52:37 UTC 2000 82° 84° 79° 37° 3848mV 1500MHz Sat Jan 1 04:52:38 UTC 2000 77° 79° 73° 37° 3813mV 1500MHz Sat Jan 1 04:52:40 UTC 2000 90° 78° 75° 37° 3836mV 1500MHz Sat Jan 1 04:52:41 UTC 2000 97° 84° 80° 38° 3822mV 1500MHz Sat Jan 1 04:52:43 UTC 2000 101° 86° 84° 38° 3964mV 750MHz Sat Jan 1 04:52:44 UTC 2000 104° 89° 86° 38° 4019mV 250MHz Sat Jan 1 04:52:47 UTC 2000 85° 86° 81° 37° 3820mV 1500MHz Sat Jan 1 04:52:48 UTC 2000 78° 81° 75° 37° 3832mV 1500MHz Sat Jan 1 04:52:50 UTC 2000 93° 81° 78° 37° 3797mV 1500MHz Sat Jan 1 04:52:51 UTC 2000 100° 86° 83° 37° 4029mV 250MHz Sat Jan 1 04:52:53 UTC 2000 104° 89° 86° 37° 3986mV 750MHz Sat Jan 1 04:52:55 UTC 2000 83° 84° 79° 37° 3815mV 1500MHz Sat Jan 1 04:52:57 UTC 2000 78° 80° 75° 37° 3824mV 1500MHz Sat Jan 1 04:52:58 UTC 2000 94° 82° 78° 37° 3799mV 1500MHz Sat Jan 1 04:53:00 UTC 2000 100° 87° 84° 39° 4056mV 500MHz Sat Jan 1 04:53:01 UTC 2000 105° 90° 87° 39° 4004mV 500MHz Sat Jan 1 04:53:04 UTC 2000 87° 88° 83° 39° 3816mV 1500MHz Sat Jan 1 04:53:05 UTC 2000 81° 83° 77° 38° 3797mV 1500MHz Sat Jan 1 04:53:07 UTC 2000 95° 83° 79° 38° 3824mV 1500MHz Sat Jan 1 04:53:08 UTC 2000 103° 88° 85° 38° 4106mV 250MHz ^Ckill 2653 2654
root@letux:~# cpufreq-info cpufrequtils 008: cpufreq-info (C) Dominik Brodowski 2004-2009 Report errors and bugs to cpufreq@vger.kernel.org, please. analyzing CPU 0: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 500 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:24.54%, 500 MHz:21.39%, 750 MHz:5.83%, 1000 MHz:11.23%, 1.25 GHz:4.45%, 1.50 GHz:32.56% (142) analyzing CPU 1: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 500 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:24.54%, 500 MHz:21.39%, 750 MHz:5.83%, 1000 MHz:11.23%, 1.25 GHz:4.45%, 1.50 GHz:32.56% (142) root@letux:~#
What I have changed was to apply Matthijs' impedance calibration patch
http://git.goldelico.com/?p=gta04-uboot.git;a=commit;h=918c4c46104714a3cfede...
and enabling the 1.5GHz OPP in DT. Plus recompile u-boot and kernel and then boot.
I always said it could be a software bug but nobody believed that it could be one. And I never believed that changing 1 bit in U-Boot could be sufficient (I thought we need to patch the kernel).
A realistic explanation is that our DDR3 is a little more sensitive to noise by 1.5GHz operation than the EVM and hence we get the problems but don't see them on the EVM if it is more robust.
At least my prototype was now running for 5 minutes @ 200% load @ 1.5GHz (cycling between 1.5GHz and 250MHz and 80°C and 110°C with a period of ca. 10 seconds) until I got
thermal thermal_zone0: critical temperature reached(125 C),shutting down
We should evaluate this further!
BR and please cross-check, Nikolaus
Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
Am 13.09.2019 um 22:40 schrieb H. Nikolaus Schaller hns@goldelico.com:
Hi, I have digged out an old mail describing OMAP5 temperature throttling experiments (using some 4.x kernel):
It looks as if throttling starts at 100°C and may go downwards by multiple OPPs (I had additional ones at 750, 500, 250 MHz).
Since TJ cools down quickly (<1 second) the clock rate goes up almost immediately to the highest OPP again.
I now think we do not need to do it differently for OMAP3, just different temperature.
So in total it may be simpler as we thought:
- just add a 90° trip point (and 125°C "critcial" but no mapping)
- map it to <&cpu THERMAL_NO_LIMIT THERMAL_NO_LIMIT>
- if >90° it will go from OPP1G (state 3) -> OPP130 (state 2)
- if this is not enough it will go OPP130 -> OPP100 (state 1)
- if this is not enough it will go OPP100 -> OPP50 (state 0)
So we just need simlar to omap4-cpu-thermal.dtsi:
cpu_thermal: cpu_thermal { polling-delay-passive = <250>; /* milliseconds */ polling-delay = <1000>; /* milliseconds */
/* sensor ID */ thermal-sensors = <&bandgap 0>;
cpu_trips: trips { cpu_alert0: cpu_alert { temperature = <90000>; /* millicelsius */ hysteresis = <2000>; /* millicelsius */ type = "passive"; }; cpu_crit: cpu_crit { temperature = <105000>; /* millicelsius */ hysteresis = <2000>; /* millicelsius */ type = "critical"; }; };
cpu_cooling_maps: cooling-maps { map0 { trip = <&cpu_alert0>; cooling-device = <&cpu0 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>; }; }; };
And contrary to my thinking that we should do that only for omap36xx we can simply do it for any OMAP3. This makes sure that none overheats...
BR, Nikolaus
BR, Nikolaus
Anfang der weitergeleiteten Nachricht:
Von: "H. Nikolaus Schaller" hns@goldelico.com Betreff: 1.5GHz operation surprise Datum: 25. Februar 2017 um 17:28:48 MEZ An: kernel@pyra-handheld.com
Hi all, here is a very surprising result which needs independent confirmation.
root@letux:~# cpufreq-info cpufrequtils 008: cpufreq-info (C) Dominik Brodowski 2004-2009 Report errors and bugs to cpufreq@vger.kernel.org, please. analyzing CPU 0: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 250 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:10.27%, 500 MHz:37.39%, 750 MHz:9.76%, 1000 MHz:24.02%, 1.25 GHz:6.92%, 1.50 GHz:11.64% (60) analyzing CPU 1: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 250 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:10.29%, 500 MHz:37.38%, 750 MHz:9.76%, 1000 MHz:24.01%, 1.25 GHz:6.92%, 1.50 GHz:11.63% (60) root@letux:~# ./high-load -n 100% load stress test for 2 cores Sat Jan 1 04:52:17 UTC 2000 51° 52° 49° 36° 4002mV 1000MHz Sat Jan 1 04:52:17 UTC 2000 51° 52° 49° 36° 3991mV 1500MHz Sat Jan 1 04:52:18 UTC 2000 52° 53° 51° 36° 3854mV 1500MHz Sat Jan 1 04:52:20 UTC 2000 64° 54° 53° 36° 3876mV 1500MHz Sat Jan 1 04:52:21 UTC 2000 76° 62° 61° 36° 3855mV 1500MHz Sat Jan 1 04:52:23 UTC 2000 81° 69° 66° 37° 3852mV 1500MHz Sat Jan 1 04:52:24 UTC 2000 85° 71° 71° 37° 3832mV 1500MHz Sat Jan 1 04:52:26 UTC 2000 90° 75° 74° 37° 3829mV 1500MHz Sat Jan 1 04:52:27 UTC 2000 90° 75° 77° 37° 3824mV 1500MHz Sat Jan 1 04:52:29 UTC 2000 92° 78° 77° 38° 3843mV 1500MHz Sat Jan 1 04:52:30 UTC 2000 95° 81° 79° 38° 3813mV 1500MHz Sat Jan 1 04:52:31 UTC 2000 97° 84° 82° 38° 3825mV 1500MHz Sat Jan 1 04:52:33 UTC 2000 101° 86° 83° 38° 4021mV 500MHz Sat Jan 1 04:52:35 UTC 2000 102° 87° 84° 37° 4023mV 500MHz Sat Jan 1 04:52:37 UTC 2000 82° 84° 79° 37° 3848mV 1500MHz Sat Jan 1 04:52:38 UTC 2000 77° 79° 73° 37° 3813mV 1500MHz Sat Jan 1 04:52:40 UTC 2000 90° 78° 75° 37° 3836mV 1500MHz Sat Jan 1 04:52:41 UTC 2000 97° 84° 80° 38° 3822mV 1500MHz Sat Jan 1 04:52:43 UTC 2000 101° 86° 84° 38° 3964mV 750MHz Sat Jan 1 04:52:44 UTC 2000 104° 89° 86° 38° 4019mV 250MHz Sat Jan 1 04:52:47 UTC 2000 85° 86° 81° 37° 3820mV 1500MHz Sat Jan 1 04:52:48 UTC 2000 78° 81° 75° 37° 3832mV 1500MHz Sat Jan 1 04:52:50 UTC 2000 93° 81° 78° 37° 3797mV 1500MHz Sat Jan 1 04:52:51 UTC 2000 100° 86° 83° 37° 4029mV 250MHz Sat Jan 1 04:52:53 UTC 2000 104° 89° 86° 37° 3986mV 750MHz Sat Jan 1 04:52:55 UTC 2000 83° 84° 79° 37° 3815mV 1500MHz Sat Jan 1 04:52:57 UTC 2000 78° 80° 75° 37° 3824mV 1500MHz Sat Jan 1 04:52:58 UTC 2000 94° 82° 78° 37° 3799mV 1500MHz Sat Jan 1 04:53:00 UTC 2000 100° 87° 84° 39° 4056mV 500MHz Sat Jan 1 04:53:01 UTC 2000 105° 90° 87° 39° 4004mV 500MHz Sat Jan 1 04:53:04 UTC 2000 87° 88° 83° 39° 3816mV 1500MHz Sat Jan 1 04:53:05 UTC 2000 81° 83° 77° 38° 3797mV 1500MHz Sat Jan 1 04:53:07 UTC 2000 95° 83° 79° 38° 3824mV 1500MHz Sat Jan 1 04:53:08 UTC 2000 103° 88° 85° 38° 4106mV 250MHz ^Ckill 2653 2654
root@letux:~# cpufreq-info cpufrequtils 008: cpufreq-info (C) Dominik Brodowski 2004-2009 Report errors and bugs to cpufreq@vger.kernel.org, please. analyzing CPU 0: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 500 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:24.54%, 500 MHz:21.39%, 750 MHz:5.83%, 1000 MHz:11.23%, 1.25 GHz:4.45%, 1.50 GHz:32.56% (142) analyzing CPU 1: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 500 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:24.54%, 500 MHz:21.39%, 750 MHz:5.83%, 1000 MHz:11.23%, 1.25 GHz:4.45%, 1.50 GHz:32.56% (142) root@letux:~#
What I have changed was to apply Matthijs' impedance calibration patch
http://git.goldelico.com/?p=gta04-uboot.git;a=commit;h=918c4c46104714a3cfede...
and enabling the 1.5GHz OPP in DT. Plus recompile u-boot and kernel and then boot.
I always said it could be a software bug but nobody believed that it could be one. And I never believed that changing 1 bit in U-Boot could be sufficient (I thought we need to patch the kernel).
A realistic explanation is that our DDR3 is a little more sensitive to noise by 1.5GHz operation than the EVM and hence we get the problems but don't see them on the EVM if it is more robust.
At least my prototype was now running for 5 minutes @ 200% load @ 1.5GHz (cycling between 1.5GHz and 250MHz and 80°C and 110°C with a period of ca. 10 seconds) until I got
thermal thermal_zone0: critical temperature reached(125 C),shutting down
We should evaluate this further!
BR and please cross-check, Nikolaus
Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
Am 13.09.2019 um 22:43 schrieb H. Nikolaus Schaller hns@goldelico.com:
Am 13.09.2019 um 22:40 schrieb H. Nikolaus Schaller hns@goldelico.com:
Hi, I have digged out an old mail describing OMAP5 temperature throttling experiments (using some 4.x kernel):
It looks as if throttling starts at 100°C and may go downwards by multiple OPPs (I had additional ones at 750, 500, 250 MHz).
Since TJ cools down quickly (<1 second) the clock rate goes up almost immediately to the highest OPP again.
I now think we do not need to do it differently for OMAP3, just different temperature.
So in total it may be simpler as we thought:
- just add a 90° trip point (and 125°C "critcial" but no mapping)
- map it to <&cpu THERMAL_NO_LIMIT THERMAL_NO_LIMIT>
- if >90° it will go from OPP1G (state 3) -> OPP130 (state 2)
- if this is not enough it will go OPP130 -> OPP100 (state 1)
- if this is not enough it will go OPP100 -> OPP50 (state 0)
So we just need simlar to omap4-cpu-thermal.dtsi:
cpu_thermal: cpu_thermal { polling-delay-passive = <250>; /* milliseconds */ polling-delay = <1000>; /* milliseconds */
/* sensor ID */ thermal-sensors = <&bandgap 0>;cpu_trips: trips { cpu_alert0: cpu_alert { temperature = <90000>; /* millicelsius */ hysteresis = <2000>; /* millicelsius */ type = "passive"; }; cpu_crit: cpu_crit { temperature = <105000>; /* millicelsius */ hysteresis = <2000>; /* millicelsius */ type = "critical"; }; };
cpu_cooling_maps: cooling-maps { map0 { trip = <&cpu_alert0>; cooling-device = <&cpu0 THERMAL_NO_LIMIT THERMAL_NO_LIMIT>; }; }; };
And contrary to my thinking that we should do that only for omap36xx we can simply do it for any OMAP3. This makes sure that none overheats...
Well, no OMAP3 will then use the not explicitly forbidden temperature range of 90° .. 105° at lower OPPs. But IMHO this is unlikely than anyone is missing such system operation states.
BR, Nikolaus
BR, Nikolaus
Anfang der weitergeleiteten Nachricht:
Von: "H. Nikolaus Schaller" hns@goldelico.com Betreff: 1.5GHz operation surprise Datum: 25. Februar 2017 um 17:28:48 MEZ An: kernel@pyra-handheld.com
Hi all, here is a very surprising result which needs independent confirmation.
root@letux:~# cpufreq-info cpufrequtils 008: cpufreq-info (C) Dominik Brodowski 2004-2009 Report errors and bugs to cpufreq@vger.kernel.org, please. analyzing CPU 0: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 250 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:10.27%, 500 MHz:37.39%, 750 MHz:9.76%, 1000 MHz:24.02%, 1.25 GHz:6.92%, 1.50 GHz:11.64% (60) analyzing CPU 1: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 250 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:10.29%, 500 MHz:37.38%, 750 MHz:9.76%, 1000 MHz:24.01%, 1.25 GHz:6.92%, 1.50 GHz:11.63% (60) root@letux:~# ./high-load -n 100% load stress test for 2 cores Sat Jan 1 04:52:17 UTC 2000 51° 52° 49° 36° 4002mV 1000MHz Sat Jan 1 04:52:17 UTC 2000 51° 52° 49° 36° 3991mV 1500MHz Sat Jan 1 04:52:18 UTC 2000 52° 53° 51° 36° 3854mV 1500MHz Sat Jan 1 04:52:20 UTC 2000 64° 54° 53° 36° 3876mV 1500MHz Sat Jan 1 04:52:21 UTC 2000 76° 62° 61° 36° 3855mV 1500MHz Sat Jan 1 04:52:23 UTC 2000 81° 69° 66° 37° 3852mV 1500MHz Sat Jan 1 04:52:24 UTC 2000 85° 71° 71° 37° 3832mV 1500MHz Sat Jan 1 04:52:26 UTC 2000 90° 75° 74° 37° 3829mV 1500MHz Sat Jan 1 04:52:27 UTC 2000 90° 75° 77° 37° 3824mV 1500MHz Sat Jan 1 04:52:29 UTC 2000 92° 78° 77° 38° 3843mV 1500MHz Sat Jan 1 04:52:30 UTC 2000 95° 81° 79° 38° 3813mV 1500MHz Sat Jan 1 04:52:31 UTC 2000 97° 84° 82° 38° 3825mV 1500MHz Sat Jan 1 04:52:33 UTC 2000 101° 86° 83° 38° 4021mV 500MHz Sat Jan 1 04:52:35 UTC 2000 102° 87° 84° 37° 4023mV 500MHz Sat Jan 1 04:52:37 UTC 2000 82° 84° 79° 37° 3848mV 1500MHz Sat Jan 1 04:52:38 UTC 2000 77° 79° 73° 37° 3813mV 1500MHz Sat Jan 1 04:52:40 UTC 2000 90° 78° 75° 37° 3836mV 1500MHz Sat Jan 1 04:52:41 UTC 2000 97° 84° 80° 38° 3822mV 1500MHz Sat Jan 1 04:52:43 UTC 2000 101° 86° 84° 38° 3964mV 750MHz Sat Jan 1 04:52:44 UTC 2000 104° 89° 86° 38° 4019mV 250MHz Sat Jan 1 04:52:47 UTC 2000 85° 86° 81° 37° 3820mV 1500MHz Sat Jan 1 04:52:48 UTC 2000 78° 81° 75° 37° 3832mV 1500MHz Sat Jan 1 04:52:50 UTC 2000 93° 81° 78° 37° 3797mV 1500MHz Sat Jan 1 04:52:51 UTC 2000 100° 86° 83° 37° 4029mV 250MHz Sat Jan 1 04:52:53 UTC 2000 104° 89° 86° 37° 3986mV 750MHz Sat Jan 1 04:52:55 UTC 2000 83° 84° 79° 37° 3815mV 1500MHz Sat Jan 1 04:52:57 UTC 2000 78° 80° 75° 37° 3824mV 1500MHz Sat Jan 1 04:52:58 UTC 2000 94° 82° 78° 37° 3799mV 1500MHz Sat Jan 1 04:53:00 UTC 2000 100° 87° 84° 39° 4056mV 500MHz Sat Jan 1 04:53:01 UTC 2000 105° 90° 87° 39° 4004mV 500MHz Sat Jan 1 04:53:04 UTC 2000 87° 88° 83° 39° 3816mV 1500MHz Sat Jan 1 04:53:05 UTC 2000 81° 83° 77° 38° 3797mV 1500MHz Sat Jan 1 04:53:07 UTC 2000 95° 83° 79° 38° 3824mV 1500MHz Sat Jan 1 04:53:08 UTC 2000 103° 88° 85° 38° 4106mV 250MHz ^Ckill 2653 2654
root@letux:~# cpufreq-info cpufrequtils 008: cpufreq-info (C) Dominik Brodowski 2004-2009 Report errors and bugs to cpufreq@vger.kernel.org, please. analyzing CPU 0: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 500 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:24.54%, 500 MHz:21.39%, 750 MHz:5.83%, 1000 MHz:11.23%, 1.25 GHz:4.45%, 1.50 GHz:32.56% (142) analyzing CPU 1: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 500 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:24.54%, 500 MHz:21.39%, 750 MHz:5.83%, 1000 MHz:11.23%, 1.25 GHz:4.45%, 1.50 GHz:32.56% (142) root@letux:~#
What I have changed was to apply Matthijs' impedance calibration patch
http://git.goldelico.com/?p=gta04-uboot.git;a=commit;h=918c4c46104714a3cfede...
and enabling the 1.5GHz OPP in DT. Plus recompile u-boot and kernel and then boot.
I always said it could be a software bug but nobody believed that it could be one. And I never believed that changing 1 bit in U-Boot could be sufficient (I thought we need to patch the kernel).
A realistic explanation is that our DDR3 is a little more sensitive to noise by 1.5GHz operation than the EVM and hence we get the problems but don't see them on the EVM if it is more robust.
At least my prototype was now running for 5 minutes @ 200% load @ 1.5GHz (cycling between 1.5GHz and 250MHz and 80°C and 110°C with a period of ca. 10 seconds) until I got
thermal thermal_zone0: critical temperature reached(125 C),shutting down
We should evaluate this further!
BR and please cross-check, Nikolaus
Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
On Fri, Sep 13, 2019 at 3:40 PM H. Nikolaus Schaller hns@goldelico.com wrote:
Hi, I have digged out an old mail describing OMAP5 temperature throttling experiments (using some 4.x kernel):
It looks as if throttling starts at 100°C and may go downwards by multiple OPPs (I had additional ones at 750, 500, 250 MHz).
Since TJ cools down quickly (<1 second) the clock rate goes up almost immediately to the highest OPP again.
I now think we do not need to do it differently for OMAP3, just different temperature.
So in total it may be simpler as we thought:
- just add a 90° trip point (and 125°C "critcial" but no mapping)
- map it to <&cpu THERMAL_NO_LIMIT THERMAL_NO_LIMIT>
- if >90° it will go from OPP1G (state 3) -> OPP130 (state 2)
- if this is not enough it will go OPP130 -> OPP100 (state 1)
- if this is not enough it will go OPP100 -> OPP50 (state 0)
From what I can tell, this patch does this:
https://patchwork.kernel.org/patch/11144941/
The 2nd patch in the series associates it to all omap3 boards (34xx, 36xx, and 3517) https://patchwork.kernel.org/patch/11144939/
If there is anything you think that should be done differently, please let me know.
adam
BR, Nikolaus
Anfang der weitergeleiteten Nachricht:
Von: "H. Nikolaus Schaller" hns@goldelico.com Betreff: 1.5GHz operation surprise Datum: 25. Februar 2017 um 17:28:48 MEZ An: kernel@pyra-handheld.com
Hi all, here is a very surprising result which needs independent confirmation.
root@letux:~# cpufreq-info cpufrequtils 008: cpufreq-info (C) Dominik Brodowski 2004-2009 Report errors and bugs to cpufreq@vger.kernel.org, please. analyzing CPU 0: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 250 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:10.27%, 500 MHz:37.39%, 750 MHz:9.76%, 1000 MHz:24.02%, 1.25 GHz:6.92%, 1.50 GHz:11.64% (60) analyzing CPU 1: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 250 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:10.29%, 500 MHz:37.38%, 750 MHz:9.76%, 1000 MHz:24.01%, 1.25 GHz:6.92%, 1.50 GHz:11.63% (60) root@letux:~# ./high-load -n 100% load stress test for 2 cores Sat Jan 1 04:52:17 UTC 2000 51° 52° 49° 36° 4002mV 1000MHz Sat Jan 1 04:52:17 UTC 2000 51° 52° 49° 36° 3991mV 1500MHz Sat Jan 1 04:52:18 UTC 2000 52° 53° 51° 36° 3854mV 1500MHz Sat Jan 1 04:52:20 UTC 2000 64° 54° 53° 36° 3876mV 1500MHz Sat Jan 1 04:52:21 UTC 2000 76° 62° 61° 36° 3855mV 1500MHz Sat Jan 1 04:52:23 UTC 2000 81° 69° 66° 37° 3852mV 1500MHz Sat Jan 1 04:52:24 UTC 2000 85° 71° 71° 37° 3832mV 1500MHz Sat Jan 1 04:52:26 UTC 2000 90° 75° 74° 37° 3829mV 1500MHz Sat Jan 1 04:52:27 UTC 2000 90° 75° 77° 37° 3824mV 1500MHz Sat Jan 1 04:52:29 UTC 2000 92° 78° 77° 38° 3843mV 1500MHz Sat Jan 1 04:52:30 UTC 2000 95° 81° 79° 38° 3813mV 1500MHz Sat Jan 1 04:52:31 UTC 2000 97° 84° 82° 38° 3825mV 1500MHz Sat Jan 1 04:52:33 UTC 2000 101° 86° 83° 38° 4021mV 500MHz Sat Jan 1 04:52:35 UTC 2000 102° 87° 84° 37° 4023mV 500MHz Sat Jan 1 04:52:37 UTC 2000 82° 84° 79° 37° 3848mV 1500MHz Sat Jan 1 04:52:38 UTC 2000 77° 79° 73° 37° 3813mV 1500MHz Sat Jan 1 04:52:40 UTC 2000 90° 78° 75° 37° 3836mV 1500MHz Sat Jan 1 04:52:41 UTC 2000 97° 84° 80° 38° 3822mV 1500MHz Sat Jan 1 04:52:43 UTC 2000 101° 86° 84° 38° 3964mV 750MHz Sat Jan 1 04:52:44 UTC 2000 104° 89° 86° 38° 4019mV 250MHz Sat Jan 1 04:52:47 UTC 2000 85° 86° 81° 37° 3820mV 1500MHz Sat Jan 1 04:52:48 UTC 2000 78° 81° 75° 37° 3832mV 1500MHz Sat Jan 1 04:52:50 UTC 2000 93° 81° 78° 37° 3797mV 1500MHz Sat Jan 1 04:52:51 UTC 2000 100° 86° 83° 37° 4029mV 250MHz Sat Jan 1 04:52:53 UTC 2000 104° 89° 86° 37° 3986mV 750MHz Sat Jan 1 04:52:55 UTC 2000 83° 84° 79° 37° 3815mV 1500MHz Sat Jan 1 04:52:57 UTC 2000 78° 80° 75° 37° 3824mV 1500MHz Sat Jan 1 04:52:58 UTC 2000 94° 82° 78° 37° 3799mV 1500MHz Sat Jan 1 04:53:00 UTC 2000 100° 87° 84° 39° 4056mV 500MHz Sat Jan 1 04:53:01 UTC 2000 105° 90° 87° 39° 4004mV 500MHz Sat Jan 1 04:53:04 UTC 2000 87° 88° 83° 39° 3816mV 1500MHz Sat Jan 1 04:53:05 UTC 2000 81° 83° 77° 38° 3797mV 1500MHz Sat Jan 1 04:53:07 UTC 2000 95° 83° 79° 38° 3824mV 1500MHz Sat Jan 1 04:53:08 UTC 2000 103° 88° 85° 38° 4106mV 250MHz ^Ckill 2653 2654
root@letux:~# cpufreq-info cpufrequtils 008: cpufreq-info (C) Dominik Brodowski 2004-2009 Report errors and bugs to cpufreq@vger.kernel.org, please. analyzing CPU 0: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 500 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:24.54%, 500 MHz:21.39%, 750 MHz:5.83%, 1000 MHz:11.23%, 1.25 GHz:4.45%, 1.50 GHz:32.56% (142) analyzing CPU 1: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 500 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:24.54%, 500 MHz:21.39%, 750 MHz:5.83%, 1000 MHz:11.23%, 1.25 GHz:4.45%, 1.50 GHz:32.56% (142) root@letux:~#
What I have changed was to apply Matthijs' impedance calibration patch
http://git.goldelico.com/?p=gta04-uboot.git;a=commit;h=918c4c46104714a3cfede...
and enabling the 1.5GHz OPP in DT. Plus recompile u-boot and kernel and then boot.
I always said it could be a software bug but nobody believed that it could be one. And I never believed that changing 1 bit in U-Boot could be sufficient (I thought we need to patch the kernel).
A realistic explanation is that our DDR3 is a little more sensitive to noise by 1.5GHz operation than the EVM and hence we get the problems but don't see them on the EVM if it is more robust.
At least my prototype was now running for 5 minutes @ 200% load @ 1.5GHz (cycling between 1.5GHz and 250MHz and 80°C and 110°C with a period of ca. 10 seconds) until I got
thermal thermal_zone0: critical temperature reached(125 C),shutting down
We should evaluate this further!
BR and please cross-check, Nikolaus
Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
Am 13.09.2019 um 22:50 schrieb Adam Ford aford173@gmail.com:
On Fri, Sep 13, 2019 at 3:40 PM H. Nikolaus Schaller hns@goldelico.com wrote:
Hi, I have digged out an old mail describing OMAP5 temperature throttling experiments (using some 4.x kernel):
It looks as if throttling starts at 100°C and may go downwards by multiple OPPs (I had additional ones at 750, 500, 250 MHz).
Since TJ cools down quickly (<1 second) the clock rate goes up almost immediately to the highest OPP again.
I now think we do not need to do it differently for OMAP3, just different temperature.
So in total it may be simpler as we thought:
- just add a 90° trip point (and 125°C "critcial" but no mapping)
- map it to <&cpu THERMAL_NO_LIMIT THERMAL_NO_LIMIT>
- if >90° it will go from OPP1G (state 3) -> OPP130 (state 2)
- if this is not enough it will go OPP130 -> OPP100 (state 1)
- if this is not enough it will go OPP100 -> OPP50 (state 0)
From what I can tell, this patch does this:
https://patchwork.kernel.org/patch/11144941/
The 2nd patch in the series associates it to all omap3 boards (34xx, 36xx, and 3517) https://patchwork.kernel.org/patch/11144939/
If there is anything you think that should be done differently, please let me know.
Ah, yes. The mails arrived in different order...
This patches above came before I received the "[RFC v2] ARM: dts: omap36xx: Enable thermal throttling" with cooling-device = <&cpu 1 2>;
So in my current evaluation we simply can skip that omap36xx specific patch, because the general omap3 patch should already do it right...
In other words, I think we are done :)
BR and thanks, Nikolaus
adam
BR, Nikolaus
Anfang der weitergeleiteten Nachricht:
Von: "H. Nikolaus Schaller" hns@goldelico.com Betreff: 1.5GHz operation surprise Datum: 25. Februar 2017 um 17:28:48 MEZ An: kernel@pyra-handheld.com
Hi all, here is a very surprising result which needs independent confirmation.
root@letux:~# cpufreq-info cpufrequtils 008: cpufreq-info (C) Dominik Brodowski 2004-2009 Report errors and bugs to cpufreq@vger.kernel.org, please. analyzing CPU 0: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 250 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:10.27%, 500 MHz:37.39%, 750 MHz:9.76%, 1000 MHz:24.02%, 1.25 GHz:6.92%, 1.50 GHz:11.64% (60) analyzing CPU 1: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 250 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:10.29%, 500 MHz:37.38%, 750 MHz:9.76%, 1000 MHz:24.01%, 1.25 GHz:6.92%, 1.50 GHz:11.63% (60) root@letux:~# ./high-load -n 100% load stress test for 2 cores Sat Jan 1 04:52:17 UTC 2000 51° 52° 49° 36° 4002mV 1000MHz Sat Jan 1 04:52:17 UTC 2000 51° 52° 49° 36° 3991mV 1500MHz Sat Jan 1 04:52:18 UTC 2000 52° 53° 51° 36° 3854mV 1500MHz Sat Jan 1 04:52:20 UTC 2000 64° 54° 53° 36° 3876mV 1500MHz Sat Jan 1 04:52:21 UTC 2000 76° 62° 61° 36° 3855mV 1500MHz Sat Jan 1 04:52:23 UTC 2000 81° 69° 66° 37° 3852mV 1500MHz Sat Jan 1 04:52:24 UTC 2000 85° 71° 71° 37° 3832mV 1500MHz Sat Jan 1 04:52:26 UTC 2000 90° 75° 74° 37° 3829mV 1500MHz Sat Jan 1 04:52:27 UTC 2000 90° 75° 77° 37° 3824mV 1500MHz Sat Jan 1 04:52:29 UTC 2000 92° 78° 77° 38° 3843mV 1500MHz Sat Jan 1 04:52:30 UTC 2000 95° 81° 79° 38° 3813mV 1500MHz Sat Jan 1 04:52:31 UTC 2000 97° 84° 82° 38° 3825mV 1500MHz Sat Jan 1 04:52:33 UTC 2000 101° 86° 83° 38° 4021mV 500MHz Sat Jan 1 04:52:35 UTC 2000 102° 87° 84° 37° 4023mV 500MHz Sat Jan 1 04:52:37 UTC 2000 82° 84° 79° 37° 3848mV 1500MHz Sat Jan 1 04:52:38 UTC 2000 77° 79° 73° 37° 3813mV 1500MHz Sat Jan 1 04:52:40 UTC 2000 90° 78° 75° 37° 3836mV 1500MHz Sat Jan 1 04:52:41 UTC 2000 97° 84° 80° 38° 3822mV 1500MHz Sat Jan 1 04:52:43 UTC 2000 101° 86° 84° 38° 3964mV 750MHz Sat Jan 1 04:52:44 UTC 2000 104° 89° 86° 38° 4019mV 250MHz Sat Jan 1 04:52:47 UTC 2000 85° 86° 81° 37° 3820mV 1500MHz Sat Jan 1 04:52:48 UTC 2000 78° 81° 75° 37° 3832mV 1500MHz Sat Jan 1 04:52:50 UTC 2000 93° 81° 78° 37° 3797mV 1500MHz Sat Jan 1 04:52:51 UTC 2000 100° 86° 83° 37° 4029mV 250MHz Sat Jan 1 04:52:53 UTC 2000 104° 89° 86° 37° 3986mV 750MHz Sat Jan 1 04:52:55 UTC 2000 83° 84° 79° 37° 3815mV 1500MHz Sat Jan 1 04:52:57 UTC 2000 78° 80° 75° 37° 3824mV 1500MHz Sat Jan 1 04:52:58 UTC 2000 94° 82° 78° 37° 3799mV 1500MHz Sat Jan 1 04:53:00 UTC 2000 100° 87° 84° 39° 4056mV 500MHz Sat Jan 1 04:53:01 UTC 2000 105° 90° 87° 39° 4004mV 500MHz Sat Jan 1 04:53:04 UTC 2000 87° 88° 83° 39° 3816mV 1500MHz Sat Jan 1 04:53:05 UTC 2000 81° 83° 77° 38° 3797mV 1500MHz Sat Jan 1 04:53:07 UTC 2000 95° 83° 79° 38° 3824mV 1500MHz Sat Jan 1 04:53:08 UTC 2000 103° 88° 85° 38° 4106mV 250MHz ^Ckill 2653 2654
root@letux:~# cpufreq-info cpufrequtils 008: cpufreq-info (C) Dominik Brodowski 2004-2009 Report errors and bugs to cpufreq@vger.kernel.org, please. analyzing CPU 0: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 500 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:24.54%, 500 MHz:21.39%, 750 MHz:5.83%, 1000 MHz:11.23%, 1.25 GHz:4.45%, 1.50 GHz:32.56% (142) analyzing CPU 1: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 500 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:24.54%, 500 MHz:21.39%, 750 MHz:5.83%, 1000 MHz:11.23%, 1.25 GHz:4.45%, 1.50 GHz:32.56% (142) root@letux:~#
What I have changed was to apply Matthijs' impedance calibration patch
http://git.goldelico.com/?p=gta04-uboot.git;a=commit;h=918c4c46104714a3cfede...
and enabling the 1.5GHz OPP in DT. Plus recompile u-boot and kernel and then boot.
I always said it could be a software bug but nobody believed that it could be one. And I never believed that changing 1 bit in U-Boot could be sufficient (I thought we need to patch the kernel).
A realistic explanation is that our DDR3 is a little more sensitive to noise by 1.5GHz operation than the EVM and hence we get the problems but don't see them on the EVM if it is more robust.
At least my prototype was now running for 5 minutes @ 200% load @ 1.5GHz (cycling between 1.5GHz and 250MHz and 80°C and 110°C with a period of ca. 10 seconds) until I got
thermal thermal_zone0: critical temperature reached(125 C),shutting down
We should evaluate this further!
BR and please cross-check, Nikolaus
Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
On Fri, Sep 13, 2019 at 4:01 PM H. Nikolaus Schaller hns@goldelico.com wrote:
Am 13.09.2019 um 22:50 schrieb Adam Ford aford173@gmail.com:
On Fri, Sep 13, 2019 at 3:40 PM H. Nikolaus Schaller hns@goldelico.com wrote:
Hi, I have digged out an old mail describing OMAP5 temperature throttling experiments (using some 4.x kernel):
It looks as if throttling starts at 100°C and may go downwards by multiple OPPs (I had additional ones at 750, 500, 250 MHz).
Since TJ cools down quickly (<1 second) the clock rate goes up almost immediately to the highest OPP again.
I now think we do not need to do it differently for OMAP3, just different temperature.
So in total it may be simpler as we thought:
- just add a 90° trip point (and 125°C "critcial" but no mapping)
- map it to <&cpu THERMAL_NO_LIMIT THERMAL_NO_LIMIT>
- if >90° it will go from OPP1G (state 3) -> OPP130 (state 2)
- if this is not enough it will go OPP130 -> OPP100 (state 1)
- if this is not enough it will go OPP100 -> OPP50 (state 0)
From what I can tell, this patch does this:
https://patchwork.kernel.org/patch/11144941/
The 2nd patch in the series associates it to all omap3 boards (34xx, 36xx, and 3517) https://patchwork.kernel.org/patch/11144939/
If there is anything you think that should be done differently, please let me know.
Ah, yes. The mails arrived in different order...
This patches above came before I received the "[RFC v2] ARM: dts: omap36xx: Enable thermal throttling" with cooling-device = <&cpu 1 2>;
So in my current evaluation we simply can skip that omap36xx specific patch, because the general omap3 patch should already do it right...
In other words, I think we are done :)
Awesome! I will wait for Tony and/or any TI people to make any feedback on the accuracy of the temperature sensor. I am also concerned we may get some push-back because when I update the series, I'll have to explicitly update the defconfig to enable the temperature sensor, and sometimes people get uncomfortable with messages like "This OMAP thermal sensor is unreliable. You've been warned"
If you can reply to the RFC with some feedback like tested-by or review-by, etc. I'll add your name to the 'suggested-by' and whatever other feedback you give, because you inspired me to do this.
Now, if we can get the Smartreflex patches that Andre did accepted by Tony, we should be able to reduce the power consumption which should also help with heat.
adam
BR and thanks, Nikolaus
adam
BR, Nikolaus
Anfang der weitergeleiteten Nachricht:
Von: "H. Nikolaus Schaller" hns@goldelico.com Betreff: 1.5GHz operation surprise Datum: 25. Februar 2017 um 17:28:48 MEZ An: kernel@pyra-handheld.com
Hi all, here is a very surprising result which needs independent confirmation.
root@letux:~# cpufreq-info cpufrequtils 008: cpufreq-info (C) Dominik Brodowski 2004-2009 Report errors and bugs to cpufreq@vger.kernel.org, please. analyzing CPU 0: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 250 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:10.27%, 500 MHz:37.39%, 750 MHz:9.76%, 1000 MHz:24.02%, 1.25 GHz:6.92%, 1.50 GHz:11.64% (60) analyzing CPU 1: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 250 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:10.29%, 500 MHz:37.38%, 750 MHz:9.76%, 1000 MHz:24.01%, 1.25 GHz:6.92%, 1.50 GHz:11.63% (60) root@letux:~# ./high-load -n 100% load stress test for 2 cores Sat Jan 1 04:52:17 UTC 2000 51° 52° 49° 36° 4002mV 1000MHz Sat Jan 1 04:52:17 UTC 2000 51° 52° 49° 36° 3991mV 1500MHz Sat Jan 1 04:52:18 UTC 2000 52° 53° 51° 36° 3854mV 1500MHz Sat Jan 1 04:52:20 UTC 2000 64° 54° 53° 36° 3876mV 1500MHz Sat Jan 1 04:52:21 UTC 2000 76° 62° 61° 36° 3855mV 1500MHz Sat Jan 1 04:52:23 UTC 2000 81° 69° 66° 37° 3852mV 1500MHz Sat Jan 1 04:52:24 UTC 2000 85° 71° 71° 37° 3832mV 1500MHz Sat Jan 1 04:52:26 UTC 2000 90° 75° 74° 37° 3829mV 1500MHz Sat Jan 1 04:52:27 UTC 2000 90° 75° 77° 37° 3824mV 1500MHz Sat Jan 1 04:52:29 UTC 2000 92° 78° 77° 38° 3843mV 1500MHz Sat Jan 1 04:52:30 UTC 2000 95° 81° 79° 38° 3813mV 1500MHz Sat Jan 1 04:52:31 UTC 2000 97° 84° 82° 38° 3825mV 1500MHz Sat Jan 1 04:52:33 UTC 2000 101° 86° 83° 38° 4021mV 500MHz Sat Jan 1 04:52:35 UTC 2000 102° 87° 84° 37° 4023mV 500MHz Sat Jan 1 04:52:37 UTC 2000 82° 84° 79° 37° 3848mV 1500MHz Sat Jan 1 04:52:38 UTC 2000 77° 79° 73° 37° 3813mV 1500MHz Sat Jan 1 04:52:40 UTC 2000 90° 78° 75° 37° 3836mV 1500MHz Sat Jan 1 04:52:41 UTC 2000 97° 84° 80° 38° 3822mV 1500MHz Sat Jan 1 04:52:43 UTC 2000 101° 86° 84° 38° 3964mV 750MHz Sat Jan 1 04:52:44 UTC 2000 104° 89° 86° 38° 4019mV 250MHz Sat Jan 1 04:52:47 UTC 2000 85° 86° 81° 37° 3820mV 1500MHz Sat Jan 1 04:52:48 UTC 2000 78° 81° 75° 37° 3832mV 1500MHz Sat Jan 1 04:52:50 UTC 2000 93° 81° 78° 37° 3797mV 1500MHz Sat Jan 1 04:52:51 UTC 2000 100° 86° 83° 37° 4029mV 250MHz Sat Jan 1 04:52:53 UTC 2000 104° 89° 86° 37° 3986mV 750MHz Sat Jan 1 04:52:55 UTC 2000 83° 84° 79° 37° 3815mV 1500MHz Sat Jan 1 04:52:57 UTC 2000 78° 80° 75° 37° 3824mV 1500MHz Sat Jan 1 04:52:58 UTC 2000 94° 82° 78° 37° 3799mV 1500MHz Sat Jan 1 04:53:00 UTC 2000 100° 87° 84° 39° 4056mV 500MHz Sat Jan 1 04:53:01 UTC 2000 105° 90° 87° 39° 4004mV 500MHz Sat Jan 1 04:53:04 UTC 2000 87° 88° 83° 39° 3816mV 1500MHz Sat Jan 1 04:53:05 UTC 2000 81° 83° 77° 38° 3797mV 1500MHz Sat Jan 1 04:53:07 UTC 2000 95° 83° 79° 38° 3824mV 1500MHz Sat Jan 1 04:53:08 UTC 2000 103° 88° 85° 38° 4106mV 250MHz ^Ckill 2653 2654
root@letux:~# cpufreq-info cpufrequtils 008: cpufreq-info (C) Dominik Brodowski 2004-2009 Report errors and bugs to cpufreq@vger.kernel.org, please. analyzing CPU 0: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 500 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:24.54%, 500 MHz:21.39%, 750 MHz:5.83%, 1000 MHz:11.23%, 1.25 GHz:4.45%, 1.50 GHz:32.56% (142) analyzing CPU 1: driver: cpufreq-dt CPUs which run at the same hardware frequency: 0 1 CPUs which need to have their frequency coordinated by software: 0 1 maximum transition latency: 400 us. hardware limits: 250 MHz - 1.50 GHz available frequency steps: 250 MHz, 500 MHz, 750 MHz, 1000 MHz, 1.25 GHz, 1.50 GHz available cpufreq governors: conservative, userspace, powersave, ondemand, performance current policy: frequency should be within 250 MHz and 1.50 GHz. The governor "ondemand" may decide which speed to use within this range. current CPU frequency is 500 MHz (asserted by call to hardware). cpufreq stats: 250 MHz:24.54%, 500 MHz:21.39%, 750 MHz:5.83%, 1000 MHz:11.23%, 1.25 GHz:4.45%, 1.50 GHz:32.56% (142) root@letux:~#
What I have changed was to apply Matthijs' impedance calibration patch
http://git.goldelico.com/?p=gta04-uboot.git;a=commit;h=918c4c46104714a3cfede...
and enabling the 1.5GHz OPP in DT. Plus recompile u-boot and kernel and then boot.
I always said it could be a software bug but nobody believed that it could be one. And I never believed that changing 1 bit in U-Boot could be sufficient (I thought we need to patch the kernel).
A realistic explanation is that our DDR3 is a little more sensitive to noise by 1.5GHz operation than the EVM and hence we get the problems but don't see them on the EVM if it is more robust.
At least my prototype was now running for 5 minutes @ 200% load @ 1.5GHz (cycling between 1.5GHz and 250MHz and 80°C and 110°C with a period of ca. 10 seconds) until I got
thermal thermal_zone0: critical temperature reached(125 C),shutting down
We should evaluate this further!
BR and please cross-check, Nikolaus
Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel
-
Adam Ford -
Andreas Lausch -
aTc -
Grazvydas Ignotas -
H. Nikolaus Schaller -
Matthijs van Duin -
Michael Mrozek