Like its predecessors, the omap5 has a special bank of I/Os for interfacing with sdcards, which (unlike other I/O) may operate at 3V in addition to 1.8V, with support for switching at runtime.
Unfortunately the designated LDO for this supply in the PMIC (Palmas), ldo9, can only supply 50 mA while cards are known to require more than that in practice. The omap5-uevm solved this by using a separate 3.3v supply for the sdcard, but disturbingly they continued to use ldo9 for the I/O supply on the omap5...
The device tree right now permits ldo9 to be lowered to 1.8V, unaware of the fact that the sdcard will continue to drive 3.3V signals into the now very unhappy I/Os of the omap5 !
Hence, the most urgent fix would be:
&ldo9_reg { regulator-min-microvolt = <3000000>; };
To make things worse, according to the Recommended Operating Conditions in the omap5 datasheet, vdds_sdcard should not be operated at 3.3V at all but 3.0V nominal (3.06V MAX DC).
This means you have to choose between: - configure ldo9 in bypass. Both sides use 3.3v, but this exceeds the max value specified by the omap5 datasheet. - configure ldo9 at 3.0V. The sdcard will be driving signals 0.3v higher than the omap5's supply (+ overshoot, if any).
Pick your poison.
Afaik bypass is the power-up config, while u-boot changes it to 3.0V. I'm not sure what linux does currently since omap5-board-common.dtsi incorrectly claims that &vmmcsd_fixed is 3.0V instead of 3.3V.
I'm guessing that ldo9 at 3.0V is indeed the intended modus operandi.
(side-note: ldo8 can provide 200 mA, which would have been more than enough, and is not connected to anything whatsoever... nicely done)
A final problem I noticed is power switching. The DT as-is implies that the sdcard can be powered off by switching ldo9 but this is plain wrong. Powering down the sdcard should be done by switching &vmmcsd_fixed via REGEN3. An obstacle is that eMMC (&mmc2) uses it as supply, but fortunately DT is simply wrong: eMMC is powered directly from the main 3.3v supply.
So, roughly:
vcc_3v3_main: fixedregulator-vcc_3v3_main { compatible = "regulator-fixed"; regulator-name = "vcc_3v3_main"; vin-supply = <&vmain>; regulator-min-microvolt = <3300000>; regulator-max-microvolt = <3300000>; };
&vmmcsd_fixed { /* called vcc_3v3_sdio on omap5-uevm */ vin-supply = <&vcc_3v3_main>; regulator-min-microvolt = <3300000>; regulator-max-microvolt = <3300000>; gpio = /* FIXME palmas REGEN3 */; };
&mmc1 { /* card supply is actually &vmmcsd_fixed */ vmmc-supply = <&ldo9_reg>; };
&mmc2 { vmmc-supply = <&vcc_3v3_main>; };
The details remaining are how to actually control REGEN3, and how to make sure linux leaves ldo9 alone and switches the supply just using &vmmcsd_fixed. I'm hoping someone who understands this stuff better than me has a suggestion?
Matthijs
Am Fri, 3 Mar 2017 10:08:41 +0100 hat Matthijs van Duin matthijsvanduin@gmail.com geschrieben:
Interesting find. Could that have something to do with the weird hangs while booting? Depending on the power usage of the SD Card, it could be that the SD Card you're booting from simply freezes because it doesn't get enough power... so instability might be caused by that.
Is that something we need to fix with software or does the hardware need to be changed as well.
Like its predecessors, the omap5 has a special bank of I/Os for interfacing with sdcards, which (unlike other I/O) may operate at 3V in addition to 1.8V, with support for switching at runtime.
Unfortunately the designated LDO for this supply in the PMIC (Palmas), ldo9, can only supply 50 mA while cards are known to require more than that in practice. The omap5-uevm solved this by using a separate 3.3v supply for the sdcard, but disturbingly they continued to use ldo9 for the I/O supply on the omap5...
The device tree right now permits ldo9 to be lowered to 1.8V, unaware of the fact that the sdcard will continue to drive 3.3V signals into the now very unhappy I/Os of the omap5 !
Hence, the most urgent fix would be:
&ldo9_reg { regulator-min-microvolt = <3000000>; };
To make things worse, according to the Recommended Operating Conditions in the omap5 datasheet, vdds_sdcard should not be operated at 3.3V at all but 3.0V nominal (3.06V MAX DC).
This means you have to choose between:
- configure ldo9 in bypass. Both sides use 3.3v, but this exceeds the max value specified by the omap5 datasheet.
- configure ldo9 at 3.0V. The sdcard will be driving signals 0.3v higher than the omap5's supply (+ overshoot, if any).
Pick your poison.
Afaik bypass is the power-up config, while u-boot changes it to 3.0V. I'm not sure what linux does currently since omap5-board-common.dtsi incorrectly claims that &vmmcsd_fixed is 3.0V instead of 3.3V.
I'm guessing that ldo9 at 3.0V is indeed the intended modus operandi.
(side-note: ldo8 can provide 200 mA, which would have been more than enough, and is not connected to anything whatsoever... nicely done)
A final problem I noticed is power switching. The DT as-is implies that the sdcard can be powered off by switching ldo9 but this is plain wrong. Powering down the sdcard should be done by switching &vmmcsd_fixed via REGEN3. An obstacle is that eMMC (&mmc2) uses it as supply, but fortunately DT is simply wrong: eMMC is powered directly from the main 3.3v supply.
So, roughly:
vcc_3v3_main: fixedregulator-vcc_3v3_main { compatible = "regulator-fixed"; regulator-name = "vcc_3v3_main"; vin-supply = <&vmain>; regulator-min-microvolt = <3300000>; regulator-max-microvolt = <3300000>; };
&vmmcsd_fixed { /* called vcc_3v3_sdio on omap5-uevm */ vin-supply = <&vcc_3v3_main>; regulator-min-microvolt = <3300000>; regulator-max-microvolt = <3300000>; gpio = /* FIXME palmas REGEN3 */; };
&mmc1 { /* card supply is actually &vmmcsd_fixed */ vmmc-supply = <&ldo9_reg>; };
&mmc2 { vmmc-supply = <&vcc_3v3_main>; };
The details remaining are how to actually control REGEN3, and how to make sure linux leaves ldo9 alone and switches the supply just using &vmmcsd_fixed. I'm hoping someone who understands this stuff better than me has a suggestion?
Matthijs
(I dropped linux-omap from the recipients list for this subthread since it's pyra-specific)
On Fri, Mar 03, 2017 at 12:03:56PM +0100, Michael Mrozek wrote:
Could that have something to do with the weird hangs while booting?
No, not really, it's just a concern with the DT content and how linux might act upon it. Specifically if the sd card supports 1.8V operation then linux might (I presume) want to change voltage from 3.0V to 1.8V to save power. This is not supported by the (rather sub-optimal) way the power supplies have been handled here, and if linux attempts it this could be harmful to the omap5.
The fix is mostly adjusting DT so linux won't try to do this.
Matthijs
* Michael Mrozek EvilDragon@openpandora.org [170303 04:24]:
Am Fri, 3 Mar 2017 10:08:41 +0100 hat Matthijs van Duin matthijsvanduin@gmail.com geschrieben:
Interesting find. Could that have something to do with the weird hangs while booting? Depending on the power usage of the SD Card, it could be that the SD Card you're booting from simply freezes because it doesn't get enough power... so instability might be caused by that.
Is that something we need to fix with software or does the hardware need to be changed as well.
Are you seeing hangs during boot on omap5-uevm also or just on pyra?
Regards,
Tony
Am Fri, 3 Mar 2017 08:19:58 -0800 hat Tony Lindgren tony@atomide.com geschrieben:
Interesting find. Could that have something to do with the weird hangs while booting? Depending on the power usage of the SD Card, it could be that the SD Card you're booting from simply freezes because it doesn't get enough power... so instability might be caused by that. Is that something we need to fix with software or does the hardware need to be changed as well.
Are you seeing hangs during boot on omap5-uevm also or just on pyra?
Right now, Matthijs is mostly the one working on the current-gen Pyra and we haven't tested it extensively before shipping it to him. So the instability could be a faulty hardware. I'll ship out more prototype units next week (also a new one to Matthijs), so maybe it's just that.
However, we have some weird behaviour with reboots and resets where some settings are being remembered, some aren't, etc., and we're currently investigating that.
I don't think the EVM has these issues, but we're using different hardware for the power supply and a different DT-file, so it can't really be compared.
* Michael Mrozek EvilDragon@openpandora.org [170303 08:31]:
Am Fri, 3 Mar 2017 08:19:58 -0800 hat Tony Lindgren tony@atomide.com geschrieben:
Interesting find. Could that have something to do with the weird hangs while booting? Depending on the power usage of the SD Card, it could be that the SD Card you're booting from simply freezes because it doesn't get enough power... so instability might be caused by that. Is that something we need to fix with software or does the hardware need to be changed as well.
Are you seeing hangs during boot on omap5-uevm also or just on pyra?
Right now, Matthijs is mostly the one working on the current-gen Pyra and we haven't tested it extensively before shipping it to him. So the instability could be a faulty hardware. I'll ship out more prototype units next week (also a new one to Matthijs), so maybe it's just that.
OK
However, we have some weird behaviour with reboots and resets where some settings are being remembered, some aren't, etc., and we're currently investigating that.
OK recall seeing igepv5 reboot not always work or the omap5-uevm reboot always not work depending on the u-boot version. Not sure if that is still happening but that hints to something not being properly re-initialized by the kernel for clocks.
I don't think the EVM has these issues, but we're using different hardware for the power supply and a different DT-file, so it can't really be compared.
OK
Regards,
Tony
Am 03.03.2017 um 18:05 schrieb Tony Lindgren tony@atomide.com:
- Michael Mrozek EvilDragon@openpandora.org [170303 08:31]:
Am Fri, 3 Mar 2017 08:19:58 -0800 hat Tony Lindgren tony@atomide.com geschrieben:
Interesting find. Could that have something to do with the weird hangs while booting? Depending on the power usage of the SD Card, it could be that the SD Card you're booting from simply freezes because it doesn't get enough power... so instability might be caused by that. Is that something we need to fix with software or does the hardware need to be changed as well.
Are you seeing hangs during boot on omap5-uevm also or just on pyra?
Right now, Matthijs is mostly the one working on the current-gen Pyra and we haven't tested it extensively before shipping it to him. So the instability could be a faulty hardware. I'll ship out more prototype units next week (also a new one to Matthijs), so maybe it's just that.
OK
However, we have some weird behaviour with reboots and resets where some settings are being remembered, some aren't, etc., and we're currently investigating that.
OK recall seeing igepv5 reboot not always work or the omap5-uevm reboot always not work depending on the u-boot version. Not sure if that is still happening but that hints to something not being properly re-initialized by the kernel for clocks.
I have just tried and my EVM seems now to properly "reboot" (with Linux-4.10.0 and SPL/U-Boot-2016.11) while I still remember this problem existed earlier.
Pyra using the same kernel binary, but different DTB and different SPL/U-Boot binaries now "restarts" as well, but I have the impression that it takes longer after "Starting kernel ..."
And then I get:
Starting kernel ...
[ 0.000000] Booting Linux on physical CPU 0x0 [ 0.000000] Linux version 4.10.0-letux+ (hns@iMac.local) (gcc version 4.9.2 (GCC) ) #851 SMP PREEMPT Fri Mar 3 18:21:44 CET 2017 [ 0.000000] CPU: ARMv7 Processor [412fc0f2] revision 2 (ARMv7), cr=10c5387d [ 0.000000] CPU: div instructions available: patching division code [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache [ 0.000000] OF: fdt:Machine model: Pyra-Handheld-V5.1 [ 0.000000] debug: ignoring loglevel setting. [ 0.000000] cma: Reserved 16 MiB at 0xfd800000 [ 0.000000] Memory policy: Data cache writealloc [ 0.000000] OMAP4: Map 0xfee00000 to fe600000 for dram barrier [ 0.000000] On node 0 totalpages: 519680 [ 0.000000] free_area_init_node: node 0, pgdat c0ca3f80, node_mem_map ef019000 [ 0.000000] Normal zone: 1536 pages used for memmap [ 0.000000] Normal zone: 0 pages reserved [ 0.000000] Normal zone: 196608 pages, LIFO batch:31 [ 0.000000] HighMem zone: 323072 pages, LIFO batch:31 [ 0.000000] OMAP5432 ES2.0 [ 0.000000] percpu: Embedded 15 pages/cpu @eefb5000 s31680 r8192 d21568 u61440 [ 0.000000] pcpu-alloc: s31680 r8192 d21568 u61440 alloc=15*4096 [ 0.000000] pcpu-alloc: [0] 0 [0] 1 [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 518144 [ 0.000000] Kernel command line: console= root=PARTUUID=00000000-02 rw rootfstype=ext4 rootwait console=ttyO2,115200n8 vram=12M omapfb.vram=0:8M,1:4M omapfb.rotate_type=0 omapdss.def_disp=lcd rootwait twl4030_charger.allow_usb=1 musb_hdrc.preserve_vbus=1 log_buf_len=8M ignore_loglevel earlyprintk [ 0.000000] log_buf_len: 8388608 bytes [ 0.000000] early log buf free: 63860(97%) [ 0.000000] PID hash table entries: 4096 (order: 2, 16384 bytes) [ 0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes) [ 0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes) [ 0.000000] Memory: 2016588K/2078720K available (7168K kernel code, 679K rwdata, 2204K rodata, 1024K init, 8060K bss, 45748K reserved, 16384K cma-reserved, 1275904K highmem) [ 0.000000] Virtual kernel memory layout: [ 0.000000] vector : 0xffff0000 - 0xffff1000 ( 4 kB) [ 0.000000] fixmap : 0xffc00000 - 0xfff00000 (3072 kB) [ 0.000000] vmalloc : 0xf0800000 - 0xff800000 ( 240 MB) [ 0.000000] lowmem : 0xc0000000 - 0xf0000000 ( 768 MB) [ 0.000000] pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB) [ 0.000000] modules : 0xbf000000 - 0xbfe00000 ( 14 MB) [ 0.000000] .text : 0xc0008000 - 0xc0800000 (8160 kB) [ 0.000000] .init : 0xc0b00000 - 0xc0c00000 (1024 kB) [ 0.000000] .data : 0xc0c00000 - 0xc0ca9ef8 ( 680 kB) [ 0.000000] .bss : 0xc0ca9ef8 - 0xc1488f1c (8061 kB) [ 0.000000] Running RCU self tests [ 0.000000] Preemptible hierarchical RCU implementation. [ 0.000000] RCU lockdep checking is enabled. [ 0.000000] Build-time adjustment of leaf fanout to 32. [ 0.000000] NR_IRQS:16 nr_irqs:16 16 [ 0.000000] clock: dpll_abe_ck failed transition to 'locked' [ 0.000000] OMAP clockevent source: timer1 at 32768 Hz [ 0.000000] arm_arch_timer: Architected cp15 timer(s) running at 6.14MHz (phys). [ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x16ac02862, max_idle_ns: 440795202218 ns [ 0.000005] sched_clock: 56 bits at 6MHz, resolution 162ns, wraps every 4398046511085ns [ 0.000016] Switching to timer-based delay loop, resolution 162ns [ 0.000366] clocksource: 32k_counter: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 58327039986419 ns [ 0.000375] OMAP clocksource: 32k_counter at 32768 Hz [ 0.001482] Console: colour dummy device 80x30 [ 0.001505] Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar [ 0.001513] ... MAX_LOCKDEP_SUBCLASSES: 8 [ 0.001519] ... MAX_LOCK_DEPTH: 48 [ 0.001526] ... MAX_LOCKDEP_KEYS: 8191 [ 0.001532] ... CLASSHASH_SIZE: 4096 [ 0.001537] ... MAX_LOCKDEP_ENTRIES: 32768 [ 0.001543] ... MAX_LOCKDEP_CHAINS: 65536 [ 0.001549] ... CHAINHASH_SIZE: 32768 [ 0.001555] memory used by lock dependency info: 5167 kB [ 0.001561] per task-struct memory footprint: 1536 bytes [ 0.001582] Calibrating delay loop (skipped), value calculated using timer frequency.. 12.28 BogoMIPS (lpj=61440) [ 0.001597] pid_max: default: 32768 minimum: 301 [ 0.001830] Security Framework initialized [ 0.001917] Mount-cache hash table entries: 2048 (order: 1, 8192 bytes) [ 0.001930] Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes) [ 0.004366] CPU: Testing write buffer coherency: ok [ 0.005653] /cpus/cpu@0 missing clock-frequency property [ 0.005676] /cpus/cpu@1 missing clock-frequency property [ 0.005688] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000 [ 0.030191] Setting up static identity map for 0x80100000 - 0x80100058 [ 0.100410] smp: Bringing up secondary CPUs ... [ 0.171431] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001 [ 0.171939] smp: Brought up 1 node, 2 CPUs [ 0.171953] SMP: Total of 2 processors activated (24.57 BogoMIPS). [ 0.171961] CPU: All CPU(s) started in HYP mode. [ 0.171967] CPU: Virtualization extensions available. [ 0.174659] devtmpfs: initialized [ 0.234205] VFP support v0.3: implementor 41 architecture 4 part 30 variant f rev 0 [ 0.331721] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns [ 0.331758] futex hash table entries: 512 (order: 3, 32768 bytes) [ 0.332912] pinctrl core: initialized pinctrl subsystem [ 0.334597] _regulator_do_enable regulator-dummy [ 0.336940] NET: Registered protocol family 16 [ 0.342193] DMA: preallocated 256 KiB pool for atomic coherent allocations [ 0.344036] omap_hwmod: l3_main_3 using broken dt data from ocp [ 0.345812] omap_hwmod: l3_main_2 using broken dt data from ocp [ 1.748394] clock: dpll_abe_ck failed transition to 'locked' [ 3.047550] clock: dpll_abe_ck failed transition to 'locked' [ 4.346752] clock: dpll_abe_ck failed transition to 'locked' [ 5.645936] clock: dpll_abe_ck failed transition to 'locked' [ 5.648581] ------------[ cut here ]------------ [ 5.648604] WARNING: CPU: 0 PID: 1 at drivers/clk/clk.c:594 clk_core_disable_lock+0x18/0x24 [ 5.648611] Modules linked in: [ 5.648625] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.10.0-letux+ #851 [ 5.648632] Hardware name: Generic OMAP5 (Flattened Device Tree) [ 5.648652] [<c010f368>] (unwind_backtrace) from [<c010b914>] (show_stack+0x10/0x14) [ 5.648669] [<c010b914>] (show_stack) from [<c042ec60>] (dump_stack+0x98/0xd0) [ 5.648683] [<c042ec60>] (dump_stack) from [<c012f844>] (__warn+0xd0/0x100) [ 5.648694] [<c012f844>] (__warn) from [<c012f918>] (warn_slowpath_null+0x1c/0x24) [ 5.648707] [<c012f918>] (warn_slowpath_null) from [<c0486594>] (clk_core_disable_lock+0x18/0x24) [ 5.648722] [<c0486594>] (clk_core_disable_lock) from [<c011ca9c>] (_disable_clocks+0x18/0x70) [ 5.648735] [<c011ca9c>] (_disable_clocks) from [<c011d0f4>] (_enable+0x1e0/0x224) [ 5.648748] [<c011d0f4>] (_enable) from [<c0b0c690>] (_setup+0xc4/0x3ec) [ 5.648760] [<c0b0c690>] (_setup) from [<c011d5a0>] (omap_hwmod_for_each+0x3c/0x64) [ 5.648772] [<c011d5a0>] (omap_hwmod_for_each) from [<c0b0ccec>] (__omap_hwmod_setup_all+0x30/0x40) [ 5.648784] [<c0b0ccec>] (__omap_hwmod_setup_all) from [<c0101938>] (do_one_initcall+0xa8/0x150) [ 5.648797] [<c0101938>] (do_one_initcall) from [<c0b00d88>] (kernel_init_freeable+0x124/0x1f0) [ 5.648810] [<c0b00d88>] (kernel_init_freeable) from [<c073a994>] (kernel_init+0x8/0x110) [ 5.648822] [<c073a994>] (kernel_init) from [<c01070d0>] (ret_from_fork+0x14/0x24) [ 5.648856] ---[ end trace 158d0428f201436f ]--- [ 5.648866] omap_hwmod: dmic: _wait_target_ready failed: -16 [ 5.648878] omap_hwmod: dmic: cannot be enabled for reset (3) [ 6.974755] clock: dpll_abe_ck failed transition to 'locked' [ 8.273895] clock: dpll_abe_ck failed transition to 'locked' [ 8.276541] ------------[ cut here ]------------ [ 8.276556] WARNING: CPU: 0 PID: 1 at drivers/clk/clk.c:594 clk_core_disable_lock+0x18/0x24 [ 8.276562] Modules linked in: [ 8.276575] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G W 4.10.0-letux+ #851 [ 8.276582] Hardware name: Generic OMAP5 (Flattened Device Tree) [ 8.276595] [<c010f368>] (unwind_backtrace) from [<c010b914>] (show_stack+0x10/0x14) [ 8.276609] [<c010b914>] (show_stack) from [<c042ec60>] (dump_stack+0x98/0xd0) [ 8.276621] [<c042ec60>] (dump_stack) from [<c012f844>] (__warn+0xd0/0x100) [ 8.276631] [<c012f844>] (__warn) from [<c012f918>] (warn_slowpath_null+0x1c/0x24) [ 8.276644] [<c012f918>] (warn_slowpath_null) from [<c0486594>] (clk_core_disable_lock+0x18/0x24) ... later the kernel hangs after initializing several peripherals incl. twl6040.
So to me it looks as if some ABE clocks are not reset/restarted/locked. And this blocks logging for the first seconds (you notice the big delays for each attempt with dpll_abe_ck).
I wonder how this can be different from the OMAP5-EVM.
BR, Nikolaus
* H. Nikolaus Schaller hns@goldelico.com [170303 10:47]:
Am 03.03.2017 um 18:05 schrieb Tony Lindgren tony@atomide.com:
- Michael Mrozek EvilDragon@openpandora.org [170303 08:31]:
Am Fri, 3 Mar 2017 08:19:58 -0800 hat Tony Lindgren tony@atomide.com geschrieben:
Interesting find. Could that have something to do with the weird hangs while booting? Depending on the power usage of the SD Card, it could be that the SD Card you're booting from simply freezes because it doesn't get enough power... so instability might be caused by that. Is that something we need to fix with software or does the hardware need to be changed as well.
Are you seeing hangs during boot on omap5-uevm also or just on pyra?
Right now, Matthijs is mostly the one working on the current-gen Pyra and we haven't tested it extensively before shipping it to him. So the instability could be a faulty hardware. I'll ship out more prototype units next week (also a new one to Matthijs), so maybe it's just that.
OK
However, we have some weird behaviour with reboots and resets where some settings are being remembered, some aren't, etc., and we're currently investigating that.
OK recall seeing igepv5 reboot not always work or the omap5-uevm reboot always not work depending on the u-boot version. Not sure if that is still happening but that hints to something not being properly re-initialized by the kernel for clocks.
I have just tried and my EVM seems now to properly "reboot" (with Linux-4.10.0 and SPL/U-Boot-2016.11) while I still remember this problem existed earlier.
Pyra using the same kernel binary, but different DTB and different SPL/U-Boot binaries now "restarts" as well, but I have the impression that it takes longer after "Starting kernel ..."
And then I get:
Yup this is similar to what I recall seeing where the clocks don't always lock on reboot. Adding Tero to Cc if he has a fix in mind for this one.
Regards,
Tony
Starting kernel ...
[ 0.000000] Booting Linux on physical CPU 0x0 [ 0.000000] Linux version 4.10.0-letux+ (hns@iMac.local) (gcc version 4.9.2 (GCC) ) #851 SMP PREEMPT Fri Mar 3 18:21:44 CET 2017 [ 0.000000] CPU: ARMv7 Processor [412fc0f2] revision 2 (ARMv7), cr=10c5387d [ 0.000000] CPU: div instructions available: patching division code [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache [ 0.000000] OF: fdt:Machine model: Pyra-Handheld-V5.1 [ 0.000000] debug: ignoring loglevel setting. [ 0.000000] cma: Reserved 16 MiB at 0xfd800000 [ 0.000000] Memory policy: Data cache writealloc [ 0.000000] OMAP4: Map 0xfee00000 to fe600000 for dram barrier [ 0.000000] On node 0 totalpages: 519680 [ 0.000000] free_area_init_node: node 0, pgdat c0ca3f80, node_mem_map ef019000 [ 0.000000] Normal zone: 1536 pages used for memmap [ 0.000000] Normal zone: 0 pages reserved [ 0.000000] Normal zone: 196608 pages, LIFO batch:31 [ 0.000000] HighMem zone: 323072 pages, LIFO batch:31 [ 0.000000] OMAP5432 ES2.0 [ 0.000000] percpu: Embedded 15 pages/cpu @eefb5000 s31680 r8192 d21568 u61440 [ 0.000000] pcpu-alloc: s31680 r8192 d21568 u61440 alloc=15*4096 [ 0.000000] pcpu-alloc: [0] 0 [0] 1 [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 518144 [ 0.000000] Kernel command line: console= root=PARTUUID=00000000-02 rw rootfstype=ext4 rootwait console=ttyO2,115200n8 vram=12M omapfb.vram=0:8M,1:4M omapfb.rotate_type=0 omapdss.def_disp=lcd rootwait twl4030_charger.allow_usb=1 musb_hdrc.preserve_vbus=1 log_buf_len=8M ignore_loglevel earlyprintk [ 0.000000] log_buf_len: 8388608 bytes [ 0.000000] early log buf free: 63860(97%) [ 0.000000] PID hash table entries: 4096 (order: 2, 16384 bytes) [ 0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes) [ 0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes) [ 0.000000] Memory: 2016588K/2078720K available (7168K kernel code, 679K rwdata, 2204K rodata, 1024K init, 8060K bss, 45748K reserved, 16384K cma-reserved, 1275904K highmem) [ 0.000000] Virtual kernel memory layout: [ 0.000000] vector : 0xffff0000 - 0xffff1000 ( 4 kB) [ 0.000000] fixmap : 0xffc00000 - 0xfff00000 (3072 kB) [ 0.000000] vmalloc : 0xf0800000 - 0xff800000 ( 240 MB) [ 0.000000] lowmem : 0xc0000000 - 0xf0000000 ( 768 MB) [ 0.000000] pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB) [ 0.000000] modules : 0xbf000000 - 0xbfe00000 ( 14 MB) [ 0.000000] .text : 0xc0008000 - 0xc0800000 (8160 kB) [ 0.000000] .init : 0xc0b00000 - 0xc0c00000 (1024 kB) [ 0.000000] .data : 0xc0c00000 - 0xc0ca9ef8 ( 680 kB) [ 0.000000] .bss : 0xc0ca9ef8 - 0xc1488f1c (8061 kB) [ 0.000000] Running RCU self tests [ 0.000000] Preemptible hierarchical RCU implementation. [ 0.000000] RCU lockdep checking is enabled. [ 0.000000] Build-time adjustment of leaf fanout to 32. [ 0.000000] NR_IRQS:16 nr_irqs:16 16 [ 0.000000] clock: dpll_abe_ck failed transition to 'locked' [ 0.000000] OMAP clockevent source: timer1 at 32768 Hz [ 0.000000] arm_arch_timer: Architected cp15 timer(s) running at 6.14MHz (phys). [ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x16ac02862, max_idle_ns: 440795202218 ns [ 0.000005] sched_clock: 56 bits at 6MHz, resolution 162ns, wraps every 4398046511085ns [ 0.000016] Switching to timer-based delay loop, resolution 162ns [ 0.000366] clocksource: 32k_counter: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 58327039986419 ns [ 0.000375] OMAP clocksource: 32k_counter at 32768 Hz [ 0.001482] Console: colour dummy device 80x30 [ 0.001505] Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar [ 0.001513] ... MAX_LOCKDEP_SUBCLASSES: 8 [ 0.001519] ... MAX_LOCK_DEPTH: 48 [ 0.001526] ... MAX_LOCKDEP_KEYS: 8191 [ 0.001532] ... CLASSHASH_SIZE: 4096 [ 0.001537] ... MAX_LOCKDEP_ENTRIES: 32768 [ 0.001543] ... MAX_LOCKDEP_CHAINS: 65536 [ 0.001549] ... CHAINHASH_SIZE: 32768 [ 0.001555] memory used by lock dependency info: 5167 kB [ 0.001561] per task-struct memory footprint: 1536 bytes [ 0.001582] Calibrating delay loop (skipped), value calculated using timer frequency.. 12.28 BogoMIPS (lpj=61440) [ 0.001597] pid_max: default: 32768 minimum: 301 [ 0.001830] Security Framework initialized [ 0.001917] Mount-cache hash table entries: 2048 (order: 1, 8192 bytes) [ 0.001930] Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes) [ 0.004366] CPU: Testing write buffer coherency: ok [ 0.005653] /cpus/cpu@0 missing clock-frequency property [ 0.005676] /cpus/cpu@1 missing clock-frequency property [ 0.005688] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000 [ 0.030191] Setting up static identity map for 0x80100000 - 0x80100058 [ 0.100410] smp: Bringing up secondary CPUs ... [ 0.171431] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001 [ 0.171939] smp: Brought up 1 node, 2 CPUs [ 0.171953] SMP: Total of 2 processors activated (24.57 BogoMIPS). [ 0.171961] CPU: All CPU(s) started in HYP mode. [ 0.171967] CPU: Virtualization extensions available. [ 0.174659] devtmpfs: initialized [ 0.234205] VFP support v0.3: implementor 41 architecture 4 part 30 variant f rev 0 [ 0.331721] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns [ 0.331758] futex hash table entries: 512 (order: 3, 32768 bytes) [ 0.332912] pinctrl core: initialized pinctrl subsystem [ 0.334597] _regulator_do_enable regulator-dummy [ 0.336940] NET: Registered protocol family 16 [ 0.342193] DMA: preallocated 256 KiB pool for atomic coherent allocations [ 0.344036] omap_hwmod: l3_main_3 using broken dt data from ocp [ 0.345812] omap_hwmod: l3_main_2 using broken dt data from ocp [ 1.748394] clock: dpll_abe_ck failed transition to 'locked' [ 3.047550] clock: dpll_abe_ck failed transition to 'locked' [ 4.346752] clock: dpll_abe_ck failed transition to 'locked' [ 5.645936] clock: dpll_abe_ck failed transition to 'locked' [ 5.648581] ------------[ cut here ]------------ [ 5.648604] WARNING: CPU: 0 PID: 1 at drivers/clk/clk.c:594 clk_core_disable_lock+0x18/0x24 [ 5.648611] Modules linked in: [ 5.648625] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.10.0-letux+ #851 [ 5.648632] Hardware name: Generic OMAP5 (Flattened Device Tree) [ 5.648652] [<c010f368>] (unwind_backtrace) from [<c010b914>] (show_stack+0x10/0x14) [ 5.648669] [<c010b914>] (show_stack) from [<c042ec60>] (dump_stack+0x98/0xd0) [ 5.648683] [<c042ec60>] (dump_stack) from [<c012f844>] (__warn+0xd0/0x100) [ 5.648694] [<c012f844>] (__warn) from [<c012f918>] (warn_slowpath_null+0x1c/0x24) [ 5.648707] [<c012f918>] (warn_slowpath_null) from [<c0486594>] (clk_core_disable_lock+0x18/0x24) [ 5.648722] [<c0486594>] (clk_core_disable_lock) from [<c011ca9c>] (_disable_clocks+0x18/0x70) [ 5.648735] [<c011ca9c>] (_disable_clocks) from [<c011d0f4>] (_enable+0x1e0/0x224) [ 5.648748] [<c011d0f4>] (_enable) from [<c0b0c690>] (_setup+0xc4/0x3ec) [ 5.648760] [<c0b0c690>] (_setup) from [<c011d5a0>] (omap_hwmod_for_each+0x3c/0x64) [ 5.648772] [<c011d5a0>] (omap_hwmod_for_each) from [<c0b0ccec>] (__omap_hwmod_setup_all+0x30/0x40) [ 5.648784] [<c0b0ccec>] (__omap_hwmod_setup_all) from [<c0101938>] (do_one_initcall+0xa8/0x150) [ 5.648797] [<c0101938>] (do_one_initcall) from [<c0b00d88>] (kernel_init_freeable+0x124/0x1f0) [ 5.648810] [<c0b00d88>] (kernel_init_freeable) from [<c073a994>] (kernel_init+0x8/0x110) [ 5.648822] [<c073a994>] (kernel_init) from [<c01070d0>] (ret_from_fork+0x14/0x24) [ 5.648856] ---[ end trace 158d0428f201436f ]--- [ 5.648866] omap_hwmod: dmic: _wait_target_ready failed: -16 [ 5.648878] omap_hwmod: dmic: cannot be enabled for reset (3) [ 6.974755] clock: dpll_abe_ck failed transition to 'locked' [ 8.273895] clock: dpll_abe_ck failed transition to 'locked' [ 8.276541] ------------[ cut here ]------------ [ 8.276556] WARNING: CPU: 0 PID: 1 at drivers/clk/clk.c:594 clk_core_disable_lock+0x18/0x24 [ 8.276562] Modules linked in: [ 8.276575] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G W 4.10.0-letux+ #851 [ 8.276582] Hardware name: Generic OMAP5 (Flattened Device Tree) [ 8.276595] [<c010f368>] (unwind_backtrace) from [<c010b914>] (show_stack+0x10/0x14) [ 8.276609] [<c010b914>] (show_stack) from [<c042ec60>] (dump_stack+0x98/0xd0) [ 8.276621] [<c042ec60>] (dump_stack) from [<c012f844>] (__warn+0xd0/0x100) [ 8.276631] [<c012f844>] (__warn) from [<c012f918>] (warn_slowpath_null+0x1c/0x24) [ 8.276644] [<c012f918>] (warn_slowpath_null) from [<c0486594>] (clk_core_disable_lock+0x18/0x24) ... later the kernel hangs after initializing several peripherals incl. twl6040.
So to me it looks as if some ABE clocks are not reset/restarted/locked. And this blocks logging for the first seconds (you notice the big delays for each attempt with dpll_abe_ck).
I wonder how this can be different from the OMAP5-EVM.
BR, Nikolaus
On 06/03/17 19:03, Tony Lindgren wrote:
- H. Nikolaus Schaller hns@goldelico.com [170303 10:47]:
Am 03.03.2017 um 18:05 schrieb Tony Lindgren tony@atomide.com:
- Michael Mrozek EvilDragon@openpandora.org [170303 08:31]:
Am Fri, 3 Mar 2017 08:19:58 -0800 hat Tony Lindgren tony@atomide.com geschrieben:
Interesting find. Could that have something to do with the weird hangs while booting? Depending on the power usage of the SD Card, it could be that the SD Card you're booting from simply freezes because it doesn't get enough power... so instability might be caused by that. Is that something we need to fix with software or does the hardware need to be changed as well.
Are you seeing hangs during boot on omap5-uevm also or just on pyra?
Right now, Matthijs is mostly the one working on the current-gen Pyra and we haven't tested it extensively before shipping it to him. So the instability could be a faulty hardware. I'll ship out more prototype units next week (also a new one to Matthijs), so maybe it's just that.
OK
However, we have some weird behaviour with reboots and resets where some settings are being remembered, some aren't, etc., and we're currently investigating that.
OK recall seeing igepv5 reboot not always work or the omap5-uevm reboot always not work depending on the u-boot version. Not sure if that is still happening but that hints to something not being properly re-initialized by the kernel for clocks.
I have just tried and my EVM seems now to properly "reboot" (with Linux-4.10.0 and SPL/U-Boot-2016.11) while I still remember this problem existed earlier.
Pyra using the same kernel binary, but different DTB and different SPL/U-Boot binaries now "restarts" as well, but I have the impression that it takes longer after "Starting kernel ..."
And then I get:
Yup this is similar to what I recall seeing where the clocks don't always lock on reboot. Adding Tero to Cc if he has a fix in mind for this one.
Haven't faced this issue myself, but ABE is notorious for this kind of issues I recall, due to custom clocking setup for that specific DPLL. Are you tweaking the source of that DPLL?
I can give omap5uevm a shot at some point and see what happens.
-Tero
Regards,
Tony
Starting kernel ...
[ 0.000000] Booting Linux on physical CPU 0x0 [ 0.000000] Linux version 4.10.0-letux+ (hns@iMac.local) (gcc version 4.9.2 (GCC) ) #851 SMP PREEMPT Fri Mar 3 18:21:44 CET 2017 [ 0.000000] CPU: ARMv7 Processor [412fc0f2] revision 2 (ARMv7), cr=10c5387d [ 0.000000] CPU: div instructions available: patching division code [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache [ 0.000000] OF: fdt:Machine model: Pyra-Handheld-V5.1 [ 0.000000] debug: ignoring loglevel setting. [ 0.000000] cma: Reserved 16 MiB at 0xfd800000 [ 0.000000] Memory policy: Data cache writealloc [ 0.000000] OMAP4: Map 0xfee00000 to fe600000 for dram barrier [ 0.000000] On node 0 totalpages: 519680 [ 0.000000] free_area_init_node: node 0, pgdat c0ca3f80, node_mem_map ef019000 [ 0.000000] Normal zone: 1536 pages used for memmap [ 0.000000] Normal zone: 0 pages reserved [ 0.000000] Normal zone: 196608 pages, LIFO batch:31 [ 0.000000] HighMem zone: 323072 pages, LIFO batch:31 [ 0.000000] OMAP5432 ES2.0 [ 0.000000] percpu: Embedded 15 pages/cpu @eefb5000 s31680 r8192 d21568 u61440 [ 0.000000] pcpu-alloc: s31680 r8192 d21568 u61440 alloc=15*4096 [ 0.000000] pcpu-alloc: [0] 0 [0] 1 [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 518144 [ 0.000000] Kernel command line: console= root=PARTUUID=00000000-02 rw rootfstype=ext4 rootwait console=ttyO2,115200n8 vram=12M omapfb.vram=0:8M,1:4M omapfb.rotate_type=0 omapdss.def_disp=lcd rootwait twl4030_charger.allow_usb=1 musb_hdrc.preserve_vbus=1 log_buf_len=8M ignore_loglevel earlyprintk [ 0.000000] log_buf_len: 8388608 bytes [ 0.000000] early log buf free: 63860(97%) [ 0.000000] PID hash table entries: 4096 (order: 2, 16384 bytes) [ 0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes) [ 0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes) [ 0.000000] Memory: 2016588K/2078720K available (7168K kernel code, 679K rwdata, 2204K rodata, 1024K init, 8060K bss, 45748K reserved, 16384K cma-reserved, 1275904K highmem) [ 0.000000] Virtual kernel memory layout: [ 0.000000] vector : 0xffff0000 - 0xffff1000 ( 4 kB) [ 0.000000] fixmap : 0xffc00000 - 0xfff00000 (3072 kB) [ 0.000000] vmalloc : 0xf0800000 - 0xff800000 ( 240 MB) [ 0.000000] lowmem : 0xc0000000 - 0xf0000000 ( 768 MB) [ 0.000000] pkmap : 0xbfe00000 - 0xc0000000 ( 2 MB) [ 0.000000] modules : 0xbf000000 - 0xbfe00000 ( 14 MB) [ 0.000000] .text : 0xc0008000 - 0xc0800000 (8160 kB) [ 0.000000] .init : 0xc0b00000 - 0xc0c00000 (1024 kB) [ 0.000000] .data : 0xc0c00000 - 0xc0ca9ef8 ( 680 kB) [ 0.000000] .bss : 0xc0ca9ef8 - 0xc1488f1c (8061 kB) [ 0.000000] Running RCU self tests [ 0.000000] Preemptible hierarchical RCU implementation. [ 0.000000] RCU lockdep checking is enabled. [ 0.000000] Build-time adjustment of leaf fanout to 32. [ 0.000000] NR_IRQS:16 nr_irqs:16 16 [ 0.000000] clock: dpll_abe_ck failed transition to 'locked' [ 0.000000] OMAP clockevent source: timer1 at 32768 Hz [ 0.000000] arm_arch_timer: Architected cp15 timer(s) running at 6.14MHz (phys). [ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x16ac02862, max_idle_ns: 440795202218 ns [ 0.000005] sched_clock: 56 bits at 6MHz, resolution 162ns, wraps every 4398046511085ns [ 0.000016] Switching to timer-based delay loop, resolution 162ns [ 0.000366] clocksource: 32k_counter: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 58327039986419 ns [ 0.000375] OMAP clocksource: 32k_counter at 32768 Hz [ 0.001482] Console: colour dummy device 80x30 [ 0.001505] Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar [ 0.001513] ... MAX_LOCKDEP_SUBCLASSES: 8 [ 0.001519] ... MAX_LOCK_DEPTH: 48 [ 0.001526] ... MAX_LOCKDEP_KEYS: 8191 [ 0.001532] ... CLASSHASH_SIZE: 4096 [ 0.001537] ... MAX_LOCKDEP_ENTRIES: 32768 [ 0.001543] ... MAX_LOCKDEP_CHAINS: 65536 [ 0.001549] ... CHAINHASH_SIZE: 32768 [ 0.001555] memory used by lock dependency info: 5167 kB [ 0.001561] per task-struct memory footprint: 1536 bytes [ 0.001582] Calibrating delay loop (skipped), value calculated using timer frequency.. 12.28 BogoMIPS (lpj=61440) [ 0.001597] pid_max: default: 32768 minimum: 301 [ 0.001830] Security Framework initialized [ 0.001917] Mount-cache hash table entries: 2048 (order: 1, 8192 bytes) [ 0.001930] Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes) [ 0.004366] CPU: Testing write buffer coherency: ok [ 0.005653] /cpus/cpu@0 missing clock-frequency property [ 0.005676] /cpus/cpu@1 missing clock-frequency property [ 0.005688] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000 [ 0.030191] Setting up static identity map for 0x80100000 - 0x80100058 [ 0.100410] smp: Bringing up secondary CPUs ... [ 0.171431] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001 [ 0.171939] smp: Brought up 1 node, 2 CPUs [ 0.171953] SMP: Total of 2 processors activated (24.57 BogoMIPS). [ 0.171961] CPU: All CPU(s) started in HYP mode. [ 0.171967] CPU: Virtualization extensions available. [ 0.174659] devtmpfs: initialized [ 0.234205] VFP support v0.3: implementor 41 architecture 4 part 30 variant f rev 0 [ 0.331721] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns [ 0.331758] futex hash table entries: 512 (order: 3, 32768 bytes) [ 0.332912] pinctrl core: initialized pinctrl subsystem [ 0.334597] _regulator_do_enable regulator-dummy [ 0.336940] NET: Registered protocol family 16 [ 0.342193] DMA: preallocated 256 KiB pool for atomic coherent allocations [ 0.344036] omap_hwmod: l3_main_3 using broken dt data from ocp [ 0.345812] omap_hwmod: l3_main_2 using broken dt data from ocp [ 1.748394] clock: dpll_abe_ck failed transition to 'locked' [ 3.047550] clock: dpll_abe_ck failed transition to 'locked' [ 4.346752] clock: dpll_abe_ck failed transition to 'locked' [ 5.645936] clock: dpll_abe_ck failed transition to 'locked' [ 5.648581] ------------[ cut here ]------------ [ 5.648604] WARNING: CPU: 0 PID: 1 at drivers/clk/clk.c:594 clk_core_disable_lock+0x18/0x24 [ 5.648611] Modules linked in: [ 5.648625] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.10.0-letux+ #851 [ 5.648632] Hardware name: Generic OMAP5 (Flattened Device Tree) [ 5.648652] [<c010f368>] (unwind_backtrace) from [<c010b914>] (show_stack+0x10/0x14) [ 5.648669] [<c010b914>] (show_stack) from [<c042ec60>] (dump_stack+0x98/0xd0) [ 5.648683] [<c042ec60>] (dump_stack) from [<c012f844>] (__warn+0xd0/0x100) [ 5.648694] [<c012f844>] (__warn) from [<c012f918>] (warn_slowpath_null+0x1c/0x24) [ 5.648707] [<c012f918>] (warn_slowpath_null) from [<c0486594>] (clk_core_disable_lock+0x18/0x24) [ 5.648722] [<c0486594>] (clk_core_disable_lock) from [<c011ca9c>] (_disable_clocks+0x18/0x70) [ 5.648735] [<c011ca9c>] (_disable_clocks) from [<c011d0f4>] (_enable+0x1e0/0x224) [ 5.648748] [<c011d0f4>] (_enable) from [<c0b0c690>] (_setup+0xc4/0x3ec) [ 5.648760] [<c0b0c690>] (_setup) from [<c011d5a0>] (omap_hwmod_for_each+0x3c/0x64) [ 5.648772] [<c011d5a0>] (omap_hwmod_for_each) from [<c0b0ccec>] (__omap_hwmod_setup_all+0x30/0x40) [ 5.648784] [<c0b0ccec>] (__omap_hwmod_setup_all) from [<c0101938>] (do_one_initcall+0xa8/0x150) [ 5.648797] [<c0101938>] (do_one_initcall) from [<c0b00d88>] (kernel_init_freeable+0x124/0x1f0) [ 5.648810] [<c0b00d88>] (kernel_init_freeable) from [<c073a994>] (kernel_init+0x8/0x110) [ 5.648822] [<c073a994>] (kernel_init) from [<c01070d0>] (ret_from_fork+0x14/0x24) [ 5.648856] ---[ end trace 158d0428f201436f ]--- [ 5.648866] omap_hwmod: dmic: _wait_target_ready failed: -16 [ 5.648878] omap_hwmod: dmic: cannot be enabled for reset (3) [ 6.974755] clock: dpll_abe_ck failed transition to 'locked' [ 8.273895] clock: dpll_abe_ck failed transition to 'locked' [ 8.276541] ------------[ cut here ]------------ [ 8.276556] WARNING: CPU: 0 PID: 1 at drivers/clk/clk.c:594 clk_core_disable_lock+0x18/0x24 [ 8.276562] Modules linked in: [ 8.276575] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G W 4.10.0-letux+ #851 [ 8.276582] Hardware name: Generic OMAP5 (Flattened Device Tree) [ 8.276595] [<c010f368>] (unwind_backtrace) from [<c010b914>] (show_stack+0x10/0x14) [ 8.276609] [<c010b914>] (show_stack) from [<c042ec60>] (dump_stack+0x98/0xd0) [ 8.276621] [<c042ec60>] (dump_stack) from [<c012f844>] (__warn+0xd0/0x100) [ 8.276631] [<c012f844>] (__warn) from [<c012f918>] (warn_slowpath_null+0x1c/0x24) [ 8.276644] [<c012f918>] (warn_slowpath_null) from [<c0486594>] (clk_core_disable_lock+0x18/0x24) ... later the kernel hangs after initializing several peripherals incl. twl6040.
So to me it looks as if some ABE clocks are not reset/restarted/locked. And this blocks logging for the first seconds (you notice the big delays for each attempt with dpll_abe_ck).
I wonder how this can be different from the OMAP5-EVM.
BR, Nikolaus
On Tue, Mar 07, 2017 at 11:19:53AM +0200, Tero Kristo wrote:
On 06/03/17 19:03, Tony Lindgren wrote:
Yup this is similar to what I recall seeing where the clocks don't always lock on reboot. Adding Tero to Cc if he has a fix in mind for this one.
Haven't faced this issue myself, but ABE is notorious for this kind of issues I recall, due to custom clocking setup for that specific DPLL. Are you tweaking the source of that DPLL?
I don't think we are, not on purpose anyway :)
I can give omap5uevm a shot at some point and see what happens.
I'd offer to try it on my pyra prototype, but I'm still trying to coax it around ddr3 hw leveling failures. Apparently having 6 out of 8 bytelanes working is not enough ;-)
Matthijs
On Tue, Mar 07, 2017 at 11:19:53AM +0200, Tero Kristo wrote:
On 06/03/17 19:03, Tony Lindgren wrote:
Yup this is similar to what I recall seeing where the clocks don't always lock on reboot.
ok, I've gotten the locking failures too, and managed to get a snapshot of some registers after warm reset and compared them to cold reset.
Haven't faced this issue myself, but ABE is notorious for this kind of issues I recall, due to custom clocking setup for that specific DPLL. Are you tweaking the source of that DPLL?
No differences in dpll_abe refclk or other settings in CKGEN_PRM.
CM_ABE_CLKSTCTRL is showing clock activity for: DPLL_ABE_X2_CLK ABE_GICLK ABE_24M_GFCLK no other differences within the ABE_CM register block
dpll_abe control is requesting low-power idle-bypass, which is also the reset default. The differences are that the (reset insensitive) multiplier is set to 560, and the status register says: bit 0: 0 DPLL is not locked bits 1-3: 0 transient state bit 4: 1 DPLL has been initialized
Clearly it can't be good that the PLL is being observed in transient state. It seems the PLL is "stuck" somehow.
Note that this doesn't always seem to happen, often enough the status reports it is in "low power stop", same as after cold reset.
If I manage to catch it in this state again while in u-boot I'll see if I can unwedge it by meddling with registers. Suggestions are welcome.
Matthijs
On Fri, Mar 03, 2017 at 07:45:45PM +0100, H. Nikolaus Schaller wrote:
Am 03.03.2017 um 18:05 schrieb Tony Lindgren tony@atomide.com:
OK recall seeing igepv5 reboot not always work or the omap5-uevm reboot always not work depending on the u-boot version. Not sure if that is still happening but that hints to something not being properly re-initialized by the kernel for clocks.
I have just tried and my EVM seems now to properly "reboot" (with Linux-4.10.0 and SPL/U-Boot-2016.11) while I still remember this problem existed earlier.
Caution: the omap5-uevm has been wired to turn every warm reset into a full hwreset of palmas by default, which power cycles the omap5. This means that it has really very little excuse to fail to reboot (unless it's simply stochastic on every boot and not just reboot).
To allow palmas and omap5 to retain settings across warm reset (as they would in a mobile device), install 0 ohm resistor R195 (on bottom side, in the general neighbourhood of palmas' belly and power measurement connector J19). At your own risk, I have not tried this myself yet. :)
I have no idea what the IGEPv5 does, I don't have access to one nor its schematic. Check the device reset status register after warm reset: root:~# omapconf read 0x4ae07c04 00000001 This indicates a cold/power-on reset occurred.
So to me it looks as if some ABE clocks are not reset/restarted/locked.
Which is still strange... although ABE does seem to be somewhat special, you'd expect that either linux configures the clock source on each boot, or it never does but then it also doesn't matter whether that gets reset or not.
(Hmm, for some reason I'm suddenly thinking it's going to be fun when it's time to get device low power modes working...)
And this blocks logging for the first seconds (you notice the big delays for each attempt with dpll_abe_ck).
Note: to get real-time log output instead of a long delay after "Starting kernel", enable earlycon / early printk (is there a difference?). It's working on the uevm with my config: https://github.com/mvduin/linux/blob/work/merged/arch/arm/configs/my_omap5_d... relevant settings are probably among these: CONFIG_SERIAL_EARLYCON=y CONFIG_SERIAL_8250=y CONFIG_SERIAL_8250_CONSOLE=y CONFIG_SERIAL_8250_OMAP=y CONFIG_SERIAL_8250_OMAP_TTYO_FIXUP=y CONFIG_SERIAL_OF_PLATFORM=y CONFIG_SERIAL_CORE=y CONFIG_SERIAL_CORE_CONSOLE=y # CONFIG_SERIAL_OMAP is not set CONFIG_DEBUG_LL=y CONFIG_DEBUG_OMAP4UART3=y CONFIG_DEBUG_OMAP2PLUS_UART=y CONFIG_EARLY_PRINTK=y
I wonder how this can be different from the OMAP5-EVM.
The uEVM took the lazy option by making every warm reset into a full palmas+omap5 hard reset instead of fixing warm reset issues.
Matthijs
* Matthijs van Duin matthijsvanduin@gmail.com [170303 05:21]:
Like its predecessors, the omap5 has a special bank of I/Os for interfacing with sdcards, which (unlike other I/O) may operate at 3V in addition to 1.8V, with support for switching at runtime.
Unfortunately the designated LDO for this supply in the PMIC (Palmas), ldo9, can only supply 50 mA while cards are known to require more than that in practice. The omap5-uevm solved this by using a separate 3.3v supply for the sdcard, but disturbingly they continued to use ldo9 for the I/O supply on the omap5...
The device tree right now permits ldo9 to be lowered to 1.8V, unaware of the fact that the sdcard will continue to drive 3.3V signals into the now very unhappy I/Os of the omap5 !
Hence, the most urgent fix would be:
&ldo9_reg { regulator-min-microvolt = <3000000>; };
Sounds like we should do that ASAP. Care to send a patch for that?
To make things worse, according to the Recommended Operating Conditions in the omap5 datasheet, vdds_sdcard should not be operated at 3.3V at all but 3.0V nominal (3.06V MAX DC).
This means you have to choose between:
- configure ldo9 in bypass. Both sides use 3.3v, but this exceeds the max value specified by the omap5 datasheet.
- configure ldo9 at 3.0V. The sdcard will be driving signals 0.3v higher than the omap5's supply (+ overshoot, if any).
Pick your poison.
Afaik bypass is the power-up config, while u-boot changes it to 3.0V. I'm not sure what linux does currently since omap5-board-common.dtsi incorrectly claims that &vmmcsd_fixed is 3.0V instead of 3.3V.
I'm guessing that ldo9 at 3.0V is indeed the intended modus operandi.
OK
(side-note: ldo8 can provide 200 mA, which would have been more than enough, and is not connected to anything whatsoever... nicely done)
A final problem I noticed is power switching. The DT as-is implies that the sdcard can be powered off by switching ldo9 but this is plain wrong. Powering down the sdcard should be done by switching &vmmcsd_fixed via REGEN3. An obstacle is that eMMC (&mmc2) uses it as supply, but fortunately DT is simply wrong: eMMC is powered directly from the main 3.3v supply.
So, roughly:
vcc_3v3_main: fixedregulator-vcc_3v3_main { compatible = "regulator-fixed"; regulator-name = "vcc_3v3_main"; vin-supply = <&vmain>; regulator-min-microvolt = <3300000>; regulator-max-microvolt = <3300000>; };
&vmmcsd_fixed { /* called vcc_3v3_sdio on omap5-uevm */ vin-supply = <&vcc_3v3_main>; regulator-min-microvolt = <3300000>; regulator-max-microvolt = <3300000>; gpio = /* FIXME palmas REGEN3 */; };
&mmc1 { /* card supply is actually &vmmcsd_fixed */ vmmc-supply = <&ldo9_reg>; };
&mmc2 { vmmc-supply = <&vcc_3v3_main>; };
The details remaining are how to actually control REGEN3, and how to make sure linux leaves ldo9 alone and switches the supply just using &vmmcsd_fixed. I'm hoping someone who understands this stuff better than me has a suggestion?
I guess for one option is making ldo9 "regulator-always-on" and "regulator-boot-on" if you want Linux not touch it. If you set it with status = "disabled" then no struct device is ever created for it. But isn't the vcc_3v3_main also input for ldo9?
For measuring, you should be able to configure regen3 pin as GPIO regulator with palmas gpio2, then toggle it via /sys/class/gpio/.
The earlier PMICs would allow controlling REGEN pins(s) manually or via software loaded to the PMIC based on SoC PM state changes. Not sure what twl6030 has for that.
For your example above, if REGEN3 is already configured as a regulator in palmas-regulator.c, maybe all you have to do is:
regen3_reg: regen3 { regulator-name = "regen3"; regulator-min-microvolt = <3300000>; regulator-max-microvolt = <3300000>; };
&mmc1 { /* card supply is actually &vmmcsd_fixed */ vmmc-supply = <&ldo9_reg>; vmmc_aux-supply = <®en3_reg>; };
&mmc2 { vmmc-supply = <&vcc_3v3_main>; };
If there are other users of REGEN3 or regen3_reg they would need to be mapped too as with both cards powered off the use count goes to 0.
Regards,
Tony
On 3 March 2017 at 17:19, Tony Lindgren tony@atomide.com wrote:
Sounds like we should do that ASAP. Care to send a patch for that?
Never mind!
I'm more familiar with (e)MMC than with SD and incorrectly assumed the card supply voltage is also its I/O voltage, but apparently an SD card is required to generate its I/O supply voltage internally, and this is switched to 1.8V by software command.
Now it all makes much more sense
main 3.3V ---[REGEN3]---> switched 3.3V to sdcard and LDO9 sdcard 3.3V ---[LDO9]---> host-side I/O supply sdcard 3.3V ---[card-internal LDO]---> card-side I/O supply
Sorry for the false alarm.
There are still things that should be fixed to accurately reflect the schematics, including the use of REGEN3 to be able to power-cycle the card, but they are less urgent now and can wait until I've had time to test them.
Matthijs
* Matthijs van Duin matthijsvanduin@gmail.com [170306 10:26]:
On 3 March 2017 at 17:19, Tony Lindgren tony@atomide.com wrote:
Sounds like we should do that ASAP. Care to send a patch for that?
Never mind!
I'm more familiar with (e)MMC than with SD and incorrectly assumed the card supply voltage is also its I/O voltage, but apparently an SD card is required to generate its I/O supply voltage internally, and this is switched to 1.8V by software command.
Now it all makes much more sense
main 3.3V ---[REGEN3]---> switched 3.3V to sdcard and LDO9 sdcard 3.3V ---[LDO9]---> host-side I/O supply sdcard 3.3V ---[card-internal LDO]---> card-side I/O supply
Sorry for the false alarm.
OK good to hear there's no urgent need for fixes.
There are still things that should be fixed to accurately reflect the schematics, including the use of REGEN3 to be able to power-cycle the card, but they are less urgent now and can wait until I've had time to test them.
OK
Regards,
Tony
-
H. Nikolaus Schaller -
Matthijs van Duin -
Michael Mrozek -
Tero Kristo -
Tony Lindgren