Hi, unfortunately every year or so some upstream change breaks our (still out-of tree) DSI panel driver for the Pyra(Phone). This time it happened with v5.2-rc1.
v5.2-rc1 also broke the GTA04 display and the L3704/L7004 displays, but those are DPI panels and the reasons were two separate issues which have nothing to do with the Pyra DSI panel.
There have been some API changes for DSI panel drivers and I have done a
git diff v5.2-rc1 v5.1 drivers/gpu/drm/omapdrm/displays/panel-dsi-cm.c
to identify them and tried to apply similar patches to our panel. But still the driver is successfully probed, but shows not activity. No calls to enable() or connect(). Backlight stays black. And as a result there is also no /dev/fb0, text console or X11.
Unfortunately, it has not improved after upgrading to v5.2-rc2.
So once again it appears that I have to git bisect upstream patches to find out what is going on...
Maybe I should look for more attractive and innovative projects than hunting such bugs in my spare time. Or look for someone who takes care of maintaining the panel driver...
Anyways, ideas what could be wrong or missing are welcome.
I have pushed all code to the pyra-minimal tree so that it is just a handful of commits on top of v5.2-rc2 (can be compiled with omap2plus_defconfig as well).
http://git.goldelico.com/?p=letux-kernel.git;a=shortlog;h=refs/heads/letux/p...
BR, Nikolaus
Am 27.05.2019 um 14:46 schrieb H. Nikolaus Schaller hns@goldelico.com:
Hi, unfortunately every year or so some upstream change breaks our (still out-of tree) DSI panel driver for the Pyra(Phone). This time it happened with v5.2-rc1.
v5.2-rc1 also broke the GTA04 display and the L3704/L7004 displays, but those are DPI panels and the reasons were two separate issues which have nothing to do with the Pyra DSI panel.
There have been some API changes for DSI panel drivers and I have done a
git diff v5.2-rc1 v5.1 drivers/gpu/drm/omapdrm/displays/panel-dsi-cm.c
to identify them and tried to apply similar patches to our panel. But still the driver is successfully probed, but shows not activity. No calls to enable() or connect(). Backlight stays black. And as a result there is also no /dev/fb0, text console or X11.
Unfortunately, it has not improved after upgrading to v5.2-rc2.
So once again it appears that I have to git bisect upstream patches to find out what is going on...
After spending more hours... it does not seem to be in our patches and the panel driver is likely almost correct now.
I believe this because I also found not working: * dwc3 (usb3/otg port) * bq24297 battery charger driver (my Pyra fortunately runs well enough with U-Boot initialization) * tca6424 (gpio expander - we use the pcal9524 variant)
The tca6424 driver seems to be broken (not loaded).
And since it is used for interrupt detection of the bq24297, the charger driver defers probing.
Because the charger driver does not load, there is no otg booster to successfully probe dwc3.
So this looks like a clear failure chain.
Additionally, the tca6424 is used for the HDMI port and maybe a missing tca6424 is sufficient to stop omapdss initialization although probed.
So far this analysis.
Help (at least discussion) is really welcome because I don't know how much voluntary time I can and want to invest into such bugs.
BR and thanks, Nikolaus
On Mon, 27 May 2019 21:25:41 +0200 "H. Nikolaus Schaller" hns@goldelico.com wrote:
[...]
So this looks like a clear failure chain.
Additionally, the tca6424 is used for the HDMI port and maybe a missing tca6424 is sufficient to stop omapdss initialization although probed.
so for testing, we can #ifdef out the hdmi parts of the devicetree. For gta04 mainline testing I have also done such a thing with the dac in the period when there was something broken there.
Regards, Andreas
On Mon, 27 May 2019 21:25:41 +0200 "H. Nikolaus Schaller" hns@goldelico.com wrote:
Am 27.05.2019 um 14:46 schrieb H. Nikolaus Schaller hns@goldelico.com:
Hi, unfortunately every year or so some upstream change breaks our (still out-of tree) DSI panel driver for the Pyra(Phone). This time it happened with v5.2-rc1.
v5.2-rc1 also broke the GTA04 display and the L3704/L7004 displays, but those are DPI panels and the reasons were two separate issues which have nothing to do with the Pyra DSI panel.
There have been some API changes for DSI panel drivers and I have done a
git diff v5.2-rc1 v5.1 drivers/gpu/drm/omapdrm/displays/panel-dsi-cm.c
to identify them and tried to apply similar patches to our panel. But still the driver is successfully probed, but shows not activity. No calls to enable() or connect(). Backlight stays black. And as a result there is also no /dev/fb0, text console or X11.
Unfortunately, it has not improved after upgrading to v5.2-rc2.
So once again it appears that I have to git bisect upstream patches to find out what is going on...
After spending more hours... it does not seem to be in our patches and the panel driver is likely almost correct now.
I believe this because I also found not working:
- dwc3 (usb3/otg port)
- bq24297 battery charger driver (my Pyra fortunately runs well enough with U-Boot initialization)
- tca6424 (gpio expander - we use the pcal9524 variant)
The tca6424 driver seems to be broken (not loaded).
[ 6.581432] ts3a227e 4-003b: Cannot request irq 0 (-22) [ 6.597252] ts3a227e: probe of 4-003b failed with error -22
so somehow it insists of having an irq and we are not providing one in the dtb, at least not in pyra-mainboard-v5.1.dtsi
Regards, Andreas
Am 28.05.2019 um 23:40 schrieb Andreas Kemnade andreas@kemnade.info:
On Mon, 27 May 2019 21:25:41 +0200 "H. Nikolaus Schaller" hns@goldelico.com wrote:
Am 27.05.2019 um 14:46 schrieb H. Nikolaus Schaller hns@goldelico.com:
Hi, unfortunately every year or so some upstream change breaks our (still out-of tree) DSI panel driver for the Pyra(Phone). This time it happened with v5.2-rc1.
v5.2-rc1 also broke the GTA04 display and the L3704/L7004 displays, but those are DPI panels and the reasons were two separate issues which have nothing to do with the Pyra DSI panel.
There have been some API changes for DSI panel drivers and I have done a
git diff v5.2-rc1 v5.1 drivers/gpu/drm/omapdrm/displays/panel-dsi-cm.c
to identify them and tried to apply similar patches to our panel. But still the driver is successfully probed, but shows not activity. No calls to enable() or connect(). Backlight stays black. And as a result there is also no /dev/fb0, text console or X11.
Unfortunately, it has not improved after upgrading to v5.2-rc2.
So once again it appears that I have to git bisect upstream patches to find out what is going on...
After spending more hours... it does not seem to be in our patches and the panel driver is likely almost correct now.
I believe this because I also found not working:
- dwc3 (usb3/otg port)
- bq24297 battery charger driver (my Pyra fortunately runs well enough with U-Boot initialization)
- tca6424 (gpio expander - we use the pcal9524 variant)
The tca6424 driver seems to be broken (not loaded).
[ 6.581432] ts3a227e 4-003b: Cannot request irq 0 (-22) [ 6.597252] ts3a227e: probe of 4-003b failed with error -22
so somehow it insists of having an irq and we are not providing one in the dtb, at least not in pyra-mainboard-v5.1.dtsi
The reference is set in pyra-mainboard-v5.2.dtsi
interrupt-parent = <&gpio99>; interrupts = <14 IRQ_TYPE_EDGE_RISING>;
But: this is another symptom of the tca6424 (gpio99) not being available. It is not only a gpio expander but also an interrupt expander.
So we should focus on finding out why the tca6424 / pcal6524 does not probe.
BR and thanks, Nikolaus
On Wed, 29 May 2019 06:39:22 +0200 "H. Nikolaus Schaller" hns@goldelico.com wrote:
Am 28.05.2019 um 23:40 schrieb Andreas Kemnade andreas@kemnade.info:
On Mon, 27 May 2019 21:25:41 +0200 "H. Nikolaus Schaller" hns@goldelico.com wrote:
Am 27.05.2019 um 14:46 schrieb H. Nikolaus Schaller hns@goldelico.com:
Hi, unfortunately every year or so some upstream change breaks our (still out-of tree) DSI panel driver for the Pyra(Phone). This time it happened with v5.2-rc1.
v5.2-rc1 also broke the GTA04 display and the L3704/L7004 displays, but those are DPI panels and the reasons were two separate issues which have nothing to do with the Pyra DSI panel.
There have been some API changes for DSI panel drivers and I have done a
git diff v5.2-rc1 v5.1 drivers/gpu/drm/omapdrm/displays/panel-dsi-cm.c
to identify them and tried to apply similar patches to our panel. But still the driver is successfully probed, but shows not activity. No calls to enable() or connect(). Backlight stays black. And as a result there is also no /dev/fb0, text console or X11.
Unfortunately, it has not improved after upgrading to v5.2-rc2.
So once again it appears that I have to git bisect upstream patches to find out what is going on...
After spending more hours... it does not seem to be in our patches and the panel driver is likely almost correct now.
I believe this because I also found not working:
- dwc3 (usb3/otg port)
- bq24297 battery charger driver (my Pyra fortunately runs well enough with U-Boot initialization)
- tca6424 (gpio expander - we use the pcal9524 variant)
The tca6424 driver seems to be broken (not loaded).
[ 6.581432] ts3a227e 4-003b: Cannot request irq 0 (-22) [ 6.597252] ts3a227e: probe of 4-003b failed with error -22
so somehow it insists of having an irq and we are not providing one in the dtb, at least not in pyra-mainboard-v5.1.dtsi
The reference is set in pyra-mainboard-v5.2.dtsi
... which is as far as I understand not used on a 5.1 board.
interrupt-parent = <&gpio99>; interrupts = <14 IRQ_TYPE_EDGE_RISING>;But: this is another symptom of the tca6424 (gpio99) not being available. It is not only a gpio expander but also an interrupt expander.
so it an independant problem.
So we should focus on finding out why the tca6424 / pcal6524 does not probe.
yes, I disabled the hdmi stuff completely. Result: Display works.
Regards, Andreas
Am 29.05.2019 um 07:16 schrieb Andreas Kemnade andreas@kemnade.info:
On Wed, 29 May 2019 06:39:22 +0200 "H. Nikolaus Schaller" hns@goldelico.com wrote:
Am 28.05.2019 um 23:40 schrieb Andreas Kemnade andreas@kemnade.info:
On Mon, 27 May 2019 21:25:41 +0200 "H. Nikolaus Schaller" hns@goldelico.com wrote:
Am 27.05.2019 um 14:46 schrieb H. Nikolaus Schaller hns@goldelico.com:
Hi, unfortunately every year or so some upstream change breaks our (still out-of tree) DSI panel driver for the Pyra(Phone). This time it happened with v5.2-rc1.
v5.2-rc1 also broke the GTA04 display and the L3704/L7004 displays, but those are DPI panels and the reasons were two separate issues which have nothing to do with the Pyra DSI panel.
There have been some API changes for DSI panel drivers and I have done a
git diff v5.2-rc1 v5.1 drivers/gpu/drm/omapdrm/displays/panel-dsi-cm.c
to identify them and tried to apply similar patches to our panel. But still the driver is successfully probed, but shows not activity. No calls to enable() or connect(). Backlight stays black. And as a result there is also no /dev/fb0, text console or X11.
Unfortunately, it has not improved after upgrading to v5.2-rc2.
So once again it appears that I have to git bisect upstream patches to find out what is going on...
After spending more hours... it does not seem to be in our patches and the panel driver is likely almost correct now.
I believe this because I also found not working:
- dwc3 (usb3/otg port)
- bq24297 battery charger driver (my Pyra fortunately runs well enough with U-Boot initialization)
- tca6424 (gpio expander - we use the pcal9524 variant)
The tca6424 driver seems to be broken (not loaded).
[ 6.581432] ts3a227e 4-003b: Cannot request irq 0 (-22) [ 6.597252] ts3a227e: probe of 4-003b failed with error -22
so somehow it insists of having an irq and we are not providing one in the dtb, at least not in pyra-mainboard-v5.1.dtsi
The reference is set in pyra-mainboard-v5.2.dtsi
... which is as far as I understand not used on a 5.1 board.
interrupt-parent = <&gpio99>; interrupts = <14 IRQ_TYPE_EDGE_RISING>;But: this is another symptom of the tca6424 (gpio99) not being available. It is not only a gpio expander but also an interrupt expander.
so it an independant problem.
So we should focus on finding out why the tca6424 / pcal6524 does not probe.
yes, I disabled the hdmi stuff completely. Result: Display works.
Ok, fine! So we at least have an idea that the broken display is this time just a symptom and not a bug...
Strangely I do not see a printk() added to pca953x_probe()...
BR and thanks, Nikolaus
Am 29.05.2019 um 07:55 schrieb H. Nikolaus Schaller hns@goldelico.com:
Am 29.05.2019 um 07:16 schrieb Andreas Kemnade andreas@kemnade.info:
On Wed, 29 May 2019 06:39:22 +0200 "H. Nikolaus Schaller" hns@goldelico.com wrote:
Am 28.05.2019 um 23:40 schrieb Andreas Kemnade andreas@kemnade.info:
On Mon, 27 May 2019 21:25:41 +0200 "H. Nikolaus Schaller" hns@goldelico.com wrote:
Am 27.05.2019 um 14:46 schrieb H. Nikolaus Schaller hns@goldelico.com:
Hi, unfortunately every year or so some upstream change breaks our (still out-of tree) DSI panel driver for the Pyra(Phone). This time it happened with v5.2-rc1.
v5.2-rc1 also broke the GTA04 display and the L3704/L7004 displays, but those are DPI panels and the reasons were two separate issues which have nothing to do with the Pyra DSI panel.
There have been some API changes for DSI panel drivers and I have done a
git diff v5.2-rc1 v5.1 drivers/gpu/drm/omapdrm/displays/panel-dsi-cm.c
to identify them and tried to apply similar patches to our panel. But still the driver is successfully probed, but shows not activity. No calls to enable() or connect(). Backlight stays black. And as a result there is also no /dev/fb0, text console or X11.
Unfortunately, it has not improved after upgrading to v5.2-rc2.
So once again it appears that I have to git bisect upstream patches to find out what is going on...
After spending more hours... it does not seem to be in our patches and the panel driver is likely almost correct now.
I believe this because I also found not working:
- dwc3 (usb3/otg port)
- bq24297 battery charger driver (my Pyra fortunately runs well enough with U-Boot initialization)
- tca6424 (gpio expander - we use the pcal9524 variant)
The tca6424 driver seems to be broken (not loaded).
[ 6.581432] ts3a227e 4-003b: Cannot request irq 0 (-22) [ 6.597252] ts3a227e: probe of 4-003b failed with error -22
so somehow it insists of having an irq and we are not providing one in the dtb, at least not in pyra-mainboard-v5.1.dtsi
The reference is set in pyra-mainboard-v5.2.dtsi
... which is as far as I understand not used on a 5.1 board.
interrupt-parent = <&gpio99>; interrupts = <14 IRQ_TYPE_EDGE_RISING>;But: this is another symptom of the tca6424 (gpio99) not being available. It is not only a gpio expander but also an interrupt expander.
so it an independant problem.
So we should focus on finding out why the tca6424 / pcal6524 does not probe.
yes, I disabled the hdmi stuff completely. Result: Display works.
Ok, fine! So we at least have an idea that the broken display is this time just a symptom and not a bug...
Strangely I do not see a printk() added to pca953x_probe()...
Well, it turned out that my dmesg buffer was too small to find it after booting and login.
Here is something related:
[ 7.457864] omap_i2c 4807a000.i2c: bus 3 rev0.12 at 400 kHz [ 7.463749] bus: 'platform': really_probe: bound device 4807a000.i2c to driver omap_i2c [ 7.472247] bus: 'platform': driver_probe_device: matched device 4807c000.i2c with driver omap_i2c [ 7.481649] really_probe: driver omap_i2c [ 7.485860] bus: 'platform': really_probe: probing driver omap_i2c with device 4807c000.i2c [ 7.495402] bus: 'i2c': driver_probe_device: matched device 4-0022 with driver pca953x [ 7.503697] really_probe: driver pca953x [ 7.507827] bus: 'i2c': really_probe: probing driver pca953x with device 4-0022 [ 7.515531] pca953x_probe: tca6424 [ 7.519129] vdds_1v8_main: supplied by smps7 [ 7.524125] pca953x 4-0022: failed writing register [ 7.529444] pca953x: probe of 4-0022 failed with error -22 [ 7.535545] omap_i2c 4807c000.i2c: bus 4 rev0.12 at 400 kHz [ 7.541420] bus: 'platform': really_probe: bound device 4807c000.i2c to driver omap_i2c [ 7.549934] bus: 'platform': driver_probe_device: matched device 40132000.target-module with driver ti-sysc
So this maybe has a different reason. Maybe, the tca6424 is kept in RESET state?
No. It is responding:
root@letux:~# i2cdetect -r -y 4 0 1 2 3 4 5 6 7 8 9 a b c d e f 00: -- -- -- -- -- -- -- -- -- -- -- -- -- 10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 20: -- -- 22 -- -- -- -- -- -- -- -- -- -- -- -- -- 30: -- -- -- -- -- -- -- -- -- -- -- 3b -- -- -- -- 40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 60: -- -- -- -- UU -- -- -- -- -- -- -- -- -- -- -- 70: -- -- -- -- -- -- -- -- root@letux:~# i2cdump -y 4 0x22 No size specified (using byte-data access) 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef 00: 00 48 02 XX 00 00 00 XX 00 00 00 XX 00 48 0f XX .H?X...X...X.H?X 10: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 20: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 30: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 40: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 50: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 60: XX XX XX XX XX XX ^C root@letux:~#
Well, this are not the real register values since the tca6424 uses a "pointer register" that must be written before reading.
It is like an adventure game to find out what is going wrong...
BR, Nikolaus
Am 29.05.2019 um 09:05 schrieb H. Nikolaus Schaller hns@goldelico.com:
Am 29.05.2019 um 07:55 schrieb H. Nikolaus Schaller hns@goldelico.com:
Am 29.05.2019 um 07:16 schrieb Andreas Kemnade andreas@kemnade.info:
On Wed, 29 May 2019 06:39:22 +0200 "H. Nikolaus Schaller" hns@goldelico.com wrote:
Am 28.05.2019 um 23:40 schrieb Andreas Kemnade andreas@kemnade.info:
On Mon, 27 May 2019 21:25:41 +0200 "H. Nikolaus Schaller" hns@goldelico.com wrote:
> Am 27.05.2019 um 14:46 schrieb H. Nikolaus Schaller hns@goldelico.com: > > Hi, > unfortunately every year or so some upstream change breaks > our (still out-of tree) DSI panel driver for the Pyra(Phone). > This time it happened with v5.2-rc1. > > v5.2-rc1 also broke the GTA04 display and the L3704/L7004 displays, > but those are DPI panels and the reasons were two separate > issues which have nothing to do with the Pyra DSI panel. > > There have been some API changes for DSI panel drivers and > I have done a > > git diff v5.2-rc1 v5.1 drivers/gpu/drm/omapdrm/displays/panel-dsi-cm.c > > to identify them and tried to apply similar patches to our > panel. But still the driver is successfully probed, but shows > not activity. No calls to enable() or connect(). Backlight stays > black. And as a result there is also no /dev/fb0, text console > or X11. > > Unfortunately, it has not improved after upgrading to v5.2-rc2. > > So once again it appears that I have to git bisect upstream > patches to find out what is going on...
After spending more hours... it does not seem to be in our patches and the panel driver is likely almost correct now.
I believe this because I also found not working:
- dwc3 (usb3/otg port)
- bq24297 battery charger driver (my Pyra fortunately runs well enough with U-Boot initialization)
- tca6424 (gpio expander - we use the pcal9524 variant)
The tca6424 driver seems to be broken (not loaded).
[ 6.581432] ts3a227e 4-003b: Cannot request irq 0 (-22) [ 6.597252] ts3a227e: probe of 4-003b failed with error -22
so somehow it insists of having an irq and we are not providing one in the dtb, at least not in pyra-mainboard-v5.1.dtsi
The reference is set in pyra-mainboard-v5.2.dtsi
... which is as far as I understand not used on a 5.1 board.
interrupt-parent = <&gpio99>; interrupts = <14 IRQ_TYPE_EDGE_RISING>;But: this is another symptom of the tca6424 (gpio99) not being available. It is not only a gpio expander but also an interrupt expander.
so it an independant problem.
So we should focus on finding out why the tca6424 / pcal6524 does not probe.
yes, I disabled the hdmi stuff completely. Result: Display works.
Ok, fine! So we at least have an idea that the broken display is this time just a symptom and not a bug...
Strangely I do not see a printk() added to pca953x_probe()...
Well, it turned out that my dmesg buffer was too small to find it after booting and login.
Here is something related:
[ 7.457864] omap_i2c 4807a000.i2c: bus 3 rev0.12 at 400 kHz [ 7.463749] bus: 'platform': really_probe: bound device 4807a000.i2c to driver omap_i2c [ 7.472247] bus: 'platform': driver_probe_device: matched device 4807c000.i2c with driver omap_i2c [ 7.481649] really_probe: driver omap_i2c [ 7.485860] bus: 'platform': really_probe: probing driver omap_i2c with device 4807c000.i2c [ 7.495402] bus: 'i2c': driver_probe_device: matched device 4-0022 with driver pca953x [ 7.503697] really_probe: driver pca953x [ 7.507827] bus: 'i2c': really_probe: probing driver pca953x with device 4-0022 [ 7.515531] pca953x_probe: tca6424 [ 7.519129] vdds_1v8_main: supplied by smps7 [ 7.524125] pca953x 4-0022: failed writing register [ 7.529444] pca953x: probe of 4-0022 failed with error -22 [ 7.535545] omap_i2c 4807c000.i2c: bus 4 rev0.12 at 400 kHz [ 7.541420] bus: 'platform': really_probe: bound device 4807c000.i2c to driver omap_i2c [ 7.549934] bus: 'platform': driver_probe_device: matched device 40132000.target-module with driver ti-sysc
So this maybe has a different reason. Maybe, the tca6424 is kept in RESET state?
No. It is responding:
root@letux:~# i2cdetect -r -y 4 0 1 2 3 4 5 6 7 8 9 a b c d e f 00: -- -- -- -- -- -- -- -- -- -- -- -- -- 10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 20: -- -- 22 -- -- -- -- -- -- -- -- -- -- -- -- -- 30: -- -- -- -- -- -- -- -- -- -- -- 3b -- -- -- -- 40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 60: -- -- -- -- UU -- -- -- -- -- -- -- -- -- -- -- 70: -- -- -- -- -- -- -- -- root@letux:~# i2cdump -y 4 0x22 No size specified (using byte-data access) 0 1 2 3 4 5 6 7 8 9 a b c d e f 0123456789abcdef 00: 00 48 02 XX 00 00 00 XX 00 00 00 XX 00 48 0f XX .H?X...X...X.H?X 10: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 20: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 30: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 40: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 50: XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XX XXXXXXXXXXXXXXXX 60: XX XX XX XX XX XX ^C root@letux:~#
Well, this are not the real register values since the tca6424 uses a "pointer register" that must be written before reading.
It is like an adventure game to find out what is going wrong...
Next finding using linus/v5.2-rc2 and omap2plus_defconfig on the OMAP5 EVM:
[ 5.212678] reg-fixed-voltage modem_power_reg: No GPIO consumer (null) found [ 5.212998] pca953x 3-0022: GPIO lookup for consumer reset [ 5.223364] at24 0-0050: 256 byte 24c02 EEPROM, writable, 1 bytes/write [ 5.225311] pca953x 3-0022: using device tree for GPIO lookup [ 5.237702] of_get_named_gpiod_flags: can't parse 'reset-gpios' property of node '/ocp/interconnect@48000000/segment@0/target-module@7a000/i2c@0/tca6424@22[0]' [ 5.238956] omap_hsmmc 480ad000.mmc: GPIO lookup for consumer wp [ 5.252040] of_get_named_gpiod_flags: can't parse 'reset-gpio' property of node '/ocp/interconnect@48000000/segment@0/target-module@7a000/i2c@0/tca6424@22[0]' [ 5.252047] pca953x 3-0022: using lookup tables for GPIO lookup [ 5.258074] omap_hsmmc 480ad000.mmc: using device tree for GPIO lookup [ 5.272308] pca953x 3-0022: No GPIO consumer reset found [ 5.278259] of_get_named_gpiod_flags: can't parse 'wp-gpios' property of node '/ocp/interconnect@48000000/segment@0/target-module@ad000/mmc@0[0]' [ 5.284961] pca953x 3-0022: failed writing register [ 5.290141] of_get_named_gpiod_flags: can't parse 'wp-gpio' property of node '/ocp/interconnect@48000000/segment@0/target-module@ad000/mmc@0[0]'
So there is also failure:
[ 5.284961] pca953x 3-0022: failed writing register
We are coming closer... Something in the regmap or i2c or pca953x address calculation is going wrong.
BR, Nikolaus
Hi,
* H. Nikolaus Schaller hns@goldelico.com [190529 08:57]:
[ 7.524125] pca953x 4-0022: failed writing register [ 7.529444] pca953x: probe of 4-0022 failed with error -22
I saw something similar with a BBB cape (early Baylibre ACME cape) and it was device_pca95xx_init() somehow trying to write to a read-only register causing an error.
I did not debug further and I do not have it handy, and I could not easily figure out what caused the regression.. But maybe take a look and see if commenting out the register write device_pca95xx_init() helps.
Regards,
Tony
On Wed, 29 May 2019 02:12:27 -0700 Tony Lindgren tony@atomide.com wrote:
Hi,
- H. Nikolaus Schaller hns@goldelico.com [190529 08:57]:
[ 7.524125] pca953x 4-0022: failed writing register [ 7.529444] pca953x: probe of 4-0022 failed with error -22
I saw something similar with a BBB cape (early Baylibre ACME cape) and it was device_pca95xx_init() somehow trying to write to a read-only register causing an error.
I did not debug further and I do not have it handy, and I could not easily figure out what caused the regression.. But maybe take a look and see if commenting out the register write device_pca95xx_init() helps.
root@letux:/sys/bus/i2c/drivers/pca953x# echo 4-0022 >bind [ 176.917013] pca953x 4-0022: writing reg 02 -> 88 = 00 -> apparently inversion register
[ 176.922460] pca953x 4-0022: failed writing register: 02 [ 176.929805] pca953x: probe of 4-0022 failed with error -22 -bash: echo: write error: No such device root@letux:/sys/bus/i2c/drivers/pca953x# i2cset -y 4 0x22 0x88 0 root@letux:/sys/bus/i2c/drivers/pca953x# i2cset -y 4 0x22 0x2 0 root@letux:/sys/bus/i2c/drivers/pca953x#
debug output by this patch: diff --git a/drivers/gpio/gpio-pca953x.c b/drivers/gpio/gpio-pca953x.c index b7ef33f63392..161c5d5ee8c8 100644 --- a/drivers/gpio/gpio-pca953x.c +++ b/drivers/gpio/gpio-pca953x.c @@ -338,10 +338,11 @@ static int pca953x_write_regs(struct pca953x_chip *chip, int reg, u8 *val) { u8 regaddr = pca953x_recalc_addr(chip, reg, 0, true, true); int ret; - + + dev_info(&chip->client->dev, "writing reg %02x -> %02x = %02x", reg, regaddr, *val); ret = regmap_bulk_write(chip->regmap, regaddr, val, NBANK(chip)); if (ret < 0) { - dev_err(&chip->client->dev, "failed writing register\n"); + dev_err(&chip->client->dev, "failed writing register: %02x\n", reg); return ret; }
perhaps this rings a bell for someone.
Regards, Andreas
On Wed, 29 May 2019 19:51:21 +0200 Andreas Kemnade andreas@kemnade.info wrote:
On Wed, 29 May 2019 02:12:27 -0700 Tony Lindgren tony@atomide.com wrote:
Hi,
- H. Nikolaus Schaller hns@goldelico.com [190529 08:57]:
[ 7.524125] pca953x 4-0022: failed writing register [ 7.529444] pca953x: probe of 4-0022 failed with error -22
I saw something similar with a BBB cape (early Baylibre ACME cape) and it was device_pca95xx_init() somehow trying to write to a read-only register causing an error.
I did not debug further and I do not have it handy, and I could not easily figure out what caused the regression.. But maybe take a look and see if commenting out the register write device_pca95xx_init() helps.
root@letux:/sys/bus/i2c/drivers/pca953x# echo 4-0022 >bind [ 176.917013] pca953x 4-0022: writing reg 02 -> 88 = 00 -> apparently inversion register
[ 176.922460] pca953x 4-0022: failed writing register: 02 [ 176.929805] pca953x: probe of 4-0022 failed with error -22 -bash: echo: write error: No such device root@letux:/sys/bus/i2c/drivers/pca953x# i2cset -y 4 0x22 0x88 0 root@letux:/sys/bus/i2c/drivers/pca953x# i2cset -y 4 0x22 0x2 0 root@letux:/sys/bus/i2c/drivers/pca953x#
debug output by this patch: diff --git a/drivers/gpio/gpio-pca953x.c b/drivers/gpio/gpio-pca953x.c index b7ef33f63392..161c5d5ee8c8 100644 --- a/drivers/gpio/gpio-pca953x.c +++ b/drivers/gpio/gpio-pca953x.c @@ -338,10 +338,11 @@ static int pca953x_write_regs(struct pca953x_chip *chip, int reg, u8 *val) { u8 regaddr = pca953x_recalc_addr(chip, reg, 0, true, true); int ret;
dev_info(&chip->client->dev, "writing reg %02x -> %02x = %02x", reg, regaddr, *val); ret = regmap_bulk_write(chip->regmap, regaddr, val, NBANK(chip)); if (ret < 0) {
dev_err(&chip->client->dev, "failed writing register\n");
dev_err(&chip->client->dev, "failed writing register: %02x\n", reg); return ret; }perhaps this rings a bell for someone.
ok, commented out the invert access in the init: @@ -865,7 +866,7 @@ static int device_pca95xx_init(struct pca953x_chip *chip, u32 invert) else memset(val, 0, NBANK(chip));
- ret = pca953x_write_regs(chip, chip->regs->invert, val); +// ret = pca953x_write_regs(chip, chip->regs->invert, val); out: return ret; }
and it probes successfully. Maybe we should just ignore that error there.
Regards, Andreas
Hi,
Am 29.05.2019 um 20:04 schrieb Andreas Kemnade andreas@kemnade.info:
On Wed, 29 May 2019 19:51:21 +0200 Andreas Kemnade andreas@kemnade.info wrote:
On Wed, 29 May 2019 02:12:27 -0700 Tony Lindgren tony@atomide.com wrote:
Hi,
- H. Nikolaus Schaller hns@goldelico.com [190529 08:57]:
[ 7.524125] pca953x 4-0022: failed writing register [ 7.529444] pca953x: probe of 4-0022 failed with error -22
I saw something similar with a BBB cape (early Baylibre ACME cape) and it was device_pca95xx_init() somehow trying to write to a read-only register causing an error.
I did not debug further and I do not have it handy, and I could not easily figure out what caused the regression.. But maybe take a look and see if commenting out the register write device_pca95xx_init() helps.
root@letux:/sys/bus/i2c/drivers/pca953x# echo 4-0022 >bind [ 176.917013] pca953x 4-0022: writing reg 02 -> 88 = 00 -> apparently inversion register
[ 176.922460] pca953x 4-0022: failed writing register: 02 [ 176.929805] pca953x: probe of 4-0022 failed with error -22 -bash: echo: write error: No such device root@letux:/sys/bus/i2c/drivers/pca953x# i2cset -y 4 0x22 0x88 0 root@letux:/sys/bus/i2c/drivers/pca953x# i2cset -y 4 0x22 0x2 0 root@letux:/sys/bus/i2c/drivers/pca953x#
debug output by this patch: diff --git a/drivers/gpio/gpio-pca953x.c b/drivers/gpio/gpio-pca953x.c index b7ef33f63392..161c5d5ee8c8 100644 --- a/drivers/gpio/gpio-pca953x.c +++ b/drivers/gpio/gpio-pca953x.c @@ -338,10 +338,11 @@ static int pca953x_write_regs(struct pca953x_chip *chip, int reg, u8 *val) { u8 regaddr = pca953x_recalc_addr(chip, reg, 0, true, true); int ret;
dev_info(&chip->client->dev, "writing reg %02x -> %02x = %02x", reg, regaddr, *val); ret = regmap_bulk_write(chip->regmap, regaddr, val, NBANK(chip)); if (ret < 0) {
dev_err(&chip->client->dev, "failed writing register\n");
dev_err(&chip->client->dev, "failed writing register: %02x\n", reg); return ret; }perhaps this rings a bell for someone.
ok, commented out the invert access in the init: @@ -865,7 +866,7 @@ static int device_pca95xx_init(struct pca953x_chip *chip, u32 invert) else memset(val, 0, NBANK(chip));
ret = pca953x_write_regs(chip, chip->regs->invert, val);+// ret = pca953x_write_regs(chip, chip->regs->invert, val); out: return ret; }
and it probes successfully. Maybe we should just ignore that error there.
No...
I have now been able to confirm your findings. But the problem is much deeper. The reason is that chip->regs->invert is 0x02 but translates into 0x88 as your first dev_info shows. This 0x88 is beyond the number of regmap registers!
Why is it? Because the 0x80 component is not a register number but an autoincrement flag.
This means that device_pca95xx_init tries to write a register with autoincrement enabled through regmap. IMHO this can't ever work...
Why did it not appear in v5.1 although the driver is the same?
I have checked the log of drivers/base/regmap/regmap.c and there is one which may explain:
8b9f9d4dc511 regmap: verify if register is writeable before writing operations
This is not in 5.1 but in 5.2-rc1. This means that the pca953x_writeable_register() callback is now never called in device_pca95xx_init but was before and could allow the register write...
I have tried to revert 8b9f9d4dc511 and
Why did not anyone else discover this? This autoincrement flag is special for 24 bit expanders and I guess there are much more 16 bit gpio expanders in the wild than 24 bit...
Now how can it be solved? I have no idea. Expanding the regmap to 256 addresses and passing this REG_ADDR_AI to regmap seems to be wrong anyways since there will be two cache entries for one register.
Maybe changing reg_bits to 7 in regmap_config may help - but I don't know if regmap simply passes the REG_ADDR_AI flag when doing physical i2c access.
I will ask the regmap and pca953x maintainers...
BR, Nikolaus
Am 31.05.2019 um 18:31 schrieb H. Nikolaus Schaller hns@goldelico.com:
Hi,
Am 29.05.2019 um 20:04 schrieb Andreas Kemnade andreas@kemnade.info:
I have tried to revert 8b9f9d4dc511 and
... it works.
I will ask the regmap and pca953x maintainers...
done.
I think until there is a good fix upstream, we will add the revert of 8b9f9d4dc511 to letux/hacks. As soon as the pca953x.c driver is really repaired we can remove it.
So I can now finally look at the Pyra panel driver...
The panel is now showing something which indicates that the panel is being programmed now. But it is not yet a recognizable picture but changes when writing to the frame buffer. Maybe some tiny bit in the panel timings is wrong.
So it is almost solved.
BR, Nikolaus
Hi,
Am 29.05.2019 um 11:12 schrieb Tony Lindgren tony@atomide.com:
Hi,
- H. Nikolaus Schaller hns@goldelico.com [190529 08:57]:
[ 7.524125] pca953x 4-0022: failed writing register [ 7.529444] pca953x: probe of 4-0022 failed with error -22
I saw something similar with a BBB cape (early Baylibre ACME cape) and it was device_pca95xx_init() somehow trying to write to a read-only register causing an error.
I did not debug further and I do not have it handy, and I could not easily figure out what caused the regression.. But maybe take a look and see if commenting out the register write device_pca95xx_init() helps.
I did a quick check and there are significant recent changes
git diff v4.19 v5.2-rc1 drivers/gpio/gpio-pca953x.c
even going down to register address calculations. Most changes were introduced in v5.0 for some regulator handling and wakeup_path, irq-handling and allowing to define pullup/down (if the gpio expander chip permits).
Maybe there is a bug that was hidden by something else before v5.2-rc1.
Anyways I seem to have found a recipe to be able to bisect. The problem is that mainline does not boot and the sets of LetuxOS patches to make it boot do either apply cleanly for v5.1 or for v5.2-rc1 but not both. Sometimes the compile fails and sometimes the Pyra does not boot to login and hangs with a kernel panic.
But now I seem to have found a good mix where v5.1 boots and is good (4-0022 is listed as a gpiochip), while v5.2-rc1 boots and is bad (4-0022 missing) and intermediate bisect points seem to compile and boot as well.
Now, ca. 13 rounds of bisect are needed.
BR and thanks, Nikolaus
Am 30.05.2019 um 12:34 schrieb H. Nikolaus Schaller hns@goldelico.com:
Hi,
Am 29.05.2019 um 11:12 schrieb Tony Lindgren tony@atomide.com:
Hi,
- H. Nikolaus Schaller hns@goldelico.com [190529 08:57]:
[ 7.524125] pca953x 4-0022: failed writing register [ 7.529444] pca953x: probe of 4-0022 failed with error -22
I saw something similar with a BBB cape (early Baylibre ACME cape) and it was device_pca95xx_init() somehow trying to write to a read-only register causing an error.
I did not debug further and I do not have it handy, and I could not easily figure out what caused the regression.. But maybe take a look and see if commenting out the register write device_pca95xx_init() helps.
I did a quick check and there are significant recent changes
git diff v4.19 v5.2-rc1 drivers/gpio/gpio-pca953x.c
even going down to register address calculations. Most changes were introduced in v5.0 for some regulator handling and wakeup_path, irq-handling and allowing to define pullup/down (if the gpio expander chip permits).
Maybe there is a bug that was hidden by something else before v5.2-rc1.
Anyways I seem to have found a recipe to be able to bisect. The problem is that mainline does not boot and the sets of LetuxOS patches to make it boot do either apply cleanly for v5.1 or for v5.2-rc1 but not both. Sometimes the compile fails and sometimes the Pyra does not boot to login and hangs with a kernel panic.
But now I seem to have found a good mix where v5.1 boots and is good (4-0022 is listed as a gpiochip), while v5.2-rc1 boots and is bad (4-0022 missing) and intermediate bisect points seem to compile and boot as well.
Now, ca. 13 rounds of bisect are needed.
It is finished now but the result is not convincing:
433b8dd7672be1140ffbb17eacba776298bf4733 is the first bad commit commit 433b8dd7672be1140ffbb17eacba776298bf4733 Author: Steve French stfrench@microsoft.com Date: Tue Mar 26 13:53:21 2019 -0500
SMB3: Track total time spent on roundtrips for each SMB3 command
Also track minimum and maximum time by command in /proc/fs/cifs/Stats
Signed-off-by: Steve French stfrench@microsoft.com
Especially since this is not the first bad commit if I test 433b8dd7672be1140ffbb17eacba776298bf4733~1. This means git bisect has not done its work properly (or I did report one test result wrongly). Most likely the real bug is in a parallel branch and older than v5.1 which was merged in by v5.1-rc1.
Unfortunately using v4.19 or v4.20 with my current Letux patch set makes the boot process hang when trying to find the SD card. Therefore I can not yet simply bisect between v4.19 and v5.2-rc1.
So it might be quicker to come to a solution by hacking the register read/write in the pca953x driver to better understand what it is trying and what it should do instead.
@Tony: if you can confirm that the omap5uevm is also broken with mainline v5.2-rc1 we could extend the discussion to the linux-omap list and gpio. But before it could simply be a special Pyra effect where nobody without hardware could easily help.
BR, Nikolaus
On Wed, 29 May 2019 06:39:22 +0200 "H. Nikolaus Schaller" hns@goldelico.com wrote:
Am 28.05.2019 um 23:40 schrieb Andreas Kemnade andreas@kemnade.info:
On Mon, 27 May 2019 21:25:41 +0200 "H. Nikolaus Schaller" hns@goldelico.com wrote:
Am 27.05.2019 um 14:46 schrieb H. Nikolaus Schaller hns@goldelico.com:
Hi, unfortunately every year or so some upstream change breaks our (still out-of tree) DSI panel driver for the Pyra(Phone). This time it happened with v5.2-rc1.
v5.2-rc1 also broke the GTA04 display and the L3704/L7004 displays, but those are DPI panels and the reasons were two separate issues which have nothing to do with the Pyra DSI panel.
There have been some API changes for DSI panel drivers and I have done a
git diff v5.2-rc1 v5.1 drivers/gpu/drm/omapdrm/displays/panel-dsi-cm.c
to identify them and tried to apply similar patches to our panel. But still the driver is successfully probed, but shows not activity. No calls to enable() or connect(). Backlight stays black. And as a result there is also no /dev/fb0, text console or X11.
Unfortunately, it has not improved after upgrading to v5.2-rc2.
So once again it appears that I have to git bisect upstream patches to find out what is going on...
After spending more hours... it does not seem to be in our patches and the panel driver is likely almost correct now.
I believe this because I also found not working:
- dwc3 (usb3/otg port)
- bq24297 battery charger driver (my Pyra fortunately runs well enough with U-Boot initialization)
- tca6424 (gpio expander - we use the pcal9524 variant)
The tca6424 driver seems to be broken (not loaded).
[ 6.581432] ts3a227e 4-003b: Cannot request irq 0 (-22) [ 6.597252] ts3a227e: probe of 4-003b failed with error -22
so somehow it insists of having an irq and we are not providing one in the dtb, at least not in pyra-mainboard-v5.1.dtsi
The reference is set in pyra-mainboard-v5.2.dtsi
interrupt-parent = <&gpio99>; interrupts = <14 IRQ_TYPE_EDGE_RISING>;But: this is another symptom of the tca6424 (gpio99) not being available. It is not only a gpio expander but also an interrupt expander.
after disabling that invert write I see still root@letux:~# dmesg | grep ts3a [ 6.913371] ts3a227e 4-003b: Cannot request irq 0 (-22) [ 6.932368] ts3a227e: probe of 4-003b failed with error -22
so it is an indepandant problem an least on board 5.1. It might work on board version 5.2
Regards, Andreas
Am 29.05.2019 um 20:38 schrieb Andreas Kemnade andreas@kemnade.info:
On Wed, 29 May 2019 06:39:22 +0200 "H. Nikolaus Schaller" hns@goldelico.com wrote:
Am 28.05.2019 um 23:40 schrieb Andreas Kemnade andreas@kemnade.info:
On Mon, 27 May 2019 21:25:41 +0200 "H. Nikolaus Schaller" hns@goldelico.com wrote:
Am 27.05.2019 um 14:46 schrieb H. Nikolaus Schaller hns@goldelico.com:
Hi, unfortunately every year or so some upstream change breaks our (still out-of tree) DSI panel driver for the Pyra(Phone). This time it happened with v5.2-rc1.
v5.2-rc1 also broke the GTA04 display and the L3704/L7004 displays, but those are DPI panels and the reasons were two separate issues which have nothing to do with the Pyra DSI panel.
There have been some API changes for DSI panel drivers and I have done a
git diff v5.2-rc1 v5.1 drivers/gpu/drm/omapdrm/displays/panel-dsi-cm.c
to identify them and tried to apply similar patches to our panel. But still the driver is successfully probed, but shows not activity. No calls to enable() or connect(). Backlight stays black. And as a result there is also no /dev/fb0, text console or X11.
Unfortunately, it has not improved after upgrading to v5.2-rc2.
So once again it appears that I have to git bisect upstream patches to find out what is going on...
After spending more hours... it does not seem to be in our patches and the panel driver is likely almost correct now.
I believe this because I also found not working:
- dwc3 (usb3/otg port)
- bq24297 battery charger driver (my Pyra fortunately runs well enough with U-Boot initialization)
- tca6424 (gpio expander - we use the pcal9524 variant)
The tca6424 driver seems to be broken (not loaded).
[ 6.581432] ts3a227e 4-003b: Cannot request irq 0 (-22) [ 6.597252] ts3a227e: probe of 4-003b failed with error -22
so somehow it insists of having an irq and we are not providing one in the dtb, at least not in pyra-mainboard-v5.1.dtsi
The reference is set in pyra-mainboard-v5.2.dtsi
interrupt-parent = <&gpio99>; interrupts = <14 IRQ_TYPE_EDGE_RISING>;But: this is another symptom of the tca6424 (gpio99) not being available. It is not only a gpio expander but also an interrupt expander.
after disabling that invert write I see still root@letux:~# dmesg | grep ts3a [ 6.913371] ts3a227e 4-003b: Cannot request irq 0 (-22) [ 6.932368] ts3a227e: probe of 4-003b failed with error -22
so it is an indepandant problem an least on board 5.1. It might work on board version 5.2
The interrupt-parent and interrupts = are commented out in -v5.1.dtsi and active in -v5.2.dtsi.
Maybe the ts3a227e driver simply assumes that the interrupt is defined and has no fallback for missing interrupt.
That it can not be requested on v5.2 board is the missing tca6424 (gpio99) device.
BR, Nikolaus
-
Andreas Kemnade -
H. Nikolaus Schaller -
Tony Lindgren