Prototype mainboards coming soon

[TrashyMG]

Have you tried stacking other caps on top of the existing ones? With >=0603 and tweezers this could work for testing (without waiting for new pcbs).

This would have been solved long ago if it's only a matter of changing cap values. From what I gathered from the behind the scenes discussions is that some supplies never got decoupling caps in the CPU board design, most likely to save space, now a design change to the PCB needs to be made.

 

[Black Sliver]

This would have been solved long ago if it's only a matter of changing cap values.

It's not only about values, It's about the effective impedance, which also means "quality". Besides that, I haven't seen a post about what has been done, so this was just a suggestion.

From what I gathered from the behind the scenes discussions is that some supplies never got decoupling caps in the CPU board design, most likely to save space, now a design change to the PCB needs to be made.

In the schematics I don't see any non-phy VCC that has no decoupling capacitor? Some VCC pins span across the entire chip, so multiple caps / physical locations would be preferable, but that's not necessarily required and depends a lot on the actual board buildup (the actual resistance and inductance between the caps' leads and the SoC's leads).
It's always best to compare to a working reference design, which we (or at least I) don't have access to.

 

[levi]

Is there an issue with the mainboard that I've missed? As far as I knew the 1.xGHz issue (or whatever it should be called) is on the CPU board, not the main board (using the terminology ED has settled on as I understand him).

 

[Grench]

Is there an issue with the mainboard that I've missed? As far as I knew the 1.xGHz issue (or whatever it should be called) is on the CPU board, not the main board (using the terminology ED has settled on as I understand him).

To my understanding... Mainboards had a revision change. It included adding a through post microUSB 3.0 connector, slightly moving the battery connectors so they match the case & battery better, and other tweaks and improvements. To my understanding this is the batch of mainboards to be used in the upcoming prototype production & assembly. They will probably be very close if not the same as what should eventually be production mainboards.

From what I can tell:
The case is going back for another round of tweaks.
The current '1.2Ghz' SoC boards are getting continued scrutiny to figure out why they aren't 1.5Ghz SoC boards.
The keymat is nearly there - but is undergoing another round of tweaks.
That there are 1000 displays with touchscreen layers on them 'ready to go' is good news.
Batteries are waiting & ready.

It still feels like prototypes are 1-2 months out. I would be happily surprised to see one arrive the first week of February. In the grand terms of this project, though, that's, "getting close".

 

[SNESFAN]

2 months?

 

[levi]

To my understanding... Mainboards had a revision change. It included adding a through post microUSB 3.0 connector, slightly moving the battery connectors so they match the case & battery better, and other tweaks and improvements. To my understanding this is the batch of mainboards to be used in the upcoming prototype production & assembly. They will probably be very close if not the same as what should eventually be production mainboards.

Yes, those patches (and the ones mentioned in the last case video) are going to be in this round, which should be all that's need for the prototype builds. Boom...PROTOTYPES!

 

[notaz]

Some VCC pins span across the entire chip, so multiple caps / physical locations would be preferable, but that's not necessarily required and depends a lot on the actual board buildup (the actual resistance and inductance between the caps' leads and the SoC's leads).

The uEVM (eval board) has multiple caps, so that indeed is one departure from the reference design. AFAIK it ended up that way because of lack of space on the CPU board.
Do you think it may be the cause of the issue?
If you have experience in this area I think @hns would be interested to hear you.

 

[Silent-Hunter]

The uEVM (eval board) has multiple caps, so that indeed is one departure from the reference design. AFAIK it ended up that way because of lack of space on the CPU board.
Do you think it may be the cause of the issue?
If you have experience in this area I think @hns would be interested to hear you.

If there's a lack of space, is this going to prevent the issue from being resolved without a major redesign?

 

[Black Sliver]

Do you think it may be the cause of the issue?

I'd say my guess is as good as anyone's.

If you have experience in this area I think @hns would be interested to hear you.

Luckily I never had to design stuff around such a huge and fast SoC, but the schematics' 10µF sound like high-ESR / low voltage; we do more low-level (non-linux) stuff with less than 120MHz. That being said, we still had our share of problems with "bad" capacitors.

Anyway, the mathematics are easy:
First you need a bulk capacitor since the DC/DC Converter is slower than the change in low frequency current consumption: the (average) consumption may hop from 100mA to 200mA within less than 1/fSW,DCDC.
If the effective value of bulk capacitors equals the reference design, you should be on the safe side there (e.g. 2x10µF @ 10V can be replaced by 1x22µF @ 10V but cannot always be replaced by 1x22µF @ 5V because of the capacitance change with applied DC voltage; see cap's datasheet). Luckily, you can measure/verify this with any oscilloscope: ripple on caps.

The "smaller" caps are for "near-RF" current consumption (well... 1.5GHz IS RF): each bit change at each clock circle requires a certain current, which has to come from somewhere and may be non-constant. If you need "additional" 100mA over 1ns, a 100nF cap will discharge by only 1mV, so it's not really a question of "size".
BUT from the SoC's point of view, there is a resistance between the current source (the cap) and the sink (the register's FET), which is mainly an inductance and so depends highly on the frequency. Z = R+2*pi*f*L. That very inductance L consists of the bonding wire inside the chip, the trace between the chip and the cap and the ESR of the cap itself.
Since we can't change the bonding wire and - without reordering new PCBs - the trace, we can only try to reduce the cap's ESR by choosing another dielectric or rated voltage.
Knowing the exact dielectric properties of the PCB (between VCC and GND plane) and copper layout, you could calculate (or simulate) the voltage drop to see if an ESR of zero would make any difference. But you can't really measure it since you'd need a impedance-matched line from inside the SoC to a scope, that has a bandwidth of >3GHz.

TL;DR: trial and error; may or may not work to stack 100nF X7R@>16V or 1..10nF C0G on top. Reasons are clear. I'd try.

 

[WizardStan]

I'd say my guess is as good as anyone's.

My guess is gremlins. Literal gremlins. They eat the magic smoke that resides inside all electronics.

 

[Agricola]

My guess is gremlins. Literal gremlins. They eat the magic smoke that resides inside all electronics.

 

[levi]

Anyway, the mathematics are easy:

If you say so

Anyway, good to have you on board!

 

[edgex004]

TL;DR: trial and error; may or may not work to stack 100nF X7R@>16V or 1..10nF C0G on top. Reasons are clear. I'd try.

I'm glad I did read. That was a great summary of the potential issue. Thanks for that. Electronics was always one of my most difficult courses, but what you wrote almost looked like English to me.

 

[Eight Bit]

TL;DR: trial and error; may or may not work to stack 100nF X7R@>16V or 1..10nF C0G on top. Reasons are clear. I'd try.

I'm glad I did read. That was a great summary of the potential issue. Thanks for that. Electronics was always one of my most difficult courses, but what you wrote almost looked like English to me.

Indeed, even I thought I understood what I read. Are you a teacher Black Silver?

 

[Black Sliver]

Indeed, even I thought I understood what I read. Are you a teacher Black Silver?

Nope, just working in electronics.

 

[rv6502-2]

I'd say my guess is as good as anyone's.

@hns , @notaz

May I suggest another possibility? Just throwing a wild theory.

What if a trace or power plane catches the 1.5Ghz RF signal like an antenna?
Quarter-wave of 1.5Ghz is 5cm. The space between the CPU board connectors looks close to that size.

I've recently hit an issue at work, a strip of smart LEDs happened to have its power traces pretty much the exact 1/2 wave length of a transmitter and it caused the LEDs to go nuts.
I put my hand between the two and the issue went away. Putting any probe on the power traces also made the issue go away due to changing the trace length.
Even holding my hand in different spot close to the LED strips would lessen or worsen the issue.

Was (not) a fun one to figure out.

The CPU might be blasting it's 1.5Ghz clock signal into the main board at just the right trace length to resonate with something, like an accidental LC circuit, that when combined with normally sufficient power decoupling may just be enough to occasionally crash.

Can it be tried with a grounded piece of metal around the CPU board, going between the two boards to disrupt/shield the RF? (covered in electric tape to not short anything out)
Doesn't look like there's much space tho.

That in itself may act as a big capacitor against the power plane as well so it wouldn't be super conclusive but when stomped...

I feel for you having to debug this. Best of luck.

 

[ashcloud]

@hns , @notazMay I suggest another possibility? Just throwing a wild theory.

Congratulations! You finally understood the issue. That's exactly what the blocking capacitors are for.

 

[EvilDragon]

The board works perfectly fine with 1,5GHz, if you only use ONE CPU.

 

[Eight Bit]

Still no progress then?

 

[Black Sliver]

What if a trace or power plane catches the 1.5Ghz RF signal like an antenna?

I've managed to fry an µC with RF imission (30V/m (3x industrial limit)) once. That moment when you hold a measuring tape with λ/2 instead of cm or inch
But stuff like that would depend on location (external RF source) and not really on load/core frequency.

Quarter-wave of 1.5Ghz is 5cm. The space between the CPU board connectors looks close to that size.

If GND "swings" at an offset to PE, that should not be a problem for the CPU itself, since other traces should then just have the same offset to PE (on multi-layer). Assuming the DC/DC blocks noise from supply lines, the relevant distance is from the DC/DC converter to the CPU (you can apply noise between VCC and GND only there).

The CPU might be blasting it's 1.5Ghz clock signal into the main board

This would be more like cross-talk, which normally only affects high-impedance non-burried traces. So you'd probably see random SD/SPI/I²C errors before any cpu crash.

resonate with something, like an accidental LC circuit

(Assuming we are still talking about VCC,) the caps are way to big to get a (high-Q) resonator at GHz (or MHz). Besides that, the caps were already changed which would have shifted the frequency.