Ahh, while I was driving home I just realized one thing:
As our capacitors are all on the same side of OMAP5, we don't have any vias and the power pin / via / pad ratio doesn't have an effect in our design as well.
Stupid me!
But that's good - maybe we're lucky and just changing the amount, type and capacity of the capacitors we have on the PCB might work!
@Nikolaus: We have a ton of SMD capacitors here. Can you let me know which ones are the ones for the MPU and which value they have? Maybe I can try to change some here.
On November 9, 2016 8:20:41 PM GMT+01:00, Michael Mrozek EvilDragon@openpandora.org wrote:
Am Wed, 9 Nov 2016 18:43:16 +0100 hat "H. Nikolaus Schaller" hns@goldelico.com geschrieben:
Hi,
Hm, in the document Matthijs mentioned, I often fine "must be", etc.
Yes sometimes. But I see weakening words, e.g.:
• Whenever possible • Minimizing the separation distance • as close as possible it is recommended to consider • must be as wide as possible
In no such case it is clear when it is "good enough".
Well, yes, the guide first is an overall strategy guide for a design based around the power. Later it has be more clear examples.
Dealing with transients is still the job of decoupling caps as usual. TI gives quite a bit of guidance on the power distribution nets in general (sections 7.2.1-7.2.4, 7.3 of swps051k)
I didn't find that one but swps050i.
A quick google showed up this one:
http://software-dl.ti.com/secure/techdocs/swps051k/swps051k?__gda__=14795634...
You just need to login with your TI account and tell them you don't want to nuclear bomb Trump.
Says "Access Denied". Maybe they know more about me than I do...
Well, probably because direct linking is not allowed ;) But as mentioned entering swps051k in google gave me a direct link to the TI site where you can download the document. But I've attached it :)
I had studied that as well as some Palmas Layout guide. But this is
2 years ago so that I don't remember all details and especially I
do not remember the reasons for all decisions.
That's understandable.
Not sure if there's a difference, but swps051k quite often has "must" in their text, not should be or similar.
"must be $value" is different from "must be as good as possible".
They have some values later in the guide, AFAIR, and it's mostly "must be at least" or similar.
So this is something objective we can check.
And we have quite strong space constraints so that not everything can be copied from the EVM (e.g. placing decoupling capacitors on the backside under the OMAP5) or guide lines (there isn't much space for adding more VIAs or moving traces to optimal positions).
The backside is completely unpopulated right now, but we have 0,5mm space between it and the case. Should be enough for capacitors, so why, for example, couldn't we put decoupling capacitors there?
Because
- we didn't know before :)
- we might at some time want to make the pyra thinner without
limitations by the CPU board 3. we need to redesign the whole board
Well, the case was being built around the PCB design, so during the design phase, you could've easily added some things there.
The PCB could've also been made more wide (there's space between the CPU board and the MicroSD/SIM Card reader). But we can't start to redesign now.
(This is just an example though, because according to the
Datasheet:
"2. Place Dcaps on “same-side” as component within their power
plane
outline to minimize “decoupling loop inductance”. Target distance to power pin should be less than ~500mils depending upon PCB
500mil = 12.7mm
Thanks. I didn't find anything about that on the net :D
This is one of these contradictions where we might have to trade off. With a chip dimension of 17x17 mm this more or less means "on the
same
side as the BGA balls and as close as possible to the edge of the chip"). It looks as we have that.
Good :)
How many mils are we away from the power pin?
There is no single power pin...
Well, I guess they the distance of every capacitor to the correct power pin. I guess the capacitors are not connected to multiple power pins, are they?
If I understand correctly, what we need to check for are the IR drop and capacitance-inductance loop requirements... I have no idea how these can be checked though (I'm not really experienced here). Is that something you simulate from the PCB design files?
If you spend me >10 k€ for a high end tool like OrCAD or PADS and a modern Windows workstation, we can easily simulate this.
And there is no company that offers that service? For our TV services, we often rent expensive tools (like 100k EUR measurement devices) if we need them for a job.
For me it's totally normal booking external companies for services I need... or do you plan to buy a population machine as well so we don't need GC? ;)
"It is highly recommended to run board frequency domain analysis on the vdd_mpu, vdd_mm, and vdd_core power supplies to check if proper board decoupling is implemented and confirm that the respective target impedances are met. For more information on target impedance, see , External Capacitors." (Page 272)
"highly recommended" sounds like "it won't work otherwise" to me. Does someone know a company that could check that and give us results?
And something we should also check: "Use minimum 10mil trace width for all voltage and gnd planes connections (i.e. Dcap pads, component power pins, etc.)."
10 mil = 250 µm
The minimum width used for all GND and power wires is 0.2 mm - but there is additional GND filling.
Okay, that should be good then.
I.e. big GND/power-strips like this for VDD-MPU: One quite weak thing I see now is that there is only a single VIA to connect this big area running in some inner layer to the surface. The problem is the space in this area is also heavily used for signal wiring:
Ouch... I really do NOT hope this is the issue!
The sentence "Do not share vias among multiple capacitors." on page 274 sounds like that could cause problems.
On that page they also show a nice "Bad design" and "Good design" example, where the good design simply has more vias added. They don't even cut any traces, they just add vias.
Well it could be possible to squeeze in one or two more VIAs in
there.
And we could use one of the 2 layers we had added for getting the layout done to add another connection strip between Palmas and OMAP. This would reduce resistance definitively to less than 50% of the current value.
50%! This is quite a lot! Is there anything we can to to test that manually? (soldering a wire somewhere?)
That such things are not optimal here is the result of finishing the CPU design in a hurry last year to get the layout done for the first attempt. And a second hurry this year as well. We likely have suboptimal trace layout because there was not enough time (and manpower) to think about and peer-review such details. And to be fair to ourselves - we didn't know for sure that it could be a potential source of a problem.
Well, yes and no. If you remember, the first thing we suspected after the production when that happened were weaknessess in the power supply (that was in February).
I asked you to triple check everything that has to do with the power supply of the CPU board so we can be as sure as possible, it's not a hardware issue.
And yes, while we always need to hurry, we do not need to RUSH things. That's my motto: Hurry but don't rush. So if you told me you need two weeks more time to recheck the power supply circuit, then I would've said "Okay, do that".
Just do that next time!
Sadly, I'm not competent enough to do all that, I wouldn't even have known where I'd have to look for such design guides, otherwise I would've checked that earlier myself.
Improvements here would need a change in board design. I am not sure if this is within financial and time budgets.
I can't possible produce 3000 Pyras which do not work as advertised. Especially as it might be possible they don't even work reliable in all situations with 1GHz.
Fixing this issue is mandatory, that's what I said in February as well.
Hopefully, we can find another workaround in software or without changing the layout...
And it could be a solution but as well it could remain unsolved if it is not the problem.
True, but the more I read about it and think about it, the more I think this is the problem. Well, not particularly the design (maybe just the capacitors), but something with the power here.
Working with 1 Core at 1,5GHz, working with 2 cores at 1 GHz, but crashing with 2 cores when going higher?
That doesn't sound like a kernel issue but like a power supply issue to me.
"Use a “Power/Gnd pad/pin to via” ratio of 1:1 whenever possible. Do not exceed 2:1 ratio for small number of vias within restricted PCB areas (i.e. underneath BGA components)."
Maybe someone could help to translate this...
Doesn't sound too complicated to understand to me, unless I understood it wrong:
When you're using a via, the trace from the Power or ground pin should have the same length as the trace from the via (on the other side) to the capacitor pad.
So e.g. if the distance from the power pin to the via is 100mil, it should also be 100mil from the via to the capacitor.
It should not exceed 2:1 (so 200mil and 100mil, in our example).
It can also be seen on page 274 (with examples).
The recommend various decoupling capacitors, so we might have some ways to play around here. And capacitors can be changed without changing the board design.
Yes, indeed.
So this is basically our last straw trying to fix it here... :/
"For power IC which can support more than 10uF close to processor,
a
bulk capacitor of at least 22uF is strongly recommended for VDD_MPU power domains." Do we have that? :)
I have checked
- we did have 3x10µF at the Palmas side (measured as 22µF)
- now we have 3x10µF at the Palmas side and 10µF at the OMAP side
(measured as 34µF) Seems to be 20% below. Or my capacitance measurements is imprecise. So we have already increased the capacitors by ~50% with no significant success @ 1.5GHz.
Well, it seems to me the amount and size of capacitors DOES matter. In the PDF, 7.3.4 Dynamic PDN Analysis (Page 287)
In their example, they use: 5x 100nF, 3x470nF, 4x1µF, 1x2.2µF, 1x22µF
So they are suggesting 14 capacitors while you are using 4...
There has to be a difference between using one or more capacitors!
I guess these are some clues (all still page 287): "Maximum Loop Inductance per Capacitor (nH) - Values do not include ESL of the capacitor, and the ESL of the capacitor should not exceed 0.5nH"
And "Fpcb (Frequency of Interest) is defined to be a power rail’s max frequency after which adding a reasonable number of decoupling capacitors no longer significantly reduces the power rail impedance below the desired impedance target (Zt2). This is due to the dominance of the PCB’s parasitic planar spreading and internal package inductances."
So it's not only the total value of the capacitors that matters. It seems it's important to use multiple capacitors and make sure the maximum loop inductance is being kept. And that maximum loop inductance per capacitor...
Some more clues could be "PDN Impedance characteristics": "The PDN (Power Delivery Network) impedance characteristics are defined versus the device activity (that runs at different frequency) based on , Recommended Operating Conditions."
This tells us that different frequencies can have an effect of the impedance, if I understand that correctly.
There's an interesting sentence on page 283: "– At high frequencies current follows the path of least inductance. – At low frequencies current follows the path of least resistance."
Could that be our effect, running at more than 1GHz?
There is more explanation on page 276/277:
"Decoupling capacitors must be mounted with minimum impact to inductance. A real capacitor has characteristics not only of capacitance but also inductance and resistance. Figure 7-10 shows the parasitic model of a real capacitor. A real capacitor must be treated as an RLC circuit with effective series resistance (ESR) and effective series inductance (ESL)."
Maybe the capacitors we're using have too high impact on the inductance and that's why higher frequencies fail?
Maybe our capacitor value is TOO HIGH? Or we are using too few capacitors.
If we're lucky, we might be able to fix that without extra vias...
Is it possible that we can test increasing the amount of capacitors on the CPU board with some manual soldering?
At least we should change some more different values / capacitors...
I have picked up some unpopulated CPU PCBs this morning and I can try to measure resistances. If they are in the mOhm range (which is to be expected) the precision might not be good.
Okay!
Here are the first results:
- Palmas A33 (SMPS3 feedback) *is* cut open in the new boards
Good!
- VDD-MPU and GND resistance between Palmas and OMAP is <0.3 Ohm (my Multimeters are not more precise)
Not sure what the maximum allowed value is. :)
-- Mit freundlichen Grüßen,
Michael Mrozek
OpenPandora GmbH Geschäftsführer: Michael Mrozek
Schäffbräustr. 11 85049 Ingolstadt Deutschland Tel.: 0841 / 990 5548 http://www.openpandora.de/ HRB 4879, Amtsgericht Ingolstadt
eMail: mrozek@openpandora.org
Kernel mailing list Kernel@pyra-handheld.com http://pyra-handheld.com/cgi-bin/mailman/listinfo/kernel