The tale of the cases

Discussion in 'Pyra News' started by EvilDragon, Oct 14, 2017.

  1. daveshah

    daveshah Still Fresh

    Joined:
    Aug 17, 2008
    Messages:
    23
    I'm talking more about power supplies, not the data signals where 1s and 0s come into it - although they are a whole other can of worm - when it comes to high speed interfaces there is a much bigger gap between working all the time and not working at all. As an example, a DDR3 chip might register Vref+160mV as high and Vref-160mV as low. If at the time the clock transitions the input is only Vref+80mV eg because your trace delay is too high, it's anyone guess what happens - it could depend on chip to chip variations, temperature, etc.

    Going back to power issues, I reiterate the problem is not the steady state DC voltage that you are talking about, which is what the DC resistance of the trace affects and I have no doubt is in spec, the likely problem is transient drops on the supply line as the devices switches. How the supply will respond to these depends on not just the length of the trace but also the presence and spacing to of a ground plane, decoupling capacitors and their placement and value.

    The reason that this only affects 4GB boards is the larger memory devices have more transistors to switch, and therefore cause greater transients on the power supply lines, pushing them out of spec for a tiny period of time during that transient, enough to cause issues inside the chip.

    The resistor issue I mentioned was not intended to be an example of how sensitive DDR chips are, just how hard it is to debug them and work out what is software and what is hardware.

    Hence even I wouldn't be totally certain about the power issue but it seems pretty likely. Maybe you've done this and I've missed it, but I know you can buy special active scope probes for measuring power rail quality. They are expensive, and you would need access to a suitable good - also expensive, but easier to find someone with - oscilloscope but scrapping boards is also expensive... If you do get stuck it would be interesting to see. I saw some simulations have been done but sometimes measurement is the only way, particularly the exact characteristics of the capacitors and RAM supply demands won't be taken into account in a sim.
     
  2. Letalis Sonus

    Letalis Sonus Advanced Member

    Joined:
    Mar 5, 2009
    Messages:
    1,055
    Throwing DC formulas at high frequency signals is absolutely useless - digital or analogue, that doesn't work, you're effectively working with an AC signal that suffers the more from external influences the higher its frequency gets. The effective resistance that causes such a voltage drop is not just caused by the physical properties of the signal line, you have to take all of the electromagnetic influences around it into consideration as well, there's all sorts of virtual electronic components popping up in your calculations due to the effects that a high frequency signal has on nearby conductive elements. It can't be avoided that these effects bleed into the power supply lines as well, a high frequency signal powered by a DC voltage will inevitably cause high frequency noise on the supply line.
    If you haven't already seen what's involved to calculate AC circuits, look up some basic formulas. That's some serious nightmare fuel for someone who hasn't studied math or physics.

    Have you ever spent more than a second to think about the idea that a DDR interface is more than a simple address/data bus? The actual structure of the available memory matters a lot, DDR memory is neither accessed atomically nor is it organized in a planar configuration. There's a reason motherboard manufacturers still provide a list of compatible RAM modules nowadays, every single parameter about DDR RAM that changes has a high probability to break things. Compared to things like USB 3 or PCIe lanes a DDR interface is still highly parallelized, the distortions that such a high amount of side by side signal lines cause to each other are enormous.
     
    Djhg2000 and daveshah like this.
  3. daveshah

    daveshah Still Fresh

    Joined:
    Aug 17, 2008
    Messages:
    23
    Thinking about it some more, I saw at one point that turning off the DLL solves the problem. That could point further to a power issue, as it could be that the extra supply demand and hence noise caused by the extra logic of the DLL drives the power rail to the point of unreliability - clearly something is marginal and very much on the border of working. Equally I suppose it could also indicate a timing issue.

    I know the people working on this have a lot more experience with this stuff than I do, and it sounds like what's happening so far is sensible. Although more decoupling capacitors aren't always a solution squeezing in more capacitors close to the memory would almost certainly help. Have you already checked if switching to a smaller size of capacitor so you can fit more closer to the memory would cost a lot more to assemble with GC?
     
  4. EvilDragon

    EvilDragon Administrator Staff Member

    Joined:
    Mar 4, 2003
    Messages:
    19,786
    Location:
    Ingolstadt
    Nope, totally different voltage line (VDDR3).
    Besides, the VSYS (power for the OMAP5) was still having a lot of noise and voltage drops, so improving it was mandatory.
    Otherwiese, it could still have caused instabilities when more power was actually being used (like NEON, 3D and a high clockrate together).

    It's only guaranteed to work if everything is inside specs, and it clearly wasn't.

    Yeah, right.
    It seems you don't even have the most basic understanding of electronics.

    If a wire (or trace) is too thin the for power you want to squeeze through, it will have an increased resistance, which will lead to a voltage drop.
    The thinner the wire is, the higher the drop - and that has NOTHING to do with the length of the trace and wire.

    Besides, you could even see NO wire as a wire with a diameter of 0 (which is really really thin), and now please don't tell me you don't lose all the voltage if no wire is there.

    Here is a nice result of the simulation.
    Here it shows we've got too high noise on the VDDR3 plane:

    Noise.jpg

    And here is where you can clearly see the nice voltage drops:

    DDR3_Noise_and_Voltage_Drop.jpg

    But yeah, these professional simulations are all bullshit. They were probably created by us using Microsoft Paint so we have some pretty pictures.

    What you are explaining is a COMPLETELY different type of tests. These are tests to prove your assumptions.

    There are also tests that you make to find out exactly what's going on if there are multiple reasons something can happen. This is what we did.
    Of course, you still need to know WHAT to test and how to interpret the results. Which we did.

    What we knew:
    • It could be a software issue (wrong timings, wrong setup, etc.)
    • It could be a hardware issue (noise, voltage drop).
    • It could be a general incompatibility between the memory chip and the OMAP5
    So, simulation showed the hardware is out of specs. But as before with the OMAP5: Out of specs doesn't necessarily mean that's the problem. Because the specs GUARANTEE that everything will work. Therefore, the manufacturers add A LOT of tolerance to them.
    So even if we're out of spec, it could've been that it would still be fine.

    Therefore, we also played around with the timings and settings, as this can give more clues.
    For example, as the RAM worked fine with higher voltage (with the same timings), it's very likely the voltage drop is indeed an issue.

    Why it worked with DLL Off and not with DLL On (which is the default) is a bit trickier to figure out.
    The most likely reason is that the too high noise affects the internal clock divider of the DRAM (which is the DLL).

    To rule out an issue with the memory, we also tried SingleRank Samsung memory (the 2GB version of the 4GB RAM we're using) and DualRank IM memory (which is the 4GB version of the 2GB RAM we used).

    The 2GB Samsung memory works fine, whereas the 4GB IM memory works a bit better than the 4GB Samsung, but still crashes and reports memory errors. So while it's not as sensitive as the Samsung memory, it still doesn't work properly.

    All these tests showed that the memory itself is fine and the voltage drop and noise are too far out of specs. So that's what we need to fix.

    Besides, we've got support from some memory experts (for example, this company: www.eyeknowhow.de, as well as Juliano who did the simulation for us). And even eyeknowhow could only give some assumptions but said without tests, nothing can be said for sure.
    And that's what why we did the tests we could, and it showed us what the issue is.

    See? This is the reason why companies usually don't let anyone look into their development process, but rather present the finished product.

    Do you think other companies don't have these things to figure out?
    Why do you think companies like www.eyeknowhow.de do exist, if they don't have any clients (as apparently, it's not needed to run any tests and everything works without issues from the first try)?

    Do you think companies like Nintendo or Sony don't do tons of redesigns, testings, etc. before they publish their final product?
    Dream on!

    Sure, they've got a LOT more experience and know-how than we do, as they have a lot more experts, a LOT more money and they have already produced TONS of products using memory, so they might have some reference designs already for that.

    BTW: Nikolaus (the designer of the hardware) has worked for one of the largest electronic companies in Germany before... and they usually even had more prototype revisions than we have.

    Sure, it's easier. Instead of spending the time to test whether it's the hardware, we could've just assumed it is the hardware (as it was the most likely case), produced new CPU boards and test if that would fix it.
    But that's also a lot more expensive. No prob for a big company, but certainly not possible for a smaller one like us.
     
    szr, eric, johnicboom and 25 others like this.
  5. EvilDragon

    EvilDragon Administrator Staff Member

    Joined:
    Mar 4, 2003
    Messages:
    19,786
    Location:
    Ingolstadt
    It's actually up to 300 - 350mV in worst case, if you take a look at the simulation I posted.
    So yep, that's way too much.

    I think you understood me wrong. This is not against any greek person. They're always friendly and nice, and the new case company is a good example.
    However, I know FormAction was not an exception.

    When we had lunch, it was nigh impossible for me to get an invoice. I had to do a lot of discussion, and when I received that invoice, it was Nr. 1, so the first invoice they issued for this day.
    They probably calculated most of their prices without tax.
    And the government seems to be unable to do something against that... that's a mess.

    Additionally, I know that about 50% of the companies at least in the Thessaloniki area closed down during the last years, because of financial misorganization.
    I've talked a lot about all these things with my contact, and I know most companies don't really care about bookings, tax or anything.

    They just produce, sell (often without any invoice). They can do that because government doesn't really care.
    But it doesn't work like that if you sell outside of Germany - as I HAVE to have an invoice for every payment I make.

    So yes, sadly, it's not just FormAction. There's a reason these financial issues exist.
    And yes, I know it's not the fault of most of the people living there. But it's still real. This has nothing to do with racism.
    --- Double Post Merged, Oct 17, 2017, Original Post Date: Oct 17, 2017 ---
    Actually, yes, with voltages involved, there is a rise and fall, so something between 0 and 1.
    And it's actually important there's not too much noise in there, as this would cause issues the faster the timing settings are. That's the signal integrity.

    The test for that is called an eye pattern, but we already knew that's not the issue, as that had been simulated before.

    newomap_ibis.jpg
     
    szr, johnicboom, Akko and 10 others like this.
  6. Grench

    Grench Forum Addict!

    Joined:
    Oct 3, 2008
    Messages:
    6,139
    If everything else is more or less working on that board with 2GB Samsung RAM, can I call dibs on it for my prototype? <3
     
    Djhg2000 and WizardStan like this.
  7. WizardStan

    WizardStan Mega GP Mania

    Joined:
    May 24, 2008
    Messages:
    16,713
    I'm honestly just a touch surprised you haven't gotten one yet. I know ED's trying to make them as good as possible, but they're prototypes, they're supposed to be questionably stable :p Otherwise what is even the point?
     
    ilo_pona and rygD like this.
  8. levi

    levi Still fresh, damnit!

    Joined:
    Oct 6, 2008
    Messages:
    11,227
    Location:
    Somewhere off the coast of the EU
    The modern usage of the term 'racism' also applies if you're discriminating against a specific tribe, race or nationality, I've heard. Still, I miss the old 'irishman, scot and an englishman walk into a bar' jokes, but perhaps the internet has never really been the place for them.

    OT: Thanks ED, for the pretty pictures!
     
    Djhg2000 likes this.
  9. tarator

    tarator Very Active Member

    Joined:
    Jun 4, 2016
    Messages:
    150
    Location:
    Vienna
    pocak, Dark Pulse, levi and 3 others like this.
  10. Zwerg01

    Zwerg01 Member

    Joined:
    Feb 15, 2017
    Messages:
    66
    Interesting, that you exactly know what my profession is. It goes along with all the prejudice about people from Greek.

    Without a layout, a BOM, all datasheets and the information what is assembled, the only thing i can do is guess. And thats not my style.

    Also interesting, that i get response to my posts soon, whereas other users here dont get an answer at all.

    Now i am convinced. It has to be the voltage drop. Have fun.
     
    Last edited: Oct 17, 2017
    ThinkPad likes this.
  11. ClockworkCoder

    ClockworkCoder Chaotic Neutral

    Joined:
    Jan 21, 2016
    Messages:
    1,095
    Location:
    Menzoberranzan
    Guys, can we please keep this thread on topic? At least for now, while there are relevant and interesting conversations going on. Thanks.
     
  12. ashcloud

    ashcloud Lactose Intolerant Volcano

    Joined:
    Oct 5, 2010
    Messages:
    196
    Location:
    guru.awakes.dormant
    ClockworkCoder likes this.
  13. TrashyMG

    TrashyMG Sarcasm Dispenser Staff Member

    Joined:
    Jan 18, 2010
    Messages:
    10,558
    Only if we had all these arm chair engineers back when the CPU board was designed.
     
    Djhg2000, drock, Kippykip and 9 others like this.
  14. Grench

    Grench Forum Addict!

    Joined:
    Oct 3, 2008
    Messages:
    6,139
    I suspect that some of the other prototype preorders in line ahead of me may be more interested in electronically & aesthetically 'perfect' units in unique colors with a 'prototype' pedigree.

    My requirements are to have a unit where every piece physically works - i.e. electronically & mechanically sound. There are still a few pieces to the puzzle that I'm still uneasy about.
    1. Thermal solution. We've seen a prototype of a heat pipe and a chunk of copper - so that has been worked on somewhat.
    2. WiFi. It would be nice if I were able to get decent WiFi without using a USB dongle. I understand that has been worked on and may be pending a new version of the mainboard.
    3. Siring capacitor? I want to be able to use my prototype in an office environment. If the device sits there and screams continuously at 18000Hz, not so much. I understand that there were options being looked into, but I'm not sure if the exact component that was singing was identified.

    They are all known issues. There just doesn't seem to be enough human power to get those sorted while dealing with the 4GB and case issues.

    The project is definitely getting close to where it would nee to be for my prototype. I'm not sure and can't say how close or far it is from acceptable for the other 7 people in line in front of me though.
     
    Djhg2000 and levi like this.
  15. Splintercat

    Splintercat Member

    Joined:
    Oct 3, 2015
    Messages:
    51
    Location:
    United States
    So with things looking like a hardware issue, what's the plan?
    It sounds like to make 4GB work the cpu boards will need another round of prototypes with some added capacitors.
    So what about the ones that have already been produced? If the already produced 4GB boards are stable when limited to 2GB, I personally would be happy with that as my 4G order and would be fine with later paying additionally for a redesigned working 4G board.

    What I'm saying is that I get that there's a limited budget and I'm open to options from the Pyra team that allows the project to keep moving.
     
  16. Silent-Hunter

    Silent-Hunter Hardcore Member

    Joined:
    May 29, 2010
    Messages:
    3,053
    Count me in for this too.
     
    Grench likes this.
  17. rygD

    rygD Nihilistic Mystic

    Joined:
    Feb 28, 2014
    Messages:
    7,360
    Location:
    Everywhere
    If they get 4GB working, would 8GB work?

    JK, I am fine with 2GB when the cases get made.
     
    Djhg2000, ilo_pona and Silent-Hunter like this.
  18. levi

    levi Still fresh, damnit!

    Joined:
    Oct 6, 2008
    Messages:
    11,227
    Location:
    Somewhere off the coast of the EU
    As I'm reading it, it's looking more like it might be down to who's made the chips. AFAIK while PCBs have been made I don't think they've all been populated yet.

    It's still not clear to me whether 4GB boards limited to 2GB would be more or less stable. Perhaps if they only use the chip nearer to the power supply, it'll be more stable, but remember AIUI it'll have half the memory bandwidth of genuine 2GB units, as those will have four 512MB chips (while 4GB have 4 1GB chips, with one pair disabled in your plan), giving you double the pin count.

    Edit: Fixed; it's four chips of half the relative capacity (two pairs) not two chips. Thanks @Grench!
     
    Last edited: Oct 17, 2017
  19. Grench

    Grench Forum Addict!

    Joined:
    Oct 3, 2008
    Messages:
    6,139
    Just a clarification to what I understand... There are 2 controllers and 4 chip positions on the CPU card. Each controller has 2 chips. Each active controller needs 2 chips Possible configurations are:
    (4) 512MB chips = 2GB full bandwidth
    (2) 512MB chips = 1GB half bandwidth (one controller off) (This configuration does not exist to my knowledge, but could)
    (4) 1GB chips = 4GB full bandwidth
    (2) 1GB chips = 2GB half bandwidth (one controller off)

    The working 2GB units are (4) 512MB chips.
    The 4GB units are (4) 1GB chips.

    There is no such animal as Pyra with (2) 2GB chips = 4GB.
    There is no such animal as Pyra with (1) ANY chips = failure

    Hypothetically a (4) 2GB chips = 8GB version could exist, but I don't think there are any compatible 2GB per chip parts to work from.

    I don't think ED and crew have ever said anything about configuring any units to (2) 1GB chips for 2GB units.
     
    Djhg2000 and levi like this.
  20. daveshah

    daveshah Still Fresh

    Joined:
    Aug 17, 2008
    Messages:
    23
    If the OMAP5 supports each channel having a different size, you could even create a 3GB Pyra - 2x1GB on one channel and 2x512MB on the other...
     

Share This Page

Loading...