Investigating Reddit's Exploded 9800X3D CPU | AMD Ryzen Post-Mortem

Overview

The used AMD Ryzen 9800X3D and MSI MAG X870 Tomahawk WiFi motherboard we purchased are heavily damaged and likely totally dead. We bought them from the Reddit user, who posted it to r/pcmasterrace after the incident.

The user said this:

“I built a new PC last night, and I couldn't figure out why it wouldn't POST. When checking for bent pins, I found that it killed itself. :(“

The situation immediately reminded us of the 7800X3D failures following its launch. Since that came down to a fix in the motherboard BIOS, other users naturally started asking questions. When asked about BIOS, the user said:

“I don't know what the BIOS rev. was because it never worked, unfortunately. It has whatever BIOS it shipped with.

It burned up before I ever entered the BIOS. The socket was brand new when I put the chip in, no bent pins or anything.

Edit: Since people have many theories about pins being bent, here are some more photos: https://imgur.com/a/b5x4BVh

The plastic is melted and I think this deformed them. It's a chicken or egg thing.

This was not watercooled and there was no water around this PC.

I don't see any stickers with the BIOS rev on it.”

The above images are some photos the user provided. As you can see, it’s bad. But we have our own evidence and microscope shots to get into because the Reddit user was willing to sell us the components.

The root cause last time, back with the 7800X3D, mainly came down to excessive voltage over an extended period of time, possibly in conjunction with poor internal thermal protections, and motherboard overcurrent protection (OCP) being set way too high. This led to dielectric layer breakdown at the transistor level in a runaway failure state, causing shorts and internal temperatures hitting the point of melting silicon.

In simple terms, too big zap force make CPU go boom.

Ultimately, the issue was fixed via BIOS updates leveraging stricter controls around VSOC, as handed down from AMD to the board manufacturers.

But this new failure likely isn’t the same. 

A lot of the internet pointed toward some potential user-caused damage from improper installation. We had some tell us not to buy the part as they were worried we’d be paying for someone’s mistake. While we really appreciate the concern, it’s the audience that puts us in a position to be able to afford to buy out catastrophic failures like this when they happen. We view it as part of the business: If there’s a chance of something educational coming out of it, we want to be there to learn it. Worst case, some guy installed his CPU wrong and he gets helped out, never makes the mistake again, and we all get some cool shots of a crazy failure. 

Best case: You never know when someone has discovered the next massive failure that could affect thousands of users in an anti-consumer cascade, like Intel’s instability, and it’s important for us to treat all of these events seriously and get the parts before they disappear into the hands of manufacturers with motive and opportunity to hide the results. Third-party investigation is important to transparency.

So, that’s why we’re OK with spending money on something that unveils nothing every now and then, because every time it unveils something important, we can help keep a company honest. And when it doesn’t, someone at least got a bailout.

Let’s get into the damage report.

Damage Report

The bottom of the 9800X3D (read our review) has two major burn sites with discoloration, charring, a hardened, enamel-like substance, and individual points of concentrated scorching where the tips of the LGA socket pins were located. Also, it smells bad.

The motherboard’s socket has damaged areas directly matching the pattern from the bottom of the CPU. The pins themselves are slightly bent out of alignment and discolored, with a few showing an intense bluing as seen on heat-treated metal. Solder from under the socket flowed up onto some of the pins, and the bed of plastic is scorched, appearing molten. There’s also one pin near the bottom that’s badly bent over, without visible burn marks. It, too, smells bad.

The plastic edge of the socket is damaged in two major locations. The bottom left corner shows marring along a slight angle on both the left and bottom borders of the socket area. Along the bottom edge, the plastic alignment pin is clearly misshapen, appearing crushed.

Moving to the bottom of the socket actuation mechanism (SAM, as AMD calls it, also known as ILM in Intel-speak), the hole in the base shows two points of physical wear accompanied by discoloration on the PCB itself. This area differs in design between the AM5 socket manufactured by Foxconn, which this is, and the other AM5 socket, made by Lotes. We’ll get into that later.

A final sign of damage is a missing chunk of plastic between pins 3 and 4 on the ATX 24-pin header, which we wouldn’t have expected.

In technical terms, this board is f***ed up.

We haven’t observed any damage to the back side of the board underneath the socket or anywhere else. That includes the top part of the SAM, which some people thought looked bent in the user’s photos. It is not bent. That may have been lighting and those users are incorrect.

The CPU and motherboard are irreversibly and catastrophically damaged.

Knowing all of the aftermath, we need some testimony.

Other key pieces of first-hand testimony from the Reddit user include the following:

In reference to the pins, the user said, “They were definitely not bent prior to installing the CPU. The CPU is bulging where it's burnt and the mobo socket is slightly melted.”

When asked if all power cables were plugged in, the user replied, “Yes everything was in place. I checked all connections when it wouldn't post and everything was fine. Nothing was in the socket when I put the chip in. That was the first time it was ever opened.

I did not update BIOS prior to trying to boot.”

This is about the time we stepped in, offering to purchase the parts from the user for their full retail price.

Hypothesis

Moving on to our hypothesis, we thought that the cause of this failure was a dead short to ground, and not an inherent flaw in the CPU or the motherboard BIOS.

We thought this could have happened two ways and kept an open mind.

Option one would be a defect in the socket or the motherboard as a whole – manufacturers can always screw up. If the factory misaligned the socket in relation to the underlying PCB, or damaged the plastic surround of the socket, then it could have led to misalignment and a short.

Option two is improper installation. If the user installed the CPU into the motherboard in such a way that caused the outer socket damage and misalignment of the CPU in relation to the socket pins, then that would be sufficient to cause a short as well.

Detailed Support and Evidence

It’s possible that the issue was caused by something else we can’t account for, but given the evidence we have, we believe that improper installation is the most likely cause.

Now for the details and evidence to support our hypothesis.

The CPU appears to have been installed with an offset within the socket by ~1.01mm diagonally to the left and down at the bottom edge where the socket’s lower guide pin is. This is enough to cause the pins to mate with the wrong pads.

To determine this, Jeremy on our team took a measurement based on converting the known width of the guide pin (1.3mm) and then pixel measuring the microscope shots by pixel counting.

It took a while. There’s a lot of pixels.

This got us the distance between where the CPU’s guide cutout is supposed to sit versus where the plastic is crushed on the guide pin, then we converted it to millimeters. It’s not exact, but it’s very close, and establishes the basis of our hypothesis.

This is generally supported by the marred edge of the socket at the bottom left, and possibly faint rounding off along almost the entire bottom interior edge. This would also result in the bottom edge of the CPU being raised away from the socket in the Z-axis, adding light or poor pin-to-pad contact into the mix, i.e., higher resistance.

Taking a closer look at the bottom of the CPU shows that it was rotated by 1.0-1.4° clockwise, relative to the socket, when the burn occurred. This is based on the pattern of pinpoint scorch marks left by the pins. We carefully inspected the burn marks on the pads where you can see the point of contact, which is evident not only from the natural scratches caused by contact, but by the residue surrounding the burn sites. They should contact the pins in parallel rows, but the marks clearly show an angle. As you go along the pins, you see it diverging. Measuring in software, we came to the 1-1.4 degree offset.

Analysis of the burned pins in the socket shows that all of them are VDDCR (core voltage) pins that are down and/or left of VSS (ground) pins. If the CPU was shifted down and left in relation to the socket, then the burned VDDCR pins would have been in contact with VSS pads on the CPU, causing a catastrophic short circuit.

The fact that the top row of pins in the bottom half of the socket has no burns at all supports this, as those pins would have been in contact with the CPU substrate, not any pads. There’s nothing else conductive to touch in that direction until you get across the gap to the upper field of pads.

That said, not all VDDCR pins that were nearby to ground got burned. We can speculate on a few reasons for this. 

The skewed angle of the CPU in the Z-axis and the unknown pivot point would make the offset inconsistent across the 2D plane of pins. Also, it’s possible that some pads on the single-CCD 9800X3D are not connected internally.

The marring and discoloration where the tip of the SAM (or ILM) touches the motherboard PCB is less conclusive evidence as compared to the rest. 

Best case scenario, it’s just physical wear unrelated to the burning incident. Worst case, it’s a location where core voltage was able to find another path to ground and scorched the PCB.

Finally, the SAM doesn’t show any signs of damage or bending that we can detect. It’s made of springy steel that readily returns to its original state after flexing – assuming it isn’t pushed past elastic deformation into plastic deformation, or in other words, made to permanently bend. 

We think the area suspected of damage in the user’s photo is an optical illusion – a reflection or light shining through the top of the case.

Through all of this, we want to make it clear –  the user shouldn’t be bashed for this. This sort of thing can happen to anyone, and that’s especially true with newer platforms. We’re grateful that the user was willing to sell us the parts to inspect, because we had fun working through the diagnostic.

We’ll use it as a learning opportunity.

Concern about SAM/ILM Styles

All of this led to one observation that we hadn’t realized before that we’d like to briefly cover.

We have a new concern about the two different styles of AM5 socket by Foxconn and Lotes. Research on this piece led us to realize that there are actually a lot of differences between them, in ways that should mostly be immaterial to functionality. 

For example, the Foxconn socket has solid pins and solid plastic, while the Lotes socket has split pins and a more skeletonized plastic molding.

The discoloration on the burned motherboard where the tip of the SAM touches down raises our concern in general, even outside the scope of this piece, because the metal comes in direct contact with the PCB, directly on top of surface-level traces. 

We don’t think this is what happened here, however: Repeated socket actuations could theoretically cause the SAM to abrade through the mask and create a short.

The Lotes version of the socket has the insulation sheet run across this entire hole, adding a barrier of mechanical protection. 

We aren’t saying that the Foxconn socket will have problems, just that we would like to see more conformity to a single style between vendors.

We took a brand-new sample of the same model X870 Tomahawk motherboard and repeatedly opened and shut the SAM 16 times with a CPU in the socket to see if this area of the board would show similar wear or any change at all.

At the beginning of the test, we saw some disturbance of the protective sheet on the left, and a small spot of what looked like adhesive on the right, surrounded by an extremely faint discoloration. At the end of the process, it looked virtually unchanged, and nothing like the burned motherboard.

This pushes us to think these spots were subjected to heat, or maybe a direct short as we mentioned before.

Conclusion

The short version of this is that, after looking through everything, we think that the result of all this is improper installation, which is something a lot of Reddit users were pointing out when looking at the photos online. 

Regardless, we definitely think this story was worth doing. First of all, it was a lot of fun to piece together a mystery. It also allowed us to possibly learn something major as you never know when some random PC builder might encounter something like the next Intel instability issue, where it might initially be perceived as user error but then proves to be a big, valid issue. 

In this case, this issue doesn’t seem to be something for people to worry about, though users should pay attention to how they're installing their CPUs. It’s possible that a situation like this could have happened if you tried to install the CPU with the motherboard oriented vertically. It’s important to lay the system flat to install it properly and to be careful with where the guides are aligned. You can also slightly wiggle the CPU a bit once it’s installed, but shouldn’t do it hard, to see if there’s something wrong.