May 2016
Volume 31 Number 5
[Editor's Note]
Going Solid
By Michael Desmond | May 2016
.jpg "Michael Desmond")
The nuclear meltdown at Three Mile Island (TMI) in 1979 was a seminal event that’s informed disciplines from crisis management and crew training to man-machine interface design. One fascinating aspect of the TMI disaster—and a very similar 1977 incident at the Davis-Besse plant in Ohio that foreshadowed it—was how the plant crew’s response was guided by wisdom received from another domain.
A lot went wrong after the turbines tripped and shut down at TMI, but crucial to the event was a stuck valve that, unknown to the crew, was releasing coolant out the top of the reactor’s pressurizer tank. That tank helps control the pressure of water in the reactor cooling system, ensuring it stays liquid even at extreme temperatures. It also maintains a bubble of compressible steam to absorb damaging forces—known as a water hammer—that can occur when fluid flow is disrupted, such as when a valve closes or a pump turns on. The TMI plant operators, many of whom were trained and served as reactor operators in the U.S. Navy, were drilled from day one to never let the pressurizer fill completely with water—a condition known as “going solid.”
James Mahaffey, author of the book “Atomic Accidents” (Pegasus Books, 2015), says the naval proscription against going solid reflects the unique character and operating environment of submarine reactors and cooling systems, which are compact, lightweight and (compared to massive commercial reactors) very low power.
“Characteristics of a small reactor do not scale up perfectly to a big reactor,” he says. “The primary coolant system on a power reactor is made of very big, thick pipes, and you try to reduce cost by minimizing the number of turns. The turbine hall is huge.”
By contrast, the lightweight plumbing and tight bends in a submarine reactor cooling system pose a vulnerability. Says Mahaffey: “A primary coolant loop water hammer in a sub would possibly mean loss of the boat and everyone in it.”
It’s deadly serious stuff, and it colored the choices the Navy-trained crews at TMI and Davis-Besse were forced to make. Both reactor vessels reported low pressure and rising temperatures, indicators of dangerous coolant loss, yet the pressurizer tanks reported high water levels that would mean the reactor vessels were full of water. Unaware of the stuck-open pressurizer relief valve and a plant design flaw that trapped steam in the reactor vessels, each crew had to choose—cool the reactor or prevent the system from going solid. In each instance, plant operators chose the latter, overriding the automatic emergency core cooling system (ECCS) and denying critically needed water to the reactor core.
Said Mahaffey of the TMI incident: “They were under strict guidelines to never let the pressurizer go solid, and yet it was. The internal stress to meet this guideline was so severe, they left the rails and violated another guideline (shutting down the ECCS).”
Hidden among the hard lessons of TMI is the risk that occurs when received wisdom is applied across domains. The reactor operators were confronted with this risk in the most stark possible terms, but the challenge exists in every discipline. Developers moving among platforms, languages, organizations and projects must be aware of the preconceptions and habits they bring with them, and be careful not to fall into traps disguised as good practice.
Michael Desmond is the Editor-in-Chief of MSDN Magazine.