The northeast blackout of 2003 really really opened utilities eyes to the potential of a grid down event. It was followed by a software bug in the alarm system. The blackouts proximate cause was a software bug in the alarm system at the control room of firstenergy, an akron, ohio. Imagine the severity and scale of damage that could have transpired had the blackout been caused intentionally by a hacker. Software process methodology for code development and many other ways to find problems and increase confidence no single activity or approach can guarantee software quality beware of bugs in the above code. The northeast blackout of 2003 was a widespread power outage throughout parts of the northeastern and midwestern united states, and the canadian province of ontario on august 1428, 2003, beginning just after 4. Canada power system outage task force final report on the august 14, 2003 blackout in the united states and canada. In this page, i collect a list of wellknown software failures.
Reliability research for sequential software has made tremendous progress recently 9, 17, 41. A technician had shut it off to upgrade the software, and then gone to lunch. Software bug contributed to 2003 blackout ars technica. The northeast blackout of 2003 was one of the most critical blackouts in the history which affected more than 50 million people in both the united states and canada.
Concurrency bugs are serious 2 northeast blackout of 2003 the northeast blackout of 2003 was a widespread power outage that occurred throughout parts of the northeastern and midwestern united states and the canadian province of ontario on thursday, august 14, 2003, just after 4. Software failure cited in august blackout investigation. Most places restored power by midnight, as early as 6 p. The task force responsible for investigating the cause of the aug. From electronic voting to online shopping, a significant part of our daily life is mediated by software. Feb 11, 2004 turns out after all the various speculation on reasons, the big northeast blackout from last summer was the result of a software bug that was only just discovered and not as many had speculated. Blog exploitable vulnerabilities in cyber systems rma. It turns out that a deeply embedded bug was triggered by a series of events on august 14th, and when a backup server kicked in, it could not keep up. Turns out after all the various speculation on reasons, the big northeast blackout from last summer was the result of a software bug that was only just discovered. A massive outage like argentinas could happen in the us.
Its been 16 years since the 2003 blackout that left metro. The operators didnt fully understand their system the monitoring computers failed power lines failedand as some failed, other had to carry more of the load, so they heated up and sagged. In other areas, it took nearly a week or two for power to be restored. Most did not get their power back until two days later. This category has the following 4 subcategories, out of 4 total.
According to news reports in april of 2004, a software bug was determined to be a major contributor to the 2003 northeast blackout, the worst power system failure in north american history. Technical analysis of the august 14, 2003, blackout nerc. However, a nerc report dated june 20, 2003, shows that the slammer worm had a significant impact on some utilities. The report makes clear that this blackout could have been prevented and that immediate actions must be taken in both the united states and canada to ensure that our electric system is more reliable. Argentinas blackout and the stormbattered future of the. In 2003, the infamous northeast blackout left 50 million people in eight states without power. Software bugs have caused many realworld problems, e. Finding bugs is critical to solving those problems. Today, they are of increasing concern due to the pervasiveness of multicore machines. A case study of the 2003 north american power outage. The southwest blackout of 2003 1218 words bartleby. But to expose situations that could negatively impact the customer, maintainability, and usability. How to easily make your aws based applications analyze themselves for reliability. Impact of northeast blackout continues to emerge by jeremy johnson and alex lefebvre 20 august 2003 the blackout that began last thursday, cutting electrical power to more than 50 million.
Tougher regulatory measures are in place, but were still a long way from a smart power grid. Every so often i teach an electrical engineering class, and towards the end we discuss engineering ethics. Almost forty years later, the same area was hit by the great northeast blackout of 2003, where over 55 million people experienced power outages as a result of a software bug. On thursday, august 14, 2003 a power outage affected an estimated 55 million people in canada and the united states. August 14, 2003 blackout in the united states and canada. Impact of northeast blackout continues to emerge world. Software bug contributed to blackout no, nothing to do with blaster, or microsoft. The cases include the 1977 new york city blackout, the 2003 northeast blackout, multiyear national annual lightningrelated electrical damage. Northeast blackout of 2003 your expert root cause analysis. Northeast blackout of 2003 a bug in general electrics energy management system stalled a local utilitys control room. The blackout was, in fact, triggered on a hot day by an untrimmed tree in ohio and was aided by a hung alarm system.
On august 14, 2003, more than 50 million people across eight u. Status of the northeastern power grid before the blackout sequence began. In 2003, at the time of the blackout, the north american electricity grid had grown to include four distinct grids, called interconnections. Feb 14, 2006 tracking the blackout bug kevin poulsen, securityfocus 20040407. At first, power providers may only notice a cascade of overloaded transmission lines failing in rapid succession something that happened during the 2003 blackout, which was caused by an. The redundancy in the american electric grid is astounding.
Resilience metrics in energy master planning for communities. A collection of wellknown software failures software systems are pervasive in all aspects of society. The blackout s primary cause was a software bug in the alarm system at a control room of the firstenergy corporation in ohio. On august 14, 2003, an enormous electrical blackout struck the northeast and upper midwest of the united states and a large part of western ontario, canada, leaving about or aboot in the great white north 55 million people without electricity for a period lasting between 7 hours and 2 days. Is current energy systems designed for resilience to unexpected hazard. A massive power outage like argentinas could happen in the.
The us got a taste of massive grid failure in 2003, when an overgrown tree in ohio tripped a transmission line and a software bug failed to alert the utility. What began as a software bug in the electrical grid of ohio soon escalated into a catastrophic event that left nearly 45 million americans without power, ultimately becoming the worst blackout in north. Effective approaches to detecting concurrency bugs before software is released are sorely desired. Its been 15 years since the 2003 blackout that left metro detroit in the dark. The blackouts primary cause was a software bug in the alarm system at the control room what should have been a manageable local blackout cascaded into collapse of the entire northeast region. Northeast blackout of 1965 or north america blackout of 1965. The 2003 northeast blackoutfive years later scientific. Blackout page 2 july 7th peoples independent inquiry forum. Grid better prepared to prevent a repeat of the 2003 blackout. To protect against all types of threats to the grid including natural and.
American blackout is the best representation of what would. We examined the public health effects of the northeast blackout of august 2003 and the emergency response to the blackout by the new york city department of health and mental hygiene dohmh. Back in 2003, a bug in one plants control room alarm system ultimately resulted in the forced shutdown of more than 100 power plants, cutting power to an estimated 55 million people in the us and canada. Software crisis in software engineering ecomputernotes. Northeast blackout of 2003 or north america blackout of 2003. The northeast blackout in 2003 has been one of the major power system failures in the history of north america. Unfortunately, many existing bug detectors suffer from high runtime and space overheads as well as scalability and precision issues. The northeast blackout of 2003 was a widespread power outage throughout parts of the northeastern and midwestern united states, and the canadian province of ontario on august 14, 2003, beginning just after 4. At the time, it was the second most widespread blackout in history, after the 1999 southern brazil blackout. Indeed, the fragility of the american power grid was starkly illuminated by the 2003 northeast blackout in which a software bug caused a simple local power issue to cascade into an outage.
Detecting concurrency bugs through sequential errors. The bug resulted in performance problems for many of the sites simultaneously and required disabling of the software until the bug was fixed. Due to this blackout, large number of power plants failed and approximately fifty million customers faced power loss which resulted in a huge financial threat to currency. It was a warm day and the wind died, so there was less cooling. The northeast blackout of 2003 was a widespread power outage that occurred throughout parts of the northeastern and midwestern united states and the.
A previouslyunknown software flaw in a widelydeployed general electric energy management system contributed to the devastating scope of the august 14th northeastern u. The blackout occurred when a software bug in firstenergys energy management system was triggered and prevented controllers from hearing any alarms. Why a power grid attack is a nightmare scenario thehill. See figure 4 generation had expanded to include a range of energy types, although coal remained the single largest source. If you remember the northeast blackout of 2003 in which a software bug plunged 55 million in to the dark then you know the possibility of computer failure taking down the grid. Virtually every power plant in the eastern half of the usa experienced frequency and voltage perturbations. A number of factors and failings came together to make the august 14th northeastern blackout the worst outage in north american history. The massive power outage affected areas in eight northeastern and midwestern states, as well as ontario, canada.
Feb 12, 2004 software bug contributed to blackout no, nothing to do with blaster, or microsoft. The lack of an alarm left operators unaware of the need to redistribute power after overloaded transmission lines hit unpruned foliage, triggering a race condition in the energy management system what would have been a manageable. New york news coverage of the northeast blackout of 2003. The blackouts primary cause was a bug in the alarm system.
A software bug at firstenergy corporation in ohio caused this power outage. A bug that can occur in both an electronic or software system. The north american power grid is structured in such a way that this single failure cascaded to larger and larger geographic areas, leading to the loss of power to more than 55 million people. The blackout that exposed the flaws in the grid the new. By thinkreliability staff on august 14, 2003, over 50 million people in the u. A surge of electricity to western new york and canada touched off a series of power failures and enforced blackouts yesterday that left parts of at least eight states in the northeast. Designing practical software bug detectors using commodity.
Wikimedia commons has media related to software bugs pertains to software bugs and techniquestools used to manage them subcategories. History has been witness to many such failures, some of which are listed below. Jan 05, 2015 the northeast blackout of 2003 1 ranks high on the list. Unfortunately, many existing bug detectors suffer from high runtime and space overheads as. While everything you just posted is accurate, none of it rises to shtf. The nations electrical system has undergone some revampingmaybe still not enoughsince tens of millions in the. A software bug failed to redistribute the overloaded transmission lines. It is hard to believe 15 years have passed since a massive power outage left 50 million people in the northeast and midwest in the dark, including metro detroit. Software bug contributed to blackout kevin poulsen, securityfocus 20040211.
For instance, in the 2003 northeast blackout in usa, a software bug in the alarm system at a control room of the firstenergy corporation, located in ohio, eventually caused a widespread power outage throughout parts of the northeastern and midwestern united states and the canadian province of ontario, affecting an estimated 10 million people in. The therac25 radiation therapy machine, and the 2003 northeast blackout are both examples of a race condition with disastrous consquences. Technological software failures bugs, code problems, unknown loopholes. Software failures in these technologyoriented areas have led to considerable loss in terms of time, money, and even human lives. Blaster worm linked to severity of blackout computerworld. I will start with a study of economic cost of software bugs. Internet service issues internet service provider isp failures can. Blackout hits northeast united states a major outage knocked out power across the eastern united states and parts of canada on august 14, 2003. Neglect and error cause all kinds of problems in industrial operations, but. Northeast blackout caused by a software bug techdirt. The main cause of this disaster was a software bug in the energy management system used by. Consulting specifying engineer network controls for. The 2003 northeast blackout was one of the most important power system failures in the north american history.
August 1416, 2003 a software bug in the energy companys alarm system left causing operators to remain unaware of the need to redistribute load after overloaded transmission lines drooped into foliage. While most power outages are caused by storms or utility company mistakes, a growing number of these outages are being blamed on everything from our deteriorating. Fifteen years ago, a software bug resulted in over 50 million people losing power, crippling the northeast power grid. Final report on the august 14, 2003 blackout in the united.
Northeast blackout of 2003 how can a software bug of an energy company in ohio lead to a blackout in new york city. Considered one of the worst blackouts in history, the northeast blackout was caused by a software bug. Jul 04, 2016 software crisis in software engineering. Photo of a toronto street corner during the 2003 blackout by john r.
The blackout that exposed the flaws in the grid the new york. Systemic risk just like the system that allowed the 2003 northeast blackout, the financial system has become real time by virtue of todays instruments and activities marktomarket valuations, etc. The northeast blackout of 2003 is the second most widespread power outage in. Investigators found several factorsalarm system issues, software bugs, and an overloaded. Mar 11, 20 the beginnings of one of the largest power outages in u. The recordbreaking outage, caused by a software bug in the control room of an energy company in ohio, had a catastrophic effect on the entire. On august 14, 2003 there was a widespread power outage that occurred throughout parts of the northeastern and midwestern united states and the canadian province of ontario. Power grid security fears surge since 2003 blackout. Threats and attacks computer science and engineering. The overall objective of testing is not to find every software bug that exists. Race catcher race condition analysis application reliability monitoring. A massive power outage like argentinas could happen in. Jul 20, 2011 eight years ago a computer virus accidentally killed power to the entire northeast. Its been 15 years since the 2003 blackout that left metro.
Ten years later, we are still grappling with concerns over the vulnerability of the power grid. The blackouts proximate cause was a software bug in the alarm system at the control room of firstenergy, an akron, ohiobased. I bring up the usual examples three mile island, challenger, and the hyatt regency walkway collapse but id like to use an example or two thats more specific to ee rather than a mechanical general design problem. A massive power outage like argentinas could happen in the us 4. One of them was buried in a massive piece of software compiled from four million lines of c code and running on an. Some recent software failures caused by software bugs. Canada power system outage task force final report on the august 14, 2003 blackout. Failure to implement the recommendations would threaten the. The northeast blackout struck 15 years ago today extremetech. A surge of electricity to western new york and canada touched off a series of power failures and enforced blackouts yesterday that left parts of at least eight states in the northeast and the. Is your simulators electrical model meeting industry.