Topics in this article
At the beginning of September 2022, northern California experienced an unprecedented heatwave. Temperatures in the otherwise temperate region reached more than 110 degrees Fahrenheit (43 degrees Celsius) in places. Understandably, residents turned up their air-conditioning, pushing the region’s power grid to its breaking point.
That it didn’t actually break is in part thanks to the rapid response of a brand-new public–private partnership between large energy customers and the California Energy Commission – not to mention the tireless work of NTT’s data center operations teams and many others in the region.
New law incentivizes reduced power consumption during extreme events
To understand how and why the response was so rapid and impressive, we have to back up a bit and head over to the California legislature. Earlier this year, Assembly Bill No. 205 became law. This particular law, designed to expand the development of renewable energy sources, also enabled the creation of the Demand Side Grid Support (DSGS) Program, which offers incentives to electricity customers to minimize their consumption during extreme events in order to reduce their load on the grid and the risk of blackouts.
Organizations can do this in many ways. A factory can shut down for one day, for example. But for data centers, that's not an option: they need uninterrupted power to keep vital equipment running and to prevent that equipment from overheating and failing. In Sacramento, for example, our three fully integrated facilities offer 52.7 megawatts (MW) of critical, high-density IT capacity.
So, we needed to help in a different way.
When temperatures rise, every megawatt counts
Under normal circumstances, data centers operate with multiple levels of redundancy. If grid power stops flowing for any reason, diesel-powered backup generators protect against service interruption for data center clients. However, to reduce pollution, there are legal limits to how often a given facility can run on generators. We can operate them only when there’s a utility failure, and then only for a certain number of hours.
In the event of major electric grid stress, every megawatt counts – and this is what gave impetus to California’s DSGS Program. It’s designed to be a bulwark against rolling blackouts as the state weans itself off electricity powered by fossil fuel.
At the start of September, we were still hammering out details with the state on our agreement to join the Program, so specific protocols and processes hadn’t yet been put in place – and then the temperatures started rising.
The Thursday before Labor Day in September, Nick Jones, our Senior Director for Critical Facilities Regional Operations West, got a call from the Executive Director of the California Energy Commission asking if we’d be able to shed our load on the grid and move it to generators should the Governor declare a state of emergency. Because of the statutes on why and for how long data center generators could run, we had to ensure that we complied with the law.
We had meetings with the government all weekend to finalize the plan. By Labor Day on Monday, all the permissions had been signed off and our team was ready to go to generators.
The only hurdle left was to actually start running the generators – something that data center operations teams don’t tend to do intentionally. We normally have to be reactive to utility loss, but in this case, we were proactively moving to generators.
NTT staff came together to work on the data center power switchover from the grid to generators
Generating backup, one engine at a time
The task before our operations team was to make the grid the backup to the generators, which is not as easy as you may think. Each generator is independent, so our team had to put a process in place to go to each of the engines one at a time, get it started, confirm it was operating properly and then transfer the load from the utility onto the generator.
Turning on all the generators at once isn’t just impractical. If one were to fail for whatever reason, it could put the entire system at risk. We opted for redundancy – a phased approach that would enable us to control the transition. If there was a fault or an issue with a piece of equipment, the team could just move on to the next system. So, we proceeded cautiously in the extremely hot and difficult conditions.
That week, we moved 33MW of power to generators during peak periods every day – enough to power nearly 33,000 homes – and there wasn’t a single service interruption for our clients. We needed all hands on deck, and they all jumped at the opportunity, even when the heatwave was affecting them in other ways. Some of our people were dealing with wildfires near their homes and still arrived to support us and their fellow Californians.
Through the lessons we learned this year, we know that we’ll be ready if this situation arises again next summer, and we’ll continue to find ways to be good stewards in the community.
Read more about NTT’s Global Data Centers and view our data center locations.
Ben Stewart is Senior Vice President of Operations at NTT Global Data Centers