When Microsoft announced its plan to build an underwater data center, Lathish Kumar Chaparala was enthused.
"During the initial implementation of Project Natick, I went to their website and saw the live feed from the underwater camera that was mounted in the data center," said Chaparala, senior program manager in Microsoft Digital's networking team, Microsoft's engineering organization, which develops and manages the products, processes, and services that Microsoft runs.
Little did he know that he and his team would later be hired to expand the network connectivity of this underwater data center so that it could be safely fished in the sea.
But the story begins much earlier.
We've seen the potential benefit [of developing an underwater data center] for the industry and for Microsoft. People reacted to our work as if we were going to the moon. In our eyes, we have only lived up to our charter: we accept challenging problems and find solutions.
– Mike Shepperd, senior research and development engineer in the Microsoft Research team
The idea for an underwater data center came out of ThinkWeek, a Microsoft event where employees shared offbeat ideas they felt the company should pursue. A creative idea came from collaborators Sean James and Todd Rawlings, who proposed building an underwater data center powered by renewable ocean energy that would bring super-fast cloud services to the crowded coastal population.
His idea appealed to Norm Whitaker, who was then running special projects for Microsoft Research.
From this,Natick Projectborn.
"Norm's team was responsible for making the impossible possible, so he began investigating the feasibility of an underwater data center that could be powered by renewable energy," says Mike Shepperd, senior research and development engineer at Norm's research team. Microsoft. to support research on the feasibility of subsea data centers.
It quickly became an all-Microsoft effort spanning engineering, research, and IT.
"We recognized the potential benefit to the industry and Microsoft," says Shepperd. “People reacted to our work as if we were going to the moon. In our eyes, we have only fulfilled our letter: take on challenging problems and find solutions.”
Project researchers hypothesized that a sealed container on the seabed in a nitrogen environment with low humidity and cool, stable temperatures would better protect servers and increase reliability.
"Once you're 20 to 30 meters underwater, you're weatherproof," says Shepperd. "You could have a hurricane whipping at your head and an underwater data center wouldn't be any smarter."
[Read how Microsoft is reducing its carbon footprint by tracking its internal use of Microsoft Azure. Find out how Microsoft Digital leverages modern network infrastructure to drive transformation at Microsoft.]
Strengthened internal engineering team
The Project Natick team worked with the network and security teams from Microsoft Digital and Arista to create a secure wide area network (WAN) connection from the subsea data center to the corporate network.
“We needed the connectivity they provided to successfully complete our project,” says Shepperd. "We also needed this connectivity to support the actual decommissioning process, which was very challenging because we deployed the data center in such a remote location."
In the spring of 2018, they installed a fully connected, secure data center 117 feet below sea level in the Orkney Islands, off the coast of Scotland. After being designed, erected, and gently lowered to the bottom of the sea, it would remain intact for two years. Chakri Thammineni, a network engineer at Microsoft Digital, assisted in this effort.
"Project Natick was my first engagement after coming to Microsoft and it was a great opportunity to work with a lot of people to develop a network solution," says Thammineni.
Earlier this year, the experiment was completed without interruption. And yes, the team learned that placing a data center underwater is actually a more sustainable and efficient way to bring cloud to coastal areas and achieve better data center responsiveness.
At the end of the experiment, the team needed to restore the data center so they could analyze all the data collected during their time underwater.
This is where Microsoft's internal engineering teams stepped in.
"To make sure we didn't lose any data, we needed to keep the data center connected to Microsoft's corporate network during our extraction," says Shepperd. "We accomplished this with a dedicated leased line that we use to connect the data center to our Microsoft facility in London."
For the same reasons, the extraction had to be done at exactly the right time.
"The Orkney seas throw up waves that can be 30 to 30 feet high most of the year," he says. "The team chose this location because of the extreme conditions, as it is a good place to demonstrate Natick's ability to deploy data centers virtually anywhere."
And then, as with so many other projects, COVID-19 forced the team to change plans. While developing a new data center recovery plan, the team discovered that corporate connectivity was down at the end of May 2020 and could not be extended.
“Requesting the equipment would have taken two to three months, and we had a much shorter timeline,” says Chaparala.
Shepperd challenged the team at Platform Engineering, a division of Microsoft Digital, to quickly transform corporate connectivity from Microsoft's London facility to the port of Natick, while ensuring the connection was secure.
Die Mission?
He made sure the servers were online until the data center could be rescued from the water, all without additional hardware.
“My role was to make sure he understood the importance of the request versus the schedule and to bring in the necessary teams and expertise to keep the data center online until it was safely pulled out of the water,” Chaparala says.
The stakes were high, especially with the investigation at stake.
"If we lost connectivity and shut down the data center, it could have affected the profitability of the research we've been doing up to that point," says Shepperd.
A seamless collaboration between Microsoft Research and IT
To solve this problem, the Core Platform Engineering and Microsoft Research teams needed to align their vision and workflows.
"IT teams can plan their work months or years in advance," says Shepperd. "Our research has a different timeline because we don't know where the technology will take us, so we had to work together, and fast."
Unable to get the hardware to the data center location, Chaparala, Thammineni, and the Microsoft research team had to come up with a redesign of the network. This led to the implementation of software-based encryption using a virtual network operating system in Windows virtual machines.
It's exciting to play a role in bringing the right engineers and program managers together around a common goal, especially so quickly. Once we had the right equipment, we knew there was nothing we couldn't handle.
- Chakri Thammineni, Network Engineer at Microsoft Digital
(Video) The fight over the internet, under the sea | CNBC Explains
With this solution in tow, the team was able to extend network connectivity from the Microsoft Docklands facility in London to the Natick data center on the Scottish coast.
"Chakri and Lathish constantly worked with us to close the gaps between what our research team knew and what these Microsoft networking experts needed to meet the needs of this project," says Shepperd. "Without the help of your teams, we could not have achieved our research goals so quickly and efficiently."
Lessons learned from the world's second underwater data center
HeResearch on Project Natick pays off in Microsoft's future work, particularly in terms of running more sustainable data centers that could support Microsoft Azure cloud services.
"Whether a data center is on land or at sea, the size and scope of the Natick Project is a viable model for the data center of the future," says Shepperd. "Instead of clearing acres of land for data centers, our customers and competitors are looking for ways to power their home computing and storage in more sustainable ways."
This experience taught Chaparala to assess the needs of its partner teams.
“We work with clients to understand their needs and develop goals and key deliverables that align,” Chaparala says.
Ultimately, the Project Natick story is one of interdisciplinary collaboration, and just in time.
"It's exciting to play a role in bringing the right engineers and program managers together around a common goal, especially so quickly," says Chaparala. "Once we had the right equipment, we knew there was nothing we couldn't handle."
Find out how Microsoft rebuilt its VPN infrastructure.
Find out how Microsoft Digital leverages modern network infrastructure to drive transformation at Microsoft.
hang tags:Azure data and storage,connectivity,data center,Microsoft research,red,Natick Project,sustainability,underwater data center