The US Department of Energy (DoE) has boosted its high-speed research communications network by extending 100G connectivity across the Atlantic. The new capabilities will improve network speeds between US research sites and European facilities in readiness for big data transfers from the Large Hadron Collider (LHC) at Cern, near Geneva Switzerland.
The DoE's Energy Sciences Network (ESnet) has deployed four new high-speed transatlantic links to giving researchers at US national laboratories and universities ultra-fast access to scientific data not only from the LHC but from other research sites in Europe. ESnet's transatlantic extension will deliver a total capacity of 340 gigabits-per-second (Gbps), and serve dozens of scientific collaborations. To maximise the resiliency of the new infrastructure, ESnet equipment in Europe will be interconnected by dedicated 100 Gbps links from the European networking organisation Géant.
ESnet is funded by the DOE's Office of Science, and managed by Lawrence Berkeley National Laboratory. It provides advanced networking capabilities and tools to support US national laboratories, experimental facilities, and supercomputing centres.
Because most large scientific collaborations are international in scale, undersea cables are especially important for research networks such as ESnet, and national networks around the world have been collaborating to test high-speed undersea connections for the past 18 months. Last year, six organisations (ESnet, Internet2 and Canarie in North America, with Surfnet, Nordunet and Géant in Europe) deployed the world's first 100 Gbps research link across the Atlantic. More recently, New Zealand's Reannz deployed an experimental 100 Gbps link across the Pacific.
ESnet installed its first European network node at CERN in mid-September, and is now deploying other equipment necessary to bring the first link online. The plan is for all links to be commissioned and in production by January 2015. The LHC is expected to resume operations next spring, at which point it will be generating significantly more scientific data every day.
When ESnet engineers began to study the idea of building a new 100 Gbps network between the United States and Europe, a primary concern was ensuring that the service would be robust and built from multiple underlying links, so that if one went down, researchers could still rely on sufficient bandwidth. Based on data collected for years by Caltech physicist and networking pioneer Harvey Newman, the ESnet team understood that multiple cables are sometimes cut simultaneously.
ESnet's transatlantic extension will use network capacity leased from the owners of four undersea cables. Each cable has several pairs of optical fibre, with one fibre in each pair used for sending data from the US to Europe, and the other used for data moving in the reverse direction. Each fibre is lit using dense wavelength division multiplexing (DWDM), allowing between 40 and 80 different signals to be sent on a single fibre simultaneously.
Ship's anchors cause most cable cuts near the coastline. Even though undersea cables are designed with extra armour near land, and are buried up to a metre deep, the anchors dropped by supertankers riding out storms can still damage the fibre optics inside the cables. Undersea earthquakes pose another hazard. According to Joe Metzger, ESnet's lead engineer for the project, one such quake triggered an underwater landslide that took out several cables all laid through the same channel. However, while each cable operator knows the locations of its own cables, there's no accurate database covering all cable paths.
Another major issue is the time needed to repair a break. ESnet can usually have an engineer at the site within four hours of a problem being reported in its network connecting sites across the terrestrial US. With an undersea break, it can take a repair ship up to a week just to find the problem. Once located, it can take four weeks or longer to repair the undersea break.
Because the scope of the ESnet project breaks new ground for the international networking community, Metzger gave a presentation on the two-year project at the 2014 Technology Exchange conference in late October 2014.
