The electrical distribution system in a data center is an important concept that many IT professionals overlook. Understanding the basics of your electrical distribution system can save downtime and aid in troubleshooting power problems in your cabinets. It's easy to understand if you follow the flow.
As with many introductory lessons in electricity, I will use the analogy of a flowing river to help describe the flow of electricity in a data center. The river is akin to wires, the amount of water is the voltage and the speed the water moves is the current flow also known as amps. So, when looking at an electrical system, think about a flowing river and the paths that it must take to get to and from its source to the ocean.
External Power Sources
The preferred source of electrical power is delivered to a data center by the local utility company. Once that utility power enters the building, its first stop is usually going to be the ATS or Automatic Transfer Switch. This electro-mechanical device is fed power from two or more sources – a Primary and an Emergency. While the primary source is available, it sits happily and flows power to a series of distribution breakers, often called "switch gear." These large breakers are designed to carry hundreds or thousands of amps and pass that power to your uninterruptible power supply (UPS) units and other facility infrastructure: lighting, HVAC, fire life safety systems, etc.
If the Primary source becomes unavailable, the ATS triggers the emergency source. In our data center example, that means our on-site generators start up. It typically takes between 9 to 12 seconds for the generators to come up to speed to allow for full power generation. Once the ATS sees that the generators have started and are ready to supply power, it will switch the load from the primary source to the emergency source. This is called an open transition because the load is removed from the primary source during the switch to the emergency source.
Once the power leaves the ATS and switch gear, it is no longer important to know whether you are connected to the primary or emergency sources. The next step in the power flow is the UPS. Like a dam, the UPS system takes an untamed river and transforms it into something safe and usable: An uninterruptable source of power to your server cabinet.
This is achieved by a bank of batteries sized to support the IT load. The batteries are connected in-line with the supply and load, so while the ATS senses a utility outage and starts the emergency generators, the IT load is still supplied power. A typical UPS battery system is designed to support the IT load for a maximum of 10 minutes.
Another benefit of the UPS system is the ability to clean the incoming utility power. Normal utility power voltages vary wildly depending on what other loads the service is supplying. These voltage fluctuations are detrimental to power supplies in servers and can shorten their life spans or worse: destroy them. This is why most home computers have a surge suppressor to prevent power spikes from damaging your equipment. UPS units clean electrical power by converting utility power from AC to DC and back to AC again:
Power Distribution Units
After protecting and cleaning the power, the UPS power will flow to a group of power distribution units (PDUs). At this point, the voltage will normally be 480vac which is too high for most IT equipment. The PDU or a separate transformer has to to convert the 480 volts to a more usable voltage like 120vac or 277vac. Once the voltage is converted, the power is then distributed to electrical outlets via a common electrical breaker.
PDU technology has advanced, like all data center equipment, from simple breaker panels to complex devices capable of measuring IT loads, load balancing, alarm and fault monitoring and even automatic switching between two power sources instantly during an outage.
The final piece of equipment in the data center electrical system before your server is a power strip. Power strips are often mistakenly referred to as PDUs. The power strip is mounted in a cabinet and contains multiple electrical outlets, not electrical breakers. You plug the server power cord into the power strip, not the PDU. And from here, the flow of electricity finally reaches the sea of servers.
Here's a basic for a data center electrical distribution system:
Our data centers are complex, and the entire building infrastructure is critical to its continuous operation. The electrical distribution system is at the heart of any critical facility, and it's vital that everyone working in and around critical sites knows at least the basics of your electrical distribution system.
In Part 2 of our "Building a Data Center" series, we'll cover how we keep the facility cool.