On April 7th, the OpenSSL Project released an update to address a serious security flaw (CVE-2014-0160), which allows remote attackers to obtain sensitive information from process memory via crafted packets that trigger a buffer over-read, as demonstrated by reading private keys, related to d1_both.c and t1_lib.c, aka the Heartbleed bug.
After notification of this vulnerability we began a close examination of our services to determine any that may have been affected. Both the SoftLayer customer portal and API are serviced behind hardware load balancers and neither the hardware load balancers nor the software running on the servers behind them were found to be running vulnerable versions of OpenSSL. This was confirmed by the hardware vendor and direct testing as well. During these tests it was discovered that certain nodes of our Object Storage cluster were running a vulnerable version of OpenSSL. The software was immediately patched to remediate the issue. Although there is no indication that this vulnerability was exploited, the subset of customers potentially affected has been advised of precautionary measures to ensure continued security.
Additionally, our team forced updates to all of our internal operating system update mirrors as soon as patched versions were released by their publishers. Our system automatically checks for and updates all operating system versions hosted on our mirrors, but due to the urgency of this exploit, manual updates were run as quickly as possible to have patched versions available sooner.
Due to the nature, surface area, and severity of this vulnerability, we recommend revoking all possibly compromised keys and reissuing new certificates for any service secured using the OpenSSL library. The rekeying process can vary depending on your Certificate Authority (CA) and you should contact them if you have questions on how to complete this process. This OpenSSL vulnerability has major security implications for a wide range of operating systems and applications and may necessitate rebooting your hardware (or restarting services) to ensure all services linking against the affected code use the updated version of the OpenSSL library. We also recommend that you patch all of your servers and change passwords as soon as possible. Take this opportunity to review your overall password strategy including password strength and password sharing across sites.