Blog

Security Notice (Updated)

This is an update to the security notice sent out Thursday, February 21, 2008.

After reviewing security logs and comparing file systems and source code to known backups as part of our security audit, we are relieved to report that we have not found any evidence that any systems or customer information were successfully compromised during the period that firewall rules were partially disabled and some internet-facing systems were under attack. Additionally it appears that at least some of what was reported as an attack was the work of worms targeting Windows vulnerabilities, which we do not use for our servers–traffic we do not normally see due to firewalls and other protections.

The main avenue for attack that we were most concerned with was certain systems that were being brought up to migrate services to that might not have been fully patched or locked down yet while they were in transition. Though it doesn’t appear there’s been any compromise, as an added precaution, we will still be reinstalling these systems and services, so you may expect some brief periods of down time over the weekend (much of this we’re doing anyway, to sync up deployed software versions with versions being installed on the new systems to ease transition). We’re also revoking all authentication and encryption keys and have requested a new SSL certificate, and will take additional steps to improve our auditing procedures and response time.

Once we have finished upgrading critical server software and have received the new SSL certificate, we will bring the shopping system back online. You will not be able to log into your Muonics web site account until this time.

Please accept our apology for any inconvenience or worry caused by our previous notice and the downtime involved. Though it appears to have been a false alarm, one can never be too cautious about these things.

Should you have any lingering concerns, we will be more than happy to reimburse anyone who has placed orders with us online, at any time in the past, for up to 12 months of credit activity monitoring from your choice of provider. You can contact me directly by phone or email any time between now and March 31, 2008 to arrange.

Thank you for your patience.

Michael Kirkham, President & CEO

What was this about?

Due to an apparent bug in a new WAN-edge firewall/switch we recently deployed, there was a window of about 30 hours in which some of the firewall rules were not active and several internet-facing systems were under attack. Although we did not have any immediate evidence that any systems or customer information were compromised, we sent notice to all all registered users of the Muonics web site and posted a notice on our web site apprising them of the situation, and took our ordering system offline to conduct a thorough security audit of our network and systems.

What happened?We’re going through major infrastructure expansion at this time, including further separating and moving server roles and network segmentation. As part of this, we deployed a new WAN-edge switch a few weeks ago, with more capacity and firewall capabilities to offload that role from older systems that we’re retiring. Some time within 48 hours of the notice, one of several ACLs (Access Control Lists, or firewall rule sets) became disabled on a route, the switch reported conflicting information about whether or not it was enabled, and it wouldn’t re-enable until the switch was rebooted. During this time, some traffic was allowed through that is normally blocked, and the affected systems began reporting an in progress attack.

Aren’t there other security measures in place?

Yes. We have several layers of security in place, including encryption of sensitive information, virus scanners, additional layers of firewall protection, authentication, and restrictions on which systems can talk to each other. Also, there are Port ACLs, VLAN ACLs, and Routed ACLs deployed on the WAN-edge switch; only the Routed ACL was affected. As some of these measures may be in transition during our infrastructure expansion, we felt it was prudent to be extra cautious about possible impact from the firewall issue, and notify all parties of possible concern.

Also, please keep in mind that we do not store credit card information beyond the last four digits after an order is completed, and session data (such as shopping cart information) is encrypted. Hence the only credit card information that would have been at risk would be those of orders placed during or after the window in which the firewall rules were disabled (of which there were none) and only if the attackers managed to breach several layers of security during that window. We took the shopping cart offline to prevent orders while our audit was underway, which included a thorough review of security logs and comparing the systems and source code to known backups.

Was any customer information been leaked?

After conducting a thorough security audit of our network and systems, we have not found any evidence of successful compromise. Nevertheless, we are taking additional precautionary measures such as reinstalling and patching certain systems and obtaining a new SSL certificate for the web site, and taking the opportunity to evaluate and improve our security measures.

If the risk was so low, why did you mention it?

We take security and privacy of customer information extremely seriously, and we believe you have the right to know, even if your information is not at risk.

When can I log in, and when will the shopping cart be back online?

We expect users will be able to log into the system and the shopping cart to be back online as soon as a few software upgrades are complete and we have received a new SSL certificate from our issuer. We require https to access to these pages, and have disabled the https server while we are waiting for the issuer to revoke the existing certificate and issue a new one.