December 31, 2003

more server crash info

Early Monday afternoon, we had a report of a hacking attempt by one of our clients on the (dyketech.com) server. We immediately changed the passwords and set our security response procedures in place commencing with a security audit for the entire server and a thorough examination of the server?s logs.

We determined when the breech took place and we were in the process of tracing the culprit?s IP addresses when some of the server's files started reading as corrupt. The primary hard drive failed about four hours into our investigation (10:00 p.m. Monday). At that point, we immediately proceeded with an OS installation and CPanel recovery.

As the hard drive has failed, we don?t have information available to diagnose if the failure was a result of the hack attempts or a coincidence. We have submitted what evidence we have to the authorities, and we will work with them to bring the culprit to justice.

During the recovery from our backup drive, we encountered a corrupt password file. As sorting though the file to recover individual passwords would have added hours to our recovery time, we made a decision to change all passwords on this box.

Accounts on the (dyketech.com) box started becoming available around 2:00 a.m. Tuesday. The recovery finished around 9:00 p.m. Tuesday. Since then we have also been cleaning up minor mySQL, secure certificate and subdomain problems.

We will be sending out additional information regarding this event in the next 48 hours.

Posted by Elaine at December 31, 2003 07:09 PM