On Easter Sunday, the data center where our server is located suffered:

A power surge, which caused
A power outage, which caused
Backup generators to deploy, which caused
A fire in one of the backup generators, which caused
Fire sprinklers to come on.
News from the datacenter was slim, except to tell us that they were awaiting a city inspector on Monday to approve turning the power back on.

Unfortunately, that bit of info caused us to stand by and wait for our current server to get turned back on, which we hoped would be imminently.

As it turns out, there was more water damage at the data center than originally let on, and this morning we received the message:

"Unfortunately, this server appears to be in the section with a high probability of water damage. So it will need to be physically inspected before it can be powered on.

I'm afraid that we do not have an ETA for repair at this time."

I sure wish I would have known that Sunday or Monday!

Anyway, with that news, we decided to put our contingency plans into effect immediately. The good news is that we have a safe backup. Since this morning's news, we have deployed a new dedicated server, and we are in the process of configuring it and loading the backup. That should take ~24hours. We will lose a few hours worth of data ... the period of time between the backup and the server crashing ... but at this point I have made the decision that we cannot wait any longer to be back online, especially in light of the not-so-encouraging ETA.

We thank you for your patience. The clock is now ticking on a (belated) quick return.

UPDATE: And just as soon as I ran my card on the new server, bam, we're back up at the old datacenter ... (so no data loss ... everyhting up until the moment the datacenter went down is preserved.)

Rick Bin

"What we obtain too cheap, we esteem too lightly: it is dearness only that gives every thing its value. Heaven knows how to put a proper price upon its goods; and it would be strange indeed if so celestial an article as freedom should not be highly rated." Thomas Paine