This did not turn into PD, but it sure came close.
Years ago I worked for a small IT consulting firm. It had one major client so naturally the owner of the company wanted to make sure we gave top notch service to the customer.
We were responsible for the email infrastructure for the client which was based on a Microsoft platform, at that time it was Microsoft Exchange version5.5 (this was back in the late 90's) Microsoft had come out with a new service pack for the Exchange5.5 platform. A service pack is basically a bunch fixes and upgrades to remedy problems that had been identified with the application.
Wanting to make sure we did our due diligence, a time was scheduled to apply the service pack to the email servers. My boss decided that to make sure the email environment was clean, he was going to run some utilities on the email servers. One of the utilities is something called eseutil, which defragments the database. The idea is that a fragmented database slows things down since the information is not easily readable since the information is not in consecutive order. Defragmenting the database will clean things up and increase performance.
However, if my boss had actually done real due diligence, he would have found out, that one of the problems that the new service pack was going to fix, was a problem with running eseutil against an email database that was larger than 16 GB. If you tried to defrag a database that was 16GB, it basically destroyed the database which would take down the entire email infrastructure and all of the email for all of the employees would be lost.
the email database on the clients email servers was just over 16GB, my boss was the one who ran the utility, and he crashed their email system. There were backups, but there were problems being able to read the backups to rebuild the email environment. It took over a week, but eventually, with the help of Microsoft, they were able to rebuild and restore most of the mail.
My boss was literally in the bathroom getting sick the first few days of trying to fix things. The quirk in all this, is that the client never really knew what happened, they didn't know why the system crashed, and there was no post mortem to find out exactly what went wrong. My boss actually made a lot of money since he had over 12 employees working practically around the clock for a week to fix the problem, and he charged the client for all the time spent fixing the problem.
the client not only paid the invoice for all the time, they actually thanked my boss for all the hard work and diligence at working at fixing the problem, and he came out of it smelling like a rose