I recently had a strange problem to troubleshoot. A client's ESXi 5.1 box crashed hard. Unfortunately, the server (a Dell PowerEdge 2950 g3) had no iDrac card.
When I arrived, I discovered that someone had unplugged two of the drives in a RAID 5 set of 5 drives. There was a hot spare, but both drives were pulled at the same time (long story.) The RAID 5 volume was down. I looked into the Perc BIOS. Two of the five drives now showed up as foreign. While I've certainly imported foreign sets, I've rarely seen this situation on some of the drives and not all of them.
When a foreign disk is detected in the Dell Perc BIOS (this is, of course, a rebadged LSI RAID card), a new tab shows up - foreign. I went into the foreign tab and imported the configs from both drives. The volume was now online (rebuilding, of course.) I waited a while then let ESXi boot. I stopped the VMs from starting and waited till the rebuild finished, just to be safe - a power outage at that moment might have been particularly bad (this building has unreliable power and the UPS can only keep a server up so long, of course.) After the disks had quieted down, I started the VMs. There was some data loss, of course, and the startup took over an hour (there was a snapshot in place - so the hard crash probably damaged the snapshot). However, the server was back. There's something to be said for lockable bezels.
No comments:
Post a Comment