How-To : Replace dead / faulty drive in Dell PERC RAID array

dell

Introduction

The goal of this post is to help you in case you need to replace a faulty disk that is a part of a RAID array built over a PERC controller.

You may find some tutorials to help you achieving this task using the GUI, but there is not many about the command line usage for this task, this is my contribution.

I will start at the point where the faulty drive has already been removed and the new one is also already in place on the server.

 

1) Checking the current state of your newly added drive

Once your new drive is set up in your server it might be seen either as “Foreign” or “Ready“. The “Ready” state is the one we are aiming at.

See the following the command in order to check the current state of the new drive :

[root@server ~]# omreport storage pdisk controller=0
List of Physical Disks on Controller PERC 5/i Integrated (Embedded)

Controller PERC 5/i Integrated (Embedded)

[ output stripped ... ]

ID : 0:0:1
Status : Non-Critical
Name : Physical Disk 0:0:1
State : Foreign
Failure Predicted : No
Progress : Not Applicable
Type : SATA
Capacity : 931.00 GB (999653638144 bytes)
Used RAID Disk Space : 0.00 GB (0 bytes)
Available RAID Disk Space : 931.00 GB (999653638144 bytes)
Hot Spare : No
Vendor ID : DELL
Product ID : ST31000340NS
Revision : MA98MA08
Serial No. : 9QJ4F4G4
Negotiated Speed : Not Available
Capable Speed : Not Available
Manufacture Day : Not Available
Manufacture Week : Not Available
Manufacture Year : Not Available
SAS Address : 1221000001000000

ID : 0:0:2
Status : Ok
Name : Physical Disk 0:0:2
State : Online
Failure Predicted : No
Progress : Not Applicable
Type : SAS
Capacity : 931.00 GB (999653638144 bytes)
Used RAID Disk Space : 931.00 GB (999653638144 bytes)
Available RAID Disk Space : 0.00 GB (0 bytes)
Hot Spare : No
Vendor ID : DELL
Product ID : ST31000640SS
Revision : MS04
Serial No. : 9QJ71332
Negotiated Speed : Not Available
Capable Speed : Not Available
Manufacture Day : 07
Manufacture Week : 03
Manufacture Year : 2010
SAS Address : 5000C5001033CD3D
[ output stripped ... ]

As you can see in the above output excerpt our new drive is seen as “Foreign” (line #11), which means it might has been a part of another RAID array before and it already has RAID meta-data written somewhere.

 

 

2) Changing the drive status from Foreign to Ready

We need to get rid of these meta-data before we can add it to the existing array, in the DELL terminology it is called “clearing the foreign config“, here is the command that allows you to do that :

[root@server ~]# omconfig storage controller action=clearforeignconfig controller=0
Command successful!

In the above command you need to know the controller ID, which can be found using the information given here and / or here.

Check the new status :

[root@server ~]# omreport storage pdisk controller=0
List of Physical Disks on Controller PERC 5/i Integrated (Embedded)

Controller PERC 5/i Integrated (Embedded)

[ output stripped ... ]

ID                        : 0:0:1
Status                    : Ok
Name                      : Physical Disk 0:0:1
State                     : Ready
Failure Predicted         : No
Progress                  : Not Applicable
Type                      : SATA
Capacity                  : 931.00 GB (999653638144 bytes)
Used RAID Disk Space      : 0.00 GB (0 bytes)
Available RAID Disk Space : 931.00 GB (999653638144 bytes)
Hot Spare                 : No
Vendor ID                 : DELL
Product ID                : ST31000340NS
Revision                  : MA08MA08
Serial No.                : 9QJ4F4G4
Negotiated Speed          : Not Available
Capable Speed             : Not Available
Manufacture Day           : Not Available
Manufacture Week          : Not Available
Manufacture Year          : Not Available
SAS Address               : 1221000001000000

ID                        : 0:0:2
Status                    : Ok
Name                      : Physical Disk 0:0:2
State                     : Online
Failure Predicted         : No
Progress                  : Not Applicable
Type                      : SAS
Capacity                  : 931.00 GB (999653638144 bytes)
Used RAID Disk Space      : 931.00 GB (999653638144 bytes)
Available RAID Disk Space : 0.00 GB (0 bytes)
Hot Spare                 : No
Vendor ID                 : DELL
Product ID                : ST31000640SS
Revision                  : MS04
Serial No.                : 9QJ71332
Negotiated Speed : Not Available
Capable Speed : Not Available
Manufacture Day : 07
Manufacture Week : 03
Manufacture Year : 2010
SAS Address : 5000C5001033CD3D
[ output stripped ... ]

We can see the drive has become “Ready” (line #11), it is now ready to be added to the existing array.

 

 

3) Adding the “ready” drive to the existing array

Most PERC controller will automatically rebuild a “degraded” array when a proper Hdd is plugged in, but in the case you do have to handle this manually the Dell recommended way (as stated by the Dell official support) is to assign the new Hdd as a “Global Hot spare“.

Once it is assigned as a global hot spare the PERC controller should automatically start the rebuilding process…

The command to assign a new disk as “Global Hot Spare” is :

[root@server ~]# omconfig storage pdisk action=assignglobalhotspare controller=0 pdisk=0:0:1 assign=yes
Command successful!

Check the resulting state of the 0:0:1 physical disk (line #12) :

[ output stripped ... ]
ID                        : 0:0:1
Status                    : Ok   
Name                      : Physical Disk 0:0:1
State                     : Ready
Failure Predicted         : No
Progress                  : Not Applicable
Type                      : SAS
Capacity                  : 1,862.50 GB (1999844147200 bytes)
Used RAID Disk Space      : 1,862.50 GB (1999844147200 bytes)
Available RAID Disk Space : 0.00 GB (0 bytes)
Hot Spare                 : Global
[ output stripped ... ]

 

Note :

You can check the correct physical address for your new drive by using the “blink” action as :

[root@server ~]# omconfig storage pdisk action=blink controller=0 pdisk=0:0:1
Command successful!

Then stop the blinking :

[root@server ~]# omconfig storage pdisk action=unblink controller=0 pdisk=0:0:1
Command successful!

 

 

 

4) Dell RAID related definitions

Here is some details about the Dell terminology you may encounter in the physical disk “State” section.

Online : This means a drive is already a part of a RAID array

Ready : Ready to be used (either in a RAID array or as a HotSpare etc)

Foreign : This means the drive contains some RAID meta-data (coming from an old RAID array the disk used to be a part of), this does not allow the use of this disk, you may firts clean the foreign config (see this section)

 

More “Sysadmin” posts

 

10 thoughts on “How-To : Replace dead / faulty drive in Dell PERC RAID array

  1. alan boxberger

    Hi Pier
    Followed instructions above but array is not rebuilding.
    Had a raid 6 array with 8 disks. Two failed. Replaced the disks. Showed as foreign.
    After steps above, disks show as State=ready.

    How do I configure the new global hot spares into the array?

    1. Pier Post author

      Hey Alan,

      Sorry for the late reply (dunno if you’ll ever come back here :) ). Your disks in “ready” state, which is good, as you noticed you only have to add them as hot spares so they will be automagically added to the existing array, in the above guide the recomended command is :

      omconfig storage pdisk action=assignglobalhotspare controller=0 pdisk=0:0:1 assign=yes

      Did you try this ?

      Otherwise this command might have been updated for the newer omreport version, and I do not deal with Dell server anymore (changed compnay and switched to HP :) )

      Anyway this might not be too far from the truth…. I am sure you(re going to fix this ! (if not already)

      Pier

  2. Paul

    Hi,
    Thanks for your time.
    Got a H310 Mini controller & 3 SAS disks already in a raid array.
    So, i’ve got one 5,588.00Go RAID5 vdisk and i’ve been asked to expand it with one more disk.

    1. Pier Post author

      Hi Paul,

      Your welcome, here is a doc that contains useful information about what you’re trying to achieve :

      http://www.dell.com/support/article/us/en/19/SLN156106

      The interesting line in your case is the 9th line of the table, as :

      RAID 5 | RAID 5 | 3 | 4 or more | Yes | Increases capacity by adding disks.

      Let me know if you want a specific guideline to achieve your goal, but the answer to your question is YES, you can expand a PERC H310 based RAID5 array by adding Hdd.

      This page may help you too :
      http://www.partition-magic.org/hardware-raid/add-disk-to-raid-5.html

      Tips : The “reconfigure” option is the one you want to use !

      Let me know if you want to do it using the Open Manage CLI.

      Beside this part of the job, you may have to expand your file system or any higher level container, depends on what is your OS etc …
      Have a good day Paul.

  3. Paul

    Is it possible expand an existing array ?

    Once my drive is ready, is there a way to use ot to expand an existing array with data on it ?

    1. Pier Post author

      Hi Paul,

      I need more information about your setup to give you a correct answer, such as :

      – Your controller
      – Your Hdd number
      – Your RAID level

      As a global advice, regarding “Hdd with data” manipulation, ALWAYS BACKUP YOUR DATA !

      I would be glad to help if you give me more details about your setup.

Leave a Reply to Pier Cancel reply

Your email address will not be published. Required fields are marked *

This site supports SyntaxHighlighter via WP SyntaxHighlighter. It can highlight your code.
How to highlight your code: Paste your code in the comment form, select it and then click the language link button below. This will wrap your code in a <pre> tag and format it when submitted.