Raw Device Mapping of local SATA disks on ESXi

My brand new ESXi machine is a “standard” Intel Z68 powered PC. It has three disks inside: two SSD and a SATA disk. My idea is to use the first SSD to host some VM (production VM, virtual ESXi, virtual Hyper-V…) and to connect the two other disks to a FreeNAS virtual instance. This enables to present a ZFS storage to the virtual hypervisors.

Here’s the way to attach two of the SATA drives to a single virtual machine using RDM technics. BTW, I’m using ESXi v5.0.0.

First of all, you’ll need an SSH access to the ESXi. SSH is enabled from the VI client, in the Configuration/Security Profile section. When “SSH” and “ESXi Shell” services are enabled, connect to the ESXi using your preferred SSH client.

# uname -a VMkernel esxi.tumfatig.net 5.0.0 #1 SMP Release build-469512 Aug 18 2011 18:32:24 x86_64 unknown

The first thing to do is to get the identifiers for your local storage. What you will need is the path to the devices:

# esxcfg-scsidevs -l | egrep '^[^ ]|Display Name|Vendor|Devfs Path' mpx.vmhba32:C0:T0:L0 Display Name: Local USB Direct-Access (mpx.vmhba32:C0:T0:L0) Devfs Path: /vmfs/devices/disks/mpx.vmhba32:C0:T0:L0 Vendor: Generic- Model: SD/MMC Revis: 1.00 t10.ATA_____M42DCT064M4SSD2__________________________00000000111703068787 Display Name: Local ATA Disk (t10.ATA_____M42DCT064M4SSD2__________________________00000000111703068787) Devfs Path: /vmfs/devices/disks/t10.ATA_____M42DCT064M4SSD2__________________________00000000111703068787 Vendor: ATA Model: M4-CT064M4SSD2 Revis: 0309 t10.ATA_____SAMSUNG_MMDOE28G5MPP2D0VA________________SE853A9829__________ Display Name: Local ATA Disk (t10.ATA_____SAMSUNG_MMDOE28G5MPP2D0VA________________SE853A9829__________) Devfs Path: /vmfs/devices/disks/t10.ATA_____SAMSUNG_MMDOE28G5MPP2D0VA________________SE853A9829__________ Vendor: ATA Model: SAMSUNG MMDOE28G Revis: VAM0 t10.ATA_____WDC_WD2500BEVT2D75ZCT2________________________WD2DWX90EC9JDT19 Display Name: Local ATA Disk (t10.ATA_____WDC_WD2500BEVT2D75ZCT2________________________WD2DWX90EC9JDT19) Devfs Path: /vmfs/devices/disks/t10.ATA_____WDC_WD2500BEVT2D75ZCT2________________________WD2DWX90EC9JDT19 Vendor: ATA Model: WDC WD2500BEVT-7 Revis: 11.0

In this example, I did a quite complex `grep`. You may want to use “esxcfg-scsidevs -l | grep 'Devfs Path'” to only get the device paths.

So in my configuration, I want to attach the Crucial M4 SSD and the Western Digital SATA disks to a virtual instance. The next thing to do is to create the RDM passthrough devices. What’s to be done is create the VMDK file that links to the physical device. That link is stored in some ESXi datastore and will be attached to the virtual machine:

# vmkfstools -z /vmfs/devices/disks/t10.ATA_____M42DCT064M4SSD2__________________________00000000111703068787 /vmfs/volumes/SSD_datastore/rdmssd.vmdk -a lsilogic # vmkfstools -z /vmfs/devices/disks/t10.ATA_____WDC_WD2500BEVT2D75ZCT2________________________WD2DWX90EC9JDT19 /vmfs/volumes/SSD_datastore/rdmsata.vmdk -a lsilogic

Let’s have a look at what’s been created:

# cd /vmfs/volumes/SSD_datastore /vmfs/volumes/4f46bbfc-4a0314f1-2663-00012e3d309f # ls -alh rdm* -rw------- 1 root root 232.9G Feb 25 12:16 rdmsata-rdmp.vmdk -rw------- 1 root root 496 Feb 26 12:15 rdmsata.vmdk -rw------- 1 root root 59.6G Feb 25 12:15 rdmssd-rdmp.vmdk -rw------- 1 root root 494 Feb 26 12:15 rdmssd.vmdk /vmfs/volumes/4f46bbfc-4a0314f1-2663-00012e3d309f #

You will see the links and the files used for direct attaching. The “rdmp” files claim to use some of the space on the datastore but they don’t actually do.

Now, you can attach the RDM to the virtual machine. Use the vSphere Client to edit the virtual machine settings. In the “Hardware” section, add a new “Hard Disk”. Choose the “Use an existing virtual disk” option and select one of the vmdk you just created. I attached “rdmssd.vmdk” and “rdmsata.vmdk” in that particular order and use the “SCSI(1:0)” and “SCSI (1:1)” Virtual Device Node ; just to be able to refer to them easily in the virtual machine.

Once this is done, boot the virtual machine and check for the disks. In mine, what I see is:

# dmesg | grep "^da" da0 at mpt0 bus 0 scbus2 target 0 lun 0 da0: <VMware Virtual disk 1.0> Fixed Direct Access SCSI-2 device da0: 320.000MB/s transfers (160.000MHz, offset 127, 16bit) da0: Command Queueing enabled da0: 8192MB (16777216 512 byte sectors: 255H 63S/T 1044C) da1 at mpt1 bus 0 scbus3 target 0 lun 0 da1: <ATA M4-CT064M4SSD2 0309> Fixed Direct Access SCSI-5 device da1: 6.600MB/s transfers (16bit) da1: 61057MB (125045424 512 byte sectors: 255H 63S/T 7783C) da2 at mpt1 bus 0 scbus3 target 1 lun 0 da2: <ATA WDC WD2500BEVT-7 11.0> Fixed Direct Access SCSI-5 device da2: 6.600MB/s transfers (16bit) da2: 238475MB (488397168 512 byte sectors: 255H 63S/T 30401C)

As you can see, the first disk in a “standard” virtual disk image. The two other disks appears as “real” disks thanks to the Raw Device Mapping. Now the storage can be configured as wished in the virtual machine.

Happy storing!

Source: Creating RDMs on SATA drives


  • Escaping the Apple ecosystem: part 0 | Tactical Grace Sun, 11 Nov 2012 13:45:37 +0000 Reply

    […] links: fan speed of the ATI card with ESXi 5.1 Raw Device Mapping of local SATA disks on ESXi ESXi as a Desktop with VMDirectPath I/O Share window._wp_rp_static_base_url = […]

  • Mac OS X Benchmark: native vs virtualized | Tactical Grace Mon, 03 Dec 2012 12:40:01 +0000 Reply

    […] workstation setup is that I’ve created the Mac OS X VM using my very own hard drives, hooked as raw devices (RDM: raw device mapping). So I can boot exactly the same OS directly from the hard drive, or from […]

Leave a Reply

%d bloggers like this: