Create VMFS Datastore on Boot USB Device

For my Home Lab, I wanted to use my internal Hard-drives for vSAN. Thus, I had to install ESXi on an USB Drive. Since I used 64 GB USB Drives, and ESXi requires only 10-15 GB, I wanted to make the remaining space usable as VMFS Datastores. Not to run VMs on it, but for syslog, maybe ISOs, …
This turned out to be a bit more tricky than expected. There are a bunch of great articles out there, that show how to create VMFS Datastores on empty USB drives. But I haven’t found an article on how to install a VMFS datastore next to a ESXi installation on the same USB device.

So, in order to make the remaining space on my USB drive usable as VMFS Datastore, I had to perform the following steps:

1. First stop (and disable) the usbarbitrator service
(Note – this will prevent all USB devices from being passed-through to any VMs)

[root@esxi3:~] /etc/init.d/usbarbitrator stop
[root@esxi3:~] chkconfig usbarbitrator off

2. Next, let’s have a look at the current partition table of our USB device, found in /dev/disks/:

[root@esxi3:~] ls -aslh
total 2505694880
     0 drwxr-xr-x    2 root     root         512 Aug  1 10:31 .
     0 drwxr-xr-x   19 root     root         512 Aug  1 10:31 ..
60063744 -rw-------    1 root     root       57.3G Aug  1 10:31 mpx.vmhba32:C0:T0:L0
  4064 -rw-------    1 root     root        4.0M Aug  1 10:31 mpx.vmhba32:C0:T0:L0:1
255984 -rw-------    1 root     root      250.0M Aug  1 10:31 mpx.vmhba32:C0:T0:L0:5
255984 -rw-------    1 root     root      250.0M Aug  1 10:31 mpx.vmhba32:C0:T0:L0:6
112624 -rw-------    1 root     root      110.0M Aug  1 10:31 mpx.vmhba32:C0:T0:L0:7
292848 -rw-------    1 root     root      286.0M Aug  1 10:31 mpx.vmhba32:C0:T0:L0:8
2789376 -rw-------    1 root     root        2.7G Aug  1 10:31 mpx.vmhba32:C0:T0:L0:9
[remaining output omitted]

As you can see, “mpx.vmhba32:C0:T0:L0” is my USB drive. Though, the output is already showing the partitions (lines with “:<number>” at the end) we need a bit more detailed information.

3. List partitions using partedUtil

[root@esxi3:~] partedUtil getptbl /dev/disks/mpx.vmhba32\:C0\:T0\:L0
gpt
7477 255 63 120127488
1 64 8191 C12A7328F81F11D2BA4B00A0C93EC93B systemPartition 128
5 8224 520191 EBD0A0A2B9E5443387C068B6B72699C7 linuxNative 0
6 520224 1032191 EBD0A0A2B9E5443387C068B6B72699C7 linuxNative 0
7 1032224 1257471 9D27538040AD11DBBF97000C2911D1B8 vmkDiagnostic 0
8 1257504 1843199 EBD0A0A2B9E5443387C068B6B72699C7 linuxNative 0
9 1843200 7421951 9D27538040AD11DBBF97000C2911D1B8 vmkDiagnostic 0

On a high level, the output means the following:
<PartitionNumber> – <StartingSector< – >EndingSector> – <Type(vmfs/vsan/…)> – <Attribute>
If you are interested in further details, check out https://kb.vmware.com/s/article/1036609
From line 9, we know already the last used sector, which is 7421951. That means, our new partition can’t start before sector 7421952.

4. Let’s find out, which is the last usable Sector:

[root@esxi3:~] partedUtil getUsableSectors /dev/disks/mpx.vmhba32\:C0\:T0\:L0
34 120127454

120127454 is the last sector of the device. In order to keep this command working, even after our modifications, we have to substract 1 sector.

Now, we know that our partition can span across sectors 7421952 to 120127453.

5. Next, we need to find out what type we need for our VMFS datastore:

[root@esxi3:~] partedUtil showGuids
 Partition Type       GUID
 vmfs                 AA31E02A400F11DB9590000C2911D1B8
 vmkDiagnostic        9D27538040AD11DBBF97000C2911D1B8
 vsan                 381CFCCC728811E092EE000C2911D0B2
 virsto               77719A0CA4A011E3A47E000C29745A24
 VMware Reserved      9198EFFC31C011DB8F78000C2911D1B8
 Basic Data           EBD0A0A2B9E5443387C068B6B72699C7
 Linux Swap           0657FD6DA4AB43C484E50933C84B4F4F
 Linux Lvm            E6D6D379F50744C2A23C238F2A3DF928
 Linux Raid           A19D880F05FC4D3BA006743F0F84911E
 Efi System           C12A7328F81F11D2BA4B00A0C93EC93B
 Microsoft Reserved   E3C9E3160B5C4DB8817DF92DF00215AE
 Unused Entry         00000000000000000000000000000000

In theory, we now have all required information:

Partition Number: 10 (just the next usable)
Start Sector: 7421952
End Sector: 120127453
Type: AA31E02A400F11DB9590000C2911D1B8
Attribute: 0

6. Next, construct the command to create the new partition.
Important
– you have to enter the information for all partitions, as the command overwrites the partition table.

[root@esxi3:~] partedUtil setptbl /vmfs/devices/disks/mpx.vmhba32\:C0\:T0\:L0  gpt "1 64 8191 C12A7328F81F11D2BA4B00A0C93EC93B 128" "5 8224 520191 EBD0A0A2B9E5443387C068B6B72699C7 0" "6 520224 1032191 EBD0A0A2B9E5443387C068B6B72699C7 0" "7 1
032224 1257471 9D27538040AD11DBBF97000C2911D1B8 0" "8 1257504 1843199 EBD0A0A2B9E5443387C068B6B72699C7 0" "9 1843200 7421951 9D27538040AD11DBBF97000C2911D1B8 0" "10 7421952 120127453 AA31E02A400F11DB9590000C2911D1B8 0"

[output omitted]

Error: Read-only file system during write on /dev/disks/mpx.vmhba32:C0:T0:L0
SetPtableGpt: Unable to commit to disk

To find out, what went wrong, took me quite some time. But eventually, I stumbled across this thread in the VMware Forum:
https://communities.vmware.com/message/2472738
It outlines, that the issue is caused by the vmkdiagnostics partitions (7 and 9 in our example), which were automatically created during the installation and are used to store coredump information.

7. So, let’s check the partitions being used for coredump:

root@esxi3:~] esxcli system coredump partition list
Name                    Path                                        Active  Configured
----------------------  ------------------------------------------  ------  ----------
mpx.vmhba32:C0:T0:L0:7  /vmfs/devices/disks/mpx.vmhba32:C0:T0:L0:7   false       false
mpx.vmhba32:C0:T0:L0:9  /vmfs/devices/disks/mpx.vmhba32:C0:T0:L0:9    true        true

They are indeed.

8. Disable Coredump

esxcli system coredump partition set --enable false

9. Try again to create the new partition table

[root@esxi3:~] partedUtil setptbl /vmfs/devices/disks/mpx.vmhba32\:C0\:T0\:L0  gpt "1 64 8191 C12A7328F81F11D2BA4B00A0C93EC93B 128" "5 8224 520191 EBD0A0A2B9E5443387C068B6B72699C7 0" "6 520224 1032191 EBD0A0A2B9E5443387C068B6B72699C7 0" "7 1
032224 1257471 9D27538040AD11DBBF97000C2911D1B8 0" "8 1257504 1843199 EBD0A0A2B9E5443387C068B6B72699C7 0" "9 1843200 7421951 9D27538040AD11DBBF97000C2911D1B8 0" "10 7421952 120127453 AA31E02A400F11DB9590000C2911D1B8 0"
gpt
0 0 0 0
1 64 8191 C12A7328F81F11D2BA4B00A0C93EC93B 128
5 8224 520191 EBD0A0A2B9E5443387C068B6B72699C7 0
6 520224 1032191 EBD0A0A2B9E5443387C068B6B72699C7 0
7 1032224 1257471 9D27538040AD11DBBF97000C2911D1B8 0
8 1257504 1843199 EBD0A0A2B9E5443387C068B6B72699C7 0
9 1843200 7421951 9D27538040AD11DBBF97000C2911D1B8 0
10 7421952 120127453 AA31E02A400F11DB9590000C2911D1B8 0

Now it worked.

10. Last thing to do is to create the VMFS6 Datastore

[root@esxi3:~] vmkfstools -C vmfs6 -S ESXi3-LocalUSB /dev/disks/mpx.vmhba32\:C0\:T0\:L0\:10
create fs deviceName:'/dev/disks/mpx.vmhba32:C0:T0:L0:10', fsShortName:'vmfs6', fsName:'ESXi3-LocalUSB'
deviceFullPath:/dev/disks/mpx.vmhba32:C0:T0:L0:10 deviceFile:mpx.vmhba32:C0:T0:L0:10
ATS on device /dev/disks/mpx.vmhba32:C0:T0:L0:10: not supported
.
Checking if remote hosts are using this device as a valid file system. This may take a few seconds...
Creating vmfs6 file system on "mpx.vmhba32:C0:T0:L0:10" with blockSize 1048576, unmapGranularity 1048576, unmapPriority default and volume label "ESXi3-LocalUSB".
Successfully created new volume: 5d42cb1c-a9f53c50-4bca-54b20310b9e1

The newly created Datastore is now visible in vSphere GUI.

Leave a Reply

Your email address will not be published. Required fields are marked *