While working on a vMotion issue i came across a scenario where the vMotion Operation was getting failed with the Error: “A specified parameter was not correct:”. While working i saw that the vMotion operation is getting failed at 80 Percent.
vCenter:
Product: VMware vCenter Server Appliance Version: 6.5.0
Build: 13638625 Release: vCenter Server Appliance 6.5 Update 2g (vCenter Server
Appliance 6.5 Update 2g)
Node Type: embedded
Configured Timezone: “Etc/UTC” (+00:00)
Hosts and VMs in Inventory: 6/21
Resource Configuration: CPU: 4 Memory: 16 GB
VM Name: NvCenter
VMware Logs:
- Does not have many details about the vMotion Operation.
vCenter Logs:
VPXD Logs:
· On the VPXD Logs we can see the below vMotion Operation Flow.
· We used this to find the OP-ID
· Here we can see that the failure happened while creating a dummy VM on the
Destination.
2022-05-10T07:23:14.811Z
info vpxd[7F01125CB700] [Originator@6876 sub=vpxLro
opID=l2zto750-4t-h5:70000105-5c] [VpxLRO] -- BEGIN task-9503 -- vm-33 --
vim.VirtualMachine.relocate -- 52a8cbb4-4263-0302-907e-48d69da23616(526da896-2df8-3ff1-cfc8-0bf79b0f69f4)
2022-05-10T07:23:14.813Z
info vpxd[7F01125CB700] [Originator@6876 sub=vpxLro
opID=l2zto750-4t-h5:70000105-5c-01] [VpxLRO] -- BEGIN lro-1546 -- --
VmprovWorkflow --
2022-05-10T07:23:14.813Z
info vpxd[7F01125CB700] [Originator@6876 sub=VmProv
opID=l2zto750-4t-h5:70000105-5c-01] Execute local action
vpx.vmprov.InvokePrechecks (5%)
2022-05-10T07:23:14.826Z
info vpxd[7F01125CB700] [Originator@6876 sub=vpxCrypt
opID=l2zto750-4t-h5:70000105-5c-01] Failed to read X509 cert; err: 151441516
2022-05-10T07:23:15.127Z
info vpxd[7F01125CB700] [Originator@6876 sub=VmProv
opID=l2zto750-4t-h5:70000105-5c-01] Execute local action
vpx.vmprov.SelectDestination (1%)
2022-05-10T07:23:15.129Z
info vpxd[7F01125CB700] [Originator@6876 sub=VmProv
opID=l2zto750-4t-h5:70000105-5c-01] Datastore changing LocalVC Migrate of NvCenter
on vim.HostSystem:host-26(10.74.1.123) with ds
ds:///vmfs/volumes/5d4b625e-3883346c-4eed-e26ee8500010/ to
vim.HostSystem:host-30(10.74.1.321) with ds
ds:///vmfs/volumes/5d4b65d2-28fd39fa-d8b6-e26ee8500014/ started
2022-05-10T07:23:15.129Z
info vpxd[7F01125CB700] [Originator@6876 sub=VmProv
opID=l2zto750-4t-h5:70000105-5c-01] Execute local action
vpx.vmprov.CheckCompatibility (1%)
2022-05-10T07:23:15.160Z warning vpxd[7F01125CB700]
[Originator@6876 sub=pbm opID=l2zto750-4t-h5:70000105-5c-01] PBMCallback:
FillConfigSpec: No linked clone disk profiles
2022-05-10T07:23:15.160Z warning vpxd[7F01125CB700]
[Originator@6876 sub=drmLogger opID=l2zto750-4t-h5:70000105-5c-01]
[VpxdDrmInterface::GetVmotionResSettings] Doing full invocation.
2022-05-10T07:23:15.166Z
info vpxd[7F0112C58700] [Originator@6876 sub=vpxLro
opID=l2zto750-4t-h5:70000105-5c-01-01] [VpxLRO] -- BEGIN lro-1551 -- --
DrmExecute --
2022-05-10T07:23:15.171Z
info vpxd[7F0112C58700] [Originator@6876 sub=vpxLro
opID=l2zto750-4t-h5:70000105-5c-01-01] [VpxLRO] -- FINISH lro-1551
2022-05-10T07:28:01.530Z
info vpxd[7F01125CB700] [Originator@6876 sub=VmProv
opID=l2zto750-4t-h5:70000105-5c-01] Execute local action
vpx.vmprov.CreateDestinationVm (1%)
2022-05-10T07:28:01.659Z
error vpxd[7F01125CB700] [Originator@6876 sub=VmProv
opID=l2zto750-4t-h5:70000105-5c-01] Get exception while executing action
vpx.vmprov.CreateDestinationVm: N5Vmomi5Fault15InvalidArgument9ExceptionE(vmodl.fault.InvalidArgument)
2022-05-10T07:28:01.662Z
info vpxd[7F01125CB700] [Originator@6876 sub=VmProv
opID=l2zto750-4t-h5:70000105-5c-01] Workflow context:
2022-05-10T07:28:01.689Z
info vpxd[7F01125CB700] [Originator@6876 sub=VmProv
opID=l2zto750-4t-h5:70000105-5c-01] Invoking callback type 2 on Iofilter
2022-05-10T07:28:01.689Z
info vpxd[7F01125CB700] [Originator@6876 sub=VmProv
opID=l2zto750-4t-h5:70000105-5c-01] Invoking callback type 2 on MetadataHandler
2022-05-10T07:28:01.689Z
info vpxd[7F01125CB700] [Originator@6876 sub=VmProv
opID=l2zto750-4t-h5:70000105-5c-01] Invoking callback type 2 on PbmCallBack
2022-05-10T07:28:01.689Z
warning vpxd[7F01125CB700] [Originator@6876 sub=pbm opID=l2zto750-4t-h5:70000105-5c-01]
PBMCallback: ShouldSkipPostMigrateCallback: post migrate callback is skipped -
VM migration failed
2022-05-10T07:28:01.689Z
info vpxd[7F01125CB700] [Originator@6876 sub=VmProv
opID=l2zto750-4t-h5:70000105-5c-01] Invoking callback type 2 on
PermissionsHandler
2022-05-10T07:28:01.689Z
info vpxd[7F01125CB700] [Originator@6876 sub=VmProv
opID=l2zto750-4t-h5:70000105-5c-01] Invoking callback type 2 on PmemValidator
2022-05-10T07:28:02.328Z
info vpxd[7F01125CB700] [Originator@6876 sub=VmProv
opID=l2zto750-4t-h5:70000105-5c-01] Done undo action
vpx.vmprov.ReserveDirectory with output:
2022-05-10T07:28:02.328Z warning vpxd[7F01125CB700]
[Originator@6876 sub=VpxProfiler opID=l2zto750-4t-h5:70000105-5c-01]
VpxLro::LroMain [TotalTime] took 287514 ms
2022-05-10T07:28:02.328Z
error vpxd[7F01125CB700] [Originator@6876 sub=vpxLro
opID=l2zto750-4t-h5:70000105-5c-01] [VpxLRO] Unexpected Exception: N5Vmomi5Fault15InvalidArgument9ExceptionE(vmodl.fault.InvalidArgument)
2022-05-10T07:28:02.331Z
info vpxd[7F01125CB700] [Originator@6876 sub=vpxLro
opID=l2zto750-4t-h5:70000105-5c-01] [VpxLRO] -- FINISH lro-1546
2022-05-10T07:28:02.331Z
info vpxd[7F01125CB700] [Originator@6876 sub=Default
opID=l2zto750-4t-h5:70000105-5c-01] [VpxLRO] -- ERROR lro-1546 -- --
VmprovWorkflow: vmodl.fault.InvalidArgument:
2022-05-10T07:28:02.332Z warning vpxd[7F01201A7700]
[Originator@6876 sub=VpxProfiler opID=l2zto750-4t-h5:70000105-5c-01-TaskLoop-64b2ff8e]
TaskLoop [TotalTime] took 286680 ms
2022-05-10T07:28:02.333Z
info vpxd[7F01125CB700] [Originator@6876 sub=vpxLro
opID=l2zto750-4t-h5:70000105-5c] [VpxLRO] -- FINISH task-9503
2022-05-10T07:28:02.333Z
info vpxd[7F01125CB700] [Originator@6876 sub=Default
opID=l2zto750-4t-h5:70000105-5c] [VpxLRO] -- ERROR task-9503 -- vm-33 --
vim.VirtualMachine.relocate: vmodl.fault.InvalidArgument:
-->
Result:
-->
(vmodl.fault.InvalidArgument) {
-->
faultCause = (vmodl.MethodFault) null,
-->
faultMessage = <unset>,
-->
invalidProperty = <unset>
-->
msg = "A specified parameter was not correct: "
Destination ESXi Host:
· On the Destination ESXi Host we used the same OP ID which we captured from the
VPXD logs to understand the process flow:
VPXA Logs:
· Here we can see that the task initiated and failed with the Invalid Argument.
2022-05-10T07:23:15.128Z
info vpxa[5AFD8B70] [Originator@6876 sub=vpxLro
opID=l2zto750-4t-h5:70000105-5c-01-2d] [VpxLRO] -- BEGIN lro-119953 -- vpxa --
vpxapi.VpxaService.reserveName -- 5284a843-f94e-434b-4f92-bc5582239929
2022-05-10T07:23:15.128Z
verbose vpxa[5AFD8B70] [Originator@6876 sub=vpxaNameReserve opID=l2zto750-4t-h5:70000105-5c-01-2d]
[ReserveFileWithGivenRootDir] Directory
/vmfs/volumes/5d4b65d2-28fd39fa-d8b6-e26ee8500014 is available for use
2022-05-10T07:23:15.135Z
verbose vpxa[5AFD8B70] [Originator@6876 sub=vpxaNameReserve
opID=l2zto750-4t-h5:70000105-5c-01-2d] [ReserveFileWithGivenRootDir] Reserved
directory only, name /vmfs/volumes/5d4b65d2-28fd39fa-d8b6-e26ee8500014/NvCenter,
stable name /vmfs/volumes/5d4b65d2-28fd39fa-d8b6-e26ee8500014/NvCenter
2022-05-10T07:23:15.135Z
verbose vpxa[5AFD8B70] [Originator@6876 sub=journal
opID=l2zto750-4t-h5:70000105-5c-01-2d] Deleting Journal
/var/log/vmware/journal/1652167395.18.
2022-05-10T07:23:15.135Z
info vpxa[5AFD8B70] [Originator@6876 sub=vpxLro
opID=l2zto750-4t-h5:70000105-5c-01-2d] [VpxLRO] -- FINISH lro-119953
2022-05-10T07:28:01.535Z
info vpxa[5B03BB70] [Originator@6876 sub=vpxLro
opID=l2zto750-4t-h5:70000105-5c-01-3a] [VpxLRO] -- BEGIN task-22 -- vpxa --
vpxapi.VpxaService.createVm -- 5284a843-f94e-434b-4f92-bc5582239929
2022-05-10T07:28:01.536Z
warning vpxa[5B03BB70] [Originator@6876 sub=vpxaVmprovUtil
opID=l2zto750-4t-h5:70000105-5c-01-3a] [VpxaVmprov] UndoNameReservation: Failed
to open file /vmfs/volumes/5d4b65d2-28fd39fa-d8b6-e26ee8500014/NvCenter/NvCenter.vmx
2022-05-10T07:28:01.543Z
verbose vpxa[7BD0B70] [Originator@6876 sub=vpxLro
opID=l2zto750-4t-h5:70000105-5c-01-3a] [VpxLRO] LRO task-22 has been resumed
2022-05-10T07:28:01.543Z
error vpxa[7BD0B70] [Originator@6876 sub=vpxaVmprov
opID=l2zto750-4t-h5:70000105-5c-01-3a] [CreateVmFailed] Received unexpected
exception from host agent while creating VM: 07bcfe94
2022-05-10T07:28:01.544Z
error vpxa[7BD0B70] [Originator@6876 sub=vpxLro opID=l2zto750-4t-h5:70000105-5c-01-3a]
[LroMainContinuation] Unexpected Exception:
N5Vmomi5Fault15InvalidArgument9ExceptionE(vmodl.fault.InvalidArgument)
2022-05-10T07:28:01.545Z
verbose vpxa[7BD0B70] [Originator@6876 sub=PropertyProvider
opID=l2zto750-4t-h5:70000105-5c-01-3a] RecordOp ASSIGN: info.state, task-22.
Applied change to temp map.
2022-05-10T07:28:01.545Z
verbose vpxa[7BD0B70] [Originator@6876 sub=PropertyProvider
opID=l2zto750-4t-h5:70000105-5c-01-3a] RecordOp ASSIGN: info.cancelable,
task-22. Applied change to temp map.
2022-05-10T07:28:01.545Z
verbose vpxa[7BD0B70] [Originator@6876 sub=PropertyProvider
opID=l2zto750-4t-h5:70000105-5c-01-3a] RecordOp ASSIGN: info.error, task-22.
Applied change to temp map.
2022-05-10T07:28:01.545Z
verbose vpxa[7BD0B70] [Originator@6876 sub=PropertyProvider
opID=l2zto750-4t-h5:70000105-5c-01-3a] [CommitChangesAndNotify] Updating cached
values
2022-05-10T07:28:01.545Z
info vpxa[7BD0B70] [Originator@6876 sub=vpxLro
opID=l2zto750-4t-h5:70000105-5c-01-3a] [VpxLRO] -- FINISH task-22
2022-05-10T07:28:01.545Z
info vpxa[7BD0B70] [Originator@6876 sub=Default
opID=l2zto750-4t-h5:70000105-5c-01-3a] [VpxLRO] -- ERROR task-22 -- vpxa --
vpxapi.VpxaService.createVm: vmodl.fault.InvalidArgument:
-->
key = "cdrom.showIsoLockWarning",
2022-05-10T07:28:01.678Z
info vpxa[5AF54B70] [Originator@6876 sub=vpxLro
opID=l2zto750-4t-h5:70000105-5c-01-4c] [VpxLRO] -- BEGIN lro-119976 -- vpxa --
vpxapi.VpxaService.nfcMakeSystemTicket -- 5284a843-f94e-434b-4f92-bc5582239929
2022-05-10T07:28:01.679Z
info vpxa[5AF54B70] [Originator@6876 sub=vpxLro
opID=l2zto750-4t-h5:70000105-5c-01-4c] [VpxLRO] -- FINISH lro-119976
2022-05-10T07:28:01.923Z
verbose vpxa[7C96B70] [Originator@6876 sub=NfcManager
opID=l2zto750-4t-h5:70000105-5c-01-65] [NfcUtil] Specs are the same but
non-local.
2022-05-10T07:28:01.923Z
verbose vpxa[7C96B70] [Originator@6876 sub=NfcManager
opID=l2zto750-4t-h5:70000105-5c-01-65] [NfcUtil] CopySpecs are equal.
2022-05-10T07:28:01.923Z
warning vpxa[7C96B70] [Originator@6876 sub=Libs opID=l2zto750-4t-h5:70000105-5c-01-65]
[NFC ERROR] NfcFile_Open: Open failed:
2022-05-10T07:28:01.923Z
warning vpxa[7C96B70] [Originator@6876 sub=Libs
opID=l2zto750-4t-h5:70000105-5c-01-65] [NFC ERROR] NfcFile_GetInfo: Failed to
open file
2022-05-10T07:28:01.923Z
error vpxa[7C96B70] [Originator@6876 sub=NfcManager
opID=l2zto750-4t-h5:70000105-5c-01-65] [NfcClient] Unable to get file info for
file ds:///vmfs/volumes/5d4b65d2-28fd39fa-d8b6-e26ee8500014/NvCenter//vmware.log:
No file exists for given path -- File open failed: Could not find the file
2022-05-10T07:28:01.924Z
error vpxa[7C96B70] [Originator@6876 sub=NfcManager
opID=l2zto750-4t-h5:70000105-5c-01-65] [NfcWorker] Error encountered while
pre-processing copy spec for file ds:///vmfs/volumes/5d4b65d2-28fd39fa-d8b6-e26ee8500014/NvCenter//vmware.log:
2022-05-10T07:28:01.925Z
warning vpxa[7C96B70] [Originator@6876 sub=NfcManager
opID=l2zto750-4t-h5:70000105-5c-01-65] [NfcWorker] FailOnError is false.
Proceeding to the next spec...
2022-05-10T07:28:01.925Z
verbose vpxa[7C96B70] [Originator@6876 sub=NfcManager
opID=l2zto750-4t-h5:70000105-5c-01-65] [NfcClient] Closing NFC connection to
server
2022-05-10T07:28:01.926Z
info vpxa[7C96B70] [Originator@6876 sub=NfcManager opID=l2zto750-4t-h5:70000105-5c-01-65]
[NfcManagerImpl] Copy operation completed successfully
2022-05-10T07:28:01.926Z
verbose vpxa[7C96B70] [Originator@6876 sub=PropertyProvider
opID=l2zto750-4t-h5:70000105-5c-01-65] RecordOp ASSIGN: info.progress, task-23.
Applied change to temp map.
2022-05-10T07:28:02.313Z
info vpxa[7B6DB70] [Originator@6876 sub=vpxLro
opID=l2zto750-4t-h5:70000105-5c-01-d7] [VpxLRO] -- BEGIN lro-119980 -- vpxa --
vpxapi.VpxaService.unregisterVm -- 5284a843-f94e-434b-4f92-bc5582239929
2022-05-10T07:28:02.313Z
verbose vpxa[7B6DB70] [Originator@6876 sub=vpxaVmprov
opID=l2zto750-4t-h5:70000105-5c-01-d7] Unregister vm -1 and remove 1 files
2022-05-10T07:28:02.314Z warning vpxa[7B6DB70]
[Originator@6876 sub=vpxaVmprov opID=l2zto750-4t-h5:70000105-5c-01-d7]
[UnregisterVm] Ignoring non-empty dir:
/vmfs/volumes/5d4b65d2-28fd39fa-d8b6-e26ee8500014/NvCenter
2022-05-10T07:28:02.314Z
info vpxa[7B6DB70] [Originator@6876 sub=vpxLro
opID=l2zto750-4t-h5:70000105-5c-01-d7] [VpxLRO] -- FINISH lro-119980
Hostd Logs:
· While looking into the Hostd Logs I can see that the reason for the operation failure is that the VM Already exist. Due to which the create VM operation failed.
2022-05-10T07:23:15.164Z
verbose hostd[B340B70] [Originator@6876 sub=PropertyProvider
opID=l2zto750-4t-h5:70000105-5c-01-02-d-88bc user=vpxuser] RecordOp ASSIGN:
info, haTask-ha-host-vim.option.OptionManager.queryView-111715918. Applied
change to temp map.
2022-05-10T07:28:01.541Z
info hostd[B381B70] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/5d4b65d2-28fd39fa-d8b6-e26ee8500014/NvCenter/NvCenter.vmx
opID=l2zto750-4t-h5:70000105-5c-01-3a-8917
user=vpxuser:VSPHERE.LOCAL\Administrator] Virtual machine object cleanup
2022-05-10T07:28:01.541Z
info hostd[B381B70] [Originator@6876 sub=Vmsvc
opID=l2zto750-4t-h5:70000105-5c-01-3a-8917
user=vpxuser:VSPHERE.LOCAL\Administrator] Vmsvc::CreateVm: Exception thrown vim.fault.AlreadyExists
2022-05-10T07:28:01.541Z
info hostd[B381B70] [Originator@6876 sub=Vcsvc.LLPM opID=l2zto750-4t-h5:70000105-5c-01-3a-8917
user=vpxuser:VSPHERE.LOCAL\Administrator] CreateVm: transient VM creation
failed -- vim.fault.AlreadyExists
2022-05-10T07:28:01.541Z
info hostd[B381B70] [Originator@6876 sub=Default
opID=l2zto750-4t-h5:70000105-5c-01-3a-8917 user=vpxuser:VSPHERE.LOCAL\Administrator]
AdapterServer caught exception: vmodl.fault.InvalidArgument
2022-05-10T07:28:01.541Z
info hostd[B381B70] [Originator@6876 sub=Vimsvc.TaskManager opID=l2zto750-4t-h5:70000105-5c-01-3a-8917
user=vpxuser:VSPHERE.LOCAL\Administrator] Task Completed :
haTask--vim.host.LowLevelProvisioningManager.createVm-111715986 Status error
2022-05-10T07:28:01.541Z
verbose hostd[B381B70] [Originator@6876 sub=PropertyProvider
opID=l2zto750-4t-h5:70000105-5c-01-3a-8917
user=vpxuser:VSPHERE.LOCAL\Administrator] RecordOp ASSIGN: info,
haTask--vim.host.LowLevelProvisioningManager.createVm-111715986. Applied change
to temp map.
2022-05-10T07:28:01.929Z
verbose hostd[BF40B70] [Originator@6876 sub=PropertyProvider
opID=l2zto750-4t-h5:70000105-5c-01-b9-891c
user=vpxuser:VSPHERE.LOCAL\Administrator] RecordOp ASSIGN: info, haTask--
2022-05-10T07:28:01.940Z
info hostd[B340B70] [Originator@6876 sub=Libs opID=l2zto750-4t-h5:70000105-5c-01-62-891d
user=vpxuser:VSPHERE.LOCAL\Administrator] CopyFromEntry: Hostlog_Dump: Hostlog
/vmfs/volumes/5d4b65d2-28fd39fa-d8b6-e26ee8500014/NvCenter/NvCenter-6b89aee5.hlog
2022-05-10T07:28:01.940Z
info hostd[B340B70] [Originator@6876 sub=Libs opID=l2zto750-4t-h5:70000105-5c-01-62-891d
user=vpxuser:VSPHERE.LOCAL\Administrator] UUID:
c22039aa-9462-4804-afcf-c92f82343a1d
2022-05-10T07:28:01.940Z
info hostd[B340B70] [Originator@6876 sub=Libs
opID=l2zto750-4t-h5:70000105-5c-01-62-891d user=vpxuser:VSPHERE.LOCAL\Administrator]
MigID: 6059858064348623433
2022-05-10T07:28:01.940Z
info hostd[B340B70] [Originator@6876 sub=Libs
opID=l2zto750-4t-h5:70000105-5c-01-62-891d
user=vpxuser:VSPHERE.LOCAL\Administrator] HLState: failure
2022-05-10T07:28:01.940Z
info hostd[B340B70] [Originator@6876 sub=Libs
opID=l2zto750-4t-h5:70000105-5c-01-62-891d
user=vpxuser:VSPHERE.LOCAL\Administrator] ToFrom: none
2022-05-10T07:28:01.940Z
info hostd[B340B70] [Originator@6876 sub=Libs opID=l2zto750-4t-h5:70000105-5c-01-62-891d
user=vpxuser:VSPHERE.LOCAL\Administrator] MigType: invalid
2022-05-10T07:28:01.940Z
info hostd[B340B70] [Originator@6876 sub=Libs
opID=l2zto750-4t-h5:70000105-5c-01-62-891d
user=vpxuser:VSPHERE.LOCAL\Administrator] OpType: nfc
2022-05-10T07:28:01.940Z
info hostd[B340B70] [Originator@6876 sub=Libs
opID=l2zto750-4t-h5:70000105-5c-01-62-891d
user=vpxuser:VSPHERE.LOCAL\Administrator] WorldID: 0
2022-05-10T07:28:01.940Z
info hostd[B340B70] [Originator@6876 sub=vm: Item Vm F
"/vmfs/volumes/5d4b65d2-28fd39fa-d8b6-e26ee8500014/NvCenter/NvCenter.vmx
opID=l2zto750-4t-h5:70000105-5c-01-62-891d
user=vpxuser:VSPHERE.LOCAL\Administrator]
2022-05-10T07:28:02.285Z
info hostd[B340B70] [Originator@6876 sub=Libs
opID=l2zto750-4t-h5:70000105-5c-01-62-891d
user=vpxuser:VSPHERE.LOCAL\Administrator] OBJLIB-LIB: Failed to get VCFS
root path for '/vmfs/volumes/5d4b65d2-28fd39fa-d8b6-e26ee8500014/NvCenter': No
such file or directory (131076).
2022-05-10T07:28:02.286Z
info hostd[B340B70] [Originator@6876 sub=Libs
opID=l2zto750-4t-h5:70000105-5c-01-62-891d
user=vpxuser:VSPHERE.LOCAL\Administrator] OBJLIB-LIB: Failed to get VCFS
root path for '/vmfs/volumes/5d4b65d2-28fd39fa-d8b6-e26ee8500014/NvCenter': No
such file or directory (131076).
2022-05-10T07:28:02.293Z
info hostd[B340B70] [Originator@6876 sub=Vcsvc.LLPM
opID=l2zto750-4t-h5:70000105-5c-01-62-891d
user=vpxuser:VSPHERE.LOCAL\Administrator] VM [] /vmfs/volumes/5d4b65d2-28fd39fa-d8b6-e26ee8500014/NvCenter/NvCenter.vmx
is currently being loaded
2022-05-10T07:28:02.293Z
warning hostd[B340B70] [Originator@6876 sub=Vcsvc.OCM
opID=l2zto750-4t-h5:70000105-5c-01-62-891d
user=vpxuser:VSPHERE.LOCAL\Administrator] OCMCleanupHostlogItem: Failed to
delete VM /vmfs/volumes/5d4b65d2-28fd39fa-d8b6-e26ee8500014/NvCenter/NvCenter.vmx:
vim.fault.InvalidState
2022-05-10T07:28:02.293Z
warning hostd[B340B70] [Originator@6876 sub=vm:Hostlog_ExecuteCleanup: Callback
failed to delete object /vmfs/volumes/5d4b65d2-28fd39fa-d8b6-e26ee8500014/NvCenter/NvCenter.vmx
opID=l2zto750-4t-h5:70000105-5c-01-62-891d
user=vpxuser:VSPHERE.LOCAL\Administrator] Hlog:
/vmfs/volumes/5d4b65d2-28fd39fa-d8b6-e26ee8500014/NvCenter/NvCenter-6b89aee5.hlog
2022-05-10T07:28:02.293Z
info hostd[B340B70] [Originator@6876 sub=Vimsvc.TaskManager
opID=l2zto750-4t-h5:70000105-5c-01-62-891d
user=vpxuser:VSPHERE.LOCAL\Administrator] Task Completed :
haTask--vim.host.OperationCleanupManager.updateEntry-111715988 Status success
2022-05-10T07:28:02.293Z
verbose hostd[B340B70] [Originator@6876 sub=PropertyProvider
opID=l2zto750-4t-h5:70000105-5c-01-62-891d
user=vpxuser:VSPHERE.LOCAL\Administrator] RecordOp ASSIGN: info,
haTask--vim.host.OperationCleanupManager.updateEntry-111715988. Applied change
to temp map.
Conclusion:
· Based on the review we can see that we are getting the below error:
2022-05-10T07:28:01.541Z
info hostd[B381B70] [Originator@6876 sub=Vmsvc
opID=l2zto750-4t-h5:70000105-5c-01-3a-8917
user=vpxuser:VSPHERE.LOCAL\Administrator] Vmsvc::CreateVm: Exception thrown
vim.fault.AlreadyExists
2022-05-10T07:28:01.541Z
info hostd[B381B70] [Originator@6876 sub=Vcsvc.LLPM opID=l2zto750-4t-h5:70000105-5c-01-3a-8917
user=vpxuser:VSPHERE.LOCAL\Administrator] CreateVm: transient VM creation
failed -- vim.fault.AlreadyExists
· While looking at the vminventory.xml of the Destination Host, I can see that there are duplicate entries for the virtual machine present. This is the reason why we are seeing the error already exist.
<ConfigRoot> <ConfigEntry id="0000"> <objID>1</objID> <secDomain/> <vmxCfgPath>/vmfs/volumes/5d4b65d2-28fd39fa-d8b6-e26ee8500014/NvCenter/NvCenter.vmx</vmxCfgPath> </ConfigEntry> <ConfigEntry id="0001"> <objID>10</objID> <secDomain>7</secDomain> <vmxCfgPath>/vmfs/volumes/5d4b65d2-28fd39fa-d8b6-e26ee8500014/NvCenter1SC-2/NvCenter1SC-2.vmx</vmxCfgPath> </ConfigEntry> <ConfigEntry id="0002"> <objID>12</objID> <secDomain>8</secDomain> <vmxCfgPath>/vmfs/volumes/5d4b65d2-28fd39fa-d8b6-e26ee8500014/NvCenter1PL-6/NvCenter1PL-6.vmx</vmxCfgPath> </ConfigEntry> </ConfigRoot>
· To remove the duplicate entry for the virtual machine:
- Connect to the host using SSH. For more
information, see Using Tech Support Mode in ESXi 4.1, ESXi 5.x, and ESXi 6.x. - Navigate to
the /etc/vmware/hostd directory using this command:cd /etc/vmware/hostd
- Open the vmInventory.xml file
using a Text Editor. - Create a backup of
the vmInventory.xml file. - Identify the duplicate entries for the
virtual machine in the vmInventory.xml file.
- Remove all entries
within <ConfigRoot> and </ConfigRoot> tags.
In our case
<ConfigEntry id="0000">
<objID>1</objID>
<secDomain/>
<vmxCfgPath>/vmfs/volumes/5d4b65d2-28fd39fa-d8b6-e26ee8500014/NvCenter/NvCenter.vmx</vmxCfgPath>
</ConfigEntry>
- Restart management agents on the host using this command:
/etc/init.d/hostd restart
/etc/init.d/vpxa restart
- Perform
the vMotion operation again and let me know the status.
Note: Please note that this plan will not cause any downtime with the ESXi Host, However the Restart of the Management service might disconnect the Host from vCenter for a very short interval.
Let me know if you have any questions regarding this.
