Capture Network Trace from an ESXi Host

  • Post category:VMware / Vmware vSphere
  • Post last modified:July 25, 2024

Performing a network trace on an ESXi host can help diagnose network-related issues by capturing and analyzing network traffic. The process involves using the command-line interface (CLI) to capture packets and then analyzing these packets using tools like Wireshark.

You might have encountered a situation where your In-House Networking Team is giving you a hard time accepting that there are no issues on their end and since you have a problem with the ESXi Host, you are stuck and need help capturing network traces for them to review.

In this article, we are going to learn exactly how to make that happen.

First, we need to understand exactly where we need to capture the network traces. To simplify this we can have this in Two Ways:

Investigating Packet drops for a Virtual Machine:

As we know a Virtual Machine that is running on an ESXi host will have some connectivity with the Physical Network Interfaces.

  • To find that connectivity ID or in VMware’s term Switch Port Number we can follow the below steps:
    • Log in to the ESXi host where the VM is running and gather all the below information, 
esxcli network vm list | grep -i VM_Name
esxcli network vm port list -w World_ID 
    • World_ID could be found from the first command.
  • Once you have captured the switch Port Number to which the VM is connected you can run the below command to capture the Incoming and Outgoing Traffic.
    Please note that you will have to run it separately
  • The switchport_number for the below commands can be obtained from the second command.
    • Open to SSH session to the same host and run the below command by replacing the right parameters.
      pktcap-uw --switchport switchport_number --dir 1 -o /vmfs/volumes/datastore/switchportnumber_dir1.pcap
      pktcap-uw --switchport switchport_number --dir 0 -o /vmfs/volumes/datastore/switchportnumber_dir0.pcap
    • This would capture all the inbound and outbound packets and save it to /vmfs/volumes/datastore/

 

Investigating Packet drops for a Physical Network Adapter:

Packet capture on pnic (assumed as vmnic0, modify accordingly)

pktcap-uw --uplink vmnicX --dir 0 -o /vmfs/volumes/datastore/vmnicX_dir0.pcap
pktcap-uw --uplink vmnicX --dir 1 -o /vmfs/volumes/datastore/vmnicX_dir1.pcap

Investigating Packet drops for a vNic:

Packet capture on vNic:

  • Run net-stats -l and find the problematic VM’s switchport.
    For e.g. 33554927 is the switchport used by VM1
PortNum          Type SubType SwitchName       MACAddress         ClientName
33554927            5       7 DvsPortset-0     00:50:56:8a:24:c8  VM1.eth0
  • Run the command:
pktcap-uw ---switchport 33554927 -o -dir 0 -o /vmfs/volumes/datastore/vnicX_dir0.pcap
pktcap-uw ---switchport 33554927 -o -dir 1 -o /vmfs/volumes/datastore/vnicX_dir0.pcap

Lab Test:

  • Pinged the vCenter IP: 192.168.50.10 from ESXi Host 192.168.1.2
  • On DIR-1 We can see the response going to the Destination IP 50.10
Machine generated alternative text: vmnicO dirl .pcap File Edit View Go Capture Analyze Statistics Telephony 192.168 192.168 192.168 192.168 192.168 192.168 192.168 192.168 192.168 192.168 192.168 192.168 192.168 192.168 Wireless Tools p. addr 6 ø.øsø757 1.ø63123 213 2.1364726 28 3.e66821 4.'36905 53 5.1371267 67 6.1373468 69 6.215685 7.'374525 97 8.e76845 1131 9.1378775 le.e81842 113 11. e83937 124 12. ø87ø48 136 13. e8921e 141 14. egeg82 icmp Source 192.168.1.2 192.168.1.2 Destnaton 192.168.5ø.1ø 192.168.5ø.1ø 192.168.5e.1e 192.168.5e.1e 192.168.5e.1e 192.168.5e.1e 192.168.5e.1e 192.168.5ø.1ø 192.168.5e.1e 192.168.5e.1e 192.168.5e.1e 192.168.5e.1e 192.168.5e.1e 192.168.5e.1e 192.168.5e.1e 192.168.5e.1e Pr o tocol lcmp lcmp lcmp lcmp lcmp lcmp lcmp lcmp lcmp lcmp lcmp lcmp lcmp lcmp lcmp Help Length 98 98 98 98 98 98 98 98 98 98 Echo Echo Echo Echo Echo Echo Echo (ping) (ping) (ping) (ping) (ping) (ping) (ping) request request request request request request request Len=333 request request request request request request request request id=øxacf6 , id=øxa c f6 , id=øxacf6, id=øxacf6, id=øxacf6, id=øxacf6, id=øxacf6, id=øxacf6, id=øxacf6, id=øxacf6, id=øxacf6, id=øxacf6, id=øxacf6, id=øxacf6, id=øxacf6, seq=335/2ø225, seq=336/2ø481, seq=337/2ß737, seq=338/2egg3, seq=33g/21249, seq=34e/215ß5, seq=341/21761, seq=342/22ø17, seq=343/22273, seq=345/22785, seq=346/23e41, seq=347/23297, seq=348/23553, seq=34g/238eg, ttI=64 375 * 9132 98 98 98 98 98 Echo Echo Echo Echo Echo Echo Echo Echo (ping) (ping) (ping) (ping) (ping) (ping) (ping) (ping) ttl ttl ttl ttl ttl ttl ttl ttl ttl ttl ttl ttl ttl ttl = 64 = 64 = 64 = 64 = 64 = 64 = 64 = 64 = 64 = 64 = 64 = 64 = 64 = 64 ( no (no (no (no (no (no (no (no (no (no (no (no (no (no (no response response response response response response response response response response response response response response response fou nd ! ) found!) found ! ) found ! ) found ! ) found ! ) found ! ) found ! ) found ! ) found ! ) found ! ) found ! ) found ! ) found ! ) found ! )
  • To check the Response when we check the DIR-0 Pcap we can see below:
  • There is a reply to the Ping which we have initiated with the vCenter Server.
Machine generated alternative text: vmnicO dirO.pcap File Edit View Go Capture 192.168 tt1=62 192.168 192. Isa 192.168 tt1=62 192.168 tt1=62 192.168 192.168. 192. Isa 192.168 192. Isa 192.168 tt1=62 192.168 tt1=62 192.168 tt1=62 192.168 Analyze Statistics Telephony Destnaton lcmp icmpv6 6 1.73%49 2. 732756 3. 73413113 4.736227 5. 738e48 7. 7422øe 8.744314 g. 746421 Source 18 26 28 32 34 35 41 44 46 64 85 192.168.5ø.1ø 192.168. sø.lø 192.16a.se.23 192.168.se.1e 192.168.5e.1e 192.168.5e.1e 192.168. se.le 192.16a.se.23 192.16a.se.1e 192.16a.se.23 192.168.se.1e 192.168.5e.1e 192.168.5e.1e 192.168.5e.1e Wireless Tools Pr o tocol lcmp lcmp ICY? lcmp lcmp ICY? lcmp lcmp lcmp Help Length 98 98 98 98 98 98 2 24 289 98 98 98 98 Echo (ping) Echo (ping) Destination Echo (ping) Echo (ping) Echo (ping) Echo (ping) Destination Echc (ping) Destination reply id=øxacf6, seq=332/19457, reply id=øxacf6, seq=333/19713, unreachable (Pct-t unreachable) reply id=exacf6, seq=334/19969, reply id=exacf6, seq=335/2ß225, reply id=øxacf6, seq=336/2ß481, reply id=exacf6, seq=337/2e737, unreachable (Pct-t unreachable) reply id=exacfE, seq=33a/2C993, unreachable (Pct-t unreachable) ttI=62 ttl ttl ttl ttl le. 747493 Echo Echo Echo Echo (ping) (ping) (ping) (ping) reply reply reply reply id=exacf6, id=øxacf6, id=øxacf6, id=øxacf6, =339/21249, seq seq=34e/215ß5, seq=341/21761, seq=342/22e17,

Ashutosh Dixit

I am currently working as a Senior Technical Support Engineer with VMware Premier Services for Telco. Before this, I worked as a Technical Lead with Microsoft Enterprise Platform Support for Production and Premier Support. I am an expert in High-Availability, Deployments, and VMware Core technology along with Tanzu and Horizon.

Leave a Reply