Welcome to Knowledge Base!

Symptoms

You present new LUN(s) to the ESXi Host / Cluster from the Pure Storage FlashArray. Upon initiating a rescan you verify that the expected LUN(s) are not seen on the ESXi Host as available devices under the "Storage Adapters" section. While investigating the issue you are able to confirm other LUNs are successfully mapped to the ESXi Host without issue.

Note: If checking from the command line of the ESXi Host the LUN(s) would be missing the following command:

'esxcli storage core device list'

Diagnosis

Upon reviewing the ESXi Host 'vmkernel' logs you are able to locate the following error message being reported at the time of rescan:

2016-04-23 22:21:43.483Z cpu13:34281087)WARNING: ScsiPath: 903: The number of paths allocated has reached the maximum: 1024. Path: vmhba2:C0:T14:L0 will not be allocated. This warning won't be repeated until some paths are removed.

Additionally, when running the following command from the ESXi Host command line interface it reports a value of '1024':

grep vmhba esxcfg-mpath -b |wc -l

Solution

The error message above indicates that the the maximum number of total paths allowed to an ESXi Host has been reached. You will not be allowed to add additional LUN(s) to the ESXi Host until this is addressed.

This can be resolved the following ways:

• Reduce the number of paths to each LUN so they can add the additional LUN(s) as needed.
• Reduce the number of total LUNs (if possible) to the ESXi Host so they can have desired number of paths.
• Map the LUNs to a new ESXi Host / Cluster.

This decision needs to be made by the customer to determine what the best scenario for their environment is. We do not have control over this variable within the ESXi Host. If they have additional directions kindly direct them to VMware Support as needed.

Review the following VMware links below for additional information:

• Maximum number of paths reached (1020654)
• VMware vSphere 5.1 Maximum Configurations - Page 3
• VMware vSphere 5.5 Maximum Configurations - Page 3
• VMware vSphere 6.0 Maximum Configurations - Page 12

How to capture an esxtop output

There are a couple of very important things to keep in mind when capturing an esxtop:

• The esxtop should be captured from the ESXi host where you suspect possible performance issues are originating from. It is host specific and does not capture for a cluster.
• If there are particular VMs you suspect as the cause of performance anomalies, the esxtop needs to be captured on the ESXi host where the VM resides at the time the esxtop is run.
• When specifying the location of the esxtop.csv file, please be sure you send it to a datastore that has at least 5GB of space available. If you try to send the file to the tmp folder, you risk filling it up and causing the ESXi Host management services to stall. This would require a host reboot and possible outage, so please be mindful of this.
• SSH must be enabled on the ESXi host since esxtop can only be ran / captured from the ESXi host CLI.
• The output file should always be a .csv format for easy review of the data.

esxtop -b -a -d 2 -n 300 > /vmfs/volumes/pure_datastore/esxtop-[hostname]-[date].csv

[root@ac-esxi-6:~] mkdir /vmfs/volumes/sn1-x70-b05-vmfs-scale-ds/esxtop-captures/
[root@ac-esxi-6:~]
[root@ac-esxi-6:~] esxtop -b -a -d 2 -n 300 > /vmfs/volumes/sn1-x70-b05-vmfs-scale-ds/esxtop-captures/esxtop-ac-esxi-6-June-2019.csv
[root@ac-esxi-6:~]
[root@ac-esxi-6:~] ls -lh /vmfs/volumes/sn1-x70-b05-vmfs-scale-ds/esxtop-captures/
total 87040
-rw-r--r--    1 root     root       84.1M Jun 12 15:15 esxtop-ac-esxi-6-June-2019.csv

The above runs the esxtop capture in batch mode ( -b ), captures all available counters ( -a ), at 2 second intervals ( -d ), for 300 iterations ( -n ).  Which means that this will capture 10 minutes of performance data with 2 second intervals (shortest interval VMware supports) and output it to the 'esxtop.csv' file for review.

You will know the esxtop capture is completed once the CLI prompt returns and you are able to type again. After the esxtop has completed running, you can navigate to the path the file was saved and SCP it off the ESXi host and upload to FTPS for Pure Storage review.

Related Links and References

Here are some useful links for unlocking the power of the esxtop output that was captured.

• How to upload files to Pure Storage FTPS Service
• Exporting esxtop performance data as a CSV file
• Using esxtop to identify storage performance issues
• Using visualesxtop to troubleshoot performance issues
• Using visualesxtop on a Mac

Problem

The Pure Storage icon is not showing up in the vCenter Web Client. This can be caused the JDK not being properly installed on the VMware vCenter server.  We can verify whether the JDK is installed correctly, and which Java version is being used by the Pure Plugin in the VMware vSphere Web Client main log file (vsphere_client_virgo.log).

Impact

An incorrect installation and configuration of the JDK will cause issues with the Pure Plugin.

Solution

The Java version can be found in the vshere_client_virgo.log, and will only show up in the log when the vSphere web client is restarted.

In this example, the JDK 1.7u17 is installed on the Windows Server 2008 R2 for vCenter Server 5.1.

[2016-07-15 10:22:41.225] INFO  [INFO ] start-signalling-1            com.vmware.vise.util.debug.SystemUsageMonitor                     System info :
 OS - Windows Server 2008 R2
 Arch - amd64
 Java Version - 1.7.0_17 
[2016-07-15 10:22:41.256] INFO  [INFO ] Timer-2                       com.vmware.vise.util.debug.SystemUsageMonitor                     
 Heap     : init = 201292928(196575K) used = 309326640(302076K) committed = 672727040(656960K) max = 954466304(932096K)
 non-Heap : init = 136773632(133568K) used = 82368472(80437K) committed = 142344192(139008K) max = 318767104(311296K)
 No of loaded classes : 13796 

Problem

When viewing the Performance under the Pure Storage tab in the vSphere Web Client, you get an error "Content was blocked because it was not signed by a valid security certificate. For more information, see Certificate Errors in Internet Explorer Help".

This issue can occur in any web browser, not just Internet Explorer.

Impact

On the vSphere Web Client, the user is unable to view the Performance graph in the Pure Storage tab.

Solution

Open a new browser window or tab to log in to the Pure FlashArray GUI.  Once you are able to see the login screen of the Pure FlashArray, then the browser has accepted the SSL certificate from the Pure FlashArray.

Then go back to the vSphere Web Client, the Performance graph should now be displayed under the Pure Storage tab.

Problem

When attempting to install the Pure Storage vSphere plugin in a vSphere 5.1 environment, the following errors are reported in the vSphere virgo client logs:

[2016-06-30 09:44:02.629] ERROR [ERROR] P Connection(2)-170.92.17.57 org.eclipse.virgo.kernel.deployer.management.StandardDeployer Exception filtered from JMX invocation org.eclipse.virgo.kernel.deployer.core.DeploymentException: Error creating bean with name 'FlashArrayDataAdapter': Invocation of init method failed; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'FlashArrayDataAdapterImpl' defined in URL [bundleentry://243.fwk971840267/META-INF/spring/bundle-context.xml]: Instantiation of bean failed; nested exception is java.lang.UnsupportedClassVersionError: com/purestorage/rest/exceptions/PureException : Unsupported major.minor version 51.0
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'FlashArrayDataAdapterImpl' defined in URL [bundleentry://243.fwk971840267/META-INF/spring/bundle-context.xml]: Instantiation of bean failed; nested exception is java.lang.UnsupportedClassVersionError: com/purestorage/rest/exceptions/PureException : Unsupported major.minor version 51.0
Caused by: java.lang.UnsupportedClassVersionError: com/purestorage/rest/exceptions/PureException : Unsupported major.minor version 51.0
at com.purestorage.FlashArrayDataAdapter.<clinit>(Unknown Source)

Impact

vSphere Plugin Pure Storage icon does not show up on vSphere Web Client.

Solution

The error message "Unsupported major.minor version 51.0" is coming from Java and does not mean that vCenter 5.1 is not supported. This message indicates that the vCenter 5.1 is not using JDK 7.

See https://en.wikipedia.org/wiki/Java_class_file for a list of mapping from JDK version to the major version of the class file:

major version number of the class file format being used.

Java SE 7 = 51 (0x33 hex)

vCenter 5.1 comes with JDK 6 by default, which would cause this exception. The solution is to update to JDK 7u17 as per the vSphere Plugin FAQ for the vSphere plugin to work.

Link to the JDK 7u17:

http://www.oracle.com/technetwork/java/javase/downloads/java-archive-downloads-javase7-521261.html#jdk-7u17-oth-JPR