Welcome to Knowledge Base!

KB at your finger tips

This is one stop global knowledge base where you can learn about all the products, solutions and support features.

Categories
All

Storage and Backups-Purestorage

SRM User Guide: Operation Log Locations

Pure Storage Storage Replication Adapter Log File Cheat Sheet

This sheet lists all of the common Site Recovery Manager management or recovery operations and the relevant SRA operations initiated by them. The location of the respective log for the SRA operation is listed, is one of the following:

  • Protected SRM server in respect to the Recovery Plan
  • Recovery SRM server in respect to the Recovery Plan
  • Initiating SRM server—in other words where the operation was started from (create an array manager on site A SRM server it will be on site A)
  • Both—this operation will create a log for that operation on both SRM servers
  • Former Recovery Site (in the case of reprotect where the recovery/protected servers have switched for that recovery plan this would be the original, now protected, SRM server)

Photon SRM Appliance Logs

The SRM logs are located at:

/var/log/vmware/srm/

The SRA logs are located at:

/var/log/vmware/srm/SRAs/sha256{RandomCharacters}

Windows SRM Appliance Logs

SRM logs are located at:

C:\ProgramData\VMware\VMware vCenter Site Recovery Manager\Logs\vmware-dr*

SRA logs are located at:

C:\ProgramData\VMware\VMware vCenter Site Recovery Manager\Logs\SRAs\purestorage\

Photon SRM Appliance Logs

The SRM logs are located at:

/var/log/vmware/srm/

The SRA logs are located at:

/var/log/vmware/srm/SRAs/sha256{RandomCharacters}

Windows SRM Appliance Logs

SRM logs are located at:

C:\ProgramData\VMware\VMware vCenter Site Recovery Manager\Logs\vmware-dr*

SRA logs are located at:

C:\ProgramData\VMware\VMware vCenter Site Recovery Manager\Logs\SRAs\purestorage\

SRM Operations

SRA Discovery

SRM Operation

SRA Operation

Log Location

SRA Discover

QueryInfo

Initiating SRM server

QueryCapabilities

QueryConnectionParameters

QueryErrorDefinitions

DiscoverArrays   [i]

DiscoverDevice   [ii]

QueryReplicationSettings  [iii]

Create Array Manager

SRM Operation

SRA Operation

Log Location

Discover Arrays

DiscoverArrays

Initiating SRM server

Discover New Arrays

SRM Operation

SRA Operation

Log Location

Discover Arrays

DiscoverArrays

Initiating SRM server

Enable Array Pair

SRM Operation

SRA Operation

Log Location

Discover Devices

DiscoverDevices

Both

Discover New Devices

SRM Operation

SRA Operation

Log Location

Discover Devices

DiscoverDevices

Both

Test Recovery Start

SRM Recovery Plan Step

SRA Operation

Log Location

Synchronize Storage

QueryReplicationSettings  [iv]

Protected

SyncOnce  [v]

Protected

QuerySyncStatus  [vi]

Protected

Create Writeable Storage Snapshot

TestFailoverStart

Recovery

DiscoverDevices

Recovery

Test Recovery Cleanup

SRM Recovery Plan Step

SRA Operation

Log Location

Discard test data and reset storage

TestFailoverStop

Recovery

DiscoverDevices

Recovery

Recovery (Planned Migration, in DR some operations may not occur)

SRM Recovery Plan Step

SRA Operation

Log Location

Pre-synchronize Storage

QueryReplicationSettings

Protected

SyncOnce

Protected

QuerySyncStatus

Protected

Prepare Protected VMs for Migration

PrepareFailover

Protected

DiscoverDevices

Protected

Synchronize Storage

SyncOnce

Protected

QuerySyncStatus

Protected

Change Recovery Site Storage to Writeable

Failover

Recovery

DiscoverDevices

Recovery

Reprotect

SRM Recovery Plan Step

SRA Operation

Log Location

Configure Storage to Reverse Direction

ReverseReplication

Former Recovery Site

DiscoverDevices

Both

Synchronize Storage

QueryReplicationSettings

Former Recovery Site

SyncOnce

Former Recovery Site

QuerySyncStatus

Former Recovery Site

SRA Discovery

SRM Operation

SRA Operation

Log Location

SRA Discover

QueryInfo

Initiating SRM server

QueryCapabilities

QueryConnectionParameters

QueryErrorDefinitions

DiscoverArrays   [i]

DiscoverDevice   [ii]

QueryReplicationSettings  [iii]

Create Array Manager

SRM Operation

SRA Operation

Log Location

Discover Arrays

DiscoverArrays

Initiating SRM server

Discover New Arrays

SRM Operation

SRA Operation

Log Location

Discover Arrays

DiscoverArrays

Initiating SRM server

Enable Array Pair

SRM Operation

SRA Operation

Log Location

Discover Devices

DiscoverDevices

Both

Discover New Devices

SRM Operation

SRA Operation

Log Location

Discover Devices

DiscoverDevices

Both

Test Recovery Start

SRM Recovery Plan Step

SRA Operation

Log Location

Synchronize Storage

QueryReplicationSettings  [iv]

Protected

SyncOnce  [v]

Protected

QuerySyncStatus  [vi]

Protected

Create Writeable Storage Snapshot

TestFailoverStart

Recovery

DiscoverDevices

Recovery

Test Recovery Cleanup

SRM Recovery Plan Step

SRA Operation

Log Location

Discard test data and reset storage

TestFailoverStop

Recovery

DiscoverDevices

Recovery

Recovery (Planned Migration, in DR some operations may not occur)

SRM Recovery Plan Step

SRA Operation

Log Location

Pre-synchronize Storage

QueryReplicationSettings

Protected

SyncOnce

Protected

QuerySyncStatus

Protected

Prepare Protected VMs for Migration

PrepareFailover

Protected

DiscoverDevices

Protected

Synchronize Storage

SyncOnce

Protected

QuerySyncStatus

Protected

Change Recovery Site Storage to Writeable

Failover

Recovery

DiscoverDevices

Recovery

Reprotect

SRM Recovery Plan Step

SRA Operation

Log Location

Configure Storage to Reverse Direction

ReverseReplication

Former Recovery Site

DiscoverDevices

Both

Synchronize Storage

QueryReplicationSettings

Former Recovery Site

SyncOnce

Former Recovery Site

QuerySyncStatus

Former Recovery Site

Glossary of SRA Operations

This section lists all of the relevant SRM to SRA operations. Each operation has a definition in accordance to what SRM expects to happen and then also a definition of what the Pure Storage SRA actually does to fulfill SRMs expectations.

  • queryInfo
    • SRM: Queries the SRA for basic properties such as name and version
    • SRA: Returns SRA name, version number, company and website
  • queryCapabilities
    • SRM: Queries the SRA for supported models of storage arrays and supported SRM commands
    • SRA: Returns FA 400 series, Purity 4.0, supported protocols (FC and iSCSI) and supported SRM commands: failover, discoverArrays, discoverDevices, prepareFailover, prepareRestoreReplication, queryCapabilities, queryConnectionParameters, queryErrorDefinitions, queryReplicationSettings, querySyncStatus, restoreReplication, reverseReplication, syncOnce, testFailoverStart, testFailoverStop, queryInfo.
  • queryErrorDefinitions
    • SRM: Queries the SRA for pre-defined array specific errors
    • SRA: Returns error messages relating specifically to the FlashArray
  • queryConnectionParameters
    • SRM: Queries the SRA for parameters needed to connect to the array management system to perform array management operations
    • SRA: Returns questions for array manager configuration to request connection/credential information for the local and remote FlashArray.
  • discoverArrays
    • SRM: Discovers storage array pairs configured for replication
    • SRA: Returns FlashArray information, Purity level, controller information and serial number/name.
  • discoverDevices
    • SRM: Discovers replicated devices on a given storage array
    • SRA: Returns local and remotely replicated devices (name and state), hosts on FlashArray information, initiator and storage port identifiers. Also looks for demoted devices, devices used for test failover and recovered volumes that are not yet replicated.
  • queryReplicationSettings
    • SRM: Queries replication settings for a list of devices
    • SRA: Returns host grouping information for local, replicated devices
  • syncOnce
    • SRM: Requests immediate replication
    • SRA: Starts a FlashRecover “replicatenow” operation and creates new remote snapshots for the given devices on the remote array through their protection groups. If there are multiple protection groups involved they will be all replicated. The source device and the new snapshot name will be returned to SRM along with the replication progress.
  • querySyncStatus
    • SRM: Queries the status of a replication initiated by syncOnce
    • SRA: Returns the source device and the new snapshot name to SRM along with the replication progress.
  • testFailoverStart
    • SRM: Creates writable temporary copies of the target devices
    • SRA: The SRA identifies the latest snapshot for each volume, identifies the ESXi connectivity information and correlates it with the configured hosts on the FlashArray. Favors attaching to a hostgroup over a host. Then creates a new volume for each source volume with the suffix of –puresra-testfailover and associates snapshots and then connects the volumes to the host or host group.
  • testFailoverStop
    • SRM: Deletes the temporary copies created by testFailoverStart
    • SRA: Disconnects and eradicates volumes created for a test recovery. Only volumes with the original prefix (the source name) and the puresra-testfailover suffix will be identified and eradicated. Replica names are returned to SRM.
  • prepareFailover
    • SRM: Makes source devices read-only and optionally takes a snapshot of the source devices in anticipation of a failover
    • SRA: SRA renames the source volumes with a suffix of –puresra-demoted and disconnects them from the hosts. Returns original volume name to SRM.
  • failover
    • SRM: Promotes target devices by stopping replication for those devices and making them writable
    • SRA: The SRA identifies the latest snapshot for each volume, identifies the ESXi connectivity information and correlates it with the configured hosts on the FlashArray. Favors attaching to a hostgroup over a host. Then either creates a new volume for each source volume or it identifies if there is a former volume from a previous recovery with the –puresra-demoted suffix and either names it or renames it with the suffix of –puresra-failover and associates snapshots and then connects the volumes to the host or host group.
  • reverseReplication
    • SRM: Reverses array replication so that the original target array becomes the source array and vice versa
    • SRA: Renames recovery volumes to remove –puresra-failover suffix, recreates the protection groups for the original source volumes on the target side and adds the volumes into it.

Footnotes

[i] Only is created if there are already existing array managers created in SRM for the Pure Storage SRA.

[ii] Only is created if one or more array pairs are enabled

[iii] Only is created if one or more array pairs are enabled

[iv] Only created if “Replicate Recent Changes” is selected at the start of the test recovery

[v] Only created if “Replicate Recent Changes” is selected at the start of the test recovery

[vi] Only created if “Replicate Recent Changes” is selected at the start of the test recovery

Troubleshooting when Formatting iSCSI VMFS Datastore Generates Error

Symptoms

vSphere reports the following error while attempting to format a VMFS datastore using a Pure Storage iSCSI LUN:

"HostDatastoreSystem.CreateVmfsDatastore" for object "<...>" on vCenter Server "<...>" failed

The LUN will report as online and available under the "Storage Adapters" section in the vSphere Client.

Diagnosis

This error can be due to improper configuration in the network path causing jumbo frames to be fragmented from the ESXi Host to the FlashArray.

How to confirm Jumbo Frames can pass through the network

Run the following command from the ESXi Host in question via SSH:

vmkping -d -s 8972 <target portal ipaddress>

If no response is received, or the following message is returned, then jumbo frames are not successfully traversing the network:

sendto() failed (Message too long)

sendto() failed (Message too long)

sendto() failed (Message too long)

There is an L2 device between the ESXi host and FlashArray that is not allowing jumbo frames to properly pass. Please have the customer check virtual and physical switches on the subnet to ensure jumbo frames are configured from end-to-end.

Solution

Make sure all network devices allow jumbo frames to pass from the ESXi host to the Pure Storage FlashArray.

Read article

Troubleshooting ESXi iSCSI Configuration Issue: Pure Shows Different Hosts in GUI and CLI

Read article

Troubleshooting when ESXi Hosts Disconnect with CHAP Enabled

Problem

Enabling CHAP authentication leads to ESXi hosts disconnecting and they are unable to reconnect.

Scenario

The array has CHAP authentication enabled and is unable to reconnect after configuring CHAP on the ESXi host.

Cause

Purity does not support Dynamic Discovery with CHAP.

Solution

Follow this blog post for a more detailed guide.

Configure the ESXi host to use static CHAP, confirm Dynamic CHAP is not set up, and inherit from parent is not checked.

Screen Shot 2015-07-13 at 10.16.52 AM.png


Two methods of configuring CHAP to the pure array:

Procedure 1 - Manually enter Static Discovery targets

  1. Configure array to use CHAP by entering Host User and Host Password in GUI > Storage Tab > Host > Gear > Configure CHAP .
  2. Confirm iSCSI Adapter > Properties > Dynamic Discovery does NOT contain array target.
  3. Configure iSCSI initiator to use CHAP by selecting Use CHAP and entering Name and Secret that matches array CHAP settings in vSphere Client > iSCSI Adapter > Properties > General tab > CHAP .
  4. Add Static Discovery array targets in vSphere Client > iSCSI Adapter > Properties > Static Discovery . Confirm the CHAP settings for each target are set to Inherit from parent.
  5. Rescan adapter

Procedure 2 - Enter CHAP settings for each discovered Static Discovery target

  1. Configure array to use CHAP by entering Host User and Host Password in GUI > Storage Tab > Host > Gear > Configure CHAP .
  2. Enter a single array iSCSI port IP address in iSCSI Adapter > Properties > Dynamic Discovery .
  3. Confirm Static Discovery list is populated with array iSCSI targets in iSCSI Adapter > Properties > Dynamic Discovery .
  4. For each array Static Discovery target configure CHAP settings to NOT inherit from parent and Use CHAP in iSCSI Adapter > Properties > Static Discovery > <target> > Settings > CHAP .
  5. Rescan adapter

Procedure 1 - Manually enter Static Discovery targets

  1. Configure array to use CHAP by entering Host User and Host Password in GUI > Storage Tab > Host > Gear > Configure CHAP .
  2. Confirm iSCSI Adapter > Properties > Dynamic Discovery does NOT contain array target.
  3. Configure iSCSI initiator to use CHAP by selecting Use CHAP and entering Name and Secret that matches array CHAP settings in vSphere Client > iSCSI Adapter > Properties > General tab > CHAP .
  4. Add Static Discovery array targets in vSphere Client > iSCSI Adapter > Properties > Static Discovery . Confirm the CHAP settings for each target are set to Inherit from parent.
  5. Rescan adapter

Procedure 2 - Enter CHAP settings for each discovered Static Discovery target

  1. Configure array to use CHAP by entering Host User and Host Password in GUI > Storage Tab > Host > Gear > Configure CHAP .
  2. Enter a single array iSCSI port IP address in iSCSI Adapter > Properties > Dynamic Discovery .
  3. Confirm Static Discovery list is populated with array iSCSI targets in iSCSI Adapter > Properties > Dynamic Discovery .
  4. For each array Static Discovery target configure CHAP settings to NOT inherit from parent and Use CHAP in iSCSI Adapter > Properties > Static Discovery > <target> > Settings > CHAP .
  5. Rescan adapter
Read article

Troubleshooting when Unable to Increase Datastore in ESXi vSphere

Problem

When trying to increase an existing volume you will see the new volume size in vCenter. However, when the increase button is clicked for the datastore, the available capacity on the volume does not show up.

Solution

If the correct volume size is not being reflected, the management services on the ESX hosts that aren't reflecting the volume size correctly may need to be restarted and storage rescanned.

Please refer to VMware support for this recommendation as restarting management services has the potential to impact tasks that are running on the ESXi host at the time of the restart.

There are a few reasons this may be happening, you may have one fo the following issues , please refer to the following:

  1. Try VMware KB Unable to grow or expand a VMFS volume or datastore (1017662)
  2. Make sure that the Host Personality is configured in Purity//FA.
  3. Try VMware KB Cannot extend datastore through vCenter Server (1011754)

If none of these references fix the issue, please open a VMware Support case and reach out to Pure Storage Support for further assistance.

Read article

Capturing VMware Support Bundles

Description

The Pure Storage FlashArray is consistently used with VMware environments and there's a good chance that Support will have cases where they need to troubleshoot and diagnose how the FlashArray interacts with the VMware environment.

During live troubleshooting, both the customer and Pure Support can look at the logs live as needed, however for an investigation into events that have already occurred VMware Support Logs will need to be provided to Support to move that investigation forward.


Procedure to Gather vCenter and ESXi Logs

VMware has a detailed KB on how to Gather vCenter and ESXi Logs.  Please review VMware's documentation.

With vCenter 6.7 release, there has been more adoption of the HTML5 Client now.  There is no export option in the monitor tab, so the process is a little different.

Here is an example of the HTML5 Client.

Right Click on the vCenter and Select Export System Logs
Export Logs - 02 - HTML5 - 01 -  vCenter - Right Click.png
Check the box to include vCenter logs.  If the Support case is related to vVols, SRM or Plugins, this is very important to gather.
Export Logs - 02 - HTML5 - 02 - vCenter - Export Logs.png
Then you can export the logs.  The default selections for the hosts are usually enough for Pure Support.
Export Logs - 02 - HTML5 - 03 - vCenter - Export Logs.png
Keep in mind, the HTML5 Log exports will be named a little different then the from the flash client.
Export Logs -03 - File Types.png

Then here is an example of the Flash client and the monitor tab.

Navigate to the vCenter that the logs are being gathered for.
Go to the Monitor tab and click on system logs.
Then click on export system logs.
Export Logs - 01 - Flash - 01 - vCenter - Monitor - System Logs.png
Check the box to include vCenter logs.  If the Support case is related to vVols, SRM or Plugins, this is very important to gather.
Export Logs - 01 - Flash - 02 - vCenter - Export Logs.png
Then click Finish.  Another window will pop up asking where to save the compressed file.
Export Logs - 01 - Flash - 03 - vCenter - Export Logs.png

Once the logs are downloaded, they can be uploaded via Pure1 for Support.

There is a Uploading Files to a Support Case KB that outlines that process.


Read article