EMC Storage Performance Monitoring: PowerMax VMAX VPLEX Unity ECS VNX Isilon DataDomain XtremIO CLARiiON Celerra VNXe ECS ME4

Installation

Dell EMC storage installation

PowerMax / VMAX
VPLEX
Unity
VNX / CLARiiON
VNX file / Celerra
DataDomain
XtremIO
Isilon
VNXe
ME4 Series
ECS

In case of usage of Virtual Appliance

Use local account lpar2rrd for hosting of STOR2RRD on the virtual appliance
Use /home/stor2rrd/stor2rrd as the product home

This install procedure is identical for VNX block and CLARiiON CX4 storage devices.

Prerequisites

Allow access from the STOR2RRD host to the storage service processors ports 6389,443 (TCP).
Test if ports are open:

$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 6389
  Connection to "192.168.1.1" on port "6389" is ok
$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 443
  Connection to "192.168.1.1" on port "443" is ok

Install Navisphere Secure CLI. Download it from EMC support page:
(Note: use always the latest version to avoid compatibility issues)
- AIX download of Navisphere CLI (AIX) 7.33.9.1.84-1
  Install it as a lpp source, it will be installed into /usr/bin/naviseccli
```
# smitty installp ➡ select actuall directory ➡ find package F4 ➡ install
```
- Linux download of Navisphere CLI (Linux x64) 7.33.9.1.84-1
  It will be installed into /opt/Navisphere/bin/naviseccli)
```
# rpm -i NaviCLI-Linux-64-x86-en_US-7.33.9.1.84-1.x86_64.rpm
```
- Debian, Ubuntu packages:
  Use either Debian download or conver it via "alien" cmd
```
# alien NaviCLI-Linux-64-x86-en_US-7.33.9.1.84-1.x86_64.rpm
  NaviCLI-Linux-64-x86-en_US-7.33.9.1.84-1.x86_64.deb generated
```
During installation select low verifying level.
In case you select "medium" then accept and store certificate during first manuall connection to the storage.
Create stor2rrd user on the storage (lpar2rrd on Virtual Appliance)
Open the storage UI and navigate to storage ➡ settings ➡ user management ➡ local users ➡ add.
Create user account stor2rrd with role operator.
Do not use shell special characters like #!?|.*[]\{} in the password
Allow statistics logging on the storage
UI ➡ Storage ➡ System properties ➡ Statistics logging ➡ yes

Configure naviseccli, on AIX use /usr/bin/naviseccli (lpar2rrd user on Virtual Appliance)
User your own scope <0 - global; 1 - local; 2 - LDAP>

# su - stor2rrd
$ /opt/Navisphere/bin/naviseccli -addusersecurity -scope 1 -password XXXXXXXXXXX -user stor2rrd 
$ /opt/Navisphere/bin/naviseccli -h <storage controller IP/hostname> getcontrol -all
  System Fault LED:              OFF
  Statistics Logging:            ON
                ...

Statistics logging must be ON

If you get bellow error assure you use right user/password combination and scope (loca/global/LDAP)

$ /opt/Navisphere/bin/naviseccli -h <storage controller IP/hostname> getcontrol -all
  Error returned from the Management Server on 192.168.1.1
  Authentication failed. Possible reasons for failure are invalid security file, invalid username, password and/or scope.

Remove user and define it again with different scope (like below) or user credentials:

$ /opt/Navisphere/bin/naviseccli -RemoveUserSecurity 
$ /opt/Navisphere/bin/naviseccli -addusersecurity -scope 0 -password XXXXXXXXXXX -user stor2rrd

If you have selected medium security level then accept and store certificate

$ /opt/Navisphere/bin/naviseccli -h <storage controller IP/hostname> getsptime
  Unable to validate the identity of the server.  There are issues with the certificate presented.
  Only import this certificate if you have reason to believe it was sent by a trusted source.
  Certificate details:
  Subject:        CN=192.168.1.1,CN=VNX46EM_SPA,OU=CLARiiON
  Issuer: CN=192.168.1.1,CN=VNX46EM_SPA,OU=CLARiiON
  Serial#:        2c3c9999
  Valid From:     20140724193051Z
  Valid To:       20190723193051Z
  Would you like to [1]Accept the certificate for this session, [2] Accept and store ,[3] Reject the certificate?
  Please input your selection(The default selection is [1]):
  --> select "2"

  Time on SP A: 04/04/17 09:53:54
  Time on SP B: 04/04/17 09:49:00

If you get below error then you probaly use older NaviCLI 7.30.
Upgrade to 7.33.8.1.19-1

$ /opt/Navisphere/bin/naviseccli -h <storage controller IP/hostname> getcontrol -all
  Exception: End of data stream

STOR2RRD storage configuration

All actions below under stor2rrd user (lpar2rrd on Virtual Appliance)
STOR2RRD v2.60+:
Add storage into configuration from the UI:
Settings icon ➡ Storage ➡ New ➡ Vendor:device ➡ Dell EMC VNX block

STOR2RRD v2.50 and olders:

Configure storage in etc/storage-list.cfg
Uncomment (remove the hash) example line and adjust it:

$ vi /home/stor2rrd/stor2rrd/etc/storage-list.cfg

#
# EMC VNX block
#
# Storage Alias:VNX:IP address SPA:IP address SPB:VOLUME_AGG_DATA_LIM:VOLUME_AGG_IO_LIM
#EMC-VNX-block_alias:VNX:192.168.1.2:192.168.1.3:0:5

EMC-VNX-block_alias01:VNX:192.168.1.2:192.168.1.3

Assure you have enough of disk space on the filesystem where is STOR2RRD installed
Roughly you might count 2 - 30 GB per a storage (it depends on number of volumes, 30GB for 5000 volumes)
```
$ df -g /home   # AIX
$ df -h /home   # Linux
```

Test the storage connectivity:

$ cd /home/stor2rrd/stor2rrd
$ ./bin/config_check.sh 
  ===============================
  STORAGE: VNX-block_alias01: VNX
  ===============================
  first IP address : 192.168.1.2
  /opt/Navisphere/bin/naviseccli -h 192.168.1.2 getsptime
    Time on SP A: 11/03/16 12:56:45
    Time on SP B: 11/03/16 12:56:57
  connection ok

  second IP address : 192.168.1.3
  /opt/Navisphere/bin/naviseccli -h 192.168.1.3 getsptime
    Time on SP A: 11/03/16 12:56:47
    Time on SP B: 11/03/16 12:56:58
  connection ok

Schedule to run storage agent from stor2rrd crontab (lpar2rrd on Virtual Appliance, it might already exist there)

$ crontab -l | grep load_vnxperf.sh
$

Add if it does not exist as above

$ crontab -e

# EMC VNX storage agent
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/stor2rrd/stor2rrd/load_vnxperf.sh > /home/stor2rrd/stor2rrd/load_vnxperf.out 2>&1

Assure there is already an entry with the UI creation running once an hour in crontab

$ crontab -e

# STOR2RRD UI (just ONE entry of load.sh must be there)
5 * * * * /home/stor2rrd/stor2rrd/load.sh > /home/stor2rrd/stor2rrd/load.out 2>&1

Let run the storage agent for 15 - 20 minutes to get data, then:
```
$ cd /home/stor2rrd/stor2rrd
$ ./load.sh
```
Go to the web UI: http://<your web server>/stor2rrd/
Use Ctrl-F5 to refresh the web browser cache.

In case of usage of Virtual Appliance

Use local account lpar2rrd for hosting of STOR2RRD on the virtual appliance
Use /home/stor2rrd/stor2rrd as the product home

Prerequisites

Allow access from the STOR2RRD host to the storage on port 22 (do not use data mover IP!).
Test if port is open:

$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 22
  Connection to "192.168.1.1" on port "22" is ok

Create a user on the storage for download of perf data (lpar2rrd on Virtual Appliance)
- user name: stor2rrd
- expiration: 99999
- CLI access allowed
- roles: opadmin (operator)
EMC Celerra note: use fullnas (nasadmin) role.

Create ssh-keys on STOR2RRD host under stor2rrd user if do not exist yet.
It should already exist on the Virtual Appliance, skip it there.
Type enter instead of passphrase

# su - stor2rrd # (use lpar2rrd user on the Appliance)
$ ls -l ~/.ssh/id_rsa.pub
$ ssh-keygen -t rsa -N ""
  Generating public/private rsa key pair.
  Enter file in which to save the key (/home/stor2rrd/.ssh/id_rsa):
  ...
$ ls -l ~/.ssh/id_rsa.pub
  -rw-r--r-- 1 stor2rrd stor2rrd 382 Jun  1 12:47 /home/stor2rrd/.ssh/id_rsa.pub

Copy ssh-key into the storage under storage stor2rrd

$ scp ~/.ssh/id_rsa.pub stor2rrd@<storage controller>:/tmp/id_rsa.pub
$ ssh stor2rrd@<storage controller>
  cat /tmp/id_rsa.pub >> ~/.ssh/authorized_keys
  rm /tmp/id_rsa.pub
  exit

test connectivity, you should no be prompted for password:

$ ssh -q stor2rrd@<storage controller> "export NAS_DB=/nas; nas_server -i server_2"
  EMC VNX Control Station Linux release 3.0 (NAS 7.1.79) 
  id      type  acl  slot groupID  state  name 
  1        1    0     2              0    server_2 
  2        4    0     3              0    server_3

STOR2RRD storage configuration

All actions below under stor2rrd user (lpar2rrd on Virtual Appliance)
STOR2RRD v2.60+:
Add storage into configuration from the UI:
Settings icon ➡ Storage ➡ New ➡ Vendor:device ➡ Dell EMC VNX file

STOR2RRD v2.50 and olders:

Configure storage in etc/storage-list.cfg
Uncomment (remove the hash) example line and adjust it:

$ vi /home/stor2rrd/stor2rrd/etc/storage-list.cfg

#
# EMC VNX for file (Celerra)
#
# Storage Alias:VNXF:IP address:USER:ssh_key_file:VOLUME_AGG_DATA_LIM:VOLUME_AGG_IO_LIM:SAMPLE_RATE_MINS
#VNX-file_alias:VNXF:192.168.2.1:stor2rrd:/home/stor2rrd/.ssh/id_rsa:0:0:5
VNX-file_alias01:VNXF:192.168.2.1:stor2rrd

Assure you have enough of disk space on the filesystem where is STOR2RRD installed
Roughly you might count 2 - 30 GB per a storage (it depends on number of volumes, 30GB for 5000 volumes)
```
$ df -g /home   # AIX
$ df -h /home   # Linux
```

Test the storage connectivity:

$ cd /home/stor2rrd/stor2rrd
$ ./bin/config_check.sh
  ==================================
  STORAGE: VNX-file_alias01e01: VNXF
  ==================================
  connection ok

Schedule to run storage agent from stor2rrd crontab (lpar2rrd on Virtual Appliance, it might already exist there)

$ crontab -l | grep load_vnxfperf.sh
$

Add if it does not exist as above

$ crontab -e

# EMC VNX for file storage agent
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/stor2rrd/stor2rrd/load_vnxfperf.sh > /home/stor2rrd/stor2rrd/load_vnxfperf.out 2>&1

Assure there is already an entry with the UI creation running once an hour in crontab

$ crontab -e

# STOR2RRD UI (just ONE entry of load.sh must be there)
5 * * * * /home/stor2rrd/stor2rrd/load.sh > /home/stor2rrd/stor2rrd/load.out 2>&1

Let run the storage agent for 15 - 20 minutes to get data, then:
```
$ cd /home/stor2rrd/stor2rrd
$ ./load.sh
```
Go to the web UI: http://<your web server>/stor2rrd/
Use Ctrl-F5 to refresh the web browser cache.

In case of usage of Virtual Appliance

Use local account lpar2rrd for hosting of STOR2RRD on the virtual appliance
Use /home/stor2rrd/stor2rrd as the product home

Note: if you have STOR2RRD 2.52 and lower and Unity firmware 4.5+ then applay this hot fix

Prerequisites

Allow access from the STOR2RRD host to the storage on port 443 (HTTPS).
Test if port is open:

$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 443
  Connection to "192.168.1.1" on port "443" is ok

Unity Unisphere UEM CLI download
https://support.emc.com/downloads
➡ Unity ➡ Unity family ➡ PRODUCT TOOL ➡ more ➡ Unity Unisphere UEM CLI ➡ RedHat Linux 64-bit
Note above location already is not available, we actually do not know where to download the package, ask EMC support, let us know ...
Supported platforms (unfortunately there is no AIX support for UEM CLI as per November 2016 at least)
- Linux RedHat
- SuSE Linux
- Solaris (x86 & Sparc)
- Debian, Ubuntu package:
  Convert RedHat rpm package via "alien" cmd
```
# alien UnisphereCLI-Linux-64-x86-en_US-4.0.1.1.3166-1.x86_64.rpm
  UnisphereCLI-Linux-64-x86-en_US-4.0.1.1.3166-1.i386.deb generated
```

Unisphere API installation

# umask 022
# rpm -i UnisphereCLI-Linux-64-x86-en_US-4.0.1.1.3166-1.x86_64.rpm

Setting security level under root user:

# /opt/emc/uemcli/bin/setlevel.sh low
  Setting low verifying level.....

You can even use "medium" level, then you have to accept and store certificate at the first connection.

under root:

# /opt/emc/uemcli/bin/setlevel.sh medium
  Setting medium verifying level.....

under stor2rrd user (lpar2rrd on the Virtual appliance):

# su - stor2rrd  
$ uemcli -d 192.168.1.1 -u Local/admin -p "XXXXXX" /user/account show
  Remote certificate:
  Issuer:     CN=EMC-Unity
  Subject:    CN=EMC-Unity
  Valid from: 2016-07-15 09:54:57 Z
  Valid to:   2019-07-15 09:54:57 Z
  Serial:     0094aea927526b466e
  Id:         2595584890
  Would you like to:
    [1] Accept the certificate for this session
    [2] Reject the certificate
    [3] Accept and store
  Please input your selection (The default selection is [1]): 3

  Storage system address: 192.168.1.1
  Storage system port: 443
  HTTPS connection
  1:    ID   = user_admin
        Name = admin
        Role = administrator
        Type = local

create user stor2rrd under operator role on the storage.
You might skip it and use already existing account with minimal rights.
Do not use shell special characters like #!?|$*[]\{}`"'& in the password, use ;:.+-%@ instead.
Cmd returns "Error code: 0x6000068" if you do not pass them.
Do not use directly /opt/emc/uemcli/bin/uemcli, use always /usr/bin/uemcli.

$ uemcli -d 192.168.1.1 -u Local/admin -p "XXXXXX" /user/account show
  Storage system address: 192.168.1.1
  Storage system port: 443
  HTTPS connection
  1:    ID   = user_admin
        Name = admin
        Role = administrator
        Type = local
...

$ uemcli -d 192.168.1.1 -u Local/admin -p "XXXXXX" /user/account create -name stor2rrd -role operator -type local -passwd "Test2test_"
  Storage system address: 192.168.1.1
  Storage system port: 443
  HTTPS connection

  ID = user_stor2rrd
  Operation completed successfully.

Test storage connection for stor2rrd account:

$ /usr/bin/uemcli -d 192.168.1.1 -u Local/stor2rrd -p "XXXXXX" /sys/general show -detail
  Storage system address: 192.168.1.1
  Storage system port: 443
  HTTPS connection

  1:    System name            = EMC-Unity
        Model                  = UnityVSA
        Platform type          = EMC Storage System
        System UUID            = 420FFE1D-0234-FD02-976E-9217DC6135ED
        License activation key = ELMEUV0716HN0K
        Product serial number  = VIRT162859DQXZ
        Health state           = OK (5)
        Health details         = "The system is operating normally."

STOR2RRD storage configuration

All actions below under stor2rrd user (lpar2rrd on Virtual Appliance)
STOR2RRD v2.60+:
Add storage into configuration from the UI:
Settings icon ➡ Storage ➡ New ➡ Vendor:device ➡ Dell EMC Unity

STOR2RRD v2.50 and olders:

Configure storage in etc/storage-list.cfg
Uncomment (remove the hash) example line and adjust it:

$ vi /home/stor2rrd/stor2rrd/etc/storage-list.cfg

#
# EMC Unity
#
# Storage Alias:UNITY:IP address or hostname:user:password:VOLUME_AGG_DATA_LIM:VOLUME_AGG_IO_LIM:SAMPLE_RATE_MINS
# to encrypt password use: perl ./bin/spasswd.pl
#EMC-Unity:UNITY:192.168.1.1:stor2rrd:KT4mXVI9N0BUPjZdVQo=:0:0:5
#EMC-Unity:UNITY:192.168.1.1:stor2rrd:KT4mXVI9N0BUPjZdVQo=

EMC-Unity_alias01:UNITY:192.168.1.1:stor2rrd:KT4mXVI9N0BUPjZdVQo=

Above shows the storage with IP 192.168.1.1 which will be visible in the UI as EMC-Unity_alias01

Use encrypted password in storage line above:

$ cd /home/stor2rrd/stor2rrd
$ perl bin/spasswd.pl

  Encode password for storage authentication:
  -------------------------------------------
  Enter password:
  Re-enter password:

  Copy the following string to the password field of the corresponding line in etc/storage-list.cfg:

  KT4mXVI9N0BUPjZdVQo=

Assure you have enough of disk space on the filesystem where is STOR2RRD installed
Roughly you might count 2 - 30 GB per a storage (it depends on number of volumes, 30GB for 5000 volumes)
```
$ df -g /home   # AIX
$ df -h /home   # Linux
```

Test the storage connectivity:

$ cd /home/stor2rrd/stor2rrd
$ ./bin/config_check.sh
  =====================================
  STORAGE: EMC-Unity_alias01: UNITY
  =====================================
  connection ok

Schedule to run storage agent from stor2rrd crontab (lpar2rrd on Virtual Appliance, it might already exist there)

$ crontab -l | grep load_unityperf.sh
$

Add if it does not exist as above

$ crontab -e

# EMC Unity storage agent
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/stor2rrd/stor2rrd/load_unityperf.sh > /home/stor2rrd/stor2rrd/load_unityperf.out 2>&1

Assure there is already an entry with the UI creation running once an hour in crontab

$ crontab -e

# STOR2RRD UI (just ONE entry of load.sh must be there)
5 * * * * /home/stor2rrd/stor2rrd/load.sh > /home/stor2rrd/stor2rrd/load.out 2>&1

Let run the storage agent for 15 - 20 minutes to get data, then:
```
$ cd /home/stor2rrd/stor2rrd
$ ./load.sh
```
Go to the web UI: http://<your web server>/stor2rrd/
Use Ctrl-F5 to refresh the web browser cache.

Troubleshooting

about 50% of performance counters are not working.
You have probably hit into a Unity bug (926416/944550/9013962), upgrade Unity firmware to 4.2.2.x.

In case of usage of Virtual Appliance

Use local account lpar2rrd for hosting of STOR2RRD on the virtual appliance
Use /home/stor2rrd/stor2rrd as the product home

Storage agent uses the Unisphere as data source, therefore only connection to the Unisphere is necessary to get performance statistic from all VMAX boxes attached there.
Unispheres 1.6.x, 8.x and 9.x are supported.
Read why TDEVs are not monitored.

Prerequisites

Allow access from the STOR2RRD host to the Unisphere on port 8443 (TCP).
Test if port is open:

$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 8443
  Connection to "192.168.1.1" on port "8443" is ok

Create user stor2rrd under "PerfMonitor" role on Unisphere.
Allow statistics logging on the Unisphere
UI ➡ Performance ➡ Settings ➡ System Registrations ➡ Diagnostics ➡ On (Diagnostics interval 5 minutes)

Unisphere 1.6

Unisphere 8

STOR2RRD storage configuration

All actions below under stor2rrd user (lpar2rrd on Virtual Appliance)
STOR2RRD v2.60+:
Add storage into configuration from the UI:
Settings icon ➡ Storage ➡ New ➡ Vendor:device ➡ Dell EMC VMAX

STOR2RRD v2.50 and olders:

Configure storage in etc/storage-list.cfg
Uncomment (remove the hash) example line and adjust it.
Fill in Symmetrix ID. You can find it in Unisphere.
Each VMAX has 1 configuration line. (unique storage name alias and unique Symmetrix ID).

$ vi /home/stor2rrd/stor2rrd/etc/storage-list.cfg

#
# EMC VMAX
#
# Storage Alias:VMAX:Symmetrix id:Unisphere IP or hostname:Unisphere port:storage user:encrypted password:VOLUME_AGG_DATA_LIM:VOLUME_AGG_IO_LIM:SAMPLE_RATE_MINS
# to encrypt password use: perl ./bin/spasswd.pl
#EMC-VMAX-01-alias:VMAX:0002957XXXXX:192.168.15.155:8443:stor2rrd:KT4mXVI9N0BUPjZdVQo=

EMC-VMAX-01:VMAX:0002957XXXXX:192.168.1.2:8443:stor2rrd:KT4mXVI9N0BUPjZdVQo=

Above shows the storage with IP 192.168.1.1 which will be visible in the UI as EMC-VMAX-01

Use encrypted password in storage line above:

$ cd /home/stor2rrd/stor2rrd
$ perl bin/spasswd.pl

  Encode password for storage authentication:
  -------------------------------------------
  Enter password:
  Re-enter password:

  Copy the following string to the password field of the corresponding line in etc/storage-list.cfg:

  KT4mXVI9N0BUPjZdVQo=

Assure you have enough of disk space on the filesystem where is STOR2RRD installed
Roughly you might count 2 - 30 GB per a storage (it depends on number of volumes, 30GB for 5000 volumes)
```
$ df -g /home   # AIX
$ df -h /home   # Linux
```
Test the storage connectivity:
```
$ cd /home/stor2rrd/stor2rrd
$ ./bin/config_check.sh VMAX_alias01
  ===============================
  STORAGE: VMAX_alias01: VMAX
  ===============================
  connection ok
```
Possible errors and solutions:
- Error "Missing LWP::Protocol::https", "Connection reset by peer", "500 SSL negotiation failed": follow this.
- Error "Array keys not found!": attach this VMAX through its local Unisphere

Schedule to run storage agent from stor2rrd crontab (lpar2rrd on Virtual Appliance, it might already exist there)

$ crontab -l | grep load_vmaxperf.sh
$

Add if it does not exist as above

$ crontab -e

# VMAX storage agent
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/stor2rrd/stor2rrd/load_vmaxperf.sh > /home/stor2rrd/stor2rrd/load_vmaxperf.out 2>&1

Assure there is already an entry with the UI creation running once an hour in crontab

$ crontab -e

# STOR2RRD UI (just ONE entry of load.sh must be there)
5 * * * * /home/stor2rrd/stor2rrd/load.sh > /home/stor2rrd/stor2rrd/load.out 2>&1

Let run the storage agent for 15 - 20 minutes to get data, then:
```
$ cd /home/stor2rrd/stor2rrd
$ ./load.sh
```
Go to the web UI: http://<your web server>/stor2rrd/
Use Ctrl-F5 to refresh the web browser cache.

In case of usage of Virtual Appliance

Use local account lpar2rrd for hosting of STOR2RRD on the virtual appliance
Use /home/stor2rrd/stor2rrd as the product home

STOR2RRD storage agent uses these APIs:

Performance data via ssh: default VPLEX data collectors available through ssh (port 22) : read only access
Configuration data via REST API: VPLEX Element Manager API (port 443)

Storage access options

VPLEX 5 full admin access
VPLEX 5 non storage UI neither VPLEXcli access
This kind of access requires only ssh access to VPLEX management host to get files with perf data.
This will miss in data then:
- Host section
- Host mapping (HOST vrs Volume relationship)
- Host TOP
- Health status
- Capacity
VPLEX 6 access under vplexuser role

Prerequisites

Allow access from the STOR2RRD host to the VPLEX on ports 22 & 443 (TCP).

$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 22
  Connection to "192.168.1.1" on port "22" is ok
$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 443
  Connection to "192.168.1.1" on port "443" is ok

Create user stor2rrd on the VPLEX host under "users" group and exchange ssh-keys.
This will allow download perfromance data and some basic configuration.
Use vplexuser role on VPLEX 6.

Create ssh keys if it does not exist yet in /home/stor2rrd/.ssh/id_rsa (type enter instead of passphrase)

$ ssh-keygen -t rsa -N ""
  Generating public/private rsa key pair.
  Enter file in which to save the key (/home/stor2rrd/.ssh/id_rsa):
  ...
$ cat ~/.ssh/id_rsa.pub
AF5afPdciUTJ0PYzB6msRxFrCuDSqDwPshq.....

Create stor2rrd user via VPlexcli under admin priviliges

# ssh service@VPLEX01

service@VPLEX01:~> vplexcli 
    Trying ::1... 
    Connected to localhost. 
    Escape character is '^]'. 
    Enter User Name: admin 
    Password:

VPLEX5

VPlexcli:/> user add -u stor2rrd
  admin password: 
  New password: 
  Confirm password:

VPLEX6
User creation with role assigment

VPlexcli:/> user add -r vplexuser -u stor2rrd
  Enter admin password:
  Enter password for stor2rrd:
  Confirm password for stor2rrd:

Allow shell access

VPlexcli:/> cd /management-server/users/local/stor2rrd/
VPlexcli:/management-server/users/local/stor2rrd> set shell-access true
  Enter admin password:

user rights

VPlexcli:/> ll /management-server/users/local/stor2rrd/
  Name          Value
  ------------  --------
  role-name     vplexuser
  shell-access  true
  user-name     stor2rrd

Exchange SSH keys

admin@VPLEX01:~> su - stor2rrd
  Password:
  Password change requested. Choose a new password.
  Old Password:
  New Password:
  Reenter New Password:
stor2rrd@VPLEX01:~> pwd
  /home/stor2rrd
stor2rrd@VPLEX01:~> mkdir .ssh 
stor2rrd@VPLEX01:~> chmod 700 .ssh
  echo "ssh-rsa ...." >> .ssh/authorized_keys
  chmod 600 .ssh/authorized_keys
stor2rrd@VPLEX01:~> grep stor2rrd /etc/passwd
  stor2rrd:x:1003:100::/home/stor2rrd:/bin/bash 
stor2rrd@VPLEX01:~> egrep "^user|stor2rrd" /etc/group|grep 100 
  users:x:100: 
  groupAllUsr:!:1002:service,admin,stor2rrd

User stor2rrd must be created on all VPLEX systems (clusters).

Test connection, you should login without typing of password:
```
$ ssh vplex01 -l stor2rrd
stor2rrd@VPLEX01:~> exit
```
You have to end up in shell! If you end up in VPLEX CLI then it is wrnong, use above "set shell-access true" cmd.
UI access to https://vplex01 as stor2rrd does not work
Allow access to the storage UI (via REST API) under admin priviliges on VPLEX 5
If you do want to allow it then it will work but without this data:
- Host section
- Host mapping (HOST vrs Volume relationship)
- Host TOP
- Health status
- Capacity
VPLEX 6 apparently starts VPLEXcli automatically after log on.
This must be disabled to allow login only to operating system.
There is a way (perhaps commenting out starting VPLEXcli in .bash_profile or something like that), not sure here as for now.

Assure that all monitors we use are running (collecting data), they are all default monitors and should run:

VPlexcli:/> ll /monitoring/directors/*/monitors/
  /monitoring/directors/director-1-1-A/monitors:
  Name                                                            Ownership  Collecting  Period  Average  Idle   Bucket  Bucket  Bucket  Bucket
  --------------------------------------------------------------  ---------  Data        ------  Period   For    Min     Max     Width   Count
  --------------------------------------------------------------  ---------  ----------  ------  -------  -----  ------  ------  ------  ------
  director-1-1-A_PERPETUAL_vplex_sys_perf_mon_v19                 true       true        5s      4.97s    2s     -       -       -       64
  director-1-1-A_VIRTUAL_VOLUMES_PERPETUAL_MONITOR                true       true        1min    1min     33s    -       -       -       64

  /monitoring/directors/director-1-1-B/monitors:
  Name                                              Ownership  Collecting  Period  Average  Idle  Bucket  Bucket  Bucket  Bucket
  ------------------------------------------------  ---------  Data        ------  Period   For   Min     Max     Width   Count
  ------------------------------------------------  ---------  ----------  ------  -------  ----  ------  ------  ------  ------
  director-1-1-B_PERPETUAL_vplex_sys_perf_mon_v19   true       true        5s      5.05s    4s    -       -       -       64
  director-1-1-B_VIRTUAL_VOLUMES_PERPETUAL_MONITOR  true       true        1min    60s      33s   -       -       -       64
  ...

If some do not running then use this procedure to enable it.

STOR2RRD storage configuration

All actions below under stor2rrd user (lpar2rrd on Virtual Appliance)
STOR2RRD v2.60+:
Add storage into configuration from the UI:
Settings icon ➡ Storage ➡ New ➡ Vendor:device ➡ Dell EMC VPLEX

STOR2RRD v2.50 and olders:

Configure storage in etc/storage-list.cfg
Uncomment (remove the hash) example line and adjust it.
Fill in VPLEX cluster name. You can find it in the UI.
Each VPLEX cluster has 1 configuration line. (unique storage name alias and unique VPLEX cluster name).

$ vi /home/stor2rrd/stor2rrd/etc/storage-list.cfg

#
# EMC VPLEX
#
# Storage Alias:VPLEX:Cluster name:Mgmt server IP or hostname:Mgmt server ssh port:Mgmt server user:use API [1|0] ([yes|no]):Unisphere/API IP or hostname:API port:API user name:API encrypted password:VOLUME_AGG_DATA_LIM:VOLUME_AGG_IO_LIM:SAMPLE_RATE_MINS
# to encrypt password use: perl ./bin/spasswd.pl
# If you do not use VPLEX API access then you do not need to fill in API fields, leave them empty 
#EMC-VPLEX_alias01:VPLEX:VPLEX-cluster-name:vplex_mgmt.example.com:22:stor2rrd:0:vplex_api.example.com:443:stor2rrd:KT4mXVI9N0BUPjZdVQo=:1024:10:
#EMC-VPLEX_alias01:VPLEX:VPLEX-cluster-name:vplex_mgmt.example.com:22:stor2rrd:0:vplex_api.example.com:443:stor2rrd:KT4mXVI9N0BUPjZdVQo=

EMC-VPLEX-01:VPLEX:VPLEX-cluster01:192.168.1.1:22:stor2rrd:0:::
EMC-VPLEX-02:VPLEX:VPLEX-cluster01:192.168.1.2:22:stor2rrd:1:192.168.1.2:443:stor2rrd:KT4mXVI9N0BUPjZdVQo=

Above shows two VPLEXes

one with IP 192.168.1.1 which will be visible in the UI as EMC-VPLEX-01
It does not have VPLEX API access configured
one with IP 192.168.1.2 which will be visible in the UI as EMC-VPLEX-02
It has VPLEX API access configured. HOST, HW event log and mapping will work there, but admin role

Use encrypted password in storage line above:

$ cd /home/stor2rrd/stor2rrd
$ perl bin/spasswd.pl

  Encode password for storage authentication:
  -------------------------------------------
  Enter password:
  Re-enter password:

  Copy the following string to the password field of the corresponding line in etc/storage-list.cfg:

  KT4mXVI9N0BUPjZdVQo=

Assure you have enough of disk space on the filesystem where is STOR2RRD installed
Roughly you might count 2 - 30 GB per a storage (it depends on number of volumes, 30GB for 5000 volumes)
```
$ df -g /home   # AIX
$ df -h /home   # Linux
```

Test the storage connectivity:

$ cd /home/stor2rrd/stor2rrd
$ ./bin/config_check.sh EMC-VPLEX-01
  ===============================
  STORAGE: EMC-VPLEX-01: VPLEX
  ===============================
  connection ok

If you get an error about missing LWP::Protocol::https then install as per this manual

Schedule to run storage agent from stor2rrd crontab (lpar2rrd on Virtual Appliance, it might already exist there)

$ crontab -l | grep load_vplexperf.sh
$

Add if it does not exist as above

$ crontab -e

# VPLEX storage agent
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/stor2rrd/stor2rrd/load_vplexperf.sh > /home/stor2rrd/stor2rrd/load_vplexperf.out 2>&1

Assure there is already an entry with the UI creation running once an hour in crontab

$ crontab -e

# STOR2RRD UI (just ONE entry of load.sh must be there)
5 * * * * /home/stor2rrd/stor2rrd/load.sh > /home/stor2rrd/stor2rrd/load.out 2>&1

Let run the storage agent for 15 - 20 minutes to get data, then:
```
$ cd /home/stor2rrd/stor2rrd
$ ./load.sh
```
Go to the web UI: http://<your web server>/stor2rrd/
Use Ctrl-F5 to refresh the web browser cache.

In case of usage of Virtual Appliance

Use local account lpar2rrd for hosting of STOR2RRD on the virtual appliance
Use /home/stor2rrd/stor2rrd as the product home

VNXe support

STOR2RRD supports these SAN/NAS VNXe storage:

VNXe 1600
VNXe 3200

STOR2RRD does not support VNXe series1 models which do not have block implementation:

VNXe 3100
VNXe 3150
VNXe 3300

Such storage do not provide performance data.

Prerequisites

Allow access from the STOR2RRD host to the storage service processors on port 443 (TCP).
Test if port is open:

$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 443
  Connection to "192.168.1.1" on port "443" is ok

VNXe Unisphere UEM CLI download
If you have also EMC Unity in place then use Unity UEM CLI which works also for VNXe.
VNXe UEM CLI does not support Unity.
https://support.emc.com/downloads
VNX/VNXe family ➡ VNXe1/2 series ➡ ADD-ONS ➡ More ➡ VNXe Unisphere CLI ➡ Linux 64-bit
Supported platforms (unfortunatelly there is no AIX support for UEM CLI as per November 2016 at least)
- Linux RedHat
- SuSE Linux
- Solaris (x86 & Sparc)
- Debian, Ubuntu package:
  Convert RedHat rpm package via "alien" cmd
```
# alien UnisphereCLI-Linux-64-x86-en_US-4.0.1.1.3166-1.i386.rpm
  UnisphereCLI-Linux-64-x86-en_US-4.0.1.1.3166-1.i386.deb generated
```

Unisphere API installation

# umask 022
# rpm -i UnisphereCLI-Linux-64-x86-en_US-4.0.1.1.3166-1.x86_64.rpm

Setting security level under root user:

# /opt/emc/uemcli/bin/setlevel.sh low
  Setting low verifying level.....

You can even use "medium" level, then you have to accept and store certificate at the first connection.

under root:

# /opt/emc/uemcli/bin/setlevel.sh medium
  Setting medium verifying level.....

under stor2rrd user:

$ uemcli -d 192.168.1.1 -u Local/admin -p "XXXXXX" /user/account show
  Remote certificate:
  Issuer:     CN=EMC-Unity
  Subject:    CN=EMC-Unity
  Valid from: 2016-07-15 09:54:57 Z
  Valid to:   2019-07-15 09:54:57 Z
  Serial:     0094aea927526b466e
  Id:         2595584890
  Would you like to:
    [1] Accept the certificate for this session
    [2] Reject the certificate
    [3] Accept and store
  Please input your selection (The default selection is [1]): 3

  Storage system address: 192.168.1.1
  Storage system port: 443
  HTTPS connection
  1:    ID   = user_admin
        Name = admin
        Role = administrator
        Type = local

create user stor2rrd under operator role on the storage.
You might skip it and use already existing account with minimal rights.
Do not use shell special characters like #!?|.*[]\{} in the password.
Cmd returns "Error code: 0x6000068" if you do not pass them.
Do not use directly /opt/emc/uemcli/bin/uemcli, use always /usr/bin/uemcli.

$ uemcli -d 192.168.1.1 -u Local/admin -p "XXXXXX" /user/account show
  Storage system address: 192.168.1.1
  Storage system port: 443
  HTTPS connection
  1:    ID   = user_admin
        Name = admin
        Role = administrator
        Type = local
...

$ uemcli -d 192.168.1.1 -u Local/admin -p "XXXXXX" /user/account create -name stor2rrd -role operator -type local -passwd "Test2test_"
  Storage system address: 192.168.1.1
  Storage system port: 443
  HTTPS connection

  ID = user_stor2rrd
  Operation completed successfully.

STOR2RRD storage configuration

All actions below under stor2rrd user (lpar2rrd on Virtual Appliance)
STOR2RRD v2.60+:
Add storage into configuration from the UI:
Settings icon ➡ Storage ➡ New ➡ Vendor:device ➡ Dell EMC VNXe

STOR2RRD v2.50 and olders:

Configure storage in etc/storage-list.cfg
Uncomment (remove the hash) example line and adjust it:

$ vi /home/stor2rrd/stor2rrd/etc/storage-list.cfg

#
# EMC VNXe 
#
# Storage Alias:VNXE:IP address or hostname:user:password:VOLUME_AGG_DATA_LIM:VOLUME_AGG_IO_LIM:SAMPLE_RATE_MINS
# to encrypt password use: perl ./bin/spasswd.pl
#EMC-VNXe:VNXE:192.168.1.1:stor2rrd:KT4mXVI9N0BUPjZdVQo=:0:0:5
#EMC-VNXe:VNXE:192.168.1.1:stor2rrd:KT4mXVI9N0BUPjZdVQo=

EMC-VNXe_alias01:UNITY:192.168.1.1:stor2rrd:KT4mXVI9N0BUPjZdVQo=

Above shows the storage with IP 192.168.1.1 which will be visible in the UI as EMC-VNXe_alias01

Use encrypted password in storage line above:

$ cd /home/stor2rrd/stor2rrd
$ perl bin/spasswd.pl

  Encode password for storage authentication:
  -------------------------------------------
  Enter password:
  Re-enter password:

  Copy the following string to the password field of the corresponding line in etc/storage-list.cfg:

  KT4mXVI9N0BUPjZdVQo=

Assure you have enough of disk space on the filesystem where is STOR2RRD installed
Roughly you might count 2 - 30 GB per a storage (it depends on number of volumes, 30GB for 5000 volumes)
```
$ df -g /home   # AIX
$ df -h /home   # Linux
```

Test the storage connectivity:

$ cd /home/stor2rrd/stor2rrd
$ ./bin/config_check.sh
  =====================================
  STORAGE: EMC-VNXe_alias01: VNXE 
  =====================================
  connection ok

Schedule to run storage agent from stor2rrd crontab (lpar2rrd on Virtual Appliance, it might already exist there)

$ crontab -l | grep load_vnxeperf.sh
$

Add if it does not exist as above

$ crontab -e

# EMC VNXe storage agent
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/stor2rrd/stor2rrd/load_vnxeperf.sh > /home/stor2rrd/stor2rrd/load_vnxeperf.out 2>&1

Assure there is already an entry with the UI creation running once an hour in crontab

$ crontab -e

# STOR2RRD UI (just ONE entry of load.sh must be there)
5 * * * * /home/stor2rrd/stor2rrd/load.sh > /home/stor2rrd/stor2rrd/load.out 2>&1

Let run the storage agent for 15 - 20 minutes to get data, then:
```
$ cd /home/stor2rrd/stor2rrd
$ ./load.sh
```
Go to the web UI: http://<your web server>/stor2rrd/
Use Ctrl-F5 to refresh the web browser cache.

In case of usage of Virtual Appliance

Use local account lpar2rrd for hosting of STOR2RRD on the virtual appliance
Use /home/stor2rrd/stor2rrd as the product home

Storage agent uses Data Domain CLI accessed via ssh to get all necesary data from the storage.
DataDomain firmware versions 5.5+ and 6.x are supported.
DataDomain firmware versions 7.x+ is supported since STOR2RRD v2.70.
DataDomain 6.0.2.50+ and 6.1.2.40+ (6.2 is not affected) do not provide disk stats community.emc.com/docs/DOC-78887

Prerequisites

Allow access from the STOR2RRD host to the storage on port 22 (TCP).
Test if port is open:

$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 22
  Connection to "192.168.1.1" on port "22" is ok

create user stor2rrd under user role on the storage.

Exchange ssh keys
Print public key:

$ cat .ssh/id_rsa.pub
  ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPpQlm0Z/x1X4txd2GgZgvg1Fm7QHeusQ6LqBp/Sb1doAoiDmXErk+lNY27aJAqb4YiGnqejioqCJ9N93NiKpffxH+L3

Log on to the storage and add above key:

$ ssh <data domain host> -l stor2rrd
  stor2rrd@data_domain_host> adminaccess add ssh-keys
    Enter the key and then press Control-D, or press Control-C to cancel.
    ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPpQlm0Z/x1X4txd2GgZgvg1Fm7QHeusQ6LqBp/Sb1doAoiDmXErk+lNY27aJAqb4YiGnqejioqCJ9N93NiKpffxH+L3

Test connection:

$ ssh <data domain host> -l stor2rrd
  EMC Data Domain Virtual Edition
  Welcome to Data Domain OS 6.1.0.11-571028

STOR2RRD storage configuration

All actions below under stor2rrd user (lpar2rrd on Virtual Appliance)
STOR2RRD v2.60+:
Add storage into configuration from the UI:
Settings icon ➡ Storage ➡ New ➡ Vendor:device ➡ Dell EMC DataDomain

STOR2RRD v2.50 and olders:

Configure storage in etc/storage-list.cfg
Uncomment (remove the hash) example line and adjust it:

$ vi /home/stor2rrd/stor2rrd/etc/storage-list.cfg

#
# EMC DataDomain
#
# Storage Alias:VNXE:IP address or hostname:user:VOLUME_AGG_DATA_LIM:VOLUME_AGG_IO_LIM:SAMPLE_RATE_MINS
#EMC-DataDomain01:DATADOMAIN:dd_host.example.com:stor2rrd:1024:10:

EMC-DataDomain01_alias01:DATADOMAIN:192.168.1.1:stor2rrd

Above shows the storage with IP 192.168.1.1 which will be visible in the UI as EMC-DataDomain01_alias01

Assure you have enough of disk space on the filesystem where is STOR2RRD installed
Roughly you might count 2 - 30 GB per a storage (it depends on number of volumes, 30GB for 5000 volumes)
```
$ df -g /home   # AIX
$ df -h /home   # Linux
```

Test the storage connectivity:

$ cd /home/stor2rrd/stor2rrd
$ ./bin/config_check.sh EMC-DataDomain01_alias01
  =====================================
  STORAGE: EMC-DataDomain01_alias01: DATADOMAIN
  =====================================
  connection ok

Schedule to run storage agent from stor2rrd crontab (lpar2rrd on Virtual Appliance, it might already exist there)

$ crontab -l | grep load_datadomainperf.sh 
$

Add if it does not exist as above

$ crontab -e

# EMC DataDomain storage agent
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/stor2rrd/stor2rrd/load_datadomainperf.sh > /home/stor2rrd/stor2rrd/load_datadomainperf.out 2>&1

Assure there is already an entry with the UI creation running once an hour in crontab

$ crontab -e

# STOR2RRD UI (just ONE entry of load.sh must be there)
5 * * * * /home/stor2rrd/stor2rrd/load.sh > /home/stor2rrd/stor2rrd/load.out 2>&1

Let run the storage agent for 15 - 20 minutes to get data, then:
```
$ cd /home/stor2rrd/stor2rrd
$ ./load.sh
```
Go to the web UI: http://<your web server>/stor2rrd/
Use Ctrl-F5 to refresh the web browser cache.

In case of usage of Virtual Appliance

Use local account lpar2rrd for hosting of STOR2RRD on the virtual appliance
Use /home/stor2rrd/stor2rrd as the product home

Isilon v7.x and v8.x firmwares are supported.
The product uses REST API interface natively provided by the storage to get all necessary data.
Support of Isilon 8.1.0.3+ firmware where was a major change in storage REST API is in STOR2RRD 2.30+.

Storage connectivity

Allow access from the STOR2RRD host to the storage on port 8080.

$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 8080
  Connection to "192.168.1.1" on port "8080" is ok

Create user stor2rrd on the storage with new read-only role called stor2rrd.

$ ssh -l admin 192.168.1.1
isilon01-1% isi auth users create stor2rrd --enabled yes --password-expires no --password XXXXXXX
isilon01-1% isi auth roles create --name stor2rrd --description "Read-only role for STOR2RRD"
isilon01-1% isi auth roles modify stor2rrd --add-user=stor2rrd --add-priv-ro=ISI_PRIV_EVENT --add-priv-ro=ISI_PRIV_LOGIN_PAPI --add-priv-ro=ISI_PRIV_STATISTICS --add-priv-ro=ISI_PRIV_SMARTPOOLS --add-priv-ro=ISI_PRIV_DEVICES

STOR2RRD storage configuration

All actions below under stor2rrd user (lpar2rrd on Virtual Appliance)
STOR2RRD v2.60+:
Add storage into configuration from the UI:
Settings icon ➡ Storage ➡ New ➡ Vendor:device ➡ Dell EMC Isilon

STOR2RRD v2.50 and olders:

Uncomment (remove the hash) example line and adjust it:

$ vi /home/stor2rrd/stor2rrd/etc/storage-list.cfg

#
# Dell EMC Isilon
#
# Storage Alias:ISILON:IP address or hostname:storage user:REST API port:encrypted password:VOLUME_AGG_DATA_LIM:VOLUME_AGG_IO_LIM:SAMPLE_RATE_MINS
# to encrypt password use: perl ./bin/spasswd.pl
#isilon-alias01:ISILON:isilon_host.example.com:8080:stor2rrd:KT4mXVI9N0BUPjZdVQo:1024:10

isilon-alias01:ISILON:192.168.1.1:8080:stor2rrd:KT4mXVI9N0BUPjZdVQo

Above shows the storage with IP 192.168.1.1 which will be visible in the UI as isilon-alias01

Use encrypted password in storage line above, encrypt it like this:

$ cd /home/stor2rrd/stor2rrd
$ perl bin/spasswd.pl 

  Encode password for storage authentication:
  -------------------------------------------
  Enter password: 
  Re-enter password: 

  Copy the following string to the password field of the corresponding line in etc/storage-list.cfg:

  KT4mXVI9N0BUPjZdVQo=

Assure you have enough of disk space on the filesystem where is STOR2RRD installed
Roughly you might count 2 - 30 GB per a storage (it depends on number of volumes, 30GB for 5000 volumes)
```
$ df -g /home   # AIX
$ df -h /home   # Linux
```

Check storage connectivity

$ cd /home/stor2rrd/stor2rrd
$ ./bin/config_check.sh isilon-alias01
  =====================
  STORAGE: isilon-alias01 : ISILON
  =====================
  connection ok

If you get an errors like "Missing LWP::Protocol::https", "Connection reset by peer" or "500 SSL negotiation failed" then follow this.

Schedule to run storage agent from stor2rrd crontab (lpar2rrd on Virtual Appliance, it might already exist there)

$ crontab -l | grep load_isilonperf.sh
$

Add if it does not exist as above

$ crontab -e

# Dell EMC Isilon
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/stor2rrd/stor2rrd/load_isilonperf.sh > /home/stor2rrd/stor2rrd/load_isilonperf.out 2>&1

Assure there is already an entry with the UI creation running once an hour in crontab

$ crontab -e

# STOR2RRD UI (just ONE entry of load.sh must be there)
5 * * * * /home/stor2rrd/stor2rrd/load.sh > /home/stor2rrd/stor2rrd/load.out 2>&1

Let run the storage agent for 15 - 20 minutes to get data, then:
```
$ cd /home/stor2rrd/stor2rrd
$ ./load.sh
```
Go to the web UI: http://<your web server>/stor2rrd/
Use Ctrl-F5 to refresh the web browser cache.

In case of usage of Virtual Appliance

Use local account lpar2rrd for hosting of STOR2RRD on the virtual appliance
Use /home/stor2rrd/stor2rrd as the product home

The product uses REST API natively provided by XtremIO XMS management module to get all necessary data.

Storage connectivity

Allow access from the STOR2RRD host to the storage on port 443 (https)

$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 443
  Connection to "192.168.1.1" on port "443" is ok

Create user stor2rrd in storage XMS with role Read-Only

STOR2RRD storage configuration

All actions below under stor2rrd user (lpar2rrd on Virtual Appliance)
STOR2RRD v2.60+:
Add storage into configuration from the UI:
Settings icon ➡ Storage ➡ New ➡ Vendor:device ➡ XtremIO

STOR2RRD v2.50 and olders:

Uncomment (remove the hash) example line and adjust it:

$ vi /home/stor2rrd/stor2rrd/etc/storage-list.cfg

#
# Dell EMC XtremIO
#
# Storage Alias:XTREMIO:Cluster name:XMS IP or hostname:XMS UI/API port:XMS user:encrypted XMS user password:VOLUME_AGG_DATA_LIM:VOLUME_AGG_IO_LIM:SAMPLE_RATE_MINS
# to encrypt password use: perl ./bin/spasswd.pl
#xtremio-alias01:XTREMIO:Cluster01:XMS_host.example.com:443:stor2rrd:KT4mXVI9N0BUPjZdVQo=:1024:10:
#xtremio-alias01:XTREMIO:Cluster01:XMS_host.example.com:443:stor2rrd:KT4mXVI9N0BUPjZdVQo=

xtremio-XIO-01-alias01:XTREMIO:XIO-01:192.168.1.1:443:stor2rrd:KT4mXVI9N0BUPjZdVQo=
xtremio-XIO-02-alias01:XTREMIO:XIO-02:192.168.1.1:443:stor2rrd:KT4mXVI9N0BUPjZdVQo=

You have to define each monitored cluster separately (like above clusters XIO-01 and XIO-02 on single XMS).
Above shows clusters XIO-01 and XIO-02 on XMS with IP 192.168.1.1 which will be visible in the UI as xtremio-XIO-01-alias01 and xtremio-XIO-02-alias01

Use encrypted password in storage line above, encrypt it like this:

$ cd /home/stor2rrd/stor2rrd
$ perl bin/spasswd.pl 

  Encode password for storage authentication:
  -------------------------------------------
  Enter password: 
  Re-enter password: 

  Copy the following string to the password field of the corresponding line in etc/storage-list.cfg:

  KT4mXVI9N0BUPjZdVQo=

Assure you have enough of disk space on the filesystem where is STOR2RRD installed
Roughly you might count 2 - 30 GB per a storage (it depends on number of volumes, 30GB for 5000 volumes)
```
$ df -g /home   # AIX
$ df -h /home   # Linux
```

Check storage connectivity

$ cd /home/stor2rrd/stor2rrd
$ ./bin/config_check.sh xtremio-XIO-01-alias01
  =====================
  STORAGE: xtremio-XIO-01-alias01 : XTREMIO
  =====================
  connection ok

If you get an errors like "Missing LWP::Protocol::https", "Connection reset by peer" or "500 SSL negotiation failed" then follow this.

Schedule to run storage agent from stor2rrd crontab (lpar2rrd on Virtual Appliance, it might already exist there)

$ crontab -l | grep load_xtremioperf.sh
$

Add if it does not exist as above

$ crontab -e

# Dell EMC XtremIO
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/stor2rrd/stor2rrd/load_xtremioperf.sh > /home/stor2rrd/stor2rrd/load_xtremioperf.out 2>&1

Assure there is already an entry with the UI creation running once an hour in crontab

$ crontab -e

# STOR2RRD UI (just ONE entry of load.sh must be there)
5 * * * * /home/stor2rrd/stor2rrd/load.sh > /home/stor2rrd/stor2rrd/load.out 2>&1

Let run the storage agent for 15 - 20 minutes to get data, then:
```
$ cd /home/stor2rrd/stor2rrd
$ ./load.sh
```
Go to the web UI: http://<your web server>/stor2rrd/
Use Ctrl-F5 to refresh the web browser cache.

Troubleshooting

No performance data in STOR2RRD UI:
Check storage XMS console if it has performance data in its graphs.
if not then contact Dell EMC support.
As soon as data appears in XMS console then STOR2RRD starts working as well

In case of usage of Virtual Appliance

Use local account lpar2rrd for hosting of STOR2RRD on the virtual appliance
Use /home/stor2rrd/stor2rrd as the product home

The product uses SMI-S and XML API interfaces natively provided by the storage.

Storage connectivity

Allow access from the STOR2RRD host to the storage on ports 80 (HTTP) and 5988 (SMI-S).
If you use storage cmd line configuration like in examples below (you can use the storage UI instead) then port 22 (ssh) must be open.
If you use https and secure SMI-S then test ports 443 and 5989 instead below ones.
```
$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 80
  Connection to "192.168.1.1" on port "80" is ok
$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 5988
  Connection to "192.168.1.1" on port "5988" is ok
```

Assure that SMI-S 5988 and HTTP protocols are enabled on the storage

$ ssh -l manage <storage_hostname>
# show protocols                     
  Service and Security Protocols
  ------------------------------
  Web Browser Interface (HTTP): Enabled 
  Secure Web Browser Interface (HTTPS): Enabled
  Command Line Interface (Telnet): Disabled
  Secure Command Line Interface (SSH): Enabled
  Storage Management Initiative Specification (SMI-S): Disabled
  Unsecure Storage Management Initiative Specification (SMI-S 5988): Enabled
  ...

If SMI-S 5988 or HTTP is not enabled:

# set protocols usmis enabled
# set protocols http enabled

Create user stor2rrd on the storage with monitor role and allow SMI-S and XML API.
Adjust password to your prefered one from used Test1234.
```
$ ssh -l manage <storage_hostname>
# create user interfaces smis,wbi locale English password Test1234 roles monitor stor2rrd
```
Note newer firmwares might require role "standard" for WBI access.
In case you want to use SSL access to the storage
enable ports 5989 and 443 and start up SSL services on the storage instead of non SSL ones
```
# set protocols smis enabled
# set protocols https enabled
```

Install prerequisites

Skip that in case of Virtual Appliance

Linux:
Install sblim-wbemcli package:

# yum list |grep sblim-wbemcli
  sblim-wbemcli.x86_64                       1.6.2-9.el7                 base
# yum install sblim-wbemcli     # RedHat
# apt-get install sblim-wbemcli # Debian based (Ubuntu etc)
# /usr/bin/wbemcli -noverify -nl ei https://<storage_hostname>:5988/root/dsh:cim_computersystem
  *
  * ./wbemcli: Http Exception: Username/password required.
  *

If you prefere to use https on port 5989 then you must use wbemcli with this patch 0002742
Download the package from:

wbemcli-1.6.3 x86_64 for Redhat 6
wbemcli-1.6.3 x86_64 for Redhat 7
wbemcli-1.6.3 for other Linuxes: ➡ Select your OS in the header ➡ "Grab binary packages directly"
wbemcli_x86_64.gz as general Linux binary, gunzip and replace /usr/bin/wbemcli by this

Test the package:

# rpm -q sblim-wbemcli
  sblim-wbemcli-1.6.3-31.1.x86_64
# /usr/bin/wbemcli -v
  * wbemcli Version 1.6.3
# /usr/bin/wbemcli -noverify -nl ei http://<storage_hostname>:5988/root/dsh:cim_computersystem
  *
  * ./wbemcli: Http Exception: Username/password required.
  *

AIX:
Download and install

# rpm -Uvh libffi-3.1-1.aix5.1.ppc.rpm --nodeps
# rpm -Uvh libiconv-1.14-2.aix5.1.ppc.rpm --nodeps
# rpm -Uvh libidn-1.29-1.aix5.1.ppc.rpm --nodeps
# rpm -Uvh libssh2-1.4.3-2.aix5.1.ppc.rpm --nodeps
# rpm -Uvh openssl-1.0.1j-1.aix5.1.ppc.rpm
# rpm -Uvh openldap-2.4.23-0.3.aix5.1.ppc.rpm
# rpm -Uvh curl-7.27.0-1.aix5.1.ppc.rpm
# rpm -Uvh wbemcli-1.6.3-1.aix5.1.ppc.rpm

# LIBPATH=/opt/freeware/lib; export LIBPATH
# /opt/freeware/bin/wbemcli -noverify -nl ei http://<storage_hostname>:5988/root/dsh:cim_computersystem
  *
  * ./wbemcli: Http Exception: Username/password required.
  *

STOR2RRD storage configuration

All actions below under stor2rrd user (lpar2rrd on Virtual Appliance)
STOR2RRD v2.60+:
Add storage into configuration from the UI:
Settings icon ➡ Storage ➡ New ➡ Vendor:device ➡ Dell EMC ME4

STOR2RRD v2.50 and olders:

Uncomment (remove the hash) example line and adjust it:

$ vi /home/stor2rrd/stor2rrd/etc/storage-list.cfg

#
# Dot Hill AssuredSAN and its rebrands like Lenovo S2200 or HPE MSA2000/P2000, Dell EMC PowerVault ME4 Series, Quantum StorNex
#
# Storage Alias:DOTHILL:Controller_A IP address or hostname:Controller_B IP address or hostname:[http|https]:storage user:encrypted password:VOLUME_AGG_DATA_LIM:VOLUME_AGG_IO_LIM:SAMPLE_RATE_MINS
# to encrypt password use: perl ./bin/spasswd.pl
#DOTHILL_AssuredSAN-alias01:DOTHILL:ctla_host.example.com:ctlb_host.example.com:http:stor2rrd:KT4mXVI9N0BUPjZdVQo=:1024:10:
#LENOVO_S2000-alias01:DOTHILL:ctla_host.example.com:ctlb_host.example.com:http:stor2rrd:KT4mXVI9N0BUPjZdVQo=
#HPE_MSA2000-alias01:DOTHILL:ctla_host.example.com:ctlb_host.example.com:http:stor2rrd:KT4mXVI9N0BUPjZdVQo=

EMC-ME4-alias01:DOTHILL:192.168.1.1:192.168.1.2:http:stor2rrd:KT4mXVI9N0BUPjZdVQo=

Above shows the storage with IP 192.168.1.1 which will be visible in the UI as EMC-ME4-alias01

Use encrypted password in storage line above, encrypt it like this:
Do not use shell special characters like @#!?|$*[]\{}`"'& in the password, use rather ;:.+- instead.

$ cd /home/stor2rrd/stor2rrd
$ perl bin/spasswd.pl 

  Encode password for storage authentication:
  -------------------------------------------
  Enter password: 
  Re-enter password: 

  Copy the following string to the password field of the corresponding line in etc/storage-list.cfg:

  KT4mXVI9N0BUPjZdVQo=

Assure you have enough of disk space on the filesystem where is STOR2RRD installed
Roughly you might count 2 - 30 GB per a storage (it depends on number of volumes, 30GB for 5000 volumes)
```
$ df -g /home   # AIX
$ df -h /home   # Linux
```
Check storage connectivity
```
$ cd /home/stor2rrd/stor2rrd
$ ./bin/config_check.sh 
  =====================
  STORAGE: EMC-ME4-alias01 : DOTHILL
  =====================
  connection ok
```
If you use HTTPS and do not have installed Perl SSL support then you get an error:
"Missing LWP::Protocol::https", "Connection reset by peer" or "500 SSL negotiation failed"
Follow this or switch to HTTP on the storage line in etc/storage-list.cfg.

Schedule to run storage agent from stor2rrd crontab (lpar2rrd on Virtual Appliance, it might already exist there)

$ crontab -l | grep load_dothillperf.sh
$

Add if it does not exist as above

$ crontab -e

# DotHill AssuredSAN and its rebrands like Lenovo S2200 or HPE MSA2000/P2000, Dell EMC PowerVault ME4 Series, Quantum StorNex
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/stor2rrd/stor2rrd/load_dothillperf.sh > /home/stor2rrd/stor2rrd/load_dothillperf.out 2>&1

Assure there is already an entry with the UI creation running once an hour in crontab

$ crontab -e

# STOR2RRD UI (just ONE entry of load.sh must be there)
5 * * * * /home/stor2rrd/stor2rrd/load.sh > /home/stor2rrd/stor2rrd/load.out 2>&1

Let run the storage agent for 15 - 20 minutes to get data, then:
```
$ cd /home/stor2rrd/stor2rrd
$ ./load.sh
```
Go to the web UI: http://<your web server>/stor2rrd/
Use Ctrl-F5 to refresh the web browser cache.

Troubleshooting

Assure SMI-S is working for user stor2rrd. Adjust password and storage_hostname in example below:

$ /usr/bin/wbemcli -nl -t -noverify ein 'http://stor2rrd:<password>@<storage_hostnamew>:5988/root/dsh:SMI_Controller'
  192.168.1.1:5988/root/smis:SMI_Controller.Name="208000c0ff261f53:A",CreationClassName="SMI_Controller"
  192.168.1.2:5988/root/smis:SMI_Controller.Name="208000c0ff261f53:B",CreationClassName="SMI_Controller"

When it does not work then check user rights of stor2rrd user (must have "smis"), network visibility (firewall), user/password combination.

In case of usage of Virtual Appliance

Use local account lpar2rrd for hosting of STOR2RRD on the virtual appliance
Use /home/stor2rrd/stor2rrd as the product home

STOR2RRD uses REST API interface natively provided by the storage to get all necessary data.
All data is downloaded from VDC nodes.

Storage connectivity

Allow access from the STOR2RRD host to each VDC node on port 4443

$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 4443
  Connection to "192.168.1.1" on port "4443" is ok

Create Storage Management user "stor2rrd" with System Monitor role on each monitored VDC.

STOR2RRD storage configuration

Add storage into configuration from the UI:
Settings icon ➡ Storage ➡ New ➡ Vendor:device ➡ Dell EMC : Elastic Cloud Storage
Assure you have enough of disk space on the filesystem where is STOR2RRD installed
Roughly you might count 2 - 30 GB per a storage (it depends on number of volumes, 30GB for 5000 volumes)
```
$ df -g /home   # AIX
$ df -h /home   # Linux
```

Schedule to run storage agent from stor2rrd crontab (lpar2rrd on Virtual Appliance, it might already exist there)

$ crontab -l | grep load_ecsperf.sh
$

Add if it does not exist as above

$ crontab -e

# Dell EMC ECS
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/stor2rrd/stor2rrd/load_ecsperf.sh > /home/stor2rrd/stor2rrd/load_ecsperf.out 2>&1

Assure there is already an entry with the UI creation running once an hour in crontab

$ crontab -e

# STOR2RRD UI (just ONE entry of load.sh must be there)
5 * * * * /home/stor2rrd/stor2rrd/load.sh > /home/stor2rrd/stor2rrd/load.out 2>&1

Let run the storage agent for 15 - 20 minutes to get data, then:
```
$ cd /home/stor2rrd/stor2rrd
$ ./load.sh
```
Go to the web UI: http://<your web server>/stor2rrd/
Use Ctrl-F5 to refresh the web browser cache.