Lenovo monitoring: Storage V/S Series, ThinkSystem DS/DE/DM/DG Series

V Series
S Series
(WBI)
S Series
(SMI-S)
DS Series
(WBI)
DS Series
(SMI-S)
DE Series
DM / DG Series
REST API
DM / DG Series
ONTAPI

In case of usage of Virtual Appliance

Use local account lpar2rrd for hosting of STOR2RRD on the virtual appliance
Use /home/stor2rrd/stor2rrd as the product home

Prerequisites

Allow access from the STOR2RRD host to the IBM SVC / Storwize / FlashSystem storage on port 22 (ssh).
Test if port is open:
```
$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 22
  Connection to "192.168.1.1" on port "22" is ok
```

Storage access

There are 2 possibilities how to connect the storage:

Use user/password
Create new user "stor2rrd" on the storage

Storage firmware 8.3.1.2+: use "Monitor" role

$ ssh superuser@<storage control enclosure IP address> "svctask mkuser -name stor2rrd -usergrp "Monitor"

Storage firmware 8+ up to 8.3.1.2: use "Restricted Administrator"

$ ssh superuser@<storage control enclosure IP address> "svctask mkuser -name stor2rrd -usergrp RestrictedAdmin"

Older firmwares: use "Administrator" role.
It is the lowest role which allows retrieve of statistics data from the storage. Read this for explanation.
```
$ ssh superuser@<storage control enclosure IP address> "svctask mkuser -name stor2rrd -usergrp Administrator"
```

SSH-keys

Create SSH-keys on STOR2RRD host under stor2rrd user (lpar2rrd on the XoruX Virtual Appliance) if do not exist yet.
It should already exist on the Virtual Appliance, skip it there.
Type enter instead of passphrase

# su - stor2rrd # (use lpar2rrd user on the Appliance)
$ ls -l ~/.ssh/id_rsa.pub
$ ssh-keygen -t rsa -N ""
  Generating public/private rsa key pair.
  Enter file in which to save the key (/home/stor2rrd/.ssh/id_rsa):
  ...
$ ls -l ~/.ssh/id_rsa.pub
  -rw-r--r-- 1 stor2rrd stor2rrd 382 Jun  1 12:47 /home/stor2rrd/.ssh/id_rsa.pub

New storage devices might require stronger key, use this one instead of above 'RSA'

$ ssh-keygen -t ecdsa -b 521 -N ""

Copy ssh-key into the storage under storage superuser
In case of IBM Storwize V7000 Unified use one of "Control Enclosure" IPs (definitely not "File Module" IP).
Detailed description of IBM Storwize V7000 Unified.
```
$ scp ~/.ssh/id_rsa.pub superuser@<storage control enclosure IP address>:/tmp/123tmp
```

Create new user "stor2rrd" on the storage

Storage firmware 8.3.1.2+: use "Monitor" role

$ ssh superuser@<storage control enclosure IP address> "svctask mkuser -name stor2rrd -usergrp Monitor -keyfile /tmp/123tmp"

Storage firmware 8+ up to 8.3.1.2: use "Restricted Administrator"

$ ssh superuser@<storage control enclosure IP address> "svctask mkuser -name stor2rrd -usergrp RestrictedAdmin -keyfile /tmp/123tmp"

Older firmwares: use "Administrator" role.
It is the lowest role which allows retrieve of statistics data from the storage. Read this for explanation.
```
$ ssh superuser@<storage control enclosure IP address> "svctask mkuser -name stor2rrd -usergrp Administrator -keyfile /tmp/123tmp"  
```

If stor2rrd user already exists on the storage then assign him new ssh-keys:

$ ssh superuser@<storage control enclosure IP address> "svctask chuser -keyfile /tmp/123tmp stor2rrd"

Check if that works:

$ ssh -i ~/.ssh/id_rsa stor2rrd@<storage control enclosure IP address> "svcinfo lssystem"
  id 00000100C0906BAA
  name SVC
  location local
  partnership
  bandwidth
  ...

STOR2RRD storage configuration

All actions below under stor2rrd user (lpar2rrd on Virtual Appliance)
Add storage into configuration from the UI:
Settings icon ➡ Storage ➡ New ➡ Vendor:device ➡ Lenovo V series
Make sure you have enough of disk space on the filesystem where is STOR2RRD installed
Roughly you might count 2 - 30 GB per a storage (it depends on number of volumes, 30GB for 5000 volumes)
```
$ df -g /home   # AIX
$ df -h /home   # Linux
```

Test storage connectivity, either from the UI or from the command line:

$ cd /home/stor2rrd/stor2rrd
$ ./bin/config_check.sh 
  =========================
  STORAGE: storwize01: SWIZ
  =========================
  ssh -o ConnectTimeout=15 -i /home/stor2rrd/.ssh/id_rsa  stor2rrd@192.168.1.1 "lscurrentuser"
  connection ok

Schedule to run storage agent from stor2rrd crontab (lpar2rrd on Virtual Appliance, it might already exist there)

$ crontab -l | grep load_svcperf.sh
$

Add if it does not exist as above

$ crontab -e  

# IBM Storwize/SVC storage agent and Lenovo Storwize
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/stor2rrd/stor2rrd/load_svcperf.sh > /home/stor2rrd/stor2rrd/load_svcperf.out 2>&1

Assure there is already an entry with the UI creation running once an hour in crontab

$ crontab -e

# STOR2RRD UI (just ONE entry of load.sh must be there)
5 * * * * /home/stor2rrd/stor2rrd/load.sh > /home/stor2rrd/stor2rrd/load.out 2>&1

Let run the storage agent for 15 - 20 minutes to get data, then:
```
$ cd /home/stor2rrd/stor2rrd
$ ./load.sh
```
Go to the web UI: http://<your web server>/stor2rrd/
Use Ctrl-F5 to refresh the web browser cache.

Troubleshooting

In case you find below error during initial configuration then follow this page

$ cd /home/stor2rrd/stor2rrd/logs
$ tail -1 error.log-<storage name>
  Can't locate object method "new" via package "XML::LibXML::SAX" at /usr/opt/perl5/lib/site_perl/5.10.1/XML/SAX/ParserFactory.pm line 43.

In case of a problem check our forum or contact us via support@stor2rrd.com

In case of usage of Virtual Appliance

Use local account lpar2rrd for hosting of STOR2RRD on the virtual appliance
Use /home/stor2rrd/stor2rrd as the product home

The product uses SMI-S and WBI API interfaces natively provided by the storage.

Storage connectivity

Allow access from the STOR2RRD host to the storage on ports 80 (WBI) and 5988 (SMI-S).
If you use https and secure SMI-S then test ports 443 and 5989 instead.
If you use storage cmd line configuration like in examples below (you can use the storage UI instead) then port 22 (ssh) must be open.
```
$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 443
  Connection to "192.168.1.1" on port "443" is ok
$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 5989
  Connection to "192.168.1.1" on port "5989" is ok
```

Make sure that secure SMI-S (port 5989) and WBI HTTPS (port 443) protocols are enabled on the storage

CMD line:

$ ssh -l manage <storage_hostname>
# show protocols
  Service and Security Protocols
  ------------------------------
  Web Browser Interface (HTTP): Enabled
  Secure Web Browser Interface (HTTPS): Enabled
  Command Line Interface (Telnet): Disabled
  Secure Command Line Interface (SSH): Enabled
  Storage Management Initiative Specification (SMI-S): Enabled
  Unsecure Storage Management Initiative Specification (SMI-S 5988): Disabled
  ...

If secure SMI-S or WBI HTTPS are not enabled:

# set protocols smis enabled
# set protocols https enabled

Create user stor2rrd on the storage with monitor and standard roles, allow SMI-S and WBI.
- CMD line method:
  Adjust password to your preferred one from used Test1234.
```
$ ssh -l manage <storage_hostname>
# create user interfaces smis,wbi locale English password Test1234 roles monitor,standard stor2rrd
```
- UI:
Note for some firmwares role "standard" is necessary for WBI access.

Install prerequisites

Skip that in case of Virtual Appliance

Linux:
Install sblim-wbemcli package:

# yum list |grep sblim-wbemcli
  sblim-wbemcli.x86_64                       1.6.2-9.el7                 base
# yum install sblim-wbemcli     # RedHat
# apt-get install sblim-wbemcli # Debian based (Ubuntu etc)
# /usr/bin/wbemcli -noverify -nl ei https://<storage_hostname>:5988/root/smis:cim_computersystem
  *
  * ./wbemcli: Http Exception: Username/password required.
  *

If you prefere to use https on port 5989 then you must use wbemcli with this patch 0002742
Download the package from:

wbemcli-1.6.3 x86_64 for Redhat 6
wbemcli-1.6.3 x86_64 for Redhat 7
wbemcli-1.6.3 for other Linuxes: ➡ Select your OS in the header ➡ "Grab binary packages directly"
wbemcli_x86_64.gz as general Linux binary, gunzip and replace /usr/bin/wbemcli by this

Test the package:

# rpm -q sblim-wbemcli
  sblim-wbemcli-1.6.3-31.1.x86_64
# /usr/bin/wbemcli -v
  * wbemcli Version 1.6.3
# /usr/bin/wbemcli -noverify -nl ei http://<storage_hostname>:5988/root/smis:cim_computersystem
  *
  * ./wbemcli: Http Exception: Username/password required.
  *

AIX:
Download and install

# rpm -Uvh libffi-3.1-1.aix5.1.ppc.rpm --nodeps
# rpm -Uvh libiconv-1.14-2.aix5.1.ppc.rpm --nodeps
# rpm -Uvh libidn-1.29-1.aix5.1.ppc.rpm --nodeps
# rpm -Uvh libssh2-1.4.3-2.aix5.1.ppc.rpm --nodeps
# rpm -Uvh openssl-1.0.1j-1.aix5.1.ppc.rpm
# rpm -Uvh openldap-2.4.23-0.3.aix5.1.ppc.rpm
# rpm -Uvh curl-7.27.0-1.aix5.1.ppc.rpm
# rpm -Uvh wbemcli-1.6.3-1.aix5.1.ppc.rpm

# LIBPATH=/opt/freeware/lib; export LIBPATH
# /opt/freeware/bin/wbemcli -noverify -nl ei http://<storage_hostname>:5988/root/smis:cim_computersystem
  *
  * ./wbemcli: Http Exception: Username/password required.
  *

STOR2RRD storage configuration

All actions below under stor2rrd user (lpar2rrd on Virtual Appliance)

Add storage into configuration from the UI:
Settings icon ➡ Storage ➡ New ➡ Vendor:device ➡ Lenovo S series ➡ SMI-S

If you set https in above storage properties then this must be enabled on the storage:

$ ssh -l manage <storage_hostname>
# show protocols
  ...
  Secure Web Browser Interface (HTTPS): Enabled
  Storage Management Initiative Specification (SMI-S): Enabled
  ...

If you use http then this:

  Web Browser Interface (HTTP): Enabled
  Unsecure Storage Management Initiative Specification (SMI-S 5988): Enabled

Do not use shell special characters like @:;#!?|$*[]\{}`"'& in the password, use rather .+-=_% instead.
Make sure you have enough of disk space on the filesystem where is STOR2RRD installed
Roughly you might count 2 - 30 GB per a storage (it depends on number of volumes, 30GB for 5000 volumes)
```
$ df -g /home   # AIX
$ df -h /home   # Linux
```
Test storage connectivity, either from the UI or from the command line:
```
$ cd /home/stor2rrd/stor2rrd
$ ./bin/config_check.sh 
  =====================
  STORAGE: Lenovo-alias01 : DOTHILL
  =====================
  connection ok
```
If you use HTTPS and do not have installed Perl SSL support then you get an error:
"Missing LWP::Protocol::https", "Connection reset by peer" or "500 SSL negotiation failed"
Follow this or switch to HTTP on the storage line in etc/storage-list.cfg.

Newer storage firmware releases (GL225R003) might require higher user role for SMI-S access.
Try to use "manage" role if you get this error:
```
$ /usr/bin/wbemcli -noverify -nl ei https://stor2rrd@<IP/hostname>:5989/root/smis:cim_computersystem
$ /usr/bin/wbemcli -noverify -nl ei http://stor2rrd@<IP/hostname>:5988/root/smis:cim_computersystem
  *
  * /usr/bin/wbemcli: Http Exception: Invalid username/password.
  *
```

Schedule to run storage agent from stor2rrd crontab (lpar2rrd on Virtual Appliance, it might already exist there)

$ crontab -l | grep load_dothillperf.sh
$

Add if it does not exist as above

$ crontab -e

# DotHill AssuredSAN and its rebrands like Lenovo  S series or HPE MSA2000/P2000, Dell EMC PowerVault ME4 Series, Quantum StorNex
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/stor2rrd/stor2rrd/load_dothillperf.sh > /home/stor2rrd/stor2rrd/load_dothillperf.out 2>&1

Assure there is already an entry with the UI creation running once an hour in crontab

$ crontab -e

# STOR2RRD UI (just ONE entry of load.sh must be there)
5 * * * * /home/stor2rrd/stor2rrd/load.sh > /home/stor2rrd/stor2rrd/load.out 2>&1

Let run the storage agent for 15 - 20 minutes to get data, then:
```
$ cd /home/stor2rrd/stor2rrd
$ ./load.sh
```
Go to the web UI: http://<your web server>/stor2rrd/
Use Ctrl-F5 to refresh the web browser cache.

Troubleshooting

Assure SMI-S is working for user stor2rrd. Adjust password and storage_hostname in example below:

$ /usr/bin/wbemcli -nl -t -noverify ein 'http://stor2rrd:<password>@<storage_hostnamew>:5988/root/smis:SMI_Controller'
  192.168.1.1:5988/root/smis:SMI_Controller.Name="208000c0ff261f53:A",CreationClassName="SMI_Controller"
  192.168.1.2:5988/root/smis:SMI_Controller.Name="208000c0ff261f53:B",CreationClassName="SMI_Controller"

When it does not work then check user rights of stor2rrd user (must have "smis"), network visibility (firewall), user/password combination.

In case of usage of Virtual Appliance

Use local account lpar2rrd for hosting of STOR2RRD on the virtual appliance
Use /home/stor2rrd/stor2rrd as the product home

The product uses SMI-S and WBI API interfaces natively provided by the storage.

Storage connectivity

Allow access from the STOR2RRD host to the storage on ports 80 (WBI) and 5988 (SMI-S).
If you use https and secure SMI-S then test ports 443 and 5989 instead.
If you use storage cmd line configuration like in examples below (you can use the storage UI instead) then port 22 (ssh) must be open.
```
$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 443
  Connection to "192.168.1.1" on port "443" is ok
$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 5989
  Connection to "192.168.1.1" on port "5989" is ok
```

Make sure that secure SMI-S (port 5989) and WBI HTTPS (port 443) protocols are enabled on the storage

CMD line:

$ ssh -l manage <storage_hostname>
# show protocols
  Service and Security Protocols
  ------------------------------
  Web Browser Interface (HTTP): Enabled
  Secure Web Browser Interface (HTTPS): Enabled
  Command Line Interface (Telnet): Disabled
  Secure Command Line Interface (SSH): Enabled
  Storage Management Initiative Specification (SMI-S): Enabled
  Unsecure Storage Management Initiative Specification (SMI-S 5988): Disabled
  ...

If secure SMI-S or WBI HTTPS are not enabled:

# set protocols smis enabled
# set protocols https enabled

Create user stor2rrd on the storage with monitor and standard roles, allow SMI-S and WBI.
- CMD line method:
  Adjust password to your preferred one from used Test1234.
```
$ ssh -l manage <storage_hostname>
# create user interfaces smis,wbi locale English password Test1234 roles monitor,standard stor2rrd
```
- UI:
Note for some firmwares role "standard" is necessary for WBI access.

Install prerequisites

Skip that in case of Virtual Appliance

Linux:
Install sblim-wbemcli package:

# yum list |grep sblim-wbemcli
  sblim-wbemcli.x86_64                       1.6.2-9.el7                 base
# yum install sblim-wbemcli     # RedHat
# apt-get install sblim-wbemcli # Debian based (Ubuntu etc)
# /usr/bin/wbemcli -noverify -nl ei https://<storage_hostname>:5988/root/smis:cim_computersystem
  *
  * ./wbemcli: Http Exception: Username/password required.
  *

If you prefere to use https on port 5989 then you must use wbemcli with this patch 0002742
Download the package from:

wbemcli-1.6.3 x86_64 for Redhat 6
wbemcli-1.6.3 x86_64 for Redhat 7
wbemcli-1.6.3 for other Linuxes: ➡ Select your OS in the header ➡ "Grab binary packages directly"
wbemcli_x86_64.gz as general Linux binary, gunzip and replace /usr/bin/wbemcli by this

Test the package:

# rpm -q sblim-wbemcli
  sblim-wbemcli-1.6.3-31.1.x86_64
# /usr/bin/wbemcli -v
  * wbemcli Version 1.6.3
# /usr/bin/wbemcli -noverify -nl ei http://<storage_hostname>:5988/root/smis:cim_computersystem
  *
  * ./wbemcli: Http Exception: Username/password required.
  *

AIX:
Download and install

# rpm -Uvh libffi-3.1-1.aix5.1.ppc.rpm --nodeps
# rpm -Uvh libiconv-1.14-2.aix5.1.ppc.rpm --nodeps
# rpm -Uvh libidn-1.29-1.aix5.1.ppc.rpm --nodeps
# rpm -Uvh libssh2-1.4.3-2.aix5.1.ppc.rpm --nodeps
# rpm -Uvh openssl-1.0.1j-1.aix5.1.ppc.rpm
# rpm -Uvh openldap-2.4.23-0.3.aix5.1.ppc.rpm
# rpm -Uvh curl-7.27.0-1.aix5.1.ppc.rpm
# rpm -Uvh wbemcli-1.6.3-1.aix5.1.ppc.rpm

# LIBPATH=/opt/freeware/lib; export LIBPATH
# /opt/freeware/bin/wbemcli -noverify -nl ei http://<storage_hostname>:5988/root/smis:cim_computersystem
  *
  * ./wbemcli: Http Exception: Username/password required.
  *

STOR2RRD storage configuration

All actions below under stor2rrd user (lpar2rrd on Virtual Appliance)
Add storage into configuration from the UI:
Settings icon ➡ Storage ➡ New ➡ Vendor:device ➡ Lenovo S series ➡ SMI-S
Do not use shell special characters like @#!?|$*[]\{}`"'& in the password, use rather ;:.+- instead.
Make sure you have enough of disk space on the filesystem where is STOR2RRD installed
Roughly you might count 2 - 30 GB per a storage (it depends on number of volumes, 30GB for 5000 volumes)
```
$ df -g /home   # AIX
$ df -h /home   # Linux
```
Test storage connectivity, either from the UI or from the command line:
```
$ cd /home/stor2rrd/stor2rrd
$ ./bin/config_check.sh 
  =====================
  STORAGE: Lenovo-alias01 : DOTHILL
  =====================
  connection ok
```
If you use HTTPS and do not have installed Perl SSL support then you get an error:
"Missing LWP::Protocol::https", "Connection reset by peer" or "500 SSL negotiation failed"
Follow this or switch to HTTP on the storage line in etc/storage-list.cfg.

Newer storage firmware releases (GL225R003) might require higher user role for SMI-S access.
Try to use "manage" role if you get this error:
```
*
* /usr/bin/wbemcli: Http Exception: Invalid username/password.
*
```

Schedule to run storage agent from stor2rrd crontab (lpar2rrd on Virtual Appliance, it might already exist there)

$ crontab -l | grep load_dothillperf.sh
$

Add if it does not exist as above

$ crontab -e

# DotHill AssuredSAN and its rebrands like Lenovo S series, Lenovo ThinkSystem DS series or HPE MSA2000/P2000
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/stor2rrd/stor2rrd/load_dothillperf.sh > /home/stor2rrd/stor2rrd/load_dothillperf.out 2>&1

Assure there is already an entry with the UI creation running once an hour in crontab

$ crontab -e

# STOR2RRD UI (just ONE entry of load.sh must be there)
5 * * * * /home/stor2rrd/stor2rrd/load.sh > /home/stor2rrd/stor2rrd/load.out 2>&1

Let run the storage agent for 15 - 20 minutes to get data, then:
```
$ cd /home/stor2rrd/stor2rrd
$ ./load.sh
```
Go to the web UI: http://<your web server>/stor2rrd/
Use Ctrl-F5 to refresh the web browser cache.

Troubleshooting

Assure SMI-S is working for user stor2rrd. Adjust password and storage_hostname in example below:

$ /usr/bin/wbemcli -nl -t -noverify ein 'http://stor2rrd:<password>@<storage_hostnamew>:5988/root/smis:SMI_Controller'
  192.168.1.1:5988/root/smis:SMI_Controller.Name="208000c0ff261f53:A",CreationClassName="SMI_Controller"
  192.168.1.2:5988/root/smis:SMI_Controller.Name="208000c0ff261f53:B",CreationClassName="SMI_Controller"

When it does not work then check user rights of stor2rrd user (must have "smis"), network visibility (firewall), user/password combination.

In case of usage of Virtual Appliance

Use local account lpar2rrd for hosting of STOR2RRD on the virtual appliance
Use /home/stor2rrd/stor2rrd as the product home

Storage access can be done in 2 different ways:

REST API use it on new models which support it, it provides more data metrics and is not necessary install 3rd part SW
ThinkSystem Storage Manager for DE Series CLI works on all models

Read detailed comparison of both methods.

REST API

Allow access from the STOR2RRD host to the storage on port 8443.
Test if port is open:

$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 8443
  Connection to "192.168.1.1" on port "8443" is ok

Storage user: use predefined user monitor which has monitor role. Change its password if necessary through the storage UI.
Add storage into configuration from the UI:
Settings icon ➡ Storage ➡ New ➡ Vendor:device ➡ Lenovo DE Series ➡ REST API
All actions below under stor2rrd user on STOR2RRD server (lpar2rrd on Virtual Appliance)
Assure you have enough of disk space on the filesystem where is STOR2RRD installed
Roughly you might count 2 - 30 GB per a storage (it depends on number of volumes, 30GB for 5000 volumes)
```
$ df -g /home   # AIX
$ df -h /home   # Linux
```

Schedule to run storage agent from stor2rrd crontab (lpar2rrd on Virtual Appliance, it might already exist there)

$ crontab -l | grep load_ds5perf.sh 
$

Add if it does not exist as above

$ crontab -e  

# NetApp E-series (use same script load_ds5perf.sh as IBM DS3000/DS4000/DS5000)
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/stor2rrd/stor2rrd/load_ds5perf.sh > /home/stor2rrd/stor2rrd/load_ds5perf.out 2>&1

Assure there is already an entry with the UI creation running once an hour in crontab

$ crontab -e

# STOR2RRD UI (just ONE entry of load.sh must be there)
5 * * * * /home/stor2rrd/stor2rrd/load.sh > /home/stor2rrd/stor2rrd/load.out 2>&1

Let run the storage agent for 15 - 20 minutes to get data, then:
```
$ cd /home/stor2rrd/stor2rrd
$ ./load.sh
```
Go to the web UI: http://<your web server>/stor2rrd/
Use Ctrl-F5 to refresh the web browser cache.

ThinkSystem Storage Manager for DE Series CLI

Storage Manager installation

Download and install storage manager package from Lenovo download site.

If there is already installed SMcli from IBM to monitor DS3000/4000/5000 storage then it must be removed:

# /opt/IBM_DS/Uninstall\ IBM\ System\ Storage\ DS\ Storage\ Manager\ 10/Uninstall\ IBM\ System\ Storage\ DS\ Storage\ Manager\ 10

Install storage manager under root user :

# umask 022
# chmod 775 lnvgy_utl_storagemanager_11.52.0A54.0003_linux_x86-64.bin
# ./lnvgy_utl_storagemanager_11.52.0A54.0003_linux_x86-64.bin  -i console

  ===============================================================================
  Select Installation Type
  ------------------------

  Please choose the Install Set to be installed by this installer.

  ->1- Typical (Full Installation)
    2- Management Station
    3- Host

    4- Customize...

  ENTER THE NUMBER FOR THE INSTALL SET, OR PRESS <ENTER> TO ACCEPT THE DEFAULT
   : 2

  ===============================================================================
  Installation Complete
  ---------------------

  Congratulations. Lenovo ThinkSystem Storage Manager has been successfully
  installed to:

   /opt/ThinkSystem/SMgr

# chmod 755 /opt/ThinkSystem/SMgr/client/SMcli
# ls -l /opt/ThinkSystem/SMgr/client/SMcli
  -rwxr-xr-x 1 root root 996 Dec 10 23:02 /opt/ThinkSystem/SMgr/client/SMcli

In case of any problem check Install logs in /opt/ThinkSystem/SMgr/InstallLogs

Stop SMmonitor service

# /opt/ThinkSystem/SMgr/client/SMmonitor stop 
or
# systemctl stop smmonitor.service
# systemctl stop smagent.service

Disable SMmonitor service, use one of below method applicable to your OS

# chkconfig --list SMmonitor
# chkconfig --level 2345 SMmonitor off

# systemctl disable smmonitor.service
# systemctl disable smagent.service

# rm -f /etc/rc2.d/S99SMmonitor /etc/rc.d/init.d/SMmonitor

Allow user stor2rrd run SMcli API (lpar2rrd user on Virtual Appliance)
Under root execute following:

# rm -f /var/opt/SM/emwdata_v04_bin.lock /var/opt/SM/pref_01_bin.lock /var/opt/SM/PMServer.lock
# chown stor2rrd /opt/ThinkSystem/SMgr/client/SMcli /var/opt/SM/prefbackup_01.bin
# chown stor2rrd /var/opt/SM/emwback_v04.bin /var/opt/SM /var/opt/SM/emwdata_v04.bin
# chown stor2rrd /var/opt/SM /var/opt/SM/pref_01.bin
# chmod -f 644 /opt/ThinkSystem/SMgr/jre/lib/security/java.security /usr/SMruntime/jre/lib/security/java.security
# chmod -f 755 /opt/ThinkSystem/SMgr/client/lib/*

If you do not do it then you get such error by running it under someone else than root:
(Do not forget to deactivate SMmonitor as above as it can change file rights back to root itself)

# su - stor2rrd  # (lpar2rrd user on the Virtual Appliance)
$ SMcli -d
  An internal error, error code 12, has occurred.  This is possibly due to
  initialization problems when loading the necessary internal files.  Please check
  your installation of the storage management client. If problems continue 
  to persist, re-install the storage management client.

Allow "The legacy management interface" via ThinkSystem System Manager (it is necessary only on some firmare releases)

Storage configuration

Allow access from the STOR2RRD host to the storage on both controllers on port 2463.
Test if port is open:

$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 2463
  Connection to "192.168.1.1" on port "2463" is ok

Connect new storage, you must connect both storage controllers

# su - stor2rrd  # (lpar2rrd user on the Virtual Appliance)
$ SMcli -d
  There are currently no storage subsystems listed in the configuration file. Add
  storage subsystems using the Add Storage Subsystem option in the storage management
  software or by command line.
$ SMcli -A 192.168.1.1 192.168.1.2
  New storage subsystem was discovered at address 192.168.1.1
  New storage subsystem was discovered at address 192.168.1.2
  SMcli completed successfully
$ SMcli -d
  DE6000H  ControlerA_hostname  ControlerB_hostname

Use "DE6000H" further for etc/storage-list.cfg configuration

Check if the storage needs user authorization (new firmwares allows to specify password for monitor user role):

$ SMcli -n DE6000H -e -c "show hostTopology;"

When it fails then use "monitor" role credetials (password can be set in the storage UI):

$ SMcli -n DE6000H -e -c "show storageArray hostTopology;"            
  The operation needs either the 'Administrator' or 'Monitor' password to be set.
$ SMcli -n DE6000H -p <password> -R monitor -e -c "show storageArray hostTopology;"

Use that credentials in future etc/storage-list.cfg configuration

STOR2RRD storage configuration

All actions below under stor2rrd user (lpar2rrd on Virtual Appliance)
Add storage into configuration from the UI:
Settings icon ➡ Storage ➡ New ➡ Vendor:device ➡ Lenovo DE series

Note that user/password is required only when it is configured on the storage.

If you get this error during connection test
```
API authorization: NOK

SMcli command failed: "/opt/IBM_DS/client/SMcli -n DE6000H -e -c "show hostTopology;""
sh: /opt/IBM_DS/client/SMcli: No such file or directory
```
Then change DS5_CLIDIR variable in /home/stor2rrd/stor2rrd/etc/stor2rrd.cfg to below path and repeat test
DS5_CLIDIR=/opt/ThinkSystem/SMgr/client
Make sure you have enough of disk space on the filesystem where is STOR2RRD installed
Roughly you might count 2 - 30 GB per a storage (it depends on number of volumes, 30GB for 5000 volumes)
```
$ df -g /home   # AIX
$ df -h /home   # Linux
```
Test storage connectivity, either from the UI or from the command line:
```
$ cd /home/stor2rrd/stor2rrd
$ ./bin/config_check.sh 
  =======================
  STORAGE: DE6000H : DS5K
  =======================
  /usr/SMclient/SMcli -n DE6000H -e -c "storageSubsystem summary;"
  connection ok
```
When connection failed and direct command says this:
```
$ SMcli -n DE6000H -e -c "show hostTopology;"
  Warning! No Monitor password is set for the storage subsystem.
```
Then it looks like that the latest firmwares require enabling of password authorization.
Set password for monitor account in the storage UI and then place same password into etc/storage-list.cfg
Do not use shell special characters like #!?|$*[]\{}`"'& in the password, use ;:.+-%@ instead.
```
DE6000H:DS5K:monitor:password01:
```
Then verify connection via:
```
$ ./bin/config_check.sh DE6000H
...
```

Schedule to run storage agent from stor2rrd crontab (lpar2rrd on Virtual Appliance, it might already exist there)

$ crontab -l | grep load_ds5perf.sh 
$

Add if it does not exist as above

$ crontab -e  

# Lenovo DE series (use same script load_ds5perf.sh as IBM DS3000/DS4000/DS5000 and NetApp E-series)
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/stor2rrd/stor2rrd/load_ds5perf.sh > /home/stor2rrd/stor2rrd/load_ds5perf.out 2>&1

Assure there is already an entry with the UI creation running once an hour in crontab

$ crontab -e

# STOR2RRD UI (just ONE entry of load.sh must be there)
5 * * * * /home/stor2rrd/stor2rrd/load.sh > /home/stor2rrd/stor2rrd/load.out 2>&1

Let run the storage agent for 15 - 20 minutes to get data, then:
```
$ cd /home/stor2rrd/stor2rrd
$ ./load.sh
```

Go to the web UI: http://<your web server>/stor2rrd/
Use Ctrl-F5 to refresh the web browser cache.

In case of usage of Virtual Appliance

Use local account lpar2rrd for hosting of STOR2RRD on the virtual appliance
Use /home/stor2rrd/stor2rrd as the product home

The program uses Lenovo CLI (ssh) interface for getting of performance data and Lenovo API (http[s]) for getting of configuration data from the storage.

Allow access from the STOR2RRD host to the storage on port 22 (ssh) and 80 (443 if you use SSL).
Test if ports are open:

$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 22
  Connection to "192.168.1.1" on port "22" is ok
$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 80
  Connection to "192.168.1.1" on port "80" is ok

Create ssh-keys on STOR2RRD host under stor2rrd user if do not exist yet.
It should already exist on the Virtual Appliance, skip it there.
Type enter instead of passphrase

# su - stor2rrd # (use lpar2rrd user on the Appliance)
$ ls -l ~/.ssh/id_rsa.pub
$ ssh-keygen -t rsa -N ""
  Generating public/private rsa key pair.
  Enter file in which to save the key (/home/stor2rrd/.ssh/id_rsa):
  ...
$ ls -l ~/.ssh/id_rsa.pub
  -rw-r--r-- 1 stor2rrd stor2rrd 382 Jun  1 12:47 /home/stor2rrd/.ssh/id_rsa.pub

New storage devices might require stronger key, use this one instead of above 'RSA'

$ ssh-keygen -t ecdsa -b 521 -N ""

Cmode::> security login role create -role monitor -cmddirname DEFAULT -access readonly
Cmode::> security login role create -role monitor -cmddirname set -access all
Cmode::> security login role create -role monitor -cmddirname statistics -access all

# You can check the role you've just created
Cmode::> security login role show -role monitor

             Role          Command/                                      Access
  Vserver    Name          Directory                               Query Level
  ---------- ------------- --------- ----------------------------------- --------
  xorux      monitor       DEFAULT                                       readonly
  xorux      monitor       set                                           all
  xorux      monitor       statistics                                    all
  3 entries were displayed.

# Create stor2rrd user, enable SSH, ONTAPI
Cmode::> security login create -username stor2rrd -role monitor -application ssh -authmethod password
Cmode::> security login create -username stor2rrd -role monitor -application ontapi -authmethod password
Cmode::> security login create -username stor2rrd -role monitor -application ssh -authmethod publickey

# Assign stor2rrd user for monitoring role
Cmode::> security login modify stor2rrd -role monitor *

Cmode::> security login show
                               Authentication             Acct   Is-Nsswitch
  User/Group Name  Application Method    Role Name        Locked Group
  ---------------- ----------- --------- ---------------- ------ -----------
  stor2rrd         ontapi      password  monitor          no     no
  stor2rrd         ssh         password  monitor          no     no
  stor2rrd         ssh         publickey monitor          -      no

Enable ssh key authentication.
On the STOR2RRD server show the public key (id_rsa.pub or id_rsa.pub), under stor2rrd (lpar2rrd user on the Appliance)

$ cat ~/.ssh/id_rsa.pub
  ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCu7y41DTAqJwqA4KJfZQWiqU7IMbs8lbK0nvDgZ9dMWmd3l+uawDKxkUdvnRUxBrLqGrxHZx3CjcLoNDqvlFl0AKE3DRTmv6o7plD3b1jRwsCABpNob5knZ/A3E4uVWytJPz036zjkn4vUSrGKJon3oUgLkmtSJoT2FCLvmPbYONPEqs+tQ/yCCkIpVcZwCxwPCbC7nnuuXhjMmRQNzNooREgBHr4eUWr3rlUPNxsSBjg6Mcbfb3u3O9+e49KJKUvMoD40AULW3h3j6bJ6bTTyhKavETpnkryeEq3936oVJdKdd4noraR54rK8lLrErFYgAG8ugLrxzmOYt6dN1phb

Note: If you is your storage in FIPS mode then you must use more secure ssh kyes, DSA/RSA keys will not work
Generate it by (type enter when you are prompted):

$ ssh-keygen -t ecdsa -b 521

use this pub key: cat ~/.ssh/id_ecdsa.pub

Place the key into this command (still under admin user)

Cmode::> security login publickey create -username stor2rrd -publickey "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCu7y41DTAqJwqA4KJfZQWiqU7IMbs8lbK0nvDgZ9dMWmd3l+uawDKxkUdvnRUxBrLqGrxHZx3CjcLoNDqvlFl0AKE3DRTmv6o7plD3b1jRwsCABpNob5knZ/A3E4uVWytJPz036zjkn4vUSrGKJon3oUgLkmtSJoT2FCLvmPbYONPEqs+tQ/yCCkIpVcZwCxwPCbC7nnuuXhjMmRQNzNooREgBHr4eUWr3rlUPNxsSBjg6Mcbfb3u3O9+e49KJKUvMoD40AULW3h3j6bJ6bTTyhKavETpnkryeEq3936oVJdKdd4noraR54rK8lLrErFYgAG8ugLrxzmOYt6dN1phb" -comment STOR2RRD

Assure ssh-key access works, it should not promt for password
Log out from the storage and try to log in as stor2rrd user

$ ssh stor2rrd@192.168.1.1
Cmode::>

Assure that web services are started, again under admin account

Cmode::> system services web show
  External Web Services: true
                 Status: online
     HTTP Protocol Port: 80
    HTTPS Protocol Port: 443
           HTTP Enabled: true

Assure that storage build in firewall does not filter it:

Cmode::> firewall policy show -policy mgmt -instance -service http,https
  Vserver Name: Cmode
      Policy: mgmt
     Service: http
  Allowed IPs: 0.0.0.0/0
     IPspace: Default

  Vserver Name: Cmode
      Policy: mgmt
     Service: https
  Allowed IPs: 0.0.0.0/0
     IPspace: Default

Cmode::> net int show -vserver Cmode -lif cluster_mgmt
  --> interfaces must have: Firewall Policy: mgmt

STOR2RRD storage configuration

Add storage into configuration from the UI:
Settings icon ➡ Storage ➡ New ➡ Vendor:device ➡ Lenovo ThinkSystem DM / DG Series
Assure you have enough of disk space on the filesystem where is STOR2RRD installed
Roughly you might count 2 - 30 GB per a storage (it depends on number of volumes, 30GB for 5000 volumes)
```
$ df -g /home   # AIX
$ df -h /home   # Linux
```

Schedule to run storage agent from stor2rrd crontab (lpar2rrd on Virtual Appliance, it might already exist there)

$ crontab -l | grep load_netappperf.sh
$

Add if it does not exist as above

$ crontab -e

# Lenovo DM / DG series, NetApp FAS : C-mode & 7-mode (same code, use it just once)
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/stor2rrd/stor2rrd/load_netappperf.sh > /home/stor2rrd/stor2rrd/load_netappperf.out 2>&1

Assure there is already an entry with the UI creation running once an hour in crontab

$ crontab -e

# STOR2RRD UI (just ONE entry of load.sh must be there)
5 * * * * /home/stor2rrd/stor2rrd/load.sh > /home/stor2rrd/stor2rrd/load.out 2>&1

Let run the storage agent for 15 - 20 minutes to get data, then:
```
$ cd /home/stor2rrd/stor2rrd
$ ./load.sh
```

Go to the web UI: http://<your web server>/stor2rrd/
Use Ctrl-F5 to refresh the web browser cache.

In case of usage of Virtual Appliance

Use local account lpar2rrd for hosting of STOR2RRD on the virtual appliance
Use /home/stor2rrd/stor2rrd as the product home

The product uses storage WBI API interfaces natively provided by the storage.
WBI access must be used on new storage models which do not support SMI-S protocol.

Storage connectivity

Allow access from the STOR2RRD host to the storage on ports 443.

$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 443
  Connection to "192.168.1.1" on port "443" is ok

Make sure that secure and WBI HTTPS (port 443) protocol is enabled on the storage

CMD line:

$ ssh -l manage <storage_hostname>
# show protocols
  Service and Security Protocols
  ------------------------------
  ...
  Secure Web Browser Interface (HTTPS): Enabled
  ...

If secure WBI HTTPS is not enabled:

# set protocols https enabled

Create user stor2rrd on the storage with monitor and standard roles, allow WBI.
- CMD line method:
  Adjust password to your preferred one from used Test1234.
```
$ ssh -l manage <storage_hostname>
# create user interfaces wbi locale English password Test1234 roles monitor,standard stor2rrd
```
- UI:
Note for some newer firmwares role "standard" is necessary for WBI access.

STOR2RRD storage configuration

All actions below under stor2rrd user (lpar2rrd on Virtual Appliance)
Add storage into configuration from the UI:
Settings icon ➡ Storage ➡ New ➡ Vendor:device ➡ Lenovo Storage S series ➡ WBI

If you set https in above storage properties then this must be enabled on the storage:
```
$ ssh -l manage <storage_hostname>
# show protocols
  ...
  Secure Web Browser Interface (HTTPS): Enabled
  ...
```
If you use http then this:
```
  Web Browser Interface (HTTP): Enabled
```
Make sure you have enough of disk space on the filesystem where is STOR2RRD installed
Roughly you might count 2 - 30 GB per a storage (it depends on number of volumes, 30GB for 5000 volumes)
```
$ df -g /home   # AIX
$ df -h /home   # Linux
```
Test storage connectivity, either from the UI or from the command line:
```
$ cd /home/stor2rrd/stor2rrd
$ ./bin/config_check.sh 
  =====================
  STORAGE: DotHill
  =====================
  connection ok
```
If you use HTTPS and do not have installed Perl SSL support then you get an error:
"Missing LWP::Protocol::https", "Connection reset by peer" or "500 SSL negotiation failed"
Follow this or switch to HTTP on the storage line in etc/storage-list.cfg.

Schedule to run storage agent from stor2rrd crontab (lpar2rrd on Virtual Appliance, it might already exist there)

$ crontab -l | grep load_dothillperf.sh
$

Add if it does not exist as above

$ crontab -e

# DotHill AssuredSAN and its rebrands like Lenovo  S series or HPE MSA2000/P2000, Dell EMC PowerVault ME4 Series, Quantum StorNex
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/stor2rrd/stor2rrd/load_dothillperf.sh > /home/stor2rrd/stor2rrd/load_dothillperf.out 2>&1

Ensure there is already an entry with the UI creation running once an hour in crontab

$ crontab -e

# STOR2RRD UI (just ONE entry of load.sh must be there)
5 * * * * /home/stor2rrd/stor2rrd/load.sh > /home/stor2rrd/stor2rrd/load.out 2>&1

Let run the storage agent for 15 - 20 minutes to get data, then:
```
$ cd /home/stor2rrd/stor2rrd
$ ./load.sh
```
Go to the web UI: http://<your web server>/stor2rrd/
Use Ctrl-F5 to refresh the web browser cache.

In case of usage of Virtual Appliance

Use local account lpar2rrd for hosting of STOR2RRD on the virtual appliance
Use /home/stor2rrd/stor2rrd as the product home

The product uses storage WBI API interfaces natively provided by the storage.
WBI access must be used on new storage models which do not support SMI-S protocol.

Storage connectivity

Allow access from the STOR2RRD host to the storage on ports 443.

$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 443
  Connection to "192.168.1.1" on port "443" is ok

Make sure that secure and WBI HTTPS (port 443) protocol is enabled on the storage

CMD line:

$ ssh -l manage <storage_hostname>
# show protocols
  Service and Security Protocols
  ------------------------------
  ...
  Secure Web Browser Interface (HTTPS): Enabled
  ...

If secure WBI HTTPS is not enabled:

# set protocols https enabled

Create user stor2rrd on the storage with monitor and standard roles, allow WBI.
- CMD line method:
  Adjust password to your preferred one from used Test1234.
```
$ ssh -l manage <storage_hostname>
# create user interfaces wbi locale English password Test1234 roles monitor,standard stor2rrd
```
- UI:
Note for some newer firmwares role "standard" is necessary for WBI access.

STOR2RRD storage configuration

All actions below under stor2rrd user (lpar2rrd on Virtual Appliance)
Add storage into configuration from the UI:
Settings icon ➡ Storage ➡ New ➡ Vendor:device ➡ Lenovo Storage S series ➡ WBI

If you set https in above storage properties then this must be enabled on the storage:
```
$ ssh -l manage 
# show protocols
  ...
  Secure Web Browser Interface (HTTPS): Enabled
  ...
```
If you use http then this:
```
  Web Browser Interface (HTTP): Enabled
```
Make sure you have enough of disk space on the filesystem where is STOR2RRD installed
Roughly you might count 2 - 30 GB per a storage (it depends on number of volumes, 30GB for 5000 volumes)
```
$ df -g /home   # AIX
$ df -h /home   # Linux
```
Test storage connectivity, either from the UI or from the command line:
```
$ cd /home/stor2rrd/stor2rrd
$ ./bin/config_check.sh 
  =====================
  STORAGE: DotHill
  =====================
  connection ok
```
If you use HTTPS and do not have installed Perl SSL support then you get an error:
"Missing LWP::Protocol::https", "Connection reset by peer" or "500 SSL negotiation failed"
Follow this or switch to HTTP on the storage line in etc/storage-list.cfg.

Schedule to run storage agent from stor2rrd crontab (lpar2rrd on Virtual Appliance, it might already exist there)

$ crontab -l | grep load_dothillperf.sh
$

Add if it does not exist as above

$ crontab -e

# DotHill AssuredSAN and its rebrands like Lenovo  S series or HPE MSA2000/P2000, Dell EMC PowerVault ME4 Series, Quantum StorNex
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/stor2rrd/stor2rrd/load_dothillperf.sh > /home/stor2rrd/stor2rrd/load_dothillperf.out 2>&1

Ensure there is already an entry with the UI creation running once an hour in crontab

$ crontab -e

# STOR2RRD UI (just ONE entry of load.sh must be there)
5 * * * * /home/stor2rrd/stor2rrd/load.sh > /home/stor2rrd/stor2rrd/load.out 2>&1

Let run the storage agent for 15 - 20 minutes to get data, then:
```
$ cd /home/stor2rrd/stor2rrd
$ ./load.sh
```
Go to the web UI: http://<your web server>/stor2rrd/
Use Ctrl-F5 to refresh the web browser cache.

In case of usage of Virtual Appliance

Use local account lpar2rrd for hosting of STOR2RRD on the virtual appliance
Use /home/stor2rrd/stor2rrd as the product home

The program uses REST API data from the storage.
Note REST API is available since ONTAP 9.12.1, use ONTAPI method for older firmwares
STOR2RRD monitoring after change from ONTAPI to REST API.

Allow access from the STOR2RRD host to the NetApp storage on port 443.
Test if ports are open:

$ perl /home/stor2rrd/stor2rrd/bin/conntest.pl 192.168.1.1 443
  Connection to "192.168.1.1" on port "443" is ok

# Create stor2rrd user, enable REST API access
Cmode::> security login create -username stor2rrd -role readonly -application http -authmethod password

Cmode::> security login show -user-or-group-name stor2rrd
                                                                 Second
User/Group                 Authentication                 Acct   Authentication
Name           Application Method        Role Name        Locked Method
-------------- ----------- ------------- ---------------- ------ --------------
stor2rrd       http        password      readonly         no     none

Or you can use ONTAP System Manager to do the same:

Make sure that web services are started, again under admin account

Cmode::> system services web show
  External Web Services: true
                 Status: online
     HTTP Protocol Port: 80
    HTTPS Protocol Port: 443
           HTTP Enabled: true

Make sure that NetApp build in firewall does not filter it:

Cmode::> firewall policy show -policy mgmt -instance -service http,https
  Vserver Name: Cmode
      Policy: mgmt
     Service: http
  Allowed IPs: 0.0.0.0/0
     IPspace: Default

  Vserver Name: Cmode
      Policy: mgmt
     Service: https
  Allowed IPs: 0.0.0.0/0
     IPspace: Default

Cmode::> net int show -vserver Cmode -lif cluster_mgmt
  --> interfaces must have: Firewall Policy: mgmt

STOR2RRD storage configuration

All actions below under stor2rrd user (lpar2rrd on Virtual Appliance)
Add storage into configuration from the UI:
Settings icon ➡ Storage ➡ New ➡ Vendor:device ➡ Lenovo ThinkSystem DM / DG Series (REST API)
If you get errors such as "Missing LWP::Protocol::https", "Connection reset by peer" or "500 SSL negotiation failed" then follow this.
Make sure you have enough of disk space on the filesystem where is STOR2RRD installed
Roughly you might count 2 - 30 GB per a storage (it depends on number of volumes, 30GB for 5000 volumes)
```
$ df -g /home   # AIX
$ df -h /home   # Linux
```

Test storage connectivity, either from the UI or from the command line:

$ cd /home/stor2rrd/stor2rrd
$ ./bin/config_check.sh 
  =====================================
  STORAGE: Lenovo-DM-01 : NetApp
  =====================================
  connection ok

Schedule to run storage agent from stor2rrd crontab (lpar2rrd on Virtual Appliance, it might already exist there)

$ crontab -l | grep load_netappperf.sh
$

Add if it does not exist as above

$ crontab -e

# NetApp FAS : C-mode, 7-mode, Lenovo ThinkSystem DM / DG Series, REST API (same code, use it just once)
0,5,10,15,20,25,30,35,40,45,50,55 * * * * /home/stor2rrd/stor2rrd/load_netappperf.sh > /home/stor2rrd/stor2rrd/load_netappperf.out 2>&1

Assure there is already an entry with the UI creation running once an hour in crontab

$ crontab -e

# STOR2RRD UI (just ONE entry of load.sh must be there)
5 * * * * /home/stor2rrd/stor2rrd/load.sh > /home/stor2rrd/stor2rrd/load.out 2>&1

Let run the storage agent for 15 - 20 minutes to get data, then:
```
$ cd /home/stor2rrd/stor2rrd
$ ./load.sh
```
Go to the web UI: http://<your web server>/stor2rrd/
Use Ctrl-F5 to refresh the web browser cache.