Upgrade the Data Repository

Upgrade the data repository if you are upgrading from
DX NetOps Performance Management
3.7, 20.2, or 21.2.2 and lower to
DX NetOps Performance Management
21.2.3 or higher.
For more information, see New Features and Enhancements.
Use the following process to upgrade the data repository:
Verify the Prerequisites
Before you upgrade the data repository, ensure that you have completed the following prerequisite steps:
  • To prevent the validation and installation scripts from failing, you have verified that the
    dialog
    ,
    chrony
    ,
    zip
    , and
    unzip
    packages are installed on each data repository host by issuing the following command:
    rpm -qa | grep ^dialog
    rpm -qa | grep ^chrony
    rpm -qa | grep ^zip
    rpm -qa | grep ^unzip
    For more information about the packages that the installer requires for the data repository, see Installation Requirements and Considerations.
    If a package is missing, you have installed the package by issuing the following command:
    If you are not the root user, use the sudo prefix.
    yum -y install dialog
    yum -y install chrony
    yum -y install zip
    yum -y install unzip
    SLES:
    zypper install dialog -y
    zypper install zip -y
    zypper install unzip -y
  • You have verified that the data repository host has at least 2 GB of swap space.
  • You have verified that the data repository hosts use the ext4 file system for data and catalog directories. If they are not, remake the file system as ext4.
    The default file system for RHEL 7.x and OL 7.x is the XFS file system. The default file system for SLES is btrfs. Vertica does not support XFS or btrfs. The database supports only the ext4 file system.
  • You have verified that you are not using Logical Volume Manager (LVM) for data and catalog directories.
Verify Hardware and Network Performance
Verify that the environment is ideal for the database by testing the performance of your hardware for the data repository.
For more information, see Run Data Repository Diagnostic Utilities.
Verify the Limit on the Number of Open Files
Verify that the user that is installing data repository has a value of at least 65536 on the number of open files. Set this value permanently. Complete this procedure for each node in the data repository cluster.
Follow these steps:
  1. As the root or sudo user, log in to the data repository host node.
  2. Issue the following command:
    su dradmin
  3. Verify the number of open files:
    ulimit -n
    The command returns the ulimit number. This number must be at least 65536.
    The number must be the same on all nodes in the cluster.
  4. If this number is not at least 65536, do the following steps:
    1. Change the ulimit for the open files limit to 65536 by issuing the following command:
      ulimit -n 65536
    2. Open the
      /etc/security/limits.conf
      file on each data repository node, and add the following lines:
      # Added by Vertica
      * soft nofile 65536
      # Added by Vertica
      * hard nofile 65536
      # Added by Vertica
      * soft fsize unlimited
      # Added by Vertica
      * hard fsize unlimited
      Restart the sshd service on each data repository node for these changes to take effect:
      service sshd restart
      For RHEL 7.x, and OL 7.x,
      service
      invokes
      systemctl
      . You can use
      systemctl
      instead.
      If you do not have the restart argument, stop and start sshd by issuing the following command:
      service sshd stop
      service sshd start
    3. Verify that the number of open files is set properly by issuing the following command:
      ulimit -n
      The command returns the limit that you specified earlier.
Disable the Automatic Recovery of the Data Aggregator Process
The recovery process restarts the data aggregator. Disable the automatic recovery of the data aggregator process.
Follow these steps:
  1. Log in to the computer where the data aggregator is installed as the root user.
  2. Open a console and issue the following command:
    crontab -e
    A vi session opens.
  3. If the following line exists, comment out the line by adding the pound symbol (#) to the beginning of the line:
    * * * * * /sbin/service dadaemon start > /dev/null
    For example:
    #
    * * * * * /sbin/service dadaemon start > /dev/null
The automatic recovery of the data aggregator process is disabled.
(Fault-tolerant Environment) Put the Data Aggregators into Maintenance Mode
If you are using
DX NetOps Performance Management
in a fault-tolerant environment, prevent the data aggregators from restarting until the data repository upgrade is complete by putting them into maintenance mode.
Log in to each data aggregator host as the root user or a sudo user, and issue
one
of the following commands:
  • RHEL 6.x:
    service dadaemon maintenance
  • RHEL 7.x, SLES, or OL:
    <installation_directory>
    /scripts/dadaemon maintenance
Upgrade the Database
You can initiate the data repository upgrade from any host in the cluster. Upgrade as one of the following users:
Upgrade as Root User with Passwordless SSH Configured
In a cluster installation, initiate the data repository upgrade from any of the hosts that participates in the cluster. The upgrade pushes the required software components to the additional nodes.
Follow these steps:
  1. Log in to
    any
    host in the data repository cluster as the root user.
  2. Determine on which hosts Vertica is running by completing the following steps:
    1. As the database administrator user, open the Vertica Administration Tools utility,
      adminTools
      :
      /opt/vertica/bin/adminTools
    2. Select option
      6 (Configuration Menu)
      .
    3. Select option
      3 (View Database)
      .
    4. Select the database.
    5. Note the IP addresses and database name for use later in this procedure.
    6. Exit the
      adminTools
      utility, and revert to the root user.
  3. Copy the
    installDR.bin
    file locally.
  4. Change permissions for the installation file by issuing the following command:
    chmod u+x installDR.bin
  5. Extract the installation file by issuing the following command:
    ./installDR.bin
  6. Follow the instructions in the console.
  7. When prompted, specify the installation directory to which to extract the associated scripts.
    The default installation directory is
    /opt/CA/IMDataRepository_vertica
    Version
    /
    .
  8. Press the
    Return/Enter
    key on your keyboard twice.
    The following scripts are extracted from the installation file:
    • dr_validate.sh
      This validation script verifies the OS settings and modifies the settings if necessary.
      If this script detects issues that you must resolve before the upgrade, you can restart your system while you resolve the issue.
    • dr_install.sh
      This installation script upgrades the Vertica database.
  9. Copy the
    <installation_directory>
    /drinstall.properties
    file to the location where you extracted the installation scripts:
  10. Change directories to the location where you extracted the installation scripts by issuing the following command:
    cd /opt/CA/IMDataRepository_vertica
    Version
  11. Verify that the following parameters in the
    drinstall.properties
    file reflect upgrade-specific values:
    • DbAdminLinuxUser
      The Linux user that is created to serve as the Vertica database administrator.
      Default:
      dradmin
    • DbAdminLinuxUserHome
      The Vertica database administrator user home directory.
      Default:
      /export/dradmin
    • DbDataDir
      The location of the
      data
      directory.
      Default:
      /data
    • DbCatalogDir
      The location of the
      catalog
      directory.
      Default:
      /catalog
      You can verify the
      data
      and
      catalog
      directories by completing the following steps:
      1. Open the
        /opt/vertica/config/admintools.conf
        file.
      2. Scroll down until you see the
        [Nodes]
        section.
      3. Locate a line that begins with
        v_
        dbname
        _node
        ####
        .
        This line contains the IP address of the node, the location of the catalog directory, and the location of the data directory.
        Example:
        v_drdata_node0001 = 10.42.1.1,/catalog,/data
    • DbHostNames
      The comma-delimited list of hostnames for data repository.
      Default:
      yourhostname1,yourhostname2,yourhostname3
    • DbName
      The database name.
      Default:
      drdata
      Case sensitive:
      Yes
    • DbPwd
      The database password. You can use special characters (except for single quotation marks) in passwords. If the script does not find the
      DbPwd
      property or if it is blank, the script prompts for this information at runtime.
      Default:
      dbpass
    The
    InstallDestination
    parameter is not used. You can safely remove it.
  12. Verify that the data repository is running. On the data repository host, issue the following command:
    /opt/vertica/bin/vsql -U
    dauser
    -w
    dapass
    -c 'select version()'
    The following response is expected:
    version
    ------------------------------------
    Vertica Analytic Database vx.x.x-x
  13. On the data aggregator host, open a command prompt. Do
    one
    of the following steps:
    • (Non-fault-tolerant environment) Stop the Data Aggregator service by issuing the following command:
      service dadaemon stop
    • (Fault-tolerant environments) If the local data aggregator is running, put it into maintenance mode and prevent it from restarting until maintenance is complete by issuing
      one
      the following commands:
      • RHEL 6.x:
        service dadaemon maintenance
      • RHEL 7.x, SLES, or OL:
        <installation_directory>
        /scripts/dadaemon maintenance
  14. Run the validation script with the
    -p
    option, on
    each
    node in the data repository cluster using the same location by issuing the following command:
    ./dr_validate.sh -p drinstall.properties
    You can use the following options with the command:
    • The
      -l
      option: Allows
      localhost
      as the value for the
      DbHostNames
      property.
    • The
      -n
      option: Skips database connectivity checks.
    The script validates the system settings.
  15. Review on-screen output for failures or warnings. You can run the validation script multiple times after you fix any failures or warnings. The validation script might prompt you to reboot. If this script fails, you can restart your system with the previous version to reduce downtime.
  16. Stop the data repository by completing the following steps:
    1. Switch to the database admin user.
    2. Open the
      adminTools
      utility:
      /opt/vertica/bin/adminTools
    3. Select option
      4 (Stop Database)
      .
    4. Select the database, confirm the selection, and provide the password.
    5. Exit the
      adminTools
      utility, and revert to the root user or sudo user.
  17. Run the installation script with the
    -p
    option by issuing the following command:
    ./dr_install.sh -p drinstall.properties
The data repository is upgraded and the unnecessary Vertica processes are disabled. You might be prompted for the database administrator system account (the database administrator user) password.
If this user account is not configured with passwordless SSH, you are prompted, sometimes multiple times, for a password.
Upgrade as Sudo User with Passwordless SSH Configured
You set up the data repository by installing and configuring the Vertica database as the sudo user.
RHEL 6.x does not support this functionality.
Follow these steps:
  1. Log in to
    each
    node in the data repository cluster as the sudo user.
  2. Determine on which hosts Vertica is running by completing the following steps:
    1. As the database administrator user, open the Vertica Administration Tools utility,
      adminTools
      :
      /opt/vertica/bin/adminTools
    2. Select option
      6 (Configuration Menu)
      .
    3. Select option
      3 (View Database)
      .
    4. Select the database.
    5. Note the IP addresses and database name for use later in this procedure.
    6. Exit the
      adminTools
      utility, and revert to the sudo user.
  3. Copy the
    installDR.bin
    file locally.
  4. Change permissions for the installation file by issuing the following command:
    chmod u+x installDR.bin
  5. Extract the installation file by issuing the following command:
    sudo ./installDR.bin
  6. Follow the instructions in the console.
  7. Press the
    Return/Enter
    key on your keyboard twice.
    The following scripts are extracted from the installation file:
    • dr_validate.sh
      This validation script verifies the OS settings and modifies the settings if necessary.
      If this script detects issues that you must resolve before the upgrade, you can restart your system while you resolve the issue.
    • dr_install.sh
      This installation script upgrades the Vertica database.
  8. Copy the
    <installation_directory>
    /drinstall.properties
    file to the location where you extracted the installation scripts:
    The default installation directory is
    /opt/CA/IMDataRepository_vertica
    Version
    /
    .
  9. Change directories to the location where you extracted the installation scripts by issuing the following command:
    cd /opt/CA/IMDataRepository_vertica
    Version
  10. Verify that the following parameters in the
    drinstall.properties
    file reflect upgrade-specific values:
    • DbAdminLinuxUser
      The Linux user that is created to serve as the Vertica database administrator.
      Default
      : dradmin
    • DbAdminLinuxUserHome
      The Vertica database administrator user home directory.
      Default:
      /export/dradmin
    • DbDataDir
      The location of the
      data
      directory.
      Default:
      /data
    • DbCatalogDir
      The location of the
      catalog
      directory.
      Default:
      /catalog
      You can verify the
      data
      and
      catalog
      directories by completing the following steps:
      1. Open the
        /opt/vertica/config/admintools.conf
        file.
      2. Scroll down until you see the
        [Nodes]
        section.
      3. Locate a line that begins with
        v_
        dbname
        _node
        ####
        .
        This line contains the IP address of the node, the location of the catalog directory, and the location of the data directory.
        Example:
        v_drdata_node0001 = 10.42.1.1,/catalog,/data
    • DbHostNames
      The comma-delimited list of hostnames for data repository. List only the local hostname. You add the other nodes in a later step.
      Default:
      yourhostname1,yourhostname2,yourhostname3
    • DbName
      The database name.
      Default:
      drdata
      Case sensitive:
      Yes
    • DbPwd
      The database password. You can use special characters (except for single quotation marks) in passwords. If the script does not find the
      DbPwd
      property or if it is blank, the script prompts for this information at runtime.
      Default:
      dbpass
    The
    InstallDestination
    parameter is not used. You can safely remove it.
  11. Verify that the data repository is running. On the data repository host, issue the following command:
    /opt/vertica/bin/vsql -U
    dauser
    -w
    dapass
    -c 'select version()'
    The following response is expected:
    version
    ------------------------------------
    Vertica Analytic Database vx.x.x-x
  12. On the data aggregator host, open a command prompt. Do
    one
    of the following steps:
    • (Non-fault-tolerant environment) Stop the Data Aggregator service by issuing the following command:
      service dadaemon stop
    • (Fault-tolerant environments) If the local data aggregator is running, put it into maintenance mode and prevent it from restarting until maintenance is complete by issuing
      one
      the following commands:
      • RHEL 6.x:
        service dadaemon maintenance
      • RHEL 7.x, SLES, or OL:
        <installation_directory>
        /scripts/dadaemon maintenance
  13. Run the validation script with the
    -sp
    option by issuing the following command:
    ./dr_validate.sh -sp drinstall.properties
    You can use the following options with the command:
    • The
      -l
      option: Allows
      localhost
      as the value for the
      DbHostNames
      property.
    • The
      -n
      option: Skips database connectivity checks.
    The script validates the system settings.
  14. Review on-screen output for failures or warnings. You can run the validation script multiple times after you fix any failures or warnings. The validation script might prompt you to reboot. If this script fails, you can restart your system with the previous version to reduce downtime.
  15. Stop the data repository by completing the following steps:
    1. Switch to the database admin user.
    2. Open the
      adminTools
      utility:
      /opt/vertica/bin/adminTools
    3. Select option
      4 (Stop Database)
      .
    4. Select the database, confirm the selection, and provide the password.
    5. Exit the
      adminTools
      utility, and revert to the root user or sudo user.
  16. Go to the first node, and edit the
    DbHostNames
    parameter in the
    drinstall.properties
    file to includes all the nodes in the cluster.
  17. Run the installation script with the
    -sp
    option by issuing the following command:
    ./dr_install.sh -sp drinstall.properties
The data repository is upgraded and the unnecessary Vertica processes are disabled. You might be prompted for the database administrator system account (the database administrator user) password.
If this user account is not configured with passwordless SSH, you are prompted, sometimes multiple times, for a password.
Verify the Database Upgrade
Verify that the installation script has upgraded the data repository successfully. Use
adminTools
.
Follow these steps:
  1. Log in to the database server as the database administrator (dradmin) user by issuing the following command:
    su - dradmin
  2. Open the
    adminTools
    utility:
    /opt/vertica/bin/adminTools
  3. Open
    adminTools
    from the
    /opt/vertica/bin/adminTools
    directory.
  4. Verify that the top of the banner indicates that the database version is correct.
    NetOps Portal
    21.2.1 and 21.2.2 include Vertica 9.1.1-5.
    NetOps Portal
    21.2.3 and higher includes Vertica 10.1.1.
Next Steps
After you have upgraded the data repository, you can do the following: