Add a Node to the Data Repository Cluster

To increase system performance or replace a node, add a node to the Data Repository cluster.
You can increase system performance or replace a node by adding nodes to the data repository cluster.
This process applies to expanding a node that was set up with a hostname. If a node was set up using localhost, the nodes table in Vertica has node_address or node_ip set to 127.0.0.1. To expand a node that was set up using localhost, contact Broadcom Support.
In this article:
For more details about this process, see the Vertica documentation.
Complete the Prerequisites
Before you add the new node to the data repository cluster, complete the prerequisites:
  • Stop the data aggregator by issuing the following command:
    service dadaemon stop
    Keep the data aggregator stopped until you complete the entire process.
  • Prepare the new host for the data repository installation.
    For more information see, Prepare to Install the Data Repository.
  • Back up the data repository.
    For more information, see Back Up the Data Repository.
Prepare the New Host
Before you add the host to the cluster, configure and validate the
drinstall.properties
file.
Follow these steps:
  1. Copy the
    installDR.bin
    file to the
    /tmp
    directory on the new host.
  2. On the new host, extract the installation files from the
    BIN
    file by issuing the following command:
    ./installDR.bin
  3. Configure the
    drinstall.properties
    file.
    The parameters must match the properties file from the original cluster. Copy the
    drinstall.properties
    file from an active node, and modify only the necessary parameters.
    • DbAdminLinuxUser
      The Linux user that is created to serve as the Vertica database administrator.
      Default:
      dradmin
    • DbAdminLinuxUserHome
      The Vertica Linux database administrator user home directory.
      Default:
      /export/dradmin
      This directory is created if the Vertical installer creates the user. Be sure that the directory leading up to the home account already exists on the system. For example, if you are using
      /export/dradmin
      , verify that
      /export
      exists.
    • DbDataDir
      The location of the data directory.
      Default:
      /data
    • DbCatalogDir
      The location of the catalog directory.
      Default:
      /catalog
    • DbHostNames
      The names of the new hosts.
    • DbName
      The database name.
      Default:
      drdata
      Case-sensitive:
      Yes
    • DbPwd
      The database password.
      Default:
      dbpass
      The database password that you define here is used during the installation of data aggregator.
  4. Run the validation script by issuing the following command:
    ./dr_validate.sh -p drinstall.properties
    The validation script might ask you to reboot.
Add a Host to the Cluster
After you have prepared the new host, add the host to the cluster.
Use the same configuration for the new cluster as for the source cluster. For example, the Vertica version, node count, database name, administrator, user, catalog directory, and data directory must be the same as the original data repository.
Follow these steps:
  1. Log in to one of the existing data repository hosts.
  2. To add the host to the cluster, do one of the following tasks:
    • If you have root access, issue the following command and specify the italic values:
      /opt/vertica/sbin/update_vertica --add-hosts
      hostname
      -u
      DbAdminLinuxUser
      -l
      DbAdminLinuxUserHome
      -L
      location
      /resources/vlicense.dat --rpm
      location
      /resources/
      vertica-release.rpm
      -T -S default
      • hostname
        Specify the name of the new host to add to the cluster.
      • DbAdminLinuxUser
        The database administrator user name for the cluster as specified in the properties file.
      • DbAdminLinuxUserHome
        The database administrator user home directory.
      • Location
        The location where you extracted
        installDR.bin
        file
      • vertica-release.rpm
        The current RPM file that exists in the extracted installation directory.
    • If you do not have root access, issue the command as the sudo user:
      export SUDO_USER=root
      /opt/vertica/sbin/update_vertica --add-hosts
      hostname
      -u
      DbAdminLinuxUser
      -l
      DbAdminLinuxUserHome
      -L
      location
      /resources/vlicense.dat --rpm
      location
      /resources/
      vertica-release.rpm
      -T -S default
    The script installs Vertica on the new host and incorporates the host into the cluster.
    If the database administrator user does not already exist, the installation script creates the user. The script prompts you to assign a new password. If the database administrator user exists, but passwordless SSH is not set up, the script prompts for the password to set up.
  3. Ensure that the Vertica database administrator can write to the data and catalog directories by issuing the following commands:
    chown
    DbAdminLinuxUser
    .verticadba
    DbDataDir DbCatalogDir
    chmod 755
    DbDataDir DbCatalogDir
    If expanding from one node, segment the database after expansion is complete.
    For more information, see Segment Database Tables.
Add a Node to the Database
After you add the host to the cluster, add the node to the database. If you are adding multiple nodes to the cluster, prepare all nodes before you add any nodes to the database.
Follow these steps:
  1. Log in to any host in the cluster as the data repository administrator.
  2. Open the Vertica
    adminTools
    utility by issuing the following command:
    /opt/vertica/bin/adminTools
  3. Select
    Advanced Menu
    , and then press the
    Enter
    key on your keyboard.
  4. Select
    Cluster Management
    , and then press the
    Enter
    key on your keyboard.
  5. Select
    Add Host(s)
    , and then press the
    Enter
    key on your keyboard.
  6. Select the database, and then press the
    Enter
    key on your keyboard.
    The console displays a list of unused hosts.
  7. Select the new host, and then press the
    Enter
    key on your keyboard.
  8. Confirm the selection, and follow the instructions in the console.
    The data repository adds the node to the cluster. This process can take a long time.
Create the Buddy Projection
Before you can make an expanded cluster k-safety of 1, you must create the buddy projection for the
migration_status table.
  1. Log into vsql as the
    dradmin
    user.
  2. Issue the following command:
    select export_objects('/tmp/migration_status.sql', '
    SCHEMA
    .migration_status');
    • SCHEMA
      Defines the schema, for example,
      dauser
      .
  3. Edit the
    /tmp/migration_status.sql
    file.
  4. Remove the
    CREATE TABLE
    definition and everything above it.
  5. Edit the
    CREATE PROJECTION
    property as follows:
    1. Change the name from
      migration_status_super
      to
      migration_status_super_b1
    2. After
      ALL NODES
      and before the semi-colon (;) add
      OFFSET 1
    3. Before the
      SELECT MARK_DESIGN_SAFE
      line, add
      SELECT START_REFRESH();
    4. Change
      MARK_DESIGN_KSAFE(0)
      to
      MARK_DESIGN_KSAFE(1)
  6. Save the
    migration_status.sql
    file.
  7. Issue the following command:
    vsql -U dradmin -w
    dradminPassword
    -f /tmp/migration_status.sql
    A prompt should confirm that the refresh background process has begun.
Update the Data Repository Configuration
On the data aggregator host, ensure that the
dbconnection.cfg
file points to the correct data repository hosts.
Follow these steps:
  1. Open the
    <data_aggregator_installation_directory>
    /apache-karaf-<
    version
    >/etc/dbconnection.cfg
    file on the data aggregator host.
  2. Modify the following line in the
    dbconnection.cfg
    file. Modify the line to reference the hostname or IP address of each data repository host:
    dbHostNames=
    dbNode1Hostname
    ,
    dbNode2Hostname
    ,
    dbNode3Hostname
  3. Save the
    dbconnection.cfg
    file.
  4. Do one of the following steps:
    • Start the Data Aggregator service by issuing the following command:
      service dadaemon start
      For RHEL 7.x or OL,
      service
      invokes
      systemctl
      . You can use
      systemctl
      instead.
    • (Fault-tolerant environment) Enable the fault-tolerant data aggregator so that it can start when necessary by issuing
      one
      the following commands:
      • RHEL 6.x:
        service dadaemon activate
      • RHEL 7.x, SLES, or OL:
        <data_aggregator_installation_directory>
        /scripts/dadaemon activate