Configuring Cluster Database Replication

The gateway cluster depends on a reliable MySQL database. The standard mechanism uses the MASTER-MASTER replication, in which each node is a SLAVE of the other node. Any changes to either database node are automatically replayed in the other database. This ensures that an up-to-date database is always available for the processing nodes of the Gateway cluster. In this configuration, one node is considered the “Primary” database, while the other node the “Secondary” (or “Failover”) node.
gateway10
The
Layer7 API Gateway
cluster depends on a reliable MySQL database. The standard mechanism uses the MASTER-MASTER replication, in which each node is a SLAVE of the other node. Any changes to either database node are automatically replayed in the other database. This ensures that an up-to-date database is always available for the processing nodes of the Gateway cluster. In this configuration, one node is considered the “Primary” database, while the other node the “Secondary” (or “Failover”) node.
Database replication applies to Appliance Gateways only. It does not apply to Software Gateways.
During normal operation, all Gateway processing devices connect to the Primary database. If the Primary database fails, then all nodes switch to the Secondary database if steps to configure the cluster database failover are followed.
Regardless of the number of Gateway processing nodes in a cluster, a maximum of two MySQL database servers can be configured for database replication in a Gateway cluster (for example, "DBServer1" and "DBServer2"). Each peered database unit becomes both a slave and master to the other unit. The Gateway cluster uses one as the Primary database node and then fails over to the Secondary database node in case of problems.
Background replication between the database nodes occurs constantly, with updates to the Secondary Node happening milliseconds after the primary node. If the Primary Node fails, then the Gateway processing nodes invalidates previous connections to that database node before automatically connecting to the Secondary Node.
Configure replication before configuring any Gateway nodes. If you have already configured a primary node before replication is set up, do the following steps to recover before configuring replication:
  1. You do not need to configure the secondary node as a standalone node. If it is already configured, then delete the Gateway using option
    2
    ("Display Layer7 API Gateway configuration menu") and option
    5
    ("Delete the Layer7 API Gateway") to delete the database as well.
  2. Return to the Gateway main menu and then select option
    2
    ("Display
    Layer7 API Gateway
    configuration menu"), then option
    3
    ("Configure the
    Layer7 API Gateway
    "). See Gateway Configuration Menu (Appliance). Point the database to the primary node and ensure that the secondary database is set up.
Contents:
2
2
Prerequisites
Ensure that you have:
  • Host names for DBServer1 and DBServer2 are configured in DNS or "/etc/hosts"; you may use IP addresses instead of hostnames
  • MySQL service for DBServer1 and DBServer2 is running
  • Both Gateway services are stopped
  • Time synchronization is configured for all the Gateway nodes
Replication Flow
When a change is made to a MASTER database, the replication configuration replays the change in the SLAVE database. The following diagram illustrates the replay mechanism.
The diagram only describes one "direction" of the MASTER-MASTER replication story. Full replication includes an identical flow in the reverse direction.
Replication Flow
Replication Flow
Notes:
  1. A change (INSERT, UPDATE, ALTER...) is made to the MASTER database (DB1).
  2. The MASTER database sends this change to the MySQL binlog (binary log) where it is recorded as an "event".
  3. The replication SLAVE I/O thread connects to the MASTER, reads the event from the MASTER binlog, then copies the event to the SLAVE relay log.
  4. The SQL thread on SLAVE reads the event in the SLAVE relay log and applies the changes to the SLAVE database (DB2).
How to Configure Replication
  1. Run the following script against the local database on each node of the cluster:
    # /opt/SecureSpan/Appliance/bin/add_slave_user.sh
    When executed, the system shall check the Gateway configuration for running MySQL:
    Checking configuration of running MySQL... MySQL appears to be properly configured with server_id=1 Do you want to continue? [Y]
    Enter
    Y
    to proceed.
    The
    add_slave_user.sh
    script grants slave permissions for users to MySQL.
    Enabling replication requires changes to the
    /etc/my.cnf
    file. There are four lines in the file that pertain to replication, all of which are commented out in a newly-installed
    Layer7 API Gateway
    . When you run the
    add_slave_user.sh
    script, it enables these lines and restarts the MySQL daemon.
    The four lines in question are grouped within the
    /etc/my.cnf
    file on the Gateway appliance:
    # Uncommment log-bin, log_slave_update and log_bin_trust_function_creators # if a clustered db server log-bin=/var/lib/mysql/ssgbin-log log_bin_trust_function_creators=1 log_slave_update # uncomment the next item on 1st db master server server-id=1 # uncomment the next item on 2nd db master servers # server-id=2
    Each line is described below:
    • log-bin
      sets up the location of the binary replay log on a MASTER system
    • log_bin_trust_function_creators
      controls whether stored function creators can be trusted not to create stored functions that causes unsafe events to be written to the binary log
    • log_slave_update
      instructs the SLAVE to create its own binary replay log for all changes made if a slave chain is configured (A → B → C)
    • server-id
      is set to a unique identifier for the node in the replication scenario
    For more information, refer to the MySQL documentation at: http://dev.mysql.com/doc/refman/8.0/en/
  2. Complete the following prompts in the script:
    1. Enter hostname or IP for the <target>:
      Enter the hostname for the target machine being configured. In almost all cases, this is the other peer node. For example, if running the
      add_slave_user.sh
      script on the primary node, you enter the hostname of the secondary node.
      Note:
      Hostnames are preferable to prevent DNS resolution issues. Use an IP address only if the hostname is not known.
    2. Enter replication user:
      Enter the user account in the MySQL database that is used for replication. The default username is
      repluser
      .
    3. Enter replication password:
      Enter the password for the replication user.
    4. Enter MySQL root user:
      Enter the user account for the root MySQL user.
    5. Enter MySQL root password:
      Enter the user account for the root MySQL password.
    6. Is this the Primary (1) or Secondary (2) database node?
      Enter
      1
      or
      2
      to indicate the type of node.
    A message confirms that MySQL is properly configured for replication and that slave permissions have been granted.
  3. Stop the slave on the primary node before performing this step:
    # mysqladmin stop-slave
    Not performing this step may wipe the Gateway database from the primary node after running
    create_slave.sh
    (next step).
  4. Run the following script against each database node of the cluster:
    # /opt/SecureSpan/Appliance/bin/create_slave.sh
    The
    create_slave.sh
    script sets up the replication to run between the two databases, using the user configured in
    add_slave_user.sh.
    The
    create_slave.sh
    script is always run against the other node in a two-node database cluster. In other words, you are setting up the other node as the MASTER. Since this script is run on both nodes, each node gets the other one set up as its master, thus creating the MASTER-SLAVE relationship.
  5. Complete the following prompts in the script:
    1. Enter hostname or IP for the MASTER:
      Enter the hostname for the target machine being configured. In almost all cases, this is the other peer node. For example, if running the
      create_slave_user.sh
      script on the primary node, you would enter the hostname of the secondary node.
      Note:
      Hostnames are preferable to prevent DNS resolution issues. Use an IP address only if the hostname is not known.
    2. Enter replication user:
      Enter the account that is used in MySQL for replication. This should be the same user as entered when configuring the replication user in the first script (
      add_slave_user.sh)
      .
    3. Enter replication password:
      Enter the password for the replication user.
    4. Enter MySQL root user:
      Enter the user account for the root MySQL user.
    5. Enter MySQL root password:
      Enter the user account for the root MySQL password.
    6. Do you want to clone a database?
      • If you have a pre-existing database and wish to keep its contents, enter
        yes
        and then enter the name of the database to clone. Be sure the slave is not currently running on the MASTER.
      • If you are following these steps for the first time or if you want to discard the existing database and create a new one, enter
        no
        .
    This script then clones the database and starts the SLAVE.
  6. Run the following command on the primary node to start replication on the replica server:
    # mysqladmin start-slave
  7. Verify replication status on all nodes by following the steps outlined in Check Replication Status.