Connect to an External Hazelcast Datastore

This topic describes how to configure the to use an external in-memory Hazelcast® data store. Such a configuration can improve performance in large data flow scenarios.
gateway10
This topic describes how to configure the
Layer7 API Gateway
to use an external in-memory Hazelcast® data store. Such a configuration can improve performance in large data flow scenarios.
The sample files provided on this page are not for production use and must be customized to your Gateway configuration.
Contents:
2
2
What is Hazelcast?
Hazelcast-IMDG-Logo-Orange_Dark_200px.png
Hazelcast is an in-memory data grid service. In a Hazelcast grid, data is distributed evenly between the nodes of a computer cluster, allowing for horizontal scaling of processing and available storage. Backups are distributed among the nodes to protect against failure of any single node. Hazelcast provides central, predictable scaling of applications through in-memory access to frequently-used data and across an elastically scalable data grid. These techniques reduce the query load on databases and improve speed.
You can configure Hazelcast to run in the following configurations:
  • On-premise
  • Cloud (Amazon Web Services, Microsoft Azure, Cloud Foundry, Google Cloud Platform, OpenShift® )
  • Virtual (VMware)
  • Docker containers
Create a Hazelcast Client Configuration File
Creating the Hazelcast client configuration file is required for all Gateway form factors. The file contains the Hazelcast server address plus additional client configuration.
The Gateway does not support all Hazelcast client configuration parameters. See Supported Hazelcast Client Configuration.
The following is an example of a client-side Hazelcast configuration file
:
<hazelcast-client xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.hazelcast.com/schema/client-config http://www.hazelcast.com/schema/client-config/hazelcast-client-config-3.12.xsd" xmlns="http://www.hazelcast.com/schema/client-config"> <instance-name>gatewayHazelcastClient</instance-name> <network> <cluster-members> <!-- Set your Hazelcast server node addresses here --> <address>hazelcast_host_ip:5701</address> </cluster-members> <connection-attempt-limit>0</connection-attempt-limit> <redo-operation>true</redo-operation> </network> <connection-strategy async-start="false" reconnect-mode="ON" /> </hazelcast-client>
Supported Hazelcast Client Configuration
The
Layer7 API Gateway
relies on the configuration within
hazelcast-client.xml
. However, the
Layer7 API Gateway
requires certain parameter values and does not support all configurations of the Hazelcast client.
Configuration elements apply to all Gateway form factors unless noted.
Configuration Parameters
Example
Description
instance-name
<instance-name>gatewayHazelcastClient</instance-name>
The name of the Hazelcast client. The instance-name is required and used by Gateway code.
connection-strategy
<client-connection-strategy async-start="false" reconnect-mode="ON" />
Two attributes in this configuration specify how a Hazelcast client connects to the Hazelcast cluster at startup and after disconnections.
The attributes must be set to the following values:
async-start
:
false
(default value).
Synchronizes the creation of the Hazelcast client with the establishment of a Hazelcast cluster connection. When async-start is set to false, synchronization is required, which ensures that a client/server Hazelcast connection exists before the Gateway starts.
reconnect-mode
:
ON
Reconnection attempts are allowed.
The ASYNC value is not supported and throws exceptions. The OFF value prevents reconnection attempts.
connection-attempt-limit
<connection-attempt-limit>0</connection-attempt-limit>
Defines the number of connection attempts the Hazelcast client makes for any connection scenario. Connection attempts include both startup and reconnection attempts if the client disconnects from the Hazelcast cluster. If a client/server connection is not established with all Hazelcast server nodes within the attempt limit, then the Hazelcast client shuts down.
Recommended value: 0
(zero is used for max number of attempts, equivalent to 2147483647)
Hazelcast documentation recommends tuning connection attempt parameters to define client behavior for Hazelcast cluster failures.
Related configuration for connection-timeout has a default of 5000 ms, which is reasonable. It represents how long a Hazelcast client waits for a server node to respond to its connection request, after which it marks the request as a failed connection attempt.
connection-timeout
<connection-timeout>5000<connection-timeout>
The amount of time a Hazelcast client waits for a server node to respond to its connection request. When the connection-timeout value is exceeded, the request is considered a failed connection attempt. Value is represented in milliseconds.
smart-routing
<smart-routing>true<smart-routing>
The default setting of
true
results in better performance.
This parameter is supported only by Container Gateways on OpenShift. The Hazelcast service and the Gateways must be in the same OpenShift environment.
Smart routing refers to the Hazelcast client being aware of all Hazelcast server nodes, and then sending its requests to the best Hazelcast node (for example, the node owning the data).
redo-operation
<redo-operation>true</redo-operation>
Hazelcast client requests can fail for any number of reasons. Read-only operations are retried by default. If set to
true
, then all operations (read-only and non-read-only) are retried.
Default:
false
near-cache
Not supported.
The
near-cache
setting configures Hazelcast clients to cache data locally on the client.
WARNING:
This configuration is not supported by the Gateway and should not be present. The Gateway will not start up if this setting is present.
Enable External Hazelcast Storage for Other Gateway Form Factors
To configure the all other form factors of the Gateway:
  1. Create the hazelcast-client.xml file locally. See Create a Hazelcast Client Configuration File.
  2. Access the privileged shell.
  3. Navigate to this directory:
    /opt/SecureSpan/Gateway/node/default/etc
  4. Create the following folder structure:
    # mkdir -p bootstrap/assertions/ExternalHazelcastSharedStateProviderAssertion
  5. Place the Hazelcast client configuration file named
    hazelcast-client.xml
    in the ExternalHazelcastSharedStateProviderAssertion directory. The name is case sensitive.
  6. Run this command to set proper owner of the directory and files:
    # chown -R layer7:gateway /opt/SecureSpan/Gateway/node/default/etc/bootstrap
  7. Run this command to set the appropriate permission for the directory:
    # chmod -R 2750 /opt/SecureSpan/Gateway/node/default/etc/bootstrap/
  8. Run this command to set the appropriate permissions for the file:
    # chmod 644 /opt/SecureSpan/Gateway/node/default/etc/bootstrap/assertions/ExternalHazelcastSharedStateProviderAssertion/hazelcast-client.xml
  9. Set the following properties to
    externalhazelcast
    (case sensitive):
    • com.l7tech.server.extension.sharedKeyValueStoreProvider
    • com.l7tech.server.extension.sharedCounterProvider
    • com.l7tech.server.extension.sharedClusterInfoProvider
      Switching between providers will not migrate existing data to the newly configured provider.
  10. Restart the Gateway to connect to the External Hazelcast Storage.