Configure Enterprise Managers and Clusters

In a clustered environment, an Enterprise Manager serves as a Manager of Managers (MOM), managing the other Enterprise Managers in the cluster. The managed Enterprise Managers are named Collectors
apmdevops106
In a clustered environment, an Enterprise Manager serves as a Manager of Managers (MOM), managing the other Enterprise Managers in the cluster. The managed Enterprise Managers are named Collectors
Workstation users logged in to a MOM can do the following:
  • See and access the data reported by the agents that report to each of the Collectors that the MOM manages.
  • Perform all usual Workstation functions on those Collectors.
An optional Enterprise Manager serves as a Cross-cluster Data Viewer (CDV), which can connect to Collectors across clusters.
Workstation users who are logged in to a CDV can do the following:
  • See and access dashboards and the agent and customer experience metric data in the metric browser tree. This data is reported to each of the Collectors to which the CDV is connected.
  • Perform a subset of the usual Workstation functions on those Collectors including these capabilities:
    • Using the Management Module editor to create dashboards.
    • Run a Transaction Trace.
    • Use a dynamic instrumentation.
To configure Enterprise Managers and clusters for your APM environment, perform these steps:
Clustering is useful in large Introscope environments that consist of multiple Enterprise Managers and many agents -- a cluster configuration reduces the complexity and cost of managing a multiple-Enterprise Manager environment.
MOMs Manage Collectors
When you start a MOM, consider the following information:
  • The MOM attempts to establish a connection with each of the collectors it manages.
  • Every agent in the Investigator automatically has an Enterprise Manager host metric. This metric is helpful when running a cluster, so you can tell to which collector Enterprise Manager the agent is connected.
  • The MOM uses a built-in, preconfigured user account for connecting to collectors. The password is protected using public/private key encryption.
  • When the initial attempt of MOM to connect to one of its collectors results in an unrecoverable error, the collector is removed from the cluster. The MOM stops trying to connect to that collector until the next restart of the MOM.
  • For recoverable errors, such as a lost connection, or a java.rmi.ConnectionException, the MOM repeats the connection attempt every 10 seconds.
Logging of connection status occurs as follows:
  • If the logging level is DEBUG, a stack trace of the connection failure is written to the log.
  • If the logging level is VERBOSE, a message is written to the log for the first ten consecutive failed attempts.
  • A message is always written to the log when the MOM connects or reconnects to a collector.
As long as it remains running, the MOM polls the collectors that it manages.
For more information about public/private key encryption, see the Configure Enterprise Manager.
Clustering and Enterprise Manager Clock Synchronization
In an Enterprise Manager cluster, collector system clocks must be within 3 seconds of the MOM clock setting. If at startup, the clock of a collector is not synchronized with the clock of a MOM, the collector fails to join the cluster. During an operation, if a previously synchronized clock of a collector becomes more than 3 seconds off, the MOM no longer accepts data.
The clock synchronization among the MOM and the collector Enterprise Managers is not automatic.
To keep the clocks synchronized, we recommend that all Enterprise Managers synchronize their clocks at regular intervals to an NTP server. If a collector clock becomes more than 3 seconds off from the clock of a MOM, error messages appear in the log. The data is not collected. If this situation occurs, increase the frequency of synchronization with the NTP server. No restart of Enterprise Managers is necessary after clock synchronization. The MOM automatically reconnects to the Collector when it detects the clocks are synchronized.
To synchronize the clocks of all servers in the cluster, run the time server software at regular intervals.
Clustering and Agent Naming
Although each Enterprise Manager ensures that the agents have a unique name, you must ensure that the agents across the collectors in the cluster have unique names. If more than one agent in the cluster has the same name, only the last agent encountered by the MOM appears in Workstation views. Configuring the agents to obtain their names from the application server is a technique for assigning unique agent names. For more information about assigning unique agent names, see Java Agent or Microsoft .NET Agent.
A collector automatically renames custom metric agents, adding a unique identifier to the agent name to identify the collector of an agent. The default identifier has the form ([email protected]), where "host" is the hostname of the collector and "port" is the connection port of the collector. The identifier is defined in the introscope.enterprisemanager.port.<
ChannelName
> property.
Alternatively, the identifier can be set using the introscope.enterprisemanager.clustering.collector.identifier property. The identifier is unique. For example, the name of a custom metric agent in the Super Domain in a collector on
myhost
that listens on port 5001 is:
Custom Metric Agent ([email protected]) (*SuperDomain*)
Clustering and Management Modules
In an Enterprise Manager cluster, only the Management Modules that are installed on the MOM are used. For performance reasons, no Management Modules resides on the Collector Enterprise Managers.
Deploying Extension Files in a Cluster Environment
If you have activated extensions in your CA APM deployment, extension files must be administered in a certain way when deployed in a clustered environment.
Follow these steps:
  1. Locate the extension folder where it has been installed in
    <EM_Home>/
    examples/<
    Extension_Home>
    , where
    <Extension_Home>
    is the name of an extension.
  2. Copy the entire contents of this folder to the
    <EM_Home>/
    location on the MOM.
  3. Copy the entire contents of this folder, except the directory
    <EM_Home>/
    examples/
    <Extension_Home>
    /config, to the
    <EM_Home>/
    location on each Collector Enterprise Manager.
    The
    <EM_Home>
    /examples/
    <Extension_Home>
    /config directory is not copied because Management Module
    .jar
    files must only reside on the MOM.