docker_monitor AC Configuration

This article is for probe versions 1.2 or later.
uimpga-ga
docker_monitor_AC
This article describes the configuration concepts and procedures to set up the Docker Monitoring (docker_monitor) probe. You can create monitoring profiles to locally or remotely access your Docker environment. You can then generate QoS and alarm messages for the health and availability of your hosts, containers, images, and volumes.
This article is for probe versions 1.2 or later.
The following diagram outlines the process to configure the probe.
Configure docker_monitor through AC
Configure docker_monitor through AC
Contents
3
Verify Prerequisites
Verify that the required hardware and software are available before you configure the probe. For more information, see docker_monitor (Docker Monitoring) Release Notes - Pre-release.
(Optional) Configure Logging Properties
The probe is deployed with default logging configuration that you can change, as applicable.
Follow these steps:
  1. Select the
    docker_monitor
    node.
    The
    Probe Information
    section provides information about the probe name, probe version, start time of the probe, and the probe vendor.
  2. In the
    Probe Setup
    section >
    Log level
    field, specify the level of details that are written to the log file. You can select the following log levels:
    • 0 - Logs only severe information
    • 1 - Logs error information
    • 2 - Logs warning information
    • 3 - Logs general information (default)
    • 4 - Logs debugging information
    • 5 - Logs tracing/low-level debugging information
      Log as little as possible during normal operation to minimize disk consumption, and increase the amount of detail when debugging.
  3. In the
    Log size
    field, specify the maximum size of the probe log file.
  4. Click
    Save
    .
Create a Profile
Create at least one profile to monitor each Docker Engine. Each profile uses a connection to the Docker Engine API for the probe to collect and store monitoring information. You can connect to your environment as follows:
  • Local:
    uses a socket connection to the Docker Engine API
  • Remote:
    uses either TLS or non-TLS TCP connection to the Docker Engine API
Follow these steps:
  1. Click the
    Options (icon)
    next to the
    docker_monitor
    node and click
    Add New Profile
    .
  2. Complete the following information in the
    Add New Profile
    dialog to specify the identification details of the Docker Engine:
    • Profile Name:
      specifies a unique name for the profile.
    • Connection Type:
      specifies the type of connection to the host system with the Docker API. You can select from the following options:
      • (Default) Unix Domain Socket:
        connects locally to the Docker API.
      • TCP:
        connects remotely to the Docker API.
        From version 1.45, the probe allows secure TCP connection using TLS.
    • API URI
      : specifies the URI for the host system with the Docker API.
      Default: /var/run/docker.sock
      Supported Formats
      • (Recommended) The path of the socket file; for example, /var/run/docker.sock; requires root access to the Docker socket file
      • A URL to an IP address; for example, http://192.168.99.100
      • An IP address; for example, 192.168.99.100; specify a non-loopback address for TCP connections
    • (Optional) Application Port
      : specifies the application port to receive the correlation data.
  3. (TCP connections only)
    Complete the following information to specify the connection parameters of the profile:
    • Port:
      specifies the valid port for TCP connections.
      Default: 4243
  4. Complete the following information to specify the monitoring parameters of the profile:
    • Active
      : activates the profile for monitoring, on creation.
      Default: Selected
    • Interval (seconds)
      : specifies the time interval (in seconds) after which the probe collects the data from the Docker Engine API.
      Default: 300
      Reduce this interval to generate alarms faster. A shorter interval can also increase the system load.
    • Alarm Message
      : specifies the alarm that is generated when the profile does not respond. For example, the profile does not respond if there is a connection failure or inventory update failure.
      Default: ResourceCritical
  5. (Optional - from version 1.45)
    Complete the following information to configure the TLS secured TCP connection properties of the profile:
    • TLS Secure:
      enables the profile to create a secure connection.
    • Certificate Path:
      specifies the path of the
      keystore PKS12 (*.P12)
      certificate file on the robot host system. You can also browse for the file.
    • Keystore Password:
      specifies the password for the certificate file.
  6. Click
    Submit
    .
    The profile appears as the
    Profile name
    node under the
    docker_monitor
    node.
  7. Navigate to the
    Profile name
    node and select
    Actions > Verify Selection
    to verify the connection between the probe and the Docker Engine. The probe displays a dialog with verification information.
  8. Click
    Save
    .
    The probe automatically displays the monitored storage objects as child nodes under the
    Profile name
    node.
    In a large configuration, the probe can take some time before it displays all the components on the interface.
Docker Components for Monitoring
Each profile node is associated with an inventory of various Docker components. Expand and click nodes in a profile to see the inventory components and any associated monitors. Each profile node contains the following nodes:
  • Resource name
    : A node that represents the resource running the docker_monitor probe. Click this node to view available monitors.
  • Detached Configuration: An organizational node that contains information about resources with active monitors where data could not be collected.
  • Hosts: An organizational node for one or more Docker host resources
  • Host name
    : A node that represents a specific Docker resource. Click this node to view available monitors.
  • Containers: An organizational node that contains the Docker images for a specific resource. All containers appear in the probe inventory even if the container is in an inactive state. (From version 1.21, the icon representing the container in the probe inventory indicates the container state.) Each node includes monitors for:
    • CPU
    • Disk
    • Memory
    • Network
    • Processes
  • (Version 1.31 or later)
    Images: An organizational node that contains the Docker images. Expand an image node to see what containers are using the image. Click these nodes to view available monitors.
  • (Version 1.31 or later)
    Volumes: An organizational node that contains the Docker volumes. Expand a volume node to see what containers are using the volume. Each volume node contains monitors for:
    • CPU
    • Disk
    • Memory
    • Network
    • Processes
The following figure shows the probe configuration GUI navigation pane inventory. In this example, the Docker environment is configured as a single host. You might see some differences in your GUI depending on your environment, probe, and Admin Console versions.
docker_monitorProbeGUI.png
Apply Individual Monitoring Parameters for Each Profile with the Probe Configuration Interface
Configure the monitors and generate alarms and QoS messages to view the data in USM. We recommend that you use this procedure only to collect data for a few components. To collect similar types of data for many components, use a template to configure monitoring and alarms. The nodes in the profile tree have monitors for applicable nodes. You can filter through the nodes to configure the monitors for the required component.
Follow these steps:
  1. Navigate to the required node.
  2. Click a node in the resource tree. If necessary, expand one or more nodes to view the monitors for any components.
    The available monitors appear in a table on the details pane.
  3. Select the monitor that you want to modify in the
    Monitors
    section.
  4. Select the
    Publish Data
    and
    Publish Alarms
    fields to retrieve QoS data and generate alarm messages, respectively.
  5. (Status Monitors Only)
    Update the following information to configure the monitoring parameters for status alarms:
    Status monitors have numeric thresholds that correspond to the applicable state. For more information about the thresholds, see docker_monitor Metrics - Pre-release.
    • Value Definition
      : indicates that the probe uses the current value for the alarms.
    • High Operator:
      specifies the operator that is used to set the high alarm threshold for measured value.
      Default: >=
      • High Threshold:
        defines the value of the high alarm threshold.
      • High Message Name:
        defines the alarm message that the probe generates when the monitored value breaches the specified high threshold.
    • Low Operator:
      specifies the operator that is used to set the low alarm threshold for measured value.
      Default: disabled
      • Low Threshold:
        defines the value of the low alarm threshold.
      • Low Message Name:
        defines the alarm message that the probe generates when the monitored value breaches the specified low threshold.
  6. (Numeric Monitors Only)
    Update the following information to configure the monitoring parameters for numeric alarms:
    • Value Definition
      : specifies from the following type of values that the probe uses for the alarms:
      • (Default) Current Value:
        specifies that the probe uses the last measured value for the alarm.
      • Average Value Last n Samples:
        specifies that the probe uses the average of the last
        n
        measured values for the alarm. The value of
        n
        is specified in the
        Number of Samples
        field.
      • Delta Value (Current - Previous):
        specifies that the probe uses difference between the last two measured values for the alarm.
      • Delta Per Second:
        specifies that the probe uses difference between the last two measured values for the alarm for each second of the interval.
        Delta Per Second =
        Delta Value (Current - Previous) / Interval (in seconds)
    • (For Average Calculations) Number Of Samples:
      specifies the number of values that are measured to calculate the average value.
    • Compute Baseline:
      enables thresholds. This option might not be available depending on your CA Unified Infrastructure Management configuration. For more information, see Configuring Alarm Thresholds.
  7. Click
    Save
    .
Apply Consistent Monitoring Parameters Across Multiple Profiles with the Template Editor
You can create templates in the probe to configure multiple profiles on a single instance of the probe with the same monitor configuration. Templates reduce the time that is required to configure the monitors and provide consistent monitoring across all the profiles in your probe. The
Template Editor
interface allows you to create and apply monitoring templates. Expand and click the nodes in a template to find the template components and any associated monitors.
The sections of the profiles that are configured using templates are not available for individual configuration. Clear the
Active
checkbox to deactivate the template on the profile to unlock it. You can also exclude a profile using filter rules.
Each template contains the following types of nodes:
  • Template name
    node:
    Each template is created as a
    Template name
    node. You can activate or deactivate a template using this node.
  • Organizational nodes
    : These nodes allow you to group related components and do not contain any configuration settings.
  • Filter name
    nodes:
    These nodes contain the filter settings and rules.
  • Probe inventory nodes
    : These nodes contain the monitoring configuration settings.
You can specify
Precedence
values in
Template name
and
Filter name
nodes to define the order in which the node configurations are applied. Precedence controls the order of template application. The probe applies a template with precedence one before a template with precedence two. If configurations between two templates overlap, the template with a lower precedence overrides the other template. If the precedence numbers are equal, then the templates are applied in alphabetical order.
The following figure shows an example of the nodes that appear in the Template Editor navigation pane inventory. You might see some differences in your template editor interface depending on your probe and Admin Console versions.
docker_monitorTemplateGUI.png
Apply Default Template with Recommended Monitor Configuration
The default template has the monitors that are configured for the default USM view. You can use these default configurations to start collecting data for your environment.
You can only create a copy of the default template, to use as required.
Follow these steps
:
  1. Open the probe configuration interface.
  2. Click
    Template Editor
    .
    The
    Template Editor -
    <probeName> <Version>
    page is displayed.
  3. Click the
    Options (icon)
    next to the
    Factory Template
    node and click
    Copy
    .
    The
    Copy
    dialog appears.
  4. Enter the name and a description of the template.
  5. Enter a precedence value for the template. The rules for setting the precedence are as follows:
    • A numeric value is set as precedence.
    • The default value is 0 (highest precedence).
    • The precedence is applied on multiple templates. The scenarios are described as follows:
      • When the precedence is different for the templates:
        The precedence of a template decreases as the value increases.
        Example: 1 has higher precedence than 2, and so on.
      • When precedence is same for all templates:
        The precedence works in alphabetical order of template name.
      • When filters are applied on templates:
        The precedence works according to the applied filters. If no filter is applied, the precedence is applied on available templates.
  6. Select the
    Active
    checkbox to activate the template, on creation.
  7. Click
    Submit
    >
    Save
    .
The probe applies the template with the default monitor configurations.
Create a Custom Template
You can create a template to configure multiple existing profiles with the same monitor configuration.
Multiple templates can increase the system load. We recommend you to use template rules within a single template. Rules can apply specific configurations to groups of target profiles in your environment.
Follow these steps:
  1. Click
    Template Editor
    .
    The
    Template Editor -
    <probeName> <Version>
    page appears.
  2. Click the
    Options (icon)
    next to the
    docker_monitor probe
    node.
  3. Click
    Create Template
    .
    The
    Template
    dialog appears.
  4. Specify a name and description for the template.
  5. Enter the precedence value if you want to modify the default precedence setting. The rules for setting the precedence are as follows:
    You can specify Precedence values in
    Template name
    and
    Filter name
    nodes to define the order in which the node configurations are applied. Precedence controls the order of template application. The probe applies a template with precedence one before a template with precedence two. If configurations between two templates overlap, the template with a lower precedence overrides the other template. If the precedence numbers are equal, then the templates are applied in alphabetical order.
    • A numeric value is set as precedence.
    • The default value is 0 (highest precedence).
    • The precedence is applied on multiple templates. The scenarios are described as follows:
      • When the precedence is different for the templates:
        The precedence of a template decreases as the value increases.
        Example: 1 has higher precedence than 2, and so on.
      • When precedence is same for all templates:
        The precedence works in alphabetical order of template name.
      • When filters are applied on templates:
        The precedence works according to the applied filters. If no filter is applied, the precedence is applied on available templates.
  6. Click
    Submit
    to create the template.
    The
    <templateName>
    node appears including all the profile nodes of the probe.
  7. Click the
    <templateName>
    node to activate or deactivate the template.
  8. Click the
    Options (icon)
    next to nodes representing multiple possible values (such as Resource or Hosts monitors) to create filters. Filters allow you to control how the probe applies monitors that are based on attributes of the target device. An
    Auto Filter
    node is available with all nodes that can be filtered.
  9. Create rules for filters, as required. For more information, see
    Create Template Filter Rules
    .
  10. Navigate to the applicable node with the monitors that require configuration.
  11. Select
    Include in Template
    to include and enable configuration of a monitor or section in the template. This checkbox is available for static nodes and within filters for dynamic nodes.
  12. Select the
    Publish Data
    and
    Publish Alarms
    fields to retrieve QoS data and generate alarm messages, respectively.
  13. (Status Monitors Only)
    Update the following information to configure the monitoring parameters:
    Status monitors have numeric thresholds that correspond to the applicable state. For more information about the thresholds, see docker_monitor Metrics - Pre-release.
    • Value Definition
      : indicates that the probe uses the current value for the alarms.
    • High Operator:
      specifies the operator that is used to set the high alarm threshold for measured value.
      Default: >=
      • High Threshold:
        defines the value of the high alarm threshold.
      • High Message Name:
        defines the alarm message that the probe generates when the monitored value breaches the specified high threshold.
    • Low Operator:
      specifies the operator that is used to set the low alarm threshold for measured value.
      Default: disabled
      • Low Threshold:
        defines the value of the low alarm threshold.
      • Low Message Name:
        defines the alarm message that the probe generates when the monitored value breaches the specified low threshold.
  14. (Numeric Monitors Only)
    Update the following information to configure the monitoring parameters for numeric alarms:
    • Value Definition
      : specifies from the following type of values that the probe uses for the alarms:
      • (Default) Current Value:
        specifies that the probe uses the last measured value for the alarm.
      • Average Value Last n Samples:
        specifies that the probe uses the average of the last
        n
        measured values for the alarm. The value of
        n
        is specified in the
        Number of Samples
        field.
      • Delta Value (Current - Previous):
        specifies that the probe uses difference between the last two measured values for the alarm.
      • Delta Per Second:
        specifies that the probe uses difference between the last two measured values for the alarm for each second of the interval.
        Delta Per Second =
        Delta Value (Current - Previous) / Interval (in seconds)
    • (For Average Calculations) Number Of Samples:
      specifies the number of values that are measured to calculate the average value.
    • Compute Baseline:
      enables thresholds. This option might not be available depending on your CA Unified Infrastructure Management configuration. For more information, see Configuring Alarm Thresholds.
  15. Click
    Save
    .
    The template is created and is applied to the applicable profiles at the next probe interval.
    The sections of the profiles that are configured using templates are not available for individual configuration. Clear the
    Active
    checkbox to deactivate the template on the profile to unlock it. You can also exclude a profile using filter rules.
Create Template Filter Rules
Filters contain one or more rules to define the types of components for the template. You can add rules to a filter to group multiple components or can exclude components from monitoring. You can add rules to a filter to identify a configuration for specific components. For example, you can add a rule to apply the monitor
Resource Response Time
to all profiles.
A filter applies to components that match all the defined rules. If you do not add any rules to a filter, the configuration of the filter applies to all the components.
Follow these steps:
  1. Click the
    Filter name
    node to be configured.
  2. Click
    New
    in the
    Rules
    section of a filter.
    A new rule is created.
  3. Specify the condition that the probe uses to match against the label. All the conditions are described as follows:
    • Contains
      : indicates that the label contains the specified value.
    • Does not Contain
      : indicates that the label does not contain the specified value.
    • Ends With
      : indicates that the label ends with the specified value.
    • Equals
      : indicates that the label is the same as the specified value.
    • Not Equals
      : indicates that the label is not the specified value.
    • Regex
      : indicates that the label matches the specified regular expression.
      For more information, see
      Use Regular Expressions in Templates
      .
    • Starts With
      : indicates that the label starts with the specified value.
  4. Specify the value against which the probe matches the label.
  5. Enter a precedence value for the filter.
    The rules for setting the precedence value for filters are same as setting precedence for templates.
  6. Click
    Save
    to save the rule for the filter. The filter is applicable to all instances that match the created rule.
Activate the template for the probe to apply the monitor configuration. When you change the template state to
active
, the probe immediately applies all template configuration, including filters, rules, and monitors.
Use Regular Expressions in Templates
A regular expression (RegEx for short) is a special text string for describing a search pattern. Constructing regular expression and pattern matching requires meta characters. The probe supports Perl Compatible Regular Expression (PCRE) which are enclosed within forward slash (/). For example, the expression /[0-9A-C]/ matches any character in the range 0 through 9 in the target string.
You can also use simple text with some wildcard operators for matching the target string. For example, *test* expression matches the text test in target string.
The following table describes some examples of RegEx and pattern matching for the probe.
Regular expression
Type of regular expression
Explanation
[A-Z]
Standard (PCRE)
Matches any uppercase alpha character
.
Standard (PCRE)
Matches against any character
\d*
Custom
Matches for the name which starts from letter d
View Probe Data
You can view docker_monitor probe data as applicable for your requirements.
Verify that you are viewing the correct information for your CA UIM version.
Docker Unified Dashboard
The Unified Management Portal (UMP) includes a predefined dashboard to view the probe data. The default template in the
Template Editor
interface includes the monitors that are configured for the Unified Dashboard. For detailed descriptions and required data sources, see Docker Unified Dashboards.
CABI for CA UIM Dashboards
The Unified Management Portal (UMP) includes a predefined Docker CABI dashboard to view the probe data. The default template in the
Template Editor
interface includes the monitors that are configured for the dashboard. For detailed descriptions and required data sources, see Docker CABI for CA UIM Dashboards and the Metrics for
CA Business Intelligence Dashboards
section in docker_monitor Metrics - Pre-release.
Docker Groups in USM
You can configure Docker groups in Unified Service Manager (USM) to view Docker metric data. For more information, see Create and Manage Groups in USM.
Follow these steps:
  1. Verify that the probe configuration is complete.
  2. Go to USM and create a group.
  3. Add at least one advanced filter to the group.
  4. Use from the following filters, as applicable:
    Filter
    Description
    docker.CgroupDriver
    Filters using the name of the Cgroup Driver; for example, cgroupfs
    docker.clusterId
    Filters using the unique cluster ID of the docker swarm
    docker.ClusterVersion
    Filters using the cluster version of the docker swarm
    docker.ControlAvailable
    Filters using the availability status of control; possible values are TRUE and FALSE
    docker.HostName
    Filters using the host name of the system with the Docker engine
    docker.KernelVersion
    Filters using the kernel version of the OS with the Docker engine; for example, 4.4.0-66-generic
    docker.NodeID
    Filters using the unique ID of the docker node in a swarm
    docker.NodeState
    Filters using the status of the node in a swarm
    docker.OperatingSystem
    Filters using the OS name of the Docker engine; for example, Ubuntu 16.04.1 LTS
    docker.OSType
    Filters using the OS type of the Docker engine; for example, linux
    docker.ProfileName
    Filters using the name of the profile in the probe
    docker.RemoteManager_IPAddress
    Filters using the IP Address of the Remote Manager
    docker.RemoteManagerNodeID
    Filters using the unique node ID of the Remote Manager
    docker.Role
    Filters using the role of the Docker host system; possible value is docker_monitor
    docker.ServerVersion
    Filters using the version of the docker host system; for example, 1.13.1
    docker.swarm
    Filters using the status of the Docker swarm; possible values are Active and Inactive
    docker.Swarm_Role
    Filters using the role of the node in the Docker swarm; possible values are worker and manager
    IP Address
    Filters using the IP Address of the Docker engine
    userprop.docker
    Filters using the current monitoring status of the probe; possible values are TRUE and FALSE
docker_monitor FAQs
Why do I see “Throwing InterruptedException to allow shutdown” messages in my probe log after I change the configuration on a running probe?
This message is generated when the probe is attempting to process data, and a configuration change causes the probe to restart. For example, the probe might be waiting for a response from the Docker API. This message is most frequently seen when an MCS configuration profile is applied to a running probe, but this message might be generated for any configuration change that restarts the probe. This is not a serious problem and only occurs during the probe configuration process.
What is the fastest way to configure monitors for the docker_monitor probe dashboard in USM?
Create a copy of the Docker Factory Template. Then activate and save the new template. It might take a few minutes for the data to appear in the probe dashboard.
How do I modify or delete a profile in the probe configuration interface?
  • Modify:
    Click a profile node in the probe configuration interface. Go to the details pane, make your changes, and then click
    Save
    .
  • Delete:
    Click a profile node in the probe configuration interface. Select
    Options (icon) > Delete Profile
    .
Why did my configuration settings disappear after I refreshed or reopened the probe configuration interface?
Verify that you clicked
Save
to load your configuration changes in the probe configuration interface.
Can I verify my profile credentials in the docker_monitor probe configuration GUI?
Yes. Go to
docker_monitor >
profile name
and then in the details pane click
Actions > Verify Selection
.
Why is my profile in a pending state?
The probe inventory can take some time to appear in the interface or your profile might contain incorrect information and is unable to connect to the Docker API. You can verify if the probe has successfully made a connection. Click a profile node in the probe configuration interface. Go to the details pane and click
Actions > Verify Selection
. The probe displays a
Request was successful
message when you specify correct details.
Why can't I change a monitor setting in the docker_monitor probe configuration GUI?
Verify that there is not a "This configuration is managed by Templates", message. You cannot modify a configured monitor in the probe configuration GUI once you activate a template.
How can I tell what monitors are included in a docker_monitor template?
Go to
Template Editor > docker_monitor probe >
template name
. Go to the details pane and look at the monitors Included in
Template
table.
Can I modify a docker_monitor template?
Yes. Go to the probe configuration GUI and click
Template Editor
. Verify that the template you want to edit is inactive. Make your changes to the appropriate template item and click
Save
.
How do I stop receiving alerts of a monitored component that I have deleted from the server?
Go to the probe configuration GUI and click
Profile name > Detached Configuration
. Select the appropriate component and click
Options (icon) > Delete
.
How do I delete a probe template or filter?
Verify that the template is inactive, and perform one of the following steps:
  • To delete a template, click the applicable template name in the Template Editor and then click
    Options (icon) > Delete Template
    .
  • To delete a filter, click the applicable filter name in the Template Editor and then click
    Options (icon) > Delete Filter
    .
Why can't I change a monitor setting in the probe configuration interface?
Verify that there is not a
"This configuration is managed by Templates"
message. Once you apply a template, you cannot modify a configured monitor in the probe configuration interface. To configure an individual monitor, remove the monitor from the template for exclude using filter rules.
How can I tell what monitors are included in a template?
Go to
Template Editor > docker_monitor probe >
template name
. Go to the details pane and look at the monitors Included in Template table.
Can I modify a template?
Yes. Go to the probe configuration interface and click
Template Editor
. Verify that the template you want to edit is inactive. Make your changes to the applicable template item and click
Save
.
Can I copy a filter?
Yes. Select the Options (icon) next to the
Filter name
node and select
Copy
.