log_monitoring_service AC Configuration

The log_monitoring_service probe periodically queries log data that is stored in CA Analytics data store (Jarvis) and raises notifications based on predefined queries. The Monitoring Service queries Jarvis at the predefined schedule and provides the following output:
uimpga-ga
log_monitoring_service_AC
The log_monitoring_service probe periodically queries log data that is stored in CA Analytics data store (Jarvis) and raises notifications based on predefined queries. The Monitoring Service queries Jarvis at the predefined schedule and provides the following output:
  • Match_Count metric for the count of matches found
  • Alarm if the match count exceeds a predefined threshold
  • Alarms containing sample matched logs lines (number of sample lines configurable)
You can create one or more profiles. Each profile includes a query to be executed for a particular log type and interval. For example, "response_time:[10 TO *] AND url:*ServiceDesk*" for apache access logs scheduled every 5 minutes. You can also forward the Log Monitoring Service alarms as email or SNMP notifications using the UIM Email Gateway (emailgtw) and SNMP Gateway (snmpgtw) probes respectively.
 
Contents:
 
 
 
Verify Prerequisites
Configure General Properties
You can configure the following properties of the probe:
 
Follow these steps:
 
  1. Open the 
    log_monitoring_service
     node. The Probe Information section provides information about the probe name, probe version, start time of the probe, and the probe vendor. 
  2. In the 
    Probe Setup
     section, update the following information to configure the log properties. The log file of the probe (excluding the system log files) contains information about the activity of the probe itself.
    •  
      Tenant ID.
       A unique Tenant ID provided during the Agile Operations Analytics Base Platform on-boarding process. You can configure the tenant ID at the profile level too. The setup level Tenant ID is used if the profile level Tenant ID is not provided. During the probe upgrade, a blank field is added for the profile level tenant ID field  to all the existing profiles.
    •  
      Log Level.
       Specifies the level of details that are written to the log file. Log as little as possible during normal operation to minimize disk consumption, and increase the amount of detail when debugging. Default: 3 - INFO
    •  
      Log Size (KBytes).
       Specifies the size of the log file where the internal log messages of the probe are written, in kilobytes. When this size is reached, new log file entries are added and the older entries are deleted. Default: 2000
  3. In the 
    Elasticsearch Configuration
     section, update the following information to configure AXA Elasticsearch:
    •  
      Host.
       Hostname of Elasticsearch. On a single node setup, the host name is same as the Agile Operations Analytics Base Platform server hostname.
    •  
      Port.
       Port of Elasticsearch. Default: 9200
    •  
      Health Interval (In minutes)
      . Frequency at which probe checks for connection status with AXA Elasticsearch. Default: 5 minutes
  4. Click 
    Save
Create a Profile
Create a configuration profile to query and monitor log file data stored in Elasticsearch. The probe periodically queries Elasticsearch to fetch log data based on the specified query string and generates Alarms based on the query output.
 
Follow these steps:
 
  1. Select a group from the navigation pane.
  2. Click the 
    Options (...)
     and select + 
    Add New Profile
    .
  3. Enter the following information in the + 
    Add New Profile
     section:
    •  
      Profile Name.
       Name of the profile.
    •  
      Active.
       Activates the monitoring profile.
    •  
      Tenant ID
      . Tenant ID for Agile Operations Analytics – Base Platform. The setup level Tenant ID would be used if the profile level Tenant is not provided.
    •  
      Check Interval (seconds).
       The time interval at which the profile runs. Default: 300 seconds
    •  
      Search Time Window (seconds
       
      ).
       
    •  
      Data Category
      Values: 
      all, alarms, events, inventory, logs, metrics
    •  
      Data Type.
       Select the blank option to search data over the entire data category.
      Values:
       alarms_spectrum, alarms_uim, all, all alarms, all events, all inventory, all logs, all metrics, blank, evets_spectrum, inventory_spectrum, inventory_uim, logs_apache_access, logs_apache_error, logs_aws_cloudtrail, logs_docker, logs_eventlog, logs_generic, logs_iis, logs_log4j, logs_nginx_access, logs_oracle_alert, logs_oracle_audit, logs_sqlserver, logs-syslog, logs_tomcat, logs_tomcataccess, logs_zos_syslog, metrics_spectrum_availability, metrics_spectrum_devicecount, metrics-uim
    •  
      Search String. 
      A phrase or a query to search the log data. For example,
      • Enter Exception to find all logs that contain the word Exception.
      • Enter request:*servicedesk* AND response_code: [500 TO 599] in the Web Server logs to find all requests with the server error codes.
        In this example 
         
        request
         
         and 
         
        response_code
         
         are fields for a specific log type. You can look at the available fields for any log type in the AXA DataStudio Discover tab (after selecting the appropriate index pattern for the log type). The query syntax is same as you use in AXA DataStudio while doing a search. It uses the Elasticsearch (internally Lucene) query syntax. For more information, see .
         
    •  
      Send Alarm on Each Match.
       If selected, the probe generates a separate alarm for each match found in the log data. Default: Selected
      •  
        Match Alarm Message.
        Default: 
        Match found for $profileName search string $query in message: $result
      •  
        Match Alarm Severity
        Values: 
        CRITICAL, INFORMATION, MAJOR, MINOR, WARNING
    •  
      Set Maximum Alarms Limit section:
       Controls the maximum number of match alarms to be sent if 
       
      Send Alarm on Each Match
       
       is selected.
      •  
        Maximum Alarm Limit:
         Maximum number of match alarms to be sent. Default: 5
      •  
        Publish Alarms on Max Limit:
         Probe generates an alarm when the maximum alarm limit is reached. That is, when more matches are found in the log data than the maximum alarm limit.
      •  
        Maximum Alarm Message:
         Alarm text for Max Limit alarm. Default:  Maximum alarm reached for profile $profileName.
  4. Click 
    Submit
    .
  5. After the profile is saved, you can configure the 
    Set Maximum Alarms Limit
     section. This section controls the maximum number of match alarms to be sent if 
     
    Send Alarm on Each Match
     
     is selected.
    •  
      Description:
       Limits the maximum number of alarms sent on the string match.
    •  
      Metric Type:
       
    •  
      Unit:
       
    •  
      Maximum Alarm Limit:
       Maximum number of match alarms to be sent. Default: 5
    •  
      Publish Alarms on Max Limit:
       Probe generates an alarm when the maximum alarm limit is reached. That is, when more matches are found in the log data than the maximum alarm limit.
    •  
      Maximum Alarm Message:
       Alarm text for Max Limit alarm. Default: Maximum alarm reached for profile $profileName.
Deploy Log Monitoring Service on the Secondary Hub
To post alarms from the Secondary hub to the Primary hub, you must create a tunnel from Secondary hub to Primary hub.
 
Follow these steps:
 
  1. Create an Alarm queue on the Primary Hub.
    1. Go to Primary hub of the primary robot and double-click on the 
      hub
       probe.
    2. Click on the 
      Queues
       tab and create a queue named 
      alarm.
       
    3. Select 
      get
       as the type. Set the 
      Name
       and 
      Subject
       as alarm.
    4. Select the Address of the robot in dropdown menu.
    5. Perform the same steps to create another queue named 
      alarm2.
       
  2. Create an Alarm queue on the Secondary Hub.
    1. Go to Secondary hub of the primary robot and double click on the 
      hub
       probe.
    2. Click on the 
      Queues
       tab and create a queue named
       alarm.
       
    3. Select 
      attach
       as the type. Set the 
      Name
       and 
      Subject
       as alarm.
    4. Perform the same steps to create another queue named 
      alarm2.