logmon Advanced IM Configuration

This article describes the advanced configuration concepts and procedures to set up the Log Monitoring (logmon) probe.
uimpga-ga
This article describes the advanced configuration concepts and procedures to set up the Log Monitoring (logmon) probe.
 
Contents
 
 
 
Create Format Rules
To identify a required text string in a record-oriented (multiple lines) log file, you can configure a format rule. The rule uses start and end expressions to identify strings in the monitored log file.
You can then define a watcher to identify the required text string in the block. If more than one format rule is defined, the format rules run sequentially. The first format rule identifies a block of text then next format rule runs from the line where the first format rule was ended. If one format rule scans through the entire log file but is unable to find the pattern, then the subsequent format rules do not run.
Format rules are not applicable for 
URL
 mode.
 
Follow these steps: 
 
  1. Select the required profile and open the 
    Format Rules
     tab.
  2. Right-click and select 
    New
    .
    The 
    Add a New Format Definition
     dialog appears. 
  3. Enter a name for the format rule.
  4. Click 
    OK
    .
    The format rule is created.
  5. Specify a start expression or select one from the 
    Start Expression
     drop-down list. The start expression defines the starting point from where the probe begins to monitor the log file.
  6. Set or modify one of the following fields to configure the end rule:
    •  
      End Expression: 
      allows you to specify a end expression or select one from the 
      End Expression
       drop-down list. The end expression defines the ending point when the probe stops monitoring the log file.
      (From logmon 3.70)
       You can use a positive lookahead regex in the 
      End Expression
       field where the the last line of the record is not defined, and the next line after the record is defined. next line For example, the next line after the record starts with Heartbeat or heartbeat. Add /(?=[Hh]eartbeat)/ in the 
      End Expression
       field. When the word Heartbeat or heartbeat is found, the probe excludes the line starting with Heartbeat or heartbeat, and only includes the previous line.
       If the probe does not find the end point in the file within the configured number of scan intervals, it matches the end expression to an empty line.
    •  
      End After:
       defines a fixed number of lines to specify the end of the log-file entry.
    The end rule defines the point when the probe stops monitoring the log file. 
  7. Select the checkbox next the rule to enable the format rule.
  8. Click 
    Apply
     to save the configuration.
Create Exclude Rules
You can configure exclude rules to define text or expressions to be excluded for monitoring. The probe ignores these text blocks or lines where the excluded text or expression exists. You can define more than one exclude rules.
 You cannot create exclude rules for URL mode profiles.
 
Follow these steps: 
 
  1. Select the required profile.
  2. Open the 
    Exclude Rules
     tab.
  3. Right-click and select 
    New
    .
    The 
    Add New Exclude Definition
     dialog appears. 
  4. Enter a name for the rule.
  5. Click 
    OK
    .
    The exclude rule is created.
  6. Specify text or expression to exclude lines or blocks of input from monitoring.
     Each block or line of input is checked to see if the expression entered in the 
    Exclude Expression
     field is found. If a match occurs, the line or block is ignored and does not trigger any action.
  7.  
    (From logmon 3.60)
     Update the following information to use multiple strings in a regular expression. For more information, see 
    Using Pattern Files
     in logmon Hints and Examples.
     The probe only supports text files in which the each string is listed as a new line.
    •  
      Enable regex from external file:
       allows you to use the same regular expression with multiple strings.
       8-bit PCRE supports a maximum length of 30000 characters for the regex and string combination. Ensure that each regex and string combination is a valid regular expression.
    •  
      Token to be replaced:
       replaces the specified text in the match expression with each string in the file and executes the rule.
    •  
      Path to pattern file:
       specifies the path of the file with the pattern strings to include in the expression. You can also browse and select the pattern file with the strings for the regular expression.
       Restart the probe if you modify the pattern file. The probe only supports UTF-8 encoding for the pattern file.
  8. Select the checkbox next the rule to enable the rule.
  9. Click 
    Apply
     to save the configuration.
Using the ExcludeEndTag property to monitor XML logs
 This enhancement supports only the xml logs that append new content right before the end tag, and not for general xml updates which can be across the entire file.
By default, the logmon probe only monitors content added after the end of file (EOF) mark. In case of some XML log types (like the ones generated by tools like Dell® OMSA®), the log entries are added right before the last tag in the xml file, instead of being appended at the end of file. In such cases the probe ends up reading the whole file again, generating alarms for all old events that were already contained in the xml file. 
To handle these XML log entries, you will need to add a property called 'ExcludeEndTag' in the probe's Raw Configuration, with end tag name as its value. This will ensure that the probe reads the newest entries right before the last tag, and captures only the updates to the xml file, instead of reading the file again from the beginning.
 
To use the ExcludeEndTag, follow these steps:
 
  1. Create a profile and navigate to Raw Configure, through AC or IM.
  2. Add/ set the following key/value pair, for the selected profile:
    •  
      Key 
      = ExcludeEndTag
    •  
      Value
       = The XML close/ end Tag you wish to specify. For e.g. </EventLog> 
  3. Click OK.
  4. Restart the probe for the configuration changes to be effective.
 The Exclude End Tag will only work if the log file you are monitoring is in 'update' mode.
If you intend to append the log file after the specified end tag, CA recommends that you remove the ExcludeEndTag property, or change it's value to null/ blank.  
Use Time Formatting Primitives
The log file name, command or path is constructed using text and special primitives for the system-call 
strftime
. The special primitives on different platforms are as follows:
 From version 3.90, the probe does not expand the % symbol in URL profiles to time formatting primitives.
 
Primitive
 
 
Windows
 
 
Unix
 
 
%a
 
The abbreviated weekday name
The abbreviated name of the day of the week according to the current locale
 
%A
 
The full weekday name
The full name of the day of the week according to the current locale
 
%b
 
The abbreviated month name
The abbreviated month name according to the current locale
 
%B
 
The full month name
The full month name according to the current locale
 
%c
 
Date and time representation appropriate for the locale
 The preferred date and time representation for the current locale
 
%C
 
The century number (year/100) as a 2-digit integer
 
%d
 
Day of the month as the decimal number (01 - 31)
The day of the month as a decimal number (range 01 to 31)
 
%D
 
Equivalent to %m/%d/%y
 
%e
 
Like %d, the day of the month as a decimal number, but a leading zero is replaced by a space
 
 %E
 
Modifier
 
%F
 
Equivalent to %Y-%m-%d
 
%G
 
The ISO 8601 week-based year with century as a decimal number. The 4-digit year corresponding to the ISO week number (see %V). This has the same format and value as %Y, except that if the ISO week number belongs to the previous or next year, that year is used instead.
 
%g
 
Like %G, but without century, that is, with a 2-digit year (00-99)
 
%h
 
Equivalent to %b
 
%H
 
Hour in 24-hour format (00 - 23)
The hour as a decimal number using a 24-hour clock (range 00 to 23)
 
%I
 
Hour in 12-hour format (01 - 12)
The hour as a decimal number using a 12-hour clock (range 01 to 12)
 
%j
 
Day of the year as the decimal number (001 - 366)
The day of the year as a decimal number (range 001 to 366)
 
%k
 
The hour (24-hour clock) as a decimal number (range 0 to 23); single digits are preceded by a blank
 
%l
 
The hour (12-hour clock) as a decimal number (range 1 to 12); single digits are preceded by a blank
 
%m
 
Month as the decimal number (01 - 12)
The month as a decimal number (range 01 to 12)
 
%M
 
Minute as the decimal number (00 - 59)
The minute as a decimal number (range 00 to 59)
 
%n
 
A newline character
 
%O
 
Modifier
 
%p
 
The current locale A.M./P.M. indicator for the 12-hour clock
Either "AM" or "PM" according to the given time value, or the corresponding strings for the current locale. Noon is treated as "PM" and midnight as "AM"
 
%P
 
Like %p but in lowercase: "am" or "pm" or a corresponding string for the current locale
 
%r
 
The time in a.m. or p.m. notation. In the POSIX locale this is equivalent to %I:%M:%S %p
 
%R
 
The time in 24-hour notation (%H:%M)
 
%s
 
The number of seconds since the Epoch, 1970-01-01 00:00:00 +0000 (UTC)
 
%S
 
Second as the decimal number (00 - 59)
The second as a decimal number (range 00 to 60). (The range is up to 60 to allow for occasional leap seconds.)
 
%t
 
A tab character
 
%T
 
The time in 24-hour notation (%H:%M:%S)
 
%u
 
The day of the week as a decimal, range 1 to 7, Monday being 1
 
%U
 
Week of the year as the decimal number, with Sunday as the first day of the week (00 - 53)
The week number of the current year as a decimal number, range 00 to 53, starting with the first Sunday as the first day of week 01
 
%V
 
The ISO 8601 week number of the current year as a decimal number, range 01 to 53, where week 1 is the first week that has at least 4 days in the new year
 
%w
 
Weekday as the decimal number (0 - 6; Sunday is 0)
The day of the week as a decimal, range 0 to 6, Sunday being 0
 
%W
 
Week of the year as the decimal number, with Monday as the first day of the week (00 - 53)
The week number of the current year as a decimal number, range 00 to 53, starting with the first Monday as the first day of week 01
 
%x
 
The date representation for the current locale
The preferred date representation for the current locale without the time
 
%X
 
The time representation for the current locale
The preferred time representation for the current locale without the date
 
%y
 
Year without the century as the decimal number (00 - 99)
The year as a decimal number without a century (range 00 to 99)
 
%Y
 
Year with the century as the decimal number
The year as a decimal number including the century
 
%z
 
The time zone name or abbreviation; no characters when the time zone is unknown
The +hhmm or -hhmm numeric timezone (that is, the hour and minute offset from UTC)
 
%Z
 
The time zone name or abbreviation; no characters when the time zone is unknown
The timezone name or abbreviation
%+
The date and time in date(1) format
 
%%
 
The percent sign
A literal '%' character
 
Example: 
You can monitor log files that switch every day in the form of 
ddmmyy.log
:
C: \Program files\mylogs\%d%m%y.log
Configure Command Exit Code Alarms
You can add a new threshold, copy existing threshold details to a new one, modify an existing one, and delete a threshold.
The 
Threshold [New]
 dialog contains the following fields:
  •  
    Threshold Value: 
    specifies an exit code.
  •  
    Condition: 
    specifies relational operator.
  •  
    Severity: 
    specifies the alarm message severity.
  •  
    ID: 
    displays the numeric sequence of the threshold.
  • Subsystem ID:
     defines the alarm subsystem ID. 
  •  
    Message: 
    defines the alarm message text.
Configure URL Settings
The 
URL Settings
 dialog allows you to configure additional properties to access the URL content. These properties include proxy settings, user authentication settings, and timeout settings.
The 
URL Settings 
dialog contains the following fields:
  •  
    URL: 
    defines the complete URL of the web page you want to scan. The probe sends an alarm if the specified URL is not found or the probe is unable to contact the URL. The probe also sends an alarm, if the specified URL exists, its http response is 200 but its fetch data is 0.
  •  
    Timeout (sec.): 
    specifies the wait time for the probe for loading the complete web page. If the page takes more than the specified time to load, an alarm is generated.
  •  
    Retries before failure: 
    specifies the number of attempts for accessing the URL before giving up and then sends a failure alarm.
  •  
    Failure alarm level: 
    specifies the severity level of the alarm to be sent if an error occurs (timeout or too many retries).
  •  
    Windows NT Authentication: 
    allows you to configure the username and password for impersonating the Windows user account for authenticating the probe for accessing the URL. The proxy setting of the specified user is fetched from the registry, as saved by Internet Explorer.
  •  
    Proxy Properties (optional):
     specifies the proxy details for accessing the URL over a proxy server. These fields appear when the Windows NT authentication is not selected.
  •  
    Host: 
    defines the IP address or hostname of the proxy server, which forwards the URL requests.
  •  
    Port: 
    defines the HTTP port of the proxy server.
  •  
    User:
     defines the username of the proxy server.
  •  
    Password: 
    defines the password for the corresponding proxy user.
  •  
    SSL Settings:
     specifies the degree of encryption of the data traffic.
  •  
    User Authentication: 
    configures the user authentication detail. This option is required when the web server hosting the URL requires you to log in.
    •  
      User:
       defines the user name of the webserver host for accessing the web page.
    •  
      Password: 
      defines the password for the corresponding user.
Variable Expansion in Alarms
You can enter a $ symbol in the alarm message text to select from the list of variables. The values of these variables are retrieved from the monitored system.
 The following variables are available in the alarm message text:
  • PROFILE: returns the profile name.
  • FILENAME: returns the name of log file, queue, url, or command.
  • WATCHER: returns the watcher rule name.
  • WATCHERMATCHEDLINE: displays the line which matches the given expression. 
Exit code monitoring alarms in Command profiles have additional variables available in the alarm message text. For more information, see Configure Command Exit Code Alarms.
  • EXITCODE: returns the exit code used to end the command.
  • OPER: returns the operator specified for the thresholds to generate alarms for exit codes.
  • THR: returns the specified threshold to generate alarms for exit codes.
(From logmon 3.70) Pattern match threshold alarms in Update profiles have additional variables available in the alarm message text.
  • PATTERN: returns the regular expression defined to match in the file.
  • THRESHOLD: returns the specified threshold value for the probe to generate alarms on the breach.
  • STARTTIME: returns the last modified time of the file in the previous interval. When you start monitoring, the probe uses the existing modified time of the file in the first interval. 
  • ENDTIME: returns the last modified time of the file in the same interval.
The ${Watcher Variable} returns the variable value which is defined in the watcher rule. For example, you can use
 ${var}
 to retrieve the value of a 
var
 variable that is configured for the watcher rule. If the specified variable is not defined in the watcher rule, then the variable name is displayed as-is. For more information, see the 
Configure Custom Variables
 section of 
Create Watcher Rules
 in logmon IM Configuration.