Syslog Collector
Syslog
collector is a Python application that collects log messages generated by various devices
and applications within a network. This protocol is widely used in network management and
monitoring to track and troubleshoot system events.
The Syslog collector operates in two modes:
- Query mode: In this mode, the Syslog collector establishes an SSH connection to the network agent or syslog server to tail the logs. Each incoming log is dynamically processed and published as an event in theVMware Telco Cloud Service Assurance.
- Listen mode: In this mode, the Syslog collector starts a UDP server with a NodePort exposed to collect Syslog messages from multiple devices within a network. Each device pushes the Syslog messages to the configured logging server (Syslog collector) through the exposedVMware Telco Cloud Service AssuranceNodePort.
The standard format of the sample Syslog message
is:
Timestamp: %Facility-Severity-Mnemonic: Message Text
For
example:
Feb 14 12:02:38: %LINK-5-CHANGED: Interface FastEthernet0/1, changed state to administratively down
Regular Expressions
(regex)
A regular expression (commonly referred
to as regex) is a sequence of characters that specifies a search pattern in
text.
The regex examples below can be used to
extract values for the notification attributes:
Regex | Attributes | Description |
|---|---|---|
.*%(.*?)-.* | Class Name | For the above example Syslog message, the provided Regex
extracts the value - LINK |
$set_on_match(".*", AVAILABILITY) | Category | For the above example Syslog message, the provided Regex
extracts the value - AVAILABILITY |
.*-(\w+):.* | Event Name, Event Text | For the above example Syslog message, the provided Regex
extracts the value - CHANGED. |
.*:\s*([^,]+) | Instance Name | For the above example Syslog message, the provided Regex
extracts the value - Interface FastEthernet0/1. |
.*-(\d+)-.* | Severity | For the above example Syslog message, the provided Regex
extracts the value - 5 (which translates to Normal as per the
standards). |
$set_on_match(".*:\s*(.*\bup\b.*)$","INACTIVE",".*:\s*(.*\benabled\b.*)$","INACTIVE",".*-(UPLINK):.*","INACTIVE",".*:\s*(.*Failed.*)$","ACTIVE") | EventState | Each condition is treated as an argument and checked against
incoming syslog messages until a match is found. If no messages
match, the default value is used for the EventState. The method
$set_on_match can be used to set
the EventState to INACTIVE based on specific conditions that
can be leveraged to clear the Events.Syntax: $set_on_match("regex1", "value1", "regex2",
"value2",,,,,) For
example: ".*:\s*(.*\bup\b.*)$" checks if the syslog
message has string up , if it matches sets value INACTIVE as specified.
".*:\s*(.*Failed.*)$" checks if the syslog
message has string Failed , if it matches then sets the value ACTIVE
as specified. |
$transform_date("DD-MM-YYYYTHH:mm:SS.FZ") | Timestamp | The method,
$transform_date("input_date_format") , is
used to extract the timestamp present in the syslog message and
convert it into epoch time.Example Syslog:
2024-06-13T10:32:50.52Z %VIRTIO-3-INIT_FAIL: Failed to
initialize device, PCI 0/6/0/1002 , device is disabled,
not supported Here is how you
can add the regex and input date format to the Timestamp
field of the Syslog collector: As per RFC
5424 , VMware Telco
Cloud Service Assurance supports
timestamps in the default format
YYYY-MM-DDTHH:mm:SS.FZ . For any
deviations from this format, you can specify the input date
format as shown below:$transform_date("DD-MM-YYYYTHH:mm:SS.FZ") This function
extracts the timestamp
2024-06-13T10:32:50.52Z from the Syslog
message, which is in the format
YYYY-MM-DDTHH:mm:SS.FZ , and converts it
into epoch time.Sample Date
Input formats:
|
The regular expression can also include
regex to extract the field and filter the value simultaneously from ClassName,
EventName and InstanceName.
For example:
.*-(\w+):.*Interface.*
: This type of complex regex
allows extraction of the eventName, such as "CHANGED" (from the above message), only
if it is followed by or contains "Interface" in the Syslog message text.To create a Syslog collector, the
Class Name, Event Name, and Instance Name parameters are mandatory. The
notification name is derived from these parameters in the format:
ClassName_InstanceName_EventName. Therefore, filters can be applied, or the user
can filter syslog messages using the provided regex based on these parameters
only.