Scheduling Monitor Checks

 uses a system that is named the Second Opinion to double-check unsuccessful probe results that are sent from monitoring stations. A temporary network glitch or other issues can affect probes. ASM automatically double-checks an unsuccessful probe by sending more probes from geographically similar locations. If the Second Opinion probes are also unsuccessful ASM reports the error.
cacm82
Second Opinion
DX APP Synthetic Monitor
 uses a system that is named the Second Opinion to double-check unsuccessful probe results that are sent from monitoring stations. A temporary network glitch or other issues can affect probes. 
DX APP Synthetic Monitor
 automatically double-checks an unsuccessful probe by sending more probes from geographically similar locations. If the Second Opinion probes are also unsuccessful, 
DX APP Synthetic Monitor
 reports the error.
Probe Scheduling
A check is executed at an interval that is specified in monitor settings. The scheduler issues up to three probes per check. Each check typically runs on a different monitoring station. The check result is then obtained as a combination of the results of these probes. Multiple probes eliminate false positives.
A single probe delivers one of four results:
  • Pass
  • Fail
    conditions mandatory for 
    pass
    , as defined in the monitor, were not satisfied.
  • Unconfirmed
    A special case of fail.
    The system verifies the result by running a second opinion. Each monitor type has different rules on how to classify as fail. Factors other than a failure within customer infrastructure can cause errors.
  • Inconclusive
     execution of the probe failed due to an internal error. For example, network failure between the scheduler and a monitoring station.
The dashboard log displays the results of probes, except 
inconclusive
 results, which are hidden. The two most common scenarios for a check run are:
  • The first probe returns a pass or fail. No other probes are required. The result of the check is the result of this probe.
  • The first probe returns a timeout, which is classified as an unconfirmed error. A second probe is issued. If it returns a timeout too, the result of the check is an error. If enabled in monitor settings, an alert is sent.
Log entries for probes have different colors, which are based on status. In the two probe scenario, the first result is shown in yellow (Unconfirmed error), the second probe in red (Error). If the result triggered an alert, the log entry also contains an icon with the envelope symbol.
Alerts are generated by the checks, not the probes. Therefore, a monitor set-up to send an alert 
immediately
 generates the alert after two probes consecutively report timeout errors. If the monitor is configured to send an alert after two errors, the monitor sends the alert after four probes have sent timeout error reports.
This table describes the rules for issuing probes on public monitoring stations, and for returning results:
Firstprobe
Secondprobe
Thirdprobe
Check Result
pass
ok
fail
error
unconfirmed
pass
ok
unconfirmed
fail
error
unconfirmed
confirmed
error
unconfirmed
inconclusive
pass
ok
unconfirmed
inconclusive
fail
error
unconfirmed
inconclusive
confirmed
error
unconfirmed
inconclusive
inconclusive
error
inconclusive
pass
ok
inconclusive
fail
error
inconclusive
unconfirmed
pass
ok
inconclusive
unconfirmed
fail
error
inconclusive
unconfirmed
confirmed
error
inconclusive
unconfirmed
inconclusive
error
inconclusive
inconclusive
pass
ok
inconclusive
inconclusive
fail
error
inconclusive
inconclusive
unconfirmed
error
inconclusive
inconclusive
inconclusive
inconclusive
For OPMS with only two monitoring stations available in a namespace, the rules are:
Firstprobe
Secondprobe
Check Result
pass
ok
fail
error
unconfirmed
pass
ok
unconfirmed
fail
error
unconfirmed
confirmed
error
unconfirmed
inconclusive
error
inconclusive
pass
ok
inconclusive
fail
error
inconclusive
unconfirmed
error
inconclusive
inconclusive
error
For a single OPMS installation:
probe
Check Result
pass
ok
fail
error
unconfirmed
error
inconclusive
error
ASM selects a monitoring station for the second opinion automatically from the 
order algorithm
 you select for a given rule:
 
Algorithm
 
 
Next Monitor Station
 
Master
The nearest monitor station
Random
A random monitor station
Sequential
Next monitor station in the sequence
Sticky
If the previous probe returned an error, use the same monitor station. Otherwise pick one at random.
Classification of Probe Failures
This section lists probe failures that do not require a second opinion. Any error code that is not listed here is classified as 
unconfirmed.
Simple monitors
These include HTTP, HTTPS, FTP, FTPS, CONNECT, SMTP, TELNET, LDAP, SCP, SFTP, IMAP, POP3.
  • 95xx –content did not match a specified string or regular expression
DNS
  • 8003 – no name servers found
  • 8006 – name server error
  • 8007 – invalid name server list
  • 9501 – record not matched
Domain
  • -12 – invalid name server list
  • 8003 – no name servers found
  • 8005 – inconsistent responses
  • 8006 – name server error
  • 8007 – UDP reply truncated
Traceroute
  • 7125 – timeout inside perimeter / Number of hops exceeded
  • 7100 – host unreachable
  • 7101 – network unreachable
  • 7102 – protocol unreachable
  • 7103 – port unreachable
  • 7104 – fragmentation that is needed and DF set
  • 7105 – source route failed
  • 7106 – destination network unknown
  • 7107 – destination host unknown
  • 7111 – network unreachable for Type of Service
  • 7112 – host unreachable for Type of Service
  • 7113 – communication administratively prohibited
  • 7114 – host precedence violation
  • 7115 – precedence cutoff in effect
  • 7120 – fragmentation needed
  • 7121 – host unreachable
Script
  • 1060 – bandwidth allowance exceeded
  • 1061 – unsupported bandwidth size
  • 7001-7010 – assertion failed / JMeter errors
  • 7016 – request limit exceeded
  • 7018 – only HTTP samplers supported
  • 7021 – script did not perform any requests
Full Page Monitor
  • <response code> – HTTP response code not from 200 through 399, inclusive.
  • 9501 – matched string not found
  • 9502 – invalid regular expression
  • 7001-7010 – JavaScript errors