Evaluate the Performance of a SpectroSERVER
Examining Thread Latency
The topics in this section describe how to determine whether thread latency indicates performance issues. The types of threads and indicators of latency are also discussed.
Threads and Thread Latency
DX NetOps Spectrumperformance partially depends on timely thread allocation. A
threadis a set of commands that perform a function or a set of functions. Each thread can run independently from other threads.
SpectroSERVERis a single-threaded application regarding the CPU, but a multi-threaded application internally. Within its own process, the
SpectroSERVERcreates and manages multiple threads that run simultaneously for tasks such as polling, logging, notifications, timers, and more.
The Archive Manager runs in its own thread. As a result, you can use multiple CPUs on a server: one for the
SpectroSERVERand another for the Archive Manager. However, deploying three or more CPUs can degrade performance.
In a multi-threaded context, while some threads are waiting (for user input, responses from devices, or data retrieval, for example), other threads can be running. A thread runs to log data, respond to traps, and to connect to SSAPI applications, for example. As each thread runs in the
SpectroSERVERprocess, it takes control of the CPU for a few microseconds and then relinquishes control to allow other threads to run.
SpectroSERVERmaintains a pool of threads that are shared by the
DX NetOps Spectrumprocesses that perform polling, logging, client requests, inference handler timer, inference handler notification, model activation, and model destruction. A
SpectroSERVERsubsystem uses threads from the pool - up to their individual limits - during periods of increased processing activity. These limits prevent any one
SpectroSERVERsubsystem from dominating resources and consuming all the available threads.
When the common pool of threads is exhausted, new threads are created. The pool grows to meet the needs of the increased activity. Threads that a process no longer requires are returned to the common pool for later use. When a thread remains unused for a specified period, it is removed from the pool, and its resources are returned to the system. This process is called aging.
Thread latencyis the amount of time between when a thread is supposed to complete and when it actually completes. It can cause problems when the number of outstanding threads accumulates as the threads take more time to complete. If
DX NetOps Spectrumruns for a prolonged period with high thread latencies, delays occur in device polling, logging, and other tasks. As a result, delays contribute to the response time. For example, if a critical network device became inoperable, a delay would occur before
DX NetOps Spectrumnotified a Network Administrator of the problem.
Thread latency is a symptom, not a cause, of performance degradation.
Types of Threads That Affect Performance
Poll, log, timer, and notification threads can affect the performance of the
SpectroSERVER. Performance View provides usage and latency statistics for these threads.
By default, the Poll_Log_Ratio attribute on device models is set to 0, which effectively disables native
DX NetOps Spectrumlogging. To log device, attribute, and port statistics, we recommend using SSLogger instead of the native method, which writes the information to the Archive Manager database. SSLogger is a
DX NetOps Spectrumcommand-line application that logs statistics directly to ASCII files. This type of logging reduces the load on the Archive Manager database and eliminates the need to export the data. SSLogger also provides increased control over the type and frequency of data that is logged.
For more information about SSLogger, see
- Poll threadsPoll devices on the network.DX NetOps Spectrumuses polling to manage the operation and performance of the network. TheSpectroSERVERcode that manages poll threads is named the Poll Manager.
- Log threadsLog data from the network intoDX NetOps Spectrumdatabase archive files.DX NetOps Spectrumcan use data logging to store information about the operation and performance of the network.
- Timer threadsNotify inference handlers that have registered timers, also known as wake-up calls.
- Notification threadsNotify inference handlers about changes in an attribute for which the inference handlers have registered.
Access the Thread Information View
Access the Thread Information view to see information about thread performance and status.
Follow these steps:
- Click the Topology tab in the OneClick Console.The Topology opens.
- Click the VNM icon for theSpectroSERVER.
- Click the Information tab in the Component Details panel.
- Expand theSpectroSERVERControl, Thread Information subview.The Thread Information view opens.
Thread Information View
The Thread Information subview appears within the
SpectroSERVERControl subview in the OneClick Console. This subview shows the threads in use, threads available, and peak value for the threads within the
SpectroSERVERprocess. While it shows a list of important threads that are used in
DX NetOps Spectrum, the list is not exhaustive. Some of the threads available in this list include the following types:
- Poll ThreadsUsed to read the polled attributes for a model on the Polling_Interval of that model.
- Log ThreadsUsed to read and log the logged attributes for a model on the Polling_Interval * Poll_Log_Ratio.
- Notification ThreadsUsed to send notifications of attribute changes to inference handlers andDX NetOps Spectrumclient applications.
- IH Timer ThreadsUsed to trigger timers in inference handlers.
- Destroy ThreadsUsed to send model destruction notifications to inference handlers and client applications.
- Model Activate ThreadsUsed to send model activation notifications to inference handlers and client applications.
- Relation Activate ThreadsUsed to send relation change notifications to inference handlers and client applications.
- Client Request Threads(*)Used to handle client application requests.
- Multi-Request Threads(*)Used to handle multi-model requests that originate from inference handlers and client applications.
You can use the Thread Information view to change the available value for each thread type. However, the values are typically left at their defaults. If one of the thread types is consistently running at the limit, and CPU cycles are available, increasing a limit can reduce the latency. However, if CPU utilization is already above 80percent, increasing thread limits does not increase throughput. By contrast, the combination of high CPU utilization and increased thread limits can reduce throughput by increasing thread overhead.
If you find that all available threads are being used for one specific type of thread, contact CA Support for assistance.