Logging

Logging is the act of keeping an append-only record of events which can later be read for troubleshooting purposes or forensic analysis. OSDx provides support for two forms of logging: the system journal and syslog.

Events

Events are occurrences located at a single point in time and contain a plain-text message containing information about the cause and context surrounding it. They have a severity level or log level that hints at how the message should be interpreted and reacted to.

Events are generated by processes in the system, for example the CLI or services such as the firewall. By default, lower log levels are hidden to avoid having too much noise in the log, but this cut-off log level can be configured on a feature-by-feature basis through the feature’s log-level or logging configuration parameters, for example service ssh log-level, vpn ipsec logging log-types any log-level, or system cli configuration logging global.

Log levels

Log levels are the predefined categories events may be in. In descending order of severity they are:

Emergency: Panic messages. They indicate the system is in an unusable state and requires immediate attention.
Alert: Alert messages which command immediate action to resolve a critical issue.
Critical: Critical error messages which demand intervention to prevent system failure.
Error: Error messages which signify non-critical conditions that may affect normal operations.
Warning: Warnings which may lead to errors or unexpected behavior if not addressed.
Notice: Informational messages that may hint at an issue or abnormal state.
Informational: Informational messages seen in a normally functioning system.
Debug: Verbose messages intended for debugging specific issues.

Journal

The system journal is a volatile in-memory log, meaning it gets reset every time the system is rebooted. It is a useful tool for quickly inspecting the state of the system and investigate possible issues due to a faulty configuration.

To view the system journal, run the system journal show command.

Navigating the journal

The journal uses a pager in order to allow the user to navigate the potentially massive log. It might be cumbersome to navigate at first, but with some basic shortcuts it becomes an extremely powerful tool.

The most important shortcut to remember is the command for quitting the pager: This is done by simply pressing the q key.

Navigating the log can be done using the up and down arrow keys. If the log messages are cut off due to being too long, the right and left arrow keys can be used for horizontal scrolling. The g and G keys can be used to go to the beginning and end of the journal, respectively.

Finally, the logs can be searched through by pressing /, followed by the text and the Enter key. Jumping to the next search result is done by pressing n, and the previous, N.

For more information, pressing h will show a full reference for every shortcut available.

Monitoring the journal

If the goal is to show the journaled events as they arrive, showing the system journal will not suffice as it only shows a snapshot of the events when the command is run, instead, the system journal monitor command provides a live view of events as they are logged.

Syslog

OSDx supports the syslog message logging standard, which allows it to send events to remote syslog servers as well as store logs persistently on the system.

Unlike the journal, syslog is a service. This means that it needs to start up before logging can begin, causing it to miss some events like the ones generated at boot up time.

For more information about syslog, check out the Syslog chapter.