> >

Linux system logs (logging management). Syslog system and logs in Linux How to create your own log file in linux

If you take up Linux administration, be prepared for the fact that viewing and analyzing log files will take up the lion's share of the time that you spend in the console. Log analysis is the main (and most often the only) way to understand the behavior of the server.

Often, the log contains thousands of lines, so moreover, it can increase by several more entries every second. And it is desirable to watch live, tracking the reaction to certain actions. Two utilities tail and less will help us here.

Viewing logs on Linux with tail .

tail (translated as tail) allows you to view the last 10 lines of a file. If you need more, just say so. For example, 25:

# tail -n 25 /var/log/httpd/error_log

[hc@host~] # tail -n 25 /var/log/httpd/error_log

Or you can simply:

# tail -25 /var/log/httpd/error_log

[hc@host~] # tail -25 /var/log/httpd/error_log

The second useful parameter that can be passed is -f . With this parameter, tail prints the specified number of last lines and continues to read the added lines until you press Ctrl+C - i.e. you can track changes in the log file live:

# tail -5f /home/sites/web/site.ru/logs/site.ru.log 178.45.0.0 - - "GET / HTTP/1.0" 200 17401 "http://site.ru/allsorts/" "Mozilla /5.0 (iPad; CPU OS 8_0_2 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12A405 Safari/600.1.4" 217.118.0.0 - - "GET /allsorts HTTP/1.0" 301 - "http://site.ru/" 217.118.0.0 - - "GET /allsorts/ HTTP/1.0" 200 16663 "http://site.ru/" "Mozilla/5.0 (iPhone; CPU iPhone OS 9_2 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13C75 Safari/601.1" 85.95.0.0 - - "GET / HTTP/1.0" 200 17401 "http://vk.com/away.php " "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.111 Safari/537.36OPR/34.0.2036.50" 85.95.0.0 - - "GET /allsorts/ HTTP/1.0" 200 16663 "http://site.ru/" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.111 Safari/537.36OPR/34.0.2036.50"

[hc@host~] # tail -5f /home/sites/web/site.ru/logs/site.ru.log

178.45.0.0 - - [ 23 / Jan / 2016 : 19 : 45 : 02 + 0300 ] "GET / HTTP/1.0" 200 17401 "http://site.ru/allsorts/" "Mozilla/5.0 (iPad; CPU OS 8_0_2 like Mac OS X) AppleWebKit/600.1.4 (KHTML, like Gecko) Version/8.0 Mobile/12A405 Safari/600.1.4"

217.118.0.0 - - [ 23 / Jan / 2016 : 19 : 46 : 34 + 0300 ] "GET /allsorts HTTP/1.0" 301 - "http://site.ru/" "Mozilla/5.0 (iPhone; CPU iPhone OS 9_2 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13C75 Safari/601.1"

217.118.0.0 - - [ 23 / Jan / 2016 : 19 : 46 : 35 + 0300 ] "GET /allsorts/ HTTP/1.0" 200 16663 "http://site.ru/" "Mozilla/5.0 (iPhone; CPU iPhone OS 9_2 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13C75 Safari/601.1"

85.95.0.0 - - [ 23 / Jan / 2016 : 19 : 47 : 34 + 0300 ] "GET /allsorts/ HTTP/1.0" 200 16663 "http://site.ru/" "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/47.0.2526.111 Safari/537.36 OPR/34.0.2036.50"

But the biggest convenience of tail is that multiple files can be passed as parameters, i.e. you can monitor multiple related log files at the same time:

# tail -f -n 5 -s 3 /home/sites/web/site.ru/logs/site.ru.log /var/log/httpd/domains/site.ru.error.log ==> /home/ sites/web/site.ru/logs/site.ru.log<== 37.113.0.0 - - "GET /wp-content/themes/bigroll/img/delivery-feature-6.png HTTP/1.1" 200 18165 "http://site.ru/delivery/" "Mozilla/5.0 (iPhone; CPU iPhone OS 9_2 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13C75 Safari/601.1" 37.113.0.0 - - "GET /wp-content/themes/bigroll/img/delivery-truck.png HTTP/1.1" 200 15243 "http://site.ru/delivery/" "Mozilla/5.0 (iPhone; CPU iPhone OS 9_2 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13C75 Safari/601.1" 37.113.0.0 - - "GET /wp-content/themes/bigroll/img/scooter-icon-small.png HTTP/1.1" 200 4322 "http://site.ru/delivery/" "Mozilla/5.0 (iPhone; CPU iPhone OS 9_2 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13C75 Safari/601.1" 37.113.0.0 - - "GET /wp-content/themes/bigroll/img/link-button-shadow.png HTTP/1.1" 200 7792 "http://site.ru/delivery/" "Mozilla/5.0 (iPhone; CPU iPhone OS 9_2 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13C75 Safari/601.1" 37.113.0.0 - - "GET /wp-content/themes/bigroll/img/header-bg.png HTTP/1.1" 200 194475 "http://site.ru/delivery/" "Mozilla/5.0 (iPhone; CPU iPhone OS 9_2 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13C75 Safari/601.1" ==>/var/log/httpd/domains/site.ru.error.log<== client denied by server configuration: /home/sites/web/site.ru/public_html/xmlrpc.php client denied by server configuration: /home/sites/web/site.ru/public_html/xmlrpc.php client denied by server configuration: /home/sites/web/site.ru/public_html/xmlrpc.php client denied by server configuration: /home/sites/web/site.ru/public_html/xmlrpc.php client denied by server configuration: /home/sites/web/site.ru/public_html/xmlrpc.php

[hc@host~] # tail -f -n 5 -s 3 /home/sites/web/site.ru/logs/site.ru.log /var/log/httpd/domains/site.ru.error.log

==> /home/sites/web/site. ru/logs/site. ru. log<= =

37.113.0.0 - - [ 23 / Jan / 2016 : 19 : 55 : 45 + 0300 ] "GET /wp-content/themes/bigroll/img/delivery-feature-6.png HTTP/1.1" 200 18165 "http://site.ru/delivery/" "Mozilla/5.0 (iPhone; CPU iPhone OS 9_2 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13C75 Safari/601.1"

"GET /wp-content/themes/bigroll/img/delivery-truck.png HTTP/1.1" 200 15243 "http://site.ru/delivery/" "Mozilla/5.0 (iPhone; CPU iPhone OS 9_2 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13C75 Safari/601.1"

37.113.0.0 - - [ 23 / Jan / 2016 : 19 : 55 : 46 + 0300 ] "GET /wp-content/themes/bigroll/img/scooter-icon-small.png HTTP/1.1" 200 4322 "http://site.ru/delivery/" "Mozilla/5.0 (iPhone; CPU iPhone OS 9_2 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13C75 Safari/601.1"

37.113.0.0 - - [ 23 / Jan / 2016 : 19 : 55 : 46 + 0300 ] "GET /wp-content/themes/bigroll/img/link-button-shadow.png HTTP/1.1" 200 7792 "http://site.ru/delivery/" "Mozilla/5.0 (iPhone; CPU iPhone OS 9_2 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13C75 Safari/601.1"

37.113.0.0 - - [ 23 / Jan / 2016 : 19 : 55 : 46 + 0300 ] "GET /wp-content/themes/bigroll/img/header-bg.png HTTP/1.1" 200 194475 "http://site.ru/delivery/" "Mozilla/5.0 (iPhone; CPU iPhone OS 9_2 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobile/13C75 Safari/601.1"

==> /var/log/httpd/domains/site. ru. error . log<= =

[ Wed Jan 20 04 : 38 : 36 2016 ] [ error ] [ client 46.166.0.0 ] client denied by server configuration : / home / sites / web / site . en/public_html/xmlrpc. php

[ Wed Jan 20 13:15:43 2016 ] [ error ] [ client 69.84.0.0 ] client denied by server configuration : / home / sites / web / site . en/public_html/xmlrpc. php

[ Wed Jan 20 20:48:58 2016 ] [ error ] [ client 185.112.0.0 ] client denied by server configuration : / home / sites / web / site . en/public_html/xmlrpc. php

[ Wed Jan 20 22:42:02 2016 ] [ error ] [ client 5.255.0.0 ] client denied by server configuration : / home / sites / web / site . en/public_html/xmlrpc. php

Please note that I used the -s 3 construction - using this key, the update interval is set in seconds, i.e. in this case, the files will be reread every 3 seconds.

Viewing logs on Linux with less .

For all its simplicity and convenience, tail has one significant drawback - it only shows the last lines and you do not have the opportunity to see what happened before these entries or at a certain point in time. For such situations, the less utility is better suited.

Let's open the file:

# less /var/log/httpd/domains/big-roll.ru.error.log

[hc@host~] # less /var/log/httpd/domains/big-roll.ru.error.log

As you can see, we can easily scroll through the contents of the file using the cursor arrows and the PgUp and PgDn keys.
A few more useful keys to remember:
q – output;
g - go to the beginning of the file;
G - go to the end of the file;
Ng - go to line N (for example, line 14 14g);
N% - move to position N% (for example, 35%);
F - switch to the file update tracking mode, i.e. similar to tail -f ;
/pattern - direct (down the file) search by pattern;
?pattern - reverse (up the file);
n is the next match of the search pattern;
N is the previous match;
Exit tracking mode Ctrl+C ;
You can see a complete list of all other keys and modes on your own - by pressing h - help (I strongly advise you to press).

Preceding the filename with a pipe character (|) will allow you to use fifo (first in - first out, first in, first out) or named pipe as a message receiver. Before starting (or restarting) syslogd, a fifo must be created using the mkfifo command. Sometimes fifo is used for debugging.

Terminal and console

Terminal such as /dev/console.

remote machine

To forward messages to another host, precede the host name with an at sign (@). Note that messages are not forwarded from the receiving host. (for this assignment to work on the client and server in the file /etc/services line must be written syslog 514/udp, and open UTP port 514)

a list of users

Comma-separated list of users receiving messages (if the user is logged in). This often includes the root user.

All registered users

To notify all registered users using the wall command, use the asterisk character (*).

An example of a simple syslog.conf:

# Print all kernel messages to the console. #kern.* /dev/console # All logs of info level or higher, except emails, and # don't log authentication messages and cron daemon messages! *.info;mail.none;authpriv.none;cron.none /var/log/messages # Log messages containing confidential # authentication information to a separate file, regardless of their level. authpriv.* /var/log/secure # All messages from the mail system are also written to a separate file. mail.* -/var/log/maillog # Log scheduler messages to file /var/log/cron cron.* /var/log/cron # Emergency messages should be received immediately # by all system users *.emerg * # Save messages news of crit level and above into a separate file. uucp,news.crit /var/log/spooler # Store boot messages in boot.log local7.* /var/log/boot.log

As with many configuration files, the syntax is:

  • lines beginning with # and empty lines are ignored.
  • The * symbol can be used to indicate all categories or all priorities.
  • The special keyword none indicates that logging for this category should not be performed for this action.
  • A hyphen before the file name (like -/var/log/maillog in this example) indicates that the log should not be synchronized after each write. In the event of a system crash, you may lose information, but disabling synchronization will improve performance.

In the syntax of the configuration file, you can put before the priority sign! to show that the action should not be applied, from this level and up. Similarly, the priority can be prefixed sign = to indicate that the rule only applies to that level, or != to show that the rule applies to all levels except this one. Below are a few examples (man syslog.conf you can find many more examples):

# Send all kernel messages to /var/log/kernel. # Send all critical and higher level messages to the remote sysloger machine and console # Send all info, notice and warning level messages to /var/log/kernel-info # kern.* /var/log/kernel kern.crit @sysloger kern .crit /dev/console kern.info;kern.!err /var/log/kernel-info # Send all mail system messages except info level to /var/log/mail. mail.*;mail.!=info /var/log/mail

I tried to show the work of syslogd as clearly as possible on the diagram:

Starting the syslogd daemon

The launch of the logging daemon is started at the stage of system initialization by means of a script /etc/rc.d/init.d/syslog, however, in order to set launch options, there is no need to modify this script - starting from version 7.2, launch options are read from a separate configuration file /etc/sysconfig/syslog (/etc/default/syslogin debian).

Here are some possible syslogd daemon startup parameters:

  • -a /folder/socket- specifying an additional listening socket (do not forget to create a socket beforehand)
  • -d- debug mode. In this case, the daemon does not go into the background and issues all messages to the current terminal;
  • -f config-file-name. Specifies the name of an alternate configuration file to be used instead of the default /etc/syslog.conf;
  • -l host-list- setting a list of hosts whose names should not be written with the full domain name (FQDN - Full Qwalified Domain Name);
  • -m minutes- running without this option, sysklogd logs messages of category mark (time stamps) every 20 minutes. With the -m option, you can either change the interval between marks, or completely stop issuing such messages;
  • -p socket- setting an alternative UNIX socket (instead of the default listening /dev/log);
  • -r- permission to receive messages from remote hosts;
  • -x- prohibition of determining the host name by its address to prevent freezing when working on the same host as the DNS server.
  • -v- show version and finish work

After starting the syslogd daemon, a status file is created /var/lock/subsys/syslog zero length, and a file with process ID /var/run/syslogd.pid.

With the command
kill -SIGNAL `cat /var/run/syslogd.pid`

can be sent syslogd daemon one of the following signals: SIGHUP- restarting the daemon; SIGTERM- completion of work; SIGUSR1- enable/disable debug mode.

Actually, two logging daemons are launched in the system - syslogd and klogd. Both daemons are included in the package syslogd.

klogd daemon responsible for logging events that occur in system core. The need for a separate klogd daemon is because the kernel cannot use the standard syslog function. This is because the standard C libraries (including the library that contains the syslog function) are intended to be used only by common applications. Since the kernel also needs logging functions, it includes its own libraries that are not available to applications. Therefore, the kernel uses its own message generation mechanism.

klogd daemon designed to organize the processing of these messages. In principle, it can do this processing entirely on its own and independent of syslogd, such as writing these messages to a file, but in most cases the default klogd setting is used, in which all messages from the kernel are forwarded to the same syslogd daemon.

Automatic rotation (updating filled files) and archiving logs

Over time, the log file tends to grow, especially when a service is running intensively. Accordingly, it is necessary to be able to control the size of the logs. This is done using logrotate commands, which is usually performed cron daemon. I will talk about the work of cron in the following articles. the main objective logrotate commands is to periodically back up the logs and create new clean logs. Several generations of logs are saved and when the last generation log expires, it can be archived (compressed). The result can be sent by mail, for example, to the person responsible for archiving.

To determine the order of rotation and archiving of logs, use configuration file /etc/logrotate.conf . For different logs, you can set different periodicity, for example, daily, weekly or monthly, in addition, you can adjust the number of generations accumulated, as well as specify whether copies of the archives will be sent to the archive manager and, if so, when. Below is shown example /etc/logrotate.conf file:

# first set default parameters (global options) # update log files weekly weekly # store log archive for the last 4 weeks rotate 4 # create new (empty) file after rotation (update) create # uncomment if you want saved files were compressed #compress # enable rotation settings from specified directory include /etc/logrotate.d # don't store wtmp, or btmp -- log data rotation settings are as follows: /var/log/wtmp ( missingok monthly create 0664 root utmp rotate 1 ) / var/log/btmp ( missingok monthly create 0664 root utmp rotate 1 ) # specific syslogs can be configured below

Global options are placed at the beginning file logrotate.conf. They are used by default unless something more specific is specified elsewhere. In the example, the logs are rotated weekly and backups are kept for four weeks. As soon as a journal is rotated, a new one is automatically created in place of the old journal. logrotate.conf file may contain specifications from other files. So, it includes all files from the directory /etc/logrotate.d.

This example also contains special rules for /var/log/wtmp and /var/log/btmp(storing information about successful and unsuccessful attempts to enter the system), the rotation of which occurs monthly. If the files are missing, no error message is given. A new file is created and only one backup is kept.

In this example, when the backup reaches the last generation, it is deleted because it is not defined what to do with it.

Log backups can also be generated when the logs reach a certain size, and scripts can be generated from command sets to run before or after a backup operation. Example:

/var/log/messages ( rotate 5 mail [email protected] size 100k postrotate /usr/bin/killall -HUP syslogd endscript )

In this example the rotation /var/log/messages is produced when it reaches the size of 100 KB. Five backups are accumulated, and when the oldest backup expires, it is mailed to [email protected] The postrotate command word includes a script that restarts the syslogd daemon after rotation is complete by sending a HUP signal. The endscript command word is required to terminate the script, and also if there is a prerotate script. See the man pages for logrotate for more information.

Options, set in the configuration file logrotate.conf:

  • compress| no compression(older versions compress or not compress with gzip)
  • compress cmd(specifies the compression program, default is gzip)
  • uncompress cmd(sets the decompressor, default is ungzip)
  • compressext(sets the suffix for compressed files)
  • compressoptions(sets the parameters of the compression program; default is "-9", i.e. the maximum compression for gzip)
  • copytruncate| nocopytruncate(usually the old version is renamed and a new version of the log is created; with this option, logrotate copies the log to a new file and then truncates the old one; used if the program that creates the log does not know how to close it; entries made in the interval between copying and truncation are lost ; would it help if the logging program instead of using append mode just writes to the file using an internal pointer?)
  • create[permission-owner-group] | nocreate(immediately after renaming the old version of the journal and before calling postrotate, a new journal is created with the specified attributes - permissions are set in octal, as in chmod.2; if the attributes are not specified, then they are taken from the old journal)
  • daily(change of versions in the series occurs daily)
  • delaycompress| nodelaycompress(some programs do not immediately close the log, in which case compression should be postponed until the next cycle)
  • errorsemail(to whom to report bugs)
  • extensionsuffix(specifies the suffix added to the file names during rotation before the compression suffix)
  • ifempty| notifempty(change versions even if file is empty; default)
  • includeFile name| directory-name (textually substitute a file or all files from the specified directory; subdirectories, special files and files with suffixes from the exclusion list are not included; cannot be used inside a section)
  • mailaddress| nomail(when a version change necessitates deleting the old journal, then send it to the specified address)
  • mailfirst(send not the deleted version of the log, but the first one)
  • maillast(send log version to be deleted; default)
  • missingok| nomissingok(don't send error messages if the log is missing)
  • monthly(version changes monthly)
  • olddirdirectory| noolddir(during a version change, the log is moved to the specified directory; must be on the same physical device)
  • postrotate(all further lines up to the endscript line are executed as shell commands after the version change process)
  • prerotate(all further lines up to the endscript line are executed before the version change process)
  • rotatenumber(how many old versions to keep; if 0, then none)
  • sizebyte(version change occurs if the log size exceeds the specified number; you can use the suffixes "k" - kilobyte - and "M" - megabyte)
  • shared scripts| nosharedscripts(execute the prerotate and postrotate commands only once for all files described in the section)
  • tabooext[+] suffix-list(setting a list of exclusion suffixes for include; if the plus sign is specified, then addition, otherwise replacement; default: .rpmorig, .rpmsave, .rpmnew, ",v", .swp and "~")
  • weekly(version changes weekly)

Studying and monitoring journals

Log entries typically contain a timestamp, the name of the host on which the described process is running, and the name of the process. View logs You can use a paging program, for example, less, you can search for specific entries (for example, kernel messages from a specific daemon) using the command grep:

# less /var/log/messages # grep "ppp" /var/log/messages | tail Dec 17 16:34:25 proxy pppd: Connection terminated. Dec 17 16:34:25 proxy pppd: Exit. Dec 17 16:35:57 proxy pppd: LCP terminated by peer (^P]kV^@

The computer may not work constantly and turn off, say at night. Therefore, entries in /var/log/messages are stored cyclically from computer startup to shutdown, this can be seen from the messages:

Dec 17 08:32:56 syslog-server syslogd 1.4-0: restart. Dec 17 08:32:56 syslog-server syslog: syslogd started succeeded Dec 17 08:32:56 syslog-server kernel: klogd 1.4-0, log source = /proc/kmsg started. Dec 17 08:32:56 syslog-server syslog: starting klogd succeeded

Dec 17 08:32:56 syslog-server kernel: Kernel command line: auto BOOT_IMAGE=linux ro root=303 BOOT_FILE=/boot/vmlinuz-2.4.2-2 Dec 17 08:32:56 syslog-server kernel: Memory: 125652k/130560k available (1365k kernel code, 4200k reserved, 92k data, 236k init, 0k highmem) Dec 17 08:32:56 syslog-server kernel: CPU: Intel(R) Pentium(R) 4 CPU 1.60GHz stepping 02

Also, in this file you can find information about disk memory (including information about disk geometry, partition structure and interrupts used), information about peripheral devices, about starting individual services and services, information about connecting file systems and user logon messages. , as well as error messages.

Sometimes it may be necessary system log monitoring to search for current events. For example, you can try to catch a rarely occurring event at the moment it happened. In this case, you can use the command tail with option -f to monitor the contents of the system log. Example:

# tail -f /var/log/messages | grep syslog-server Dec 17 16:46:09 syslog-server pppd: pptpd-logwtmp.so ip-up ppp0 maikop 94.77.0.150 Dec 17 16:46:09 syslog-server pppd: Script /etc/ppp/ip-up finished (pid 12552), status = 0x0 Dec 17 16:46:49 syslog-server pptpd: CTRL: Client 85.175.197.65 control connection started Dec 17 16:46:49 syslog-server pptpd: CTRL: Starting call (launching pppd, opening GRE) Dec 17 16:46:49 syslog-server pppd: Plugin /usr/lib/pptpd/pptpd-logwtmp.so loaded.

In addition to the log files specified in /etc/syslog.conf, there are also other files, such as a file that stores information about the boot process of the system before syslogd was started, as well as files that have a binary format and store information about the last login user login, all successful user logins, and all unsuccessful user logins, respectively. The /var/log/ directory can also contain log files for daemons such as a web server or a proxy server. The format of these files is similar to syslogd logs.

Finally, I would like to emphasize that this protocol is not very secure, because. syslog does not contain any protection against message forgery. Even worse, the use of the UDP protocol allows attackers to send messages on behalf of any host. Your LAN must be shielded from receiving packets with spoofed local addresses (although this does not prevent spoofed messages from being sent from within the LAN) and from receiving packets from outside on port 514/udp. Cases of disk overflow with false messages are known.

The syslog protocol and UDP do not provide guaranteed delivery (messages can be lost due to network congestion or intercepted, corrupted messages are deleted without warning), correct delivery sequence (a process termination message may arrive before a process start message), priority delivery.

Confidentiality of messages is not ensured, as they are transmitted in clear text.

If you specify an incorrect collector or relay address when setting up the message generator, then there will be no error messages - the messages will be deleted (or written to someone else's log).

Several projects have been proposed to improve the syslog protocol. For example, RFC 3195 proposes a TCP-based logging system (syslog-conn) that ensures that messages are delivered in the correct sequence. The syslog-sign project proposes to provide authentication, message ordering, message integrity, and detection of missing messages by generating special messages containing a digital signature of a block of previous messages while maintaining the standard syslog protocol and format and using UDP.

Let's summarize a little:

Linux has a single daemon responsible for logging the events of the local system and remote systems. All events are collected from socket /dev/log, UDP port - 514, as well as from the "helper" klogd daemon, which sends messages from the kernel. All collected messages are filtered by the syslogd daemon through the rules in the /etc/syslog.conf file and distributed to the appropriate destinations according to the rules. Log files are periodically "cut off". The frequency is determined by the logrotate.conf file and the logrotate command, which is run by the system scheduler - cron.

That's all for today. I hope I described everything as clearly as possible. Over time, I will supplement the article!

Sincerely, Mc.Sim!


When viewing log files (log files) in Linux, it is sometimes necessary to monitor new entries in the logs in real time. That is, you specify which log file (or files) you want to view and monitor new entries in this file in real time.

Monitoring log files with the tail command

If you run the command tail FileName without any additional arguments, the last 10 lines of the file will be printed and the command will exit.

In order for the tail command to continuously display the latest records in the file, that is, if new records appeared in the file, then the information on the screen was updated, use the -f option:
tail -f LogFileName

Run the tail -f command to display the log file /var/log/syslog

tail -f /var/log/syslog

Since the option is used -f , the tail command does not exit, but waits for new entries to appear in the log file. As soon as new entries are added to the log file, they will be immediately displayed in the terminal.

To abort the command, press Ctrl+C

tailf command

Tail -f command is analogous to tailf command.

Usage:

Tailf /var/log/mylogfile.log

An important difference between tailf and tail -f is that tailf does not access a file when it is not being modified. As a result, the file access time is not updated and the system does not permanently flush the file to disk when the file is not updated.

The description of the tailf command indicates that it is useful for monitoring log files on laptops. Since unnecessarily there is no access to the disk and battery life is preserved.

tail-F. If the file has been renamed or deleted

Usually in Linux, log files are not written indefinitely, otherwise such a file would be very inconvenient to use in the future. Instead, so-called file rotation is used. When a log file becomes large, it is either deleted or renamed (a backup copy of the file is created), and further messages are written to a new, empty file.

The tail command has two options: -f and -F

  • If used -f option and the tracked file is renamed, the tail command continues to track the already renamed file. Team tail in this case, it is bound to the identifier (inode) of the file.
  • If used option -F and the tracked file is being renamed, then the tail command will determine this, and as soon as a new log file is created (with the name that we specified to the tail command tail), team tail will start tracking this new file.

Consider an example.

We will monitor the log file /var/log/apache2/error.log . Run the tail command with the -F option

Tail -F /var/log/apache2/error.log

If the system moves (renames) the error.log file to the error.log.1 file and creates a new error.log file, then our tail command will continue to monitor the already new error.log file

If we were to use the option in this example -f , then the tail command would continue to monitor the error.log.1 file, which is no longer relevant for us when viewing the logs in real time.

Tracking multiple log files at the same time

The tail command supports tracking multiple files at the same time. To do this, you must specify the file names separated by spaces.

Tail -f /var/log/apache2/error.log /var/log/apache2/access.log

As soon as any of the files changes, the name of this file and new entries in it are displayed on the screen.

To track several log files at the same time, there is a very handy multitail utility.

It does not just display data, but creates its own window (area) for each file and displays data in this window. With its help, it is very convenient to track many log files at once and see them in one terminal window.

The multitail utility can be installed from the regular repositories of your distribution. To install, run the command (choose the appropriate command for your distribution):

sudo apt install multitail sudo yum install multitail sudo dnf install multitail

Usage:

Multitail /var/log/apache2/access.log /var/log/apache2/error.log

Press the q key to exit the utility

Before starting with this guide, it is recommended that you log into your Linux and follow this guide by looking directly at the files, as this is the best way to learn and remember a topic.

Learn about log files when your system is running smoothly, how this understanding of log files will help you successfully diagnose and fix any problems that may come up later.

And finally, ‘ Linux log files‘ is a fairly broad topic and is unlikely to be fully covered in one article. This article can probably only serve as a general guide. Each application installed on the system has its own logging mechanism. Whenever you need application specific information, the application documentation is the best place to look for it.

General Log Files

As a general standard on almost every Linux system, log files are found in the /var/log directory. Any other applications you may later install on your system will probably leave their log files here. After logging into your system, type the command

Ls -l /var/log

To view the contents of this directory.

/var/log/messages– Most of the general system messages are logged here, including messages during system startup.
/var/log/cron– The cron daemon messages are logged here. Tasks are created and stopped, as well as error messages.
/var/log/maillog or /var/log/mail.log– Registering the information of the mail server running on the server.
/var/log/wtmp– Contains a history of all entries and exits.
/var/log/btmp– Records of failed login attempts.
/var/run/utmp– Logging the actual login to the state of each user.
/var/log/dmesg– Contains very important messages about the ring buffer core. In human terms, this means that when the nucleus spins up, it writes all the information here. Team dmesg can be used to view the messages in this file.
/var/log/secure– Security-related messages will be stored here. This includes messages from the SSH daemon, password failures, non-existent users, and so on.
/var/log/mariadb– If MariaDB is installed on the system, then this is where it will drop the logs by default
/var/log/mysql– If the database is installed, this is the default data entry directory.

View and manage log files

The primary registration of the Linux daemon is the rsyslogd process and its configuration is located in /etc/rsyslog.conf.

For all plain text log files, the logs can be viewed using the cat command. However, if the log file is very large, then you may want to use the tail command, which can only show the last part of the log.

To view the last 500 entries in a file, type the following command:

Tail -n 500 /var/log/messages

For monitoring logs in real time, tail -f is also a very useful command that will track messages as they are entered. This is especially useful when troubleshooting mail flow and mail delivery errors.

Tail -f /var/log/maillog

Some Linux logs are like binary files that need to be parsed by another application specially adapted to view these logs. These logs are stored in /var/log/wtmp /var/log/btmp and /var/run/utmp .

To view the contents of /var/log/wtmp use: last
To view the contents of /var/log/btmp use: lastb
To view the contents of /var/run/utmp use: who

cpanel specific log files

Apache log files:

/usr/local/apache/logs/ - General logs.
/usr/local/apache/domlogs/ - Domain specific logs.

Exim log files:

/var/log/exim_mainlog
/var/log/exim_rejectlog

cpanel log files:

/usr/local/cpanel/logs/ - All cpanel related logs in this location.

DirectAdmin specific log files

DirectAdmin log files

/var/log/directadmin/ - DirectAdmin related logs.

Log files

/var/log/httpd/ - The Apache web server logged into the standard directory.
/var/log/httpd/domains/ - For all other domains, the logs are in this subdirectory.

Log files

/var/log/proftpd/ - If using ProFTPd.
/var/log/pureftpd.log - If using PureFTPd.

Exim log files

/var/log/exim/ – Logs of the Exim Mail Transfer Agent in this directory.

Log files

/var/lib/mysql/server.hostname.com.err - This is the directory logging for errors related to .

Log files

/var/log/yum.log - Logging the Yum package manager.
/var/log/httpd - On /RedHat CentOS based systems, this is where the Apache web server will store logs by default.

Log files

/var/log/apache2/ - On Ubuntu systems, Apache web server logs are stored in this directory.
/var/log/apt/ - Logs from package management in Ubuntu.

All log files are located in /var/log directory. In that directory, there are specific files for each type of logs. For example, system logs, such as kernel activities are logged in syslog file.

Some of the most common log files in that directory is:

    In directory apt there is a file history.log which saves all the package installation and removal information even the initial system build as Live CD. You can open this file to see this very interesting file.

    In directory dist-upgrade there is a file apt.log which logs the information during distribution upgrades

    In directory installer the log files which are created during installation can be found.

    There is an apport.log file which saves information about crashes in your system and reporting them.

    The file auth.log includes information about the authentication activities such as when you authenticate as root user via sudo.

    The file dpkg.log saves the low level details of package installation and removal related to dpkg . You might be aware that the apt system depends on dpkg for package installation and removal.

    boot.log includes information of each booting.

    kern.log saves kernel information such as warnings, errors etc.

    alternatives.log includes the history of all the alternatives set by various packages and their removal via update-alternatives command.

    Another important log file is Xorg.log which include information about the graphics driver, its failures, warnings etc.

Some other types of Log files may be there depending on your installed packages. For example, My system also includes a log files epoptes.log which will only be there if you install epoptes package.

Changes after systemd

With the advent of systemd , logging is mostly handled by journalctl utility and store the logs in binary format in /var/lib/systemd/catalog/database file. This file enumerates all logs including kernel, boot and application logs and provides required logs via journalctl utility.

Here is a good article on journalctl on how you can use it to fetch required log info.

A computer
The most interesting:
How to connect the printer via LAN
Instagram self-promotion tools
How to make money on Glopart