webpointmorpheus Linux Info
Backups and Log Files

     Introduction      Common Log Files      Log File Configuration      Log File Management
     File Compression      Backup      Backup Devices      Backup Utilities      Documents in This Series

©2005 - material compiled by Bob Carnaghi, www.webpointmorpheus.com

Introduction     Top of Page

Administering a Linux system is facilitated by several log files and daemons that log a wealth of system information to the log files. Listed below are some of the more common log files. There are more log files available on the system that are software specific. Check your system for these additional log files. Additionally, system backups are one of the most important elements to the stability of an organization. In many cases, the position of a system administrator is as secure as his most recent backup, and how well he can get it restored during a time of need.

At system boot, the System Log Daemon (syslogd) is initialized. This daemon opens a socket (/dev/log) and receives information that other processes write to the socket. The log information is then written to the appropriate log file as determined in the /etc/syslog.conf file. Use the cat on this file to see what activity is to be logged where on the system. Information can be logged to remote systems by entering the server in the syslog.conf file using the @servername syntax in place of the logfile as listed.

Common Linux System Log Files
File	Description
boot.log	This log file is generated at system boot, and contains information about daemon initialization processes.
cron	This log file contains error messages that have been generated by the cron and at daemons.
dmesg	This log file contains hardware information that is generated during system boot.
maillog	This log file contains error messages that have been generated by the sendmail utility.
secure	This log file is generated in response to network access errors by the sshd and xinetd daemons.
wtmp	This log file contains a history of all login sessions. Use the utmpdump wtmp command to view this binary log file.
rpmpkgs	This log file reflects the activity of the Red Hat Package Manager and the errors that are generated.
xferlog	This log file contains information and error messages generated by the FTP daemon.
Xorg.0.log XFree86	This log file contains information and error messages generated by the XWindows System.
lastlog	This log file contains a list of users and their last login time, and must be viewed using the lastlog command.
messages	This log file contains information about daemon startup and runtime errors, etc.
Notes: These log files are typically located in the /var/log directory. Check the directory /var/log/ for the location of additional software-specific log files, such as mail, httpd, etc.

 

Log File Configuration     Top of Page

The syslog.conf file permits logging information based upon a facility and the priority given to the message. Edit the file to alter the information that is logged to the appropriate log file. The list below shows which settings can be altered and to what degree. See the /etc/syslog.conf file for comments, format, etc.

System Log Facilities     Top of Page

1. auth, or security - from the login system, getty program, or the su command.
2. authpriv - from the login system when authenticating users across the network or from a database.
3. cron - from the cron and at daemons.
4. daemon - from the system daemons.
5. kern - from the kernel.
6. lpr - from the lpd printer daemon.
7. mail - from the mail daemon.
8. mark - an internal syslogd daemon.
9. news - from news daemons.
10. syslog - from the syslog system.
11. user - from user processes.
12. uucp - from the UNIX to UNIX copy daemon.
13. local0-7 - local messages that can be defined for custom use.

System Log Priorities     Top of Page

1. debug - configures all information from a specific utility to be logged.
2. info - normal messages only.
3. notice - notifications that may prove useful at a later time, not critical.
4. warning, or warn - facility warnings, not critical.
5. error, or err - something that has gone wrong.
6. crit - system errors that demand attention.
7. alert - an error that needs immediate attention.
8. emerg, or panic - indicates broadcast type emergencies.

Log File Management     Top of Page

Log files tend to grow, and can consume valuable disk space. Good practice calls for clearing the log files periodically by using the  > 

redirection symbol. Additionally, the log files can be cleared by scheduling the logrotate utility, which is configured in the /etc/logrotate.conf file, and stored in the /etc/logrotate.d directory. The logrotate utility can be configured to create new log files, delete and rename the older ones, and much more. See the configuration file for more info.

File Compression     Top of Page

File compression is the process whereby a file is reduced in size ty a specific compression algorithm. There are several popular compression algorithms, and each has advantages and disadvantages, such as the compression ratio, etc. Read this wikipedia compression article for more information on data compression.

The Linux operating System offers three popular compression utilities, as listed. Each is discussed further below.

1. compress & uncompress - an older compression utility capable of 40-50% compression ratio.
2. gzip & gunzip - a newer compression utility that is similar to compress and capable of 60-70% compression ratio. Capable of different levels of compression, see the command flags for details.
3. bzip2 - differs from compress and gzip, and uses a different compression algorithm.

Listed below are commands for viewing the contents of compressed archives. For more details, see the man page for the compression utility, or for the view command.

1. zcat - command that lists the contents of a compressed archive.
2. zmore - command that lists the contents of a compressed archive page by page.
3. zless - command that lists the contents of a compressed archive page by page.
4. bzcat - command that must be used to view archives created with the bzip2 utility.
5. bzmore - command that must be used to view archives created with the bzip2 utility page by page.
6. bzless - command that must be used to view archives created with the bzip2 utility page by page.

Backup Theory and Routine     Top of Page

When files are backed up, they are typically stored in archives. File archiving has essentially two parts, one is collecting the files into an archive of some sort, the other is optionally compressing the data. Linux provides the tar (tape archive) command that offers options for performing either or both operations in one pass, as well as options to use the various compression utilities listed above. The typical options for the tar command are listed below. The mt command permits archives to be written directly to a tape, CD, or DVD drive using the ouptut file as listed in the table below. See the man page for details.

Not all data on a Linux system needs to be backed up. Backup files can easily become too large to work with and store. A typical system backup would include user home folders, configuration files, and other system critical data. Configuration files that are contained in the /etc directory, as well as those for specific applications, will need to be considered as part of a good system backup. Backup schedules and types (Full, Differential, Incremental) are to be considered as well. The backup files should be restored in a non-critical setting to verify that the backed up data is complete and correct. In many situations, a system administrator's job is as secure as his most recent backup as well as his ability to get that data back into operation quickly and effectively.

System Files That Represent Tape Devices
File	Description
/dev/st0	First rewinding SCSI tape device
/dev/st1	Second rewinding SCSI tape device
/dev/st2	Third rewinding SCSI tape device
/dev/nst0	First nonrewinding SCSI tape device
/dev/ht0	First rewinding ATAPI IDE tape device
/dev/nht0	First nonrewinding ATAPI IDE tape device
/dev/ftape	First floppy tape device

 

Three Backup Utilities     Top of Page

Linux offers the three utilities listed below for system backup. Each has advantages and disadvantates, as noted. Note that backing up large amounts of data to archives on disk is not always the best option. Backing up to tape and offsite storage is often the best approach to disaster recovery.

•  tar  - flexible, can create an archive on disk or on a tape device as listed above. See the command options listed below. tar cannot backup device files.
•  cpio  - also known as 'copy in/copy out'. Has more options than tar, namely support for long filenames and the ability to backup device files.
•  dump/restore  - a more specialized backup utility that is designed to backup entire file systems and store the listing of the filesystem in /etc/dumpdates. This utility only works on ext2 and ext3 filesystems, and is capable of performing full and incremental backups.

tar  Utility Options
Command Option	Description
-A	Appends archives together.
-c	Creates a new archive.
-f filename	Specifies the name or the device file where the archive is to be placed.
-h	Forces tar to back up the targets of symbolic links, and not the links themselves.
-j	Uses the bzip2 utility for compression and decompression of archives.
-P	Forces the use of absolute pathnames in an archive.
-r	Will append files to an archive.
-t	Will list the table of contents of an archive.
-u	Updates an existing archive by appending files only if they are newer than the ones inside the archive.
-v	Displays verbose file and directory information during operation.
-w	Will prompt the user for confirmation of actions that are to be performed.
-W	Will verify the contents of each archive after creation.
-x	Extracts the contents of an archive.
-z	Uses the gzip utility to compress/decompress an archive.
-Z	Uses the compress utility to compress/decompress an archive.
Notes: The tar utility makes available POSIX style command options. See the man page for details.

1. Introduction and History
2. Installation, Advanced Installation, and Usage
3. The Linux Kernel and the Boot Process
4. Filesystems - Management & Administration
5. The BASH and Other Shells
6. System Initialization and the X Environment
7. Linux Processes
8. Linux Administration, Peripherals, and Hardware
9. Software Installation and Management
10. Backups and Log Files
11. Performance and Problems
12. Network Configuration
13. Security
14. Key Linux Commands
15. Essential Linux Definitions