Paging vs Swapping

These two terms are associated with the virtual filesystem.

In the past, the cost of RAM (physical memory) is relatively high compared to the cost of hard disk drives.

During then, it was almost always preferrable to use the virtual file system.

In a nut shell, virtual filesystem is combining the use of  physical RAM and a particular slice or file in the hard disk  as the total memory.

In the event the system runs out of physical memory and a more important job is waiting to be executed it transfer some of its load in the RAM to the swap memory.

The term paging and swapping refers to the transfer of data from RAM to the HDD. They only differ on the bulk of data they transfer. In paging only portion on the data in the memory is being transferred while in swapping ALL data is being transfered.

Paging happens in normal operation. In many cases, it is totally inevitable not to page some of the memory contents to the swap space. Especially, on development servers where in the coreadm is using the swap space for its corefiles.

On the other hand, swapping happens when there is an abnormal event in the system. This happens very rarely, and if it does there must be something wrong with the configuration of the server.

Its very unlikely for a unix box to experience swapping.

SVR4

Just incase you are reading the admin guide and came across the term SVR4, here is a quick wrap-up of the story.

SVR4 - System V Release 4


Unix System V, commonly abbreviated SysV (and usually pronounced—though rarely written—as "System Five"), is one of the first commercial versions of the Unix operating system. It was originally developed by American Telephone & Telegraph (AT&T) and first released in 1983. Four major versions of System V were released, termed Releases 1, 2, 3 and 4. System V Release 4, or SVR4, was commercially the most successful version, being the result of an effort, marketed as Unix System Unification, which solicited the collaboration of the major Unix vendors. It was the source of several common commercial Unix features.


While AT&T sold their own hardware that ran System V (see AT&T Computer Systems), most customers ran a version from a reseller, based on AT&T's reference implementation. A standards document called the System V Interface Definition outlined the default features and behavior of implementations. The most widely used versions of System V today are IBM's AIX, based on System V Release 3, and Sun's Solaris and Hewlett-Packard's HP-UX, both based on System V Release 4.


In the 1980s and early-1990s, System V was considered one of the two major "flavors" of UNIX, the other being Berkeley Unix (BSD). During the period of the Unix wars System V was known for being the primary choice of manufacturers of large multiuser systems, in opposition to BSD's dominance of desktop workstations. However, with standardization efforts such as POSIX and the commercial success of Linux, this generalization is not as accurate as it once was.
 Read the rest...

System Logs

One of the major jobs that a typical UNIX administrator performs is the system monitoring.  He or she uses a lot of utilities, tricks, and systems regularly to aid in the process of administration.
 

Most UNIX systems use the syslog, a generic logging system that is used by the operating system and applications and services to log information. The syslog records a whole host of data, including logins, performance information, and failures reported by different hardware and systems. 

In addition to the syslog, systems also have a variety of service, environment, and application logs that record information about the machine and its operation. Syslog allows you to encode messages by level and by facility. Levels can be considered various levels of a problem whereas facilities are considered to be service areas. Syslog also allows you to forward log entries to another machine for processing, in this way syslog functions as a distributed error manager.

For the Solaris 10 OS, the syslog function, the syslogd daemon, and input from the /etc/syslog.conf file work together to facilitate system messaging.
The daemon can write messages to a system log, forward messages to a centralized log host, forward messages to a list of users, write messages to the system console.

In actual, the syslogd daemon runs the m4 macro processor to parse the /etc/syslog.conf file into statements that can be used by the syslogd process.
Comments in the /etc/syslog.conf file are not sent to the syslogd daemon.
If an ifdef statement is encountered it is evaluated for a True or False condition. A different value is sent to the syslogd daemon for each condition.
Statements that are not comments or ifdef statements are passed to the syslogd daemon as a selector field and an action field.

The location of the various log files varies from system to system. For Solaris 10 OS, most of the logs are in /var/adm and in /var/log

The levels available are the following:

 LOG_EMERG    A panic condition.  This is normally broadcast to all users.
 LOG_ALERT    A condition that should be corrected immediately such as a corrupted system database.
 LOG_CRIT    Critical conditions, e.g., hard device errors.
 LOG_ERR    Errors.
 LOG_WARNING    Warning messages.
 LOG_NOTICE    Conditions that are not error conditions, but should possibly be handled specially.
 LOG_INFO    Informational messages.
 LOG_DEBUG    Messages that contain information normally of use only when debugging a program.

And here are the facilities under which you can log messages:

 LOG_KERN    Messages generated by the kernel.  These cannot be generated by any user processes.
 LOG_USER    Messages generated by random user processes.  This is the default facility identifier if none is specified.
 LOG_MAIL    The mail system.
 LOG_DAEMON    System daemons, such as routed(1M), ftpd(1M), rshd(1M), etc.
 LOG_AUTH    The authorization system:  login(1), su(1M), getty(1M), etc.  ftpd(1M), and rshd(1M) also use LOG_AUTH.
 LOG_LPR    The line printer spooling system:  lpr(1), lpd(1M), etc.
 LOG_LOCAL0    Reserved for local use.  Similarly for LOG_LOCAL1 through LOG_LOCAL7.

Sample /var/adm/messages

Jun  13 09:16:59 reciosys01ata: [ID 496167 kern.info] cmdk2 at ata1 target 0 lun 0
Jun  13 09:16:59 reciosys01genunix: [ID 936769 kern.info] cmdk2 is /pci@0,0/pci-ide@1f,1/ide@1/cmdk@0,0
Jun  13 09:16:60 reciosys01asy: [ID 267298 kern.notice] asy0: UART @ 3f8 scratch register: expected 0x5a, got 0xff
Jun  13 09:16:60 reciosys01asy: [ID 702181 kern.notice] Cannot identify UART chip at 3f8
Jun  13 09:16:60 reciosys01asy: [ID 267298 kern.notice] asy1: UART @ 2f8 scratch register: expected 0x5a, got 0xff
Jun  13 09:16:60 reciosys01asy: [ID 702181 kern.notice] Cannot identify UART chip at 2f8
Jun  13 09:17:02 reciosys01genunix: [ID 314293 kern.info] device pciclass,030000@2(display#0) keeps up device sd@1,0(sd#1), but the latter is not power managed
Jun  13 09:17:02 reciosys01/usr/lib/power/powerd: [ID 387247 daemon.error]  Able to open /dev/srn
Jun  13 09:17:09 reciosys01/sbin/dhcpagent[164]: [ID 778557 daemon.warning]  configure_v4_lease: no IP broadcast specified for ni0, making best guess
Jun  13 09:17:32 reciosys01sendmail[503]: [ID 702911 mail.crit] My unqualified host name  (reciosys01) unknown; sleeping for retry
Jun  13 09:17:33 reciosys01sendmail[507]: [ID 702911 mail.crit] My unqualified host name (reciosys01) unknown; sleeping for retry
Jun  13 09:17:49 reciosys01svc.startd[7]: [ID 652011 daemon.warning]  svc:/system/webconsole:console: Method "/lib/svc/method/svc-webconsole start" failed with exit status 95.
Jun  13 09:17:49 reciosys01svc.startd[7]: [ID 748625 daemon.error] system/webconsole:console failed fatally: transitioned to maintenance (see 'svcs -xv' for details)
Jun  13 09:17:56 reciosys01pseudo: [ID 129642 kern.info] pseudo-device: devinfo0
Jun  13 09:17:56 reciosys01genunix: [ID 936769 kern.info] devinfo0 is /pseudo/devinfo@0
Jun  13 09:18:32 reciosys01sendmail[503]: [ID 702911 mail.alert] unable to qualify my own domain name (reciosys01) -- using short name
Jun  13 09:18:33 reciosys01sendmail[507]: [ID 702911 mail.alert] unable to qualify my own domain name (reciosys01) -- using short name


And in this example you can see some faults on tape drive being reported by a running filesystem.

Jun 11 14:08:29 reciosys01 bptm[10949]: [ID 842812 daemon.warning] TapeAlert Code: 0x03, Type: Warning, Flag: HARD ERROR, from drive MWC-30-F04-D03 (index 39), Media Id J15113
Jun 11 14:08:30 reciosys01 bptm[10949]: [ID 988370 daemon.warning] TapeAlert Code: 0x27, Type: Warning, Flag: DIAGNOSTICS REQ., from drive MWC-30-F04-D03 (index 39), Media Id J15113
Jun 11 14:08:30 reciosys01 bptm[10949]: [ID 325195 daemon.crit] TapeAlert Code: 0x37, Type: Critical, Flag: LOADING FAILURE, from drive MWC-30-F04-D03 (index 39), Media Id J15113

We’ll add some more in the future.

How to Find Large Files and Directories

One of the most repetitive tasks in an SA's work shift is finding large files and directories.
Here are some tips you can use to find the components that are taking up space in a UNIX file system.
This one liners works on most of the UNIX flavors but not all.

Finding Large Directories

To find large directories, use the du command and sort the output.

#du -k /var|sort -n | tail -10

This outputs the 10 largest directories in /var, sorted in ascending size order.

Solaris has this d option to avoid crossing file system boundaries, that is, to see the directory usage in / but not in the other mounted files systems (/var, /opt, and so on).

#du -kod /var|sort -n | tail -10

Finding Large Files

Use the find command to find large files,

#find / -xdev -type f -size +100000 -ls -exec du -sk {} \;

Now here is a sample that sorts the output.

#find / -size +100000 -type f -ls | sort -n

This finds all plain files in a file system larger than 100,000 512-byte blocks (approximately 50 Mbytes) and sorts the output.


For Solaris you can add a keydef argument. It restricts the sort key field definition. Here is an example that finds all plain files in a /var file system larger than 1,000 512-byte blocks and sort on field 7 which is the file size. It numerically ignores leading blanks.

#find /var -size +1000 -type f -ls | sort -k 7,7 -n

And here is a similar script for HPUX

#find . -type f -print|xargs ls -l|sort -r -n -k 5,5


There you go! Hope I can add some more in the future. Enjoy!