Unix Tip: Using SCAT (Solaris CAT) for analyzing crash dumps

April 23, 2008, 01:20 PM —  ITworld.com — 

Want to do something with those crash dump files other than remove them? Want to extract some useful information without a lot of work? Take a look at Sun's free crash analysis tool, scat.

While "scat" in this context stands for "Solaris Crash Analysis Tool", I find myself drawing an analogy to the other meaning of the word. "Scat" refers to the dung of certain animals. Analysis of animal scat can tell you a lot about the health and the diet of the particular animals and can be useful in identifying the species as well. Similarly, scat analysis of the system variety can provide useful information for diagnosing the ailments of your Solaris servers -- in particular, what caused them to crash. Examining the "droppings" of a system can require considerable skill if you end up delving deeply into the crash dump files. At the same time, even the simplest use of the scat command can provide you with some of the most telling information about a crash. You simply give the tool the numeric identifier of the crash dump you want to know about. If your crash dump files are unix.0 and vmcore.0 (i.e., the first set of a possible collection of crash dump files), for example, you would issue commands such as these:

# cd /var/crash/boson
# /opt/SUNWscat/bin/scat 0

Notice that, since the scat program wants only the numeric identifier for the crash dump files, you must first cd to the directory in which these files are stored.

The unix.X and vmcore.X files are generated when a Solaris system panics and are normally stored in /var/crash/`uname -n`. The initial output from scat will identify the system and provide information on the date and time the system crashed, how long the system had been up at the time, the system type, hostid, panic string, etc. Here's an example:

  Solaris[TM] CAT 4.1 (build 526) for Solaris 9 64-bit SPARC(sun4u)

  Copyright © 2003 Sun Microsystems, Inc. All rights reserved.
  Patents Pending. Use is subject to license terms.
  Sun Microsystems proprietary - DO NOT RE-DISTRIBUTE!

  Feedback regarding the tool should be sent to SolarisCAT_Feedback@Sun.COM

opening vmcore.0 ...dumphdr...symtab...core...done
loading core data: modules...panic...memory...time...misc...done
loading stabs...read_type_db: Wrong number of lines in database, or database
doesn't end in a newline
unable to load any stabs file
patches... - NOT AVAILABLE (No such file or directory) done

core file:      /var/crash/boson/vmcore.0
user:           Super-User (root:0)
release:        5.9 (64-bit)
version:        Generic_112233-11
machine:        sun4u
node name:      boson
domain:         lab.particles.org
hw_provider:    Sun_Microsystems
system type:    SUNW,Sun-Fire-V210
hostid:         837844c7
time of crash:  Tue Apr 22 11:49:52 EDT 2008
age of system:  22 hours 5 minutes 4.48 seconds
panic cpu:      0 (ncpus: 1)
panic string:   free: freeing free block, dev:0x200000016e, block:32032, ino:6057255, 
                fs:/homes

running sanity checks.../etc/system...ndd...sysent...misc...done
SolarisCAT(vmcore.0)>
I like it!
Post a comment
The content of this field is kept private and will not be shown publicly.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.
Resources
White Paper

Symantec Backup Exec 12 and Backup Exec System Recovery 8 deliver industry leading Windows data protection and system recovery. Download this whitepaper to find out the top reasons to upgrade and how to get continuous data protection and complete system recovery.

Webcast

Data and system loss — from a hard drive failure, malicious attack, natural disaster, or simple human error — can happen anytime. Don’t leave your business vulnerable. Make sure you have a secure recovery strategy in place. Symantec's latest backup and system recovery technology can efficiently restore critical applications, individual emails and documents and even restore your entire system in minutes in the event of a loss.

White Paper

Businesses face a growing challenge to ensure that the IT environment is properly protected. Backup Exec 12 integrates with other applications in the Symantec family of products, to complement your current data protection strategy, keep your data securely backed up and make it recoverable when you need it most.

Free stuff

VMware ESX Server in the Enterprise
By Edward L. Haletky
Published Dec 29, 2007 by Prentice Hall.
Enter now! | Official rules | Sample chapter

Green IT
By Toby Velte, Anthony Velte, Robert C. Elsenpeter
To be published Oct. 10, 2008 by McGraw Hill Professional
Enter now! | Official rules | About the book

Featured Sponsor

AISO founders envisioned a Web hosting company that was environmentally friendly. While the company employed energy-efficient innovations like solar panels, its infrastructure produced unacceptable power and cooling requirements. Find out how AISO leveraged AMD technology to overcome their challenge in this case study white paper.

In this whitepaper, Scalar explores the opportunity to change the landscape with respect to mission critical databases built around Oracle. Leveraging technologies such as Linux, high-end commodity processing power and Oracle RAC technology to architect, design, build and maintain database infrastructure that delivers maximum availability, reliability and performance at a fraction of traditional cost.

On a typical day, weather.com, the Web site for The Weather Channel in Atlanta, serves up between 15 million and 20 million page views. But in September 2004, when back-to-back hurricanes ransacked Florida, the peak traffic on one day more than tripled: over 70 million page views by more than 7 million unique visitors. Read the full success story now.

More Resources