Introduction to Core Dump Analysis

The Introduction to Crash Dump Analysis and the SunOS Kernel course provides students with essential skills to diagnose and resolve system problems, beginning with whether a system problem is due to hardware or software. If it's hardware, which piece of hardware is causing the problem? If it's software, is there a patch that fixes this problem? What tools can be used to help identify the causes of system problems? Where can they be obtained, and how are they used? This course helps students to answer these questions and to identify and resolve basic Solaris Operating System (Solaris OS) problems successfully.

Students who can benefit from this course

• Experienced System Administrators, Support personnel, and Driver Developers

Audience
Course Topics Gathering Information

• Collect system crash data
• Determine if a system is hung
• Obtain a crash dump from a hung system
• Force crash dumps for hard hangs
• Describe the mechanism and system resources necessary to create crash dumps
• Describe how to configure a system to save crash dumps
• Troubleshoot crash dump failures
• Describe system information used to analyze a system crash

Analyzing Information

• Differentiate between software and hardware problems
• Interpret the output from the prtdiag utility
• Interpret the output from the Sun Explorer software
• Analyze patch information
• Describe the types of device names and their components
• Display the device tree
• Interpret information in the messages file
• Interpret memory error messages

Understanding Data Structures

• Describe how a C language structure looks in memory
• Describe how a pointer is used in the C language
• Describe linked lists
• Describe stacks
• Describe hashing

Analyzing System Crash Dumps

• Describe the information used for initial crash dump analysis
• Describe information sources within the SunSolve program that are useful for resolving system crashes
• Use the SunSolve program to locate technical information, bugs, and patches related to system crashes
• Use the adb debugger to obtain initial crash dump information
• Use the Modular Debugger (MDB) to obtain initial crash dump information
• Use Solaris Crash Analysis Tool (CAT) to obtain initial crash dump information

Introducing Kernel Features and Organization

• Describe the location of kernel modules on disk
• Describe the components of the SunOS software
• Describe user processes
• Describe interrupts
• Describe multithreading
• Describe the location of structure definitions

Exploring Kernel Services

• Describe what traps are and how they are processed
• Describe how system calls are processed
• Describe what synchronization and protection are and why they are needed
• Describe the Solaris OS locking mechanisms
• Describe signals and how they are processed
• Describe what callouts are and how they are stored and processed
• Describe what cyclics are and what parts of the kernel are clients of this service

Examining the Process Subsystem

• Describe multithreading in the kernel and processes
• Identify the structures used by the kernel to support a process
• Describe the process life cycle, including thread creation
• Describe how threads are scheduled
• Describe priority inversion and how the kernel resolves the problem
• Describe how time stamps are used in the kernel
• Describe how the kernel controls processors

Examining the Memory Subsystem

• Describe memory management issues from the kernel perspective
• Identify the process structures used in address translation
• Describe anonymous memory
• Describe kernel page structures and their use
• Describe the MMU and the HAT layer
• Describe the kernel address space
• Describe kernel memory allocation
• Describe paging and swapping

Examining the File and I/O Subsystem

• Describe the basic file system structure
• Describe the virtual file system framework
• Describe how file system mounting and path name resolution are accomplished
• Describe the use of the Directory Name Lookup Cache (DNLC)
• Describe the UFS file system specific data structures
• Describe process file management
• Describe the use of the specfs file system
• Describe the kernel device tree

Course Objectives

• Distinguish software-caused system crashes from hardware-caused system crashes
• Retrieve initial analysis information from a crash dump using Solaris CAT, ACT, the adb debugger, and the mdb debugger
• Distinguish among unique panic string, bad trap, and hang crash dumps
• Identify appropriate SunSolve search strings for unique panic string and bad trap crash dumps
• appropriate SunSolve search strings for unique panic string and bad trap crash dumps Use the SunSolve program to find bugs and cases related to the crash dumps or error messages
• Given a C structure declaration, describe how the data structure is stored in memory and how it looks when displayed by Solaris CAT, the adb debugger, and the mdb debugger
• Describe the major kernel subsystems and use Solaris CAT, the adb debugger, and the mdb debugger to display the subsystems' related data structures