Process Photography on Mac OS X
(Handcrafting Process Core Dumps)

© Amit Singh. All Rights Reserved. Written in May 2007

Introduction

Depending on one’s development and debugging scenario, one could get by without ever having to “photograph” a process—that is, take a snapshot of the memory and register state(s) of an existing process and save it to a file. Such a file, a “core” file, can be invaluable if the existence of a process is geographically or temporally separated from the debugging or analysis you need to perform on that process. For example, it could be that the person who needs to debug is somewhere else currently, or that you need to otherwise perform offline analysis, and so on.

In this document, we will discuss the process core dumping mechanism in Mac OS X. We will also take a historical look at core files in ancient UNIX. We will then discuss some drawbacks of the existing mechanism in Mac OS X and how we could improve it. Finally, a full-blown open-source program that implements these improvements will be provided.

Looking Back

In late 1969, the very first version of UNIX, essentially 0th Edition UNIX, ran on the PDP-7, a minicomputer sold by Digital Equipment Corporation. The PDP (Programmed Data Processor) series of machines used magnetic core memory—an early form of random access memory developed around 1950. It was called so because it physically consisted of small ferromagnetic rings or “cores”.

1969 was the year that the first ARPANET node became operational. The first Internet RFC—a “Request for Comments” document—was also published in 1969.

Soon afterwards, the UNIX “group” at Bell Laboratories made efforts to convince their management to acquire a better machine, the PDP-11. They promised to deliver a document-editing and formatting system meant to run standalone, without UNIX, while using UNIX only as a development platform. The first PDP-11 they received was an 11/20 with 24KB of core memory. UNIX ran on the PDP-11 in early 1971. Of the available memory, 12KB was used by the kernel, some was used by user programs, and the rest was used as a RAM disk.

In comparison, the size of the executable generated for an “empty” C program—main(){}—is a little over 12KB on the x86 version of Mac OS X.

“Dumping” process memory (or kernel memory, for that matter) contents to a file has long been useful in debugging and analysis. The mechanism for obtaining such a file—a core dump file or simply a core file—was present even in ancient UNIX. In the UNIX paper, Ritchie and Thompson said:

“The PDP-11 hardware detects a number of program faults, such as references to nonexistent memory, unimplemented instructions, and odd addresses used where an even address is required. Such faults cause the processor to trap to a system routine. When an illegal action is caught, unless other arrangements have been made, the system terminates the process and writes the user’s image on file core in the current directory. A debugger can be used to determine the state of the program at the time of fault.”

The UNIX Time Sharing System, Dennis M. Ritchie and Ken Thompson, Fourth ACM Symposium on Operating System Principles, October 15-17, 1973

Copyright © 1974, Association for Computing Machinery, Inc.

Besides “illegal actions” causing core dumps, the user could send the quit signal to a process to force a core image to be produced. This general logic remains the same in today’s systems. As is also the case with typical modern-day core files, ancient UNIX core files contained the actual contents of the memory at the time of fault along with the register state of the process. Of course, today’s core files would be several orders of magnitude larger.

Core Dumps on Mac OS X

The only way a process core dump can be generated in Mac OS X is through and during signal processing. When a signal is delivered to a process, what happens thereafter depends on the current disposition for that signal in the context of that process. Section 9.8.7 (Signal Generation and Delivery) of the Mac OS X Internals book discusses in detail how signals work in the Mac OS X kernel.

Signals that Cause Core Dumps

Without going into the excruciating minutiae of signal handling, suffice it to know that unless a process changes (say, by catching or blocking the signal) the disposition of a signal, the kernel arranges for the default disposition to take effect. Some signals, specifically those listed in Table 1, generate a core image by default on Mac OS X. In the case of these signals, the kernel will dump core and terminate the process.

Table 1. Signals that Generate a Core Image

SignalNumberDescription
SIGQUIT3quit program
SIGILL4illegal instruction
SIGTRAP5trace trap
SIGABRT6abort program
SIGEMT7emulate instruction executed
SIGFPE8floating-point exception
SIGBUS10bus error
SIGSEGV11segmentation violation
SIGSYS12non-existent system call invoked

If you are running a program as a foreground process from a shell, you can typically send several useful signals from the shell itself, without even using the kill program. The stty command can be used to display what signals could be sent. For example, Figure 1 shows that in you could type ^\ to send a SIGQUIT.

$ stty -a ... cchars: discard = ^O; dsusp = ^Y; eof = ^D; eol = <undef>; eol2 = <undef>; erase = ^?; intr = ^C; kill = ^U; lnext = ^V; min = 1; quit = ^\; reprint = ^R; start = ^Q; status = ^T; stop = ^S; susp = ^Z; time = 0; werase = ^W;

Figure 1. Viewing terminal characteristics from a shell


Core File Size Limit

Triggering a core dump explicitly or implicitly will actually result in a core file only if the per-process resource limit allows that. The RLIMIT_CORE limit specifies the largest size (in bytes) core file that may be created. Figure 2 shows how you can view from a shell the current settings of various resource limits.

$ limit # zsh cputime unlimited filesize unlimited datasize 6MB stacksize 8MB coredumpsize 0kB addressspace unlimited memorylocked unlimited maxproc 266 descriptors 256 ... $ ulimit -c # bash, csh, ksh, zsh 0

Figure 2. Viewing the core file size limit in various Mac OS X command shells


As Figure 2 shows, the coredumpsize value is 0 for this shell. This is the default on Mac OS X. Resource limits are copy-on-write shared in the kernel across processes. During bootstrapping, the BSD portion of the kernel creates process 0, which represents the kernel itself. As Figure 3 shows, the RLIMIT_CORE value for this process is set to 0. Unless you change this value for a process (either programmatically or in a shell), any given process will continue to have an allowed core dump size of 0 bytes. In other words, a core file will simply not be generated.

// xnu/bsd/[i386|ppc]/vmparam.h #define DFLCSIZ (0) /* initial core size limit */ #define MAXCSIZ (RLIM_INFINITY) /* maximum core size limit */ // xnu/bsd/kern/bsd_init.c struct rlimit vm_initial_limit_core = { DFLCSIZ, MAXCSIZ }; void bsd_init() { register struct proc *p; ... kernproc = &proc0; /* process 0 */ p = kernproc; ... limit0.pl_rlimit[RLIMIT_CORE] = vm_initial_limit_core; ... }

Figure 3. The system-default core file size limit of 0 being initialized at bootstrap time


Figure 4 shows how you can set the core dump file size to “unlimited” from a shell prompt. Any processes that you run from this shell will get the new resource limit.

$ ulimit -c unlimited $ ulimit -c unlimited

Figure 4. Setting the resource limit on core file size from the shell command line


Figure 5 shows how you can set RLIMIT_CORE programmatically in C. The rlimit structure contains two limit values: a soft limit and a hard limit. The value RLIM_INFINITY is used to specify “infinity”.

#include <sys/types.h> #include <sys/time.h> #include <sys/resource.h> ... int ret; struct rlimit rl = { RLIM_INFINITY, RLIM_INFINITY }; ret = setrlimit(RLIMIT_CORE, &rl);

Figure 5. Programmatically setting the resource limit on core file size


Core File Name

Besides determining how large a core file to write, the kernel needs to determine the core file's location, including the file name itself. The kern.corefile sysctl variable influences this decision on Mac OS X. Figure 6 shows how you can view the value of this variable using the sysctl command-line program.

$ sysctl kern.corefile kern.corefile = /cores/core.%P

Figure 6. Viewing the core file name pattern maintained by the kernel


We see that the default value of kern.corefile is “/cores/core.%P”. The kernel consults this variable prior to initiating a core dump. In this case, the %P will be replaced by the process identifier (pid) of the target process. For example, a process with pid 1234 will have a core file called /cores/core.1234. Figure 7 shows an example of this behavior. Note that the assumption is that you already have set a non-zero RLIMIT_CORE.

$ sleep 60 ^\zsh: quit (core dumped) sleep 60 $ ls -lh /cores total 306304 -r-------- 1 singh admin 149M May 12 15:57 core.8173 $ rm -f /cores/core.8173

Figure 7. Causing a core dump


The kernel allows a little more flexibility in specifying core file name patterns. Besides %P, you can use %N (process name) and %U (user identifier) in the pattern. You also don't have to put all core files in the /cores/ directory. Figure 8 shows an example of using a non-default pattern.

$ sudo sysctl -w kern.corefile="%N-%P-%U.core" kern.corefile: /cores/core.%P -> %N-%P-%U.core $ cd /tmp $ sleep 60 ^\zsh: quit (core dumped) sleep 60 $ ls -lh *.core -r-------- 1 singh wheel 149M May 12 16:00 sleep-8178-501.core $ rm -f sleep-8178-501.core

Figure 8. Customizing the core file name pattern


Now that we know how to obtain a core file in Mac OS X, let us look at using one—it is operationally quite similar to debugging a live process. Figure 9 shows an example. We compile a simple C program called sleeper.c, cause it to dump core, and use gdb to “look” at the core file in conjunction with the executable. As you normally would while debugging, you can look at the stack traces of the various threads, examine memory, and so on.

$ cat sleeper.c /* sleeper.c */ #include <stdio.h> #include <stdlib.h> #include <unistd.h> int main(void) { printf("%d\n", getpid()); sleep(60); exit(0); } $ gcc -Wall -o sleeper sleeper.c 10213 ^\zsh: quit (core dumped) ./sleeper $ gdb ./sleeper /cores/core.10213 ... Core was generated by `./sleeper'. #0 0x90041518 in mach_wait_until () (gdb) info threads * 1 core thread 0 0x90041518 in mach_wait_until () (gdb) where #0 0x90041518 in mach_wait_until () #1 0x900412e4 in nanosleep () #2 0x90041110 in sleep () #3 0x00002b4c in main ()

Figure 9. Examining a core file with gdb


The example shown in Figure 9 was run on the PowerPC version of Mac OS X. As of this writing (May 2007), the x86 version of Mac OS X has some problems that make it not-so-useful for analysis with gdb. Figure 10 shows an example. More specifically, the thread states that are dumped by the kernel during core file generation are not what gdb is expecting. This is easily fixed in the operating system, but there's not much you can do as an “end developer”.

$ uname -m i386 ... $ gdb ./sleeper /cores/core.8294 ... Core was generated by `./sleeper'. Core file contained no thread-specific data (gdb) where #0 0x00000000 in ?? () (gdb)

Figure 10. Examining a core file with gdb fails on the x86 version of Mac OS X (as of version 10.4.9, mid 2007)


Making Mac OS X Processes Dump Core Better

Given what we have discussed so far about core dumping on Mac OS X, several aspects of the mechanism could be nicer. For example:

This is where the power of low-level user-space programming interfaces in Mac OS X comes into play. “User space” is key—it is particularly interesting and useful that developers can perform major system-level operations without having to write any kernel code. In fact, it is possible to implement a fully functional core-dumping user-space program that can safely generate a core dump file from a running process. To achieve this, we need to be able to do several things, the most important of which are enumerated below.

Details on how to perform most of these operations can be found in the book Mac OS X Internals: A Systems Approach.

We will call our program gcore in honor of a utility of the same name and purpose that appeared in 4.2BSD (circa 1984). The source for gcore can be downloaded through the download link towards the end of this document. Figure 11 shows how to unpack the tarball and compile the source. Note that gcore should work on both the x86 and PowerPC platforms.

$ tar -xzf gcore-<version>.tar.gz $ cd gcore-<version> $ make gcc -O2 -arch ppc -arch i386 -Wall -o gcore gcore.c $ ./gcore usage: gcore [-c <corefile>] <pid>

Figure 11. Compiling the gcore program as a Universal binary


Figure 12 shows the gcore program in action. You do need superuser access to run gcore because some of the information retrieved during core-file generation can only be retrieved through superuser privileges. By default, gcore produces a core file named core.%P where %P is the pid of the target process. Alternatively, you can use the -c argument to specify another location for the core file. gcore dumps the “proper” thread states on the x86 version of Mac OS X, so gdb does work as it should.

$ cd /work/macfuse/filesystems/sshfs $ ps -hax PID TT STAT TIME COMMAND ... 8463 ?? Ss 0:00.00 ./sshfs user@host:/remote/path /local/volume ... $ sudo gcore 8463 $ ls -lh -rw------- 1 singh singh 153M May 12 16:54 core.8463 $ gdb ./sshfs ./core.8463 ... Core was generated by `./sshfs'. #0 0x9000598a in syscall () (gdb) info threads 4 core thread 3 0x9001075c in read () 3 core thread 2 0x9001075c in read () 2 core thread 1 0x9001075c in read () * 1 core thread 0 0x9000598a in syscall () (gdb) thread 1 [Switching to thread 1 (core thread 0)] 0x9000598a in syscall () (gdb) where #0 0x9000598a in syscall () #1 0x9003b3ef in sigsuspend () #2 0x9003b3cd in sigpause () #3 0x9003b38a in pause () #4 0x0002faf4 in g_mem_set_vtable (vtable=0x1800a00) at gmem.c:273 ... #11 0x000021d1 in do_chown (f=0xbffffa8c, req=0xbffffa94, path=... (gdb) up 11 ... (gdb) print path $1 = 0xbffffaad "/local/volume"

Figure 12. Using the gcore program on the x86 version of Mac OS X


Source Download

The gcore program is available in source form:

gcore-1.3.tar.gz

If you read the source of gcore, it would be interesting to compare it with the in-kernel core-dumping code in ancient UNIX. Reproduced in Figure 13 is the source of the core() function from Third Edition UNIX (early 1973). To help you understand the code better, I’ve reformatted and annotated it. Note that this is essentially the entire relevant code.

/* UNIX Third Edition, circa early 1973 */ /* ken/sig.c */ core() { int s, *ip; extern schar; /* u is the user area */ u.u_error = 0; /* reset error code to "no error" */ u.u_dirp = "core"; /* file name to search for */ ip = namei(&schar, 1); /* do search; schar means it's a kernel string */ if (ip == NULL) { /* failed to find */ if (u.u_error) /* because of some error */ return(0); /* so bail out */ ip = maknode(0666); /* didn't exist; so create it */ } if (!access(ip, IWRITE)) { /* check "write" permission; 0 means OK */ itrunc(ip); /* truncate the core file */ /* first we write the user area */ u.u_offset[0] = 0; /* offset for I/O */ u.u_offset[1] = 0; /* offset for I/O */ u.u_base = &u; /* base address for I/O (user area itself) */ u.u_count = USIZE*64; /* bytes remaining for I/O; USIZE=8 */ u.u_segflg = 1; /* specify kernel address space */ writei(ip); /* do the write */ /* * u_procp points to the process structure * p_size is the size of the process's swappable image (x 64 bytes) */ */ s = u.u_procp->p_size - USIZE; /* compute size left to write */ /* * This sets up software prototype segmentation registers to implement * text(=0 here), data(=s here), and stack(=0 here) sizes specified. */ estabur(0, s, 0); u.u_base = 0; /* base address for I/O (start of space) */ u.u_count = s*64; /* s is in units of 64 bytes, so adjust */ u.u_segflg = 0; /* specify user address space */ writei(ip); /* do the write */ } iput(ip); /* decrement inode reference count */ return(u.u_error==0); /* done */ }

Figure 13. Annotated core dumping code from the Third Edition UNIX kernel