A Debugger for HFS Plus Volumes

© Amit Singh. All Rights Reserved. Written in May 2004

What's New (and Old)?

hfsdebug is obsolete and has been retired because it can behave incorrectly under several circumstances on modern versions of Mac OS X. For the next generation of HFS Plus debugging, check out fileXray—hfsdebug's functionality is a small subset of what fileXray can do.

hfsdebug

A debugger is a program that facilitates debugging, which may be casually defined as the process of finding and fixing "bugs" in the object of interest: usually a piece of software, firmware, or hardware (although debugging may apply to any domain).

hfsdebug is a filesystem debugger for Apple's HFS Plus (hereafter referred to as HFS+) volume format. The term "debugger" may be somewhat of a misnomer for hfsdebug, because a key feature (rather, a limitation) of it is that it operates on an HFS+ volume only in read-only mode. Therefore, it cannot currently be used by itself to repair any defects in a volume, or to even write to a volume in any manner whatsoever. Nevertheless, hfsdebug is meant to be a useful tool in exploring HFS+ internals, as it allows you to browse, inspect, and analyze various aspects of HFS+ volumes. It can also calculate and display certain filesystem statistics. My original intent in creating hfsdebug was to quantify filesystem fragmentation under HFS+.

hfsdebug will often display many more details than are viewable via "usual" methods. Moreover, read-only operations that involve going through all objects on a volume should be faster with hfsdebug (for example, creating a list of the N largest, or the N most fragmented files on a volume). Still, hfsdebug is not supposed to be a replacement for standard file utilities, or of any existing API.

Volume Formats Supported

hfsdebug supports only the HFS+ volume format. The older HFS format is not supported. It does, however, support the following HFS+ variants:

Note that hfsdebug should work with HFS+ volumes regardless of whether they reside on a real, physical disk (or disk slice), or on a "virtual" disk (such as a disk image).

Architecture

hfsdebug aims to provide accurate and complete details of various HFS+ internal structures, as well as filesystem statistics, most of which are not available through an API such as POSIX (or even the Carbon File Manager, which is far more flexible than POSIX when dealing with HFS+ volumes). hfsdebug works on an HFS+ volume by accessing its raw device directly. This means several things:

When you run hfsdebug as a superuser (via sudo, say), it "drops" its privileges once it has opened a volume's raw device. Specifically, it sets its user and group id's to 99 and 99, respectively (that is, to those of the "unknown" user on Mac OS X).

Features

You can use hfsdebug to:

Usage

Specifying a volume

hfsdebug needs an HFS+ volume to work on. This volume may be specified in three ways:

# hfsdebug /mach_kernel OPTIONS ... # hfsdebug OPTIONS ...

# hfsdebug -d /dev/rdisk0s9 OPTIONS ...

# hfsdebug -V /Volumes/Music OPTIONS ... ... # hfsdebug -V /Volumes/Music/somefile.mp3 OPTIONS ...

Viewing the Volume Header

The following command displays the volume header of the specified device (in this example, the root device). If the volume has an HFS wrapper, the master directory block is also displayed. This option is similar to "hdiutil hfsanalyze", except that hfsdebug shows some additional details, while not showing the "alternate" structure(s).

# hfsdebug -v # HFS Plus Volume with HFS Wrapper Embedded offset = 11984 bytes Wrapper volume size = 78149032.00 KB/76317.41 MB/74.53 GB Embedded volume size = 78143052.00 KB/76311.57 MB/74.52 GB # HFS Wrapper Master Directory Block drSigWord = $4244 (BD) ... # HFS Plus Volume Header signature = 0x482b (H+) ... # Finder Info finderInfo[0] = 0x1077 (Macintosh HD:/System/Library/CoreServices) finderInfo[1] = 0 finderInfo[2] = 0 finderInfo[3] = 0x30894 (Macintosh HD:/System Folder) ... # Catalog File logicalSize = 251658240 bytes totalBlocks = 61440 fork temperature = /* Metadata Zone */ clumpSize = 8388608 bytes extents = startBlock blockCount % of file 0xc01 0x5800 36.67 % 0xed8c 0x800 3.33 % 0x1e70a 0x800 3.33 % 0x2add4 0x1000 6.67 % 0x4d4b4 0x800 3.33 % 0x55c39 0x800 3.33 % 0xdddb2 0x800 3.33 % 0x140b0e 0x800 3.33 % 38912 allocation blocks in 8 extents total. 4864.00 allocation blocks per extent on an average. ...

Viewing headers of the Volume B-Trees

HFS+ uses several B-Trees for storing metadata. hfsdebug allows you to view the contents of the header nodes of these data structures:

# hfsdebug -b catalog ... # hfsdebug -b extent ... # hfsdebug -b hotfile # HFS+ Hot File Clustering (HFC) B-Tree # B-Tree Node Descriptor fLink = 0 bLink = 0 kind = 1 (kBTHeaderNode) height = 0 numRecords = 3 reserved = 0 # B-Tree Header Record treeDepth = 2 rootNode = 3 leafRecords = 21244 firstLeafNode = 88 lastLeafNode = 1 nodeSize = 4096 bytes maxKeyLength = 10 bytes totalNodes = 144 freeNodes = 28 reserved1 = 0 clumpSize = 65536 (ignored) btreeType = 128 (kUserBTreeType) keyCompareType = 0 (unspecified/default) attributes = 00000000000000000000000000000010 . kBTBigKeys (keyLength is UInt16) # User Data Record magic = 0XFF28FF26 version = 1 duration = 216000 seconds timebase = Sun May 2 09:02:27 2004 timeleft = 79068 seconds threshold = 16 maxfileblks = 2560 blocks maxfilecnt = 1000 tag = CLUSTERED HOT FILES B-TREE

Note that hfsdebug only knows about the on-disk structures -- not their in-memory counterparts. If some information exists only in memory, hfsdebug will not display it until it is flushed to disk.

Viewing records contained in the Volume B-Trees

hfsdebug lets you list details of all leaf records residing in the Volume B-Trees. Since some trees have more than one type of leaf records, you may also specify the type(s) of records to list:

# hfsdebug -b catalog -l file ... # hfsdebug -b catalog -l file -l folder ... # hfsdebug -b catalog -l filethread ... # hfsdebug -b hotfile -l hfcfile ... # hfsdebug -b hotfile -l hfcthread ... # hfsdebug -b extent -l any ...

The "any" keyword causes all records to be listed for the specified B-Tree. Note that this may result in a very large amount of output. For example, running "hfsdebug -b catalog -l any" produces almost a Gigabyte of data for a volume with about 600,000 files.

Viewing details of filesystem objects

hfsdebug can be used to view details of files, folders, and other kinds of objects on HFS+ volumes, including those that are not exposed by the filesystem to the user (for example, indirect node files, hard link files, journal files, etc.) You may specify an object in three ways:

  1. Its POSIX visible path (specified implicitly as the last argument, or using the -p option)

    # hfsdebug /usr/bin/gcc ... # hfsdebug -p /bin/ls ...

  2. Its Catalog Node ID (specified using the -c option)

    # hfsdebug -c 2 ...

  3. Its name and the Catalog Node ID of its parent (specified using the -F option)

    # hfsdebug -F 2:.journal ...

The third way is needed when an object's POSIX path is not visible at the user-level, and you don't know its Catalog Node ID: for example, the journal files.

# hfsdebug /System/Library/CoreServices/BootX path = Macintosh HD:/System/Library/CoreServices/BootX # Catalog File Record type = file file ID = 186444 flags = 0000000000000010 . File has a thread record in the catalog. reserved1 = 0 createDate = Mon Oct 6 19:28:33 2003 contentModDate = Fri Mar 26 04:23:24 2004 attributeModDate = Fri Mar 26 04:23:24 2004 accessDate = Mon Oct 6 20:08:00 2003 backupDate = Fri Jan 1 00:00:00 1904 # BSD Info ownerID = 0 (root) groupID = 0 (wheel) adminFlags = 00000000 ownerFlags = 00000000 fileMode = -rw-r--r-- linkCount = 0 textEncoding = 0 reserved2 = 0 # Finder Info fdType = 0x74627869 (tbxi) fdCreator = 0x63687270 (chrp) fdFlags = 0000000000000000 fdLocation = (v = 0, h = 0) opaque = 0 # Data Fork logicalSize = 157892 bytes totalBlocks = 39 fork temperature = not a hot file clumpSize = 0 bytes extents = startBlock blockCount % of file 0x4bfd6e 0x27 100.00 % 39 allocation blocks in 1 extents total. 39.00 allocation blocks per extent on an average. # Resource Fork logicalSize = 0 bytes

Note that as would be the correct behavior for a "debugger" of this kind, hfsdebug does not follow hard links and symbolic links automatically. It does, however, display the link target of a hard or symbolic link.

Viewing a journaled volume's journal

hfsdebug can be used to view information about, and dump the contents of, a volume's journal.

# hfsdebug -j # HFS+ Journal # Journal Info Block flags = 00000000000000000000000000000001 . Journal resides on local volume itself. device_signature = 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 offset = 104861696 bytes size = 8388608 bytes reserved = 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 0x00000000 # Journal Header magic = 0x4a4e4c78 endian = 0x12345678 start = 5090816 bytes end = 3207680 bytes size = 8388608 bytes blhdr_size = 12288 bytes checksum = 0x802a54c0 jhdr_size = 512 bytes # Block List Header max_blocks = 767 num_blocks = 2 bytes_used = 20480 checksum = 0 pad = 0 binfo[0].bp = 0 block_info[ 1] { bnum 0000000000a0b010 bsize 8192 bytes bp 0x17d5477c } ... output continues ...

Viewing the "hottest" files on a volume

We saw that you can list hot file records using "hfsdebug -b hotfile -l ...". You can also use hfsdebug to list the top N hottest files, as tracked and determined by Apple's Hot File Clustering scheme, that is available on Mac OS X 10.3.x boot volumes:

# hfsdebug -H -t 10 # Top 10 Hottest Files on the Volume rank temperature cnid path 1 469 9165 Macintosh HD:/private/var/db/netinfo/local.nidb/Store.128 2 372 237719 Macintosh HD:/usr/libexec/gcc/darwin/ppc/3.3/specs 3 359 1047016 Macintosh HD:/private/var/vm/app_profile/0_names 4 337 238241 Macintosh HD:/usr/libexec/gcc/darwin/ppc/3.3-fast/specs 5 305 9133 Macintosh HD:/usr/share/zoneinfo/US/Pacific 6 305 9166 Macintosh HD:/private/var/db/netinfo/local.nidb/Store.160 7 282 250281 Macintosh HD:/usr/include/ppc/types.h 8 282 250287 Macintosh HD:/usr/include/sys/appleapiopts.h 9 282 250295 Macintosh HD:/usr/include/sys/cdefs.h 10 282 250310 Macintosh HD:/usr/include/sys/fcntl.h 610 active Hot Files.

Viewing volume usage summary

hfsdebug can be used to calculate and display various usage statistics about a volume. It can optionally be instructed to list the top N largest sized files on the volume.

# Volume Summary Information files = 563142 folders = 89762 aliases = 4 hard links = 4394 symbolic links = 9856 invisible files = 602 empty files = 5269 # Data Forks non-zero data forks = 556267 fragmented data forks = 408 allocation blocks used = 11629019 allocated storage = 47632461824 bytes (46516076.00 KB/45425.86 MB/44.36 GB) actual usage = 46214263456 bytes (45131116.66 KB/44073.36 MB/43.04 GB) total extent records = 556319 total extent descriptors = 557328 overflow extent records = 52 overflow extent descriptors = 347 # Resource Forks non-zero resource forks = 7570 fragmented resource forks = 18 allocation blocks used = 93739 allocated storage = 383954944 bytes (374956.00 KB/366.17 MB/0.36 GB) actual usage = 363034266 bytes (354525.65 KB/346.22 MB/0.34 GB) total extent records = 7570 total extent descriptors = 7598 overflow extent records = 0 overflow extent descriptors = 0 # Largest Files on the Volume # Using 1 KB = 1024 bytes, 1 MB = 1024 KB, 1 GB = 1024 MB rank size cnid path 1 700.20 MB 988570 Macintosh HD:/work/Linux/KNOPPIX.toast ... output continues ...

Quantifying fragmentation on a volume

hfsdebug can be used to quantify how fragmented an HFS+ volume is, which was my original reason for creating it.

The -f option causes hfsdebug to list all files (forks, to be precise) that have non-zero fragmentation. Thus, any data or resource fork that occupies more than one extent is listed, along with the number and size of each of its fragments. Moreover, you can instruct hfsdebug to create a list of the top N files with the most extents.

Refer to Fragmentation in HFS+ Volumes for a more detailed discussion on this issue.

# hfsdebug -f -t 5 ... cnid=2364913 fork=data map=:1728:773: bytes=10241935 blocks=2501 extents=2 \ avg=1250.50 blks/ext path=Macintosh HD:/Applications/3rd Party/Microsoft \ Office 2004/Shared Applications/Proofing Tools/German Grammar Dictionary cnid=2364937 fork=data map=:192:168: bytes=1473424 blocks=360 extents=2 \ avg=180.00 blks/ext path=Macintosh HD:/Applications/3rd Party/Microsoft \ Office 2004/Shared Applications/Proofing Tools/Norwegian NYN Dictionary cnid=2364952 fork=data map=:240:333: bytes=2346969 blocks=573 extents=2 \ avg=286.50 blks/ext path=Macintosh HD:/Applications/3rd Party/Microsoft \ Office 2004/Shared Applications/Proofing Tools/Portuguese Thesaurus Dictionary cnid=2365831 fork=data map=:33:5:97:1:30:67:3:1:2:1:7:2:3:7:1:5:36: \ bytes=1229704 blocks=301 extents=17 avg=17.71 blks/ext path=Macintosh \ HD:/private/var/vm/app_profile/0_data ... # Top 10 Files with the Most Extents on the Volume rank extents blk/extents cnid path 1 694 28.43 1881709 Macintosh HD:/sw/var/mysql/lxr/useage.MYI 2 371 71.37 1881710 Macintosh HD:/sw/var/mysql/lxr/useage.MYD 3 281 104.16 1881707 Macintosh HD:/sw/var/mysql/lxr/symbols.MYD 4 267 61.36 2369116 Macintosh HD:/private/tmp/hfs+.dmg 5 104 78.33 45746 Macintosh HD:/System/Library/Caches/fontTablesAnnex Out of 527454 data forks total, 526428 (99.81 %) have no fragmentation. Out of 8119 resource forks total, 8081 (99.53 %) have no fragmentation.

Qualifying All Free Space

hfsdebug can go through the HFS+ Allocation File and enumerate all free extents: an indicator of "free space fragmentation", if you will.

The -0 option will print a list of triplets for the volume, with each triplet consisting of the number of free contiguous extents, the block on the volume at which the extent starts, and the amount of space it represents. For example:

# hfsdebug -0 # Free Contiguous Starting @ Space 128928 68520 503.62 MB 1 197456 4.00 KB 1 197462 4.00 KB 1 197920 4.00 KB 1 198191 4.00 KB ... 15412 12819025 60.20 MB 6701325 12834438 25.56 GB Allocation block size = 4096 bytes Allocation blocks total = 19535763 (0x12a1793) Allocation blocks free = 8163575 (0x7c90f7)

Note that the free extents list is printed in block order: it is not sorted by the amount of free space the extents represent. You can obtain a sorted listed by piping the output through sort:

# hfsdebug -0 | sort -n ... 31850 12611790 124.41 MB 45861 12689954 179.14 MB 81709 213206 319.18 MB 128928 68520 503.62 MB 6701325 12834438 25.56 GB

This way, you can list the free extents with the biggest ones last. In this example, we can see that the largest amount of contiguous free space on the volume is 25.56 GB.

Further Reading

I add new features to hfsdebug every once in a while. To know what more you can do with hfsdebug, take a look at the following links.

Summary of Options

hfsdebug: Command-line Options

Download


You must read and agree to this site's terms and conditions before downloading or using any software or other material available from this site.

hfsdebug is obsolete and has been retired. Take a look at fileXray. You may also want to look at this comparison between fileXray and hfsdebug.