When you install the official MacFUSE package, you see the following rebooting-related notice.

To begin with, it is recommended (not required) that you restart your system. Unlike in the case of Mac OS X software installations that do necessitate a restart, you can simply close the Installer window after you are done installing MacFUSE.

Now, why is it even recommended that you restart? Let us get some context.

If you are installing MacFUSE for the first time (that is, you are not upgrading a MacFUSE installation), restarting is entirely unnecessary.

If you are upgrading a MacFUSE installation, restarting is a heavy-handed way for users to avoid the confusing situation described below—it is still not required to restart if you know how to avoid the confusing situation and are willing to deal with the extra steps needed.

MacFUSE consists of both kernel-resident and user-space-resident software components. The MacFUSE kernel extension is both dynamically loadable and dynamically unloadable. That is, you can simply unload it if it is not in use and reload a new version.

There is one important thing to know: the kernel-space and user-space components of MacFUSE are in lockstep. You cannot use an older user-space MacFUSE component with a newer MacFUSE kernel extension and vice versa. The aforementioned confusing situation arises when you upgrade MacFUSE while you have a MacFUSE-based file system mounted. The busy kernel extension cannot be unloaded, and the upgraded user-space components will refuse to work with the old kernel extension.

The user-space MacFUSE components actually ensure that the loaded kernel extension is the matching one. In fact, if you upgrade an installation while the MacFUSE kernel extension is loaded, the user-space components will try to unload the old extension and then try to load the new extension. This works, except in the case when the already loaded kernel extension is "busy," which would be the case if you do have a mounted MacFUSE-based file system.

If you first ensure that no MacFUSE-based file systems are mounted, you can upgrade MacFUSE, not restart, and once upgraded, start using the new MacFUSE. The lsvfs command-line tool can be used to determine if a MacFUSE-based file system is mounted.

$ lsvfs
Filesystem                        Refs Flags
-------------------------------- ----- ---------------
ufs                                  0 local
nfs                                  1
fdesc                                1
cd9660                               0 local
unionfs                              0
hfs                                  2 local, dovolfs
devfs                                1
autofs                               2
msdos                                0 local
fusefs                               0

If there is no entry for fusefs or if the corresponding value in the Refs column is 0, you shouldn’t have to do anything. If the value is non-zero, you need to unmount that many MacFUSE-based volumes.

To sum up, a restart is recommended so that end users don’t have to understand these details. Even then, the only time a restart will actually matter is the case when you upgrade MacFUSE while a MacFUSE-based file system is mounted and then you attempt to mount yet another MacFUSE-based file system using the upgraded version of MacFUSE.

Now that the talk’s video is available. I’ve written some notes on the new file systems themselves. There is a lot of new information and new code for those interested in file systems, Mac OS X, and operating systems in general. Much of it is academically and practically useful, and some could have significant potential down the road. You could also think of it as a new year gift to the Mac OS X and open-source communities.

This is a meta note on how you can go about discovering and understanding the what, how, and why of everything I’m talking about.

1. MacFUSE State of the Union Talk (2008) Video

If you were not at the talk, watching the video would provide useful context and should help greatly in understanding what follows.

2. AncientFS

Next, you could look at AncientFS, which I introduced, among other things, during the talk. AncientFS lets you mount ancient (and in some cases, current-day) "data containers" as regular volumes on Mac OS X. It supports the following formats.

• v1tap—DECtape tap tape archive; UNIX V1
• v2tap—DECtape tap tape archive; UNIX V2
• v3tap—DECtape tap tape archive; UNIX V3
• ntap—DECtape/magtape tap tape archive; 1970 epoch
• tp—DECtape/magtape tp tape archive
• itp—UNIX itp tape archive
• dtp—UNIX dtp tape archive
• dump—Incremental file system dump (512-byte blocks, V7/bsd)
• dump1kI—Incremental file system dump (1024-byte blocks, V7/bsd)
• dump-vn—Incremental file system dump (512-byte blocks, bsd-vn)
• dump1k-vn—Incremental file system dump (1024-byte blocks, bsd-vn)
• v1ar—Very old (0177555) archive (.a) from First Edition UNIX
• v2ar—Very old (0177555) archive (.a) from Second Edition UNIX
• v3ar—Very old (0177555) archive (.a) from Third Edition UNIX
• ar—Current (!<arch>\n), old (0177545), or very old (0177555) archive (.a); use (v1|v2|v3)ar for UNIX V1/V2/V3 archives
• bcpio—Binary cpio archive (old); may be byte-swapped
• cpio_odc—ASCII (odc) cpio archive
• cpio_newc—New ASCII (newc) cpio archive
• cpio_newcrc—New ASCII (newc) cpio archive with checksum
• tar—ustar, pre-POSIX ustar, or V7 tar archive
• v1—First Edition UNIX file system
• v2—Second Edition UNIX file system
• v3—Third Edition UNIX file system
• v4—Fourth Edition UNIX file system
• v5—Fifth Edition UNIX file system
• v6—Sixth Edition UNIX file system
• v7—Seventh Edition UNIX file system
• v10—Tenth Edition UNIX file system
• 32v—UNIX/32V file system
• bsd—BSD file system (V7-style with fixed-length file names; e.g. 2.9BSD or 4.0BSD)
• bsd-vn—BSD file system (pre fast-file-system “UFS” with variable-length file names; e.g. 2.11BSD for PDP-11)

To learn more about AncientFS and how to use it, please read the AncientFS article. Enjoy seeing ancient data seamlessly in the modern namespaces of Mac OS X!

3. The UnixFS Layer

AncientFS also led to UnixFS, a general-purpose abstraction layer that proved useful in getting several other "alien" file systems up and running on Mac OS X. It is particularly useful for "Unix-style" file systems, where you need concepts such as on-disk and in-memory superblocks and inodes. As I mentioned during the talk, I used UnixFS in conjunction with ad-hoc "Linux emulation" to bring support for the UFS, System V, and Minix file system families on Mac OS X. That amounts to a large number of new file systems, but it was easier than it sounds because the idea was to take existing Linux kernel implementations of these file system families and make them work in user space on Mac OS X!

UnixFS is currently rather "beta" and not a formal API by any means. Since it is a programming interface and doesn’t do anything by itself, it will be of interest only to developers at this point. In future, it may also evolve into a "LinuxFS" layer that could make it even easier and faster to systematically make Linux kernel-based file systems work on other platforms with very few code changes. For now, you can browse some bits of code.

You can, however, use the aforementioned UnixFS-based file system families: UFS, System V, and Minix. To do so, check out the relevant part of the MacFUSE source tree and compile one or more (or all) of the UnixFS-based file systems—it is quite straightforward: a single make in the filesystems/unixfs/ subdirectory in the MacFUSE source tree should build all of them. (You must have MacFUSE installed, of course.)

$ svn co http://macfuse.googlecode.com/svn/trunk/filesystems/unixfs
$ cd unixfs
$ ls -F
Makefile	common/		sysvfs/
ancientfs/	minixfs/	ufs/
$ make
...
$

4. The UFS Family

This is a user-space implementation (read-only) of the UFS file system family. Most of the UFS-specific code comes from the Linux kernel and is largely unchanged. Specific UFS flavors supported are as follows.

• old—the oldest UFS format
• sun—used in SunOS/Solaris
• sunx86—used in the x86 versions of SunOS/Solaris
• hp—used in HP-UX
• nextstep—used in NEXTSTEP
• nextstep-cd—used in NEXTSTEP CDROMs
• openstep—used in OPENSTEP
• 44bsd—used in FreeBSD, NetBSD, OpenBSD, and Mac OS X
• ufs2—used in FreeBSD 5.x

5. The System V Family

This is a user-space implementation (read-only) of the System V file system family. Most of the sysvfs-specific code comes from the Linux kernel and is largely unchanged. Specific sysvfs flavors supported are as follows.

• svr2—used in SVR2
• svr4—used in SVR4
• xenix—used in Xenix
• coherent—used in Coherent Unix

6. The Minix Family

This is a user-space implementation (read-only) of the Minix file system family. Most of the minixfs-specific code comes from the Linux kernel and is largely unchanged.

The following image summarizes the new file system capabilities Mac OS X gets as a result of these exercises.

7. The "One More Thing" Thing

Although it isn’t directly file system related, the last thing I demonstrated during the talk was the ability to run ancient Unix (PDP-11) binaries seamlessly on Mac OS X. PDP-11 aside, the techniques used to do so are generally useful for research and experimentation because unlike Linux, Mac OS X does not allow developers to extend the set of binary formats that can be "natively" executed by the kernel. The demo shows the Fifth Edition Unix kernel being compiled on Mac OS X using the original C compiler toolchain from a Fifth Edition disk image mounted using AncientFS. Additionally, there’s an "authentic" reproduction of the following ominous error message by running the original mv executable from Sixth Edition Unix.

	values of β will give rise to dom!

For more details, please watch the last section of the talk video. This is very preliminary work for which no source code or binaries are available yet.

Such automatic mounting is orthogonal to and possible with MacFUSE. (NB: You will need MacFUSE 2.0 or above for this to work properly since older versions of MacFUSE filter out the “automounted” mount-time argument.) Consider sshfs, a user-space SFTP file system implementation that works with MacFUSE. The following is a quick-and-dirty example of how you could set up an autofs mount for sshfs. (There are other ways to set up autofs mounts.)

Create an /etc/fstab file (or add to an existing one) with the following entry. We will create what’s called a “static map” in autofs parlance.

$ cat /etc/fstab
dummy:user@host:/remotedir /Network/name sshfs volname=volname,allow_other 0 0

You will have to customize the green highlighted components. user, host, and remotedir are the SFTP user name, the SFTP server host name, and the remote directory on the SFTP server, respectively. You can choose some reasonable value for name and volname. The local mount point will be /Network/name.

Next, to keep things simple, configure key-based authentication to the SFTP server so you can log in without having to type your password.

The keyword “sshfs” in the /etc/fstab entry is the type of the file system. Given a file system type foo, the automounter will expect a mount_foo file-system-specific mounting program to exist. In our case, we don’t have a separate mounting program for sshfs. However, because of the format of the entry and how the automounter passes arguments to the mounting program, it will work if you simply copy the command-line sshfs program to /sbin/mount_sshfs. Alternatively, you can create a symbolic link as follows.

$ which sshfs
$ /usr/local/bin/sshfs
$ sudo ln -s /usr/local/bin/sshfs /sbin/mount_sshfs

That should be it. Run the automount program to update the state of things. The -c argument tells the automount daemon to flush any cached information.

$ sudo automount -c

If everything went well, the new mount should appear in the output of the mount command. In the following example, we used SSH as the name component of the mount point.

$ mount
...
map -static on /Network/SSH (autofs, automounted)

Now, if you simply access /Network/SSH, the SFTP file system should be automatically mounted.

$ ls /Network/SSH
Applications		Volumes			work
Desktop DB		bin			private
...

If there is an error in mounting (say, the remote host is not reachable), you will not be permitted to access the /Network/SSH directory.

$ ls /Network/SSH
ls: SSH: Operation not permitted

You can specify a timeout period after which an automounted file system will be unmounted if it has not been accessed within that period. Either use the -t argument of automount or see the /etc/autofs.conf file.

This is useful if you want to see how a program will deal with unusually large files. Since it is extremely unlikely for the majority of us to be able to actually create files that are 512TB in size any time in the near future, faking is the way to go. Assuming you have MacFUSE installed, here is how you can try this file system.

$ ls
Makefile	verybigfs.c
$ make
...
$ mkdir "/Volumes/Very Big HD"
$ ./verybigfs "/Volumes/Very Big HD"
$ ls -lh
...
-r--r--r--  1 singh  wheel   512T Nov 28 05:38 copyme.txt

As an aside, note that HFS+ does not support sparse files. In fact, if you create a large scratch file on HFS+, it will be zero filled “soon”. This can take a long time depending upon the file size, the hardware, and the resources available. Mac OS X provides a way for a privileged process to set a file’s size without zero filling: it’s the F_SETSIZE command of the fcntl() system call.

Here is what the Finder tells you about the file copyme.txt inside the volume.

Note that although the tantalizingly named file is fictitious in that it doesn’t occupy any real disk space, you will read 512TB of zero-filled data if you choose to do so.

$ od -Xv copyme.txt
...
1357500          00000000        00000000        00000000        00000000
1357520          00000000        00000000        00000000        00000000
1357540          00000000        00000000        00000000        00000000
1357560          00000000        00000000        00000000        00000000
1357600          00000000        00000000        00000000        00000000
...
# A very, very, very long undetermined length of time...

MacFUSE 2.0 is a major update to MacFUSE. We’ll be discussing what’s new in MacFUSE 2.0 in a talk at Google’s Mountain View headquarters today. For those who can’t be at the talk, there are two versions of the “what’s new” description.

The Apple-style version is easy to state: “improvements and bug-fixes.”

Alternatively, you can read the CHANGELOG in its entirety at the project’s web site.

Enjoy.

The relevant wiki page has all the details.

Note that instead of Tiger- and Leopard-specific downloads, now there’s a single downloadable disk image containing a single installable package. The package, which third parties can choose to include within their metapackages, knows how to install the latest version of MacFUSE for your platform.

The CHANGELOG has details of what’s new.

In a pinch, GrabFS is a file system that shows you a live view of the window contents of currently running applications. In a GrabFS volume, folders represent running applications and image files represent instant screenshots (“grabs”) of the applications’ windows. You simply copy a file or just open it in place, and you have a screenshot. Open it again, and you have a new screenshot!

Go here to read more about GrabFS and to download it. GrabFS requires Mac OS X "Leopard" and MacFUSE.

Downloads: http://code.google.com/p/macfuse/downloads/list

Documentation: http://code.google.com/p/macfuse/w/list

I recently added several new features to this procfs implementation. Some of these features are "cool" in that they put a new twist on certain types of visual information.

For example, there’s a folder /proc/system/hardware/displays/ that contains a subfolder each for connected displays. Subfolder 0 represents the first display, 1 is the second display (if any), and so on. Within each such subfolder, there’s a file called info that contains information about that particular display: its resolution, bits-per-pixel, bytes-per-row, whether the display is built-in, whether it supports OpenGL acceleration, and so on. There’s another file called screenshot.tiff that contains a TIFF rendition of what’s on that display at that moment—an always-live screenshot, if you will. You copy this file and you get a screenshot. Copy it again, and you get a new screenshot. You can just open it in place too.

Along similar lines, there’s another folder /proc/system/hardware/camera/ and a file screenshot.tiff within it. When you open this file, procfs activates the camera momentarily, takes a picture, deactivates the camera, and makes the picture available as a TIFF file. You can copy the file and you get an image of what the camera’s seeing at that moment. Copy it again and you get another "live" image.

Besides these, the updated procfs has other (not-so-visual) interesting features.

More details, source code, and a precompiled binary available here:

Making procfs Cooler

Video on YouTube

Here’s an abstract: