The Future of LibZFS

Last week we had a very successful Illumos meetup, hosted at Delphix HQ in Menlo Park.  Thanks to all who participated!

Last week we had a very successful Illumos meetup, hosted at Delphix HQ in Menlo Park.  Thanks to all who participated! In honor of the ZFS 10 year anniversary, my colleagues Chris Siden and John Kennedy gave great talks about ZFS Feature Flags, and testing strategies.  My contribution was to present plans for a new programmatic interface to ZFS.

I’ve been running into a lot of problems with libzfs in my work at Delphix, generally falling into 2 categories: making changes to libzfs, and using libzfs.

The ZFS community has added many new capabilities to libzfs since it was integrated on October 31, 2005.  Unfortunately, some of them to not fit into the original design.  For example, back in the day there were only “normal” properties — the statically defined ones in zfs_prop_t (e.g. quota, used, compression).  Now we have several flavors of dynamic properties: user properties (e.g. “com.delphix:dbname”), userquota-type props (e.g. “userused@mahrens”, “userquota@csiden”), and written props (e.g. “written@prevsnap”).  Each of these flavors requires special-case code to be added in several different places.  We need to consolidate all the handling of one property “flavor” in one place.

LibZFS has outgrown its original design, to the point that even simple enhancements are overly complicated and risky.

We are using libzfs from our Java stack, via JNA.  We’ve run into a number of difficulties: libzfs is not thread safe; the interface is unstable; there are different functions for manipulating each “flavor” of property.

To address these issues and more, I propose that we create a new library, libzfs_core, which will generally be a thin wrapper around the kernel ioctls.  Error handling and thread safety issues will be pushed down into the kernel, and user interface issues will be pushed up into the libzfs_core consumers.  Our goals for libzfs_core are:

  • Thread safe
    • Avoid global data (e.g. caching)
  • Committed interface
    • Consumers will work on future releases
    • Interfaces won’t be changed
  • Programatic error handling
    • Routines return defined error numbers or error nvlists
    • No printing to stdout/stderr
  • Thin layer
    • Generally just marshalls arguments to/from kernel ioctls
    • Generally 1:1 libzfs_core function -> ioctl
  • Clear Atomicity
    • Generally, every function call will be atomic (because ioctls are generally atomic)

For more details, check out my slides, and the video of my presentation.  (In the video I refer to libzfs_core as “libzfs2”.  There was some discussion about the name, and we decided to rename it to libzfs_core.)

If you are using libzfs, I’d like to hear from you!  Will libzfs_core meet your needs?  Do you have other ideas for improving libzfs?  Contact me or the mailing list.