OpenZFS Developer Summit

From OpenZFS
Revision as of 20:32, 20 November 2013 by Grahamperrin (talk | contribs) (Past tense for some of the text. Needs further attention.)
Jump to navigation Jump to search

Hosted by Delphix and originally named OpenZFS Day, the event was renamed OpenZFS Developer Summit to reflect the target audience of ZFS developers. The goals were:

  • to foster cross-platform community discussions of OpenZFS work
  • to make progress on some of the projects proposed for this community.

Video recordings of the presentations are in the OpenZFS channel on YouTube.

A huge THANK YOU to our generous sponsors

Delphix logo.png Ddrdrive logo dev 2013.png Logo freebsdfoundation.png
SageCloud-logo-RGB-large-optimized.jpg Ixsystems.jpg CS logo withTagline 150px.png
SpectraLogicLogo.jpg Joyent-logo.png New hac logo.jpg
Nexenta logo.jpg


Sunday, November 17th

6:30pm Optional dinner at Rosamunde Sausage Grill on Mission St.

Location is a 3 minute walk from the 24th Street BART station.

Monday, November 18th

8:30am - 9:00am Registration and breakfast
9:00am - 9:15am Introductions
9:15am - 9:45am State of OpenZFS & Event overview (Matt Ahrens)
9:45am - 10:00am List and prioritize discussion topics (lead by Matt Ahrens & Karyn Ritter)
10:00am - 11:00am Discussion sessions
11:00am - 11:15am Coffee & snack break
11:15am - 12:15pm Discussion sessions
12:15am - 1:00pm Lunch
1:00pm - 1:30pm Discussion session: Community Management (lead by Karyn Ritter)
1:30pm - 3:00pm Discussion sessions
3:00pm - 3:30pm Coffee & snack break
3:30pm - 5:30pm Discussion sessions
5:30pm - 5:45pm Closing (Matt Ahrens)
5:45pm Beer bash & casual dinner at Delphix office

Tuesday, November 19th

9:00am - 9:30am Breakfast
9:30am - 10:00am Discuss Hackathon ideas & form teams
10:00am - 12:30pm Hackathon
12:30pm - 1:30pm Lunch
1:30pm - 5:00pm Hackathon
5:00pm - 5:45pm Hackathon presentations
6:00pm Optional dinner at Thirsty Bear

Location is 5 minute walk from Delphix office.

Note that the optional, off-site dinners on Sunday and Tuesday are at your own expense.


The Delphix office in San Francisco:

604 Mission Street, 4th floor, San Francisco, CA 94105

Use the keypad to call Delphix and you will be buzzed in. Google maps has pretty good directions. The easiest way to reach the Delphix SF office is by public transportation. Transportation options, easiest listed first:

  • BART to the Montgomery St. Station + 3 minute walk
  • Caltrain to the 4th & King station + 25 minutes walk
  • Caltrain to the 4th & King station + bus (routes 10, 30, or 45)
  • drive + park in a lot (costs about $25/day e.g. 41 Tehama Street, 509 Howard Street, or 55 Hawthorne Street)

Discussion topics

Note: If you are in the room, you are potentially a speaker. This will be done in unconference style, so every attendee should come prepared to present or otherwise lead a discussion or session of up to 45 minutes. The most important things to discuss in this setting are:

  1. Contentious topics that we need to get widespread agreement on
  2. Reports of work in progress that need input from others

Proposed topics

  • OpenZFS code repository (Matt)
    • what code changes are needed to make this a success
    • process & procedure (code review, testing, committers)
  • Performance on full or fragmented pools (George, Ned, Alfred)
    • block allocator improvements
  • Portable implementation of FMA functionality for Linux & FreeBSD (e.g. for hot spare activation) (Brian)
  • OpenZFS community organization (Karyn, Jordan, Matt)
  • Storage tiering (Boris)
  • How to investigate ZFS performance problems (Adam)
  • Test suite (John K)
  • ZFS Channel Programs (Chris, Max G, Matt)
  • how to examine the ZFS on-disk format with MDB (Max B)
  • Interactions with virtual memory subsystem (Brian, Jorgen)
    • dealing with swap on zfs
    • how the VM system handles large memory allocations
  • Scalability on large systems, known bottlenecks (Kirill)
  • How to educate new ZFS developers (Max G)
  • Disk management - identification, drive spin down/up (John L)
  • Resumable send/recv (Boris, Max G, Chris)

Topics that were discussed

  • Platform Panel (w/ representatives from illumos, FreeBSD, Linux, and OSX)
  • Platform-independent code repository (Matt Ahrens)
  • Storage tiering (Boris Protopopov)
  • Vendor Panel (all represented companies sharing their work)
  • Community planning (Karyn Ritter)
  • Channel programs (Chris Siden & Max Grossman)
  • Test coverage (John Kennedy)
  • Performance investigations (Adam Leventhal)
  • Performance on full/fragmented pools (George Wilson)
  • Scalability (Kirill Davydychev)
  • Virtual memory interactions (Brian Behlendorf)
  • Multi tenancy (Rob Mustacchi)
  • Examining on-disk format (Max Bruning)


The goals of the hackathon are:

  • to get people working with ZFS engineers from outside their normal circles
  • to start work on some of the projects we'd like to accomplish

Therefore, we should plan to work in small teams of 2-3 engineers. The team members could be pair-programming, splitting up the work into discrete tasks for each person, or advising / implementing.

Hackathon projects

These projects were worked on at the hackathon:

  • Testrunner suite running on other platforms: linux, freebsd (+ illumos libzpool)
  • channel programs: pass instruction stream into kernel
  • ZFS send progress reporting with "set proc title"
  • dedicated kmem_cache for dedup table entries
  • larger (1MB) blocksize
  • compressed ARC
  • upstreaming dbuf_user & dbuf state machine (spectra DMU changes)
  • limit DDT size
  • onexit for osx

Hackathon ideas

Add your idea and your name below. Be prepared to explain your idea at the beginning of the hackathon and enlist/entice others to help with it.

  • Userland ioctls: support for send/recv (fd operations over the wire) User:mahrens
  • larger (1MB) blocksize: shouldn't be too hard if we ignore performance impact User:mahrens
  • dedicated kmem_cache for dedup table entries
  • stdin / stdout buffer to improve performance of zfs send and receive
  • ZFS mechanism to override devices' reported sector size (512 vs 4k) (alex @ delphix)
  • feature flags enhancements for send streams User:csiden
  • set owner of all files within a filesystem with a ZFS property User:csiden
  • dynamic taskqs for ZIO User:ahl
  • towards Oracle Solaris ZFS compatibility User:ahl
  • RAIDzN -- full, arbitrary RAID-7 User:ahl
  • Testrunner suite running on other platforms (John Kennedy)
    • or at least identify what is preventing it from running


We are expecting everyone will be there to contribute/participate.

Name Affiliation What do you want to talk about? Your experience with ZFS Travel information T-shirt Size Food allergies/preferences? (please list)
Matt Ahrens Delphix OpenZFS repo: what interfaces should be supported Founded ZFS, OpenZFS Local to SF M none
Sanjeev Bagewadi Pluribus Networks Was part of the RPE team in Sun which fixed ZFS issues faced by customers Will be in the Bay Area around that time M none
Ned Bass LLNL Common code base, block allocator improvements ZFS on Linux port, Lustre integration from East Bay XL vegetarian preferred
Brian Behlendorf LLNL Portable memory management & FMA infrastructure ZFS on Linux Maintainer Local to east Bay Area XL none
Max Bruning Joyent examining ZFS on-disk format with MDB leads ZFS training classes ? ? ?
Steven Burgess Datto Inc. openZFS as a customer Working as a developer at a company whose product uses ZOL from NYC M none
Kirill Davydychev Nexenta Scalability on large systems, major known bottlenecks. Debugging, performance analysis/tuning Houston M none
Justin Gibbs Spectra Logic Ref-Links, Resilvering Algorithm, Spares, auto-configuration DMU performance work flying in from CO L vegetarian
Max Grossman Delphix Making it easier to get new ZFS developers up to speed (retrospective on the hole birth feature) 3 months so far, still so much to learn... Local L none
Jordan Hubbard iXsystems OpenZFS community organization, independent code repo processes ? Local ? ?
Jeremy Jones Delphix ? ZFS bookmarks; debugging techniques local ? ?
John Wren Kennedy Delphix Uniform test coverage for OpenZFS Supported ZFS in RPE at Sun/Oracle, and test ZFS in QA at Delphix. Denver L none
John Layman SageCloud Interested in disk management (spin up/down disks, drive identification, etc.) Been working with ZFS on Linux for a few months, contributed small bugfix Boston XL none
Adam Leventhal Delphix ZFS performance investigations? RAID-Z2/3, performance work, some small ZFS tasks here and there local L none
Xin Li iXsystems ZFS developer for FreeBSD, TrueNAS and FreeNAS products FreeBSD deputy security officer / release engineering team member local Bay Area L none
Jorgen Lundman GMO Internet ZFS, portability ZFS on OS X Tokyo XL none
Maxim Martynov M none
Robert Mustacchi Joyent Misc. debugging Local to SF L None
Tony Nguyen Nexenta Misc. debugging, import and I/O performance flying in from Denver M None
Alfred Perlstein iXsystems ZFS performance and debugging FreeNAS (FreeBSD+ZFS) developer ? ? ?
Boris Protopopov Nexenta Resumable send/recv, defrag, storage tiering, spare management, doing fancy things with vdevs ZFS dev lead at Nexenta, ZFS extensions for enterprise class features and performance Boston M None
Alex Reece Delphix small bug fixes local M No marinera
Karyn Ritter Delphix Community management (discussion) local M
Craig Rodrigues iXsystems ZFS developer for TrueNAS and FreeNAS products FreeBSD release engineering team member local L none
Chris Siden Delphix ZFS Channel Programs, feature flags enhancements for send streams Worked on feature flags and async destroy local L none
Alex Solomatnikov M none
Prashanth Sreenivasa I would like to learn more about ZIL, ZFS I/O scheduler & pipeline and impact of using flash-disks instead of conventional hard-disks. Worked on various PDE (product development engineering) groups including Network Storage, Systems & Solaris kernel. Recently I'm working on ZFS based storage appliances, trying to improve zfs performance on illumos/x86 platform and looking forward to contribute to the open source zfs. M
Prakash Surya LLNL Misc. work on Linux port. Much to learn, still. Local to east Bay Area M none
Rafael Vanoni Pluribus I'm not a ZFS developer, but would like to work my way towards that. I worked on the thread scheduler/dispatcher at Sun/Oracle ? M ?
George Wilson Delphix performance on full & fragmented pools Implemented lots of SPA & VDEV stuff flying in from Atlanta ~Monday - Wednesday L none
Matt Youds High Availability Experience with ZFS configuration at High-Availability. L Fish
Sam Zaydel ? ? ? ? ? ?


The event was fully sponsored! Thank you to our sponsors:

What do you get for sponsoring:

  • Your name/link on this page!
  • Thank you mention during the opening and closing
  • Space to put your pop-up banner at the conference venue
  • An opportunity to be known as a ZFS supporter in a very talented and influential community.