Difference between revisions of "OpenZFS Developer Summit"

Jump to navigation Jump to search
14,332 bytes removed ,  17:58, 24 March 2020
redirect to 2020 page
(redirect to 2020 page)
(15 intermediate revisions by 4 users not shown)
Line 1: Line 1:
Delphix is hosting a developer-focused event for OpenZFS. The goal of the event is to foster cross-platform community discussions of OpenZFS work, and to make progress on some of the projects proposed for this community.
{{:OpenZFS Developer Summit 2020}}
 
Video recordings of the presentations are available [https://www.youtube.com/playlist?list=PLaUVvul17xSdWMBt5tAC8Hu7bbeWskD_q here].
 
''We were previously calling this event "OpenZFS Day", but decided to rename it to reflect its target audience of ZFS developers.''
 
== Huge THANK YOU to our generous sponsors ==
{|cellpadding="10"
|-
|[[File: Delphix logo.png|200x200px|link=http://delphix.com]]
|[[File: Ddrdrive logo dev 2013.png|200x200px|link=http://www.ddrdrive.com]]
|[[File: Logo freebsdfoundation.png|200x200px|link=http://www.freebsd.org]]
|-
|[[FIle: SageCloud-logo-RGB-large-optimized.jpg|200x200px|link=http://sagecloud.com/]]
|[[File: Ixsystems.jpg|200x200px|link=http://www.ixsystems.com/]]
|[[File: CS logo withTagline 150px.png|200x200px|link=http://cloudscaling.com/]]
|-
|[[File: SpectraLogicLogo.jpg|200x200px|link=http://www.spectralogic.com/]]
|[[File: Joyent-logo.png|200x200px|link=http://joyent.com]]
|[[File: New_hac_logo.jpg|200x200px|link=http://www.high-availability.com/]]
|-
|[[File: Nexenta logo.jpg|200x200px|link=http://www.nexenta.com/]]
|}
 
== Schedule ==
 
If you are unable to attend in person, you can view the video stream live, on Monday November 18th from 9am - 6pm Pacific time.  The URL for the stream will be posted here Monday, and a recording will be available after the event.
 
=== Sunday, November 17th ===
{|border="1" cellpadding="10"
|-
|6:30pm
| Optional dinner at [http://goo.gl/943i5i Rosamunde Sausage Grill] on Mission St.
Location is a 3 minute walk from the 24th Street BART station.
|}
 
=== Monday, November 18th ===
{|border="1" cellpadding="10"
|-
|valign="top"|8:30am - 9:00am
| Registration and breakfast
|-
|valign="top"|9:00am - 9:15am
| Introductions
|-
|valign="top"|9:15am - 9:45am
| State of OpenZFS & Event overview (Matt Ahrens)
|-
|valign="top"|9:45am - 10:00am
| List and prioritize discussion topics (lead by Matt Ahrens & Karyn Ritter)
|-
|valign="top"|10:00am - 11:00am
| Discussion sessions
|-
|valign="top"|11:00am - 11:15am
| Coffee & snack break
|-
|valign="top"|11:15am - 12:15pm
| Discussion sessions
|-
|valign="top"|12:15am - 1:00pm
| Lunch
|-
|valign="top"|1:00pm - 1:30pm
| Discussion session: Community Management (lead by Karyn Ritter)
|-
|valign="top"|1:30pm - 3:00pm
| Discussion sessions
|-
|valign="top"|3:00pm - 3:30pm
| Coffee & snack break
|-
|valign="top"|3:30pm - 5:30pm
| Discussion sessions
|-
|valign="top"|5:30pm - 5:45pm
| Closing (Matt Ahrens)
|-
|valign="top"|5:45pm
| Beer bash & casual dinner at Delphix office
|}
 
=== Tuesday, November 19th ===
{|border="1" cellpadding="10"
|-
|valign="top"|9:00am - 9:30am
| Breakfast
|-
|valign="top"|9:30am - 10:00am
| Discuss Hackathon ideas & form teams
|-
|valign="top"|10:00am - 12:30pm
|Hackathon
|-
|valign="top"|12:30pm - 1:30pm
|Lunch
|-
|valign="top"|1:30pm - 5:00pm
|Hackathon
|-
|valign="top"|5:00pm - 5:45pm
| Hackathon presentations
|-
|valign="top"|6:00pm
| Optional dinner at [https://www.google.com/maps/preview#!q=Thirsty+Bear+Brewery%2C+Howard+St%2C+San+Francisco%2C+CA&data=!1m4!1m3!1d3309!2d-122.4011122!3d37.7868869!4m15!2m14!1m13!1s0x8085807d0bcfd0df%3A0xdfac6306f9e18402!3m8!1m3!1d13236!2d-122.3926051!3d37.7913156!3m2!1i1261!2i806!4f13.1!4m2!3d37.785502!4d-122.399664 Thirsty Bear]
Location is 5 minute walk from Delphix office.
|}
 
Note that the optional, off-site dinners on Sunday and Tuesday are at your own expense.
 
== Location ==
The event will be in the [http://www.delphix.com/company/about-delphix/ Delphix office] in San Francisco:
 
[https://www.google.com/maps/preview#!q=604+Mission+St.+Suite+400%2C+San+Francisco%2C+CA+94105&data=!1m4!1m3!1d26473!2d-122.4224157!3d37.7867658!4m12!2m11!1m10!1s0x80858062b5adcc23%3A0xd7584093492239e6!3m8!1m3!1d12612!2d-122.3926051!3d37.7913156!3m2!1i1024!2i768!4f13.1 604 Mission Street, 4th floor, San Francisco, CA 94105]
 
Use the keypad to call Delphix and you will be buzzed in.  Google maps has pretty good [http://goo.gl/QyUHUj directions].  The easiest way to reach the Delphix SF office is by public transportation.  Transportation options, easiest listed first:
* BART to the Montgomery St. Station + 3 minute walk
* Caltrain to the 4th & King station + 25 minutes walk
* Caltrain to the 4th & King station + bus (routes 10, 30, or 45)
* drive + park in a lot (costs about $25/day e.g. 41 Tehama Street, 509 Howard Street, or 55 Hawthorne Street)
 
== Discussion Topics ==
'''Note: If you are in the room, you are potentially a speaker.''' This will be done in unconference style, so every attendee should come prepared to present or otherwise lead a discussion or session of up to 45 minutes.  The most important things to discuss in this setting are:
 
# Contentious topics that we need to get widespread agreement on
# Reports of work in progress that need input from others
 
Topics that have been proposed:
* OpenZFS code repository (Matt)
** what code changes are needed to make this a success
** process & procedure (code review, testing, committers)
* Performance on full or fragmented pools (George, Ned, Alfred)
** block allocator improvements
* Portable implementation of FMA functionality for Linux & FreeBSD (e.g. for hot spare activation) (Brian)
* OpenZFS community organization (Karyn, Jordan, Matt)
* Storage tiering (Boris)
* How to investigate ZFS performance problems (Adam)
* Test suite (John K)
* ZFS Channel Programs (Chris, Max G, Matt)
* how to examine the ZFS on-disk format with MDB (Max B)
* Interactions with virtual memory subsystem (Brian, Jorgen)
** dealing with swap on zfs
** how the VM system handles large memory allocations
* Scalability on large systems, known bottlenecks (Kirill)
* How to educate new ZFS developers (Max G)
* Disk management - identification, drive spin down/up (John L)
* Resumable send/recv (Boris, Max G, Chris)
 
== Hackathon ==
The goals of the hackathon are:
* to get people working with ZFS engineers from outside their normal circles
* to start work on some of the [[projects]] we'd like to accomplish
Therefore, we should plan to work in small teams of 2-3 engineers.  The team members could be pair-programming, splitting up the work into discrete tasks for each person, or advising / implementing.
 
=== Hackathon projects ===
These projects were worked on at the hackathon:
 
* Testrunner suite running on other platforms: linux, freebsd (+ illumos libzpool)
* channel programs: pass instruction stream into kernel
* ZFS send progress reporting with "set proc title"
* dedicated kmem_cache for dedup table entries
* larger (1MB) blocksize
* compressed ARC
* upstreaming dbuf_user & dbuf state machine (spectra DMU changes)
* limit DDT size
* onexit for osx
 
=== Hackathon ideas ===
''Add your idea and your name below.  Be prepared to explain your idea at the beginning of the hackathon and enlist/entice others to help with it.''
* Userland ioctls: support for send/recv (fd operations over the wire)  [[User:mahrens]]
* larger (1MB) blocksize: shouldn't be too hard if we ignore performance impact [[User:mahrens]]
* dedicated kmem_cache for dedup table entries
* stdin / stdout buffer to improve performance of zfs send and receive
* ZFS mechanism to override devices' reported sector size (512 vs 4k) (alex @ delphix)
* feature flags enhancements for send streams [[User:csiden]]
* set owner of all files within a filesystem with a ZFS property [[User:csiden]]
* dynamic taskqs for ZIO [[User:ahl]]
* towards Oracle Solaris ZFS compatibility [[User:ahl]]
* RAIDzN -- full, arbitrary RAID-7 [[User:ahl]]
* Testrunner suite running on other platforms (John Kennedy)
** or at least identify what is preventing it from running
 
== Registration ==
We are expecting everyone will be there to contribute/participate.
 
'''The sign up deadline has passed'''.  A few spots are available for late registration.  Mail [mailto:admin@open-zfs.org admin@open-zfs.org] if you would like to attend.
 
{|border="1" cellpadding="10"
|-
!| Name
!| Affiliation
!| What do you want to talk about?
!| Your experience with ZFS
!| Travel information
!| T-shirt Size
!| Food allergies/preferences? (please list)
|-
| [[User:mahrens | Matt Ahrens]]
| Delphix
| OpenZFS repo: what interfaces should be supported
| Founded ZFS, OpenZFS
| Local to SF
| M
| none
|-
|[[User:sanjeev | Sanjeev Bagewadi]]
| Pluribus Networks
|
| Was part of the RPE team in Sun which fixed ZFS issues faced by customers
| Will be in the Bay Area around that time
| M
| none
|-
|-
| Ned Bass
| LLNL
| Common code base, block allocator improvements
| ZFS on Linux port, Lustre integration
| from East Bay
| XL
| vegetarian preferred
|-
| [[User:behlendorf | Brian Behlendorf]]
| LLNL
| Portable memory management & FMA infrastructure
| ZFS on Linux Maintainer
| Local to east Bay Area
| XL
| none
|-
| Max Bruning
| Joyent
| examining ZFS on-disk format with MDB
| leads ZFS training classes
| ?
| ?
| ?
|-
| Steven Burgess
| Datto Inc.
| openZFS as a customer
| Working as a developer at a company whose product uses ZOL
| from NYC
| M
| none
|-
| [[User:kdavyd | Kirill Davydychev]]
| Nexenta
| Scalability on large systems, major known bottlenecks.
| Debugging, performance analysis/tuning
| Houston
| M
| none
|-
| Justin Gibbs
| Spectra Logic
| Ref-Links, Resilvering Algorithm, Spares, auto-configuration
| DMU performance work
| flying in from CO
| L
| vegetarian
|-
| [[User:Max | Max Grossman]]
| Delphix
| Making it easier to get new ZFS developers up to speed (retrospective on the hole birth feature)
| 3 months so far, still so much to learn...
| Local
| L
| none
|-
| Jordan Hubbard
| iXsystems
| OpenZFS community organization, independent code repo processes
| ?
| Local
| ?
| ?
|-
| Jeremy Jones
| Delphix
| ?
| ZFS bookmarks; debugging techniques
| local
| ?
| ?
|-
| [[User:jkennedy | John Wren Kennedy]]
| Delphix
| Uniform test coverage for OpenZFS
| Supported ZFS in RPE at Sun/Oracle, and test ZFS in QA at Delphix.
| Denver
| L
| none
|-
| [[User:jml | John Layman]]
| SageCloud
| Interested in disk management (spin up/down disks, drive identification, etc.)
| Been working with ZFS on Linux for a few months, contributed small bugfix
| Boston
| XL
| none
|-
| [[User:ahl | Adam Leventhal]]
| Delphix
| ZFS performance investigations?
| RAID-Z2/3, performance work, some small ZFS tasks here and there
| local
| L
| none
|-
| [[User:delphij | Xin Li]]
| iXsystems
| ZFS developer for FreeBSD, TrueNAS and FreeNAS products
| FreeBSD deputy security officer / release engineering team member
| local Bay Area
| L
| none
|-
| [[User:lundman | Jorgen Lundman]]
| GMO Internet
| ZFS, portability
| ZFS on OS X
| Tokyo
| XL
| none
|-
| [[User:maxim | Maxim Martynov]]
|
|
|
|
| M
| none
|-
| Robert Mustacchi
| Joyent
|
| Misc. debugging
| Local to SF
| L
| None
|-
| [[User:tonyn | Tony Nguyen]]
| Nexenta
|
| Misc. debugging, import and I/O performance
| flying in from Denver
| M
| None
|-
| Alfred Perlstein
| iXsystems
| ZFS performance and debugging
| FreeNAS (FreeBSD+ZFS) developer
| ?
| ?
| ?
|-
| Boris Protopopov
| Nexenta
| Resumable send/recv, defrag, storage tiering, spare management, doing fancy things with vdevs
| ZFS dev lead at Nexenta, ZFS extensions for enterprise class features and performance
| Boston
| M
| None
|-
| [[User:awreece | Alex Reece]]
| Delphix
|
| small bug fixes
| local
| M
| No marinera
|-
| [[User:kritter | Karyn Ritter]]
| Delphix
| Community management (discussion)
|
| local
| M
|
|-
| [[User:rodrigc | Craig Rodrigues]]
| iXsystems
| ZFS developer for TrueNAS and FreeNAS products
| FreeBSD release engineering team member
| local
| L
| none
|-
| [[User:csiden | Chris Siden]]
| Delphix
| ZFS Channel Programs, feature flags enhancements for send streams
| Worked on feature flags and async destroy
| local
| L
| none
|-
| Alex Solomatnikov
|
|
|
|
| M
| none
|-
| [[User:pks | Prashanth Sreenivasa ]]
|
| I would like to learn more about ZIL, ZFS I/O scheduler & pipeline and impact of using flash-disks instead of conventional hard-disks.
| Worked on various PDE (product development engineering) groups including Network Storage, Systems & Solaris kernel. Recently I'm working on ZFS based storage appliances, trying to improve zfs performance on illumos/x86 platform and looking forward to contribute to the open source zfs.
|
| M
|
|-
| [[User:prakashsurya | Prakash Surya]]
| LLNL
|
| Misc. work on Linux port. Much to learn, still.
| Local to east Bay Area
| M
| none
<!--
|-
| Ilya Usvyatsky
| Nexenta
| resumable send/recv, defrag
| ?
| ?
| ?
| ?
-->
|-
| Rafael Vanoni
| Pluribus
| I'm not a ZFS developer, but would like to work my way towards that.
| I worked on the thread scheduler/dispatcher at Sun/Oracle
| ?
| M
| ?
|-
| [[User:gwilson | George Wilson]]
| Delphix
| performance on full & fragmented pools
| Implemented lots of SPA & VDEV stuff
| flying in from Atlanta ~Monday - Wednesday
| L
| none
|-
| Matt Youds
| High Availability
|
| Experience with ZFS configuration at High-Availability.
|
| L
| Fish
|-
| Sam Zaydel
| ?
| ?
| ?
| ?
| ?
| ?
|}
 
– ''please keep the table above sorted by last name.''
 
== Sponsors ==
The event is fully sponsored!  Thank you to our sponsors:
 
* Breakfast (2 days) - Sponsored!  Thank you [http://www.cloudscaling.com Cloudscaling] and [http://www.ixsystems.com iXsystems]
* Lunch  (2 days) - Sponsored!  Thank you [http://sagecloud.com Sage Cloud], [http://www.spectralogic.com Spectra Logic] and [http://www.freebsdfoundation.org FreeBSD Foundation]
* Coffee Breaks (2 days) - Sponsored!  Thank you [http://www.ixsystems.com iXsystems]
* Beer Bash (day 1) - Sponsored!  Thank you [http://www.ddrdrive.com DDRdrive]
* T-Shirts  - Sponsored!  Thank you [http://www.dexlphix.com Delphix]
* Video Recordings - Thank you [http://www.joyent.com Joyent]
* Cash Donation for expenses - Thank you [http://www.nexenta.com Nexenta] and [http://www.high-availability.com/ High-Availability]
 
What do you get for sponsoring:
 
* Your name/link on this page!
* Thank you mention during the opening and closing
* Space to put your pop-up banner at the conference venue
* An opportunity to be known as a ZFS supporter in a very talented and influential community.

Navigation menu