Difference between revisions of "OpenZFS Developer Summit 2016"

From OpenZFS
Jump to navigation Jump to search
(typo)
 
(30 intermediate revisions by 3 users not shown)
Line 1: Line 1:
[[File:OpenZFS_2016_338px_White.png| right | 370px | OpenZFS Dev Summit 2016]] The fourth annual OpenZFS Developer Summit will be held in San Francisco, California. All OpenZFS developers are invited to participate!
[[File:OpenZFS_2016_338px_White.png| right | 370px | OpenZFS Dev Summit 2016]] The fourth annual OpenZFS Developer Summit was held in San Francisco, California.
The goal of the event is to foster cross-community discussions of OpenZFS work and to make progress on some of the projects we have proposed. This 2-day event consists of a day of presentation and a 1-day hackathon.
 
Check out the following blog posts about the event:
<ul>
<li>[https://www.delphix.com/blog/delphix-engineering/thank-you-openzfs-2016-developers-summit-sponsors Delphix Blog: Thank you OpenZFS 2016 Developers Summit Sponsors!]</li>
<li>[https://www.ixsystems.com/blog/openzfs-devsummit-2016/ iXsystems Blog: OpenZFS: Stronger Than Ever]</li>
</ul>


The goal of the event is to foster cross-community discussions of OpenZFS work and to make progress on some of the projects we have proposed. This 2-day event consisted of 1 day of presentation and a 1-day hackathon.


==Thank You 2016 Sponsors!==
==Thank You 2016 Sponsors!==
Line 9: Line 15:
<div><ul>  
<div><ul>  
<li style="display: inline-block; padding-bottom: 70px;"> [[File: Delphix logo.png|170x150px|link=http://delphix.com | right | Delphix]] </li>
<li style="display: inline-block; padding-bottom: 70px;"> [[File: Delphix logo.png|170x150px|link=http://delphix.com | right | Delphix]] </li>
<li style="display: inline-block; padding: 20px;"> [[File: Intel.png|120x120px|link=http://www.high-availability.com/ | right | Intel ]]</li>
<li style="display: inline-block; padding: 20px;"> [[File: Intel.png|120x120px|link=http://www.intel.com/ | right | Intel ]]</li>
<li style="display: inline-block; padding: 20px; padding-bottom: 60px;"> [[File: OsNexus.png|150x150px|link=http://www.osnexus.com/ | right | OSNexus ]] </li>
<li style="display: inline-block; padding: 20px; padding-bottom: 60px;"> [[File: OsNexus.png|150x150px|link=http://www.osnexus.com/ | right | OSNexus ]] </li>
<li style="display: inline-block; padding: 20px; padding-bottom: 60px;"> [[File: Nexenta.png|150x150px|link=http://www.nexenta.com/ | right | Nexenta ]] </li>
<li style="display: inline-block; padding: 20px; padding-bottom: 60px;"> [[File: Nexenta.png|150x150px|link=http://www.nexenta.com/ | right | Nexenta ]] </li>
Line 30: Line 36:
</ul></div>
</ul></div>


 
==Presentations==
==Event==
 
===When===
September 26 and 27, 2016
 
Detailed Schedule is below.
 
===Where===
The conference will take place in San Francisco, with each day held at a different location within the city:
* Monday, September 26th (presentations) - [https://www.google.com/maps/place/Children's+Creativity+Museum/@37.7833214,-122.4043141,17z/data=!3m1!4b1!4m5!3m4!1s0x80858087438c54b7:0x9229072a5964c7fb!8m2!3d37.7833172!4d-122.4021201 Children's Creativity Museum, 221 4th St, San Francisco, CA 94103.]
* Tuesday, September 27th (hackathon) - [https://www.google.com/maps/place/343+Sansome+St,+San+Francisco,+CA+94104/@37.7939169,-122.4052801,16.54z/data=!4m5!3m4!1s0x8085808a7989b1d9:0x878bc14303216c5e!8m2!3d37.7936971!4d-122.4017307 Delphix SF Office, 343 Sansome St, Suite 900, San Francisco, CA 94104]
 
===Registration===
[https://www.eventbrite.com/e/openzfs-developer-summit-2016-tickets-24968191533|OpenZFS Developer Summit 2016 registration] is now open!
* There will be no registration deadline - the registration will remain open until capacity is reached. There will be a waitlist in case tickets become available.
* Registration is $50 per attendee. If your company is a sponsor (as listed below), you may be eligible for a free registration. Please contact admin at open-zfs dot org once if that is the case. Speakers will also receive free registration.
All attendees are expected to contribute/participate. Your ideas and questions are what make the event exciting!
 
===Schedule===
 
====Day 1: September 26th====


Click on the title of the presentation for more details.
Click on the title of the presentation for more details.
Line 57: Line 42:
{| class="wikitable"
{| class="wikitable"
|-
|-
!Start!!End!!Title!!Speaker!!Company
!Title!!Speaker!!Company!!Video!!Slides
|-
|9:00 am||9:25 am||State of the Union||Matt Ahrens||[http://www.delphix.com/ Delphix]
|-
|9:25 am|| 10:05 am||Keynote||Dustin Kirkland||[http://canonical.com/ Canonical]
|-
|10:05 am||10:35 am||colspan=3|'''Break'''
|-
|-
|10:35 am|| 11:15 am||Lustre, Supercomputers, and ZFS||Brian Behlendorf||[http://llnl.gov LLNL]
|State of the Union||Matt Ahrens||[http://www.delphix.com/ Delphix]||[https://youtu.be/CIJGUNDZ9yE Video]|| [https://drive.google.com/file/d/0B5hUzsxe4cdmcVpoR0N6cEJLeFU/view?usp=sharing Slides]
|-
|-
|11:15 am|| 11:40 am||[[ZFS and Containers]]||Michael Crogan||
|Keynote||Dustin Kirkland||[http://canonical.com/ Canonical]||[https://youtu.be/nDmvwevnJNc Video]||N/A
|-
|-
|11:40 am||11:55 am||Channel Programs||Sara Hartse & Chris Williamson||[http://delphix.com Delphix]
|Lustre, Supercomputers, and ZFS||Brian Behlendorf||[http://llnl.gov LLNL]||[https://youtu.be/RoyrIocAByU Video]||[https://drive.google.com/file/d/0B5hUzsxe4cdmLVRwZy1WY19LYkE/view?usp=sharing Slides]
|-
|-
|11:55 am||12:45 pm||colspan=3|'''Lunch'''
|[[ZFS and Containers]]||Michael Crogan|| ||[https://youtu.be/HOCihVmsbwQ Video]||[https://drive.google.com/file/d/0B5hUzsxe4cdmbTRBXzZPN3BrbDQ/view?usp=sharing Slides]
|-
|-
|12:45 pm||1:10 pm||ZFS First Mount||Mark Shellenbaum||[http://www.oracle.com Oracle]
|Channel Programs||Sara Hartse & Chris Williamson||[http://delphix.com Delphix]||[https://youtu.be/ogOwo4mZxMA Video]||[https://drive.google.com/file/d/0B5hUzsxe4cdmTVFUb2ItekRaU2s/view?usp=sharing Slides]
|-
|-
|1:10 pm||1:50pm||[[Scrub/Resilver Performance]]||Saso Kiselkov||[http://www.nexenta.com/ Nexenta]
|ZFS First Mount||Mark Shellenbaum||[http://www.oracle.com Oracle]||[https://youtu.be/xMH5rCL8S2k Video]||[https://drive.google.com/file/d/0B5hUzsxe4cdmT1lDS3JaUlhwc1E/view?usp=sharing Slides]
|-
|-
|1:50 pm||2:20 pm||colspan=3|'''Break'''
|[[Scrub/Resilver Performance]]||Saso Kiselkov||[http://www.nexenta.com/ Nexenta]||[https://youtu.be/SZFwv8BdBj4 Video]||[https://drive.google.com/file/d/0B5hUzsxe4cdmVU91cml1N0pKYTQ/view?usp=sharing Slides]
|-
|-
|2:20 pm||3:00 pm||[[ZFS-Native Encryption]]||Tom Caputi||[http://www.datto.com Datto]
|[[ZFS-Native Encryption]]||Tom Caputi||[http://www.datto.com Datto]||[https://youtu.be/frnLiXclAMo Video]||[https://drive.google.com/file/d/0B5hUzsxe4cdmU3ZTRXNxa2JIaDQ/view?usp=sharing Slides]
|-
|-
|3:00 pm||3:40 pm||[[Fault Management]]||Don Brady & Justin Gibbs||[http://www.intel.com/ Intel] & [https://www.freebsdfoundation.org/ FreeBSD Foundation]
|[[Fault Management]]||Don Brady & Justin Gibbs||[http://www.intel.com/ Intel] & [https://www.freebsdfoundation.org/ FreeBSD Foundation]||[https://youtu.be/zJbQiqdcpDg Video]||[https://drive.google.com/file/d/0B5hUzsxe4cdmeGFNRzZKV2J0blE/view?usp=sharing Slides]
|-
|-
|3:40 pm||4:10 pm||colspan=3|'''Break'''
|[[ZFS Validation & QA]]||Sydney Vanda & John Salinas||[http://www.intel.com/ Intel]||[https://youtu.be/N0OZN8Z2IX0 Video]||[https://drive.google.com/file/d/0B5hUzsxe4cdmalBiay1EdHdMbWs/view?usp=sharing Slides]
|-
|-
|4:10 pm|| 4:50 pm||[[ZFS Validation & QA]]||Sydney Vanda & John Salinas||[http://www.intel.com/ Intel]
|Closing||Matt Ahrens||[http://www.delphix.com/ Delphix]||[https://youtu.be/kn5xnqKehfY Video]||N/A
|-
|4:50 pm||5:15 pm||Noms database||Adam Leventhal||[https://www.shv.com/ Sutter Hill Ventures]
|-
|5:15 pm||5:30 pm||colspan=3|'''Break'''
|-
|5:30 pm||8:00 pm||colspan=3|'''Dinner'''
|}
|}


====Day 2: September 26th====
==Hackathon==


{| class="wikitable"
|-
!Start!!End!!Event
|-
|9:00 am||9:45 am||Lightning Talks
|-
|9:45 am||10:00 am|| '''Break'''
|-
|10:00 am||12:00 pm||Hackathon
|-
|12:00 pm||1:00 pm||'''Lunch'''
|-
|1:00 pm||4:00 pm||Hackathon
|-
|4:00 pm||5:00 pm||Hackathon Presentations
|}


We will have lightning (5 minute) updates on the following projects which have been discussed at previous conferences:
We had lightning (5 minute) updates on the following projects which have been discussed at previous conferences:


{| class="wikitable"
{| class="wikitable"
Line 123: Line 80:
|Eager Zero||George Wilson||[https://drive.google.com/file/d/0B4BF1vnv6p0-T21WN0N2N0pCY0E/view?usp=sharing Slides] [https://youtu.be/oVh6l4BTntw Video] (2015)
|Eager Zero||George Wilson||[https://drive.google.com/file/d/0B4BF1vnv6p0-T21WN0N2N0pCY0E/view?usp=sharing Slides] [https://youtu.be/oVh6l4BTntw Video] (2015)
|-
|-
|Compressend Send and Receive || Dan Kimmel||[https://drive.google.com/file/d/0B4BF1vnv6p0-cHZEeEJaZmNpZVE/view?usp=sharing Slides] [https://youtu.be/LaNgoAZeTww Video] (2015)
|Compressed Send and Receive || Dan Kimmel||[https://drive.google.com/file/d/0B4BF1vnv6p0-cHZEeEJaZmNpZVE/view?usp=sharing Slides] [https://youtu.be/LaNgoAZeTww Video] (2015)
|-
|-
|Device Removal || Matt Ahrens || [http://open-zfs.org/w/images/b/b4/Device_Removal-Alex_Reece_%26_Matt_Ahrens.pdf Slides] [http://www.youtube.com/watch?v=Xs6MsJ9kKKE&list=PLaUVvul17xSdOhJ-wDugoCAIPJZHloVoq&index=12 Video] (2014)
|Device Removal || Matt Ahrens || [http://open-zfs.org/w/images/b/b4/Device_Removal-Alex_Reece_%26_Matt_Ahrens.pdf Slides] [http://www.youtube.com/watch?v=Xs6MsJ9kKKE&list=PLaUVvul17xSdOhJ-wDugoCAIPJZHloVoq&index=12 Video] (2014)
Line 134: Line 91:
|-
|-
|Persistent L2ARC and TRIM||Saso Kiselkov||[https://drive.google.com/file/d/0B4BF1vnv6p0-RnBoQXc3MVdLRGM/view?usp=sharing Slides] [https://youtu.be/kt91t4sBWi8 Video] (2015)
|Persistent L2ARC and TRIM||Saso Kiselkov||[https://drive.google.com/file/d/0B4BF1vnv6p0-RnBoQXc3MVdLRGM/view?usp=sharing Slides] [https://youtu.be/kt91t4sBWi8 Video] (2015)
|-
|SPA import and pool recovery||Pavel Zakharov||
|}
|}


<!--
=== Hackathon Projects===
==Sponsorship==
 
Thank you for considering sponsoring the OpenZFS Developers Summit!
Congratulations to our 4 "best in show" winners, who received [http://www.intel.com/content/www/us/en/compute-stick/intel-compute-stick.html Intel Compute Sticks], donated by Intel.


As an OpenZFS Dev Summit sponsor, you are providing critical support to our annual event, without which it would not be able to continue. Your generous contributions pay for the venue, food and beverage expenses, live streaming service, and other miscellaneous goods and services. There are multiple sponsorship opportunities for each package.
* zpool checkpoint (Dan Kimmel, Serapheim Dimitropoulos).  Allows reverting the entire pool to a previous state.
* preferred ashift (George Wilson).  Allows dynamically switching to 4K sector aligned allocations
* zstd compression algo (Saso Kiselkov). Better compression ratio & faster than zlib/gzip.
* testing encryption with Illumos debug kernel (Tom Caputi & Dan McDonald). Using ::findleaks and kmem_flags=0xf to find bugs in encryption implementation


Please note that we are happy to customize a sponsorship package if you are willing to provide more or less support than specified by our packages.
Thanks to everyone who participated at the hackathon!


We are also happy to accept non-monetary support, such as by webcasting and video recording the event, creating the badges, providing hackathon prizes, or designing the t-shirts. For this type of support, we match the sponsorship benefits to the level of effort/support you provide.
* John Kennedy - zpool wait
* Pavel, Don, Steve, George - special metadata vdev classes
* Richard Elling - timing zil_replay
* Prakash - automated pull request testing for openzfs
* Alek P - async delete, port dnode backfill patch from Linux to OpenZFS
* Steven Burgess & Intel QA folks- platform-independent test suite changes
* Richard Laager - Linux pull requests, Ubuntu Root-on-ZFS HOWTO LUKS updates
* David Chen - ABD Linux code testing
* Alexander Motin - slog performance investigations
* Prashanth - large dnode from Linux to Illumos
* SMART data in pool stats
* ZoL buildbot check style
* Chris W - upstreaming channel programs
* Paul D - compatibility layer
* Matt - upstreaming device removal
* Jim Salter - Updating OpenZFS [[Roadmap]]


For more information or questions, please contact admin@open-zfs.org.
<!--
==== Simpler hackathon projects ====
* cppcheck errors (reported on mailing list, DLPX-47345)
* ::metaslab_trace and ::metaslab_weight dcmds should be documented (DLPX-45762
* break zfs/zpool manpages into one page per subcommand
* zfs send —current-format
* “zfs API” subcommand to enumerate software capabilities (e.g. what CLI flags are allowed, when things change (zfs send -nv now outputs to stdout instead of stderr))
* more type safety in ZFS: typedef struct { uint64_t doi_id} dmu_object_id_t
* metaslab size can exceed offset addressable by spacemap (DLPX-39390)


===Sponsor Deadlines===
==== Medium-size hackathon projects ====
{| class="wikitable" style="vertical-align:top;"
* zstd compression algo (saso)
|-
* storage pool checkpoint (DLPX-46652)
| style="width: 15% | August 29, 2016 || Attendee registration deadline for reserved sponsor tickets. <br/> '''Attendees registered after this deadline may need to be placed on a waitlist.'''
* port "zpool iostat" enhancements from Linux to illumos/FreeBSD [https://github.com/zfsonlinux/zfs/pull/4433 ZoL PR]
|-
* add subcommand to wait for background zfs activity to complete (DLPX-44432)
| September 1, 2016 || Sponsorship confirmed, logo provided, and documentation returned in order to have logo included on OpenZFS Dev Summit 2016 conference t-shirt and conference signage
* compress last block of each object, even if compression is disabled (DLPX-42061)
|-
* store total # blocks on disk (of each type, in each ds, etc) (George W)
| September 1, 2016 || Sizes and quantities provided for extra t-shirts
* port "Backfill metadnode more intelligently" from Linux to illumos/FreeBSD [https://github.com/zfsonlinux/zfs/pull/4711 ZoL PR]
|}
* port large dnode support from Linux to illumos/FreeBSD [https://github.com/zfsonlinux/zfs/pull/3542 ZoL PR]
* port SIMD-vectorized fletcher4 and RAIDZ code from Linux to illumos/FreeBSD
* cross-platform compatibility layer (Paul D)
* use refcount_t for v_count (fileystem-ey but not ZFS)
* on-disk map from dataset GUID -> set of object IDs
* scrub specific dataset
* channel programs enhancements
** open-context channel program
** setting properties
** new sync tasks
* finish reviewing / push LZ4HC compression [https://github.com/zfsonlinux/zfs/pull/3908 ZoL PR]
* allow receive of -L / -e / -c streams on systems with the required feature flags disabled
* allow spacemap to represent larger range with one entry (increase entry size)


===Sponsorship Packages===
==== Backlog ideas ====
{| class="wikitable" style="text-align:center;"
* error counts in per-vdev object
! || Platinum<br />Beer Bash<br />Hackathon || Gold || Silver || Bronze
* scrub on readonly pool
|-
* “resilver groups”/“vdev-directed resilver” where additional failure does not cause scrub to restart
| Amount || $4,000 || $2,000 || $1,000 || $500
* preferred allocation alignment (george)
|-
* promote prefetch reads to sync reads
| Logo on the back of OpenZFS Dev Summit t-shirts || X || || ||
* runtime lock analyzer (like freebsd WITNESS)
|-
* test suite for system attributes
| A dedicated table at venue for promotional purposes to<br />display sponsor branded swag and any provided tablecloths || X || || ||
* feature flags support for disabling inactive features
|-
* ZFS should prefer gethrtime() to ddi_get_lbolt()
| Space to put up the sponsoring company’s<br />pop-up banner for display at venue || X || || ||
* create vdev in syncing context (with sync task)
|-
* Add interface for creating file with specific recordsize
| Home page acknowledgement with<br />corporate name, logo, and link on open-zfs.org || X || || ||
* does ::abuf_find still work?
|-
* JSON command output
| Top of event page acknowledgement with<br />corporate name, logo, and link on open-zfs.org || X || X || ||
* zfs help command
|-
* zfs set print usage message specific to property you're trying to set
| Ability to provide a number of sponsor branded<br />swag types for promotional purposes || Unlimited || 2 || 1 ||
* make zfs recv less trusting
|-
* convert some AVL trees to b-trees or hash tables
| Reserved Free Conference Tickets || 4 || 3 || 2 || 1
* cross platform xattr
|-
| Extra t-shirts in chosen sizes sent to the sponsor || 5 || 3 || 2 || 1
|-
| An opportunity to be known as an OpenZFS supporter<br />in a very talented and influential community || X || X || X || X
|-
|Inclusion of company logo on a<br />sponsor “thank you” banner for display at venue || X || X || X || X
|-
|Event page acknowledgement with<br />corporate name, logo, and link on open-zfs.org || X || X || X || X
|-
|Permission to use OpenZFS Dev Summit logo<br />on sponsor website || X || X || X || X
|-
|Mentioned as a “thank you” during<br />the opening and closing of the event || X || X || X || X
|}
-->
-->
==Sponsorship==
A huge thank you for all the companies sponsoring the OpenZFS Developers Summit!
As an OpenZFS Dev Summit sponsor, you are providing critical support to our annual event, without which it would not be able to continue. Your generous contributions pay for the venue, food and beverage expenses, live streaming service, and other miscellaneous goods and services.


==Harrasment Policy==
==Harrasment Policy==

Latest revision as of 19:45, 27 June 2017

OpenZFS Dev Summit 2016

The fourth annual OpenZFS Developer Summit was held in San Francisco, California.

The goal of the event is to foster cross-community discussions of OpenZFS work and to make progress on some of the projects we have proposed. This 2-day event consists of a day of presentation and a 1-day hackathon.

Check out the following blog posts about the event:


Thank You 2016 Sponsors!

Platinum Sponsors

  • Delphix
  • Intel
  • OSNexus
  • Nexenta
  • datto

Gold Sponsors

  • Syneto
  • iXSystems

Silver Sponsors

  • FreeBSD Foundation
  • High Availability
  • Canonical

Presentations

Click on the title of the presentation for more details.

Title Speaker Company Video Slides
State of the Union Matt Ahrens Delphix Video Slides
Keynote Dustin Kirkland Canonical Video N/A
Lustre, Supercomputers, and ZFS Brian Behlendorf LLNL Video Slides
ZFS and Containers Michael Crogan Video Slides
Channel Programs Sara Hartse & Chris Williamson Delphix Video Slides
ZFS First Mount Mark Shellenbaum Oracle Video Slides
Scrub/Resilver Performance Saso Kiselkov Nexenta Video Slides
ZFS-Native Encryption Tom Caputi Datto Video Slides
Fault Management Don Brady & Justin Gibbs Intel & FreeBSD Foundation Video Slides
ZFS Validation & QA Sydney Vanda & John Salinas Intel Video Slides
Closing Matt Ahrens Delphix Video N/A

Hackathon

We had lightning (5 minute) updates on the following projects which have been discussed at previous conferences:

Title Speaker Previous Talk
ABD solves large/fragmented memory Dan Kimmel & David Chen Slides Video (2015)
Eager Zero George Wilson Slides Video (2015)
Compressed Send and Receive Dan Kimmel Slides Video (2015)
Device Removal Matt Ahrens Slides Video (2014)
Parity Declustered RAID for ZFS (DRAID) Isaac Huang Slides Video (2015)
SPA Metadata Allocation Classes Don Brady Slides Video (2015)
Redacted send/receive Paul Dagnelie Slides Video (2015)
Persistent L2ARC and TRIM Saso Kiselkov Slides Video (2015)
SPA import and pool recovery Pavel Zakharov

Hackathon Projects

Congratulations to our 4 "best in show" winners, who received Intel Compute Sticks, donated by Intel.

  • zpool checkpoint (Dan Kimmel, Serapheim Dimitropoulos). Allows reverting the entire pool to a previous state.
  • preferred ashift (George Wilson). Allows dynamically switching to 4K sector aligned allocations
  • zstd compression algo (Saso Kiselkov). Better compression ratio & faster than zlib/gzip.
  • testing encryption with Illumos debug kernel (Tom Caputi & Dan McDonald). Using ::findleaks and kmem_flags=0xf to find bugs in encryption implementation

Thanks to everyone who participated at the hackathon!

  • John Kennedy - zpool wait
  • Pavel, Don, Steve, George - special metadata vdev classes
  • Richard Elling - timing zil_replay
  • Prakash - automated pull request testing for openzfs
  • Alek P - async delete, port dnode backfill patch from Linux to OpenZFS
  • Steven Burgess & Intel QA folks- platform-independent test suite changes
  • Richard Laager - Linux pull requests, Ubuntu Root-on-ZFS HOWTO LUKS updates
  • David Chen - ABD Linux code testing
  • Alexander Motin - slog performance investigations
  • Prashanth - large dnode from Linux to Illumos
  • SMART data in pool stats
  • ZoL buildbot check style
  • Chris W - upstreaming channel programs
  • Paul D - compatibility layer
  • Matt - upstreaming device removal
  • Jim Salter - Updating OpenZFS Roadmap


Sponsorship

A huge thank you for all the companies sponsoring the OpenZFS Developers Summit!

As an OpenZFS Dev Summit sponsor, you are providing critical support to our annual event, without which it would not be able to continue. Your generous contributions pay for the venue, food and beverage expenses, live streaming service, and other miscellaneous goods and services.

Harrasment Policy

OpenZFS Developer Summit is dedicated to providing a harassment-free conference experience for everyone, regardless of gender, gender identity and expression, sexual orientation, disability, physical appearance, body size, race, age or religion. We do not tolerate harassment of conference participants in any form. Sexual language and imagery is not appropriate for any conference venue, including talks. Conference participants violating these rules may be sanctioned or expelled from the conference at the discretion of the conference organizers.

For information about last year's event, see the OpenZFS Developer Summit 2015 page.