Difference between revisions of "OpenZFS Developer Summit 2013"

From OpenZFS
Jump to navigation Jump to search
Line 44: Line 44:


* Testrunner suite running on other platforms: linux, freebsd (+ illumos libzpool)
* Testrunner suite running on other platforms: linux, freebsd (+ illumos libzpool)
* channel programs: pass instruction stream into kernel
** John Kennedy, Steven Burgess, Ned Bass, Tony Nguyen, Kirill Davydychev, Matt Ahrens
** Got several tests running on Linux
*** "it's a giant hack" -- lots of work needed to get this to integration quality
** Got several tests running against libzpool on illumos (using userland ioctl shim)
* [[Projects/ZFS Channel Programs | channel programs]]: pass instruction stream into kernel
** Chris Siden, Max Grossman, Alex Reece
** See [[Projects/ZFS Channel Programs | project page]], [http://www.slideshare.net/MatthewAhrens/openzfs-channel-programs slides] and [https://www.youtube.com/watch?v=EGKek5sZ2Xw&list=PLaUVvul17xSdWMBt5tAC8Hu7bbeWskD_q video] describing project
** Got snapshotting via a Lua program working all the way through the stack
* ZFS send progress reporting with "set proc title"
* ZFS send progress reporting with "set proc title"
** Xin Li
** Finished and sent for code review on FreeBSD
** "ps" and "top" can display zfs send progress: "sending pool/fs@snap X MB done (24%)"
** small amount of work needed to hook into title-updating routines on Linux and illumos
* dedicated kmem_cache for dedup table entries
* dedicated kmem_cache for dedup table entries
* larger (1MB) blocksize  
** John Layman, Matt Youds, Kirill
** goals: increase diagnosability, reduce memory consumption
** Finished this and also added kmem_cache for l2arc header
* larger (1MB) blocksize
** Brian Behlendorf
** Got it working, can set recordsize property to 1MB!
** Next steps: performance investigation, thorough review of all uses of SPA_MAXBLOCKSIZE
* compressed ARC
* compressed ARC
** George Wilson, Rob Mustacchi
** got it working! (modulo some refcount bugs)
** Next steps: figure out better policy for when to compress/uncompress
* upstreaming dbuf_user & dbuf state machine (spectra DMU changes)
* upstreaming dbuf_user & dbuf state machine (spectra DMU changes)
** Justin Gibbs, Tony Nguyen, Boris Protopopov
** Lots of progress, got it compiling against latest illumos code and passes several tests
* limit DDT size
* limit DDT size
** Sanjeev Bagewadi, Prashanth Sreenivasa
** got it sort of working w/ limit set by global tunable
* onexit for osx
* onexit for osx
** Jorgen Lundman
** pretty annoying to implement since OSX doesn't give much visibility into file handles, vnodes, or device handles


== Event Planning Info ==
== Event Planning Info ==
See [[OpenZFS Developer Summit 2013 planning]]
See [[OpenZFS Developer Summit 2013 planning]]

Revision as of 18:44, 22 November 2013

The first OpenZFS Developer Summit was held November 18-19, 2013. The goals of the event were:

  • to foster cross-platform community discussions of OpenZFS work
  • to make progress on some of the projects proposed for this community.

Video recordings of the presentations are in the OpenZFS channel on YouTube.

Participant Survey: Please complete by December 6 end of day.

Topics that were discussed

Title Speaker Slides Video
Introduction Matt Ahrens slides video
Platform Panel Brian Behlendorf, Jorgen Lundman, Chris Siden, Xin Li - video
Platform-independent code repository Matt Ahrens slides video
Storage Tiering Boris Protopopov slides video
Vendor Lightning Talks representatives from each participating company - video
Community Planning Karyn Ritter slides video
Channel Programs Chris Siden, Max Grossman slides video
Test Coverage John Kennedy slides video
Performance Investigation Adam Leventhal slides video
Performance on full & fragmented pools George Wilson - video
Scalability Kirill Davydychev - video
Virtual Memory Interactions Brian Behlendorf - video
Multi Tenancy Rob Mustacchi - video
Examining On-disk Format Max Bruning - video

Hackathon projects

These projects were worked on at the hackathon:

  • Testrunner suite running on other platforms: linux, freebsd (+ illumos libzpool)
    • John Kennedy, Steven Burgess, Ned Bass, Tony Nguyen, Kirill Davydychev, Matt Ahrens
    • Got several tests running on Linux
      • "it's a giant hack" -- lots of work needed to get this to integration quality
    • Got several tests running against libzpool on illumos (using userland ioctl shim)
  • channel programs: pass instruction stream into kernel
    • Chris Siden, Max Grossman, Alex Reece
    • See project page, slides and video describing project
    • Got snapshotting via a Lua program working all the way through the stack
  • ZFS send progress reporting with "set proc title"
    • Xin Li
    • Finished and sent for code review on FreeBSD
    • "ps" and "top" can display zfs send progress: "sending pool/fs@snap X MB done (24%)"
    • small amount of work needed to hook into title-updating routines on Linux and illumos
  • dedicated kmem_cache for dedup table entries
    • John Layman, Matt Youds, Kirill
    • goals: increase diagnosability, reduce memory consumption
    • Finished this and also added kmem_cache for l2arc header
  • larger (1MB) blocksize
    • Brian Behlendorf
    • Got it working, can set recordsize property to 1MB!
    • Next steps: performance investigation, thorough review of all uses of SPA_MAXBLOCKSIZE
  • compressed ARC
    • George Wilson, Rob Mustacchi
    • got it working! (modulo some refcount bugs)
    • Next steps: figure out better policy for when to compress/uncompress
  • upstreaming dbuf_user & dbuf state machine (spectra DMU changes)
    • Justin Gibbs, Tony Nguyen, Boris Protopopov
    • Lots of progress, got it compiling against latest illumos code and passes several tests
  • limit DDT size
    • Sanjeev Bagewadi, Prashanth Sreenivasa
    • got it sort of working w/ limit set by global tunable
  • onexit for osx
    • Jorgen Lundman
    • pretty annoying to implement since OSX doesn't give much visibility into file handles, vnodes, or device handles

Event Planning Info

See OpenZFS Developer Summit 2013 planning