Difference between revisions of "Main Page"

From OpenZFS
Jump to navigation Jump to search
 
(103 intermediate revisions by 4 users not shown)
Line 2: Line 2:
__NOTOC__
__NOTOC__


OpenZFS brings together developers and users from various open-source forks of the original ZFS on different platforms and we're always looking to grow our community. There are a multitude of ways to contribute to the project and contributors are happy to help newcomers. As we add diverse perspectives, our community becomes richer, and we're committed to creating an environment where all people feel welcome.  [[Newcomers | Information for new contributors is available here.]]


== Summary ==
== [[OpenZFS Developer Summit 2023]] ==


OpenZFS was [[announcement | announced in September 2013]] as the truly '''open source''' successor to the [[History | ZFS project]]. Our community '''brings together developers''' from the illumos, FreeBSD, Linux, and OS X [[distributions | platforms]], and a wide range of [[companies]] that build products on top of OpenZFS.  
The eleventh annual OpenZFS Developer Summit was held in San Francisco '''Oct 16-17 (Mon-Tue), 2023'''.


OpenZFS is an '''outstanding storage platform''' that encompasses the functionality of traditional filesystems, volume managers, and more, with consistent reliability, functionality and performance across all [[distributions]]:
===Presentations===
 
{| class="wikitable"
| '''illumos'''
|[http://wiki.illumos.org/display/illumos/ZFS Webpage]
|[https://github.com/illumos/illumos-gate/ GitHub]
|-
| '''FreeBSD'''
|[https://wiki.freebsd.org/ZFS Webpage]
|[https://github.com/freebsd/freebsd/ GitHub]
|-
| '''ZFS on Linux'''
|[http://zfsonlinux.org/ Webpage]
|[https://github.com/zfsonlinux/zfs/ GitHub]
|-
| '''OpenZFS on OS X'''
|[https://openzfsonosx.org/ Webpage]
|[https://github.com/openzfsonosx/ GitHub]
|}
 
== News ==
 
Matt Ahrens and George Wilson gave a talk on [https://www.socallinuxexpo.org/scale/16x/presentations/openzfs-basics OpenZFS Basics] at SCALE16x, March 2018.  [https://docs.google.com/presentation/d/1P2Q_eAgJH_6YorgTHDVgTZQfrBgtGs6VwLBvX7W23FA/edit?usp=sharing Slides] and [https://www.youtube.com/watch?v=MsY-BafQgj4 Video]
 
The [http://zfs.datto.com/ ZFS User Conference] was held April 2018.  Videos now posted.
 
Matt Ahrens gave a talk at BSDCAN: [https://www.bsdcan.org/2018/schedule/events/960.en.html Flexible Disk Use in OpenZFS?], June 2018
 
Sara Hartse gave a talk at BSDCAN: [https://www.bsdcan.org/2018/schedule/events/937.en.html 100x Faster Clone Deletion for ZFS], June 2018
 
[[OpenZFS_Developer_Summit_2018 | OpenZFS Developer Summit]] was held September 10-11, 2018.
 
The first [https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnWHhV-BM/edit?ts=5bb3b66c# OpenZFS Leadership Meeting] will be held October 9, 2018
 
== [[OpenZFS Developer Summit 2018]] <-- click for more info ==
 
The sixth annual [[OpenZFS Developer Summit 2018|OpenZFS Developer Summit]] was held '''September 10-11, 2018''' in San Francisco.
 
===Talks: Day 1===


{| class="wikitable"
{| class="wikitable"
Line 52: Line 13:
!Title!!Speaker!!Company!!Slides!!Video
!Title!!Speaker!!Company!!Slides!!Video
|-
|-
|State of the Union||Matt Ahrens||Delphix||[https://docs.google.com/presentation/d/144mwRPMznHL3xUxYuohIc0IZrDJDMj3s3_36_L9DJh8/edit#slide=id.p slides]||[https://www.youtube.com/watch?v=DearbVpOTro&index=2&list=PLaUVvul17xSe0pC6sCirlZXYqICP09Y8z&t=0s video]
| [[OpenZFS_Developer_Summit_2023_Talks#Introducing_Fast_Dedup_(Allan_Jude) | Introducing Fast Dedup]] ||Allan Jude || Klara Inc.||[https://openzfs.org/w/images/3/3b/Klara_-_Introducing_Fast_Dedup_-_OpenZFS2023.pdf slides] || [https://www.youtube.com/watch?v=_T2lkb49gc8&list=PLaUVvul17xSegbzdTT2Mo-QCDzEn5Z3XC&index=1&pp=iAQB video]
|-
|ZIO Pipeline Explained||George Wilson||Delphix||[https://docs.google.com/presentation/d/1ohdmjsp9mejuSRKwDeU83o9297KaiHrqC-tV__kjO6E/edit?usp=sharing slides]||[https://www.youtube.com/watch?v=qkA5HhfzsvM&index=3&list=PLaUVvul17xSe0pC6sCirlZXYqICP09Y8z&t=0s video]
|-
|[[zrepl]], a one-stop ZFS replication solution||Christian Schwarz||Student||[https://docs.google.com/presentation/d/1HXJ_9Q0kiHffeoQ7PlFn1qAfUcWyzedRgFlvL3nUlMU/edit?usp=sharing slides]||[https://www.youtube.com/watch?v=U4TUPQzZzPk&index=4&list=PLaUVvul17xSe0pC6sCirlZXYqICP09Y8z&t=0s video]
|-
|[[Observing and Monitoring ZFS Metrics Using Open Source Tools]]||Richard Elling||Newisys||[https://drive.google.com/open?id=1Q-I4xD6q_wWkmhCy5oJ0Gyt00EgvfCec slides]||[https://www.youtube.com/watch?v=b5awF6SVNwI&index=5&list=PLaUVvul17xSe0pC6sCirlZXYqICP09Y8z&t=0s video]
|-
|-
|[[Managing ZVOLs with vzvol]]||Rainbow||Moogsoft||[https://docs.google.com/presentation/d/1BFgMUoUu5NxotqVbbn-ci-gAF3ylpzNyy9Dj1iwY_F0/edit?usp=sharing slides]||[https://www.youtube.com/watch?v=PoHsYzzzp8Q&index=6&list=PLaUVvul17xSe0pC6sCirlZXYqICP09Y8z&t=0s video]
| [[OpenZFS_Developer_Summit_2023_Talks#Z.I.A._Accelerates_ZFS_Compression,_Checksumming,_and_RAIDZ_(Jason_Lee) | Z.I.A. Accelerates ZFS Compression, Checksumming, and RAIDZ]] || Jason Lee || Los Alamos National Labs || [https://openzfs.org/w/images/a/a2/Z.I.A..pdf slides] || [https://www.youtube.com/watch?v=mKDDKG0yVRg&list=PLaUVvul17xSegbzdTT2Mo-QCDzEn5Z3XC&index=3 video]
|-
|-
|Hackathon Info|| Serapheim Dimitropoulos & Sara Hartse ||||[https://drive.google.com/file/d/1gx_UmSWhUhdNMRCWFpw2L7Zd_GpwuvvT/view slides]||[https://www.youtube.com/watch?v=5xG5WjPazlk&index=7&list=PLaUVvul17xSe0pC6sCirlZXYqICP09Y8z&t=0s video]
| [[OpenZFS_Developer_Summit_2023_Talks#Idmapped_Mount_Support_in_ZFS_and_its_Application_(Youzhong_Yang) | Idmapped Mount Support in ZFS and its Application]] ||Youzhong Yang || MathWorks || [https://openzfs.org/w/images/5/56/Idmapped-mount-support-and-its-application-2.pptx slides] || [https://www.youtube.com/watch?v=tTHUtegxfro&list=PLaUVvul17xSegbzdTT2Mo-QCDzEn5Z3XC&index=3&pp=iAQB video]
|-
|-
|[[ZFS Hardware Acceleration with QAT]]||Weigang Li||Intel||[https://drive.google.com/file/d/0B_J4mRfoVJQRV3ZOd1ZMWkphcV9OYXdWT0FBblVHbVZpSmZj/view?usp=sharing slides]||[https://www.youtube.com/watch?v=4zWTU_hnGp0&index=8&list=PLaUVvul17xSe0pC6sCirlZXYqICP09Y8z&t=0s video]
| [[OpenZFS_Developer_Summit_2023_Talks#RAIDZ_Expansion_(Matt_Ahrens_&_Don_Brady) | RAIDZ Expansion]] || Matt Ahrens; Don Brady || Delphix, FreeBSD Foundation; Klara Inc, iXsystems || [https://openzfs.org/w/images/5/5e/RAIDZ_Expansion_2023.pdf slides] || [https://www.youtube.com/watch?v=tqyNHyq0LYM&list=PLaUVvul17xSegbzdTT2Mo-QCDzEn5Z3XC&index=3&pp=iAQB video]
|-
|-
|ZoL Releases||Tony Hutter||LLNL||[https://drive.google.com/file/d/0B_J4mRfoVJQRMzNPb1hBZ3RYcGg4NzFBSHd1clk4enVob01r/view?usp=sharing slides]||[https://www.youtube.com/watch?v=TyA0EKhGRsE&index=9&list=PLaUVvul17xSe0pC6sCirlZXYqICP09Y8z&t=0s video]
| [[OpenZFS_Developer_Summit_2023_Talks#OpenZFS_at_scale:_learning,_challenges,_and_awesome_customers_(Sam_Atkinson) | OpenZFS at scale: learning, challenges, and awesome customers]] ||Sam Atkinson || AWS || [https://openzfs.org/w/images/a/a0/OpenZFS_At_Scale_-_AWS.pptx slides] || [https://www.youtube.com/watch?v=6Jt9LQcobXM&list=PLaUVvul17xSegbzdTT2Mo-QCDzEn5Z3XC&index=4&pp=iAQB video]
|-
|-
|Device Removal||Matt Ahrens||Delphix||[https://docs.google.com/presentation/d/1u4gIGHJCbKAUxpFjU6VpUJaTX_pHAyT_HVpQJxUOFR8/edit?usp=sharing slides]||[https://www.youtube.com/watch?v=KGnFhmG8gT0&index=10&list=PLaUVvul17xSe0pC6sCirlZXYqICP09Y8z&t=0s video]
| [[OpenZFS_Developer_Summit_2023_Talks#Shared_Log_Pool_(Paul_Dagnelie) | Shared Log Pool]] || Paul Dagnelie ||Delphix || [https://openzfs.org/w/images/0/03/Shared_Log_Pool.pdf slides] || [https://www.youtube.com/watch?v=Z5L44a2gJdg&list=PLaUVvul17xSegbzdTT2Mo-QCDzEn5Z3XC&index=6&pp=iAQB video]
|-
|-
|[[iRAID]]||Shailendra Tripathi||Tegile, a Western Digital Brand||[https://docs.google.com/presentation/d/1pO10vS_6xY1MqFDnO1Op-BkAeVI5N4quigZHGVgBvD0/edit?usp=sharing slides]||[https://www.youtube.com/watch?v=OaZmkq5jgfk&index=11&list=PLaUVvul17xSe0pC6sCirlZXYqICP09Y8z&t=0s video]
|}
|}


===Talks: Day 2===
See [[OpenZFS Developer Summit 2022]] for presentations and information about last year's conference.


The bulk of this day will be spent on the hackathon but in the morning we'll have a few shorter talks and discussions, around 10-20 minutes each:
== Introduction to OpenZFS ==


{| class="wikitable"
'''OpenZFS''' is an open-source storage platform. It includes the functionality of both traditional file systems and volume manager. It has many advanced [[Features | features]] including:
|-
* Protection against data corruption. Integrity checking for both data and metadata.
!Title!!Speaker!!Company!!Slides!!Video
* Continuous integrity verification and automatic “self-healing” repair
|-
** Data redundancy with mirroring, RAID-Z1/2/3 [and DRAID]
|[[Log Spacemap: Flushing algorithm and performance]]|| Serapheim Dimitropoulos ||Delphix||[https://docs.google.com/presentation/d/1qxsbZGt1jCwhz-eHmilZS0ZATxGvvk7VLYBd1JnCCtI/edit?usp=sharing slides]||[https://youtu.be/x6D2dHRjkxw video]
* Support for high storage capacities — up to 256 trillion yobibytes (2^128 bytes)
|-
* Space-saving with transparent compression using LZ4, GZIP or ZSTD
|[[DRAID Rebuild Performance]]||Carles Mateo||Newisys||[https://docs.google.com/presentation/d/1GZXj0u13_FcgfnFCZNbKemlvC0NPZ7I56UVNQNHJwIc/edit?usp=sharing slides]||[https://youtu.be/CVST0t0X9YU video]
* Hardware-accelerated native encryption
|-
* Efficient storage with snapshots and copy-on-write clones
|Send Dedup||Paul Dagnelie||Delphix||[https://docs.google.com/presentation/d/1Ecwg3L52FQ7B86QPleVqXDJe9g5Jv_S4TJsNyK5uJLI/edit?usp=sharing slides]||[https://youtu.be/yPObFI-QU0o video]
* Efficient local or remote replication — send only changed blocks with ZFS send and receive
|-
|Allocation Classes||Don Brady||Delphix||[https://docs.google.com/presentation/d/17nYRgs-TAIOPODOMaq-VwuJ0LqJHEXBfM9sUDxJUJ54/edit?usp=sharing slides]||[https://youtu.be/Wg0VRAgaARI video]
|-
|[[Vdev Properties]]||Allan Jude||Klara Systems||[https://docs.google.com/presentation/d/14dr-s9ScAHvAwuDAZBNj8W0r5z77xLCGbdZFggtFT5U/edit?usp=sharing slides]||[https://youtu.be/hD0uJOOCDb0 video]
|}
 
===Hackathon===
 
Hackathon presentation [https://youtu.be/zN_tGxCpTBU video]
 
Audience Choice Awards:


{| class="wikitable"
== Contributing to OpenZFS ==
!Prize!!Project!!Participants
The OpenZFS project brings together developers from the Linux, FreeBSD, illumos, MacOS, and Windows platforms.  OpenZFS is supported by a wide range of companies.
|-
|1st place||zpool status in color / 🔥|| Tony Hutter
|-
|2nd place||vdev properties||Pavel Zahkarov, Allan Jude, Brian Behlendorf, and others
|-
|3rd place|| Newcomer track; how to ask for money; AND setting props from channel progs ||Sara Hartse
|}


===Diamond Sponsors===
There are many ways to contribute to OpenZFS including:
* OpenZFS uses [https://github.com/openzfs/zfs GitHub] to track [https://github.com/openzfs/zfs/issues bug reports] and [https://github.com/openzfs/zfs/pulls feature development]
* A monthly [[OpenZFS_Leadership_Meeting | OpenZFS Leadership Meeting]] Zoom call to discussion active development
* The annual [[OpenZFS_Developer_Summit | OpenZFS Developer Summit]]


<div><ul>
== Get Started with OpenZFS ==
<li style="display: inline-block; padding: 20px; padding-bottom: 28px;"> [[File: Datto_logo_2015.jpg|80x80px|link=http://www.datto.com/ | right | datto ]] </li>
[https://openzfs.github.io/openzfs-docs/Project%20and%20Community/FAQ.html Frequently Asked Questions]
<li style="display: inline-block; padding: 20px; padding-bottom: 28px;"> [[File: Delphix logo.png|170x150px|link=http://delphix.com | right | Delphix]] </li>
<li style="display: inline-block; padding: 20px; padding-bottom: 15px;">[[File: OsNexus.png|180x180px|link=http://www.osnexus.com/ | right | OSNEXUS ]] </li>
</ul></div>


===Platinum Sponsors===
[[Newcomers | OpenZFS concepts]]


<div><ul>
A non-exhaustive list of [[Features | OpenZFS features]]
<li style="display: inline-block; padding: 20px; padding-bottom: 15px;"> [[File:Newisys_Logo.png|200x200px|link=http://www.newisys.com/ | right | Newisys]] </li>
</ul></div>


== Automated Testing ==
== OpenZFS Technical Resources ==
[https://openzfs.github.io/openzfs-docs/Getting%20Started/index.html How to install OpenZFS]


Pull requests submitted to our [https://github.com/openzfs/openzfs/pulls GitHub page] are automatically tested, by running ztest and the ZFS Test Suite.  We use illumos VM's running on AWS to perform the tests, and results are available publicly.
[https://openzfs.github.io/openzfs-docs/man/index.html OpenZFS manual pages]


Thanks to infrastructure sponsor Nexenta for covering our AWS hosting costs! [[File: Nexenta-global-leader.jpg|180x180px|link=http://www.nexenta.com/ | Nexenta ]]
Feature Details. Detailed subsystem/feature blogs, on-disk format specifications: [[Developer_resources | Developer Resources]]
 
== Goals ==
 
The high-level goals of OpenZFS are:
* to '''raise awareness''' of the quality, utility, and availability of open source implementations of ZFS
* to encourage '''open communication''' about ongoing efforts to improve open source ZFS
* to ensure '''consistent reliability''', functionality, and performance of all distributions of ZFS.
 
The main technical goal of OpenZFS is '''easier sharing of code''' between platforms. Strategies include:
* creating a platform-independent [[mailing list]] for developers to review ZFS code and architecture changes from all platforms
* smoothing the [[illumos integration process]]
* making it easy to run both ztest and the ZFS test suite (TestRunner or STF based) on each platform
* [[Reduce code differences | reducing code differences]] between the platforms.


== Donate ==
== Donate ==
Line 145: Line 67:
We accept donations to cover our ongoing costs.
We accept donations to cover our ongoing costs.


OpenZFS is an associated project of SPI ([https://www.spi-inc.org/projects/openzfs/ Software in the Public Interest]).  SPI is a 501(c)3 nonprofit organization which handles our donations, finances, and legal holdings.  You can donate through paypal using the link below:
OpenZFS is an associated project of SPI ([https://www.spi-inc.org/projects/openzfs/ Software in the Public Interest]).  SPI is a 501(c)(3) nonprofit organization which handles our donations, finances, and legal holdings.  You can donate through paypal using the link below:


<html>
<html>
<br>
<form action="https://www.paypal.com/cgi-bin/webscr" method="post">
<form action="https://www.paypal.com/cgi-bin/webscr" method="post">
<fieldset>
<input type="hidden" name="cmd" value="_s-xclick" />
<input type="hidden" name="cmd" value="_s-xclick" />
<input type="hidden" name="hosted_button_id" value="X6KB4BFPRFAG6" />
<input type="hidden" name="hosted_button_id" value="X6KB4BFPRFAG6" />
<input type="image" src="https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif" name="submit" alt="Donate to OpenZFS via PayPal" />
<input type="image" src="https://www.paypalobjects.com/en_US/i/btn/btn_donateCC_LG.gif" name="submit" alt="Donate to OpenZFS via PayPal" />
<img alt="" src="https://www.paypalobjects.com/en_US/i/scr/pixel.gif" width="1" height="1" />
<img alt="" src="https://www.paypalobjects.com/en_US/i/scr/pixel.gif" width="1" height="1" />
</fieldset>
</form>
</form>
</html>
== Site orientation ==
* [[FAQ]]
* How to [[participate]]
* [[Companies]] with products that are based on OpenZFS
* Download [[distributions]] that include OpenZFS
* [[Events]] – the forthcoming [[OpenZFS Developer Summit]] and more
* [[Developer resources]]
** [[mailing list]]
** [[illumos integration process]]
** [[contributors]]
** [[platform code differences]]
** [[OpenZFS Office Hours]]
** implementation
*** [[Documentation/ZfsSend | ZFS send]]
*** [[Documentation/Administrative Commands | administrative commands]]
* [[System Administration|Documentation]] for users/sysadmins
** [[features]]
** [[Feature Flags| feature flags]]
** [[Publications| publications and conference talks]]
** [[history]] – from ZFS to OpenZFS
*** OpenZFS launch [[announcement]] – September 2013
** [[performance tuning]]
* [[Projects]]
** [[reduce code differences]]
** the [[Projects/ZFS Channel Programs | ZFS Channel Programs]] (ZCP) proposal
* [[About OpenZFS]]
[http://open-zfs.org/w/index.php?title=Special%3AAllPages&from=&to=&namespace=0&hideredirects=1 All main pages] (alphabetical order) …
== Twitter ==
<html>
<a class="twitter-timeline" width="720" height="437" href="https://twitter.com/OpenZFS" data-widget-id="378639669799288832">Tweets by @OpenZFS</a>
<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+"://platform.twitter.com/widgets.js";fjs.parentNode.insertBefore(js,fjs);}}(document,"script","twitter-wjs");</script>
<span>
<br>
<a href="https://twitter.com/openzfs" class="twitter-follow-button" data-show-count="true">Follow @openzfs</a>
<br>
<br>
<a href="https://twitter.com/share" class="twitter-share-button" data-via="OpenZFS">Tweet</a>
</span>
<script>!function(d,s,id){var js,fjs=d.getElementsByTagName(s)[0],p=/^http:/.test(d.location)?'http':'https';if(!d.getElementById(id)){js=d.createElement(s);js.id=id;js.src=p+'://platform.twitter.com/widgets.js';fjs.parentNode.insertBefore(js,fjs);}}(document, 'script', 'twitter-wjs');</script>
</html>
</html>

Latest revision as of 05:20, 7 November 2023

Welcome to OpenZFS

OpenZFS Developer Summit 2023

The eleventh annual OpenZFS Developer Summit was held in San Francisco Oct 16-17 (Mon-Tue), 2023.

Presentations

Title Speaker Company Slides Video
Introducing Fast Dedup Allan Jude Klara Inc. slides video
Z.I.A. Accelerates ZFS Compression, Checksumming, and RAIDZ Jason Lee Los Alamos National Labs slides video
Idmapped Mount Support in ZFS and its Application Youzhong Yang MathWorks slides video
RAIDZ Expansion Matt Ahrens; Don Brady Delphix, FreeBSD Foundation; Klara Inc, iXsystems slides video
OpenZFS at scale: learning, challenges, and awesome customers Sam Atkinson AWS slides video
Shared Log Pool Paul Dagnelie Delphix slides video

See OpenZFS Developer Summit 2022 for presentations and information about last year's conference.

Introduction to OpenZFS

OpenZFS is an open-source storage platform. It includes the functionality of both traditional file systems and volume manager. It has many advanced features including:

  • Protection against data corruption. Integrity checking for both data and metadata.
  • Continuous integrity verification and automatic “self-healing” repair
    • Data redundancy with mirroring, RAID-Z1/2/3 [and DRAID]
  • Support for high storage capacities — up to 256 trillion yobibytes (2^128 bytes)
  • Space-saving with transparent compression using LZ4, GZIP or ZSTD
  • Hardware-accelerated native encryption
  • Efficient storage with snapshots and copy-on-write clones
  • Efficient local or remote replication — send only changed blocks with ZFS send and receive

Contributing to OpenZFS

The OpenZFS project brings together developers from the Linux, FreeBSD, illumos, MacOS, and Windows platforms. OpenZFS is supported by a wide range of companies.

There are many ways to contribute to OpenZFS including:

Get Started with OpenZFS

Frequently Asked Questions

OpenZFS concepts

A non-exhaustive list of OpenZFS features

OpenZFS Technical Resources

How to install OpenZFS

OpenZFS manual pages

Feature Details. Detailed subsystem/feature blogs, on-disk format specifications: Developer Resources

We accept donations to cover our ongoing costs.

OpenZFS is an associated project of SPI (Software in the Public Interest). SPI is a 501(c)(3) nonprofit organization which handles our donations, finances, and legal holdings. You can donate through paypal using the link below: