Difference between revisions of "Projects"
Grahamperrin (talk | contribs) (More sentences, fewer bullets. Awareness-raising ideas.) |
Grahamperrin (talk | contribs) (illumos (not Illumos). More sentences, fewer bullets, addressable subheadings …) |
||
| Line 21: | Line 21: | ||
== Inter-platform coordination ideas == | == Inter-platform coordination ideas == | ||
Ideas for projects that would help coordinate changes between | Ideas for projects that would help coordinate changes between platforms … | ||
=== Mechanism for pull changes from one place to another === | |||
Make it easier to build, test, code review, and integrate ZFS changes into illumos. | |||
=== Cross-platform test suite === | |||
One sourcebase, rather than porting STF to every platform? | |||
* | |||
* | Maybe integrate [https://github.com/behlendorf/xfstests XFS Test Suite]. | ||
* | |||
=== Userland ZFS === | |||
We already have ztest / libzpool and want to: | |||
* expand this to also be able to test more of zfs in userland | |||
* be able to run /sbin/zfs, /sbin/zpool against userland implementation | |||
* be able to run most of testrunner (and/or STF) test suite against userland implementation | |||
=== ZFS (ZPL) version feature flags === | |||
Import ZFS on Linux sa=xattr into illumos. | |||
=== /dev/zfs ioctl interface versioning === | |||
Ensure that future additions/changes to the interface maintain maximum compatibility with userland tools. | |||
Enable FreeBSD Linux jails / illumos lx brandz to use ZFS on Linux utilities. | |||
=== Port ZPIOS from ZFS on Linux to illumos === | |||
[http://zfsonlinux.org/example-zpios.html ZPIOS example] | |||
This would require a rewrite to not use Linux interfaces. | |||
=== Virtual machine images with OpenZFS === | |||
To easily try OpenZFS on a choice of distributions within a virtual machine: | |||
* images could be built for running on public clouds | |||
* images for installing to real hardware. | |||
[[Talk:Project_Ideas#Virtual_machine_images | Discuss]] … | |||
== General feature ideas == | == General feature ideas == | ||
=== [[Projects/ZFS channel programs | ZFS channel programs]] === | |||
=== Device removal === | |||
Based on indirect vdevs, rather than bprewrite. | |||
=== Reflink support === | |||
[http://lwn.net/Articles/331808/ The two sides of reflink() <nowiki>[LWN.net]</nowiki>] | |||
=== Unified ashift handling === | |||
[http://www.listbox.com/member/archive/182191/2013/07/search/YXNoaWZ0/sort/subj/page/3/entry/7:58/20130703201427:AEA03DD0-E43E-11E2-A883-F4AAC72FE4D2/ <nowiki>[illumos-zfs]</nowiki> Specifying ashift when creating vdevs] (2013-07-03) | |||
=== 1MB blocksize === | |||
* | Preferably compatible with pool version 32, as pool-feature-flag. | ||
=== RAID-Z hybrid allocator === | |||
Preferably compatible with pool version 29 for Solaris 10u11 compatibility. | |||
=== Replace larger ZIO caches with explicit pages === | |||
Subproject: document useful kernel interfaces for page manipulation on various platforms | |||
=== Improved SPA namespace collision management === | |||
Needed mostly by virtual machine hosts. Work in progress in Gentoo. | |||
Temporary pool names in zpool import | |||
* [http://www.listbox.com/member/archive/182191/2013/07/search/YXNoaWZ0/sort/subj/page/3/entry/6:58/20130701131204:56A77554-E271-11E2-8F75-EDC51164E148/ <nowiki>[illumos-zfs]</nowiki> RFC: zpool import -t for temporary pool names] (2013-07-01) | |||
Temporary pool names in zpool create. | |||
=== TRIM support === | |||
Realtime TRIM. | |||
Free space TRIM: | |||
* walk metaslab space maps and issue discard commands to the vdevs. | |||
=== Platform agnostic encryption support === | |||
Preferably compatible with pool version 30, as pool-feature-flag. | |||
The early ZFS encryption code published in the zfs-crypto repository of OpenSolaris.org could be a starting point. A copy is available from [[User:Ryao | Richard Yao]] upon request. | |||
=== Deduplication improvements === | |||
[http://www.listbox.com/member/archive/182191/2013/02/search/Ymxvb20gZmlsdGVycw/sort/time_rev/page/1/entry/8:16/20130212183221:70E13332-756C-11E2-996D-F0C715E11FC0/ Bloom filter]. | |||
Use dedicated kmem_cache for deduplication table entries: | |||
* easy to implement | |||
* will reduce DDT entries from 512-bytes to 320-bytes. | |||
== Lustre feature ideas == | == Lustre feature ideas == | ||
The [http://www.lustre.org/ Lustre project] supports the use of ZFS as an Object Storage Target. They maintain their [http://wiki.lustre.org/index.php/Architecture_-_ZFS_for_Lustre#ZFS_Features_Required_by_Lustre own feature request page] with ZFS project ideas. Below is a list of project ideas that are well defined, benefit Lustre and have no clear benefit outside of that context. | The [http://www.lustre.org/ Lustre project] supports the use of ZFS as an Object Storage Target. They maintain their [http://wiki.lustre.org/index.php/Architecture_-_ZFS_for_Lustre#ZFS_Features_Required_by_Lustre own feature request page] with ZFS project ideas. Below is a list of project ideas that are well defined, benefit Lustre and have no clear benefit outside of that context. | ||
=== Collapsible ZAP objects === | |||
E.g. fatzap -> microzap downgrades. | |||
=== Data on separate devices === | |||
[http://wiki.lustre.org/index.php/Architecture_-_ZFS_for_Lustre#Data_on_Separate_Devices Architecture - ZFS for Lustre] … | |||
=== Large dnodes === | |||
[http://wiki.lustre.org/index.php/Architecture_-_ZFS_large_dnodes Architecture - ZFS large dnodes] … | |||
=== TinyZAP === | |||
[http://wiki.lustre.org/index.php/Architecture_-_ZFS_TinyZAP Architecture - ZFS TinyZAP] … | |||
== Awareness-raising ideas == | == Awareness-raising ideas == | ||
| Line 95: | Line 160: | ||
A very short video for ZFS and OpenZFS might throw in all that's good, using plain english wherever possible, including: | A very short video for ZFS and OpenZFS might throw in all that's good, using plain english wherever possible, including: | ||
* very close to the beginning, the word ''resilience'' | * very close to the beginning, the word ''resilience'' | ||
* ''verifiable integrity of data'' and so on | * ''verifiable integrity of data'' and so on | ||
* some basic comparisons (NTFS, HFS Plus, ReFS) | * some basic comparisons (NTFS, HFS Plus, ReFS) | ||
Revision as of 06:12, 27 September 2013
Active projects
Resumable send/receive
illumos gate - Feature #2605: Partial/incremental ZFS send/receive - illumos.org
Work in progress by Chris Siden.
Storage of small files in dnode
Work in progress by Matt Ahrens.
Raspberry Pi® support
Based on ZFS on Linux, which already works on ARM.
Unresolved issues include: running out of kernel virtual address space.
Work in progress by Ryao.
Inter-platform coordination ideas
Ideas for projects that would help coordinate changes between platforms …
Mechanism for pull changes from one place to another
Make it easier to build, test, code review, and integrate ZFS changes into illumos.
Cross-platform test suite
One sourcebase, rather than porting STF to every platform?
Maybe integrate XFS Test Suite.
Userland ZFS
We already have ztest / libzpool and want to:
- expand this to also be able to test more of zfs in userland
- be able to run /sbin/zfs, /sbin/zpool against userland implementation
- be able to run most of testrunner (and/or STF) test suite against userland implementation
ZFS (ZPL) version feature flags
Import ZFS on Linux sa=xattr into illumos.
/dev/zfs ioctl interface versioning
Ensure that future additions/changes to the interface maintain maximum compatibility with userland tools.
Enable FreeBSD Linux jails / illumos lx brandz to use ZFS on Linux utilities.
Port ZPIOS from ZFS on Linux to illumos
This would require a rewrite to not use Linux interfaces.
Virtual machine images with OpenZFS
To easily try OpenZFS on a choice of distributions within a virtual machine:
- images could be built for running on public clouds
- images for installing to real hardware.
Discuss …
General feature ideas
ZFS channel programs
Device removal
Based on indirect vdevs, rather than bprewrite.
Reflink support
The two sides of reflink() [LWN.net]
Unified ashift handling
[illumos-zfs] Specifying ashift when creating vdevs (2013-07-03)
1MB blocksize
Preferably compatible with pool version 32, as pool-feature-flag.
RAID-Z hybrid allocator
Preferably compatible with pool version 29 for Solaris 10u11 compatibility.
Replace larger ZIO caches with explicit pages
Subproject: document useful kernel interfaces for page manipulation on various platforms
Improved SPA namespace collision management
Needed mostly by virtual machine hosts. Work in progress in Gentoo.
Temporary pool names in zpool import
Temporary pool names in zpool create.
TRIM support
Realtime TRIM.
Free space TRIM:
- walk metaslab space maps and issue discard commands to the vdevs.
Platform agnostic encryption support
Preferably compatible with pool version 30, as pool-feature-flag.
The early ZFS encryption code published in the zfs-crypto repository of OpenSolaris.org could be a starting point. A copy is available from Richard Yao upon request.
Deduplication improvements
Use dedicated kmem_cache for deduplication table entries:
- easy to implement
- will reduce DDT entries from 512-bytes to 320-bytes.
Lustre feature ideas
The Lustre project supports the use of ZFS as an Object Storage Target. They maintain their own feature request page with ZFS project ideas. Below is a list of project ideas that are well defined, benefit Lustre and have no clear benefit outside of that context.
Collapsible ZAP objects
E.g. fatzap -> microzap downgrades.
Data on separate devices
Architecture - ZFS for Lustre …
Large dnodes
Architecture - ZFS large dnodes …
TinyZAP
Awareness-raising ideas
… awareness of the quality, utility, and availability of open source implementations of ZFS.
Quality
Please add or discuss your ideas.
Utility
ZFS and OpenZFS in three minutes (or less)
A very short and light video/animation to grab the attention of people who don't yet realise why ZFS is an extraordinarily good thing.
For an entertaining example of how a little history (completely unrelated to storage) can be taught in ninety seconds, see Hohenwerfen Fortress - The Imprisoned Prince Bishop (context) (part of the ZONE Media portfolio).
A very short video for ZFS and OpenZFS might throw in all that's good, using plain english wherever possible, including:
- very close to the beginning, the word resilience
- verifiable integrity of data and so on
- some basic comparisons (NTFS, HFS Plus, ReFS)
– with the 2010 fork in the midst but (blink and you'll miss that milestone) the lasting impression from the video is that ZFS is great (years ahead of the alternatives) and OpenZFS is rapidly making it better for a broader user base.
Hint: there exist many ZFS-related videos but many are a tad dry, and cover a huge amount of content. Aim for two minutes :-) … discuss…
Availability
Please add or discuss your ideas.
General
An OpenZFS channel on YouTube. Something more appealing and naturally better organised than the automatically generated ZFS channel.