Difference between revisions of "Projects/ZFS Channel Programs"

From OpenZFS
Jump to navigation Jump to search
(Created page with "A ZFS Channel Program (ZCP) permits the execution of user programs inside the kernel. A ZFS Channel Program manipulate ZFS internals in a single, atomically-visible operation....")
 
Tags: Mobile edit Mobile web edit
 
(18 intermediate revisions by 4 users not shown)
Line 1: Line 1:
A ZFS Channel Program (ZCP) permits the execution of user programs inside the kernel. A ZFS Channel Program manipulate ZFS internals in a single, atomically-visible operation. For instance, to delete all snapshots of a filesystem a channel program could be written which 1) generates the list of snapshots, 2) traverses that list, and 3) destroys each snapshot unconditionally. Because each of these statements would be evaluated from within the kernel, channel programs can guarantee safety from interference with other concurrent ZFS modifications. Executing from inside the kernel allows us to guarantee atomic visibility of these operations (correctness) and allows them to be performed in a single transaction group (performance).
See also [http://www.slideshare.net/MatthewAhrens/openzfs-channel-programs slides] and [https://www.youtube.com/watch?v=EGKek5sZ2Xw&list=PLaUVvul17xSdWMBt5tAC8Hu7bbeWskD_q video] from talk at [[OpenZFS Developer Summit 2013]], and [http://open-zfs.org/w/images/d/db/Channel_Programs-Chris_Siden.pdf slides] and [http://www.youtube.com/watch?v=RMTxyqcomPA&list=PLaUVvul17xSdOhJ-wDugoCAIPJZHloVoq&index=14 video] from the [[OpenZFS Developer Summit 2014]]


A successful implementation of ZCP will:
== Proposal ==
 
A ZFS channel program (ZCP) is a small script written in a domain-specific language that manipulates ZFS internals in a single, atomically-visible operation. For instance, to delete all snapshots of a filesystem a ZCP could be written which 1) generates the list of snapshots, 2) traverses that list, and 3) destroys each snapshot unconditionally. Because each of these statements would be evaluated from within the kernel, ZCPs can guarantee safety from interference with other concurrent ZFS modifications. Executing from inside the kernel allows us to guarantee atomic visibility of these operations (correctness) and allows them to be performed in a single transaction group (performance).
 
Successful implementation of ZCP will:
# Support equivalent functionality for all of the current ZFS commands with improved performance and correctness from the point of view of the user of ZFS.
# Support equivalent functionality for all of the current ZFS commands with improved performance and correctness from the point of view of the user of ZFS.
# Facilitate the quick addition of new and useful commands as ZCP enables the implementation of more powerful operations which previously would have been unsafe to implement in user programs, or would require modifications to the kernel. Since the ZCP layer guarantees the atomicity of each channel program, we no longer need to write elaborate new sync_tasks for each new IOCTL.
# Facilitate the quick addition of new and useful commands as ZCP enables the implementation of more powerful operations which previously would have been unsafe to implement in user programs, or would require modifications to the kernel for correctness. Since the ZCP layer guarantees the atomicity of each ZCP, we only need to write new sync_tasks for individual simple operations, then can use ZCPs to chain those simple operations together into more complicated operations.
# Allow ZFS users to safely implement their own ZFS operations without breaking ZFS or performing operations they don’t have the privileges for.
# Allow ZFS users to safely implement their own ZFS operations without performing operations they don’t have the privileges for.
# Improve the performance and correctness of existing applications built on ZFS operations.
# Improve the performance and correctness of existing applications built on ZFS operations.
== Outcome ==
Integrated in 2018, see [https://openzfs.github.io/openzfs-docs/man/8/zfs-program.8.html man-page].

Latest revision as of 19:28, 11 February 2022

See also slides and video from talk at OpenZFS Developer Summit 2013, and slides and video from the OpenZFS Developer Summit 2014

Proposal

A ZFS channel program (ZCP) is a small script written in a domain-specific language that manipulates ZFS internals in a single, atomically-visible operation. For instance, to delete all snapshots of a filesystem a ZCP could be written which 1) generates the list of snapshots, 2) traverses that list, and 3) destroys each snapshot unconditionally. Because each of these statements would be evaluated from within the kernel, ZCPs can guarantee safety from interference with other concurrent ZFS modifications. Executing from inside the kernel allows us to guarantee atomic visibility of these operations (correctness) and allows them to be performed in a single transaction group (performance).

Successful implementation of ZCP will:

  1. Support equivalent functionality for all of the current ZFS commands with improved performance and correctness from the point of view of the user of ZFS.
  2. Facilitate the quick addition of new and useful commands as ZCP enables the implementation of more powerful operations which previously would have been unsafe to implement in user programs, or would require modifications to the kernel for correctness. Since the ZCP layer guarantees the atomicity of each ZCP, we only need to write new sync_tasks for individual simple operations, then can use ZCPs to chain those simple operations together into more complicated operations.
  3. Allow ZFS users to safely implement their own ZFS operations without performing operations they don’t have the privileges for.
  4. Improve the performance and correctness of existing applications built on ZFS operations.


Outcome

Integrated in 2018, see man-page.