Skip to content

DOCS-2770 / 28 / DOCS-2770 Update L2ARC Reference Article Content #4872

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
linzi-ix merged 1 commit into from
Oct 15, 2026
Merged

DOCS-2770 / 28 / DOCS-2770 Update L2ARC Reference Article Content #4872

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 33 additions & 9 deletions content/References/L2ARC.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ aliases:

ZFS has several features to help improve performance for frequent access data read operations.
One is Adaptive Replacement Cache (ARC), which uses the server memory (RAM).
The other is second level adaptive replacement cache (L2ARC), which uses cache drives added to ZFS storage pools.
The other is second-level adaptive replacement cache (L2ARC), which uses cache drives added to ZFS storage pools.
These cache drives are multi-level cell (MLC) SSD drives and, while slower than system memory, are still much faster than standard hard drives.
ZFS (including TrueNAS) uses all of the RAM installed in a system to make the ARC as large as possible, but this can be very expensive.
Cache drives provide a cheaper alternative to RAM for frequently accessed data.
Expand All @@ -28,15 +28,40 @@ Cached drives are always striped, not mirrored.
To increase an existing L2ARC size, stripe another cache device with it.
You cannot share dedicated L2ARC devices between ZFS pools.

A cache device failure does not affect the integrity of the pool, but it might impact read performance depending on the workload and the dataset size to cache size ratio.
A cache device failure does not affect the integrity of the pool, but it might impact read performance depending on the workload and the dataset size-to-cache size ratio.

Beginning in TrueNAS 27 (OpenZFS 2.4), L2ARC uses a revised write algorithm designed to produce steadier write rates and that better protects cache device endurance.

### Multi-Sublist Scanning

Previous versions scanned ARC eviction lists one sublist at a time, and somewhat randomly, which could cause bursty and inconsistent L2ARC population. New algorithm scans across all sublists evenly within each write burst, distributing headroom consistently and reducing gaps in cache coverage.

### Persistent Scan Positions

L2ARC now saves its scan position between iterations on larger cache devices (usage at or above half of arc_c_max), instead of always re-scanning from the tail of the eviction list. This reduces CPU overhead. Smaller devices retain the original head/tail scanning behavior based on ARC warmup state.

### Per-Device Feed Threads

Each L2ARC device now runs its own feed thread instead of sharing a single global thread, allowing devices to write in parallel. Write throughput scales with the number of cache devices in the pool.

### Drive Writes Per Day (DWPD) Rate Limiting

L2ARC write speed is now governed by a drive writes per day (DWPD) budget instead of the older boost/interval-based approach.
This is intended to produce stable, predictable write rates and reduce unnecessary wear on cache SSDs. In this DWPD:

* The l2arc_dwpd_limit tunable sets the allowed writes per day, as a multiple of device capacity (default: 1).
* The unused write budget accumulates over a rolling 24-hour period and carries over.
* The writes occur in controlled bursts (up to 50 MB) with adaptive intervals to hit the target rate.
* The rate limiting applies only after the cache device's initial fill, once total L2ARC usage reaches half of arc_c_max. First-pass fill writes are unrestricted.
* The l2arc_write_boost and l2arc_write_interval tunables are removed and no longer apply.

### Persistent L2ARC in TrueNAS

By default, the L2ARC cache empties when the system restarts.
When Persistent L2ARC is enabled, a sysctl repopulates the cache device mapping during the restart process.
Persistent L2ARC preserves L2ARC performance even after a system restarts.

However, persistent L2ARC for large data pools can drastically slow the restarts process, degrading middleware and web interface performance.
However, persistent L2ARC for large data pools can drastically slow the restart process, degrading middleware and web interface performance.
Because of this, we have disabled persistent L2ARC by default in TrueNAS, but you can manually activate it.

### Activating Persistent L2ARC
Expand All @@ -46,6 +71,9 @@ Users must manually enable this function.
Refer to the Documentation Hub archive for information on enabling L2ARC persistence related to your system release, or consider upgrading to the latest publicly available TrueNAS release.

{{< tabs "L2ARC" >}}
{{< tab "TrueNAS 24.04 or later" >}}
TrueNAS enables persistent L2ARC by default. We do not recommend disabling it.
{{< /tab >}}
{{< tab "TrueNAS 13.0" >}}
Go to **System > Tunables** and click **ADD**.
For the **Variable**, enter **vfs.zfs.l2arc.rebuild_enabled**. Set the **Value** to **1** and the **Type** to **sysctl**.
Expand All @@ -62,9 +90,6 @@ In a command line, enter `sysctl vfs.zfs.l2arc.rebuild_enabled=1`.
When successful, the output reads: `vfs.zfs.l2arc.rebuild_enabled: 0 -> 1`
{{< /expand >}}
{{< /tab >}}
{{< tab "TrueNAS 24.04 or later" >}}
TrueNAS enables persistent L2ARC by default. We do not recommend disabling it.
{{< /tab >}}
{{< /tabs >}}

## Device Recommendations
Expand All @@ -75,16 +100,15 @@ However, we have a few recommendations for L2ARC devices:

* Using multiple L2ARC devices helps reduce latency and improve performance.

* Using large capacity L2ARC SSDs can benefit random Read Heavy workloads. L2ARC SSDs are faster than the existing data storage drives.
* Using large-capacity L2ARC SSDs can benefit random read-heavy workloads. L2ARC SSDs are faster than the existing data storage drives.

* Using an L2ARC device that is much faster than the data storage devices makes better use of its larger capacity.
Sequential or streaming workloads need very fast, low-latency L2ARC devices.
[We recommend Enterprise-grade NVMe devices](https://www.snia.org/sites/default/files/SDC/2019/presentations/File_Systems/McKenzie_Ryan_Best_Practices_for_OpenZFS_L2ARC_in_the_Era_of_NVMe.pdf). L2ARC device capacity depends on how much faster it is than the data storage devices.


## Resources

* "Best Practices for OpenZFS L2ARC in the Era of NVMe" : https://www.snia.org/sites/default/files/SDC/2019/presentations/File_Systems/McKenzie_Ryan_Best_Practices_for_OpenZFS_L2ARC_in_the_Era_of_NVMe.pdf [Talk Video : Best Practices for OpenZFS L2ARC in the Era of NVMe (SDC 2019)](https://www.youtube.com/watch?v=yHgSU6iqrlE)
* "Best Practices for OpenZFS L2ARC in the Era of NVMe": https://www.snia.org/sites/default/files/SDC/2019/presentations/File_Systems/McKenzie_Ryan_Best_Practices_for_OpenZFS_L2ARC_in_the_Era_of_NVMe.pdf [Talk Video : Best Practices for OpenZFS L2ARC in the Era of NVMe (SDC 2019)](https://www.youtube.com/watch?v=yHgSU6iqrlE)
* Open ZFS Repository: https://github.com/openzfs/zfs
* "ARC: A Self-Tuning, Low Overhead Replacement Cache": https://www.usenix.org/conference/fast-03/arc-self-tuning-low-overhead-replacement-cache
* "ZFS L2ARC": https://www.brendangregg.com/blog/2008-07-22/zfs-l2arc.html
Expand Down
Loading