man: Update L2ARC tunables for DWPD and parallel writes

Add l2arc_dwpd_limit, remove l2arc_write_boost, update related tunables.

Reviewed-by: Alexander Motin <alexander.motin@TrueNAS.com>
Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Ameer Hamza <ahamza@ixsystems.com>
Closes #18093

man/man4/zfs.4

@@ -109,8 +109,7 @@ Seconds between L2ARC writing.
 .
 .It Sy l2arc_headroom Ns = Ns Sy 8 Pq u64
 How far through the ARC lists to search for L2ARC cacheable content,
-expressed as a multiplier of
+expressed as a multiplier of the effective write size.
-.Sy l2arc_write_max .
 ARC persistence across reboots can be achieved with persistent L2ARC
 by setting this parameter to
 .Sy 0 ,
@@ -125,6 +124,19 @@ A value of
 .Sy 100
 disables this feature.
 .
+.It Sy l2arc_dwpd_limit Ns = Ns Sy 100 Pq uint
+Drive Writes Per Day limit for L2ARC devices to protect SSD endurance,
+specified as a percentage where 100 equals 1.0 DWPD.
+A value of 100 means each L2ARC device can write its own capacity once per day.
+Lower values support fractional DWPD
+(50 = 0.5 DWPD, 30 = 0.3 DWPD for QLC SSDs).
+Higher values allow more writes (300 = 3.0 DWPD).
+The effective write rate is always bounded by
+.Sy l2arc_write_max .
+A value of 0 disables DWPD rate limiting entirely.
+DWPD limiting only applies after the initial fill pass completes and when
+total L2ARC capacity is at least twice arc_c_max.
+.
 .It Sy l2arc_exclude_special Ns = Ns Sy 0 Ns | Ns 1 Pq int
 Controls whether buffers present on special vdevs are eligible for caching
 into L2ARC.
@@ -179,9 +191,8 @@ can render it slow or unusable.
 This parameter limits L2ARC writes and rebuilds to achieve the target.
 .
 .It Sy l2arc_trim_ahead Ns = Ns Sy 0 Ns % Pq u64
-Trims ahead of the current write size
+Trims ahead of the current write size on L2ARC devices by this percentage
-.Pq Sy l2arc_write_max
+of write size if we have filled the device.
-on L2ARC devices by this percentage of write size if we have filled the device.
 If set to
 .Sy 100
 we TRIM twice the space required to accommodate upcoming writes.
@@ -216,13 +227,12 @@ to enable caching/reading prefetches to/from L2ARC.
 .It Sy l2arc_norw Ns = Ns Sy 0 Ns | Ns 1 Pq int
 No reads during writes.
 .
-.It Sy l2arc_write_boost Ns = Ns Sy 33554432 Ns B Po 32 MiB Pc Pq u64
-Cold L2ARC devices will have
-.Sy l2arc_write_max
-increased by this amount while they remain cold.
-.
 .It Sy l2arc_write_max Ns = Ns Sy 33554432 Ns B Po 32 MiB Pc Pq u64
-Max write bytes per interval.
+Maximum write rate in bytes per second for each L2ARC device.
+Used directly during initial fill, when DWPD limiting is disabled,
+or for non-persistent L2ARC.
+When DWPD limiting is active, writes are capped by this rate.
+Total L2ARC throughput scales with the number of cache devices in a pool.
 .
 .It Sy l2arc_rebuild_enabled Ns = Ns Sy 1 Ns | Ns 0 Pq int
 Rebuild the L2ARC when importing a pool (persistent L2ARC).

man/man7/zpoolconcepts.7

@@ -426,10 +426,24 @@ This can be changed with
 The cache device header
 .Pq Em 512 B
 is updated even if no metadata structures are written.
+.Pp
+L2ARC operates in one of two modes depending on total cache capacity.
+When total L2ARC capacity is less than twice
+.Sy arc_c_max ,
+L2ARC uses exclusive caching,
+writing buffers to cache as they are evicted from ARC.
+When total capacity is at least twice
+.Sy arc_c_max ,
+L2ARC switches to inclusive caching with persistent markers
+that track scan positions,
+attempting to duplicate ARC contents as much as write throughput allows.
 Setting
 .Sy l2arc_headroom Ns = Ns Sy 0
 will result in scanning the full-length ARC lists for cacheable content to be
 written in L2ARC (persistent ARC).
+In inclusive mode, markers progress toward the head across iterations,
+naturally covering the full list.
+.Pp
 If a cache device is added with
 .Nm zpool Cm add ,
 its label and header will be overwritten and its contents will not be