zed additional features

This commit adds two features to zed, that macOS desires. The first
is that when you unload the kernel module, zed would enter into a
cpubusy loop calling zfs_events_next() repeatedly. We now look for
ENODEV, returned by kernel, so zed can exit gracefully.

Second feature is -I (idle) (alas -P persist was taken) is for the
deamon to;

1; if started without ZFS kernel module, stick around waiting for it.
2; if kernel module is unloaded, go back to 1.

This is due to daemons in macOS is started by launchctl, and is
expected to stick around.

Currently, the busy loop only exists when errno is ENODEV. This is
to ensure that functionality that upstream expects is not changed.
It did not care about errors before, and it still does not. (with the
exception of ENODEV).

However, it is probably better that all errors
(ERESTART notwithstanding) exits the loop, and the issues complaining
about zed taking all CPU will go away.

Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Jorgen Lundman <lundman@lundman.net>
Closes #10476

man/man8/zed.8.in

@@ -24,6 +24,7 @@ ZED \- ZFS Event Daemon
 [\fB\-f\fR]
 [\fB\-F\fR]
 [\fB\-h\fR]
+[\fB\-I\fR]
 [\fB\-L\fR]
 [\fB\-M\fR]
 [\fB\-p\fR \fIpidfile\fR]
@@ -66,6 +67,12 @@ Lock all current and future pages in the virtual memory address space.
 This may help the daemon remain responsive when the system is under heavy
 memory pressure.
 .TP
+.BI \-I
+Request that the daemon idle rather than exit when the kernel modules are
+not loaded. Processing of events will start, or resume, when the kernel
+modules are (re)loaded. Under Linux the kernel modules cannot be unloaded
+while the daemon is running.
+.TP
 .BI \-Z
 Zero the daemon's state, thereby allowing zevents still within the kernel
 to be reprocessed.