OpenZFS 9963 - Separate tunable for disabling ZIL vdev flush

Porting Notes:
* Add options to zfs-module-parameters(5) man page.
* zfs_nocacheflush move to vdev.c instead of vdev_disk.c, since
  the latter doesn't get built for user space.

Authored by: Prakash Surya <prakash.surya@delphix.com>
Reviewed by: Matt Ahrens <matt@delphix.com>
Reviewed by: Brad Lewis <brad.lewis@delphix.com>
Reviewed by: Patrick Mooney <patrick.mooney@joyent.com>
Reviewed by: Tom Caputi <tcaputi@datto.com>
Reviewed by: George Melikov <mail@gmelikov.ru>
Approved by: Dan McDonald <danmcd@joyent.com>
Ported-by: Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>

OpenZFS-issue: https://www.illumos.org/issues/9963
OpenZFS-commit: https://github.com/openzfs/openzfs/commit/f8fdf68125
Closes #8186

man/man5/zfs-module-parameters.5

@@ -1846,8 +1846,9 @@ Use \fB1\fR for yes and \fB0\fR for no (default).
 \fBzfs_nocacheflush\fR (int)
 .ad
 .RS 12n
-Disable cache flush operations on disks when writing. Beware, this may cause
-corruption if disks re-order writes.
+Disable cache flush operations on disks when writing.  Setting this will
+cause pool corruption on power loss if a volatile out-of-order write cache
+is enabled.
 .sp
 Use \fB1\fR for yes and \fB0\fR for no (default).
 .RE
@@ -1903,8 +1904,6 @@ A value of zero will disable this throttle.
 Default value: \fB30\fR and \fB0\fR to disable.
 .RE
 
-
-
 .sp
 .ne 2
 .na
@@ -2527,6 +2526,19 @@ value of 100% will create a maximum of one thread per cpu.
 Default value: \fB100\fR%.
 .RE
 
+.sp
+.ne 2
+.na
+\fBzil_nocacheflush\fR (int)
+.ad
+.RS 12n
+Disable the cache flush commands that are normally sent to the disk(s) by
+the ZIL after an LWB write has completed. Setting this will cause ZIL
+corruption on power loss if a volatile out-of-order write cache is enabled.
+.sp
+Use \fB1\fR for yes and \fB0\fR for no (default).
+.RE
+
 .sp
 .ne 2
 .na