Add device rebuild feature

The device_rebuild feature enables sequential reconstruction when
resilvering.  Mirror vdevs can be rebuilt in LBA order which may
more quickly restore redundancy depending on the pools average block
size, overall fragmentation and the performance characteristics
of the devices.  However, block checksums cannot be verified
as part of the rebuild thus a scrub is automatically started after
the sequential resilver completes.

The new '-s' option has been added to the `zpool attach` and
`zpool replace` command to request sequential reconstruction
instead of healing reconstruction when resilvering.

    zpool attach -s <pool> <existing vdev> <new vdev>
    zpool replace -s <pool> <old vdev> <new vdev>

The `zpool status` output has been updated to report the progress
of sequential resilvering in the same way as healing resilvering.
The one notable difference is that multiple sequential resilvers
may be in progress as long as they're operating on different
top-level vdevs.

The `zpool wait -t resilver` command was extended to wait on
sequential resilvers.  From this perspective they are no different
than healing resilvers.

Sequential resilvers cannot be supported for RAIDZ, but are
compatible with the dRAID feature being developed.

As part of this change the resilver_restart_* tests were moved
in to the functional/replacement directory.  Additionally, the
replacement tests were renamed and extended to verify both
resilvering and rebuilding.

Original-patch-by: Isaac Huang <he.huang@intel.com>
Reviewed-by: Tony Hutter <hutter2@llnl.gov>
Reviewed-by: John Poduska <jpoduska@datto.com>
Co-authored-by: Mark Maybee <mmaybee@cray.com>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Closes #10349

tests/zfs-tests/include/libtest.shlib

@@ -2222,26 +2222,27 @@ function check_pool_status # pool token keyword <verbose>
 	if [[ $verbose == true ]]; then
 		log_note $scan
 	fi
-	echo $scan | grep -i "$keyword" > /dev/null 2>&1
+	echo $scan | egrep -i "$keyword" > /dev/null 2>&1
 
 	return $?
 }
 
 #
 # The following functions are instance of check_pool_status()
-#	is_pool_resilvering - to check if the pool is resilver in progress
-#	is_pool_resilvered - to check if the pool is resilver completed
-#	is_pool_scrubbing - to check if the pool is scrub in progress
-#	is_pool_scrubbed - to check if the pool is scrub completed
-#	is_pool_scrub_stopped - to check if the pool is scrub stopped
-#	is_pool_scrub_paused - to check if the pool has scrub paused
-#	is_pool_removing - to check if the pool is removing a vdev
-#	is_pool_removed - to check if the pool is remove completed
-#	is_pool_discarding - to check if the pool has checkpoint being discarded
+#	is_pool_resilvering - to check if the pool resilver is in progress
+#	is_pool_resilvered - to check if the pool resilver is completed
+#	is_pool_scrubbing - to check if the pool scrub is in progress
+#	is_pool_scrubbed - to check if the pool scrub is completed
+#	is_pool_scrub_stopped - to check if the pool scrub is stopped
+#	is_pool_scrub_paused - to check if the pool scrub has paused
+#	is_pool_removing - to check if the pool removing is a vdev
+#	is_pool_removed - to check if the pool remove is completed
+#	is_pool_discarding - to check if the pool checkpoint is being discarded
 #
 function is_pool_resilvering #pool <verbose>
 {
-	check_pool_status "$1" "scan" "resilver in progress since " $2
+	check_pool_status "$1" "scan" \
+	    "resilver[ ()0-9A-Za-z_-]* in progress since" $2
 	return $?
 }
 
@@ -3487,7 +3488,7 @@ function wait_scrubbed
 	typeset pool=${1:-$TESTPOOL}
 	while true ; do
 		is_pool_scrubbed $pool && break
-		log_must sleep 1
+		sleep 1
 	done
 }