Subclass tq_lock to eliminate a lockdep warning

When taskq_dispatch() calls taskq_thread_spawn() to create a new thread
for a taskq, linux lockdep warns of possible recursive locking.  This is
a false positive.

One such call chain is as follows, when a taskq needs more threads:
	taskq_dispatch->taskq_thread_spawn->taskq_dispatch

The initial taskq_dispatch() holds tq_lock on the taskq that needed more
worker threads.  The later call into taskq_dispatch() takes
dynamic_taskq->tq_lock.  Without subclassing, lockdep believes these
could potentially be the same lock and complains.  A similar case occurs
when taskq_dispatch() then calls task_alloc().

This patch uses spin_lock_irqsave_nested() when taking tq_lock, with one
of two new lock subclasses:

subclass              taskq
TQ_LOCK_DYNAMIC       dynamic_taskq
TQ_LOCK_GENERAL       any other

Signed-off-by: Olaf Faaland <faaland1@llnl.gov>
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Issue #480

include/sys/taskq.h

@@ -55,6 +55,14 @@
 #define TQ_NEW			0x04000000
 #define TQ_FRONT		0x08000000
 
+/* spin_lock(lock) and spin_lock_nested(lock,0) are equivalent,
+ * so TQ_LOCK_DYNAMIC must not evaluate to 0
+ */
+typedef enum tq_lock_role {
+	TQ_LOCK_GENERAL =	0,
+	TQ_LOCK_DYNAMIC =	1,
+} tq_lock_role_t;
+
 typedef unsigned long taskqid_t;
 typedef void (task_func_t)(void *);
 
@@ -81,6 +89,7 @@ typedef struct taskq {
 	struct list_head	tq_delay_list; /* delayed task_t's */
 	wait_queue_head_t	tq_work_waitq; /* new work waitq */
 	wait_queue_head_t	tq_wait_waitq; /* wait waitq */
+	tq_lock_role_t		tq_lock_class; /* class used when taking tq_lock */
 } taskq_t;
 
 typedef struct taskq_ent {