]> asedeno.scripts.mit.edu Git - linux.git/commit
rcu/nocb: Reduce nocb_cb_wait() leaf rcu_node ->lock contention
authorPaul E. McKenney <paulmck@linux.ibm.com>
Mon, 15 Jul 2019 08:09:04 +0000 (01:09 -0700)
committerPaul E. McKenney <paulmck@linux.ibm.com>
Tue, 13 Aug 2019 21:38:24 +0000 (14:38 -0700)
commit1d5a81c18dc68fc38a52e8dab1992a043a358927
tree348ced2b4230c636b203b7213bf6b7a12226d4b9
parent23651d9b9616060cf86af5e3b15defcf3bcd2642
rcu/nocb: Reduce nocb_cb_wait() leaf rcu_node ->lock contention

Currently, nocb_cb_wait() advances callbacks on each pass through its
loop, though only if it succeeds in conditionally acquiring its leaf
rcu_node structure's ->lock.  Despite the conditional acquisition of
->lock, this does increase contention.  This commit therefore avoids
advancing callbacks unless there are callbacks in ->cblist whose grace
period has completed.

Note that nocb_cb_wait() doesn't worry about callbacks that have not
yet been assigned a grace period.  The idea is that the only reason for
nocb_cb_wait() to advance callbacks is to allow it to continue invoking
callbacks.  Time will tell whether this is the correct choice.

Signed-off-by: Paul E. McKenney <paulmck@linux.ibm.com>
kernel/rcu/tree_plugin.h