IB/mlx5: Fix how advise_mr() launches async work

Work must hold a kref on the ib_device otherwise the dev pointer can
become free before the work runs. This can happen because the work is
being pushed onto the system work queue which is not flushed during driver
unregister.

Remove the bogus use of 'reg_state':
 - While in uverbs the reg_state is guaranteed to always be
   REGISTERED
 - Testing reg_state with no locking is bogus. Use ib_device_try_get()
   to get back into a region that prevents unregistration.

For now continue with a flow that is similar to the existing code.

Fixes: 813e90b1aeaa ("IB/mlx5: Add advise_mr() support")
Signed-off-by: Jason Gunthorpe <jgg@mellanox.com>
Reviewed-by: Moni Shoua <monis@mellanox.com>

diff --git a/drivers/infiniband/hw/mlx5/odp.c b/drivers/infiniband/hw/mlx5/odp.c
index 01e0f62006314f63b5cda5580366cdeb16f1736e..4ee32964e1dd9cf7f8ce5094fdd8673c98af714a 100644 (file)
--- a/drivers/infiniband/hw/mlx5/odp.c
+++ b/drivers/infiniband/hw/mlx5/odp.c
@@ -1595,10 +1595,12 @@ static void mlx5_ib_prefetch_mr_work(struct work_struct *work)
        struct prefetch_mr_work *w =
                container_of(work, struct prefetch_mr_work, work);
 
-       if (w->dev->ib_dev.reg_state == IB_DEV_REGISTERED)
+       if (ib_device_try_get(&w->dev->ib_dev)) {
                mlx5_ib_prefetch_sg_list(w->dev, w->pf_flags, w->sg_list,
                                         w->num_sge);
-
+               ib_device_put(&w->dev->ib_dev);
+       }
+       put_device(&w->dev->ib_dev.dev);
        kfree(w);
 }
 
@@ -1617,15 +1619,13 @@ int mlx5_ib_advise_mr_prefetch(struct ib_pd *pd,
                return mlx5_ib_prefetch_sg_list(dev, pf_flags, sg_list,
                                                num_sge);
 
-       if (dev->ib_dev.reg_state != IB_DEV_REGISTERED)
-               return -ENODEV;
-
        work = kvzalloc(struct_size(work, sg_list, num_sge), GFP_KERNEL);
        if (!work)
                return -ENOMEM;
 
        memcpy(work->sg_list, sg_list, num_sge * sizeof(struct ib_sge));
 
+       get_device(&dev->ib_dev.dev);
        work->dev = dev;
        work->pf_flags = pf_flags;
        work->num_sge = num_sge;