mtd: fix Coverity integer handling issue

A Coverity robot reported an integer handling issue
(OVERFLOW_BEFORE_WIDEN) in the potentially overflowing expression:

    (mtd_div_by_ws(mtd->size, mtd) - mtd_div_by_ws(offs, mtd)) *
    mtd_oobavail(mtd, ops)

While such overflow will certainly never happen due to the numbers
handled, it is cleaner to fix this operation anyway.

The problem is that all the maths include 32-bit quantities, while the
result is stored in an explicit 64-bit value.

As maxooblen will just be compared with a size_t, let's change the
type of the variable to a size_t. This will not fix anything but will
clarify a bit the situation. Then, do an explicit cast to fix Coverity
warning.

Signed-off-by: Miquel Raynal <miquel.raynal@bootlin.com>
Signed-off-by: Boris Brezillon <boris.brezillon@bootlin.com>

diff --git a/drivers/mtd/mtdcore.c b/drivers/mtd/mtdcore.c
index 97ac219c082e7ef78993084ffcc8c9e98abaaac4..afb4b17fb670e9f4b3161219e9a0a0878ddebbe9 100644 (file)
--- a/drivers/mtd/mtdcore.c
+++ b/drivers/mtd/mtdcore.c
@@ -1136,13 +1136,13 @@ static int mtd_check_oob_ops(struct mtd_info *mtd, loff_t offs,
                return -EINVAL;
 
        if (ops->ooblen) {
-               u64 maxooblen;
+               size_t maxooblen;
 
                if (ops->ooboffs >= mtd_oobavail(mtd, ops))
                        return -EINVAL;
 
-               maxooblen = ((mtd_div_by_ws(mtd->size, mtd) -
-                             mtd_div_by_ws(offs, mtd)) *
+               maxooblen = ((size_t)(mtd_div_by_ws(mtd->size, mtd) -
+                                     mtd_div_by_ws(offs, mtd)) *
                             mtd_oobavail(mtd, ops)) - ops->ooboffs;
                if (ops->ooblen > maxooblen)
                        return -EINVAL;