In the Linux kernel, the following vulnerability has been resolved: RDMA/mlx5: Fix integer overflow of user QP buffer size set_user_buf_size() computes the QP buffer size by left-shifting the user-supplied rq.wqe_cnt and rq.wqe_shift values as signed integers. A sufficiently large rq.wqe_cnt causes signed integer overflow, which is undefined behavior, and yields a small or negative buf_size, causing ib_umem_get() to map a buffer smaller than the hardware will actually write into. Replace the shifts and addition with check_shl_overflow() and check_add_overflow(), rejecting invalid user inputs. Moreover, guard the identical shift computing qp->sq.offset in _create_user_qp() before set_user_buf_size() is reached.
RDMA/mlx5: Fix integer overflow of user QP buffer size