]> git.itanic.dy.fi Git - linux-stable/commit
io_uring: adjust defer tw counting
authorPavel Begunkov <asml.silence@gmail.com>
Wed, 17 Jan 2024 00:57:26 +0000 (00:57 +0000)
committerGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Thu, 25 Jan 2024 23:36:00 +0000 (15:36 -0800)
commit8836df02124f037cdcd821623c6ac45dcedcb593
tree77c89c6e4d9deb8398921f0b0d3953343d9175df
parentb019406e5ad933676258af7f6b13ced24df78d56
io_uring: adjust defer tw counting

[ Upstream commit dc12d1799ce710fd90abbe0ced71e7e1ae0894fc ]

The UINT_MAX work item counting bias in io_req_local_work_add() in case
of !IOU_F_TWQ_LAZY_WAKE works in a sense that we will not miss a wake up,
however it's still eerie. In particular, if we add a lazy work item
after a non-lazy one, we'll increment it and get nr_tw==0, and
subsequent adds may try to unnecessarily wake up the task, which is
though not so likely to happen in real workloads.

Half the bias, it's still large enough to be larger than any valid
->cq_wait_nr, which is limited by IORING_MAX_CQ_ENTRIES, but further
have a good enough of space before it overflows.

Fixes: 8751d15426a31 ("io_uring: reduce scheduling due to tw")
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/108b971e958deaf7048342930c341ba90f75d806.1705438669.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Sasha Levin <sashal@kernel.org>
io_uring/io_uring.c