io_uring: recycle kbuf recycle on tw requeue

[ Upstream commit 336d28a8f38013a069f2d46e73aaa1880ef17a47 ]

When we queue a request via tw for execution it's not going to be
executed immediately, so when io_queue_async() hits IO_APOLL_READY
and queues a tw but doesn't try to recycle/consume the buffer some other
request may try to use the the buffer.

Fixes: c7fb19428d67 ("io_uring: add support for ring mapped supplied buffers")
Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
Link: https://lore.kernel.org/r/a19bc9e211e3184215a58e129b62f440180e9212.1662480490.git.asml.silence@gmail.com
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Sasha Levin <sashal@kernel.org>

diff --git a/io_uring/io_uring.c b/io_uring/io_uring.c
index cd155b7e1346d6c0da256ce7908bba5578b4fbc9..effe3570a051ff0cf3198160b663f315b87f1be6 100644 (file)
--- a/io_uring/io_uring.c
+++ b/io_uring/io_uring.c
@@ -8260,6 +8260,7 @@ static void io_queue_async(struct io_kiocb *req, int ret)
 
        switch (io_arm_poll_handler(req, 0)) {
        case IO_APOLL_READY:
+               io_kbuf_recycle(req, 0);
                io_req_task_queue(req);
                break;
        case IO_APOLL_ABORTED: