Skip to content

Commit

Permalink
cli: tunnels can sporadically become unresponsive (#181286)
Browse files Browse the repository at this point in the history
Last iteration I moved some RPC logic to use Tokios "codecs" to give
them cancellation safety. These operate on streams of input data.

While this worked at first, I failed to take into account that the byte
buffer we read from the stream could have _more_ data than just the
current message under load scenarios. We were discarding any extra data
from the subsequent message. In most cases caused the next message
"length" to be read from the middle of the next message, which usually
(when parsed as a u32) was some number of gigabytes, then causing the
connection to stall forever.

Fixes #181284
  • Loading branch information
connor4312 authored May 2, 2023
1 parent c74c003 commit 657bb89
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions cli/src/msgpack_rpc.rs
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
* Licensed under the MIT License. See License.txt in the project root for license information.
*--------------------------------------------------------------------------------------------*/

use bytes::Buf;
use tokio::{
io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt, BufReader},
pin,
Expand Down Expand Up @@ -124,8 +125,8 @@ impl tokio_util::codec::Decoder for U32PrefixedCodec {
return Ok(None);
}

let msg = src[U32_SIZE..].to_vec();
src.resize(0, 0);
let msg = src[U32_SIZE..required_len].to_vec();
src.advance(required_len);
Ok(Some(msg))
}
}

0 comments on commit 657bb89

Please sign in to comment.