RabbitMQ: JSONReader in the default JSON-RPC mapper never terminates on truncated input, causing DoS

Summary

com.rabbitmq.tools.json.JSONReader.read() never returns when its input ends inside a quoted string or a // line comment. Both scanners walk the input with StringCharacterIterator.next() but only compare against a delimiter, so once the iterator reaches CharacterIterator.DONE (￿) they loop forever. The string scanner (string(), line 210, while (c != sep)) also appends ￿ to a StringBuilder every iteration, so it fills the heap and throws OutOfMemoryError, taking down the JVM. The comment scanner (skipWhiteSpace(), lines 89-92, while (c != '\n')) pins a thread at 100% CPU with no allocation.

This is reachable with a single message. JsonRpcServer and JsonRpcClient fall back to DefaultJsonRpcMapper whenever no mapper is passed (JsonRpcServer.java:84 and :114, JsonRpcClient.java:186), and that mapper hands the raw message body straight to JSONReader.read() (DefaultJsonRpcMapper.java:42 for the server request, :52 for the client reply). A caller that can publish to the RPC request queue hangs the server; a malicious or MITM'd JSON-RPC service does the same to a client.

Proof of concept

Against amqp-client 5.36.0 from Maven Central:

import com.rabbitmq.tools.jsonrpc.DefaultJsonRpcMapper;

public class Poc {
    public static void main(String[] args) {
        DefaultJsonRpcMapper mapper = new DefaultJsonRpcMapper();
        mapper.parse("{\"method\":\"x", String.class); // unterminated string
        // mapper.parse("//", String.class);           // unterminated // comment
        System.out.println("unreachable");
    }
}

java -Xmx64m -cp amqp-client-5.36.0.jar:. Poc throws OutOfMemoryError: Java heap space in about 0.1s and never prints. Swapping in the // line spins at 100% CPU and never returns. A well-formed body such as {"method":"x"} returns immediately.

Impact

Availability. One small, unauthenticated message stops a JSON-RPC endpoint: the unterminated string exhausts the heap, the unterminated comment pins a thread forever. Neither is recoverable per request - JsonRpcServer.doCall only catches ClassCastException, and an OutOfMemoryError affects the whole process.

Scope and fix

Only applications using the JSON-RPC-over-AMQP tooling (com.rabbitmq.tools.jsonrpc) with the default DefaultJsonRpcMapper are affected. DefaultJsonRpcMapper and JSONReader are deprecated in favour of JacksonJsonRpcMapper, but both still ship and remain the default when no mapper is supplied. The fix is to stop both loops at CharacterIterator.DONE.