Websocket disconnect happening when uploading a base64 file with size > 1MB

[MadhusudhanGude]

We are using Django-channels to handle WebSockets and Uvicorn as ASGI server.
Everything is working fine when normal text messages are sent over WebSocket, but the WebSocket disconnect is happening when we send a base64 field ith size greater than 1MB. It is able to handle the files which are less than 1MB.

Same is the issue when uviron is run independently or by configuring it along with gunicorn(like the way how it is mentioned in the deployment documentation)

[euri10]

I added this test that confirms it, it passes fine at 1048576

@pytest.mark.parametrize("protocol_cls", WS_PROTOCOLS)
def test_send_binary_data_big_size(protocol_cls):
    SIZE = 1048577

    class App(WebSocketResponse):
        async def websocket_connect(self, message):
            await self.send({"type": "websocket.accept"})

        async def websocket_receive(self, message):
            _bytes = message.get("bytes")
            await self.send({"type": "websocket.send", "bytes": _bytes})

    async def send_text(url):
        async with websockets.connect(url) as websocket:

            data = b'\x01' * SIZE
            await websocket.send(data)
            return await websocket.recv()

    with run_server(App, protocol_cls=protocol_cls) as url:
        loop = asyncio.new_event_loop()
        data = loop.run_until_complete(send_text(url))
        assert data == b'\x01' * SIZE
        loop.close()

I'm not familiar enough with websocket protocol but it looks like from https://github.com/aaugustin/websockets/blob/46ddc64b3ab02f38579880a812b9c04da6d89ae1/src/websockets/protocol.py#L123-L127

The max_size parameter enforces the maximum size for incoming messages
in bytes. The default value is 1 MiB. None disables the limit. If a
message larger than the maximum size is received, :meth:recv will
raise :exc:~websockets.exceptions.ConnectionClosedError and the
connection will be closed with code 1009.

in case of WSProtocol test it ends with

>           raise self.connection_closed_exc()
E           websockets.exceptions.ConnectionClosedError: code = 1006 (connection closed abnormally [internal]), no reason

/home/lotso/.local/share/virtualenvs/uvicorn-gEHaMp2C/lib/python3.7/site-packages/websockets/protocol.py:780: ConnectionClosedError

in case of WebSocketProtocol it ends with

>           raise self.connection_closed_exc()
E           websockets.exceptions.ConnectionClosedError: code = 1009 (message too big), no reason

/home/lotso/.local/share/virtualenvs/uvicorn-gEHaMp2C/lib/python3.7/site-packages/websockets/protocol.py:780: ConnectionClosedError

[MadhusudhanGude]

When sending text
DEBUG: ('127.0.0.1', 49614) - ASGI [10] Sent {'type': 'websocket.receive', 'text': '<76 chars>'}
DEBUG: ('127.0.0.1', 49614) - ASGI [10] Received {'type': 'websocket.send', 'text': '<412 chars>'}

When sending image < 1MB
DEBUG: ('127.0.0.1', 49614) - ASGI [10] Sent {'type': 'websocket.receive', 'text': '<7238 chars>'}
DEBUG: ('127.0.0.1', 49614) - ASGI [10] Received {'type': 'websocket.send', 'text': '<819 chars>'}

When sending image > 1MB
DEBUG: ('127.0.0.1', 49614) - ASGI [10] Sent {'type': 'websocket.disconnect', 'code': 1006}
DEBUG: ('127.0.0.1', 49614) - ASGI [10] Received {'type': 'websocket.close'}
DEBUG: ('127.0.0.1', 49614) - ASGI [10] Completed

Attaching logs.

[euri10]

Seems like you can pass arbitrary size on WSProtocol using kwarg max_size in connect methoid, oddly it fails on WebSocketProtocol

@pytest.mark.parametrize("protocol_cls", WS_PROTOCOLS)
def test_send_binary_data_big_size(protocol_cls):
    SIZE = 1048577

    class App(WebSocketResponse):
        async def websocket_connect(self, message):
            await self.send({"type": "websocket.accept"})

        async def websocket_receive(self, message):
            _bytes = message.get("bytes")
            await self.send({"type": "websocket.send", "bytes": _bytes})

    async def send_text(url):
        async with websockets.connect(url, max_size=SIZE) as websocket:

            data = b'\x01' * SIZE
            await websocket.send(data)
            return await websocket.recv()

    with run_server(App, protocol_cls=protocol_cls) as url:
        loop = asyncio.new_event_loop()
        data = loop.run_until_complete(send_text(url))
        assert data == b'\x01' * SIZE
        loop.close()

[MadhusudhanGude]

Thought there would be any setting in the Uvicorn configuration.

Closing this as a duplicate of #306

[oogghh]

@madhu150894 did you ever find a solution to this? I'm also using channels and getting the exact same issue. Cannot find anything helpful on the matter and would appreciate any knowledge you found!

[MadhusudhanGude]

@machineslearnoliver We are using Hypercorn as the ASGI server now, it has an option to customize the max_size limit.
This can be helpful to you.

[oogghh]

@madhu150894 You’ve saved my week, thank you!