Gunicorn w/ Django on nginx + unix socket: REMOTE Adress is "b''" (a string containing the three characters b'')

[anx-ckreuzberger]

I just had a huge "wtf moment" when I was setting up a Django Middleware to parse the HTTP_X_FORWARDED_FOR header. For security reasons (more a precaution) my middleware would check the content of the REMOTE_ADDR header, to verify that the request is actually coming from a trusted proxy (e.g., 127.0.0.1).

Now, as already discussed in issue #797 gunicorn does not provide the REMOTE_ADDR anymore when used with an unix socket, and instead an empty string is being sent (see comment by @benoitc on Oct 9, 2014: "To summarise the context, the remote address is right now empty by default when using the unix socket. ").

So I thought I'll just make this little thing in a Django Middleware:

if request.META['REMOTE_ADDR'] == "": # empty string
    request.META['REMOTE_ADDR'] = request.META['HTTP_X_FORWARDED_FOR'].split(",")[0].strip()

Oh well... It seems that I was wrong.

Would you like to know what request.META['REMOTE_ADDR'] contains? It's actually the three characters b'' (no, not the byte representation b'', but actually the three characters!).

I believe this is not the intended behaviour, though I'm not involved enough with the whole gunicorn/nginx unix socket setup to point out where the error might be.

Here are some Details of the software that is being used:

• Debian
• Python 3.5.3
• gunicorn 19.7.1
• nginx 1.10.3
• Django 1.11.12

Gunicorn is started via systemd with the following command:
gunicorn --workers 3 --bind unix:/run/gunicorn/socket application.wsgi

Nginx is configured as follows:

upstream web {
        ip_hash;
        server unix:/run/gunicorn/socket fail_timeout=0;
}

        location @proxy {
                # proxy stuff
                proxy_read_timeout 600;
                proxy_connect_timeout 600;
                proxy_pass_request_headers on;
                proxy_set_header        X-Real-IP $remote_addr;
                proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
                proxy_set_header        Host $http_host;
                proxy_pass              http://web;
                proxy_redirect          off;
                # tell the upstream that we are using https (or whatever scheme)
                proxy_set_header        X-Forwarded-Proto $scheme;
        }

[anx-ckreuzberger]

edit: I've added several details into the ticket.

[tilgovi]

I've tried this locally with Python 3.6 and environ['REMOTE_ADDR'] is empty for me. I have not tested with Django. This could be a bug with Django. If you can provide a minimal test I can take a look.

[anx-ckreuzberger]

I'll try to provide a docker setup where this bug happens, stay tuned.

[anx-ckreuzberger]

I've created an example repo:
https://github.com/anx-ckreuzberger/gunicorn-nginx-socket-docker

Feel free to check it out, it should be easy to reproduce.

[anx-ckreuzberger]

This issue seems to be appearing with Python 3.5.

Python 3.6.0 also has this issue.
Python 3.6.5 does not have this issue. I'm trying to see at which version of Python 3.6.* this issue was introduced.

[anx-ckreuzberger]

It seems like this issue is related to the python version, rather than gunicorn.

Affected Versions:
Python 3.5.2, 3.5.3
Python 3.6.0, 3.6.1

Non-affected versions:
Python 3.5.4+
Python 3.6.2+

I'm just making an educated guess here, but I believe that python issue 30205 is the cause (patch has been included with Python 3.5.4 and Python 3.6.2).

I'm not 100% sure if you can fix this in gunicorn. However, the problem is appearing on a debian 9.4, so this issue is probably affecting quite a bunch of users.

[tilgovi]

Closed by #1769.

[ichux]

It is also worth noting that the REMOTE_ADDR will be completely empty if you bind Gunicorn to a UNIX socket and not a TCP host:port tuple.

https://docs.gunicorn.org/en/stable/deploy.html