iOS notifications not working on 5G, but works on Wi-Fi

[davidamacey]

🐞 Describe the bug
iOS App has different results between WiFi and Cell network (5G) connections.
Setup: Locally hosted docker image, routed through cloudflare tunnel with my domain name (as described by networkChuck)

1. Unable to subscribe to new topics unless I enter the locally hosted server IP address on local Wi-Fi then subscribe to topic. Then I change the server URL to my ntfy..com, and it works with the subscribed topic (on Wi-Fi). When using my personal URL the iOS app waits and then says "Request timed out"

2. When subscribed to channels on my ntfy..com server I do not get notifications over 5G, but when I am connected to the local Wi-Fi notifications are instant. When switching back to Wi-Fi I instantly receive all my test messages pushed at once, otherwise nothing happens on 5G.

💻 Components impacted
iOS Application

💡 Screenshots and/or logs
No error logs at this time.

🔮 Additional context

• Locally hosted Docker container with ntfy on Ubuntu 22.04 OS, only container using port 80; cloudflare tunnel for server address
• Initially had PiHole running, which could have caused other issues on the network, but disabled it.
• iOS 16.3.1

[Haitham-ghaida]

Hi,

I have a very similar setup. Everything works for me. Although I did have issues setting it up so maybe I can help.

A) did you setup a server.yml file in /etc/ntfy ? if not do that and when running the docker image you have to pass it like this docker run -v /var/cache/ntfy:/var/cache/ntfy -v /etc/ntfy:/etc/ntfy -p 80:80 -itd binwiederhier/ntfy serve --cache-file /var/cache/ntfy/cache.db
B) if you setup the server.yml file then remember to uncomment the bit where it says ios notifications.
C) in the server.yml file do not use the local IP. use the actual domain name so HTTP://ntfy.yourdomain.com

[davidamacey]

@Haitham-ghaida Thank for the response.

A) I used a docker compose file, only difference is I mapped the cache and etc to a permanent folder for saving information between instances

B) Yes I created the server.yml file and edited the base url to the local host IP address.

C) I tried both local IP and the cloudflare url http://ntfy.mydomain.com, but I got the same effect. I wonder if there is a setting I am missing in cloudflare. The tunnel is a CNAME and the docker image locally says its healthy in cloudflare. Are there any additional settings in cloudflare?

[LogicalSoIutions]

Having similar issues ~ Everything was just fine for 4-5 days. Also setup on a Docker Container w/ CF Tunnel.
Now, I just got the "New Message" notification and nothing else. Even force reloading the app doesn't pull in new data.

Edit: Just changed my server IP to ntfy.sh instead of my self hosted server and now the messages are coming through just fine. Something must be messed up on NTFY backend side for the upstream?

[Haitham-ghaida]

@davidamacey

I doubt it's a Cloudflare issue. like I found that having tiny mistakes in the config file can miss the whole thing for me I added HTTP:ntfy.website.com/ instead of HTTP:ntfy.website.com so just the "/" was enough to mess things up.

Make sure the upstream server for IOS is configured correctly.

@LogicalSoIutions

Mine has been working fine for one week now. But I did notice a small lag for IOS.

Sorry, I couldn't help further.

[LogicalSoIutions]

Without changing ANYTHING - it just started to work ~1 hour ago.

[davidamacey]

All, thank you for the responses. I am still troubleshooting.

1. settings.yml, created a folder at root of docker-compose file with file

• changed base-rurl to 'http://ntfy.mydoman.com'
• uncommented 'upstream-base-url: https://ntfy.sh'

2. Created a cloudflare tunnel, same as base-url, installed via docker

Tests:

1. LOCAL: Using local IP http://serverip the notifications post in both iOS (requires pull to refresh on home network) and website using serverip
2. CF TUNNEL: Using url domain http://ntfy.mydomain.com I cannot subscribe to topics (Request has timed out)
3. CF TUNNEL with HTTPS: Using url https://ntfy.mydomain.com I cannot subscribe in iOS app or post topics via curl. With Curl I get '502 Bad Geteway, Unable to reach origin service.'

From Cloudflare Tunnel Logs:
This is when I attempt to post a message: curl -d "hi test" https://ntfy.mydomain.com/test01

2023-09-27T13:13:14Z ERR Request failed error="Unable to reach the origin service. The service may be down or it may not be responding to traffic from cloudflared: dial tcp 192.XXX.XX.XX:81: i/o timeout" connIndex=0 dest=https://ntfy.mydomain.com/test01 event=0 ip=198.XX.XX.XX type=http

Docker-compose file

version: "2.3"

services:
  ntfy:
    image: binwiederhier/ntfy
    container_name: ntfy
    command:
      - serve
    environment:
      - TZ='America/New_York' # optional: set desired timezone
    # user: UID:GID # optional: replace with your own user/group or uid/gid
    volumes:
      - ./ntfy-cache:/var/cache/ntfy
      - ./ntfy-settings:/etc/ntfy
    ports:
      - 81:80
    healthcheck:
      # optional: remember to adapt the host:port to your environment
      test:
        [
          "CMD-SHELL",
          "wget -q --tries=1 http://localhost:80/v1/health -O - | grep -Eo '\"healthy\"\\s*:\\s*true' || exit 1"
        ]
      interval: 60s
      timeout: 10s
      retries: 3
      start_period: 40s
    restart: unless-stopped

I wonder if there is a mismatch with the routing between the cloudflare container and the internal configuration of ntfy container? As it seems I have followed all of the documentation and tutorials. Did I miss something?

[wunter8]

First, change your base-url to include "https" instead of just "http".

Next, I'm not very familiar with Cloudflare, but if it and ntfy are running in docker containers, I'm pretty sure you need to make sure they are on the same docker network so that they can communicate. I also think that docker containers on the same network communicate using internal ports (80 in this case), instead of the exposed port (81).

[LogicalSoIutions]

Sigh... Yea... This is frustrating. Probably just going back to Twilio SMS. This is too inconsistent on iOS.

Just randomly broke again.

[wunter8]

Have any of you tried the PWA on iOS instead of the native app? People have much more success with the PWA

[LogicalSoIutions]

I have not yet due to having too many devices that we use these notifications for.
I just am confused as to why things work when I use https://ntfy.sh as the server and test

curl -H "Title: This is a test" -H "Priority: 5" -H "Tags: warning" -d "This is a sample message" https://ntfy.sh/diff ~ works
curl -H "Title: This is a test" -H "Priority: 5" -H "Tags: warning" -d "This is a sample message" https://ntfy.mydomain/diff ~ gives generic "New Message" notification with no data showing relating to the alert.

I just don't get why this works for a few hours and then breaks without any config changes.

[wunter8]

The native iOS app is known to be buggy. The iOS PWA works much better

With your specific example, sending messages with priority headers has been known to cause problems with iOS, particularly with Cloudflare. A fix for this specific issue has been made and should be released in the next version

[davidamacey]

Hi Everyone, I really appreciate the conversation and feedback!

I believe I have figure it out! It has to do with cloudflare tunnel docker container hitting the ntfy docker container.

Steps to configure

1. Create folder with docker-compose file (see below for content)
2. In same folder create ntfy-cache and ntfy-settings
3. In ntfy-settings include the server.yml file
4. Change server.yml base-url to https://ntfy.mydomain.com and uncommented 'upstream-base-url: https://ntfy.sh/'
5. Setup cloudflare tunnel via cloudflare website with subdomain as ntfy, domain as mydomain.com, path (empty), type as HTTP, and url as ntfy
6. Once all settings are done then run docker-compose up -d

Notes for why to setup this way

Based on the cloudflare error, stating it cannot resolve the ntfy docker IP address; it actually cannot because they were run on different docker networks.

By setting the url in cloudflare, we are using the container dynamic IP of ntfy, docker will automatically resolve the local docker container IP of ntfy.

By doing this setup I am able to run ntfy in a container on my home server with instant notifications:

Tests:

1. On home server command line: `curl -d "test message 123" https://ntfy.mydomain.com/test55'
2. On home laptop connected to home WiFi using server ntfy IP (serverip/test55) post via my web browser

Both instances generate an automatic notification on my iOS phone which is NOT connected to my home WiFi, using 5G

Repo

In order to share my code more easily I created a simple repo with my docker-compose setup
run_ntfy

Please feel free to clone the repo to setup your ntfy on your home server!

For those that just want the file

version: "2.3"

services:
  ntfy:
    image: binwiederhier/ntfy
    container_name: ntfy
    command:
      - serve
    environment:
      - TZ='America/New_York' # optional: set desired timezone
    # user: UID:GID # optional: replace with your own user/group or uid/gid
    volumes:
      - ./ntfy-cache:/var/cache/ntfy
      - ./ntfy-settings:/etc/ntfy
    ports:
      - 80:80
    healthcheck:
      # optional: remember to adapt the host:port to your environment
      test:
        [
          "CMD-SHELL",
          "wget -q --tries=1 http://localhost:80/v1/health -O - | grep -Eo '\"healthy\"\\s*:\\s*true' || exit 1"
        ]
      interval: 60s
      timeout: 10s
      retries: 3
      start_period: 40s
    restart: unless-stopped
    networks:
      - ntfy_net

  tunnel:
    image: cloudflare/cloudflared:latest
    container_name: cloudflare-tunnel
    command: tunnel --no-autoupdate run --token mysecretkey
    networks:
      - ntfy_net

networks:
  ntfy_net:

[wunter8]

Yay! I'm glad you got it working! And thank you for the thorough instructions. Hopefully they are helpful to others

By the way, you should be able to use https://ntfy.mydomain.com everywhere; you shouldn't ever have to use ntfy server IP

[davidamacey]

@wunter8 Thank you for the follow up.

Correct the url of https://ntfy.mydomain.com will work outside the network. I was suggesting the serverip/topic as a test example to ensure the topics were actually showing up on the local server before going external.

Thanks again!

[z3k0sec]

@davidamacey I have the exact same setup, but notifications are not working for me.

[binwiederhier]

Folks, apologies that you are having issues. iOS selfhosted is a bit tricky, and there are still a few issues. I have not read through the entire thread, but here are some pointers.

1. The upstream-base-url must be configured to match exactly what you have configured as a default base URL in your iOS app. Otherwise messages will not arrive at all. See https://docs.ntfy.sh/config/#ios-instant-notifications
2. If you have different results between 5G and WiFI, it's probably because the iOS app cannot reach your local ntfy server. It must be able to reach it instantly when a new message arrives.
3. If you have intermittent delivery issues, you are likely being rate-limited by ntfy.sh. Please check your logs and enable debugging/tracing.
4. Please read the known issues to see if anything applies to your situation.

In general, troubleshooting like this is best done via Discord/Matrix.

I am closing this for now, because OP got their issues resolved, but I am happy to chat about other issues in chat or in other tickets.