gatsby develop can only serve 5 pages at a time

[jeffrafter]

Description

When running gatsby develop everything works perfectly and fast for the first 5 pages I open (in multiple tabs). Subsequent pages opened in new tabs hang indefinitely. When I close one of the previously opened tabs the last pages load immediately. I've tried to see if this is related to similar issues including (#12143, #11747 #10257 and #11727) but it seems different.

Steps to reproduce

Start gatsby develop, open a bunch of tabs simultaneously.

Expected result

All of them load, HMR is ready.

Actual result

Only the first four load, the others hang.

Environment

  System:
    OS: macOS 10.14.2
    CPU: (12) x64 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz
    Shell: 3.2.57 - /bin/bash
  Binaries:
    Node: 10.13.0 - ~/.nvm/versions/node/v10.13.0/bin/node
    Yarn: 1.12.3 - ~/.nvm/versions/node/v10.13.0/bin/yarn
    npm: 6.4.1 - ~/.nvm/versions/node/v10.13.0/bin/npm
  Languages:
    Python: 2.7.15 - /usr/local/bin/python
  Browsers:
    Chrome: 72.0.3626.119
    Firefox: 65.0.1
    Safari: 12.0.2
  npmPackages:
    gatsby: ^2.0.85 => 2.1.17 
    gatsby-image: ^2.0.30 => 2.0.30 
    gatsby-plugin-canonical-urls: ^2.0.5 => 2.0.10 
    gatsby-plugin-feed: ^2.0.13 => 2.0.13 
    gatsby-plugin-google-analytics: ^2.0.14 => 2.0.14 
    gatsby-plugin-manifest: ^2.0.19 => 2.0.19 
    gatsby-plugin-react-helmet: ^3.0.0 => 3.0.6 
    gatsby-plugin-sharp: ^2.0.5 => 2.0.22 
    gatsby-plugin-styled-components: ^3.0.5 => 3.0.6 
    gatsby-plugin-typescript: ^2.0.0 => 2.0.9 
    gatsby-remark-autolink-headers: ^2.0.14 => 2.0.14 
    gatsby-remark-copy-linked-files: ^2.0.5 => 2.0.9 
    gatsby-remark-images: ^2.0.1 => 2.0.6 
    gatsby-remark-prismjs: ^3.0.0 => 3.2.4 
    gatsby-remark-responsive-iframe: ^2.0.5 => 2.0.9 
    gatsby-remark-smartypants: ^2.0.5 => 2.0.8 
    gatsby-source-filesystem: ^2.0.1 => 2.0.22 
    gatsby-transformer-json: ^2.1.1 => 2.1.8 
    gatsby-transformer-remark: ^2.1.3 => 2.2.6 
    gatsby-transformer-sharp: ^2.1.1 => 2.1.14 
  npmGlobalPackages:
    gatsby: 2.1.4

[adamclifford]

@jeffrafter This is a limitation of the browser you are using.

When you are developing with GatsbyJS there is a connection to the GraphQL endpoint open and all of the pages queries are then rendered based on the live queries, this is how getting live data to the pages you are editing works in the back ground.

However, each browser has a limited set of connections that it uses to connect to the same end point. In chrome, this limit is 5, when you exceed that limit, chrome restricts the connection until one has been freed up and can connect.

I hope this helps.

[DSchau]

@adamclifford whoa! Today I learned!

Could you provide a source on that? Would be helpful to document for posterity.

In any case--this doesn't seem like an issue we can resolve within Gatsby, and we're not doing anything outside the ordinary (apart from opening up a web socket) that would cause any weirdness here!

Going to close as answered--but please reply if we missed something or if the above is off-base.

Thanks for using Gatsby! 💜

[adamclifford]

@DSchau the HTTL/1.1 specification stipulates 2 connections as a limit, as can be read here.
Most modern browsers now allow more than that, as the official limit has hindered the effectiveness of said browsers, an RF7230 replaced the previous limit, and told browser vendors to not be greedy.

This is a form of protection against DDoS attacks.

I cant find an official list of available limits, but Pushtechnology has a list of the current ones.

[jeffrafter]

@adamclifford thanks for those links, this is super helpful; I never realized that limit was per browser not per tab!

When you are developing with GatsbyJS there is a connection to the GraphQL endpoint open and all of the pages queries are then rendered based on the live queries, this is how getting live data to the pages you are editing works in the back ground.

Is this just for HMR? Do you know if there is a config/fallback strategy for polling instead of open connections?

[DSchau]

Is this just for HMR

Yep! We build out the data from the local GraphQL schema, so there isn't a trace of that in the production code.

Do you know if there is a config/fallback strategy for polling instead of open connections?

Not really. Generally - is this something you do often? 5 open tabs with the same content seems to be a little overkill, but maybe there's a use case we're missing here with our current strategy!

[jeffrafter]

@DSchau it isn't a great use case... I just keep a lot of tabs open and forget some on a desktop here or there. Then I open a new tab and it is empty and I wait and then 10-15 seconds later realize "Oh, I must have 5 other tabs open somewhere" and go hunt them down through my mess of desktops and tabs and disorganization. Often it is different pages with different styling and I want to see the impact of a new header or something across them. So I have to open/close open/close.

My understanding is that this is not a problem in HTTP/2 because of multiplexing. I wonder if there is a way to move to HTTP/2 or some other connection mechanism that is more forgiving of disorganized people like myself 🤹‍♂️

[KyleAMathews]

Huh, this is an unfortunate bug. We could change from websockets to long-polling to solve this. It shouldn't have much effect on performance.

[jeffrafter]

Actually, I spent some time digging into this... if we upgrade the development experience to SPDY instead of HTTP the problem is resolved. Unfortunately, doing this (effectively) requires running the local development environment behind TLS.

I changed this line to:

     server = require(`spdy`).createServer(
      {
        key: program.ssl.key,
        cert: program.ssl.cert,
      },
      app
    )

Then I ran gatsby develop --https and everything worked perfectly. It works and you can simultaneously open as many pages as you want.

Note, I tried to run SPDY in non-TLS (this does not work):

 let server = require(`spdy`).createServer(
      {
        spdy: {
          plain: true,
          ssl: false,
        },
      },
      app,
    )

Running in plain mode with no SSL correctly serves pages with no https but in this mode Chrome falls back to normal HTTP/1 rules and the concurrent connection limit kicks in.

[jeffrafter]

@DSchau would you consider re-opening this, considering #12302

[DSchau]

Of course! Consider it done!

[gatsbot]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

Thanks for being a part of the Gatsby community! 💪💜

[gatsbot]

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.

Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.

Thanks again for being part of the Gatsby community!

[jlengstorf]

This showed up in Sizzy. cc @kitze

[kitze]

Yup. It'd be cool if at least I can change something in Sizzy so it works.

[jlengstorf]

It looks like there's a workaround in Electron (--ignore-connections-limit) that's been implemented for Sizzy — are we able to do that by default in Gatsby so the workaround isn't required?

[kitze]

@jlengstorf I think this has to be implemented on the browser side. Firefox has this setting in about:config, but Chrome only supports it via flag. Unless Gatsby changes from SSE to WebSockets I'm afraid this won't be possible.

https://tools.ietf.org/html/rfc6202#section-5.1

[coreyward]

Could Gatsby listen at multiple ports or hostnames (is the limit per-port or per-hostname)?

If so, we could potentially monitor the total number of connections per browser (doesn't have to be perfect: a user-agent heuristic probably works fine) and automatically add another port/hostname to the pool when we get close to the limit.

Alternately, there are approaches to allow for sharing data across tabs which would limit the need for multiple websocket connections, but this is enough of an edge case that I don't know that it would merit the complications that such an approach would bring.

[tu4mo]

Many component libraries show examples in iframes, so they are isolated from the documentation page. I guess there isn't any workaround to enable opening multiple Gatsby pages in iframes inside one page?

[LekoArts]

Hi!

I'm closing this as a stale issue as in the meantime Gatsby 4 and related packages were released. You can check our Framework Version Support Page to see which versions currently receive active support.

Please try the mentioned issue on the latest version (using the next tag) and if you still see this problem, open a new bug report. It must include a minimal reproduction.

Thanks!

[limichange]

I'm using the latest version of GATSBY (4.15.2) and the problem still exists, this issue needs to be reopened.

I turned on HTTPS mode, but I still have the same problem.

[FMCorz]

In Firefox, increasing the config value network.http.max-persistent-connections-per-server (from about:config) seems to help.