net: make stdout & stderr block on all platforms #1771

Fishrock123 · 2015-05-22T21:34:50Z

Fixes #784

@vkurchatkin suggested a test here, but I'm not sure how to adapt to be only in javascript. Maybe with two child processes? I'm not sure how to pipe them without the unix |.

R=@bnoordhuis

bnoordhuis · 2015-05-22T21:41:08Z

lib/net.js

-        (this._handle instanceof Pipe) &&
-        process.platform === 'win32') {
-      // Make stdout and stderr blocking on Windows
+        (this._handle instanceof Pipe) {


This doesn't look like valid JS to me (mismatched parentheses.)

Doh, forgot to run the tests.

bnoordhuis · 2015-05-22T21:41:56Z

Maybe with two child processes?

Yes, that should work.

Fishrock123 · 2015-05-22T21:45:57Z

Oh, I could just pipe the stdout from one to the sdin of the other via .pipe, I think. Also doh.

Fishrock123 · 2015-05-25T16:39:02Z

@bnoordhuis updated ptal

bnoordhuis · 2015-05-25T17:32:25Z

test/fixtures/stdout-producer.js

+const add = 'test\n';
+
+var str = '';
+for (var i = 0; i < 100000; i++) {


Suggestion: use 1e5 here. Personally, I find scientific notation easier to read for large numbers.

Fishrock123 · 2015-05-25T18:23:23Z

.eslintrc

@@ -9,6 +9,8 @@ ecmaFeatures:
  binaryLiterals: true
  generators: true
  forOf: true
+  objectLiteralShorthandMethods: true
+  objectLiteralShorthandProperties: true


used here: https://github.com/nodejs/io.js/pull/1771/files#diff-7718507eb7a71106a68401cae76c5268R14

I don't see a good reason not to enable these other than backporting, maybe.

+1, we had to enable one of these in #1760 already.

Yeah, I only needed one here, but might as well enable both.

bnoordhuis · 2015-05-25T21:01:03Z

LGTM if the CI is happy.

Fishrock123 · 2015-05-25T21:06:11Z

Good call. https://jenkins-iojs.nodesource.com/view/iojs/job/iojs+any-pr+multi/711/

Fishrock123 · 2015-05-26T03:00:34Z

Smartos seems unrelated, will land tomorrow.

bnoordhuis · 2015-05-26T09:49:41Z

One final suggestion: maybe you can go into a little more detail in the commit log about the ramifications of this change.

Fishrock123 · 2015-06-25T00:16:47Z

@bnoordhuis updated, ptal at code comment(s) + commit message. I think I'm correct in what is happening.

.. Although, I'm a little less sure this is actually a bug rather than expected behaviour when the process exits "early"? Again, it doesn't fail without the text producer exiting it early.

Fishrock123 · 2015-07-01T17:25:16Z

ping @bnoordhuis

Fishrock123 · 2015-07-01T18:43:33Z

See also #1541

bnoordhuis · 2015-07-06T11:39:38Z

lib/net.js

-        (this._handle instanceof Pipe) &&
-        process.platform === 'win32') {
-      // Make stdout and stderr blocking on Windows
+        this._handle instanceof Pipe) {


Fits on a single line now, I think?

bnoordhuis · 2015-07-06T11:46:33Z

LGTM with comments. Can you rebase and run the CI?

Refs: nodejs#1771 Refs: nodejs#6456 Refs: nodejs#6773 Refs: nodejs#6816

OS X has a tiny 1kb hard-coded buffer size for stdout / stderr to TTYs (terminals). Output larger than that causes chunking, which ends up having some (very small but existent) delay past the first chunk. That causes two problems: 1. When output is written to stdout and stderr at similar times, the two can become mixed together (interleaved). This is especially problematic when using control characters, such as \r. With interleaving, chunked output will often have lines or characters erased unintentionally, or in the wrong spots, leading to broken output. CLI apps often extensively use such characters for things such as progress bars. 2. Output can be lost if the process is exited before chunked writes are finished flushing. This usually happens in applications that use `process.exit()`, which isn't infrequent. See nodejs#6980 for more info. This became an issue as result of the Libuv 1.9.0 upgrade. A fix to an unrelated issue broke a hack previously required for the OS X implementation. This resulted in an unexpected behavior change in node. The 1.9.0 upgrade was done in c3cec1e, which was included in v6.0.0. Full details of the Libuv issue that induced this are at nodejs#6456 (comment) Refs: nodejs#1771 Refs: nodejs#6456 Refs: nodejs#6773 Refs: nodejs#6816 PR-URL: nodejs#6895 Reviewed-By: Rod Vagg <rod@vagg.org> Reviewed-By: Anna Henningsen <anna@addaleax.net>

OS X has a tiny 1kb hard-coded buffer size for stdout / stderr to TTYs (terminals). Output larger than that causes chunking, which ends up having some (very small but existent) delay past the first chunk. That causes two problems: 1. When output is written to stdout and stderr at similar times, the two can become mixed together (interleaved). This is especially problematic when using control characters, such as \r. With interleaving, chunked output will often have lines or characters erased unintentionally, or in the wrong spots, leading to broken output. CLI apps often extensively use such characters for things such as progress bars. 2. Output can be lost if the process is exited before chunked writes are finished flushing. This usually happens in applications that use `process.exit()`, which isn't infrequent. See #6980 for more info. This became an issue as result of the Libuv 1.9.0 upgrade. A fix to an unrelated issue broke a hack previously required for the OS X implementation. This resulted in an unexpected behavior change in node. The 1.9.0 upgrade was done in c3cec1e, which was included in v6.0.0. Full details of the Libuv issue that induced this are at #6456 (comment) Refs: #1771 Refs: #6456 Refs: #6773 Refs: #6816 PR-URL: #6895 Reviewed-By: Rod Vagg <rod@vagg.org> Reviewed-By: Anna Henningsen <anna@addaleax.net>

OS X has a tiny 1kb hard-coded buffer size for stdout / stderr to TTYs (terminals). Output larger than that causes chunking, which ends up having some (very small but existent) delay past the first chunk. That causes two problems: 1. When output is written to stdout and stderr at similar times, the two can become mixed together (interleaved). This is especially problematic when using control characters, such as \r. With interleaving, chunked output will often have lines or characters erased unintentionally, or in the wrong spots, leading to broken output. CLI apps often extensively use such characters for things such as progress bars. 2. Output can be lost if the process is exited before chunked writes are finished flushing. This usually happens in applications that use `process.exit()`, which isn't infrequent. See nodejs#6980 for more info. This became an issue as result of the Libuv 1.9.0 upgrade. A fix to an unrelated issue broke a hack previously required for the OS X implementation. This resulted in an unexpected behavior change in node. The 1.9.0 upgrade was done in c3cec1e, which was included in v6.0.0. Full details of the Libuv issue that induced this are at nodejs#6456 (comment) Refs: nodejs#1771 Refs: nodejs#6456 Refs: nodejs#6773 Refs: nodejs#6816 PR-URL: nodejs#6895 Reviewed-By: Rod Vagg <rod@vagg.org> Reviewed-By: Anna Henningsen <anna@addaleax.net>

OS X has a tiny 1kb hard-coded buffer size for stdout / stderr to TTYs (terminals). Output larger than that causes chunking, which ends up having some (very small but existent) delay past the first chunk. That causes two problems: 1. When output is written to stdout and stderr at similar times, the two can become mixed together (interleaved). This is especially problematic when using control characters, such as \r. With interleaving, chunked output will often have lines or characters erased unintentionally, or in the wrong spots, leading to broken output. CLI apps often extensively use such characters for things such as progress bars. 2. Output can be lost if the process is exited before chunked writes are finished flushing. This usually happens in applications that use `process.exit()`, which isn't infrequent. See #6980 for more info. This became an issue as result of the Libuv 1.9.0 upgrade. A fix to an unrelated issue broke a hack previously required for the OS X implementation. This resulted in an unexpected behavior change in node. The 1.9.0 upgrade was done in c3cec1e, which was included in v6.0.0. Full details of the Libuv issue that induced this are at #6456 (comment) Refs: #1771 Refs: #6456 Refs: #6773 Refs: #6816 PR-URL: #6895 Reviewed-By: Rod Vagg <rod@vagg.org> Reviewed-By: Anna Henningsen <anna@addaleax.net>

Refs: nodejs#1771 Refs: nodejs#6456 Refs: nodejs#6773 Refs: nodejs#7743 PR-URL: nodejs#6816

Refs: #1771 Refs: #6456 Refs: #6773 Refs: #7743 PR-URL: #6816 Reviewed-By: Fedor Indutny <fedor.indutny@gmail.com> Reviewed-By: Saúl Ibarra Corretgé <saghul@gmail.com> Reviewed-By: Anna Henningsen <anna@addaleax.net> Reviewed-By: Rod Vagg <rod@vagg.org> Reviewed-By: Michaël Zasso <mic.besace@gmail.com> Reviewed-By: James M Snell <jasnell@gmail.com>

Fishrock123 added the net Issues and PRs related to the net subsystem. label May 22, 2015

bnoordhuis reviewed May 22, 2015
View reviewed changes

Fishrock123 force-pushed the fix-chunked-output branch from e62bc62 to 1602b9b Compare May 25, 2015 16:38

bnoordhuis reviewed May 25, 2015
View reviewed changes

Fishrock123 force-pushed the fix-chunked-output branch from 1602b9b to e56d03b Compare May 25, 2015 18:22

Fishrock123 reviewed May 25, 2015
View reviewed changes

Fishrock123 force-pushed the fix-chunked-output branch from e56d03b to a98eb8a Compare May 25, 2015 18:49

Fishrock123 mentioned this pull request May 26, 2015

Why does node/io.js hang when printing to the console inside a loop? #1741

Closed

bnoordhuis force-pushed the master branch from 311c3e4 to b926718 Compare June 2, 2015 17:25

Fishrock123 force-pushed the fix-chunked-output branch from a98eb8a to e646cbc Compare June 25, 2015 00:14

Fishrock123 mentioned this pull request Jun 25, 2015

Chunked stdout/stderr drops writes if terminated early. #784

Closed

rvagg force-pushed the master branch from 9d21135 to 628a3ab Compare June 25, 2015 09:18

jshkurti mentioned this pull request Jun 25, 2015

Logging before process.exit does not work Unitech/pm2#1149

Closed

Fishrock123 force-pushed the fix-chunked-output branch from e646cbc to 6f05c6a Compare July 1, 2015 17:24

bnoordhuis reviewed Jul 6, 2015
View reviewed changes

Fishrock123 mentioned this pull request May 20, 2016

tty: use blocking mode on OS X #6895

Merged

4 tasks

Fishrock123 added a commit to Fishrock123/node that referenced this pull request May 26, 2016

tty: use blocking mode on OS X

f4dcdfa

Refs: nodejs#1771 Refs: nodejs#6456 Refs: nodejs#6773 Refs: nodejs#6816

Fishrock123 added a commit to Fishrock123/node that referenced this pull request Jul 18, 2016

tty: set the handle to blocking mode

09cc780

Refs: nodejs#1771 Refs: nodejs#6456 Refs: nodejs#6773 Refs: nodejs#7743 PR-URL: nodejs#6816

ScottFreeCode mentioned this pull request Nov 3, 2016

Avoid output truncation in node 5.12.1-6.2.0 mochajs/mocha#2566

Closed

Fishrock123 mentioned this pull request May 11, 2017

process: always make stdio blocking #12970

Closed

4 tasks

CMCDragonkai mentioned this pull request Aug 1, 2022

Standardise Asynchronous Logging and Message Encoding on Windows MatrixAI/js-logger#22

Open

ForsakenHarmony mentioned this pull request Jun 20, 2023

failed printing to stdout: Resource temporarily unavailable (os error 35) on MacOS napi-rs/napi-rs#1630

Open

github-actions bot mentioned this pull request Apr 24, 2024

Adding VersionStream for nodejs-22 wolfi-dev/os#17646

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

net: make stdout & stderr block on all platforms #1771

net: make stdout & stderr block on all platforms #1771

Fishrock123 commented May 22, 2015

bnoordhuis May 22, 2015

Fishrock123 May 22, 2015

bnoordhuis commented May 22, 2015

Fishrock123 commented May 22, 2015

Fishrock123 commented May 25, 2015

bnoordhuis May 25, 2015

Fishrock123 May 25, 2015

silverwind May 25, 2015

Fishrock123 May 25, 2015

bnoordhuis commented May 25, 2015

Fishrock123 commented May 25, 2015

Fishrock123 commented May 26, 2015

bnoordhuis commented May 26, 2015

Fishrock123 commented Jun 25, 2015

Fishrock123 commented Jul 1, 2015

Fishrock123 commented Jul 1, 2015

bnoordhuis Jul 6, 2015

bnoordhuis commented Jul 6, 2015

net: make stdout & stderr block on all platforms #1771

net: make stdout & stderr block on all platforms #1771

Conversation

Fishrock123 commented May 22, 2015

bnoordhuis May 22, 2015

Choose a reason for hiding this comment

Fishrock123 May 22, 2015

Choose a reason for hiding this comment

bnoordhuis commented May 22, 2015

Fishrock123 commented May 22, 2015

Fishrock123 commented May 25, 2015

bnoordhuis May 25, 2015

Choose a reason for hiding this comment

Fishrock123 May 25, 2015

Choose a reason for hiding this comment

silverwind May 25, 2015

Choose a reason for hiding this comment

Fishrock123 May 25, 2015

Choose a reason for hiding this comment

bnoordhuis commented May 25, 2015

Fishrock123 commented May 25, 2015

Fishrock123 commented May 26, 2015

bnoordhuis commented May 26, 2015

Fishrock123 commented Jun 25, 2015

Fishrock123 commented Jul 1, 2015

Fishrock123 commented Jul 1, 2015

bnoordhuis Jul 6, 2015

Choose a reason for hiding this comment

bnoordhuis commented Jul 6, 2015