Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RangeError: Invalid string length #1919

Closed
kubijo opened this issue Oct 11, 2020 · 25 comments · Fixed by #1920
Closed

RangeError: Invalid string length #1919

kubijo opened this issue Oct 11, 2020 · 25 comments · Fixed by #1920
Labels

Comments

@kubijo
Copy link

kubijo commented Oct 11, 2020

Describe the bug

I've started the migration to webpack@5 and from what
I can summise it seems that the compilation is able to
pretty much finish, but than it fails with following error.

By what the linked sources are about I've tried
to set stats: false, however nothing changed.

What is the current behavior?

[webpack-cli] Promise rejection: RangeError: Invalid string length
[webpack-cli] RangeError: Invalid string length
    at JSON.stringify (<anonymous>)
    at Compiler.compilerCallback (.yarn/$$virtual/webpack-cli-virtual-6b0393a6f7/0/cache/webpack-cli-npm-4.0.0-ccb32dab5d-6c693d4cc9.zip/node_modules/webpack-cli/lib/utils/Compiler.js:91:36)
    at .yarn/$$virtual/webpack-cli-virtual-6b0393a6f7/0/cache/webpack-cli-npm-4.0.0-ccb32dab5d-6c693d4cc9.zip/node_modules/webpack-cli/lib/utils/Compiler.js:110:46
    at .yarn/cache/neo-async-npm-2.6.2-75d6902586-34a8f53091.zip/node_modules/neo-async/async.js:2830:7
    at done (.yarn/cache/neo-async-npm-2.6.2-75d6902586-34a8f53091.zip/node_modules/neo-async/async.js:2865:11)
    at .yarn/cache/neo-async-npm-2.6.2-75d6902586-34a8f53091.zip/node_modules/neo-async/async.js:2818:7
    at .yarn/$$virtual/webpack-virtual-de8dd232be/0/cache/webpack-npm-5.0.0-72465b5b59-bbd9df6389.zip/node_modules/webpack/lib/HookWebpackError.js:69:3
    at Hook.eval [as callAsync] (eval at create (.yarn/cache/tapable-npm-2.0.0-1b1ecde87d-e6321280f9.zip/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:9:1)
    at Hook.CALL_ASYNC_DELEGATE [as _callAsync] (.yarn/cache/tapable-npm-2.0.0-1b1ecde87d-e6321280f9.zip/node_modules/tapable/lib/Hook.js:18:14)
    at Cache.shutdown (.yarn/$$virtual/webpack-virtual-de8dd232be/0/cache/webpack-npm-5.0.0-72465b5b59-bbd9df6389.zip/node_modules/webpack/lib/Cache.js:150:23)

To Reproduce

Hard to say really, but I'll gladly collect and provide any further information when given pointers.

Please paste the results of webpack-cli info here, and mention other relevant information

  System:
    OS: Linux 5.4 Ubuntu 20.04.1 LTS (Focal Fossa)
    CPU: (12) x64 Intel(R) Core(TM) i7-8700K CPU @ 3.70GHz
    Memory: 7.94 GB / 31.29 GB
  Binaries:
    Node: 13.12.0 - /tmp/xfs-eeeb95f6/node
    Yarn: 2.1.1 - /tmp/xfs-eeeb95f6/yarn
    npm: 6.14.4 - ~/.virtualenvs/pool/bin/npm
  Browsers:
    Chrome: 84.0.4147.125
    Firefox: 81.0
@kubijo kubijo added the Bug label Oct 11, 2020
@alexander-akait
Copy link
Member

Without reproducible steps hard to fix it

@kubijo
Copy link
Author

kubijo commented Oct 11, 2020

Without reproducible steps hard to fix it

I'm well aware that this not a good bug report and I'm ready to collect as much traces as needed, but I need something to get going.

These are the last two trace locations:

/**
 * @param {CallbackCache<void>} callback signals when the call finishes
 * @returns {void}
 */
endIdle(callback) {
	this.hooks.endIdle.callAsync(
		makeWebpackErrorCallback(callback, "Cache.hooks.endIdle")
	);
}
    compilerCallback(err, stats, lastHash, options, outputOptions) {
        const statsErrors = [];

        if (!outputOptions.watch || err) {
            // Do not keep cache anymore
            this.compiler.purgeInputFileSystem();
        }
        if (err) {
            lastHash = null;
            logger.error(err.stack || err);
            process.exit(1);
        }
        if (!outputOptions.watch && stats.hasErrors()) {
            process.exitCode = 1;
        }
        if (outputOptions.json === true) {
            process.stdout.write(JSON.stringify(stats.toJson(outputOptions), null, 2) + '\n');
        } else if (stats.hash !== lastHash) {
            lastHash = stats.hash;
            if (stats.compilation && stats.compilation.errors.length !== 0) {
                const errors = stats.compilation.errors;
                errors.forEach((statErr) => {
                    const errLoc = statErr.module ? statErr.module.resource : null;
                    statsErrors.push({ name: statErr.message, loc: errLoc });
                });
            }

            //
            // THIS IS THE EXCEPTION LOCATION
            //
            const JSONStats = JSON.stringify(stats.toJson(outputOptions), null, 2);
            if (typeof outputOptions.json === 'string') {
                try {
                    writeFileSync(outputOptions.json, JSONStats);
                    logger.success(`stats are successfully stored as json to ${outputOptions.json}`);
                } catch (err) {
                    logger.error(err);
                }
            }
            return this.generateOutput(outputOptions, stats, statsErrors);
        }
    }

@alexander-akait
Copy link
Member

Can you try to use it without PnP?

@kubijo
Copy link
Author

kubijo commented Oct 11, 2020

Can you try to use it without PnP?

I'm trying to… it doesn't seem to fail on the reported error, but fails differently (some errors are related to my migration to new dependencies versions, some by disabling PnP), so bare with me on this....

@alexander-akait
Copy link
Member

I think it is bug on our side, so I want to get better error reporting to search the problem

@kubijo
Copy link
Author

kubijo commented Oct 11, 2020

I think it is bug on our side, so I want to get better error reporting to search the problem

I think so as well, but the project is pretty big, everything takes a long time & I want to first remove our bugs … so yeah, give me some time and I'll get there ;-)

@kubijo
Copy link
Author

kubijo commented Oct 11, 2020

The compilation is VERY slow now ... but that may be related to the fact, that now I'm running the production build instead of development watch one (you already have a bug in that reported elsewhere)

@kubijo
Copy link
Author

kubijo commented Oct 11, 2020

Ok… so it seems the only trailing issues I have now with disabled PnP are

  • the CLI output is not colored anymore
  • Module build failed (from ./node_modules/mini-css-extract-plugin/dist/loader.js): Error: Didn't get a result from child compiler

Neither of which seem to really be related to this… so I guess that what's to do now is to wait for your linked PR to land & build again with PnP enabled right?

@alexander-akait
Copy link
Member

the CLI output is not colored anymore

Already in tracker

Module build failed (from ./node_modules/mini-css-extract-plugin/dist/loader.js): Error: Didn't get a result from child compiler

Do you use latest mini-css-extract-plugin version?

@kubijo
Copy link
Author

kubijo commented Oct 11, 2020

$ yarn why mini-css-extract-plugin
└─ root-workspace-0b6124@workspace:.
   └─ mini-css-extract-plugin@npm:1.0.0 [dc3fc] (via npm:^1.0.0 [dc3fc])

@alexander-akait
Copy link
Member

alexander-akait commented Oct 11, 2020

@kubijo Maybe you can create reproducible test repo? Error: Didn't get a result from child compiler is not related to webpack-cli, I think something wrong in mini-css-extract-plugin options

@alexander-akait
Copy link
Member

I think it is fixed in master, but I want more information about mini-css-extract-plugin

@kubijo
Copy link
Author

kubijo commented Oct 11, 2020 via email

@kubijo
Copy link
Author

kubijo commented Oct 12, 2020

So…

  • the good one is that I've somehow managed to make a very minimal setup
  • the bad one is that the error is not there 🤦

On with the archeology then…

@kubijo
Copy link
Author

kubijo commented Oct 12, 2020

So I've tried a watch mode to se if there will be a difference and… not sure what is the correct place, so I'll drop it here

Compilation A starting…
Compilation B starting…
Compilation C starting…
Compilation D starting…

(node:38481) [DEP_WEBPACK_WATCH_WITHOUT_CALLBACK] DeprecationWarning: A 'callback' argument need to be provided to the 'webpack(options, callback)' function when the 'watch' option is set. There is no way to handle the 'watch' option without a callback.
    at webpack (node_modules/webpack/lib/webpack.js:145:5)
    at f (node_modules/webpack/lib/index.js:35:15)
    at Compiler.createCompiler (node_modules/webpack-cli/lib/utils/Compiler.js:129:35)
    at WebpackCLI.run (node_modules/webpack-cli/lib/webpack-cli.js:220:32)
    at async runCLI (node_modules/webpack-cli/lib/bootstrap.js:74:24)

Compilation  finished
Compilation  finished

Compilation C starting…
Compilation D starting…

Compilation  finished
Compilation B starting…

Compilation  finished

<--- Last few GCs --->

[38481:0x4bf7290]   305780 ms: Scavenge 3297.3 (4115.0) -> 3295.4 (4130.3) MB, 20.2 / 0.0 ms  (average mu = 0.064, current mu = 0.019) allocation failure 
[38481:0x4bf7290]   306437 ms: Mark-sweep 3310.8 (4130.3) -> 3310.7 (4119.3) MB, 643.2 / 0.0 ms  (average mu = 0.058, current mu = 0.051) allocation failure scavenge might not succeed


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x3040d5c40921 <JSObject>
    0: builtin exit frame: stringify(this=0x3040d5c5efd9 <Object map = 0x11d1f7443641>,2,0x395ddb8c01c1 <null>,0x0053fa3c1341 <Object map = 0x28a43f01f8e1>,0x3040d5c5efd9 <Object map = 0x11d1f7443641>)

    1: compilerCallback [0x19b34e9f2b9] [node_modules/webpack-cli/lib/utils/Compiler.js:91] [bytecode=0x35cd1ccba161 offset=314](this=0x10859d6197d9 <Compiler map = 0...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0xa0c020 node::Abort() [webpack]
 2: 0xa0c42d node::OnFatalError(char const*, char const*) [webpack]
 3: 0xb8130e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [webpack]
 4: 0xb81689 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [webpack]
 5: 0xd2fb95  [webpack]
 6: 0xd30226 v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector) [webpack]
 7: 0xd3ea99 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [webpack]
 8: 0xd3f8d5 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [webpack]
 9: 0xd423ac v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [webpack]
10: 0xd07b5d  [webpack]
11: 0xd19abd v8::internal::Factory::NewRawTwoByteString(int, v8::internal::AllocationType) [webpack]
12: 0x10d501b v8::internal::IncrementalStringBuilder::Extend() [webpack]
13: 0xe2753b v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [webpack]
14: 0xe2af21 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<false>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [webpack]
15: 0xe28970 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [webpack]
16: 0xe28da5 v8::internal::JsonStringifier::SerializeJSReceiverSlow(v8::internal::Handle<v8::internal::JSReceiver>) [webpack]
17: 0xe2a4f2 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<false>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [webpack]
18: 0xe2b503 v8::internal::JsonStringifier::SerializeArrayLikeSlow(v8::internal::Handle<v8::internal::JSReceiver>, unsigned int, unsigned int) [webpack]
19: 0xe27a77 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [webpack]
20: 0xe2af21 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<false>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [webpack]
21: 0xe28970 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<true>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [webpack]
22: 0xe2af21 v8::internal::JsonStringifier::Result v8::internal::JsonStringifier::Serialize_<false>(v8::internal::Handle<v8::internal::Object>, bool, v8::internal::Handle<v8::internal::Object>) [webpack]
23: 0xe2b8d7 v8::internal::JsonStringify(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>) [webpack]
24: 0xc349ab v8::internal::Builtin_JsonStringify(int, unsigned long*, v8::internal::Isolate*) [webpack]
25: 0x13d8c59  [webpack]

not an improvement to say the least (I know that the watch error is aleady being fixed, but I've included it for completness)

@kubijo
Copy link
Author

kubijo commented Oct 12, 2020

also one nitpick… it seems that the Compilation finished should have a name as well, but there are just two spaces

@alexander-akait
Copy link
Member

DeprecationWarning: A 'callback' argument need to be provided to the 'webpack(options, callback)' function when the 'watch' option is set. There is no way to handle the 'watch' option without a callback.

Already reported, WIP

@kubijo
Copy link
Author

kubijo commented Oct 12, 2020

@evilebottnawi … it seems to me that the problem with css must be either in sass-loader (last one)…

{
    loader: 'sass-loader',
    options: {
        // eslint-disable-next-line global-require
        implementation: require('sass'),
        // webpackImporter: typeof conf.sassWebpackImporter === 'boolean' ? conf.sassWebpackImporter : true,
        sassOptions: {
            // eslint-disable-next-line global-require
            fiber: require('fibers'),
            sourceMap: !isProduction,
        },
    },
}

…or css-loader (one before sass-loader; normally I have a postcss-loader between them, but when I comment it out, the error persists)

{
    loader: 'css-loader',
    options: {
        url: false, // URL calls are already handled by PostCSS
        import: true,
        modules: !withModules
            ? false
            : {
                  localIdentContext: paths.styles,
                  // - In production, short (yet still unique) CSS identifiers are the goal.
                  // - In Developmnet we want the classes to be be readable.
                  localIdentName:
                      isProduction && !inline ? '[md5:hash:base64:8]' : '[name]_[local]_[md5:hash:base64:3]',
              },
        sourceMap: !isProduction,

        // Required because we have postcss-loader pre-processing
        importLoaders: 1,
    },
}

Would you be able to assist me with this somehow?

@alexander-akait
Copy link
Member

@kubijo sorry, no, without full reproducible test repo I can't help

@mizchi
Copy link

mizchi commented Oct 12, 2020

I have same problem with webpack 5.

[webpack-cli] Uncaught exception: RangeError: Invalid string length
[webpack-cli] RangeError: Invalid string length
    at JSON.stringify (<anonymous>:null:null)
    at Compiler.compilerCallback (<my repo root>/node_modules/webpack-cli/lib/utils/Compiler.js:91:36)

that code is here.

// lib/utils/Compiler.js

            const JSONStats = JSON.stringify(stats.toJson(outputOptions)); // here
            if (typeof outputOptions.json === 'string') {
                try {
                    writeFileSync(outputOptions.json, JSONStats);
                    logger.success(`stats are successfully stored as json to ${outputOptions.json}`);
                } catch (err) {
                    logger.error(err);
                }
            }

Turning it off will work by edit this directly.

I have huge size project (total size of chunks have <100MB on prod). I doubt something wrong on stats.toJson() in this case.

@alexander-akait
Copy link
Member

Fixed in master, hope release will be today, WIP on other issues

@kubijo
Copy link
Author

kubijo commented Oct 12, 2020

@kubijo Maybe you can create reproducible test repo? Error: Didn't get a result from child compiler is not related to webpack-cli, I think something wrong in mini-css-extract-plugin options

I've already burnt two days on this and I'm just not able to create a minimal repro, so I'm droping this for now as need to do something usefull for the company…

Thanks for the other fix though and I'll possibly revisit this at some point…

@francescocaveglia
Copy link

Thanks for the fix @evilebottnawi ! Will be released soon?

@alexander-akait
Copy link
Member

@francescocaveglia yes, today/tomorrow

@alexander-akait
Copy link
Member

I want to close this issue, because original problem was solved, anyway @kubijo Can you provide more information to help your migration?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants