mocha does not stop on failure in beforeEach anymore

[c8y3]

Prerequisites

• Checked that your issue hasn't already been filed by cross-referencing issues with the faq label
• Checked next-gen ES issues and syntax problems by using the same environment and/or transpiler configuration without Mocha to ensure it isn't just a feature that actually isn't supported in the environment in question or a bug in your code.
• 'Smoke tested' the code to be tested by running it outside the real test suite to get a better sense of whether the problem is in the code under test, your usage of Mocha, or Mocha itself
• Ensured that there is no discrepancy between the locally and globally installed versions of Mocha. You can find them with: node node_modules/.bin/mocha --version(Local) and mocha --version(Global). We recommend that you not install Mocha globally.

Description

I am using karma with mocha and the firefox launcher.
Before mocha version 7.1.0, when there was an exception during the execution of beforeEach, the test would fail correctly.
But since version 7.1.0, the test execution blocks until disconnected from firefox after the absence of any message in 30000 ms :

Firefox 75.0 (Ubuntu 0.0.0) ERROR
  Disconnected, because no message in 30000 ms.
Firefox 75.0 (Ubuntu 0.0.0): Executed 0 of 1 DISCONNECTED (30.017 secs / 0 secs)
npm ERR! Test failed.  See above for more details.

Steps to Reproduce

Have a directory with the 3 following files :

• test.js
• package.json
• karma.conf.js

Here is the content of each file.
test.js

describe('Test which should fail nicely during beforeEach', function() {
    beforeEach(function() {
        throw 'an exception on purpose';
    });

    it('should fail nicely before even reaching this code', function() {
        console.log('hello');
    });
});

package.json

{
  "name": "karma-mocha-bug",
  "version": "1.0.0",
  "description": "",
  "scripts": {
    "test": "karma start"
  },
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "karma": "^5.0.1",
    "karma-firefox-launcher": "^1.3.0",
    "karma-mocha": "^1.3.0",
    "mocha": "^7.1.1"
  }
}

karma.conf.js

module.exports = function(config) {
  config.set({

    // base path that will be used to resolve all patterns (eg. files, exclude)
    basePath: '',


    // frameworks to use
    // available frameworks: https://npmjs.org/browse/keyword/karma-adapter
    frameworks: ['mocha'],


    // list of files / patterns to load in the browser
    files: [
      'test.js'
    ],


    // list of files / patterns to exclude
    exclude: [
    ],


    // preprocess matching files before serving them to the browser
    // available preprocessors: https://npmjs.org/browse/keyword/karma-preprocessor
    preprocessors: {
    },


    // test results reporter to use
    // possible values: 'dots', 'progress'
    // available reporters: https://npmjs.org/browse/keyword/karma-reporter
    reporters: ['progress'],


    // web server port
    port: 9876,


    // enable / disable colors in the output (reporters and logs)
    colors: true,


    // level of logging
    // possible values: config.LOG_DISABLE || config.LOG_ERROR || config.LOG_WARN || config.LOG_INFO || config.LOG_DEBUG
    logLevel: config.LOG_INFO,


    // enable / disable watching file and executing tests whenever any file changes
    autoWatch: false,


    // start these browsers
    // available browser launchers: https://npmjs.org/browse/keyword/karma-launcher
    browsers: ['Firefox'],


    // Continuous Integration mode
    // if true, Karma captures browsers, runs the tests and exits
    singleRun: true,

    // Concurrency level
    // how many browser should be started simultaneous
    concurrency: Infinity
  })
}

Then run the tests:

npm test

Expected behavior: the test should run quickly (without blocking), print that the beforeEach hook failed and then stop. The correct output (which works with version 7.0.1 of mocha), should look like this:

> karma-mocha-bug@1.0.0 test /home/ubuntu/tmp/karma-mocha-bug
> karma start

14 04 2020 09:03:55.706:INFO [karma-server]: Karma v5.0.1 server started at http://0.0.0.0:9876/
14 04 2020 09:03:55.708:INFO [launcher]: Launching browsers Firefox with concurrency unlimited
14 04 2020 09:03:55.712:INFO [launcher]: Starting browser Firefox
14 04 2020 09:04:00.190:INFO [Firefox 75.0 (Ubuntu 0.0.0)]: Connected on socket NjDuW0jyTxYRa2REAAAA with id 96024665
Firefox 75.0 (Ubuntu 0.0.0) Test which should fail nicely during beforeEach "before each" hook for "should fail nicely before even reaching this code" FAILED
	the string "an exception on purpose" was thrown, throw an Error :)
Firefox 75.0 (Ubuntu 0.0.0): Executed 1 of 1 (1 FAILED) (0.097 secs / 0 secs)
TOTAL: 1 FAILED, 0 SUCCESS
npm ERR! Test failed.  See above for more details.

Actual behavior: test execution blocks for 30000 ms, until the timeout is reached and disconnection from firefox happens. Here is the full trace :

> karma-mocha-bug@1.0.0 test /home/ubuntu/tmp/karma-mocha-bug
> karma start

14 04 2020 08:59:46.194:INFO [karma-server]: Karma v5.0.1 server started at http://0.0.0.0:9876/
14 04 2020 08:59:46.196:INFO [launcher]: Launching browsers Firefox with concurrency unlimited
14 04 2020 08:59:46.202:INFO [launcher]: Starting browser Firefox
14 04 2020 08:59:51.129:INFO [Firefox 75.0 (Ubuntu 0.0.0)]: Connected on socket a4NTmvFcARbO-805AAAA with id 16623341
Firefox 75.0 (Ubuntu 0.0.0): Executed 0 of 1 SUCCESS (0 secs / 0 secs)
Firefox 75.0 (Ubuntu 0.0.0) ERROR
  Disconnected, because no message in 30000 ms.
Firefox 75.0 (Ubuntu 0.0.0): Executed 0 of 1 DISCONNECTED (30.003 secs / 0 secs)
npm ERR! Test failed.  See above for more details.

Reproduces how often: 100%

Versions

For the versions, check the package.json copy-pasted in the previous section.
This regression happens in mocha 7.1.0 and 7.1.1.
There is no bug with mocha 7.0.1.

[juergba]

Running your test sample in Chrome (without Karma), everything works as it should.
Please try and set Mocha's allow-uncaught option to true.

[juergba]

duplicate #4250