Skip to content

Latest commit

 

History

History
89 lines (65 loc) · 5.05 KB

TESTING.md

File metadata and controls

89 lines (65 loc) · 5.05 KB

Guide to Web3's tests and CI

Web3 is used in Node.js and browser contexts to interact with a wide variety of clients. Its tests try to cover as much of this domain as possible.

If you're looking for a fixture, test pattern or common execution context to validate a change, you should be able find it in the existing test suite. (Caveats include Parity / Quorum clients and MetaMask specific tests.)

Required Tests

These should pass for PRs to merge:

Test type npm command Example Description CI Only
unit test eth.accounts.sign.js For discrete pieces of logic
integration test:e2e:clients e2e.contract.events.js Tests using geth and ganache-cli, (insta-mining and interval mining.) Easy to write and good for modeling complex use-cases
browser test:e2e:browsers The integration tests run in a headless browser using web3.min.js (browserified, vs. ganache-cli)
typescript dtslint -- TS type definitions tests
dependencies depcheck -- Verifies every dependency is listed correctly in the module package
bundle test:e2e:min e2e.minified.js Verifies minified bundle loads in a headless browser without being webpacked / browserified
cdn test:e2e:cdn e2e.cdn.sh Visual inspection check: publishes an (un-webpacked) site that uses web3.min.js at https://web3-staging.netlify.app/
windows -- e2e.windows.sh Verifies Web3 installs on Windows OS / Node 12 and can connect to Infura over wss and https

Optional Tests

CI also has tests that install Web3's state at an arbitrary commit in an external real-world project and run their unit tests with it. This strategy is borrowed from ethereum/solidity which checks latest Solidity against OpenZeppelin and others to keep abreast of how local changes might affect critical projects downstream from them.

Examples include:

These tests are "allowed failures". They're:

  • a pre-publication sanity check that discovers how Web3 performs in the wild
  • useful for catching problems which are difficult to anticipate
  • exposed to failure for reasons outside of Web3's control, ex: when fixes here surface bugs in the target.

Implementation Details

Code coverage

Coverage is measured by aggregating the results of tests run in the unit_and_e2e_clients CI job.

Tests which use an Ethereum client

The npm script test:e2e:clients greps all tests with an [ @E2E ] tag in their mocha test description and runs them against:

  • ganache-cli
  • geth stable (POA, single instance, instamining)
  • geth stable (POA, single instance, mining at 2s intervals)

These tests are grouped in files prefixed by "e2e", ex: test/e2e.method.call.js.

Additionally, there are conventional unit tests postfixed .ganache.js which spin up a ganache server programatically within mocha. This pattern is useful if you want to control client configuration for a specific case, test against multiple independent providers, etc.

"Real world" tests

The tests which install Web3's current state in an external real-world project and run their unit tests accomplish this by publishing the monorepo to an ephemeral private npm registry which is spun up in CI using verdaccio. (Implementation details can be seen in scripts/e2e.npm.publish.sh)

The real world target is then cloned and npm or yarn are used to replace its existing Web3 version with the version published to the the private registry. A simple example can be seen at scripts/e2e.ganache.core.sh.

In practice, complex projects can have many versions of Web3 nested in their dependency tree. It's important to coerce all of them to the virtually published package's version for the test to be valid. This can be done with scripts/js/resolutions.js which modifies the target's package.json to take advantage of Yarn's selective dependency resolutions. An example of its use can be seen at scripts/e2e.mosaic.sh.