Internal: Parallel smoke testing

[caendesilva]

We've updated our smoke tests workflow to run tests in parallel, improving efficiency while maintaining compatibility with our existing processes. Here are the key changes:

1. Renamed the workflow to "🔥 Parallel Smoke Tests" to reflect its new parallel nature.
2. Added a step to prepare separate directories for unit and feature tests.
3. Introduced a "Run Parallel Smoke Tests" step that:
   • Defines a function run_tests to execute tests for each suite.
   • Runs unit and feature tests in parallel using background processes.
   • Uses sed to prefix each line of output with either [UNIT] or [FEATURE] for clarity, with proper padding for alignment.
   • Generates separate JUnit XML reports for each test suite.
4. Added a "Merge JUnit XML Reports" step that combines the unit and feature test reports into a single report.xml file, ensuring compatibility with existing ping scripts.
5. Retained the rest of the workflow, using the combined report.xml for subsequent steps.
6. Removed the separate CI smoke test feature as this parallel approach is now fast enough.

This parallel approach significantly speeds up the smoke tests while maintaining compatibility with our existing CI processes. We get interleaved, real-time output from both test suites.

graph TD
  A[Start] --> B[Set up environment]
  B --> C[Checkout code]
  C --> D[Validate composer files]
  D --> E[Cache Composer packages]
  E --> F[Install dependencies]
  F --> G[Prepare test directories]
  G --> H[Run Unit Tests]
  G --> I[Run Feature Tests]
  H --> J[Generate Unit Test Report]
  I --> K[Generate Feature Test Report]
  J --> L[Merge JUnit XML Reports]
  K --> L
  L --> N[Ping statistics server]
  N --> O[End]
  style H fill:#f9f,stroke:#333,stroke-width:2px,color:#333
  style I fill:#f9f,stroke:#333,stroke-width:2px,color:#333
  style J fill:#ccf,stroke:#333,stroke-width:2px,color:#333
  style K fill:#ccf,stroke:#333,stroke-width:2px,color:#333
  style L fill:#fcc,stroke:#333,stroke-width:2px,color:#333

Loading

[caendesilva]

Baseline test execution time 2337a94

• 1m 2s for test execution time
• 1m 22s for test succeeded time

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (305a6fc) to head (766b7e6).
Report is 4 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff             @@
##              master     #1928   +/-   ##
===========================================
  Coverage     100.00%   100.00%           
  Complexity      1766      1766           
===========================================
  Files            182       182           
  Lines           4707      4707           
===========================================
  Hits            4707      4707           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[caendesilva]

Merging coverage adds a lot of complexity, so pivoting to do smoke tests in parallel as that's where the biggest pain point is anyway, and we don't need to assemble so much data.

Baseline execution time (spot test)
26s for test execution time
34s for test succeeded time

New execution time (spot test)
20s for test execution time
31s for test succeeded time