-
Notifications
You must be signed in to change notification settings - Fork 78
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Deployment Validation with coverage-formatters writes to nested coverage/ folder #2816
Comments
Hello @codefriar 👋 It looks like you didn't include the full Salesforce CLI version information in your issue. A few more things to check:
Thank you! |
Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support. |
{
"architecture": "linux-x64",
"cliVersion": "@salesforce/cli/2.35.6",
"nodeVersion": "node-v18.20.1",
"osVersion": "Linux 6.5.0-1017-azure",
"rootPath": "/usr/local/lib/nodejs/lib/node_modules/@salesforce/cli",
"shell": "bash",
"pluginVersions": [
"@oclif/plugin-autocomplete 3.0.[13](https://github.com/department-of-veterans-affairs/va-teams/actions/runs/8635526082/job/23673436239#step:14:14) (core)",
"@oclif/plugin-commands 3.2.2 (core)",
"@oclif/plugin-help 6.0.20 (core)",
"@oclif/plugin-not-found 3.1.1 (core)",
"@oclif/plugin-plugins 5.0.1 (core)",
"@oclif/plugin-search 1.0.20 (core)",
"@oclif/plugin-update 4.2.2 (core)",
"@oclif/plugin-version 2.0.16 (core)",
"@oclif/plugin-warn-if-update-available 3.0.[15](https://github.com/department-of-veterans-affairs/va-teams/actions/runs/8635526082/job/23673436239#step:14:16) (core)",
"@oclif/plugin-which 3.1.7 (core)",
"@salesforce/cli 2.35.6 (core)",
"apex 3.1.0 (core)",
"auth 3.5.0 (core)",
"data 3.2.2 (core)",
"deploy-retrieve 3.4.0 (core)",
"info 3.1.0 (core)",
"limits 3.2.0 (core)",
"marketplace 1.1.0 (core)",
"org 3.6.0 (core)",
"packaging 2.2.0 (core)",
"schema 3.2.0 (core)",
"settings 2.1.0 (core)",
"sobject 1.2.0 (core)",
"source 3.2.0 (core)",
"telemetry 3.1.[17](https://github.com/department-of-veterans-affairs/va-teams/actions/runs/8635526082/job/23673436239#step:14:18) (core)",
"templates 56.1.0 (core)",
"trust 3.4.0 (core)",
"user 3.4.0 (core)",
"sfdx-git-delta 5.34.0 (user)"
]
} You should change your info-required-bot to accept a docker image tag as well. |
hey @codefriar - nice catch - we'll get this fixed |
@WillieRuemmele - at the risk of being told off here, it'd be nice if i could specify the folder / filename ;) |
@codefriar more than just update on the PR, but this seems to be edging towards a breaking change, or at least breaking an existing bug that everyone's got accustomed to... how about making the message correct, e.g. Code Coverage formats, [cobertura], written to /coverage/coverage |
@WillieRuemmele Apologies. I didn't realize --output-dir was an option. It's not listed here: https://developer.salesforce.com/docs/atlas.en-us.sfdx_cli_reference.meta/sfdx_cli_reference/cli_reference_project_commands_unified.htm#cli_reference_project_deploy_start_unified Nor is it listed in the command output: I understand this is likely a breaking change. is there any other solution than just updating the output of where it's stored? |
sorry, I mislead you, it's the |
ah, then it's my bad for not having caught |
@codefriar - I'm not affiliated with Salesforce, but this is consistent with what I've seen developing a plugin to convert the code coverage output for SonarQube and what I've listed in that plugin's readme (not to advertise it or suggest you use it - https://github.com/mcarvin8/apex-code-coverage-transformer). The I usually append this to my deploy command (my plugin converts the JSON output over the cobertura output but I think the directory logic works the same) - This still creates the coverage file in I've tested I think it would be nice if this extra sub-folder created by the CLI was removed since I've definitely had failed pipelines when passing the coverage file path to my plugin. |
@mcarvin8 is there any ongoing work by Salesforce to support sf run test format (test-result-codecoverage.json) while doing sf project deploy ? Did you create an issue to request that ? |
@mathpaquette - I'm not aware of any ongoing work by Salesforce to have the same coverage JSON formats for both deployments and test run. That would be ideal. But I haven't created an issue to request that, but let me dig first to see if anyone else has submitted a request for that. I did submit a case before I went on travel to have Salesforce fix the deployment command coverage reports since the deployment coverage reports inaccurately report In terms of this original issue with how deployments write to deeply nested coverage folders, this behavior isn't the same with the tests commands. The test command --output-dir dumps them in the desired folder, where the deployment command adds an additional sub-folder.
IMO, it would be nice for both commands to have the same flag names for coverage format and results directory, and use the same format for the coverage reports as much as possible. |
Summary
We use Github actions. One of our steps is to do a trial deployment to validate and do code coverage calculations. Specifically we use this command:
sf project deploy start --verbose --dry-run -x package/package.xml --target-org stage --ignore-conflicts --wait 45 --test-level RunSpecifiedTests --tests ${{ steps.related-tests.outputs.result }} --coverage-formatters json --coverage-formatters cobertura
This command executes normally. Here's the log output from that command:
This results in the cobertura file being written to
coverage/coverage/cobertura.xml' not
coverage/cobertura.xml` as I would expect, given the output line saying "Code Coverage formats, [json, cobertura], written to coverage/"Steps To Reproduce
I'm unable to create a full repo, for security reasons. However, the operant bits of the workflow file are as follows:
Expected result
cobertura.xml should be found in the WorkingDir's coverage/ folder.
Actual result
cobertura.xml (and it's sister Json file) are written to coverage/coverage/ folder.
System Information
Github Actions using the official SF cli docker image:
Additional information
The text was updated successfully, but these errors were encountered: