Make coverage report optional in case it fails to submit

[niccokunzmann]

Sometimes, we get this error:
coveralls.exception.CoverallsException: Could not submit coverage: 422 Client Error: Unprocessable Entity for url: https://coveralls.io/api/v1/jobs

This PR allows the error to happen while also maintaining the test result and making sure that we actually installed all dependencies.

This is a follow up from
#644
main branch failed:
https://github.com/collective/icalendar/actions/runs/9663149658/job/26654629821

[stevepiercy]

I don't know whether this change had the intended effect. I don't see any Coveralls comments or email notifications in this pull request as I do with other pull requests.

[niccokunzmann]

Hm.
Step 1: the parallel build is uploaded:
https://github.com/collective/icalendar/actions/runs/9663354421/job/26655307801#step:6:18

Run coveralls --service=github-actions || which coveralls
Submitting coverage to coveralls.io...
Coverage submitted!
Coverage for parallel build uploaded
https://coveralls.io/builds/68286429

Step 2:

  coveralls --service=github-actions --finish || which coveralls
  shell: sh -e {0}
  env:
    GITHUB_TOKEN: ***
    COVERALLS_REPO_TOKEN: 
Finishing parallel jobs...
Done

Step 2 output in main branch:
https://github.com/collective/icalendar/actions/runs/9664432038/job/26659037653#step:6:16

Submitting coverage to coveralls.io...

Hm... It says "Finishing parallel jobs..."
That should be it though.
So, I do not know what this means exactly: It is submitted but it does not turn up.

If you look at the main branch:

https://github.com/collective/icalendar/commits/main/

We have several broken commits.

The broken commits are on the main branch but the Pull Requests actually are green.
Even Merge branch 'main' into remove-pytz is green - merging main into the other branch.
So, the failure is in the main branch only?

Another thing I just saw:
https://github.com/collective/icalendar/actions/runs/9662268189/job/26651793581#step:6:19

Received 422 submitting job via Github Actions. By default, coveralls-python uses the "github" service name, which requires you to set the $GITHUB_TOKEN environment variable. If you want to use a COVERALLS_REPO_TOKEN instead, please manually override $COVERALLS_SERVICE_NAME to "github-actions". For more info, see https://coveralls-python.readthedocs.io/en/latest/usage/configuration.html#github-actions-support

And this:
https://github.com/collective/icalendar/actions/runs/9663354421/job/26655308106#step:6:11

     GITHUB_TOKEN: ***
    COVERALLS_REPO_TOKEN: 

Makes me think that the COVERALLS_REPO_TOKEN is actually not set and ${{ secrets.CODECOV_TOKEN }} is empty.

So, maybe, I should switch it back to --service=github I will do this and hope that a report is submitted. If that is so, I would say we could merge it. If that still breaks main, I would make anonther PR and see if I can just skip coverage on main.

[niccokunzmann]

Finishing parallel jobs...
Done

This is the report:
https://coveralls.io/builds/68314915
This is the last report:
https://coveralls.io/builds/68286429

Reporting works:

I do not know why there is not a comment in this PR.

[stevepiercy]

Perhaps with the rename of master to main, Coveralls got lost in the woods? I logged in to the Coveralls admin settings, and clicked the Rescyn button.

Maybe try again?

You could also compare build logs between the last known good one and the current one.

[stevepiercy]

I found this doc: https://docs.coveralls.io/common-issues-and-troubleshooting#issue-not-getting-pr-comments

[stevepiercy]

I see this in the GitHub collective organization audit log: https://github.com/organizations/collective/settings/audit-log?q=actor%3A+coveralls

How was Coveralls set up initially? There are at least a couple of ways, and there are both the repo and organization scopes.

We might need to remove it and set it up again, as a last resort.

[niccokunzmann]

I think, when the main branch fails, no coverage is finalized. In that case, we would compare to a broken commit and thus coveralls cannot create a comparison.

This can happen when Coveralls can’t find the base build for your PR build.

So, since main is failing, we cannot expect a report.

I would like to change it so that the coverage is submitted even if one test fails: c452128

I restarted the main build after you did your refresh. Coveralls seems to see main as the base branch now.
I still get an error: https://github.com/collective/icalendar/actions/runs/9664432038/job/26809777846#step:6:36

[niccokunzmann]

This should not break anything and just in case it does: Let's merge this with a squash commit so we can easily revert this. I would like to get main green again.

[stevepiercy]

@niccokunzmann I'm not 100% sure, but I think that PRs that come from forks of this repo are not affected by this, but when they come from a branch in this repo, they are always affected. See https://github.com/collective/icalendar/actions/runs/9737293945/job/26869188526?pr=674.

Anyway, I don't want to do the honors of merging. I think it best that you finish what you started.

[niccokunzmann]

I will merge it and see what happens... It does not break anything and we can revert it.. Squash merge...