Add CodeQL workflow for GitHub code scanning and fix few bugs it detected

[lgtm-com]

Hi dandi/dandi-cli!

This is a one-off automatically generated pull request from LGTM.com 🤖. You might have heard that we’ve integrated LGTM’s underlying CodeQL analysis engine natively into GitHub. The result is GitHub code scanning!

With LGTM fully integrated into code scanning, we are focused on improving CodeQL within the native GitHub code scanning experience. In order to take advantage of current and future improvements to our analysis capabilities, we suggest you enable code scanning on your repository. Please take a look at our blog post for more information.

This pull request enables code scanning by adding an auto-generated codeql.yml workflow file for GitHub Actions to your repository — take a look! We tested it before opening this pull request, so all should be working ✔️. In fact, you might already have seen some alerts appear on this pull request!

Where needed and if possible, we’ve adjusted the configuration to the needs of your particular repository. But of course, you should feel free to tweak it further! Check this page for detailed documentation.

Questions? Check out the FAQ below!

FAQ

Click here to expand the FAQ section

How often will the code scanning analysis run?

By default, code scanning will trigger a scan with the CodeQL engine on the following events:

• On every pull request — to flag up potential security problems for you to investigate before merging a PR.
• On every push to your default branch and other protected branches — this keeps the analysis results on your repository’s Security tab up to date.
• Once a week at a fixed time — to make sure you benefit from the latest updated security analysis even when no code was committed or PRs were opened.

What will this cost?

Nothing! The CodeQL engine will run inside GitHub Actions, making use of your unlimited free compute minutes for public repositories.

What types of problems does CodeQL find?

The CodeQL engine that powers GitHub code scanning is the exact same engine that powers LGTM.com. The exact set of rules has been tweaked slightly, but you should see almost exactly the same types of alerts as you were used to on LGTM.com: we’ve enabled the security-and-quality query suite for you.

How do I upgrade my CodeQL engine?

No need! New versions of the CodeQL analysis are constantly deployed on GitHub.com; your repository will automatically benefit from the most recently released version.

The analysis doesn’t seem to be working

If you get an error in GitHub Actions that indicates that CodeQL wasn’t able to analyze your code, please follow the instructions here to debug the analysis.

How do I disable LGTM.com?

If you have LGTM’s automatic pull request analysis enabled, then you can follow these steps to disable the LGTM pull request analysis. You don’t actually need to remove your repository from LGTM.com; it will automatically be removed in the next few months as part of the deprecation of LGTM.com (more info here).

Which source code hosting platforms does code scanning support?

GitHub code scanning is deeply integrated within GitHub itself. If you’d like to scan source code that is hosted elsewhere, we suggest that you create a mirror of that code on GitHub.

How do I know this PR is legitimate?

This PR is filed by the official LGTM.com GitHub App, in line with the deprecation timeline that was announced on the official GitHub Blog. The proposed GitHub Action workflow uses the official open source GitHub CodeQL Action. If you have any other questions or concerns, please join the discussion here in the official GitHub community!

I have another question / how do I get in touch?

Please join the discussion here to ask further questions and send us suggestions!

[codecov]

Codecov Report

Base: 88.27% // Head: 63.35% // Decreases project coverage by -24.91% ⚠️

Coverage data is based on head (1137dbd) compared to base (0439caf).
Patch coverage: 83.33% of modified lines in pull request are covered.

Additional details and impacted files

@@             Coverage Diff             @@
##           master    #1165       +/-   ##
===========================================
- Coverage   88.27%   63.35%   -24.92%     
===========================================
  Files          73       73               
  Lines        8817     8809        -8     
===========================================
- Hits         7783     5581     -2202     
- Misses       1034     3228     +2194     

Flag	Coverage Δ
unittests	63.35% <83.33%> (-24.92%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
dandi/cli/tests/test_command.py	100.00% <ø> (ø)
dandi/cli/tests/test_ls.py	100.00% <ø> (ø)
dandi/dandiapi.py	49.04% <ø> (-39.24%)	⬇️
dandi/support/pyout.py	90.00% <50.00%> (ø)
dandi/tests/fixtures.py	73.47% <100.00%> (-24.79%)	⬇️
dandi/tests/test_upload.py	19.35% <0.00%> (-80.65%)	⬇️
dandi/tests/test_delete.py	22.69% <0.00%> (-77.31%)	⬇️
dandi/upload.py	9.13% <0.00%> (-76.65%)	⬇️
dandi/tests/test_helpers.py	25.00% <0.00%> (-75.00%)	⬇️
dandi/cli/tests/test_service_scripts.py	25.00% <0.00%> (-75.00%)	⬇️
... and 25 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[yarikoptic]

oh boy,

New alerts
Security Alerts:

1 high
Other Alerts:

9 errors
11 warnings
63 notes

@jwodder could you please review and address one way (fix all) or another (skip some) or state your verdict on why we should not bother with CodeQL.

[jwodder]

@yarikoptic Where are you seeing that text?

[yarikoptic]

on the run for added workflow action: https://github.com/dandi/dandi-cli/pull/1165/checks?check_run_id=9750642454

[jwodder]

@yarikoptic Please explain the comment here:

dandi-cli/dandi/tests/fixtures.py

Lines 259 to 261 in 0439caf

	path = tempfile.mktemp() # not using pytest's tmpdir fixture to not
	# collide in different scopes etc. But we
	# would need to remove it ourselves

CodeQL complains about the use of mktemp(). Couldn't whatever scope issue you refer to be solved by using tmp_path_factory to create a directory instead?

[jwodder]

@TheChymera Please explain what the point of the error_message += ... lines is supposed to be in the below code, particularly given that (a) error_message is unset at those points, and (b) error_message is overwritten before it's used.

dandi-cli/dandi/cli/cmd_validate.py

Lines 196 to 213 in b3b7909

	elif len(errors) == 1:
	fg = _get_severity_color(severities)
	group_message = (
	f"{errors[0]}: detected in {pluralize(len(purviews), 'purviews')}"
	)
	if len(set(messages)) == 1:
	error_message += f" — {messages[0]}."
	click.secho(group_message, fg=fg)
	for purview, severity in zip(purviews, severities):
	error_message = f" {purview}"
	fg = _get_severity_color([severity])
	click.secho(error_message, fg=fg)
	else:
	error_message += "."
	for purview, severity, message in zip(purviews, severities, messages):
	error_message = f" {purview} — {message}"
	fg = _get_severity_color([severity])
	click.secho(error_message, fg=fg)

[jwodder]

@yarikoptic Aside from the issues related to the above two comments, I fixed a few of the issues and marked the rest as either false positives or "won't fix" (because the given code is that way deliberately). Now that those issues have been dismissed, they hopefully shouldn't come back.

[yarikoptic]

@yarikoptic Please explain the comment here:

dandi-cli/dandi/tests/fixtures.py

Lines 259 to 261 in 0439caf

	path = tempfile.mktemp() # not using pytest's tmpdir fixture to not
	# collide in different scopes etc. But we
	# would need to remove it ourselves

CodeQL complains about the use of mktemp(). Couldn't whatever scope issue you refer to be solved by using tmp_path_factory to create a directory instead?

I guess so - please refactor as you see fit. Unfortunately I am not even sure I understand exactly what collision across scopes I had in mind

[TheChymera]

Please explain what the point of the error_message += ... lines is supposed to be in the below code,

@jwodder There was no point, I used the wrong variable name. This is part of the logic which isn't really tested, but to explain the concept which I had intended for whenever we will have a use case, the idea is to auto-determine grouping by the length of the inputs, so that the grouping logic is handled by the cli function and doesn't need to pass an argument further down the line. Ideally you can mix it up in any way you want and the function should be able to figure out how to show it.

The group_message reports whichever list is length one and then details to which things it applies to, which are then listed prefixed with an indent via the error_messages. The line you highlighted was intended to simply append the error message, if there is only one, directly to the error — elsewise it appends a period and then proceeds to list the messages and the purviews together in the individual error_messages. I have a fix PR here: #1166

[yarikoptic]

I just merged #1466 -- since it should cover those two outstanding errors, let's be brave&bold and just merge this PR. Thank you @jwodder