Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create tests to evaluate the number of false positives and false negatives #222

Merged
merged 11 commits into from
May 9, 2023

Conversation

romain-dd
Copy link
Contributor

@romain-dd romain-dd commented Apr 14, 2023

What is done in this PR:

  • Download ~500 legit pypi packages
  • Download ~1000 malicious packages
  • Add folder scanning capability to guarddog to analyze packages
  • Dockerfile to test malicious packages
  • Clean up json result to avoid empty category like "result": {"npm-install-script": {}, "steganography": {}}. So we''ll only get the categories if something malicious/suspicious has been found, if nothing found, we'll get "result": {}

Test round 1:

  • In a container, I tested with 100 legit packages, it took ~5min (which is too slow), and found 12 false positives
  • On my local machine, same result except it took 1min 30s.

Test round 2:

  • Test ~500 legit pypi packages, I got the result after 8 min and found 46 false positives (I didn't analyze them), so this give us a ratio of ~9% of false positives

@romain-dd romain-dd force-pushed the rma/test_false_positives branch 3 times, most recently from ebfcb02 to 4deb421 Compare April 17, 2023 07:36
@romain-dd romain-dd marked this pull request as ready for review April 28, 2023 11:31
@romain-dd romain-dd force-pushed the rma/test_false_positives branch 2 times, most recently from 0f4758c to cf2bd19 Compare April 28, 2023 12:03
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
@christophetd christophetd merged commit 8814db5 into main May 9, 2023
@christophetd christophetd deleted the rma/test_false_positives branch May 9, 2023 13:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants