-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor Evasion Detectors #1993
Refactor Evasion Detectors #1993
Conversation
Signed-off-by: Farhan Ahmed <Farhan.Ahmed@ibm.com>
Signed-off-by: Farhan Ahmed <Farhan.Ahmed@ibm.com>
Signed-off-by: Farhan Ahmed <Farhan.Ahmed@ibm.com>
Signed-off-by: Farhan Ahmed <Farhan.Ahmed@ibm.com>
Signed-off-by: Farhan Ahmed <Farhan.Ahmed@ibm.com>
Signed-off-by: Farhan Ahmed <Farhan.Ahmed@ibm.com>
Signed-off-by: Farhan Ahmed <Farhan.Ahmed@ibm.com>
Signed-off-by: Farhan Ahmed <Farhan.Ahmed@ibm.com>
Signed-off-by: Farhan Ahmed <Farhan.Ahmed@ibm.com>
Codecov Report
📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more @@ Coverage Diff @@
## dev_1.14.0 #1993 +/- ##
==============================================
- Coverage 85.41% 85.41% -0.01%
==============================================
Files 290 291 +1
Lines 25437 25453 +16
Branches 4573 4587 +14
==============================================
+ Hits 21728 21740 +12
+ Misses 2541 2539 -2
- Partials 1168 1174 +6
|
Signed-off-by: Farhan Ahmed <Farhan.Ahmed@ibm.com>
Signed-off-by: Farhan Ahmed <Farhan.Ahmed@ibm.com>
tests/defences/detector/evasion/test_binary_activation_detector.py
Outdated
Show resolved
Hide resolved
tests/defences/detector/evasion/test_binary_activation_detector.py
Outdated
Show resolved
Hide resolved
tests/defences/detector/evasion/test_binary_activation_detector.py
Outdated
Show resolved
Hide resolved
tests/defences/detector/evasion/test_binary_activation_detector.py
Outdated
Show resolved
Hide resolved
tests/defences/detector/evasion/test_binary_activation_detector.py
Outdated
Show resolved
Hide resolved
tests/defences/detector/evasion/test_subsetscanning_detector.py
Outdated
Show resolved
Hide resolved
Signed-off-by: Farhan Ahmed <Farhan.Ahmed@ibm.com>
Signed-off-by: Farhan Ahmed <Farhan.Ahmed@ibm.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @f4str Thank you very much for refactoring the evasion detectors, this should be very useful for many users! I also like how you made the detector tests framework independent!
Description
Refactor the
art.defences.detector.evasion
module by creating a base abstract classEvasionDetector
which all of the evasion detectors now inherit from. This base class has a.detect()
method which will return a report and binary numpy array for which samples are detected adversarial. TheBinaryInputDetector
,BinaryActivationDetector
, andSubsetScanningDetector
classes have been refactored accordingly to inherit fromEvasionDetector
and use the.detect()
method.The unit tests have been updated accordingly and now use the pytest framework, so they have been removed from legacy tests.
This is technically a "breaking change" since the module has been refactored, but existing functionality is unchanged. Documentation has been updated along with updating
notebooks/detection_adversarial_samples_cifar10.ipynb
to use the new evasion detector structure.Fixes #1913
Type of change
Please check all relevant options.
Testing
Please describe the tests that you ran to verify your changes. Consider listing any relevant details of your test configuration.
BinaryInputDetector
using Keras, TensorFlow, and PyTorchBinaryActivationDetector
using Keras, TensorFlow, and PyTorchSubsetScanningDetector
using Keras, TensorFlow, and PyTorchTest Configuration:
Checklist