Package for rule based dependency scanning and service fingerprinting via DNS.
This package provides utilities for writing and evaluating verbose and easy to read rule definitions in YAML-format. There some builtin rule formats with more available as plugins.
Due to needing the implementation to be less bloated, teh structure was changed. There is now much less code and useless features were removed.
Here is the gist:
- The API has been slimmed down
- Almost no "hacked" attributes
- Only the
context
is "injected" into rules when processing - Context contains
mule
,rules
,storage
andbackend
- Only the
- Record has had its
result
property removed - Record has been simplified
- Everything is now optionally a context manager
- Allows easy initialization and de-initialization
- Storage, Mule and Backend are entered together
- Rules are entered once per scan
- Rules were reworked to be simple callables
- New signature
__call__(Record, Result) -> None
- Scan result is now explicitly given to the rule
- Result is modified by the rule and nothing is returned to reduce complexity
- Removed rule priority
- Now simply respects insertion order
- Added batch rules
- Signature
__call__(List[Record], Result) -> None
- Batches are run at the end of processing
- Signature
- Added ANY matching rules
- Will get called using all produced records
- Note: Only record types with active rules are scanned
- Created additional built-in rule types
- Storage was slimmed down
- Now a storage simply has
store
andfetch
- Additional storage types for databses
- Now a storage simply has
- Backends were simplified
- Added some utility backends
- New signature
- Almost no "hacked" attributes
- Redesigned the YAML loader and Plugin handling
- Now there is no hacky loading going on
- Plugins are implicit and do no hooking
- Plugins only extend rule, storage and backend type scan options
- Example server was removed completely
- Did not have time to rewrite it yet
- Nearly useless without search
- Adds a command line runner for rules (WIP)
pip install dnsmule[full] dnsmule_plugins[full]
This will install everything available for DNSMule. You can also choose to install components as necessary.
For installing from the repo you can use:
pip install -e . ./plugins
The DNSMule tool takes YAML config as input and uses it to scan a domain:
python -m dnsmule --config rules/rules.yml example.com
This will give output like:
{
"name": "example.com",
"types": [
"TXT",
"A"
],
"tags": [],
"data": {}
}
Check out the examples in the examples folder. They should get you up and running quickly. The examples show some of the features available in DNSMule.
The tool configuration is done through one or multiple rule configuration files. The file structure is defined in the schema file. In addition to some builtin rule types, it is possible to create new types by registering handlers or rules programmatically.
Rules support registration per DNS record type, as well as batch or any types:
plugins:
- 'sample_module'
storage:
type: 'dict'
config: { }
backend:
type: 'data'
config:
'example.com':
- name: 'example.com'
type: 'A'
data: '127.0.0.1'
rules:
- name: 'Sample'
type: 'sample'
record: 'any'
- name: 'Match digits'
type: 'regex'
record: 'A'
config:
name: 'TEST'
regex: '\d+'
label: 'label'
- name: 'Add timestamp'
type: 'timestamp'
record: 'batch'
- name: 'Mark as test'
type: 'dynamic'
record: 'any'
config:
code: |
from dnsmule.utils import extend_set
def process(record: Record, result: Result):
result.data['test'] = True
It is possible to register the schema file as a custom JSON schema in IntelliJ editors. This will give access to typehints and schema validation inside rule files and is especially nice for dynamic rule definitions as you get full editor features for python inside the snippets.
- settings...
- Languages & Frameworks > Schemas and DTDs > JSON Schema Mappings
- Add a new mapping with the schema file and specified file or pattern
This is configured in the schema using the custom intellij language injection tag:
x-intellij-language-injection:
language: Python
prefix: |+0
code hints go here
check the schema for more info
suffix: ''
Currently, this supports dns.regex
pattern regex language injection and dns.dynamic
rule code language injection.
Type hints and quick documentation are available.
Dynamic rules are defined as code snippets with one or two methods
An init method that is invoked once after creation
def init() -> None:
...
A process function that is invoked once for each record
def process(record: Record, result: Result) -> None:
...
The only globals passed to these methods are:
- __builtins__
- RRType, Record, Result, Domain, Config
- The Config contains the
config
property passed to the rule from YAML
- The Config contains the
- Any additional globals created by the code itself
When the code is exec'd the result is inspected for:
- init function
- process function
Some notes:
- The init function is invoked every time the rule context is entered
- Take note that this happens on every scan
- The process function is invoked exactly once for a single Record
- Any rules created from the init method will be invoked for every suitable record
This package is under development. Pull requests are welcome, but will only be accepted after 0.8.0 is complete and my thesis is done.