Allow to parse JSON/XML inside specific variables

[blotus]

Summary

Currently, it is only possible to parse JSON/XML at the body level and not for specific variables.
This makes it a bit tricky (or impossible, especially in the case of XML) to write rules to match on specific variables containing JSON or XML.

Basic example

With a request like this:

POST / HTTP/1.1
...

-----------------------------ABCDE
Content-Disposition: form-data; name="jsonVar"

{"foo": "bar"}
-----------------------------ABCDE--

I'd like to be able to access the parsed JSON as any over variable with something like ARGS_POST:jsonVar.json.foo

[jptosso]

+1

[jcchavezs]

QQ: couldn't you achieve the same with a custom multipart body processor

coraza/internal/bodyprocessors/multipart.go

Lines 20 to 21 in 476a178

	type multipartBodyProcessor struct{}

?

[blotus]

I thought about this, but it seems a bit impractical: you would need one custom processor for multipart+XML, one for multipart+JSON, one for urlencoded+XML, and one for urlencoded+JSON no?
This would also not allow you (AFAIK ?) to handle query strings args with JSON/XML in them.

And probably the most difficult thing, you have to set the body processor very early when processing a request, but this is not something that would always be possible.
For example, let's say you have a (weird) API with only one endpoint, and an action parameter that dictates what the rest of the body looks like:

POST / HTTP/1.1
...

action=foo&jsonFoo={"bla":1}

and

POST / HTTP/1.1
...
action=bar&JsonBar={"bla": 2}

In this case, you would need to first process the body before knowing which var should be decoded.
(I know this example is very far-fetched, but I've seen way worse in real life).

Without looking at the code, I think maybe having some kind of lazy parsing would be the best bet: If the user tries to access ARGS:foo.json.whatever, trigger the JSON parsing of the foo arg (same for XML).

[jptosso]

I think we should update the body processor interfaces and add some feature to handle other sources of data.
A ctl or an operator might make sense to update the ARGS variable, but also it doesn't make sense to use post or get, as the new source of ARGS could be the url too or even a header

[jcchavezs]

So we are talking about an ARGS processor? Is there any standard situation for this? Like all the examples you mentioned look like specific behaviours so I wonder if we should be declaring this programatically rather than through a seclang.

Without looking at the code, I think maybe having some kind of lazy parsing would be the best bet: If the user tries to access ARGS:foo.json.whatever, trigger the JSON parsing of the foo arg (same for XML).

I remember proposing this to @jptosso but he had a point about variables being flattened for a reason.

Pinging @airween and @dune73 in case they have some ideas.

[jcchavezs]

Some discussion already happened in #456