Chowda cannot handle multiple parameters in app endpoint query string

[owencking]

Description

When I run a batch against an app/endpoint defined with multiple query string parameters, not all the parameters get correctly parsed and sent to the app.

Depending on the app and the parameters, this might cause the app to disregard some parameters, or crash completely.

This is a super-important issue, since it affects the core ability to run CLAMS apps.

Reproduction steps

The problem can be observed by looking at a couple of Chowda batches, their pipelines, and the apps in those pipelines.

For example, look at Batch 49.
Follow links over to the relevant metaflow run. See this line in stderr:

mario.utils.CLAMSAppError: http://app-swt-detection?sampleRate=750\u0026minFrameCount=3 failed: 500 - b'{"message": "Internal Server Error"}\n'

For another example, look at Batch 51
In the metaflow run, we see:

mario.utils.CLAMSAppError: http://app-swt-detection?startAt=0\u0026stopAt=300\u0026sampleRate=2000 failed: 500 - b'{"message": "Internal Server Error"}\n'

It looks like the ampersand in the query string is getting replaced by a unicode escape sequence.

Expected behavior

Chowda should correctly handle &-separated parameters in the querystring.

Screenshots

No response

Browsers

No response

OS

No response

Additional context

No response

[mrharpo]

Found the issue

It's a matter of how Metaflow creates kubernetes files for an argo-events Sensor:

In Metaflow, the argo-workflow plugin _compile_sensor adds parameters as "{{ .Input.body.payload.%s | toJson }}".

But digging further, the toJson function is a simple wrapper around Go's json.Marshal which escapes & (and < >) for HTML compatible formatting.

Have written this to the Metaflow slack group. Will let you know what I hear.