Cyberchef cli

[Boolean263]

This fairly simple script leverages the Node API provided by CyberChef to allow performing operations on local filesystem objects without needing to copy them into and out of a browser window.

It requires a recipe to be specified at runtime as a JSON file.

Also includes a simple TCP server for use by other local processes. This server is hard-coded to use localhost in order to discourage production use of the script.

This is my first time programming in Node, but the presence of the Node API inspired me to try. CyberChef includes many tools that are frequently subjected to "reinventing the wheel," but it makes more sense to leverage the existing tools whenever possible.

[Boolean263]

I chose this version of commander because it's also the version pulled by codepage, in the hopes of minimizing impact of adding a dependency.

[Boolean263]

One quirk of Node I haven't yet been able to work out is why I need the failure handler on this then() call in addition to the catch() call.

[mattnotmitt]

You don't need the failure handler on it, you can either use that or .catch().

[d98762625]

Hi @Boolean263 Thanks a lot for this PR. I have never considered running chef in a cli and I didn't know about the bin property of package.json, so thank you.

I have cloned this and tried running it locally - I am having some issues running it. Here's what I did

// node  version is 10.13.0
// clone repo into cli
// cd cli
git checkout cyberchef-cli
npm install
npm link
cyberchef -r recipe.json

// => TypeError: program.version(...).description(...).usage(...).requiredOption is not a function

You've stuck with commander version 2.x.x in this PR, but it looks like requiredOption doesn't exist in that version of commander.

Otherwise, for listening for TCP requests, I feel like this overlaps a lot with the CyberChef-server project, which accepts recipes and input over TCP and returns the output.

More generally, I think this CLI could benefit from living in its own repository. This would then allow you to wrap the CyberChef-server into it, if you were to use that. It would also allow you to use a more modern version of commander! Any thoughts on this? @n1474335 ?

[Boolean263]

Hi @d98762625, thank you for considering my PR!

If you like the idea of the script for CyberChef, then I'm okay with using a more recent version of commander (which apparently I was using anyway without realizing) if you folks are okay with adding one more dependency.

I hadn't known about CyberChef-server when I added the TCP option. I threw that in there after noting the (relatively) long startup time of my script, both for the learning experience and as a way to amortize the startup time. If you like the script idea apart from that, I can remove the server option and look more closely at CyberChef-server.

I feel like the script is too simple to merit its own repo, but of course that'll be my next option if you decline this PR. :)

[d98762625]

Can you also give a full example of how you would run this, with a recipe and some input?

I'll spend some time thinking about whether we want to add the script to this repo or not. In the mean time, please also take a look at the repl that's already part of this repo, which you can run with npm run repl. Can you tell me if that functionality is useful? You would be able to bake some recipe like so:

npm run repl
chef> recipe = JSON.parse(fs.readFileSync('recipe.json'))
chef> chef.bake('some input', recipe)

For context, the load time of this script is slow because we're using require and all of CyberChef is written for the ES Module ecosystem rather than node's CommonJS ecosystem. Therefore the whole thing is wrapped in a CJS export, which is what slows it down. I've tried to improve this in this PR, prompted by comments in issues linked to that PR.

[Boolean263]

When you ask for a full example, are you looking for proper written documentation? The script itself has a help message, and it's intended to be straightforward:

sh$ cyberchef -r recipe.json -o output_dir/ input_dir/*.bin

I hadn't known of the REPL either, but that's less convenient to me because it involves having to add programming to each iteration of my workflow, rather than doing so once with the script, and focusing on what I want to find after that. (Or, to put it another way, you can view the script as being a thin wrapper around the REPL.)

CyberChef (specifically, the web-based version) is very popular where I work, but I'm more of a command-line type of person. I wrote the script because my workflow can involve hundreds or even thousands of files, and dragging them into and out of a browser interrupts my flow. It also seems a bit over-complicated to me to require the extra overhead of a web browser when all I need is the code you've written.

[mattnotmitt]

If you're looking for speed, why not adapt this script to use a cyberchef-server instance running under docker - first time you run it it checks if the container and server are up, and if not then it starts it. Every time after that it just uses the already spawned docker instance.

[d98762625]

Yes I've seen the script help message. Maybe I'm misunderstanding, but when I run:

cyberchef -r recipe.json -o out/ hi.txt

I get

Run with "--help" for usage

and nothing in the out dir.

It might be worth you doing a fresh clone / checkout of your branch to see if you can replicate these issues I'm having?

[Boolean263]

@d98762625: Looking at the script, it shows the Run with "--help" for usage if there's a problem parsing the command line. That may be another issue with commander, combined with the fact that my script doesn't actually print the exception that gets it to that point. I'll push an update that shows the exception message, but for troubleshooting purposes, could you do the same in your test checkout and see if that gives you more information?

Running from a fresh checkout (and doing npm install), cli.js isn't able to require("cyberchef"). I'll report back on your request to reproduce the error from this fresh checkout once I've figured out how to generate src/node/index.mjs. (Did I mention I'm new to Node? This has been a fun learning experience, and I appreciate your patience so far.)

@mattnotmitt: I'll be looking into an approach like that, definitely. Like I told @d98762625, I hadn't known about the existence of CyberChef-server when I originally created the script. My next update will remove the TCP support for this reason.

[d98762625]

I forgot, I also ran npm link && npm link cyberchef in the repo to get around that require issue. I think the actual fix is just to require the .cjs file at src/node/cjs.js instead of cyberchef, seeing as you're already in the cyberchef repo.

[Boolean263]

Thank you for the extra information! This latest push uses that requirement.

Building from scratch in a fresh checkout didn't let me reproduce the problem you had. This latest push should print e.message when an exception is thrown during command-line parsing.

(By the way @d98762625 , I've also dropped you a note on gmail.)

[Boolean263]

In discussion with @d98762625, we decided this is probably better discussed as a ticket than as a PR, so see feature request #1046. The script in this PR could possibly still be included as an example program for using the Node API.

[BillC-wfh]

Neat. Seems like a good discussion.

…

On Sat, Jul 30, 2022, 11:28 David Perry ***@***.***> wrote: Closed #1043 <#1043>. — Reply to this email directly, view it on GitHub <#1043 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AO26RI5DCZ6MWSJUAWE4GGDVWVC2RANCNFSM4NNMB23Q> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>