filter programs

[Byron]

Follow up of #913.

Driving of filter programs with two different protocols - statehandling is interesting here as well when supporting long-running procresses.

Tasks

Generally it's interesting if we can handle big files here, so streaming of files to the filter (huge files are replaced by small ones), and the other way around when small files are replaced by big ones. So it's a two-way streaming of input and output that should be reflected in the interface.

To support delays, it must be possible to run all standard filters but the filter process.

• one-off clean and smudge filters
• long-runnnig processes
• allow partial configuration and partial capabilities (without making this an error)
• control whether or not to wait for ongoing processes, and initiate graceful shutdowns.
• ability to restart long-running processes if they crash
• delay support for long-running processes
• a test to validate the ident filter against actual git

Next PR

• pipeline support to handle attribute logic with streaming support
• integration with gix

Research

• general directions are for checking out from git and for checking in to git
• it's vital to see if filtering would apply before having to apply it. This could be native to the relevant functions.
• optional round-trip checking
• if gitattribute is text=auto, git is set to auto-detect text mode for all files.
• to-git conversions are convert_to_git() namely: clean-filter | encode_to_git | crlf | ident
• to-worktree conversions are convert_to_working_tree…(), namely: ident | crlf | encode_to_worktree | smudge-filter
• normalization is special enough to be mentioned in the pipeline, but I don't see why it affects anything.
• there are streaming versions only of apply-filter effectively, with encoding, crlf and ident changes operating on memory buffers