eCap Stream

This is an eCap adapter that receives body data from the host and forwards it to an external library, to be then recorded, adapted or filtered at will. The processed -- or unprocessed -- data is then sent back to the host.

The intent of this library is to provide a simpler interface than fully implementing the eCap protocol, and thus also support FFI implementations.

Building

$ docker build -t ecap-stream .

Testing

The default entrypoint runs a simple test case, so all that is needed is to execute the docker image.

$ docker run -it ecap-stream

Using

Ecap-stream works by loading a configurable shared object library that implements the api described below, and delegating calls to it as some particular events take place in the host and are sent down to ecap-stream via the ecap interface.

These events are:

• new transaction is initialized: in the context of a web or proxy server, these could be requests and responses.
• new data is available: again in the web/proxy context, these will translate into request or response body data.
• host requests data: the host is requesting a block of data. This can be the data as just sent by the host, or an adapted version of it.
• no more data is available from the host: this would mean that all of the request/response data has been transferred in a web/proxy context.
• transaction is finished.

The docker image built can be used as the base image for a Dockerfile used in another project.

FROM ecap-stream

...

This will make both the libecap and libecap-stream dynamic shard objects available for linking.

The next steps are:

1. implement the external client module that will be loaded and receive the API calls with the host data to be then analyzed, adapted, or filtered, depending on the module's implementation requirements.
2. this module must be packaged in another shared object and also made available for linking.
3. configure the host to load both the ecap-stream shared object, and pass along this configuration the required path for the module that will in turn be loaded by eCap stream to fully implement your adapter/filter.

One such implementation of a shared object library in Rust can be referrenced here.

The following section details which endpoints need to be implemented, and their purpose, and the final one will provide an example configuration for loading ecap-stream in the squid proxy.

API

The following functions need to be implemented in the client module that will be loaded by libecap-stream, to complete an end-to-end implementation of the client module.

void init()

This function should perform any module wide initialization required, such as specific logging or runtime setups that may be needed. It is called once per eCap-service initialization, which usually translates into a single call per host lifecycle.

void uri(int id, const char* uri, int mode, const char* method)

This function is the first called for each transaction. In the context of web requests or responses, note that each flow of information is separately handled by different transactions. So, there could be an initial transaction for handling the request and a matching subsequent transaction for handling the respective response.

Parameters:

• id (int): this is an integer that represents the transaction identifier. It will be passed down throughout all other interactions between eCap stream and the client module.
• uri (const char*): a C null-terminated string, containing the full uri of the request (inluding query string parameters).
• mode (int): an integer signalling which ICAP mode this transaction is related to. 0 is for REQMOD (i.e. requests), and 1 is for RESPMOD (i.e. responses).
• method (const char*): a C null-terminated string containing the request method. For a web/proxy context, this can be any usual valid HTTP verb or proxying related verbs, such as CONNECT.

void header(int id, const char* name, const char* value)

This function will notify about every request or response header that is seen in the transaction. There will be one call per header.

Parameters:

• id (int): the transaction id to which the header belongs to.
• name (const char*): a null-terminated string representing the header name.
• value (const char*): a null-terminated string representing the header value.

void receive(int id, const void* data, libecap::size_type size)

This function notifies the client module of raw data available from the host, which usually is a part of a request or response body. None, a single, or multiple chunks may be sent, and the only guarantees made are on the order and completeness, meaning that chunks are sent ordered and all data in the body will be eventually sent down as chunks, if any data exists -- empty bodies may result in no calls being made.

Parameters:

• id (int): the transaction id to which the chunk of data belongs to.
• data (const void*): a pointer to an array of bytes that contains the available data from the host.
• size (libecap::size_type): the length of data available.

Chunk send(int id, libecap::size_type offset, libecap::size_type size)

This function is called when the host expects to receive data. The auxiliar parameters offset and size can be used to determine where the data transmission should start or resume, and how many bytes are expected to be sent back to the host.

Parameters:

• id (int): the transaction id to which the chunk of data belongs to.
• offset (libecap::size_type): the data offset of the data chunk to send back to the host.
• size (libecap::size_type): the number of bytes of data to send back to the host.

Return value:

• Chunk: a struct containing a const void* named bytes, pointing to the data to be sent back to the host, and a size_t size attribute representing the amount of bytes available in the data. See "chunk.h" for details. It should usually be acceptable to return a null bytes value along with a 0 size -- but this ultimately depends on what the host expects, which falls outside the scope of eCap Stream.

void done(int id)

This function signifies that the host has no additional raw data to be sent to the client module. There will be usually one last call to send so that any trailing data (such as stream terminators) can be properly sent back to the host.

Parameters:

• id (int): the transaction id for which data has been fully transmitted.

void cleanup(int id)

This function signifies transaction termination, and serves to cleanup any transaction related resources in the client module. No other calls are to be expected after this, and it should be called only once.

Parameters:

• id (int): the transaction id to be terminated.

Configuration

The modulePath configuration parameter must be passed to eCap-stream, so it can find the client module implementation. This varies depending on the eCap host eCap-stream is to run on, so we will provide an example for configuring the Squid proxy server. Always refer to the documentation for the host you want to use eCap-stream with.

Example Squid configuration (squid.conf):

...
loadable_modules /usr/local/lib/libecap-stream.so

ecap_enable on
ecap_service lens_respmod respmod_precache \
                 uri=ecap://github.com/51390/ecap-stream/respmod \
                 modulePath=/usr/local/lib/libclient-module.so
ecap_service lens_reqmod reqmod_precache \
                 uri=ecap://github.com/51390/ecap-stream/reqmod \
                 modulePath=/usr/local/lib/libclient-module.so

adaptation_access lens_respmod allow all
adaptation_access lens_reqmod allow all

...

This example would load a client module in the path /usr/local/lib/libclient-module.so, that should implement the API described above. After the host initializes the eCap interface and service, transactions should induce the data exchange between host, eCap-stream and finally, the client module.

By configuring both REQMOD and RESPMOD modes, the client module should then start seeing transactions both in the request as well as response sides of the communication. Squid -- and thus eCap stream -- will usually send 2 transactions related to the request leg, and 1 transaction related to the response leg of the communication, but this may vary depending on the flow between client -> proxy -> server.

References

• eCap documentation
• Squid eCap support