NOTE: the project is no longer supported. All the functionality has been integrated into EVA ICS v4.
Pulr is a small command-line tool, which can pull data from the device and convert it into events. Meaning the data is outputted ONLY when it's changed.
The data is always being outputted to STDOUT, you can grep it or use any pipeline processor (I use vector) to push it via HTTP, into databases etc.
Before the output, any data part can be transformed: converted, divided, multiplied, rounded and so on.
The data can be outputted as plain text or as nd-json.
pulr -F modbus.yml -L
{"id":"sensor:axon/din1.value","value":0}
{"id":"sensor:axon/din2.value","value":1}
{"id":"sensor:axon/din3.value","value":1}
{"id":"sensor:axon/din4.value","value":0}
{"id":"unit:axon/dout1.status","value":1}
{"id":"unit:axon/dout2.status","value":1}
{"id":"unit:axon/dout3.status","value":0}
{"id":"unit:axon/dout4.status","value":0}
{"id":"unit:axon/aout.value","value":0.00619}
{"id":"sensor:axon/ain.value","value":5.2045}
{"id":"unit:tests/u1.status","value":0}
{"id":"unit:tests/u2.status","value":0}
Download a static binary from releases page and enjoy.
libplctag is statically linked with Linux binaries. For Windows binaries, plctag.dll is included into the release zip archive.
Ethernet/IP support requires libplctag, download and install it:
git clone https://github.com/libplctag/libplctag
cd libplctag
cmake CMakeLists.txt
make
sudo make install
sudo ldconfig
After, either use Makefile targets or run cargo manually:
cargo build --release # or custom options
Look in ./examples for the example configurations.
One Pulr instance pulls one piece of the hardware equipment. The goal is to pull and process the data as fast as possible, but die as soon as any errors occur. Pulr is built to be started by supervisor, which collects the data from it and restarts the process on crashes.
Pulr is written in Rust. So it's rocket-fast and super memory efficient. You can start a hundred of Pulr processes on a single machine and barely notice any load.
The first (draft) Pulr version was written in Python, it isn't supported any longer, but kept in "legacy0" branch.
Currently supported:
- Modbus (TCP/UDP only)
- SNMP (v2)
- Ethernet/IP (Allen Bradley-compatible)
Need to pull data from Modbus RTU? Use our free Modbus gateway server.
-
calc_speed - calculate value growing speed, useful for SNMP interface counters
-
multiply, divide, round
- text (aliases: stdout, plain, "-") - output the data as plain text, default
- ndjson (alias: json) - output the data as newline delimited JSON
- csv - comma-separated values
- eva/datapuller - specific type for EVA ICS
Optional field "time-format" adds time to data output. Valid values are: "rfc3339", "timestamp" (alias: raw).
By default, data in JSON is outputted as
{ "time": "time rfc 3339/timestamp", "id": "metric id", "value": "event value" }
{ "time": "time rfc 3339/timestamp", "id": "metric id", "value": "event value" }
{ "time": "time rfc 3339/timestamp", "id": "metric id", "value": "event value" }
Specifying output format as ndjson/short (aliases: ndjson/s, json/short, json/s), this can be switched to "short" format, with only 2 fields: "time" and "metric id":
{ "time": "time rfc 3339/timestamp", "metric id": "event value" }
{ "time": "time rfc 3339/timestamp", "metric id": "event value" }
{ "time": "time rfc 3339/timestamp", "metric id": "event value" }
Consider you have a device, want to collect metrics from it and store them into InfluxDB. Pulr comes with a tiny tool called ndj2influx, which allows parsing NDJSON data and storing it directly into InfluxDB.
It's highly recommended to set "time-format" (rfc3339 or raw, doesn't matter) to have metrics stored with the same timestamp the pull request has been performed.
And full command to get and store metrics will be:
pulr -F /path/to/pulr-config.yml -L -O ndjson | \
ndj2influx http://<INFLUXDB-IP-OR-HOST>:8086 \
<DATABASE_NAME> @device1 -U <DB_USERNAME>:<PASSWORD> -M id -v
Let's explain all options:
-
pulr option -L tells Pulr to work in loop, continuously pulling the device.
-
pulr option -O ndjson makes sure Pulr will output data in NDJSON format.
-
first two ndj2influx options specify InfluxDB API URL and database name
-
the next option should specify base metric column (device name). As pulr doesn't output it, set it for all metrics to @device1
-
option -U is used to pass InfluxDB authentication. If auth isn't turned on, the option is not requited.
-
option -M id is to use "id" column as metric id.
-
option -v is for verbose output and can be omitted in production.
The same result is produced with commands:
pulr -F /path/to/pulr-config.yml -L -O ndjson/short | \
ndj2influx http://<INFLUXDB-IP-OR-HOST>:8086 \
<DATABASE_NAME> @device1 -U <DB_USERNAME>:<PASSWORD> -v
everything is almost the same, except Pulr is told to produce "short" (id=value) JSON output and option -M id for ndj2influx can be omitted.
It's recommended to hide password from command line arguments on production using INFLUXDB_AUTH env variable:
pulr -F /path/to/pulr-config.yml -L -O ndjson | \
INFLUXDB_AUTH=<DB_USERNAME>:<PASSWORD> ndj2influx \
http://<INFLUXDB-IP-OR-HOST>:8086 <DATABASE_NAME> @device1 -M id -v
Both tools will crash as soon as any problem occurs. They're made this way, because in production "it's better to crash and be restarted, than freeze".
To automatically restart the tools, let's use any supervisor, e.g. Supervisord (available in almost all Linux distros).
Create a simple config and put it to /etc/supervisor/conf.d/pulr-device1.conf
[program:pulr-device1]
command=sh -c "sleep 1 && pulr -F /path/to/pulr-config.yml -L -O ndjson | ndj2influx http://<dbhost>:8086 <DATABASE_NAME> @router1 -M id -v"
environment=INFLUXDB_AUTH=<DB_USERNAME>:<PASSWORD>
autorestart=true
autostart=true
priority=100
events=PROCESS_STATE
That's all. Supervisord will monitor the processes and restart them if necessary.
As it was mentioned above, Rust version is fast. It's very fast and efficient. However it differs from the draft Python version:
-
"version" field in configuration file should be set to "2"
-
"transform" syntax was changed a little bit, see config examples. "speed" function renamed to "calc_speed".
-
more command line args.
-
No more "bit2int" and "int2bit" transformers, it's hardly to imagine where they could be useful.
Before reporting a problem, try running Pulr with verbose output ("-v" command line flag).
Run pulr as:
PULR_VERBOSE_WARNINGS=1 pulr <options>
You welcome.
Just:
-
Outputs. No outputs are planned, except STDOUT. Use pipeline converters.
-
Pulling more than one device simultaneously. Isn't planned, start another Pulr process instead.