waltti-apc-vehicle-position-splitter

Split Vehicle Position feed snapshots by vehicle and send onwards only the data for vehicles with APC devices onboard, each vehicle in an individual message.

This repository has been created as part of the Waltti APC project.

Development

1. Create a suitable .env file for configuration. Check below for the configuration reference.

2. Create any necessary secrets that the .env file points to.

3. Install dependencies:

   npm install

4. Run linters and tests and build:

   npm run check-and-build

5. Load the environment variables:

   set -a
   source .env
   set +a

6. Run the application:

   npm start

Docker

You can use the Docker image tvvlmj/waltti-apc-vehicle-position-splitter:edge. Check out the available tags.

Configuration

Environment variable	Required?	Default value	Description
FEED_MAP	✅ Yes		A map from Pulsar topics to feed publisher IDs. The map is given in the form of a stringified JSON array of strings in the shape [[pulsarTopic1, feedPublisher1], [pulsarTopic2, feedPublisher2], ...] like the output of Map.prototype.entries(). The feed publisher ID is a unique ID for the authority or the GTFS feed publisher whose APC data will be handled. The format is <country-code>:<name> where <country-code> follows a lowercase version of ISO 3166-1 alpha-2 and <name> is unique within the country. An example could be fi:jyvaskyla. The corresponding Waltti OpenData Authority IDs are listed here: https://opendata.waltti.fi/docs#gtfs-static-packages . An example of the whole list could be [[\"persistent://apc-dev/source/gtfs-realtime-vp-fi-kuopio\",\"fi:kuopio\"],[\"persistent://apc-dev/source/gtfs-realtime-vp-fi-jyvaskyla\",\"fi:jyvaskyla\"]].
CACHE_WINDOW_IN_SECONDS	❌ No	172800	How old messages to read from PULSAR_PRODUCER_TOPIC to warm up the cache when the service starts, maybe after serious downtime.
HEALTH_CHECK_PORT	❌ No	8080	Which port to use to respond to health checks.
PINO_LOG_LEVEL	❌ No	info	The level of logging to use. One of "fatal", "error", "warn", "info", "debug", "trace" or "silent".
PULSAR_BLOCK_IF_QUEUE_FULL	❌ No	true	Whether the send operations of the producer should block when the outgoing message queue is full. If false, send operations will immediately fail when the queue is full.
PULSAR_COMPRESSION_TYPE	❌ No	ZSTD	The compression type to use in the topic where messages are sent. Must be one of Zlib, LZ4, ZSTD or SNAPPY.
PULSAR_GTFSRT_CONSUMER_TOPICS_PATTERN	✅ Yes		The topic pattern to consume GTFS Realtime messages from.
PULSAR_GTFSRT_SUBSCRIPTION	✅ Yes		The name of the subscription for reading messages from PULSAR_GTFSRT_CONSUMER_TOPICS_PATTERN.
PULSAR_OAUTH2_AUDIENCE	✅ Yes		The OAuth 2.0 audience.
PULSAR_OAUTH2_ISSUER_URL	✅ Yes		The OAuth 2.0 issuer URL.
PULSAR_OAUTH2_KEY_PATH	✅ Yes		The path to the OAuth 2.0 private key JSON file.
PULSAR_PRODUCER_TOPIC	✅ Yes		The topic to send messages to.
PULSAR_VEHICLE_READER_TOPIC	✅ Yes		The topic to read vehicle catalogue from.
PULSAR_CACHE_READER_NAME	✅ Yes		The name of the reader for reading messages from PULSAR_PRODUCER_TOPIC to fill the initial cache.
PULSAR_VEHICLE_READER_NAME	✅ Yes		The name of the reader for reading messages from PULSAR_VEHICLE_READER_TOPIC to fill the initial cache.
PULSAR_SERVICE_URL	✅ Yes		The service URL.
PULSAR_TLS_VALIDATE_HOSTNAME	❌ No	true	Whether to validate the hostname on its TLS certificate. This option exists because some Apache Pulsar hosting providers cannot handle Apache Pulsar clients setting this to true.