ZeroMQ plugin framework

This document describes details of the MicroDrop ZeroMQ plugin framework that is being introduced as part of MicroDrop v2.0 development. It will be updated as progress proceeds along the development roadmap.

Plugin structure

A MicroDrop plugin is simply a file folder (which can be installed from an archived *.zip or *.tar.gz file) containing the following files (at a minimum):

/
  __init__.py
  properties.yml
  hooks /
          Windows /
                    on_plugin_install.bat
          Linux /
                    on_plugin_install.sh

__init__.py

This file is necessary for PyUtilib based plugins, in which plugins are loaded by importing the plugin root directory. This should probably be replaced by an on_plugin_start hook (or similar) which will start a separate plugin process in a way that is language agnostic. See #197.

properties.yml

A YAML file which contains plugin metadata. For example, here are the contents of the properties.yml file from the dmf_control_board_plugin:

{
  package_name: dmf_control_board_plugin,
  plugin_name: wheelerlab.dmf_control_board_plugin,
  version: 2.2.4
}

Here's a description for each of the metadata keys:

• package_name: name of the folder in which the plugin will be extracted (must be a valid Python package name and unique across all plugins). Note: maybe we don't need this if we're not importing plugins as Python packages?
• plugin_name: MicroDrop plugin name (must be unique across all plugins)
• version: plugin version

hooks

The following plugin hooks are defined. For Windows, these are *.bat files stored in the hooks/Windows folder. For Linux/Mac OS, they are *.sh files stored in the hooks/Linux folder. (See also #197)

on_plugin_install

Called immediately after a plugin has been extracted but before it is launched. This file is commonly used to install plugin requirements (e.g., using pip). This script receives the path to the Python executable as it's first argument.

Note: how can we generalize this? Plugins written in other languages will probably require similar information (i.e., if we had a node.js plugin, it would need the path to node.exe). See #198 for more info.

Packaging plugins

We currently use a release.py file included in each plugin's code repository to generate the properties.yml metadata file and package the plugin in a *.tar.gz archive file. The plugin version field is generated from each plugin's git repository (currently using the microdrop_utility.Version.from_git_repository method).

Here are some ideas to support current use and language agnostic plugins down the road (note that not all of these need to be done all at once, or at all):

1. Formally define our plugin version scheme (this might also be an opportunity to adopt the same versioning as the Conda build tool uses?). See #198
2. Write a command line tool to return the version string given a path to a git repository root and package this command line tool as a Conda package. See #198
3. Create a tool similar to Conda build (maybe we can figure out a way to actually use Conda build? See #216) to package up a plugin release (replacing release.py). Even if we used Conda build to package plugins, we could still manually host and install those plugins like we are doing now.