# Writing Plugins

Plugins for Clusterio are JavaScript modules that run under Node.js.
Each entrypoint of a plugin exports a function that attaches handlers to hooks called during various stages and operations of Clusterio.


## Contents

- [Plugin Structure](#plugin-structure)
- [Defining the plugin entrypoint](#defining-the-plugin-entrypoint)
- [Logging Messages](#logging-messages)
- [Plugin Configuration](#plugin-configuration)
- [Plugin Permissions](#plugin-permissions)
- [Migrating Stored Data](#migrating-stored-data)
- [Communicating with Factorio](#communicating-with-factorio)
- [Defining Link Messages](#defining-link-messages)
- [Sending Link Messages](#sending-link-messages)
    - [Handling connection events](#handling-connection-events)
- [Collecting Statistics](#collecting-statistics)
- [Adding Custom Commands to clusterioctl](#adding-custom-commands-to-clusterioctl)


## Plugin Structure

The basic file structure of a plugin is the following.

    plugin_name/
      +- index.js
      +- package.json
      +- controller.js
      +- instance.js
      +- ctl.js
      +- test/
      |  +- plugin.js
      +- module/
         +- module.json
         +- plugin.lua

Clusterio plugins are Node.js packages that can be published on npm for ease of installation and distribution.
The usual guides for creating such packages apply.
At minimum the `package.json` file must contain a version entry.
For automatic discovery, include `"clusterio-plugin"` in the package's `"keywords"` array.
Installed npm plugins are discovered from the installation's dependencies when controller, host or ctl loads its configuration, and added to `plugin-list.json`.
Discovery starts with the root package's `dependencies`, not `devDependencies`, and follows dependencies of recognized plugins.
It does not search arbitrary nested dependency trees for plugins.
For tooling that needs the same discovery behavior, reuse [loadPluginList](../packages/lib/src/load_plugin_list.ts) rather than maintaining another package scanner.

A possible workflow for developing plugins is to place the plugin in a sub-directory of where clusterio has been installed, and rely on Node.js searching up the folder heirarchy for it to find `@clusterio/lib`.
After creating a directory for your plugin, the boilerplate can be generated automaticlly from a set of tempaltes using `npm exec @clusterio/create -y -- --plugin-template` after which you will be asked which areas your plugin will interact with and your plugin's name.
To add it to `plugin-list.json` so that it gets loaded use the `plugin add <path>` sub-command to either clusteriocontroller, clusteriohost or clusterioctl.
Note that it's important that the path starts with ./ or ../ (use .\ or ..\ on Windows).

For a plugin to be recognised by Clusterio it needs to export an entry named `plugin` from its main entrypoint.
By default the main entrypoint is the `index.js` file, but this may be changed by setting the `"main"` entry to a different file in `package.json`.
Keep this `"main"` entry when using a package `"exports"` map: Clusterio currently loads the plugin implementation from `"main"` (or `index.js`), rather than selecting a conditional export.

If a plugin published on npm defines an `"exports"` map, it must also expose `"./package.json"`.
Both automatic discovery and runtime loading resolve `<package-name>/package.json` to read the plugin's metadata.
Packages that hide this file are currently skipped by discovery, and manually adding their package names to `plugin-list.json` does not make them loadable.
The generated plugin templates omit `"exports"`, which leaves `package.json` accessible.
For example, a plugin that needs an export map can use:

```json
{
    "name": "example-clusterio-plugin",
    "version": "1.0.0",
    "type": "module",
    "main": "./index.js",
    "keywords": ["clusterio-plugin"],
    "exports": {
        ".": "./index.js",
        "./package.json": "./package.json"
    }
}
```

Here's an example of `index.js`:

```js
import * as messages from "./messages.js";

export const plugin = {
    name: "foo_frobber",
    title: "Foo Frobber",
    description: "Does advanced frobnication",
    instanceEntrypoint: "instance.js",
    controllerEntrypoint: "controller.js",
    messages: [messages.Frobnicate],
};
```

The following properties are recognized:

**name**:
    Stable internal identifier of the plugin, used as its key in the plugin list.
    For local plugins automatically discovered under `plugins/` or `external_plugins/`, it must match the plugin's directory name.
    It does not have to match the npm package name.
    For example, the package `@clusterio/plugin-subspace_storage` declares the name `subspace_storage`.

**title**:
    Name of the plugin as shown to users.
    Currently not used.

**description**:
    Brief description of what the plugin does.
    Currently not used.

**instanceEntrypoint**:
    Path to a Node.js module relative to the plugin directory which exports the instance entrypoint function for this plugin.
    This is an optional paramater.
    A plugin may have code only for instances but it must still be loaded on the controller in order for it to be possible to load it on an instance.

**instanceConfigFields**:
    Object mapping instance configuration fields to `FieldDefinitions`for this plugin.
    See [Plugin Configuration](#plugin-configuration)

**controllerEntrypoint**:
    Path to a Node.js module relative to the plugin directory which exports the controller entrypoint function for this plugin.
    This is an optional parameter.
    A plugin can be made that only runs on the controller.

**controllerConfigFields**:
    Object mapping controller configuration fields to `FieldDefinition` objects for this plugin.
    See [Plugin Configuration](#plugin-configuration)

**hostEntrypoint**:
    Optional path to the Node.js module exporting the host entrypoint function.

**hostConfigFields**:
    Object mapping host configuration fields to `FieldDefinition` objects.

**ctlEntrypoint**:
    Path to a Node.js module relative to the plugin directory which exports the clusterioctl entrypoint function for this plugin.
    This is an optional paramater.
    A plugin can be made that only runs on the clusterioctl side.

**ctlConfigFields**:
    Object mapping ctl configuration fields to `FieldDefinition` objects.

**webEntrypoint**:
    Optional key of the web entrypoint in the plugin's web bundle, this should be `"./web"`.
    The `clusterioPlugin` Vite plugin exposes `web/index.tsx` (or `.ts`, `.jsx`, `.js`) under this key; it is not a Node.js file path.

**migrationEntrypoint**:
    Path to a Node.js module relative to the plugin directory which exports the migration entrypoint function for this plugin.
    This is an optional parameter.
    See [Migrating Stored Data](#migrating-stored-data)

**features**:
    Array of features the plugin needs from the instances it is loaded on, any of `"SavePatching"`, `"ScriptCommands"` and `"LuaUdp"`.
    See [Handling Invalid Configuration](#handling-invalid-configuration)

**messages**:
    Array of request and event classes for this plugin.
    See guide for [defining link messages](#defining-link-messages) below.

**errors**:
    Array of error classes for this plugin.
    See [Request errors](#request-errors).

**permissions**:
    Array of permission definitions for this plugin.
    See [Plugin Permissions](#plugin-permissions)

The optional module folder contains a Clusterio module that will be patched into the save when the plugin is loaded.
See the section on [Clusterio Modules](developing-for-clusterio.md) in the Developing for Clusterio document.
The only restriction imposed on modules embedded into plugins is that they must be named the same as the plugin.

While there is no standard for how to organize a plugin it's recommended to put the controller entrypoint into controller.js and the instance entrypoint into instance.js.
You can put them into whatever file you want (even the same one for both).

Node.js entrypoint paths must include the `.js` extension and use forward slashes for directory separators.
This rule does not apply to the `webEntrypoint` exposure key.


## Defining the plugin entrypoint

Each entrypoint module exports a default async function which is called once when the plugin is loaded.
It is passed a context object with what the plugin needs to interact with Clusterio, and attaches handlers to the hooks it is interested in.
For example, a controller entrypoint can look like this:

```js
export default async function(context) {
    const { controller, logger, plugin } = context;

    controller.hooks.save.attach(plugin.name, async () => {
        await saveFrobnicationData();
    });

    controller.hooks.shutdown.attach(plugin.name, async () => {
        logger.info("Stopping frobnication");
    });
}
```

The context has the following properties:

**plugin**:
    The plugin's own info, as exported from the main entrypoint.

**logger**:
    Logger for this plugin, see [Logging Messages](#logging-messages).

**controller**, **host**, **instance**, **hooks** or **control**:
    The object the entrypoint is loaded for.
    Controller entrypoints get `controller`, host entrypoints get `host`, instance entrypoints get both `instance` and `host`, clusterioctl entrypoints get `hooks` and web entrypoints get `control`.

The available hooks are documented on the `ControllerHooks`, `HostHooks`, `InstanceHooks`, `CtlHooks` and `WebHooks` classes.
Handlers are attached under a name, which should be the name of the plugin, and a hook only accepts one handler per name.
An error thrown by a handler is logged and does not affect the other handlers, the same goes for handlers that take more than 15 seconds to complete.

For TypeScript the context types are exported as `ControllerPluginContext`, `HostPluginContext`, `InstancePluginContext`, `CtlPluginContext` and `WebPluginContext` from their respective packages.

### Class based plugins

Plugins written as classes deriving from `BaseControllerPlugin`, `BaseHostPlugin`, `BaseInstancePlugin`, `BaseCtlPlugin` or `BaseWebPlugin` still load, but this way of defining plugins is deprecated and a warning is logged when such a plugin is loaded.
The base class constructor attaches the `on*` methods overridden by the plugin to the corresponding hooks, and the async `init` method is called immediately after.
See [Migrating Plugins to 2.0](devs/plugin-migration-to-2.0.md) for how to convert them.


## Logging Messages

The load context provides a winston logger for logging messages to the shared cluster log.
For instances a copy of the log is also stored on the host the instance is on.
To use it, pass a string to one of the log levels functions, for example:

```js
export default async function(context) {
    context.logger.info("Initializing frobbing");
}
```

Metadata covering which plugin and instance (for instance entrypoints) the log event happened on will automatically be attached to the log message.

The available levels are fatal, error, warn, audit, info, and verbose.
These levels have roughly the following meanings:

- fatal: Unrecoverable error that prevents continuing execution of the Node.js process.
  Note that Plugins shouldn't have fatal errors.
- error: An unexpected error condition occured, usually an exception thrown when it wasn't expected.
  Should be investigated and fixed.
- warn: An potential error was detected, but operation continued gracefully.
  These may indicate that something is running sub-optimally, or is not working as intedend.
- audit: Record of actions performed by users of the cluster.
  This is usually emitted in response to management requests by a control link.
- info: A general log message.
  May indicae a status or operation completed.
  This should not be emitted by periodic tasks that run often.
- verbose: A verbose log message which might be useful when investigating issues, does not show by default.
  Note that logging messages is not a cheap operation even when the messages do not show up and spammy logging should be avoided for the verbose level.
  If the logs are still valuable in certain cases consider putting them behind a config option.

For guarding unexpected errors the best option is to log a short description along with the`stack` property of the Error:

```js
try {
    // Operation that should not throw but may end up throwing
} catch (err) {
    logger.error(`Operation failed:\n${err.stack}`);
}
```

For plugin hooks expections thrown are automatically catched and logged, but for event handlers registered on EventEmitters it's critical that exceptions are catched and handled appropriately.


## Plugin Configuration

Clusterio provides a configuration system that handles storing, distributing, editing and validating config fields for you.
Declare fields under `controllerConfigFields`, `hostConfigFields`, `instanceConfigFields`, or `ctlConfigFields` in the `plugin` export.
For example in index.js:

```ts
import type * as lib from "@clusterio/lib";

declare module "@clusterio/lib" {
    // Extend the interface defining available fields so that instance.config.get(...)
    // will have our own options.
    export interface ControllerConfigFields {
        "foo_frobber.level": number, // If optional is true then this should have | null in addition.
    }
}

export const plugin = {
    ...
    controllerConfigFields: {
        "foo_frobber.level": {
            description: "Level of frobnication done",
            type: "number",
            initialValue: 2,
        },
    }
} satisfies lib.PluginDeclaration;
```

The controller entrypoint will then be able to access the level config field through the `Config` object at `controller.config`:

```ts
export default async function(context: ControllerPluginContext) {
    const level = context.controller.config.get("foo_frobber.level");
    context.logger.info(`I got a frobnication level of ${level}`);
}
```

The same applies for instance configs, replace "controller" with "instance" where appropriate.
See [Configuration System](config-system.md) for more details on how this system works.


### Handling Invalid Configuration

If the plugin requires save patching, script commands or Lua UDP to function, which can be disabled to run vanilla or scenarios not compatible with Clusterio, declare it under `features` in the `plugin` export:

```js
export const plugin = {
    name: "foo_frobber",
    // ...
    features: ["SavePatching"],
};
```

Enabling the plugin on an instance that has the feature turned off then fails config validation, so the mistake is caught when the config is edited instead of when the instance starts.

Other constraints on config fields are expressed with a `validator` on the field definition, which throws when the value is not acceptable.
Use `dependsOn` when the check involves other fields so that the validator runs when those change too.
See [Configuration System](config-system.md) for details.


## Plugin Permissions

Permissions are granted to roles and checked on users, both on the controller and in the web UI.
A plugin declares the permissions it uses under `permissions` in the `plugin` export.
Names must start with the plugin name followed by a dot.
For example in index.ts:

```ts
import type * as lib from "@clusterio/lib";

declare module "@clusterio/lib" {
    // Extend the interface of known permission names so that
    // user.checkPermission(...) accepts our permission.
    export interface Permissions {
        "foo_frobber.frobnicate": never;
    }
}

export const plugin = {
    ...
    permissions: [
        {
            name: "foo_frobber.frobnicate",
            title: "Frobnicate",
            description: "Run frobnication on instances.",
            grantByDefault: false, // Whether the generated Player role is granted this permission
        },
    ],
} satisfies lib.PluginDeclaration;
```

The `Permissions` interface only exists at the type level, its values are always `never` and only the keys are used.
Adding your permission names to it gives you autocomplete and a compile error on typos when calling `user.checkPermission`, `account.hasPermission` or setting the `permission` of a web UI page.
In JavaScript the `declare module` block is not needed.

Permissions declared this way are registered when the plugin is loaded.
The older `lib.definePermission()` function still works, but a plugin should not use both for the same permission name.
Controller-side permission checks are authoritative; hiding a web UI control does not authorize or protect its underlying request.
Declare the [permission on control-origin messages](#static-permission) and test denied requests as well as allowed ones.


## Migrating Stored Data

Clusterio stores the config fields and permissions of plugins, so a plugin that renames them needs to migrate the stored values or users lose what they had set.
This is done from the migration entrypoint, which is loaded on the controller and hosts before any config or role is read, for all plugins in the plugin list including those disabled with `load_plugin`.
It is passed a context with `hooks` and attaches handlers for the data it needs to migrate:

```js
import * as lib from "@clusterio/lib";

export default function(context) {
    const { hooks, plugin } = context;
    // Plugin was renamed from frobber in 2.0.0
    const renameFields = config => lib.renameConfigPrefix(config, "frobber.", "foo_frobber.");
    hooks.controllerConfig.attach(plugin.name, renameFields);
    hooks.hostConfig.attach(plugin.name, renameFields);
    hooks.instanceConfig.attach(plugin.name, renameFields);
    hooks.role.attach(plugin.name, role => lib.renamePermissionPrefix(role, "frobber.", "foo_frobber."));
}
```

The handlers get the serialized config or role after the core migrations and return the migrated version.
They run synchronously, a handler that throws stops the controller or host from starting.
`lib.renameConfigField`, `lib.renameConfigPrefix`, `lib.renamePermission` and `lib.renamePermissionPrefix` cover the common case of renaming.

Handlers run every time the data is loaded and must leave data that is already migrated unchanged.
For changes that can't be repeated, controller, host, and instance configs have a hidden `<plugin>.version` field with the version of the plugin they were last loaded with, which is updated after the migrations have run:

```js
hooks.instanceConfig.attach(plugin.name, config => {
    const version = config["foo_frobber.version"];
    // The default delay went from 1 to 2 in 1.2.0, update configs still on the old default
    if (
        version && lib.integerFullVersion(version) < lib.integerFullVersion("1.2.0")
        && config["foo_frobber.delay"] === 1
    ) {
        config["foo_frobber.delay"] = 2;
    }
    return config;
});
```

The version field is missing or empty in configs saved before Clusterio tracked it, and in configs last loaded in recovery mode.

Roles have no version of their own.
They are stored on the controller, so use the version in the controller config, which is loaded before the roles:

```js
let controllerVersion;
hooks.controllerConfig.attach(plugin.name, config => {
    controllerVersion ??= config["foo_frobber.version"] ?? "";
    return config;
});
hooks.role.attach(plugin.name, role => {
    // foo_frobber.edit was split out of foo_frobber.view in 1.2.0
    if (
        controllerVersion && lib.integerFullVersion(controllerVersion) < lib.integerFullVersion("1.2.0")
        && role.permissions.includes("foo_frobber.view")
    ) {
        role.permissions.push("foo_frobber.edit");
    }
    return role;
});
```

Data the plugin stores itself, like files in the database directory or values in the save, is up to the plugin to migrate from its other entrypoints.
Modules don't need migrating, save patching replaces the files of every module when an instance starts.


## Communicating with Factorio

For pushing data into Factorio there's RCON, which lets you send arbitrary Lua commands to invoke whatever code you want in the game.
This is done by calling the `sendRcon` method on the instance.
For example:

```js
instance.hooks.start.attach(plugin.name, async () => {
    const response = await instance.sendRcon(
        "/sc rcon.print('data')"
    );

    // Do stuff with response.
});
```

Because data into Factorio is streamed at a rate of 3-6 kB/s by default, it is recommended to avoid sending large commands as much as possible, and to strip down the data on the ones you send to only what's strictly necessary.
You can have lua code injected into the game via the module system and call that from RCON to avoid having to send code through the commands.

For getting data out from Factorio there's both RCON and the `send_json` API of the Clusterio module.
Returning data via RCON is prefered if the action is initiated from the Node.js side.
The `send_json` API allows sending JSON payloads on channels that plugins can listen to.
From a plugin you listen for an event named `ipc-channel_name` in order to get data sent by `send_json`.
For example in the plugin code:

```js
export default async function(context) {
    const { instance, logger } = context;
    instance.server.on("ipc-my_plugin_foo", content =>
        handleFoo(content).catch(err => logger.error(
            `Error handling foo:\n${err.stack}`
        ))
    );
}

async function handleFoo(content) {
    // Do stuff with content
}
```

And then in the module for the plugin:

```lua
local clusterio_api = require("modules/clusterio/api")

-- inside some event handler
clusterio_api.send_json("my_plugin_foo", { data = 123 })
```

For convience a helper function is exposed on server specifically for handling IPC events.

```js
export default async function(context) {
    context.instance.server.handle("my_plugin_foo", handleFoo);
}

async function handleFoo(content) {
    // Do stuff with content
}
```

**Note:** This only handles IPC events, all other events must use `on`. You should not include the `ipc-` prefix when using this method. Error logging is handled for you, if you want custom error handling use `on`.

It's recommended to either use the plugin name as the channel name or to prefix the channel name with the name of the plugin if you need multiple channels.
With raw `server.on(...)` listeners, catch errors in the handler; `server.handle(...)` already handles error logging for IPC handlers.

Data out from Factorio does not have the same limits as data into Factorio, RCON responses can be in 100kB range without causing issues, and payloads to the `send_json` API can be in the 4MB range provided the server has a fast enough storage system.

**Note:** both `send_json` and RCON can operate out of order.
For `send_json` it's possible that payloads greater than 4kB are received after payloads that were sent at a later point in time.
For RCON, commands longer than 50 characters may end up being executed after shorter commands sent after it.

### Auto pause

Each RCON command makes a paused game run one tick, so commands sent on a timer keep a server with `auto_pause` on running while nobody is online.
`instance.autoPaused` is true while the instance was started with `auto_pause` on and no players are online, and the `autoPauseChanged` hook is invoked with the new value when it changes.
Skip periodic commands while it is true:

```js
instance.hooks.start.attach(plugin.name, async () => {
    timer = setInterval(() => {
        if (!instance.autoPaused) {
            instance.sendRcon("/sc my_plugin.update()").catch(handleError);
        }
    }, 1000);
});
```

Commands sent in response to events from elsewhere in the cluster are fine, they cost one tick each.
A game paused with `game.tick_paused` does not run ticks for commands.

### Communicating over UDP

Instances can optionally exchange UDP packets with the Factorio server for low latency fire-and-forget communication, using the `--enable-lua-udp` option of Factorio.
This needs Factorio 2.1.10 or later, earlier versions crash when a packet is received on a headless server.
This is disabled by default and enabled by setting `factorio.enable_lua_udp` on the instance.
Sending data out of Factorio also needs `factorio.enable_script_commands`, as the port to send to is passed to the module with a script command on startup.
Plugins requiring it should declare the `LuaUdp` feature.
Note that UDP does not guarantee delivery or ordering, and packets should be kept well below 32 kB.

Data out from Factorio is sent with the `send_udp` API of the Clusterio module.
Unlike `send_json` the payload is an arbitrary string, and it is delivered to plugins as a `udp-channel_name` event with the payload as a Buffer:

```lua
clusterio_api.send_udp("my_plugin_foo", "123")
```

```js
export default async function(context) {
    const { instance, logger } = context;
    instance.server.handleUdp("my_plugin_foo", async data => {
        logger.info(`got ${data.toString()}`);
    });
}
```

Like `handle` for `send_json`, `handleUdp` logs errors thrown by the handler.
Use `instance.server.on("udp-my_plugin_foo", ...)` for custom error handling.

Data into Factorio is sent with the `sendUdp` method on the instance and delivered in-game as `defines.events.on_udp_packet_received` events with the packet in `event.payload`.
The Clusterio module polls for received packets every tick, all modules and mods receive every packet so make sure the payload identifies your plugin.

```js
instance.hooks.start.attach(plugin.name, async () => {
    await instance.sendUdp("my_plugin_foo:123");
});
```


## Defining Link Messages

You will most likely have to communicate with the controller or other instances in your plugin for it to do anything useful.
For this there's a WebSocket communication channel established between the hosts and the controller that plugins can define their own messages to send over it.
This channel is bi-directional and all messages sent over it are validated with a JSON schema (see [this guide][guide] for an introduction to writing JSON schema).

[guide]: https://json-schema.org/learn/getting-started-step-by-step.html

There are currently two kinds of messages that can be defined: events and requests.
Events are simple one-way notifications that invoke a handler on the target it's sent to.
Requests are pairs of request and response messages where the request is sent to the target and the response is the reply back from the target.
The requests are similar to HTTP requests, except both parties of a link may initiate one.

Export message classes from `messages.js` and list them in the `plugin.messages` array in `index.js`, as in the [plugin structure example](#plugin-structure).
For example, `messages.js` can contain:

```js
export class Frobnicate {
    static plugin = "foo_frobber";
    static type = "request";
    static src = "instance";
    static dst = "controller";
    static jsonSchema = {
        type: "object",
        properties: {
            verbosity: { type: "integer" },
            special: { type: "boolean" },
        },
        required: ["verbosity"],
        additionalProperties: false,
    };
    constructor(json) {
        this.verbosity = json.verbosity;
        this.special = json.special;
    }
    static fromJSON(json) {
        return new Frobnicate(json);
    }
    toJSON() {
        return {
            verbosity: this.verbosity,
            special: this.special,
        };
    }
    static Response = {
        jsonSchema: {
            type: "object",
            properties: {
                report: {
                    type: "array",
                    items: { type: "string" },
                },
            },
            required: ["report"],
            additionalProperties: false,
        },
        fromJSON(json) {
            return json;
        },
    };
}
```

This specifies a request sent from an instance to the controller through its host.
The request data must contain the property `verbosity` with an integer number as the value, and it may also contain a boolean `special` property.
The response must contain a `report` property containing an array of strings.

The following properties are recognized as part of a message class:

#### static plugin

Must contain the name of the plugin this message is defined by.
This property is not used in core messages.

#### static type

A string describing the message type sent over the wire.
This can be either `"request"` or `"event"`.
The primary difference is whether they allow for a response.

#### static src

A string or an array of strings describing where this message is allowed to originate from.
The available endpoints are `"controller"`, `"host"`, `"instance"`, or `"control"`.
Messages sent from a different source will be rejected.

#### static dst

A string or array of strings describing where this message may be addressed to.
The available endpoints are `"controller"`, `"host"`, `"instance"`, or `"control"`.
If the target is multiple hops away it the message will be forwarded towards its destination by the intermediaries.
For example a message sent from a control connection an instance will be forwarded by the controller to the relevant host and from the host to the correct instance.
Messages sent to a different destination will be rejected.

#### static permission

Required for request and event classes whose `src` includes `"control"`.
Use a permission name, a synchronous permission-checking function, or `null` when every authenticated control client is deliberately allowed to send the message.
Registration rejects control-origin messages with no permission declaration.

A callback receives `(user, message)`, where `message` is the raw wire envelope.
It runs on the controller before destination payload validation and deserialization, so `message.data` is untrusted and may be missing or malformed.
Check any required payload shape before inspecting fields, and use `user.checkPermission(...)` or throw a `ClientError` such as `PermissionError` for a normal denial.
For example, this checks the payload shape needed to choose a permission; the destination schema still validates the full payload:

```js
import { ValidationError } from "@clusterio/lib";

function checkFrobnicatePermission(user, message) {
    const data = message.data;
    if (!data || typeof data !== "object" || typeof data.special !== "boolean") {
        throw new ValidationError("Expected a boolean special field");
    }
    user.checkPermission(data.special ? "foo_frobber.special" : "foo_frobber.frobnicate");
}
```

To use this callback, declare both permissions, require `special` in the message schema, and set `static permission = checkFrobnicatePermission` on a class that accepts `"control"` sources.
The `Frobnicate` example above only accepts instances and therefore does not need a permission declaration.
Do not make permission callbacks async: their result is not awaited, and returning `false` does not deny access.
Unexpected exceptions are contained and logged as errors; they are not a substitute for intentional permission denials.

#### static jsonSchema

A JSON schema that specifies what's valid to send in the message.

#### toJSON

Optional method to serialize an instance of the class to a JSON object, see the [documentation for `JSON.stringify()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify#description) for the semantics of this method.
This should return an object that's valid according to `jsonSchema` and accepted by `fromJSON`, and may be omitted if the default serialisation by `JSON.stringify()` already does this.

#### static fromJSON

Function to restore the class from a JSON object; it must be provided together with `jsonSchema` for messages carrying data.
Messages without data can omit both and are constructed without arguments.
This function is called when the message reaches its destination and is expected to return an identical instance of the class.
I.e. the result of `fromJSON(JSON.parse(JSON.stringify(instance)))` should be a complete copy of the message.

#### static Response

Optional [JSON sterilisable class](/docs/devs/json-serialisable-classes.md) that specifies the response to the request.
Required properties are `jsonSchema` and `fromJSON`.

Test JSON round trips, invalid payloads, allowed source/destination types, and authorization failures when changing messages.
Keep registration, schemas, serialization, permissions, and handlers consistent.


## Sending Link Messages

Link messages are sent by calling the `.sendTo()` method on a connection object with the destination and the data you want to send, where the data is an instance of one of the defined message classes in plugin.messages.
For example:

```js
import * as messages from "./messages.js";

// In an instance entrypoint
async function frobnicate(instance) {
    const response = await instance.sendTo("controller", new messages.Frobnicate({
        verbosity: 2,
        special: false,
    }));
    console.log(response); // { report: [...] }
}
```
Register the corresponding handler on the controller:

```js
import * as messages from "./messages.js";

export default async function({ controller }) {
    controller.handle(messages.Frobnicate, async request => ({
        report: [`Frobnication verbosity: ${request.verbosity}`],
    }));
}
```

The send and handle methods depend on `this`.
Call them on their connection object, or explicitly bind them when passing a method as a value:

```js
const send = instance.sendTo.bind(instance);
const response = await send("controller", new messages.Frobnicate({ verbosity: 2, special: false }));
```

Extracting an unbound method loses its receiver; storing a returned response does not.
For classes with `static type = "request"` the send method is async and returns a promise that resolves to the response data received from the target it was sent to, or rejects with an error if the request failed.
The destination can identify a particular control, host, or instance.
The `allHosts`, `allControls`, and `allInstances` broadcast destinations are for events, not requests.

Example destinations:

```js
connection.sendTo("controller", message);
connection.sendTo("allHosts", event);
connection.sendTo("allControls", event);
connection.sendTo("allInstances", event);
connection.sendTo({ hostId: 123 }, message);
connection.sendTo({ instanceId: 123 }, message);
connection.sendTo({ controlId: 123 }, message);
```

### Request errors

An error thrown by a request handler is sent back to the requester, and the promise returned by `.sendTo()` rejects with an instance of the same class.
The error classes follow the HTTP status codes:

```
Error
├── ClientError (4xx)
│   ├── ValidationError (400)
│   │   ├── InvalidValueError
│   │   └── InvalidFieldError
│   ├── ForbiddenError (403)
│   │   ├── PermissionError
│   │   └── InvalidAccessError
│   ├── NotFoundError (404)
│   ├── TimeoutError (408)
│   └── ConflictError (409)
└── ServerError (5xx)
    ├── InternalServerError (500)
    └── UnavailableError (503)
```

Throw a `ClientError` subclass when the request can't be done as asked, and `UnavailableError` when the host or instance that should handle it is not connected or not running, for example:

```js
controller.handle(messages.Frobnicate, async request => {
    if (!frobnicators.has(request.id)) {
        throw new lib.NotFoundError(`Frobnicator ${request.id} does not exist`);
    }
    // ...
});
```

Errors of a class that is not registered are logged as unexpected by the side that threw them and are received as `InternalServerError`.
Registered errors are not logged.
The message and stack trace of the original error are kept in both cases.

Plugins can define their own error classes by listing them in `errors` of the plugin declaration.
They need a static `plugin` property set to the plugin name and must take the message as the first constructor argument:

```js
export class FrobnicatorJammed extends lib.ConflictError {
    static plugin = "foo_frobber";
}
```

A subclass that is not registered is received as its closest registered parent class.

### Handling connection events

There are a few connection related events that plugins neeed to repsond to in order to avoid data loss and connection problems.
The most important is the prepare disconnect for the link between controller and host.
This is signaled to controller plugins via the `prepareHostDisconnect` hook and to instance plugins via the `prepareControllerDisconnect` hook.

After the prepare disconnect the connection will be closed, which will result in pending requests and events being dropped.
Plugins must respond to the prepare disconnect by stopping any processess it does that send events or requests over the link in question.
This can be accomplished either through listening for the prepare disconnect hook, or by checking the `connected` property of the `HostConnection` class and `Host` class on the controller and host respectively.
For example, an instance plugin can avoid starting a request while the host is disconnected or preparing to disconnect:

```js
if (host.connected) {
    await instance.sendTo("controller", new messages.Frobnicate({ verbosity: 2, special: false }));
}
```

If the event or request needs to be sent to the controller it can be put into a queue and sent out when the connection is established again.
The connection hooks report `connect` for a new session and `resume` for a retained session.
Keep application queues separate from messages already accepted into the connector buffer; blindly sending both again can duplicate an operation.

The second connection event which is of lesser importance to respond to is the `drop` connection event served through `controllerConnectionEvent` for instance plugins and through `hostConnectionEvent` for controller plugins.
This is raised when the connection between the controller and host in question is lost, most likely due to networking issues.
When in the dropped state the host retries the connection and attempts to resume the retained session.
Successful resumption replays unacknowledged buffered messages; expiry, invalidation, or buffer overflow can instead end the session.
This is not an application-level exactly-once guarantee; stateful plugins need their own recovery and duplicate-handling strategy.
However while in the dropped state any requests and events sent gets queued up in memory until the connection is either re-established or the session times out.
This means that if your plugin sends a lot of events or requests, they can end up being queued up in a buffer and sent out all at once the connection is re-estabished.
To avoid this you should be throtteling and/or stopping your requests/events after `drop` has been raised, and continue back as normal when `resume` is raised.


## Collecting Statistics

Clusterio comes with its own Prometheus client implementation, one part due to Not Invented Here and another part due to collectors in prom-client being difficult to get to work nicely with collecting data from plugins optionally loaded at runtime on different computers.

In its simplest form collecting data from plugins consists of defining the metric and updating it somewhere in the plugin code.
For example:

```js
import { Counter } from "@clusterio/lib";

const fooMetric = new Counter(
    "clusterio_foo_frobber_foo_metric", "Measures the level of foo",
);

// Somewhere in the controller plugin code
fooMetric.inc();
```

This works for controller plugins, and the metric will be automatically available through the /metrics HTTP endpoint.
It's recommended that plugin metrics follow `clusterio_<plugin_name>_<metric_name>` as the naming scheme.

For metrics that are per-instance, you must define an `instance_id` label and set it accordingly, for example:

```js
import { Gauge } from "@clusterio/lib";

const barMetric = new Gauge(
    "clusterio_foo_frobber_bar_metric", "Bar instance level",
    { labels: ["instance_id"] }
);

// Somewhere in the instance plugin code
barMetric.labels(String(instance.id)).set(someValue);
```

Metrics are automatically registered to the default registry, and this default registry is automatically polled by the controller on hosts.
This means that it's important that you place the definition of the metric at module level so that it's not created more than once over the lifetime of a host.
Since the metrics remember their values and would continue to be exported after an instance is shutdown, there's code at instance shutdown that removes all the values where the `instance_id` label matches the id of the instance shut down.

For statistics you need to update on collection there's a `metrics` hook on the controller, host and instance that is run before the metrics in the default registry are collected.

```js
import { Gauge } from "@clusterio/lib";

const frobnicationsMetric = new Gauge(
    "clusterio_foo_frobber_frobnications", "Frobnications currently in progress"
);

// In the controller entrypoint
controller.hooks.metrics.attach(plugin.name, async () => {
    frobnicationsMetric.set(frobnications.size);
});
```

A handler may also return the results of collecting collectors that are not in the default registry, and they are added to the response.

```js
import { CollectorRegistry, Counter } from "@clusterio/lib";

const fooRegistry = new CollectorRegistry();
const fooCounter = new Counter("clusterio_foo_frobber_foos", "Foos frobbed", { register: false });
fooRegistry.register(fooCounter);

controller.hooks.metrics.attach(plugin.name, async () => fooRegistry.collect());
```


## Adding Custom Commands to clusterioctl

The ctl entrypoint for plugins allows you to extend clustectl with your own commands.
The creation of custom commands typically starts with defining a command tree for the plugin:

```js
import { Command, CommandTree } from "@clusterio/lib";
const fooFrobberCommands = new CommandTree({
    name: "foo-frobber", description: "Foo Frobber Plugin commands"
});
```

Then commands are added to the the plugin's command tree:

```js
import * as messages from "./messages.js";

fooFrobberCommands.add(new Command({
    definition: ["frobnicate <type>", "Do frobnications", (yargs) => {
        yargs.positional("type", {
            describe: "type of frobnication", type: "string"
        });
    }],
    handler: async function(args, ctl) {
        await ctl.sendTo("controller", new messages.FrobnicateRequest(args.type));
    },
}));
```

For a command the `definition` is the arguments to pass to [yargs.command](http://yargs.js.org/docs/#api-reference-commandcmd-desc-builder-handler) (see also [yargs.positional](http://yargs.js.org/docs/#api-reference-positionalkey-opt) and [yargs.options](http://yargs.js.org/docs/#api-reference-optionskey-opt) for setting up positional and optional arguments to commands).
The `handler` is an async function that's invoked when the command is executed and it's passed the parsed command line arguments and a reference to the `Ctl` class of clusterioctl.
Note that messages sent from clusterioctl needs to have `"control"` as a part of `src` for it to be accepted by the controller, see [Defining Link Messages](#defining-link-messages) for how to define the messages that can be sent to the controller.

To have the command tree become part of clusterioctl it needs to be added to the rootCommand tree in the `addCommands` hook of the ctl entrypoint:

```js
export default async function(context) {
    context.hooks.addCommands.attach(context.plugin.name, async (rootCommand) => {
        rootCommand.add(fooFrobberCommands);
    });
}
```