diff --git a/README.md b/README.md index 381a0657..d0ecb813 100644 --- a/README.md +++ b/README.md @@ -86,6 +86,8 @@ There's also plugins developed and maintained by the community: Want to make your own plugin? Check out the documentation on [Writing Plugins](/docs/writing-plugins.md) for where to start. +If you have a plugin written for an earlier 2.0 alpha, see [Migrating Plugins to 2.0](/docs/devs/plugin-migration-to-2.0.md). +Note that from 2.0 onwards minor versions of Clusterio may contain breaking changes for plugins, while breaking changes for users running a cluster are held back for major versions. ## Installation diff --git a/docs/decisions.md b/docs/decisions.md index 67d32831..37ec7467 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -361,16 +361,16 @@ While we have considered alternatives since, transitioning to another solution w ### Breaking Changes -🧾 Time Immemorial +🧾 7th October 2026 -> We follow symver versioning for breaking changes. -> In our alpha stage, we allow every alpha to contain breaking changes as long as their are documented and provide migration actions. -> Once we enter beta, breaking changes will be more restricted. +> From 2.0.0 onwards, major versions contain breaking changes for end users, while minor versions may contain breaking changes for the plugin API. +> All breaking changes are documented in the changelog along with migration actions. We aim to be highly transparent with breaking changes and maintain compatibility with as many existing installations as possible. -However, as we are currently in the alpha stage, things are unstable, and breaking changes occur frequently. -As such, we have decided to document all breaking changes and provide migration steps where the changes are non-trivial. -Once we enter the beta stage, we will implement stricter restrictions on breaking changes. +During the alpha stage every alpha was allowed to contain breaking changes, as long as they were documented and provided migration actions. +From 2.0 onwards changes which require action from people running a cluster, such as changes to configs, stored data or the command line, are held back for the next major version. +Changes to the plugin API may still be made in minor versions. +Plugin authors should check the changelog when a new minor version is released, non-trivial changes come with a migration guide such as [Migrating Plugins to 2.0](/docs/devs/plugin-migration-to-2.0.md). ### Distribution diff --git a/docs/devs/plugin-migration-to-2.0.md b/docs/devs/plugin-migration-to-2.0.md new file mode 100644 index 00000000..4faec692 --- /dev/null +++ b/docs/devs/plugin-migration-to-2.0.md @@ -0,0 +1,327 @@ +# Migrating Plugins to 2.0 + +This guide covers the changes needed to bring a plugin written for 2.0.0-alpha.27 up to 2.0.0. +If your plugin targets an older alpha, go through the Breaking Changes sections in the [changelog](/CHANGELOG.md) for the versions in between first. +Plugins still on alpha 13 should start with the [alpha 14 guide](plugin-migration-to-alpha-14.md). + + +## Versioning from 2.0 + +From 2.0.0 onwards a major version (3.0.0) contains breaking changes for users running a cluster, while a minor version (2.1.0) may contain breaking changes for the plugin API. +This means a plugin written for 2.0 is not guaranteed to work with 2.1 without changes. +Check the Breaking Changes section of the changelog when a new minor version is released and publish an update to your plugin if needed. +See [Breaking Changes](/docs/decisions.md#breaking-changes) in the decision log. + + +## Convert to ES modules + +Clusterio's packages are now ES modules and plugins should be too. +Add `"type": "module"` to the plugin's `package.json` and replace `require()` and `module.exports` with `import` and `export`. + +Relative imports must include the full filename with the `.js` extension. +This also applies in TypeScript, where you import `./messages.js` even though the source file is `messages.ts`. +Importing a directory no longer resolves to its `index.js`. + +**Alpha 27** +```js +"use strict"; +const lib = require("@clusterio/lib"); +const messages = require("./messages"); + +const plugin = { + name: "foo_frobber", + controllerEntrypoint: "controller", + messages: [messages.Frobnicate], +}; + +module.exports = { + plugin, +}; +``` + +**2.0** +```js +import * as lib from "@clusterio/lib"; +import * as messages from "./messages.js"; + +export const plugin = { + name: "foo_frobber", + controllerEntrypoint: "controller.js", + messages: [messages.Frobnicate], +}; +``` + +The entrypoint paths in the plugin declaration must include the `.js` extension as well, for TypeScript plugins this means `"./dist/node/controller.js"` instead of `"./dist/node/controller"`. + +`__dirname` and `__filename` do not exist in ES modules, use `import.meta.dirname` and `import.meta.filename` instead. +JSON files are imported with `import packageJson from "./package.json" with { type: "json" };`. + +A `webpack.config.js` written with `require()` stops working once the package is an ES module. +Rename it to `webpack.config.cjs`, webpack-cli finds it under that name without further changes. +If your `prepare` script passes `--config webpack.config.js` update it to the new name. + +TypeScript plugins using `"module": "node16"` in their tsconfig, as the plugin templates do, emit ES modules once `"type": "module"` is set and need no further config changes. + +Node.js can still import a CommonJS plugin, but it only finds the `plugin` export if it can detect it from the source, which works for `module.exports = { plugin };` and fails for other forms such as an object literal written in place. +Converting the plugin avoids the problem. + + +## Replace plugin classes with entrypoint functions + +Entrypoints now export a default async function instead of a class deriving from `BaseControllerPlugin`, `BaseHostPlugin`, `BaseInstancePlugin`, `BaseCtlPlugin` or `BaseWebPlugin`. +The function is called once when the plugin loads and attaches handlers to the hooks it needs. +Class exports still load, but they are deprecated and a warning is logged for each one. +See [Defining the plugin entrypoint](/docs/writing-plugins.md#defining-the-plugin-entrypoint) for the details. + +**Alpha 27** +```js +const { BaseControllerPlugin } = require("@clusterio/controller"); + +class ControllerPlugin extends BaseControllerPlugin { + async init() { + this.controller.handle(Frobnicate, this.handleFrobnicate.bind(this)); + } + + async onSaveData() { + await this.saveFrobnicationData(); + } + + async onInstanceStatusChanged(instance, prev) { + this.logger.info(`Instance ${instance.id} went from ${prev} to ${instance.status}`); + } + + async handleFrobnicate(request) { /* ... */ } +} + +module.exports = { + ControllerPlugin, +}; +``` + +**2.0 (JS)** +```js +export default async function(context) { + const { controller, logger, plugin } = context; + + controller.handle(Frobnicate, async (request) => { /* ... */ }); + + controller.hooks.save.attach(plugin.name, async () => { + await saveFrobnicationData(); + }); + + controller.hooks.instanceStatusChanged.attach(plugin.name, async (instance, prev) => { + logger.info(`Instance ${instance.id} went from ${prev} to ${instance.status}`); + }); +} +``` + +**2.0 (TS)** +```ts +import type { ControllerPluginContext } from "@clusterio/controller"; + +export default async function(context: ControllerPluginContext) { + // Same as above +} +``` + +The code from `init()` goes into the body of the function, and state that lived on `this` becomes local variables. +The context types are `ControllerPluginContext`, `HostPluginContext`, `InstancePluginContext`, `CtlPluginContext` and `WebPluginContext` from the respective packages. + + +### Keeping a class for state + +If the plugin has enough state that a class is still the better fit, give the class a private constructor and an async `static fromContext` that creates it from the context and attaches its hooks. +`fromContext` can then be the default export of the entrypoint. +Instances are kept in a static `WeakMap` keyed by the controller, host or instance they belong to, which lets other code look them up without holding a reference to them. + +```ts +import type { Controller, ControllerPluginContext } from "@clusterio/controller"; +import type * as lib from "@clusterio/lib"; + +export class Frobber { + private static instances = new WeakMap(); + private frobnications = new Map(); + + private constructor( + private controller: Controller, + private logger: lib.Logger, + ) { } + + static async fromContext(context: ControllerPluginContext) { + const frobber = new Frobber(context.controller, context.logger); + await frobber.load(); + context.controller.hooks.save.attach(context.plugin.name, () => frobber.save()); + Frobber.instances.set(context.controller, frobber); + return frobber; + } + + static get(controller: Controller) { + return Frobber.instances.get(controller); + } + + async load() { /* ... */ } + async save() { /* ... */ } +} + +export default Frobber.fromContext; +``` + +The loader calls the default export without `this`, so refer to the class by name inside `fromContext` rather than using `this`. +Don't extend the deprecated base classes for this, they attach `on*` methods to hooks themselves and log a warning. + + +### Method to hook names + +Each `on*` method has a hook of the same name without the `on` prefix and with the first letter lowercased, so `onInstanceStatusChanged` becomes `controller.hooks.instanceStatusChanged`. +The exceptions are: + +- `onSaveData` on the controller is `controller.hooks.save`. +- `addCommands` on ctl plugins is `context.hooks.addCommands`. +- The `pages`, `loginForms` and `inputComponents` properties of web plugins are the `pages`, `loginForms` and `inputComponents` hooks on `control.hooks`, with handlers that return the value. +- The `componentExtra` property of web plugins is the `control.hooks.extensionComponents` hook. + +Controller hooks are on `controller.hooks`, host hooks on `host.hooks`, instance hooks on `instance.hooks`, and web hooks on `control.hooks`. +The ctl hooks are passed directly as `context.hooks`. + +**Alpha 27** +```jsx +class WebPlugin extends BaseWebPlugin { + async init() { + this.pages = [ + { path: "/foo_frobber", sidebarName: "Foo Frobber", content: }, + ]; + } +} +``` + +**2.0** +```jsx +export default async function(context) { + const { control, plugin } = context; + + control.hooks.pages.attach(plugin.name, () => [ + { path: "/foo_frobber", sidebarName: "Foo Frobber", content: }, + ]); +} +``` + + +### Replacing other class members + +The class properties are on the context object passed to the entrypoint: + +- `this.info` is `context.plugin`. +- `this.logger` is `context.logger`. +- `this.controller`, `this.host`, `this.instance` and `this.control` are `context.controller`, `context.host`, `context.instance` and `context.control`. +- `this.metrics` on controller plugins is `context.metrics`. +- `this.package` on web plugins is `context.package`. + +The helper methods of the base classes have the following replacements: + +- `this.sendRcon(message, expectEmpty)` is `instance.sendRcon(message, expectEmpty, plugin.name)`. + The plugin name is used to label the RCON metrics. +- `this.broadcastEventToHosts(event)` is `controller.sendTo("allHosts", event)`. +- `this.sendOrderedRcon(message)` has no replacement. + If your plugin needs commands to execute in order, queue them in the plugin and send the next one after the previous one resolves. + + +### Changes to hook behaviour + +Handlers attached by different plugins to the same hook now run concurrently, where the `on*` methods were called one plugin at a time. +Don't rely on another plugin's handler having finished before yours runs. + +A handler that throws or takes more than 15 seconds is logged and does not prevent the other handlers from running. +Each hook accepts one handler per name, attach it under `plugin.name`. + + +## Accessing other plugins + +The `plugins` map on `Controller`, `Host` and `Instance` has been removed. +It's replaced with `loadedPlugins`, a set of the plugin info objects for the plugins that loaded. +The set holds info objects rather than names, so look the info up in `pluginInfos` by name first to check if a plugin is loaded on the controller or host: + +```js +const info = controller.pluginInfos.find(info => info.name === "other_plugin"); +const loaded = info !== undefined && controller.loadedPlugins.has(info); +``` + +In the web interface `control.loadedPlugins` is a map from plugin name to plugin info. +`control.plugins` is deprecated and only holds plugins still using the class export. + +With no plugin class instances, `controller.plugins.get("other_plugin")` has no direct replacement. +If one plugin needs to call into another, have the other plugin export its class and use the static lookup from [Keeping a class for state](#keeping-a-class-for-state): + +```ts +import { Frobber } from "other_plugin/dist/node/controller.js"; + +const frobber = Frobber.get(controller); +if (!frobber) { + throw new Error("other_plugin is not loaded"); +} +``` + +This only works if both plugins resolve `other_plugin` to the same file on disk, otherwise the lookup goes to a different copy of the class and returns undefined. + +For state shared between components of a web plugin, assign it to a module level variable in the entrypoint. +The web plugin template created by `@clusterio/create` shows this for a subscriber. + + +## Declare permissions on the plugin + +Permissions can now be declared under `permissions` in the plugin declaration instead of with `lib.definePermission()`. +`lib.definePermission()` still works, but don't use both for the same permission name. + +For TypeScript plugins `user.checkPermission()`, `account.hasPermission()` and the `permission` of web pages now only accept known permission names. +Add your permission names to the `Permissions` interface or these calls will fail to compile, even if you keep using `lib.definePermission()`: + +```ts +declare module "@clusterio/lib" { + export interface Permissions { + "foo_frobber.frobnicate": never; + } +} +``` + +The plugin template's `tsconfig.browser.json` now includes `index.ts` so the web build also sees the declaration, add it to the `include` list if your plugin has its own tsconfig for the browser. +See [Plugin Permissions](/docs/writing-plugins.md#plugin-permissions). + + +## Renamed ctl classes and config + +The remaining uses of "control" for clusterioctl were renamed to "ctl": + +- `controlConfigFields` in the plugin declaration is `ctlConfigFields`. + Fields under the old name are silently ignored. +- The `ControlConfigFields` interface is `CtlConfigFields`. +- `lib.ControlConfig` is `lib.CtlConfig`. +- The `Control` class exported by `@clusterio/ctl` is `Ctl`. + +The term "control" is still used in message `src` and `dst` and in config field locations, where it covers both ctl and the web interface. + + +## Static assets in the web interface + +`staticRoot` now includes the `static/` prefix. +Plugins that fetch assets with `${staticRoot}static/...` should drop the `static/` part. + + +## Package dependencies + +npm installs peer dependencies by default, so listing `@clusterio/web_ui` under `peerDependencies` installs it everywhere the plugin is installed even though it's only needed to build the web bundle. +Move `@clusterio/web_ui` to `devDependencies` and mark `@clusterio/controller`, `@clusterio/host` and `@clusterio/ctl` as optional in `peerDependenciesMeta`: + +```json +"peerDependencies": { + "@clusterio/controller": "^2.0.0", + "@clusterio/host": "^2.0.0", + "@clusterio/lib": "^2.0.0" +}, +"peerDependenciesMeta": { + "@clusterio/controller": { + "optional": true + }, + "@clusterio/host": { + "optional": true + } +} +``` diff --git a/docs/managing-a-cluster.md b/docs/managing-a-cluster.md index 06483749..66aaa3ff 100644 --- a/docs/managing-a-cluster.md +++ b/docs/managing-a-cluster.md @@ -5,7 +5,7 @@ This document uses the shorthand `ctl> foo` to indicate `npx clusterioctl foo` s Mandatory parameters are shown in `` and optional pameters are in `[square brackets]`. Before `clusterioctl` can be used it needs to be configured for the cluster it will connect to. -The easiest way to do this is to run `npx clusteriocontroller bootstrap create-ctl-config ` on the controller, which creates the necessary `config-control.json` for managing the cluster as the given user. +The easiest way to do this is to run `npx clusteriocontroller bootstrap create-ctl-config ` on the controller, which creates the necessary `config-ctl.json` for managing the cluster as the given user. ## Hosts diff --git a/docs/writing-plugins.md b/docs/writing-plugins.md index 5b9c02f8..bf878972 100644 --- a/docs/writing-plugins.md +++ b/docs/writing-plugins.md @@ -7,7 +7,7 @@ Each entrypoint of a plugin exports a function that attaches handlers to hooks c ## Contents - [Plugin Structure](#plugin-structure) -- [Defining the plugin class](#defining-the-plugin-class) +- [Defining the plugin entrypoint](#defining-the-plugin-entrypoint) - [Logging Messages](#logging-messages) - [Plugin Configuration](#plugin-configuration) - [Plugin Permissions](#plugin-permissions) @@ -162,6 +162,7 @@ For TypeScript the context types are exported as `ControllerPluginContext`, `Hos Plugins written as classes deriving from `BaseControllerPlugin`, `BaseHostPlugin`, `BaseInstancePlugin`, `BaseCtlPlugin` or `BaseWebPlugin` still load, but this way of defining plugins is deprecated and a warning is logged when such a plugin is loaded. The base class constructor attaches the `on*` methods overridden by the plugin to the corresponding hooks, and the async `init` method is called immediately after. +See [Migrating Plugins to 2.0](devs/plugin-migration-to-2.0.md) for how to convert them. ## Logging Messages @@ -176,7 +177,7 @@ export default async function(context) { } ``` -Metadata covering which plugin and instance (for instance plugin classes) the log event happened on will automatically be attached to the log message. +Metadata covering which plugin and instance (for instance entrypoints) the log event happened on will automatically be attached to the log message. The available levels are fatal, error, warn, audit, info, and verbose. These levels have roughly the following meanings: @@ -226,7 +227,7 @@ declare module "@clusterio/lib" { } } -export default { +export const plugin = { ... controllerConfigFields: { "foo_frobber.level": { @@ -288,7 +289,7 @@ declare module "@clusterio/lib" { } } -export default { +export const plugin = { ... permissions: [ { @@ -592,7 +593,7 @@ const fooMetric = new Counter( fooMetric.inc(); ``` -This works for controller plugins, and the metric will be automatically available through the /metric HTTP endpoint. +This works for controller plugins, and the metric will be automatically available through the /metrics HTTP endpoint. It's recommended that plugin metrics follow `clusterio__` as the naming scheme. For metrics that are per-instance, you must define an `instance_id` label and set it accordingly, for example: @@ -656,26 +657,23 @@ const fooFrobberCommands = new CommandTree({ Then commands are added to the the plugin's command tree: ```js -import { messages } from "./index.js"; +import * as messages from "./messages.js"; fooFrobberCommands.add(new Command({ definition: ["frobnicate ", "Do frobnications", (yargs) => { - yargs.positional("level", { + yargs.positional("type", { describe: "type of frobnication", type: "string" }); }], handler: async function(args, ctl) { - await info.messages.frobnicate.send(ctl, { - instance_name: "Console", - content: args.message, - }); + await ctl.sendTo("controller", new messages.FrobnicateRequest(args.type)); }, })); ``` For a command the `definition` is the arguments to pass to [yargs.command](http://yargs.js.org/docs/#api-reference-commandcmd-desc-builder-handler) (see also [yargs.positional](http://yargs.js.org/docs/#api-reference-positionalkey-opt) and [yargs.options](http://yargs.js.org/docs/#api-reference-optionskey-opt) for setting up positional and optional arguments to commands). The `handler` is an async function that's invoked when the command is executed and it's passed the parsed command line arguments and a reference to the `Ctl` class of clusterioctl. -Note that messages sent from clusterioctl needs to have `"control-controller"` as a part of the links array for it to be accepted by the controller, see [Defining Link Messages](#defining-link-messages) for how to define the messages that can be sent to the controller. +Note that messages sent from clusterioctl needs to have `"control"` as a part of `src` for it to be accepted by the controller, see [Defining Link Messages](#defining-link-messages) for how to define the messages that can be sent to the controller. To have the command tree become part of clusterioctl it needs to be added to the rootCommand tree in the `addCommands` hook of the ctl entrypoint: