From b0f6c95fcab5a62c2ee917a0ad02adbd2558d6d7 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Thu, 18 Jan 2024 20:31:52 +0100 Subject: [PATCH 01/84] Move export for plugin declaration (#555) Change the expected exported name of plugin declarations to be "plugin" at the main entrypoint instead of the default export. This makes the plugin's code organisation more flexible especially when using CommonJS JavaScript modules as they can export additional symbols from the plugin declaration file. --- docs/config-system.md | 2 +- docs/writing-plugins.md | 26 +++++++++---------- .../controller/src/BaseControllerPlugin.ts | 4 +-- packages/ctl/src/BaseCtlPlugin.ts | 2 +- packages/host/src/BaseHostPlugin.ts | 2 +- packages/host/src/BaseInstancePlugin.ts | 2 +- packages/lib/src/plugin.ts | 3 +-- packages/lib/src/plugin_loader.ts | 8 +++++- packages/lib/src/shared_commands.ts | 8 +++++- packages/web_ui/src/bootstrap.tsx | 2 +- plugins/global_chat/{info.ts => index.ts} | 4 +-- plugins/global_chat/package.json | 2 +- plugins/global_chat/test/plugin.js | 2 +- plugins/global_chat/webpack.config.js | 2 +- plugins/inventory_sync/{info.ts => index.ts} | 4 +-- plugins/inventory_sync/package.json | 2 +- plugins/inventory_sync/webpack.config.js | 2 +- plugins/player_auth/{info.ts => index.ts} | 4 +-- plugins/player_auth/package.json | 2 +- plugins/player_auth/test/plugin.js | 2 +- plugins/player_auth/webpack.config.js | 2 +- plugins/research_sync/{info.ts => index.ts} | 4 +-- plugins/research_sync/package.json | 2 +- plugins/research_sync/webpack.config.js | 2 +- .../statistics_exporter/{info.ts => index.ts} | 4 +-- plugins/statistics_exporter/package.json | 2 +- plugins/statistics_exporter/test/plugin.js | 2 +- plugins/statistics_exporter/webpack.config.js | 2 +- .../subspace_storage/{info.ts => index.ts} | 4 +-- plugins/subspace_storage/package.json | 2 +- plugins/subspace_storage/webpack.config.js | 2 +- test/integration/index.js | 10 +++---- test/lib/plugin_loader.js | 10 +++---- 33 files changed, 71 insertions(+), 62 deletions(-) rename plugins/global_chat/{info.ts => index.ts} (85%) rename plugins/inventory_sync/{info.ts => index.ts} (96%) rename plugins/player_auth/{info.ts => index.ts} (95%) rename plugins/research_sync/{info.ts => index.ts} (89%) rename plugins/statistics_exporter/{info.ts => index.ts} (92%) rename plugins/subspace_storage/{info.ts => index.ts} (97%) diff --git a/docs/config-system.md b/docs/config-system.md index a1f4018a..e822e2b7 100644 --- a/docs/config-system.md +++ b/docs/config-system.md @@ -6,7 +6,7 @@ Clusterio uses a field based configuration system to manage settings for the con Plugins can add their own config fields for the controller and instances configs. The fields are defined early in the startup, and fields for disabled plugins will still be created. -The built-in config fields are defined in [packages/lib/config/definitions.js](/packages/lib/config/definitions.js) while plugins can add their own config field definitions to the `controllerConfigFields` and `instanceConfigFields` properties of the `info.js` export. +The built-in config fields are defined in [packages/lib/config/definitions.js](/packages/lib/config/definitions.js) while plugins can add their own config field definitions to the `controllerConfigFields` and `instanceConfigFields` properties of the `plugin` export. A `.load_plugin` field is automatically created for each plugin and it's used to enable/disable the loading plugins on startup. diff --git a/docs/writing-plugins.md b/docs/writing-plugins.md index 5aab0f3a..2b7e6566 100644 --- a/docs/writing-plugins.md +++ b/docs/writing-plugins.md @@ -25,7 +25,7 @@ The plugin classes have pre-defined hooks that are called during various stages The basic file structure of a plugin is the following. plugin_name/ - +- info.js + +- index.js +- package.json +- controller.js +- instance.js @@ -44,14 +44,12 @@ A possible workflow for developing plugins is to place the plugin in a sub-direc To add it to `plugin-list.json` so that it gets loaded use the `plugin add ` sub-command to either clusteriocontroller, clusteriohost or clusterioctl. Note that it's important that the path starts with ./ or ../ (use .\ or ..\ on Windows). -For a plugin the most important file is the `info.js` file. -Without it the plugin will not recognized by Clusterio. -Here's an example of it: +For a plugin to be recognised by Clusterio it needs to export an entry named `plugin` from its main entrypoint. +By default the main entrypoint is the `index.js` file, but this may be changed by setting the `"main"` entry to a different file in `package.json`. +Here's an example of `index.js`: ```js -const lib = require("@clusterio/lib"); // For messages - -module.exports = { +module.exports.plugin = { name: "foo_frobber", title: "Foo Frobber", description: "Does advanced frobnication", @@ -202,8 +200,8 @@ For plugin hooks expections thrown are automatically catched and logged, but for ## Plugin Configuration Clusterio provides a configuration system that handles storing, distributing, editing and validating config fields for you. -You can take advantage of it by declaring config fields under either `controllerConfigFields` or `instanceConfigFields` in the `info.js` export. -For example in info.js: +You can take advantage of it by declaring config fields under either `controllerConfigFields` or `instanceConfigFields` in the `plugin` export. +For example in index.js: ```ts import type * as lib from "@clusterio/lib"; @@ -330,9 +328,9 @@ The requests are similar to HTTP requests, except both parties of a link may ini ### Defining Events -Events are defined as properties of the messages object exported by `info.js` that map to instances of the `Event` class from `lib/link`. +Events are defined as properties of the messages object exported on `plugin` that map to instances of the `Event` class from `lib/link`. The name of the property correspond to the handler invoked on the plugin class. -The Event constructor takes an object of properties that define the event, for example the following could be defined in `info.js`: +The Event constructor takes an object of properties that define the event, for example the following could be defined in `plugin`: ```js messages: { @@ -404,10 +402,10 @@ Use a request if you need a confirmation that the message was received. ### Definining Requests -Requests are defined as properties of the messages object exported by `info.js` that map to instances of the `Request` class from `lib/link`. +Requests are defined as properties of the messages object exported by `plugin` that map to instances of the `Request` class from `lib/link`. The name of the property corresponds to the handler invoked on the plugin class. The Request constructor takes an object of properties that define the event. -For example, the following could be defined in `info.js`: +For example, the following could be defined in `plugin`: ```js messages: { @@ -484,7 +482,7 @@ Same as the requestProperties only for the response sent back by the target. Link messages are sent by calling the `.send()` method on the Event/Request instance with the link you want to send it over and the data to send. For `InstancePlugin` code the link to the host is the `instance` itself, which is accessible through the `.instance` property of the `InstancePlugin`. -The `.info` property of the plugin class exposes the data exported from the plugin's `info.js` module. +The `.info` property of the plugin class exposes the data exported from the plugin's `plugin` export. In other words: ```js diff --git a/packages/controller/src/BaseControllerPlugin.ts b/packages/controller/src/BaseControllerPlugin.ts index 281810c9..92e019db 100644 --- a/packages/controller/src/BaseControllerPlugin.ts +++ b/packages/controller/src/BaseControllerPlugin.ts @@ -13,8 +13,8 @@ import type HostConnection from "./HostConnection"; * Controller plugins are subclasses of this class which get instantiated by * the controller on startup when the plugin is enabled in the config. * To be discovered the class must be exported under the name `ControllerPlugin` - * in the module specified by the `controllerEntrypoint` in the plugin's info.js - * file. + * in the module specified by the `controllerEntrypoint` in the plugin's + * `plugin` export. */ export default class BaseControllerPlugin { /** diff --git a/packages/ctl/src/BaseCtlPlugin.ts b/packages/ctl/src/BaseCtlPlugin.ts index 7171f86a..bed7534b 100644 --- a/packages/ctl/src/BaseCtlPlugin.ts +++ b/packages/ctl/src/BaseCtlPlugin.ts @@ -6,7 +6,7 @@ import type { CommandTree, Logger, PluginNodeEnvInfo } from "@clusterio/lib"; * Ctl plugins are subclasses of this class which get instantiated by * clusterioctl in order to extend its functionallity. To be discovered the * class must be exported under the name `CtlPlugin` in the module - * specified by the `ctlEntrypoint` in the plugin's info.js file. + * specified by the `ctlEntrypoint` in the plugin's `plugin` export. */ export default class BaseCtlPlugin { /** diff --git a/packages/host/src/BaseHostPlugin.ts b/packages/host/src/BaseHostPlugin.ts index 87664cff..131da3db 100644 --- a/packages/host/src/BaseHostPlugin.ts +++ b/packages/host/src/BaseHostPlugin.ts @@ -9,7 +9,7 @@ import type Host from "./Host"; * Host plugins are subclasses of this class which get instantiated by * the host when it starts up with the plugin enabled in the config. To be * discovered the class must be exported under the name `HostPlugin` in the - * module specified by the `hostEntrypoint` in the plugin's info.js file. + * module specified by the `hostEntrypoint` in the plugin's `plugin` export. */ export default class BaseHostPlugin { /** diff --git a/packages/host/src/BaseInstancePlugin.ts b/packages/host/src/BaseInstancePlugin.ts index 2792f404..515af1ad 100644 --- a/packages/host/src/BaseInstancePlugin.ts +++ b/packages/host/src/BaseInstancePlugin.ts @@ -11,7 +11,7 @@ import type Host from "./Host"; * the host when it brings up an instance with the plugin enabled in the * config. To be discovered the class must be exported under the name * `InstancePlugin` in the module specified by the `instanceEntrypoint` in - * the plugin's info.js file. + * the plugin's `plugin` export. * * Instances may be started and stopped many times, and many instances may * be running at the same time, each of which will have their own instance diff --git a/packages/lib/src/plugin.ts b/packages/lib/src/plugin.ts index a08cac9a..ca878706 100644 --- a/packages/lib/src/plugin.ts +++ b/packages/lib/src/plugin.ts @@ -7,8 +7,7 @@ import type { Logger } from "./logging"; import type { FieldDefinition } from "./config"; import type { PlayerStats } from "./data"; -// TODO Add proper typing for plugins -/* Used to define the export of info.ts from plugins */ +/** Used to define the plugin export in plugins */ export type PluginDeclaration = { name: string; title: string; diff --git a/packages/lib/src/plugin_loader.ts b/packages/lib/src/plugin_loader.ts index 207e17ed..3c1af67f 100644 --- a/packages/lib/src/plugin_loader.ts +++ b/packages/lib/src/plugin_loader.ts @@ -27,12 +27,18 @@ export async function loadPluginInfos(pluginList: Map) { let pluginPackage: { name?: string, version: string, main?: string, private?: boolean }; try { - pluginInfo = require(pluginPath).default; + pluginInfo = require(pluginPath).plugin; pluginPackage = require(path.posix.join(pluginPath, "package.json")); } catch (err: any) { throw new libErrors.PluginError(pluginName, err); } + if (typeof pluginInfo !== "object") { + throw new libErrors.EnvironmentError( + `Expected plugin at ${pluginPath} to export an object named 'plugin' but got ${typeof pluginInfo}`, + ); + } + if (pluginInfo.name !== pluginName) { throw new libErrors.EnvironmentError( `Expected plugin at ${pluginPath} to be named ${pluginName} but got ${pluginInfo.name}` diff --git a/packages/lib/src/shared_commands.ts b/packages/lib/src/shared_commands.ts index 6b9ea138..13be8c32 100644 --- a/packages/lib/src/shared_commands.ts +++ b/packages/lib/src/shared_commands.ts @@ -59,13 +59,19 @@ export async function handlePluginCommand( let pluginInfo: { name: string }; try { // eslint-disable-next-line @typescript-eslint/no-var-requires - pluginInfo = require(pluginPath).default; + pluginInfo = require(pluginPath).plugin; } catch (err: any) { logger.error(`Unable to import plugin info from ${args.path}:\n${err.stack}`); process.exitCode = 1; return; } + if (typeof pluginInfo !== "object") { + logger.error("Plugin does not correctly export a 'plugin' object"); + process.exitCode = 1; + return; + } + if (pluginList.has(pluginInfo.name)) { logger.error(`Plugin with the same ${pluginInfo.name} already exists`); process.exitCode = 1; diff --git a/packages/web_ui/src/bootstrap.tsx b/packages/web_ui/src/bootstrap.tsx index 5d45395d..9c018e7d 100644 --- a/packages/web_ui/src/bootstrap.tsx +++ b/packages/web_ui/src/bootstrap.tsx @@ -52,7 +52,7 @@ async function loadPluginInfos(): Promise { throw new Error(`Plugin did not expose its container via plugin_${meta.name}`); } await container.init(__webpack_share_scopes__.default); - let pluginInfo = (await container.get("./info"))().default; + let pluginInfo = (await container.get("./"))().plugin; pluginInfo.container = container; pluginInfo.package = (await container.get("./package.json"))(); pluginInfo.enabled = meta.enabled; diff --git a/plugins/global_chat/info.ts b/plugins/global_chat/index.ts similarity index 85% rename from plugins/global_chat/info.ts rename to plugins/global_chat/index.ts index 092a3287..620a8ecc 100644 --- a/plugins/global_chat/info.ts +++ b/plugins/global_chat/index.ts @@ -2,7 +2,7 @@ import type * as lib from "@clusterio/lib"; import { ChatEvent } from "./messages"; -export default { +export const plugin: lib.PluginDeclaration = { name: "global_chat", title: "Global Chat", description: "Forwards chat between instances.", @@ -12,4 +12,4 @@ export default { messages: [ ChatEvent, ], -} satisfies lib.PluginDeclaration; +}; diff --git a/plugins/global_chat/package.json b/plugins/global_chat/package.json index 240af799..648e8ab5 100644 --- a/plugins/global_chat/package.json +++ b/plugins/global_chat/package.json @@ -6,7 +6,7 @@ "homepage": "https://github.com/clusterio/clusterio#readme", "license": "MIT", "repository": "clusterio/clusterio", - "main": "dist/plugin/info.js", + "main": "dist/plugin/index.js", "scripts": { "prepare": "tsc --build && webpack-cli --env production", "test": "echo \"Error: run tests from root\" && exit 1" diff --git a/plugins/global_chat/test/plugin.js b/plugins/global_chat/test/plugin.js index 0708bbf1..d8b9b860 100644 --- a/plugins/global_chat/test/plugin.js +++ b/plugins/global_chat/test/plugin.js @@ -5,7 +5,7 @@ const lib = require("@clusterio/lib"); const mock = require("../../../test/mock"); const lines = require("../../../test/lib/factorio/lines"); const instance = require("../dist/plugin/instance"); -const info = require("../dist/plugin/info").default; +const info = require("../dist/plugin/index").plugin; const { ChatEvent } = require("../dist/plugin/messages"); diff --git a/plugins/global_chat/webpack.config.js b/plugins/global_chat/webpack.config.js index 6870a94b..39150485 100644 --- a/plugins/global_chat/webpack.config.js +++ b/plugins/global_chat/webpack.config.js @@ -16,7 +16,7 @@ module.exports = (env = {}) => merge(common(env), { name: "global_chat", library: { type: "var", name: "plugin_global_chat" }, exposes: { - "./info": "./info.ts", + "./": "./index.ts", "./package.json": "./package.json", }, shared: { diff --git a/plugins/inventory_sync/info.ts b/plugins/inventory_sync/index.ts similarity index 96% rename from plugins/inventory_sync/info.ts rename to plugins/inventory_sync/index.ts index 38a42735..029e3a46 100644 --- a/plugins/inventory_sync/info.ts +++ b/plugins/inventory_sync/index.ts @@ -17,7 +17,7 @@ lib.definePermission({ grantByDefault: true, }); -export default { +export const plugin: lib.PluginDeclaration = { name: "inventory_sync", title: "Inventory sync", description: "Synchronizes players inventories between instances", @@ -54,4 +54,4 @@ export default { ], webEntrypoint: "./web", routes: ["/inventory"], -} satisfies lib.PluginDeclaration; +}; diff --git a/plugins/inventory_sync/package.json b/plugins/inventory_sync/package.json index 3f2bba9b..fe6b7738 100644 --- a/plugins/inventory_sync/package.json +++ b/plugins/inventory_sync/package.json @@ -6,7 +6,7 @@ "homepage": "https://github.com/clusterio/factorioClusterio#readme", "license": "MIT", "repository": "clusterio/factorioClusterio", - "main": "dist/plugin/info.js", + "main": "dist/plugin/index.js", "scripts": { "prepare": "tsc --build && webpack-cli --env production", "test": "echo \"Error: run tests from root\" && exit 1" diff --git a/plugins/inventory_sync/webpack.config.js b/plugins/inventory_sync/webpack.config.js index 90f35fef..6f4df3b1 100644 --- a/plugins/inventory_sync/webpack.config.js +++ b/plugins/inventory_sync/webpack.config.js @@ -16,7 +16,7 @@ module.exports = (env = {}) => merge(common(env), { name: "inventory_sync", library: { type: "var", name: "plugin_inventory_sync" }, exposes: { - "./info": "./info.ts", + "./": "./index.ts", "./package.json": "./package.json", "./web": "./web/index.tsx", }, diff --git a/plugins/player_auth/info.ts b/plugins/player_auth/index.ts similarity index 95% rename from plugins/player_auth/info.ts rename to plugins/player_auth/index.ts index ac1386bf..a181c508 100644 --- a/plugins/player_auth/info.ts +++ b/plugins/player_auth/index.ts @@ -11,7 +11,7 @@ declare module "@clusterio/lib" { } } -export default { +export const plugin: lib.PluginDeclaration = { name: "player_auth", title: "Player Auth", description: "Provides authentication to the cluster via logging into a Factorio server.", @@ -37,4 +37,4 @@ export default { messages.FetchPlayerCodeRequest, messages.SetVerifyCodeRequest, ], -} satisfies lib.PluginDeclaration; +}; diff --git a/plugins/player_auth/package.json b/plugins/player_auth/package.json index 7faf3f69..da721e6c 100644 --- a/plugins/player_auth/package.json +++ b/plugins/player_auth/package.json @@ -6,7 +6,7 @@ "homepage": "https://github.com/clusterio/clusterio#readme", "license": "MIT", "repository": "clusterio/clusterio", - "main": "dist/plugin/info.js", + "main": "dist/plugin/index.js", "scripts": { "prepare": "tsc --build && webpack-cli --env production", "test": "echo \"Error: run tests from root\" && exit 1" diff --git a/plugins/player_auth/test/plugin.js b/plugins/player_auth/test/plugin.js index 5e7e81fb..fcbcf060 100644 --- a/plugins/player_auth/test/plugin.js +++ b/plugins/player_auth/test/plugin.js @@ -7,7 +7,7 @@ const mock = require("../../../test/mock"); const controller = require("../dist/plugin/controller"); const instance = require("../dist/plugin/instance"); -const info = require("../dist/plugin/info").default; +const info = require("../dist/plugin/index").plugin; const { FetchPlayerCodeRequest, SetVerifyCodeRequest } = require("../dist/plugin/messages"); const lib = require("@clusterio/lib"); diff --git a/plugins/player_auth/webpack.config.js b/plugins/player_auth/webpack.config.js index 6a3fc528..af39fcba 100644 --- a/plugins/player_auth/webpack.config.js +++ b/plugins/player_auth/webpack.config.js @@ -16,7 +16,7 @@ module.exports = (env = {}) => merge(common(env), { name: "player_auth", library: { type: "var", name: "plugin_player_auth" }, exposes: { - "./info": "./info.ts", + "./": "./index.ts", "./package.json": "./package.json", "./web": "./web/index.tsx", }, diff --git a/plugins/research_sync/info.ts b/plugins/research_sync/index.ts similarity index 89% rename from plugins/research_sync/info.ts rename to plugins/research_sync/index.ts index b6692e12..af7c8364 100644 --- a/plugins/research_sync/info.ts +++ b/plugins/research_sync/index.ts @@ -2,7 +2,7 @@ import type * as lib from "@clusterio/lib"; import * as messages from "./messages"; -export default { +export const plugin: lib.PluginDeclaration = { name: "research_sync", title: "Research Sync", description: "Synchronises technology research progress between instances.", @@ -15,4 +15,4 @@ export default { messages.FinishedEvent, messages.SyncTechnologiesRequest, ], -} satisfies lib.PluginDeclaration; +}; diff --git a/plugins/research_sync/package.json b/plugins/research_sync/package.json index eeaa70a3..9fb317e1 100644 --- a/plugins/research_sync/package.json +++ b/plugins/research_sync/package.json @@ -6,7 +6,7 @@ "homepage": "https://github.com/clusterio/clusterio#readme", "license": "MIT", "repository": "clusterio/clusterio", - "main": "dist/plugin/info.js", + "main": "dist/plugin/index.js", "scripts": { "prepare": "tsc --build && webpack-cli --env production", "test": "echo \"Error: run tests from root\" && exit 1" diff --git a/plugins/research_sync/webpack.config.js b/plugins/research_sync/webpack.config.js index 6c04b3d1..52c861de 100644 --- a/plugins/research_sync/webpack.config.js +++ b/plugins/research_sync/webpack.config.js @@ -16,7 +16,7 @@ module.exports = (env = {}) => merge(common(env), { name: "research_sync", library: { type: "var", name: "plugin_research_sync" }, exposes: { - "./info": "./info.ts", + "./": "./index.ts", "./package.json": "./package.json", }, shared: { diff --git a/plugins/statistics_exporter/info.ts b/plugins/statistics_exporter/index.ts similarity index 92% rename from plugins/statistics_exporter/info.ts rename to plugins/statistics_exporter/index.ts index d8276358..8748224c 100644 --- a/plugins/statistics_exporter/info.ts +++ b/plugins/statistics_exporter/index.ts @@ -6,7 +6,7 @@ declare module "@clusterio/lib" { } } -export default { +export const plugin: lib.PluginDeclaration = { name: "statistics_exporter", title: "Statistics Exporter", description: @@ -22,4 +22,4 @@ export default { initialValue: 1, }, }, -} satisfies lib.PluginDeclaration; +}; diff --git a/plugins/statistics_exporter/package.json b/plugins/statistics_exporter/package.json index 50a09732..23c93577 100644 --- a/plugins/statistics_exporter/package.json +++ b/plugins/statistics_exporter/package.json @@ -6,7 +6,7 @@ "homepage": "https://github.com/clusterio/clusterio#readme", "license": "MIT", "repository": "clusterio/clusterio", - "main": "dist/plugin/info.js", + "main": "dist/plugin/index.js", "scripts": { "prepare": "tsc --build && webpack-cli --env production", "test": "echo \"Error: run tests from root\" && exit 1" diff --git a/plugins/statistics_exporter/test/plugin.js b/plugins/statistics_exporter/test/plugin.js index 43356fcb..e54184a4 100644 --- a/plugins/statistics_exporter/test/plugin.js +++ b/plugins/statistics_exporter/test/plugin.js @@ -3,7 +3,7 @@ const assert = require("assert").strict; const mock = require("../../../test/mock"); const instance = require("../dist/plugin/instance"); -const info = require("../dist/plugin/info").default; +const info = require("../dist/plugin/index").plugin; describe("statistics_exporter plugin", function() { diff --git a/plugins/statistics_exporter/webpack.config.js b/plugins/statistics_exporter/webpack.config.js index 3f768eaa..1e25f204 100644 --- a/plugins/statistics_exporter/webpack.config.js +++ b/plugins/statistics_exporter/webpack.config.js @@ -16,7 +16,7 @@ module.exports = (env = {}) => merge(common(env), { name: "statistics_exporter", library: { type: "var", name: "plugin_statistics_exporter" }, exposes: { - "./info": "./info.ts", + "./": "./index.ts", "./package.json": "./package.json", }, shared: { diff --git a/plugins/subspace_storage/info.ts b/plugins/subspace_storage/index.ts similarity index 97% rename from plugins/subspace_storage/info.ts rename to plugins/subspace_storage/index.ts index a7d6ad54..6543cd6c 100644 --- a/plugins/subspace_storage/info.ts +++ b/plugins/subspace_storage/index.ts @@ -18,7 +18,7 @@ lib.definePermission({ grantByDefault: true, }); -export default { +export const plugin: lib.PluginDeclaration = { name: "subspace_storage", title: "Subspace Storage", description: "Provides shared storage across instances for the Subspace Storage mod", @@ -58,4 +58,4 @@ export default { ], webEntrypoint: "./web", routes: ["/storage"], -} satisfies lib.PluginDeclaration; +}; diff --git a/plugins/subspace_storage/package.json b/plugins/subspace_storage/package.json index 0049ebd9..eecb9a89 100644 --- a/plugins/subspace_storage/package.json +++ b/plugins/subspace_storage/package.json @@ -6,7 +6,7 @@ "homepage": "https://github.com/clusterio/clusterio#readme", "license": "MIT", "repository": "clusterio/clusterio", - "main": "dist/plugin/info.js", + "main": "dist/plugin/index.js", "scripts": { "prepare": "tsc --build && webpack-cli --env production", "test": "echo \"Error: run tests from root\" && exit 1" diff --git a/plugins/subspace_storage/webpack.config.js b/plugins/subspace_storage/webpack.config.js index a2669ef6..ae97b74c 100644 --- a/plugins/subspace_storage/webpack.config.js +++ b/plugins/subspace_storage/webpack.config.js @@ -16,7 +16,7 @@ module.exports = (env = {}) => merge(common(env), { name: "subspace_storage", library: { type: "var", name: "plugin_subspace_storage" }, exposes: { - "./info": "./info.ts", + "./": "./index.ts", "./package.json": "./package.json", "./web": "./web/index.tsx", }, diff --git a/test/integration/index.js b/test/integration/index.js index 33db6e9c..6932e283 100644 --- a/test/integration/index.js +++ b/test/integration/index.js @@ -12,11 +12,11 @@ const lib = require("@clusterio/lib"); const { LineSplitter, ConsoleTransport, logger } = lib; // Make sure permissions from plugins are loaded -require("../../plugins/global_chat/dist/plugin/info"); -require("../../plugins/player_auth/dist/plugin/info"); -require("../../plugins/research_sync/dist/plugin/info"); -require("../../plugins/statistics_exporter/dist/plugin/info"); -require("../../plugins/subspace_storage/dist/plugin/info"); +require("../../plugins/global_chat/dist/plugin/index"); +require("../../plugins/player_auth/dist/plugin/index"); +require("../../plugins/research_sync/dist/plugin/index"); +require("../../plugins/statistics_exporter/dist/plugin/index"); +require("../../plugins/subspace_storage/dist/plugin/index"); const subspaceStorageVersion = "1.99.8"; const subspaceStorageGitHubTag = "1.99.6"; // Tag subspaceStorageVersion was released under on GitHub diff --git a/test/lib/plugin_loader.js b/test/lib/plugin_loader.js index 2e113210..b8f86ea4 100644 --- a/test/lib/plugin_loader.js +++ b/test/lib/plugin_loader.js @@ -18,18 +18,18 @@ describe("lib/plugin_loader", function() { before(async function() { async function writePlugin(pluginPath, infoName) { await fs.outputFile( - path.join(pluginPath, "info.js"), - `module.exports.default = { name: "${infoName}" };` + path.join(pluginPath, "index.js"), + `module.exports.plugin = { name: "${infoName}" };` ); await fs.outputFile( path.join(pluginPath, "package.json"), - '{ "version": "0.0.1", "main":"info.js" }' + '{ "version": "0.0.1" }' ); } await writePlugin(testPlugin, "test"); await writePlugin(brokenPlugin, "broken"); - await fs.outputFile(path.join(brokenPlugin, "info.js"), "Syntax Error"); + await fs.outputFile(path.join(brokenPlugin, "index.js"), "Syntax Error"); await writePlugin(invalidPlugin, "wrong"); }); @@ -49,7 +49,7 @@ describe("lib/plugin_loader", function() { let brokenMessage; try { // eslint-disable-next-line node/global-require - require(path.resolve(brokenPlugin, "info.js")); + require(path.resolve(brokenPlugin)); } catch (err) { brokenMessage = err.message; } From d5626d31e6a660a958242a93957c55bdeec47c2d Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Wed, 17 Jan 2024 04:08:09 +0100 Subject: [PATCH 02/84] Fix rendering of ModDetails --- packages/web_ui/src/components/ModDetails.tsx | 28 ++++++++++++------- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/packages/web_ui/src/components/ModDetails.tsx b/packages/web_ui/src/components/ModDetails.tsx index f630f01d..973480fb 100644 --- a/packages/web_ui/src/components/ModDetails.tsx +++ b/packages/web_ui/src/components/ModDetails.tsx @@ -14,7 +14,11 @@ export default function ModDetails(props: let screens = useBreakpoint(); const mod: Partial & Partial = props.mod; - return + return {mod.error && Error} span={2} @@ -29,10 +33,14 @@ export default function ModDetails(props: && {props.actions(props.mod)} } - {mod.title && {mod.title}} - {mod.version && {mod.version}} - {mod.description && {mod.description} } - {mod.author && {mod.author}} + {Boolean(mod.title) && {mod.title}} + {Boolean(mod.version) + && {mod.version} + } + {Boolean(mod.description) + && {mod.description} + } + {Boolean(mod.author) && {mod.author}} @@ -40,17 +48,17 @@ export default function ModDetails(props: - {mod.contact && {mod.contact}} - {mod.homepage && {mod.homepage}} + {Boolean(mod.contact) && {mod.contact}} + {Boolean(mod.homepage) && {mod.homepage}} {mod.name} - {mod.factorioVersion + {Boolean(mod.factorioVersion) && {mod.factorioVersion} } - {mod.filename && {mod.filename}} + {Boolean(mod.filename) && {mod.filename}} {mod.size !== undefined && {lib.formatBytes(mod.size)} } - {mod.sha1 && {mod.sha1}} + {Boolean(mod.sha1) && {mod.sha1}} {mod.dependencies && mod.dependencies.length && {mod.dependencies From adc2d9cb4d78df6ff6f96f9c180af2021cd04b91 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Fri, 19 Jan 2024 19:17:03 +0100 Subject: [PATCH 03/84] Fix HostInfo.name description --- packages/controller/src/HostInfo.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/controller/src/HostInfo.ts b/packages/controller/src/HostInfo.ts index 7a7a105e..93f41fd3 100644 --- a/packages/controller/src/HostInfo.ts +++ b/packages/controller/src/HostInfo.ts @@ -8,7 +8,7 @@ export default class HostInfo { constructor( /** Id of this host */ public id: number, - /** Version this host last connected with */ + /** Name of this host */ public name: string, /** Version this host last connected with */ public version: string, From 5e2221796a6968a82ff8f75d73a7326d6b137af7 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sat, 20 Jan 2024 01:22:29 +0100 Subject: [PATCH 04/84] Rename sharedMods to mods on migration The place for storing mods was changed from sharedMods to mods when mod management was implemented. Rename the old sharedMods folder to mods when the migration script is invoked. --- packages/create/create.js | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/packages/create/create.js b/packages/create/create.js index a50b5dbd..e5f27d24 100755 --- a/packages/create/create.js +++ b/packages/create/create.js @@ -281,6 +281,14 @@ async function migrateRename(args) { await migrateLogsDir(path.join("logs", "cluster-prerename"), path.join("logs", "cluster")); } + if ( + await fs.pathExists("sharedMods") + && !await fs.pathExists("mods") + ) { + logger.info("Moving sharedMods/ to mods/"); + await fs.rename("sharedMods", "mods"); + } + if (!args.dev) { let pkg = JSON.parse(await fs.readFile("package.json")); if (pkg.dependencies) { From 671ff23085a955392d5a802e3c7dbd31a8781883 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sat, 20 Jan 2024 01:30:24 +0100 Subject: [PATCH 05/84] Write new run scripts during migration If the old run-master or run-slave scripts are present write the new run-controller and/or run-host scripts. --- packages/create/create.js | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/packages/create/create.js b/packages/create/create.js index e5f27d24..2ccfb8b7 100755 --- a/packages/create/create.js +++ b/packages/create/create.js @@ -305,9 +305,27 @@ async function migrateRename(args) { await execFile(`npm${scriptExt}`, ["update"]); } + const hasRunMaster = await fs.pathExists("run-master.sh") || await fs.pathExists("run-master.cmd"); + const hasRunSlave = await fs.pathExists("run-slave.sh") || await fs.pathExists("run-slave.cmd"); + if (hasRunMaster || hasRunSlave) { + logger.info("Writing run scripts"); + let mode = "standalone"; + if (!hasRunSlave) { mode = "controller"; } + if (!hasRunMaster) { mode = "host"; } + await writeScripts(mode); + } + logger.info( "Migration complete, you may now delete the following left over files and directories (if present):" + - "\n- config-master.json\n- config-slave.json\n- logs/master\n- logs/slave\n- logs/cluster-prerename" + "\n- config-master.json" + + "\n- config-slave.json" + + "\n- logs/master" + + "\n- logs/slave" + + "\n- logs/cluster-prerename" + + "\n- systemd/clusteriomaster.service" + + "\n- systemd/clusterioslave.service" + + "\n- run-master.sh / run-master.cmd" + + "\n- run-slave.sh / run-slave.cmd" ); } From 96f48ca1020283cc20dc4a3f2f068e9318c95d88 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sat, 20 Jan 2024 01:34:33 +0100 Subject: [PATCH 06/84] Fix installer breaking on setting up plugins The entrypoint of plugins were changed in 50f12c7 and b0f6c95 but the installer's loading of plugin info to get the correct plugin name was not updated for these changes. Update installer plugin loading to use the correct entrypoint. --- packages/create/create.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/create/create.js b/packages/create/create.js index 2ccfb8b7..7fcc62bc 100755 --- a/packages/create/create.js +++ b/packages/create/create.js @@ -374,7 +374,7 @@ async function installClusterio(mode, plugins) { for (let plugin of plugins) { if (!pluginList.has(plugin)) { // eslint-disable-next-line node/global-require - let pluginInfo = require(require.resolve(path.posix.join(plugin, "info"), { paths: [process.cwd()] })); + let pluginInfo = require(require.resolve(plugin, { paths: [process.cwd()] })).plugin; pluginList.set(pluginInfo.name, plugin); } } From bb1401a6832322f4fcdce25761274af35db8156a Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sat, 20 Jan 2024 02:03:23 +0100 Subject: [PATCH 07/84] Update jsonwebtoken used by player_auth This was missed during 1a9cbd2 which updated jsonwebtoken in other packages. --- plugins/player_auth/package.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/plugins/player_auth/package.json b/plugins/player_auth/package.json index da721e6c..d3ff154a 100644 --- a/plugins/player_auth/package.json +++ b/plugins/player_auth/package.json @@ -23,7 +23,7 @@ "dependencies": { "@sinclair/typebox": "^0.30.4", "express": "^4.16.2", - "jsonwebtoken": "^8.2.1" + "jsonwebtoken": "^9.0.1" }, "devDependencies": { "@clusterio/lib": "^2.0.0-alpha.13", From 1e66f1191747f41bd820bebaf0e7cc2664c181e9 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sat, 20 Jan 2024 02:16:19 +0100 Subject: [PATCH 08/84] Reorder members of Address The logical order of clusterio's components are in most places controller, host, instance, control. It is confusing that Address breaks this convention and I do not remember I wrote it that way. Reorder and renumber the host and instance members of Address to be consistent with most other places. --- packages/lib/src/data/messages_core.ts | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/packages/lib/src/data/messages_core.ts b/packages/lib/src/data/messages_core.ts index 4a84e9c4..67ea64a5 100644 --- a/packages/lib/src/data/messages_core.ts +++ b/packages/lib/src/data/messages_core.ts @@ -19,10 +19,10 @@ export type AddressShorthand = export class Address { /** Controller address */ static controller = 0; - /** Instance address */ - static instance = 1; /** Host address */ - static host = 2; + static host = 1; + /** Instance address */ + static instance = 2; /** Control address */ static control = 3; /** Broadcast */ From c3c2528c30294ed5b20c0889d3d8a78f5e4e6212 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sat, 20 Jan 2024 02:44:55 +0100 Subject: [PATCH 09/84] Add missing updates to instance.updatedAt Fix instances not properly updating their status to unassigned and unknown due to the updatedAt field not being updated with the current timestamp. --- packages/controller/src/Controller.ts | 1 + packages/controller/src/HostConnection.ts | 2 ++ 2 files changed, 3 insertions(+) diff --git a/packages/controller/src/Controller.ts b/packages/controller/src/Controller.ts index 89214e0f..4ad9e6f0 100644 --- a/packages/controller/src/Controller.ts +++ b/packages/controller/src/Controller.ts @@ -937,6 +937,7 @@ export default class Controller { ); } else { instance.status = "unassigned"; + instance.updatedAt = Date.now(); this.instanceDetailsUpdated([instance]); } } diff --git a/packages/controller/src/HostConnection.ts b/packages/controller/src/HostConnection.ts index f4cf1d84..ec078553 100644 --- a/packages/controller/src/HostConnection.ts +++ b/packages/controller/src/HostConnection.ts @@ -57,6 +57,7 @@ export default class HostConnection extends BaseConnection { } this.connector.on("close", () => { + const now = Date.now(); // Update status to unknown for instances on this host. const instances: InstanceInfo[] = []; for (let instance of this._controller.instances.values()) { @@ -67,6 +68,7 @@ export default class HostConnection extends BaseConnection { instances.push(instance); let prev = instance.status; instance.status = "unknown"; + instance.updatedAt = now; lib.invokeHook(this._controller.plugins, "onInstanceStatusChanged", instance, prev); } this._controller.instanceDetailsUpdated(instances); From f702c956a5df3cc738792e5774620cb1e71df7a9 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sat, 20 Jan 2024 05:13:55 +0100 Subject: [PATCH 10/84] Document migration of plugins to alpha 14 (#558) Co-authored-by: Cooldude2606 <25043174+Cooldude2606@users.noreply.github.com> --- docs/devs/plugin-migration-to-alpha-14.md | 523 ++++++++++++++++++++++ 1 file changed, 523 insertions(+) create mode 100644 docs/devs/plugin-migration-to-alpha-14.md diff --git a/docs/devs/plugin-migration-to-alpha-14.md b/docs/devs/plugin-migration-to-alpha-14.md new file mode 100644 index 00000000..91962795 --- /dev/null +++ b/docs/devs/plugin-migration-to-alpha-14.md @@ -0,0 +1,523 @@ +# Migrating Plugins to Alpha 14 + +Between version 2.0.0-alpha.13 and 2.0.0-alpha.14 the codebase underwent major refactoring and was ported to TypeScript. +In order to support alpha 14 there are several major changes that need to be done to existing plugins, these are in no particular order + + +## Rename master and slave + +All references to slave and master in the code was renamed to controller and host respectively which means that code referencing these in plugins must be renamed. +In short this means performing a global search and replace on the codebase replacing: + +- `master` with `controller` +- `Master` with `Controller` +- `slave` with `host` +- `Slave` with `Host` + +in all places it occurs, even as part of longer names such as changing `BaseMasterPlugin` to `BaseControllerPlugin`. + + +## Change lib imports to import directly from the top level package + +Previously code could import sub packages of lib such as lib/lua_tools. +These imports need to be changed to all import from lib directly. + +**Alpha 13** +```js +const libLuaTools = require("@clusterio/lib/lua_tools"); +const libErrors = require("@clusterio/lib/errors"); + +// Using imports +new libErrors.RequestError(...) +libLuaTools.escapeString(...) +``` + +**Alpha 14 (JS)** +```js +const lib = require("@clusterio/lib"); + +// Using library components +new lib.RequestError(...) +lib.escapeString(...) +``` + +**Alpha 14 (TS)** +```ts +import * as lib from "@clusterio/lib"; + +// Using library components +new lib.RequestError(...) +lib.escapeString(...) +``` + + +## Change import of base plugins + +Previously `BaseMasterPlugin` and `BaseInstancePlugin` were imported from `@clusterio/lib`, these have been moved to `@clusterio/controller` and `@clusterio/host` respectively. + +**Alpha 13** +```js +const { BaseMasterPlugin } = require("@clusterio/lib"); +const { BaseInstancePlugin } = require("@clusterio/lib"); +``` + +**Alpha 14 (JS)** +```js +const { BaseControllerPlugin } = require("@clusterio/controller"); +const { BaseInstancePlugin } = require("@clusterio/host"); +``` + +**Alpha 14 (TS)** +```ts +import { BaseControllerPlugin } from "@clusterio/controller"; +import { BaseInstancePlugin } from "@clusterio/host"; +``` + + +## Change main entrypoint of plugin + +Previously the info of a plugin would be loaded by requiring `./plugin_dir/info`, this has been changed to require `./plugin_dir` which in turn means that `info.js` is no longer the default entrypoint. +To solve this you can either add `"main": "info.js",` to package.json or rename `info.js` to `index.js` (the rest of this guide assumes the entrypoint was renamed to `index.js`). +The motivation behind this change was to allow TypeScript to output its build to a sub-folder of the plugin, if you're using TypeScript you'll likely have the entrypoint be located in `dist/index.js` instead. + +The expected place where the declaration of a plugin is exported has also changed from `module.exports` to `module.exports.plugin`: + +**Alpha 13** +```js +// info.js +module.exports = { + name: "plugin_name", + ... +} +``` + +**Alpha 14 (JS)** +```js +// index.js +const plugin = { + name: "plugin_name", + ... +} + +module.exports = { + plugin, +}; +``` + +**Alpha 14 (TS)** +```ts +// index.ts +import type * as lib from "@clusterio/lib"; + +export const plugin: lib.PluginDeclaration = { + name: "plugin_name", + ... +}; +``` + + +### Modify exposed path in webpack.config.js + +The `./info` import path exposed in the ModuleFederationPlugin in the `webpack.config.js` needs to be changed to `./`. +E.g.: + +**Alpha 13** +```js +// In webpack.config.js +new webpack.container.ModuleFederationPlugin({ + ... + exposes: { + "./info": "./info.js", + ... + }, +}), +``` + +**Alpha 14** +```js +// In webpack.config.js +new webpack.container.ModuleFederationPlugin({ + ... + exposes: { + "./": "./index.js", + ... + }, +}), +``` + +## Remove usage of ConfigGroup + +The Config system was refactored and the `ConfigGroup` concept was completely removed. +Config fields are now defined directly on the exported plugin info. + +**Alpha 13** +```js +const lib = require("@clusterio/lib"); + +class MasterConfigGroup extends lib.PluginConfigGroup { } +MasterConfigGroup.groupName = "plugin_name"; +MasterConfigGroup.defaultAccess = ["master", "control"]; +MasterConfigGroup.define({ + name: "option", + title: "Option", + description: "A config option.", + type: "string", + optional: true, +}); + +module.exports = { + name: "plugin_name", + MasterConfigGroup, + ... +} +``` + +**Alpha 14 (JS)** +```js +const plugin = { + name: "plugin_name", + controllerConfigFields: { + "plugin_name.option": { + title: "Option", + description: "A config option.", + type: "string", + initial_value: "foo", + optional: true, + }, + }, + ... +} + +module.exports = { + plugin, +}; +``` + +**Alpha 14 (TS)** +```ts +import type * as lib from "@clusterio/lib"; + +// Declare the type of the field to make the type of controller.config.get(field) correctly deduced. +declare module "@clusterio/lib" { + export interface ControllerConfigFields { + "plugin_name.option": null | string; + } +} + +export const plugin: lib.PluginDeclaration = { + name: "plugin_name", + controllerConfigFields: { + "plugin_name.option": { + title: "Option", + description: "A config option.", + type: "string", + initialValue: "foo", + optional: true, + }, + }, + ... +}; +``` + + +### Use onControllerConfigFieldChanged instead of listering to the fieldChanged event + +**Alpha 13** +```js +this.controller.config.on("fieldChanged", (group, field, prev) => { + if (group.name === "plugin_name" && field === "option") { + const curr = group.get(field); + // Do stuff with curr and or prev + } +} +``` + +**Alpha 14 (JS)** +```js +async onControllerConfigFieldChanged(field, curr, prev) { + if (field === "plugin_name.option") { + // Do stuff with curr and or prev + } +} +``` + +**Alpha 14 (TS)** +```ts +async onControllerConfigFieldChanged(field: string, curr: unknown, prev: unknown) { + if (field === "plugin_name.option") { + // Do stuff with curr and or prev + } +} +``` + +## Redefine link messages as classes + +The link layer has been majorly refactored and Request and Events are now defined as [JSON Serialisable Classes](./json-serialisable-classes.md). + +**Alpha 13** +```js +module.exports = { + name: "plugin_name", + messages: { + forwarded: new lib.Event({ + type: "plugin_name:forwarded", + links: ["instance-slave", "slave-master"], + forwardTo: "master", + eventProperties: { + "string_value": { type: "string" }, + "number_value": { type: "number" }, + }, + }), + simple: new lib.Request({ + type: "plugin_name:simple", + links: ["master-slave", "slave-instance"], + forwardTo: "instance", + requestProperties: { + "instance_id": { type: "number" }, + "string_value": { type: "string" }, + }, + responseProperties: { + "number_value": { type: "number" }, + }, + }), + }, +}; +``` + +**Alpha 14 (JS)** +```js +const lib = require("@clusterio/lib"); + +class ForwardedEvent { + static type = "event"; + static src = "instance"; + static dst = "controller"; + static plugin = "plugin_name"; + + constructor(stringValue, numberValue) { + this.stringValue = stringValue; + this.numberValue = numberValue; + } + + static jsonSchema = { + type: "object", + required: ["stringValue", "numberValue"], + properties: { + "stringValue": { type: "string" }, + "numberValue": { type: "number" }, + }, + }; + + static fromJSON(json) { + return new this(json.stringValue, json.numberValue); + } +} + +class SimpleRequest { + static type = "request"; + static src = "controller"; + static dst = "instance"; + static plugin = "plugin_name"; + + constructor(stringValue) { + this.stringValue = stringValue; + } + + static jsonSchema = { + type: "object", + required: ["stringValue"], + properties: { + "stringValue": { type: "string" }, + }, + }; + + static Response = lib.JsonNumber +} + +const plugin = { + name: "plugin_name", + messages: [ + ForwardedEvent, + SimpleRequest, + ], + ... +}; + +module.exports = { + plugin, + ForwardedEvent, + SimpleRequest, +} +``` + +**Alpha 14 (TS)** +```ts +import * as lib from "@clusterio/lib"; +import { Static, Type } from "@sinclair/typebox"; + +export class ForwardedEvent { + declare ["constructor"]: typeof ForwardedEvent; + static type = "event" as const; + static src = "instance" as const; + static dst = "controller" as const; + static plugin = "plugin_name" as const; + + constructor( + public stringValue: string, + public numberValue: number, + ) { } + + static jsonSchema = Type.Object({ + "stringValue": Type.String(), + "numberValue": Type.Number(), + }); + + static fromJSON(json: Static) { + return new this(json.stringValue, json.numberValue); + } +} + +export class SimpleRequest { + declare ["constructor"]: typeof SimpleRequest; + static type = "request" as const; + static src = "controller" as const; + static dst = "instance" as const; + static plugin = "plugin_name" as const; + + constructor( + public stringValue: string, + ) { } + + static jsonSchema = Type.Object({ + "stringValue": Type.String(), + }); + + static fromJSON(json: Static) { + return new this(json.stringValue); + } + + static Response = lib.JsonNumber +} + +export const plugin: lib.PluginDeclaration = { + name: "plugin_name", + messages: [ + ForwardedEvent, + SimpleRequest, + ], + ... +} +``` + +### Refactor message sending + +Message handlers are now explicitly registered instead of relying on magic name lookups that don't make sense unless you know them. + +**Alpha 13** +```js +const lib = require("@clusterio/lib"); + +class ControllerPlugin extends lib.BaseControllerPlugin { + // event name is magically created by concatenating EventHandler or + // RequestHandler to the name in the messages export in in info.js. + async forwardedEventHandler(message) { + // message.data contains the raw JSON data sent. + if (message.data.string_value) { + // ... + } + } +} +``` + +**Alpha 14 (JS)** +```js +const { BaseControllerPlugin } = require("@clusterio/lib"); +const { ForwardedEvent } = require("./index.js"); + +class ControllerPlugin extends BaseControllerPlugin { + async init() { + // Explicit binding between message class and the handler for it + this.controller.handle(ForwardedEvent, this.handleForwardedEvent.bind(this)); + // Note: On an instance plugin you would use this.instance.handle + } + + async handleForwardedEvent(event, src, dst) { + // Event is an instance of the ForwardedEvent class + if (event.stringValue) { + // ... + } + + // src and dst are instances of lib.Address and describe where + // the message came from and who it was addressed to. + } + + ... +} + +module.exports = { + ControllerPlugin, +} +``` + +**Alpha 14 (TS)** +```ts +import * as lib from "@clusterio/lib"; +import { BaseControllerPlugin } from "@clusterio/controller"; +import { ForwardedEvent } from "./index.js"; + +export class ControllerPlugin extends BaseControllerPlugin { + async init() { + // When using TypeScript this call is type checked to make sure the handler accepts the right class. + this.controller.handle(ForwardedEvent, this.handleForwardedEvent.bind(this)); + // Note: On an instance plugin you would use this.instance.handle + } + + async handleForwardedEvent(event: ForwardedEvent, src: lib.Address, dst: lib.Address) { + // Event is an instance of the ForwardedEvent class + if (event.stringValue) { + // ... + } + + // src and dst describe where the message came from and who it was addressed to. + } + + ... +} +``` + + +### Refactor message sending + +Message are now addressed end to end instead of using the confusing forwardTo and broadcastTo properties of the message definition. +When broadcasting events the special `"allInstances"`, `"allHosts"`, and `"allControls"` shorthand addresses can be used. +There's two methods for sending messages `.send` and `.sendTo` where `.send` is a shorthand for `.sendTo` with an address filled in as the other side of the link, this is mostly only useful in control plugins where the other side of the link is the controller. + +**Alpha 13** +```js +// from instance plugin +this.info.messages.forwarded.send(this.instance, { + string_value: "foo", + number_value: 123, +}); + +// from master plugin +const instance_id = ... +const response = await this.forwardRequestToInstance(this.info.messages.simple, { + instance_id, + string_value: "foo", +}); +const value = response.number_value; +``` + +**Alpha 14** +```js +const { ForwardedEvent } = require("./info.js"); + +// from instance plugin +this.instance.sendTo("controller", new ForwardedEvent("foo", 123)); + +// from controller plugin +const instanceId = ... +const value = await this.controller.sendTo({ instanceId }, new SimpleRequest("foo")); +// if using TypeScript then the type of value will be inferred as number here +``` From 9842705eb9f6e742dff4b2c034aecfb92d5acb4e Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sat, 20 Jan 2024 05:20:24 +0100 Subject: [PATCH 11/84] Improve error reporting of plugins in the Web UI (#557) Keep the error message and display it in the plugin page when loading the plugin info or the plugin itself fails to load in the Web UI. --- packages/lib/src/plugin.ts | 5 ++-- packages/web_ui/src/bootstrap.tsx | 14 ++++++++-- .../web_ui/src/components/PluginViewPage.tsx | 28 +++++++++++++------ .../web_ui/src/components/PluginsPage.tsx | 19 ++++++++----- packages/web_ui/src/util/websocket.ts | 1 + 5 files changed, 46 insertions(+), 21 deletions(-) diff --git a/packages/lib/src/plugin.ts b/packages/lib/src/plugin.ts index ca878706..98953b63 100644 --- a/packages/lib/src/plugin.ts +++ b/packages/lib/src/plugin.ts @@ -43,9 +43,10 @@ export type PluginNodeEnvInfo = PluginDeclaration & { }; export type PluginWebpackEnvInfo = PluginDeclaration & { - container: any; - package: any; + container?: any; + package?: any; enabled?: boolean; + error?: string; }; /** diff --git a/packages/web_ui/src/bootstrap.tsx b/packages/web_ui/src/bootstrap.tsx index 9c018e7d..be280ecf 100644 --- a/packages/web_ui/src/bootstrap.tsx +++ b/packages/web_ui/src/bootstrap.tsx @@ -38,7 +38,7 @@ async function loadPluginInfos(): Promise { pluginList = []; } - let pluginInfos = []; + let pluginInfos: lib.PluginWebpackEnvInfo[] = []; await __webpack_init_sharing__("default"); for (let meta of pluginList) { if (meta.web.error) { @@ -59,6 +59,12 @@ async function loadPluginInfos(): Promise { pluginInfos.push(pluginInfo); } catch (err: any) { + pluginInfos.push({ + name: meta.name, + title: meta.name, + enabled: false, + error: err.message, + }); logger.error(`Failed to load plugin info for ${meta.name}`); if (err.stack) { logger.error(err.stack); @@ -79,7 +85,8 @@ async function loadPlugins(pluginInfos: lib.PluginWebpackEnvInfo[], control: Con if (pluginInfo.webEntrypoint) { let webModule = (await pluginInfo.container.get(pluginInfo.webEntrypoint))(); if (!webModule.WebPlugin) { - throw new Error("Plugin webEntrypoint does not export WebPlugin class"); + pluginInfo.error = "Plugin webEntrypoint does not export WebPlugin class"; + throw new Error(pluginInfo.error); } WebPluginClass = webModule.WebPlugin; } @@ -89,6 +96,7 @@ async function loadPlugins(pluginInfos: lib.PluginWebpackEnvInfo[], control: Con plugins.set(pluginInfo.name, plugin); } catch (err: any) { + pluginInfo.error = `Error loading plugin: ${err.message}`; logger.error(`Failed to load plugin ${pluginInfo.name}`); if (err.stack) { logger.error(err.stack); @@ -124,7 +132,7 @@ export default async function bootstrap() { let wsUrl = new URL(webRoot, document.location.href); let controlConnector = new ControlConnector(wsUrl.href, 120, undefined); - let control = new Control(controlConnector); + let control = new Control(controlConnector, new Map(pluginInfos.map(p => [p.name, p]))); control.plugins = await loadPlugins(pluginInfos, control); control.inputComponents = inputComponentsFromPlugins(control.plugins); diff --git a/packages/web_ui/src/components/PluginViewPage.tsx b/packages/web_ui/src/components/PluginViewPage.tsx index 01aef936..8b9281fd 100644 --- a/packages/web_ui/src/components/PluginViewPage.tsx +++ b/packages/web_ui/src/components/PluginViewPage.tsx @@ -10,7 +10,7 @@ import notify from "../util/notify"; export default function PluginViewPage() { let params = useParams(); - let plugins = useContext(ControlContext).plugins; + const control = useContext(ControlContext); let [pluginList, setPluginList] = useState(); let pluginName = String(params.name); @@ -25,8 +25,9 @@ export default function PluginViewPage() { })(); }, []); - let plugin = plugins.get(pluginName); - let pluginTitle = plugin ? plugin.info.title : pluginName; + const plugin = control.plugins.get(pluginName); + const pluginInfo = control.pluginInfos.get(pluginName); + const pluginTitle = pluginInfo ? pluginInfo.title : pluginName; let nav = [{ name: "Plugins", path: "/plugins" }, { name: pluginTitle }]; if (!pluginList) { return @@ -43,33 +44,42 @@ export default function PluginViewPage() { ; } + const pluginError = pluginMeta.web.error ?? pluginInfo?.error; if (!plugin) { return {pluginMeta.version} - {pluginMeta.loaded ? "Yes" : "No"} + No + {pluginMeta.loaded ? "Yes" : "No"} + + {pluginMeta.enabled ? "Yes" : "No"} + - + />} ; } return {pluginMeta.version} - {pluginMeta.loaded ? "Yes" : "No"} + Yes + {pluginMeta.loaded ? "Yes" : "No"} + + {pluginMeta.enabled ? "Yes" : "No"} + {plugin.info.description} {plugin.package.homepage ? {plugin.package.homepage} diff --git a/packages/web_ui/src/components/PluginsPage.tsx b/packages/web_ui/src/components/PluginsPage.tsx index bd976f66..f12b3d00 100644 --- a/packages/web_ui/src/components/PluginsPage.tsx +++ b/packages/web_ui/src/components/PluginsPage.tsx @@ -1,6 +1,8 @@ import React, { useContext, useEffect, useState } from "react"; import { useNavigate } from "react-router-dom"; import { Table } from "antd"; +import CloseCircleFilled from "@ant-design/icons/CloseCircleFilled"; +import InfoCircleFilled from "@ant-design/icons/InfoCircleFilled"; import type { PluginWebApi } from "@clusterio/lib"; @@ -12,7 +14,7 @@ const strcmp = new Intl.Collator(undefined, { numeric: true, sensitivity: "base" export default function PluginsPage() { - let plugins = useContext(ControlContext).plugins; + const control = useContext(ControlContext); let navigate = useNavigate(); let [pluginList, setPluginList] = useState([]); @@ -29,8 +31,8 @@ export default function PluginsPage() { let tableContents = []; for (let meta of pluginList) { - if (plugins.has(meta.name)) { - let plugin = plugins.get(meta.name)!; + if (control.plugins.has(meta.name)) { + let plugin = control.plugins.get(meta.name)!; tableContents.push({ meta, info: plugin.info, @@ -39,6 +41,7 @@ export default function PluginsPage() { } else { tableContents.push({ meta, + info: control.pluginInfos.get(meta.name), }); } } @@ -58,8 +61,11 @@ export default function PluginsPage() { title: "Version", key: "version", render: (_, plugin) => { + if (!plugin.meta.enabled) { + return <> Disabled on controller; + } if (!plugin.package) { - return "Error loading module"; + return <> Error loading module; } if (plugin.package.version !== plugin.meta.version) { return "Version missmatched"; @@ -70,9 +76,8 @@ export default function PluginsPage() { }, { title: "Loaded", - dataIndex: ["meta", "loaded"], - render: loaded => (loaded ? "Yes" : null), - sorter: (a, b) => Number(a.meta.loaded) - Number(b.meta.loaded), + render: (_, plugin) => (plugin.package ? "Yes" : null), + sorter: (a, b) => Number(Boolean(a.package)) - Number(Boolean(b.package)), responsive: ["sm"], }, ]} diff --git a/packages/web_ui/src/util/websocket.ts b/packages/web_ui/src/util/websocket.ts index dee94dac..49573b22 100644 --- a/packages/web_ui/src/util/websocket.ts +++ b/packages/web_ui/src/util/websocket.ts @@ -72,6 +72,7 @@ export class Control extends lib.Link { constructor( connector: ControlConnector, + public pluginInfos = new Map(), ) { super(connector); From c53781d16cebf004d6ef8efe5ba6c0c50ac49c6e Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sat, 20 Jan 2024 13:26:55 +0100 Subject: [PATCH 12/84] Add time unit suffixes to variables (#561) Add suffixes like Ms and Ns to variables whos time unit is not in seconds. This is aligning with a new code style of putting units on variables that are not in SI base units. --- docs/contributing.md | 13 ++++++- packages/controller/src/Controller.ts | 40 ++++++++++---------- packages/controller/src/ControllerUser.ts | 2 +- packages/controller/src/HostConnection.ts | 10 ++--- packages/controller/src/HostInfo.ts | 10 ++--- packages/controller/src/HttpCloser.ts | 6 +-- packages/controller/src/InstanceInfo.ts | 8 ++-- packages/controller/src/routes.ts | 4 +- packages/host/src/Instance.ts | 4 +- packages/host/src/server.ts | 8 ++-- packages/lib/src/ExponentialBackoff.ts | 8 ++-- packages/lib/src/ModStore.ts | 2 +- packages/lib/src/RateLimiter.ts | 10 ++--- packages/lib/src/data/ModInfo.ts | 10 ++--- packages/lib/src/data/ModPack.ts | 12 +++--- packages/lib/src/data/PlayerStats.ts | 25 ++++++------ packages/lib/src/data/User.ts | 10 ++--- packages/lib/src/data/messages_controller.ts | 6 +-- packages/lib/src/data/messages_host.ts | 6 +-- packages/lib/src/data/messages_instance.ts | 16 ++++---- packages/lib/src/helpers.ts | 12 +++--- packages/lib/src/link/connectors.ts | 30 +++++++-------- packages/lib/src/prometheus.ts | 18 ++++----- packages/lib/src/subscriptions.ts | 20 +++++----- packages/web_ui/src/util/time_format.ts | 4 +- plugins/inventory_sync/controller.ts | 14 +++---- plugins/player_auth/controller.ts | 16 ++++---- plugins/player_auth/test/plugin.js | 35 +++++++++-------- plugins/statistics_exporter/instance.ts | 4 +- test/controller/InstanceInfo.js | 2 +- test/host/Instance.js | 12 +++--- test/host/server.js | 2 +- test/integration/clusterio.js | 6 +-- test/lib/ExponentialBackoff.js | 6 +-- test/lib/data/ModInfo.js | 2 +- test/lib/data/User.js | 14 +++---- test/lib/helpers.js | 4 +- test/lib/subscriptions.js | 11 ++++-- 38 files changed, 221 insertions(+), 201 deletions(-) diff --git a/docs/contributing.md b/docs/contributing.md index 34ac895d..44582f4a 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -19,6 +19,7 @@ Though we have some conventions and workflows that we kindly ask you to follow. - [Supported Node.js and Factorio Version](#supported-node.js-and-factorio-version) - [Code Style](#code-style) - [Naming Style](#naming-style) + - [Units](#units) - [Indenting](#indenting) - [Line Length](#line-length) - [Markdown](#markdown) @@ -202,7 +203,7 @@ But there are at least a few things that have been agreed upon. JavaScript variables in general uses camelCase. The exception is for classes/constructors which use PascalCase and variables destructured from over the wire objects. -Fields in configs and messages sent over links uses lowercase_underscore. +Fields in configs uses lowercase_underscore. JavaScript files are named using lowercase_underscore for files that export multiple items. Files containing and exporting a single class should be named the same as the class in PascalCase. @@ -217,6 +218,16 @@ Note that no sub-modules of the lib package is imported directly from the outsid For lua code lowercase_underscore is used for everything. +### Units + +When naming variables whos unit is not a base SI unit like seconds or metres the unit used should be suffixed in the variable name: +E.g.: + + const startTimeMs = date.now(); + const endNs = process.hrtime(); + +This makes it easier to spot different unit bases are being mixed in the code. + ### Strings diff --git a/packages/controller/src/Controller.ts b/packages/controller/src/Controller.ts index 4ad9e6f0..8930f854 100644 --- a/packages/controller/src/Controller.ts +++ b/packages/controller/src/Controller.ts @@ -596,7 +596,7 @@ export default class Controller { const host = HostInfo.fromJSON(json); if (host.connected) { host.connected = false; - host.updatedAt = Date.now(); + host.updatedAtMs = Date.now(); } hosts.set(id, host); } @@ -617,13 +617,13 @@ export default class Controller { ) as Static[]; for (let json of serialized) { if (!json.config) { // migrate: from pre Alpha 14 format. - json = { config: json as any, status: "running" }; // Use running to force updatedAt + json = { config: json as any, status: "running" }; // Use running to force updatedAtMs } const instance = InstanceInfo.fromJSON(json, "controller"); const status = instance.config.get("instance.assigned_host") === null ? "unassigned" : "unknown"; if (instance.status !== status) { instance.status = status; - instance.updatedAt = Date.now(); + instance.updatedAtMs = Date.now(); } instances.set(instance.id, instance); } @@ -719,14 +719,14 @@ export default class Controller { static addAppRoutes(app: Application, pluginInfos: any[]) { app.use((req: Request, res: Response, next) => { - let start = process.hrtime.bigint(); + let startNs = process.hrtime.bigint(); stream.finished(res, () => { let routePath = "static"; if (req.route && req.route.path) { routePath = req.route.path; } - let end = process.hrtime.bigint(); - endpointDurationSummary.labels(routePath).observe(Number(end - start) / 1e9); + let endNs = process.hrtime.bigint(); + endpointDurationSummary.labels(routePath).observe(Number(endNs - startNs) / 1e9); }); next(); }); @@ -759,7 +759,7 @@ export default class Controller { async handleSystemInfoSubscription(request: lib.SubscriptionRequest) { const systems = [...this.systems.values()].filter( - metric => metric.updatedAt > request.lastRequestTime, + metric => metric.updatedAtMs > request.lastRequestTimeMs, ); return systems.length ? new lib.SystemInfoUpdateEvent(systems) : null; } @@ -780,7 +780,7 @@ export default class Controller { hostsUpdated(hosts: HostInfo[]) { const now = Date.now(); for (const host of hosts) { - host.updatedAt = now; + host.updatedAtMs = now; } this.hostsDirty = true; let updates = hosts.map(host => host.toHostDetails()); @@ -789,7 +789,7 @@ export default class Controller { async handleHostSubscription(request: lib.SubscriptionRequest) { const hosts = [...this.hosts.values()].filter( - host => host.updatedAt > request.lastRequestTime, + host => host.updatedAtMs > request.lastRequestTimeMs, ).map(host => host.toHostDetails()); return hosts.length ? new lib.HostUpdatesEvent(hosts) : null; } @@ -937,7 +937,7 @@ export default class Controller { ); } else { instance.status = "unassigned"; - instance.updatedAt = Date.now(); + instance.updatedAtMs = Date.now(); this.instanceDetailsUpdated([instance]); } } @@ -963,7 +963,7 @@ export default class Controller { let prev = instance.status; instance.status = "deleted"; - instance.updatedAt = Date.now(); + instance.updatedAtMs = Date.now(); this.instanceDetailsUpdated([instance]); await lib.invokeHook(this.plugins, "onInstanceStatusChanged", instance, prev); } @@ -991,7 +991,7 @@ export default class Controller { addInstanceHooks(instance: InstanceInfo) { instance.config.on("fieldChanged", (field: string, curr: any, prev: any) => { - instance.updatedAt = Date.now(); + instance.updatedAtMs = Date.now(); if (field === "instance.name") { this.instanceDetailsUpdated([instance]); } @@ -1008,7 +1008,7 @@ export default class Controller { async handleInstanceDetailsSubscription(request: lib.SubscriptionRequest) { const instances = [...this.instances.values()].filter( - instance => instance.updatedAt > request.lastRequestTime, + instance => instance.updatedAtMs > request.lastRequestTimeMs, ).map(instance => instance.toInstanceDetails()); return instances.length ? new lib.InstanceDetailsUpdatesEvent(instances) : null; } @@ -1019,7 +1019,7 @@ export default class Controller { } async handleInstanceSaveDetailsSubscription(request: lib.SubscriptionRequest) { - const saves = [...this.saves.values()].filter(save => save.updatedAt > request.lastRequestTime); + const saves = [...this.saves.values()].filter(save => save.updatedAtMs > request.lastRequestTimeMs); return saves.length ? new lib.InstanceSaveDetailsUpdatesEvent(saves) : null; } @@ -1027,7 +1027,7 @@ export default class Controller { modPacksUpdated(modPacks: lib.ModPack[]) { const now = Date.now(); for (const modPack of modPacks) { - modPack.updatedAt = now; + modPack.updatedAtMs = now; } this.modPacksDirty = true; this.subscriptions.broadcast(new lib.ModPackUpdatesEvent(modPacks)); @@ -1036,20 +1036,20 @@ export default class Controller { async handleModPackSubscription(request: lib.SubscriptionRequest) { const modPacks = [...this.modPacks.values()].filter( - modPack => modPack.updatedAt > request.lastRequestTime, + modPack => modPack.updatedAtMs > request.lastRequestTimeMs, ); return modPacks.length ? new lib.ModPackUpdatesEvent(modPacks) : null; } modsUpdated(mods: lib.ModInfo[]) { - // ModStore sets updatedAt for mods + // ModStore sets updatedAtMs for mods this.subscriptions.broadcast(new lib.ModUpdatesEvent(mods)); lib.invokeHook(this.plugins, "onModsUpdated", mods); } async handleModSubscription(request: lib.SubscriptionRequest) { const mods = [...this.modStore.files.values()].filter( - mod => mod.updatedAt > request.lastRequestTime, + mod => mod.updatedAtMs > request.lastRequestTimeMs, ); return mods.length ? new lib.ModUpdatesEvent(mods) : null; } @@ -1057,7 +1057,7 @@ export default class Controller { usersUpdated(users: ControllerUser[]) { const now = Date.now(); for (const user of users) { - user.updatedAt = now; + user.updatedAtMs = now; } this.userManager.dirty = true; this.subscriptions.broadcast(new lib.UserUpdatesEvent(users)); @@ -1065,7 +1065,7 @@ export default class Controller { async handleUserSubscription(request: lib.SubscriptionRequest) { const users = [...this.userManager.users.values()].filter( - user => user.updatedAt > request.lastRequestTime, + user => user.updatedAtMs > request.lastRequestTimeMs, ); return users.length ? new lib.UserUpdatesEvent(users) : null; } diff --git a/packages/controller/src/ControllerUser.ts b/packages/controller/src/ControllerUser.ts index 5af3b1c2..b8e83d9f 100644 --- a/packages/controller/src/ControllerUser.ts +++ b/packages/controller/src/ControllerUser.ts @@ -35,7 +35,7 @@ export default class ControllerUser extends User implements IControllerUser { json.is_banned, json.is_whitelisted, json.ban_reason, - json.updated_at, + json.updated_at_ms, json.is_deleted, playerStats, instanceStats, diff --git a/packages/controller/src/HostConnection.ts b/packages/controller/src/HostConnection.ts index ec078553..564b5b96 100644 --- a/packages/controller/src/HostConnection.ts +++ b/packages/controller/src/HostConnection.ts @@ -68,7 +68,7 @@ export default class HostConnection extends BaseConnection { instances.push(instance); let prev = instance.status; instance.status = "unknown"; - instance.updatedAt = now; + instance.updatedAtMs = now; lib.invokeHook(this._controller.plugins, "onInstanceStatusChanged", instance, prev); } this._controller.instanceDetailsUpdated(instances); @@ -185,7 +185,7 @@ export default class HostConnection extends BaseConnection { let prev = instance.status; instance.status = request.status; instance.gamePort = request.gamePort; - instance.updatedAt = Date.now(); + instance.updatedAtMs = Date.now(); logger.verbose(`Instance ${instance.config.get("instance.name")} State: ${instance.status}`); this._controller.instanceDetailsUpdated([instance]); await lib.invokeHook(this._controller.plugins, "onInstanceStatusChanged", instance, prev); @@ -222,7 +222,7 @@ export default class HostConnection extends BaseConnection { let prev = controllerInstance.status; controllerInstance.status = instanceData.status; controllerInstance.gamePort = instanceData.gamePort; - controllerInstance.updatedAt = Date.now(); + controllerInstance.updatedAtMs = Date.now(); logger.verbose(`Instance ${instanceConfig.get("instance.name")} State: ${instanceData.status}`); instanceUpdates.push(controllerInstance); await lib.invokeHook( @@ -283,7 +283,7 @@ export default class HostConnection extends BaseConnection { if (existingSave && save.equals(existingSave)) { continue; } - save.updatedAt = now; + save.updatedAtMs = now; this._controller.saves.set(save.id, save); updates.push(save); } @@ -292,7 +292,7 @@ export default class HostConnection extends BaseConnection { for (const [id, save] of this._controller.saves) { if (save.instanceId === event.instanceId && !updatedSaves.has(id)) { save.isDeleted = true; - save.updatedAt = now; + save.updatedAtMs = now; updates.push(save); this._controller.saves.delete(id); } diff --git a/packages/controller/src/HostInfo.ts b/packages/controller/src/HostInfo.ts index 93f41fd3..5d876e86 100644 --- a/packages/controller/src/HostInfo.ts +++ b/packages/controller/src/HostInfo.ts @@ -23,7 +23,7 @@ export default class HostInfo { /** Unix timestamp in seconds host token must be issued after to be valid */ public tokenValidAfter: number = 0, /** Millisecond Unix timestamp this entry was last updated at */ - public updatedAt: number = 0, + public updatedAtMs: number = 0, /** True if this host has been deleted */ public isDeleted: boolean = false, ) { } @@ -37,7 +37,7 @@ export default class HostInfo { "remote_address": Type.String(), "public_address": Type.String(), "token_valid_after": Type.Number(), - "updated_at": Type.Number(), + "updated_at_ms": Type.Number(), "is_deleted": Type.Boolean(), }); @@ -51,7 +51,7 @@ export default class HostInfo { json.remote_address, json.public_address, json.token_valid_after, - json.updated_at, + json.updated_at_ms, json.is_deleted, ); } @@ -66,7 +66,7 @@ export default class HostInfo { remote_address: this.remoteAddress, public_address: this.publicAddress, token_valid_after: this.tokenValidAfter, - updated_at: this.updatedAt, + updated_at_ms: this.updatedAtMs, is_deleted: this.isDeleted, }; } @@ -80,7 +80,7 @@ export default class HostInfo { this.remoteAddress, this.publicAddress, this.tokenValidAfter, - this.updatedAt, + this.updatedAtMs, this.isDeleted, ); } diff --git a/packages/controller/src/HttpCloser.ts b/packages/controller/src/HttpCloser.ts index 2631930c..599a9e86 100644 --- a/packages/controller/src/HttpCloser.ts +++ b/packages/controller/src/HttpCloser.ts @@ -67,10 +67,10 @@ export default class HttpCloser { * If there's requests in progress that still haven't been completed * after the timeout value passed they will be aborted. * - * @param timeout - + * @param timeoutMs - * Time in ms to wait for connections to close. */ - async close(timeout:number = 5000): Promise { + async close(timeoutMs: number = 5000): Promise { let activeSockets = new Set(); for (let response of this._responses) { if (!response.headersSent) { @@ -97,7 +97,7 @@ export default class HttpCloser { // Node.js, see nodejs/node#27428. socket.destroy(); } - }, timeout); + }, timeoutMs); this._server.close(err => { clearTimeout(timer); if (err) { diff --git a/packages/controller/src/InstanceInfo.ts b/packages/controller/src/InstanceInfo.ts index e7db6cbf..0534c20d 100644 --- a/packages/controller/src/InstanceInfo.ts +++ b/packages/controller/src/InstanceInfo.ts @@ -11,7 +11,7 @@ export default class InstanceInfo { public config: lib.InstanceConfig, public status: lib.InstanceStatus, public gamePort?: number, - public updatedAt = 0, + public updatedAtMs = 0, ) { this.config = config; this.status = status; @@ -21,7 +21,7 @@ export default class InstanceInfo { "config": lib.InstanceConfig.jsonSchema, "status": lib.InstanceStatus, "gamePort": Type.Optional(Type.Number()), - "updatedAt": Type.Optional(Type.Number()), + "updatedAtMs": Type.Optional(Type.Number()), }); static fromJSON(json: Static, location: lib.ConfigLocation) { @@ -29,7 +29,7 @@ export default class InstanceInfo { lib.InstanceConfig.fromJSON(json.config, location), json.status, json.gamePort, - json.updatedAt, + json.updatedAtMs, ); } @@ -40,7 +40,7 @@ export default class InstanceInfo { this.config.get("instance.assigned_host") ?? undefined, this.gamePort, this.status, - this.updatedAt, + this.updatedAtMs, ); } diff --git a/packages/controller/src/routes.ts b/packages/controller/src/routes.ts index ab67aeaf..51b25bee 100644 --- a/packages/controller/src/routes.ts +++ b/packages/controller/src/routes.ts @@ -56,7 +56,7 @@ async function getMetrics(req: Request, res: Response, next: any) { } let requests: Promise | null>[] = []; - let timeout = controller.config.get("controller.metrics_timeout") * 1000; + let timeoutMs = controller.config.get("controller.metrics_timeout") * 1000; for (let [hostId, hostConnection] of controller.wsServer.hostConnections) { if (!hostConnection.connected) { continue; @@ -68,7 +68,7 @@ async function getMetrics(req: Request, res: Response, next: any) { } return null; }), - timeout, null + timeoutMs, null )); } diff --git a/packages/host/src/Instance.ts b/packages/host/src/Instance.ts index b91d2a8b..c4e3d473 100644 --- a/packages/host/src/Instance.ts +++ b/packages/host/src/Instance.ts @@ -495,7 +495,7 @@ rcon.print(game.table_to_json(players))`.replace(/\r?\n/g, " "); } async _loadPlugin(pluginInfo: lib.PluginNodeEnvInfo, host: Host) { - let pluginLoadStarted = Date.now(); + let pluginLoadStartedMs = Date.now(); let InstancePluginClass = await lib.loadPluginClass( pluginInfo.name, path.posix.join(pluginInfo.requirePath, pluginInfo.instanceEntrypoint!), @@ -506,7 +506,7 @@ rcon.print(game.table_to_json(players))`.replace(/\r?\n/g, " "); this.plugins.set(pluginInfo.name, instancePlugin); await instancePlugin.init(); - this.logger.info(`Loaded plugin ${pluginInfo.name} in ${Date.now() - pluginLoadStarted}ms`); + this.logger.info(`Loaded plugin ${pluginInfo.name} in ${Date.now() - pluginLoadStartedMs}ms`); } async _loadStats() { diff --git a/packages/host/src/server.ts b/packages/host/src/server.ts index 81d6936f..b34af3b9 100644 --- a/packages/host/src/server.ts +++ b/packages/host/src/server.ts @@ -427,7 +427,7 @@ export interface FactorioServerOptions { */ maxConcurrentCommands?: number /** Timeout in ms to wait for a response to a shutdown command. */ - hangTimeout?: number, + hangTimeoutMs?: number, } /** @@ -463,7 +463,7 @@ export class FactorioServer extends events.EventEmitter { /** Enable verbose logging */ verboseLogging: boolean; /** Timeout in ms to wait for a response to a shutdown command */ - hangTimeout = 5000; + hangTimeoutMs = 5000; _factorioDir: string; _writeDir: string; @@ -519,7 +519,7 @@ export class FactorioServer extends events.EventEmitter { /** Maximum number of RCON commands transmitted in parallel on the RCON connection */ this.maxConcurrentCommands = options.maxConcurrentCommands || 5; /** Timeout in ms to wait for a response to a shutdown command */ - this.hangTimeout = options.hangTimeout ?? 5000; + this.hangTimeoutMs = options.hangTimeoutMs ?? 5000; if (options.stripPaths) { let charSet = new Set(path.resolve(this.writePath("temp"))); @@ -1012,7 +1012,7 @@ export class FactorioServer extends events.EventEmitter { } this._server!.kill("SIGKILL"); } - }, this.hangTimeout); + }, this.hangTimeoutMs); // It's possible the server decides to exit on its own, in which // case the stop operation has to be cancelled. diff --git a/packages/lib/src/ExponentialBackoff.ts b/packages/lib/src/ExponentialBackoff.ts index 38b13eb9..06616381 100644 --- a/packages/lib/src/ExponentialBackoff.ts +++ b/packages/lib/src/ExponentialBackoff.ts @@ -30,7 +30,7 @@ export default class ExponentialBackoff { reset: number; private _exp = 0; - private _lastInvocationTime = Date.now(); + private _lastInvocationTimeMs = Date.now(); /** * Construct ExponentialBackoff helper @@ -65,9 +65,9 @@ export default class ExponentialBackoff { * @returns time in milliseconds to wait. */ delay() { - let invocationTime = Date.now(); - let interval = (invocationTime - this._lastInvocationTime) / 1000; - this._lastInvocationTime = invocationTime; + let invocationTimeMs = Date.now(); + let interval = (invocationTimeMs - this._lastInvocationTimeMs) / 1000; + this._lastInvocationTimeMs = invocationTimeMs; if (interval > this.reset) { this._exp = 0; diff --git a/packages/lib/src/ModStore.ts b/packages/lib/src/ModStore.ts index 8240d8f5..1819e645 100644 --- a/packages/lib/src/ModStore.ts +++ b/packages/lib/src/ModStore.ts @@ -56,7 +56,7 @@ export default class ModStore extends TypedEventEmitter void; - private _lastRun = 0; + private _lastRunMs = 0; private _runTimeout?: ReturnType; /** @@ -60,15 +60,15 @@ export default class RateLimiter { */ activate() { let now = Date.now(); - if (now < this._lastRun + 1000 / this.maxRate) { + if (now < this._lastRunMs + 1000 / this.maxRate) { if (!this._runTimeout && this.action) { this._runTimeout = setTimeout(() => { this._runTimeout = undefined; - this._lastRun = Date.now(); + this._lastRunMs = Date.now(); if (this.action) { this.action(); } - }, this._lastRun + 1000 / this.maxRate - now); + }, this._lastRunMs + 1000 / this.maxRate - now); } return false; } @@ -79,7 +79,7 @@ export default class RateLimiter { this._runTimeout = undefined; } - this._lastRun = now; + this._lastRunMs = now; if (this.action) { this.action(); } diff --git a/packages/lib/src/data/ModInfo.ts b/packages/lib/src/data/ModInfo.ts index fb29a8d4..b555f0bb 100644 --- a/packages/lib/src/data/ModInfo.ts +++ b/packages/lib/src/data/ModInfo.ts @@ -122,7 +122,7 @@ export default class ModInfo { sha1?: string; /** Millisecond Unix timestamp this entry was last updated at */ - updatedAt = 0; + updatedAtMs = 0; /** * True if this mod has been deleted @@ -148,7 +148,7 @@ export default class ModInfo { ...this.infoJsonSchema.properties, "size": Type.Optional(Type.Integer()), "sha1": Type.Optional(Type.String()), - "updated_at": Type.Optional(Type.Number()), + "updated_at_ms": Type.Optional(Type.Number()), "is_deleted": Type.Optional(Type.Boolean()), }); @@ -171,7 +171,7 @@ export default class ModInfo { // Additional data if (json.size) { modInfo.size = json.size; } if (json.sha1) { modInfo.sha1 = json.sha1; } - if (json.updated_at) { modInfo.updatedAt = json.updated_at; } + if (json.updated_at_ms) { modInfo.updatedAtMs = json.updated_at_ms; } if (json.is_deleted) { modInfo.isDeleted = json.is_deleted; } return modInfo; @@ -193,7 +193,7 @@ export default class ModInfo { } if (this.size) { json.size = this.size; } if (this.sha1) { json.sha1 = this.sha1; } - if (this.updatedAt) { json.updated_at = this.updatedAt; } + if (this.updatedAtMs) { json.updated_at_ms = this.updatedAtMs; } if (this.isDeleted) { json.is_deleted = this.isDeleted; } return json; } @@ -229,7 +229,7 @@ export default class ModInfo { size: stat.size, sha1: await libHash.hashFile(modPath), - updated_at: stat.mtimeMs, + updated_at_ms: stat.mtimeMs, is_deleted: false, }); } diff --git a/packages/lib/src/data/ModPack.ts b/packages/lib/src/data/ModPack.ts index f5761137..2aec7b63 100644 --- a/packages/lib/src/data/ModPack.ts +++ b/packages/lib/src/data/ModPack.ts @@ -102,7 +102,7 @@ export default class ModPack { exportManifest?: ExportManifest; /** Millisecond Unix timestamp this entry was last updated at */ - updatedAt = 0; + updatedAtMs = 0; /** * True if this mod pack has been deleted from the list of mod packs. @@ -118,7 +118,7 @@ export default class ModPack { clone.mods = this.mods; clone.settings = this.settings; clone.exportManifest = this.exportManifest; - clone.updatedAt = this.updatedAt; + clone.updatedAtMs = this.updatedAtMs; clone.isDeleted = this.isDeleted; return clone; } @@ -135,7 +135,7 @@ export default class ModPack { "runtime-per-user": ModSettingsJsonSchema, }), "export_manifest": Type.Optional(ExportManifest.jsonSchema), - "updated_at": Type.Optional(Type.Number()), + "updated_at_ms": Type.Optional(Type.Number()), "is_deleted": Type.Optional(Type.Boolean()), }); @@ -161,7 +161,7 @@ export default class ModPack { }; } if (json.export_manifest) { modPack.exportManifest = ExportManifest.fromJSON(json.export_manifest); } - if (json.updated_at) { modPack.updatedAt = json.updated_at; } + if (json.updated_at_ms) { modPack.updatedAtMs = json.updated_at_ms; } if (json.is_deleted) { modPack.isDeleted = json.is_deleted; } if (!modPack.mods.has("base")) { @@ -185,7 +185,7 @@ export default class ModPack { }, }; if (this.exportManifest) { json.export_manifest = this.exportManifest; } - if (this.updatedAt) { json.updated_at = this.updatedAt; } + if (this.updatedAtMs) { json.updated_at_ms = this.updatedAtMs; } if (this.isDeleted) { json.is_deleted = this.isDeleted; } return json; } @@ -194,7 +194,7 @@ export default class ModPack { const json = this.toJSON(); delete json.id; delete json.export_manifest; - delete json.updated_at; + delete json.updated_at_ms; delete json.is_deleted; // eslint-disable-next-line node/no-sync diff --git a/packages/lib/src/data/PlayerStats.ts b/packages/lib/src/data/PlayerStats.ts index d03569c3..61ca3aed 100644 --- a/packages/lib/src/data/PlayerStats.ts +++ b/packages/lib/src/data/PlayerStats.ts @@ -37,30 +37,33 @@ export default class PlayerStats { static jsonSchema = Type.Object({ "join_count": Type.Optional(Type.Integer()), "online_time_ms": Type.Optional(Type.Number()), - "first_join_at": Type.Optional(Type.Number()), + "first_join_at_ms": Type.Optional(Type.Number()), + "last_join_at_ms": Type.Optional(Type.Number()), + "last_leave_at_ms": Type.Optional(Type.Number()), + "last_leave_reason": Type.Optional(Type.String()), + // migrate: renamed fields from alpha 13 "last_join_at": Type.Optional(Type.Number()), "last_leave_at": Type.Optional(Type.Number()), - "last_leave_reason": Type.Optional(Type.String()), }); constructor(json: Static = {}) { if (json["join_count"]) { this.joinCount = json["join_count"]; } - if (json["first_join_at"]) { - this.firstJoinAt = new Date(json["first_join_at"]); + if (json["first_join_at_ms"]) { + this.firstJoinAt = new Date(json["first_join_at_ms"]); } if (json["online_time_ms"]) { this.onlineTimeMs = json["online_time_ms"]; } - if (json["last_join_at"]) { - this.lastJoinAt = new Date(json["last_join_at"]); + if (json["last_join_at_ms"] ?? json["last_join_at"]) { + this.lastJoinAt = new Date(json["last_join_at_ms"] ?? json["last_join_at"]!); if (!this.firstJoinAt) { // migrate: pre-alpha 14 did not have this field. this.firstJoinAt = this.lastJoinAt; } } - if (json["last_leave_at"]) { - this.lastLeaveAt = new Date(json["last_leave_at"]); + if (json["last_leave_at_ms"] ?? json["last_leave_at"]) { + this.lastLeaveAt = new Date(json["last_leave_at_ms"] ?? json["last_leave_at"]!); this.lastLeaveReason = json["last_leave_reason"] || "quit"; } } @@ -78,13 +81,13 @@ export default class PlayerStats { json["online_time_ms"] = this.onlineTimeMs; } if (this.firstJoinAt) { - json["first_join_at"] = this.firstJoinAt.getTime(); + json["first_join_at_ms"] = this.firstJoinAt.getTime(); } if (this.lastJoinAt) { - json["last_join_at"] = this.lastJoinAt.getTime(); + json["last_join_at_ms"] = this.lastJoinAt.getTime(); } if (this.lastLeaveAt) { - json["last_leave_at"] = this.lastLeaveAt.getTime(); + json["last_leave_at_ms"] = this.lastLeaveAt.getTime(); json["last_leave_reason"] = this.lastLeaveReason; } return json; diff --git a/packages/lib/src/data/User.ts b/packages/lib/src/data/User.ts index 3db13af8..9cae6fde 100644 --- a/packages/lib/src/data/User.ts +++ b/packages/lib/src/data/User.ts @@ -24,7 +24,7 @@ export default class User { /** Reason for being banned. Ignored if isBanned is false. */ public banReason = "", /** Millisecond Unix timestamp this entry was last updated at */ - public updatedAt = 0, + public updatedAtMs = 0, /** True if this user object has been removed from the cluster. */ public isDeleted = false, /** Combined statistics for the player this user account is tied to. */ @@ -42,7 +42,7 @@ export default class User { is_banned: Type.Optional(Type.Boolean()), is_whitelisted: Type.Optional(Type.Boolean()), ban_reason: Type.Optional(Type.String()), - updated_at: Type.Optional(Type.Number()), + updated_at_ms: Type.Optional(Type.Number()), is_deleted: Type.Optional(Type.Boolean()), instance_stats: Type.Optional( Type.Array( @@ -67,7 +67,7 @@ export default class User { json.is_banned, json.is_whitelisted, json.ban_reason, - json.updated_at, + json.updated_at_ms, json.is_deleted, playerStats, instanceStats, @@ -103,8 +103,8 @@ export default class User { json.ban_reason = this.banReason; } - if (this.updatedAt) { - json.updated_at = this.updatedAt; + if (this.updatedAtMs) { + json.updated_at_ms = this.updatedAtMs; } if (this.isDeleted) { diff --git a/packages/lib/src/data/messages_controller.ts b/packages/lib/src/data/messages_controller.ts index 11477a53..e24d0be4 100644 --- a/packages/lib/src/data/messages_controller.ts +++ b/packages/lib/src/data/messages_controller.ts @@ -239,7 +239,7 @@ export class SystemInfo { public diskAvailable: number, public canRestart: boolean, /** Millisecond Unix timestamp this entry was last updated at */ - public updatedAt: number, + public updatedAtMs: number, public isDeleted: boolean, ) { } @@ -256,7 +256,7 @@ export class SystemInfo { "diskCapacity": Type.Number(), "diskAvailable": Type.Number(), "canRestart": Type.Boolean(), - "updatedAt": Type.Number(), + "updatedAtMs": Type.Number(), "isDeleted": Type.Boolean(), }); @@ -274,7 +274,7 @@ export class SystemInfo { json.diskCapacity, json.diskAvailable, json.canRestart, - json.updatedAt, + json.updatedAtMs, json.isDeleted, ); } diff --git a/packages/lib/src/data/messages_host.ts b/packages/lib/src/data/messages_host.ts index 62a95647..5061724d 100644 --- a/packages/lib/src/data/messages_host.ts +++ b/packages/lib/src/data/messages_host.ts @@ -84,7 +84,7 @@ export class HostDetails { public publicAddress: string = "", public tokenValidAfter: number = 0, /** Millisecond Unix timestamp this entry was last updated at */ - public updatedAt = 0, + public updatedAtMs = 0, public isDeleted = false, ) { } @@ -96,7 +96,7 @@ export class HostDetails { "remoteAddress": Type.Optional(Type.String()), "publicAddress": Type.Optional(Type.String()), "tokenValidAfter": Type.Optional(Type.Number()), - "updatedAt": Type.Optional(Type.Number()), + "updatedAtMs": Type.Optional(Type.Number()), "isDeleted": Type.Optional(Type.Boolean()), }); @@ -109,7 +109,7 @@ export class HostDetails { json.remoteAddress, json.publicAddress, json.tokenValidAfter, - json.updatedAt, + json.updatedAtMs, json.isDeleted, ); } diff --git a/packages/lib/src/data/messages_instance.ts b/packages/lib/src/data/messages_instance.ts index b943ea6c..ff29d886 100644 --- a/packages/lib/src/data/messages_instance.ts +++ b/packages/lib/src/data/messages_instance.ts @@ -43,7 +43,7 @@ export class InstanceDetails { public gamePort: number | undefined, public status: InstanceStatus, /** Millisecond Unix timestamp this entry was last updated at */ - public updatedAt = 0, + public updatedAtMs = 0, ) { } static jsonSchema = Type.Object({ @@ -55,7 +55,7 @@ export class InstanceDetails { "unknown", "unassigned", "stopped", "starting", "running", "stopping", "creating_save", "exporting_data", "deleted", ]), - "updatedAt": Type.Optional(Type.Number()), + "updatedAtMs": Type.Optional(Type.Number()), }); static fromJSON(json: Static) { @@ -65,7 +65,7 @@ export class InstanceDetails { json.assignedHost, json.gamePort, json.status, - json.updatedAt, + json.updatedAtMs, ); } @@ -289,7 +289,7 @@ export class SaveDetails { public loaded: boolean, public loadByDefault: boolean, /** Millisecond Unix timestamp this entry was last updated at */ - public updatedAt: number, + public updatedAtMs: number, public isDeleted: boolean, ) { } @@ -301,7 +301,7 @@ export class SaveDetails { "mtimeMs": Type.Number(), "loaded": Type.Boolean(), "loadByDefault": Type.Boolean(), - "updatedAt": Type.Number(), + "updatedAtMs": Type.Number(), "isDeleted": Type.Boolean(), }); @@ -314,7 +314,7 @@ export class SaveDetails { json.mtimeMs, json.loaded, json.loadByDefault, - json.updatedAt, + json.updatedAtMs, json.isDeleted, ); } @@ -330,8 +330,8 @@ export class SaveDetails { * @returns true if this save is identical to the provided save */ equals(other: SaveDetails) { - // Note that updatedAt is not included in the equality check because - // instances send the whole list of saves with updatedAt set to zero + // Note that updatedAtMs is not included in the equality check because + // instances send the whole list of saves with updatedAtMs set to zero // to the controller whenever any save changes and this equals check // is used to filter whether a save was updated. return ( diff --git a/packages/lib/src/helpers.ts b/packages/lib/src/helpers.ts index 24cb3477..1c92c4d7 100644 --- a/packages/lib/src/helpers.ts +++ b/packages/lib/src/helpers.ts @@ -22,10 +22,10 @@ export function basicType(value: unknown) { /** * Asynchronously wait for the given duration * - * @param duration - Time to wait for in milliseconds. + * @param durationMs - Time to wait for in milliseconds. */ -export async function wait(duration: number) { - await new Promise(resolve => { setTimeout(resolve, duration); }); +export async function wait(durationMs: number) { + await new Promise(resolve => { setTimeout(resolve, durationMs); }); } @@ -33,16 +33,16 @@ export async function wait(duration: number) { * Resolve a promise with a timeout. * * @param {Promise} promise - Promise to wait for. - * @param {number} time - Maximum time im milliseconds to wait for. + * @param {number} limitMs - Maximum time im milliseconds to wait for. * @param {*=} timeoutResult - Value to return if the operation timed out. */ -export async function timeout(promise: Promise, time: number, timeoutResult: T) { +export async function timeout(promise: Promise, limitMs: number, timeoutResult: T) { let timer: ReturnType | undefined; try { return await Promise.race([ promise, new Promise(resolve => { - timer = setTimeout(() => resolve(timeoutResult), time); + timer = setTimeout(() => resolve(timeoutResult), limitMs); }), ]); diff --git a/packages/lib/src/link/connectors.ts b/packages/lib/src/link/connectors.ts index 918fbb6d..2bc15f19 100644 --- a/packages/lib/src/link/connectors.ts +++ b/packages/lib/src/link/connectors.ts @@ -95,7 +95,7 @@ export abstract class WebSocketBaseConnector extends BaseConnector { _state: ConnectorState = "closed"; _closing = false; _socket: WebSocketClusterio | null = null; - _lastHeartbeat: number | null = null; + _lastHeartbeatMs: number | null = null; _heartbeatId: ReturnType | null = null; _heartbeatInterval: number | null = null; _lastReceivedSeq = undefined; @@ -105,7 +105,7 @@ export abstract class WebSocketBaseConnector extends BaseConnector { this._state = "closed"; this._closing = false; assert(this._socket === null); - this._lastHeartbeat = null; + this._lastHeartbeatMs = null; assert(this._heartbeatId === null); this._heartbeatInterval = null; this._lastReceivedSeq = undefined; @@ -117,7 +117,7 @@ export abstract class WebSocketBaseConnector extends BaseConnector { this._state = "connecting"; assert(this._closing === false); assert(this._socket); - this._lastHeartbeat = null; + this._lastHeartbeatMs = null; assert(this._heartbeatId === null); this._heartbeatInterval = null; this._lastReceivedSeq = undefined; @@ -192,7 +192,7 @@ export abstract class WebSocketBaseConnector extends BaseConnector { _doHeartbeat() { this._check("connected"); - if (Date.now() - this._lastHeartbeat! > 2000 * this._heartbeatInterval!) { + if (Date.now() - this._lastHeartbeatMs! > 2000 * this._heartbeatInterval!) { logger.verbose("Connector | closing after heartbeat timed out"); // eslint-disable-next-line node/no-process-env if (process.env.APP_ENV === "browser") { @@ -207,7 +207,7 @@ export abstract class WebSocketBaseConnector extends BaseConnector { } _processHeartbeat(message: libData.MessageHeartbeat) { - this._lastHeartbeat = Date.now(); + this._lastHeartbeatMs = Date.now(); this._dropSendBufferSeq(message.seq); } @@ -218,7 +218,7 @@ export abstract class WebSocketBaseConnector extends BaseConnector { if (this._heartbeatId) { throw new Error("heartbeat is already running"); } - this._lastHeartbeat = Date.now(); + this._lastHeartbeatMs = Date.now(); this._heartbeatId = setInterval(() => { this._doHeartbeat(); }, this._heartbeatInterval! * 1000); @@ -327,7 +327,7 @@ export abstract class WebSocketClientConnector extends WebSocketBaseConnector { _reconnectId?: ReturnType; _sessionToken: string | null = null; _sessionTimeout: number | null = null; - _startedResuming: number | null = null; + _startedResumingMs: number | null = null; _backoff: ExponentialBackoff; constructor( @@ -350,7 +350,7 @@ export abstract class WebSocketClientConnector extends WebSocketBaseConnector { this._reconnectId = undefined; this._sessionToken = null; this._sessionTimeout = null; - this._startedResuming = null; + this._startedResumingMs = null; super._reset(); this.src = undefined as any; } @@ -359,7 +359,7 @@ export abstract class WebSocketClientConnector extends WebSocketBaseConnector { assert(this._reconnectId === undefined); this._sessionToken = null; this._sessionTimeout = null; - this._startedResuming = null; + this._startedResumingMs = null; super._invalidate(); this.src = undefined as any; } @@ -473,10 +473,10 @@ export abstract class WebSocketClientConnector extends WebSocketBaseConnector { logger.error("Unexpected double call to reconnect"); } - let delay = this._backoff.delay(); + let delayMs = this._backoff.delay(); if ( this._state === "resuming" - && this._startedResuming! + this._sessionTimeout! * 1000 < Date.now() + delay + && this._startedResumingMs! + this._sessionTimeout! * 1000 < Date.now() + delayMs ) { logger.error("Connector | Session timed out trying to resume"); this._reset(); @@ -489,12 +489,12 @@ export abstract class WebSocketClientConnector extends WebSocketBaseConnector { } } logger.verbose( - `Connector | waiting ${(Math.round(delay / 10) / 100)} seconds for reconnect` + `Connector | waiting ${(Math.round(delayMs / 10) / 100)} seconds for reconnect` ); this._reconnectId = setTimeout(() => { this._reconnectId = undefined; this._doConnect(); - }, delay); + }, delayMs); } _attachSocketHandlers() { @@ -516,7 +516,7 @@ export abstract class WebSocketClientConnector extends WebSocketBaseConnector { } else { this._state = "resuming"; - this._startedResuming = Date.now(); + this._startedResumingMs = Date.now(); this.reconnect(); this.emit("drop"); } @@ -625,7 +625,7 @@ export abstract class WebSocketClientConnector extends WebSocketBaseConnector { for (let bufferedMessage of this._sendBuffer) { this._sendInternal(bufferedMessage); } - this._startedResuming = null; + this._startedResumingMs = null; this.emit("resume"); } else if (type === "invalidate") { diff --git a/packages/lib/src/prometheus.ts b/packages/lib/src/prometheus.ts index d3359fe7..13c94ee0 100644 --- a/packages/lib/src/prometheus.ts +++ b/packages/lib/src/prometheus.ts @@ -740,10 +740,10 @@ class GaugeChild extends ValueCollectorChild { * seconds from when the timer was started */ startTimer() { - let start = process.hrtime.bigint(); + let startNs = process.hrtime.bigint(); return () => { - let end = process.hrtime.bigint(); - this.set(Number(end - start) / 1e9); + let endNs = process.hrtime.bigint(); + this.set(Number(endNs - startNs) / 1e9); }; } } @@ -979,10 +979,10 @@ class SummaryChild { * when the timer was started into the metric. */ startTimer() { - let start = process.hrtime.bigint(); + let startNs = process.hrtime.bigint(); return () => { - let end = process.hrtime.bigint(); - this.observe(Number(end - start) / 1e9); + let endNs = process.hrtime.bigint(); + this.observe(Number(endNs - startNs) / 1e9); }; } } @@ -1260,10 +1260,10 @@ class HistogramChild { * when the timer was started into the metric. */ startTimer() { - let start = process.hrtime.bigint(); + let startNs = process.hrtime.bigint(); return () => { - let end = process.hrtime.bigint(); - this.observe(Number(end - start) / 1e9); + let endNs = process.hrtime.bigint(); + this.observe(Number(endNs - startNs) / 1e9); }; } } diff --git a/packages/lib/src/subscriptions.ts b/packages/lib/src/subscriptions.ts index abf237cd..de155090 100644 --- a/packages/lib/src/subscriptions.ts +++ b/packages/lib/src/subscriptions.ts @@ -33,7 +33,7 @@ export class SubscriptionRequest { constructor( public eventName: string, public subscribe: boolean, - public lastRequestTime: number = 0, + public lastRequestTimeMs: number = 0, ) { if (!Link._eventsByName.has(eventName)) { throw new Error(`Unregistered Event class ${eventName}`); @@ -47,7 +47,7 @@ export class SubscriptionRequest { ]); toJSON() { - return [this.eventName, this.subscribe, this.lastRequestTime]; + return [this.eventName, this.subscribe, this.lastRequestTimeMs]; } static fromJSON(json: Static): SubscriptionRequest { @@ -152,7 +152,7 @@ export class SubscriptionController { export interface SubscribableValue { id: number | string, - updatedAt: number, + updatedAtMs: number, isDeleted: boolean, } @@ -175,8 +175,8 @@ export class EventSubscriber< /** True if this subscriber is currently synced with the source */ synced = false; _snapshot?: readonly [ReadonlyMap>, boolean]; - _snapshotLastUpdated = 0; - lastResponseTime = -1; + _snapshotLastUpdatedMs = 0; + lastResponseTimeMs = -1; constructor( private Event: EventClass, @@ -203,7 +203,7 @@ export class EventSubscriber< */ async _handle(event: EventSubscribable) { for (const value of event.updates) { - this.lastResponseTime = Math.max(this.lastResponseTime, value.updatedAt); + this.lastResponseTimeMs = Math.max(this.lastResponseTimeMs, value.updatedAtMs); if (value.isDeleted) { this.values.delete(value.id as K); } else { @@ -257,8 +257,8 @@ export class EventSubscriber< * @returns tuple of values map snapshot and synced property. */ getSnapshot() { - if (this._snapshotLastUpdated !== this.lastResponseTime) { - this._snapshotLastUpdated = this.lastResponseTime; + if (this._snapshotLastUpdatedMs !== this.lastResponseTimeMs) { + this._snapshotLastUpdatedMs = this.lastResponseTimeMs; this._snapshot = [new Map(this.values), this.synced]; } return this._snapshot!; @@ -277,10 +277,10 @@ export class EventSubscriber< await this.control.send(new SubscriptionRequest( entry.name, this._callbacks.length > 0, - this.lastResponseTime + this.lastResponseTimeMs )); this.synced = this._callbacks.length > 0; - this._snapshotLastUpdated = 0; + this._snapshotLastUpdatedMs = 0; for (let callback of this._callbacks) { callback([], this.synced); } diff --git a/packages/web_ui/src/util/time_format.ts b/packages/web_ui/src/util/time_format.ts index e071a6ec..69082573 100644 --- a/packages/web_ui/src/util/time_format.ts +++ b/packages/web_ui/src/util/time_format.ts @@ -31,7 +31,7 @@ export function formatDuration(ms: number): string { return result; } -export function formatTimestamp(timestamp: number): string { +export function formatTimestamp(timestampMs: number): string { const options = { year: "numeric", month: "numeric", @@ -39,5 +39,5 @@ export function formatTimestamp(timestamp: number): string { hour: "numeric", minute: "numeric", } as const; - return new Intl.DateTimeFormat(undefined, options).format(new Date(timestamp)); + return new Intl.DateTimeFormat(undefined, options).format(new Date(timestampMs)); } diff --git a/plugins/inventory_sync/controller.ts b/plugins/inventory_sync/controller.ts index 33f0f192..6ce19850 100644 --- a/plugins/inventory_sync/controller.ts +++ b/plugins/inventory_sync/controller.ts @@ -36,7 +36,7 @@ async function saveDatabase( } export class ControllerPlugin extends BaseControllerPlugin { - acquiredPlayers!: Map; + acquiredPlayers!: Map; playerDatastore!: Map; playerDatastoreDirty = false; @@ -62,18 +62,18 @@ export class ControllerPlugin extends BaseControllerPlugin { } if (["unknown", "stopped"].includes(instance.status)) { - let timeout = this.controller.config.get("inventory_sync.player_lock_timeout") * 1000; + let timeoutMs = this.controller.config.get("inventory_sync.player_lock_timeout") * 1000; for (let acquisitonRecord of this.acquiredPlayers.values()) { - if (acquisitonRecord.instanceId === instanceId && !acquisitonRecord.expires) { - acquisitonRecord.expires = Date.now() + timeout; + if (acquisitonRecord.instanceId === instanceId && !acquisitonRecord.expiresMs) { + acquisitonRecord.expiresMs = Date.now() + timeoutMs; } } } if (instance.status === "running") { for (let acquisitonRecord of this.acquiredPlayers.values()) { - if (acquisitonRecord.instanceId === instanceId && acquisitonRecord.expires) { - delete acquisitonRecord.expires; + if (acquisitonRecord.instanceId === instanceId && acquisitonRecord.expiresMs) { + delete acquisitonRecord.expiresMs; } } } @@ -85,7 +85,7 @@ export class ControllerPlugin extends BaseControllerPlugin { !acquisitionRecord || acquisitionRecord.instanceId === instanceId || !this.controller.instances.has(acquisitionRecord.instanceId) - || acquisitionRecord.expires && acquisitionRecord.expires < Date.now() + || acquisitionRecord.expiresMs && acquisitionRecord.expiresMs < Date.now() ) { this.acquiredPlayers.set(playerName, { instanceId }); return true; diff --git a/plugins/player_auth/controller.ts b/plugins/player_auth/controller.ts index bc0a494b..838c6eab 100644 --- a/plugins/player_auth/controller.ts +++ b/plugins/player_auth/controller.ts @@ -26,7 +26,7 @@ async function generateCode(length: number): Promise { } -type PlayerCode = { playerCode: string, verifyCode: string | null, expires: number }; +type PlayerCode = { playerCode: string, verifyCode: string | null, expiresMs: number }; export class ControllerPlugin extends BaseControllerPlugin { players!: Map; @@ -39,7 +39,7 @@ export class ControllerPlugin extends BaseControllerPlugin { setInterval(() => { let now = Date.now(); for (let [player, entry] of this.players) { - if (entry.expires < now) { + if (entry.expiresMs < now) { this.players.delete(player); } } @@ -88,13 +88,13 @@ export class ControllerPlugin extends BaseControllerPlugin { } for (let entry of this.players.values()) { - if (entry.playerCode === playerCode && entry.expires > Date.now()) { + if (entry.playerCode === playerCode && entry.expiresMs > Date.now()) { let verifyCode = await generateCode(this.controller.config.get("player_auth.code_length")); let secret = Buffer.from(this.controller.config.get("controller.auth_secret"), "base64"); let verifyToken = jwt.sign( { aud: "player_auth.verify_code", - exp: Math.floor(entry.expires / 1000), + exp: Math.floor(entry.expiresMs / 1000), verify_code: verifyCode, player_code: playerCode, }, @@ -150,7 +150,7 @@ export class ControllerPlugin extends BaseControllerPlugin { } for (let [player, entry] of this.players) { - if (entry.playerCode === playerCode && entry.expires > Date.now()) { + if (entry.playerCode === playerCode && entry.expiresMs > Date.now()) { if (entry.verifyCode === verifyCode) { let user = this.controller.userManager.users.get(player); if (!user) { @@ -174,8 +174,8 @@ export class ControllerPlugin extends BaseControllerPlugin { async handleFetchPlayerCodeRequest(request: FetchPlayerCodeRequest) { let playerCode = await generateCode(this.controller.config.get("player_auth.code_length")); - let expires = Date.now() + this.controller.config.get("player_auth.code_timeout") * 1000; - this.players.set(request.player, { playerCode, verifyCode: null, expires }); + let expiresMs = Date.now() + this.controller.config.get("player_auth.code_timeout") * 1000; + this.players.set(request.player, { playerCode, verifyCode: null, expiresMs }); return { player_code: playerCode, controller_url: this.controller.getControllerUrl() }; } @@ -183,7 +183,7 @@ export class ControllerPlugin extends BaseControllerPlugin { let { player, verifyCode } = request; let entry = this.players.get(player); - if (!entry || entry.expires < Date.now()) { + if (!entry || entry.expiresMs < Date.now()) { throw new lib.RequestError("invalid player"); } diff --git a/plugins/player_auth/test/plugin.js b/plugins/player_auth/test/plugin.js index fcbcf060..1d523fbc 100644 --- a/plugins/player_auth/test/plugin.js +++ b/plugins/player_auth/test/plugin.js @@ -123,8 +123,8 @@ describe("player_auth", function() { } }); it("should return invalid player_code if the code is expired", async function() { - let expires = Date.now() - 1000; - controllerPlugin.players.set("expired", { playerCode: "expried", verifyCode: null, expires }); + let expiresMs = Date.now() - 1000; + controllerPlugin.players.set("expired", { playerCode: "expired", verifyCode: null, expiresMs }); let result = await phin({ url: `${controllerUrl}/api/player_auth/player_code`, method: "POST", @@ -134,8 +134,8 @@ describe("player_auth", function() { assert.deepEqual(result.body, { error: true, message: "invalid player_code" }); }); it("should return a verify code and token if code is valid", async function() { - let expires = Date.now() + 1000; - controllerPlugin.players.set("valid", { playerCode: "valid", verifyCode: null, expires }); + let expiresMs = Date.now() + 1000; + controllerPlugin.players.set("valid", { playerCode: "valid", verifyCode: null, expiresMs }); let result = await phin({ url: `${controllerUrl}/api/player_auth/player_code`, method: "POST", @@ -214,8 +214,8 @@ describe("player_auth", function() { await verify({ aud: "not_player_auth" }); }); it("should return invalid player_code if the code is expired", async function() { - let expires = Date.now() - 1000; - controllerPlugin.players.set("expired", { playerCode: "expried", verifyCode: "verify", expires }); + let expiresMs = Date.now() - 1000; + controllerPlugin.players.set("expired", { playerCode: "expired", verifyCode: "verify", expiresMs }); let result = await phin({ url: `${controllerUrl}/api/player_auth/verify`, method: "POST", @@ -233,8 +233,11 @@ describe("player_auth", function() { assert.deepEqual(result.body, { error: true, message: "invalid player_code" }); }); it("should return verified false if verify code has not yet been set", async function() { - let expires = Date.now() + 1000; - controllerPlugin.players.set("unverified", { playerCode: "unverified", verifyCode: null, expires }); + let expiresMs = Date.now() + 1000; + controllerPlugin.players.set( + "unverified", + { playerCode: "unverified", verifyCode: null, expiresMs }, + ); let result = await phin({ url: `${controllerUrl}/api/player_auth/verify`, method: "POST", @@ -252,8 +255,8 @@ describe("player_auth", function() { assert.deepEqual(result.body, { verified: false }); }); it("should return error if user is missing", async function() { - let expires = Date.now() + 1000; - controllerPlugin.players.set("missing", { playerCode: "missing", verifyCode: "verify", expires }); + let expiresMs = Date.now() + 1000; + controllerPlugin.players.set("missing", { playerCode: "missing", verifyCode: "verify", expiresMs }); let result = await phin({ url: `${controllerUrl}/api/player_auth/verify`, method: "POST", @@ -271,8 +274,8 @@ describe("player_auth", function() { assert.deepEqual(result.body, { error: true, message: "invalid user" }); }); it("should return verified true with token if valid verification", async function() { - let expires = Date.now() + 1000; - controllerPlugin.players.set("player", { playerCode: "player", verifyCode: "verify", expires }); + let expiresMs = Date.now() + 1000; + controllerPlugin.players.set("player", { playerCode: "player", verifyCode: "verify", expiresMs }); let result = await phin({ url: `${controllerUrl}/api/player_auth/verify`, method: "POST", @@ -299,8 +302,8 @@ describe("player_auth", function() { ); assert(typeof result.player_code === "string", "no code returned"); assert(result.player_code.length === 10, "incorrect code length returned"); - let expires = controllerPlugin.players.get("test").expires; - let msFromExpected = Math.abs(expires - Date.now() - 1000); + let expiresMs = controllerPlugin.players.get("test").expiresMs; + let msFromExpected = Math.abs(expiresMs - Date.now() - 1000); assert(msFromExpected < 100, `expiry time expected outside window (${msFromExpected}ms)`); }); }); @@ -315,8 +318,8 @@ describe("player_auth", function() { ); }); it("should throw if player code has expired", async function() { - let expires = Date.now() - 1000; - controllerPlugin.players.set("expired", { playerCode: "expried", verifyCode: null, expires }); + let expiresMs = Date.now() - 1000; + controllerPlugin.players.set("expired", { playerCode: "expired", verifyCode: null, expiresMs }); await assert.rejects( controllerPlugin.handleSetVerifyCodeRequest( new SetVerifyCodeRequest("expired", "expired") diff --git a/plugins/statistics_exporter/instance.ts b/plugins/statistics_exporter/instance.ts index edf53ae2..76502173 100644 --- a/plugins/statistics_exporter/instance.ts +++ b/plugins/statistics_exporter/instance.ts @@ -109,8 +109,8 @@ export class InstancePlugin extends BaseInstancePlugin { // take a long time for the command to go through if the command // stream is overloaded. Should the timeout be exceeded the // previous values for the metrics will end up being sent to controller. - let timeout = this.instance.config.get("statistics_exporter.command_timeout") * 1000; - await lib.timeout(this.gatherMetrics(), timeout, undefined); + let timeoutMs = this.instance.config.get("statistics_exporter.command_timeout") * 1000; + await lib.timeout(this.gatherMetrics(), timeoutMs, undefined); } } diff --git a/test/controller/InstanceInfo.js b/test/controller/InstanceInfo.js index 67939189..fda15835 100644 --- a/test/controller/InstanceInfo.js +++ b/test/controller/InstanceInfo.js @@ -22,7 +22,7 @@ describe("controller/InstanceInfo", function() { check(InstanceInfo.fromJSON({ config, status: "stopped" }, "controller")); check(InstanceInfo.fromJSON({ config, status: "running" }, "controller")); check(InstanceInfo.fromJSON({ config, status: "running", gamePort: 34197 }, "controller")); - check(InstanceInfo.fromJSON({ config, status: "running", updatedAt: Date.now() }, "controller")); + check(InstanceInfo.fromJSON({ config, status: "running", updatedAtMs: Date.now() }, "controller")); // All at once check(InstanceInfo.fromJSON({ diff --git a/test/host/Instance.js b/test/host/Instance.js index 723bd776..b899a335 100644 --- a/test/host/Instance.js +++ b/test/host/Instance.js @@ -64,7 +64,7 @@ describe("class Instance", function() { "player", new PlayerStats({ join_count: 1, - last_join_at: stats.lastJoinAt.getTime(), + last_join_at_ms: stats.lastJoinAt.getTime(), }) ) ), @@ -111,8 +111,8 @@ describe("class Instance", function() { new PlayerStats({ join_count: 1, online_time_ms: stats.onlineTimeMs, - last_join_at: stats.lastJoinAt.getTime(), - last_leave_at: stats.lastLeaveAt.getTime(), + last_join_at_ms: stats.lastJoinAt.getTime(), + last_leave_at_ms: stats.lastLeaveAt.getTime(), last_leave_reason: "quit", }), "quit", @@ -162,7 +162,7 @@ describe("class Instance", function() { "foo", new PlayerStats({ join_count: 1, - last_join_at: stats.lastJoinAt.getTime(), + last_join_at_ms: stats.lastJoinAt.getTime(), }) ) ) @@ -185,8 +185,8 @@ describe("class Instance", function() { new PlayerStats({ join_count: 1, online_time_ms: stats.onlineTimeMs, - last_join_at: stats.lastJoinAt.getTime(), - last_leave_at: stats.lastLeaveAt.getTime(), + last_join_at_ms: stats.lastJoinAt.getTime(), + last_leave_at_ms: stats.lastLeaveAt.getTime(), last_leave_reason: "quit", }), "quit", diff --git a/test/host/server.js b/test/host/server.js index 637838b8..12ade2e0 100644 --- a/test/host/server.js +++ b/test/host/server.js @@ -214,7 +214,7 @@ describe("host/server", function() { if (process.platform === "win32") { this.skip(); } - server.hangTimeout = 20; + server.hangTimeoutMs = 20; server._server = new events.EventEmitter(); server._server.kill = () => true; server._state = "running"; diff --git a/test/integration/clusterio.js b/test/integration/clusterio.js index 7025efd7..b99e912f 100644 --- a/test/integration/clusterio.js +++ b/test/integration/clusterio.js @@ -113,7 +113,7 @@ describe("Integration of Clusterio", function() { let controlB = new TestControl(connectorB); connectorB._sessionToken = connectorA._sessionToken; connectorB._sessionTimeout = connectorA._sessionTimeout; - connectorB._startedResuming = Date.now(); + connectorB._startedResumingMs = Date.now(); connectorB._state = "resuming"; connectorB._doConnect(); await events.once(connectorB, "resume"); @@ -898,7 +898,7 @@ describe("Integration of Clusterio", function() { reference.settings["runtime-global"].set("MyInt", { value: 1235 }); reference.settings["runtime-global"].set("MyDouble", { value: 12.25 }); reference.settings["runtime-per-user"].set("MyString", { value: "a-string" }); - reference.updatedAt = modPack.updatedAt; + reference.updatedAtMs = modPack.updatedAtMs; assert.deepEqual(modPack, reference); }); }); @@ -1003,7 +1003,7 @@ describe("Integration of Clusterio", function() { "dependencies:\n" + `size: ${stat.size}\n` + `sha1: ${hash}\n` + - `updatedAt: ${stat.mtimeMs}\n` + + `updatedAtMs: ${stat.mtimeMs}\n` + "isDeleted: false\n", ); }); diff --git a/test/lib/ExponentialBackoff.js b/test/lib/ExponentialBackoff.js index 790b1afa..5949deb2 100644 --- a/test/lib/ExponentialBackoff.js +++ b/test/lib/ExponentialBackoff.js @@ -36,7 +36,7 @@ describe("lib/ExponentialBackoff", function() { for (let max of [0.1, 0.5, 1, 2, 50, 100, 1000, 100000]) { let backoff = new ExponentialBackoff({ base: 1, max }); for (let j = 0; j < 100; j++) { - let value = backoff.delay() / 1000; // Divide by 100 for ms to s conversion + let value = backoff.delay() / 1000; // Divide by 1000 for ms to s conversion assert(value <= max, `delay returned a value greater than max: ${value} > ${max}`); } } @@ -46,10 +46,10 @@ describe("lib/ExponentialBackoff", function() { let backoff = new ExponentialBackoff({ reset: 100 }); backoff.delay(); assert.equal(backoff._exp, 1); - backoff._lastInvocationTime = Date.now() - 99 * 1000; + backoff._lastInvocationTimeMs = Date.now() - 99 * 1000; backoff.delay(); assert.equal(backoff._exp, 2); - backoff._lastInvocationTime = Date.now() - 101 * 1000; + backoff._lastInvocationTimeMs = Date.now() - 101 * 1000; backoff.delay(); assert.equal(backoff._exp, 1); }); diff --git a/test/lib/data/ModInfo.js b/test/lib/data/ModInfo.js index 8e0cb640..50e104db 100644 --- a/test/lib/data/ModInfo.js +++ b/test/lib/data/ModInfo.js @@ -92,7 +92,7 @@ describe("lib/data/ModInfo", function() { dependencies: [], filename: "empty_mod_1.0.0.zip", size: stat.size, - updated_at: stat.mtimeMs, + updated_at_ms: stat.mtimeMs, sha1: hash, }) ); diff --git a/test/lib/data/User.js b/test/lib/data/User.js index 3b13ece3..5ae52b7d 100644 --- a/test/lib/data/User.js +++ b/test/lib/data/User.js @@ -28,15 +28,15 @@ describe("lib/data/User", function() { [1, { join_count: 1, online_time_ms: 60e3, - last_join_at: new Date("2020-05T12:00Z").getTime(), - last_leave_at: new Date("2020-05T12:01Z").getTime(), + last_join_at_ms: new Date("2020-05T12:00Z").getTime(), + last_leave_at_ms: new Date("2020-05T12:01Z").getTime(), last_leave_reason: "quit", }], [2, { join_count: 1, online_time_ms: 0, - first_join_at: new Date("2020-05T12:00Z").getTime(), - last_join_at: new Date("2020-05T12:02Z").getTime(), + first_join_at_ms: new Date("2020-05T12:00Z").getTime(), + last_join_at_ms: new Date("2020-05T12:02Z").getTime(), }], ]}); @@ -52,9 +52,9 @@ describe("lib/data/User", function() { lib.PlayerStats.fromJSON({ join_count: 2, online_time_ms: 120e3, - first_join_at: new Date("2020-05T11:00Z").getTime(), - last_join_at: new Date("2020-05T13:00Z").getTime(), - last_leave_at: new Date("2020-05T13:02Z").getTime(), + first_join_at_ms: new Date("2020-05T11:00Z").getTime(), + last_join_at_ms: new Date("2020-05T13:00Z").getTime(), + last_leave_at_ms: new Date("2020-05T13:02Z").getTime(), last_leave_reason: "afk", }), ); diff --git a/test/lib/helpers.js b/test/lib/helpers.js index b9398dcb..b38e50f7 100644 --- a/test/lib/helpers.js +++ b/test/lib/helpers.js @@ -21,10 +21,10 @@ describe("lib/helpers", function() { it("should wait the approximate amount of time given", async function() { let startNs = hrtime(); await lib.wait(100); - let duration = Number(hrtime() - startNs) / 1e6; + let durationMs = Number(hrtime() - startNs) / 1e6; // On windows the time waited is normally up to 15 ms late with outliers in the 20ms range. - assert(duration > 90 && duration < 130, `duration (${duration}ms) not within (90-130)`); + assert(durationMs > 90 && durationMs < 130, `duration (${durationMs}ms) not within (90-130)`); }); }); diff --git a/test/lib/subscriptions.js b/test/lib/subscriptions.js index f345f906..4367f858 100644 --- a/test/lib/subscriptions.js +++ b/test/lib/subscriptions.js @@ -347,7 +347,7 @@ describe("lib/subscriptions", function() { it("should allow subscriptions to an event", async function() { let calledWith = null; let calledWithTwo = null; - const item = { id: 1, updatedAt: 1, isDeleted: false }; + const item = { id: 1, updatedAtMs: 1, isDeleted: false }; const event = new RegisteredEvent([item]); registeredEvent.subscribe(function(updates, synced) { calledWith = [updates, synced]; }); registeredEvent.subscribe(function(updates, synced) { calledWithTwo = [updates, synced]; }); @@ -382,9 +382,12 @@ describe("lib/subscriptions", function() { describe("getSnapshot()", function() { it("should give an immutable copy of the tracked values", async function() { - const update1 = [{ id: 1, updatedAt: 1, isDeleted: false }]; - const update2 = [{ id: 1, updatedAt: 2, isDeleted: false }, { id: 2, updatedAt: 3, isDeleted: false }]; - const update3 = [{ id: 1, updatedAt: 4, isDeleted: true }]; + const update1 = [{ id: 1, updatedAtMs: 1, isDeleted: false }]; + const update2 = [ + { id: 1, updatedAtMs: 2, isDeleted: false }, + { id: 2, updatedAtMs: 3, isDeleted: false }, + ]; + const update3 = [{ id: 1, updatedAtMs: 4, isDeleted: true }]; const snap0 = registeredEvent.getSnapshot(); assert.deepEqual(snap0, [new Map(), false]); await registeredEvent._handle(new RegisteredEvent(update1)); From 90969bd90ba3b594df20f097a976095365729610 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sat, 20 Jan 2024 20:33:45 +0100 Subject: [PATCH 13/84] Add changelog for 2.0.0-alpha.14 (#562) Add changelog and migration guide for alpha 14. Co-authored-by: Cooldude2606 <25043174+Cooldude2606@users.noreply.github.com> --- CHANGELOG.md | 86 ++++++++++++++++++++++++++- docs/migrating-from-2.0.0-alpha.13.md | 40 +++++++++++++ 2 files changed, 123 insertions(+), 3 deletions(-) create mode 100644 docs/migrating-from-2.0.0-alpha.13.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 7ee98779..f3c0cd60 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,87 @@ # Changelog +## Version 2.0.0-alpha.14 + +This release does not retain backwards compatibility with 2.0.0-alpha.13 meaning all parts of the cluster needs to updated at the same time. +See the [migration guide](/docs/migrating-from-2.0.0-alpha.13.md) for how to update. + +### Major Features + +- Added mod management. Mods can be uploaded to the controller and included in mod packs that can be assigned to instances and run. +- Added host port ranges which the instance ports will be automatically assigned a port from if factorio.game_port is not set. +- Added system info such as CPU model, hostname and kernel along with basic metrics for CPU usage, memory usage and disk usage for the controller and hosts. +- Added ability to remotely view and modify the configuration of hosts. + +### Features + +- Added logging of Clusterio version on startup +- Added `clusterio_controller_log_bytes` metric counting the size of the log files on the controller. +- Added warning logged when plugins on a connecting host does not match the ones installed on the controller. +- Added ability to revoke user tokens. +- Added restart required indicator to config fields in the Web UI. +- Added ability to revoke host tokens. +- Added support for plugins to run in the context of a host. +- Added more feedback when the web UI and or its plugins fails to load. +- Added autosaving of data held in memory on the controller such as configs, instances, users, mods to prevent large data loss on crashing. +- Added `system_info` metric with kernel, instruction set, CPU Model, hostname and Node.js version of the controller. +- Added `clusterio_host_system_info` metric with kernel, instruction set, CPU Model, hostname and Node.js version of hosts. +- Added `system_cpu_seconds_total`, `system_cpu_idle_seconds_total`, `system_memory_capasity_bytes`, `system_memory_available_bytes`, `system_disk_capasity_bytes`, `system_disk_available_bytes` metrics for CPU, memory and disk usage on the controller. +- Added `clusterio_host_system_cpu_seconds_total`, `clusterio_host_system_cpu_idle_seconds_total`, `clusterio_host_system_memory_capasity_bytes`, `clusterio_host_system_memory_available_bytes`, `clusterio_host_system_disk_capasity_bytes`, `clusterio_host_system_disk_available_bytes` metrics for CPU, memory and disk usage on hosts. +- Added support for reading the client's IP address from the `X-Forwarded-For` header when the source address is in the `controller.trusted_proxies` config option. +- Added field showing the IP address a host is connecting from. +- Added per player statistic tracking when they were first seen joining a particular server. +- Added ability to remotely restart and/or stop the controller and hosts. + +### Changes + +- Fixed filenames with extensions that are invalid on Windows such as "con.zip" not being treated as invalid. +- Fixed generate host token modal not using a random id when the optional id field is unset. +- Exported Factorio data is now tied to mod packs instead of being per cluster. +- Fixed export failing if a mod uses a different root folder mod_version in its zip file. +- Fixed sorting of user last seen column in the Web UI. +- The list users permission is now granted by default to players. +- Fixed incorrect handling of hosts and instance with an id of 0 causing them to not update properly. +- Fixed unable to unassign instances in the Web UI. +- Fixed host crashing if the Factorio server is not marked executable. +- Fixed errors logged from instance not being attributed to the instance in certain situations. +- Hosts no longer print messages from instances into their console. +- Clusterio now treat unhandled Node.js errors and promise rejections as fatal and exit immediately. +- Fixed a potential for data loss after a hard system crash due to fsync not being called on files written. +- Fixed Factorio server crashing in the research_sync plugin when a mod enables a previously disabled technology and a player attempts to research it. +- Fixed Factorio server crashing when the game starts with tick_paused = true +- Removed migration of alpha 10's single file log format. +- Log errors linking mods on Windows instead of preventing startup. +- Added locale support for modules patched into to the game save. +- Added support for defining a module_exports.lua file for modules patched into the game save. +- Fixed controller logs being split on server's local time day boundary instead of UTC day boundary. +- Fixed Unicode BOM in mod locale files breaking instance data export. +- Added a warning to the server-setting.json that is automatically written to instance folders stating changes will be overwritten. +- Changed save patching to use a `.tmp.zip` extension for the temporary file created instead of `zip.tmp`. +- Fixed Inventory Sync wiping the player's inventory when logging out while in the Space Exploration remote view mode. +- Fixed the public /api/plugins endponit leaking filesystem paths on the controller. +- Refactored list subscriptions in the Web UI to cache and only fetch updated entries when navigating to previously seen list. +- Added caching of save lists on the controller, enabling the saves of instances on offline hosts to still be seen. +- Hosts in the assign instance dropdown is now sorted by name. +- Added `non_blocking_saving` as one of the default settings added to `factorio.settings` on new instances. + +### Breaking Changes + +- Renamed usage of the terms master and slave to controller and host in all places they occur. +- Mods located in `sharedMods` are no longer copied to instances' mods folder. +- The instances' mods folder is now managed through mod packs, mods not part of the mod pack will be automatically deleted. +- Removed ability to import sub packages like `@clusterio/lib/link` from. +- Replaced `clusterio_controller_http_hits_total` metric with `clusterio_controller_http_endpoint_duration_seconds`. +- Replaced `clusterio_controller_last_query_log_duration_seconds` metric with `clusterio_controller_query_log_duration_seconds` summary. +- Reworked the network protocol to be simpler to work and reason with. +- Bumped minimum supported Node.js version to v18. +- Ported the codebase to TypeScript +- Changed the config format stored on disk to be much simpler. + +### Contributors + +Many thanks to the following for contributing code in this release: +[@Hornwitser](https://github.com/Hornwitser) [@Luciole0x](https://github.com/Luciole0x) [@Cooldude2606](https://github.com/Cooldude2606) [@Danielv123](https://github.com/Danielv123) [@Reavershark](https://github.com/Reavershark) [@Laar](https://github.com/Laar) [@rstrom1763](https://github.com/rstrom1763) + ## Version 2.0.0 ### Major Features @@ -77,8 +159,7 @@ - Removed UPSdisplay plugin. UPS statistics is exported by the statistics exporter plugin. - Controller now defaults to hosting on https on port 8443. -- Renamed master to controller. -- Renamed slave/client to host. +- Renamed client to slave. - Removed rotation of factorio-current.log files. ### Breaking Changes @@ -145,7 +226,6 @@ - Removed automatic creation of self-signed TLS certificate. - Node.js versions below 12 are no longer supported. - ## Version 1.2.4 - Removed broken remote combinator signaling. diff --git a/docs/migrating-from-2.0.0-alpha.13.md b/docs/migrating-from-2.0.0-alpha.13.md new file mode 100644 index 00000000..5472ab19 --- /dev/null +++ b/docs/migrating-from-2.0.0-alpha.13.md @@ -0,0 +1,40 @@ +# Migrating from 2.0.0-alpha.13 to 2.0.0-alpha.14 + +Alpha 14 comes with significant breaking changes for existing clusters, most notably the rename of master and slave to controller and host. +Please study the entire migration guide carefully before attempting to update an existing cluster. + +## Before updating + +If you're using mods you need to create mod pack string for the mod configurations you're currently running. +You can do this by running the follow RCON command on an instance that is currently running the mod configuration you want to use (you can also run this command in single player and find the string printed in Factorio's [Log file](https://wiki.factorio.com/Log_file)): + + /sc local m={name="pack",description="",factorio_version=game.active_mods.base,mods={},settings={startup={},["runtime-global"]={},["runtime-per-user"]={}}} for n,v in pairs(game.active_mods) do table.insert(m.mods,{name=n,enabled=true,version=v}) end local function set(s,c) for n,v in pairs(s) do m.settings[c][n]=v end end set(settings.startup,"startup") set(settings.global,"runtime-global") if game.player then set(game.player.mod_settings,"runtime-per-user") else set(settings.player,"runtime-per-user") end local e=game.encode_string(game.table_to_json(m)) log(e) rcon.print(e) + +This should produce a [Mod Pack String](https://forums.factorio.com/viewtopic.php?f=96&t=103578) which you need to copy and paste somewhere to keep it for later. +Mod Pack strings looks something like this: + + eNqFketuwyAMhd/Fv0OUVpW25lWmqSLESdAIRNh0l4h3n9mlitZt5QdCx/7so8MKXs8ILSzaPEEFPZKJdmEbvIgiDNpwiDaczhjpU93Vu/r+Tmpz6Anah/V7RqcJRUavO4c9tBwTVnAF5upCGJeIsYx3tvsPbQTdb0lKHYlnPJHY0+ONvcdjvW8gP1ZAyGz9KL5XINaR0yLPXEFMnu2ManSh0+6j/GOFitqXG4mjNSUidVm5wlm7JMYG7Qhl3BX8FjyqZ9vztOk+NM2fvRPaceLbzbN+UejQFFOWXzfArvk6v2HWD9ZLv5JfxA1UwsubPBaMKhHGklLO76vAwfQ= + +If you have multiple mod configuration create a mod pack string for each and note them down. + + +## Migrate installations + +**Note:** Check that plugins you are using have updates avaialble for alpha 14 before attempting to update, if they do not either wait for the plugin to be updated or uninstall the plugin from all places it is installed on first. + +Shutdown all slaves and the master server and then migrate each installations by running the following command: + + npm exec @clusterio/create --migrate-rename --log-level=verbose + +This will convert the existing installation's usage of the master and slave names in packages, configs, database and logs to the new controller and host terms and update the installed packages. + +If you're using systemd then copy the generated clusteriocontroller.service and/or clusteriohost.service files from the systemd folder to /etc/systemd/system/ and remove the old clusteriomaster.service and clusterioslave.service files. + +Once the installations have been migrated start up the controller and hosts again. + + +## Post installation + +Import the Mod Pack strings that was created beforehand on the controller and upload the mods needed by those mod packs to the controller. +You can assign a default mod pack in the controller config which instances use by default or set the mod pack on the per instance basis in the instance config. +Once you have assigned a mod pack to an instance run a data export on the instance in order to properly populate the mod settings in the mod and get item icons and item names resolved in the web interface. From 5ae19042f1526d72c225fb5f8e54f77418934946 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sat, 20 Jan 2024 15:52:06 +0100 Subject: [PATCH 14/84] Fix capacity typo in metrics --- CHANGELOG.md | 4 ++-- packages/lib/src/system_collectors.ts | 8 ++++---- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index f3c0cd60..56ee3a6f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,8 +25,8 @@ See the [migration guide](/docs/migrating-from-2.0.0-alpha.13.md) for how to upd - Added autosaving of data held in memory on the controller such as configs, instances, users, mods to prevent large data loss on crashing. - Added `system_info` metric with kernel, instruction set, CPU Model, hostname and Node.js version of the controller. - Added `clusterio_host_system_info` metric with kernel, instruction set, CPU Model, hostname and Node.js version of hosts. -- Added `system_cpu_seconds_total`, `system_cpu_idle_seconds_total`, `system_memory_capasity_bytes`, `system_memory_available_bytes`, `system_disk_capasity_bytes`, `system_disk_available_bytes` metrics for CPU, memory and disk usage on the controller. -- Added `clusterio_host_system_cpu_seconds_total`, `clusterio_host_system_cpu_idle_seconds_total`, `clusterio_host_system_memory_capasity_bytes`, `clusterio_host_system_memory_available_bytes`, `clusterio_host_system_disk_capasity_bytes`, `clusterio_host_system_disk_available_bytes` metrics for CPU, memory and disk usage on hosts. +- Added `system_cpu_seconds_total`, `system_cpu_idle_seconds_total`, `system_memory_capacity_bytes`, `system_memory_available_bytes`, `system_disk_capacity_bytes`, `system_disk_available_bytes` metrics for CPU, memory and disk usage on the controller. +- Added `clusterio_host_system_cpu_seconds_total`, `clusterio_host_system_cpu_idle_seconds_total`, `clusterio_host_system_memory_capacity_bytes`, `clusterio_host_system_memory_available_bytes`, `clusterio_host_system_disk_capacity_bytes`, `clusterio_host_system_disk_available_bytes` metrics for CPU, memory and disk usage on hosts. - Added support for reading the client's IP address from the `X-Forwarded-For` header when the source address is in the `controller.trusted_proxies` config option. - Added field showing the IP address a host is connecting from. - Added per player statistic tracking when they were first seen joining a particular server. diff --git a/packages/lib/src/system_collectors.ts b/packages/lib/src/system_collectors.ts index 6d8e6628..fa689c97 100644 --- a/packages/lib/src/system_collectors.ts +++ b/packages/lib/src/system_collectors.ts @@ -67,8 +67,8 @@ export const systemCpuIdleSecondsTotal = new Gauge( }, ); -export const systemMemoryCapasityBytes = new Gauge( - "system_memory_capasity_bytes", +export const systemMemoryCapacityBytes = new Gauge( + "system_memory_capacity_bytes", "Total system memory installed", { callback: function(collector) { @@ -87,8 +87,8 @@ export const systemMemoryAvailableBytes = new Gauge( }, ); -export const systemDiskCapasityBytes = new Gauge( - "system_disk_capasity_bytes", +export const systemDiskCapacityBytes = new Gauge( + "system_disk_capacity_bytes", "Size of the filesystem of the current working directory Node.js runs on", { callback: async function(collector) { From cafafbd540b6393a3d9653eba119aea5192ea8ab Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sat, 20 Jan 2024 21:46:01 +0100 Subject: [PATCH 15/84] Fix handleFetchPlayerCodeRequest returning incorrect properties --- plugins/player_auth/controller.ts | 2 +- plugins/player_auth/test/plugin.js | 10 +++++----- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/plugins/player_auth/controller.ts b/plugins/player_auth/controller.ts index 838c6eab..77832ee7 100644 --- a/plugins/player_auth/controller.ts +++ b/plugins/player_auth/controller.ts @@ -176,7 +176,7 @@ export class ControllerPlugin extends BaseControllerPlugin { let playerCode = await generateCode(this.controller.config.get("player_auth.code_length")); let expiresMs = Date.now() + this.controller.config.get("player_auth.code_timeout") * 1000; this.players.set(request.player, { playerCode, verifyCode: null, expiresMs }); - return { player_code: playerCode, controller_url: this.controller.getControllerUrl() }; + return { playerCode, controllerUrl: this.controller.getControllerUrl() }; } async handleSetVerifyCodeRequest(request: SetVerifyCodeRequest) { diff --git a/plugins/player_auth/test/plugin.js b/plugins/player_auth/test/plugin.js index 1d523fbc..8dc59142 100644 --- a/plugins/player_auth/test/plugin.js +++ b/plugins/player_auth/test/plugin.js @@ -300,8 +300,8 @@ describe("player_auth", function() { let result = await controllerPlugin.handleFetchPlayerCodeRequest( new FetchPlayerCodeRequest("test") ); - assert(typeof result.player_code === "string", "no code returned"); - assert(result.player_code.length === 10, "incorrect code length returned"); + assert(typeof result.playerCode === "string", "no code returned"); + assert(result.playerCode.length === 10, "incorrect code length returned"); let expiresMs = controllerPlugin.players.get("test").expiresMs; let msFromExpected = Math.abs(expiresMs - Date.now() - 1000); assert(msFromExpected < 100, `expiry time expected outside window (${msFromExpected}ms)`); @@ -332,14 +332,14 @@ describe("player_auth", function() { describe("integration", function() { it("should verify a full login flow", async function() { let app = controllerPlugin.controller.app; - let { player_code } = await controllerPlugin.handleFetchPlayerCodeRequest( + let { playerCode } = await controllerPlugin.handleFetchPlayerCodeRequest( new FetchPlayerCodeRequest("test") ); let playerCodeResult = await phin({ url: `${controllerUrl}/api/player_auth/player_code`, method: "POST", - data: { player_code }, + data: { player_code: playerCode }, parse: "json", }); @@ -351,7 +351,7 @@ describe("player_auth", function() { let verifyResult = await phin({ url: `${controllerUrl}/api/player_auth/verify`, method: "POST", - data: { player_code, verify_code, verify_token }, + data: { player_code: playerCode, verify_code, verify_token }, parse: "json", }); assert.equal(verifyResult.body.verified, true); From 1be1b029a08307b20f867ce091b52f364d634e49 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sat, 20 Jan 2024 20:48:52 +0100 Subject: [PATCH 16/84] 2.0.0-alpha.14 --- packages/controller/package.json | 6 +++--- packages/create/package.json | 2 +- packages/ctl/package.json | 4 ++-- packages/host/package.json | 4 ++-- packages/lib/package.json | 2 +- packages/web_ui/package.json | 4 ++-- plugins/global_chat/package.json | 6 +++--- plugins/inventory_sync/package.json | 6 +++--- plugins/player_auth/package.json | 6 +++--- plugins/research_sync/package.json | 6 +++--- plugins/statistics_exporter/package.json | 6 +++--- plugins/subspace_storage/package.json | 6 +++--- 12 files changed, 29 insertions(+), 29 deletions(-) diff --git a/packages/controller/package.json b/packages/controller/package.json index 042a53ad..6f967415 100644 --- a/packages/controller/package.json +++ b/packages/controller/package.json @@ -3,7 +3,7 @@ "description": "Implementation of Clusterio controller", "author": "Hornwitser", "license": "MIT", - "version": "2.0.0-alpha.13", + "version": "2.0.0-alpha.14", "main": "dist/index.js", "scripts": { "prepare": "tsc --build && webpack-cli --env production", @@ -19,7 +19,7 @@ "node": ">=18" }, "dependencies": { - "@clusterio/lib": "^2.0.0-alpha.13", + "@clusterio/lib": "^2.0.0-alpha.14", "@sinclair/typebox": "^0.30.4", "busboy": "^1.6.0", "compression": "^1.7.4", @@ -35,7 +35,7 @@ "yargs": "^17.7.2" }, "devDependencies": { - "@clusterio/web_ui": "^2.0.0-alpha.13", + "@clusterio/web_ui": "^2.0.0-alpha.14", "@types/busboy": "^1.5.0", "@types/compression": "^1.7.2", "@types/express": "^4.17.17", diff --git a/packages/create/package.json b/packages/create/package.json index 448b7ee5..6f320a27 100644 --- a/packages/create/package.json +++ b/packages/create/package.json @@ -3,7 +3,7 @@ "description": "Installer for clusterio", "author": "Hornwitser", "license": "MIT", - "version": "2.0.0-alpha.13b", + "version": "2.0.0-alpha.14", "main": "create.js", "bin": { "clusteriocreate": "./create.js" diff --git a/packages/ctl/package.json b/packages/ctl/package.json index abadf277..18c95a2e 100644 --- a/packages/ctl/package.json +++ b/packages/ctl/package.json @@ -3,7 +3,7 @@ "description": "Clusterio command line management tool", "author": "Hornwitser", "license": "MIT", - "version": "2.0.0-alpha.13", + "version": "2.0.0-alpha.14", "main": "dist/index.js", "scripts": { "prepare": "tsc --build", @@ -19,7 +19,7 @@ "node": ">=18" }, "dependencies": { - "@clusterio/lib": "^2.0.0-alpha.13", + "@clusterio/lib": "^2.0.0-alpha.14", "as-table": "^1.0.55", "fs-extra": "^11.1.1", "phin": "^3.7.0", diff --git a/packages/host/package.json b/packages/host/package.json index 2df4aab9..54e963e2 100644 --- a/packages/host/package.json +++ b/packages/host/package.json @@ -3,7 +3,7 @@ "description": "Implementation of Clusterio host server", "author": "Hornwitser", "license": "MIT", - "version": "2.0.0-alpha.13", + "version": "2.0.0-alpha.14", "main": "dist/index.js", "scripts": { "prepare": "tsc --build", @@ -19,7 +19,7 @@ "node": ">=18" }, "dependencies": { - "@clusterio/lib": "^2.0.0-alpha.13", + "@clusterio/lib": "^2.0.0-alpha.14", "@sinclair/typebox": "^0.30.4", "fs-extra": "^11.1.1", "jimp": "^0.22.8", diff --git a/packages/lib/package.json b/packages/lib/package.json index dc981417..6cd1aebc 100644 --- a/packages/lib/package.json +++ b/packages/lib/package.json @@ -1,7 +1,7 @@ { "name": "@clusterio/lib", "description": "Shared library for Clusterio", - "version": "2.0.0-alpha.13", + "version": "2.0.0-alpha.14", "repository": "https://github.com/clusterio/clusterio", "license": "MIT", "scripts": { diff --git a/packages/web_ui/package.json b/packages/web_ui/package.json index 07cbe1f5..6608a14c 100644 --- a/packages/web_ui/package.json +++ b/packages/web_ui/package.json @@ -3,7 +3,7 @@ "description": "Clusterio web interface implementation", "author": "Hornwitser", "license": "MIT", - "version": "2.0.0-alpha.13", + "version": "2.0.0-alpha.14", "browser": "src/index.ts", "main": "dist/src/index.js", "scripts": { @@ -19,7 +19,7 @@ "@ant-design/icons": "^5.1.4", "@babel/core": "^7.22.9", "@babel/preset-react": "^7.22.5", - "@clusterio/lib": "^2.0.0-alpha.13", + "@clusterio/lib": "^2.0.0-alpha.14", "antd": "^5.13.0", "assert": "^2.0.0", "babel-loader": "^9.1.3", diff --git a/plugins/global_chat/package.json b/plugins/global_chat/package.json index 648e8ab5..b438bec3 100644 --- a/plugins/global_chat/package.json +++ b/plugins/global_chat/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-global_chat", - "version": "2.0.0-alpha.13", + "version": "2.0.0-alpha.14", "description": "Clusterio plugin forwarding between Factorio servers", "author": "Hornwitser ", "homepage": "https://github.com/clusterio/clusterio#readme", @@ -21,8 +21,8 @@ "@clusterio/lib": "^2.0.0-alpha.0" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.13", - "@clusterio/web_ui": "^2.0.0-alpha.13", + "@clusterio/lib": "^2.0.0-alpha.14", + "@clusterio/web_ui": "^2.0.0-alpha.14", "@types/node": "^20.4.5", "@types/react": "^18.2.21", "typescript": "^5.1.6", diff --git a/plugins/inventory_sync/package.json b/plugins/inventory_sync/package.json index fe6b7738..5664be56 100644 --- a/plugins/inventory_sync/package.json +++ b/plugins/inventory_sync/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-inventory_sync", - "version": "2.0.0-alpha.13", + "version": "2.0.0-alpha.14", "description": "Clusterio plugin forwarding between Factorio servers", "author": "Danielv ", "homepage": "https://github.com/clusterio/factorioClusterio#readme", @@ -25,8 +25,8 @@ "fs-extra": "^8.1.0" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.13", - "@clusterio/web_ui": "^2.0.0-alpha.13", + "@clusterio/lib": "^2.0.0-alpha.14", + "@clusterio/web_ui": "^2.0.0-alpha.14", "@types/fs-extra": "^11.0.1", "@types/node": "^20.4.5", "@types/react": "^18.2.21", diff --git a/plugins/player_auth/package.json b/plugins/player_auth/package.json index d3ff154a..a7f78e0a 100644 --- a/plugins/player_auth/package.json +++ b/plugins/player_auth/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-player_auth", - "version": "2.0.0-alpha.13", + "version": "2.0.0-alpha.14", "description": "Clusterio plugin authenticating logged in players to the web interface", "author": "Hornwitser ", "homepage": "https://github.com/clusterio/clusterio#readme", @@ -26,8 +26,8 @@ "jsonwebtoken": "^9.0.1" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.13", - "@clusterio/web_ui": "^2.0.0-alpha.13", + "@clusterio/lib": "^2.0.0-alpha.14", + "@clusterio/web_ui": "^2.0.0-alpha.14", "@types/express": "^4.17.17", "@types/jsonwebtoken": "^9.0.2", "@types/node": "^20.4.5", diff --git a/plugins/research_sync/package.json b/plugins/research_sync/package.json index 9fb317e1..19ab6b70 100644 --- a/plugins/research_sync/package.json +++ b/plugins/research_sync/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-research_sync", - "version": "2.0.0-alpha.13", + "version": "2.0.0-alpha.14", "description": "Clusterio plugin for synchronizing research between Factorio servers", "author": "Hornwitser ", "homepage": "https://github.com/clusterio/clusterio#readme", @@ -25,8 +25,8 @@ "fs-extra": "^8.1.0" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.13", - "@clusterio/web_ui": "^2.0.0-alpha.13", + "@clusterio/lib": "^2.0.0-alpha.14", + "@clusterio/web_ui": "^2.0.0-alpha.14", "@types/fs-extra": "^11.0.1", "@types/node": "^20.4.5", "@types/react": "^18.2.21", diff --git a/plugins/statistics_exporter/package.json b/plugins/statistics_exporter/package.json index 23c93577..a98facb8 100644 --- a/plugins/statistics_exporter/package.json +++ b/plugins/statistics_exporter/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-statistics_exporter", - "version": "2.0.0-alpha.13", + "version": "2.0.0-alpha.14", "description": "Clusterio plugin for gathering statistics from Factorio servers", "author": "Hornwitser ", "homepage": "https://github.com/clusterio/clusterio#readme", @@ -21,8 +21,8 @@ "@clusterio/lib": "^2.0.0-alpha.0" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.13", - "@clusterio/web_ui": "^2.0.0-alpha.13", + "@clusterio/lib": "^2.0.0-alpha.14", + "@clusterio/web_ui": "^2.0.0-alpha.14", "@types/node": "^20.4.5", "@types/react": "^18.2.21", "typescript": "^5.1.6", diff --git a/plugins/subspace_storage/package.json b/plugins/subspace_storage/package.json index eecb9a89..7cd98cf9 100644 --- a/plugins/subspace_storage/package.json +++ b/plugins/subspace_storage/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-subspace_storage", - "version": "2.0.0-alpha.13", + "version": "2.0.0-alpha.14", "description": "Clusterio plugin for sharing storage between Factorio servers", "author": "Hornwitser ", "homepage": "https://github.com/clusterio/clusterio#readme", @@ -25,8 +25,8 @@ "fs-extra": "^8.1.0" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.13", - "@clusterio/web_ui": "^2.0.0-alpha.13", + "@clusterio/lib": "^2.0.0-alpha.14", + "@clusterio/web_ui": "^2.0.0-alpha.14", "@types/express": "^4.17.17", "@types/fs-extra": "^11.0.1", "@types/node": "^20.4.5", From 0aa330561b3f28fdd72790261c1d7000a6fbef85 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sat, 20 Jan 2024 22:44:28 +0100 Subject: [PATCH 17/84] Update migration guide Add a note about double checking which version of Node.js is installed to avoid accidentally updating on an outdated version, and fix migrate arguments. --- docs/migrating-from-2.0.0-alpha.13.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/migrating-from-2.0.0-alpha.13.md b/docs/migrating-from-2.0.0-alpha.13.md index 5472ab19..adddebdd 100644 --- a/docs/migrating-from-2.0.0-alpha.13.md +++ b/docs/migrating-from-2.0.0-alpha.13.md @@ -17,6 +17,8 @@ Mod Pack strings looks something like this: If you have multiple mod configuration create a mod pack string for each and note them down. +Double check that you are on a supported version of Node.js by running `node --version`, the version should be v18 or later, if not you will need to update Node.js before continuing. + ## Migrate installations @@ -24,7 +26,7 @@ If you have multiple mod configuration create a mod pack string for each and not Shutdown all slaves and the master server and then migrate each installations by running the following command: - npm exec @clusterio/create --migrate-rename --log-level=verbose + npm exec @clusterio/create -y -- --migrate-rename --log-level=verbose This will convert the existing installation's usage of the master and slave names in packages, configs, database and logs to the new controller and host terms and update the installed packages. From 5fdc5852129e6390693189e75106288f2944b75d Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sat, 20 Jan 2024 22:48:20 +0100 Subject: [PATCH 18/84] Add known issues section Some problems were not caught during testing and is not straight forward to fix. Add them to the known issues section for now. --- README.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/README.md b/README.md index ee76ddfa..2ecce36d 100644 --- a/README.md +++ b/README.md @@ -39,6 +39,7 @@ If you are starting a new cluster, it's highly recommended to use the 2.0 alpha. * [Running Clusterio](#running-clusterio) * [Setting up remote hosts](#setting-up-remote-hosts) * [Setting up clusterioctl](#setting-up-clusterioctl) +* [Known issues](#known-issues) * [Common problems](#Common-problems) @@ -265,6 +266,13 @@ If you want to use a different user for the command line interface, you can gene npx clusteriocontroller bootstrap generate-user-token +## Known Issues + +- The `instance.auto_start` option is currently is broken and will not work, turn it off and do not use it for now. +- When the `server_select` plugin is installed you may see `[error] Unhandled event server_select:UpdateInstancesEvent` logged when starting an instance. + This should be ignored as it does not indicate an error occured. + + ## Common problems ### EACCESS [...] LISTEN 443 From 901ce4e08ec000a7d0d764fd0648445dc4b23d28 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sun, 21 Jan 2024 01:01:08 +0100 Subject: [PATCH 19/84] Add broken instance data export to known issues --- README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/README.md b/README.md index 2ecce36d..07bf2765 100644 --- a/README.md +++ b/README.md @@ -271,6 +271,9 @@ If you want to use a different user for the command line interface, you can gene - The `instance.auto_start` option is currently is broken and will not work, turn it off and do not use it for now. - When the `server_select` plugin is installed you may see `[error] Unhandled event server_select:UpdateInstancesEvent` logged when starting an instance. This should be ignored as it does not indicate an error occured. +- After migrating from Alpha 13 to Alpha 14 running an instance data export fails with the error `Upload failed: 401 Unauthorized` due to the host token being issued to a slave. + To workaround this find the id of the host in the host config and generate a new host token with the same id. + Stop the host and replace the value of the `host.controller_token` field in `config-host.json` with the new host token before starting it up again. ## Common problems From d6d6c052aa08730005e70094c6c70a89324f33ba Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sun, 21 Jan 2024 02:17:12 +0100 Subject: [PATCH 20/84] Add known issue about being kicked out of the Web UI --- README.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/README.md b/README.md index 07bf2765..10be24e6 100644 --- a/README.md +++ b/README.md @@ -274,6 +274,11 @@ If you want to use a different user for the command line interface, you can gene - After migrating from Alpha 13 to Alpha 14 running an instance data export fails with the error `Upload failed: 401 Unauthorized` due to the host token being issued to a slave. To workaround this find the id of the host in the host config and generate a new host token with the same id. Stop the host and replace the value of the `host.controller_token` field in `config-host.json` with the new host token before starting it up again. +- After migrating from Alpha 13 to Alpha 14 you get kicked out from the web interface due to the master to controller rename. + If you don't have the authentication token readily available you can manually migrate the token by opening the Inspect/Developer Tools on the page and running the following piece of JavaScript in the Console: + ``` + localStorage.controller_token = localStorage.master_token + ``` ## Common problems From 72264f0c5309f4729f9a95b401318b130938b671 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sun, 21 Jan 2024 23:48:26 +0100 Subject: [PATCH 21/84] Add suport for color mod setting (#568) Implement the color-setting type in ModPack and display a color picker for it in the Web UI. This type of setting was added in Factorio 1.1.77. --- CHANGELOG.md | 7 ++++ packages/ctl/src/commands.ts | 19 ++++++++- packages/lib/src/data/ModPack.ts | 14 +++++-- packages/lib/src/data/index.ts | 2 +- .../web_ui/src/components/ModPackViewPage.tsx | 33 +++++++++++++-- test/integration/clusterio.js | 12 ++++-- test/lib/data/ModPack.js | 40 ++++++++++++++++++- 7 files changed, 114 insertions(+), 13 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 56ee3a6f..5013726a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,12 @@ # Changelog +## Version 2.0.0-alpha.15 + +### Features + +- Added support for `color-settings` Mod settings (#568)[https://github.com/clusterio/clusterio/issues/568]. + + ## Version 2.0.0-alpha.14 This release does not retain backwards compatibility with 2.0.0-alpha.13 meaning all parts of the cluster needs to updated at the same time. diff --git a/packages/ctl/src/commands.ts b/packages/ctl/src/commands.ts index 8e892a50..76150cc3 100644 --- a/packages/ctl/src/commands.ts +++ b/packages/ctl/src/commands.ts @@ -1097,7 +1097,13 @@ modPackCommands.add(new lib.Command({ function setModPackSettings( modPack: lib.ModPack, - args: { boolSetting?: string[], intSetting?: string[], doubleSetting?: string[], stringSetting?: string[] }, + args: { + boolSetting?: string[], + intSetting?: string[], + doubleSetting?: string[], + stringSetting?: string[], + colorSetting?: string[], + }, ) { function doSettings(settings: string[], cast: (value: string) => (string | number | boolean)) { for (let i = 0; i + 2 < settings.length; i += 3) { @@ -1134,6 +1140,13 @@ function setModPackSettings( return number; }); doSettings(args.stringSetting || [], value => value); + doSettings(args.colorSetting || [], value => { + let color = JSON.parse(value); + if (typeof color !== "object" || color === null) { + throw new lib.CommandError("color value must be a JSON object with r, g, b, a properties"); + } + return color; + }); } function setModPackMods(modPack: lib.ModPack, mods: string[] | undefined) { @@ -1173,6 +1186,7 @@ modPackCommands.add(new lib.Command({ "int-setting": { describe: "Set int setting", array: true, nargs: 3, type: "string" }, "double-setting": { describe: "Set double setting", array: true, nargs: 3, type: "string" }, "string-setting": { describe: "Set string setting", array: true, nargs: 3, type: "string" }, + "color-setting": { describe: "Set color setting", array: true, nargs: 3, type: "string" }, }); }], handler: async function( @@ -1186,6 +1200,7 @@ modPackCommands.add(new lib.Command({ intSetting?: string[], doubleSetting?: string[], stringSetting?: string[], + colorSetting?: string[], }, control: Control, ) { @@ -1244,6 +1259,7 @@ modPackCommands.add(new lib.Command({ "int-setting": { describe: "Set int setting", array: true, nargs: 3, type: "string" }, "double-setting": { describe: "Set double setting", array: true, nargs: 3, type: "string" }, "string-setting": { describe: "Set string setting", array: true, nargs: 3, type: "string" }, + "color-setting": { describe: "Set color setting", array: true, nargs: 3, type: "string" }, "remove-setting": { describe: "Remove a setting", array: true, nargs: 2, type: "string" }, }); }], @@ -1261,6 +1277,7 @@ modPackCommands.add(new lib.Command({ intSetting?: string[], doubleSetting?: string[], stringSetting?: string[], + colorSetting?: string[], removeSetting?: string[], }, control: Control, diff --git a/packages/lib/src/data/ModPack.ts b/packages/lib/src/data/ModPack.ts index 2aec7b63..094addb4 100644 --- a/packages/lib/src/data/ModPack.ts +++ b/packages/lib/src/data/ModPack.ts @@ -8,16 +8,24 @@ import { integerFactorioVersion } from "./version"; import type { Logger } from "../logging"; +export const ModSettingColor = Type.Object({ + r: Type.Number(), + g: Type.Number(), + b: Type.Number(), + a: Type.Number(), +}); +export type ModSettingColor = Static; + /** * A setting for a mod. */ export interface ModSetting { /** Value of the given mod setting. */ - value: boolean | number | string; + value: boolean | number | string | ModSettingColor; } const ModSettingJsonSchema = Type.Object({ - "value": Type.Union([Type.Boolean(), Type.Number(), Type.String()]), + "value": Type.Union([Type.Boolean(), Type.Number(), Type.String(), ModSettingColor]), }); const ModSettingsJsonSchema = Type.Record(Type.String(), ModSettingJsonSchema); @@ -335,7 +343,7 @@ export default class ModPack { * @param logger - Logger used to report warnings on. */ fillDefaultSettings(settingPrototypes: Record, logger: Logger) { - const knownTypes = ["bool-setting", "int-setting", "double-setting", "string-setting"]; + const knownTypes = ["bool-setting", "int-setting", "double-setting", "string-setting", "color-setting"]; let prototypes = Object.entries(settingPrototypes) .filter(([type, _]) => knownTypes.includes(type)) .flatMap(([_, settings]) => Object.values(settings)) diff --git a/packages/lib/src/data/index.ts b/packages/lib/src/data/index.ts index 86668cdc..0ca5f47a 100644 --- a/packages/lib/src/data/index.ts +++ b/packages/lib/src/data/index.ts @@ -5,7 +5,7 @@ */ export { default as ExportManifest } from "./ExportManifest"; export { default as ModInfo } from "./ModInfo"; -export { default as ModPack, ModSetting, ModRecord } from "./ModPack"; +export { default as ModPack, ModSetting, ModSettingColor, ModRecord } from "./ModPack"; export { default as ModuleInfo } from "./ModuleInfo"; export { default as Permission } from "./Permission"; export { default as PlayerStats } from "./PlayerStats"; diff --git a/packages/web_ui/src/components/ModPackViewPage.tsx b/packages/web_ui/src/components/ModPackViewPage.tsx index ae2d2821..47c86fa5 100644 --- a/packages/web_ui/src/components/ModPackViewPage.tsx +++ b/packages/web_ui/src/components/ModPackViewPage.tsx @@ -1,7 +1,7 @@ import React, { Fragment, memo, useCallback, useEffect, useContext, useRef, useState } from "react"; import { useNavigate, useParams } from "react-router-dom"; import { - Button, Card, Checkbox, Col, ConfigProvider, Descriptions, Form, Input, Pagination, + Button, Card, ColorPicker, Checkbox, Col, ConfigProvider, Descriptions, Form, Input, Pagination, Popconfirm, Row, Table, Tag, Typography, Select, Skeleton, Space, Spin, Switch, Modal, Tooltip, TablePaginationConfig, } from "antd"; @@ -404,7 +404,7 @@ function hasChange(changes: ModChange[], field:Partial) { } // picks out the value property from settings. -function pickValue(map: Map): [string, string|number|boolean][] { +function pickValue(map: Map): [string, lib.ModSetting["value"]][] { return [...map].map(([name, value]) => [name, value.value]); } @@ -420,6 +420,25 @@ function groupToMap(array: any[], fn: (...args:any) => any) { return map; } +function InputColor(props: { + value: lib.ModSettingColor, + onChange: (value: lib.ModSettingColor) => void, +}) { + const value = props.value; + return { + const rgb = color.toRgb(); + props.onChange({ + r: rgb.r / 255, + g: rgb.g / 255, + b: rgb.b / 255, + a: rgb.a, + }); + }} + />; +} type SettingsTableFieldProps = { name: string; type: string; @@ -431,11 +450,15 @@ type SettingsTableFieldProps = { const SettingsTableField = memo((props: SettingsTableFieldProps) => { const name = props.name; const isBoolean = props.type === "bool-setting"; + const isColor = props.type === "color-setting"; const isNumber = ["int-setting", "double-setting"].includes(props.type); const isSelect = Boolean(props.allowedValues); let input; if (isBoolean) { input = ; + } else if (isColor) { + const PartialInputColor = InputColor as () => React.JSX.Element; + input = ; } else if (isSelect) { const options = Object.values(props.allowedValues).map(value => ({ value, @@ -484,7 +507,7 @@ function SettingsTable(props: SettingsTableProps) { .map(mod => [mod.name, mod.title]) ); - const types = ["bool-setting", "int-setting", "double-setting", "string-setting"]; + const types = ["bool-setting", "int-setting", "double-setting", "string-setting", "color-setting"]; let prototypes = Object.entries(props.prototypes) .filter(([type, _]) => types.includes(type)) .flatMap(([_, settingPrototypes]: [string, any]) => Object.values(settingPrototypes)) @@ -492,7 +515,7 @@ function SettingsTable(props: SettingsTableProps) { function controls( scope: "startup"|"runtime-global"|"runtime-per-user", - storedFields: [string, string|number|boolean][] + storedFields: [string, lib.ModSetting["value"]][] ) { let fields = new Map(prototypes .filter(p => p.setting_type === scope) @@ -508,6 +531,8 @@ function SettingsTable(props: SettingsTableProps) { type = "double-setting"; } else if (typeof value === "string") { type = "string-setting"; + } else if (typeof value === "object" && "r" in value) { + type = "color-setting"; } else { throw Error(`Unhandled setting type ${typeof value}`); } diff --git a/test/integration/clusterio.js b/test/integration/clusterio.js index b99e912f..dcb3c36d 100644 --- a/test/integration/clusterio.js +++ b/test/integration/clusterio.js @@ -81,6 +81,10 @@ async function getUsers() { return new Map(users.map(user => [user.name, user])); } +function jsonArg(value) { + return `"${JSON.stringify(value).replace(/"/g, process.platform === "win32" ? '""' : '\\"')}"`; +} + describe("Integration of Clusterio", function() { describe("clusteriocontroller", function() { describe("bootstrap generate-user-token", function() { @@ -502,8 +506,7 @@ describe("Integration of Clusterio", function() { ]; for (let [prop, value] of testConfigs) { - value = `"${JSON.stringify(value).replace(/"/g, process.platform === "win32" ? '""' : '\\"')}"`; - let args = `test factorio.settings ${prop} ${value}`; + let args = `test factorio.settings ${prop} ${jsonArg(value)}`; await execCtl(`instance config set-prop ${args}`); } @@ -876,6 +879,7 @@ describe("Integration of Clusterio", function() { assert(modPacks.some(modPack => modPack.name === "empty-pack"), "created pack is not in the list"); }); it("should allow setting all fields", async function() { + const color = { r: 1, g: 0, b: 1, a: 0 }; await execCtl( "mod-pack create full-pack 0.17.59 " + "--description Description " + @@ -883,7 +887,8 @@ describe("Integration of Clusterio", function() { "--bool-setting startup MyBool true " + "--int-setting runtime-global MyInt 1235 " + "--double-setting runtime-global MyDouble 12.25 " + - "--string-setting runtime-per-user MyString a-string" + "--string-setting runtime-per-user MyString a-string " + + `--color-setting runtime-per-user MyColor ${jsonArg(color)}` ); let modPacks = await getControl().send(new lib.ModPackListRequest()); let modPack = modPacks.find(entry => entry.name === "full-pack"); @@ -898,6 +903,7 @@ describe("Integration of Clusterio", function() { reference.settings["runtime-global"].set("MyInt", { value: 1235 }); reference.settings["runtime-global"].set("MyDouble", { value: 12.25 }); reference.settings["runtime-per-user"].set("MyString", { value: "a-string" }); + reference.settings["runtime-per-user"].set("MyColor", { value: color }); reference.updatedAtMs = modPack.updatedAtMs; assert.deepEqual(modPack, reference); }); diff --git a/test/lib/data/ModPack.js b/test/lib/data/ModPack.js index 10134b68..14616da3 100644 --- a/test/lib/data/ModPack.js +++ b/test/lib/data/ModPack.js @@ -39,6 +39,7 @@ describe("lib/data/ModPack", function() { }, "runtime-per-user": { "string-setting": { "value": "a string" }, + "color-setting": { "value": { "r": 1, "g": 1, "b": 0, "a": 1 } }, }, }})); check(ModPack.fromJSON({ export_manifest: { assets: { setting: "settings.json" }}})); @@ -60,6 +61,7 @@ describe("lib/data/ModPack", function() { }, "runtime-per-user": { "string-setting": { "value": "a string" }, + "color-setting": { "value": { "r": 1, "g": 1, "b": 0, "a": 1 } }, }, }, export_manifest: { assets: { setting: "settings.json" }}, @@ -101,6 +103,14 @@ describe("lib/data/ModPack", function() { default_value: "a string", }, }, + "color-setting": { + "color": { + name: "color", + type: "color-setting", + setting_type: "runtime-per-user", + default_value: { "r": 1, "g": 1, "b": 1, "a": 1 }, + }, + }, }; const mockLogger = { warn: () => {} }; it("should fill in defaults for settings", function() { @@ -115,6 +125,7 @@ describe("lib/data/ModPack", function() { }, "runtime-per-user": { "string": { "value": "a string" }, + "color": { "value": { "r": 1, "g": 1, "b": 1, "a": 1 } }, }, }); }); @@ -128,6 +139,7 @@ describe("lib/data/ModPack", function() { }, "runtime-per-user": { "string": { "value": "spam" }, + "color": { "value": { "r": 0, "g": 0.5, "b": 1, "a": 1 } }, }, }}); pack.fillDefaultSettings(prototypes, mockLogger); @@ -140,6 +152,7 @@ describe("lib/data/ModPack", function() { }, "runtime-per-user": { "string": { "value": "spam" }, + "color": { "value": { "r": 0, "g": 0.5, "b": 1, "a": 1 } }, }, }); }); @@ -204,6 +217,7 @@ describe("lib/data/ModPack", function() { }, "runtime-per-user": { "string-setting": { "value": "a string" }, + "color-setting": { "value": { "r": 1, "g": 0.5, "b": 0, "a": 1 } }, }, }}); @@ -248,7 +262,7 @@ describe("lib/data/ModPack", function() { ...istr("runtime-per-user"), // name Uint8Array.from([5, 0]), // dictionary - new Uint8Array(Uint32Array.from([1]).buffer), // items + new Uint8Array(Uint32Array.from([2]).buffer), // items ...istr("string-setting"), // name Uint8Array.from([5, 0]), // dictinary @@ -257,6 +271,30 @@ describe("lib/data/ModPack", function() { ...istr("value"), // name Uint8Array.from([3, 0, 0, "a string".length]), // string Buffer.from("a string"), + + ...istr("color-setting"), // name + Uint8Array.from([5, 0]), // dictinary + new Uint8Array(Uint32Array.from([1]).buffer), // items + + ...istr("value"), // name + Uint8Array.from([5, 0]), // dictinary + new Uint8Array(Uint32Array.from([4]).buffer), // items + + ...istr("r"), // name + Uint8Array.from([2, 0]), // number + new Uint8Array(Float64Array.from([1]).buffer), + + ...istr("g"), // name + Uint8Array.from([2, 0]), // number + new Uint8Array(Float64Array.from([0.5]).buffer), + + ...istr("b"), // name + Uint8Array.from([2, 0]), // number + new Uint8Array(Float64Array.from([0]).buffer), + + ...istr("a"), // name + Uint8Array.from([2, 0]), // number + new Uint8Array(Float64Array.from([1]).buffer), ]) ); /* eslint-enable indent */ From 3d7c11a8b6274e0ad930da1a9778b849e6d033ba Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sun, 21 Jan 2024 21:48:58 +0100 Subject: [PATCH 22/84] Add missing enable source maps for host service --- CHANGELOG.md | 4 ++++ packages/create/create.js | 1 + 2 files changed, 5 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5013726a..c6ca039f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,10 @@ - Added support for `color-settings` Mod settings (#568)[https://github.com/clusterio/clusterio/issues/568]. +### Changes + +- Fixed generated systemd file for clusteriohost not enabling source maps. + ## Version 2.0.0-alpha.14 diff --git a/packages/create/create.js b/packages/create/create.js index 7fcc62bc..fca70256 100755 --- a/packages/create/create.js +++ b/packages/create/create.js @@ -488,6 +488,7 @@ Group=${await groupIdToName(os.userInfo().gid)} WorkingDirectory=${process.cwd()} KillMode=mixed KillSignal=SIGINT +Environment=NODE_OPTIONS=--enable-source-maps ExecStart=${process.cwd()}/node_modules/.bin/clusteriohost run --log-level=warn --can-restart Restart=on-failure RestartPreventExitStatus=8 From 1d1e80fbf063dcfd5be7c34d467b4302feec1802 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sun, 21 Jan 2024 21:50:38 +0100 Subject: [PATCH 23/84] Export Control and related types from ctl --- CHANGELOG.md | 1 + packages/ctl/index.ts | 1 + 2 files changed, 2 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index c6ca039f..0df07f19 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ ### Changes - Fixed generated systemd file for clusteriohost not enabling source maps. +- Fixed ctl not exporting types used for adding commands to ctl plugins. ## Version 2.0.0-alpha.14 diff --git a/packages/ctl/index.ts b/packages/ctl/index.ts index 6a70dd21..54bd5f27 100644 --- a/packages/ctl/index.ts +++ b/packages/ctl/index.ts @@ -1,4 +1,5 @@ import { bootstrap } from "./ctl"; +export * from "./ctl"; export { default as BaseCtlPlugin } from "./src/BaseCtlPlugin"; if (module === require.main) { From 2872faab98de32f54a96b82a0ece70e42673bb05 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sun, 21 Jan 2024 22:47:56 +0100 Subject: [PATCH 24/84] Skip loading plugins without entrypoint on hosts If a plugin only has entrypoints at the controller level the config will skip adding a "load_plugin" config option for the given plugin to avoid making options that do nothing. If such a plugin is added to a host it will still reach loadPlugin which in turn tries to check the load_plugin config and throw an error on startup due the field being missing. Skip trying to load plugins without an entrypoint at the instance or host level in hosts before checking the load_plugin config field. --- CHANGELOG.md | 1 + packages/host/src/Host.ts | 5 ++++- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0df07f19..9b4c4bf3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,7 @@ - Fixed generated systemd file for clusteriohost not enabling source maps. - Fixed ctl not exporting types used for adding commands to ctl plugins. +- Fixed controller only plugins throwing an error about missing a config field on startup out when added to hosts. ## Version 2.0.0-alpha.14 diff --git a/packages/host/src/Host.ts b/packages/host/src/Host.ts index 5ecfb5bd..7381ad36 100644 --- a/packages/host/src/Host.ts +++ b/packages/host/src/Host.ts @@ -377,7 +377,10 @@ export default class Host extends lib.Link { async loadPlugins() { for (let pluginInfo of this.pluginInfos) { - if (!this.config.get(`${pluginInfo.name}.load_plugin` as keyof lib.HostConfigFields)) { + if ( + !pluginInfo.hostEntrypoint && !pluginInfo.instanceEntrypoint + || !this.config.get(`${pluginInfo.name}.load_plugin` as keyof lib.HostConfigFields) + ) { continue; } From e0161114e736b878a253caa38b4b19980ed145e9 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sun, 21 Jan 2024 23:11:58 +0100 Subject: [PATCH 25/84] Fix --no-download-headless conflict in installer Fix installer incorrectly exiting with a conflicting options errors when the --no-download-headless option was used in conjunction with the --factorio-dir option. --- CHANGELOG.md | 1 + packages/create/create.js | 7 +++++-- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9b4c4bf3..ca5d4e2f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -11,6 +11,7 @@ - Fixed generated systemd file for clusteriohost not enabling source maps. - Fixed ctl not exporting types used for adding commands to ctl plugins. - Fixed controller only plugins throwing an error about missing a config field on startup out when added to hosts. +- Fixed installer incorrectly claiming the options --factorio-dir and --no-download-headless conflicts with each other. ## Version 2.0.0-alpha.14 diff --git a/packages/create/create.js b/packages/create/create.js index fca70256..c222ab04 100755 --- a/packages/create/create.js +++ b/packages/create/create.js @@ -612,7 +612,7 @@ async function inquirerMissingArgs(args) { }, ], answers); - if (answers.downloadHeadless) { + if (!answers.factorioDir && answers.downloadHeadless) { answers.factorioDir = "factorio"; } } @@ -760,7 +760,7 @@ async function main() { "Can be set to false using --no-download-headless.", type: "boolean", }) - .conflicts("factorio-dir", "download-headless"); + ; } args = args.argv; @@ -780,6 +780,9 @@ async function main() { let answers = await inquirerMissingArgs(args); if (answers.downloadHeadless) { + if (answers.factorioDir !== "factorio") { + throw new InstallError("--download-headless option requires --factorio-dir to be set to factorio"); + } await downloadLinuxServer(); } From d7edae99c5932b8f14c5fc210810efdc42138168 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 22 Jan 2024 19:29:36 +0100 Subject: [PATCH 26/84] Add metrics for pending request Track the number of requests currently awaiting a response from a request handler both from interally issued requests and requests forwarded from other parties. --- CHANGELOG.md | 3 ++- packages/controller/controller.ts | 18 ++++++++++++++++++ packages/host/host.ts | 31 ++++++++++++++++++++++++++----- packages/lib/src/link/link.ts | 7 +++++++ 4 files changed, 53 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ca5d4e2f..4dd4be7c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,7 +4,8 @@ ### Features -- Added support for `color-settings` Mod settings (#568)[https://github.com/clusterio/clusterio/issues/568]. +- Added support for `color-settings` Mod settings [#568](https://github.com/clusterio/clusterio/issues/568). +- Added `clusterio_controller_pending_requests` and `clusterio_host_pending_requests` metrics. ### Changes diff --git a/packages/controller/controller.ts b/packages/controller/controller.ts index 14b18e46..b4161f65 100644 --- a/packages/controller/controller.ts +++ b/packages/controller/controller.ts @@ -37,6 +37,24 @@ import { version } from "./package.json"; // globals let controller: Controller; +void new lib.Gauge( + "clusterio_controller_pending_requests", + "Count of pending link requests currently waiting in memory on the controller.", + { + callback: (gauge) => { + let count = 0; + if (controller) { + for (const host of controller.wsServer.hostConnections.values()) { + count += host.pendingRequestCount; + } + for (const control of controller.wsServer.controlConnections.values()) { + count += control.pendingRequestCount; + } + } + gauge.set(count); + }, + } +); void new lib.Gauge( "clusterio_controller_host_mapping", diff --git a/packages/host/host.ts b/packages/host/host.ts index 46827d85..fa195675 100755 --- a/packages/host/host.ts +++ b/packages/host/host.ts @@ -26,6 +26,27 @@ import { ConsoleTransport, levels, logger } from "@clusterio/lib"; import Host from "./src/Host"; +let host: Host | undefined; + +void new lib.Gauge( + "clusterio_host_pending_requests", + "Count of pending link requests currently waiting in memory on the host.", + { + labels: ["host_id"], + callback: (gauge) => { + if (!host) { + return; + } + + let count = host.pendingRequestCount; + for (const connection of host.instanceConnections.values()) { + count += connection.pendingRequestCount; + count += connection.instance.pendingRequestCount; + } + gauge.labels(String(host.config.get("host.id"))).set(count); + }, + } +); export class HostConnector extends lib.WebSocketClientConnector { constructor( @@ -205,7 +226,7 @@ async function startHost() { } let hostConnector = new HostConnector(hostConfig, tlsCa, pluginInfos); - let host = new Host( + host = new Host( hostConnector, args.config, hostConfig, @@ -227,7 +248,7 @@ async function startHost() { secondSigint = true; logger.info("Caught interrupt signal, shutting down"); - host.shutdown(); + host!.shutdown(); }); let secondSigterm = false; process.on("SIGTERM", () => { @@ -240,11 +261,11 @@ async function startHost() { secondSigterm = true; logger.info("Caught termination signal, shutting down"); - host.shutdown(); + host!.shutdown(); }); process.on("SIGHUP", () => { logger.info("Terminal closed, shutting down"); - host.shutdown(); + host!.shutdown(); }); try { @@ -255,7 +276,7 @@ async function startHost() { } hostConnector.once("connect", () => { - logger.add(new lib.LinkTransport({ link: host })); + logger.add(new lib.LinkTransport({ link: host! })); }); await hostConnector.connect(); diff --git a/packages/lib/src/link/link.ts b/packages/lib/src/link/link.ts index deabd028..c7365190 100644 --- a/packages/lib/src/link/link.ts +++ b/packages/lib/src/link/link.ts @@ -141,6 +141,13 @@ export class Link { }); } + /** + * Count of requests currently waiting for a response on this link + */ + get pendingRequestCount() { + return this._pendingRequests.size + this._forwardedRequests.size; + } + _clearPendingRequests(err: Error & { code?: string }) { for (let pending of this._pendingRequests.values()) { pending.reject(err); From 123148e2777ad90dca275676e58438f84497d62e Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 22 Jan 2024 19:54:41 +0100 Subject: [PATCH 27/84] Fix pending request not deleted on response Fix memory leak due to the recorded _pendingRequests entry for a request not being deleted when the response for the request has been received. --- CHANGELOG.md | 1 + packages/lib/src/link/link.ts | 2 ++ test/lib/link/link.js | 4 ++++ 3 files changed, 7 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4dd4be7c..ca5d9a34 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,6 +13,7 @@ - Fixed ctl not exporting types used for adding commands to ctl plugins. - Fixed controller only plugins throwing an error about missing a config field on startup out when added to hosts. - Fixed installer incorrectly claiming the options --factorio-dir and --no-download-headless conflicts with each other. +- Fixed a memory leak on every request sent over the WebSocket. ## Version 2.0.0-alpha.14 diff --git a/packages/lib/src/link/link.ts b/packages/lib/src/link/link.ts index c7365190..4793008b 100644 --- a/packages/lib/src/link/link.ts +++ b/packages/lib/src/link/link.ts @@ -428,6 +428,7 @@ export class Link { throw new libErrors.InvalidMessage(`Received reply from ${message.src} for message sent to ${pending.dst}`); } + this._pendingRequests.delete(message.dst.requestId!); try { pending.resolve(pending.request.responseFromJSON(message.data)); } catch (err: any) { @@ -444,6 +445,7 @@ export class Link { ); } + this._pendingRequests.delete(message.dst.requestId!); pending.reject(new libErrors.RequestError(message.data.message, message.data.code, message.data.stack)); } diff --git a/test/lib/link/link.js b/test/lib/link/link.js index 0c991031..c8bd06c4 100644 --- a/test/lib/link/link.js +++ b/test/lib/link/link.js @@ -16,6 +16,7 @@ describe("lib/link/link", function() { describe("class Link", function() { let testConnector; + /** @type {lib.Link} */ let testLink; let src = addr({ controlId: 1 }); let dst = addr("controller"); @@ -281,6 +282,7 @@ describe("lib/link/link", function() { let srcReq = new lib.Address(lib.Address.control, 1, 1); testConnector.emit("message", new lib.MessageResponse(1, dst, srcReq)); await request; + assert.deepEqual(testLink._pendingRequests, new Map()); }); it("should send request and reject with error when error response is received", async function() { let message = events.once(testConnector, "send"); @@ -297,6 +299,7 @@ describe("lib/link/link", function() { request, { message: "Error" } ); + assert.deepEqual(testLink._pendingRequests, new Map()); }); it("should send request and resolve with data when response is received", async function() { let message = events.once(testConnector, "send"); @@ -305,6 +308,7 @@ describe("lib/link/link", function() { let srcReq = new lib.Address(lib.Address.control, 1, 1); testConnector.emit("message", new lib.MessageResponse(1, dst, srcReq, 44)); assert.deepEqual(await request, new NumberRequest.Response(44)); + assert.deepEqual(testLink._pendingRequests, new Map()); }); it("should send event to the other side of the link", async function() { let message = events.once(testConnector, "send"); From 96bd5fb1c2879f50bfa0371b3c196f68a1266b27 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 22 Jan 2024 20:33:46 +0100 Subject: [PATCH 28/84] Handle virtual links in Link._clearPendingRequests When a request is forwarded from a virtual link to a WebSocket link as is the case when an instance sends a request to the controller the origin for the request is a virtual link. Handle this case in _clearPendingRequests instead of throwing an assertion error. --- CHANGELOG.md | 1 + packages/lib/src/link/link.ts | 16 +++++++++++----- 2 files changed, 12 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ca5d9a34..bfa1c742 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,7 @@ - Fixed controller only plugins throwing an error about missing a config field on startup out when added to hosts. - Fixed installer incorrectly claiming the options --factorio-dir and --no-download-headless conflicts with each other. - Fixed a memory leak on every request sent over the WebSocket. +- Fixed assertion error on closing or invalidating a link with messages forwarded from a virtual link. ## Version 2.0.0-alpha.14 diff --git a/packages/lib/src/link/link.ts b/packages/lib/src/link/link.ts index 4793008b..14b7d1ec 100644 --- a/packages/lib/src/link/link.ts +++ b/packages/lib/src/link/link.ts @@ -153,12 +153,18 @@ export class Link { pending.reject(err); } for (let pending of this._forwardedRequests.values()) { - assert(pending.origin.connector instanceof WebSocketBaseConnector); - if (pending.origin.connector.hasSession) { - pending.origin.connector.sendResponseError( - new libData.ResponseError(err.message, err.code), pending.src, pending.dst - ); + // Drop response if the origin is a WebSocket and the session + // has expired. The other end of the link will have sent an + // error response to the request in this case. + if ( + pending.origin.connector instanceof WebSocketBaseConnector + && !pending.origin.connector.hasSession + ) { + continue; } + pending.origin.connector.sendResponseError( + new libData.ResponseError(err.message, err.code), pending.src, pending.dst + ); } this._pendingRequests.clear(); this._forwardedRequests.clear(); From 256b455315bba73bc3f0f2b0f8234813dd15505c Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Tue, 23 Jan 2024 16:07:55 +0100 Subject: [PATCH 29/84] Fix host hang on stopping during create save If during the startup of an instance the Factorio server is started up to create a new save and the host shutdown in initiated then the instance will be commanded to stop while the FactorioServer state is "create". This was was not handled by Instance.stop leaving the child_process running after the host is done shutting down. Kill the FactorioServer on stop if the current state of it is stopping or create to make sure the server actually stops. --- CHANGELOG.md | 1 + packages/host/src/Instance.ts | 5 +++++ packages/host/src/server.ts | 6 ++++-- 3 files changed, 10 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index bfa1c742..1bab77a0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,7 @@ - Fixed installer incorrectly claiming the options --factorio-dir and --no-download-headless conflicts with each other. - Fixed a memory leak on every request sent over the WebSocket. - Fixed assertion error on closing or invalidating a link with messages forwarded from a virtual link. +- Fixed host hanging if stopped while an instance is in the process of creating a save during startup. ## Version 2.0.0-alpha.14 diff --git a/packages/host/src/Instance.ts b/packages/host/src/Instance.ts index c4e3d473..98d4d0eb 100644 --- a/packages/host/src/Instance.ts +++ b/packages/host/src/Instance.ts @@ -955,6 +955,11 @@ rcon.print(game.table_to_json(players))`.replace(/\r?\n/g, " "); await lib.invokeHook(this.plugins, "onStop"); await this.server.stop(); await this.sendSaveListUpdate(); + } else if ( + this.server._state === "stopping" + || this.server._state === "create" + ) { + await this.server.kill(); } } diff --git a/packages/host/src/server.ts b/packages/host/src/server.ts index b34af3b9..60089036 100644 --- a/packages/host/src/server.ts +++ b/packages/host/src/server.ts @@ -865,7 +865,9 @@ export class FactorioServer extends events.EventEmitter { try { let [code, signal] = await events.once(this._server, "exit"); - void signal; + if (signal) { + throw new Error(`Factorio exited with signal ${signal}`); + } if (code !== 0) { throw new Error(`Factorio exited with status ${code}`); } @@ -1092,7 +1094,7 @@ export class FactorioServer extends events.EventEmitter { * process. */ async kill(unexpected = false) { - this._check(["running", "stopping"]); + this._check(["running", "stopping", "create"]); this._killed = true; if (!unexpected) { this._state = "stopping"; From fc5701f24313000c37e71043decd893562e1fb18 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Tue, 23 Jan 2024 16:23:09 +0100 Subject: [PATCH 30/84] Fix instance auto start Bypass the Link layer and invoke the handleInstanceStartRequest method directly on the Instance object during the host's instance auto start phase. --- .eslintrc.js | 1 + CHANGELOG.md | 1 + README.md | 1 - packages/host/src/Host.ts | 2 +- test/integration/clusterio.js | 39 ++++++++++++++++++++++++++++++++++- 5 files changed, 41 insertions(+), 3 deletions(-) diff --git a/.eslintrc.js b/.eslintrc.js index 3cf36d34..1487649a 100644 --- a/.eslintrc.js +++ b/.eslintrc.js @@ -35,6 +35,7 @@ module.exports = { "rules": { "prefer-arrow-callback": "off", "no-invalid-this": "off", + "valid-jsdoc": "off", "node/no-unpublished-import": "off", "node/no-unpublished-require": "off", }, diff --git a/CHANGELOG.md b/CHANGELOG.md index 1bab77a0..dc84398d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,6 +16,7 @@ - Fixed a memory leak on every request sent over the WebSocket. - Fixed assertion error on closing or invalidating a link with messages forwarded from a virtual link. - Fixed host hanging if stopped while an instance is in the process of creating a save during startup. +- Fixed `instance.auto_start` option. ## Version 2.0.0-alpha.14 diff --git a/README.md b/README.md index 10be24e6..0857fc07 100644 --- a/README.md +++ b/README.md @@ -268,7 +268,6 @@ If you want to use a different user for the command line interface, you can gene ## Known Issues -- The `instance.auto_start` option is currently is broken and will not work, turn it off and do not use it for now. - When the `server_select` plugin is installed you may see `[error] Unhandled event server_select:UpdateInstancesEvent` logged when starting an instance. This should be ignored as it does not indicate an error occured. - After migrating from Alpha 13 to Alpha 14 running an instance data export fails with the error `Upload failed: 401 Unauthorized` due to the host token being issued to a slave. diff --git a/packages/host/src/Host.ts b/packages/host/src/Host.ts index 7381ad36..06a23809 100644 --- a/packages/host/src/Host.ts +++ b/packages/host/src/Host.ts @@ -1056,7 +1056,7 @@ export default class Host extends lib.Link { if (instanceInfo.config.get("instance.auto_start")) { try { let instanceConnection = await this._connectInstance(instanceId); - await instanceConnection.send(new lib.InstanceStartRequest()); + await instanceConnection.instance.handleInstanceStartRequest(new lib.InstanceStartRequest()); } catch (err: any) { logger.error( `Error during auto startup for ${instanceInfo.config.get("instance.name")}:\n${err.stack}`, diff --git a/test/integration/clusterio.js b/test/integration/clusterio.js index dcb3c36d..d34b8a0d 100644 --- a/test/integration/clusterio.js +++ b/test/integration/clusterio.js @@ -17,6 +17,7 @@ const { } = require("./index"); +/** @returns {Promise>} */ async function getInstances() { let instances = await getControl().send(new lib.InstanceDetailsListRequest()); return new Map(instances.map(instance => [instance.id, instance])); @@ -27,6 +28,10 @@ async function checkInstanceStatus(id, status) { assert.equal(instances.get(44).status, status, "incorrect instance status"); } +async function spawnAltHost(config = "alt-host-config.json") { + return await spawn("alt-host:", `node ../../packages/host run --config ${config}`, /Started host/); +} + async function startAltHost() { let config = "alt-host-config.json"; let configPath = path.join("temp", "test", config); @@ -38,7 +43,7 @@ async function startAltHost() { await exec(`node ../../packages/host --config ${config} config set host.instances_directory alt-instances`); await exec(`node ../../packages/host --config ${config} config set host.factorio_directory ../../factorio`); await exec(`node ../../packages/host --config ${config} config set host.mods_directory alt-mods`); - return await spawn("alt-host:", `node ../../packages/host run --config ${config}`, /Started host/); + return await spawnAltHost(config); } async function stopAltHost(hostProcess) { @@ -200,6 +205,38 @@ describe("Integration of Clusterio", function() { assert(await fs.pathExists(clusterioLib), "mod was not downloaded by the test"); }); }); + it("should auto start instances with auto_start enabled", async function() { + slowTest(this); + + let hostProcess; + try { + hostProcess = await startAltHost(); + await execCtl("instance create alt-start --id 97"); + await execCtl("instance assign alt-start 5"); + await execCtl("instance config set alt-start instance.auto_start true"); + await stopAltHost(hostProcess); + hostProcess = await spawnAltHost(); + // Stop the host immediatly to test the handling of stopping + // instances while automatically started and a save is being created + await stopAltHost(hostProcess); + hostProcess = await spawnAltHost(); + let status; + // Wait for instance status to become running. + for (let i = 0; i < 100; i++) { + const instances = await getInstances(); + status = instances.get(97)?.status ?? "not_present"; + if (status === "running") { + break; + } + await wait(100); // Try again 10 times per second. + } + assert.equal(status, "running"); + await execCtl("instance stop alt-start"); + await execCtl("instance delete alt-start"); + } finally { + await stopAltHost(hostProcess); + } + }); }); describe("clusterioctl", function() { From 847945cfdd5df5354e3677a13f96283ce311b043 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Wed, 24 Jan 2024 01:45:45 +0100 Subject: [PATCH 31/84] Fix plugin broadcasts not being filtered When a plugin broadcasts an event to instances there may be hosts and instances in the cluster which does not have the plugin loaded. This would cause an error to be logged for every instance which does not have the plugin loaded. Filter out broadcast events sent to hosts and instances such that instances and hosts that does not have the event do not received them. --- CHANGELOG.md | 1 + packages/controller/index.ts | 2 + packages/controller/src/BaseConnection.ts | 3 +- packages/controller/src/Controller.ts | 6 + packages/controller/src/ControllerRouter.ts | 16 +- packages/host/index.ts | 3 + packages/host/src/Host.ts | 35 +- packages/lib/src/link/link.ts | 9 +- test/integration/routing.js | 416 ++++++++++++++++++++ 9 files changed, 470 insertions(+), 21 deletions(-) create mode 100644 test/integration/routing.js diff --git a/CHANGELOG.md b/CHANGELOG.md index dc84398d..0264def0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,7 @@ - Fixed assertion error on closing or invalidating a link with messages forwarded from a virtual link. - Fixed host hanging if stopped while an instance is in the process of creating a save during startup. - Fixed `instance.auto_start` option. +- Fixed host log spammed with errors when plugins broadcast messages to instances with the plugin disabled. ## Version 2.0.0-alpha.14 diff --git a/packages/controller/index.ts b/packages/controller/index.ts index 5218034f..caebb912 100644 --- a/packages/controller/index.ts +++ b/packages/controller/index.ts @@ -1,7 +1,9 @@ import { bootstrap } from "./controller"; export { default as Controller } from "./src/Controller"; export { default as ControllerUser } from "./src/ControllerUser"; +export { default as ControllerRouter } from "./src/ControllerRouter"; export { default as ControlConnection } from "./src/ControlConnection"; +export { default as HostConnection } from "./src/HostConnection"; export { default as InstanceInfo } from "./src/InstanceInfo"; export { default as BaseControllerPlugin } from "./src/BaseControllerPlugin"; export { default as UserManager } from "./src/UserManager"; diff --git a/packages/controller/src/BaseConnection.ts b/packages/controller/src/BaseConnection.ts index 493ebbde..3ebac51e 100644 --- a/packages/controller/src/BaseConnection.ts +++ b/packages/controller/src/BaseConnection.ts @@ -6,7 +6,6 @@ import type WsServerConnector from "./WsServerConnector"; import * as lib from "@clusterio/lib"; const { logger } = lib; -import ControllerRouter from "./ControllerRouter"; import * as routes from "./routes"; @@ -24,7 +23,7 @@ export default class BaseConnection extends lib.Link { public _controller: Controller ) { super(connector); - this.router = new ControllerRouter(this._controller); + this.router = _controller.router; for (let [Request, handler] of this._controller._registeredRequests) { this.handle(Request, handler); } for (let [Request, handler] of this._controller._fallbackedRequests) { this.fallbackRequest(Request, handler); } for (let [Event, handler] of this._controller._registeredEvents) { this.handle(Event, handler); } diff --git a/packages/controller/src/Controller.ts b/packages/controller/src/Controller.ts index 8930f854..3e81c0c1 100644 --- a/packages/controller/src/Controller.ts +++ b/packages/controller/src/Controller.ts @@ -26,6 +26,7 @@ import WsServer from "./WsServer"; import HostConnection from "./HostConnection"; import HostInfo from "./HostInfo"; import BaseControllerPlugin from "./BaseControllerPlugin"; +import ControllerRouter from "./ControllerRouter"; const endpointDurationSummary = new Summary( "clusterio_controller_http_endpoint_duration_seconds", @@ -74,6 +75,7 @@ export default class Controller { /** WebSocket server */ wsServer: WsServer; + router = new ControllerRouter(this); trustedProxies: BlockList; debugEvents: events.EventEmitter = new events.EventEmitter(); private _events: events.EventEmitter = new events.EventEmitter(); @@ -1374,7 +1376,11 @@ export default class Controller { } } else if (dst.id === lib.Address.instance || dst.id === lib.Address.host) { + const plugin = event.constructor.plugin; for (let hostConnection of this.wsServer.hostConnections.values()) { + if (plugin && !hostConnection.plugins.has(plugin)) { + continue; + } hostConnection.sendEvent(event, dst); } diff --git a/packages/controller/src/ControllerRouter.ts b/packages/controller/src/ControllerRouter.ts index 84420595..4679312d 100644 --- a/packages/controller/src/ControllerRouter.ts +++ b/packages/controller/src/ControllerRouter.ts @@ -7,8 +7,6 @@ import ControlConnection from "./ControlConnection"; const { logger } = lib; export default class ControllerRouter { - pendingRequests: Map = new Map(); - constructor( public controller: Controller ) {} @@ -16,12 +14,14 @@ export default class ControllerRouter { /** * @param origin - Source link of the message. * @param message - Link the message originated from. + * @param entry - message entry if message is a request or an event. * @param hasFallback - true if fallback handling is available. * @returns true if the message was handled */ forwardMessage( origin: ControlConnection | HostConnection, message: lib.MessageRoutable, + entry: lib.RequestEntry | lib.EventEntry | undefined, hasFallback: boolean ): boolean { if (!["request", "response", "responseError", "event"].includes(message.type)) { @@ -32,7 +32,7 @@ export default class ControllerRouter { let nextHop: ControlConnection | HostConnection | undefined; let msg: string = ""; if (dst.type === lib.Address.broadcast) { - this.broadcastMessage(origin, message); + this.broadcastMessage(origin, message, entry as lib.EventEntry); return true; } else if (dst.type === lib.Address.host) { nextHop = this.controller.wsServer.hostConnections.get(dst.id); @@ -96,13 +96,17 @@ export default class ControllerRouter { broadcastMessage( origin: lib.Link, message: lib.MessageRoutable, + entry: lib.EventEntry, ) { let dst = message.dst; if (message.type !== "event") { this.warnUnrouted(message, `Unexpected broadcast of ${message.type}`); - } else if (dst.id === lib.Address.host || dst.id === lib.Address.instance) { + return; + } + const plugin = entry.Event.plugin; + if (dst.id === lib.Address.host || dst.id === lib.Address.instance) { for (let hostConnection of this.controller.wsServer.hostConnections.values()) { - if (hostConnection !== origin) { + if (hostConnection !== origin && (!plugin || hostConnection.plugins.has(plugin))) { hostConnection.connector.send(message); } } @@ -126,5 +130,3 @@ export default class ControllerRouter { logger.warn(msg ? `${baseMsg}: ${msg}.` : `${baseMsg}.`); } } - -module.exports = ControllerRouter; diff --git a/packages/host/index.ts b/packages/host/index.ts index 2b01bc21..f39ce536 100644 --- a/packages/host/index.ts +++ b/packages/host/index.ts @@ -1,6 +1,9 @@ import { bootstrap } from "./host"; export { default as BaseHostPlugin } from "./src/BaseHostPlugin"; export { default as BaseInstancePlugin } from "./src/BaseInstancePlugin"; +export { default as Host, HostRouter } from "./src/Host"; +export { default as Instance } from "./src/Instance"; +export { default as InstanceConnection } from "./src/InstanceConnection"; if (module === require.main) { bootstrap(); diff --git a/packages/host/src/Host.ts b/packages/host/src/Host.ts index 06a23809..b1b9dd41 100644 --- a/packages/host/src/Host.ts +++ b/packages/host/src/Host.ts @@ -82,7 +82,7 @@ const instanceStartingMessages = new Set([ lib.InstanceExportDataRequest.name, ]); -class HostRouter { +export class HostRouter { constructor( public host: Host ) { } @@ -93,16 +93,22 @@ class HostRouter { * @param origin - * Link the message originated from. * @param message - Message to process. + * @param entry - message entry if message is a request or an event. * @param hasFallback - is fallback available * @returns true if the message was handled, false if fallback * is requested */ - forwardMessage(origin: lib.Link, message: lib.MessageRoutable, hasFallback: boolean) { + forwardMessage( + origin: lib.Link, + message: lib.MessageRoutable, + entry: lib.RequestEntry | lib.EventEntry | undefined, + hasFallback: boolean, + ) { let dst = message.dst; let nextHop: lib.Link | undefined; let msg: string | undefined; if (dst.type === lib.Address.broadcast) { - this.broadcastMessage(origin, message); + this.broadcastMessage(origin, message, entry as lib.EventEntry); return true; } else if ( dst.type === lib.Address.controller @@ -153,27 +159,36 @@ class HostRouter { return true; } - broadcastMessage(origin: lib.Link, message: lib.MessageRoutable) { + broadcastMessage( + origin: lib.Link, + message: lib.MessageRoutable, + entry: lib.EventEntry, + ) { let dst = message.dst; if (message.type !== "event") { this.warnUnrouted(message, `Unexpected broadcast of ${message.type}`); - } else if (dst.id === lib.Address.host || dst.id === lib.Address.instance) { + return; + } + const plugin = entry.Event.plugin; + if (dst.id === lib.Address.instance) { for (let instanceConnection of this.host.instanceConnections.values()) { - if (instanceConnection !== origin) { + if (instanceConnection !== origin && (!plugin || instanceConnection.plugins.has(plugin))) { instanceConnection.connector.send(message); } } - if (this.host !== origin) { + if (this.host !== origin && (!plugin || this.host.serverPlugins.has(plugin))) { this.host.connector.send(message); } } else if (dst.id === lib.Address.control) { if (this.host !== origin) { - this.host.connector.send(message); + if (!plugin || this.host.serverPlugins.has(plugin)) { + this.host.connector.send(message); + } } else { - logger.warn(`Received control broadcast of ${(message as lib.MessageEvent).name} from master.`); + logger.warn(`Received control broadcast of ${(message as lib.MessageEvent).name} from controller.`); } } else { - this.warnUnrouted(message, `Unexpected broacdast target ${dst.id}`); + this.warnUnrouted(message, `Unexpected broadcast target ${dst.id}`); } } diff --git a/packages/lib/src/link/link.ts b/packages/lib/src/link/link.ts index 14b7d1ec..520e5fe4 100644 --- a/packages/lib/src/link/link.ts +++ b/packages/lib/src/link/link.ts @@ -72,7 +72,12 @@ export type RequestHandler = (request: Req, src: libData.Address, dst: export type EventHandler = (event: T, src: libData.Address, dst: libData.Address) => Promise; interface Router { - forwardMessage(origin: Link, message: MessageRoutable, fallback: boolean): boolean, + forwardMessage( + origin: Link, + message: MessageRoutable, + entry: RequestEntry | EventEntry | undefined, + fallback: boolean, + ): boolean, } // Some definitions for the terminology used here: @@ -343,7 +348,7 @@ export class Link { let fallback = message.type === "request" ? this._requestFallbacks.get((entry as RequestEntry).Request) : undefined ; - if (this.router.forwardMessage(this, message, Boolean(fallback))) { + if (this.router.forwardMessage(this, message, entry, Boolean(fallback))) { return; } if (!fallback) { diff --git a/test/integration/routing.js b/test/integration/routing.js new file mode 100644 index 00000000..ed31de35 --- /dev/null +++ b/test/integration/routing.js @@ -0,0 +1,416 @@ +"use strict"; +const assert = require("assert").strict; + +const lib = require("@clusterio/lib"); +const { Host, Instance, InstanceConnection } = require("@clusterio/host"); +const { ControlConnection, Controller, HostConnection, InstanceInfo } = require("@clusterio/controller"); + +const addr = lib.Address.fromShorthand; + +class Control extends lib.Link { } + +/** + * @param {Controller} controller + * @param {number} hostId + */ +function connectHost(controller, hostId, plugins) { + const [controllerSide, hostSide] = lib.VirtualConnector.makePair(addr("controller"), addr({ hostId })); + const registerData = new lib.RegisterHostData("", "0.0.0", hostId, plugins); + const hostConnection = new HostConnection(registerData, controllerSide, controller, "host-a.test"); + controller.wsServer.hostConnections.set(hostId, hostConnection); + const hostConfig = new lib.HostConfig("host"); + hostConfig.set("host.id", hostId); + return new Host(hostSide, "invalid", hostConfig, undefined, []); +} + +/** + * @param {Controller} controller + * @param {Host} host + * @param {number} instanceId + */ +function connectInstance(controller, host, instanceId) { + const hostAddress = addr({ hostId: host.config.get("host.id")}); + const instanceAddress = addr({ instanceId }); + const [instanceSide, hostSide] = lib.VirtualConnector.makePair(instanceAddress, hostAddress); + const instanceConfig = new lib.InstanceConfig("host"); + instanceConfig.set("instance.id", instanceId); + instanceConfig.set("instance.assigned_host", host.config.get("host.id")); + host.instanceInfos.set(instanceId, { path: "invalid", config: instanceConfig }); + controller.instances.set(instanceId, new InstanceInfo(instanceConfig, "stopped")); + const instance = new Instance(host, instanceSide, "invalid", "invalid", instanceConfig); + const instanceConnection = new InstanceConnection(hostSide, host, instance); + host.instanceConnections.set(instanceId, instanceConnection); + return instance; +} + +/** + * @param {Controller} controller + * @param {number} hostId + */ +function connectControl(controller, controlId) { + const [controllerSide, controlSide] = lib.VirtualConnector.makePair(addr("controller"), addr({ controlId })); + const registerData = new lib.RegisterControlData("", "0.0.0"); + let user = controller.userManager.users.get("test"); + if (!user) { + user = controller.userManager.createUser("test"); + } + const controlConnection = new ControlConnection(registerData, controllerSide, controller, user, controlId); + controller.wsServer.controlConnections.set(controlId, controlConnection); + return new Control(controlSide); +} + +class TestRequest { + static type = "request"; + static src = ["controller", "host", "instance", "control"]; + static dst = ["controller", "host", "instance", "control"]; + static permission = null; +} + +class TestEvent { + static type = "event"; + static src = ["controller", "host", "instance", "control"]; + static dst = ["controller", "host", "instance", "control"]; + static permission = null; +} + +class PluginEvent { + static type = "event"; + static src = ["controller", "host", "instance", "control"]; + static dst = ["controller", "host", "instance", "control"]; + static permission = null; + static plugin = "test"; +} + +lib.Link.register(TestRequest); +lib.Link.register(TestEvent); +lib.Link.register(PluginEvent); + +describe("Integration of link routing", function() { + /** @type {Controller} */ + let controller; + /** @type {Host} */ + let hostA; + /** @type {Host} */ + let hostB; + /** @type {Instance} */ + let instanceA1; + /** @type {Instance} */ + let instanceA2; + /** @type {Instance} */ + let instanceB1; + /** @type {Control} */ + let controlA; + /** @type {Control} */ + let controlB; + + beforeEach(function() { + controller = new Controller( + {}, + [], + "invalid", + new lib.ControllerConfig("controller") + ); + hostA = connectHost(controller, 10, []); + hostB = connectHost(controller, 20, []); + instanceA1 = connectInstance(controller, hostA, 11); + instanceA2 = connectInstance(controller, hostA, 12); + instanceB1 = connectInstance(controller, hostB, 21); + controlA = connectControl(controller, 100); + controlB = connectControl(controller, 101); + }); + + /** @returns {Controller | Host | Instance} */ + function get(name) { + return { + controller, + hostA, + hostB, + instanceA1, + instanceA2, + instanceB1, + controlA, + controlB, + }[name]; + } + + // Combinations to try. Equivalent paths have been removed. + const parties = [ + ["controller", ["hostA", "instanceA1", "controlA"]], + ["hostA", ["controller", "hostB", "instanceA1", "instanceB1", "controlA"]], + ["instanceA1", ["controller", "hostA", "hostB", "instanceA2", "instanceB1", "controlA"]], + ["controlA", ["controller", "hostA", "instanceA1", "controlB"]], + ]; + + function handle(dst, Message, handler) { + if (dst instanceof Controller) { + for (const host of dst.wsServer.hostConnections.values()) { + host.handle(Message, handler); + } + for (const control of dst.wsServer.controlConnections.values()) { + control.handle(Message, handler); + } + } else if (dst instanceof Host) { + for (const instance of dst.instanceConnections.values()) { + instance.handle(Message, handler); + } + dst.handle(Message, handler); + } else { + dst.handle(Message, handler); + } + } + + /** + * @param {Controller | Host | Instance} link + * @param {Address} dst + */ + function send(link, dst, message) { + if (link instanceof Host) { + const instance = link.instanceConnections.get(dst.id); + if (instance) { + return instance.sendTo(dst, message); + } + } + return link.sendTo(dst, message); + } + + describe("request routing", function() { + for (const [srcName, dstNames] of parties) { + it(`should route from ${srcName}`, async function() { + for (const dstName of dstNames) { + let called; + const dst = get(dstName); + handle(dst, TestRequest, async () => { called = true; }); + const dstAddr = dst instanceof Controller ? addr("controller") : dst.connector.src; + await send(get(srcName), dstAddr, new TestRequest()); + assert(called, `${dstName} handler was not called`); + } + }); + } + }); + + describe("event routing", function() { + for (const [srcName, dstNames] of parties) { + it(`should route from ${srcName}`, async function() { + for (const dstName of dstNames) { + let called; + const dst = get(dstName); + handle(dst, TestEvent, async () => { called = true; }); + const dstAddr = dst instanceof Controller ? addr("controller") : dst.connector.src; + send(get(srcName), dstAddr, new TestEvent()); + assert(called, `${dstName} handler was not called`); + } + }); + } + }); + + /** + * @param {Controller | Host | Instance | Control} link + * @param {"allHosts" | "allInstances" | "allControls"} dst + */ + function bcast(link, dst, message) { + if (link instanceof Host) { + const plugin = message.constructor.plugin; + if (dst === "allInstances") { + for (const instance of link.instanceConnections.values()) { + if (plugin && !instance.plugins.has(plugin)) { + continue; + } + instance.sendTo(dst, message); + } + } + if (plugin && !link.serverPlugins.has(plugin)) { + return; + } + } + link.sendTo(dst, message); + } + + function bhandle(srcName, dst, Event, canReach = (dstB) => true) { + const asserts = []; + const dsts = { + allHosts: ["hostA", "hostB"], + allInstances: ["instanceA1", "instanceA2", "instanceB1"], + allControls: ["controlA", "controlB"], + }; + for (const dstName of dsts[dst]) { + if (srcName === dstName) { continue; } + // TODO remove when broadcasting from instance to hosts is fixed, see #575 + if (srcName === "instanceA1" && dstName === "hostB") { continue; } + let called; + handle(get(dstName), Event, async () => { called = true; }); + const should = canReach(dstName); + asserts.push(() => assert( + !should ^ called, + `${dstName} handler was ${should ? "not " : ""}called from ${srcName}`, + )); + } + return asserts; + } + + describe("event broadcast", function() { + for (const srcName of ["controller", "hostA", "instanceA1", "controlA"]) { + it(`should broadcast from ${srcName}`, async function() { + for (const dstName of ["allHosts", "allInstances", "allControls"]) { + const dstAsserts = bhandle(srcName, dstName, TestEvent); + bcast(get(srcName), dstName, new TestEvent()); + for (const dstAssert of dstAsserts) { + dstAssert(); + } + } + }); + } + }); + + const graph = { + controller: ["hostA", "hostB", "controlA", "controlB"], + hostA: ["controller", "instanceA1", "instanceA2"], + hostB: ["controller", "instanceB1"], + instanceA1: ["hostA"], + instanceA2: ["hostA"], + instanceB1: ["hostB"], + controlA: ["controller"], + controlB: ["controller"], + }; + + // Returns true if the path from src to dst goes through via + function goesVia(src, dst, via) { + // Places we've been to + const checked = new Set(); + // Places left to check + const paths = new Set([src]); + while (paths.size) { + const [candidate] = paths; + paths.delete(candidate); + if (candidate === via) { + continue; + } + if (checked.has(candidate)) { + continue; + } + if (candidate === dst) { + return false; + } + checked.add(candidate); + for (const newCandidate of graph[candidate]) { + paths.add(newCandidate); + } + } + return true; + } + assert(goesVia("instanceA1", "instanceB1", "controller")); + assert(!goesVia("instanceA1", "instanceA2", "controller")); + assert(goesVia("instanceA1", "hostB", "hostA")); + assert(!goesVia("instanceA1", "hostB", "controlA")); + assert(goesVia("controlA", "controlB", "controller")); + assert(goesVia("controlA", "hostA", "controller")); + + function addPluginExceptFor(exName) { + const ex = get(exName); + const exAddr = ex instanceof Controller ? addr("controller") : ex.connector.src; + for (const name of [ + "controller", "hostA", "hostB", "instanceA1", "instanceA2", "instanceB1", "controlA", "controlB", + ]) { + const party = get(name); + if (party instanceof Controller) { + for (const host of party.wsServer.hostConnections.values()) { + if (!host.connector.dst.equals(exAddr)) { + host.plugins.set("test", "0.0.0"); + } + } + for (const control of party.wsServer.controlConnections.values()) { + if (!control.connector.dst.equals(exAddr)) { + // No way to set plugins + } + } + } else if (party instanceof Host) { + for (const instance of party.instanceConnections.values()) { + if (!instance.connector.dst.equals(exAddr)) { + instance.plugins.set("test", "0.0.0"); + } + } + if (!party.connector.dst.equals(exAddr)) { + party.serverPlugins.set("test", "0.0.0"); + } + } else if (party instanceof Instance) { + // Currently does not handle plugins + } else if (party instanceof Control) { + // Currently does not handle plugins + } + } + } + + // When plugin events are broadcast the party sending the event should + // filter it out if the next hop it is forwarded to does not have plugin + // the event was defined in loaded. + describe("event broadcast with controller missing plugin", function() { + for (const srcName of ["hostA", "instanceA1", "controlA"]) { + it(`should broadcast from ${srcName}`, async function() { + if (srcName === "controlA") { + this.skip(); // Not implemented + } + addPluginExceptFor("controller"); + for (const dstName of ["allHosts", "allInstances", "allControls"]) { + const dstAsserts = bhandle( + srcName, dstName, PluginEvent, dstB => !goesVia(srcName, dstB, "controller"), + ); + bcast(get(srcName), dstName, new PluginEvent()); + for (const dstAssert of dstAsserts) { + dstAssert(); + } + } + }); + } + }); + + describe("event broadcast with host missing plugin", function() { + for (const srcName of ["controller", "hostA", "instanceA1", "controlA"]) { + it(`should broadcast from ${srcName}`, async function() { + addPluginExceptFor("hostB"); + for (const dstName of ["allHosts", "allInstances", "allControls"]) { + const dstAsserts = bhandle( + srcName, dstName, PluginEvent, dstB => !goesVia(srcName, dstB, "hostB"), + ); + bcast(get(srcName), dstName, new PluginEvent()); + for (const dstAssert of dstAsserts) { + dstAssert(); + } + } + }); + } + }); + + describe("event broadcast with instance missing plugin", function() { + for (const srcName of ["controller", "hostA", "instanceA1", "controlA"]) { + it(`should broadcast from ${srcName}`, async function() { + addPluginExceptFor("instanceA2"); + for (const dstName of ["allHosts", "allInstances", "allControls"]) { + const dstAsserts = bhandle( + srcName, dstName, PluginEvent, dstB => !goesVia(srcName, dstB, "instanceA2"), + ); + bcast(get(srcName), dstName, new PluginEvent()); + for (const dstAssert of dstAsserts) { + dstAssert(); + } + } + }); + } + }); + + describe("event broadcast with control missing plugin", function() { + for (const srcName of ["controller", "hostA", "instanceA1", "controlA"]) { + it(`should broadcast from ${srcName}`, async function() { + this.skip(); // Currently not implemented. + addPluginExceptFor("controlB"); + for (const dstName of ["allHosts", "allInstances", "allControls"]) { + const dstAsserts = bhandle( + srcName, dstName, PluginEvent, dstB => !goesVia(srcName, dstB, "controlB"), + ); + bcast(get(srcName), dstName, new PluginEvent()); + for (const dstAssert of dstAsserts) { + dstAssert(); + } + } + }); + } + }); +}); From 2a57ddb02e938d63a89c388428956a11f1b12663 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Wed, 24 Jan 2024 03:08:24 +0100 Subject: [PATCH 32/84] Add note about send and bcast test functions --- test/integration/routing.js | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/test/integration/routing.js b/test/integration/routing.js index ed31de35..6881f079 100644 --- a/test/integration/routing.js +++ b/test/integration/routing.js @@ -164,6 +164,9 @@ describe("Integration of link routing", function() { * @param {Address} dst */ function send(link, dst, message) { + // This logic exists becasue .sendTo doesn't forward all kinds of + // messages to the correct location from all places. Idealy this + // would not be neccessary and .sendTo would do the right thing. if (link instanceof Host) { const instance = link.instanceConnections.get(dst.id); if (instance) { @@ -208,6 +211,9 @@ describe("Integration of link routing", function() { * @param {"allHosts" | "allInstances" | "allControls"} dst */ function bcast(link, dst, message) { + // This logic exists becasue .sendTo doesn't forward all kinds of + // messages to the correct location from all places. Idealy this + // would not be neccessary and .sendTo would do the right thing. if (link instanceof Host) { const plugin = message.constructor.plugin; if (dst === "allInstances") { From 38dcff222570d5f21524a95bc514882995ecc0d7 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 22 Jan 2024 01:23:40 +0100 Subject: [PATCH 33/84] Use h: as short form of host in log lines Before the rename from hosts were called slave which is why s: made sense. Use h: instead. --- packages/lib/src/logging.ts | 2 +- packages/lib/src/logging_utils.ts | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/lib/src/logging.ts b/packages/lib/src/logging.ts index c673f98e..d9ce0c56 100644 --- a/packages/lib/src/logging.ts +++ b/packages/lib/src/logging.ts @@ -73,7 +73,7 @@ export class WebConsoleFormat { transform(info: any, _options: object) { let src = " "; if (info.host_id !== undefined) { - src += `s:${info.host_name} - `; + src += `h:${info.host_name} - `; } if (info.instance_id !== undefined) { src += `i:${info.instance_name}`; diff --git a/packages/lib/src/logging_utils.ts b/packages/lib/src/logging_utils.ts index 6471c3f7..b3b34d35 100644 --- a/packages/lib/src/logging_utils.ts +++ b/packages/lib/src/logging_utils.ts @@ -163,7 +163,7 @@ export class TerminalFormat { src += `i:${info.instance_name} `; } else if (info.host_name) { - src += `s:${info.host_name} `; + src += `h:${info.host_name} `; } if (info.plugin) { src += `${info.plugin}: `; From e3cfc090d37c6bde174df1f837510c7bba7a7c68 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Tue, 23 Jan 2024 16:36:12 +0100 Subject: [PATCH 34/84] Fix string format of Address When host and instance id in Address was reordered in 1e66f11 the toString method Address was updated to account for this. Fix toString showing host addresses as being instance addresses and vice versa. --- packages/lib/src/data/messages_core.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/lib/src/data/messages_core.ts b/packages/lib/src/data/messages_core.ts index 67ea64a5..b4b3f91e 100644 --- a/packages/lib/src/data/messages_core.ts +++ b/packages/lib/src/data/messages_core.ts @@ -87,7 +87,7 @@ export class Address { } toString() { - const typeMap = ["controller", "instance", "host", "control"]; + const typeMap = ["controller", "host", "instance", "control"]; const type = typeMap[this.type] || this.type; if (this.type === Address.broadcast) { return `[Address ${type}:${typeMap[this.id] || this.id}]`; From 499e973db918e8e8cf32efeb46e55671a51d0b20 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Wed, 24 Jan 2024 02:27:14 +0100 Subject: [PATCH 35/84] Fix InstanceConfigSetPropRequest schema Fix the value incorrectly being set as required in the schema and thus preventing props on config field objects of the instance from being removed. --- CHANGELOG.md | 1 + packages/lib/src/config/classes.ts | 2 +- packages/lib/src/data/messages_instance.ts | 2 +- test/integration/clusterio.js | 9 +++++++++ 4 files changed, 12 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0264def0..6ece0937 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,6 +18,7 @@ - Fixed host hanging if stopped while an instance is in the process of creating a save during startup. - Fixed `instance.auto_start` option. - Fixed host log spammed with errors when plugins broadcast messages to instances with the plugin disabled. +- Fixed properties of object config fields not being possible to remove. ## Version 2.0.0-alpha.14 diff --git a/packages/lib/src/config/classes.ts b/packages/lib/src/config/classes.ts index 3e26b2c8..dae4fc8a 100644 --- a/packages/lib/src/config/classes.ts +++ b/packages/lib/src/config/classes.ts @@ -459,7 +459,7 @@ export class Config< setProp( name: Field, prop: string, - value: unknown, + value?: unknown, location = this.location, ) { let def = this.constructor.fieldDefinitions[name]; diff --git a/packages/lib/src/data/messages_instance.ts b/packages/lib/src/data/messages_instance.ts index ff29d886..f71b2a13 100644 --- a/packages/lib/src/data/messages_instance.ts +++ b/packages/lib/src/data/messages_instance.ts @@ -208,7 +208,7 @@ export class InstanceConfigSetPropRequest { "instanceId": Type.Integer(), "field": Type.String(), "prop": Type.String(), - "value": Type.Unknown(), + "value": Type.Optional(Type.Unknown()), }); static fromJSON(json: Static) { diff --git a/test/integration/clusterio.js b/test/integration/clusterio.js index d34b8a0d..4fe1f626 100644 --- a/test/integration/clusterio.js +++ b/test/integration/clusterio.js @@ -557,6 +557,15 @@ describe("Integration of Clusterio", function() { slowTest(this); await checkInstanceStatus(44, "running"); }); + it("should allow creating and removing props", async function() { + slowTest(this); + await execCtl("instance config set-prop test factorio.settings new_property new-string-value"); + let config = await getControl().send(new lib.InstanceConfigGetRequest(44)); + assert.equal(config["factorio.settings"]["new_property"], "new-string-value"); + await execCtl("instance config set-prop test factorio.settings new_property"); + config = await getControl().send(new lib.InstanceConfigGetRequest(44)); + assert.equal(config["factorio.settings"]["new_property"], undefined); + }); }); describe("user set-admin/whitelisted/banned", function() { From 6732397061d8b16519eaba67eacf8fc7b091d469 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Wed, 24 Jan 2024 04:35:24 +0100 Subject: [PATCH 36/84] Fix Web UI not clearing config props (#578) --- packages/web_ui/src/components/ControllerConfigTree.tsx | 2 ++ packages/web_ui/src/components/HostConfigTree.tsx | 2 ++ packages/web_ui/src/components/InstanceConfigTree.tsx | 2 ++ 3 files changed, 6 insertions(+) diff --git a/packages/web_ui/src/components/ControllerConfigTree.tsx b/packages/web_ui/src/components/ControllerConfigTree.tsx index 71acc44e..a369fab7 100644 --- a/packages/web_ui/src/components/ControllerConfigTree.tsx +++ b/packages/web_ui/src/components/ControllerConfigTree.tsx @@ -27,6 +27,8 @@ export default function ControllerConfigTree() { } catch (err) { return; } + } else { + value = undefined; } await control.send(new lib.ControllerConfigSetPropRequest(field, prop, value)); } diff --git a/packages/web_ui/src/components/HostConfigTree.tsx b/packages/web_ui/src/components/HostConfigTree.tsx index 6f075ac3..aef520f0 100644 --- a/packages/web_ui/src/components/HostConfigTree.tsx +++ b/packages/web_ui/src/components/HostConfigTree.tsx @@ -24,6 +24,8 @@ export default function HostConfigTree(props: { id: number, available: boolean } } catch (err) { return; } + } else { + value = undefined; } await control.sendTo({ hostId: props.id }, new lib.HostConfigSetPropRequest(field, prop, value)); } diff --git a/packages/web_ui/src/components/InstanceConfigTree.tsx b/packages/web_ui/src/components/InstanceConfigTree.tsx index 51b0d4d2..c9a45530 100644 --- a/packages/web_ui/src/components/InstanceConfigTree.tsx +++ b/packages/web_ui/src/components/InstanceConfigTree.tsx @@ -28,6 +28,8 @@ export default function InstanceConfigTree(props: InstanceConfigTreeProps) { } catch (err) { return; } + } else { + value = undefined; } await control.send(new lib.InstanceConfigSetPropRequest(props.id, field, prop, value)); } From 7d6eac65582497f30365ba02180149c4d3581930 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Wed, 24 Jan 2024 04:40:21 +0100 Subject: [PATCH 37/84] v2.0.0-alpha.15 --- packages/controller/package.json | 6 +++--- packages/create/package.json | 2 +- packages/ctl/package.json | 4 ++-- packages/host/package.json | 4 ++-- packages/lib/package.json | 2 +- packages/web_ui/package.json | 4 ++-- plugins/global_chat/package.json | 6 +++--- plugins/inventory_sync/package.json | 6 +++--- plugins/player_auth/package.json | 6 +++--- plugins/research_sync/package.json | 6 +++--- plugins/statistics_exporter/package.json | 6 +++--- plugins/subspace_storage/package.json | 6 +++--- 12 files changed, 29 insertions(+), 29 deletions(-) diff --git a/packages/controller/package.json b/packages/controller/package.json index 6f967415..87cff858 100644 --- a/packages/controller/package.json +++ b/packages/controller/package.json @@ -3,7 +3,7 @@ "description": "Implementation of Clusterio controller", "author": "Hornwitser", "license": "MIT", - "version": "2.0.0-alpha.14", + "version": "2.0.0-alpha.15", "main": "dist/index.js", "scripts": { "prepare": "tsc --build && webpack-cli --env production", @@ -19,7 +19,7 @@ "node": ">=18" }, "dependencies": { - "@clusterio/lib": "^2.0.0-alpha.14", + "@clusterio/lib": "^2.0.0-alpha.15", "@sinclair/typebox": "^0.30.4", "busboy": "^1.6.0", "compression": "^1.7.4", @@ -35,7 +35,7 @@ "yargs": "^17.7.2" }, "devDependencies": { - "@clusterio/web_ui": "^2.0.0-alpha.14", + "@clusterio/web_ui": "^2.0.0-alpha.15", "@types/busboy": "^1.5.0", "@types/compression": "^1.7.2", "@types/express": "^4.17.17", diff --git a/packages/create/package.json b/packages/create/package.json index 6f320a27..5cd7688d 100644 --- a/packages/create/package.json +++ b/packages/create/package.json @@ -3,7 +3,7 @@ "description": "Installer for clusterio", "author": "Hornwitser", "license": "MIT", - "version": "2.0.0-alpha.14", + "version": "2.0.0-alpha.15", "main": "create.js", "bin": { "clusteriocreate": "./create.js" diff --git a/packages/ctl/package.json b/packages/ctl/package.json index 18c95a2e..c4414789 100644 --- a/packages/ctl/package.json +++ b/packages/ctl/package.json @@ -3,7 +3,7 @@ "description": "Clusterio command line management tool", "author": "Hornwitser", "license": "MIT", - "version": "2.0.0-alpha.14", + "version": "2.0.0-alpha.15", "main": "dist/index.js", "scripts": { "prepare": "tsc --build", @@ -19,7 +19,7 @@ "node": ">=18" }, "dependencies": { - "@clusterio/lib": "^2.0.0-alpha.14", + "@clusterio/lib": "^2.0.0-alpha.15", "as-table": "^1.0.55", "fs-extra": "^11.1.1", "phin": "^3.7.0", diff --git a/packages/host/package.json b/packages/host/package.json index 54e963e2..2f4a4297 100644 --- a/packages/host/package.json +++ b/packages/host/package.json @@ -3,7 +3,7 @@ "description": "Implementation of Clusterio host server", "author": "Hornwitser", "license": "MIT", - "version": "2.0.0-alpha.14", + "version": "2.0.0-alpha.15", "main": "dist/index.js", "scripts": { "prepare": "tsc --build", @@ -19,7 +19,7 @@ "node": ">=18" }, "dependencies": { - "@clusterio/lib": "^2.0.0-alpha.14", + "@clusterio/lib": "^2.0.0-alpha.15", "@sinclair/typebox": "^0.30.4", "fs-extra": "^11.1.1", "jimp": "^0.22.8", diff --git a/packages/lib/package.json b/packages/lib/package.json index 6cd1aebc..1b305045 100644 --- a/packages/lib/package.json +++ b/packages/lib/package.json @@ -1,7 +1,7 @@ { "name": "@clusterio/lib", "description": "Shared library for Clusterio", - "version": "2.0.0-alpha.14", + "version": "2.0.0-alpha.15", "repository": "https://github.com/clusterio/clusterio", "license": "MIT", "scripts": { diff --git a/packages/web_ui/package.json b/packages/web_ui/package.json index 6608a14c..93ac9b45 100644 --- a/packages/web_ui/package.json +++ b/packages/web_ui/package.json @@ -3,7 +3,7 @@ "description": "Clusterio web interface implementation", "author": "Hornwitser", "license": "MIT", - "version": "2.0.0-alpha.14", + "version": "2.0.0-alpha.15", "browser": "src/index.ts", "main": "dist/src/index.js", "scripts": { @@ -19,7 +19,7 @@ "@ant-design/icons": "^5.1.4", "@babel/core": "^7.22.9", "@babel/preset-react": "^7.22.5", - "@clusterio/lib": "^2.0.0-alpha.14", + "@clusterio/lib": "^2.0.0-alpha.15", "antd": "^5.13.0", "assert": "^2.0.0", "babel-loader": "^9.1.3", diff --git a/plugins/global_chat/package.json b/plugins/global_chat/package.json index b438bec3..6e1ffcf0 100644 --- a/plugins/global_chat/package.json +++ b/plugins/global_chat/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-global_chat", - "version": "2.0.0-alpha.14", + "version": "2.0.0-alpha.15", "description": "Clusterio plugin forwarding between Factorio servers", "author": "Hornwitser ", "homepage": "https://github.com/clusterio/clusterio#readme", @@ -21,8 +21,8 @@ "@clusterio/lib": "^2.0.0-alpha.0" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.14", - "@clusterio/web_ui": "^2.0.0-alpha.14", + "@clusterio/lib": "^2.0.0-alpha.15", + "@clusterio/web_ui": "^2.0.0-alpha.15", "@types/node": "^20.4.5", "@types/react": "^18.2.21", "typescript": "^5.1.6", diff --git a/plugins/inventory_sync/package.json b/plugins/inventory_sync/package.json index 5664be56..f5530566 100644 --- a/plugins/inventory_sync/package.json +++ b/plugins/inventory_sync/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-inventory_sync", - "version": "2.0.0-alpha.14", + "version": "2.0.0-alpha.15", "description": "Clusterio plugin forwarding between Factorio servers", "author": "Danielv ", "homepage": "https://github.com/clusterio/factorioClusterio#readme", @@ -25,8 +25,8 @@ "fs-extra": "^8.1.0" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.14", - "@clusterio/web_ui": "^2.0.0-alpha.14", + "@clusterio/lib": "^2.0.0-alpha.15", + "@clusterio/web_ui": "^2.0.0-alpha.15", "@types/fs-extra": "^11.0.1", "@types/node": "^20.4.5", "@types/react": "^18.2.21", diff --git a/plugins/player_auth/package.json b/plugins/player_auth/package.json index a7f78e0a..3a9bd9d5 100644 --- a/plugins/player_auth/package.json +++ b/plugins/player_auth/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-player_auth", - "version": "2.0.0-alpha.14", + "version": "2.0.0-alpha.15", "description": "Clusterio plugin authenticating logged in players to the web interface", "author": "Hornwitser ", "homepage": "https://github.com/clusterio/clusterio#readme", @@ -26,8 +26,8 @@ "jsonwebtoken": "^9.0.1" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.14", - "@clusterio/web_ui": "^2.0.0-alpha.14", + "@clusterio/lib": "^2.0.0-alpha.15", + "@clusterio/web_ui": "^2.0.0-alpha.15", "@types/express": "^4.17.17", "@types/jsonwebtoken": "^9.0.2", "@types/node": "^20.4.5", diff --git a/plugins/research_sync/package.json b/plugins/research_sync/package.json index 19ab6b70..c2184c13 100644 --- a/plugins/research_sync/package.json +++ b/plugins/research_sync/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-research_sync", - "version": "2.0.0-alpha.14", + "version": "2.0.0-alpha.15", "description": "Clusterio plugin for synchronizing research between Factorio servers", "author": "Hornwitser ", "homepage": "https://github.com/clusterio/clusterio#readme", @@ -25,8 +25,8 @@ "fs-extra": "^8.1.0" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.14", - "@clusterio/web_ui": "^2.0.0-alpha.14", + "@clusterio/lib": "^2.0.0-alpha.15", + "@clusterio/web_ui": "^2.0.0-alpha.15", "@types/fs-extra": "^11.0.1", "@types/node": "^20.4.5", "@types/react": "^18.2.21", diff --git a/plugins/statistics_exporter/package.json b/plugins/statistics_exporter/package.json index a98facb8..a0d8fed8 100644 --- a/plugins/statistics_exporter/package.json +++ b/plugins/statistics_exporter/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-statistics_exporter", - "version": "2.0.0-alpha.14", + "version": "2.0.0-alpha.15", "description": "Clusterio plugin for gathering statistics from Factorio servers", "author": "Hornwitser ", "homepage": "https://github.com/clusterio/clusterio#readme", @@ -21,8 +21,8 @@ "@clusterio/lib": "^2.0.0-alpha.0" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.14", - "@clusterio/web_ui": "^2.0.0-alpha.14", + "@clusterio/lib": "^2.0.0-alpha.15", + "@clusterio/web_ui": "^2.0.0-alpha.15", "@types/node": "^20.4.5", "@types/react": "^18.2.21", "typescript": "^5.1.6", diff --git a/plugins/subspace_storage/package.json b/plugins/subspace_storage/package.json index 7cd98cf9..e76630bd 100644 --- a/plugins/subspace_storage/package.json +++ b/plugins/subspace_storage/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-subspace_storage", - "version": "2.0.0-alpha.14", + "version": "2.0.0-alpha.15", "description": "Clusterio plugin for sharing storage between Factorio servers", "author": "Hornwitser ", "homepage": "https://github.com/clusterio/clusterio#readme", @@ -25,8 +25,8 @@ "fs-extra": "^8.1.0" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.14", - "@clusterio/web_ui": "^2.0.0-alpha.14", + "@clusterio/lib": "^2.0.0-alpha.15", + "@clusterio/web_ui": "^2.0.0-alpha.15", "@types/express": "^4.17.17", "@types/fs-extra": "^11.0.1", "@types/node": "^20.4.5", From f5255d6f765aee79e00a4eac10ade8fd998163db Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Fri, 26 Jan 2024 13:02:26 +0100 Subject: [PATCH 38/84] Fix message seq reuse on forwarding (#579) When messages were forwarded they were sent using the low level connector .send method which accepts a message and sends it over the link. This method sends the message as is with no modification which means that at no point during the forwarding logic was the message sequence number that originated from Link A renumbered to the correct sequence number to send out on Link B which it was forwarded on. This in turn caused the logic which drops acked messages from the send buffer to break which in turn meant the resume could cause duplication of message and/or messages to be lost. With enough duplicated messages the resume of one connection could cause other connections to drop and also initiate resume. Fix by renumbering the sequence on forwarded messages. In the future the sequences sent and received should also be checked to make sure this kind of problem does not happen again, this is not done here because partially updating a running cluster with sequence problems to one that checks and enforces sequences numbers will be problematic. --- CHANGELOG.md | 7 +++++ packages/controller/src/ControllerRouter.ts | 6 ++-- packages/host/src/Host.ts | 10 +++--- packages/lib/src/link/connectors.ts | 34 +++++++++++++++++++-- packages/lib/src/link/link.ts | 5 +-- test/mock.js | 4 +++ 6 files changed, 54 insertions(+), 12 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6ece0937..7e03d59a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,12 @@ # Changelog +## Version 2.0.0-alpha.16 + +## Changes + +- Fixed message sequences not being renumbered when forwarded between links causing reconnects to send duplicated messages. + + ## Version 2.0.0-alpha.15 ### Features diff --git a/packages/controller/src/ControllerRouter.ts b/packages/controller/src/ControllerRouter.ts index 4679312d..4072c941 100644 --- a/packages/controller/src/ControllerRouter.ts +++ b/packages/controller/src/ControllerRouter.ts @@ -84,7 +84,7 @@ export default class ControllerRouter { if (message.type === "request") { nextHop.forwardRequest(message as lib.MessageRequest, origin); } else { - nextHop.connector.send(message); + nextHop.connector.forward(message); } } else { this.warnUnrouted(message, msg); @@ -107,13 +107,13 @@ export default class ControllerRouter { if (dst.id === lib.Address.host || dst.id === lib.Address.instance) { for (let hostConnection of this.controller.wsServer.hostConnections.values()) { if (hostConnection !== origin && (!plugin || hostConnection.plugins.has(plugin))) { - hostConnection.connector.send(message); + hostConnection.connector.forward(message); } } } else if (dst.id === lib.Address.control) { for (let controlConnection of this.controller.wsServer.controlConnections.values()) { if (controlConnection !== origin) { - controlConnection.connector.send(message); + controlConnection.connector.forward(message); } } } else { diff --git a/packages/host/src/Host.ts b/packages/host/src/Host.ts index b1b9dd41..c2fc5d67 100644 --- a/packages/host/src/Host.ts +++ b/packages/host/src/Host.ts @@ -173,16 +173,16 @@ export class HostRouter { if (dst.id === lib.Address.instance) { for (let instanceConnection of this.host.instanceConnections.values()) { if (instanceConnection !== origin && (!plugin || instanceConnection.plugins.has(plugin))) { - instanceConnection.connector.send(message); + instanceConnection.connector.forward(message); } } if (this.host !== origin && (!plugin || this.host.serverPlugins.has(plugin))) { - this.host.connector.send(message); + this.host.connector.forward(message); } } else if (dst.id === lib.Address.control) { if (this.host !== origin) { if (!plugin || this.host.serverPlugins.has(plugin)) { - this.host.connector.send(message); + this.host.connector.forward(message); } } else { logger.warn(`Received control broadcast of ${(message as lib.MessageEvent).name} from controller.`); @@ -208,7 +208,7 @@ export class HostRouter { } this.host._connectInstance(dst.id).then(instanceConnection => { - instanceConnection.connector.send(message); + instanceConnection.connector.forward(message); }).catch(err => { logger.error(`Error starting instance:\n${err.stack}`, this.host.instanceLogMeta(dst.id)); origin.connector.sendResponseError( @@ -222,7 +222,7 @@ export class HostRouter { if (message.type === "request") { nextHop.forwardRequest(message as lib.MessageRequest, origin); } else { - nextHop.connector.send(message); + nextHop.connector.forward(message); } } catch (err: any) { if (message.type === "request") { diff --git a/packages/lib/src/link/connectors.ts b/packages/lib/src/link/connectors.ts index 2bc15f19..999800ba 100644 --- a/packages/lib/src/link/connectors.ts +++ b/packages/lib/src/link/connectors.ts @@ -33,7 +33,30 @@ export abstract class BaseConnector extends events.EventEmitter { this._seq = 1; } - abstract send(message: libData.Message): void; + protected abstract send(message: libData.MessageRoutable): void; + + forward(message: libData.MessageRoutable) { + const seq = this._seq; + this._seq += 1; + // The message type currently can't be inferred like a tagged union + // TODO fix typing of message so it can be deduced from .type + if (message.type === "request") { + const request = message as libData.MessageRequest; + this.send(new libData.MessageRequest(seq, request.src, request.dst, request.name, request.data)); + } else if (message.type === "response") { + const response = message as libData.MessageResponse; + this.send(new libData.MessageResponse(seq, response.src, response.dst, response.data)); + } else if (message.type === "responseError") { + const error = message as libData.MessageResponseError; + this.send(new libData.MessageResponseError(seq, error.src, error.dst, error.data)); + } else if (message.type === "event") { + const event = message as libData.MessageEvent; + this.send(new libData.MessageEvent(seq, event.src, event.dst, event.name, event.data)); + } else { + throw new Error(`Cannot forward message type ${(message as any).type}`); + } + return seq; + } sendRequest( request: Request, @@ -242,7 +265,7 @@ export abstract class WebSocketBaseConnector extends BaseConnector { * * @param message - Message to send. */ - send(message: libData.MessageRoutable) { + protected send(message: libData.MessageRoutable) { if (!["connected", "resuming"].includes(this._state)) { throw new libErrors.SessionLost("No session"); } @@ -295,6 +318,13 @@ export abstract class WebSocketBaseConnector extends BaseConnector { await this.close(1000, "Disconnect"); } + /** + * Notify other end that the link is ready to disconnect + */ + sendDisconnectReady() { + this._sendInternal(new libData.MessageDisconnect("ready")); + } + /** * True if the connection is established and active */ diff --git a/packages/lib/src/link/link.ts b/packages/lib/src/link/link.ts index 520e5fe4..e3a746c0 100644 --- a/packages/lib/src/link/link.ts +++ b/packages/lib/src/link/link.ts @@ -132,7 +132,8 @@ export class Link { } ).finally( () => { - this.connector.send(new libData.MessageDisconnect("ready")); + // Only WebSocket connectors issue disconnection events. + (this.connector as WebSocketBaseConnector).sendDisconnectReady(); }, ); }); @@ -561,7 +562,7 @@ export class Link { dst: message.dst, }; this._forwardedRequests.set(message.src.index(), pending); - this.connector.send(message); + this.connector.forward(message); } sendEvent(event: Event, dst: libData.Address) { diff --git a/test/mock.js b/test/mock.js index a00729c9..171c5923 100644 --- a/test/mock.js +++ b/test/mock.js @@ -80,6 +80,10 @@ class MockConnector extends lib.BaseConnector { this.sentMessages.push(message); setImmediate(() => this.emit("send", message)); } + + sendDisconnectReady() { + this.send(new lib.MessageDisconnect("ready")); + } } class MockServer extends events.EventEmitter { From c2b6a6e1c09cc05ca1f7cce56d968d79b4ad1625 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Fri, 26 Jan 2024 13:04:07 +0100 Subject: [PATCH 39/84] v2.0.0-alpha.16 --- packages/controller/package.json | 6 +++--- packages/create/package.json | 2 +- packages/ctl/package.json | 4 ++-- packages/host/package.json | 4 ++-- packages/lib/package.json | 2 +- packages/web_ui/package.json | 4 ++-- plugins/global_chat/package.json | 6 +++--- plugins/inventory_sync/package.json | 6 +++--- plugins/player_auth/package.json | 6 +++--- plugins/research_sync/package.json | 6 +++--- plugins/statistics_exporter/package.json | 6 +++--- plugins/subspace_storage/package.json | 6 +++--- 12 files changed, 29 insertions(+), 29 deletions(-) diff --git a/packages/controller/package.json b/packages/controller/package.json index 87cff858..a3da2405 100644 --- a/packages/controller/package.json +++ b/packages/controller/package.json @@ -3,7 +3,7 @@ "description": "Implementation of Clusterio controller", "author": "Hornwitser", "license": "MIT", - "version": "2.0.0-alpha.15", + "version": "2.0.0-alpha.16", "main": "dist/index.js", "scripts": { "prepare": "tsc --build && webpack-cli --env production", @@ -19,7 +19,7 @@ "node": ">=18" }, "dependencies": { - "@clusterio/lib": "^2.0.0-alpha.15", + "@clusterio/lib": "^2.0.0-alpha.16", "@sinclair/typebox": "^0.30.4", "busboy": "^1.6.0", "compression": "^1.7.4", @@ -35,7 +35,7 @@ "yargs": "^17.7.2" }, "devDependencies": { - "@clusterio/web_ui": "^2.0.0-alpha.15", + "@clusterio/web_ui": "^2.0.0-alpha.16", "@types/busboy": "^1.5.0", "@types/compression": "^1.7.2", "@types/express": "^4.17.17", diff --git a/packages/create/package.json b/packages/create/package.json index 5cd7688d..40aa21ba 100644 --- a/packages/create/package.json +++ b/packages/create/package.json @@ -3,7 +3,7 @@ "description": "Installer for clusterio", "author": "Hornwitser", "license": "MIT", - "version": "2.0.0-alpha.15", + "version": "2.0.0-alpha.16", "main": "create.js", "bin": { "clusteriocreate": "./create.js" diff --git a/packages/ctl/package.json b/packages/ctl/package.json index c4414789..8793f1bb 100644 --- a/packages/ctl/package.json +++ b/packages/ctl/package.json @@ -3,7 +3,7 @@ "description": "Clusterio command line management tool", "author": "Hornwitser", "license": "MIT", - "version": "2.0.0-alpha.15", + "version": "2.0.0-alpha.16", "main": "dist/index.js", "scripts": { "prepare": "tsc --build", @@ -19,7 +19,7 @@ "node": ">=18" }, "dependencies": { - "@clusterio/lib": "^2.0.0-alpha.15", + "@clusterio/lib": "^2.0.0-alpha.16", "as-table": "^1.0.55", "fs-extra": "^11.1.1", "phin": "^3.7.0", diff --git a/packages/host/package.json b/packages/host/package.json index 2f4a4297..ce801590 100644 --- a/packages/host/package.json +++ b/packages/host/package.json @@ -3,7 +3,7 @@ "description": "Implementation of Clusterio host server", "author": "Hornwitser", "license": "MIT", - "version": "2.0.0-alpha.15", + "version": "2.0.0-alpha.16", "main": "dist/index.js", "scripts": { "prepare": "tsc --build", @@ -19,7 +19,7 @@ "node": ">=18" }, "dependencies": { - "@clusterio/lib": "^2.0.0-alpha.15", + "@clusterio/lib": "^2.0.0-alpha.16", "@sinclair/typebox": "^0.30.4", "fs-extra": "^11.1.1", "jimp": "^0.22.8", diff --git a/packages/lib/package.json b/packages/lib/package.json index 1b305045..538f5d75 100644 --- a/packages/lib/package.json +++ b/packages/lib/package.json @@ -1,7 +1,7 @@ { "name": "@clusterio/lib", "description": "Shared library for Clusterio", - "version": "2.0.0-alpha.15", + "version": "2.0.0-alpha.16", "repository": "https://github.com/clusterio/clusterio", "license": "MIT", "scripts": { diff --git a/packages/web_ui/package.json b/packages/web_ui/package.json index 93ac9b45..796c3242 100644 --- a/packages/web_ui/package.json +++ b/packages/web_ui/package.json @@ -3,7 +3,7 @@ "description": "Clusterio web interface implementation", "author": "Hornwitser", "license": "MIT", - "version": "2.0.0-alpha.15", + "version": "2.0.0-alpha.16", "browser": "src/index.ts", "main": "dist/src/index.js", "scripts": { @@ -19,7 +19,7 @@ "@ant-design/icons": "^5.1.4", "@babel/core": "^7.22.9", "@babel/preset-react": "^7.22.5", - "@clusterio/lib": "^2.0.0-alpha.15", + "@clusterio/lib": "^2.0.0-alpha.16", "antd": "^5.13.0", "assert": "^2.0.0", "babel-loader": "^9.1.3", diff --git a/plugins/global_chat/package.json b/plugins/global_chat/package.json index 6e1ffcf0..09d9adb6 100644 --- a/plugins/global_chat/package.json +++ b/plugins/global_chat/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-global_chat", - "version": "2.0.0-alpha.15", + "version": "2.0.0-alpha.16", "description": "Clusterio plugin forwarding between Factorio servers", "author": "Hornwitser ", "homepage": "https://github.com/clusterio/clusterio#readme", @@ -21,8 +21,8 @@ "@clusterio/lib": "^2.0.0-alpha.0" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.15", - "@clusterio/web_ui": "^2.0.0-alpha.15", + "@clusterio/lib": "^2.0.0-alpha.16", + "@clusterio/web_ui": "^2.0.0-alpha.16", "@types/node": "^20.4.5", "@types/react": "^18.2.21", "typescript": "^5.1.6", diff --git a/plugins/inventory_sync/package.json b/plugins/inventory_sync/package.json index f5530566..384a64dd 100644 --- a/plugins/inventory_sync/package.json +++ b/plugins/inventory_sync/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-inventory_sync", - "version": "2.0.0-alpha.15", + "version": "2.0.0-alpha.16", "description": "Clusterio plugin forwarding between Factorio servers", "author": "Danielv ", "homepage": "https://github.com/clusterio/factorioClusterio#readme", @@ -25,8 +25,8 @@ "fs-extra": "^8.1.0" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.15", - "@clusterio/web_ui": "^2.0.0-alpha.15", + "@clusterio/lib": "^2.0.0-alpha.16", + "@clusterio/web_ui": "^2.0.0-alpha.16", "@types/fs-extra": "^11.0.1", "@types/node": "^20.4.5", "@types/react": "^18.2.21", diff --git a/plugins/player_auth/package.json b/plugins/player_auth/package.json index 3a9bd9d5..8cf5b5c9 100644 --- a/plugins/player_auth/package.json +++ b/plugins/player_auth/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-player_auth", - "version": "2.0.0-alpha.15", + "version": "2.0.0-alpha.16", "description": "Clusterio plugin authenticating logged in players to the web interface", "author": "Hornwitser ", "homepage": "https://github.com/clusterio/clusterio#readme", @@ -26,8 +26,8 @@ "jsonwebtoken": "^9.0.1" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.15", - "@clusterio/web_ui": "^2.0.0-alpha.15", + "@clusterio/lib": "^2.0.0-alpha.16", + "@clusterio/web_ui": "^2.0.0-alpha.16", "@types/express": "^4.17.17", "@types/jsonwebtoken": "^9.0.2", "@types/node": "^20.4.5", diff --git a/plugins/research_sync/package.json b/plugins/research_sync/package.json index c2184c13..fb9d4424 100644 --- a/plugins/research_sync/package.json +++ b/plugins/research_sync/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-research_sync", - "version": "2.0.0-alpha.15", + "version": "2.0.0-alpha.16", "description": "Clusterio plugin for synchronizing research between Factorio servers", "author": "Hornwitser ", "homepage": "https://github.com/clusterio/clusterio#readme", @@ -25,8 +25,8 @@ "fs-extra": "^8.1.0" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.15", - "@clusterio/web_ui": "^2.0.0-alpha.15", + "@clusterio/lib": "^2.0.0-alpha.16", + "@clusterio/web_ui": "^2.0.0-alpha.16", "@types/fs-extra": "^11.0.1", "@types/node": "^20.4.5", "@types/react": "^18.2.21", diff --git a/plugins/statistics_exporter/package.json b/plugins/statistics_exporter/package.json index a0d8fed8..c8e5a12f 100644 --- a/plugins/statistics_exporter/package.json +++ b/plugins/statistics_exporter/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-statistics_exporter", - "version": "2.0.0-alpha.15", + "version": "2.0.0-alpha.16", "description": "Clusterio plugin for gathering statistics from Factorio servers", "author": "Hornwitser ", "homepage": "https://github.com/clusterio/clusterio#readme", @@ -21,8 +21,8 @@ "@clusterio/lib": "^2.0.0-alpha.0" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.15", - "@clusterio/web_ui": "^2.0.0-alpha.15", + "@clusterio/lib": "^2.0.0-alpha.16", + "@clusterio/web_ui": "^2.0.0-alpha.16", "@types/node": "^20.4.5", "@types/react": "^18.2.21", "typescript": "^5.1.6", diff --git a/plugins/subspace_storage/package.json b/plugins/subspace_storage/package.json index e76630bd..892e9ca9 100644 --- a/plugins/subspace_storage/package.json +++ b/plugins/subspace_storage/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-subspace_storage", - "version": "2.0.0-alpha.15", + "version": "2.0.0-alpha.16", "description": "Clusterio plugin for sharing storage between Factorio servers", "author": "Hornwitser ", "homepage": "https://github.com/clusterio/clusterio#readme", @@ -25,8 +25,8 @@ "fs-extra": "^8.1.0" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.15", - "@clusterio/web_ui": "^2.0.0-alpha.15", + "@clusterio/lib": "^2.0.0-alpha.16", + "@clusterio/web_ui": "^2.0.0-alpha.16", "@types/express": "^4.17.17", "@types/fs-extra": "^11.0.1", "@types/node": "^20.4.5", From 520007c8d1907421ac9eb3d67fbc423a3dfa002c Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 5 Feb 2024 15:51:15 +0100 Subject: [PATCH 40/84] Fix unhandled rejection on RCON lost (#580) When using .finally on a promise a new promise is created which calls the handler passed to it when the the original promise is resolved or rejected, and resolves or rejects with same value as the original promise. This means if the rejection of the new promise that was created by finally is not handled then an unhandled promise rejection will be raised. Fix by making sure the promise rejection from the finally promise created in subspace_storage's handleUpdateStorageEvent is properly handled. --- CHANGELOG.md | 7 +++++++ plugins/subspace_storage/instance.ts | 3 +-- 2 files changed, 8 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7e03d59a..4275137c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,12 @@ # Changelog +## Version 2.0.0-alpha.17 + +## Changes + +- Fixed an unhandled error in Subspace Storage crashing the host. + + ## Version 2.0.0-alpha.16 ## Changes diff --git a/plugins/subspace_storage/instance.ts b/plugins/subspace_storage/instance.ts index 9bc594a8..9771eb26 100644 --- a/plugins/subspace_storage/instance.ts +++ b/plugins/subspace_storage/instance.ts @@ -116,7 +116,6 @@ export class InstancePlugin extends BaseInstancePlugin { let itemsJson = lib.escapeString(JSON.stringify(items)); let task = this.sendRcon(`/sc __subspace_storage__ UpdateInvData("${itemsJson}")`, true); this.pendingTasks.add(task); - task.finally(() => { this.pendingTasks.delete(task); }); - await task; + await task.finally(() => { this.pendingTasks.delete(task); }); } } From 48c31b4825797df5e8840ee699c5b96b280aec14 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 5 Feb 2024 23:36:43 +0100 Subject: [PATCH 41/84] Fix unhandled rejection on disconnect (#582) If a link is closed while the prepareDisconnect action is in progress the link will still try to signal the disconnect ready over the socket when the prepareDisconnect action completes, which throws due to the socket no longer being present. This error is not caught due to the catch happening before the finally on the promise and triggers an unhandled rejection. Fix by rearranging the finally to come before the catch on the promise so that errors are always handled, and checking that the link still has a session before attempting to send the disconnect ready signal. --- CHANGELOG.md | 1 + packages/lib/src/link/link.ts | 17 ++++++++--------- test/mock.js | 1 + 3 files changed, 10 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4275137c..68e772fe 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,7 @@ ## Changes - Fixed an unhandled error in Subspace Storage crashing the host. +- Fixed an unhandled rejection in disconnect logic crashing the host. ## Version 2.0.0-alpha.16 diff --git a/packages/lib/src/link/link.ts b/packages/lib/src/link/link.ts index e3a746c0..0f8f2d32 100644 --- a/packages/lib/src/link/link.ts +++ b/packages/lib/src/link/link.ts @@ -126,16 +126,15 @@ export class Link { }); connector.on("disconnectPrepare", () => { - this.prepareDisconnect().catch( - err => { - logger.error(`Unexpected error preparing disconnect:\n${err.stack}`); + this.prepareDisconnect().finally(() => { + // Only WebSocket connectors issue disconnection events. + const webSocketConnector = this.connector as WebSocketBaseConnector; + if (webSocketConnector.hasSession) { + webSocketConnector.sendDisconnectReady(); } - ).finally( - () => { - // Only WebSocket connectors issue disconnection events. - (this.connector as WebSocketBaseConnector).sendDisconnectReady(); - }, - ); + }).catch(err => { + logger.error(`Unexpected error preparing disconnect:\n${err.stack}`); + }); }); connector.on("invalidate", () => { diff --git a/test/mock.js b/test/mock.js index 171c5923..4d2a8564 100644 --- a/test/mock.js +++ b/test/mock.js @@ -73,6 +73,7 @@ class MockConnector extends lib.BaseConnector { super(src, dst); this.connected = true; + this.hasSession = true; this.sentMessages = []; } From 40f5a6ab7cd1a0a30dc1742580d2492435952226 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 5 Feb 2024 18:02:21 +0100 Subject: [PATCH 42/84] Fix timestamp not set on clearing instance saves Set the save's updatedAtMs timestamp when clearing the save list of a reassigned or deleted instance. This fixes the Web UI not updating the saves list to remove the saves when an instance is reassigned. --- packages/controller/src/Controller.ts | 2 ++ 1 file changed, 2 insertions(+) diff --git a/packages/controller/src/Controller.ts b/packages/controller/src/Controller.ts index 3e81c0c1..d9a1332b 100644 --- a/packages/controller/src/Controller.ts +++ b/packages/controller/src/Controller.ts @@ -876,9 +876,11 @@ export default class Controller { */ clearSavesOfInstance(instanceId: number) { const updates = []; + const now = Date.now(); for (const [id, save] of this.saves) { if (save.instanceId === instanceId) { save.isDeleted = true; + save.updatedAtMs = now; updates.push(save); this.saves.delete(id); } From 7ba14f74330dbf13602b8b061629f9ee6db991ac Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 5 Feb 2024 18:12:30 +0100 Subject: [PATCH 43/84] Fix instance details not updating on reassigmnent Always send the instance details update when an instance is reassigned, not just when it is unassigned. The idea for not sending an update when assigned to a host was that the host would send an update on the status that in turn updates the instance details. However this is not actually the case; when a stopped instance is reassigned its status does not change so the update is ignored. --- CHANGELOG.md | 1 + packages/controller/src/Controller.ts | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 68e772fe..9a507824 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,7 @@ - Fixed an unhandled error in Subspace Storage crashing the host. - Fixed an unhandled rejection in disconnect logic crashing the host. +- Fixed instance screen in Web UI not updating when an instance was reassigned. ## Version 2.0.0-alpha.16 diff --git a/packages/controller/src/Controller.ts b/packages/controller/src/Controller.ts index d9a1332b..5422fedf 100644 --- a/packages/controller/src/Controller.ts +++ b/packages/controller/src/Controller.ts @@ -941,9 +941,9 @@ export default class Controller { ); } else { instance.status = "unassigned"; - instance.updatedAtMs = Date.now(); - this.instanceDetailsUpdated([instance]); } + instance.updatedAtMs = Date.now(); + this.instanceDetailsUpdated([instance]); } /** From 021af793fcece14d4e96a4eba279f2d663c0210f Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 5 Feb 2024 18:13:57 +0100 Subject: [PATCH 44/84] Handle instance status updates with port change If an instance is reassigned its port may change without the status changing, these updates should not be ignored. --- packages/controller/src/HostConnection.ts | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/packages/controller/src/HostConnection.ts b/packages/controller/src/HostConnection.ts index 564b5b96..ab566a40 100644 --- a/packages/controller/src/HostConnection.ts +++ b/packages/controller/src/HostConnection.ts @@ -178,7 +178,10 @@ export default class HostConnection extends BaseConnection { // changed. Hosts also send status updates on assignInstance which // for hacky reason is also used to push config changes and // restablish status after a connection loss. - if (instance.status === request.status) { + if ( + instance.status === request.status + && instance.gamePort === request.gamePort + ) { return; } From 2e25831b4df2ce9bd2e8dc5cd5bbe3c04cf4c91b Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 5 Feb 2024 18:16:15 +0100 Subject: [PATCH 45/84] Warn on subscription updates with same timestamp If the value in a subscribed list is updated while the updatedAtMs timestamp is kept the same then the caching logic of the subscription system will cause updates to be silently dropped. Log a warning if a subscriber catches this happening to help tracking down such cases. --- packages/lib/src/subscriptions.ts | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/packages/lib/src/subscriptions.ts b/packages/lib/src/subscriptions.ts index de155090..120cf2fe 100644 --- a/packages/lib/src/subscriptions.ts +++ b/packages/lib/src/subscriptions.ts @@ -2,6 +2,7 @@ import { Type, Static } from "@sinclair/typebox"; import { Link, Event, EventClass, RequestHandler, WebSocketClientConnector, WebSocketBaseConnector } from "./link"; import { logger } from "./logging"; import { Address, MessageRequest, IControllerUser } from "./data"; +import isDeepStrictEqual from "./is_deep_strict_equal"; export type SubscriptionRequestHandler = RequestHandler | null>; export type EventSubscriberCallback = (updates: T[], synced: boolean) => void @@ -204,6 +205,14 @@ export class EventSubscriber< async _handle(event: EventSubscribable) { for (const value of event.updates) { this.lastResponseTimeMs = Math.max(this.lastResponseTimeMs, value.updatedAtMs); + // Warn about updates which changes content but don't update updatedAtMs + const existing = this.values.get(value.id as K); + if (existing && existing.updatedAtMs === value.updatedAtMs && !isDeepStrictEqual(existing, value)) { + logger.warn( + `${this.Event.name} contains update for value with id ${value.id} that has an identical ` + + "updatedAtMs timestamp but differing content" + ); + } if (value.isDeleted) { this.values.delete(value.id as K); } else { From 6b85567bc63135e5de105d7d83234dbba40e04a4 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 5 Feb 2024 18:45:06 +0100 Subject: [PATCH 46/84] Provide theme to top level ErrorBoundary If the theme is not provided then the App's ErrorBoundary will render white text on white background, and vice versa for black. --- packages/web_ui/src/components/App.tsx | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/packages/web_ui/src/components/App.tsx b/packages/web_ui/src/components/App.tsx index 02e9c726..e4dcfdc6 100644 --- a/packages/web_ui/src/components/App.tsx +++ b/packages/web_ui/src/components/App.tsx @@ -104,16 +104,16 @@ export default function App(props: AppProps) { } return ( - - + + {page} - - + + ); } From 2df67e32daf18676c11d1b18032cd6f1f8614399 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 5 Feb 2024 18:46:55 +0100 Subject: [PATCH 47/84] Set key for top level page fragment Without a unique per page key set react will try to reuse the existing state and components of the previous page and update them to match the new page. This causes unexpected behaviour as components get turned into into a Frankenstein abomination of the previous page and the current one fused into one. Fix by setting a key on the top level fragment of the page which in turn forces React to re-render the whole page whenever the user switches to another page. --- CHANGELOG.md | 1 + packages/web_ui/src/components/SiteLayout.tsx | 6 +++--- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9a507824..fec0adec 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,7 @@ - Fixed an unhandled error in Subspace Storage crashing the host. - Fixed an unhandled rejection in disconnect logic crashing the host. - Fixed instance screen in Web UI not updating when an instance was reassigned. +- Fixed an error thrown in the rendering of one page in the Web UI causing all pages to become inaccessible. ## Version 2.0.0-alpha.16 diff --git a/packages/web_ui/src/components/SiteLayout.tsx b/packages/web_ui/src/components/SiteLayout.tsx index 2bf69d47..07a995d1 100644 --- a/packages/web_ui/src/components/SiteLayout.tsx +++ b/packages/web_ui/src/components/SiteLayout.tsx @@ -1,4 +1,4 @@ -import React, { useContext, useEffect, useState } from "react"; +import React, { Fragment, useContext, useEffect, useState } from "react"; import { Route, Routes, useNavigate } from "react-router-dom"; import { Dropdown, Layout, Menu, MenuProps } from "antd"; import UserOutlined from "@ant-design/icons/UserOutlined"; @@ -115,12 +115,12 @@ export default function SiteLayout() { {combinedPages.map(({path, sidebarPath, content}) => + element={ {content} - } + } />)} } /> From 535e43b5278b9e5c1229bc798e9132e71c60b6f7 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 5 Feb 2024 19:20:33 +0100 Subject: [PATCH 48/84] Rename save.list.subscribe permission Follow the same pattern as all the other subscribable lists by having a separate .list and .subscribe permissions for saves. This also fixes the save list not being visible due to a typo in the permission in the Web UI. --- CHANGELOG.md | 1 + packages/lib/src/data/Role.ts | 10 +++++++++- packages/lib/src/data/messages_instance.ts | 2 +- packages/lib/src/permissions.ts | 6 +++--- packages/web_ui/src/components/InstanceViewPage.tsx | 2 +- 5 files changed, 15 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fec0adec..b5a6b7b5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ - Fixed an unhandled rejection in disconnect logic crashing the host. - Fixed instance screen in Web UI not updating when an instance was reassigned. - Fixed an error thrown in the rendering of one page in the Web UI causing all pages to become inaccessible. +- Fixed Web UI not showing instance saves when the permission to list and subscribe the saves is granted but the user is not a cluster admin. ## Version 2.0.0-alpha.16 diff --git a/packages/lib/src/data/Role.ts b/packages/lib/src/data/Role.ts index 6356def8..56f104bc 100644 --- a/packages/lib/src/data/Role.ts +++ b/packages/lib/src/data/Role.ts @@ -19,12 +19,20 @@ export default class Role { permissions: Type.Array(Type.String()), }); + static remapPermission(permission: string) { + // migrate: core.instance.save.list.subscribe was renamed in alpha 17 + if (permission === "core.instance.save.list.subscribe") { + return "core.instance.save.subscribe"; + } + return permission; + } + static fromJSON(json: Static) { return new this( json.id, json.name, json.description, - new Set(json.permissions), + new Set(json.permissions.map(this.remapPermission)), ); } diff --git a/packages/lib/src/data/messages_instance.ts b/packages/lib/src/data/messages_instance.ts index f71b2a13..2a37d086 100644 --- a/packages/lib/src/data/messages_instance.ts +++ b/packages/lib/src/data/messages_instance.ts @@ -361,7 +361,7 @@ export class InstanceSaveDetailsUpdatesEvent { static type = "event" as const; static src = ["instance", "host", "controller"] as const; static dst = ["controller", "control"] as const; - static permission = "core.instance.save.list.subscribe" as const; + static permission = "core.instance.save.subscribe" as const; constructor( public updates: SaveDetails[], diff --git a/packages/lib/src/permissions.ts b/packages/lib/src/permissions.ts index cbaf9a93..306047de 100644 --- a/packages/lib/src/permissions.ts +++ b/packages/lib/src/permissions.ts @@ -239,11 +239,11 @@ definePermission({ definePermission({ name: "core.instance.save.list", title: "List saves", - description: "List the saves currently on the instance.", + description: "List the saves currently on instances.", }); definePermission({ - name: "core.instance.save.list.subscribe", - title: "Subscribe to save list updatess", + name: "core.instance.save.subscribe", + title: "Subscribe to save updates", description: "Subscribe to be notifed on updates to the list of saves of instances.", }); definePermission({ diff --git a/packages/web_ui/src/components/InstanceViewPage.tsx b/packages/web_ui/src/components/InstanceViewPage.tsx index 13258761..f5b7281c 100644 --- a/packages/web_ui/src/components/InstanceViewPage.tsx +++ b/packages/web_ui/src/components/InstanceViewPage.tsx @@ -204,7 +204,7 @@ export default function InstanceViewPage() { { - account.hasAllPermission("core.instance.save.list", "core.instance.save.list_subscribe") + account.hasAllPermission("core.instance.save.list", "core.instance.save.subscribe") && } { From d3f8c993b16e97829272ea2f21f4a4d2497cf4c0 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 5 Feb 2024 19:26:23 +0100 Subject: [PATCH 49/84] Fix bad permission checks in Web UI Fix attempts to check for permissions that don't exist in the Web UI that has gone unnoticed because the check still succeeds when the account has the cluster admin permission. --- CHANGELOG.md | 1 + packages/web_ui/src/components/HostViewPage.tsx | 2 +- packages/web_ui/src/components/ModPackViewPage.tsx | 2 +- packages/web_ui/src/components/ModsPage.tsx | 4 ++-- packages/web_ui/src/pages.tsx | 2 +- 5 files changed, 6 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b5a6b7b5..65cbac3b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ - Fixed instance screen in Web UI not updating when an instance was reassigned. - Fixed an error thrown in the rendering of one page in the Web UI causing all pages to become inaccessible. - Fixed Web UI not showing instance saves when the permission to list and subscribe the saves is granted but the user is not a cluster admin. +- Similarly fixed Web UI not showing the mod pack create, delete, list and host revoke token actions if the user is not a cluster admin. ## Version 2.0.0-alpha.16 diff --git a/packages/web_ui/src/components/HostViewPage.tsx b/packages/web_ui/src/components/HostViewPage.tsx index fc71311d..f70d9e2c 100644 --- a/packages/web_ui/src/components/HostViewPage.tsx +++ b/packages/web_ui/src/components/HostViewPage.tsx @@ -48,7 +48,7 @@ export default function HostViewPage() { let hostButtons = { - account.hasPermission("core.host.revoke_access") + account.hasPermission("core.host.revoke_token") && - {account.hasPermission("core.mod-pack.delete") && - {account.hasPermission("core.mod-pack.create") && } - {account.hasPermission("core.mod-pack.create") && } + {account.hasPermission("core.mod_pack.create") && } + {account.hasPermission("core.mod_pack.create") && } } /> account.hasAnyPermission("core.mod.list", "core.mod-pack.list"), + permission: (account: UserAccount) => account.hasAnyPermission("core.mod.list", "core.mod_pack.list"), content: , }, { From 44b0a8ff834434c6e3cd75ed232a3e24526bfb5a Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 5 Feb 2024 19:28:21 +0100 Subject: [PATCH 50/84] Throw error on bad permission check in Wed UI Throw an error if an attempt is made to check for a permission that does not exist in the Web UI to prevent future bad permission checks going unnoticed. --- packages/web_ui/src/model/account.ts | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/packages/web_ui/src/model/account.ts b/packages/web_ui/src/model/account.ts index 8086bf3a..dfce6832 100644 --- a/packages/web_ui/src/model/account.ts +++ b/packages/web_ui/src/model/account.ts @@ -22,6 +22,12 @@ export function useAccount(): UserAccount { }; }, [control]); + function checkPermissionExists(permission: string) { + if (!lib.permissions.has(permission)) { + throw new Error(`permission ${permission} does not exist`); + } + } + function hasPermission(permission: string): boolean { for (let role of roles) { if (role.permissions.includes("core.admin") || role.permissions.includes(permission)) { @@ -38,12 +44,16 @@ export function useAccount(): UserAccount { if (!roles) { return null; } + checkPermissionExists(permission); return hasPermission(permission); }, hasAnyPermission(...permissions: string[]): boolean | null { if (!roles) { return null; } + for (let permission of permissions) { + checkPermissionExists(permission); + } for (let permission of permissions) { if (hasPermission(permission)) { return true; @@ -55,6 +65,9 @@ export function useAccount(): UserAccount { if (!roles) { return null; } + for (let permission of permissions) { + checkPermissionExists(permission); + } for (let permission of permissions) { if (!hasPermission(permission)) { return false; From 6bf2bf51f23560570ee9b0d93bfa5eced165fc4a Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 5 Feb 2024 19:29:50 +0100 Subject: [PATCH 51/84] Remove deprecated showArrow prop Ant design has deprecated the Select's showArrow prop. It's now always shown by default. --- packages/web_ui/src/components/UserViewPage.tsx | 1 - 1 file changed, 1 deletion(-) diff --git a/packages/web_ui/src/components/UserViewPage.tsx b/packages/web_ui/src/components/UserViewPage.tsx index b0d138a4..8770571f 100644 --- a/packages/web_ui/src/components/UserViewPage.tsx +++ b/packages/web_ui/src/components/UserViewPage.tsx @@ -88,7 +88,6 @@ export default function UserViewPage() { onChange={() => setRolesDirty(true)} loading={!roles} mode="multiple" - showArrow={true} filterOption={(inputValue:string, option: any): boolean => { let role = roles.get(option.value); return role?.name.toLowerCase().includes(inputValue.toLowerCase()) ?? false; From dcf216ca09ba8c7c37da75053aa82a7ff58845b9 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Tue, 6 Feb 2024 20:50:19 +0100 Subject: [PATCH 52/84] Fix research sync applying progress to wrong level When the level of a technology changes either by setting level directly or by setting researched to true, the saved progress of the research is reset only if the technology is not the current research. If during the initial synchronisation with the controller an instance has progressed on a multi level research, it's the current research, and the controller's level of that research is higher, then the level is set to the higher level and the game keeps the progress. This in turn results in all of the progress made on the previous level being added to the higher level. Fix by setting the progress to zero when changing the level or researched status of a technology that's the active research. --- CHANGELOG.md | 1 + plugins/research_sync/module/sync.lua | 13 ++++++++++++- 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 65cbac3b..77598f4d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,7 @@ - Fixed an error thrown in the rendering of one page in the Web UI causing all pages to become inaccessible. - Fixed Web UI not showing instance saves when the permission to list and subscribe the saves is granted but the user is not a cluster admin. - Similarly fixed Web UI not showing the mod pack create, delete, list and host revoke token actions if the user is not a cluster admin. +- Fixed Research Sync applying progress of a previous techology level to the level on the controller when it's the active research during instance startup. ## Version 2.0.0-alpha.16 diff --git a/plugins/research_sync/module/sync.lua b/plugins/research_sync/module/sync.lua index 03a1bb9e..7abe268a 100644 --- a/plugins/research_sync/module/sync.lua +++ b/plugins/research_sync/module/sync.lua @@ -137,10 +137,21 @@ function research_sync.sync_technologies(data) for _, tech_data in pairs(game.json_to_table(data)) do local tech = force.technologies[tech_data[nameIndex]] if tech and tech.level <= tech_data[levelIndex] then - tech.level = math.min(tech_data[levelIndex], tech.prototype.max_level) + local new_level = math.min(tech_data[levelIndex], tech.prototype.max_level) + if new_level ~= tech.level then + -- when the level of the current research changes the + -- progress is not automatically reset. + if force.current_research == tech then + force.research_progress = 0 + end + tech.level = new_level + end local progress if tech_data[researchedIndex] then + if force.current_research == tech then + force.research_progress = 0 + end tech.researched = true progress = nil elseif tech_data[progressIndex] then From 99990c7a0d57ddae9c656f635c63bd1f80db9cd4 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Wed, 7 Feb 2024 00:05:13 +0100 Subject: [PATCH 53/84] Fix incorrect header level in CHANGELOG.md --- CHANGELOG.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 77598f4d..cac58b2a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,7 +2,7 @@ ## Version 2.0.0-alpha.17 -## Changes +### Changes - Fixed an unhandled error in Subspace Storage crashing the host. - Fixed an unhandled rejection in disconnect logic crashing the host. @@ -15,7 +15,7 @@ ## Version 2.0.0-alpha.16 -## Changes +### Changes - Fixed message sequences not being renumbered when forwarded between links causing reconnects to send duplicated messages. From 8e5d581b9bd126b80753e86871c4132de24dc56d Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Wed, 7 Feb 2024 17:25:42 +0100 Subject: [PATCH 54/84] Move tsconfig.web.json to tsconfig.browser.json Rename the config files for the browser build of modules to better communicate that it's purpose is to compile code for a browser environment rather than the Web UI in particular. This also aligns with the "browser" option in package.json, the existing browser.ts entrypoint in lib, and how the browser environment is typically referred to. --- ...tsconfig.web.json => tsconfig.browser.json} | 6 +++--- ...tsconfig.web.json => tsconfig.browser.json} | 2 +- ...tsconfig.web.json => tsconfig.browser.json} | 4 ++-- packages/web_ui/webpack.common.js | 2 +- plugins/global_chat/tsconfig.browser.json | 11 +++++++++++ plugins/global_chat/tsconfig.web.json | 11 ----------- plugins/inventory_sync/tsconfig.browser.json | 11 +++++++++++ plugins/inventory_sync/tsconfig.web.json | 11 ----------- plugins/player_auth/tsconfig.browser.json | 11 +++++++++++ plugins/player_auth/tsconfig.web.json | 11 ----------- plugins/research_sync/tsconfig.browser.json | 11 +++++++++++ plugins/research_sync/tsconfig.web.json | 11 ----------- .../statistics_exporter/tsconfig.browser.json | 11 +++++++++++ plugins/statistics_exporter/tsconfig.web.json | 11 ----------- plugins/subspace_storage/tsconfig.browser.json | 11 +++++++++++ plugins/subspace_storage/tsconfig.web.json | 11 ----------- tsconfig.web.json => tsconfig.browser.json | 0 tsconfig.json | 18 +++++++++--------- 18 files changed, 82 insertions(+), 82 deletions(-) rename packages/controller/{tsconfig.web.json => tsconfig.browser.json} (54%) rename packages/lib/{tsconfig.web.json => tsconfig.browser.json} (93%) rename packages/web_ui/{tsconfig.web.json => tsconfig.browser.json} (62%) create mode 100644 plugins/global_chat/tsconfig.browser.json delete mode 100644 plugins/global_chat/tsconfig.web.json create mode 100644 plugins/inventory_sync/tsconfig.browser.json delete mode 100644 plugins/inventory_sync/tsconfig.web.json create mode 100644 plugins/player_auth/tsconfig.browser.json delete mode 100644 plugins/player_auth/tsconfig.web.json create mode 100644 plugins/research_sync/tsconfig.browser.json delete mode 100644 plugins/research_sync/tsconfig.web.json create mode 100644 plugins/statistics_exporter/tsconfig.browser.json delete mode 100644 plugins/statistics_exporter/tsconfig.web.json create mode 100644 plugins/subspace_storage/tsconfig.browser.json delete mode 100644 plugins/subspace_storage/tsconfig.web.json rename tsconfig.web.json => tsconfig.browser.json (100%) diff --git a/packages/controller/tsconfig.web.json b/packages/controller/tsconfig.browser.json similarity index 54% rename from packages/controller/tsconfig.web.json rename to packages/controller/tsconfig.browser.json index 6685a163..3fff0691 100644 --- a/packages/controller/tsconfig.web.json +++ b/packages/controller/tsconfig.browser.json @@ -1,8 +1,8 @@ { - "extends": "../../tsconfig.web.json", + "extends": "../../tsconfig.browser.json", "references": [ - { "path": "../lib/tsconfig.web.json" }, - { "path": "../web_ui/tsconfig.web.json" }, + { "path": "../lib/tsconfig.browser.json" }, + { "path": "../web_ui/tsconfig.browser.json" }, ], "include": [ "web/**/*.tsx", "web/**/*.ts", "package.json" ], "compilerOptions": { diff --git a/packages/lib/tsconfig.web.json b/packages/lib/tsconfig.browser.json similarity index 93% rename from packages/lib/tsconfig.web.json rename to packages/lib/tsconfig.browser.json index a4349659..36ebfbd9 100644 --- a/packages/lib/tsconfig.web.json +++ b/packages/lib/tsconfig.browser.json @@ -1,5 +1,5 @@ { - "extends": "../../tsconfig.web.json", + "extends": "../../tsconfig.browser.json", "include": [ "browser.ts", "package.json", diff --git a/packages/web_ui/tsconfig.web.json b/packages/web_ui/tsconfig.browser.json similarity index 62% rename from packages/web_ui/tsconfig.web.json rename to packages/web_ui/tsconfig.browser.json index 65754630..8d0236e7 100644 --- a/packages/web_ui/tsconfig.web.json +++ b/packages/web_ui/tsconfig.browser.json @@ -1,7 +1,7 @@ { - "extends": "../../tsconfig.web.json", + "extends": "../../tsconfig.browser.json", "references": [ - { "path": "../lib/tsconfig.web.json" }, + { "path": "../lib/tsconfig.browser.json" }, ], "include": [ "src/**/*.tsx", "src/**/*.ts", "package.json", "global.d.ts" ], "compilerOptions": { diff --git a/packages/web_ui/webpack.common.js b/packages/web_ui/webpack.common.js index e8a5d8b7..99825c4b 100644 --- a/packages/web_ui/webpack.common.js +++ b/packages/web_ui/webpack.common.js @@ -79,7 +79,7 @@ module.exports = (env = {}) => ({ use: { loader: require.resolve("ts-loader"), options: { - configFile: "tsconfig.web.json", + configFile: "tsconfig.browser.json", projectReferences: true, }, }, diff --git a/plugins/global_chat/tsconfig.browser.json b/plugins/global_chat/tsconfig.browser.json new file mode 100644 index 00000000..a3109220 --- /dev/null +++ b/plugins/global_chat/tsconfig.browser.json @@ -0,0 +1,11 @@ +{ + "extends": "../../tsconfig.browser.json", + "references": [ + { "path": "../../packages/lib/tsconfig.browser.json" }, + { "path": "../../packages/web_ui/tsconfig.browser.json" }, + ], + "include": [ "web/**/*.tsx", "web/**/*.ts", "messages.ts", "package.json" ], + "compilerOptions": { + "outDir": "dist/web/build", + } +} diff --git a/plugins/global_chat/tsconfig.web.json b/plugins/global_chat/tsconfig.web.json deleted file mode 100644 index c37f9141..00000000 --- a/plugins/global_chat/tsconfig.web.json +++ /dev/null @@ -1,11 +0,0 @@ -{ - "extends": "../../tsconfig.web.json", - "references": [ - { "path": "../../packages/lib/tsconfig.web.json" }, - { "path": "../../packages/web_ui/tsconfig.web.json" }, - ], - "include": [ "web/**/*.tsx", "web/**/*.ts", "messages.ts", "package.json" ], - "compilerOptions": { - "outDir": "dist/web/build", - } -} diff --git a/plugins/inventory_sync/tsconfig.browser.json b/plugins/inventory_sync/tsconfig.browser.json new file mode 100644 index 00000000..a3109220 --- /dev/null +++ b/plugins/inventory_sync/tsconfig.browser.json @@ -0,0 +1,11 @@ +{ + "extends": "../../tsconfig.browser.json", + "references": [ + { "path": "../../packages/lib/tsconfig.browser.json" }, + { "path": "../../packages/web_ui/tsconfig.browser.json" }, + ], + "include": [ "web/**/*.tsx", "web/**/*.ts", "messages.ts", "package.json" ], + "compilerOptions": { + "outDir": "dist/web/build", + } +} diff --git a/plugins/inventory_sync/tsconfig.web.json b/plugins/inventory_sync/tsconfig.web.json deleted file mode 100644 index c37f9141..00000000 --- a/plugins/inventory_sync/tsconfig.web.json +++ /dev/null @@ -1,11 +0,0 @@ -{ - "extends": "../../tsconfig.web.json", - "references": [ - { "path": "../../packages/lib/tsconfig.web.json" }, - { "path": "../../packages/web_ui/tsconfig.web.json" }, - ], - "include": [ "web/**/*.tsx", "web/**/*.ts", "messages.ts", "package.json" ], - "compilerOptions": { - "outDir": "dist/web/build", - } -} diff --git a/plugins/player_auth/tsconfig.browser.json b/plugins/player_auth/tsconfig.browser.json new file mode 100644 index 00000000..a3109220 --- /dev/null +++ b/plugins/player_auth/tsconfig.browser.json @@ -0,0 +1,11 @@ +{ + "extends": "../../tsconfig.browser.json", + "references": [ + { "path": "../../packages/lib/tsconfig.browser.json" }, + { "path": "../../packages/web_ui/tsconfig.browser.json" }, + ], + "include": [ "web/**/*.tsx", "web/**/*.ts", "messages.ts", "package.json" ], + "compilerOptions": { + "outDir": "dist/web/build", + } +} diff --git a/plugins/player_auth/tsconfig.web.json b/plugins/player_auth/tsconfig.web.json deleted file mode 100644 index c37f9141..00000000 --- a/plugins/player_auth/tsconfig.web.json +++ /dev/null @@ -1,11 +0,0 @@ -{ - "extends": "../../tsconfig.web.json", - "references": [ - { "path": "../../packages/lib/tsconfig.web.json" }, - { "path": "../../packages/web_ui/tsconfig.web.json" }, - ], - "include": [ "web/**/*.tsx", "web/**/*.ts", "messages.ts", "package.json" ], - "compilerOptions": { - "outDir": "dist/web/build", - } -} diff --git a/plugins/research_sync/tsconfig.browser.json b/plugins/research_sync/tsconfig.browser.json new file mode 100644 index 00000000..a3109220 --- /dev/null +++ b/plugins/research_sync/tsconfig.browser.json @@ -0,0 +1,11 @@ +{ + "extends": "../../tsconfig.browser.json", + "references": [ + { "path": "../../packages/lib/tsconfig.browser.json" }, + { "path": "../../packages/web_ui/tsconfig.browser.json" }, + ], + "include": [ "web/**/*.tsx", "web/**/*.ts", "messages.ts", "package.json" ], + "compilerOptions": { + "outDir": "dist/web/build", + } +} diff --git a/plugins/research_sync/tsconfig.web.json b/plugins/research_sync/tsconfig.web.json deleted file mode 100644 index c37f9141..00000000 --- a/plugins/research_sync/tsconfig.web.json +++ /dev/null @@ -1,11 +0,0 @@ -{ - "extends": "../../tsconfig.web.json", - "references": [ - { "path": "../../packages/lib/tsconfig.web.json" }, - { "path": "../../packages/web_ui/tsconfig.web.json" }, - ], - "include": [ "web/**/*.tsx", "web/**/*.ts", "messages.ts", "package.json" ], - "compilerOptions": { - "outDir": "dist/web/build", - } -} diff --git a/plugins/statistics_exporter/tsconfig.browser.json b/plugins/statistics_exporter/tsconfig.browser.json new file mode 100644 index 00000000..0454f6c4 --- /dev/null +++ b/plugins/statistics_exporter/tsconfig.browser.json @@ -0,0 +1,11 @@ +{ + "extends": "../../tsconfig.browser.json", + "references": [ + { "path": "../../packages/lib/tsconfig.browser.json" }, + { "path": "../../packages/web_ui/tsconfig.browser.json" }, + ], + "include": [ "web/**/*.tsx", "web/**/*.ts", "package.json" ], + "compilerOptions": { + "outDir": "dist/web/build", + } +} diff --git a/plugins/statistics_exporter/tsconfig.web.json b/plugins/statistics_exporter/tsconfig.web.json deleted file mode 100644 index decec50c..00000000 --- a/plugins/statistics_exporter/tsconfig.web.json +++ /dev/null @@ -1,11 +0,0 @@ -{ - "extends": "../../tsconfig.web.json", - "references": [ - { "path": "../../packages/lib/tsconfig.web.json" }, - { "path": "../../packages/web_ui/tsconfig.web.json" }, - ], - "include": [ "web/**/*.tsx", "web/**/*.ts", "package.json" ], - "compilerOptions": { - "outDir": "dist/web/build", - } -} diff --git a/plugins/subspace_storage/tsconfig.browser.json b/plugins/subspace_storage/tsconfig.browser.json new file mode 100644 index 00000000..a3109220 --- /dev/null +++ b/plugins/subspace_storage/tsconfig.browser.json @@ -0,0 +1,11 @@ +{ + "extends": "../../tsconfig.browser.json", + "references": [ + { "path": "../../packages/lib/tsconfig.browser.json" }, + { "path": "../../packages/web_ui/tsconfig.browser.json" }, + ], + "include": [ "web/**/*.tsx", "web/**/*.ts", "messages.ts", "package.json" ], + "compilerOptions": { + "outDir": "dist/web/build", + } +} diff --git a/plugins/subspace_storage/tsconfig.web.json b/plugins/subspace_storage/tsconfig.web.json deleted file mode 100644 index c37f9141..00000000 --- a/plugins/subspace_storage/tsconfig.web.json +++ /dev/null @@ -1,11 +0,0 @@ -{ - "extends": "../../tsconfig.web.json", - "references": [ - { "path": "../../packages/lib/tsconfig.web.json" }, - { "path": "../../packages/web_ui/tsconfig.web.json" }, - ], - "include": [ "web/**/*.tsx", "web/**/*.ts", "messages.ts", "package.json" ], - "compilerOptions": { - "outDir": "dist/web/build", - } -} diff --git a/tsconfig.web.json b/tsconfig.browser.json similarity index 100% rename from tsconfig.web.json rename to tsconfig.browser.json diff --git a/tsconfig.json b/tsconfig.json index c9f896d7..fbb6a7ec 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -2,23 +2,23 @@ "files": [], "references": [ { "path": "./packages/lib" }, - { "path": "./packages/lib/tsconfig.web.json" }, { "path": "./packages/controller" }, - { "path": "./packages/controller/tsconfig.web.json" }, { "path": "./packages/host" }, { "path": "./packages/ctl" }, - { "path": "./packages/web_ui/tsconfig.web.json" }, { "path": "./plugins/global_chat" }, - { "path": "./plugins/global_chat/tsconfig.web.json" }, { "path": "./plugins/inventory_sync" }, - { "path": "./plugins/inventory_sync/tsconfig.web.json" }, { "path": "./plugins/player_auth" }, - { "path": "./plugins/player_auth/tsconfig.web.json" }, { "path": "./plugins/research_sync" }, - { "path": "./plugins/research_sync/tsconfig.web.json" }, { "path": "./plugins/statistics_exporter" }, - { "path": "./plugins/statistics_exporter/tsconfig.web.json" }, { "path": "./plugins/subspace_storage" }, - { "path": "./plugins/subspace_storage/tsconfig.web.json" }, + { "path": "./packages/lib/tsconfig.browser.json" }, + { "path": "./packages/controller/tsconfig.browser.json" }, + { "path": "./packages/web_ui/tsconfig.browser.json" }, + { "path": "./plugins/global_chat/tsconfig.browser.json" }, + { "path": "./plugins/inventory_sync/tsconfig.browser.json" }, + { "path": "./plugins/player_auth/tsconfig.browser.json" }, + { "path": "./plugins/research_sync/tsconfig.browser.json" }, + { "path": "./plugins/statistics_exporter/tsconfig.browser.json" }, + { "path": "./plugins/subspace_storage/tsconfig.browser.json" }, ], } From 152eb7d45433ae09bf38b3baad80345ba869b3d9 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Wed, 7 Feb 2024 17:50:48 +0100 Subject: [PATCH 55/84] Move TypeScript browser build to dist/browser Use dist/browser as the output of the browser oriented TypeScript build of packages instead of dist/web/build. This distinguishes it better from the Web UI bundle which is put together with webpack. --- packages/controller/.npmignore | 2 +- packages/controller/tsconfig.browser.json | 2 +- packages/lib/.npmignore | 2 +- packages/lib/tsconfig.browser.json | 2 +- packages/web_ui/package.json | 2 +- packages/web_ui/tsconfig.browser.json | 2 +- plugins/global_chat/.npmignore | 2 +- plugins/global_chat/tsconfig.browser.json | 2 +- plugins/inventory_sync/.npmignore | 2 +- plugins/inventory_sync/tsconfig.browser.json | 2 +- plugins/player_auth/.npmignore | 2 +- plugins/player_auth/tsconfig.browser.json | 2 +- plugins/research_sync/.npmignore | 2 +- plugins/research_sync/tsconfig.browser.json | 2 +- plugins/statistics_exporter/.npmignore | 2 +- plugins/statistics_exporter/tsconfig.browser.json | 2 +- plugins/subspace_storage/.npmignore | 2 +- plugins/subspace_storage/tsconfig.browser.json | 2 +- 18 files changed, 18 insertions(+), 18 deletions(-) diff --git a/packages/controller/.npmignore b/packages/controller/.npmignore index ebd99907..55622581 100644 --- a/packages/controller/.npmignore +++ b/packages/controller/.npmignore @@ -1,2 +1,2 @@ webpack.config.js -dist/web/build +dist/browser diff --git a/packages/controller/tsconfig.browser.json b/packages/controller/tsconfig.browser.json index 3fff0691..154bd176 100644 --- a/packages/controller/tsconfig.browser.json +++ b/packages/controller/tsconfig.browser.json @@ -7,6 +7,6 @@ "include": [ "web/**/*.tsx", "web/**/*.ts", "package.json" ], "compilerOptions": { "rootDir": "..", - "outDir": "dist/web/build", + "outDir": "dist/browser", } } diff --git a/packages/lib/.npmignore b/packages/lib/.npmignore index 5dfa68f1..55239636 100644 --- a/packages/lib/.npmignore +++ b/packages/lib/.npmignore @@ -1 +1 @@ -dist/web/build +dist/browser diff --git a/packages/lib/tsconfig.browser.json b/packages/lib/tsconfig.browser.json index 36ebfbd9..a53b11d7 100644 --- a/packages/lib/tsconfig.browser.json +++ b/packages/lib/tsconfig.browser.json @@ -25,6 +25,6 @@ ], "compilerOptions": { "rootDir": ".", - "outDir": "dist/web/build", + "outDir": "dist/browser", } } diff --git a/packages/web_ui/package.json b/packages/web_ui/package.json index 796c3242..fb77d82d 100644 --- a/packages/web_ui/package.json +++ b/packages/web_ui/package.json @@ -5,7 +5,7 @@ "license": "MIT", "version": "2.0.0-alpha.16", "browser": "src/index.ts", - "main": "dist/src/index.js", + "main": "dist/browser/src/index.js", "scripts": { "test": "echo \"Error: no test specified\" && exit 1" }, diff --git a/packages/web_ui/tsconfig.browser.json b/packages/web_ui/tsconfig.browser.json index 8d0236e7..62dbe09d 100644 --- a/packages/web_ui/tsconfig.browser.json +++ b/packages/web_ui/tsconfig.browser.json @@ -5,6 +5,6 @@ ], "include": [ "src/**/*.tsx", "src/**/*.ts", "package.json", "global.d.ts" ], "compilerOptions": { - "outDir": "dist", + "outDir": "dist/browser", } } diff --git a/plugins/global_chat/.npmignore b/plugins/global_chat/.npmignore index ebd99907..55622581 100644 --- a/plugins/global_chat/.npmignore +++ b/plugins/global_chat/.npmignore @@ -1,2 +1,2 @@ webpack.config.js -dist/web/build +dist/browser diff --git a/plugins/global_chat/tsconfig.browser.json b/plugins/global_chat/tsconfig.browser.json index a3109220..e1b39d0f 100644 --- a/plugins/global_chat/tsconfig.browser.json +++ b/plugins/global_chat/tsconfig.browser.json @@ -6,6 +6,6 @@ ], "include": [ "web/**/*.tsx", "web/**/*.ts", "messages.ts", "package.json" ], "compilerOptions": { - "outDir": "dist/web/build", + "outDir": "dist/browser", } } diff --git a/plugins/inventory_sync/.npmignore b/plugins/inventory_sync/.npmignore index ebd99907..55622581 100644 --- a/plugins/inventory_sync/.npmignore +++ b/plugins/inventory_sync/.npmignore @@ -1,2 +1,2 @@ webpack.config.js -dist/web/build +dist/browser diff --git a/plugins/inventory_sync/tsconfig.browser.json b/plugins/inventory_sync/tsconfig.browser.json index a3109220..e1b39d0f 100644 --- a/plugins/inventory_sync/tsconfig.browser.json +++ b/plugins/inventory_sync/tsconfig.browser.json @@ -6,6 +6,6 @@ ], "include": [ "web/**/*.tsx", "web/**/*.ts", "messages.ts", "package.json" ], "compilerOptions": { - "outDir": "dist/web/build", + "outDir": "dist/browser", } } diff --git a/plugins/player_auth/.npmignore b/plugins/player_auth/.npmignore index ebd99907..55622581 100644 --- a/plugins/player_auth/.npmignore +++ b/plugins/player_auth/.npmignore @@ -1,2 +1,2 @@ webpack.config.js -dist/web/build +dist/browser diff --git a/plugins/player_auth/tsconfig.browser.json b/plugins/player_auth/tsconfig.browser.json index a3109220..e1b39d0f 100644 --- a/plugins/player_auth/tsconfig.browser.json +++ b/plugins/player_auth/tsconfig.browser.json @@ -6,6 +6,6 @@ ], "include": [ "web/**/*.tsx", "web/**/*.ts", "messages.ts", "package.json" ], "compilerOptions": { - "outDir": "dist/web/build", + "outDir": "dist/browser", } } diff --git a/plugins/research_sync/.npmignore b/plugins/research_sync/.npmignore index ebd99907..55622581 100644 --- a/plugins/research_sync/.npmignore +++ b/plugins/research_sync/.npmignore @@ -1,2 +1,2 @@ webpack.config.js -dist/web/build +dist/browser diff --git a/plugins/research_sync/tsconfig.browser.json b/plugins/research_sync/tsconfig.browser.json index a3109220..e1b39d0f 100644 --- a/plugins/research_sync/tsconfig.browser.json +++ b/plugins/research_sync/tsconfig.browser.json @@ -6,6 +6,6 @@ ], "include": [ "web/**/*.tsx", "web/**/*.ts", "messages.ts", "package.json" ], "compilerOptions": { - "outDir": "dist/web/build", + "outDir": "dist/browser", } } diff --git a/plugins/statistics_exporter/.npmignore b/plugins/statistics_exporter/.npmignore index ebd99907..55622581 100644 --- a/plugins/statistics_exporter/.npmignore +++ b/plugins/statistics_exporter/.npmignore @@ -1,2 +1,2 @@ webpack.config.js -dist/web/build +dist/browser diff --git a/plugins/statistics_exporter/tsconfig.browser.json b/plugins/statistics_exporter/tsconfig.browser.json index 0454f6c4..f2dc36da 100644 --- a/plugins/statistics_exporter/tsconfig.browser.json +++ b/plugins/statistics_exporter/tsconfig.browser.json @@ -6,6 +6,6 @@ ], "include": [ "web/**/*.tsx", "web/**/*.ts", "package.json" ], "compilerOptions": { - "outDir": "dist/web/build", + "outDir": "dist/browser", } } diff --git a/plugins/subspace_storage/.npmignore b/plugins/subspace_storage/.npmignore index ebd99907..55622581 100644 --- a/plugins/subspace_storage/.npmignore +++ b/plugins/subspace_storage/.npmignore @@ -1,2 +1,2 @@ webpack.config.js -dist/web/build +dist/browser diff --git a/plugins/subspace_storage/tsconfig.browser.json b/plugins/subspace_storage/tsconfig.browser.json index a3109220..e1b39d0f 100644 --- a/plugins/subspace_storage/tsconfig.browser.json +++ b/plugins/subspace_storage/tsconfig.browser.json @@ -6,6 +6,6 @@ ], "include": [ "web/**/*.tsx", "web/**/*.ts", "messages.ts", "package.json" ], "compilerOptions": { - "outDir": "dist/web/build", + "outDir": "dist/browser", } } From 8c4d2db19211cd3bcdad847b232945ee8f6231d1 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Wed, 7 Feb 2024 18:27:46 +0100 Subject: [PATCH 56/84] Move TypeScript Node.js build to dist/node Use a separate folder for the TypeScript build for Node.js in the dist folder. This makes each type of build consistently have its own output folder instead of being interleaved with the Node.js build. --- .eslintrc.js | 2 +- docs/devs/plugin-migration-to-alpha-14.md | 2 +- packages/controller/package.json | 4 ++-- packages/controller/src/Controller.ts | 8 ++++---- packages/controller/tsconfig.json | 2 +- packages/ctl/package.json | 4 ++-- packages/ctl/tsconfig.json | 2 +- packages/host/package.json | 4 ++-- packages/host/src/Instance.ts | 2 +- packages/host/src/export.ts | 4 ++-- packages/host/tsconfig.json | 2 +- packages/lib/package.json | 2 +- packages/lib/scripts/compile_validator.js | 6 +++--- packages/lib/tsconfig.json | 2 +- plugins/global_chat/index.ts | 4 ++-- plugins/global_chat/package.json | 2 +- plugins/global_chat/test/plugin.js | 6 +++--- plugins/global_chat/tsconfig.json | 2 +- plugins/inventory_sync/index.ts | 4 ++-- plugins/inventory_sync/package.json | 2 +- plugins/inventory_sync/tsconfig.json | 2 +- plugins/player_auth/index.ts | 4 ++-- plugins/player_auth/package.json | 2 +- plugins/player_auth/test/plugin.js | 8 ++++---- plugins/player_auth/tsconfig.json | 2 +- plugins/research_sync/index.ts | 4 ++-- plugins/research_sync/package.json | 2 +- plugins/research_sync/test/missing.js | 4 ++-- plugins/research_sync/tsconfig.json | 2 +- plugins/statistics_exporter/index.ts | 2 +- plugins/statistics_exporter/package.json | 2 +- plugins/statistics_exporter/test/plugin.js | 4 ++-- plugins/statistics_exporter/tsconfig.json | 2 +- plugins/subspace_storage/index.ts | 4 ++-- plugins/subspace_storage/package.json | 2 +- plugins/subspace_storage/test/missing.js | 8 ++++---- plugins/subspace_storage/test/routes.js | 2 +- plugins/subspace_storage/tsconfig.json | 2 +- test/controller/HttpCloser.js | 2 +- test/controller/routes.js | 2 +- test/host.js | 4 ++-- test/host/Instance.js | 2 +- test/host/export.js | 4 ++-- test/host/patch.js | 2 +- test/host/server.js | 2 +- test/integration/export.js | 4 ++-- test/integration/index.js | 10 +++++----- test/integration/patch.js | 2 +- test/integration/server.js | 2 +- test/mock.js | 4 ++-- 50 files changed, 82 insertions(+), 82 deletions(-) diff --git a/.eslintrc.js b/.eslintrc.js index 1487649a..d4cbf768 100644 --- a/.eslintrc.js +++ b/.eslintrc.js @@ -391,7 +391,7 @@ module.exports = { "node/process-exit-as-throw": "error", "node/shebang": ["error", { "convertPath": { - "*.ts": ["^(.+)\\.ts$", "dist/$1.js"], + "*.ts": ["^(.+)\\.ts$", "dist/node/$1.js"], }, }], }, diff --git a/docs/devs/plugin-migration-to-alpha-14.md b/docs/devs/plugin-migration-to-alpha-14.md index 91962795..28266664 100644 --- a/docs/devs/plugin-migration-to-alpha-14.md +++ b/docs/devs/plugin-migration-to-alpha-14.md @@ -78,7 +78,7 @@ import { BaseInstancePlugin } from "@clusterio/host"; Previously the info of a plugin would be loaded by requiring `./plugin_dir/info`, this has been changed to require `./plugin_dir` which in turn means that `info.js` is no longer the default entrypoint. To solve this you can either add `"main": "info.js",` to package.json or rename `info.js` to `index.js` (the rest of this guide assumes the entrypoint was renamed to `index.js`). -The motivation behind this change was to allow TypeScript to output its build to a sub-folder of the plugin, if you're using TypeScript you'll likely have the entrypoint be located in `dist/index.js` instead. +The motivation behind this change was to allow TypeScript to output its build to a sub-folder of the plugin, if you're using TypeScript you'll likely have the entrypoint be located in `dist/node/index.js` instead. The expected place where the declaration of a plugin is exported has also changed from `module.exports` to `module.exports.plugin`: diff --git a/packages/controller/package.json b/packages/controller/package.json index a3da2405..7446edbd 100644 --- a/packages/controller/package.json +++ b/packages/controller/package.json @@ -4,13 +4,13 @@ "author": "Hornwitser", "license": "MIT", "version": "2.0.0-alpha.16", - "main": "dist/index.js", + "main": "dist/node/index.js", "scripts": { "prepare": "tsc --build && webpack-cli --env production", "test": "echo \"Error: no test specified\" && exit 1" }, "bin": { - "clusteriocontroller": "./dist/controller.js" + "clusteriocontroller": "./dist/node/controller.js" }, "keywords": [ "factorio" diff --git a/packages/controller/src/Controller.ts b/packages/controller/src/Controller.ts index 5422fedf..7bd67a52 100644 --- a/packages/controller/src/Controller.ts +++ b/packages/controller/src/Controller.ts @@ -274,7 +274,7 @@ export default class Controller { Controller.addAppRoutes(this.app, this.pluginInfos); if (!args.dev) { - let manifestPath = path.join(__dirname, "..", "web", "manifest.json"); + let manifestPath = path.join(__dirname, "..", "..", "web", "manifest.json"); let manifest = await Controller.loadJsonObject(manifestPath); if (!manifest["main.js"]) { @@ -333,7 +333,7 @@ export default class Controller { if (args.dev) { // eslint-disable-next-line node/no-missing-require - webpackConfigs.push(require("../../webpack.config")({})); // Path outside of build + webpackConfigs.push(require("../../../webpack.config")({})); // Path outside of build } if (args.devPlugin) { let devPlugins = new Map(); @@ -737,7 +737,7 @@ export default class Controller { // Set folder to serve static content from (the website) const staticOptions = { immutable: true, maxAge: 1000 * 86400 * 365 }; app.use("/static", - express.static(path.join(__dirname, "..", "..", "dist", "web", "static"), staticOptions) + express.static(path.join(__dirname, "..", "..", "web", "static"), staticOptions) ); app.use("/static", express.static("static", staticOptions)); // Used for data export files @@ -1226,7 +1226,7 @@ export default class Controller { mainBundle = stats.toJson().assetsByChunkName["main"]; } - fs.readFile(path.join(__dirname, "..", "..", "web", "index.html"), "utf8").then((content) => { + fs.readFile(path.join(__dirname, "..", "..", "..", "web", "index.html"), "utf8").then((content) => { res.type("text/html"); res.send(content .replace(/__CLUSTER_NAME__/g, res.app.locals.controller.config.get("controller.name")) diff --git a/packages/controller/tsconfig.json b/packages/controller/tsconfig.json index 335b8b16..8dcc9b8b 100644 --- a/packages/controller/tsconfig.json +++ b/packages/controller/tsconfig.json @@ -5,6 +5,6 @@ ], "include": ["index.ts", "controller.ts", "package.json", "src/**/*.ts"], "compilerOptions": { - "outDir": "dist", + "outDir": "dist/node", }, } diff --git a/packages/ctl/package.json b/packages/ctl/package.json index 8793f1bb..7d5f6707 100644 --- a/packages/ctl/package.json +++ b/packages/ctl/package.json @@ -4,13 +4,13 @@ "author": "Hornwitser", "license": "MIT", "version": "2.0.0-alpha.16", - "main": "dist/index.js", + "main": "dist/node/index.js", "scripts": { "prepare": "tsc --build", "test": "echo \"Error: no test specified\" && exit 1" }, "bin": { - "clusterioctl": "./dist/ctl.js" + "clusterioctl": "./dist/node/ctl.js" }, "keywords": [ "factorio" diff --git a/packages/ctl/tsconfig.json b/packages/ctl/tsconfig.json index 574dad9d..134019a5 100644 --- a/packages/ctl/tsconfig.json +++ b/packages/ctl/tsconfig.json @@ -5,6 +5,6 @@ ], "include": ["index.ts", "package.json", "ctl.ts", "src/**/*.ts"], "compilerOptions": { - "outDir": "dist", + "outDir": "dist/node", }, } diff --git a/packages/host/package.json b/packages/host/package.json index ce801590..c9f8446a 100644 --- a/packages/host/package.json +++ b/packages/host/package.json @@ -4,13 +4,13 @@ "author": "Hornwitser", "license": "MIT", "version": "2.0.0-alpha.16", - "main": "dist/index.js", + "main": "dist/node/index.js", "scripts": { "prepare": "tsc --build", "test": "echo \"Error: no test specified\" && exit 1" }, "bin": { - "clusteriohost": "./dist/host.js" + "clusteriohost": "./dist/node/host.js" }, "keywords": [ "factorio" diff --git a/packages/host/src/Instance.ts b/packages/host/src/Instance.ts index 98d4d0eb..f6163611 100644 --- a/packages/host/src/Instance.ts +++ b/packages/host/src/Instance.ts @@ -800,7 +800,7 @@ rcon.print(game.table_to_json(players))`.replace(/\r?\n/g, " "); // Find stand alone modules to load // XXX for now only the included clusterio module is loaded - let modulesDirectory = path.join(__dirname, "..", "..", "modules"); + let modulesDirectory = path.join(__dirname, "..", "..", "..", "modules"); for (let entry of await fs.readdir(modulesDirectory, { withFileTypes: true })) { if (entry.isDirectory()) { if (modules.has(entry.name)) { diff --git a/packages/host/src/export.ts b/packages/host/src/export.ts index dcd2b01a..938836d4 100644 --- a/packages/host/src/export.ts +++ b/packages/host/src/export.ts @@ -5,7 +5,7 @@ import Jimp from "jimp"; import JSZip from "jszip"; import * as lib from "@clusterio/lib"; -import * as libBuildMod from "@clusterio/lib/dist/build_mod"; +import * as libBuildMod from "@clusterio/lib/dist/node/build_mod"; import type { FactorioServer } from "./server"; interface Prototype { @@ -62,7 +62,7 @@ async function generateExportMod(server: FactorioServer) { clean: false, build: true, pack: true, - sourceDir: path.join(__dirname, "..", "..", "lua", "export"), + sourceDir: path.join(__dirname, "..", "..", "..", "lua", "export"), outputDir: server.writePath("mods"), bumpPatch: false, factorioVersion: server.version.replace(/\.\d+$/, ""), diff --git a/packages/host/tsconfig.json b/packages/host/tsconfig.json index bc67208a..13df203e 100644 --- a/packages/host/tsconfig.json +++ b/packages/host/tsconfig.json @@ -5,6 +5,6 @@ ], "include": ["index.ts", "host.ts", "package.json", "src/**/*.ts", "src/**/*.js"], "compilerOptions": { - "outDir": "dist", + "outDir": "dist/node", }, } diff --git a/packages/lib/package.json b/packages/lib/package.json index 538f5d75..1c5cd2d6 100644 --- a/packages/lib/package.json +++ b/packages/lib/package.json @@ -8,7 +8,7 @@ "prepare": "tsc --build && pnpm run compile_validator", "compile_validator": "node scripts/compile_validator.js" }, - "main": "dist/index.js", + "main": "dist/node/index.js", "browser": "browser.ts", "keywords": [ "factorio" diff --git a/packages/lib/scripts/compile_validator.js b/packages/lib/scripts/compile_validator.js index e2eafeb3..9dd5fabd 100644 --- a/packages/lib/scripts/compile_validator.js +++ b/packages/lib/scripts/compile_validator.js @@ -7,9 +7,9 @@ const standaloneCode = require("ajv/dist/standalone").default; /* eslint-disable node/no-sync */ // Provide an empty file for message_core to load -fs.outputFileSync(path.join(__dirname, "../dist/src/data/message_validate.js"), ""); +fs.outputFileSync(path.join(__dirname, "../dist/node/src/data/message_validate.js"), ""); -const { Message } = require("../dist/src/data/messages_core"); +const { Message } = require("../dist/node/src/data/messages_core"); const ajv = new Ajv({ allowUnionTypes: true, @@ -28,5 +28,5 @@ function write(output) { `// generated by scripts/compile_validator.js\n// @ts-nocheck\n/* eslint-disable */\n${moduleCode}` ); } -write("../dist/src/data/message_validate.js"); +write("../dist/node/src/data/message_validate.js"); write("../src/data/message_validate.js"); diff --git a/packages/lib/tsconfig.json b/packages/lib/tsconfig.json index dbc005eb..d3dab4e1 100644 --- a/packages/lib/tsconfig.json +++ b/packages/lib/tsconfig.json @@ -2,7 +2,7 @@ "extends": "../../tsconfig.node.json", "include": ["index.ts", "browser.ts", "build_mod.js", "src/**/*.ts", "src/**/*.js"], "compilerOptions": { - "outDir": "dist", + "outDir": "dist/node", "allowJs": true, }, } diff --git a/plugins/global_chat/index.ts b/plugins/global_chat/index.ts index 620a8ecc..97c0985a 100644 --- a/plugins/global_chat/index.ts +++ b/plugins/global_chat/index.ts @@ -6,8 +6,8 @@ export const plugin: lib.PluginDeclaration = { name: "global_chat", title: "Global Chat", description: "Forwards chat between instances.", - instanceEntrypoint: "dist/plugin/instance", - ctlEntrypoint: "dist/plugin/control", + instanceEntrypoint: "dist/node/instance", + ctlEntrypoint: "dist/node/control", messages: [ ChatEvent, diff --git a/plugins/global_chat/package.json b/plugins/global_chat/package.json index 09d9adb6..26518d2c 100644 --- a/plugins/global_chat/package.json +++ b/plugins/global_chat/package.json @@ -6,7 +6,7 @@ "homepage": "https://github.com/clusterio/clusterio#readme", "license": "MIT", "repository": "clusterio/clusterio", - "main": "dist/plugin/index.js", + "main": "dist/node/index.js", "scripts": { "prepare": "tsc --build && webpack-cli --env production", "test": "echo \"Error: run tests from root\" && exit 1" diff --git a/plugins/global_chat/test/plugin.js b/plugins/global_chat/test/plugin.js index d8b9b860..315df7eb 100644 --- a/plugins/global_chat/test/plugin.js +++ b/plugins/global_chat/test/plugin.js @@ -4,9 +4,9 @@ const lib = require("@clusterio/lib"); const mock = require("../../../test/mock"); const lines = require("../../../test/lib/factorio/lines"); -const instance = require("../dist/plugin/instance"); -const info = require("../dist/plugin/index").plugin; -const { ChatEvent } = require("../dist/plugin/messages"); +const instance = require("../dist/node/instance"); +const info = require("../dist/node/index").plugin; +const { ChatEvent } = require("../dist/node/messages"); describe("global_chat plugin", function() { diff --git a/plugins/global_chat/tsconfig.json b/plugins/global_chat/tsconfig.json index 087576d6..15815646 100644 --- a/plugins/global_chat/tsconfig.json +++ b/plugins/global_chat/tsconfig.json @@ -6,6 +6,6 @@ "include": ["./**/*.ts"], "exclude": ["test/*", "./dist/*"], "compilerOptions": { - "outDir": "dist/plugin", + "outDir": "dist/node", }, } diff --git a/plugins/inventory_sync/index.ts b/plugins/inventory_sync/index.ts index 029e3a46..a796ee97 100644 --- a/plugins/inventory_sync/index.ts +++ b/plugins/inventory_sync/index.ts @@ -22,7 +22,7 @@ export const plugin: lib.PluginDeclaration = { title: "Inventory sync", description: "Synchronizes players inventories between instances", - instanceEntrypoint: "dist/plugin/instance", + instanceEntrypoint: "dist/node/instance", instanceConfigFields: { "inventory_sync.rcon_chunk_size": { title: "Rcon inventory chunk size", @@ -33,7 +33,7 @@ export const plugin: lib.PluginDeclaration = { }, }, - controllerEntrypoint: "dist/plugin/controller", + controllerEntrypoint: "dist/node/controller", controllerConfigFields: { "inventory_sync.player_lock_timeout": { title: "Player Lock Timeout", diff --git a/plugins/inventory_sync/package.json b/plugins/inventory_sync/package.json index 384a64dd..2bf25a2a 100644 --- a/plugins/inventory_sync/package.json +++ b/plugins/inventory_sync/package.json @@ -6,7 +6,7 @@ "homepage": "https://github.com/clusterio/factorioClusterio#readme", "license": "MIT", "repository": "clusterio/factorioClusterio", - "main": "dist/plugin/index.js", + "main": "dist/node/index.js", "scripts": { "prepare": "tsc --build && webpack-cli --env production", "test": "echo \"Error: run tests from root\" && exit 1" diff --git a/plugins/inventory_sync/tsconfig.json b/plugins/inventory_sync/tsconfig.json index 087576d6..15815646 100644 --- a/plugins/inventory_sync/tsconfig.json +++ b/plugins/inventory_sync/tsconfig.json @@ -6,6 +6,6 @@ "include": ["./**/*.ts"], "exclude": ["test/*", "./dist/*"], "compilerOptions": { - "outDir": "dist/plugin", + "outDir": "dist/node", }, } diff --git a/plugins/player_auth/index.ts b/plugins/player_auth/index.ts index a181c508..62e7b109 100644 --- a/plugins/player_auth/index.ts +++ b/plugins/player_auth/index.ts @@ -15,8 +15,8 @@ export const plugin: lib.PluginDeclaration = { name: "player_auth", title: "Player Auth", description: "Provides authentication to the cluster via logging into a Factorio server.", - controllerEntrypoint: "dist/plugin/controller", - instanceEntrypoint: "dist/plugin/instance", + controllerEntrypoint: "dist/node/controller", + instanceEntrypoint: "dist/node/instance", webEntrypoint: "./web", controllerConfigFields: { "player_auth.code_length": { diff --git a/plugins/player_auth/package.json b/plugins/player_auth/package.json index 8cf5b5c9..733ebf7a 100644 --- a/plugins/player_auth/package.json +++ b/plugins/player_auth/package.json @@ -6,7 +6,7 @@ "homepage": "https://github.com/clusterio/clusterio#readme", "license": "MIT", "repository": "clusterio/clusterio", - "main": "dist/plugin/index.js", + "main": "dist/node/index.js", "scripts": { "prepare": "tsc --build && webpack-cli --env production", "test": "echo \"Error: run tests from root\" && exit 1" diff --git a/plugins/player_auth/test/plugin.js b/plugins/player_auth/test/plugin.js index 8dc59142..6b90348e 100644 --- a/plugins/player_auth/test/plugin.js +++ b/plugins/player_auth/test/plugin.js @@ -5,10 +5,10 @@ const jwt = require("jsonwebtoken"); const mock = require("../../../test/mock"); -const controller = require("../dist/plugin/controller"); -const instance = require("../dist/plugin/instance"); -const info = require("../dist/plugin/index").plugin; -const { FetchPlayerCodeRequest, SetVerifyCodeRequest } = require("../dist/plugin/messages"); +const controller = require("../dist/node/controller"); +const instance = require("../dist/node/instance"); +const info = require("../dist/node/index").plugin; +const { FetchPlayerCodeRequest, SetVerifyCodeRequest } = require("../dist/node/messages"); const lib = require("@clusterio/lib"); diff --git a/plugins/player_auth/tsconfig.json b/plugins/player_auth/tsconfig.json index 087576d6..15815646 100644 --- a/plugins/player_auth/tsconfig.json +++ b/plugins/player_auth/tsconfig.json @@ -6,6 +6,6 @@ "include": ["./**/*.ts"], "exclude": ["test/*", "./dist/*"], "compilerOptions": { - "outDir": "dist/plugin", + "outDir": "dist/node", }, } diff --git a/plugins/research_sync/index.ts b/plugins/research_sync/index.ts index af7c8364..42f162d5 100644 --- a/plugins/research_sync/index.ts +++ b/plugins/research_sync/index.ts @@ -6,8 +6,8 @@ export const plugin: lib.PluginDeclaration = { name: "research_sync", title: "Research Sync", description: "Synchronises technology research progress between instances.", - instanceEntrypoint: "dist/plugin/instance", - controllerEntrypoint: "dist/plugin/controller", + instanceEntrypoint: "dist/node/instance", + controllerEntrypoint: "dist/node/controller", messages: [ messages.ContributionEvent, diff --git a/plugins/research_sync/package.json b/plugins/research_sync/package.json index fb9d4424..f5a74600 100644 --- a/plugins/research_sync/package.json +++ b/plugins/research_sync/package.json @@ -6,7 +6,7 @@ "homepage": "https://github.com/clusterio/clusterio#readme", "license": "MIT", "repository": "clusterio/clusterio", - "main": "dist/plugin/index.js", + "main": "dist/node/index.js", "scripts": { "prepare": "tsc --build && webpack-cli --env production", "test": "echo \"Error: run tests from root\" && exit 1" diff --git a/plugins/research_sync/test/missing.js b/plugins/research_sync/test/missing.js index 790baadb..ae16e8f1 100644 --- a/plugins/research_sync/test/missing.js +++ b/plugins/research_sync/test/missing.js @@ -2,5 +2,5 @@ // required here in order to track coverage for them. "use strict"; -require("../dist/plugin/instance"); -require("../dist/plugin/controller"); +require("../dist/node/instance"); +require("../dist/node/controller"); diff --git a/plugins/research_sync/tsconfig.json b/plugins/research_sync/tsconfig.json index 087576d6..15815646 100644 --- a/plugins/research_sync/tsconfig.json +++ b/plugins/research_sync/tsconfig.json @@ -6,6 +6,6 @@ "include": ["./**/*.ts"], "exclude": ["test/*", "./dist/*"], "compilerOptions": { - "outDir": "dist/plugin", + "outDir": "dist/node", }, } diff --git a/plugins/statistics_exporter/index.ts b/plugins/statistics_exporter/index.ts index 8748224c..5de5fc80 100644 --- a/plugins/statistics_exporter/index.ts +++ b/plugins/statistics_exporter/index.ts @@ -12,7 +12,7 @@ export const plugin: lib.PluginDeclaration = { description: "Provides in-game item/fluid production, builds, kills, and pollution "+ "statistics to the cluster's Prometheus endpoint.", - instanceEntrypoint: "dist/plugin/instance", + instanceEntrypoint: "dist/node/instance", instanceConfigFields: { "statistics_exporter.command_timeout": { title: "Command Timeout", diff --git a/plugins/statistics_exporter/package.json b/plugins/statistics_exporter/package.json index c8e5a12f..21c9c87f 100644 --- a/plugins/statistics_exporter/package.json +++ b/plugins/statistics_exporter/package.json @@ -6,7 +6,7 @@ "homepage": "https://github.com/clusterio/clusterio#readme", "license": "MIT", "repository": "clusterio/clusterio", - "main": "dist/plugin/index.js", + "main": "dist/node/index.js", "scripts": { "prepare": "tsc --build && webpack-cli --env production", "test": "echo \"Error: run tests from root\" && exit 1" diff --git a/plugins/statistics_exporter/test/plugin.js b/plugins/statistics_exporter/test/plugin.js index e54184a4..18251e27 100644 --- a/plugins/statistics_exporter/test/plugin.js +++ b/plugins/statistics_exporter/test/plugin.js @@ -2,8 +2,8 @@ const assert = require("assert").strict; const mock = require("../../../test/mock"); -const instance = require("../dist/plugin/instance"); -const info = require("../dist/plugin/index").plugin; +const instance = require("../dist/node/instance"); +const info = require("../dist/node/index").plugin; describe("statistics_exporter plugin", function() { diff --git a/plugins/statistics_exporter/tsconfig.json b/plugins/statistics_exporter/tsconfig.json index 087576d6..15815646 100644 --- a/plugins/statistics_exporter/tsconfig.json +++ b/plugins/statistics_exporter/tsconfig.json @@ -6,6 +6,6 @@ "include": ["./**/*.ts"], "exclude": ["test/*", "./dist/*"], "compilerOptions": { - "outDir": "dist/plugin", + "outDir": "dist/node", }, } diff --git a/plugins/subspace_storage/index.ts b/plugins/subspace_storage/index.ts index 6543cd6c..1909ef29 100644 --- a/plugins/subspace_storage/index.ts +++ b/plugins/subspace_storage/index.ts @@ -22,7 +22,7 @@ export const plugin: lib.PluginDeclaration = { name: "subspace_storage", title: "Subspace Storage", description: "Provides shared storage across instances for the Subspace Storage mod", - instanceEntrypoint: "dist/plugin/instance", + instanceEntrypoint: "dist/node/instance", instanceConfigFields: { "subspace_storage.log_item_transfers": { title: "Log Item Transfers", @@ -32,7 +32,7 @@ export const plugin: lib.PluginDeclaration = { }, }, - controllerEntrypoint: "dist/plugin/controller", + controllerEntrypoint: "dist/node/controller", controllerConfigFields: { "subspace_storage.division_method": { title: "Division Method", diff --git a/plugins/subspace_storage/package.json b/plugins/subspace_storage/package.json index 892e9ca9..5dce5e9b 100644 --- a/plugins/subspace_storage/package.json +++ b/plugins/subspace_storage/package.json @@ -6,7 +6,7 @@ "homepage": "https://github.com/clusterio/clusterio#readme", "license": "MIT", "repository": "clusterio/clusterio", - "main": "dist/plugin/index.js", + "main": "dist/node/index.js", "scripts": { "prepare": "tsc --build && webpack-cli --env production", "test": "echo \"Error: run tests from root\" && exit 1" diff --git a/plugins/subspace_storage/test/missing.js b/plugins/subspace_storage/test/missing.js index 6f7e6a1b..ab3009d7 100644 --- a/plugins/subspace_storage/test/missing.js +++ b/plugins/subspace_storage/test/missing.js @@ -2,7 +2,7 @@ // required here in order to track coverage for them. "use strict"; -require("../dist/plugin/instance"); -require("../dist/plugin/controller"); -require("../dist/plugin/dole"); -require("../dist/plugin/dole_nn_base"); +require("../dist/node/instance"); +require("../dist/node/controller"); +require("../dist/node/dole"); +require("../dist/node/dole_nn_base"); diff --git a/plugins/subspace_storage/test/routes.js b/plugins/subspace_storage/test/routes.js index 00de92b0..2753b41e 100644 --- a/plugins/subspace_storage/test/routes.js +++ b/plugins/subspace_storage/test/routes.js @@ -1,7 +1,7 @@ "use strict"; const assert = require("assert").strict; -const routes = require("../dist/plugin/routes"); +const routes = require("../dist/node/routes"); const { get } = require("../../../test/integration"); diff --git a/plugins/subspace_storage/tsconfig.json b/plugins/subspace_storage/tsconfig.json index 087576d6..15815646 100644 --- a/plugins/subspace_storage/tsconfig.json +++ b/plugins/subspace_storage/tsconfig.json @@ -6,6 +6,6 @@ "include": ["./**/*.ts"], "exclude": ["test/*", "./dist/*"], "compilerOptions": { - "outDir": "dist/plugin", + "outDir": "dist/node", }, } diff --git a/test/controller/HttpCloser.js b/test/controller/HttpCloser.js index 6c48ba2a..d630600b 100644 --- a/test/controller/HttpCloser.js +++ b/test/controller/HttpCloser.js @@ -7,7 +7,7 @@ const util = require("util"); const phin = require("phin"); -const HttpCloser = require("@clusterio/controller/dist/src/HttpCloser").default; +const HttpCloser = require("@clusterio/controller/dist/node/src/HttpCloser").default; // Time to wait during async operations to ensure they happened in order. const tick = 20; diff --git a/test/controller/routes.js b/test/controller/routes.js index a598f276..e69449ca 100644 --- a/test/controller/routes.js +++ b/test/controller/routes.js @@ -7,7 +7,7 @@ const http = require("http"); const phin = require("phin"); const { wait } = require("@clusterio/lib"); -const routes = require("@clusterio/controller/dist/src/routes"); +const routes = require("@clusterio/controller/dist/node/src/routes"); const mock = require("../mock"); diff --git a/test/host.js b/test/host.js index 3aa8162e..75244b9e 100644 --- a/test/host.js +++ b/test/host.js @@ -3,8 +3,8 @@ const assert = require("assert").strict; const path = require("path"); const lib = require("@clusterio/lib"); -const Host = require("@clusterio/host/dist/src/Host").default; -const { _discoverInstances } = require("@clusterio/host/dist/src/Host"); +const Host = require("@clusterio/host/dist/node/src/Host").default; +const { _discoverInstances } = require("@clusterio/host/dist/node/src/Host"); describe("Host testing", function() { describe("discoverInstances()", function() { diff --git a/test/host/Instance.js b/test/host/Instance.js index b899a335..9df6a598 100644 --- a/test/host/Instance.js +++ b/test/host/Instance.js @@ -4,7 +4,7 @@ const path = require("path"); const lib = require("@clusterio/lib"); const { PlayerStats, wait } = lib; -const Instance = require("@clusterio/host/dist/src/Instance").default; +const Instance = require("@clusterio/host/dist/node/src/Instance").default; const { MockConnector, MockServer } = require("../mock"); const addr = lib.Address.fromShorthand; diff --git a/test/host/export.js b/test/host/export.js index f99b53ba..b0de6a0a 100644 --- a/test/host/export.js +++ b/test/host/export.js @@ -2,8 +2,8 @@ const assert = require("assert").strict; const path = require("path"); -const { FactorioServer } = require("@clusterio/host/dist/src/server"); -const { _exportLocale } = require("@clusterio/host/dist/src/export"); +const { FactorioServer } = require("@clusterio/host/dist/node/src/server"); +const { _exportLocale } = require("@clusterio/host/dist/node/src/export"); describe("host/src/export", function() { diff --git a/test/host/patch.js b/test/host/patch.js index 9998bccc..8655aabd 100644 --- a/test/host/patch.js +++ b/test/host/patch.js @@ -5,7 +5,7 @@ const JSZip = require("jszip"); const path = require("path"); const lib = require("@clusterio/lib"); -const patch = require("@clusterio/host/dist/src/patch"); +const patch = require("@clusterio/host/dist/node/src/patch"); describe("host/patch", function() { diff --git a/test/host/server.js b/test/host/server.js index 12ade2e0..ea92a06c 100644 --- a/test/host/server.js +++ b/test/host/server.js @@ -4,7 +4,7 @@ const events = require("events"); const fs = require("fs-extra"); const path = require("path"); -const hostServer = require("@clusterio/host/dist/src/server"); +const hostServer = require("@clusterio/host/dist/node/src/server"); const lib = require("@clusterio/lib"); const { wait } = lib; const { testLines } = require("../lib/factorio/lines"); diff --git a/test/integration/export.js b/test/integration/export.js index 3ab962cc..a0d8345a 100644 --- a/test/integration/export.js +++ b/test/integration/export.js @@ -2,8 +2,8 @@ const assert = require("assert").strict; const path = require("path"); -const { _exportLocale } = require("@clusterio/host/dist/src/export"); -const { FactorioServer } = require("@clusterio/host/dist/src/server"); +const { _exportLocale } = require("@clusterio/host/dist/node/src/export"); +const { FactorioServer } = require("@clusterio/host/dist/node/src/server"); describe("Integration of lib/factorio/export", function() { diff --git a/test/integration/index.js b/test/integration/index.js index 6932e283..22450cbd 100644 --- a/test/integration/index.js +++ b/test/integration/index.js @@ -12,11 +12,11 @@ const lib = require("@clusterio/lib"); const { LineSplitter, ConsoleTransport, logger } = lib; // Make sure permissions from plugins are loaded -require("../../plugins/global_chat/dist/plugin/index"); -require("../../plugins/player_auth/dist/plugin/index"); -require("../../plugins/research_sync/dist/plugin/index"); -require("../../plugins/statistics_exporter/dist/plugin/index"); -require("../../plugins/subspace_storage/dist/plugin/index"); +require("../../plugins/global_chat/dist/node/index"); +require("../../plugins/player_auth/dist/node/index"); +require("../../plugins/research_sync/dist/node/index"); +require("../../plugins/statistics_exporter/dist/node/index"); +require("../../plugins/subspace_storage/dist/node/index"); const subspaceStorageVersion = "1.99.8"; const subspaceStorageGitHubTag = "1.99.6"; // Tag subspaceStorageVersion was released under on GitHub diff --git a/test/integration/patch.js b/test/integration/patch.js index bfc1f224..3b2cc72d 100644 --- a/test/integration/patch.js +++ b/test/integration/patch.js @@ -4,7 +4,7 @@ const fs = require("fs-extra"); const JSZip = require("jszip"); const path = require("path"); -const { patch, SaveModule } = require("@clusterio/host/dist/src/patch"); +const { patch, SaveModule } = require("@clusterio/host/dist/node/src/patch"); const lib = require("@clusterio/lib"); const { slowTest } = require("./index"); diff --git a/test/integration/server.js b/test/integration/server.js index 0aeac758..9073bbb8 100644 --- a/test/integration/server.js +++ b/test/integration/server.js @@ -5,7 +5,7 @@ const fs = require("fs-extra"); const path = require("path"); const lib = require("@clusterio/lib"); -const { FactorioServer, _getVersion } = require("@clusterio/host/dist/src/server"); +const { FactorioServer, _getVersion } = require("@clusterio/host/dist/node/src/server"); const { logger } = lib; const { slowTest } = require("./index"); diff --git a/test/mock.js b/test/mock.js index 4d2a8564..1c1fb84b 100644 --- a/test/mock.js +++ b/test/mock.js @@ -5,8 +5,8 @@ const express = require("express"); const lib = require("@clusterio/lib"); -const UserManager = require("@clusterio/controller/dist/src/UserManager").default; -const ControllerUser = require("@clusterio/controller/dist/src/ControllerUser").default; +const UserManager = require("@clusterio/controller/dist/node/src/UserManager").default; +const ControllerUser = require("@clusterio/controller/dist/node/src/ControllerUser").default; const addr = lib.Address.fromShorthand; From e4f90b348036b5b8039930bb49952327617c254b Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Wed, 7 Feb 2024 19:17:11 +0100 Subject: [PATCH 57/84] Move tsconfig.json to tsconfig.node.json Distinguish the Node.js build in the same way the browser build is with the .node suffix to the tsconfig files it uses. This means tsconfig.node outputs to dist/node and tsconfig.browser outputs to dist/browser. --- .../{tsconfig.json => tsconfig.node.json} | 2 +- .../ctl/{tsconfig.json => tsconfig.node.json} | 2 +- .../{tsconfig.json => tsconfig.node.json} | 2 +- .../lib/{tsconfig.json => tsconfig.node.json} | 0 .../tsconfig.node.json} | 2 +- .../tsconfig.node.json} | 2 +- .../tsconfig.node.json} | 2 +- .../tsconfig.node.json} | 2 +- plugins/statistics_exporter/tsconfig.json | 11 ---------- .../statistics_exporter/tsconfig.node.json | 11 ++++++++++ plugins/subspace_storage/tsconfig.json | 11 ---------- plugins/subspace_storage/tsconfig.node.json | 11 ++++++++++ tsconfig.json | 20 +++++++++---------- 13 files changed, 39 insertions(+), 39 deletions(-) rename packages/controller/{tsconfig.json => tsconfig.node.json} (81%) rename packages/ctl/{tsconfig.json => tsconfig.node.json} (80%) rename packages/host/{tsconfig.json => tsconfig.node.json} (82%) rename packages/lib/{tsconfig.json => tsconfig.node.json} (100%) rename plugins/{inventory_sync/tsconfig.json => global_chat/tsconfig.node.json} (76%) rename plugins/{player_auth/tsconfig.json => inventory_sync/tsconfig.node.json} (76%) rename plugins/{research_sync/tsconfig.json => player_auth/tsconfig.node.json} (76%) rename plugins/{global_chat/tsconfig.json => research_sync/tsconfig.node.json} (76%) delete mode 100644 plugins/statistics_exporter/tsconfig.json create mode 100644 plugins/statistics_exporter/tsconfig.node.json delete mode 100644 plugins/subspace_storage/tsconfig.json create mode 100644 plugins/subspace_storage/tsconfig.node.json diff --git a/packages/controller/tsconfig.json b/packages/controller/tsconfig.node.json similarity index 81% rename from packages/controller/tsconfig.json rename to packages/controller/tsconfig.node.json index 8dcc9b8b..9dcc91db 100644 --- a/packages/controller/tsconfig.json +++ b/packages/controller/tsconfig.node.json @@ -1,7 +1,7 @@ { "extends": "../../tsconfig.node.json", "references": [ - { "path": "../lib" }, + { "path": "../lib/tsconfig.node.json" }, ], "include": ["index.ts", "controller.ts", "package.json", "src/**/*.ts"], "compilerOptions": { diff --git a/packages/ctl/tsconfig.json b/packages/ctl/tsconfig.node.json similarity index 80% rename from packages/ctl/tsconfig.json rename to packages/ctl/tsconfig.node.json index 134019a5..3a0fb105 100644 --- a/packages/ctl/tsconfig.json +++ b/packages/ctl/tsconfig.node.json @@ -1,7 +1,7 @@ { "extends": "../../tsconfig.node.json", "references": [ - { "path": "../lib" }, + { "path": "../lib/tsconfig.node.json" }, ], "include": ["index.ts", "package.json", "ctl.ts", "src/**/*.ts"], "compilerOptions": { diff --git a/packages/host/tsconfig.json b/packages/host/tsconfig.node.json similarity index 82% rename from packages/host/tsconfig.json rename to packages/host/tsconfig.node.json index 13df203e..e7a488be 100644 --- a/packages/host/tsconfig.json +++ b/packages/host/tsconfig.node.json @@ -1,7 +1,7 @@ { "extends": "../../tsconfig.node.json", "references": [ - { "path": "../lib" }, + { "path": "../lib/tsconfig.node.json" }, ], "include": ["index.ts", "host.ts", "package.json", "src/**/*.ts", "src/**/*.js"], "compilerOptions": { diff --git a/packages/lib/tsconfig.json b/packages/lib/tsconfig.node.json similarity index 100% rename from packages/lib/tsconfig.json rename to packages/lib/tsconfig.node.json diff --git a/plugins/inventory_sync/tsconfig.json b/plugins/global_chat/tsconfig.node.json similarity index 76% rename from plugins/inventory_sync/tsconfig.json rename to plugins/global_chat/tsconfig.node.json index 15815646..b6197500 100644 --- a/plugins/inventory_sync/tsconfig.json +++ b/plugins/global_chat/tsconfig.node.json @@ -1,7 +1,7 @@ { "extends": "../../tsconfig.node.json", "references": [ - { "path": "../../packages/lib" }, + { "path": "../../packages/lib/tsconfig.node.json" }, ], "include": ["./**/*.ts"], "exclude": ["test/*", "./dist/*"], diff --git a/plugins/player_auth/tsconfig.json b/plugins/inventory_sync/tsconfig.node.json similarity index 76% rename from plugins/player_auth/tsconfig.json rename to plugins/inventory_sync/tsconfig.node.json index 15815646..b6197500 100644 --- a/plugins/player_auth/tsconfig.json +++ b/plugins/inventory_sync/tsconfig.node.json @@ -1,7 +1,7 @@ { "extends": "../../tsconfig.node.json", "references": [ - { "path": "../../packages/lib" }, + { "path": "../../packages/lib/tsconfig.node.json" }, ], "include": ["./**/*.ts"], "exclude": ["test/*", "./dist/*"], diff --git a/plugins/research_sync/tsconfig.json b/plugins/player_auth/tsconfig.node.json similarity index 76% rename from plugins/research_sync/tsconfig.json rename to plugins/player_auth/tsconfig.node.json index 15815646..b6197500 100644 --- a/plugins/research_sync/tsconfig.json +++ b/plugins/player_auth/tsconfig.node.json @@ -1,7 +1,7 @@ { "extends": "../../tsconfig.node.json", "references": [ - { "path": "../../packages/lib" }, + { "path": "../../packages/lib/tsconfig.node.json" }, ], "include": ["./**/*.ts"], "exclude": ["test/*", "./dist/*"], diff --git a/plugins/global_chat/tsconfig.json b/plugins/research_sync/tsconfig.node.json similarity index 76% rename from plugins/global_chat/tsconfig.json rename to plugins/research_sync/tsconfig.node.json index 15815646..b6197500 100644 --- a/plugins/global_chat/tsconfig.json +++ b/plugins/research_sync/tsconfig.node.json @@ -1,7 +1,7 @@ { "extends": "../../tsconfig.node.json", "references": [ - { "path": "../../packages/lib" }, + { "path": "../../packages/lib/tsconfig.node.json" }, ], "include": ["./**/*.ts"], "exclude": ["test/*", "./dist/*"], diff --git a/plugins/statistics_exporter/tsconfig.json b/plugins/statistics_exporter/tsconfig.json deleted file mode 100644 index 15815646..00000000 --- a/plugins/statistics_exporter/tsconfig.json +++ /dev/null @@ -1,11 +0,0 @@ -{ - "extends": "../../tsconfig.node.json", - "references": [ - { "path": "../../packages/lib" }, - ], - "include": ["./**/*.ts"], - "exclude": ["test/*", "./dist/*"], - "compilerOptions": { - "outDir": "dist/node", - }, -} diff --git a/plugins/statistics_exporter/tsconfig.node.json b/plugins/statistics_exporter/tsconfig.node.json new file mode 100644 index 00000000..b6197500 --- /dev/null +++ b/plugins/statistics_exporter/tsconfig.node.json @@ -0,0 +1,11 @@ +{ + "extends": "../../tsconfig.node.json", + "references": [ + { "path": "../../packages/lib/tsconfig.node.json" }, + ], + "include": ["./**/*.ts"], + "exclude": ["test/*", "./dist/*"], + "compilerOptions": { + "outDir": "dist/node", + }, +} diff --git a/plugins/subspace_storage/tsconfig.json b/plugins/subspace_storage/tsconfig.json deleted file mode 100644 index 15815646..00000000 --- a/plugins/subspace_storage/tsconfig.json +++ /dev/null @@ -1,11 +0,0 @@ -{ - "extends": "../../tsconfig.node.json", - "references": [ - { "path": "../../packages/lib" }, - ], - "include": ["./**/*.ts"], - "exclude": ["test/*", "./dist/*"], - "compilerOptions": { - "outDir": "dist/node", - }, -} diff --git a/plugins/subspace_storage/tsconfig.node.json b/plugins/subspace_storage/tsconfig.node.json new file mode 100644 index 00000000..b6197500 --- /dev/null +++ b/plugins/subspace_storage/tsconfig.node.json @@ -0,0 +1,11 @@ +{ + "extends": "../../tsconfig.node.json", + "references": [ + { "path": "../../packages/lib/tsconfig.node.json" }, + ], + "include": ["./**/*.ts"], + "exclude": ["test/*", "./dist/*"], + "compilerOptions": { + "outDir": "dist/node", + }, +} diff --git a/tsconfig.json b/tsconfig.json index fbb6a7ec..e108f361 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -1,16 +1,16 @@ { "files": [], "references": [ - { "path": "./packages/lib" }, - { "path": "./packages/controller" }, - { "path": "./packages/host" }, - { "path": "./packages/ctl" }, - { "path": "./plugins/global_chat" }, - { "path": "./plugins/inventory_sync" }, - { "path": "./plugins/player_auth" }, - { "path": "./plugins/research_sync" }, - { "path": "./plugins/statistics_exporter" }, - { "path": "./plugins/subspace_storage" }, + { "path": "./packages/lib/tsconfig.node.json" }, + { "path": "./packages/controller/tsconfig.node.json" }, + { "path": "./packages/host/tsconfig.node.json" }, + { "path": "./packages/ctl/tsconfig.node.json" }, + { "path": "./plugins/global_chat/tsconfig.node.json" }, + { "path": "./plugins/inventory_sync/tsconfig.node.json" }, + { "path": "./plugins/player_auth/tsconfig.node.json" }, + { "path": "./plugins/research_sync/tsconfig.node.json" }, + { "path": "./plugins/statistics_exporter/tsconfig.node.json" }, + { "path": "./plugins/subspace_storage/tsconfig.node.json" }, { "path": "./packages/lib/tsconfig.browser.json" }, { "path": "./packages/controller/tsconfig.browser.json" }, { "path": "./packages/web_ui/tsconfig.browser.json" }, From 597d080d71577f9e3ff85e4fd0236afcbf1eff38 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Wed, 7 Feb 2024 19:29:49 +0100 Subject: [PATCH 58/84] Add tsconfig.json that references both builds Add back unqualified tsconfig.json files to each package that references both the Node.js build and the browser build. This makes tsc --build compile both builds when the variant to do is not specified. --- packages/controller/tsconfig.json | 7 ++++++ packages/ctl/tsconfig.json | 6 +++++ packages/host/tsconfig.json | 6 +++++ packages/lib/tsconfig.json | 7 ++++++ packages/web_ui/tsconfig.json | 6 +++++ plugins/global_chat/tsconfig.json | 7 ++++++ plugins/inventory_sync/tsconfig.json | 7 ++++++ plugins/player_auth/tsconfig.json | 7 ++++++ plugins/research_sync/tsconfig.json | 7 ++++++ plugins/statistics_exporter/tsconfig.json | 7 ++++++ plugins/subspace_storage/tsconfig.json | 7 ++++++ tsconfig.json | 30 +++++++++-------------- 12 files changed, 85 insertions(+), 19 deletions(-) create mode 100644 packages/controller/tsconfig.json create mode 100644 packages/ctl/tsconfig.json create mode 100644 packages/host/tsconfig.json create mode 100644 packages/lib/tsconfig.json create mode 100644 packages/web_ui/tsconfig.json create mode 100644 plugins/global_chat/tsconfig.json create mode 100644 plugins/inventory_sync/tsconfig.json create mode 100644 plugins/player_auth/tsconfig.json create mode 100644 plugins/research_sync/tsconfig.json create mode 100644 plugins/statistics_exporter/tsconfig.json create mode 100644 plugins/subspace_storage/tsconfig.json diff --git a/packages/controller/tsconfig.json b/packages/controller/tsconfig.json new file mode 100644 index 00000000..e074e85f --- /dev/null +++ b/packages/controller/tsconfig.json @@ -0,0 +1,7 @@ +{ + "files": [], + "references": [ + { "path": "./tsconfig.node.json" }, + { "path": "./tsconfig.browser.json" }, + ], +} diff --git a/packages/ctl/tsconfig.json b/packages/ctl/tsconfig.json new file mode 100644 index 00000000..20b10c53 --- /dev/null +++ b/packages/ctl/tsconfig.json @@ -0,0 +1,6 @@ +{ + "files": [], + "references": [ + { "path": "./tsconfig.node.json" }, + ], +} diff --git a/packages/host/tsconfig.json b/packages/host/tsconfig.json new file mode 100644 index 00000000..20b10c53 --- /dev/null +++ b/packages/host/tsconfig.json @@ -0,0 +1,6 @@ +{ + "files": [], + "references": [ + { "path": "./tsconfig.node.json" }, + ], +} diff --git a/packages/lib/tsconfig.json b/packages/lib/tsconfig.json new file mode 100644 index 00000000..e074e85f --- /dev/null +++ b/packages/lib/tsconfig.json @@ -0,0 +1,7 @@ +{ + "files": [], + "references": [ + { "path": "./tsconfig.node.json" }, + { "path": "./tsconfig.browser.json" }, + ], +} diff --git a/packages/web_ui/tsconfig.json b/packages/web_ui/tsconfig.json new file mode 100644 index 00000000..d96583bf --- /dev/null +++ b/packages/web_ui/tsconfig.json @@ -0,0 +1,6 @@ +{ + "files": [], + "references": [ + { "path": "./tsconfig.browser.json" }, + ], +} diff --git a/plugins/global_chat/tsconfig.json b/plugins/global_chat/tsconfig.json new file mode 100644 index 00000000..e074e85f --- /dev/null +++ b/plugins/global_chat/tsconfig.json @@ -0,0 +1,7 @@ +{ + "files": [], + "references": [ + { "path": "./tsconfig.node.json" }, + { "path": "./tsconfig.browser.json" }, + ], +} diff --git a/plugins/inventory_sync/tsconfig.json b/plugins/inventory_sync/tsconfig.json new file mode 100644 index 00000000..e074e85f --- /dev/null +++ b/plugins/inventory_sync/tsconfig.json @@ -0,0 +1,7 @@ +{ + "files": [], + "references": [ + { "path": "./tsconfig.node.json" }, + { "path": "./tsconfig.browser.json" }, + ], +} diff --git a/plugins/player_auth/tsconfig.json b/plugins/player_auth/tsconfig.json new file mode 100644 index 00000000..e074e85f --- /dev/null +++ b/plugins/player_auth/tsconfig.json @@ -0,0 +1,7 @@ +{ + "files": [], + "references": [ + { "path": "./tsconfig.node.json" }, + { "path": "./tsconfig.browser.json" }, + ], +} diff --git a/plugins/research_sync/tsconfig.json b/plugins/research_sync/tsconfig.json new file mode 100644 index 00000000..e074e85f --- /dev/null +++ b/plugins/research_sync/tsconfig.json @@ -0,0 +1,7 @@ +{ + "files": [], + "references": [ + { "path": "./tsconfig.node.json" }, + { "path": "./tsconfig.browser.json" }, + ], +} diff --git a/plugins/statistics_exporter/tsconfig.json b/plugins/statistics_exporter/tsconfig.json new file mode 100644 index 00000000..e074e85f --- /dev/null +++ b/plugins/statistics_exporter/tsconfig.json @@ -0,0 +1,7 @@ +{ + "files": [], + "references": [ + { "path": "./tsconfig.node.json" }, + { "path": "./tsconfig.browser.json" }, + ], +} diff --git a/plugins/subspace_storage/tsconfig.json b/plugins/subspace_storage/tsconfig.json new file mode 100644 index 00000000..e074e85f --- /dev/null +++ b/plugins/subspace_storage/tsconfig.json @@ -0,0 +1,7 @@ +{ + "files": [], + "references": [ + { "path": "./tsconfig.node.json" }, + { "path": "./tsconfig.browser.json" }, + ], +} diff --git a/tsconfig.json b/tsconfig.json index e108f361..df2b7168 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -1,24 +1,16 @@ { "files": [], "references": [ - { "path": "./packages/lib/tsconfig.node.json" }, - { "path": "./packages/controller/tsconfig.node.json" }, - { "path": "./packages/host/tsconfig.node.json" }, - { "path": "./packages/ctl/tsconfig.node.json" }, - { "path": "./plugins/global_chat/tsconfig.node.json" }, - { "path": "./plugins/inventory_sync/tsconfig.node.json" }, - { "path": "./plugins/player_auth/tsconfig.node.json" }, - { "path": "./plugins/research_sync/tsconfig.node.json" }, - { "path": "./plugins/statistics_exporter/tsconfig.node.json" }, - { "path": "./plugins/subspace_storage/tsconfig.node.json" }, - { "path": "./packages/lib/tsconfig.browser.json" }, - { "path": "./packages/controller/tsconfig.browser.json" }, - { "path": "./packages/web_ui/tsconfig.browser.json" }, - { "path": "./plugins/global_chat/tsconfig.browser.json" }, - { "path": "./plugins/inventory_sync/tsconfig.browser.json" }, - { "path": "./plugins/player_auth/tsconfig.browser.json" }, - { "path": "./plugins/research_sync/tsconfig.browser.json" }, - { "path": "./plugins/statistics_exporter/tsconfig.browser.json" }, - { "path": "./plugins/subspace_storage/tsconfig.browser.json" }, + { "path": "./packages/lib/" }, + { "path": "./packages/controller/" }, + { "path": "./packages/host/" }, + { "path": "./packages/ctl/" }, + { "path": "./packages/web_ui/" }, + { "path": "./plugins/global_chat/" }, + { "path": "./plugins/inventory_sync/" }, + { "path": "./plugins/player_auth/" }, + { "path": "./plugins/research_sync/" }, + { "path": "./plugins/statistics_exporter/" }, + { "path": "./plugins/subspace_storage/" }, ], } From ebb93ada9453b08ff6f836cbf129bb8b41c1bdb7 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Thu, 8 Feb 2024 03:02:35 +0100 Subject: [PATCH 59/84] Replace babel and ts-loader with swc Build and minify the Web UI using swc instead of babel, tsc and terser. This results in a significant speedup of the build process as it skips checking types and uses native code to do the minification. To prevent accidental type errors being introduced the TypeScript compiler is still used to perform type checking on the Web UI files. Due to peer dependency resolving shenanigans the swc/core package had to be added to the workspace, controller and web_ui to avoid pnpm creating a duplicate install of webpack that break the build. --- .npmrc | 1 + package.json | 1 + packages/controller/package.json | 1 + packages/web_ui/package.json | 6 ++--- packages/web_ui/webpack.common.js | 45 ++++++++++++++++++++----------- tsconfig.browser.json | 2 +- 6 files changed, 36 insertions(+), 20 deletions(-) create mode 100644 .npmrc diff --git a/.npmrc b/.npmrc new file mode 100644 index 00000000..f301fedf --- /dev/null +++ b/.npmrc @@ -0,0 +1 @@ +auto-install-peers=false diff --git a/package.json b/package.json index 336edd58..fbf50444 100644 --- a/package.json +++ b/package.json @@ -27,6 +27,7 @@ "@clusterio/ctl": "workspace:*", "@clusterio/host": "workspace:*", "@clusterio/lib": "workspace:*", + "@swc/core": "^1.4.0", "@typescript-eslint/eslint-plugin": "^6.4.1", "@typescript-eslint/parser": "^6.4.1", "eslint": "^8.45.0", diff --git a/packages/controller/package.json b/packages/controller/package.json index 7446edbd..97fd5f36 100644 --- a/packages/controller/package.json +++ b/packages/controller/package.json @@ -36,6 +36,7 @@ }, "devDependencies": { "@clusterio/web_ui": "^2.0.0-alpha.16", + "@swc/core": "^1.4.0", "@types/busboy": "^1.5.0", "@types/compression": "^1.7.2", "@types/express": "^4.17.17", diff --git a/packages/web_ui/package.json b/packages/web_ui/package.json index fb77d82d..a1f2575a 100644 --- a/packages/web_ui/package.json +++ b/packages/web_ui/package.json @@ -17,12 +17,10 @@ }, "dependencies": { "@ant-design/icons": "^5.1.4", - "@babel/core": "^7.22.9", - "@babel/preset-react": "^7.22.5", "@clusterio/lib": "^2.0.0-alpha.16", + "@swc/core": "^1.4.0", "antd": "^5.13.0", "assert": "^2.0.0", - "babel-loader": "^9.1.3", "browserify-zlib": "^0.2.0", "buffer": "^6.0.3", "clean-webpack-plugin": "^4.0.0", @@ -39,8 +37,8 @@ "react-router-dom": "^6.14.2", "stream-browserify": "^3.0.0", "style-loader": "^3.3.3", + "swc-loader": "^0.2.6", "terser-webpack-plugin": "^5.3.9", - "ts-loader": "^9.4.4", "typescript": "^5.1.6", "util": "^0.12.5", "webpack": "^5.88.2", diff --git a/packages/web_ui/webpack.common.js b/packages/web_ui/webpack.common.js index 99825c4b..73cae8e8 100644 --- a/packages/web_ui/webpack.common.js +++ b/packages/web_ui/webpack.common.js @@ -65,32 +65,47 @@ module.exports = (env = {}) => ({ }, ], }, - { - test: /\.js$/, - resolve: { - // XXX Required due to bug in babel-runtime - // see: https://github.com/babel/babel/issues/12058 - fullySpecified: false, - }, - }, { test: /\.tsx?$/, exclude: /node_modules/, use: { - loader: require.resolve("ts-loader"), + loader: require.resolve("swc-loader"), options: { - configFile: "tsconfig.browser.json", - projectReferences: true, + sourceMaps: true, + jsc: { + parser: { + syntax: "typescript", + tsx: true, + }, + target: "es2022", + transform: { + react: { + runtime: "automatic", + }, + }, + }, }, }, }, { - test: /\.jsx$/, + test: /\.jsx?$/, exclude: /node_modules/, use: { - loader: require.resolve("babel-loader"), + loader: require.resolve("swc-loader"), options: { - presets: [require.resolve("@babel/preset-react")], + sourceMaps: true, + jsc: { + parser: { + syntax: "ecmascript", + jsx: true, + }, + target: "es2022", + transform: { + react: { + runtime: "automatic", + }, + }, + }, }, }, }, @@ -123,10 +138,10 @@ module.exports = (env = {}) => ({ moduleIds: "deterministic", minimizer: [ new TerserPlugin({ + minify: TerserPlugin.swcMinify, terserOptions: { compress: { keep_classnames: true, - passes: 2, }, mangle: { keep_classnames: true, diff --git a/tsconfig.browser.json b/tsconfig.browser.json index 7edb5691..605c2c61 100644 --- a/tsconfig.browser.json +++ b/tsconfig.browser.json @@ -5,7 +5,7 @@ "module": "es2022", "moduleResolution": "bundler", "target": "es2022", - "jsx": "react", + "jsx": "react-jsx", "noEmitOnError": true, }, } From f70b4f74b160cdbd596ddf7bcac5fcb31bab23e8 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Thu, 8 Feb 2024 05:23:48 +0100 Subject: [PATCH 60/84] Add watch script to rebuild on changes --- docs/contributing.md | 3 +++ package.json | 1 + 2 files changed, 4 insertions(+) diff --git a/docs/contributing.md b/docs/contributing.md index 44582f4a..eb5b82d1 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -81,6 +81,9 @@ Note that to use `clusterioctl` you will have to create a control config first: Once you've set up the cluster you can use `node packages/controller run` to run the controller and `node packages/host run` to run the host which connects to the controller. +When editing TypeScript files runned on Node.js you need run `pnpm build` in order to compile the files to JavaScript before restarting the controller/host/ctl for changes take effect. +To have TypeScript watch for changes and rebuild immediadly you may run `pnpm watch` instead. + For web development on the controller there are also the following flags: `--dev` - Recompile the `web_ui` package on the fly providing live reloading of it. diff --git a/package.json b/package.json index fbf50444..a5f4a506 100644 --- a/package.json +++ b/package.json @@ -3,6 +3,7 @@ "private": true, "scripts": { "build": "tsc --build", + "watch": "tsc --build --watch", "test": "pnpm run build && mocha --check-leaks -R spec --recursive test \"plugins/*/test/**\"", "fast-test": "FAST_TEST=y pnpm test", "cover": "nyc pnpm test", From 96082e32b7e25c0393057c7b78a7b2c71b5d1a31 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Wed, 7 Feb 2024 16:18:33 +0100 Subject: [PATCH 61/84] Replace deprecated usage of Select bordered --- packages/web_ui/src/components/ModPackViewPage.tsx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/packages/web_ui/src/components/ModPackViewPage.tsx b/packages/web_ui/src/components/ModPackViewPage.tsx index 97f13d03..840155f2 100644 --- a/packages/web_ui/src/components/ModPackViewPage.tsx +++ b/packages/web_ui/src/components/ModPackViewPage.tsx @@ -202,7 +202,7 @@ function SearchModsTable(props: SearchModsTableProps) { align: "right", render: (_, result) => + @@ -97,7 +97,9 @@ export default function UsersPage() { title: "Roles", key: "roles", render: (_, user) => ( - [...user.roleIds].map(id => {(roles.get(id) || { name: id }).name}) + [...user.roleIds].map(id => e.stopPropagation()}> + {(roles.get(id) || { name: id }).name} + ) ), }, { From 89919f7a891aabcab682873d2bd708504cd40104 Mon Sep 17 00:00:00 2001 From: Danielv123 Date: Fri, 9 Feb 2024 06:49:17 +0300 Subject: [PATCH 68/84] Fix inconsistent indentation --- packages/web_ui/src/components/InstanceViewPage.tsx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/packages/web_ui/src/components/InstanceViewPage.tsx b/packages/web_ui/src/components/InstanceViewPage.tsx index 5597e206..c1c4e018 100644 --- a/packages/web_ui/src/components/InstanceViewPage.tsx +++ b/packages/web_ui/src/components/InstanceViewPage.tsx @@ -159,8 +159,8 @@ function InstanceButtons(props: { instance: lib.InstanceDetails }) { "core.instance.kill", "core.instance.delete", ) && - - } + + } ; } From 4aca5050b92d1ac89d5f65ded23b30007550368c Mon Sep 17 00:00:00 2001 From: Danielv123 Date: Tue, 6 Feb 2024 09:25:19 +0300 Subject: [PATCH 69/84] Allow sending instance management messages from controller plugins --- packages/lib/src/data/messages_instance.ts | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/packages/lib/src/data/messages_instance.ts b/packages/lib/src/data/messages_instance.ts index 2a37d086..d14894d4 100644 --- a/packages/lib/src/data/messages_instance.ts +++ b/packages/lib/src/data/messages_instance.ts @@ -385,7 +385,7 @@ export class InstanceSaveDetailsUpdatesEvent { export class InstanceCreateSaveRequest { declare ["constructor"]: typeof InstanceCreateSaveRequest; static type = "request" as const; - static src = "control" as const; + static src = ["control", "controller"] as const; static dst = "instance" as const; static permission = "core.instance.save.create" as const; @@ -596,7 +596,7 @@ export class InstancePushSaveRequest { export class InstanceLoadScenarioRequest { declare ["constructor"]: typeof InstanceLoadScenarioRequest; static type = "request" as const; - static src = "control" as const; + static src = ["control", "controller"] as const; static dst = "instance" as const; static permission = "core.instance.load_scenario" as const; @@ -622,7 +622,7 @@ export class InstanceLoadScenarioRequest { export class InstanceExportDataRequest { declare ["constructor"]: typeof InstanceExportDataRequest; static type = "request" as const; - static src = "control" as const; + static src = ["control", "controller"] as const; static dst = "instance" as const; static permission = "core.instance.export_data" as const; } @@ -630,7 +630,7 @@ export class InstanceExportDataRequest { export class InstanceExtractPlayersRequest { declare ["constructor"]: typeof InstanceExtractPlayersRequest; static type = "request" as const; - static src = "control" as const; + static src = ["control", "controller"] as const; static dst = "instance" as const; static permission = "core.instance.extract_players" as const; } @@ -638,7 +638,7 @@ export class InstanceExtractPlayersRequest { export class InstanceStopRequest { declare ["constructor"]: typeof InstanceStopRequest; static type = "request" as const; - static src = "control" as const; + static src = ["control", "controller"] as const; static dst = "instance" as const; static permission = "core.instance.stop" as const; } @@ -646,7 +646,7 @@ export class InstanceStopRequest { export class InstanceKillRequest { declare ["constructor"]: typeof InstanceKillRequest; static type = "request" as const; - static src = "control" as const; + static src = ["control", "controller"] as const; static dst = "instance" as const; static permission = "core.instance.kill" as const; } @@ -693,7 +693,7 @@ export class InstanceDeleteInternalRequest { export class InstanceSendRconRequest { declare ["constructor"]: typeof InstanceSendRconRequest; static type = "request" as const; - static src = "control" as const; + static src = ["control", "controller"] as const; static dst = "instance" as const; static permission = "core.instance.send_rcon" as const; From 8879f8dd80b5dff31b77ea9df78e35629933a866 Mon Sep 17 00:00:00 2001 From: Danielv123 Date: Fri, 9 Feb 2024 09:11:53 +0300 Subject: [PATCH 70/84] Replaced react Link with custom component The custom component merges the behaviour of antd Typography.Link and the react-router.dom Link after the custom component prop was removed in v6 according to this guide: https://reactrouter.com/en/main/upgrading/v5#remove-link-component-prop --- .../web_ui/src/components/InstanceList.tsx | 7 ++- .../src/components/InstanceViewPage.tsx | 12 +++-- packages/web_ui/src/components/Link.tsx | 47 +++++++++++++++++++ packages/web_ui/src/components/PageLayout.tsx | 2 +- packages/web_ui/src/components/UsersPage.tsx | 3 +- 5 files changed, 60 insertions(+), 11 deletions(-) create mode 100644 packages/web_ui/src/components/Link.tsx diff --git a/packages/web_ui/src/components/InstanceList.tsx b/packages/web_ui/src/components/InstanceList.tsx index d966ccad..c1aa76d0 100644 --- a/packages/web_ui/src/components/InstanceList.tsx +++ b/packages/web_ui/src/components/InstanceList.tsx @@ -1,5 +1,5 @@ import React from "react"; -import { Link, useNavigate } from "react-router-dom"; +import { useNavigate } from "react-router-dom"; import { message, Button, Space, Table, Typography } from "antd"; import CopyOutlined from "@ant-design/icons/CopyOutlined"; import type { SizeType } from "antd/es/config-provider/SizeContext"; @@ -10,6 +10,7 @@ import { useHosts } from "../model/host"; import InstanceStatusTag from "./InstanceStatusTag"; import StartStopInstanceButton from "./StartStopInstanceButton"; import { InstanceDetails } from "@clusterio/lib"; +import Link from "./Link"; const strcmp = new Intl.Collator(undefined, { numeric: true, sensitivity: "base" }).compare; @@ -59,9 +60,7 @@ export default function InstanceList(props: InstanceListProps) { to={`/hosts/${instance.assignedHost}/view`} onClick={e => e.stopPropagation()} > - - {hostName(instance.assignedHost)} - + {hostName(instance.assignedHost)} , sorter: (a, b) => strcmp(hostName(a.assignedHost), hostName(b.assignedHost)), responsive: ["sm"], diff --git a/packages/web_ui/src/components/InstanceViewPage.tsx b/packages/web_ui/src/components/InstanceViewPage.tsx index c1c4e018..5973cdfa 100644 --- a/packages/web_ui/src/components/InstanceViewPage.tsx +++ b/packages/web_ui/src/components/InstanceViewPage.tsx @@ -1,5 +1,5 @@ import React, { useContext, useState } from "react"; -import { Link, useNavigate, useParams } from "react-router-dom"; +import { useNavigate, useParams } from "react-router-dom"; import { Alert, Button, Descriptions, Dropdown, Menu, MenuProps, Modal, Space, Spin, Typography } from "antd"; import type { ItemType } from "antd/es/menu/hooks/useItems"; import DeleteOutlined from "@ant-design/icons/DeleteOutlined"; @@ -23,6 +23,7 @@ import { notifyErrorHandler } from "../util/notify"; import { useInstance } from "../model/instance"; import { useHost } from "../model/host"; import InstanceStatusTag from "./InstanceStatusTag"; +import Link from "./Link"; const { Title } = Typography; @@ -43,7 +44,7 @@ function InstanceDescription(props: InstanceDescriptionProps) { {!assigned ? Unassigned : - {host?.name ?? instance.assignedHost} + {host?.name ?? instance.assignedHost} } {account.hasPermission("core.instance.assign") && - - } + ) + && + + } ; } diff --git a/packages/web_ui/src/components/Link.tsx b/packages/web_ui/src/components/Link.tsx new file mode 100644 index 00000000..224e651d --- /dev/null +++ b/packages/web_ui/src/components/Link.tsx @@ -0,0 +1,47 @@ +import React from "react"; +import { Typography } from "antd"; +import { + LinkProps, + useHref, + useLinkClickHandler, +} from "react-router-dom"; +import { BaseType } from "antd/es/typography/Base"; + +const Link = React.forwardRef( + ( + { + onClick, + replace = false, + state, + target, + to, + type, + ...rest + }: LinkProps, + ref + ) => { + let href = useHref(to); + let handleClick = useLinkClickHandler(to, { + replace, + state, + target, + }) as (event: React.MouseEvent) => void; + + return ( + { + onClick?.(event as React.MouseEvent); + if (!event.defaultPrevented) { + handleClick(event); + } + }} + ref={ref as React.RefObject | null} + target={target} + type={type as BaseType} + /> + ); + } +); +export default Link; diff --git a/packages/web_ui/src/components/PageLayout.tsx b/packages/web_ui/src/components/PageLayout.tsx index 8a9ce1e7..8361827b 100644 --- a/packages/web_ui/src/components/PageLayout.tsx +++ b/packages/web_ui/src/components/PageLayout.tsx @@ -1,6 +1,6 @@ import React from "react"; -import { Link } from "react-router-dom"; import { Layout, Breadcrumb } from "antd"; +import Link from "./Link"; const { Content } = Layout; diff --git a/packages/web_ui/src/components/UsersPage.tsx b/packages/web_ui/src/components/UsersPage.tsx index c487adf5..83c605aa 100644 --- a/packages/web_ui/src/components/UsersPage.tsx +++ b/packages/web_ui/src/components/UsersPage.tsx @@ -1,5 +1,5 @@ import React, { useEffect, useContext, useState } from "react"; -import { Link, useNavigate } from "react-router-dom"; +import { useNavigate } from "react-router-dom"; import { Button, Form, Input, Modal, Space, Table, Tag } from "antd"; import * as lib from "@clusterio/lib"; @@ -12,6 +12,7 @@ import PageHeader from "./PageHeader"; import PageLayout from "./PageLayout"; import PluginExtra from "./PluginExtra"; import { formatFirstSeen, formatLastSeen, sortFirstSeen, sortLastSeen, useUsers } from "../model/user"; +import Link from "./Link"; const strcmp = new Intl.Collator(undefined, { numeric: true, sensitivity: "base" }).compare; From 3cddd2b9f828232f1ac6b6d9560949ef4f004425 Mon Sep 17 00:00:00 2001 From: Danielv123 Date: Sat, 10 Feb 2024 16:33:22 +0300 Subject: [PATCH 71/84] Export Link component for use by plugins --- packages/web_ui/src/index.ts | 2 ++ 1 file changed, 2 insertions(+) diff --git a/packages/web_ui/src/index.ts b/packages/web_ui/src/index.ts index 03e4e6cd..601e092c 100644 --- a/packages/web_ui/src/index.ts +++ b/packages/web_ui/src/index.ts @@ -28,3 +28,5 @@ export { default as PageHeader } from "./components/PageHeader"; export { default as PageLayout } from "./components/PageLayout"; export { default as SavesList } from "./components/SavesList"; export { default as SectionHeader } from "./components/SectionHeader"; + +export { default as Link } from "./components/Link"; From c9609259db644cb04b4c3cf29ca796361e8b0ef4 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sun, 11 Feb 2024 15:48:45 +0100 Subject: [PATCH 72/84] Ignore error loading mod in fetchMod If a bad download or file transfer causes a mod in a host's mod folder to become corrupt the attempt to load the mod will throw an error that's not the file not existing. This in turn causes starting the instance to always fail as the mod is never attempted to be re-downloaded. Fix by ignoring errors attempting to load mods already present in the mods folder that aren't in the mod store so that a new download is attempted after that fails. --- CHANGELOG.md | 1 + packages/host/src/Host.ts | 3 ++- 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ea1ac02f..485f1130 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -14,6 +14,7 @@ - Fixed data export with a mod using shorthand notation for the default value of a color-setting causing the ModPack data to be set to invalid data. - Fixed per user per instance player stats not being deleted when the instance it's for is deleted. - Fixed mods setting the item prototype icons property to an object mapping instead of an array breaking the data export. +- Fixed a bad mod in the host's mods folder causing instance start to fail instead of the host re-downloading the mod. ## Version 2.0.0-alpha.16 diff --git a/packages/host/src/Host.ts b/packages/host/src/Host.ts index c2fc5d67..75e80ffa 100644 --- a/packages/host/src/Host.ts +++ b/packages/host/src/Host.ts @@ -600,8 +600,9 @@ export default class Host extends lib.Link { try { modInfo = await this.modStore.loadMod(mod.name, mod.version); } catch (err: any) { + // A broken zip file may have been left in the mods folder if (err.code !== "ENOENT") { - throw err; + logger.error(`Error loading mod ${mod.name} ${mod.version}: ${err.message}`); } } if (modInfo && (!mod.sha1 || mod.sha1 === modInfo.sha1)) { From e990134b7045cf56947fe2e83dfdf36bb401954f Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Sun, 11 Feb 2024 21:40:32 +0100 Subject: [PATCH 73/84] Use suggested wording in docs/contributing.md Co-authored-by: Cooldude2606 <25043174+Cooldude2606@users.noreply.github.com> --- docs/contributing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/contributing.md b/docs/contributing.md index eb5b82d1..8f1d73fe 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -81,7 +81,7 @@ Note that to use `clusterioctl` you will have to create a control config first: Once you've set up the cluster you can use `node packages/controller run` to run the controller and `node packages/host run` to run the host which connects to the controller. -When editing TypeScript files runned on Node.js you need run `pnpm build` in order to compile the files to JavaScript before restarting the controller/host/ctl for changes take effect. +When editing TypeScript files ran by Node.js you need to run `pnpm build` to compile the files to JavaScript before restarting the controller/host/ctl for changes take effect. To have TypeScript watch for changes and rebuild immediadly you may run `pnpm watch` instead. For web development on the controller there are also the following flags: From abcbf3fddce688ae8463155c8c37c7b67402ffaa Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 12 Feb 2024 17:24:00 +0100 Subject: [PATCH 74/84] Add pagination to saves list (#592) Prevent the saves list from pushing the console and config off the screen by turning on the pagination for it. --- CHANGELOG.md | 4 ++++ packages/web_ui/src/components/SavesList.tsx | 2 +- packages/web_ui/src/style.css | 9 +++++++++ 3 files changed, 14 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 485f1130..a23be684 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,10 @@ ## Version 2.0.0-alpha.17 +### Features + +- Added pagination to the saves list. + ### Changes - Fixed an unhandled error in Subspace Storage crashing the host. diff --git a/packages/web_ui/src/components/SavesList.tsx b/packages/web_ui/src/components/SavesList.tsx index 2cbe4c8e..92ce287c 100644 --- a/packages/web_ui/src/components/SavesList.tsx +++ b/packages/web_ui/src/components/SavesList.tsx @@ -207,6 +207,7 @@ export default function SavesList(props: { instance: lib.InstanceDetails }) { let hostOffline = ["unassigned", "unknown"].includes(props.instance.status!); const saveTable =
save.name} - pagination={false} expandable={{ columnWidth: 33, expandRowByClick: true, diff --git a/packages/web_ui/src/style.css b/packages/web_ui/src/style.css index 7fa95675..3cd12ed2 100644 --- a/packages/web_ui/src/style.css +++ b/packages/web_ui/src/style.css @@ -182,10 +182,19 @@ body { white-space: pre-wrap; } +.save-list-dragger .ant-upload.ant-upload-drag { + border: none; + background: none; +} + .save-list-dragger .ant-upload.ant-upload-btn { padding: 0; } +.save-list-table .ant-table-pagination.ant-pagination { + margin-bottom: 0; +} + .rcon-result code { display: block; background: #000; From d8f1d8a3e2f00684d45e36066c9a497c609d6525 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 12 Feb 2024 17:25:10 +0100 Subject: [PATCH 75/84] Add known issue about Factorio migrations --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 0857fc07..1873cf33 100644 --- a/README.md +++ b/README.md @@ -270,6 +270,8 @@ If you want to use a different user for the command line interface, you can gene - When the `server_select` plugin is installed you may see `[error] Unhandled event server_select:UpdateInstancesEvent` logged when starting an instance. This should be ignored as it does not indicate an error occured. +- After updating Factorio you may get the error `Error: Expected empty response but got "Cannot execute command. Error: [string "clusterio_private.update_instance(..."]:1: attempt to index global 'clusterio_private' (a nil value)?` on instance start. + This is due to Factorio applying a migration replacing the patched content of the save with the original scenario, attempting to start the instance again should resolve the issue. - After migrating from Alpha 13 to Alpha 14 running an instance data export fails with the error `Upload failed: 401 Unauthorized` due to the host token being issued to a slave. To workaround this find the id of the host in the host config and generate a new host token with the same id. Stop the host and replace the value of the `host.controller_token` field in `config-host.json` with the new host token before starting it up again. From fac04adc6fd128068e3f6b6ec3f78c7657f5a1a0 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 12 Feb 2024 17:29:51 +0100 Subject: [PATCH 76/84] Add known issue about the base mod --- README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 1873cf33..c00b5ab5 100644 --- a/README.md +++ b/README.md @@ -272,6 +272,8 @@ If you want to use a different user for the command line interface, you can gene This should be ignored as it does not indicate an error occured. - After updating Factorio you may get the error `Error: Expected empty response but got "Cannot execute command. Error: [string "clusterio_private.update_instance(..."]:1: attempt to index global 'clusterio_private' (a nil value)?` on instance start. This is due to Factorio applying a migration replacing the patched content of the save with the original scenario, attempting to start the instance again should resolve the issue. +- In the mod pack view of the web interface the "base" mod is displayed with an error complaining about the mod being missing from the controller storage. + This is due to the "base" mod currently not being connectly handled as a built-in mod to Factorio in the web interface, this error message should be ignored. - After migrating from Alpha 13 to Alpha 14 running an instance data export fails with the error `Upload failed: 401 Unauthorized` due to the host token being issued to a slave. To workaround this find the id of the host in the host config and generate a new host token with the same id. Stop the host and replace the value of the `host.controller_token` field in `config-host.json` with the new host token before starting it up again. From 8ae5e11021b5b84f155d6a90b7cb353168e81c01 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 12 Feb 2024 17:33:13 +0100 Subject: [PATCH 77/84] Add changelog for #587 --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index a23be684..2c462e7c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ ### Features +- Made the host an instance is assigned to link to the host page for that host in the Web UI [#587](https://github.com/clusterio/clusterio/pull/587). - Added pagination to the saves list. ### Changes From 8a580a66be09a4c4cadaded74ace5b03aaa6946d Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 12 Feb 2024 17:37:27 +0100 Subject: [PATCH 78/84] Add breaking change note for 535e43b --- CHANGELOG.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2c462e7c..c7c8666a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,6 +21,10 @@ - Fixed mods setting the item prototype icons property to an object mapping instead of an array breaking the data export. - Fixed a bad mod in the host's mods folder causing instance start to fail instead of the host re-downloading the mod. +### Breaking Changes + +- Renamed the permission `core.instance.save.list.subscribe` to `core.instance.save.subscribe`. + ## Version 2.0.0-alpha.16 From a9c6430531e32f321ac068afd8189eb8b220d0ba Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 12 Feb 2024 17:40:25 +0100 Subject: [PATCH 79/84] Document current publishing workflow Docement the adaptations I've made to the publishing procedure which uses a less dangerous publish command and removes the dist folders before publishing to avoid accidental upload of stale build files. --- docs/contributing.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/contributing.md b/docs/contributing.md index 8f1d73fe..9a4ef6a9 100644 --- a/docs/contributing.md +++ b/docs/contributing.md @@ -319,6 +319,6 @@ You will need commit access to the repository as well publish access to the @clu 4. Publish the packages. ```sh - # Warning: Double check your current working directory is the repository before running this - pnpm -r --filter='!./external_plugins/**' publish + rm -r ./{plugins,packages}/*/dist + pnpm -r --filter='./{plugins,packages}/*' publish ``` From cf2b3db20de0caf334d142285c87d2036dc812e3 Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 12 Feb 2024 17:50:33 +0100 Subject: [PATCH 80/84] Add thank you note to contributors of Alpha 17 --- CHANGELOG.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index c7c8666a..e213fd48 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,6 +25,11 @@ - Renamed the permission `core.instance.save.list.subscribe` to `core.instance.save.subscribe`. +Many thanks to the following for contributing to this release: +[@Cooldude2606](https://github.com/Cooldude2606) +[@Danielv123](https://github.com/Danielv123) +[@Hornwitser](https://github.com/Hornwitser) + ## Version 2.0.0-alpha.16 From 5f85c0575e74e571c35ba91475dab5812cd55dde Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 12 Feb 2024 17:54:48 +0100 Subject: [PATCH 81/84] v2.0.0-alpha.17 --- packages/controller/package.json | 6 +++--- packages/create/package.json | 2 +- packages/ctl/package.json | 4 ++-- packages/host/package.json | 4 ++-- packages/lib/package.json | 2 +- packages/web_ui/package.json | 4 ++-- plugins/global_chat/package.json | 6 +++--- plugins/inventory_sync/package.json | 6 +++--- plugins/player_auth/package.json | 6 +++--- plugins/research_sync/package.json | 6 +++--- plugins/statistics_exporter/package.json | 6 +++--- plugins/subspace_storage/package.json | 6 +++--- 12 files changed, 29 insertions(+), 29 deletions(-) diff --git a/packages/controller/package.json b/packages/controller/package.json index 97fd5f36..588b39c0 100644 --- a/packages/controller/package.json +++ b/packages/controller/package.json @@ -3,7 +3,7 @@ "description": "Implementation of Clusterio controller", "author": "Hornwitser", "license": "MIT", - "version": "2.0.0-alpha.16", + "version": "2.0.0-alpha.17", "main": "dist/node/index.js", "scripts": { "prepare": "tsc --build && webpack-cli --env production", @@ -19,7 +19,7 @@ "node": ">=18" }, "dependencies": { - "@clusterio/lib": "^2.0.0-alpha.16", + "@clusterio/lib": "^2.0.0-alpha.17", "@sinclair/typebox": "^0.30.4", "busboy": "^1.6.0", "compression": "^1.7.4", @@ -35,7 +35,7 @@ "yargs": "^17.7.2" }, "devDependencies": { - "@clusterio/web_ui": "^2.0.0-alpha.16", + "@clusterio/web_ui": "^2.0.0-alpha.17", "@swc/core": "^1.4.0", "@types/busboy": "^1.5.0", "@types/compression": "^1.7.2", diff --git a/packages/create/package.json b/packages/create/package.json index 40aa21ba..7929cbb1 100644 --- a/packages/create/package.json +++ b/packages/create/package.json @@ -3,7 +3,7 @@ "description": "Installer for clusterio", "author": "Hornwitser", "license": "MIT", - "version": "2.0.0-alpha.16", + "version": "2.0.0-alpha.17", "main": "create.js", "bin": { "clusteriocreate": "./create.js" diff --git a/packages/ctl/package.json b/packages/ctl/package.json index 7d5f6707..879afe37 100644 --- a/packages/ctl/package.json +++ b/packages/ctl/package.json @@ -3,7 +3,7 @@ "description": "Clusterio command line management tool", "author": "Hornwitser", "license": "MIT", - "version": "2.0.0-alpha.16", + "version": "2.0.0-alpha.17", "main": "dist/node/index.js", "scripts": { "prepare": "tsc --build", @@ -19,7 +19,7 @@ "node": ">=18" }, "dependencies": { - "@clusterio/lib": "^2.0.0-alpha.16", + "@clusterio/lib": "^2.0.0-alpha.17", "as-table": "^1.0.55", "fs-extra": "^11.1.1", "phin": "^3.7.0", diff --git a/packages/host/package.json b/packages/host/package.json index c9f8446a..b339d0a0 100644 --- a/packages/host/package.json +++ b/packages/host/package.json @@ -3,7 +3,7 @@ "description": "Implementation of Clusterio host server", "author": "Hornwitser", "license": "MIT", - "version": "2.0.0-alpha.16", + "version": "2.0.0-alpha.17", "main": "dist/node/index.js", "scripts": { "prepare": "tsc --build", @@ -19,7 +19,7 @@ "node": ">=18" }, "dependencies": { - "@clusterio/lib": "^2.0.0-alpha.16", + "@clusterio/lib": "^2.0.0-alpha.17", "@sinclair/typebox": "^0.30.4", "fs-extra": "^11.1.1", "jimp": "^0.22.8", diff --git a/packages/lib/package.json b/packages/lib/package.json index 1c5cd2d6..82afa633 100644 --- a/packages/lib/package.json +++ b/packages/lib/package.json @@ -1,7 +1,7 @@ { "name": "@clusterio/lib", "description": "Shared library for Clusterio", - "version": "2.0.0-alpha.16", + "version": "2.0.0-alpha.17", "repository": "https://github.com/clusterio/clusterio", "license": "MIT", "scripts": { diff --git a/packages/web_ui/package.json b/packages/web_ui/package.json index a1f2575a..0c7c4cc8 100644 --- a/packages/web_ui/package.json +++ b/packages/web_ui/package.json @@ -3,7 +3,7 @@ "description": "Clusterio web interface implementation", "author": "Hornwitser", "license": "MIT", - "version": "2.0.0-alpha.16", + "version": "2.0.0-alpha.17", "browser": "src/index.ts", "main": "dist/browser/src/index.js", "scripts": { @@ -17,7 +17,7 @@ }, "dependencies": { "@ant-design/icons": "^5.1.4", - "@clusterio/lib": "^2.0.0-alpha.16", + "@clusterio/lib": "^2.0.0-alpha.17", "@swc/core": "^1.4.0", "antd": "^5.13.0", "assert": "^2.0.0", diff --git a/plugins/global_chat/package.json b/plugins/global_chat/package.json index 26518d2c..8f82dc80 100644 --- a/plugins/global_chat/package.json +++ b/plugins/global_chat/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-global_chat", - "version": "2.0.0-alpha.16", + "version": "2.0.0-alpha.17", "description": "Clusterio plugin forwarding between Factorio servers", "author": "Hornwitser ", "homepage": "https://github.com/clusterio/clusterio#readme", @@ -21,8 +21,8 @@ "@clusterio/lib": "^2.0.0-alpha.0" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.16", - "@clusterio/web_ui": "^2.0.0-alpha.16", + "@clusterio/lib": "^2.0.0-alpha.17", + "@clusterio/web_ui": "^2.0.0-alpha.17", "@types/node": "^20.4.5", "@types/react": "^18.2.21", "typescript": "^5.1.6", diff --git a/plugins/inventory_sync/package.json b/plugins/inventory_sync/package.json index 2bf25a2a..566e4006 100644 --- a/plugins/inventory_sync/package.json +++ b/plugins/inventory_sync/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-inventory_sync", - "version": "2.0.0-alpha.16", + "version": "2.0.0-alpha.17", "description": "Clusterio plugin forwarding between Factorio servers", "author": "Danielv ", "homepage": "https://github.com/clusterio/factorioClusterio#readme", @@ -25,8 +25,8 @@ "fs-extra": "^8.1.0" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.16", - "@clusterio/web_ui": "^2.0.0-alpha.16", + "@clusterio/lib": "^2.0.0-alpha.17", + "@clusterio/web_ui": "^2.0.0-alpha.17", "@types/fs-extra": "^11.0.1", "@types/node": "^20.4.5", "@types/react": "^18.2.21", diff --git a/plugins/player_auth/package.json b/plugins/player_auth/package.json index 733ebf7a..b65f9186 100644 --- a/plugins/player_auth/package.json +++ b/plugins/player_auth/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-player_auth", - "version": "2.0.0-alpha.16", + "version": "2.0.0-alpha.17", "description": "Clusterio plugin authenticating logged in players to the web interface", "author": "Hornwitser ", "homepage": "https://github.com/clusterio/clusterio#readme", @@ -26,8 +26,8 @@ "jsonwebtoken": "^9.0.1" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.16", - "@clusterio/web_ui": "^2.0.0-alpha.16", + "@clusterio/lib": "^2.0.0-alpha.17", + "@clusterio/web_ui": "^2.0.0-alpha.17", "@types/express": "^4.17.17", "@types/jsonwebtoken": "^9.0.2", "@types/node": "^20.4.5", diff --git a/plugins/research_sync/package.json b/plugins/research_sync/package.json index f5a74600..ed6c298a 100644 --- a/plugins/research_sync/package.json +++ b/plugins/research_sync/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-research_sync", - "version": "2.0.0-alpha.16", + "version": "2.0.0-alpha.17", "description": "Clusterio plugin for synchronizing research between Factorio servers", "author": "Hornwitser ", "homepage": "https://github.com/clusterio/clusterio#readme", @@ -25,8 +25,8 @@ "fs-extra": "^8.1.0" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.16", - "@clusterio/web_ui": "^2.0.0-alpha.16", + "@clusterio/lib": "^2.0.0-alpha.17", + "@clusterio/web_ui": "^2.0.0-alpha.17", "@types/fs-extra": "^11.0.1", "@types/node": "^20.4.5", "@types/react": "^18.2.21", diff --git a/plugins/statistics_exporter/package.json b/plugins/statistics_exporter/package.json index 21c9c87f..c1bb9490 100644 --- a/plugins/statistics_exporter/package.json +++ b/plugins/statistics_exporter/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-statistics_exporter", - "version": "2.0.0-alpha.16", + "version": "2.0.0-alpha.17", "description": "Clusterio plugin for gathering statistics from Factorio servers", "author": "Hornwitser ", "homepage": "https://github.com/clusterio/clusterio#readme", @@ -21,8 +21,8 @@ "@clusterio/lib": "^2.0.0-alpha.0" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.16", - "@clusterio/web_ui": "^2.0.0-alpha.16", + "@clusterio/lib": "^2.0.0-alpha.17", + "@clusterio/web_ui": "^2.0.0-alpha.17", "@types/node": "^20.4.5", "@types/react": "^18.2.21", "typescript": "^5.1.6", diff --git a/plugins/subspace_storage/package.json b/plugins/subspace_storage/package.json index 5dce5e9b..22fd77a8 100644 --- a/plugins/subspace_storage/package.json +++ b/plugins/subspace_storage/package.json @@ -1,6 +1,6 @@ { "name": "@clusterio/plugin-subspace_storage", - "version": "2.0.0-alpha.16", + "version": "2.0.0-alpha.17", "description": "Clusterio plugin for sharing storage between Factorio servers", "author": "Hornwitser ", "homepage": "https://github.com/clusterio/clusterio#readme", @@ -25,8 +25,8 @@ "fs-extra": "^8.1.0" }, "devDependencies": { - "@clusterio/lib": "^2.0.0-alpha.16", - "@clusterio/web_ui": "^2.0.0-alpha.16", + "@clusterio/lib": "^2.0.0-alpha.17", + "@clusterio/web_ui": "^2.0.0-alpha.17", "@types/express": "^4.17.17", "@types/fs-extra": "^11.0.1", "@types/node": "^20.4.5", From b3fc278b2c4c434a53c2982a2f43d670278f421f Mon Sep 17 00:00:00 2001 From: Hornwitser Date: Mon, 12 Feb 2024 18:21:46 +0100 Subject: [PATCH 82/84] Add template to CHANGELOG.md Document the format and expected sections of entries as a template in the changelog so that it's easier to add new log entries. --- CHANGELOG.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index e213fd48..dce4cadc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,28 @@ # Changelog + + ## Version 2.0.0-alpha.17 ### Features From d2775e4db9dc245768969c2237214a9dae03eb1c Mon Sep 17 00:00:00 2001 From: Danielv123 Date: Thu, 8 Feb 2024 14:37:42 +0300 Subject: [PATCH 83/84] Update message section of writing-plugins document --- docs/writing-plugins.md | 235 ++++++++++++++++------------------------ 1 file changed, 96 insertions(+), 139 deletions(-) diff --git a/docs/writing-plugins.md b/docs/writing-plugins.md index 2b7e6566..5d6a70b4 100644 --- a/docs/writing-plugins.md +++ b/docs/writing-plugins.md @@ -12,8 +12,6 @@ The plugin classes have pre-defined hooks that are called during various stages - [Plugin Configuration](#plugin-configuration) - [Communicating with Factorio](#communicating-with-factorio) - [Defining Link Messages](#defining-link-messages) - - [Defining Events](#defining-events) - - [Defining Requests](#defining-requests) - [Sending Link Messages](#sending-link-messages) - [Handling connection events](#handling-connection-events) - [Collecting Statistics](#collecting-statistics) @@ -325,175 +323,134 @@ Events are simple one-way notifications that invoke a handler on the target it's Requests are pairs of request and response messages where the request is sent to the target and the response is the reply back from the target. The requests are similar to HTTP requests, except both parties of a link may initiate one. - -### Defining Events - -Events are defined as properties of the messages object exported on `plugin` that map to instances of the `Event` class from `lib/link`. -The name of the property correspond to the handler invoked on the plugin class. -The Event constructor takes an object of properties that define the event, for example the following could be defined in `plugin`: - -```js -messages: { - startFrobnication: new lib.Event({ - type: "foo_frobber:start_frobnication", - links: ["controller-host", "host-instance"], - forwardTo: "instance", - eventRequired: ["frobnication_type"], - eventProperties: { - "frobnication_type": { type: "string" }, - "urgent": { type: "boolean" }, - }, - }), -}, -``` - -This specifies an event that can be sent from the controller to a host, and from a host to an instance. -It also specifies that the event must contain the property `frobnication_type`, with a string value in the data payload and that it may optionally contain a boolean `urgent` property. -It will also be forwarded by hosts to a specific instance. - -The following properties are recognized by the Event constructor: - -#### type - -The message type sent over the wire. -This must start with the name of the plugin followed by colon and and be unique for the plugin. -The type of the message sent over the socket will have the suffix `_event` appended to it. - -#### links - -An array of strings describing which links this event can be sent over. -Direction matters, `"controller-host"` means the event can be sent from the controller to the host, but can't be sent back the other way, unless `"host-controller"` is also present in the links array. - -The available endpoints are `controller`, `host`, `instance`, and `control`. -Controller talks with host and control, and host talks to instance. -The full chain must be specified as the individual links in order for a message to travers multiple hops, (i.e., for a message to go from controller to instance it must have both `"controller-host"` and `"host-instance"` in the links array). -See `forwardTo` and `broadcastTo` for ways to forward an event to the next link in a chain. - -#### forwardTo - -Target to forward an event to. -Can either be `"controller"`, to indicate a host should forward it to the controller, or `"instance"`, to indicate it should be forwarded to the instances specified by the `instance_id` event property. -This works by using a default handler for the event at the links that forward it. - -#### broadcastTo - -Target to broadcast this message towards. -A value of "instance" means the event will be broadcast to all instances downstream of the target it's sent to, but not back from where it came from. -Currently, only "instance" is supported. -This means that sending the event to a host from an instance will cause it to be broadcast to all instances of that host except for the instance it came from. - -#### eventRequired - -By default all properties defined in `eventProperties` are required to be present in the payload for the event. -This can be overriden by specifying an array of properties which are required here. -This is equivalent to using the `required` keyword in JSON schema. - -#### eventProperties - -Object with properties mapping to a JSON schema of that property that specifies what's valid to send in the event. -This is equivalent to using the `properties` keyword in JSON schema, except that the properties specified are by default required and additional properties are not allowed. -See [this guide][guide] for an introduction to writing JSON schemas. - -The forwardTo and broadcastTo can be combined such that specifying `"controller"` as the forwardTo value and `"instance"` as the broadcastTo value will cause the event to be broadcast to all instances in the cluster. -For this to work, you will need to specify `instance-host`, `host-controller`, `controller-host`, and `host-instance` as the links. - -Keep in mind when forwarding events that if the target an event is being forwarded to is not online, the event will be dropped. -Use a request if you need a confirmation that the message was received. - -### Definining Requests - -Requests are defined as properties of the messages object exported by `plugin` that map to instances of the `Request` class from `lib/link`. -The name of the property corresponds to the handler invoked on the plugin class. -The Request constructor takes an object of properties that define the event. +Messages are defined as properties of the `plugin.messages` array exported by `index.js`. For example, the following could be defined in `plugin`: ```js -messages: { - reportFrobnication: new lib.Request({ - type: "foo_frobber:report_frobnication", - links: ["controller-host", "host-instance"], - forwardTo: "instance", - requestRequired: ["verbosity"], - requestProperties: { - "verbosity": { type: "integer" }, - "special": { type: "boolean" }, - }, - responseProperties: { - "report": { - type: "array", - items: { type: "string" }, +messages: [ + class Frobnicate { + static plugin = "foo_frobber"; // Plugin name + static type = "request"; + static src = "controller"; + static dst = "instance"; + static jsonSchema = { + type: "object", + properties: { + "verbosity": { type: "integer" }, + "special": { type: "boolean" }, }, - }, - }), -}, + required: ["verbosity"], + additionalProperties: false, + }; + constructor(json) { + this.verbosity = json.verbosity; + this.special = json.special; + } + static fromJSON(json) { + return new Frobnicate(json); + } + toJSON() { + return { + verbosity: this.verbosity, + special: this.special, + }; + } + static Response = { + jsonSchema: { + type: "object", + properties: { + "report": { + type: "array", + items: { type: "string" }, + }, + }, + }, + fromJSON(json) { + return json + } + } + } +] ``` -This specifies a request that can be sent from the controller to a host, and from a host to an instance. -The request data must contain the property `verbosity` with an integer number as the value, as well as the `instance_id` property (implied by `forwardTo: "instance"`) and it may also contain a boolean `special` property. -It also defines that the response sent must contain a `report` property mapping to an array of strings. -When received by a host, it will also be forwarded to the instance specified by `instance_id`. +This specifies a request that can be sent from the controller to an instance, being routed through the correct host behind the scenes. +The request data must contain the property `verbosity` with an integer number as the value, and it may also contain a boolean `special` property. +It also defines that the response sent may contain a `report` property mapping to an array of strings. -The following properties are recognized by the Request constructor: +The following properties are recognized as part of a message class: + +#### plugin + +Must contain the name of the plugin this message is defined by. +This property is not used in core messages. #### type The message type sent over the wire. -This must start with the name of the plugin followed by colon and and be unique for the plugin. -The type of the message sent over the socket will have the suffix `_request` appended to it for the request and `_response` appended to it for the response. +This can be either `request` or `event`. The primary difference is whether they allow for a response. -#### links +#### src -An array of strings describing which links this request can be sent over. -Direction matters; `"controller-host"` means the request can be sent from the controller to the host and the host can reply to the controller, but the host can't send a request to the controller unless `"host-controller"` is also present in the links array. +The allowed source of the message. +The source can be specified as `controller`, `host`, `instance`, or `control`. +This property also accepts an array of strings to allow multiple sources. +Messages sent from a different source will be rejected. -The available endpoints are `controller`, `host`, `instance`, and `control`. -Controller talks with host and control, and host talks to instance. -The full chain must be specified as the individual links in order for a message to travers multiple hops, (i.e., for a message to go from controller to instance it must have both `"controller-host"` and `"host-instance"` in the links array). -See `forwardTo` for ways to forward a request to the next link in a chain. +#### dst -#### forwardTo +The allowed destination of the message. +The destination can be specified as `controller`, `host`, `instance`, or `control`. +This property also accepts an array of strings to allow multiple destinations. +Messages sent to a different destination will be rejected. -Target to forward the request to. -Can either be `"controller"` to indicate a host should forward it to the controller when receiving it from an instance, or `"instance"` to indicate it should be forwarded to the instances specified by the `instance_id` request property. -This works by using a default handler for the request by the links that forward it. +#### jsonSchema -#### requestRequired +A JSON schema that specifies what's valid to send in the message. -By default all properties defined in `requestProperties` are required to be present in the payload for the request. -This can be overriden by specifying an array of properties which are required here. -This is equivalent to using the `required` keyword in JSON schema. +#### toJSON -#### requestProperties +Optional function to serialize the class to a JSON object, bypassing the default serialization provided by `JSON.stringify()` -Object with properties mapping to a JSON schema of that property that specifies what's valid to send in the request. -This is equivalent to using the `properties` keyword in JSON schema, except that the properties specified are by default required and additional properties are not allowed. -See [this guide][guide] for an introduction to writing JSON schemas +#### fromJSON -#### responseRequired +Required function to restore the class from a JSON object. +This function is called when the message reaches its destination and is expected to return an instance of the class. -Same as the requestRequired only for the response sent back by the target. - -#### responseProperties - -Same as the requestProperties only for the response sent back by the target. +#### Response +Optional class that specifies the response to the request. +Required properties are `jsonSchema` and `fromJSON`. ## Sending Link Messages -Link messages are sent by calling the `.send()` method on the Event/Request instance with the link you want to send it over and the data to send. -For `InstancePlugin` code the link to the host is the `instance` itself, which is accessible through the `.instance` property of the `InstancePlugin`. -The `.info` property of the plugin class exposes the data exported from the plugin's `plugin` export. -In other words: +Link messages are sent by calling the `.sendTo()` method on a connection object with the destination and the data you want to send, where the data is an instance of one of the defined message classes in plugin.messages. +For example: ```js // In an InstancePlugin class async frobnicate() { - this.info.messages.exampleEvent.send(this.instance, { foo: "bar" }); + const response = await this.instance.sendTo("controller", new messages.Frobnicate({ + verbosity: 2, + special: false, + })); + console.log(response); // { report: [...] } } ``` -For the Request class the send method is async and returns the response data received from the target it was sent to, or throws an error if the request failed. +For classes with `static type = "request"` the send method is async and returns a promise that resolves to the response data received from the target it was sent to, or rejects with an error if the request failed. +The destination specification can either be the ID of a particular control, host or instance, or one of the keywords used to send to multiple targets at once. +Example destinations: + +```js +sendTo("controller", message); +sendTo("allHosts", message); +sendTo("allControls", message); +sendTo("allInstances", message); +sendTo({ hostId: 123 }, message); +sendTo({ instanceId: 123 }, message); +sendTo({ controlId: 123 }, message); +``` ### Handling connection events @@ -508,7 +465,7 @@ For example the sending of an event from an `InstancePlugin` class can be stoppe ```js if (this.host.connected) { - this.info.messages.frobnicate.send(this.instance, { foo: "bar" }); + this.instance.sendTo("controller", new messages.Frobnicate({ foo: "bar" })); } ``` @@ -617,4 +574,4 @@ class CtlPlugin extends BaseCtlPlugin { module.exports = { CtlPlugin, } -``` \ No newline at end of file +``` From 735faf4b2cfcc943b0447fe7cb9aab539412c904 Mon Sep 17 00:00:00 2001 From: Daniel Date: Wed, 14 Feb 2024 09:25:18 +0100 Subject: [PATCH 84/84] Apply suggestions from review Co-authored-by: Hornwitser --- docs/writing-plugins.md | 41 ++++++++++++++++++++++------------------- 1 file changed, 22 insertions(+), 19 deletions(-) diff --git a/docs/writing-plugins.md b/docs/writing-plugins.md index 5d6a70b4..c87b3b40 100644 --- a/docs/writing-plugins.md +++ b/docs/writing-plugins.md @@ -323,7 +323,7 @@ Events are simple one-way notifications that invoke a handler on the target it's Requests are pairs of request and response messages where the request is sent to the target and the response is the reply back from the target. The requests are similar to HTTP requests, except both parties of a link may initiate one. -Messages are defined as properties of the `plugin.messages` array exported by `index.js`. +Messages are defined as items of the `plugin.messages` array exported by `index.js`. For example, the following could be defined in `plugin`: ```js @@ -379,46 +379,49 @@ It also defines that the response sent may contain a `report` property mapping t The following properties are recognized as part of a message class: -#### plugin +#### static plugin Must contain the name of the plugin this message is defined by. This property is not used in core messages. -#### type +#### static type -The message type sent over the wire. -This can be either `request` or `event`. The primary difference is whether they allow for a response. +A string describing the message type sent over the wire. +This can be either `"request"` or `"event"`. +The primary difference is whether they allow for a response. -#### src +#### static src -The allowed source of the message. -The source can be specified as `controller`, `host`, `instance`, or `control`. -This property also accepts an array of strings to allow multiple sources. +A string or an array of strings describing where this message is allowed to originate from. +The available endpoints are `"controller"`, `"host"`, `"instance"`, or `"control"`. Messages sent from a different source will be rejected. -#### dst +#### static dst -The allowed destination of the message. -The destination can be specified as `controller`, `host`, `instance`, or `control`. -This property also accepts an array of strings to allow multiple destinations. +A string or array of strings describing where this message may be addressed to. +The available endpoints are `"controller"`, `"host"`, `"instance"`, or `"control"`. +If the target is multiple hops away it the message will be forwarded towards its destination by the intermediaries. +For example a message sent from a control connection an instance will be forwarded by the controller to the relevant host and from the host to the correct instance. Messages sent to a different destination will be rejected. -#### jsonSchema +#### static jsonSchema A JSON schema that specifies what's valid to send in the message. #### toJSON -Optional function to serialize the class to a JSON object, bypassing the default serialization provided by `JSON.stringify()` +Optional method to serialize an instance of the class to a JSON object, see the [documentation for `JSON.stringify()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify#description) for the semantics of this method. +This should return an object that's valid according to `jsonSchema` and accepted by `fromJSON`, and may be omitted if the default serialisation by `JSON.stringify()` already does this. -#### fromJSON +#### static fromJSON Required function to restore the class from a JSON object. -This function is called when the message reaches its destination and is expected to return an instance of the class. +This function is called when the message reaches its destination and is expected to return an identical instance of the class. +I.e. the result of `fromJSON(JSON.parse(JSON.stringify(instance)))` should be a complete copy of the message. -#### Response +#### static Response -Optional class that specifies the response to the request. +Optional [JSON sterilisable class](/docs/devs/json-serialisable-classes.md) that specifies the response to the request. Required properties are `jsonSchema` and `fromJSON`. ## Sending Link Messages