Compare commits

...
Author SHA1 Message Date
Arvids Godjuks 71feb86ce8 Release Clusterio 2.0.1 (#1081)
* Bump npm packages to 2.0.1

* Add changelog for Clusterio 2.0.1
2026-10-14 23:04:35 +03:00
Arvids Godjuks 346e1acc97 Fix Factorio mod release uploads with native fetch (#1080)
Fix Factorio mod release uploads with native fetch
2026-10-14 22:50:55 +03:00
Arvids Godjuks 46e6445973 Remove obsolete alpha labels from Clusterio Library mod (#1079) 2026-10-14 22:35:58 +03:00
Arvids Godjuks 28eb3a07e0 Remove obsolete 2.0 development warnings (#1078) 2026-10-14 22:24:55 +03:00
Arvids Godjuks 42c7fcd147 2.0.0 2026-10-14 19:29:05 +01:00
Arvids Godjuks 6afc3c7821 Merge pull request #1077 from clusterio/fix/factorio-server-test-port
Avoid random UDP port collisions in FactorioServer tests
2026-10-14 19:27:31 +01:00
Cooldude2606andClaude Opus 5.5 ef943f22a4 Rename inventory_sync to player_sync (#1072)
* Let plugins migrate the data core stores for them

Plugins had no way to update the config fields and permissions core stores
for them, so renaming inventory_sync needed migrations in core packages.

A plugin can now declare a migrationEntrypoint. It's loaded on the
controller and hosts before configs and roles are read, for every plugin in
the plugin list, and attaches handlers to sync hooks for the controller,
host and instance configs and roles. lib gets helpers to rename config
fields and permissions, and each config gets a hidden <plugin>.version field
set after load so migrations can tell which version saved the data.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Write plugin versions while loading configs

The staging copy made by Config.set() went through the constructor, so
plugin migrations ran a second time on the live fields. A version-gated
migration was applied twice when setPluginVersions() changed the version.
Staging now copies the fields with update() instead.

A config saved without <plugin>.version got the running version as its
default, so setPluginVersions() saw no change and nothing saved the stamp.
The constructor now writes the registered plugin versions after the
migrations run and marks the config dirty when that changes a loaded value.
Controller.bootstrap() saves the instances that came out dirty, and
setPluginVersions() is gone.

Restructure the SyncHook and plugin config tests by method, and suggest the
controller config version for role migrations in the docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Rename inventory_sync to player_sync

The plugin syncs more than inventories now (settings, appearance, recipe
notifications), so name it for what it does. The package becomes
@clusterio/plugin-player_sync and every inventory_sync identifier follows.

A migration entrypoint renames the inventory_sync config fields, including
load_plugin, and the inventory view permission. The plugin moves
inventories.json in the database directory to player_sync.json and
storage.inventory_sync in saves to storage.player_sync itself, and closes
dialogs left open under the old GUI names on startup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Let old saves through the player left guard

on_pre_player_left_game runs before on_server_startup when an old autosave
with connected players is loaded, so the data is still under inventory_sync.
The guard returned before get_script_data() could move it, and the recovery
upload was never sent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 18:54:55 +01:00
Bastiaan e33d06a5cc Merge pull request #1075 from bbassie/fix/ctl-config-fields
Register plugin ctl config fields
2026-10-14 18:54:39 +01:00
Cooldude2606 70dff9ef68 Merge pull request #1073 from bbassie/feature/plugin-migrations
Let plugins migrate the data core stores for them
2026-10-14 17:49:27 +01:00
bbassieandClaude Opus 5.5 03ff234f0f Let old saves through the player left guard
on_pre_player_left_game runs before on_server_startup when an old autosave
with connected players is loaded, so the data is still under inventory_sync.
The guard returned before get_script_data() could move it, and the recovery
upload was never sent.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 16:39:27 +00:00
bbassieandClaude Opus 5.5 8363c5f0ef Rename inventory_sync to player_sync
The plugin syncs more than inventories now (settings, appearance, recipe
notifications), so name it for what it does. The package becomes
@clusterio/plugin-player_sync and every inventory_sync identifier follows.

A migration entrypoint renames the inventory_sync config fields, including
load_plugin, and the inventory view permission. The plugin moves
inventories.json in the database directory to player_sync.json and
storage.inventory_sync in saves to storage.player_sync itself, and closes
dialogs left open under the old GUI names on startup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 16:37:12 +00:00
bbassieandClaude Opus 5.5 e455d17bb7 Write plugin versions while loading configs
The staging copy made by Config.set() went through the constructor, so
plugin migrations ran a second time on the live fields. A version-gated
migration was applied twice when setPluginVersions() changed the version.
Staging now copies the fields with update() instead.

A config saved without <plugin>.version got the running version as its
default, so setPluginVersions() saw no change and nothing saved the stamp.
The constructor now writes the registered plugin versions after the
migrations run and marks the config dirty when that changes a loaded value.
Controller.bootstrap() saves the instances that came out dirty, and
setPluginVersions() is gone.

Restructure the SyncHook and plugin config tests by method, and suggest the
controller config version for role migrations in the docs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 16:37:03 +00:00
bbassieandClaude Opus 5.5 aec0fc7a1e Let plugins migrate the data core stores for them
Plugins had no way to update the config fields and permissions core stores
for them, so renaming inventory_sync needed migrations in core packages.

A plugin can now declare a migrationEntrypoint. It's loaded on the
controller and hosts before configs and roles are read, for every plugin in
the plugin list, and attaches handlers to sync hooks for the controller,
host and instance configs and roles. lib gets helpers to rename config
fields and permissions, and each config gets a hidden <plugin>.version field
set after load so migrations can tell which version saved the data.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 15:33:27 +00:00
BastiaanandClaude Opus 5.5 24586a4b7c Redact credentials from logged request urls (#1074)
The controller logs the url and query of every HTTP request. Plugins
doing OAuth get the authorization code as a query parameter on their
callback route, so it ended up in the log. redactUrl now accepts a
path, code joins the redacted parameter names, and redactQuery covers
the parsed query in the log metadata.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 15:58:00 +01:00
Cooldude2606 2c506df38a Merge pull request #1062 from PHIDIAS0303/patch-1
Add links to Exp Cluster and Chat Relay plugins
2026-10-13 21:38:53 +01:00
Cooldude2606 04c08efb0e Update ExpCluster description 2026-10-13 21:32:54 +01:00
Cooldude2606 b4cbb1294a Fix running Clusterio from published packages (#1070)
* Fix running Clusterio from published packages

Since the move to ES modules and pnpm 12, packages installed from npm
fail to start:

- The package imports of @clusterio/lib point Node at TypeScript
  sources, which Node refuses to strip types from inside node_modules.
- The controller, host and ctl import their package.json from next to
  the compiled entrypoint, a copy pnpm leaves out of the package.
- ctl, host and web_ui have no .npmignore, so pnpm packs them using the
  repository .gitignore and leaves out nearly all of dist/.

Resolve the package imports to the compiled files at runtime while
keeping the sources for type checking, import package.json by package
name so it resolves to the real file, and add .npmignore files so pnpm
packs the same files as npm.

* Load plugins installed from npm by package name

Plugins installed from npm are listed by package name, but the plugin
loader joined that name onto the main and entrypoint paths and imported
the result as a file, which resolves relative to the working directory.
Every npm installed plugin then failed to load. Resolve these paths from
the directory of the plugin's package.json instead.
2026-10-13 21:24:16 +01:00
ILLISIS 137211d55d Merge pull request #1067 from ILLISIS/fix-game-port-in-use
Replace a stored game port that another process holds
2026-10-13 20:23:46 +01:00
Cooldude2606 48e8c53776 Merge pull request #1064 from ILLISIS/fix-ws-error-before-handshake
Handle WebSocket errors before the handshake completes
2026-10-12 22:03:08 +01:00
phidias 72360cbdf4 Update README.md 2026-10-13 01:27:35 +09:00
ILLISISandClaude Opus 5.5 f581c70ccd Log a socket error once, from WsServer
WsServer.handleConnection now listens for errors on every socket for its
whole life, and every socket a WsServerConnector holds comes from there,
new sessions and resumed ones alike. The connector's own listener only
logged the same error a second time, so it is removed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F7obMEi4L4PzUQ8w8q9Uyz
2026-10-13 03:19:55 +11:00
Daniel 9c8a308ea9 Merge pull request #1071 from clusterio/fix/subspace-initial-inventory
Send the full subspace inventory immediately on startup
2026-10-12 07:58:09 +02:00
dc9096303c Log the message of an instance start error without a stack (#1066)
HostRouter.wakeInstance() logged err.stack, which RequestError leaves
undefined, so a failed start was logged as "Error starting instance:
undefined" and the reason, such as no game port left in the range, was
lost.


Claude-Session: https://claude.ai/code/session_01F7obMEi4L4PzUQ8w8q9Uyz

Co-authored-by: ILLISIS <jayson.harris01@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-12 00:00:01 +01:00
Danielv123 7fa1e90ed7 Send the full subspace inventory immediately on startup 2026-10-11 13:22:26 +02:00
ILLISISandClaude Opus 5.5 16666c3ee0 Handle WebSocket errors before the handshake completes
WsServer attached no error listener to a new socket. WsServerConnector adds
one, but only after the handshake. ws emits an error for an invalid frame,
for example a close frame with the reserved status code 1006, and an error
event with no listener throws. That stopped the controller.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F7obMEi4L4PzUQ8w8q9Uyz
2026-10-11 12:53:13 +11:00
phidias 7ee67fc178 Add links to Exp Cluster and Chat Relay plugins 2026-10-09 22:58:22 +09:00
89 changed files with 1563 additions and 272 deletions

No files matched your search

+2 -2
View File
@@ -22,7 +22,7 @@ jobs:
node-version: 24.x
registry-url: https://registry.npmjs.org
- run: pnpm install --no-frozen-lockfile
- run: pnpm -r --filter='./{plugins,packages}/*' publish --no-git-checks
- run: pnpm -r --filter='./packages/*' --filter='./plugins/*' --fail-if-no-match publish --no-git-checks
- uses: softprops/action-gh-release@v3
with:
files: pnpm-lock.yaml
@@ -37,7 +37,7 @@ jobs:
with:
node-version: 24.x
- run: pnpm install --no-frozen-lockfile
- run: pnpm -r --filter='./{plugins,packages}/*' run build-mod --output-dir ../../dist
- run: pnpm -r --filter='./packages/*' --filter='./plugins/*' --fail-if-no-match run build-mod --output-dir ../../dist
- run: node ./publish.js factorio ./dist
env:
FACTORIO_TOKEN: ${{ secrets.FACTORIO_TOKEN }}
+132 -2
View File
@@ -31,6 +31,138 @@ Many thanks to the following for contributing to this release:
[@username](https://github.com/username)
-->
## Version 2.0.1
### Fixes
- Removed obsolete development-branch warnings from controller, host and ctl startup. [#1078](<https://github.com/clusterio/clusterio/pull/1078>)
- Removed the obsolete alpha label and development warning from the Clusterio Library mod title and description. [#1079](<https://github.com/clusterio/clusterio/pull/1079>)
- Fixed Factorio mod release uploads to use valid multipart requests and files from the requested output directory, uploading variants in version order. [#1080](<https://github.com/clusterio/clusterio/pull/1080>)
Many thanks to the following for contributing to this release:
[@psihius](<https://github.com/psihius>)
## Version 2.0.0
### Features
- Mod packs now show informational notices for disabled recommended mods and available compatible updates. [#949](<https://github.com/clusterio/clusterio/issues/949>)
- Added host, server name, save copy and start options to the create instance dialog in the web UI. [#405](<https://github.com/clusterio/clusterio/issues/405>)
- Added `InstanceSaveGameRequest` and `clusterioctl instance save-game` to save the running game of an instance. [#405](<https://github.com/clusterio/clusterio/issues/405>)
- Added `permissions` to `PluginDeclaration` and a `Permissions` type registry so plugin permission names are checked at compile time. [#987](<https://github.com/clusterio/clusterio/issues/987>)
- Added Update All to the web interface and `--all` to the ctl update commands, updating the core package and all npm installed plugins in one npm call. [#910](<https://github.com/clusterio/clusterio/issues/910>)
- Added typed hook system for plugin registration. ([#730](<https://github.com/clusterio/clusterio/issues/730>))
- Added `plugin install <package>` command to controller, host and ctl which installs a plugin from npm and adds it to the plugin list. [#1008](<https://github.com/clusterio/clusterio/issues/1008>)
- Added a guard that prevents controller and host restart requests from unintentionally downgrading Clusterio. [#950](<https://github.com/clusterio/clusterio/issues/950>)
- Added option to load static assets from a separate URL from where the controller's web API is hosted at. [#983](<https://github.com/clusterio/clusterio/pull/983>)
- Added `clusteriocontroller copy-static` command to copy static web assets to a folder to ease hosting assets on a separate server. [#983](<https://github.com/clusterio/clusterio/pull/983>)
- Added Mark Restart Required to the host and controller pages and matching ctl commands, for plugins installed by other means than the install request. [#1043](<https://github.com/clusterio/clusterio/pull/1043>)
- Added the RoleViewPageBeforePermissions web extension slot for plugin components above the permissions on the role page. [#1045](<https://github.com/clusterio/clusterio/pull/1045>)
- Added a list of permission groups to the role page, highlighting groups with granted permissions. [#1044](<https://github.com/clusterio/clusterio/pull/1044>)
- Added `clusteriohost create-scenario` to create a scenario with the Clusterio modules patched in for use outside of Clusterio. [#1014](<https://github.com/clusterio/clusterio/pull/1014>)
- Plugins can declare a `migrationEntrypoint` to migrate the config fields and role permissions core stores for them. Configs track the plugin version they were last loaded with in a hidden `<plugin>.version` field. [#1073](<https://github.com/clusterio/clusterio/pull/1073>)
### Fixes
- Deprecation warning DEP0190 is no longer printed when installing. [#851](<https://github.com/clusterio/clusterio/issues/851>)
- Events sent to subscribers are no longer addressed to the request id of the request that created the subscription. [#847](<https://github.com/clusterio/clusterio/issues/847>)
- Factorio account token no longer written to the log when a mod download fails. [#795](<https://github.com/clusterio/clusterio/issues/795>)
- Failed downloads no longer leave a temporary file and an open file handle behind, which could shift the file names used by later downloads. [#957](<https://github.com/clusterio/clusterio/pull/957>)
- Requests that could not be forwarded to the next hop are no longer answered a second time when the link disconnects. [#650](<https://github.com/clusterio/clusterio/issues/650>)
- Events broadcast from an instance to all hosts are now delivered to every host instead of only the one running the instance. [#575](<https://github.com/clusterio/clusterio/issues/575>)
- Web interface now asks to be reloaded when the controller's plugins change, instead of continuing against plugins that are no longer loaded. [#804](<https://github.com/clusterio/clusterio/issues/804>)
- Plugin list no longer reports a plugin added since the page was opened as having failed to load. [#729](<https://github.com/clusterio/clusterio/issues/729>)
- Instance console no longer hides the Factorio server's replies to console commands, or Factorio's own errors, when filtering to game activity. [#853](<https://github.com/clusterio/clusterio/issues/853>)
- Allow player online autosave slots to be disabled while autosaves are enabled. [#966](<https://github.com/clusterio/clusterio/issues/966>)
- Plugin web bundles are now loaded relative to `staticRoot` instead of `webRoot`. [#980](<https://github.com/clusterio/clusterio/issues/980>)
- Fixed inventory_sync not preserving the personal roboport toggle, the equipment movement bonus toggle, game view settings such as alt mode, and toggleable mod shortcuts between instances. [#898](<https://github.com/clusterio/clusterio/issues/898>)
- Schema validation errors are now included when an InvalidMessage is logged from IPC and event handlers. [#986](<https://github.com/clusterio/clusterio/issues/986>)
- Fixed clusterioctl not being able to reference hosts, instances, mod packs and roles by an integer like name. [#886](<https://github.com/clusterio/clusterio/issues/886>)
- Fixed the web-only plugin template referencing a permission that was never defined. [#987](<https://github.com/clusterio/clusterio/issues/987>)
- Config files are no longer world readable by default. [#1005](<https://github.com/clusterio/clusterio/issues/1005>)
- Fixed failed recipe notification names being duplicated on every upload from the same instance in inventory sync. [#964](<https://github.com/clusterio/clusterio/issues/964>)
- Instances refuse to start with a mod pack for a newer Factorio version than the instance runs, instead of Factorio silently loading default mod settings. [#1006](<https://github.com/clusterio/clusterio/issues/1006>)
- Documented the config options missing from configuration.md, marked the internal ones and noted the validation rules. [#985](<https://github.com/clusterio/clusterio/issues/985>)
- Fixed concurrent saves of the same file racing on a shared temporary file. [#1026](<https://github.com/clusterio/clusterio/pull/1026>)
- Fixed control connections leaving their log subscription and debug dump listeners attached after closing. [#1027](<https://github.com/clusterio/clusterio/pull/1027>)
- Fixed the Load Scenario modal reading the wrong field for map settings. [#1032](<https://github.com/clusterio/clusterio/pull/1032>)
- Fixed the controller exiting when a control message's permission check threw an unexpected error. [#1033](<https://github.com/clusterio/clusterio/pull/1033>)
- Fixed the host exiting on server log lines that look like actions but do not match the expected format. [#1034](<https://github.com/clusterio/clusterio/pull/1034>)
- Fixed the controller crashing when a control connection with the WebSocket debug dump active was resumed. [#1035](<https://github.com/clusterio/clusterio/pull/1035>)
- Fixed deleted users keeping their permissions on any open connection until they disconnected. [#1030](<https://github.com/clusterio/clusterio/pull/1030>)
- Restored pagination on the instance save list. [#967](<https://github.com/clusterio/clusterio/issues/967>)
- Mod dependency resolution now only picks releases made for the mod pack's Factorio version. [#965](<https://github.com/clusterio/clusterio/issues/965>)
- Inventory sync no longer uploads an empty inventory over the stored one after a failed download. [#912](<https://github.com/clusterio/clusterio/issues/912>)
- Inventory sync now releases players who leave while using a temporary inventory, instead of showing them as busy until the instance stops. [#975](<https://github.com/clusterio/clusterio/pull/975>)
- Fixed reading map exchange strings from Factorio 2.1.13 and later, which broke `instance load-scenario --map-exchange-string` on current Factorio. [#977](<https://github.com/clusterio/clusterio/pull/977>)
- Fixed instances getting stuck on a host after a failed assign, and instance create requests no longer accept `instance.assigned_host`. [#1029](<https://github.com/clusterio/clusterio/pull/1029>)
- Fixed the host exiting when an instance's whitelist or server settings config changed while it was starting, or when tags contained non-string values. [#1028](<https://github.com/clusterio/clusterio/pull/1028>)
- Fixed a host being able to modify the controller's stored save list for, or forge log messages attributed to, instances assigned to other hosts. [#1036](<https://github.com/clusterio/clusterio/pull/1036>)
- Fixed a stop request or RCON command hanging when the Factorio process exited before its RCON connection was established. [#1031](<https://github.com/clusterio/clusterio/pull/1031>)
- Fixed core plugins needlessly installing the web_ui, controller, host and ctl packages. [#1021](<https://github.com/clusterio/clusterio/pull/1021>)
- Installing a plugin on a host or the controller now marks it as needing a restart. [#1042](<https://github.com/clusterio/clusterio/pull/1042>)
- Fixed plugin manifest path resolution when loading web assets. [#1046](<https://github.com/clusterio/clusterio/issues/1046>)
- A plugin whose web build is expected but missing is reported again: `Controller.loadJsonObject(path, true)` throws on a missing file instead of returning `{}`, and a malformed JSON file always throws. [#1046](<https://github.com/clusterio/clusterio/issues/1046>)
- Preserve full configuration field names when using the CLI config editor. [#796](<https://github.com/clusterio/clusterio/issues/796>)
- Plugin metrics hook handlers may return nothing. [#1052](<https://github.com/clusterio/clusterio/pull/1052>)
- Fixed save names given when starting an instance or creating a save not being checked, allowing files outside the instance directory to be written or modified. [#1053](<https://github.com/clusterio/clusterio/pull/1053>)
- Fixed access tokens from request headers being written to the controller and cluster logs. [#1054](<https://github.com/clusterio/clusterio/pull/1054>)
- Fixed subscriptions and log following continuing after the user lost permission for them. [#1056](<https://github.com/clusterio/clusterio/pull/1056>)
- Fixed web interface data staying marked as up to date after the user lost permission to view it, and not loading after permission was granted. [#1058](<https://github.com/clusterio/clusterio/pull/1058>)
- Fixed ctl config edit showing fields that cannot be edited and leaving out field descriptions. [#1051](<https://github.com/clusterio/clusterio/issues/1051>)
- Fixed the reason an instance failed to start being logged as "undefined". [#1065](<https://github.com/clusterio/clusterio/issues/1065>)
- Fixed subspace inventory combinators missing items after instance startup. [#1069](<https://github.com/clusterio/clusterio/issues/1069>)
- Fixed the controller crashing when a WebSocket client sent an invalid frame before completing the handshake. [#1063](<https://github.com/clusterio/clusterio/issues/1063>)
- Fixed an instance reusing a stored game port that another process holds, which made it fail to start until the port was freed. [#1065](<https://github.com/clusterio/clusterio/issues/1065>)
- Fixed credentials in request query parameters, such as OAuth codes, being written to the controller's HTTP log. [#1074](<https://github.com/clusterio/clusterio/pull/1074>)
- Fixed plugin `ctlConfigFields` not being registered in the clusterioctl config. [#1075](<https://github.com/clusterio/clusterio/pull/1075>)
- Fixed startup failures and plugin loading when installing Clusterio from published npm packages after the ES module migration. [#1070](<https://github.com/clusterio/clusterio/pull/1070>)
### Changes
- Global chat now filters server-specific GPS, blueprint, train, and train-stop rich-text tags before relaying messages between instances. [#593](<https://github.com/clusterio/clusterio/issues/593>)
- Instance console filter is now a labelled Game/Instance control instead of a single-label toggle, and Factorio's output in the log consoles is prefixed with `[server]` like every other log level. [#853](<https://github.com/clusterio/clusterio/issues/853>)
- Inventory sync sends only the recipe notifications that differ from the destination instance's state on download. [#964](<https://github.com/clusterio/clusterio/issues/964>)
- Log the source of responses received without a pending request, the peer of timed out connections and the number of messages resent on session resume. [#1007](<https://github.com/clusterio/clusterio/issues/1007>)
- Plugins that are not expected to have a web build are no longer shown with an error in the plugins page. [#1004](<https://github.com/clusterio/clusterio/issues/1004>)
- Plugins should now return a function to register handlers via hooks. ([#730](<https://github.com/clusterio/clusterio/issues/730>))
- Plugin templates from `@clusterio/create` use entrypoint functions and hooks. ([#730](<https://github.com/clusterio/clusterio/issues/730>))
- Improved typing for plugin APIs and webui extension points. ([#730](<https://github.com/clusterio/clusterio/issues/730>))
- Plugin hooks run concurrently instead of one plugin at a time. ([#730](<https://github.com/clusterio/clusterio/issues/730>))
- Access tokens are also accepted as an `Authorization: Bearer` header on HTTP API endpoints. [#1009](<https://github.com/clusterio/clusterio/issues/1009>)
- `plugin install` and `plugin add` with a package name now check that they are run from a Clusterio installation directory. [#1008](<https://github.com/clusterio/clusterio/issues/1008>)
- Bumped pnpm to 12, the GitHub Actions used by CI to their current majors, and the CI Factorio version to 2.1.17. [#1022](<https://github.com/clusterio/clusterio/pull/1022>)
- Bumped dependencies, including eslint 10, yargs 18, chalk 6, @types/express 5 and @types/node 24. [#1023](<https://github.com/clusterio/clusterio/pull/1023>)
- Plugin discovery now traverses dependencies only after confirming the parent package is a Clusterio plugin. [#963](<https://github.com/clusterio/clusterio/pull/963>)
- Converted the project to using ES Modules. [#984](<https://github.com/clusterio/clusterio/pull/984>)
- Converted the bundled plugins to entrypoint functions. [#1052](<https://github.com/clusterio/clusterio/pull/1052>)
- Added `controller.max_subscription_filters` config to limit the number of filters a connection can subscribe with for one event. [#1055](<https://github.com/clusterio/clusterio/pull/1055>)
- Added `controller.max_send_buffer_size` config to limit how much the controller buffers for a connection that does not acknowledge messages. [#1057](<https://github.com/clusterio/clusterio/pull/1057>)
- Added links to the ExpCluster and Chat Relay community plugins in the README. [#1062](<https://github.com/clusterio/clusterio/pull/1062>)
### Breaking Changes
- `staticRoot` in the web interface now includes the `static/` prefix. Plugins that fetch assets with `${staticRoot}static/...` should drop the `static/` part. [#980](<https://github.com/clusterio/clusterio/issues/980>)
- `checkPermission` and the web UI `hasPermission` helpers take `PermissionName`; TypeScript plugins must augment the `Permissions` interface. [#987](<https://github.com/clusterio/clusterio/issues/987>)
- Replaced `plugins` map with `loadedPlugins` set across core classes (except WebUi). ([#730](<https://github.com/clusterio/clusterio/issues/730>))
- The `/metrics` endpoint requires a user token with the new `core.controller.metrics` permission. Concurrent scrapes share a single gather from the hosts. [#1009](<https://github.com/clusterio/clusterio/issues/1009>)
- Config files now have to be explicitly created. Use the `--create-config` flag to get the old behaviour of creating it if missing. [#548](<https://github.com/clusterio/clusterio/issues/548>)
- Renamed clusterioctl's config-control subcommand to config. [#1039](<https://github.com/clusterio/clusterio/pull/1039>)
- Renamed clusterioctl's main class from Control to Ctl. [#1039](<https://github.com/clusterio/clusterio/pull/1039>)
- Renamed clusterioctl's config entries from control.[name] to ctl.[name]. [#1039](<https://github.com/clusterio/clusterio/pull/1039>)
- Plugin entrypoints must now specify the full filename including the `.js` extension. [#984](<https://github.com/clusterio/clusterio/pull/984>)
- Renamed the inventory_sync plugin to player_sync, published as `@clusterio/plugin-player_sync`. Config fields, the inventory view permission, the controller database file and save data are migrated on load. Uninstall `@clusterio/plugin-inventory_sync` before installing the new package. [#1072](<https://github.com/clusterio/clusterio/pull/1072>)
### Meta
- Added a `NO_EXTERNAL_TEST` flag to skip tests that use external APIs. [#976](<https://github.com/clusterio/clusterio/pull/976>)
- Added a CI check that the controller and host stay compatible with the latest published release. [#978](<https://github.com/clusterio/clusterio/pull/978>)
- Added a Lua lint CI job using emmylua_check. [#979](<https://github.com/clusterio/clusterio/pull/979>)
- Added a plugin migration guide for 2.0 and documented that minor versions may contain breaking changes for the plugin API. [#1050](<https://github.com/clusterio/clusterio/pull/1050>)
Many thanks to the following for contributing to this release:
[@bbassie](<https://github.com/bbassie>), [@lntutor](<https://github.com/lntutor>), [@ColumbusLabs](<https://github.com/ColumbusLabs>), [@floze-the-genius](<https://github.com/floze-the-genius>), [@Cooldude2606](<https://github.com/Cooldude2606>), [@solarcloud7](<https://github.com/solarcloud7>), [@Hornwitser](<https://github.com/Hornwitser>), [@ILLISIS](<https://github.com/ILLISIS>), [@Danielv123](<https://github.com/Danielv123>), [@PHIDIAS0303](<https://github.com/PHIDIAS0303>)
## Version 2.0.0-alpha.27
### Features
@@ -47,14 +179,12 @@ Many thanks to the following for contributing to this release:
### Fixes
- Allow player online autosave slots to be disabled while autosaves are enabled. [#966](<https://github.com/clusterio/clusterio/issues/966>)
- Silenced two Ant Design console warnings in the Web UI: the deprecated Modal `destroyOnClose` prop, and a duplicate-value warning from the instance version selector. [#940](<https://github.com/clusterio/clusterio/pull/940>)
- Surface the mod portal's own error when downloading mods for an unsupported Factorio version. [#941](<https://github.com/clusterio/clusterio/pull/941>)
- Instances always create their config file on assignment preventing duplicate folder creation. [#945](<https://github.com/clusterio/clusterio/pull/945>)
### Changes
- Global chat now filters server-specific GPS, blueprint, train, and train-stop rich-text tags before relaying messages between instances. [#593](<https://github.com/clusterio/clusterio/issues/593>)
- Reworked the site layout into a fixed sidebar with a top bar. [#924](<https://github.com/clusterio/clusterio/pull/924>)
- The builtin "recycler" mod is now included correctly as builtin. [#925](<https://github.com/clusterio/clusterio/issues/925>)
- Mod `info.json` versions (a mod's version and its `factorio_version`) are now parsed leniently, matching how the game reads them, instead of being validated against a fixed list of Factorio versions. [#941](<https://github.com/clusterio/clusterio/pull/941>)
+8 -4
View File
@@ -10,9 +10,10 @@ Discord for development/support/play: https://discord.gg/5XuDkje
## Important notice
Clusterio 2.0 is still in alpha, however the previous stable has been abandoned and is no longer supported.
Despite being alpha it's reasonably stable now and there's no major breakages expected before a stable release of 2.0.
If you are starting a new cluster, it's highly recommended to use the 2.0 alpha.
Clusterio 2.0 is the current stable release and requires Node.js 22 or newer.
The previous stable series is no longer supported.
If you are upgrading from a 2.0 alpha, review the [changelog](CHANGELOG.md) before upgrading.
Plugin authors should also follow the [2.0 migration guide](docs/devs/plugin-migration-to-2.0.md).
### Ways to support me/the project:
@@ -75,7 +76,7 @@ These are the plugins supported and maintained by the Clusterio developers:
- [Statistics Exporter](/plugins/statistics_exporter/README.md): collect in-game statistics from all the servers and makes it available to the Prometheus endpoint on the controller.
- [Subspace Storage](https://github.com/clusterio/subspace_storage): Provide shared storage that can transport items between servers via teleport chests.
- [Player Auth](/plugins/player_auth/README.md): Provides authentication to the cluster via logging into a Factorio server.
- [Inventory Sync](/plugins/inventory_sync/README.md): Synchronizes your inventory between servers.
- [Player Sync](/plugins/player_sync/README.md): Synchronizes your inventory and player settings between servers.
There's also plugins developed and maintained by the community:
@@ -83,6 +84,9 @@ There's also plugins developed and maintained by the community:
By Hornwitser.
- [Server Select](https://github.com/Hornwitser/server_select/tree/clusterio-2.0) (@hornwitser/server_select): In-game GUI for connecting to other servers in the cluster.
Originally by Godmave, ported to 2.0 by Hornwitser.
- [ExpCluster](https://github.com/explosivegaming/ExpCluster) (@expcluster/*): A collection of plugins and scenario features including role and group syncing, moderation tools, quality of life additions, and discord account linking.
See repo readme for plugin breakdown. Maintained by Cooldude2606.
- [Chat Relay](https://github.com/PHIDIAS0303/chat_relay) (@PHIDIAS0303/chat_relay): Chat relay of Discord and Factorio, seperate channel.
Want to make your own plugin?
Check out the documentation on [Writing Plugins](/docs/writing-plugins.md) for where to start.
+7 -1
View File
@@ -11,6 +11,12 @@ If the script fails due to GitHub's 60/hour public access rate limit, you can se
## Packages
The [Publish workflow](../.github/workflows/publish.yml) publishes npm packages and Lua mods when a GitHub release is published.
For this workflow, first merge the changelog and package version updates into `master`, then create and push the release tag on that merged commit.
Create a draft GitHub release for the existing tag and use the GitHub Release Markdown from `scripts/pr_to_changelog.mts` as its notes.
Publishing the draft triggers the workflow; pushing a tag alone does not.
The steps below describe publishing npm packages manually instead.
> [!NOTE]
> You will need commit access to the repository as well publish access to the @clusterio org on npm to do this.
@@ -40,7 +46,7 @@ If the script fails due to GitHub's 60/hour public access rate limit, you can se
```sh
pnpm clean
pnpm -r --filter='./{plugins,packages}/*' publish
pnpm -r --filter='./packages/*' --filter='./plugins/*' --fail-if-no-match publish
```
## Plugins
+74
View File
@@ -11,6 +11,7 @@ Each entrypoint of a plugin exports a function that attaches handlers to hooks c
- [Logging Messages](#logging-messages)
- [Plugin Configuration](#plugin-configuration)
- [Plugin Permissions](#plugin-permissions)
- [Migrating Stored Data](#migrating-stored-data)
- [Communicating with Factorio](#communicating-with-factorio)
- [Defining Link Messages](#defining-link-messages)
- [Sending Link Messages](#sending-link-messages)
@@ -98,6 +99,11 @@ The following properties are recognized:
This is an optional paramater.
A plugin can be made that only runs on the clusterioctl side.
**migrationEntrypoint**:
Path to a Node.js module relative to the plugin directory which exports the migration entrypoint function for this plugin.
This is an optional parameter.
See [Migrating Stored Data](#migrating-stored-data)
**features**:
Array of features the plugin needs from the instances it is loaded on, `"SavePatching"` and/or `"ScriptCommands"`.
See [Handling Invalid Configuration](#handling-invalid-configuration)
@@ -310,6 +316,74 @@ Permissions declared this way are registered when the plugin is loaded.
The older `lib.definePermission()` function still works, but a plugin should not use both for the same permission name.
## Migrating Stored Data
Clusterio stores the config fields and permissions of plugins, so a plugin that renames them needs to migrate the stored values or users lose what they had set.
This is done from the migration entrypoint, which is loaded on the controller and hosts before any config or role is read, for all plugins in the plugin list including those disabled with `load_plugin`.
It is passed a context with `hooks` and attaches handlers for the data it needs to migrate:
```js
import * as lib from "@clusterio/lib";
export default function(context) {
const { hooks, plugin } = context;
// Plugin was renamed from frobber in 2.0.0
const renameFields = config => lib.renameConfigPrefix(config, "frobber.", "foo_frobber.");
hooks.controllerConfig.attach(plugin.name, renameFields);
hooks.hostConfig.attach(plugin.name, renameFields);
hooks.instanceConfig.attach(plugin.name, renameFields);
hooks.role.attach(plugin.name, role => lib.renamePermissionPrefix(role, "frobber.", "foo_frobber."));
}
```
The handlers get the serialized config or role after the core migrations and return the migrated version.
They run synchronously, a handler that throws stops the controller or host from starting.
`lib.renameConfigField`, `lib.renameConfigPrefix`, `lib.renamePermission` and `lib.renamePermissionPrefix` cover the common case of renaming.
Handlers run every time the data is loaded and must leave data that is already migrated unchanged.
For changes that can't be repeated, each config has a hidden `<plugin>.version` field with the version of the plugin it was last loaded with, which is updated after the migrations have run:
```js
hooks.instanceConfig.attach(plugin.name, config => {
const version = config["foo_frobber.version"];
// The default delay went from 1 to 2 in 1.2.0, update configs still on the old default
if (
version && lib.integerFullVersion(version) < lib.integerFullVersion("1.2.0")
&& config["foo_frobber.delay"] === 1
) {
config["foo_frobber.delay"] = 2;
}
return config;
});
```
The version field is missing or empty in configs saved before Clusterio tracked it, and in configs last loaded in recovery mode.
Roles have no version of their own.
They are stored on the controller, so use the version in the controller config, which is loaded before the roles:
```js
let controllerVersion;
hooks.controllerConfig.attach(plugin.name, config => {
controllerVersion ??= config["foo_frobber.version"] ?? "";
return config;
});
hooks.role.attach(plugin.name, role => {
// foo_frobber.edit was split out of foo_frobber.view in 1.2.0
if (
controllerVersion && lib.integerFullVersion(controllerVersion) < lib.integerFullVersion("1.2.0")
&& role.permissions.includes("foo_frobber.view")
) {
role.permissions.push("foo_frobber.edit");
}
return role;
});
```
Data the plugin stores itself, like files in the database directory or values in the save, is up to the plugin to migrate from its other entrypoints.
Modules don't need migrating, save patching replaces the files of every module when an instance starts.
## Communicating with Factorio
For pushing data into Factorio there's RCON, which lets you send arbitrary Lua commands to invoke whatever code you want in the game.
+3 -8
View File
@@ -419,6 +419,9 @@ async function initialize(): Promise<InitializeParameters> {
lib.registerPluginMessages(pluginInfos);
lib.registerPluginPermissions(pluginInfos);
lib.addPluginConfigFields(pluginInfos);
if (!args.recovery) {
await lib.loadPluginMigrations(pluginInfos);
}
const [controllerConfig, controllerConfigLock] = await lib.loadConfigFromArgs(
args,
@@ -500,14 +503,6 @@ async function startup() {
}
export function bootstrap() {
// eslint-disable-next-line no-console
console.warn(`
+==========================================================+
I WARNING: This is the development branch for the 2.0 I
I version of clusterio. Expect things to break. I
+==========================================================+
`
);
startup().catch(err => {
if (err instanceof lib.StartupError || err.code === "InstallationError") {
logger.fatal(`
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@clusterio/controller",
"version": "2.0.0-alpha.27",
"version": "2.0.1",
"description": "Implementation of Clusterio controller",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Hornwitser <github@hornwitser.no>",
+15 -7
View File
@@ -175,6 +175,12 @@ export default class Controller {
InstanceRecord.fromJSON.bind(InstanceRecord),
this.migrateInstances, this.finaliseInstances,
).bootstrap());
// Save plugin versions written while loading
for (const instance of instances.values()) {
if (instance.config.dirty) {
instances.set(instance);
}
}
const saves = new lib.SubscribableDatastore(...await new lib.JsonIdDatastoreProvider(
path.join(databaseDirectory, "saves.json"),
@@ -403,16 +409,17 @@ export default class Controller {
// Log all express errors from middleware or routes (app.use is missing the type overload)
this.app.use((err: Error, req: express.Request, res: express.Response, next: express.NextFunction): void => {
const url = lib.redactUrl(req.originalUrl);
logger.log({
level: "error",
message: "Error while handling HTTP" +
`${req.httpVersion} ${req.method} ${req.originalUrl}:\n${err.stack}`,
`${req.httpVersion} ${req.method} ${url}:\n${err.stack}`,
meta: {
method: req.method,
url: req.originalUrl,
url,
httpVersion: req.httpVersion,
headers: Controller.loggableHeaders(req.headers),
query: req.query,
query: lib.redactQuery(req.query),
},
});
@@ -821,7 +828,7 @@ export default class Controller {
}
return permission;
});
return json;
return lib.Role.pluginMigrations.reduce(json);
});
}
@@ -889,18 +896,19 @@ export default class Controller {
const endNs = process.hrtime.bigint();
const durationMs = (Number(endNs - startNs) / 1e6);
endpointDurationSummary.labels(routePath).observe(durationMs / 1e3);
const url = lib.redactUrl(req.originalUrl);
logger.log({
level: "http",
message: `HTTP${req.httpVersion} ${req.method} ${req.originalUrl}` +
message: `HTTP${req.httpVersion} ${req.method} ${url}` +
` (Status ${res.statusCode} in ${durationMs}ms)`,
meta: {
method: req.method,
url: req.originalUrl,
url,
statusCode: res.statusCode,
responseTime: durationMs,
httpVersion: req.httpVersion,
headers: Controller.loggableHeaders(req.headers),
query: req.query,
query: lib.redactQuery(req.query),
},
});
});
+7
View File
@@ -116,6 +116,13 @@ export default class WsServer {
handleConnection(socket: lib.WebSocketClusterio, req: IncomingMessage) {
logger.verbose(`WsServer | new connection from ${this.remoteAddr(req)}`);
// ws emits errors like invalid frames on the socket, and an error with
// no listener crashes the controller. The connector adds its own
// listener, but only once the handshake has completed.
socket.on("error", (err: Error) => {
logger.verbose(`WsServer | error on connection from ${this.remoteAddr(req)}: ${err.message}`);
});
// Track statistics
wsConnectionsCounter.inc();
socket.on("message", (message: WebSocket.RawData) => {
@@ -144,11 +144,6 @@ export default class WsServerConnector extends lib.WebSocketBaseConnector {
}
});
this._socket.on("error", (err: any) => {
// It's assumed that close is always called by ws
logger.verbose("Connector | Error:", err);
});
this._socket.on("open", () => {
logger.verbose("Connector | Open");
});
+1 -1
View File
@@ -30,8 +30,8 @@ if (process.platform === "win32") {
const availablePlugins = [
{ name: "Global Chat", value: "@clusterio/plugin-global_chat" },
{ name: "Inventory Sync", value: "@clusterio/plugin-inventory_sync" },
{ name: "Player Auth", value: "@clusterio/plugin-player_auth" },
{ name: "Player Sync", value: "@clusterio/plugin-player_sync" },
{ name: "Research Sync", value: "@clusterio/plugin-research_sync" },
{ name: "Statistics Exporter", value: "@clusterio/plugin-statistics_exporter" },
{ name: "Subspace Storage", value: "@clusterio/plugin-subspace_storage" },
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@clusterio/create",
"version": "2.0.0-alpha.27",
"version": "2.0.1",
"description": "Installer for clusterio",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Hornwitser <github@hornwitser.no>",
-8
View File
@@ -327,14 +327,6 @@ async function startCtl() {
}
export function bootstrap() {
// eslint-disable-next-line no-console
console.warn(`
+==========================================================+
I WARNING: This is the development branch for the 2.0 I
I version of clusterio. Expect things to break. I
+==========================================================+
`
);
startCtl().catch(err => {
if (err.errors) {
logger.fatal(JSON.stringify(err.errors, null, "\t"));
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@clusterio/ctl",
"version": "2.0.0-alpha.27",
"version": "2.0.1",
"description": "Clusterio command line management tool",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Hornwitser <github@hornwitser.no>",
+2 -2
View File
@@ -119,9 +119,9 @@ By default the freeplay scenario from the latest Factorio install in `host.facto
* `--zip` Write the scenario as a zip file instead of a directory.
Factorio does not load zipped scenarios, extract it before use.
For example, creating a scenario with only the inventory_sync module:
For example, creating a scenario with only the player_sync module:
npx clusteriohost create-scenario ~/.factorio/scenarios/inventory_sync --plugins inventory_sync
npx clusteriohost create-scenario ~/.factorio/scenarios/player_sync --plugins player_sync
#### `config list`
+3 -8
View File
@@ -177,6 +177,9 @@ async function startHost() {
lib.registerPluginMessages(pluginInfos);
lib.registerPluginPermissions(pluginInfos);
lib.addPluginConfigFields(pluginInfos);
if (!args.recovery) {
await lib.loadPluginMigrations(pluginInfos);
}
const [hostConfig, hostConfigLock] = await lib.loadConfigFromArgs(
args,
@@ -264,14 +267,6 @@ async function startHost() {
}
export function bootstrap() {
// eslint-disable-next-line no-console
console.warn(`
+==========================================================+
I WARNING: This is the development branch for the 2.0 I
I version of clusterio. Expect things to break. I
+==========================================================+
`
);
startHost().catch(err => {
if (err instanceof lib.WebSocketError) {
// Already handled by connector.on("error") within Host
@@ -1,4 +1,29 @@
---------------------------------------------------------------------------------------------------
Version: 2.1.21
Date: 2026-10-14
Changes:
- Removed the obsolete alpha label and development warning from the mod title and description.
---------------------------------------------------------------------------------------------------
Version: 2.1.20
Date: 2026-10-14
Changes:
- Removed the obsolete alpha label and development warning from the mod title and description.
---------------------------------------------------------------------------------------------------
Version: 2.1.18
Date: 2026-10-14
Changes:
- Removed the obsolete alpha label and development warning from the mod title and description.
---------------------------------------------------------------------------------------------------
Version: 2.1.17
Date: 2026-10-14
Changes:
- Removed the obsolete alpha label and development warning from the mod title and description.
---------------------------------------------------------------------------------------------------
Version: 2.1.11
Date: 2026-10-14
Changes:
- Removed the obsolete alpha label and development warning from the mod title and description.
---------------------------------------------------------------------------------------------------
Version: 2.0.20
Date: 25. 02. 2025
Features:
+7 -7
View File
@@ -2,30 +2,30 @@
"name": "clusterio_lib",
"variants": [
{
"version": "2.0.17",
"version": "2.1.17",
"factorio_version": "0.17"
},
{
"version": "2.0.18",
"version": "2.1.18",
"factorio_version": "0.18"
},
{
"version": "2.0.11",
"version": "2.1.11",
"factorio_version": "1.1"
},
{
"version": "2.0.20",
"version": "2.1.20",
"factorio_version": "2.0"
},
{
"version": "2.0.21",
"version": "2.1.21",
"factorio_version": "2.1"
}
],
"title": "Clusterio Library (Alpha)",
"title": "Clusterio Library",
"author": "Clusterio Team",
"homepage": "https://github.com/clusterio/clusterio",
"description": "Library for interfacing with Clusterio. Warning: work in progress alpha.",
"description": "Library for interfacing with Clusterio.",
"additional_files": {
"compat.lua": ["..", "..", "modules", "clusterio", "compat.lua"],
"serialize.lua": ["..", "..", "modules", "clusterio", "serialize.lua"],
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@clusterio/host",
"version": "2.0.0-alpha.27",
"version": "2.0.1",
"description": "Implementation of Clusterio host server",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Hornwitser <github@hornwitser.no>",
+59 -3
View File
@@ -1,3 +1,4 @@
import dgram from "node:dgram";
import fs from "node:fs/promises";
import path from "path";
import pidusage from "pidusage";
@@ -25,6 +26,35 @@ function checkRequestSaveName(name: string) {
}
}
/**
* Checks that no socket is bound to the given UDP port, on IPv4 or IPv6
*
* Factorio binds the game port on both 0.0.0.0 and [::].
*
* @param port - UDP port to check.
* @returns true if the port can be bound.
* @internal
*/
async function udpPortFree(port: number) {
for (const type of ["udp4", "udp6"] as const) {
const socket = dgram.createSocket({ type, ipv6Only: type === "udp6" });
try {
await new Promise<void>((resolve, reject) => {
socket.once("error", reject);
socket.bind(port, resolve);
});
} catch (err: any) {
// Other errors, such as no IPv6 support on the host, don't stop Factorio from binding the port.
if (err.code === "EADDRINUSE") {
return false;
}
} finally {
socket.close();
}
}
return true;
}
/**
* Searches for instances in the provided directory
*
@@ -218,7 +248,7 @@ export class HostRouter {
this.host._connectInstance(dst.id).then(instanceConnection => {
instanceConnection.connector.forward(message);
}).catch(err => {
logger.error(`Error starting instance:\n${err.stack}`, this.host.instanceLogMeta(dst.id));
logger.error(`Error starting instance:\n${err.stack ?? err.message}`, this.host.instanceLogMeta(dst.id));
origin.connector.sendResponseError(
new lib.ResponseError(err.message, err.code, err.stack), message.src
);
@@ -791,8 +821,11 @@ export default class Host extends lib.Link {
);
}
assignGamePort(instanceId: number) {
assignGamePort(instanceId: number, inUse = new Set<number>()) {
const availablePorts = lib.parseRanges(this.config.get("host.factorio_port_range"), 1, 2 ** 16 - 1);
for (const port of inUse) {
availablePorts.delete(port);
}
for (const [id, instance] of this.assignedInstances) {
if (id === instanceId) {
continue;
@@ -811,13 +844,36 @@ export default class Host extends lib.Link {
return assignedPort;
}
if (!availablePorts.size) {
throw new lib.RequestError("No available port left to assign from host.factorio_port_range");
const others = inUse.size ? ` (${inUse.size} in use by other processes)` : "";
throw new lib.RequestError(`No available port left to assign from host.factorio_port_range${others}`);
}
const [newPort] = availablePorts;
instanceToAssign.config.set("factorio.host_assigned_game_port", newPort);
return newPort;
}
/**
* Assign a game port that no other process is bound to
*
* Replaces the stored port if it is taken, so a port another process
* holds is not reused on every start.
*
* @param instanceId - ID of the instance to assign a game port to.
* @returns the assigned port.
*/
async assignFreeGamePort(instanceId: number) {
const inUse = new Set<number>();
for (;;) {
// Picking and storing the port before checking it keeps concurrent assignments from picking the same one.
const port = this.assignGamePort(instanceId, inUse);
if (await udpPortFree(port)) {
return port;
}
logger.warn(`Game port ${port} is in use by another process`, this.instanceLogMeta(instanceId));
inUse.add(port);
}
}
/**
* Initialize and connect an unloaded instance
*
+3
View File
@@ -678,6 +678,9 @@ end`.replace(/\r?\n/g, " ");
// After changing the status this function is responsible for calling notifyExit if it throws.
this.notifyStatus("starting");
try {
if (this.config.get("factorio.game_port") === null) {
this.server.gamePort = await this._host.assignFreeGamePort(this.id);
}
const factorioVersions = await this.sendTo("controller", new lib.FactorioVersionsRequest());
await this._loadStats();
// Resolve a release channel target (e.g. "stable") to a concrete
+1
View File
@@ -1,5 +1,6 @@
export * from "./src/api";
export * from "./src/AsyncHook";
export * from "./src/SyncHook";
export * from "./src/config";
export * from "./src/data";
export * from "./src/errors";
+2
View File
@@ -37,6 +37,8 @@ export * from "./src/load_plugin_list.js";
export * from "./src/rce_ops.js";
export * from "./src/LockFile.js";
export * from "./src/AsyncHook.js";
export * from "./src/SyncHook.js";
export * from "./src/migrations.js";
export { default as ExponentialBackoff } from "./src/ExponentialBackoff.js";
export { default as ModStore } from "./src/ModStore.js";
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@clusterio/lib",
"version": "2.0.0-alpha.27",
"version": "2.0.1",
"description": "Shared library for Clusterio",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Hornwitser <github@hornwitser.no>",
+69
View File
@@ -0,0 +1,69 @@
import { PluginError } from "./errors.js";
export type SyncHookHandler<Value> = (value: Value) => Value;
/**
* Hook that passes a value through each attached handler in turn.
*
* Handlers run synchronously in the order they were attached, for places
* that can't wait on a promise such as the Config constructor.
*/
export class SyncHook<Value> {
private _handlers = new Map<string, SyncHookHandler<Value>>();
get size() {
return this._handlers.size;
}
get attached() {
return this._handlers.keys();
}
/**
* Attach a handler to this hook.
*
* An error is thrown if another handler with the same name is already attached.
*/
attach(name: string, handler: SyncHookHandler<Value>) {
if (this._handlers.has(name)) {
throw new Error(`Handler with name ${name} is already attached`);
}
this._handlers.set(name, handler);
}
/**
* Remove a previously attached handler.
*
* If no handler exists, then no action is performed.
*/
detach(name: string) {
this._handlers.delete(name);
}
/**
* Remove all attached handlers.
*/
clear() {
this._handlers.clear();
}
/**
* Pass a value through all attached handlers.
*
* Each handler gets the value returned by the one before it. Errors are
* rethrown as a PluginError named after the handler that threw.
*
* @param value - Value given to the first handler.
* @returns Value returned by the last handler.
*/
reduce(value: Value) {
for (const [name, handler] of this._handlers) {
try {
value = handler(value);
} catch (err: any) {
throw new PluginError(name, err instanceof Error ? err : new Error(String(err)));
}
}
return value;
}
}
+41 -7
View File
@@ -10,6 +10,7 @@ import { StringEnum } from "../data/composites.js";
import { safeOutputFile } from "../file_ops.js";
import * as validators from "./validators.js";
import { RequestError } from "../errors.js";
import type { SyncHook } from "../SyncHook.js";
const ConfigLocation = StringEnum(["controller", "host", "control"]);
export type ConfigLocation = Static<typeof ConfigLocation>;
@@ -224,6 +225,18 @@ export class Config<
return config;
}
/**
* Migrations attached by plugins, run after {@link Config.migrations}.
* Declared by each subclass that plugins can add fields to.
*/
declare static pluginMigrations?: SyncHook<ConfigSchema>;
/**
* Versions of the plugins whose migrations are attached, keyed by
* plugin name. Written to `<plugin>.version` after the migrations run.
*/
static pluginVersions = new Map<string, string>();
/** Defines the default access locations for the config class */
static defaultAccess: ConfigLocation[] = ["controller", "host", "control"];
@@ -234,10 +247,15 @@ export class Config<
/** Set to true during validators to prevent edits */
private _readonly = false; // TODO replace with "using config.asReadonly()"
/** Set to true when a field in the config is changed. */
private dirty = false;
private _dirty = false;
/** Set to true when a 'restart required' field in the config is changed. */
restartRequired = false;
/** True if the config has changes that have not been saved. */
get dirty() {
return this._dirty;
}
/**
* Create a new instance of the given config
*
@@ -269,7 +287,21 @@ export class Config<
) as Fields;
if (fields) {
this.update(this.constructor.migrations(fields), false, location);
let migrated = this.constructor.migrations(fields);
if (this.constructor.pluginMigrations) {
migrated = this.constructor.pluginMigrations.reduce(migrated);
}
this.update(migrated, false, location);
}
for (const [pluginName, version] of this.constructor.pluginVersions) {
const name = `${pluginName}.version` as keyof Fields & string;
if (name in this.fields && this.fields[name] !== version) {
this.fields[name] = version as Fields[typeof name];
if (fields) {
this._dirty = true;
}
}
}
}
@@ -278,7 +310,7 @@ export class Config<
}
set filepath(filepath: string | undefined) {
this.dirty = true;
this._dirty = true;
this._filepath = filepath;
}
@@ -289,7 +321,9 @@ export class Config<
// TODO replace with "using config.startStating()"
get staging(): Config<Fields> {
if (this._staging) { return this._staging.config; }
const config = new this.constructor(this.location, this.fields);
// Copied with update so migrations don't run again on the live fields
const config = new this.constructor(this.location);
config.update(this.fields, false);
const changes = new Set<string>();
config.on("fieldChanged", name => changes.add(name));
this._staging = { config, changes };
@@ -443,7 +477,7 @@ export class Config<
if (def.restartRequired) {
this.restartRequired = true;
}
this.dirty = true;
this._dirty = true;
this.emit("fieldChanged", name, value, prev);
}
}
@@ -465,14 +499,14 @@ export class Config<
* @param force - Set to true to save the file even if no changes have been made.
*/
async save(force = false) {
if (!force && !this.dirty) {
if (!force && !this._dirty) {
return;
}
if (!this.filepath) {
throw new Error("Cannot save config which has no filepath");
}
await safeOutputFile(this.filepath, JSON.stringify(this, null, "\t"), { mode: 0o600 });
this.dirty = false;
this._dirty = false;
}
static jsonSchema = ConfigSchema;
+20
View File
@@ -4,6 +4,7 @@ import * as validators from "./validators.js";
import type { PluginNodeEnvInfo, PluginWebpackEnvInfo } from "../plugin.js";
import { Static } from "@sinclair/typebox";
import { isTargetVersion } from "../data/version.js";
import { SyncHook } from "../SyncHook.js";
type configFromJSON<T> = (...args: Parameters<typeof classes.Config.fromJSON>) => T;
type configFromFile<T> = (...args: Parameters<typeof classes.Config.fromFile>) => Promise<T>;
@@ -42,6 +43,7 @@ export interface ControllerConfigFields {
"controller.allow_plugin_updates": boolean;
"controller.allow_plugin_install": boolean;
[key: `${string}.load_plugin`]: boolean;
[key: `${string}.version`]: string;
}
/**
@@ -51,6 +53,7 @@ export interface ControllerConfigFields {
export class ControllerConfig extends classes.Config<ControllerConfigFields> {
declare static fromJSON: configFromJSON<ControllerConfig>;
declare static fromFile: configFromFile<ControllerConfig>;
static pluginMigrations = new SyncHook<classes.ConfigSchema>();
static migrations(config: Static<typeof this.jsonSchema>) {
if (config.hasOwnProperty("controller.external_address")) {
config["controller.public_url"] = config["controller.external_address"];
@@ -334,6 +337,7 @@ export interface HostConfigFields {
"host.allow_plugin_updates": boolean;
"host.allow_plugin_install": boolean;
[key: `${string}.load_plugin`]: boolean;
[key: `${string}.version`]: string;
}
/**
@@ -343,6 +347,7 @@ export interface HostConfigFields {
export class HostConfig extends classes.Config<HostConfigFields> {
declare static fromJSON: configFromJSON<HostConfig>;
declare static fromFile: configFromFile<HostConfig>;
static pluginMigrations = new SyncHook<classes.ConfigSchema>();
static fieldDefinitions: classes.ConfigDefs<HostConfigFields> = {
"host.version": {
type: "string",
@@ -468,6 +473,7 @@ export interface InstanceConfigFields {
"instance.auto_start": boolean;
"instance.exclude_from_start_all": boolean;
[key: `${string}.load_plugin`]: boolean;
[key: `${string}.version`]: string;
"factorio.version": string;
"factorio.executable_path": string | null;
@@ -500,6 +506,7 @@ export interface InstanceConfigFields {
export class InstanceConfig extends classes.Config<InstanceConfigFields> {
declare static fromJSON: configFromJSON<InstanceConfig>;
declare static fromFile: configFromFile<InstanceConfig>;
static pluginMigrations = new SyncHook<classes.ConfigSchema>();
static migrations(config: Static<typeof this.jsonSchema>) {
function boolToEnableDisable(name: string) {
if (config.hasOwnProperty(name) && typeof config[name] === "boolean") {
@@ -805,6 +812,14 @@ export function addPluginFieldDefinitions(
Config.fieldDefinitions[`${pluginInfo.name}.load_plugin`] = fieldDef;
// Version of the plugin the config was last loaded with, see Config.pluginVersions
Config.fieldDefinitions[`${pluginInfo.name}.version`] = {
type: "string",
initialValue: "",
readonly: ["controller", "host"],
hidden: true,
};
// Merge in other fields from the plugin
const fields = pluginInfo[kind];
if (fields) {
@@ -827,5 +842,10 @@ export function addPluginConfigFields(pluginInfos: PluginNodeEnvInfo[] | PluginW
if (pluginInfo.instanceEntrypoint) {
addPluginFieldDefinitions(pluginInfo, "instanceConfigFields", InstanceConfig);
}
if (pluginInfo.ctlConfigFields) {
validateFieldNames(pluginInfo.name, pluginInfo.ctlConfigFields);
Object.assign(CtlConfig.fieldDefinitions, pluginInfo.ctlConfigFields);
}
}
}
+4
View File
@@ -1,5 +1,6 @@
import { Static, Type } from "@sinclair/typebox";
import { permissions as globalPermissions } from "../permissions.js";
import { SyncHook } from "../SyncHook.js";
interface RoleCollection {
get(id: number): Role;
@@ -31,6 +32,9 @@ export default class Role {
is_deleted: Type.Optional(Type.Boolean()),
});
/** Migrations attached by plugins, run on roles loaded by the controller */
static pluginMigrations = new SyncHook<Static<typeof Role.jsonSchema>>();
static fromJSON(json: Static<typeof this.jsonSchema>) {
return new this(
json.id,
+31 -5
View File
@@ -277,6 +277,8 @@ const sensitiveSearchParams = new Set([
"access_token",
"api_key",
"apikey",
// OAuth2 authorization code, plugins doing OAuth receive it on their callback route
"code",
"password",
"secret",
"token",
@@ -290,19 +292,25 @@ const sensitiveSearchParams = new Set([
* safely be included in log output and error messages. The parameter
* names are kept intact to preserve the diagnostic value of the URL.
*
* Returns the input unchanged if it is a string which does not parse as
* a URL, as there are then no query parameters to redact.
* Accepts a path with a query string, like the url of an incoming HTTP
* request, and returns a path in that case. Returns the input unchanged
* if it is a string which does not parse as a URL, as there are then no
* query parameters to redact.
*
* @param url - URL to redact credentials from.
* @param url - URL or path to redact credentials from.
* @returns url with credentials replaced by `REDACTED`.
*/
export function redactUrl(url: URL | string) {
const isPath = typeof url === "string" && url.startsWith("/") && !url.startsWith("//");
let parsed;
try {
parsed = new URL(url);
parsed = isPath ? new URL(url, "http://localhost") : new URL(url);
} catch {
return String(url);
}
if (![...parsed.searchParams.keys()].some(name => sensitiveSearchParams.has(name.toLowerCase()))) {
return String(url);
}
const searchParams = new URLSearchParams();
for (const [name, value] of parsed.searchParams) {
@@ -310,7 +318,25 @@ export function redactUrl(url: URL | string) {
}
parsed.search = searchParams.toString();
return parsed.toString();
return isPath ? parsed.pathname + parsed.search + parsed.hash : parsed.toString();
}
/**
* Copy of parsed query parameters with credentials replaced by `REDACTED`
*
* Uses the same parameter names as {@link redactUrl}.
*
* @param query - query parameters, for example `req.query` from express.
* @returns query safe to write to the log.
*/
export function redactQuery(query: Record<string, unknown>) {
const result = { ...query };
for (const name of Object.keys(result)) {
if (sensitiveSearchParams.has(name.toLowerCase())) {
result[name] = "REDACTED";
}
}
return result;
}
function skipWhitespace(pos: number, input: string) {
+79
View File
@@ -0,0 +1,79 @@
/**
* Helpers for plugin migrations
* @module lib/migrations
*/
import type { Static } from "@sinclair/typebox";
import type { ConfigSchema } from "./config/classes.js";
import type Role from "./data/Role.js";
type SerializedRole = Static<typeof Role.jsonSchema>;
/**
* Rename a field in a serialized config
*
* The value is moved if the old field exists and the new one doesn't. The
* old field is removed either way.
*
* @param config - Serialized config to update.
* @param from - Current name of the field.
* @param to - New name of the field.
* @returns The config passed in.
*/
export function renameConfigField(config: ConfigSchema, from: string, to: string) {
if (Object.hasOwn(config, from)) {
if (!Object.hasOwn(config, to)) {
config[to] = config[from];
}
delete config[from];
}
return config;
}
/**
* Rename all fields starting with a prefix in a serialized config
*
* @example
* lib.renameConfigPrefix(config, "inventory_sync.", "player_sync.");
*
* @param config - Serialized config to update.
* @param from - Current prefix of the fields.
* @param to - New prefix of the fields.
* @returns The config passed in.
*/
export function renameConfigPrefix(config: ConfigSchema, from: string, to: string) {
for (const name of Object.keys(config)) {
if (name.startsWith(from)) {
renameConfigField(config, name, to + name.slice(from.length));
}
}
return config;
}
/**
* Rename a permission in a serialized role
*
* @param role - Serialized role to update.
* @param from - Current name of the permission.
* @param to - New name of the permission.
* @returns The role passed in.
*/
export function renamePermission(role: SerializedRole, from: string, to: string) {
return renamePermissionsWith(role, name => (name === from ? to : name));
}
/**
* Rename all permissions starting with a prefix in a serialized role
*
* @param role - Serialized role to update.
* @param from - Current prefix of the permissions.
* @param to - New prefix of the permissions.
* @returns The role passed in.
*/
export function renamePermissionPrefix(role: SerializedRole, from: string, to: string) {
return renamePermissionsWith(role, name => (name.startsWith(from) ? to + name.slice(from.length) : name));
}
function renamePermissionsWith(role: SerializedRole, rename: (name: string) => string) {
role.permissions = [...new Set(role.permissions.map(rename))];
return role;
}
+16 -2
View File
@@ -2,10 +2,12 @@
* Plugin interfaces and utilities.
* @module lib/plugin
*/
import type { Static } from "@sinclair/typebox";
import type { Logger } from "./logging.js";
import type { FieldDefinition } from "./config/index.js";
import type { ConfigSchema, FieldDefinition } from "./config/index.js";
import type { PermissionDefinition } from "./permissions.js";
import type { PlayerStats } from "./data/index.js";
import type { PlayerStats, Role } from "./data/index.js";
import type { SyncHook } from "./SyncHook.js";
export const PluginFeatureFlags = [
/** The plugin requires module code to be patched into the save */
@@ -33,6 +35,8 @@ export type PluginDeclaration = {
webEntrypoint?: string;
ctlConfigFields?: Record<string, FieldDefinition>;
migrationEntrypoint?: string;
features?: (typeof PluginFeatureFlags)[number][];
messages?: any[];
@@ -114,6 +118,16 @@ export type PluginLoadContext<
plugin: Info;
};
/** Hooks for migrating data that Clusterio stores on behalf of plugins */
export type MigrationHooks = {
controllerConfig: SyncHook<ConfigSchema>;
hostConfig: SyncHook<ConfigSchema>;
instanceConfig: SyncHook<ConfigSchema>;
role: SyncHook<Static<typeof Role.jsonSchema>>;
};
export type MigrationContext = PluginLoadContext<{ hooks: MigrationHooks }>;
export type PluginClass<
Context extends object,
Info extends PluginDeclaration,
+43
View File
@@ -10,6 +10,9 @@ import * as libErrors from "./errors.js";
import * as libPlugin from "./plugin.js";
import { type Logger, logger } from "./logging.js";
import { loadPluginEntrypoint, loadPluginClass } from "./loadPlugin.js";
import { Config } from "./config/classes.js";
import { ControllerConfig, HostConfig, InstanceConfig } from "./config/definitions.js";
import Role from "./data/Role.js";
/**
@@ -90,6 +93,46 @@ export async function loadPluginInfos(pluginList: Map<string, string>) {
return plugins;
}
/**
* Load the migration entrypoints of plugins
*
* Must be called before configs and roles are loaded for the migrations
* to run on them. Loaded for every plugin, including those disabled with
* load_plugin. Also registers the plugin versions written to each config
* after the migrations run.
*
* @param pluginInfos - Plugins to load migrations for.
*/
export async function loadPluginMigrations(pluginInfos: libPlugin.PluginNodeEnvInfo[]) {
const hooks: libPlugin.MigrationHooks = {
controllerConfig: ControllerConfig.pluginMigrations,
hostConfig: HostConfig.pluginMigrations,
instanceConfig: InstanceConfig.pluginMigrations,
role: Role.pluginMigrations,
};
for (const pluginInfo of pluginInfos) {
Config.pluginVersions.set(pluginInfo.name, pluginInfo.version);
if (!pluginInfo.migrationEntrypoint) {
continue;
}
try {
const module = await import(pathToFileURL(
path.join(path.dirname(pluginInfo.packagePath), pluginInfo.migrationEntrypoint)
).href);
const context: libPlugin.MigrationContext = {
hooks,
plugin: pluginInfo,
logger: logger.child({ plugin: pluginInfo.name }),
};
await loadPluginEntrypoint(pluginInfo, "migration", context, module);
} catch (err: any) {
throw new libErrors.PluginError(pluginInfo.name, err);
}
}
}
export async function loadPlugin<
Context extends { logger: Logger },
Class extends libPlugin.PluginClass<Context, libPlugin.PluginNodeEnvInfo>,
+1
View File
@@ -5,6 +5,7 @@
"package.json",
"./src/api.ts",
"./src/AsyncHook.ts",
"./src/SyncHook.ts",
"./src/config/*.ts",
"./src/data/*.ts",
"./src/external/*.ts",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@clusterio/web_ui",
"version": "2.0.0-alpha.27",
"version": "2.0.1",
"description": "Clusterio web interface implementation",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Hornwitser",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@clusterio/plugin-global_chat",
"version": "2.0.0-alpha.27",
"version": "2.0.1",
"description": "Clusterio plugin forwarding between Factorio servers",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Hornwitser <github@hornwitser.no>",
@@ -1,22 +0,0 @@
local compat = require("modules/clusterio/compat")
--- @param no_early_return boolean? True to always setup data
return function(no_early_return)
local inventory_sync = compat.script_data.inventory_sync
if inventory_sync and not no_early_return then
return inventory_sync
end
inventory_sync = inventory_sync or {}
inventory_sync.players = inventory_sync.players or {}
inventory_sync.players_waiting_for_acquire = inventory_sync.players_waiting_for_acquire or {}
inventory_sync.players_in_cutscene_to_sync = inventory_sync.players_in_cutscene_to_sync or {}
inventory_sync.failed_deserialization = inventory_sync.failed_deserialization or {}
inventory_sync.active_downloads = inventory_sync.active_downloads or {}
inventory_sync.finished_downloads = inventory_sync.finished_downloads or {}
inventory_sync.active_uploads = inventory_sync.active_uploads or {}
compat.script_data.inventory_sync = inventory_sync
return inventory_sync
end
@@ -1,4 +0,0 @@
{
"name": "inventory_sync",
"load": ["inventory_sync.lua", "gui/dialog_failed_download.lua"]
}
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@clusterio/plugin-player_auth",
"version": "2.0.0-alpha.27",
"version": "2.0.1",
"description": "Clusterio plugin authenticating logged in players to the web interface",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Hornwitser <github@hornwitser.no>",
File renamed without changes.
@@ -1,6 +1,6 @@
# Clusterio inventory sync plugin
# Clusterio player sync plugin
Carry over player inventory between servers
Carry over player inventories and settings between servers
As a player, it mostly just works. Most important to know is that factorio data transfer is slow, which means if you have "big" items (blueprints mostly) in your inventory it will take a long time to transfer when joining servers. For this reason it is recommended to keep blueprints in the blueprint library.
@@ -12,11 +12,24 @@ When a server crashes while you are online there won't be time to immediately up
Run the following commands in the folder Clusterio is installed to:
npm install @clusterio/plugin-inventory_sync
npx clusteriocontroller plugin add @clusterio/plugin-inventory_sync
npm install @clusterio/plugin-player_sync
npx clusteriocontroller plugin add @clusterio/plugin-player_sync
Substitute clusteriocontroller with clusteriohost or clusterioctl if this a dedicated host or ctl installation respectively.
## Upgrading from inventory_sync
This plugin was called inventory_sync before 2.0.0-alpha.28.
Remove the old package and install the new one in its place:
npm uninstall @clusterio/plugin-inventory_sync
npm install @clusterio/plugin-player_sync
npx clusteriocontroller plugin remove inventory_sync
npx clusteriocontroller plugin add @clusterio/plugin-player_sync
The `inventory_sync.*` config fields and the `inventory_sync.inventory.view` permission are renamed on load.
The controller moves `inventories.json` in the database directory to `player_sync.json`, and saves keep their player data the next time they are started with the new module.
## Method of operation
This plugin does event based synchronization of inventories.
@@ -47,7 +60,7 @@ Overall, we are able to achieve a latency between 3 and infinite ticks from serv
## Controller Configuration
### inventory_sync.player_lock_timeout
### player_sync.player_lock_timeout
Time in seconds before the exclusive access an instance holds on a player's inventory expires after the instance stops or its host disconnects.
Until it expires the player is told the inventory is in use on that instance when joining another one.
@@ -58,7 +71,7 @@ Defaults to `60`.
## Instance Configuration
### inventory_sync.rcon_chunk_size
### player_sync.rcon_chunk_size
Size in characters of the chunks the serialized inventory is split into when sent to the game over RCON.
Smaller chunks take more commands to transfer an inventory but hold up the RCON command pipe for less time each.
@@ -7,7 +7,20 @@ import * as lib from "@clusterio/lib";
import * as msg from "./messages.js";
async function loadDatabase(config: lib.ControllerConfig, logger: lib.Logger) {
let itemsPath = path.resolve(config.get("controller.database_directory"), "inventories.json");
let databaseDirectory = config.get("controller.database_directory");
let itemsPath = path.resolve(databaseDirectory, "player_sync.json");
// migrate: inventory_sync stored its data in inventories.json before alpha 28
let legacyPath = path.resolve(databaseDirectory, "inventories.json");
if (!await fs.access(itemsPath).then(() => true, () => false)) {
try {
await fs.rename(legacyPath, itemsPath);
logger.info(`Moved ${legacyPath} to ${itemsPath}`);
} catch (err: any) {
if (err.code !== "ENOENT") {
throw err;
}
}
}
logger.verbose(`Loading ${itemsPath}`);
try {
let content = await fs.readFile(itemsPath, { encoding: "utf8" });
@@ -28,7 +41,7 @@ async function saveDatabase(
logger: lib.Logger,
) {
if (playerDatastore) {
let file = path.resolve(controllerConfig.get("controller.database_directory"), "inventories.json");
let file = path.resolve(controllerConfig.get("controller.database_directory"), "player_sync.json");
logger.verbose(`writing ${file}`);
let content = JSON.stringify(Array.from(playerDatastore));
await lib.safeOutputFile(file, content);
@@ -67,7 +80,7 @@ export default async function(context: ControllerPluginContext) {
}
if (["unknown", "stopped"].includes(instance.status)) {
let timeoutMs = controller.config.get("inventory_sync.player_lock_timeout") * 1000;
let timeoutMs = controller.config.get("player_sync.player_lock_timeout") * 1000;
for (let acquisitonRecord of acquiredPlayers.values()) {
if (acquisitonRecord.instanceId === instanceId && !acquisitonRecord.expiresMs) {
acquisitonRecord.expiresMs = Date.now() + timeoutMs;
@@ -3,24 +3,25 @@ import * as messages from "./messages.js";
declare module "@clusterio/lib" {
export interface InstanceConfigFields {
"inventory_sync.rcon_chunk_size": number;
"player_sync.rcon_chunk_size": number;
}
export interface ControllerConfigFields {
"inventory_sync.player_lock_timeout": number;
"player_sync.player_lock_timeout": number;
}
export interface Permissions {
"inventory_sync.inventory.view": never;
"player_sync.inventory.view": never;
}
}
export const plugin: lib.PluginDeclaration = {
name: "inventory_sync",
title: "Inventory sync",
description: "Synchronizes players inventories between instances",
name: "player_sync",
title: "Player sync",
description: "Synchronizes player inventories and settings between instances",
migrationEntrypoint: "dist/node/migrations.js",
instanceEntrypoint: "dist/node/instance.js",
instanceConfigFields: {
"inventory_sync.rcon_chunk_size": {
"player_sync.rcon_chunk_size": {
title: "Rcon inventory chunk size",
description:
"Divide inventories into chunks of this size before sending with rcon to prevent blocking the pipe",
@@ -31,7 +32,7 @@ export const plugin: lib.PluginDeclaration = {
controllerEntrypoint: "dist/node/controller.js",
controllerConfigFields: {
"inventory_sync.player_lock_timeout": {
"player_sync.player_lock_timeout": {
title: "Player Lock Timeout",
description:
"Time in seconds before the lock on a player inventory expires after an instance stops " +
@@ -55,7 +56,7 @@ export const plugin: lib.PluginDeclaration = {
],
permissions: [
{
name: "inventory_sync.inventory.view",
name: "player_sync.inventory.view",
title: "View player inventories",
description: "View player inventories",
grantByDefault: true,
@@ -136,7 +136,7 @@ export default async function(context: InstancePluginContext) {
}
let json = lib.escapeString(JSON.stringify(response));
await instance.sendRcon(`/sc inventory_sync.acquire_response("${json}")`, true, plugin.name);
await instance.sendRcon(`/sc player_sync.acquire_response("${json}")`, true, plugin.name);
}
async function handleRelease(request: IpcPlayerName) {
@@ -178,7 +178,7 @@ export default async function(context: InstancePluginContext) {
}
await instance.sendRcon(
`/sc inventory_sync.confirm_upload("${player_data.name}", ${player_data.generation})`, true, plugin.name
`/sc player_sync.confirm_upload("${player_data.name}", ${player_data.generation})`, true, plugin.name
);
}
@@ -193,20 +193,20 @@ export default async function(context: InstancePluginContext) {
if (!response.playerData) {
await instance.sendRcon(
`/sc inventory_sync.download_inventory('${playerName}',nil,0,0)`, true, plugin.name
`/sc player_sync.download_inventory('${playerName}',nil,0,0)`, true, plugin.name
);
return;
}
await applyRecipeNotificationDelta(response.playerData, request.recipe_notifications, logger);
const chunkSize = instance.config.get("inventory_sync.rcon_chunk_size");
const chunkSize = instance.config.get("player_sync.rcon_chunk_size");
const chunks = chunkify(chunkSize, JSON.stringify(response.playerData));
logger.verbose(`Sending inventory for ${playerName} in ${chunks.length} chunks`);
for (let i = 0; i < chunks.length; i++) {
const chunk = lib.escapeString(chunks[i]);
await instance.sendRcon(
`/sc inventory_sync.download_inventory('${playerName}','${chunk}',${i + 1},${chunks.length})`,
`/sc player_sync.download_inventory('${playerName}','${chunk}',${i + 1},${chunks.length})`,
true,
plugin.name,
);
@@ -215,27 +215,27 @@ export default async function(context: InstancePluginContext) {
// Handle IPC from scenario script
instance.server.on(
"ipc-inventory_sync_acquire",
"ipc-player_sync_acquire",
(request: IpcPlayerName) => handleAcquire(request).catch(
err => logger.error(`Error handling ipc-inventory_sync_acquire:\n${err.stack}`)
err => logger.error(`Error handling ipc-player_sync_acquire:\n${err.stack}`)
),
);
instance.server.on(
"ipc-inventory_sync_release",
"ipc-player_sync_release",
(request: IpcPlayerName) => handleRelease(request).catch(
err => logger.error(`Error handling ipc-inventory_sync_release:\n${err.stack}`)
err => logger.error(`Error handling ipc-player_sync_release:\n${err.stack}`)
),
);
instance.server.on(
"ipc-inventory_sync_upload",
"ipc-player_sync_upload",
(player_data: IpcPlayerData) => handleUpload(player_data).catch(
err => logger.error(`Error handling ipc-inventory_sync_upload:\n${err.stack}`)
err => logger.error(`Error handling ipc-player_sync_upload:\n${err.stack}`)
),
);
instance.server.on(
"ipc-inventory_sync_download",
"ipc-player_sync_download",
(request: IpcDownloadRequest) => handleDownload(request).catch(
err => logger.error(`Error handling ipc-inventory_sync_download:\n${err.stack}`)
err => logger.error(`Error handling ipc-player_sync_download:\n${err.stack}`)
),
);
@@ -53,7 +53,7 @@ export class AcquireRequest {
static type = "request" as const;
static src = "instance" as const;
static dst = "controller" as const;
static plugin = "inventory_sync" as const;
static plugin = "player_sync" as const;
static Response = AcquireResponse;
constructor(
@@ -77,7 +77,7 @@ export class ReleaseRequest {
static type = "request" as const;
static src = "instance" as const;
static dst = "controller" as const;
static plugin = "inventory_sync" as const;
static plugin = "player_sync" as const;
constructor(
public instanceId: number,
@@ -100,7 +100,7 @@ export class UploadRequest {
static type = "request" as const;
static src = "instance" as const;
static dst = "controller" as const;
static plugin = "inventory_sync" as const;
static plugin = "player_sync" as const;
constructor(
public instanceId: number,
@@ -140,7 +140,7 @@ export class DownloadRequest {
static type = "request" as const;
static src = "instance" as const;
static dst = "controller" as const;
static plugin = "inventory_sync" as const;
static plugin = "player_sync" as const;
static Response = DownloadResponse;
constructor(
@@ -186,7 +186,7 @@ export class DatabaseStatsRequest {
static type = "request" as const;
static src = "control" as const;
static dst = "controller" as const;
static plugin = "inventory_sync" as const;
static permission = "inventory_sync.inventory.view" as const;
static plugin = "player_sync" as const;
static permission = "player_sync.inventory.view" as const;
static Response = DatabaseStatsResponse;
}
+13
View File
@@ -0,0 +1,13 @@
import * as lib from "@clusterio/lib";
export default function(context: lib.MigrationContext) {
const { hooks, plugin } = context;
// migrate: inventory_sync was renamed to player_sync in 2.0.0-alpha.28
const renameFields = (config: lib.ConfigSchema) => (
lib.renameConfigPrefix(config, "inventory_sync.", "player_sync.")
);
hooks.controllerConfig.attach(plugin.name, renameFields);
hooks.hostConfig.attach(plugin.name, renameFields);
hooks.instanceConfig.attach(plugin.name, renameFields);
hooks.role.attach(plugin.name, role => lib.renamePermissionPrefix(role, "inventory_sync.", "player_sync."));
}
@@ -1,7 +1,7 @@
local progress_dialog = require("modules/inventory_sync/gui/progress_dialog")
local ensure_character = require("modules/inventory_sync/ensure_character")
local restore_position = require("modules/inventory_sync/restore_position")
local get_script_data = require("modules/inventory_sync/get_script_data")
local progress_dialog = require("modules/player_sync/gui/progress_dialog")
local ensure_character = require("modules/player_sync/ensure_character")
local restore_position = require("modules/player_sync/restore_position")
local get_script_data = require("modules/player_sync/get_script_data")
local function download_inventory(player_name, data, number, total)
@@ -21,14 +21,14 @@ local function download_inventory(player_name, data, number, total)
local player_record = script_data.players[player_name]
if record.restart then
rcon.print("Restarting outdated download")
inventory_sync.initiate_inventory_download(player, player_record, record.generation)
player_sync.initiate_inventory_download(player, player_record, record.generation)
return
end
if total == 0 then
-- Download was requested but the player had no data stored.
log("ERROR: Inventory sync failed, got empty player data for " .. player.name .. " from the controller")
player.print("ERROR: Inventory sync failed, got empty player data from the controller")
log("ERROR: Player sync failed, got empty player data for " .. player.name .. " from the controller")
player.print("ERROR: Player sync failed, got empty player data from the controller")
progress_dialog.remove(player)
@@ -70,7 +70,7 @@ local function download_inventory(player_name, data, number, total)
log("Imported player data for " .. player_name .. " in " .. ticks .. " ticks")
if player.connected then
inventory_sync.finish_download(player, record)
player_sync.finish_download(player, record)
end
end
@@ -0,0 +1,28 @@
local compat = require("modules/clusterio/compat")
--- @param no_early_return boolean? True to always setup data
return function(no_early_return)
local player_sync = compat.script_data.player_sync
if player_sync and not no_early_return then
return player_sync
end
-- migrate: inventory_sync was renamed to player_sync in 2.0.0-alpha.28
if not player_sync and compat.script_data.inventory_sync then
player_sync = compat.script_data.inventory_sync
compat.script_data.inventory_sync = nil
end
player_sync = player_sync or {}
player_sync.players = player_sync.players or {}
player_sync.players_waiting_for_acquire = player_sync.players_waiting_for_acquire or {}
player_sync.players_in_cutscene_to_sync = player_sync.players_in_cutscene_to_sync or {}
player_sync.failed_deserialization = player_sync.failed_deserialization or {}
player_sync.active_downloads = player_sync.active_downloads or {}
player_sync.finished_downloads = player_sync.finished_downloads or {}
player_sync.active_uploads = player_sync.active_uploads or {}
compat.script_data.player_sync = player_sync
return player_sync
end
@@ -1,5 +1,5 @@
local ensure_character = require("modules/inventory_sync/ensure_character")
local get_script_data = require("modules/inventory_sync/get_script_data")
local ensure_character = require("modules/player_sync/ensure_character")
local get_script_data = require("modules/player_sync/get_script_data")
local dialog_failed_download = {}
@@ -21,7 +21,7 @@ function dialog_failed_download.create(player, acquire_response)
}
titlebar.drag_target = frame
local title_caption = "Inventory sync download failed"
local title_caption = "Player sync download failed"
if acquire_response.status == "busy" then
title_caption = "Inventory is in use on " .. acquire_response.message
end
@@ -43,7 +43,7 @@ function dialog_failed_download.create(player, acquire_response)
filler.drag_target = frame
local content = frame.add {
name = "inventory_sync_failed_download_content",
name = "player_sync_failed_download_content",
type = "frame",
direction = "vertical",
style = "inside_shallow_frame_with_padding",
@@ -90,7 +90,7 @@ function dialog_failed_download.create(player, acquire_response)
dialog_row.drag_target = frame
local abort = dialog_row.add {
name = "inventory_sync_failed_download_abort",
name = "player_sync_failed_download_abort",
type = "button",
style = "back_button",
caption = "Use temporary inventory",
@@ -107,7 +107,7 @@ function dialog_failed_download.create(player, acquire_response)
dialog_filler.drag_target = frame
local retry = dialog_row.add {
name = "inventory_sync_failed_download_retry",
name = "player_sync_failed_download_retry",
type = "button",
style = "forward_button",
caption = "Retry",
@@ -123,19 +123,19 @@ dialog_failed_download.events = {
return
end
if event.element.name == "inventory_sync_failed_download_abort" then
if event.element.name == "player_sync_failed_download_abort" then
-- Give the player a character and let them play with that
local player = assert(game.get_player(event.player_index))
ensure_character(player)
local inventory_sync = get_script_data()
local player_sync = get_script_data()
player.gui.screen.dialog_failed_download.destroy()
inventory_sync.players[player.name].dirty = true
inventory_sync.players[player.name].sync = false
player_sync.players[player.name].dirty = true
player_sync.players[player.name].sync = false
elseif event.element.name == "inventory_sync_failed_download_retry" then
elseif event.element.name == "player_sync_failed_download_retry" then
-- Retry acquiring the player from the controller
local player = assert(game.get_player(event.player_index))
inventory_sync.acquire(player)
player_sync.acquire(player)
player.gui.screen.dialog_failed_download.destroy()
end
end,
@@ -6,7 +6,7 @@ local progress_dialog = {}
--- @param total number
function progress_dialog.create(player, progress, total)
local frame = player.gui.screen.add {
name = "inventory_sync_progress",
name = "player_sync_progress",
type = "frame",
direction = "vertical",
}
@@ -19,7 +19,7 @@ function progress_dialog.create(player, progress, total)
local title = titlebar.add {
type = "label",
style = "frame_title",
caption = "Inventory sync",
caption = "Player sync",
}
title.drag_target = frame
@@ -33,7 +33,7 @@ function progress_dialog.create(player, progress, total)
filler.drag_target = frame
local content = frame.add {
name = "inventory_sync_progress_dialog_content",
name = "player_sync_progress_dialog_content",
type = "frame",
direction = "vertical",
style = "inside_shallow_frame_with_padding",
@@ -59,8 +59,8 @@ end
--- @param player LuaPlayer
function progress_dialog.remove(player)
if player.gui.screen.inventory_sync_progress then
player.gui.screen.inventory_sync_progress.destroy()
if player.gui.screen.player_sync_progress then
player.gui.screen.player_sync_progress.destroy()
end
end
@@ -69,12 +69,12 @@ end
--- @param total number
function progress_dialog.update(player, progress, total)
-- Be defensive here in case an old GUI is present
local frame = player.gui.screen.inventory_sync_progress
local frame = player.gui.screen.player_sync_progress
if not frame then
return false
end
local content = frame.inventory_sync_progress_dialog_content
local content = frame.player_sync_progress_dialog_content
if not content then
return false
end
+4
View File
@@ -0,0 +1,4 @@
{
"name": "player_sync",
"load": ["player_sync.lua", "gui/dialog_failed_download.lua"]
}
@@ -1,6 +1,6 @@
--[[
Inventory Sync
Player Sync
When a player leaves the game, serialize the player state and upload it to the controller.
When a player joins the game, request the player data from controller and deserialize the player state from it.
@@ -9,12 +9,12 @@ When a player joins the game, request the player data from controller and deseri
local compat = require("modules/clusterio/compat")
local clusterio_api = require("modules/clusterio/api")
local download_inventory = require("modules/inventory_sync/download_inventory")
local serialize = require("modules/inventory_sync/serialize")
local restore_position = require("modules/inventory_sync/restore_position")
local get_script_data = require("modules/inventory_sync/get_script_data")
local progress_dialog = require("modules/inventory_sync/gui/progress_dialog")
local dialog_failed_download = require("modules/inventory_sync/gui/dialog_failed_download")
local download_inventory = require("modules/player_sync/download_inventory")
local serialize = require("modules/player_sync/serialize")
local restore_position = require("modules/player_sync/restore_position")
local get_script_data = require("modules/player_sync/get_script_data")
local progress_dialog = require("modules/player_sync/gui/progress_dialog")
local dialog_failed_download = require("modules/player_sync/gui/dialog_failed_download")
local v2_remote_controller = compat.version_ge("2.0.0")
local recipe_notifications_api = compat.version_ge("2.0.67")
@@ -47,13 +47,13 @@ local function remove_player(player)
end
inventory_sync = {}
-- This function is called by instance.js as /sc inventory_sync.download_inventory("danielv", Escaped JSON string, package_number, total_packages_count) with data from the controller
inventory_sync.download_inventory = download_inventory
player_sync = {}
-- This function is called by instance.js as /sc player_sync.download_inventory("danielv", Escaped JSON string, package_number, total_packages_count) with data from the controller
player_sync.download_inventory = download_inventory
-- This function is called internally when a player leaves the game to
-- serialize the player for upload.
function inventory_sync.serialize_player(player, player_record)
function player_sync.serialize_player(player, player_record)
-- Editor mode is not supported.
if player.controller_type == defines.controllers.editor then
player.toggle_map_editor()
@@ -84,7 +84,7 @@ function inventory_sync.serialize_player(player, player_record)
return serialized_player
end
function inventory_sync.deserialize_player(player, finished_record)
function player_sync.deserialize_player(player, finished_record)
-- Editor mode is not supported.
if player.controller_type == defines.controllers.editor then
player.toggle_map_editor()
@@ -146,8 +146,8 @@ function inventory_sync.deserialize_player(player, finished_record)
end
end
inventory_sync.events = {}
inventory_sync.events[clusterio_api.events.on_server_startup] = function(event)
player_sync.events = {}
player_sync.events[clusterio_api.events.on_server_startup] = function(event)
local script_data = get_script_data(true)
script_data.players_waiting_for_acquire = {}
script_data.players_in_cutscene_to_sync = {}
@@ -156,34 +156,43 @@ inventory_sync.events[clusterio_api.events.on_server_startup] = function(event)
if not script_data.players[player.name] then
create_player(player, true)
end
-- migrate: dialogs created before inventory_sync was renamed to player_sync
local screen = player.gui.screen
if screen.inventory_sync_progress then
screen.inventory_sync_progress.destroy()
end
if screen.dialog_failed_download and screen.dialog_failed_download.inventory_sync_failed_download_content then
screen.dialog_failed_download.destroy()
end
end
end
-- Cleanup
inventory_sync.events[defines.events.on_pre_player_removed] = function(event)
player_sync.events[defines.events.on_pre_player_removed] = function(event)
remove_player(game.get_player(event.player_index))
end
inventory_sync.events[defines.events.on_player_created] = function(event)
player_sync.events[defines.events.on_player_created] = function(event)
create_player(game.get_player(event.player_index), false)
end
function inventory_sync.acquire(player)
function player_sync.acquire(player)
local script_data = get_script_data()
script_data.players_waiting_for_acquire[player.name] = {
start_tick = game.ticks_played,
}
clusterio_api.send_json("inventory_sync_acquire", {
clusterio_api.send_json("player_sync_acquire", {
player_name = player.name,
})
end
function inventory_sync.check_players_waiting_for_acquire()
function player_sync.check_players_waiting_for_acquire()
local script_data = get_script_data()
for player_name, record in pairs(script_data.players_waiting_for_acquire) do
-- Check if the acquire player request timed out
if game.ticks_played > record.start_tick + 600 then
inventory_sync.acquire_response(compat.table_to_json({
player_sync.acquire_response(compat.table_to_json({
status = "timeout",
player_name = player_name,
}))
@@ -191,11 +200,11 @@ function inventory_sync.check_players_waiting_for_acquire()
end
end
function inventory_sync.acquire_response(data)
function player_sync.acquire_response(data)
local script_data = get_script_data()
local response = assert(compat.json_to_table(data))
if not script_data.players_waiting_for_acquire[response.player_name] then
clusterio_api.send_json("inventory_sync_release", { player_name = response.player_name })
clusterio_api.send_json("player_sync_release", { player_name = response.player_name })
return
end
script_data.players_waiting_for_acquire[response.player_name] = nil
@@ -205,21 +214,21 @@ function inventory_sync.acquire_response(data)
response.player = player
script_data.players_in_cutscene_to_sync[response.player_name] = response
else
inventory_sync.sync_player(response)
player_sync.sync_player(response)
end
end
function inventory_sync.check_players_in_cutscene()
function player_sync.check_players_in_cutscene()
local script_data = get_script_data()
for player_name, response in pairs(script_data.players_in_cutscene_to_sync) do
if not is_in_cutscene(response.player) then
script_data.players_in_cutscene_to_sync[player_name] = nil
inventory_sync.sync_player(response)
player_sync.sync_player(response)
end
end
end
function inventory_sync.sync_player(acquire_response)
function player_sync.sync_player(acquire_response)
local player = assert(game.get_player(acquire_response.player_name))
-- Editor mode is not supported.
@@ -228,8 +237,8 @@ function inventory_sync.sync_player(acquire_response)
end
if player.controller_type == defines.controllers.editor then
log("ERROR: Inventory sync failed, unable to switch " .. player.name .. " out of editor mode")
player.print("ERROR: Inventory sync failed, unable to switch out of editor mode")
log("ERROR: Player sync failed, unable to switch " .. player.name .. " out of editor mode")
player.print("ERROR: Player sync failed, unable to switch out of editor mode")
return
end
@@ -237,7 +246,7 @@ function inventory_sync.sync_player(acquire_response)
local finished_record = script_data.finished_downloads[acquire_response.player_name]
if finished_record then
if acquire_response.status == "acquired" and finished_record.generation == acquire_response.generation then
inventory_sync.finish_download(player, finished_record)
player_sync.finish_download(player, finished_record)
return
end
@@ -298,7 +307,7 @@ function inventory_sync.sync_player(acquire_response)
acquire_response.has_data
and (not player_record.sync or acquire_response.generation > player_record.generation)
then
inventory_sync.initiate_inventory_download(player, player_record, acquire_response.generation)
player_sync.initiate_inventory_download(player, player_record, acquire_response.generation)
else
if acquire_response.generation == 0 then
@@ -314,12 +323,12 @@ function inventory_sync.sync_player(acquire_response)
end
end
function inventory_sync.finish_download(player, finished_record)
function player_sync.finish_download(player, finished_record)
local script_data = get_script_data()
script_data.finished_downloads[player.name] = nil
local player_record = script_data.players[player.name]
local status, result = xpcall(inventory_sync.deserialize_player, debug.traceback, player, finished_record)
local status, result = xpcall(player_sync.deserialize_player, debug.traceback, player, finished_record)
if not status then
log("ERROR: Deserializing player " .. player.name .. " failed: " .. result)
player.print("ERROR: Deserializing player data failed: " .. result)
@@ -335,43 +344,44 @@ function inventory_sync.finish_download(player, finished_record)
end
-- Download inventory from controller
inventory_sync.events[defines.events.on_player_joined_game] = function(event)
player_sync.events[defines.events.on_player_joined_game] = function(event)
local player = assert(game.get_player(event.player_index))
local script_data = get_script_data()
if not game.is_multiplayer() then
return player.print("Inventory sync deactivated: game is single player")
return player.print("Player sync deactivated: game is single player")
end
-- Send acquire request even if an active download is currently in progress
inventory_sync.acquire(player)
player_sync.acquire(player)
-- Clear active upload if it exists
script_data.active_uploads[player.name] = nil
end
inventory_sync.on_nth_tick = {}
inventory_sync.on_nth_tick[33] = function(event)
player_sync.on_nth_tick = {}
player_sync.on_nth_tick[33] = function(event)
-- Periodically check players currently in the process of syncing
inventory_sync.check_players_waiting_for_acquire()
inventory_sync.check_players_in_cutscene()
inventory_sync.check_active_downloads()
inventory_sync.check_active_uploads()
player_sync.check_players_waiting_for_acquire()
player_sync.check_players_in_cutscene()
player_sync.check_active_downloads()
player_sync.check_active_uploads()
end
inventory_sync.events[defines.events.on_cutscene_waypoint_reached] = function(event)
inventory_sync.check_players_in_cutscene()
player_sync.events[defines.events.on_cutscene_waypoint_reached] = function(event)
player_sync.check_players_in_cutscene()
end
inventory_sync.events[defines.events.on_cutscene_cancelled] = function(event)
inventory_sync.check_players_in_cutscene()
player_sync.events[defines.events.on_cutscene_cancelled] = function(event)
player_sync.check_players_in_cutscene()
end
-- Upload inventory when a player leaves the game. Triggers on restart after crash if player was online during crash.
inventory_sync.events[defines.events.on_pre_player_left_game] = function(event)
player_sync.events[defines.events.on_pre_player_left_game] = function(event)
-- for some reason, on_pre_player_left_game gets called before on_server_startup so script_data isn't ready yet
if not compat.script_data.inventory_sync then
log("ERROR: script data for inventory sync is not defined")
-- migrate: saves from before the rename still have their data under inventory_sync
if not compat.script_data.player_sync and not compat.script_data.inventory_sync then
log("ERROR: script data for player sync is not defined")
return
end
@@ -381,18 +391,18 @@ inventory_sync.events[defines.events.on_pre_player_left_game] = function(event)
-- If player has an active download release our acquisition but let the download continue
if script_data.active_downloads[player.name] then
clusterio_api.send_json("inventory_sync_release", { player_name = player.name })
clusterio_api.send_json("player_sync_release", { player_name = player.name })
return
end
if not player_record.dirty or not player_record.sync then
-- Nothing to upload, release the acquisition so other instances are not blocked
clusterio_api.send_json("inventory_sync_release", { player_name = player.name })
clusterio_api.send_json("player_sync_release", { player_name = player.name })
return
end
player_record.generation = player_record.generation + 1
local status, result = xpcall(inventory_sync.serialize_player, debug.traceback, player, player_record)
local status, result = xpcall(player_sync.serialize_player, debug.traceback, player, player_record)
if not status then
log("ERROR: Serializing player " .. player.name .. " failed: " .. result)
player.print("ERROR: Serializing player data failed: " .. result)
@@ -404,11 +414,11 @@ inventory_sync.events[defines.events.on_pre_player_left_game] = function(event)
timeout = math.random(600, 1200), -- Start with 10-20 seconds for the timeout
}
clusterio_api.send_json("inventory_sync_upload", result)
clusterio_api.send_json("player_sync_upload", result)
end
-- Invoked by the instance code to confirm the controller has received the uploaded data
function inventory_sync.confirm_upload(player_name, generation)
function player_sync.confirm_upload(player_name, generation)
local player = game.get_player(player_name)
if not player or player.connected then
log("no player, or player connected")
@@ -431,7 +441,7 @@ function inventory_sync.confirm_upload(player_name, generation)
player_record.dirty = false
end
function inventory_sync.initiate_inventory_download(player, player_record, generation)
function player_sync.initiate_inventory_download(player, player_record, generation)
player.print("Initiating inventory download...")
local record = {
started = game.ticks_played,
@@ -448,7 +458,7 @@ function inventory_sync.initiate_inventory_download(player, player_record, gener
recipe_notifications = serialize.serialize_crafting_notifications(player)
end
clusterio_api.send_json("inventory_sync_download", {
clusterio_api.send_json("player_sync_download", {
player_name = player.name,
recipe_notifications = recipe_notifications,
})
@@ -479,7 +489,7 @@ function inventory_sync.initiate_inventory_download(player, player_record, gener
progress_dialog.display(player, 0, 1)
end
function inventory_sync.check_active_downloads()
function player_sync.check_active_downloads()
-- Used for detecting the inventory download failing, like when the map is saved and
-- re-loaded in the middle of a download. The player will be returned to a playable state.
local script_data = get_script_data()
@@ -503,7 +513,7 @@ function inventory_sync.check_active_downloads()
end
end
function inventory_sync.check_active_uploads()
function player_sync.check_active_uploads()
local script_data = get_script_data()
for player_name, record in pairs(script_data.active_uploads) do
if record.last_attempt + record.timeout < game.ticks_played then
@@ -515,10 +525,10 @@ function inventory_sync.check_active_uploads()
log("Retrying upload for " .. player_name)
record.last_attempt = game.ticks_played
record.timeout = math.random(record.timeout, record.timeout * 2)
clusterio_api.send_json("inventory_sync_upload", record.serialized)
clusterio_api.send_json("player_sync_upload", record.serialized)
end
end
end
end
return inventory_sync
return player_sync
@@ -1,8 +1,8 @@
local compat = require("modules/clusterio/compat")
local clusterio_serialize = require("modules/clusterio/serialize")
local character_inventories = require("modules/inventory_sync/define_player_inventories")
local character_stat_keys = require("modules/inventory_sync/define_player_stat_keys")
local game_view_settings_keys = require("modules/inventory_sync/define_game_view_settings")
local character_inventories = require("modules/player_sync/define_player_inventories")
local character_stat_keys = require("modules/player_sync/define_player_stat_keys")
local game_view_settings_keys = require("modules/player_sync/define_game_view_settings")
local serialize = {}
local v2_logistic_api = compat.version_ge("2.0.0")
@@ -1,6 +1,6 @@
{
"name": "@clusterio/plugin-inventory_sync",
"version": "2.0.0-alpha.27",
"name": "@clusterio/plugin-player_sync",
"version": "2.0.1",
"description": "Clusterio plugin forwarding between Factorio servers",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Danielv <danielv@danielv.no>",
+53
View File
@@ -0,0 +1,53 @@
import assert from "node:assert/strict";
import fs from "node:fs/promises";
import path from "node:path";
import * as lib from "@clusterio/lib";
import * as mock from "../../../test/mock.js";
import controllerEntrypoint from "../dist/node/controller.js";
import { plugin as info } from "../dist/node/index.js";
import { DatabaseStatsRequest } from "../dist/node/messages.js";
describe("player_sync controller", function() {
const databaseDir = path.join("temp", "test", "player_sync");
before(function() {
try {
lib.registerPluginMessages([info]);
} catch (err) {
// Already registered by the full test suite
}
});
beforeEach(async function() {
await fs.rm(databaseDir, { recursive: true, force: true });
await fs.mkdir(databaseDir, { recursive: true });
});
async function load() {
return await mock.loadControllerPlugin(async (context) => {
context.controller.mockConfigEntries.set("controller.database_directory", databaseDir);
await controllerEntrypoint(context);
}, info);
}
it("should move inventories.json to player_sync.json", async function() { // Alpha 28
const playerData = { name: "test", generation: 1 };
await fs.writeFile(path.join(databaseDir, "inventories.json"), JSON.stringify([["test", playerData]]));
const controller = await load();
const stats = await mock.getHandler(controller, DatabaseStatsRequest)();
assert.equal(stats.databaseEntries, 1);
assert.equal(stats.largestEntry.name, "test");
await assert.rejects(fs.access(path.join(databaseDir, "inventories.json")), { code: "ENOENT" });
await fs.access(path.join(databaseDir, "player_sync.json"));
});
it("should not replace an existing player_sync.json", async function() {
await fs.writeFile(path.join(databaseDir, "inventories.json"), JSON.stringify([["old", { name: "old" }]]));
await fs.writeFile(path.join(databaseDir, "player_sync.json"), JSON.stringify([["new", { name: "new" }]]));
const controller = await load();
const stats = await mock.getHandler(controller, DatabaseStatsRequest)();
assert.equal(stats.databaseEntries, 1);
assert.equal(stats.largestEntry.name, "new");
});
});
@@ -19,7 +19,7 @@ async function decode(encoded) {
return JSON.parse((await inflate(Buffer.from(encoded, "base64"))).toString());
}
describe("inventory_sync", function() {
describe("player_sync", function() {
describe("recipeNotificationDelta()", function() {
it("should be empty when the lists match", function() {
assert.deepEqual(recipeNotificationDelta(["a", "b"], ["b", "a"]), {});
@@ -71,7 +71,7 @@ describe("inventory_sync", function() {
});
});
describe("ipc-inventory_sync_download", function() {
describe("ipc-player_sync_download", function() {
let instance;
before(async function() {
try {
@@ -80,7 +80,7 @@ describe("inventory_sync", function() {
// Already registered by the full test suite
}
({ instance } = await mock.loadInstancePlugin(loadInstancePlugin, info));
instance.mockConfigEntries.set("inventory_sync.rcon_chunk_size", 100000);
instance.mockConfigEntries.set("player_sync.rcon_chunk_size", 100000);
});
beforeEach(function() {
instance.server.reset();
@@ -94,7 +94,7 @@ describe("inventory_sync", function() {
});
}
async function download(request) {
instance.server.emit("ipc-inventory_sync_download", request);
instance.server.emit("ipc-player_sync_download", request);
// zlib runs on the threadpool, so poll on timers rather than setImmediate
for (let i = 0; i < 1000 && !instance.server.rconCommands.length; i++) {
await lib.wait(1);
@@ -102,7 +102,7 @@ describe("inventory_sync", function() {
}
function downloadCommand(playerData) {
const json = lib.escapeString(JSON.stringify(playerData));
return `/sc inventory_sync.download_inventory('test','${json}',1,1)`;
return `/sc player_sync.download_inventory('test','${json}',1,1)`;
}
it("should send an empty download when the controller has no data", async function() {
@@ -110,7 +110,7 @@ describe("inventory_sync", function() {
await download({ player_name: "test" });
assert.deepEqual(
instance.server.rconCommands,
["/sc inventory_sync.download_inventory('test',nil,0,0)"]
["/sc player_sync.download_inventory('test',nil,0,0)"]
);
});
it("should send everything as a delta when no snapshot was given", async function() {
+46
View File
@@ -0,0 +1,46 @@
import assert from "node:assert/strict";
import { fileURLToPath } from "node:url";
import * as lib from "@clusterio/lib";
import { Controller } from "@clusterio/controller";
import { plugin as info } from "../dist/node/index.js";
describe("player_sync migrations", function() {
before(async function() {
lib.addPluginConfigFields([info]);
const packagePath = fileURLToPath(new URL("../package.json", import.meta.url));
await lib.loadPluginMigrations([{ ...info, packagePath }]);
});
after(function() {
for (const hook of [
lib.ControllerConfig.pluginMigrations, lib.HostConfig.pluginMigrations,
lib.InstanceConfig.pluginMigrations, lib.Role.pluginMigrations,
]) {
hook.detach(info.name);
}
});
it("should rename inventory_sync config fields", function() { // Alpha 28
const controllerConfig = lib.ControllerConfig.fromJSON({
"inventory_sync.load_plugin": false,
"inventory_sync.player_lock_timeout": 120,
}, "controller");
assert.equal(controllerConfig.get("player_sync.load_plugin"), false);
assert.equal(controllerConfig.get("player_sync.player_lock_timeout"), 120);
assert.equal(controllerConfig.toJSON()["inventory_sync.player_lock_timeout"], undefined);
const hostConfig = lib.HostConfig.fromJSON({ "inventory_sync.load_plugin": false }, "host");
assert.equal(hostConfig.get("player_sync.load_plugin"), false);
const instanceConfig = lib.InstanceConfig.fromJSON({
"inventory_sync.rcon_chunk_size": 500,
}, "controller");
assert.equal(instanceConfig.get("player_sync.rcon_chunk_size"), 500);
});
it("should rename the inventory_sync permission in roles", function() { // Alpha 28
const result = Controller.migrateRoles([
{ permissions: ["core.admin", "inventory_sync.inventory.view"] },
]);
assert.deepEqual(result, [{ permissions: ["core.admin", "player_sync.inventory.view"] }]);
});
});
File renamed without changes.
@@ -17,8 +17,8 @@ function InventoryPage() {
})();
}, []);
return <PageLayout nav={[{ name: "Inventory sync" }]}>
<PageHeader title="Inventory sync" />
return <PageLayout nav={[{ name: "Player sync" }]}>
<PageHeader title="Player sync" />
{statsData && <>
<p>Database size: {Math.round(statsData.databaseSize / 1000)}kB</p>
<p>Database entries: {statsData.databaseEntries}</p>
@@ -33,8 +33,8 @@ export default async function(context: WebPluginContext) {
control.hooks.pages.attach(plugin.name, () => [
{
path: "/inventory",
sidebarName: "Inventory sync",
permission: "inventory_sync.inventory.view",
sidebarName: "Player sync",
permission: "player_sync.inventory.view",
content: <InventoryPage />,
},
]);
File renamed without changes.
@@ -13,8 +13,8 @@ module.exports = (env = {}, argv = {}) => merge(common(env, argv), {
},
plugins: [
new webpack.container.ModuleFederationPlugin({
name: "inventory_sync",
library: { type: "var", name: "plugin_inventory_sync" },
name: "player_sync",
library: { type: "var", name: "plugin_player_sync" },
exposes: {
"./": "./index.ts",
"./package.json": "./package.json",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@clusterio/plugin-research_sync",
"version": "2.0.0-alpha.27",
"version": "2.0.1",
"description": "Clusterio plugin for synchronizing research between Factorio servers",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Hornwitser <github@hornwitser.no>",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@clusterio/plugin-statistics_exporter",
"version": "2.0.0-alpha.27",
"version": "2.0.1",
"description": "Clusterio plugin for gathering statistics from Factorio servers",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Hornwitser <github@hornwitser.no>",
+3 -2
View File
@@ -99,8 +99,9 @@ export default async function(context: InstancePluginContext) {
let items = await instance.sendTo("controller", new GetStorageRequest());
// Cache items for periodic time updates
cachedInventoryItems = items;
// Send the full inventory before a delta update can replace the cached items.
const itemsJson = lib.escapeString(JSON.stringify(items));
await instance.sendRcon(`/sc __subspace_storage__ UpdateInvData("${itemsJson}", true)`, true, plugin.name);
// Ensure a payload with updated time is sent every second
timeUpdateId = setInterval(() => {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@clusterio/plugin-subspace_storage",
"version": "2.0.0-alpha.27",
"version": "2.0.1",
"description": "Clusterio plugin for sharing storage between Factorio servers",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Hornwitser <github@hornwitser.no>",
+38
View File
@@ -1,6 +1,7 @@
import assert from "node:assert/strict";
import fs from "node:fs/promises";
import path from "node:path";
import { mock as nodeMock } from "node:test";
import * as lib from "@clusterio/lib";
import * as mock from "../../../test/mock.js";
@@ -91,5 +92,42 @@ describe("subspace_storage plugin", function() {
const message = instance.connector.sentMessages.find(m => m.data instanceof RemoveRequest);
assert.equal(message, undefined);
});
it("should send the full inventory before delta updates and keep the time ticking", async function() {
nodeMock.timers.enable({ apis: ["setInterval", "Date"], now: 1000000 });
const { instance: startedInstance } = await mock.loadInstancePlugin(instanceEntrypoint, info);
const items = [
new Item("iron-plate", 100, "normal"),
new Item("iron-plate", 5, "rare"),
new Item("copper-plate", 50, "normal"),
];
startedInstance.sendTo = async (dst, request) => {
assert.equal(dst, "controller");
assert(request instanceof GetStorageRequest);
return items;
};
function updateCommand(payload, full = false) {
const json = lib.escapeString(JSON.stringify(payload));
return `/sc __subspace_storage__ UpdateInvData("${json}"${full ? ", true" : ""})`;
}
try {
await startedInstance.hooks.start.invoke();
assert.deepEqual(startedInstance.server.rconCommands, [updateCommand(items, true)]);
const changedItems = [new Item("iron-plate", 110, "normal")];
nodeMock.timers.tick(100);
await mock.getHandler(startedInstance, UpdateStorageEvent)(new UpdateStorageEvent(changedItems));
nodeMock.timers.tick(900);
nodeMock.timers.tick(1000);
assert.deepEqual(startedInstance.server.rconCommands, [
updateCommand(items, true),
updateCommand([...changedItems, new Item("signal-unixtime", 1001)]),
updateCommand([...changedItems, new Item("signal-unixtime", 1002)]),
]);
} finally {
await startedInstance.hooks.stop.invoke();
nodeMock.timers.reset();
}
});
});
});
+9 -9
View File
@@ -2,8 +2,9 @@
* Publish any mods that need to be published
*/
import FormData from "form-data";
import { openAsBlob } from "node:fs";
import fs from "node:fs/promises";
import path from "node:path";
import yargs from "yargs";
let DRY = false;
@@ -34,7 +35,7 @@ async function requestModReleases(mod) {
/**
* Upload a mod to the mod portal
*/
async function uploadModRelease(file, name) {
export async function uploadModRelease(file, name) {
// eslint-disable-next-line no-console
console.log(`Uploading new release ${file}`);
if (DRY) {
@@ -61,7 +62,7 @@ async function uploadModRelease(file, name) {
// Upload the new version of the mod
const uploadUrl = (await initResponse.json()).upload_url;
const uploadFormData = new FormData();
uploadFormData.append("file", (await fs.open(file)).createReadStream(file));
uploadFormData.append("file", await openAsBlob(file), path.basename(file));
const uploadResponse = await fetch(uploadUrl, {
method: "POST",
@@ -77,25 +78,24 @@ const regexModZip = /(\w+)_\d+.\d+.\d+\.zip/;
/**
* Publishes all mods that require publishing
*/
async function publishMods(dir) {
export async function publishMods(dir) {
const files = await fs.readdir(dir);
const mods = files
.sort((a, b) => a.localeCompare(b, "en", { numeric: true }))
.map(file => regexModZip.exec(file))
.filter(file => Boolean(file));
const uploading = [];
for (const mod of mods) {
const [file, name] = mod;
const releases = await requestModReleases(name);
if (!releases.some(release => release.file_name === file)) {
uploading.push(uploadModRelease(file, name));
// Upload in version order so the newest variant is published last.
await uploadModRelease(path.join(dir, file), name);
} else {
// eslint-disable-next-line no-console
console.log(`Skipped ${file}`);
}
}
await Promise.all(uploading);
}
/**
@@ -121,7 +121,7 @@ async function main() {
.demandOption("mods-dir", "A path to the mods dir must be specified");
}, async argv => {
DRY = argv.dry;
publishMods(argv.modsDir);
await publishMods(argv.modsDir);
})
.strict()
.parse();
+12 -1
View File
@@ -11,7 +11,7 @@ import { fileURLToPath } from "node:url";
import {
ControllerConfig, Address, RequestError,
InstanceConfig, SystemInfo, Role, ModPack, AccountUpdateEvent,
addPluginConfigFields,
addPluginConfigFields, renamePermissionPrefix,
} from "@clusterio/lib";
import "../setup_logging.js";
@@ -714,6 +714,17 @@ describe("controller/src/Controller", function() {
{ permissions: ["core.instance.save.subscribe"] },
]);
});
it("runs plugin role migrations", function() {
Role.pluginMigrations.attach("test", role => (
renamePermissionPrefix(role, "old_name.", "new_name.")
));
try {
const result = Controller.migrateRoles([{ permissions: ["foo", "old_name.view"] }]);
assert.deepEqual(result, [{ permissions: ["foo", "new_name.view"] }]);
} finally {
Role.pluginMigrations.clear();
}
});
it("does nothing for upto date data", function() {
const role = new Role(
0, "n", "d", new Set("p1", "p2"), 0, false
+19
View File
@@ -1,4 +1,5 @@
import assert from "node:assert/strict";
import events from "node:events";
import { Controller, WsServer } from "@clusterio/controller";
import { ControllerConfig } from "@clusterio/lib";
@@ -68,5 +69,23 @@ describe("controller/src/WsServer", function() {
assert.equal(ipv6, "2000::1");
});
});
describe(".handleConnection()", function() {
it("should handle socket errors before the handshake completes", function() {
// A ws socket emits "error" for e.g. a close frame with the reserved code 1006
const socket = new events.EventEmitter();
socket.send = () => {};
socket.close = () => {};
socket.terminate = () => {};
const req = { socket: { remoteAddress: "1.2.3.4", remoteFamily: "IPv4" }, headers: {} };
wsServer.handleConnection(socket, req);
try {
assert.doesNotThrow(() => {
socket.emit("error", new RangeError("Invalid WebSocket frame: invalid status code 1006"));
});
} finally {
socket.emit("message", "not json"); // ends the handshake and its timeout
}
});
});
});
});
+2 -2
View File
@@ -9,13 +9,13 @@ describe("ctl/config_ops", function() {
"# Clusterio configuration",
"",
"instance.name = Example",
"inventory_sync.load_plugin = false",
"player_sync.load_plugin = false",
"",
].join(newline);
assert.deepEqual(await configToKeyVal(data), {
"instance.name": "Example",
"inventory_sync.load_plugin": "false",
"player_sync.load_plugin": "false",
});
}
});
+68 -1
View File
@@ -1,10 +1,12 @@
import assert from "node:assert/strict";
import dgram from "node:dgram";
import events from "node:events";
import fs from "node:fs/promises";
import path from "node:path";
import { fileURLToPath } from "node:url";
import * as lib from "@clusterio/lib";
import Host, { _discoverInstances } from "@clusterio/host/dist/node/src/Host.js";
import Host, { _discoverInstances, HostRouter } from "@clusterio/host/dist/node/src/Host.js";
import Instance from "@clusterio/host/dist/node/src/Instance.js";
import { HostConnector } from "@clusterio/host/dist/node/host.js";
import "../setup_logging.js";
@@ -31,6 +33,34 @@ describe("host/src/Host", function() {
});
});
describe("class HostRouter", function() {
describe(".wakeInstance()", function() {
it("should log the message of a start error that has no stack", async function() {
// RequestError leaves its stack undefined
const err = new lib.RequestError("No available port left to assign from host.factorio_port_range");
const mockHost = {
_shuttingDown: false,
_connectInstance: async () => { throw err; },
instanceLogMeta: () => ({}),
};
const responses = [];
const origin = { connector: { sendResponseError: response => responses.push(response) } };
const message = { dst: lib.Address.fromShorthand({ instanceId: 1 }) };
const logged = [];
const originalError = lib.logger.error;
lib.logger.error = text => { logged.push(text); };
try {
new HostRouter(mockHost).wakeInstance(origin, message);
await new Promise(resolve => setImmediate(resolve));
} finally {
lib.logger.error = originalError;
}
assert.equal(responses.length, 1);
assert.equal(logged.length, 1);
assert.match(logged[0], /No available port left to assign/);
});
});
});
describe("class Host", function() {
describe(".handleSyncUserListsEvent()", function() {
const hostAddress = lib.Address.fromShorthand({ hostId: 1 });
@@ -180,6 +210,43 @@ describe("host/src/Host", function() {
assert.equal(host.config.restartRequired, true);
});
});
describe(".assignFreeGamePort()", function() {
let host;
let held;
let instanceConfig;
beforeEach(async function() {
const hostConfig = new lib.HostConfig("host");
host = new Host(new HostConnector(hostConfig, []), hostConfig, []);
instanceConfig = new lib.InstanceConfig("host", { "instance.id": 1 });
host.assignedInstances.set(1, { config: instanceConfig, path: "" });
held = dgram.createSocket("udp4");
held.bind(0, "127.0.0.1");
await events.once(held, "listening");
});
afterEach(function() {
held.close();
});
it("replaces a stored port another process is bound to", async function() {
const port = held.address().port;
host.config.set("host.factorio_port_range", `${port}-${port + 1}`);
instanceConfig.set("factorio.host_assigned_game_port", port);
assert.equal(await host.assignFreeGamePort(1), port + 1);
assert.equal(instanceConfig.get("factorio.host_assigned_game_port"), port + 1);
});
it("keeps a stored port that is free", async function() {
const port = held.address().port;
host.config.set("host.factorio_port_range", `${port}-${port + 1}`);
instanceConfig.set("factorio.host_assigned_game_port", port + 1);
assert.equal(await host.assignFreeGamePort(1), port + 1);
});
it("rejects when every port in the range is taken", async function() {
host.config.set("host.factorio_port_range", String(held.address().port));
await assert.rejects(
host.assignFreeGamePort(1),
/No available port left to assign from host.factorio_port_range \(1 in use by other processes\)/
);
});
});
describe(".handlePluginInstallRequest()", function() {
let host;
let _fetch;
+2 -2
View File
@@ -909,7 +909,7 @@ describe("Integration of Clusterio", function() {
describe("instance send-rcon", function() {
requiresFactorio(this);
const plugins = [
"global_chat", "inventory_sync", "player_auth",
"global_chat", "player_sync", "player_auth",
"research_sync", "statistics_exporter", "subspace_storage",
];
this.beforeAll(async function() {
@@ -1276,7 +1276,7 @@ describe("Integration of Clusterio", function() {
await execCtl("instance config set 44 research_sync.load_plugin false");
await execCtl("instance config set 44 statistics_exporter.load_plugin false");
await execCtl("instance config set 44 subspace_storage.load_plugin false");
await execCtl("instance config set 44 inventory_sync.load_plugin false");
await execCtl("instance config set 44 player_sync.load_plugin false");
await execCtl("instance config set 44 factorio.enable_save_patching false");
await execCtl("instance start 44");
+2 -2
View File
@@ -345,7 +345,7 @@ before(async function() {
await execCtlProcess("plugin add ../../plugins/research_sync");
await execCtlProcess("plugin add ../../plugins/statistics_exporter");
await execCtlProcess("plugin add ../../plugins/subspace_storage");
await execCtlProcess("plugin add ../../plugins/inventory_sync");
await execCtlProcess("plugin add ../../plugins/player_sync");
await execCtlProcess("plugin add ../../plugins/player_auth");
console.log("Bootstrapping");
@@ -382,7 +382,7 @@ before(async function() {
testPack.factorioVersion = latestFactorioVersion;
const [major, minor] = lib.normaliseMajorMinorVersion(latestFactorioVersion).split(".");
testPack.mods.set("clusterio_lib", { name: "clusterio_lib", enabled: true, version: `2.0.${major}${minor}` });
testPack.mods.set("clusterio_lib", { name: "clusterio_lib", enabled: true, version: `2.1.${major}${minor}` });
await control.sendTo("controller", new lib.ModPackCreateRequest(testPack));
await control.sendTo(
+2 -2
View File
@@ -43,7 +43,7 @@ describe("Clusterio Instance", function() {
await execCtl(`instance config set ${instName} instance.exclude_from_start_all true`);
for (const plugin of [
"global_chat", "research_sync", "statistics_exporter", "subspace_storage", "player_auth", "inventory_sync",
"global_chat", "research_sync", "statistics_exporter", "subspace_storage", "player_auth", "player_sync",
]) {
await execCtl(`instance config set ${instName} ${plugin}.load_plugin false`);
}
@@ -255,7 +255,7 @@ describe("start-all and stop-all commands", function() {
for (const instanceName of [testInstName1, testInstName2, testInstName3]) {
for (const plugin of [
"global_chat", "research_sync", "statistics_exporter",
"subspace_storage", "player_auth", "inventory_sync",
"subspace_storage", "player_auth", "player_sync",
]) {
await execCtl(`instance config set ${instanceName} ${plugin}.load_plugin false`);
}
+2
View File
@@ -40,6 +40,8 @@ describe("Integration of host/src/server", function() {
describe("class FactorioServer", function() {
let writePath = path.join("temp", "test", "integration");
let server = new FactorioServer(factorioDir, writePath, {});
// Let Factorio bind an OS-assigned UDP port instead of guessing a free one.
server.gamePort = 0;
let logFile;
before(async function() {
+65
View File
@@ -0,0 +1,65 @@
import assert from "node:assert/strict";
import * as lib from "@clusterio/lib";
describe("lib/SyncHook", function() {
describe("class SyncHook", function() {
let hook;
beforeEach(function() {
hook = new lib.SyncHook();
});
describe(".attach()", function() {
it("should track attached handlers", function() {
assert.equal(hook.size, 0);
hook.attach("alpha", value => value);
hook.attach("beta", value => value);
assert.equal(hook.size, 2);
assert.deepEqual([...hook.attached], ["alpha", "beta"]);
});
it("should reject duplicate handler names", function() {
hook.attach("alpha", value => value);
assert.throws(
() => hook.attach("alpha", value => value),
new Error("Handler with name alpha is already attached")
);
});
});
describe(".detach()", function() {
it("should remove the named handler", function() {
hook.attach("alpha", value => value);
hook.attach("beta", value => value);
hook.detach("alpha");
assert.deepEqual([...hook.attached], ["beta"]);
});
});
describe(".clear()", function() {
it("should remove all handlers", function() {
hook.attach("alpha", value => value);
hook.attach("beta", value => value);
hook.clear();
assert.equal(hook.size, 0);
});
});
describe(".reduce()", function() {
it("should pass the value through handlers in attach order", function() {
hook.attach("alpha", value => [...value, "alpha"]);
hook.attach("beta", value => [...value, "beta"]);
assert.deepEqual(hook.reduce([]), ["alpha", "beta"]);
});
it("should return the value unchanged with no handlers", function() {
const value = {};
assert.equal(hook.reduce(value), value);
});
it("should rethrow errors as a PluginError named after the handler", function() {
hook.attach("alpha", () => { throw new Error("failed"); });
assert.throws(
() => hook.reduce(1),
err => err instanceof lib.PluginError && err.pluginName === "alpha"
&& err.original.message === "failed"
);
});
});
});
});
+30
View File
@@ -0,0 +1,30 @@
import assert from "node:assert/strict";
import * as lib from "@clusterio/lib";
describe("lib/config/definitions", function() {
describe("addPluginConfigFields()", function() {
after(function() {
delete lib.CtlConfig.fieldDefinitions["ctl_test.field"];
});
it("should add ctl config fields", function() {
lib.addPluginConfigFields([{
name: "ctl_test",
ctlConfigFields: { "ctl_test.field": { type: "string", initialValue: "value" } },
}]);
const config = new lib.CtlConfig("control");
assert.equal(config.get("ctl_test.field"), "value");
});
it("should reject ctl config fields not named after the plugin", function() {
assert.throws(
() => lib.addPluginConfigFields([{
name: "ctl_test",
ctlConfigFields: { "other.field": { type: "string" } },
}]),
new Error("Expected name of config field 'other.field' for ctl_test to start with 'ctl_test.'")
);
});
});
});
+103
View File
@@ -45,4 +45,107 @@ describe("lib/config/definitions/migrations", function() {
describe("Ctl Config", function() {
});
describe("Plugin Config", function() {
const pluginInfo = {
name: "test_plugin",
version: "1.2.0",
instanceEntrypoint: "instance.js",
instanceConfigFields: {
"test_plugin.delay": { type: "number", initialValue: 1 },
},
};
before(function() {
lib.addPluginConfigFields([pluginInfo]);
});
afterEach(function() {
lib.InstanceConfig.pluginMigrations.clear();
lib.Config.pluginVersions.clear();
});
after(function() {
for (const Config of [lib.ControllerConfig, lib.HostConfig, lib.InstanceConfig]) {
for (const name of Object.keys(Config.fieldDefinitions)) {
if (name.startsWith("test_plugin.")) {
delete Config.fieldDefinitions[name];
}
}
}
});
describe("static .pluginMigrations", function() {
it("should run on load", function() {
lib.InstanceConfig.pluginMigrations.attach("test_plugin", config => (
lib.renameConfigPrefix(config, "old_plugin.", "test_plugin.")
));
const config = lib.InstanceConfig.fromJSON({
"old_plugin.load_plugin": false,
"old_plugin.delay": 5,
}, "controller");
assert.equal(config.get("test_plugin.load_plugin"), false);
assert.equal(config.get("test_plugin.delay"), 5);
assert.equal(config.toJSON()["old_plugin.delay"], undefined);
});
it("should run after core migrations", function() {
lib.InstanceConfig.pluginMigrations.attach("test_plugin", config => {
assert.equal(config["factorio.sync_adminlist"], "enabled");
return config;
});
lib.InstanceConfig.fromJSON({ "factorio.sync_adminlist": true }, "controller");
});
it("should see the version the config was saved with", function() {
let seen;
lib.InstanceConfig.pluginMigrations.attach("test_plugin", config => {
seen = config["test_plugin.version"];
return config;
});
lib.InstanceConfig.fromJSON({ "test_plugin.version": "1.1.0" }, "controller");
assert.equal(seen, "1.1.0");
});
it("should not run again when the config is changed", function() {
lib.Config.pluginVersions.set("test_plugin", "1.2.0");
let calls = 0;
lib.InstanceConfig.pluginMigrations.attach("test_plugin", config => {
calls += 1;
if (config["test_plugin.version"] === "1.1.0") {
config["test_plugin.delay"] *= 60;
}
return config;
});
const config = lib.InstanceConfig.fromJSON({
"test_plugin.version": "1.1.0",
"test_plugin.delay": 2,
}, "controller");
config.set("instance.name", "changed");
assert.equal(config.get("test_plugin.delay"), 120);
assert.equal(calls, 1);
});
});
describe("static .pluginVersions", function() {
it("should write the version after migrations run", function() {
lib.Config.pluginVersions.set("test_plugin", "1.2.0");
const config = lib.InstanceConfig.fromJSON({ "test_plugin.version": "1.1.0" }, "controller");
assert.equal(config.get("test_plugin.version"), "1.2.0");
assert.equal(config.dirty, true);
});
it("should mark configs saved without a version as dirty", function() {
lib.Config.pluginVersions.set("test_plugin", "1.2.0");
const config = lib.InstanceConfig.fromJSON({}, "controller");
assert.equal(config.get("test_plugin.version"), "1.2.0");
assert.equal(config.dirty, true);
});
it("should not mark configs already at the version as dirty", function() {
lib.Config.pluginVersions.set("test_plugin", "1.2.0");
const config = lib.InstanceConfig.fromJSON({ "test_plugin.version": "1.2.0" }, "controller");
assert.equal(config.dirty, false);
});
it("should write the version to new configs", function() {
lib.Config.pluginVersions.set("test_plugin", "1.2.0");
const config = new lib.InstanceConfig("controller");
assert.equal(config.get("test_plugin.version"), "1.2.0");
});
it("should leave the version empty when no versions are registered", function() {
const config = new lib.InstanceConfig("controller");
assert.equal(config.get("test_plugin.version"), "");
});
});
});
});
+19 -1
View File
@@ -241,7 +241,7 @@ describe("lib/helpers", function() {
lib.redactUrl("https://mods.factorio.com/download/mod?username=user&token=secret"),
"https://mods.factorio.com/download/mod?username=user&token=REDACTED"
);
for (const name of ["access_token", "api_key", "apikey", "password", "secret", "token"]) {
for (const name of ["access_token", "api_key", "apikey", "code", "password", "secret", "token"]) {
assert.equal(
lib.redactUrl(`https://example.com/?${name}=secret`),
`https://example.com/?${name}=REDACTED`
@@ -272,6 +272,24 @@ describe("lib/helpers", function() {
assert.equal(lib.redactUrl("not a url?token=secret"), "not a url?token=secret");
assert.equal(lib.redactUrl(""), "");
});
it("should redact a path with a query string and return a path", function() {
assert.equal(
lib.redactUrl("/api/plugin/callback?code=secret&state=abc"),
"/api/plugin/callback?code=REDACTED&state=abc"
);
assert.equal(lib.redactUrl("/api/plugin?page=2"), "/api/plugin?page=2");
});
it("should keep the original encoding when there is nothing to redact", function() {
assert.equal(lib.redactUrl("/search?q=a%20b"), "/search?q=a%20b");
});
});
describe("redactQuery()", function() {
it("should redact credential parameters and keep the rest", function() {
const query = { code: "secret", Token: "secret", state: "abc" };
assert.deepEqual(lib.redactQuery(query), { code: "REDACTED", Token: "REDACTED", state: "abc" });
assert.equal(query.code, "secret");
});
});
function parse(input, attributes) {
+50
View File
@@ -0,0 +1,50 @@
import assert from "node:assert/strict";
import * as lib from "@clusterio/lib";
describe("lib/migrations", function() {
describe("renameConfigField()", function() {
it("should move the value to the new name", function() {
const config = { "foo.bar": 1, "foo.other": 2 };
assert.equal(lib.renameConfigField(config, "foo.bar", "foo.bar_baz"), config);
assert.deepEqual(config, { "foo.bar_baz": 1, "foo.other": 2 });
});
it("should keep an existing value under the new name", function() {
const config = { "foo.bar": 1, "foo.bar_baz": 2 };
lib.renameConfigField(config, "foo.bar", "foo.bar_baz");
assert.deepEqual(config, { "foo.bar_baz": 2 });
});
it("should do nothing if the old name is missing", function() {
const config = { "foo.bar_baz": 2 };
lib.renameConfigField(config, "foo.bar", "foo.bar_baz");
assert.deepEqual(config, { "foo.bar_baz": 2 });
});
});
describe("renameConfigPrefix()", function() {
it("should rename all fields with the prefix", function() {
const config = { "old.load_plugin": false, "old.value": 1, "old_other.value": 2 };
lib.renameConfigPrefix(config, "old.", "new.");
assert.deepEqual(config, { "new.load_plugin": false, "new.value": 1, "old_other.value": 2 });
});
});
describe("renamePermission()", function() {
it("should rename the permission", function() {
const role = { permissions: ["foo.run", "foo.view"] };
assert.equal(lib.renamePermission(role, "foo.run", "foo.frobnicate"), role);
assert.deepEqual(role.permissions, ["foo.frobnicate", "foo.view"]);
});
it("should not duplicate a permission the role already has", function() {
const role = { permissions: ["foo.run", "foo.frobnicate"] };
lib.renamePermission(role, "foo.run", "foo.frobnicate");
assert.deepEqual(role.permissions, ["foo.frobnicate"]);
});
});
describe("renamePermissionPrefix()", function() {
it("should rename all permissions with the prefix", function() {
const role = { permissions: ["old.view", "old_other.view", "core.admin"] };
lib.renamePermissionPrefix(role, "old.", "new.");
assert.deepEqual(role.permissions, ["new.view", "old_other.view", "core.admin"]);
});
});
});
+46
View File
@@ -110,6 +110,52 @@ describe("lib/plugin_loader", function() {
);
});
});
describe("loadPluginMigrations()", function() {
const baseDir = path.resolve("temp", "test", "plugin", "migration_plugin");
before(async function() {
await fs.mkdir(baseDir, { recursive: true });
await fs.writeFile(path.join(baseDir, "migrations.js"), `
export default function(context) {
context.hooks.controllerConfig.attach(context.plugin.name, config => {
config.migratedBy = context.plugin.name;
return config;
});
}
`);
await fs.writeFile(path.join(baseDir, "throwing.js"), `
export default function() { throw new Error("load failed"); }
`);
});
afterEach(function() {
lib.ControllerConfig.pluginMigrations.clear();
lib.Config.pluginVersions.clear();
});
function info(entrypoint) {
return {
name: "test",
version: "1.0.0",
packagePath: path.join(baseDir, "package.json"),
migrationEntrypoint: entrypoint,
};
}
it("should attach migrations from the migration entrypoint", async function() {
await lib.loadPluginMigrations([info("migrations.js"), { name: "other" }]);
assert.deepEqual([...lib.ControllerConfig.pluginMigrations.attached], ["test"]);
assert.deepEqual(lib.ControllerConfig.pluginMigrations.reduce({}), { migratedBy: "test" });
});
it("should register the version of every plugin", async function() {
await lib.loadPluginMigrations([info("migrations.js"), { name: "other", version: "2.0.0" }]);
assert.deepEqual([...lib.Config.pluginVersions], [["test", "1.0.0"], ["other", "2.0.0"]]);
});
it("should reject with a PluginError if the entrypoint throws", async function() {
await assert.rejects(
lib.loadPluginMigrations([info("throwing.js")]),
err => err instanceof lib.PluginError && err.pluginName === "test"
);
});
});
describe("loadPlugin()", function() {
const baseDir = path.join("temp", "test", "plugin");
const functionPlugin = path.resolve(baseDir, "function_plugin");
+103
View File
@@ -0,0 +1,103 @@
import assert from "node:assert/strict";
import { execFile } from "node:child_process";
import fs from "node:fs/promises";
import path from "node:path";
import { promisify } from "node:util";
import { publishMods, uploadModRelease } from "../publish.js";
const execFileAsync = promisify(execFile);
describe("mod publishing", function() {
let originalFetch;
let directory;
beforeEach(async function() {
originalFetch = global.fetch;
await fs.mkdir("temp", { recursive: true });
directory = await fs.mkdtemp("temp/publish-test-");
});
afterEach(async function() {
global.fetch = originalFetch;
await fs.rm(directory, { recursive: true, force: true });
});
it("sends multipart mod and file fields accepted by native fetch", async function() {
const name = "multipart_test";
const filename = `${name}_1.0.0.zip`;
const contents = Buffer.from([0x50, 0x4b, 0x03, 0x04, 0x00, 0xff]);
const file = path.join(directory, filename);
await fs.writeFile(file, contents);
let calls = 0;
global.fetch = async (url, options) => {
calls += 1;
const request = new Request(url, options);
assert.match(request.headers.get("content-type"), /^multipart\/form-data; boundary=/);
const form = await request.formData();
if (calls === 1) {
assert.equal(url, "https://mods.factorio.com/api/v2/mods/releases/init_upload");
assert.equal(form.get("mod"), name);
return Response.json({ upload_url: "https://upload.example.test/release" });
}
assert.equal(url, "https://upload.example.test/release");
const uploaded = form.get("file");
assert.equal(uploaded.name, filename);
assert.deepEqual(Buffer.from(await uploaded.arrayBuffer()), contents);
return Response.json({ success: true });
};
await uploadModRelease(file, name);
assert.equal(calls, 2);
});
it("reads archives from the supplied directory and uploads unpublished versions in order", async function() {
const name = "ordered_publish_test";
for (const version of ["1.0.9", "1.0.10", "1.0.2"]) {
await fs.writeFile(path.join(directory, `${name}_${version}.zip`), version);
}
const uploaded = [];
let uploading = false;
global.fetch = async (url, options) => {
if (url === `https://mods.factorio.com/api/mods/${name}`) {
return Response.json({ releases: [{ file_name: `${name}_1.0.2.zip` }] });
}
const form = await new Request(url, options).formData();
if (url.endsWith("/init_upload")) {
assert.equal(uploading, false, "previous upload must complete first");
uploading = true;
return Response.json({ upload_url: "https://upload.example.test/release" });
}
const file = form.get("file");
uploaded.push([file.name, await file.text()]);
await new Promise(resolve => setImmediate(resolve));
uploading = false;
return Response.json({ success: true });
};
await publishMods(directory);
assert.deepEqual(uploaded, [
[`${name}_1.0.9.zip`, "1.0.9"],
[`${name}_1.0.10.zip`, "1.0.10"],
]);
});
it("runs the CLI in dry-run mode without uploading unpublished archives", async function() {
const filename = "cli_publish_test_1.0.0.zip";
await fs.writeFile(path.join(directory, filename), "dry-run archive");
const mockFetch = `
import assert from "node:assert/strict";
global.fetch = async (url, options = {}) => {
assert.equal(url, "https://mods.factorio.com/api/mods/cli_publish_test");
assert.equal(options.method ?? "GET", "GET", "dry-run must not upload");
return Response.json({ releases: [] });
};
`;
const { stdout, stderr } = await execFileAsync(process.execPath, [
"--import", `data:text/javascript,${encodeURIComponent(mockFetch)}`,
"publish.js", "factorio", directory, "--dry-run",
]);
assert.equal(stderr, "");
assert(stdout.includes("Requesting current releases of cli_publish_test"));
assert(stdout.includes(path.join(directory, filename)));
});
});
+1 -1
View File
@@ -7,7 +7,7 @@
{ "path": "./packages/ctl/" },
{ "path": "./packages/web_ui/" },
{ "path": "./plugins/global_chat/" },
{ "path": "./plugins/inventory_sync/" },
{ "path": "./plugins/player_sync/" },
{ "path": "./plugins/player_auth/" },
{ "path": "./plugins/research_sync/" },
{ "path": "./plugins/statistics_exporter/" },