When a player connects with a Factorio version and set of built in mods
no lobby runs, the lobby plugin now creates one on a connected host that
has the version installed or can download it, with a mod pack matching
the player's built in mods. This lets a cluster accept players on
several Factorio versions on the same port without any setup. It can be
turned off with lobby.auto_create.
To pick the mod pack the proxy now waits for the whole connection
confirm before routing and passes the mods listed in it to the
playerProxyRoute hook. Hosts answer the new HostFactorioVersionsRequest
with the versions they have installed.
The server list in lobbies only shows instances on the Factorio version
of the lobby, with a checkbox to also show the others greyed out.
The player proxy now listens on 34197 by default. New hosts default to
the port range 34200-34299 so they don't hand that port to instances,
and the controller logs an error and runs without the proxy if the port
is taken, as hosts with the old range only skip it when the controller
already holds it.
Lobby instances start the save lobby.zip, created on first start as a
one tile map with no autoplace and with the freeplay scenario replaced by
an empty one. This uses the new chooseSave instance hook, which lets
plugins pick or create the save to start, and setSaveScenario from
@clusterio/host, which sets the scenario the save patcher keeps loading.
Map gen settings can't use a width and height of 0 for this, as Factorio
takes 0 to mean an unlimited map.
The address players reach the proxy on moves from lobby.public_address
to controller.player_proxy_public_address. When it's set the player_auth
login page lists it once for the instances behind the proxy instead of
their own addresses, and lobbies with player_auth loaded get a button
that opens the web interface login dialog.
Players connecting through the player proxy are sent to a lobby
instance running their Factorio version, where an in-game list shows
the instances in the cluster. Picking one starts it if needed and moves
the player there with LuaPlayer.connect_to_server once it's running,
while the proxy keeps routing the player to it until they have joined,
covering the second connection Factorio makes to fetch a user key.
Players connecting to the dedicated port of a stopped instance wait in
a lobby while it starts instead of timing out.
The cluster can be listed in the Factorio server browser as one game
per lobby version with the players online in the whole cluster, and
instances with a dedicated proxy port can be listed on their own,
including while stopped. The listed build comes from probing the
instance through the proxy, as the browser only shows games with the
exact version and build of the client.
The controller now creates the player proxy before loading plugins so
they can use it from their entrypoints.
The proxy now parses the full message header, so confirms and fragmented
ConnectionRequestReplyConfirm messages (which carry the mod list) are
forwarded intact, with the server request id patched into the first
fragment. It resends the connection request until the instance answers,
starts over when a client reconnects from the same socket (as Factorio
does after LuaPlayer::connect_to_server), answers pings from the server
browser and tells clients the same 508 byte max packet size Factorio
servers use.
Routing is decided by the new playerProxyRoute controller hook, which is
given the name and version of the player, then a dedicated port set with
instance.player_proxy_port, the instance the player was last on (taken
from the user's instance stats instead of a separate routing store), and
controller.player_proxy_default_instance_id.
The proxy is now off by default, as port 34197 is in the default
host.factorio_port_range. Leftovers of the per-instance proxy on hosts
are removed.
Controller.viteServer becomes devServer, matching the devMiddleware
name it replaced. The static_url limitation is described as one of the
web build rather than webpack.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Vite serves any file under fs.allow through /@fs/, and the default
allow list is the workspace root. In a clusterio checkout that is also
the usual data directory, so with --dev any client that could reach the
controller could fetch config-controller.json, config-ctl.json and the
database without logging in. webpack-dev-middleware only served the
compiled bundles.
Allow the controller, web_ui, lib and dev plugin directories and the
node_modules directories Node.js would resolve their imports from.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace ESLint, typescript-eslint, eslint-plugin-node and the flat
config helpers with vite-plus, which runs Oxlint for vp lint. The rules
are carried over from eslint.config.mjs by @oxlint/migrate into the lint
section of the root vite.config.ts. The formatting rules ESLint
deprecated run as @stylistic rules through Oxlint's JS plugin support,
and disable comments naming them are updated to the @stylistic names.
@stylistic understands TypeScript syntax that the core rules skipped,
so the autofix adds missing semicolons and spaces in type literals and
aliases. Leading | and & in multiline types stay where they were.
Oxlint's node/no-sync also reports identifiers ending in Sync that are
not calls. The test helper techSync is renamed and the two other names
are ignored.
Refs #1127
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace webpack and Module Federation with Vite 8. The controller build
emits one entry per shared module (lib, web_ui, typebox, antd, react,
react-dom, react-router) and lists them in manifest.json. The controller
turns that into an import map in index.html, and plugins are built as
ES modules that import the shared modules by name and are loaded with
import().
The shared Vite setup lives in @clusterio/web_ui/vite.mjs. Plugins use
clusterioPlugin(), which exposes index, package.json and web/index under
the same keys the webpack template used. The --dev and --dev-plugin
flags now run the Vite dev server in middleware mode with hot module
replacement.
Browser polyfills move from web_ui to lib, which is the code that needs
them, and lib sets the process and Buffer globals itself.
Refs #1127
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Host not connected, host shutting down and instance not running are
now UnavailableError (503) instead of ConflictError. The request error
migration notes move to a new 2.1 guide, since this ships in 2.1.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Error responses now carry the registered name of the thrown error's
class, and the requester rejects with an instance of that class instead
of a generic RequestError. Error classes are registered like message
classes, and plugins list their own in `errors`.
RequestError is replaced by a hierarchy following the HTTP status codes:
ClientError (ValidationError, ForbiddenError, NotFoundError,
TimeoutError, ConflictError) and ServerError (InternalServerError).
PermissionError is PermissionDeniedError and the config errors
InvalidValue, InvalidField and InvalidAccess gain an Error suffix.
Link logs only errors whose class is not registered. These arrive on the
other side as InternalServerError with the original message and stack,
so an error forwarded through the controller is logged once, where it
was thrown.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Controller.fromConfig and Host.fromConfig now take the config, then the
plugin infos, then the logger or connector.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Controller.bootstrap and Host.bootstrap returned constructor arguments
that the caller had to spread into new Controller/Host, and
DatastoreProvider.bootstrap did the same for datastores. They are
replaced by Controller.fromConfig, Host.fromConfig and
KeyValueDatastore/SubscribableDatastore.fromProvider, which load the
data and return the finished instance.
Closes#1126
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Raise the catalog and package.json floors to the latest release within
each major, and move typebox to 0.34, react-router-dom to 7,
winston-daily-rotate-file to 5, webpack-dev-middleware to 8 and
webpack-manifest-plugin to 6. None of these need code changes.
express 5.3.0 requires proxy-addr ^2.0.8 and qs ^6.16.0, which puts a
floor under the critical proxy-addr and high qs advisories for anyone
installing the published packages. webpack-dev-middleware 8 fixes a
high advisory in the controller's dev mode, and react-router 7 fixes two
moderate ones.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Replace jszip with yauzl and yazl
jszip loads a whole archive into memory to read it and buffers the
output when writing. Saves and mods can be several hundred megabytes,
so read zips with yauzl, which only reads the central directory and
streams entries on demand, and write them with yazl, which streams the
output.
lib gains a ZipArchive wrapper around yauzl with the lookup helpers the
callers need, findRoot() now takes it instead of a JSZip. Saves are
patched by streaming every kept entry into a new zip, mod info.json is
read straight from the file, exports are streamed to the controller and
Factorio zip downloads go through a temp file instead of a buffer.
Patching now inflates and deflates each entry again where jszip reused
the compressed data, on a 336 MB synthetic save that is 7 s instead of
0.7 s, with peak memory going from 416 MB to 151 MB.
Closes#1002
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Copy save entries without recompressing them
yazl inflates and deflates every entry it is given, so patching a save
took ten times longer than with jszip, which reused the compressed
data. Add a ZipWriter to lib that streams entries out as they are
added and copies entries from a ZipArchive byte for byte, with ZIP64
structures when sizes or offsets need them, and use it in patch().
On a 336 MB synthetic save patching now takes 0.4 s at 143 MB peak RSS
against 0.75 s at 417 MB with jszip. Factorio loads the result.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Handle save write errors before the patch loop awaits them
A write error on the temporary save, such as a full disk, rejected the
pipeline promise while addEntry() or addBuffer() was still pending. Nothing
was listening yet, so the host's unhandledRejection handler exited the
process instead of failing the instance start.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Keep the export zip cache local to each export
The cache was module level, so when two instances on one host exported at
once, the first to finish closed archives the other was still reading.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Start writing zips before adding files from disk
yazl reads files passed to addFile() as soon as they are added. An
unreadable file emitted an error while the directory traversal was still
running, before the output pipeline existed, and the process exited on the
uncaught error. build_mod and zipDirectory now start the pipeline first and
reject instead. build_mod also removes the partial zip on failure.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
The staging copy made by Config.set() went through the constructor, so
plugin migrations ran a second time on the live fields. A version-gated
migration was applied twice when setPluginVersions() changed the version.
Staging now copies the fields with update() instead.
A config saved without <plugin>.version got the running version as its
default, so setPluginVersions() saw no change and nothing saved the stamp.
The constructor now writes the registered plugin versions after the
migrations run and marks the config dirty when that changes a loaded value.
Controller.bootstrap() saves the instances that came out dirty, and
setPluginVersions() is gone.
Restructure the SyncHook and plugin config tests by method, and suggest the
controller config version for role migrations in the docs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Plugins had no way to update the config fields and permissions core stores
for them, so renaming inventory_sync needed migrations in core packages.
A plugin can now declare a migrationEntrypoint. It's loaded on the
controller and hosts before configs and roles are read, for every plugin in
the plugin list, and attaches handlers to sync hooks for the controller,
host and instance configs and roles. lib gets helpers to rename config
fields and permissions, and each config gets a hidden <plugin>.version field
set after load so migrations can tell which version saved the data.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The controller logs the url and query of every HTTP request. Plugins
doing OAuth get the authorization code as a query parameter on their
callback route, so it ended up in the log. redactUrl now accepts a
path, code joins the redacted parameter names, and redactQuery covers
the parsed query in the log metadata.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Fix running Clusterio from published packages
Since the move to ES modules and pnpm 12, packages installed from npm
fail to start:
- The package imports of @clusterio/lib point Node at TypeScript
sources, which Node refuses to strip types from inside node_modules.
- The controller, host and ctl import their package.json from next to
the compiled entrypoint, a copy pnpm leaves out of the package.
- ctl, host and web_ui have no .npmignore, so pnpm packs them using the
repository .gitignore and leaves out nearly all of dist/.
Resolve the package imports to the compiled files at runtime while
keeping the sources for type checking, import package.json by package
name so it resolves to the real file, and add .npmignore files so pnpm
packs the same files as npm.
* Load plugins installed from npm by package name
Plugins installed from npm are listed by package name, but the plugin
loader joined that name onto the main and entrypoint paths and imported
the result as a file, which resolves relative to the working directory.
Every npm installed plugin then failed to load. Resolve these paths from
the directory of the plugin's package.json instead.
WsServer.handleConnection now listens for errors on every socket for its
whole life, and every socket a WsServerConnector holds comes from there,
new sessions and resumed ones alike. The connector's own listener only
logged the same error a second time, so it is removed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F7obMEi4L4PzUQ8w8q9Uyz
Since the move to ES modules and pnpm 12, packages installed from npm
fail to start:
- The package imports of @clusterio/lib point Node at TypeScript
sources, which Node refuses to strip types from inside node_modules.
- The controller, host and ctl import their package.json from next to
the compiled entrypoint, a copy pnpm leaves out of the package.
- ctl, host and web_ui have no .npmignore, so pnpm packs them using the
repository .gitignore and leaves out nearly all of dist/.
Resolve the package imports to the compiled files at runtime while
keeping the sources for type checking, import package.json by package
name so it resolves to the real file, and add .npmignore files so pnpm
packs the same files as npm.
WsServer attached no error listener to a new socket. WsServerConnector adds
one, but only after the handshake. ws emits an error for an invalid frame,
for example a close frame with the reserved status code 1006, and an error
event with no listener throws. That stopped the controller.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F7obMEi4L4PzUQ8w8q9Uyz
The old name read as an unconditional unsubscribe. Log following has
its own subscription outside SubscriptionController, so check
core.log.follow on permission changes too. Move the per-connection
work into one helper and split the tests into userPermissionsUpdated
and rolesUpdated, checking the account update event is sent.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add controller.max_send_buffer_size in MiB, default 64 and 0 for no
limit. Close with 1013 instead of 1008 on overflow: clients treat 1008
as fatal, so a host would shut down and the web UI would log out. With
1013 the client resumes, the controller answers with invalidate since
the session has ended, and the client registers a new session.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lowering the limit at runtime would leave connections holding more
filters than allowed, so read it once when the controller starts.
Move the limit tests into the subscribe and replace groups.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The controller HTTP access log and the express error log wrote the full
request headers at the http level, which both default log transports
keep. Clients send their access token in X-Access-Token or
Authorization: Bearer, so user and host tokens ended up in plain text
in controller and cluster logs, and the cluster log can be read back
over the API with core.log.query.
Mask the authorization, x-access-token, cookie and proxy-authorization
headers before logging.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Every message sent over a link is kept in the send buffer until the
other side acknowledges it with a heartbeat, so it can be resent if
the session resumes. Nothing forced a peer to acknowledge, so a client
that kept heartbeating without a sequence number while subscribed to a
busy event made the controller buffer messages until it ran out of
memory.
Store buffered messages serialised, track their total size, and close
the connection once it goes over maxSendBufferSize. The controller
sets this to 64 MiB for the connections it accepts. A client that far
behind could not resume in a reasonable time anyway.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The permission for a subscription was only checked when subscribing,
so a control connection kept receiving events after the user's roles
or role permissions changed to no longer allow it.
Add SubscriptionController.unsubscribeUnpermitted which rechecks every
subscription of a link and removes the ones the user is no longer
permitted, and call it for the affected control connections from
userPermissionsUpdated and rolesUpdated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A subscriber could keep sending subscribe requests with new filters and
the controller would add every one of them to the subscriber's filter
set without bound, growing memory until it ran out.
Add controller.max_subscription_filters (default 10000) and reject
subscribe and replace requests that would leave a subscriber holding
more filters than that for one event.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A handler that only updates gauges has nothing to yield. collect() already
dropped undefined results at runtime, now its type says so too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Controller.loadJsonObject(path, true) returned {} for a missing file, so
loadPlugins' catch never ran and a plugin whose web build was expected but
missing was not reported (the second half of #1046; #1047 fixed the path).
With throwOnMissing set, every error now throws; without it only a missing
file is ignored, so a malformed JSON file throws again as before #1047.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BGUULoiHNbAegwMK4PSjHJ
* Add a request to mark hosts and the controller as needing a restart
Not every plugin arrives through PluginInstallRequest. Plugins checked
out into external_plugins or installed by hand on the machine never
flag a restart, so the web UI shows nothing and tooling that acts on
the flag never fires.
HostMarkRestartRequiredRequest and ControllerMarkRestartRequiredRequest
set config.restartRequired, guarded by the existing restart
permissions. They are exposed as "Mark Restart Required" in the control
dropdown of the host and controller pages, hidden once the flag is set,
and as mark-restart-required ctl commands. The mark shows up on the
next system metrics poll.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Cover the mark-restart-required ctl commands
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Not every plugin arrives through PluginInstallRequest. Plugins checked
out into external_plugins or installed by hand on the machine never
flag a restart, so the web UI shows nothing and tooling that acts on
the flag never fires.
HostMarkRestartRequiredRequest and ControllerMarkRestartRequiredRequest
set config.restartRequired, guarded by the existing restart
permissions. They are exposed as "Mark Restart Required" in the control
dropdown of the host and controller pages, hidden once the flag is set,
and as mark-restart-required ctl commands. The mark shows up on the
next system metrics poll.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The behaviour of creating the config file if it's missing while
convenient causes confusion and problems when the tools are run
in the wrong folder. They end up creating a shadow installation
and misdirect the user with nonsensical error messages, instead
pointing to the real problem which is that the tool can't find
the configuration needed to actually run.
Require a config file to be present at startup and refuse to run if this
file is missing. To support the odd use-case of creating the config
when missing a --create-config flag is added. To conveniently generate
a config file to edit out from the "config create" command is added.
Danielv123