Since the move to ES modules and pnpm 12, packages installed from npm
fail to start:
- The package imports of @clusterio/lib point Node at TypeScript
sources, which Node refuses to strip types from inside node_modules.
- The controller, host and ctl import their package.json from next to
the compiled entrypoint, a copy pnpm leaves out of the package.
- ctl, host and web_ui have no .npmignore, so pnpm packs them using the
repository .gitignore and leaves out nearly all of dist/.
Resolve the package imports to the compiled files at runtime while
keeping the sources for type checking, import package.json by package
name so it resolves to the real file, and add .npmignore files so pnpm
packs the same files as npm.
The old name read as an unconditional unsubscribe. Log following has
its own subscription outside SubscriptionController, so check
core.log.follow on permission changes too. Move the per-connection
work into one helper and split the tests into userPermissionsUpdated
and rolesUpdated, checking the account update event is sent.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add controller.max_send_buffer_size in MiB, default 64 and 0 for no
limit. Close with 1013 instead of 1008 on overflow: clients treat 1008
as fatal, so a host would shut down and the web UI would log out. With
1013 the client resumes, the controller answers with invalidate since
the session has ended, and the client registers a new session.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lowering the limit at runtime would leave connections holding more
filters than allowed, so read it once when the controller starts.
Move the limit tests into the subscribe and replace groups.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The controller HTTP access log and the express error log wrote the full
request headers at the http level, which both default log transports
keep. Clients send their access token in X-Access-Token or
Authorization: Bearer, so user and host tokens ended up in plain text
in controller and cluster logs, and the cluster log can be read back
over the API with core.log.query.
Mask the authorization, x-access-token, cookie and proxy-authorization
headers before logging.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Every message sent over a link is kept in the send buffer until the
other side acknowledges it with a heartbeat, so it can be resent if
the session resumes. Nothing forced a peer to acknowledge, so a client
that kept heartbeating without a sequence number while subscribed to a
busy event made the controller buffer messages until it ran out of
memory.
Store buffered messages serialised, track their total size, and close
the connection once it goes over maxSendBufferSize. The controller
sets this to 64 MiB for the connections it accepts. A client that far
behind could not resume in a reasonable time anyway.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The permission for a subscription was only checked when subscribing,
so a control connection kept receiving events after the user's roles
or role permissions changed to no longer allow it.
Add SubscriptionController.unsubscribeUnpermitted which rechecks every
subscription of a link and removes the ones the user is no longer
permitted, and call it for the affected control connections from
userPermissionsUpdated and rolesUpdated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A subscriber could keep sending subscribe requests with new filters and
the controller would add every one of them to the subscriber's filter
set without bound, growing memory until it ran out.
Add controller.max_subscription_filters (default 10000) and reject
subscribe and replace requests that would leave a subscriber holding
more filters than that for one event.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A handler that only updates gauges has nothing to yield. collect() already
dropped undefined results at runtime, now its type says so too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Controller.loadJsonObject(path, true) returned {} for a missing file, so
loadPlugins' catch never ran and a plugin whose web build was expected but
missing was not reported (the second half of #1046; #1047 fixed the path).
With throwOnMissing set, every error now throws; without it only a missing
file is ignored, so a malformed JSON file throws again as before #1047.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BGUULoiHNbAegwMK4PSjHJ
* Add a request to mark hosts and the controller as needing a restart
Not every plugin arrives through PluginInstallRequest. Plugins checked
out into external_plugins or installed by hand on the machine never
flag a restart, so the web UI shows nothing and tooling that acts on
the flag never fires.
HostMarkRestartRequiredRequest and ControllerMarkRestartRequiredRequest
set config.restartRequired, guarded by the existing restart
permissions. They are exposed as "Mark Restart Required" in the control
dropdown of the host and controller pages, hidden once the flag is set,
and as mark-restart-required ctl commands. The mark shows up on the
next system metrics poll.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Cover the mark-restart-required ctl commands
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Not every plugin arrives through PluginInstallRequest. Plugins checked
out into external_plugins or installed by hand on the machine never
flag a restart, so the web UI shows nothing and tooling that acts on
the flag never fires.
HostMarkRestartRequiredRequest and ControllerMarkRestartRequiredRequest
set config.restartRequired, guarded by the existing restart
permissions. They are exposed as "Mark Restart Required" in the control
dropdown of the host and controller pages, hidden once the flag is set,
and as mark-restart-required ctl commands. The mark shows up on the
next system metrics poll.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The behaviour of creating the config file if it's missing while
convenient causes confusion and problems when the tools are run
in the wrong folder. They end up creating a shadow installation
and misdirect the user with nonsensical error messages, instead
pointing to the real problem which is that the tool can't find
the configuration needed to actually run.
Require a config file to be present at startup and refuse to run if this
file is missing. To support the odd use-case of creating the config
when missing a --create-config flag is added. To conveniently generate
a config file to edit out from the "config create" command is added.
Ctl ended up calling its main class for "Control" for historical
reasons, along with its config file and plugin entrypoint.
The plugin entrypoint was renamed to ctl in 808eda0513. This
finishes that renaming by also renaming the main class, default config
file, config-control command, and main entrypoint.
The config-control command is renamed to config to be consistent with
the host and controller's config commands.
The term "control" is still kept in the config and message locations as
a common specifier for both ctl and web_ui.
The instance name on a log line is whatever the host sent, so a host can
attribute a line for one of its own instances to the wrong name. Compare
it with the name the controller has on record and replace it when they
differ.
Host connections can live for days, so the warning suppression expires
after five minutes instead of latching for the life of the connection,
and the bogus instance and misaligned name warnings share it so the two
together cannot spam the log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
handleDebugDumpWsRequest sets clusterio_ignore_dump on the current
socket so the WsServer send wrapper does not echo the dumper's own
frames back into debugEvents. The flag was only set on the connector
"connect" event, but WsServerConnector.continue() installs a fresh
socket and emits "resume", not "connect", so a resumed session lost
the flag.
After a resume the next outgoing frame was echoed to debugEvents,
ws_dumper sent a DebugWsMessageEvent, the wrapper echoed that too,
and so on: infinite synchronous recursion ending in a RangeError:
Invalid string length from JSON.stringify in link.ts sendEvent. That
uncaught error made handleUnhandledErrors exit the controller.
Repro: admin sends DebugDumpWsRequest, the control connection drops
and resumes (e.g. a network blip), controller crashes on resume.
Set the flag on "resume" as well as "connect".
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Cooldude2606 <25043174+Cooldude2606@users.noreply.github.com>
Log messages share the root cause with the save details updates fixed in
the previous commit: the controller trusted host supplied instance
fields. LogMessageEvent only declares level and message in its schema,
but the Ajv instance is built without removeAdditional and the schema
does not set additionalProperties, so instance_id and instance_name sent
by a host survive validation and pass through fromJSON untouched. A host
could tag a log line with an instance assigned to another host and have
the controller store and serve it as that instance's output.
Lift the assignment check out of handleInstanceSaveDetailsUpdatesEvent
into a private method and use it from both handlers.
Strip the instance fields instead of dropping the line. A host racing a
reassignment does emit instance log lines for an instance the controller
has already moved, and dropping those loses real output exactly when a
migration is going wrong. The line is still genuine output from the host,
so it is recorded as a host level message, which grants the host nothing
it could not already log directly. Warn once per connection so a
misbehaving host cannot flood the log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Drop the special case that deferred terminating the connection issuing
the delete, so deleting yourself behaves the same as revoke-token and
every session of the deleted user is closed before the handler returns.
Set tokenValidAfter to the current time in createUser instead of 0.
Deleting a user and recreating it under the same name left tokens issued
before the deletion valid, as the check is iat < tokenValidAfter. Doing
it in createUser covers every path that makes a user. A token signed in
the same second as creation is still accepted, so create-admin and the
other production paths sign after the user exists and are unaffected.
The integration harness signed the control token at import time, before
bootstrap create-admin created the user. Sign it in the root before hook
instead and reuse it in link.js.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ControlConnection keeps the User object it authenticated with and checks
permissions against it. Deleting the user removed the record but left
its sessions open, so a deleted user kept all its permissions until the
WebSocket closed.
Repro: create "victim" with role 0 and connect as it. Another admin
sends UserDeleteRequest("victim"). Victim can still send
UserCreateRequest("mallory") and UserUpdateRolesRequest("mallory", [0]).
Terminate the deleted user's connections like revoke-token does. When
a user deletes itself, defer terminating the requesting connection so
the response is sent first.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The close handler removed the log subscription transport from the global
lib logger, but updateLogSubscriptions adds it to the controller's
clusterLogger. Every control connection that subscribed to logs stayed
attached to clusterLogger after closing, keeping the connection alive and
running every later cluster log line through its filter.
handleDebugDumpWsRequest added a new debugEvents listener on each call
and overwrote ws_dumper, so the close handler only removed the last one.
Repeated requests stacked listeners, triggered MaxListenersExceededWarning
after 11 calls, and left the earlier listeners behind. Ignore the request
if the connection is already dumping.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
handleInstanceSaveDetailsUpdatesEvent trusted the instanceId in the
event and in each SaveDetails without checking the instance was assigned
to the sending host. InstanceSaveDetailsUpdatesEvent has host in its src
list and validateIngress only checks the source host id, not the
payload, so any authenticated host could corrupt another instance's
controller-side save records.
Repro: a host owning nothing sends the event with a SaveDetails for
instance 1002 (on another host) to inject a bogus save, or sends empty
updates with instanceId 1002 to make the delete-missing sweep wipe that
instance's save list.
Guard both paths: bail if event.instanceId names an instance not
assigned to this host, and drop the whole event if any update is for an
instance not assigned here. Matches handleInstanceStatusChangedEvent's
bogus-update warning.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
assignInstance sets instance.assigned_host to the new host before
sending InstanceAssignInternalRequest. If the host fails that request
the controller kept the new host id while the host never added the
instance. Assigning to the same host again then returned early as a
no-op. Repro: make the host's instances directory read-only, assign
instance 3001 to host 10 (EACCES on mkdir), make it writable, assign
again (ok, nothing happens), start (instance does not exist on host).
Now a failed send leaves the instance unassigned and rethrows.
InstanceCreateRequest applied instance.assigned_host from the request
config, creating an instance assigned to a host that never heard of
it. Repro: InstanceCreateRequest({"instance.id": 4001, "instance.name":
"c1", "instance.assigned_host": 10}). Reject a non-null value with the
same error the config set requests use.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
yargs, @types/yargs, semver and @types/semver are used by several packages at the same version. Also point the remaining direct references to @swc/core and react-router-dom at their existing catalog entries.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Raise every caret range to the current latest within its major, in the
root package.json, the pnpm-workspace.yaml catalog, the per-package
package.json files and the create plugin template.
Majors taken: eslint 10, @eslint/js 10, globals 17, nyc 18,
@types/express 5, @types/node 24, finalhandler 2, pidusage 4,
style-loader 4, css-loader 7, yargs 18 (all packages), chalk 6.
Majors skipped: mocha 12 (integration suite fails under it, 59 tests,
starting with a 30 s timeout in clusteriocontroller run; kept at
^11.8.0), inquirer (10+ is a rewrite on @inquirer/prompts where
ui.Prompt.prototype.close no longer exists, so the win32 readline
workaround in create.js would have to go and cannot be verified here),
react/antd/react-router-dom/@types/react, typescript 7,
@sinclair/typebox, webpack-merge (exact pin), webpack-dev-middleware,
webpack-manifest-plugin, winston-daily-rotate-file.
Code adaptations: the two /api/stream/:id handlers in routes.ts are
typed Request<{ id: string }> because @types/express 5 types
req.params values as string | string[]; eslint.config.mjs adds
newIsCapExceptions: ["constructor"] because eslint 10.10 new-cap now
reports `new this.constructor()`.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The metrics endpoint fans out a gather to every connected host, so
anyone who could reach the controller could read cluster statistics and
put load on all hosts at will. Scraping now needs a user token holding
the new core.controller.metrics permission. The token is accepted via
Authorization: Bearer as well as X-Access-Token because Prometheus can
only send the former. Concurrent scrapes share one gather so hosts see
at most one metrics request at a time.
The three connection gauges in controller.ts now no-op before the
controller exists, like the other gauges there, so the endpoint can be
tested against the mock controller.
Closes#1009
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Rethrow errors other than ENOENT and ENOTDIR when reading the package
spec as a directory, mark the tests that run npm as slow so FAST_TEST
skips them, and use ./external_plugins/my_plugin in the README examples.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Plugins with only host, instance or ctl code have no reason to ship a
webpack build, yet the plugin list showed "Missing dist/web/manifest.json"
as an error for them and the controller logged a warning on every start.
Decide whether a build is expected from the plugin declaration, using the
same rule the create tool uses to decide when to generate one: a web or
controller entrypoint, or any config fields. Plugins outside that rule get
no bundle path and no error, the web UI skips loading them, and the plugin
pages show them as having no web module instead of an error or a prompt to
reload.
Fixes#1004
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Config files hold the auth secret and tokens, so the default 0o644 lets
every user on the system read them. Config.save, the ctl config written
by the controller bootstrap, ctl host create-config and the config
migration in create now pass mode 0o600.
safeOutputFile also applies the mode to the temporary file it writes,
since writeFile only honours mode on creation and a leftover temporary
from a failed write would otherwise keep its old mode.
Closes#1005
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Replace invokeHook and the per plugin on* methods with AsyncHook
collections on the controller, host, instance, clusterioctl and web
control objects. Plugin entrypoints export a default function that
receives a load context and attaches handlers to the hooks it needs.
Class exports still load through fromContext with a deprecation
warning. The create templates and docs use the entrypoint function.
This is clusterio/clusterio#921 recreated on top of master after the
ESM migration and the PluginDeclaration permissions change.
Closes#730
Co-Authored-By: Cooldude2606 <25043174+Cooldude2606@users.noreply.github.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Adds a permissions array to PluginDeclaration, a type-level Permissions
registry that plugins extend with module augmentation, and
registerPluginPermissions() called from all entrypoints. checkPermission
and the web UI hasPermission helpers now take PermissionName so typos are
compile errors. Core definitions move into a corePermissions array that
the interface derives its keys from.
Closes#987
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Cooldude2606 <25043174+Cooldude2606@users.noreply.github.com>
Migrate the usage of CommonJS specific code and imports to ESM. Roughly
summarised this:
- Changes all require() calls to import statement or expressions.
- Changes usage of module.exports to export statements.
- Replaces require.resolve with import.meta.resolve.
- Replaces conditional imports with conditional subpath imports.
- Replaces module === require.main with import.meta.main.
- Adds the full filename with extention to all relative imports.
- Changes all usage of __dirname and __filename to their import.meta
equivalents.
- Sets "type": "module" in package.json files.
- Removes usage of "use strict".
- Adds .ts an an alias of .js for the webpack build.
Due to differences in how require and import resolves files the plugin
entrypoints will have to include the filename and extension after this
migration in order to work.
Values in the assetsByChunkName object are arrays not strings. This
caused the dev mode to fail after 2f4e0c4418 added an additional step
filtering out the static/ prefix from what it expected to be a string.
The ESM import of yargs does not support the singleton way of using
yargs, instead we have to call the yargs function to get a parser that
can be augmented with the options.
The .argv property has also been deprecated. Replace it with a call to
.parse() instead.
Danielv123