Host not connected, host shutting down and instance not running are
now UnavailableError (503) instead of ConflictError. The request error
migration notes move to a new 2.1 guide, since this ships in 2.1.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Error responses now carry the registered name of the thrown error's
class, and the requester rejects with an instance of that class instead
of a generic RequestError. Error classes are registered like message
classes, and plugins list their own in `errors`.
RequestError is replaced by a hierarchy following the HTTP status codes:
ClientError (ValidationError, ForbiddenError, NotFoundError,
TimeoutError, ConflictError) and ServerError (InternalServerError).
PermissionError is PermissionDeniedError and the config errors
InvalidValue, InvalidField and InvalidAccess gain an Error suffix.
Link logs only errors whose class is not registered. These arrive on the
other side as InternalServerError with the original message and stack,
so an error forwarded through the controller is logged once, where it
was thrown.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The send buffer limit in controller.max_send_buffer_size is set in MiB, but
buffered messages were measured with String.length, which counts UTF-16 code
units. Messages with non-ASCII text could go over the configured size in bytes
before the session was ended.
Measure each serialised message with Buffer.byteLength when it is buffered,
keep that size with the entry, and subtract the same size when it is
acknowledged. The web UI build already provides Buffer through the buffer
polyfill.
Fixes#1108
Co-authored-by: ColumbusLabs <287001685+ColumbusLabs@users.noreply.github.com>
Add controller.max_send_buffer_size in MiB, default 64 and 0 for no
limit. Close with 1013 instead of 1008 on overflow: clients treat 1008
as fatal, so a host would shut down and the web UI would log out. With
1013 the client resumes, the controller answers with invalidate since
the session has ended, and the client registers a new session.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every message sent over a link is kept in the send buffer until the
other side acknowledges it with a heartbeat, so it can be resent if
the session resumes. Nothing forced a peer to acknowledge, so a client
that kept heartbeating without a sequence number while subscribed to a
busy event made the controller buffer messages until it ran out of
memory.
Store buffered messages serialised, track their total size, and close
the connection once it goes over maxSendBufferSize. The controller
sets this to 64 MiB for the connections it accepts. A client that far
behind could not resume in a reasonable time anyway.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Without a transport Winston will print annoying "Attempt to write logs
with no transports" warnings all over the place. Setup a transport in
tests that cause logging to suppress this.
A permission function runs during ingress permission checking, before
the request body is validated. UserBulkImportRequest.permission read the
raw, unvalidated message.data and threw a bare Error for an unknown
importType. That error is not a PermissionError, so _processMessage
rethrew it to the connector "message" listener, which emitted "error" on
the connector. Controller-side connectors have no "error" listener, so
Node rethrew and handleUnhandledErrors exited the process.
Repro: a control client with client-side validateSent disabled sends
UserBulkImportRequest with importType "foo"; the client gets a normal
RequestError response and the controller then logs a fatal uncaught
exception and exits.
Fix both layers: _processMessage now logs and drops an unexpected error
from the permission check (control connections only, where a response
was already sent) instead of rethrowing, and UserBulkImportRequest
throws a PermissionError for an unknown importType so it is denied
cleanly. A genuine PermissionError still denies silently as before.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Migrate the usage of CommonJS specific code and imports to ESM. Roughly
summarised this:
- Changes all require() calls to import statement or expressions.
- Changes usage of module.exports to export statements.
- Replaces require.resolve with import.meta.resolve.
- Replaces conditional imports with conditional subpath imports.
- Replaces module === require.main with import.meta.main.
- Adds the full filename with extention to all relative imports.
- Changes all usage of __dirname and __filename to their import.meta
equivalents.
- Sets "type": "module" in package.json files.
- Removes usage of "use strict".
- Adds .ts an an alias of .js for the webpack build.
Due to differences in how require and import resolves files the plugin
entrypoints will have to include the filename and extension after this
migration in order to work.
Generic error handlers such as FactorioServer.handle() and the event
handler catch in Link only log err.stack, so the ajv errors attached to
an InvalidMessage were lost unless the catch site checked for the class
explicitly. Appending the errors to the stack in the constructor makes
every stack-logging handler show which field failed validation without
special casing each one.
Fixes#986
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
An event broadcast from an instance to all hosts only reached the host
running that instance. The host side of an instance connection has the
host's own address as its source, so a broadcast aimed at hosts counts as
addressed to it, and a link that a message is addressed to handles it and
stops. The event was delivered locally and never routed onwards.
A broadcast is addressed to every link of the type it targets, so arriving
at one of them does not mean it has arrived everywhere. Keep routing such a
message after handling it locally. Only links that route are affected, as
instances and controls have no router and would otherwise throw on every
broadcast they receive.
The only links whose source type matches a broadcast target and which route
are the host and its instance connections, both matching a broadcast to
hosts, so add the missing case for that to the host router. It forwards to
the controller, which already passes broadcasts on to every other host, and
stops when the message came from the controller to begin with.
Drop the exclusion this bug had in the routing test matrix, which now
covers it along with the plugin filtering variants.
Fixes#575
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
forwardRequest records a request so that its response can be sent back the
way it came, but recorded it before handing the message to the connector.
When the connector threw, because the session to the next hop was gone, the
record stayed behind for a request that had never left.
Routers answer the origin themselves when forwarding throws, so the request
was already accounted for. The leftover record then had it answered a
second time when the link cleared its pending requests on disconnect,
producing a duplicate response and "Received response N without a pending
request" at the far end.
Remove the record when the forward throws. Doing it here rather than in the
routers covers both the host, which catches and responds, and the
controller router, which does not.
Fixes#650
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move all functions and classes currently available through lib/name and
libName to be under a single flat namespace in lib. This make
reorganising the lib namespace much simpler as the location of symbols
can move without breaking the consumers of lib.
The multi-entrypoint design was already problematic for the web plugin
system which couldn't effectively use it due to how webpack's shared
module system is awkward to set up with multiple entrypoints per package.
Format arguments to have nested structures start on a new line instead
of inline to make it easier to see where the structures starts and what
structure it is.
Validate messages received on the controller from hosts and control
nodes before handling or forwarding them. This should prevent
performing actions that are not allowed by the role system and block
simple attacks such as spoofing the source address of a message.
Split up the action of registring an event or request data class to be
decoded by the Link class and the action of registring a handler for
those requests and events. This moves the validation work from Link
instantiation time to startup initialisation which makes logic errors
fail on startup instead of failing instantiation of the link when a
connection attempt is made.
Additionally implement a partial validation logic for messages received
using the newly refactored message registration. This unfortunately
ended up merged with the registration refactor.
Reverse the direction of the API for sending messages from
messageDefinition.send(link, { content });
to
link.send(new MessageClass(content));
and convert all message definitions to use the JSON serialisable class
pattern and explicit registration of message handlers, along with a
end-to-end based routing of messages instead of link to link.
The experience I gained from the existing way messages were defined and
sent over links was that it was difficult to understand for people
unfamiliar with the system. The automatic registration of message
handlers ment that code was being called as if by magic from nowhere
with no easy way to discover it. The manual forwarding and rules for
what can be sent where and how obscured its working.
The use of pure JSON as both input and output from message handlers made
it tedious to work with. Data had to be manually converted between the
program representation and the over the wire format, which made handler
code unplesant to write. Particularly because the over the wire format
used snake_case while the program code uses camelCase for variables.
The idea behind the decision to do it this way was to make it more clear
which data came from the wire and which from the program, and while this
worked it didn't make message handlers any nicer to write.
The direction of the API and magic registration of message handlers also
meant that a future migration to TypeScript would be troublesome. While
it may be possible to implement, the degree of TypeScript magic required
would go starkly against the principle of keep it simple stupid.
By refactoring to using the JSON serialisable class pattern most of
these issues go away. There is no longer a need to distinguish the on
wire format of messages with the program representation because the only
place that deals with the wire format of messages are the toJSON and
fromJSON methods on the classes themselves. Message handlers will work
with proper instances of classes instead of raw data, something that
makes them easier to write, and trivial to type with TypeScript.
End-to-end routing also simplifies sending messages that need to do
mulitple hops to reach the destination. And removes the need for having
all messages be specified in plugin's info module, as they can be routed
without having to be parsed completely.
The explicit registration of message handlers is something that I
thought would be tedious boilerplate code that could be automated away.
But experience now shows it to be the opposite. Allthough dull
repetition, the purpose it serves is to link together the code in a way
that humans can easily find. With explicit registration searching the
name of the message handler in the editor will show the registration that
ties it together with the type of message it handles and the kind of
registration. With implicit registration searching the name of the
message handler turns up only the handler with no clue as to what calls
it.
Refine the state flow used in the connector abstractions to better
handle close while connecting and resuming. The new logic is to close
the connection immediately in these two states instead of continuing
trying to connect and then giving up some point later.
For the connectors the handshake state is split into connecting
and resuming, new is replaced by closed, and the closing state is turned
into a boolean flag as it's possible that closing is desired while
connecting, connected or resuming. The connecting state is not present
in the server connector because there's nothing for the server to do in
that connector state.
The timeout logic is completely removed, once a connection attempt is
started for the client it will continue to attempt to connect, attempt
to resume dropped connections and attempt reconnects until either an
authentication error occurs or the client explicitly requests to close
the connection.
A new session timeout is added to control how long to attempt to resume
the connection until it's considered dead and invalidated. This allows
stale connections to give up earlier, and swallow up less data while in
the dropped but not yet invalidated state.
For the client side sending messages while it's connecting has been
changed to throw an error instead of queuing up the message and
(possibly) sending it after the connection is established. Code
shouldn't try to send messages without knowing if the connection it's
trying to send them over is active or not.
Add optional arguments to Request and Event to control which properties
are required to be present in the payload. This allows gracefully
extending the protocol, reducing overhead for optional data, and leaving
properties out for additional behaviour.
Fix the validation loging for sending responses to requests not
functioning due to the message data property being passed incorrectly to
the validator.
Use lib* prefix for the modules impored from the lib package to help
distinguish them more from ordinary variables. Fix up the naming
convention for files to be consistently lowercase_underscore for
JavaScript files.
Split up codebase into the packages @clsterio/lib, @clusterio/master,
@clusterio/slave, and @clusterio/ctl using lerna to manage them. This
will allow deployment to use only the parts needed reducing install
size, and better separate the code into shared library, master, slave
and control.
Add users, roles and permissions and rework the authentication to use
separate user and slave tokens for control and slave connections
respectively.
Add commands for listing permissions, roles and users, and creating,
editing and deleting roles and users.
Send a mapping of the plugins the master server has loaded to slaves
vice versa. Limit the plugins that may be loaded by instances to only
those that the master server has loaded.
Replace socket.io with plain WebSocket using the ws library as it
doesn't really provide any useful tools over plain WebSocket when it
comes to implementing reliable communication that works across
reconnects. The widespread browser support for WebSocket also means
it has little value to add for the web interface over WebSocket.
Validate messages sent over links with the json schema provided from
them. This aids debugging as an exception will be thrown from the code
sending the bad content.
Change Instance class to be a subclass of Link and refactor the
communication to and establisment of instances to use a virtual
connection between the slave and the instance. This is simplifies the
implementation of plugin communication for instances.
Replace InstanceRequest with a forwardTo option to Request which does
the task of forwarding in a much cleaner way.
Replace the sources/targets concept with a links array in the message
definitions. Fixes the problem of the actual links the messages applies
to being ambigious when there's multiple sources and targets.
Remove connection and client subclasses of Link and put the connection
logic into a "connector" class. The connector deals with sending
and receiving messages, handling the connection and reconnection of lost
connections and giving sequence numbers to messages.
This move the logic of the actual connection away from the link
sub-classes, which in turn means the connection should be a lot easier
to mock in tests.
Rework client.js to support running multiple instances at once and be
managed remotely and implement the neccessary communication
infrastructure for this.
Due to circular dependencies on the implementation of this the master
branch is significantly broken after this.
