Split up the action of registring an event or request data class to be
decoded by the Link class and the action of registring a handler for
those requests and events. This moves the validation work from Link
instantiation time to startup initialisation which makes logic errors
fail on startup instead of failing instantiation of the link when a
connection attempt is made.
Additionally implement a partial validation logic for messages received
using the newly refactored message registration. This unfortunately
ended up merged with the registration refactor.
Reverse the direction of the API for sending messages from
messageDefinition.send(link, { content });
to
link.send(new MessageClass(content));
and convert all message definitions to use the JSON serialisable class
pattern and explicit registration of message handlers, along with a
end-to-end based routing of messages instead of link to link.
The experience I gained from the existing way messages were defined and
sent over links was that it was difficult to understand for people
unfamiliar with the system. The automatic registration of message
handlers ment that code was being called as if by magic from nowhere
with no easy way to discover it. The manual forwarding and rules for
what can be sent where and how obscured its working.
The use of pure JSON as both input and output from message handlers made
it tedious to work with. Data had to be manually converted between the
program representation and the over the wire format, which made handler
code unplesant to write. Particularly because the over the wire format
used snake_case while the program code uses camelCase for variables.
The idea behind the decision to do it this way was to make it more clear
which data came from the wire and which from the program, and while this
worked it didn't make message handlers any nicer to write.
The direction of the API and magic registration of message handlers also
meant that a future migration to TypeScript would be troublesome. While
it may be possible to implement, the degree of TypeScript magic required
would go starkly against the principle of keep it simple stupid.
By refactoring to using the JSON serialisable class pattern most of
these issues go away. There is no longer a need to distinguish the on
wire format of messages with the program representation because the only
place that deals with the wire format of messages are the toJSON and
fromJSON methods on the classes themselves. Message handlers will work
with proper instances of classes instead of raw data, something that
makes them easier to write, and trivial to type with TypeScript.
End-to-end routing also simplifies sending messages that need to do
mulitple hops to reach the destination. And removes the need for having
all messages be specified in plugin's info module, as they can be routed
without having to be parsed completely.
The explicit registration of message handlers is something that I
thought would be tedious boilerplate code that could be automated away.
But experience now shows it to be the opposite. Allthough dull
repetition, the purpose it serves is to link together the code in a way
that humans can easily find. With explicit registration searching the
name of the message handler in the editor will show the registration that
ties it together with the type of message it handles and the kind of
registration. With implicit registration searching the name of the
message handler turns up only the handler with no clue as to what calls
it.
The role of a slave as a host for Factorio servers is not well reflected
in its name and has caused confusion among users. Rename slave to host
to better reflect its role as a host of Factorio servers.
The role of the master server as the party controlling every part of the
cluster is not well reflected in its name and has caused confusion among
users on several occasions. Rename master to controller to better
reflects its role.
Store the export manifest in ModPack to allow mulitple mod packs to have
different data exported. This is necessary as the export contains the
settings prototypes needed to create a proper UI for mod settings.
For now the useLocale and useItemMetadata interfaces uses the manifest
for a default mod pack set in the master config.
Add ModPack data type with corresponding storage on the master, messages
to create, update and delete them, as well as commands to invoke them.
The ModPack data type contains a list of mods with versions and optional
sha1 hashes, and mod settings to be used.
Add the ability to manage a set of mods to be shared by the master
server to the rest of the cluster. This implements storage, uploads,
downloads, deletetion, and listing details of mods but does not actually
do anything with the stored mods.
Add command that extracts play time (and players) from the currently
running save to the recorded instance statistics. This is useful for
getting correct play time statistics when an existing save game is
imported.
Add ability to move or copy saves between instances. For instances on
the same slave the saves are transferred with filesystem operations, for
instances on two different slaves the proxy stream interface is used to
send the save via the master server.
Add the roles and their permissions for the user the control connection
is connected as along with updates when these change. This enables the
web interface to know which permissions it has available to it.
Add optional limit to the number of log records to return in the
log query request to avoid having the web interface instance console
download the entire log for only to show a few hundred lines from it.
To make it possible to query the end of the log instead of just the
start add an order option as well.
Add a generate slave token button to the web interface and modify the
generateSlaveToken message and corresponding ctl command to have the
slave id be optional. The random id is the desired behavior when adding
a slave with the installer, as it'll ask for the token for the new slave
and deduce the id from the token.
Add /api/upload-save endpoint that streams the data via the slave
assigned to the target instance to the instance saves directory, along
with a ctl command and web interface button to do the upload.
Add link messages for listing saves and listening for changes to the
list of saves for instances along with a table in the instance view page
showing the save list for the instance and a list-saves command to ctl.
Add options to pass map seed, map exchange string, map gen settings
and/or map settings when creating a save either through the create save
request or the load scenario request.
Provide the control link with the ability to subscribe to updates when
instance status and properties changes. This allows the web interface
to respond in real time to instance starts and stops that are triggered
from somewhere else.
Notify when an instance is in the process of stopping with a separate
status for it. This makes handling shutdown gracefully a bit simpler in
plugin code.
Use relative imports in lib to avoid an issue with webpack thinking
absolute imports inside @clusterio/lib are for a separate module and
giving out warnings for it being missing in package.json.
Add optional arguments to Request and Event to control which properties
are required to be present in the payload. This allows gracefully
extending the protocol, reducing overhead for optional data, and leaving
properties out for additional behaviour.
Fix the validation loging for sending responses to requests not
functioning due to the message data property being passed incorrectly to
the validator.
Add log query and log follow commands which allows inspection of the
combined cluster log. Past entries can be queried via the new queryLog
request. The log is stored into a single flat file for now, though the
performance of this may good enough.
Add winston based logger for master, slave, ctl, lib, and web_ui code.
Logging done from slaves and instances are sent to the master server and
tagged with the instance and/or slave it originated from. This allows
for example error messages to propagate back to the source that iniated
the action. The master server also stores the log to allow it to be
queried at a later point. For slaves a copy of the slave's log is also
stored locally.
Under the current system there's no distinction made on the slave side
between instance config files that are present in the filesystem and
instances actually assigned to the slave. Which could lead to problems
with determing if an instance is actually assigned to the slave.
Resolve this issue by separating discovered instances with assigned
instances. Upon startup discovered instances are put into the
discoveredInstanceInfos mapping on the Slave object and all of these are
sent to the master server. The master server in turn sends assignment
requests for all the instances which are assigned to the slave. These
are put into the instanceInfos mapping of the Slave object which can
then be relied up to include only assigned instances.
Amend the error message shown when a message handler is missing from a
plugin with the expected name of the handler and the name of the plugin
it's missing from.
Use lib* prefix for the modules impored from the lib package to help
distinguish them more from ordinary variables. Fix up the naming
convention for files to be consistently lowercase_underscore for
JavaScript files.
Send the status of the instance in the repsonse to the listInstances
request and use this status in the integration tests to check if the
instance was started/stopped.
Add the statuses unknown and unassigned for when the status has not yet
been set by a slave and replace the initialized status with starting,
creating_save and exporting_data.
Replace usage of .constructor.name with a separate property to avoid
issue with JavaScript terser mangling the class name during minification
for the web build.
Split up codebase into the packages @clsterio/lib, @clusterio/master,
@clusterio/slave, and @clusterio/ctl using lerna to manage them. This
will allow deployment to use only the parts needed reducing install
size, and better separate the code into shared library, master, slave
and control.
Hornwitser