Update non-web related dependencies to their latest version were
possible. It appears yargs has changed its parsing of command line
options to no longer remove a layer of quotes.
The LTS releases for v12 and v14 are already end of life and the v16 is
end of life in under two months from now on. Bump supported version up
to v18 as it does not make sense to continue to support EOL releases.
Even if we wanted to keep supporting them, our dependencies will not.
Note that Node.js v18 is currently not avaliable via the NodeSource PPA
on Ubuntu 19.10 and below or Debian 9 and below.
Reverse the direction of the API for sending messages from
messageDefinition.send(link, { content });
to
link.send(new MessageClass(content));
and convert all message definitions to use the JSON serialisable class
pattern and explicit registration of message handlers, along with a
end-to-end based routing of messages instead of link to link.
The experience I gained from the existing way messages were defined and
sent over links was that it was difficult to understand for people
unfamiliar with the system. The automatic registration of message
handlers ment that code was being called as if by magic from nowhere
with no easy way to discover it. The manual forwarding and rules for
what can be sent where and how obscured its working.
The use of pure JSON as both input and output from message handlers made
it tedious to work with. Data had to be manually converted between the
program representation and the over the wire format, which made handler
code unplesant to write. Particularly because the over the wire format
used snake_case while the program code uses camelCase for variables.
The idea behind the decision to do it this way was to make it more clear
which data came from the wire and which from the program, and while this
worked it didn't make message handlers any nicer to write.
The direction of the API and magic registration of message handlers also
meant that a future migration to TypeScript would be troublesome. While
it may be possible to implement, the degree of TypeScript magic required
would go starkly against the principle of keep it simple stupid.
By refactoring to using the JSON serialisable class pattern most of
these issues go away. There is no longer a need to distinguish the on
wire format of messages with the program representation because the only
place that deals with the wire format of messages are the toJSON and
fromJSON methods on the classes themselves. Message handlers will work
with proper instances of classes instead of raw data, something that
makes them easier to write, and trivial to type with TypeScript.
End-to-end routing also simplifies sending messages that need to do
mulitple hops to reach the destination. And removes the need for having
all messages be specified in plugin's info module, as they can be routed
without having to be parsed completely.
The explicit registration of message handlers is something that I
thought would be tedious boilerplate code that could be automated away.
But experience now shows it to be the opposite. Allthough dull
repetition, the purpose it serves is to link together the code in a way
that humans can easily find. With explicit registration searching the
name of the message handler in the editor will show the registration that
ties it together with the type of message it handles and the kind of
registration. With implicit registration searching the name of the
message handler turns up only the handler with no clue as to what calls
it.
The role of a slave as a host for Factorio servers is not well reflected
in its name and has caused confusion among users. Rename slave to host
to better reflect its role as a host of Factorio servers.
The role of the master server as the party controlling every part of the
cluster is not well reflected in its name and has caused confusion among
users on several occasions. Rename master to controller to better
reflects its role.
Remove ugly hack to generate and use an insecure certificate on the fly
in the HttpCloser's https tests and replace it with the already existing
properly generated test certificate. Removes the dependency on node-forge.
If only the bot was clever enough to see that node-forge is only used
for a test that has no user input it might have had the foresight to
realize that crying wolf about completely irrelevant security issues
will only lead to legitimate issues being ignored in the future. But
alas, the AI overlords are not taking over any time soon.
Mocha's recursive search feature finds all files and then removes paths
that match the exclution filter, except that the globstar filter doesn't
match directories with a dot in them which is created by pnpm. Avoid
this madness by removing the test package.
Save disk space, time and sanity by replacing npm and lerna with pnpm.
This is more performant and doesn't have developer experience ruining
bugs like npm link removing all previous links before creating the
requested one.
The Node.js specific rules for Eslint were deprecated in favour of the
node plugin for Eslint. Add node plugin for eslint and switch over to
using its rules, fixing any issues detected.
Move the depenedncies needed to run the tests into test/package.json and
manage it using lerna. This should make it easier to selectively
install only what's needed to run the software.
Add authentication plugin that does a simple challenge response protocol
to verify that the player trying to log in is who they claim to be. The
process goes as follows:
1. A player code is generated for a player that wants to log in and is
displayed in-game.
2. The player code is input into the web interface and the master
generates a verification code as a challenge back along with a token.
3. The player inputs the verification code in-game and is logged in to
the web interface in doing so.
To do the actual login the web interface polls a verify endpoint every 2
seconds until the code has either been verified or expired. The
challenge comes from being unable to control the verification code and
having to input it as a player logging into one of the servers in order
to login.
Some additional steps are taken in order to increase the difficulty for
an attacker to subvert the challenge:
- The login process must be started from in-game.
- Generated codes are only valid for a limited timespan which is
configurable and 2 minutes by default.
- The verification code comes with a token that only works with one
player code and verification code combo and must be provided to pass
validation.
When setting up the tests there were issues with code that didn't
properly clean up after itself and left lingering connections, intervals
or timers that kept the event loop running after the tests where done.
To fix this the --exit flag was passed as a temporary measure but then
subsequently forgotten about. The issues with keeping the event loop
alive have long since been fixed on Linux, but on Windows there was a
problem with the control connector not exiting because the master server
was killed instead of shutten down properly.
With that issue fixed there are no longer any connections or timers
keeping the event loop alive and --exit can thus be removed.
The removal of the automatic TLS certificate generation removed the use
of node-forge in the master server, while the addition of the HttpCloser
added it as dependency of the tests. Fix node-forge dependency being
listed in lib and master instead of as a dev dependency in package.json.
Remove the need for installing babel and the webpack loaders used in the
webpack.common.js file separately by resolving them from the web_ui
package. This allows future changes to done to the modules used without
having to update them in each plugin that uses the web_ui common config.
Fix the issue with paths resolving incorrectly for the webpack config
when running the master server with the --dev option from the project
root folder through either npx clusteriomaster or node packages/master
Split up codebase into the packages @clsterio/lib, @clusterio/master,
@clusterio/slave, and @clusterio/ctl using lerna to manage them. This
will allow deployment to use only the parts needed reducing install
size, and better separate the code into shared library, master, slave
and control.
Remove the unused dependencies to bcrypt and bcrypt-promise. These were
used by the playerManager plugin to implement password based
authentication, but this plugin is likely not going to be ported to 2.0.
Add ESLint as a dev dependency and configure it to match the preferred
style for the project. The config options set to warn are tentative and
may not be sensible as they haven't been fully investigated yet.
Upload the created export data zip file to a new /api/upload-export
endpoint on the master server which unpacks it into the appropriate
directory on the master.
Spawn master, slave, and clusterctl in separate processes in the
clusterio integration tests and leave the test master and slave process
available for the whole test duration to allow plugins to use them as
well. This removes the need for reimplementing the startup logic in the
test code and makes the existing startup code covered by tests.
Replace socket.io with plain WebSocket using the ws library as it
doesn't really provide any useful tools over plain WebSocket when it
comes to implementing reliable communication that works across
reconnects. The widespread browser support for WebSocket also means
it has little value to add for the web interface over WebSocket.
Add a library mod for wrapping the Clusterio remote API and providing
related utilities. This eases development of Clusterio related Lua code
by providing the same interface for both mods and modules and also makes
the serialize library available without having to duplicate it.
It is not possible to implement this through save patching as mods
cannot load Lua code from the level code like they can with code from
other mods.
Close active connections and http servers during master shutdown and let
node exit on its own. If this fails with an error set output streams to
blocking and report it then shutdown with process.exit.
Also add double SIGINT handling to immediately shutdown.
Hornwitser