Complete rework of the inventory synchronization to reduce the number of
times the inventory has to be downloaded and the possibilities for item
loss/duplication.
The basic concept is to acquire exclusive access to the player data
before downloading the data. This prevents the player from logging in
to two servers at the same time and getting the synced inventory twice.
The acquisition will in most cases execute faster than the player can
provide input to the game on login, and if the player data on the master
isn't what was previously uploaded then the player inventory is deleted
and the player is turned into a spectator while the updated inventory is
being transferred into the game. This model allows for the downloading
of the synced inventory to be skipped in the case the player joins the
same server they left from.
The synced data is expanded to include more than just the character and
inventory to properly handle sync of dead players. If the player leaves
dead on one server this state will be saved and when the player joins
another server they continue off as dead. The ghost, god, and spectator
controllers are supported, while the cutscene and editor controllers are
not. For players in a cutscene when joining the acquisition will be
attempted immediately, but the scenario waits until the player is out of
the cutscene before acting upon the response to the acquisition. For
players in the editor mode the scenario toggles them out of the mode
before syncing data.
Since uploads to the master can fail handling of this is also added.
The scenario will retry uploading the player data at exponentially
increasing intervals until it gets the message back that the upload
succeeded. Since it's possible for servers to go down and stay down for
a long time the exclusive access held by these servers are released after
a timeout. This means that a player could be playing and or have
uploaded newer data by the time the instance comes back online and tries
to upload the inventory, in this case the master will report the upload
succeeded but discard the data. Instances may also upload old data if
they load back to an earlier save, which will also be discarded by the
master.
Other small improvements made by this rework:
- Add a retry button to the failed download dialog
- Update the existing progress dialog instead of destroying and
recreating it every time.
- Automatically stuff the items from the temporary inventory into the
synced inventory if they fit.
- Move player back to the position they started the download from if
they were turned into a spectator for the download.
- Ensure the player has a character if the stored data is wiped.
- Add error checking of the RCON commands sent by the instance.
The ajv module was made shared to reduce package size, but it turned out
the package size was large because of a missunderstanding in how imports
to sub-paths works with module federation. Since that was fixed the ajv
module was no longer shared, but plugins incorrectly still specified it.
In order to import a path inside packages/lib/ in the webpack shared
module environment it has to be declared as shared in both the web_ui
webpack config and the plugin's webpack config. This is rather error
prone and easy to miss. Move the burden of specifying which entrypoints
are valid in lib to lib itself by changing all imports in the web
interface code to import from @clusterio/lib instead sub paths.
For web code this means imports is done like this from now on:
import { libConfig, libLink } from "@clusterio/lib";
For info.js which has to be loadable by both Node.js and web code the
imports are done as follows:
const { libConfig, libLink } = require("@clusterio/lib");
While the Node.js only code can either continue to import sub paths
directly or use the info.js style.
Create entries for players that are present in the save on server
startup to preserving their inventory and fix attempting to reference the
player object after it had been removed in on_player_removed.
Make sure the GUI element in the on_gui_click event is valid before
attempting to access this, as another module may have destroyed it
during its event handler.
When upgrading or downgrading the inventory_sync plugin it's possible
an different structure of the global.inventory_sync table is left behind
which could cause the code to crash when certain sub tables are not
present.
Be more defensive in the initialization of the table to make sure all
sub-tables are present on startup.
Convert code indents to tabs to be consistent with the rest of the
JavaScript code. This commits covers all of the lua code, some of the
hand written JSON code and the lone HTML and CSS files that were not
using tabs.
The jsonwebtoken accept strings as input for secrets which is
nonsensical from a cryptographic standpoint as the algorithms all
operate on byte sequences. Worse still, strings are encoded into bytes
using UTF-8, which disallows certain sequences of bytes from appearing
and potentially weakening the cryptography.
For legacy reasons the master secret was encoded into base64 _and passed
as a string to jsonwebtoken_. This further weakened the crypto as the
bytes that can appear in base 64 is only 1/4 of the 256 possible values
for bytes. This could significantly weaken the cryptographic strength
of the tokens.
I have known of this issue for quite some time but has put off actually
fixing it for too long now. Since the master secret is already
documented as being in base 64, decode it as base 64 and pass it as a
byte Buffer to jsonwebtoken. This will ensure every byte value is
possible in the byte sequence passed to the jsonwebtoken crypto.
The impact of this change is unfortunately that all tokens will become
invalid and will have to be reissued.
* New method avoids all dataloss and prefectly replicates the queue
* Made proper description in readme
* Added packages to make lerna build happy
* Added reference to main readme
Add logic to stop sending commands and wait for any pending commands to
complete during instance stop in subspace_storage. This should fix the
RCON connection lost errors printed in the console during stop.
The Node.js specific rules for Eslint were deprecated in favour of the
node plugin for Eslint. Add node plugin for eslint and switch over to
using its rules, fixing any issues detected.
Move the depenedncies needed to run the tests into test/package.json and
manage it using lerna. This should make it easier to selectively
install only what's needed to run the software.
Simplify the the way markdown files are formatted to make them easier to
author without editors that do auto line breaking, and make the diffs
from them simpler to read.
Add authentication plugin that does a simple challenge response protocol
to verify that the player trying to log in is who they claim to be. The
process goes as follows:
1. A player code is generated for a player that wants to log in and is
displayed in-game.
2. The player code is input into the web interface and the master
generates a verification code as a challenge back along with a token.
3. The player inputs the verification code in-game and is logged in to
the web interface in doing so.
To do the actual login the web interface polls a verify endpoint every 2
seconds until the code has either been verified or expired. The
challenge comes from being unable to control the verification code and
having to input it as a player logging into one of the servers in order
to login.
Some additional steps are taken in order to increase the difficulty for
an attacker to subvert the challenge:
- The login process must be started from in-game.
- Generated codes are only valid for a limited timespan which is
configurable and 2 minutes by default.
- The verification code comes with a token that only works with one
player code and verification code combo and must be provided to pass
validation.
Change to "auto" as the publicPath to have webpack deduce the correct
path at runtime. This fixes an issue with plugins being unable to be
loaded due to trying to load bundles from the wrong path when the web
interface is hosted under a proxied sub-path.
The web files for plugins were hosted at /plugin/plugin_name/ while the
view page for a plugin is hosted at /plugins/plugin_name/view. Reduce
confusion by making both urls use the plural version.
Add hidden development option to run the webpack compiller for the given
plugin(s) similar to the --dev option for the built-in web interface.
This requires publicPath to be set in the webpack config and an updated
version of webpack-dev-middleware to function correctly.
The module sharing in Webpack works on a per import basis and therefore
doesn't trigger when the value in the config is @clusterio/lib while the
module imported is @clusterio/lib/config. This in turn caused a copy of
lib to be bundled with each plugin. Fix by making sure each lib import
is shared and specified in the configs.
Hornwitser