- react: Update to new root rendering method
- react-router: Replace useHistory with useNavigate
- react-router: Replace <Switch> with <Routes>
- reimplement sticky notice from scratch in ModPackViewPage
- remove isomorphic-ws as it's latest version doesn't work and only
saves one line of code.
- remove import of antd.dark.css
- fix misc css breakages.
Add type object to schemas specifying properties and minItems to schemas
specifying tuples. Disable strictTuples because it's desirable to be
able to add additional items to the tuples in the future.
Update non-web related dependencies to their latest version were
possible. It appears yargs has changed its parsing of command line
options to no longer remove a layer of quotes.
The LTS releases for v12 and v14 are already end of life and the v16 is
end of life in under two months from now on. Bump supported version up
to v18 as it does not make sense to continue to support EOL releases.
Even if we wanted to keep supporting them, our dependencies will not.
Note that Node.js v18 is currently not avaliable via the NodeSource PPA
on Ubuntu 19.10 and below or Debian 9 and below.
It makes more sense to put the address first then have the large
structure to send as the second argument to sendTo. The opposite order
was originally chosen to be similar to that of sendto in BSD sockets.
Validate messages received on the controller from hosts and control
nodes before handling or forwarding them. This should prevent
performing actions that are not allowed by the role system and block
simple attacks such as spoofing the source address of a message.
Split up the action of registring an event or request data class to be
decoded by the Link class and the action of registring a handler for
those requests and events. This moves the validation work from Link
instantiation time to startup initialisation which makes logic errors
fail on startup instead of failing instantiation of the link when a
connection attempt is made.
Additionally implement a partial validation logic for messages received
using the newly refactored message registration. This unfortunately
ended up merged with the registration refactor.
Reverse the direction of the API for sending messages from
messageDefinition.send(link, { content });
to
link.send(new MessageClass(content));
and convert all message definitions to use the JSON serialisable class
pattern and explicit registration of message handlers, along with a
end-to-end based routing of messages instead of link to link.
The experience I gained from the existing way messages were defined and
sent over links was that it was difficult to understand for people
unfamiliar with the system. The automatic registration of message
handlers ment that code was being called as if by magic from nowhere
with no easy way to discover it. The manual forwarding and rules for
what can be sent where and how obscured its working.
The use of pure JSON as both input and output from message handlers made
it tedious to work with. Data had to be manually converted between the
program representation and the over the wire format, which made handler
code unplesant to write. Particularly because the over the wire format
used snake_case while the program code uses camelCase for variables.
The idea behind the decision to do it this way was to make it more clear
which data came from the wire and which from the program, and while this
worked it didn't make message handlers any nicer to write.
The direction of the API and magic registration of message handlers also
meant that a future migration to TypeScript would be troublesome. While
it may be possible to implement, the degree of TypeScript magic required
would go starkly against the principle of keep it simple stupid.
By refactoring to using the JSON serialisable class pattern most of
these issues go away. There is no longer a need to distinguish the on
wire format of messages with the program representation because the only
place that deals with the wire format of messages are the toJSON and
fromJSON methods on the classes themselves. Message handlers will work
with proper instances of classes instead of raw data, something that
makes them easier to write, and trivial to type with TypeScript.
End-to-end routing also simplifies sending messages that need to do
mulitple hops to reach the destination. And removes the need for having
all messages be specified in plugin's info module, as they can be routed
without having to be parsed completely.
The explicit registration of message handlers is something that I
thought would be tedious boilerplate code that could be automated away.
But experience now shows it to be the opposite. Allthough dull
repetition, the purpose it serves is to link together the code in a way
that humans can easily find. With explicit registration searching the
name of the message handler in the editor will show the registration that
ties it together with the type of message it handles and the kind of
registration. With implicit registration searching the name of the
message handler turns up only the handler with no clue as to what calls
it.
The role of a slave as a host for Factorio servers is not well reflected
in its name and has caused confusion among users. Rename slave to host
to better reflect its role as a host of Factorio servers.
The role of the master server as the party controlling every part of the
cluster is not well reflected in its name and has caused confusion among
users on several occasions. Rename master to controller to better
reflects its role.
Technologies that were disabled were ignored completely by research sync
due to a missunderstanding of what disabled means. Disabled does not
mean the tech is somehow inaccessible and completely out of play, it
only means it can't be selected in the technology screen.
Ignore the enabled status of technologies in research_sync to fix a
crash that happened when a mod enables a disabled by default technology
and the player attempts to research it.
Track summary statistic of total HTTP response durations using a
middleware that applies for all requests instead of having every request
needing to update the hit counter manually. Deprecates
endpointHitCounter.
Fix broken support for retrying uploads and background downloads by
making crafting queue serialisation non-destructive and only performing
the serialisation and deserialisation while the player is online.
This removes the /save-crafts and /restore-crafts commands because the
existing implementation would voids the non-saved crafting queue items
if /save-crafts is performed and then new items are added to the
crafting queue before switching server.
Fixes#419.
Count up to 100 empty slots past the last seen slot with data in it
before stopping to look for personal logistics slots in inventory_sync.
Allows unlimeted amount of logistics request slots to be used for
heavily modded games.
Put all web assets including plugin assets under a shared static/ route
in the web interface that's declared as immutable and cacheable for 1
year. This allows all JavaScript bundles to be loaded from disk instead
of having to check with the web server each time and significantly
speeds up the web interface load time. It also fixes the public path
issue when using the --dev-plugin option behind a proxy.
To prevent collisions and stale data all assets now include a hash in
the filename. This means the entrypoint is now dynamic and specified in
a manifest.json file created from the web build. For plugins this means
the filename attribute needs to be removed from webpack.config.js.
The prepare script runs before publishing a package, after installation
of a package from source, and when running npm install without
arguments in the repository. These are the times where a build of the
web interface files are required to make the web interface work and
removes a manual step from using the package(s). The "build" script on
the other hand is never automatically invoked by npm.
Write new file contents to a temporary file and then rename over the
target file to avoid data loss in the unlikely event there's an error
writing the new file contents or the process is interrupted.
Resolves#416.
The serialization and deserialization of editor mode is not supported by
the current code, and while the download correctly checked for the
player being in editor mode, the upload did not check for the player
being in editor mode causing players in editor mode to break. Fix by
adding check in upload_inventory.
Fix commands syncing researces being sent before the instance is started
up, and potential desync of research progress as a result of out of
order execution of RCON commands sent.
Fix has_data not being checked by sync_player, dialog being stuck open if
no data is received by the master and master server refusing to server
empty data.
The system for preventing items being sent without a connection to the
master and (and thus voiding the items) is to only do so if a ping from
the plugin has been received in the last 5 seconds of in-game time.
Don't send pings without being fully connected to the master to make
sure this system actually activates during connection downtime.
Ignore inventory updates and orders while the instance is not in the
running state to avoid sending commands before the server has started
up, and also ignore orders if the connection to the master server isn't
isn't online to avoid piling up requests in the buffer during session
resume attempts.
Complete rework of the inventory synchronization to reduce the number of
times the inventory has to be downloaded and the possibilities for item
loss/duplication.
The basic concept is to acquire exclusive access to the player data
before downloading the data. This prevents the player from logging in
to two servers at the same time and getting the synced inventory twice.
The acquisition will in most cases execute faster than the player can
provide input to the game on login, and if the player data on the master
isn't what was previously uploaded then the player inventory is deleted
and the player is turned into a spectator while the updated inventory is
being transferred into the game. This model allows for the downloading
of the synced inventory to be skipped in the case the player joins the
same server they left from.
The synced data is expanded to include more than just the character and
inventory to properly handle sync of dead players. If the player leaves
dead on one server this state will be saved and when the player joins
another server they continue off as dead. The ghost, god, and spectator
controllers are supported, while the cutscene and editor controllers are
not. For players in a cutscene when joining the acquisition will be
attempted immediately, but the scenario waits until the player is out of
the cutscene before acting upon the response to the acquisition. For
players in the editor mode the scenario toggles them out of the mode
before syncing data.
Since uploads to the master can fail handling of this is also added.
The scenario will retry uploading the player data at exponentially
increasing intervals until it gets the message back that the upload
succeeded. Since it's possible for servers to go down and stay down for
a long time the exclusive access held by these servers are released after
a timeout. This means that a player could be playing and or have
uploaded newer data by the time the instance comes back online and tries
to upload the inventory, in this case the master will report the upload
succeeded but discard the data. Instances may also upload old data if
they load back to an earlier save, which will also be discarded by the
master.
Other small improvements made by this rework:
- Add a retry button to the failed download dialog
- Update the existing progress dialog instead of destroying and
recreating it every time.
- Automatically stuff the items from the temporary inventory into the
synced inventory if they fit.
- Move player back to the position they started the download from if
they were turned into a spectator for the download.
- Ensure the player has a character if the stored data is wiped.
- Add error checking of the RCON commands sent by the instance.
The ajv module was made shared to reduce package size, but it turned out
the package size was large because of a missunderstanding in how imports
to sub-paths works with module federation. Since that was fixed the ajv
module was no longer shared, but plugins incorrectly still specified it.
In order to import a path inside packages/lib/ in the webpack shared
module environment it has to be declared as shared in both the web_ui
webpack config and the plugin's webpack config. This is rather error
prone and easy to miss. Move the burden of specifying which entrypoints
are valid in lib to lib itself by changing all imports in the web
interface code to import from @clusterio/lib instead sub paths.
For web code this means imports is done like this from now on:
import { libConfig, libLink } from "@clusterio/lib";
For info.js which has to be loadable by both Node.js and web code the
imports are done as follows:
const { libConfig, libLink } = require("@clusterio/lib");
While the Node.js only code can either continue to import sub paths
directly or use the info.js style.
Create entries for players that are present in the save on server
startup to preserving their inventory and fix attempting to reference the
player object after it had been removed in on_player_removed.
Make sure the GUI element in the on_gui_click event is valid before
attempting to access this, as another module may have destroyed it
during its event handler.
When upgrading or downgrading the inventory_sync plugin it's possible
an different structure of the global.inventory_sync table is left behind
which could cause the code to crash when certain sub tables are not
present.
Be more defensive in the initialization of the table to make sure all
sub-tables are present on startup.
Convert code indents to tabs to be consistent with the rest of the
JavaScript code. This commits covers all of the lua code, some of the
hand written JSON code and the lone HTML and CSS files that were not
using tabs.
The jsonwebtoken accept strings as input for secrets which is
nonsensical from a cryptographic standpoint as the algorithms all
operate on byte sequences. Worse still, strings are encoded into bytes
using UTF-8, which disallows certain sequences of bytes from appearing
and potentially weakening the cryptography.
For legacy reasons the master secret was encoded into base64 _and passed
as a string to jsonwebtoken_. This further weakened the crypto as the
bytes that can appear in base 64 is only 1/4 of the 256 possible values
for bytes. This could significantly weaken the cryptographic strength
of the tokens.
I have known of this issue for quite some time but has put off actually
fixing it for too long now. Since the master secret is already
documented as being in base 64, decode it as base 64 and pass it as a
byte Buffer to jsonwebtoken. This will ensure every byte value is
possible in the byte sequence passed to the jsonwebtoken crypto.
The impact of this change is unfortunately that all tokens will become
invalid and will have to be reissued.
Hornwitser