Hornwitser
bcd94709d2
Reverse the direction of the API for sending messages from
messageDefinition.send(link, { content });
to
link.send(new MessageClass(content));
and convert all message definitions to use the JSON serialisable class
pattern and explicit registration of message handlers, along with a
end-to-end based routing of messages instead of link to link.
The experience I gained from the existing way messages were defined and
sent over links was that it was difficult to understand for people
unfamiliar with the system. The automatic registration of message
handlers ment that code was being called as if by magic from nowhere
with no easy way to discover it. The manual forwarding and rules for
what can be sent where and how obscured its working.
The use of pure JSON as both input and output from message handlers made
it tedious to work with. Data had to be manually converted between the
program representation and the over the wire format, which made handler
code unplesant to write. Particularly because the over the wire format
used snake_case while the program code uses camelCase for variables.
The idea behind the decision to do it this way was to make it more clear
which data came from the wire and which from the program, and while this
worked it didn't make message handlers any nicer to write.
The direction of the API and magic registration of message handlers also
meant that a future migration to TypeScript would be troublesome. While
it may be possible to implement, the degree of TypeScript magic required
would go starkly against the principle of keep it simple stupid.
By refactoring to using the JSON serialisable class pattern most of
these issues go away. There is no longer a need to distinguish the on
wire format of messages with the program representation because the only
place that deals with the wire format of messages are the toJSON and
fromJSON methods on the classes themselves. Message handlers will work
with proper instances of classes instead of raw data, something that
makes them easier to write, and trivial to type with TypeScript.
End-to-end routing also simplifies sending messages that need to do
mulitple hops to reach the destination. And removes the need for having
all messages be specified in plugin's info module, as they can be routed
without having to be parsed completely.
The explicit registration of message handlers is something that I
thought would be tedious boilerplate code that could be automated away.
But experience now shows it to be the opposite. Allthough dull
repetition, the purpose it serves is to link together the code in a way
that humans can easily find. With explicit registration searching the
name of the message handler in the editor will show the registration that
ties it together with the type of message it handles and the kind of
registration. With implicit registration searching the name of the
message handler turns up only the handler with no clue as to what calls
it.
Clusterio inventory sync plugin
Carry over player inventory between servers
As a player, it mostly just works. Most important to know is that factorio data transfer is slow, which means if you have "big" items (blueprints mostly) in your inventory it will take a long time to transfer when joining servers. For this reason it is recommended to keep blueprints in the blueprint library.
A note on crashing servers:
When a server crashes while you are online there won't be time to immediately upload the inventory. Instead, the inventory is uploaded as soon as the server restarts. This means if you have been playing on a different server from the one that crashed and gathered/consumed items but are offline at the time of the first server coming online again your inventory will reset to the autosave.
Installation
Run the following commands in the folder Clusterio is installed to:
npm install @clusterio/plugin-inventory_sync
npx clusteriocontroller plugin add @clusterio/plugin-inventory_sync
Substitute clusteriocontroller with clusteriohost or clusterioctl if this a dedicated host or ctl installation respectively.
Method of operation
This plugin does event based synchronization of inventories. The greatly simplified data flow is as follows:
-
Player joins server.
-
Scenario script asks controller for exclusive access to the player.
-
Controller grants exclusive access for the player to the instance.
-
Scenario script checks the access response and then acts according to the situation:
- If the player has no inventory on the controller then the current inventory becomes the synced inventory and the process is done.
- If the player inventory was previously uploaded and is the same as the one stored on the controller then the current inventory becomes the synced inventory and the process is done.
- If the player inventory was previously uploaded but is not the same then the player inventory is deleted and the player is turned into a spectator.
- Otherwise the player inventory is kept.
-
Scenario script asks for the player inventory from the controller.
-
Controller sends it to the scenario in a stream of chunks.
-
Scenario displays and updates a progress bar as each chunk is received.
-
Once all chunks have loaded the player's synced inventory is recreated from the data and the player can start playing.
When the player leaves the inventory is uploaded if it's a synced inventory and the exclusive access the scenario script holds is released. Should an error occur during this process the player is given the option to use a temporary inventory instead, which will be merged back into the synced inventory the next time the sync succeeds on that instance.
Communication between the server and instance goes over stdout or rcon, depending on the size of the data. Communication between the instance and the controller goes over websockets. Overall, we are able to achieve a latency between 3 and infinite ticks from server join, depending on the size of the inventory. The major limiter is rcon transfer speeds with larger inventories, especially if they contain blueprints.