An instance with older data can upload the generation a save already holds, and a prefetch of it then overwrote that
save's newer state. A join now drops a prefetched copy unless the ordinary download check says the data is needed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Chunks of an earlier controller run's copy still in flight could land in a newer run's copy of the same generation. Every chunk now carries the token, and the scenario ignores a chunk whose token differs from its record's.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
If the controller crashes after sending generation N but before saving it, the destination keeps that prefetch while the controller restarts at N-1, and another instance can then upload different data as generation N. The controller now picks a random token per run, sends it with every prefetch and acquire reply, and the scenario applies a held copy only when its generation and token both match, and lets a new run's copy replace one with another token.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A prefetched copy is sent without the save's list of cleared recipes, so
its delta had only add, and a recipe cleared on the destination but not
in the stored list stayed cleared where a download would give it its
notification back. Without a current list the instance now sends the
whole stored list marked full, and the save gives the recipes it has
cleared that aren't in it their notification back, as a download's
remove does.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
player_sync.prefetch_inventory holds a copy sent ahead in active_downloads
and finished_downloads, for a player who is offline or doesn't exist on
the destination, with the components its first chunk names. Chunk 1
refuses a player who is online, a download in progress, a stream of the
same or a newer generation and a finished copy of a newer one; a finished
copy of the same generation is replaced. At the acquire reply a copy is
used only when the reply is acquired with its generation and its
components are the ones the scenario syncs now. The player is then
switched to spectator as a download does and the copy is applied, at once
or at its last chunk. Otherwise the copy is dropped and the join
continues as before.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A plugin that moves a player calls remote.call("player_sync",
"set_destination", player_name, instance_id) before connect_to_server.
It only records the destination. Every leave clears it, and a leave
that uploads with reason switching_servers adds it to the upload as
destination_instance_id. The README describes it for plugin authors.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The instance passes destination_instance_id from a leave upload on as
destinationInstanceId, and sends a PrefetchEvent's data to
player_sync.prefetch_inventory in RCON chunks, using the download's
chunk loop. As for a download, the data is stripped by the instance's
own player_sync.sync_* options and the components go with the first
chunk. A non-empty reply means the save refused or ignored the copy and
stops the transfer.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An upload can carry destinationInstanceId. Once the upload is stored,
the controller sends the stored data, the upload merged with its copy,
in a PrefetchEvent to that instance, if it is another running instance
with player_sync enabled. No lock is taken or released. A failed send is
logged and doesn't fail the upload.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The drawer actions move to its header as Export, Import and a trash
icon for Delete. Import and Delete are disabled while an instance
holds the player, since the stored data is out of date until it
uploads. The drawer also shows color and cheat mode.
Inventory tables get title case names, pagination at 10 rows, item
search, sorting and quality icons. The player list shows force with
a filter instead of generation, and the database stats use
Descriptions without the largest entry.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Imports no longer accept the cutscene controller, which needs waypoints
to restore. Logistic filters and the basic item stack fields now have the
types and limits Factorio enforces, and the inventory summary skips
malformed items and counts repeated exported items once instead of
expanding them.
Lua's JSON loses precision past 2^50, so a generation there could no
longer be counted up and uploads were discarded. Imports now start over
from generation 1 when the stored or imported generation is at that limit.
Instances holding a copy from another epoch take the stored generation
instead of keeping their own higher one when there is no data to download.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Imports and uploads could carry deleted: false, which the loader took for
a tombstone on the next restart. The schemas now reject the property, and
the loader only treats deleted: true as a tombstone and strips the key from
entries already saved with it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An import with color but no chat_color or tag assigned nil to the missing
LuaPlayer properties and failed restoration.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Wait for player list replication in integration tests
* Select available RCON ports before starting Factorio
* Document zero as automatic RCON port allocation
* Address review wording and test organization
* Restrict port collision hardening to integration fixtures
The send buffer limit in controller.max_send_buffer_size is set in MiB, but
buffered messages were measured with String.length, which counts UTF-16 code
units. Messages with non-ASCII text could go over the configured size in bytes
before the session was ended.
Measure each serialised message with Buffer.byteLength when it is buffered,
keep that size with the entry, and subtract the same size when it is
acknowledged. The web UI build already provides Buffer through the buffer
polyfill.
Fixes#1108
Co-authored-by: ColumbusLabs <287001685+ColumbusLabs@users.noreply.github.com>
The raise covered the whole gap between the serialized main inventory
and the main inventory with the armor restored, but only the bonus of
the armor's equipment arrives at the end of the tick. When the
destination force has a smaller character_inventory_slots_bonus than
the source, the rest of the gap never arrived and removing the raise on
the armor event dropped those slots on the ground. The raise is now the
equipment's inventory bonus minus what the grid already applies, at
most the gap, and the slots past it overflow into a corpse as without
the raise. Factorio 1.1 has no inventory bonus equipment and gets no
raise.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
on_player_armor_inventory_changed for the restored armor is raised at
the end of the download's tick, after the inventory bonus of the armor's
equipment applied, so the raise is removed there instead of in the next
tick's on_tick. The raise is only made for armor with a grid, as the
event needs the armor to be restored.
A player leaving on the tick of a download no longer has the raise
removed from the character first, which shrank the main inventory
before the bonus applied and spilled the extra slots. The raise is
taken off the uploaded inventory bonus instead.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The inventory bonus of equipment in the armor's grid (toolbelts) is
applied by the engine on the tick after the armor is inserted. The main
inventory was restored in the same tick, so the items of every slot past
80 plus research plus the armor's own bonus did not fit and were put in
a corpse: legendary mech armor with legendary toolbelts left 2125 of
2330 slots in the corpse.
The armor is now restored first, the character's inventory bonus is
raised to fit the saved main inventory, and the raise is removed again
on the next tick, or before an upload if the player leaves first.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Logistic slots saved before 2.0 are keyed by index strings, which
json_to_table keeps as strings, so the migration branch never ran and
the 2.0 branch failed on the missing sections. Detect the old format by
the missing sections and convert the keys to numbers.
The import schema now uses the minimums Factorio enforces for character
stats, checks the filter value type and the quick bar comparator, and
allows filters without quality or comparator.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An import with `crafting_queue: {}` passed the schema and broke the
next restore in deserialize_crafting_queue. The import schema now
checks the members module/serialize.lua reads from the crafting queue,
quick bar, personal logistic slots and character stats, while still
allowing a component to be left out.
Character stats missing from the data are left as they are instead of
being set to nil.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Raise the catalog and package.json floors to the latest release within
each major, and move typebox to 0.34, react-router-dom to 7,
winston-daily-rotate-file to 5, webpack-dev-middleware to 8 and
webpack-manifest-plugin to 6. None of these need code changes.
express 5.3.0 requires proxy-addr ^2.0.8 and qs ^6.16.0, which puts a
floor under the critical proxy-addr and high qs advisories for anyone
installing the published packages. webpack-dev-middleware 8 fixes a
high advisory in the controller's dev mode, and react-router 7 fixes two
moderate ones.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Browsers have no setImmediate. The web bundle only had one because jszip
depended on the setimmediate package, which installs it as a global.
Since jszip was replaced in #1011 every component using a subscription
threw on unmount, breaking page changes in the web interface.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A host running the plugin from before epochs never sends one, so after
an import or delete every upload it made was discarded while the
instance believed it succeeded. The Lua now always sends its epoch, and
an upload without one is compared by generation against the stored
data the way it was before, taking over the stored epoch. Those hosts
can still overwrite an import with a copy they failed to upload, which
the README now says.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The tombstones lived in player_sync_deleted.json, written after
player_sync.json. A stop between the two writes lost the tombstone of a
fresh delete, and the next upload from an old copy brought the deleted
data back. Deleted players are now entries with deleted: true in
player_sync.json, so both are written in one safeOutputFile call.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An import with an unknown controller or malformed inventories replaced
the stored snapshot and bumped its epoch, then broke the player's next
join. SetPlayerDataRequest now checks the fields the scenario restores,
leaving item stacks to the Lua deserializer and allowing the partial
snapshots instances with disabled components upload.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An import, delete or release for one player that finished after another
player was selected reloaded the drawer with the first player's data
under the second player's name. The drawer body is now keyed by player,
so a late completion can only update the view it was started from.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Generations alone could not order a repair against copies held by
instances. Deleting dropped the watermark, so a stale instance could
skip downloading the new baseline. An import could land on the same
generation as an upload that never reached the controller, and the
instance would then keep its own copy.
Imports and deletes now bump a per-player epoch, which the controller
returns on acquire and checks on upload. Instances throw away a copy
from an earlier epoch, and deleted players keep their generation and
epoch in player_sync_deleted.json. A restarted download also takes the
new generation instead of the outdated one.
Generations in player data must now be non-negative integers, so an
invalid import is refused before it reaches the datastore.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
InstanceDetailsGetRequest needs core.instance.get, which the
player-sync list and show commands are not documented to require.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Factorio's table_to_json writes an empty inventory as {"i":{}}, which
made the summary throw for show and the web drawer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A request for one player could resolve after the drawer switched to
another and show its inventory, and export it, under the wrong name.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov flagged ctl.ts at 21% patch coverage. Run each command
through a fake control link with canned responses and check what it
prints and sends.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Admins had no way to look at or fix the player data the controller
stores when a sync went wrong. The web page now lists every stored
entry with its generation, size and the instance holding the lock, and
opens a drawer per player with an inventory summary, JSON export and
import, delete and lock release. The same operations are exposed as
clusterioctl player-sync commands.
Import bumps the generation past the stored one so instances holding an
older copy download the new data on the next join. Import and delete
are refused while an instance holds the lock, since its upload on leave
would undo the change. A new player_sync.inventory.modify permission
gates the write operations.
Closes#1003
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Replace jszip with yauzl and yazl
jszip loads a whole archive into memory to read it and buffers the
output when writing. Saves and mods can be several hundred megabytes,
so read zips with yauzl, which only reads the central directory and
streams entries on demand, and write them with yazl, which streams the
output.
lib gains a ZipArchive wrapper around yauzl with the lookup helpers the
callers need, findRoot() now takes it instead of a JSZip. Saves are
patched by streaming every kept entry into a new zip, mod info.json is
read straight from the file, exports are streamed to the controller and
Factorio zip downloads go through a temp file instead of a buffer.
Patching now inflates and deflates each entry again where jszip reused
the compressed data, on a 336 MB synthetic save that is 7 s instead of
0.7 s, with peak memory going from 416 MB to 151 MB.
Closes#1002
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Copy save entries without recompressing them
yazl inflates and deflates every entry it is given, so patching a save
took ten times longer than with jszip, which reused the compressed
data. Add a ZipWriter to lib that streams entries out as they are
added and copies entries from a ZipArchive byte for byte, with ZIP64
structures when sizes or offsets need them, and use it in patch().
On a 336 MB synthetic save patching now takes 0.4 s at 143 MB peak RSS
against 0.75 s at 417 MB with jszip. Factorio loads the result.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Handle save write errors before the patch loop awaits them
A write error on the temporary save, such as a full disk, rejected the
pipeline promise while addEntry() or addBuffer() was still pending. Nothing
was listening yet, so the host's unhandledRejection handler exited the
process instead of failing the instance start.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Keep the export zip cache local to each export
The cache was module level, so when two instances on one host exported at
once, the first to finish closed archives the other was still reading.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Start writing zips before adding files from disk
yazl reads files passed to addFile() as soon as they are added. An
unreadable file emitted an error while the directory traversal was still
running, before the output pipeline existed, and the process exited on the
uncaught error. build_mod and zipDirectory now start the pipeline first and
reject instead. build_mod also removes the partial zip on failure.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
The scenario no longer sends its components with the download request. The instance strips the download by its own config and sends the components it used with the first chunk. If they differ from the ones the scenario started the download with, because the config changed in between, the scenario restarts the download.
A pending download is now marked with sync_in_progress instead of clearing dirty while spectating, and it is set for every download rather than only when inventories are synced. Uploads are skipped while it is set, so a player who leaves during a download with inventories off no longer uploads their stale state.
SyncComponent names the components in the Lua annotations.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
serialize_local_controller now respawns a player waiting to respawn for a moment to read the logistic requests hidden in the ghost, when logistics are not synced. Without this the requests were lost when the download turned the ghost into a spectator.
deserialize_player compares the physical controller with the one it restores. A player in remote view with a character was seen as changing controller, so with inventories off their crafting queue was serialized and queued a second time on the same character.
The ghost branch only sets the force when the force is synced, it errored on the stripped field otherwise.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
place_items no longer creates script inventories for what does not fit. Each serialized inventory is split at the size of its destination: the slots that fit are restored slot for slot and the rest become serialized stacks, which are inserted into the free space of the same inventory and then main, guns, ammo and armor. What is still left over is returned as a list and spill_items puts it into a corpse sized to hold it. This keeps 1.0 out of reach of game.create_inventory.
The armor-first fill order is gone, as the bonus slots from armor equipment only apply on the next tick anyway (#1106). Filters are no longer stripped for destinations, they are the same inventory type the data came from.
deserialize_inventory concatenated the filter table in its error print on 2.0, which errored out the whole restore when an inventory did not support filters.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With sync_controller on and sync_inventories off, switching to a synced
god, ghost or spectator destroyed the local character and its items.
Serialize the local items first and place them in the new controller.
Logistic requests and the flashlight live on the character, which the
download replaces when inventories are synced. Save them with the local
controller and restore them when their component is not synced.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A character without requests serialized to nil, which the controller
reads as a player with nowhere to hold them, so with inventories off
the stored requests came back. Serialize an empty table instead, clear
old slots before restoring on 1.1, and fix the set_personal_logistic_slot
typo that made any 1.1 restore fail.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
