BastiaanandClaude Opus 5.5
c28e617b22
* Replace jszip with yauzl and yazl jszip loads a whole archive into memory to read it and buffers the output when writing. Saves and mods can be several hundred megabytes, so read zips with yauzl, which only reads the central directory and streams entries on demand, and write them with yazl, which streams the output. lib gains a ZipArchive wrapper around yauzl with the lookup helpers the callers need, findRoot() now takes it instead of a JSZip. Saves are patched by streaming every kept entry into a new zip, mod info.json is read straight from the file, exports are streamed to the controller and Factorio zip downloads go through a temp file instead of a buffer. Patching now inflates and deflates each entry again where jszip reused the compressed data, on a 336 MB synthetic save that is 7 s instead of 0.7 s, with peak memory going from 416 MB to 151 MB. Closes #1002 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Copy save entries without recompressing them yazl inflates and deflates every entry it is given, so patching a save took ten times longer than with jszip, which reused the compressed data. Add a ZipWriter to lib that streams entries out as they are added and copies entries from a ZipArchive byte for byte, with ZIP64 structures when sizes or offsets need them, and use it in patch(). On a 336 MB synthetic save patching now takes 0.4 s at 143 MB peak RSS against 0.75 s at 417 MB with jszip. Factorio loads the result. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Handle save write errors before the patch loop awaits them A write error on the temporary save, such as a full disk, rejected the pipeline promise while addEntry() or addBuffer() was still pending. Nothing was listening yet, so the host's unhandledRejection handler exited the process instead of failing the instance start. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Keep the export zip cache local to each export The cache was module level, so when two instances on one host exported at once, the first to finish closed archives the other was still reading. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Start writing zips before adding files from disk yazl reads files passed to addFile() as soon as they are added. An unreadable file emitted an error while the directory traversal was still running, before the output pipeline existed, and the process exited on the uncaught error. build_mod and zipDirectory now start the pipeline first and reject instead. build_mod also removes the partial zip on failure. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Clusterio Host
Node hosting Factorio servers in a Clusterio cluster. Clusterio hosts connect to the controller and waits for commands from the controller to start up and stop instances. A cluster can have any number of hosts in it located on different computers, and each host can host any number of instances each of which is a Factorio server that talks with the rest of the cluster.
Usage
npx clusteriohost <command>
Common options:
-
--plugin-list <file>JSON file to use for storing the list of plugins that are available to the host. Defaults toplugin-list.jsonand will be created if it does not exist. See theplugincommand for managing this list. -
--config <file>JSON file to use for storing configuration for the host. Defaults toconfig-host.jsonand is required for operation. See theconfigcommand for inspecting and modifying the configuration. -
--create-configIf the config file is missing, create a new one and carry on instead of exiting with an error. Alternative to theconfig createcommand.
plugin <command>
Configure plugins available to be loaded by the host. The available plugins will be loaded unless they have been disabled in the configuration, see the config command for disabling plugins.
plugin install <package>
Install a plugin from npm and add it to the plugin list.
The package is passed to npm install --save so anything npm accepts works, including a version like @clusterio/plugin-subspace_storage@2.0.0.
This command refuses to run outside the directory Clusterio was installed in, as the plugin has to end up in that directory's node_modules to be found.
For example, installing the Subspace Storage plugin:
npx clusteriohost plugin install @clusterio/plugin-subspace_storage
Since the plugin-list.json is shared between controller, host and ctl you usually only need to do this once per machine.
plugin add <path>
Add plugin either by require path or relative/absolute path to the plugin directory.
A relative path must start with ./ or ../ (or .\ and ..\ on Windows) otherwise it will be assumed to be a require path for an installed package in node_modules.
This is mainly useful for plugins under development, packages published on npm are better installed with plugin install.
For example, adding a plugin checked out in the external_plugins directory:
npx clusteriohost plugin add ./external_plugins/my_plugin
plugin remove <name>
Remove a plugin by its name. This should be done before uninstalling the plugin, otherwise there will be an error when Clusterio tries to load the info from the plugin. Removing and unistalling a plugin is usually not neccessary as the functions provided by the plugin can be disabled in the config.
For example, uninstalling the Subspace Storage plugin:
npx clusteriohost plugin remove subspace_storage
npm uninstall @clusterio/plugin-subspace_storage
Since the plugin-list.json is shared between controller, host and ctl you usually only need to do this once per machine.
plugin list
Lists the plugins set up to be available by name followed by path.
config
Manage the host configuration offline. This should only be used when the host is stopped, otherwise the config read might be out of date and config changes will be overwritten when the host shuts down.
config create
Create a new config file with the default options inserted into if it doesn't already exist.
See also the --create-config option.
config set <config-entry> [value]
Set a config entry to the given value. If value is not provided the entry is set to null. If the config entry is of type object the value must be a valid JSON serialization of an object.
See docs/configuration.md in the main repositiory for the available configuration.
config show <config-entry>
Shows the value for a single config entry.
create-scenario <output>
Create a scenario with the Clusterio modules patched into it and write it to output.
This is the same patching that is applied to saves when an instance starts, see docs/how-it-works.md in the main repository.
The result can be used as a regular scenario outside of Clusterio by copying it to the scenarios folder of Factorio or into a mod.
The IPC between the game and the host is not available outside of Clusterio, so modules that depend on it will not work as expected.
By default the freeplay scenario from the latest Factorio install in host.factorio_directory is patched with the modules from all plugins that are enabled in the host config.
-
--scenario <path>Scenario directory, zipped scenario or save to patch instead of freeplay. This has the same restrictions as save patching, the scenario must either use the event_handler library and have been seen by Clusterio before, or already have been patched by Clusterio. -
--factorio-version <version>Factorio version to take the freeplay scenario from. Defaults tolatest. -
--plugins <name>...Only patch in modules from the given plugins. -
--zipWrite the scenario as a zip file instead of a directory. Factorio does not load zipped scenarios, extract it before use.
For example, creating a scenario with only the player_sync module:
npx clusteriohost create-scenario ~/.factorio/scenarios/player_sync --plugins player_sync
config list
Lists up all configuration entries with their currently configured values.
run
Runs the host.
See Also
The Clusterio repository for instructions on how to set up a cluster.