BastiaanandClaude Opus 5.5
c28e617b22
* Replace jszip with yauzl and yazl jszip loads a whole archive into memory to read it and buffers the output when writing. Saves and mods can be several hundred megabytes, so read zips with yauzl, which only reads the central directory and streams entries on demand, and write them with yazl, which streams the output. lib gains a ZipArchive wrapper around yauzl with the lookup helpers the callers need, findRoot() now takes it instead of a JSZip. Saves are patched by streaming every kept entry into a new zip, mod info.json is read straight from the file, exports are streamed to the controller and Factorio zip downloads go through a temp file instead of a buffer. Patching now inflates and deflates each entry again where jszip reused the compressed data, on a 336 MB synthetic save that is 7 s instead of 0.7 s, with peak memory going from 416 MB to 151 MB. Closes #1002 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Copy save entries without recompressing them yazl inflates and deflates every entry it is given, so patching a save took ten times longer than with jszip, which reused the compressed data. Add a ZipWriter to lib that streams entries out as they are added and copies entries from a ZipArchive byte for byte, with ZIP64 structures when sizes or offsets need them, and use it in patch(). On a 336 MB synthetic save patching now takes 0.4 s at 143 MB peak RSS against 0.75 s at 417 MB with jszip. Factorio loads the result. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Handle save write errors before the patch loop awaits them A write error on the temporary save, such as a full disk, rejected the pipeline promise while addEntry() or addBuffer() was still pending. Nothing was listening yet, so the host's unhandledRejection handler exited the process instead of failing the instance start. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Keep the export zip cache local to each export The cache was module level, so when two instances on one host exported at once, the first to finish closed archives the other was still reading. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Start writing zips before adding files from disk yazl reads files passed to addFile() as soon as they are added. An unreadable file emitted an error while the directory traversal was still running, before the output pipeline existed, and the process exited on the uncaught error. build_mod and zipDirectory now start the pipeline first and reject instead. build_mod also removes the partial zip on failure. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Clusterio Controller
Communication hub for Clusterio clusters. The controller forwards data between Clusterio hosts connected to it and allows the cluster to be remotely managed through WebSocket connections to it either by using the included web interface or the Clusterio ctl command line utility.
Usage
npx clusteriocontroller <command>
Common options:
-
--plugin-list <file>JSON file to use for storing the list of plugins that are available to the controller. Defaults toplugin-list.jsonand will be created if it does not exist. See theplugincommand for managing this list. -
--config <file>JSON file to use for storing configuration for the controller. Defaults toconfig-controller.jsonand is required for operation. See theconfigcommand for inspecting and modifying the configuration. -
--create-configIf the config file is missing, create a new one and carry on instead of exiting with an error. Alternative to theconfig createcommand.
plugin <command>
Configure plugins available to be loaded by the controller. The available plugins will be loaded unless they have been disabled in the configuration, see the config command for disabling plugins.
plugin install <package>
Install a plugin from npm and add it to the plugin list.
The package is passed to npm install --save so anything npm accepts works, including a version like @clusterio/plugin-subspace_storage@2.0.0.
This command refuses to run outside the directory Clusterio was installed in, as the plugin has to end up in that directory's node_modules to be found.
For example, installing the Subspace Storage plugin:
npx clusteriocontroller plugin install @clusterio/plugin-subspace_storage
Since the plugin-list.json is shared between controller, host and ctl you usually only need to do this once per machine.
plugin add <path>
Add plugin either by require path or relative/absolute path to the plugin directory.
A relative path must start with ./ or ../ (or .\ and ..\ on Windows) otherwise it will be assumed to be a require path for an installed package in node_modules.
This is mainly useful for plugins under development, packages published on npm are better installed with plugin install.
For example, adding a plugin checked out in the external_plugins directory:
npx clusteriocontroller plugin add ./external_plugins/my_plugin
plugin remove <name>
Remove a plugin by its name. This should be done before uninstalling the plugin, otherwise there will be an error when Clusterio tries to load the info from the plugin. Removing and unistalling a plugin is usually not neccessary as the functions provided by the plugin can be disabled in the config.
For example, uninstalling the Subspace Storage plugin:
npx clusteriocontroller plugin remove subspace_storage
npm uninstall @clusterio/plugin-subspace_storage
Since the plugin-list.json is shared between controller, host and ctl you usually only need to do this once per machine.
plugin list
Lists the plugins set up to be available by name followed by path.
config
Manage the controller configuration offline. This should only be used when the controller is stopped, otherwise the config read might be out of date and config changes will be overwritten when the controller shuts down.
config create
Create a new config file with the default options inserted into if it doesn't already exist.
See also the --create-config option.
config set <config-entry> [value]
Set a config entry to the given value. If value is not provided the entry is set to null. If the config entry is of type object the value must be a valid JSON serialization of an object.
See docs/configuration.md in the main repositiory for the available configuration entries.
config show <config-entry>
Shows the value for a single config entry.
config list
Lists up all configuration entries with their currently configured values.
bootstrap
Commands for setting up initial admin acount and access to the cluster.
bootstrap create-admin <name>
Creates a new user account with the given name if it does not already exist in the cluster and grant it full cluster admin permission. You should be using your own Factorio multiplayer username here.
bootstrap generate-user-token <name>
Generate an access token for logging in and possibly managing the cluster from the given user account.
This can be used both in the web interface and as the controller_token to connect with clusterioctl.
bootstrap create-ctl-config <name>
Creates a clusterioctl config for the given user with url and token set up for connecting to the cluster.
copy-static [target]
Copies the static web assets to the given target folder (which defaults to ./static/).
Useful in case you want to host the assets on a CDN or separate web server and want provide the files directly instead of fetching them from the controller.
Note that mod pack exports will create additional static files that also have to be hosted, these are written to the ./static/ folder in the clusterio installation directory.
See also the controller.public_url config option for setting where static assets are loaded from.
run
Runs the controller.
See Also
The Clusterio repository for instructions on how to set up a cluster.