# Managing a Cluster Clusterio clusters are managed through the controller by using the `clusterioctl` command line interface which is invoked by running `npx clusterioctl ` in the clusterio directory. This document uses the shorthand `ctl> foo` to indicate `npx clusterioctl foo` should be executed. Mandatory parameters are shown in `` and optional pameters are in `[square brackets]`. Before `clusterioctl` can be used it needs to be configured for the cluster it will connect to. The easiest way to do this is to run `npx clusteriocontroller bootstrap create-ctl-config ` on the controller, which creates the necessary `config-ctl.json` for managing the cluster as the given user. ## Hosts ### List hosts ctl> host list Lists all hosts. Hosts are added to the cluster by running the installer on the computer the host runs on and approving the code it shows, see [Setting up remote hosts](/README.md#setting-up-remote-hosts). Listing, approving and rejecting hosts waiting for approval needs the `core.host.approve` permission. ### List pending hosts ctl> host pending Lists hosts waiting for approval with the code they show, the name they asked for, the address they connected from and how long they have been waiting. A request goes away when its host stops waiting, and after the time set by `controller.host_pairing_timeout`, an hour by default, after which the host asks again with a new code. ### Approve host ctl> host approve [--name ] Approves the host showing the code and prints the id of the new host. The code is not case sensitive and the dash can be left out. `--name` gives the host a different name than the one it asked for. Approving gives the host full access to the cluster, check with `host pending` that the name and address match the computer you are setting up. ### Reject host ctl> host reject Rejects the host showing the code, the host then exits with an error. ### Revoke host ctl> host revoke Specify the host by name or ID. Removes the host from the cluster, disconnects it and makes its token stop working. Instances assigned to the host become unassigned, their files stay on the computer. Requires the `core.host.revoke` permission. To add the computer back to the cluster, start the host again and approve it. ### Update Clusterio ctl> host update [--all] [--restart] Specify the host by name or ID. If remote updates are enabled on the host, this will attempt to update `@clusterio/host` on the machine. Add `--all` to also update all npm-installed plugins. Changes will not be applied until the host is restarted. If the `--restart` flag is provided, the host will automatically restart once the update is complete. Remote updates are enabled by default but can be disabled on a per-machine basis at the owner's discretion. For security reasons, this configuration setting cannot be changed remotely. To modify it, run the following command locally while the host is offline: host> config set host.allow_remote_updates true/false The controller also supports remote updates, following the same logic. The corresponding commands are: ctl> controller update [--all] [--restart] controller> config set controller.allow_remote_updates true/false ### Mark restart required ctl> host mark-restart-required Marks the host as requiring a restart, the same way a plugin update does. Use it after installing or changing a plugin by other means than `host plugin install`, such as a plugin checked out into `external_plugins`. The mark shows up in the web interface on the next system metrics poll. The controller has the same command: ctl> controller mark-restart-required ## Instances ### List Instances ctl> instance list [--templates] List instances. Templates are only listed if `--templates` is passed. ### Create instance ctl> instance create [--id ] [--from ] [--template ] [--as-template] Create a new instance. `name` is the instance name. `--id` is the optional id. `--from` copies the config of an existing instance. `--template` makes the new instance inherit all config values from the given template. `--as-template` creates the instance as a template, see `instance.is_template` in the [configuration docs](configuration.md). Use `--as-template --from ` to make a template out of an existing instance. ### Kill instance ctl> instance kill Terminates the running Factorio server without giving it a chance to save or cleanup after itself. Note: This may cause loss of data. ### Save game ctl> instance save-game Saves the running game of the instance to the save it was started from and waits for the save to finish. ### Extract players from instance ctl> instance extract-players Creates a user account in the cluster for each player that has been online on the currently running save of the instance and sets the online time of that account on this instance to the online time recorded in the save. Useful when importing a save to a cluster. Note that the online time in a save is recorded differntly from how Clusterio records online time. Most notably Clusterio records wall clock time while the save time is recorded in ticks and this extraction of player time assumes the game runs at 60 ticks a second. ### Config management #### List Configuration ctl> instance config list List config values for an instance. `instance` is the instance whose config values to list. Values inherited from a template are marked with `(inherited)`, object fields also list the properties set on the instance. #### Set config value ctl> instance config set [value] [--stdin] Set field in instance config. `instance` is instance to set. `field` is field to set. `value` is value to set in field. `--stdin` signals to read value from stdin. #### Set config property of field To be written. #### Inherit config value ctl> instance config inherit [prop] Make a field in the instance config take its value from the template again, undoing a previous `set`. `instance` is the instance to update, `field` is the field to inherit, `prop` limits it to one property of an object field. #### Edit instance config ctl> instance config edit [editor] Edit instance config in a text editor. Once file is saved and editor has been exited, changed config values will be read back in. `instance` is the instance to edit, `editor` is the editor to use. If `editor` is not given, clusterioctl will read from the `EDITOR` or `VISUAL` enviroment variables. ### Assign instance to host ctl> instance assign [host] Assigns instance to host. `instance` is instance to assign, `host` is host to assign to. If `host` is not provided, instance will be unassigned. ## Users Clusterio automatically creates user accounts for all players that join an instance when save patching is enabled. These accounts are used to store per player data shared between instances, like if the player should be whitelisted, admin or banned. ### List users ctl> user list Lists all user accounts in the cluster along with some data for each. ### Create User ctl> user create Creates a new empty user account for the given Factorio player. Note that case matters here. This is usually not required as user accounts are created automatically for players that join instances with save patching enabled. ### Promote User to Server Admin ctl> user set-admin [--revoke] [--create] Promotes the user given to in-game admin on instances with the `sync_adminlist` option enabled. If the `--revoke` switch is used the user is removed from the adminlist. Since admin status is a part of the account data the account must exist for this to succeed, passing `--create` will create the account if it does not exist. **Note:** Being a server admin does not grant any access to manage the cluster. See [set-roles](#set-cluster-roles) for adding roles which grant access to managing the clusetr. ### Whitelist User ctl> user set-whitelisted [--remove] [--create] Add the user given to the whitelist on instances with the `sync_whitelist` option enabled. If the `--remove` switch is used the user is removed from the whitelist. Since whitelisted status is a part of the account data the account must exist for this to succeed, passing `--create` will create the account if it does not exist. ### Ban User ctl> user set-banned [--reason ] [--pardon] [--create] Ban user in-game from instances with the `sync_banlist` option enabled. Reason is a message that will be shown to the user when attemption to log in. If the `--pardon` switch is used it removes the ban. Since ban status is a part of the account data the account must exist for this to succeed, passing `--create` will create the account if it does not exist. Note: This bans the user from logging in to Factorio servers in the cluster, it does not revoke access to any cluster management they might have, see next section on setting roles for revoking tha. ### Bulk importing admin, whitelist and ban lists Whitelists and admin lists can be bulk imported with the following commands: ctl> user whitelist-bulk-import [--create] ctl> user admin-bulk-import [--create] The file should simply be newline seperated names: joe bob allan The `--create` option will create the user if they do not exist, and then set the relevent value. Banlists can be bulk imported with the following command: ctl> user banned-bulk-import [--create] The file should be names, followed by a ban reason: joe greifed copper mines bob stole materials allan rude language The first string on the line is used as the username, the others are used as the ban reason. The `--create` option will create the user if they do not exist, then ban them. ### Set Cluster Roles ctl> user set-roles [roles...] Replaces the roles the user has in the cluster with a new list of roles. Calling with an empty roles argument will remove all roles from the user. By default there's a roled named Cluster Admin which grants access to everything and a role named Player which grants a limited read access to the cluster, see [the section on roles](#roles) for more information about setting up roles and permissions. ### Delete user ctl> user delete Deletes everything stored on the controller for this user. Note: If the player is banned from the cluster this will effectively unban them, as the ban status is stored with the user account. Note: If the player joins the cluster again a new account will be made for them automatically. ## Roles To be written. ## Notes Free text notes can be attached to hosts, instances, users, roles, mod packs, mods and saves. They are stored on the controller and shown to everyone with access to the resource in the web interface, and can for example be used to record who runs a host or what an instance is meant for. The web interface renders notes as Markdown. A mention of the form `@/` becomes a link to that resource showing its name, for example `@instance/778528005` or `@user/cooldude2606`. This works for hosts, instances, users, roles and mod packs, and the id is shown if the name is not available. The note itself stores the mention as written. ### List notes ctl> note list [--type ] Lists all notes, optionally limited to one resource type. ### Set note ctl> note set Sets the note of a resource. The type is one of host, instance, user, role, mod_pack, mod or save. The id is the numeric id for hosts, instances, roles and mod packs, the lower case name for users, `_` for mods and `/` for saves. ### Delete note ctl> note delete Removes the note of a resource. Notes are also removed when the resource they are attached to is deleted. ## Plugins (Local) To be written. ## Plugins (Remote) The installer asks "Allow remote updates and plugin installs from the web interface?" and the answer sets `allow_remote_updates`, `allow_plugin_updates` and `allow_plugin_install` together on that machine, pass `--remote-npm` or `--no-remote-npm` to answer it from the command line. Remote updates allow ctl and the web UI to update Clusterio and plugins on machines within the cluster. Remote installs allow them to install any package from npm onto the machine, which is a risk of arbitrary code execution through malicious npm packages, so only allow them if you trust everyone with access to install plugins. In a config created without the installer remote updates are enabled and remote installs are disabled by default. See below for changing these settings after installing. ### List ctl> controller plugin list ctl> host plugin list This command lists all plugins currently known to the remote. Newly installed plugins will not be displayed until the remote is restarted. Plugins that are distributed as an npm package can be updated remotely. ### Update ctl> controller plugin update [@] [--restart] ctl> host plugin update [@] [--restart] If remote plugin updates are enabled on the remote, this will execute `npm update --save `, which stays within the version range saved in the remote's `package.json`. To update to a specific version, including one outside that range, pass it as `@`, for example `@clusterio/plugin-example@2.1.0`. The version must be published on npm, and the remote then executes `npm install --save @`. The plugin must already be installed on the remote, and only remote plugin updates need to be enabled. Changes will not be applied until the remote is restarted. If the `--restart` flag is provided, the remote will automatically restart once the update is complete. ### Update All ctl> controller update --all [--restart] ctl> host update --all [--restart] Updates the Clusterio package and all plugins installed from npm in a single `npm update --save` command. Updating everything at once avoids ending up with multiple versions of `@clusterio/lib` in the installation. Requires both remote updates and remote plugin updates to be enabled on the remote. ### Install ctl> controller plugin install [@] [--restart] ctl> host plugin install [@] [--restart] If remote plugin installs are enabled on the remote, this will execute `npm install --save `. Pass `@` to install a specific version, for example to match the version used by the rest of the cluster. The package, and the version if given, must be published on npm. Changes will not be applied until the remote is restarted. If the `--restart` flag is provided, the remote will automatically restart once the update is complete. ### Search ctl> controller plugin search [query] [--page ] [--page-size ] Searches the npm registry for packages tagged with the clusterio-plugin keyword. The search is done by the controller and does not require remote installs to be enabled. The web UI has a plugin browser on the plugins page that uses the same search. ### Concurrent Operations Only one npm operation runs at a time on a machine. A standalone controller and host share their working directory, so the operation holds the `npm-operation.lock` file there while npm runs. Other installs and updates on the same machine or installation are rejected with "Another package operation is already in progress on this machine" until it finishes. A lock file left behind by a process that is no longer running is removed automatically. ### Enable/Disable Updates host> config set host.allow_plugin_updates true/false controller> config set controller.allow_plugin_updates true/false Remote updates are enabled by default but can be disabled on a per-machine basis at the owner's discretion. For security reasons, this configuration setting cannot be changed remotely. To modify it, run the above commands locally while the remote is offline. ### Enable/Disable Installs host> config set host.allow_plugin_install true/false controller> config set controller.allow_plugin_install true/false Remote installs are disabled by default but can be enabled on a per-machine basis at the owner's discretion. For security reasons, this configuration setting cannot be changed remotely. To modify it, run the above commands locally while the remote is offline. ## Prometheus Metrics The controller exports metrics for the whole cluster on the `/metrics` endpoint. Fetching it requires a token for a user with the `core.controller.metrics` permission, sent either as an `Authorization: Bearer ` header or as an `X-Access-Token` header. Requests without a valid token get a 401 response and requests for users lacking the permission get a 403 response. Create a role with just this permission, a user to hold it, and generate a token for that user. Give the user a name that is not a valid Factorio player name, such as one starting with `#`, so a player joining the cluster can never claim the account and its token: ctl> role create Prometheus --permissions core.controller.metrics ctl> user create #prometheus ctl> user set-roles #prometheus Prometheus $ npx clusteriocontroller bootstrap generate-user-token '#prometheus' Then pass the token in the scrape job for the controller in `prometheus.yml`: scrape_configs: - job_name: clusterio authorization: credentials: static_configs: - targets: ["localhost:8080"] Concurrent scrapes share a single round of metrics gathering from the hosts, so multiple scrapers polling the controller do not multiply the load on the hosts.