Merge remote-tracking branch 'origin/master' into fix/headless-export-icons

docs/configuration.md

@@ -48,6 +48,23 @@ Must be greater than 0.
 Defaults to 60.
 
 
+### controller.host_pairing_timeout
+
+Time in seconds a host can wait for approval before its request expires, after which the host requests again with a new code.
+Set to 0 for no limit.
+
+Defaults to 3600.
+
+
+### controller.host_pairing_max_pending
+
+Maximum number of hosts that can wait for approval at the same time.
+Hosts over the limit keep trying until there is room.
+Set to 0 for no limit.
+
+Defaults to 50.
+
+
 ### controller.http_port
 
 Port to host HTTP server on.
@@ -93,7 +110,7 @@ Defaults to null meaning the header is never trusted.
 ### controller.public_url
 
 Public address the controller is accessible on.
-Used as the URL in configs generated with `clusteriocontroller bootstrap create-ctl-config` and `clusterioctl host create-config`, and by plugins like Player Auth that need to show the address of the web interface in-game.
+Used as the URL in configs generated with `clusteriocontroller bootstrap create-ctl-config`, and by plugins like Player Auth that need to show the address of the web interface in-game.
 This should be a full URL ending with a /.
 
 Defaults to null meaning assume localhost.
@@ -277,6 +294,7 @@ Defaults to 1000.
 
 Whether the `controller update` command in clusterioctl and the update button in the web interface are allowed to update the `@clusterio/controller` package with npm.
 For security reasons this can only be changed locally with `clusteriocontroller config set` while the controller is stopped, and it is hidden in the web interface.
+The installer sets this, controller.allow_plugin_updates and controller.allow_plugin_install from one question about remote updates and plugin installs.
 See [Managing a Cluster](/docs/managing-a-cluster.md) for details on remote updates.
 
 Defaults to true.
@@ -326,6 +344,7 @@ Defaults to "New Host".
 ### host.id
 
 Immutable numeric id of the host which uniquely identifies this host in the cluster.
+The controller picks the id when the host is approved and the host writes it here, the controller does not accept hosts with an id it has not approved.
 Hidden in the web interface.
 
 Defaults to a random 31 bit number.
@@ -374,7 +393,9 @@ Defaults to "http://localhost:8080/".
 ### host.controller_token
 
 Access token used for authenticating with the controller.
-You can generate an access token with `clusterioctl host generate-token --id <host-id>`, or use the `clusterioctl host create-config` to create a new host config with the correct url and token in it.
+The host saves it together with host.id when it is approved, see [Setting up remote hosts](/README.md#setting-up-remote-hosts).
+The token only works while the host exists on the controller, if the host is revoked it asks to be approved again the next time it starts.
+Set it to `enter token here` to make the host ask to join again.
 The value can only be read and changed from the host itself.
 
 Defaults to "enter token here".
@@ -427,6 +448,7 @@ Defaults to 60.
 
 Whether the `host update` command in clusterioctl and the update button in the web interface are allowed to update the `@clusterio/host` package with npm.
 For security reasons this can only be changed locally with `clusteriohost config set` while the host is stopped, and it is hidden in the web interface.
+The installer sets this, host.allow_plugin_updates and host.allow_plugin_install from one question about remote updates and plugin installs.
 See [Managing a Cluster](/docs/managing-a-cluster.md) for details on remote updates.
 
 Defaults to true.

docs/contributing.md

@@ -87,10 +87,12 @@ Install dependencies with:
 
 This installs dependencies and builds the packages.
 `pnpm-lock.yaml` is intentionally ignored; do not force-add it or assume a frozen-lockfile install is supported.
-For a fresh development cluster, run the installer and select standalone mode to create `config-controller.json`, `config-host.json`, and the credentials needed to connect them:
+For a fresh development cluster, run the installer and select standalone mode to create `config-controller.json` and `config-host.json`:
 
     node packages/create --dev
 
+The first time the host starts it prints a link for approving it in the web interface.
+
 You can skip this step if you already have a working controller and host configuration.
 
 To add plugins to use in the development environment use
@@ -123,6 +125,7 @@ For web development on the controller there are also the following flags:
 `--dev` - Serve the `web_ui` package from source through Vite's development server with hot module replacement.
 
 `--dev-plugin [name]` - Serve the web module for the given plugin from source the same way.
+This also serves the `web_ui` package from source, as the plugin has to share its modules with it.
 
 These options can be combined and multiple plugins can be actived for dev mode.
 Keep in mind if you're working on external plugins the `--dev-plugin` option is likely to only work if the plugin is placed inside the `external_plugins` directory and its dependencies are installed with pnpm as part of the clusterio repository workspace.
@@ -254,6 +257,7 @@ If there are no changelog entries to add, remove the code block and start the ch
 
 The version-compat job in CI runs the local controller with the latest published host, and the published controller with the local host.
 It fails if a link message changed its src, dst, data or response schema since that release.
+Control is left out of src and dst, and messages that only go between control and the controller are not checked, as the web UI is served by the controller it talks to.
 If the break is intended, add an entry to [scripts/version-compat-breaks.json](../scripts/version-compat-breaks.json) in the same pull request.
 Breaking changes are only accepted if the next release is intended to include them, see our decision on [breaking changes](/docs/decisions.md#breaking-changes).
 The entry is keyed by the published version directly before the breaking change was introduced, which is the latest release at the time.
@@ -338,7 +342,8 @@ The format of this log should be self explanatory, please follow it carefully. P
 
 For development, use the latest Node.js LTS release, following the [installation instructions](../README.md#installation).
 Make sure the Node.js built-in APIs and npm packages you use remain compatible with the supported versions.
-The GitHub Actions tests currently run on Node.js 24 and 26 across the Factorio matrix, with Node.js 22 used for coverage; see the [CI workflow](../.github/workflows/node.js.yml) for the version matrix.
+The GitHub Actions tests run every Factorio version on the latest Node.js, and every other supported Node.js version with the latest Factorio only, which keeps the number of jobs within GitHub's concurrent job limit.
+Coverage is collected on the oldest supported Node.js; see the [CI workflow](../.github/workflows/node.js.yml) for the version matrix.
 
 For Factorio, active support covers the latest stable version and the previous major version, using version lines such as 2.1, 2.0 and 1.1 as described in the [Factorio support policy](decisions.md#factorio).
 Older versions from 0.17 onwards receive passive support where it does not restrict actively supported versions.

docs/devs/plugin-migration-to-2.1.md

@@ -59,3 +59,23 @@ Imports of `assert`, `buffer`, `events`, `os`, `path`, `process`, `stream`, `uti
 The `container` property of `WebPluginContext` is now the plugin's ES module namespace, `await container.get("./web")` returns the module instead of a factory function.
 
 `lib.PluginWebpackEnvInfo` is renamed to `lib.PluginWebEnvInfo`.
+
+
+## Host pairing
+
+Host tokens are no longer generated by users.
+A host joins the cluster by having the code it shows when it first starts approved, see [Host pairing](protocol.md#host-pairing) for the HTTP endpoints it uses.
+
+- `HostGenerateTokenRequest` and `HostConfigCreateRequest` are removed.
+  Pending requests are listed with `HostPairingListRequest`, approved with `HostPairingApproveRequest` and rejected with `HostPairingRejectRequest`, and changes to them are sent to subscribers of `HostPairingUpdatesEvent`.
+- `HostRevokeTokensRequest` is replaced by `HostRevokeRequest`, which deletes the host instead of only invalidating its tokens.
+  Instances assigned to the host become unassigned.
+- The permissions `core.host.generate_token`, `core.host.revoke_token` and `core.host.create_config` are removed.
+  Check `core.host.approve` and `core.host.revoke` instead.
+- `clusteriocontroller bootstrap generate-host-token` is removed, scripts that set up hosts can start the host with `--user-token` to approve it with a user token.
+
+Host records are no longer created when a host connects, only by approving it, and the controller rejects hosts it has no record of.
+Revoking a host deletes its record and note.
+Plugins that keep data per host should remove it when the host is deleted.
+On the controller the `update` event of `controller.hosts` gives the record with `isDeleted` set to true, and subscribers of `HostUpdatesEvent` receive the host with `isDeleted` set to true.
+A host that is revoked and approved again gets a new id.

docs/devs/protocol.md

@@ -32,6 +32,63 @@ The [send-buffer setting](../configuration.md#controllermax_send_buffer_size) co
 ![Connection flow](../assets/connection_flow.svg)
 
 
+## Host pairing
+
+The host id and token sent in `registerHost` are requested by the host over HTTP the first time it starts, before it connects.
+Hosts are only added to the controller by approving a pairing request, and `registerHost` is rejected with close code 3000 (Unauthorized) for a host id the controller has no record of.
+Revoking a host deletes its record, so its token stops working and the host has to be approved again the next time it starts, which gives it a new id.
+See [Setting up remote hosts](/README.md#setting-up-remote-hosts) for how this looks to the user.
+
+The endpoints below take and return JSON bodies and are sent with `Cache-Control: no-store`.
+The `POST` endpoints return errors as `{"error": "<message>"}` and respond with 503 while the controller is in recovery mode.
+A request body that is not valid JSON or is larger than 4 kB is rejected with 400 or 413 before it reaches the endpoint, without a JSON body.
+`GET /api/host-pairing/check` also works in recovery mode, and responds to a token that is not valid with a plain 401 without a JSON body.
+A user or host token is sent in either an `Authorization: Bearer <token>` header or an `X-Access-Token` header.
+
+### `POST /api/host-pairing`
+
+Creates a request for a host to join the cluster.
+
+- `name: string` - Name of the host, at most 100 characters after trimming whitespace, without control characters.
+- `version: string` (optional) - Version of the host.
+
+Responds with 202 and `{"requestId": string, "code": string}`.
+The `code` is shown to the user to approve the request with, formatted as six characters like `ABC-DEF`.
+Codes are matched ignoring case and anything that is not a letter or digit, so `abcdef` matches `ABC-DEF`.
+The `requestId` is a secret used to poll for the result, it is never shown to users.
+
+If a user token is sent and the user has the `core.host.approve` permission the host is created right away and the response is 200 with the same body as an approved poll.
+An invalid token gives 401 and a user without the permission gives 403, a token is never ignored to create a pending request instead.
+
+A missing or too long name gives 400.
+Too many pending requests from the same address, or in total as set by [controller.host_pairing_max_pending](/docs/configuration.md#controllerhost_pairing_max_pending), gives 429 with a `Retry-After` header in seconds.
+
+### `POST /api/host-pairing/poll`
+
+- `requestId: string` - Id returned when the request was created.
+
+Responds with 202 and `{"status": "pending"}` while the request is waiting for approval.
+Once approved it responds with 200 and `{"hostId": integer, "token": string, "name": string}`, where `name` is the name the host was approved with, which can differ from the one asked for.
+Rejected requests give 403, and unknown or expired requests give 404.
+The result of an approved or rejected request is only returned once, after which the request is gone.
+
+A pending request expires 30 seconds after it was last polled, and after waiting for the time set by [controller.host_pairing_timeout](/docs/configuration.md#controllerhost_pairing_timeout).
+The host polls every 2 seconds, and makes a new request when the previous one has expired.
+
+### `POST /api/host-pairing/cancel`
+
+- `requestId: string` - Id returned when the request was created.
+
+Removes a pending request, sent by the host when it stops waiting.
+Responds with 204 whether or not the request existed.
+
+### `GET /api/host-pairing/check`
+
+Checks a host token.
+Responds with 200 and `{"valid": true}` if the token is valid and the host it was issued to still exists, and with 401 otherwise.
+A host with a saved token checks it with this when it starts, and requests to join again if the token is not accepted.
+
+
 ## Connector flow
 
 To manage the complexity of connecting, reconnecting, resuming or invalidating dropped sessions the WebSocket connection for a `Link` class is managed by a connector.
@@ -119,6 +176,11 @@ Host handshake for establishing a new connection session.
 - `plugins: Object<string, string>` -
     Object mapping plugin names to plugin version for plugins that are available on the host.
 
+The controller closes the connection with code 3000 (Unauthorized) if the token is invalid, was issued for another id, or if there is no host with this id, see [Host pairing](#host-pairing).
+The host checks its token before connecting and requests to join again if it is not accepted, so this normally only happens to a host that is already running.
+A host that has been rejected exits with code 8, and the run scripts created by the installer do not restart it.
+A host that is revoked while connected is disconnected, is rejected when it tries to connect again, and shuts down.
+
 #### `registerControl`
 
 Control handshake for enstablishing a new connection session.

docs/managing-a-cluster.md

@@ -16,17 +16,40 @@ The easiest way to do this is to run `npx clusteriocontroller bootstrap create-c
 
 Lists all hosts.
 
-### Generate token
+Hosts are added to the cluster by running the installer on the computer the host runs on and approving the code it shows, see [Setting up remote hosts](/README.md#setting-up-remote-hosts).
+Listing, approving and rejecting hosts waiting for approval needs the `core.host.approve` permission.
 
-    ctl> host generate-token [--id <host id>]
+### List pending hosts
 
-Generate a token for a host. `--id` is the host id.
+    ctl> host pending
 
-### Create config
+Lists hosts waiting for approval with the code they show, the name they asked for, the address they connected from and how long they have been waiting.
+A request goes away when its host stops waiting, and after the time set by `controller.host_pairing_timeout`, an hour by default, after which the host asks again with a new code.
 
-    ctl> host create-config [--id <host id>] [--name <host name>] [--generate-token <bool>] [--output <path to output>]
+### Approve host
 
-Create a host config. `--id` id the host id. `--name` is the host name. `--generate-token` decides whether to also generate a token, defaulting to `0`. `--output` is the filename for the output of the config; Default is "config-host.json", "-" for stdout.
+    ctl> host approve <code> [--name <host name>]
+
+Approves the host showing the code and prints the id of the new host.
+The code is not case sensitive and the dash can be left out.
+`--name` gives the host a different name than the one it asked for.
+Approving gives the host full access to the cluster, check with `host pending` that the name and address match the computer you are setting up.
+
+### Reject host
+
+    ctl> host reject <code>
+
+Rejects the host showing the code, the host then exits with an error.
+
+### Revoke host
+
+    ctl> host revoke <host>
+
+Specify the host by name or ID.
+Removes the host from the cluster, disconnects it and makes its token stop working.
+Instances assigned to the host become unassigned, their files stay on the computer.
+Requires the `core.host.revoke` permission.
+To add the computer back to the cluster, start the host again and approve it.
 
 ### Update Clusterio
 
@@ -270,9 +293,11 @@ To be written.
 
 ## Plugins (Remote)
 
-By default, remote updates are enabled unless you selected "no" during installation.
-Remote updates allow ctl and the web UI to update plugins on machines within the cluster.
-However, remote installations are disabled by default to prevent the risk of arbitrary code execution through the installation of malicious npm packages onto the machine. See below if you wish to enable this feature.
+The installer asks "Allow remote updates and plugin installs from the web interface?" and the answer sets `allow_remote_updates`, `allow_plugin_updates` and `allow_plugin_install` together on that machine, pass `--remote-npm` or `--no-remote-npm` to answer it from the command line.
+Remote updates allow ctl and the web UI to update Clusterio and plugins on machines within the cluster.
+Remote installs allow them to install any package from npm onto the machine, which is a risk of arbitrary code execution through malicious npm packages, so only allow them if you trust everyone with access to install plugins.
+In a config created without the installer remote updates are enabled and remote installs are disabled by default.
+See below for changing these settings after installing.
 
 ### List
 

docs/setting-up-tls.md

@@ -63,7 +63,7 @@ This creates a private key file named `key.pem` and a public certificate file na
     npx clusteriocontroller config set controller.tls_certificate cert.pem
     npx clusteriocontroller config set controller.tls_private_key key.pem
 
-Additionally you will need copy the `cert.pem` file to all of the computers you want to set up as hosts and configure node to trust it by setting the `NODE_EXTRA_CA_CERTS` environment variable to a path to this file before the host is started. You can do this by passing `--self-signed-cert path/to/cert` to the installer when setting up a host, or manually by editing run-host.cmd, run-host.sh and/or clusteriohost.service by adding the following line next to the one setting `NODE_OPTIONS`: 
+Additionally you will need copy the `cert.pem` file to all of the computers you want to set up as hosts and configure node to trust it by setting the `NODE_EXTRA_CA_CERTS` environment variable to a path to this file before the host is started. You can do this by passing `--self-signed-cert path/to/cert` to the installer when setting up a host, which also makes the installer trust the certificate when it asks the controller to approve the host, or manually by editing run-host.cmd, run-host.sh and/or clusteriohost.service by adding the following line next to the one setting `NODE_OPTIONS`:
 
 **For run-host.cmd**
 

docs/writing-plugins.md

@@ -16,6 +16,7 @@ Each entrypoint of a plugin exports a function that attaches handlers to hooks c
 - [Defining Link Messages](#defining-link-messages)
 - [Sending Link Messages](#sending-link-messages)
     - [Handling connection events](#handling-connection-events)
+- [Web Pages for a Single Resource](#web-pages-for-a-single-resource)
 - [Collecting Statistics](#collecting-statistics)
 - [Adding Custom Commands to clusterioctl](#adding-custom-commands-to-clusterioctl)
 
@@ -842,6 +843,34 @@ This means that if your plugin sends a lot of events or requests, they can end u
 To avoid this you should be throtteling and/or stopping your requests/events after `drop` has been raised, and continue back as normal when `resume` is raised.
 
 
+## Web Pages for a Single Resource
+
+A web page showing one resource from a subscription has more to handle than the resource itself.
+It can still be loading, it might not exist, it can be deleted while the page is open, and the subscription is rejected if the user lacks the permission for it.
+`useResourceState` and `ResourceState` from `@clusterio/web_ui` show these the same way as the pages for hosts, instances and the other built in resources.
+
+```tsx
+function FrobberPage() {
+    const id = Number(useParams().id);
+    const [frobbers, synced, error] = useSyncExternalStore(subscribe, () => frobberSubscriber.getSnapshot());
+    const frobber = frobbers.get(id);
+    const state = useResourceState(frobber, synced, error, id);
+
+    const nav = [{ name: "Frobbers", path: "/frobbers" }, { name: frobber?.name ?? String(id) }];
+    if (!frobber) {
+        return <ResourceState state={state} type="Frobber" id={id} nav={nav} listPath="/frobbers" error={error} />;
+    }
+
+    return <PageLayout nav={nav}>{frobber.name}</PageLayout>;
+}
+```
+
+`useResourceState` returns `"ok"` when there is a value and otherwise `"loading"`, `"missing"`, `"deleted"` or `"error"`.
+It tells deleted from missing by remembering if it has seen a value for that id.
+`ResourceState` renders the whole page for the states without a value, with a link back to the page at `listPath`.
+The `useHost`, `useInstance`, `useUser`, `useRole` and `useModPack` hooks return `[value, synced, error]` so they can be used the same way.
+
+
 ## Collecting Statistics
 
 Clusterio comes with its own Prometheus client implementation, one part due to Not Invented Here and another part due to collectors in prom-client being difficult to get to work nicely with collecting data from plugins optionally loaded at runtime on different computers.