Compare commits

...
Author SHA1 Message Date
Danielv123 95d23e9615 Create the character at spawn when it can't be made in place
create_character needs a free spot within 200 tiles and does not
generate chunks to find one, so it failed for a player teleported into
a chunk that does not exist yet, and where there is only water or void.
The download then failed and left the player as a spectator.

Resolves #1187
2026-11-10 21:48:20 +01:00
Bastiaan bd9c4a5fea Merge pull request #1188 from bbassie/fix/research-queue-message
Print only what changed in the research queue chat line
2026-11-07 14:51:42 +00:00
Cooldude2606andClaude Opus 5.5 7cf9cf8d3e Unpatch saves started with save patching disabled (#1184)
* Unpatch saves started with save patching disabled

Turning factorio.enable_save_patching off did nothing to a save that
was patched before. Its control.lua still loaded every module, so the
clusterio_api interface, player_sync's join handling and the other
module event handlers kept running with no plugin on the host side.

When patching is disabled the instance now restores the original
control.lua that patching kept as scenario.lua and removes the module
files, locale files included, as listed in clusterio.json. The patch
info stays with an empty module list so the patch number keeps
counting. Otherwise patching the save again would reuse the number
stored in the game and on_server_startup would not be raised. Data
the modules put in storage is left alone.

PatchInfo.fromSave dropped the version read from the save, so the
check against newer patch versions never fired. It now keeps it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Remove scenario.lua when unpatching a save

Unpatching moves scenario.lua back to control.lua instead of copying it, so the save only keeps clusterio.json for the patch number. Patching such a save again copies control.lua back to scenario.lua.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-05 20:58:13 +00:00
BastiaanandClaude Opus 5.5 6d9a32d28b Show localised names of mod settings in the web UI (#1181)
The mod pack page only looked up mod-setting-name.<name>, so settings
with a localised_name showed the internal name, and rich text tags in
the locale showed up as raw text.

Resolve localised_name and localised_description from the exported
settings prototypes against the exported locale, falling back to the
mod-setting-name/mod-setting-description keys and then the internal
name. The resolver handles parameters, the "" and "?" keys and
__ITEM__/__ENTITY__/__TILE__/__FLUID__ references. Rich text
formatting tags are removed and prototype icons are replaced with
their name.

Fixes #768

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 18:02:37 +00:00
bbassieandClaude Opus 5.5 513d964a8b Remove scenario.lua when unpatching a save
Unpatching moves scenario.lua back to control.lua instead of copying it, so the save only keeps clusterio.json for the patch number. Patching such a save again copies control.lua back to scenario.lua.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 17:21:19 +00:00
BastiaanandClaude Opus 5.5 c6867de13d Keep users instance filter and icon search in the URL (#1185)
The instance picker on the users page and the search box on the icon
reference page held their value in component state, so leaving the page
and coming back with the browser history reset them. Both now go through
useTableQueryState like the table filters do, as user.instance and
icon.search.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 15:22:02 +00:00
BastiaanandClaude Opus 5.5 b0fb43d678 Merge recipe notifications against the restored force (#1182)
The instance works out the recipe notification delta against the cleared
list the player had when the download started, but the save applied it
after the download restored player.force. A player whose force changed
kept notifications they had cleared elsewhere, and recipes only the old
force had dropped out of the failed list and were lost on the next
upload.

The download record now keeps the cleared list it sent. The save rebuilds
the stored list from it and the delta, then merges that against the
force the player has once the data is applied, as it did before #997.
Nothing extra goes over RCON.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 01:25:32 +00:00
Bastiaan 4a2c164188 Merge pull request #1183 from bbassie/fix/save-patching-plugin-default
Default load_plugin to off when the instance lacks required features
2026-11-04 01:19:10 +00:00
Cooldude2606andClaude Opus 5.5 308cca9b7d Add HostManager for host records and host tokens (#1180)
* Add HostManager for host records and host tokens

controller.hosts is now a HostManager wrapping the hosts datastore, the
same way controller.users wraps the users datastore. It owns creating
hosts (moved from HostPairingManager), revoking them (moved from
ControlConnection) and signing and verifying host tokens (moved from
Controller, generateHostToken is renamed signHostToken to match
signUserToken).

UserManager gains verifyUserToken, replacing the copies of the user
token check in WsServer and routes. The WsServer copy accepted a token
without iat when the user had no tokenValidAfter; the shared check
always requires iat, which jsonwebtoken adds by default.

Follows Cooldude2606's comment on #1161.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Return the host record from verifyHostToken

Matches verifyUserToken returning the user. HostConnection._saveInfo is
renamed saveRecord to match User.saveRecord.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 00:51:36 +00:00
bbassieandClaude Opus 5.5 4c6aed3cff Return the host record from verifyHostToken
Matches verifyUserToken returning the user. HostConnection._saveInfo is
renamed saveRecord to match User.saveRecord.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 00:37:37 +00:00
bbassieandClaude Opus 5.5 e6554e09c7 Add HostManager for host records and host tokens
controller.hosts is now a HostManager wrapping the hosts datastore, the
same way controller.users wraps the users datastore. It owns creating
hosts (moved from HostPairingManager), revoking them (moved from
ControlConnection) and signing and verifying host tokens (moved from
Controller, generateHostToken is renamed signHostToken to match
signUserToken).

UserManager gains verifyUserToken, replacing the copies of the user
token check in WsServer and routes. The WsServer copy accepted a token
without iat when the user had no tokenValidAfter; the shared check
always requires iat, which jsonwebtoken adds by default.

Follows Cooldude2606's comment on #1161.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 00:37:37 +00:00
BastiaanandClaude Opus 5.5 5d1e98dafd Hide unknown mod pack settings and allow removing them (#1179)
Settings stored in a mod pack that no setting prototype in the export
data defines are now listed in a collapsed "Unknown settings" panel
under each scope instead of under an "Unknown Fields" heading. Each one
has a remove link, and a revert link once removed, matching what
`clusterioctl mod-pack edit --remove-setting` already allows. They are
never removed automatically.

Without export data nothing is called unknown and stored settings are
listed as before, without a mod heading.

Fixes #773

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 00:18:32 +00:00
bbassieandClaude Opus 5.5 ca3dbab49d Unpatch saves started with save patching disabled
Turning factorio.enable_save_patching off did nothing to a save that
was patched before. Its control.lua still loaded every module, so the
clusterio_api interface, player_sync's join handling and the other
module event handlers kept running with no plugin on the host side.

When patching is disabled the instance now restores the original
control.lua that patching kept as scenario.lua and removes the module
files, locale files included, as listed in clusterio.json. The patch
info stays with an empty module list so the patch number keeps
counting. Otherwise patching the save again would reuse the number
stored in the game and on_server_startup would not be raised. Data
the modules put in storage is left alone.

PatchInfo.fromSave dropped the version read from the save, so the
check against newer patch versions never fired. It now keeps it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 00:16:52 +00:00
BastiaanandClaude Opus 5.5 d250394580 Show the mod pack an instance uses in the web UI (#1177)
InstanceDetails now carries factorio.mod_pack_id as modPackId, so the
instance list and pages can show it without fetching each config. The
web UI resolves an unset value to the controller default and tags it
"Default". The name links to the mod pack page.

Closes #852

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-04 00:07:26 +00:00
Cooldude2606andClaude Opus 5.5 6eaf48aefe Follow teleports of a syncing player in player_sync (#1168)
* Keep a syncing spectator where its body stood

A player waiting for player_sync data is a spectator who could walk away,
so restore_position teleported them back at the end. That also undid
teleports by other mods made during the sync, such as universal_edges
placing an arriving player.

The spectator is now pulled back on every tile change, and a jump of more
than 4 tiles or to another surface counts as another mod's teleport and
moves where the body comes back. A failed acquire that takes the body
keeps its place too.

Closes #908

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Rename body place to controller location

Review asked for location and controller over place and body.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Let a syncing spectator look around

Review asked for the spectator to move freely while it waits. Only a jump
of more than 16 tiles from where it was last seen, or to another surface,
counts as a teleport and moves the restore location. The body place names
from #1156 become restore_location, keep_restore_location and
controller_taken.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 23:43:42 +00:00
bbassieandClaude Opus 5.5 1c804b05b7 Let a syncing spectator look around
Review asked for the spectator to move freely while it waits. Only a jump
of more than 16 tiles from where it was last seen, or to another surface,
counts as a teleport and moves the restore location. The body place names
from #1156 become restore_location, keep_restore_location and
controller_taken.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 07:36:20 +00:00
bbassieandClaude Opus 5.5 2fcf3f10c7 Rename body place to controller location
Review asked for location and controller over place and body.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 07:34:06 +00:00
bbassieandClaude Opus 5.5 babc210a38 Keep a syncing spectator where its body stood
A player waiting for player_sync data is a spectator who could walk away,
so restore_position teleported them back at the end. That also undid
teleports by other mods made during the sync, such as universal_edges
placing an arriving player.

The spectator is now pulled back on every tile change, and a jump of more
than 4 tiles or to another surface counts as another mod's teleport and
moves where the body comes back. A failed acquire that takes the body
keeps its place too.

Closes #908

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 07:34:06 +00:00
Bastiaan cfea463c7b Merge pull request #1156 from bbassie/fix/player-sync-spectator
Download again after a stalled player_sync download
2026-11-02 22:59:03 +00:00
Cooldude2606andClaude Opus 5.5 0a23af61f9 Add instance templates for instance config inheritance (#1142)
* Add base instances for instance config inheritance

Instances can be marked as a base with instance.is_base and other
instances inherit config values from it through instance.base_id.
The controller copies base values into inheriting configs and keeps
track of inherited fields in the hidden instance.inherited_fields, so
hosts receive resolved configs and need no changes. Setting a field
stops inheriting it, `instance config inherit` undoes that.

Bases cannot be assigned to a host, are hidden from `instance list`
unless --bases is given, and cannot be deleted or unmarked while
other instances inherit from them. Chains of bases are allowed and
cycles are rejected.

Part of #200

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Rename bases to templates and add per property inheritance

Review changes for #1142:
- base becomes template everywhere (is_template, template_id,
  --template, --is-template, --templates, InstanceDetails.isTemplate).
- Object fields inherit per property. inherited_fields lists the
  overridden properties for object fields and the template object is
  merged with them.
- Linking a template also inherits fields that hold the default value.
  The server name stays per instance.
- Ports are never inherited.
- InstanceDetails carries templateId.

Web UI: inherited values are locked until unlocked, overridden values
can be inherited again, templates are listed separately, the create
modal can create templates and instances from templates, and the
instance page has "Create template" which can link the source.

Part of #200

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Give templates their own table and keep rcon_password per instance

The templates table shows name, inheritor count and a create instance
action instead of the instance status and start/stop columns.
factorio.rcon_password joins the ports as never inherited.

Part of #200

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Use link and disconnect icons for template inheritance

Overridden values show a disconnect icon. Clicking it stages a link,
shown with a link icon, which is applied together with the other
changes instead of right away.

Part of #200

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Show the template value while a link is staged

Part of #200

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Address review of template inheritance

- Unlinking a field enables Apply on its own and sends the current value.
- The link button no longer toggles the checkbox of its label.
- Link and disconnect icons for all four states, pending ones highlighted.
- instance.is_template is hidden.
- ctl create takes --as-template.
- Clearer names for the InstanceManager inheritance helpers.

Part of #200

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 22:48:55 +00:00
bbassieandClaude Opus 5.5 f14bfc3222 Address review of template inheritance
- Unlinking a field enables Apply on its own and sends the current value.
- The link button no longer toggles the checkbox of its label.
- Link and disconnect icons for all four states, pending ones highlighted.
- instance.is_template is hidden.
- ctl create takes --as-template.
- Clearer names for the InstanceManager inheritance helpers.

Part of #200

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 11:57:01 +00:00
bbassieandClaude Opus 5.5 943619a657 Show the template value while a link is staged
Part of #200

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 11:54:37 +00:00
bbassieandClaude Opus 5.5 cd04995b65 Use link and disconnect icons for template inheritance
Overridden values show a disconnect icon. Clicking it stages a link,
shown with a link icon, which is applied together with the other
changes instead of right away.

Part of #200

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 11:54:37 +00:00
bbassieandClaude Opus 5.5 b584991f5c Give templates their own table and keep rcon_password per instance
The templates table shows name, inheritor count and a create instance
action instead of the instance status and start/stop columns.
factorio.rcon_password joins the ports as never inherited.

Part of #200

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 11:54:37 +00:00
bbassieandClaude Opus 5.5 b5c372f109 Rename bases to templates and add per property inheritance
Review changes for #1142:
- base becomes template everywhere (is_template, template_id,
  --template, --is-template, --templates, InstanceDetails.isTemplate).
- Object fields inherit per property. inherited_fields lists the
  overridden properties for object fields and the template object is
  merged with them.
- Linking a template also inherits fields that hold the default value.
  The server name stays per instance.
- Ports are never inherited.
- InstanceDetails carries templateId.

Web UI: inherited values are locked until unlocked, overridden values
can be inherited again, templates are listed separately, the create
modal can create templates and instances from templates, and the
instance page has "Create template" which can link the source.

Part of #200

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 11:54:37 +00:00
bbassieandClaude Opus 5.5 01cf789f5d Add base instances for instance config inheritance
Instances can be marked as a base with instance.is_base and other
instances inherit config values from it through instance.base_id.
The controller copies base values into inheriting configs and keeps
track of inherited fields in the hidden instance.inherited_fields, so
hosts receive resolved configs and need no changes. Setting a field
stops inheriting it, `instance config inherit` undoes that.

Bases cannot be assigned to a host, are hidden from `instance list`
unless --bases is given, and cannot be deleted or unmarked while
other instances inherit from them. Chains of bases are allowed and
cycles are rejected.

Part of #200

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-02 11:54:21 +00:00
024d7b427a Rename safeOutputFile to atomicOverwriteFile (#1167)
"safe" suggested the data being written survives a crash, it only
guarantees the old file is either kept or fully replaced. The new name
says what it does. No alias, plugin API breaks are allowed in minor
versions and the 2.1 migration guide lists the rename.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Cooldude2606 <25043174+Cooldude2606@users.noreply.github.com>
2026-11-01 23:14:31 +00:00
Bastiaan 1b87497e15 Merge pull request #1154 from bbassie/fix/deleted-exports
Handle deleted export files
2026-11-01 23:10:27 +00:00
d7d1859aa0 Report missing base game icons when exporting from headless Factorio (#1153)
* Report missing base game icons when exporting from headless Factorio

The headless build has no graphics, so export-data found none of the
base game's icon files. Plain icons were skipped silently and layered
icons were drawn blank and counted as exported.

Skip a layered icon when any of its layers is missing, and log one
warning with the number of base game icon files that were not found.
Add host.factorio_graphics_directory to read builtin mod graphics from
a full install when the instance runs the headless build.

Fixes #1068

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Export without the graphics directory when it has no matching install

If host.factorio_graphics_directory had no install matching the major and
minor version of the instance, or did not exist, findVersion threw and the
whole export failed. Without the setting the same instance exports everything
except the base game icons, so it now logs a warning that names the setting
and exports without it.

* Find a full install in host.factorio_directory instead of a setting

Per review, drop host.factorio_graphics_directory. findVersion takes a
headless option that filters installs by build, telling them apart by
data/base/graphics/icons. When the instance runs the headless build the
export looks for a full install of the same major.minor version in the
host's factorio directory. Server startup is unchanged.

If none is found the export still runs, and the reason is included in
the missing icons warning.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Address review on findVersion and the export graphics lookup

findVersion takes headless as a plain optional argument, the helper is
checkInstallType, the warning is checkMissingBuiltinIcons and the full
install lookup is inlined into exportData.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Danielv123 <danielv@danielv.no>
2026-11-01 23:09:55 +00:00
Daniel 1311df5759 Keep recipe notifications when a recipe list is empty (#1170)
Factorio encodes an empty Lua table as {}, so a player with no cleared
recipes sent {} with the download request. new Set({}) threw, the
instance dropped the recipe notifications from the download, and the
next upload replaced the stored list on the controller. An empty stored
list was dropped the same way. Decode both lists treating {} as empty.
2026-11-01 23:06:16 +00:00
Daniel 03ea6accc1 Show defaults for mod settings added while the page is open (#1172)
The mod pack view gave the settings form its values through the Form's
initialValues, which rc-field-form only applies when the form is
created. Settings added by a data export while the page was open were
shown empty, and color settings black, until the page was reloaded.
Each setting field now gets its saved value as its own initialValue,
which is applied when the field is created and when the form is reset.
2026-11-01 23:06:02 +00:00
bbassieandClaude Opus 5.5 f8aafe9265 Address review on findVersion and the export graphics lookup
findVersion takes headless as a plain optional argument, the helper is
checkInstallType, the warning is checkMissingBuiltinIcons and the full
install lookup is inlined into exportData.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 22:44:43 +00:00
bbassieandClaude Opus 5.5 37d74ca2a6 Find a full install in host.factorio_directory instead of a setting
Per review, drop host.factorio_graphics_directory. findVersion takes a
headless option that filters installs by build, telling them apart by
data/base/graphics/icons. When the instance runs the headless build the
export looks for a full install of the same major.minor version in the
host's factorio directory. Server startup is unchanged.

If none is found the export still runs, and the reason is included in
the missing icons warning.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 20:53:05 +00:00
Danielv123 46c6153201 Export without the graphics directory when it has no matching install
If host.factorio_graphics_directory had no install matching the major and
minor version of the instance, or did not exist, findVersion threw and the
whole export failed. Without the setting the same instance exports everything
except the base game icons, so it now logs a warning that names the setting
and exports without it.
2026-11-01 18:33:10 +01:00
Danielv123 46fd7fcad4 Merge remote-tracking branch 'origin/master' into fix/headless-export-icons 2026-11-01 18:32:20 +01:00
Daniel e251bf0072 Merge pull request #1174 from clusterio/fix/dev-plugin-shared-modules
Serve the web interface from source with --dev-plugin
2026-11-01 18:25:08 +01:00
Daniel 2503e567f5 Merge pull request #1175 from clusterio/test/controller-start-existing-database
Test starting the controller with an existing database
2026-11-01 18:25:03 +01:00
Daniel 88855251e9 Merge pull request #1176 from clusterio/fix/export-skip-broken-icons
Skip icons that fail to export instead of failing the export
2026-11-01 18:24:58 +01:00
Daniel a98b4c84ff Merge pull request #1164 from bbassie/chore/create-prompts
Replace inquirer 8 in create with the @inquirer prompt packages
2026-11-01 18:03:38 +01:00
bbassieandClaude Opus 5.5 d2a4e9a4a1 Replace inquirer 8 with the @inquirer prompt packages in create
inquirer 8 pulls in 50 packages and 7.4 MB (rxjs, old chalk) for every
user running npm init @clusterio. The standalone select, input, confirm
and checkbox packages from the same project cover all prompts used and
cut the create install from 70 packages / 8.1 MB to 31 / 0.94 MB.

The Windows rl.close workaround now patches readline.Interface instead
of inquirer's Prompt class, since the new prompts create their own
readline interface. Ctrl+C at a prompt exits with code 130 instead of
0, and closed stdin no longer crashes with ERR_USE_AFTER_CLOSE.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-01 17:53:36 +01:00
Daniel 3abcba5991 Merge pull request #1166 from clusterio/implement-1165-separate-pr
Handle missing, deleted and failed resources the same way on web UI view pages
2026-11-01 17:40:50 +01:00
Danielv123 2e1710d3a9 Show the same page for missing, deleted and unloadable resources
Every view page handled a resource that can't be shown its own way, and
users without the subscribe permission were left on a spinner forever.

Adds useResourceState and a ResourceState component, used by the host,
instance, mod pack, icon reference, user, role and plugin pages and
exported for plugin pages. The single item hooks now also return the
subscription error.

Resolves #1165
2026-11-01 17:40:22 +01:00
Daniel 928e03ce02 Merge pull request #1161 from clusterio/host-pairing
Replace host tokens with host pairing
2026-11-01 17:40:22 +01:00
Daniel 88aeb357cd Merge pull request #1161 from clusterio/host-pairing
Replace host tokens with host pairing
2026-11-01 17:40:19 +01:00
Danielv123 8908079022 Merge remote-tracking branch 'origin/master' into host-pairing 2026-11-01 13:16:11 +01:00
Danielv123 495831da29 Stop waiting for approval when the terminal closes
SIGHUP while waiting for approval released the config lock but left the
host polling, as only SIGINT and SIGTERM stopped the wait.
2026-11-01 12:07:34 +01:00
Danielv123 7ada2e8975 Skip icons that fail to export instead of failing the export
An icon that could not be read or decoded, such as a truncated PNG, a
JPEG or a layered icon with no layers, threw out of exportIcons and
failed the whole data export. Log a warning naming the prototype and
export it without an icon instead. Also warn when a mod's zip is
missing while exporting the locale, which was skipped silently.
2026-10-31 23:26:39 +01:00
Danielv123 3c51d3b1ea Test starting the controller with an existing database
No test started a controller with instances, hosts, mod packs and users
already on disk, which is how #510 got through with all tests passing.
2026-10-31 23:24:40 +01:00
Danielv123 119c81c770 Serve the web interface from source with --dev-plugin
With --dev-plugin and no --dev the page kept the production import map,
while the development server gave the plugin its own copies of the
shared modules. The plugin then failed to load with "Attempt to import
duplicate copy of @clusterio/lib". Use the development server's web
interface and import map whenever it runs, so the plugin and the web
interface share the same modules.
2026-10-31 23:23:32 +01:00
Danielv123 49a0b27d29 Address review of host pairing
Revoking a host now unassigns its instances instead of being refused
while it has any, and removes its system metrics.

Approving and revoking no longer force a save. Hosts are saved by
autosave and on exit like everything else, which also removes the
approving state from pairing requests.

The approval code and link are logged as warnings so they show up in
the systemd service, which only prints warnings and above. The pairing
limits are now the controller.host_pairing_timeout and
controller.host_pairing_max_pending config values, 0 removes the limit.

Subscribers take an optional permitted function, used for the pairing
requests instead of overrides in a subclass.
2026-10-31 16:31:45 +01:00
Danielv123 e5701a8dae Approve standalone hosts like any other host
A standalone install no longer adds its host to the controller database
by signing a token and writing hosts.json itself. The host asks to join
when it first starts and is approved once with the link it prints.
2026-10-30 23:15:22 +01:00
Danielv123 a4894aae9d Simplify approving hosts
The list of waiting hosts opens the approval dialog instead of having its
own name input and buttons, so hosts are approved in one place. Removes
the limit on requests per minute, the limits on pending requests already
cap what an address can do.

Also updates text that still said the installer makes the requests.
2026-10-30 22:59:32 +01:00
Daniel c1826aea98 Merge pull request #1152 from bbassie/feature/live-role-updates
Live update roles on the user and role view pages
2026-10-30 22:22:23 +01:00
Danielv123 95422e6d21 Merge remote-tracking branch 'origin/master' into host-pairing
# Conflicts:
#	packages/host/host.ts
2026-10-30 21:50:38 +01:00
Danielv123 1ed46dff9c Fix issues found in review of host pairing
- Saves of a datastore run one at a time, and changes made while saving
  are saved by the next save. Overlapping saves from approving and
  revoking hosts could write an older snapshot after a newer one.
- Map subscribers fetch everything again when a new session starts, as
  the controller only sends values that exist. A host revoked while a
  browser was disconnected stayed in its list.
- The host token check accepts the same pre-rename tokens as the
  WebSocket, so upgrading a host with such a token does not make it ask
  to join again.
2026-10-30 21:49:53 +01:00
Cooldude2606andClaude Opus 5.5 9b654b1ad8 Start the controller and host CLIs faster (#1159)
* Answer the disconnect prepare in the close old connections test

The test's first connector had no Link, so the controller's disconnect
prepare went unanswered and it waited the full 10 s timeout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Start the controller and host CLIs faster

Like ctl, the controller and host now compile schema validators on first
use instead of at import, which cost about 120 ms per invocation. The
Controller and Host classes are only imported by the commands that
start them, so config, bootstrap and the other commands no longer load
express, ws and the rest of the servers. clusteriocontroller config set
goes from 413 ms to 231 ms and clusteriohost config set from 412 ms to
225 ms.

The integration tests run the bins, like npx does, instead of the
package directories.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 20:43:33 +00:00
Bastiaan e7c21fb907 Merge pull request #1150 from bbassie/fix/node-engine-range
Require Node.js 22.18 or 24.2 for import.meta.main
2026-10-30 20:41:37 +00:00
Danielv123 2cf7c66187 Merge remote-tracking branch 'origin/master' into host-pairing
# Conflicts:
#	test/integration/clusterio.js
2026-10-30 21:41:06 +01:00
Cooldude2606 cc8888fc69 Merge pull request #1160 from bbassie/ci/split-tests
Split each CI test job into two halves
2026-10-30 20:40:52 +00:00
bbassieandClaude Opus 5.5 b9d7033724 Test old Node.js versions only with the latest Factorio
Splitting the test jobs doubled them to 14, over GitHub's concurrent
job limit together with the other jobs. Every Factorio version now runs
on the latest Node.js only, and the other Node.js versions only with
the latest Factorio, which is 10 test jobs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 20:31:57 +00:00
bbassieandClaude Opus 5.5 75cdfbad8d Split the test jobs in two halves
The clusterioctl, plugin system and instance integration tests take
about half of the test time, so they run on one runner and the rest of
the tests on another, each half with its own controller and host.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 20:31:32 +00:00
Cooldude2606andClaude Opus 5.5 3707fc35fc Drop unused dependencies from web_ui and tests (#1163)
* Take FieldData from antd instead of rc-field-form

web_ui depended on rc-field-form 1.x for a single type import, while
antd 5 uses rc-field-form 2.x. Derive the type from antd's FormProps
and drop the dependency.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Move typescript to web_ui devDependencies

web_ui only needs typescript to build its own types. The runtime
dependency is a leftover from webpack's ts-loader, Vite doesn't use it,
and it pulls in a 30 MB native compiler for anyone installing web_ui.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Use native FormData in controller route tests

Node's FormData and Blob do what the form-data package did here, and
fetch sets the multipart Content-Type header itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Remove leftover typedoc.json files

TypeDoc was dropped in #1147, but these configs still ship in the
published packages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 20:29:12 +00:00
BastiaanandClaude Opus 5.5 0f7c8be060 Use only the jimp modules the host export needs (#1162)
jimp pulls in every image format and plugin, about 60 packages and
26 MB that only export.ts uses. Build the Jimp class from @jimp/core
with the PNG format and the crop and resize plugins instead. The
generated spritesheet is byte-identical to the one from jimp.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 20:24:07 +00:00
Danielv123 13c75b9e35 Link to an approval dialog from the waiting host
A host waiting for approval logs a link to the Hosts page with the code
in it. Opening it shows a dialog with the code, the address of the host,
a name prefilled with the requested one and a large Approve button. A
code that is no longer waiting shows a message saying so.
2026-10-30 21:23:03 +01:00
Danielv123 4a49a198f0 Pair hosts when they start instead of in the installer
The installer only writes the controller URL and host name. Running the
host requests to join the cluster if it has not been approved yet, or if
the controller no longer accepts its token, and waits until an admin
approves it. Once approved it saves its id and token and starts as
normal. A rejected host exits with code 8.

clusteriohost run --user-token approves the host right away with the
token of a user allowed to approve hosts. The token is not saved.

This moves the pairing client into the host package, where it uses the
same certificate setup as the host's connection to the controller, and
removes --no-pair and the rerun handling from the installer.
2026-10-30 21:13:14 +01:00
Cooldude2606 53a4fce934 Merge pull request #1151 from bbassie/fix/stop-while-starting
Fix stopping an instance while it is starting
2026-10-30 20:04:22 +00:00
Bastiaan ed2057e4a2 Merge pull request #1149 from bbassie/feature/compat-ignore-control
Ignore control in the version compat message check
2026-10-30 19:57:48 +00:00
bbassieandClaude Opus 5.5 8738c74d7f Remove leftover typedoc.json files
TypeDoc was dropped in #1147, but these configs still ship in the
published packages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 18:44:54 +00:00
bbassieandClaude Opus 5.5 c9db2fb29e Use native FormData in controller route tests
Node's FormData and Blob do what the form-data package did here, and
fetch sets the multipart Content-Type header itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 18:44:54 +00:00
bbassieandClaude Opus 5.5 41e3f50dbe Move typescript to web_ui devDependencies
web_ui only needs typescript to build its own types. The runtime
dependency is a leftover from webpack's ts-loader, Vite doesn't use it,
and it pulls in a 30 MB native compiler for anyone installing web_ui.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 18:44:54 +00:00
bbassieandClaude Opus 5.5 5df375d490 Take FieldData from antd instead of rc-field-form
web_ui depended on rc-field-form 1.x for a single type import, while
antd 5 uses rc-field-form 2.x. Derive the type from antd's FormProps
and drop the dependency.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 18:44:54 +00:00
bbassieandClaude Opus 5.5 0312a3b099 Start the controller and host CLIs faster
Like ctl, the controller and host now compile schema validators on first
use instead of at import, which cost about 120 ms per invocation. The
Controller and Host classes are only imported by the commands that
start them, so config, bootstrap and the other commands no longer load
express, ws and the rest of the servers. clusteriocontroller config set
goes from 413 ms to 231 ms and clusteriohost config set from 412 ms to
225 ms.

The integration tests run the bins, like npx does, instead of the
package directories.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 17:29:42 +00:00
bbassieandClaude Opus 5.5 a174c355d2 Answer the disconnect prepare in the close old connections test
The test's first connector had no Link, so the controller's disconnect
prepare went unanswered and it waited the full 10 s timeout.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 17:19:36 +00:00
Daniel 8a3aa56c11 Merge pull request #1157 from bbassie/refactor/plugin-state
Replace the PluginDetails booleans with a single state
2026-10-30 17:52:51 +01:00
Daniel c13b1fa78f Merge pull request #1155 from bbassie/fix/research-sync-session-loss
Resync research_sync after a new controller session
2026-10-30 17:50:58 +01:00
Daniel 31e132d4fd Merge pull request #1158 from bbassie/ci/setup-speed
Speed up CI setup and the coverage upload
2026-10-30 17:44:48 +01:00
Danielv123 37e79d9c41 Remove tests that pin implementation details
Drops tests that restate message declarations, check that removed
commands are gone, test internal helpers or repeat the same behaviour
at another layer.
2026-10-30 10:11:14 +01:00
Danielv123 87d4f5c64e Replace host tokens with host pairing
Hosts join the cluster by running the installer, which shows a code and
waits until the request is approved under Hosts > Add host in the web
interface or with clusterioctl host approve. An admin can pass their
user token to the installer to approve the host right away. The token
is only used for that and is not saved.

The controller only accepts hosts that have a record, and only
approving a host creates one. Revoking a host deletes its record, so
its token can no longer be used to reconnect. Hosts that have connected
before keep working.

Removes host generate-token, revoke-token and create-config from
clusterioctl, bootstrap generate-host-token, the host token messages and
the core.host.generate_token, revoke_token and create_config
permissions, which are also stripped from stored roles. Adds
clusterioctl host pending, approve, reject and revoke, and the
core.host.approve and core.host.revoke permissions.

Standalone installs add the host to the controller database directly,
since the controller is not running during the install.
2026-10-30 06:11:32 +01:00
bbassieandClaude Opus 5.5 b989815829 Speed up CI setup and coverage upload
pnpm/action-setup installs a bootstrap pnpm and then switches to the
packageManager version, about 10 s per job; npm installs it in 3 s.
setup-node took 4 s probing package managers for a cache it never
used. The pnpm store cache took 4 s to restore and 5 s to save, more
than downloading the packages costs, and the build cache no longer pays
off with TypeScript 7: pnpm install is 13-23 s without either cache
against 17-31 s with them.

The Codecov action searched the workspace and uploaded the raw V8
coverage in coverage/tmp next to lcov.info, 72 MB taking 11 s.

The version compatibility job ran tsc --build again after pnpm install
had already built everything.

Cancel superseded runs of branches and pull requests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 00:26:28 +00:00
bbassieandClaude Opus 5.5 814a4a201a Replace the PluginDetails booleans with a single state
loaded, enabled, pendingInstall, removed, noEntrypoint and nameTaken
allowed far more combinations than can happen, and every consumer had
to know which took priority. PluginDetails now has one state, with
loaded, loadsOnRestart and removed as getters over it.

2.0.3 web UIs and ctl send PluginListRequest straight to hosts and
require loaded and enabled, so those are still sent next to the
optional state. Plugins listed by 2.0.3 hosts get their state from
loaded and enabled.

The web UI planner gets the same treatment: MachinePlugin has one
state in place of disabled, noEntrypoint, dependencyTaken and
nameTaken. ctl host plugin list shows the state.

Closes #1134

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 00:05:23 +00:00
bbassieandClaude Opus 5.5 5589609b1b Resync research after a new controller session
While the host has no controller session, events between the instance
and the controller are dropped in both directions, and research_sync
only synced technologies on instance start. A technology finished during
the outage stayed unresearched on the controller and on every other
instance until the instance restarted.

On a new session the instance now runs the same technology sync as on
start, before taking the cluster queue. Contributions made while the
controller is unreachable are summed per technology and sent on connect
or resume, since progress from several instances can't be rebuilt from a
state sync that keeps the highest progress. A finish made while the
connection was dropped triggers the sync on resume too.

sync_technologies now runs while the game is running, so it skips
writing progress to a technology that finished after the dump.

Fixes #1123

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 00:04:15 +00:00
bbassieandClaude Opus 5.5 5238a0fbb3 Report missing base game icons when exporting from headless Factorio
The headless build has no graphics, so export-data found none of the
base game's icon files. Plain icons were skipped silently and layered
icons were drawn blank and counted as exported.

Skip a layered icon when any of its layers is missing, and log one
warning with the number of base game icon files that were not found.
Add host.factorio_graphics_directory to read builtin mod graphics from
a full install when the instance runs the headless build.

Fixes #1068

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-30 00:02:52 +00:00
bbassieandClaude Opus 5.5 78f1e8a993 Live update roles on the user and role view pages
The user page set the role selector once and never updated it. The role
page reset its whole form on every role update, wiping unsaved edits.

Both pages now follow the live data and keep local edits on top. The role
page stores only the edited fields and drops them once they match the role
or when the viewer loses core.role.update. The user page syncs the
selector while it is not dirty and drops deleted roles from a dirty
selection. A role deleted while open now says so instead of "not found".

Fixes #775, fixes #776

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 22:52:59 +00:00
bbassieandClaude Opus 5.5 666dd48412 Show the instance control button as loading while starting or stopping
The button offered Stop while the instance was starting whenever the
start came from somewhere else, such as Load save, another tab or ctl.
It now shows the action in progress as loading until the status
settles. Kill stays available from the dropdown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 22:52:09 +00:00
bbassieandClaude Opus 5.5 eb1e2e2a5e Wait for a start in progress before stopping an instance
A stop request sent while the instance was starting ran alongside the
start. During save preparation it did nothing and the instance came up
running anyway. Once Factorio was launched it put the server into the
stopping state, the start then failed with "Expected state running but
state is stopping" and its cleanup killed the server mid shutdown.

Stop and restart requests now wait for any start, stop or save in
progress to finish first, then stop the instance if it is running.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 22:52:09 +00:00
Cooldude2606 fa2883d74e Merge pull request #1147 from bbassie/build/typescript-7
Drop TypeDoc and move to TypeScript 7
2026-10-29 17:55:01 +00:00
bbassieandClaude Opus 5.5 3079426712 List node in the types of the node tsconfigs
Since TypeScript 6 the types option defaults to an empty list, so
@types/node was only loaded because winston and undici-types reference
it. Name it explicitly. The emitted files do not change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 01:11:00 +00:00
bbassieandClaude Opus 5.5 7d421847f5 Drop the self-referencing constructor type on ModPortalGetAllResponse
TypeScript 7.1 reports TS5088 for it: the inferred type of the static
Response property references itself through the declared constructor.
The other inline Response classes do not declare it either.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 01:11:00 +00:00
bbassieandClaude Opus 5.5 885de50625 Build with TypeScript 7
TypeScript 7 is the native port of the compiler. The repository builds
with it unchanged: no new type errors, the emitted JavaScript differs
only in the order of react/jsx-runtime imports, and the declarations
only in union and property order, which TypeScript 5.9 consumers read
the same way. The plugin template moves to it too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 01:11:00 +00:00
bbassieandClaude Opus 5.5 c724f144e7 Drop TypeDoc
The generated API docs were never published or built in CI, and
typedoc's peer range stops at TypeScript 6.0. TSDoc comments stay.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 01:11:00 +00:00
Cooldude2606 96dddb1526 Merge pull request #1146 from bbassie/feature/compat-breaks
Allow known breaks in the version compatibility test
2026-10-29 00:51:37 +00:00
bbassieandClaude Opus 5.5 3c2ee3e372 Require an impact for each known compat break
Entries now explain why end users are not or only minimally affected,
and the docs say which version to key by and point to the breaking
changes decision.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 00:45:27 +00:00
bbassieandClaude Opus 5.5 9bd6effd40 Allow known breaks in the version compatibility test
The version-compat job fails every pull request once master breaks
compatibility with the latest release on purpose. List those breaks in
scripts/version-compat-breaks.json keyed by the published version they
break against, each with a pr number and reason. Listed messages warn
instead of failing, connect skips the cluster test, entries for other
versions are ignored, and a listed message that did not change fails.

Add the two src changes from #1129 against 2.0.3 so master passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 00:27:37 +00:00
Cooldude2606andClaude Opus 5.5 a82af49c05 Keep auto paused instances paused (#1143)
* Keep auto paused instances paused

Every RCON command makes an auto paused Factorio server run one tick, so
subspace_storage's 1 s inventory update and 5 s ping, plus a
statistics_exporter gather on every Prometheus scrape, kept a server with
nobody online running at about 1.3 ticks per second.

The instance now records the auto_pause server setting it was started
with and exposes autoPaused, true while that setting is on and no players
are online, along with an autoPauseChanged hook. subspace_storage skips
its timers while paused and sends a ping and inventory as soon as a player
joins. statistics_exporter gathers once after the game pauses and keeps
serving those values until it resumes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Check connection and status in subspace_storage timers

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 00:05:32 +00:00
bbassieandClaude Opus 5.5 1eea756ab3 Check connection and status in subspace_storage timers
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 23:58:13 +00:00
Cooldude2606 2fb5300ec5 Merge pull request #1140 from bbassie/fix/remove-web-plugin-column
Remove the web UI status column from the plugin pages
2026-10-28 23:54:46 +00:00
bbassieandClaude Opus 5.5 11e067a933 Keep auto paused instances paused
Every RCON command makes an auto paused Factorio server run one tick, so
subspace_storage's 1 s inventory update and 5 s ping, plus a
statistics_exporter gather on every Prometheus scrape, kept a server with
nobody online running at about 1.3 ticks per second.

The instance now records the auto_pause server setting it was started
with and exposes autoPaused, true while that setting is on and no players
are online, along with an autoPauseChanged hook. subspace_storage skips
its timers while paused and sends a ping and inventory as soon as a player
joins. statistics_exporter gathers once after the game pauses and keeps
serving those values until it resumes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 23:40:30 +00:00
bbassieandClaude Opus 5.5 9aa232e2cd Refer to the web build in the plugin load error
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 23:35:35 +00:00
bbassieandClaude Opus 5.5 82b7c16dba Remove the web UI status column from the plugin pages
Users don't need the web UI load state apart from the plugin's own state now
that the web UI reloads itself. A web load error now shows under the plugin's
versions and on its view page, only for enabled plugins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 23:35:35 +00:00
Cooldude2606 ebdbb1fe56 Merge pull request #1141 from bbassie/test/plugin-integration
Finish the plugin system integration tests
2026-10-28 23:35:32 +00:00
bbassieandClaude Opus 5.5 6f70ccbf85 Add a test plugin without a module for the host tests
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 23:29:50 +00:00
Cooldude2606 188de979b6 Merge pull request #1130 from bbassie/build/vite-plus
Draft: Replace webpack with Vite and load plugins as ES modules (vote)
2026-10-28 23:17:45 +00:00
bbassieandClaude Opus 5.5 9ab36151bd Finish the plugin system integration tests
Adds the last three checks from #342 to test_plugin: an InstanceEcho
request handled on a running instance, a Lua module with a remote
interface that the request calls over RCON, and log tagging checks for
the plugin field and host/instance ids on the controller, host and
instance log lines.

test_plugin now has a module, so the create_scenario and patch unit
tests that relied on it having none use global_chat instead.

Closes #342

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-28 22:01:27 +00:00
bbassieandClaude Opus 5.5 942773c66c Use the shared TypeBox version in the plugin template
The template asked for TypeBox ^0.30.6 while lib uses ^0.34, so a new
plugin failed tsc on the first message class. TypeBox is now provided
to plugin web code through the import map, so the template should be on
the version the web interface ships.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 23:01:12 +00:00
bbassieandClaude Opus 5.5 4d4c509722 Remove the PluginWebpackEnvInfo alias
Only core uses the type, so there is no need to keep the old name
around (Cooldude in #1130).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 23:01:12 +00:00
bbassieandClaude Opus 5.5 bf847b8c74 Drop build tool names from the controller and static_url docs
Controller.viteServer becomes devServer, matching the devMiddleware
name it replaced. The static_url limitation is described as one of the
web build rather than webpack.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 23:01:12 +00:00
bbassieandClaude Opus 5.5 5c92a44e65 Limit the development server to sources and dependencies
Vite serves any file under fs.allow through /@fs/, and the default
allow list is the workspace root. In a clusterio checkout that is also
the usual data directory, so with --dev any client that could reach the
controller could fetch config-controller.json, config-ctl.json and the
database without logging in. webpack-dev-middleware only served the
compiled bundles.

Allow the controller, web_ui, lib and dev plugin directories and the
node_modules directories Node.js would resolve their imports from.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 23:01:12 +00:00
bbassieandClaude Opus 5.5 ebedf6b3b0 Run Oxlint directly instead of through Vite+
Nothing else from Vite+ is used, and Oxlint reads the same rules from
.oxlintrc.json. This drops vite-plus and with it Vitest, a second copy
of Vite and about 130 packages.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 22:53:13 +00:00
bbassieandClaude Opus 5.5 f40113e01c Rename PluginWebpackEnvInfo to PluginWebEnvInfo
The old name is kept as a deprecated alias.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 22:53:13 +00:00
bbassieandClaude Opus 5.5 a4db2dc460 Resolve browser polyfills from web_ui
Lib and plugins import Node.js modules such as events and buffer, which
the build swaps for browser packages. Vite resolves an aliased import
from the importing package, and pnpm only lets a package see its own
dependencies, so the polyfills had to be dependencies of lib.

Resolve them from web_ui with a small resolver plugin instead, also
used by the dependency optimizer. Hosts no longer install the browser
packages, and plugins get the same polyfills webpack gave them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 22:53:13 +00:00
bbassieandClaude Opus 5.5 e94abe51c9 Lint with Oxlint through Vite+ instead of ESLint
Replace ESLint, typescript-eslint, eslint-plugin-node and the flat
config helpers with vite-plus, which runs Oxlint for vp lint. The rules
are carried over from eslint.config.mjs by @oxlint/migrate into the lint
section of the root vite.config.ts. The formatting rules ESLint
deprecated run as @stylistic rules through Oxlint's JS plugin support,
and disable comments naming them are updated to the @stylistic names.

@stylistic understands TypeScript syntax that the core rules skipped,
so the autofix adds missing semicolons and spaces in type literals and
aliases. Leading | and & in multiline types stay where they were.

Oxlint's node/no-sync also reports identifiers ending in Sync that are
not calls. The test helper techSync is renamed and the two other names
are ignored.

Refs #1127

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 22:53:13 +00:00
bbassieandClaude Opus 5.5 3227179613 Build the web interface and plugins with Vite
Replace webpack and Module Federation with Vite 8. The controller build
emits one entry per shared module (lib, web_ui, typebox, antd, react,
react-dom, react-router) and lists them in manifest.json. The controller
turns that into an import map in index.html, and plugins are built as
ES modules that import the shared modules by name and are loaded with
import().

The shared Vite setup lives in @clusterio/web_ui/vite.mjs. Plugins use
clusterioPlugin(), which exposes index, package.json and web/index under
the same keys the webpack template used. The --dev and --dev-plugin
flags now run the Vite dev server in middleware mode with hot module
replacement.

Browser polyfills move from web_ui to lib, which is the code that needs
them, and lib sets the process and Buffer globals itself.

Refs #1127

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 22:52:10 +00:00
Cooldude2606andClaude Opus 5.5 35125a28ab Keep error classes across links (#999)
* Keep error classes across links

Error responses now carry the registered name of the thrown error's
class, and the requester rejects with an instance of that class instead
of a generic RequestError. Error classes are registered like message
classes, and plugins list their own in `errors`.

RequestError is replaced by a hierarchy following the HTTP status codes:
ClientError (ValidationError, ForbiddenError, NotFoundError,
TimeoutError, ConflictError) and ServerError (InternalServerError).
PermissionError is PermissionDeniedError and the config errors
InvalidValue, InvalidField and InvalidAccess gain an Error suffix.

Link logs only errors whose class is not registered. These arrive on the
other side as InternalServerError with the original message and stack,
so an error forwarded through the controller is logged once, where it
was thrown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Add UnavailableError and keep the PermissionError name

Host not connected, host shutting down and instance not running are
now UnavailableError (503) instead of ConflictError. The request error
migration notes move to a new 2.1 guide, since this ships in 2.1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Use the new error classes in code added on master

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 22:47:01 +00:00
bbassieandClaude Opus 5.5 1a03e66479 Use the new error classes in code added on master
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 22:05:14 +00:00
bbassieandClaude Opus 5.5 54e9340e61 Add UnavailableError and keep the PermissionError name
Host not connected, host shutting down and instance not running are
now UnavailableError (503) instead of ConflictError. The request error
migration notes move to a new 2.1 guide, since this ships in 2.1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 22:03:13 +00:00
bbassieandClaude Opus 5.5 1c7c00b033 Keep error classes across links
Error responses now carry the registered name of the thrown error's
class, and the requester rejects with an instance of that class instead
of a generic RequestError. Error classes are registered like message
classes, and plugins list their own in `errors`.

RequestError is replaced by a hierarchy following the HTTP status codes:
ClientError (ValidationError, ForbiddenError, NotFoundError,
TimeoutError, ConflictError) and ServerError (InternalServerError).
PermissionError is PermissionDeniedError and the config errors
InvalidValue, InvalidField and InvalidAccess gain an Error suffix.

Link logs only errors whose class is not registered. These arrive on the
other side as InternalServerError with the original message and stack,
so an error forwarded through the controller is logged once, where it
was thrown.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-27 22:03:13 +00:00
Bastiaan ffe82f81d8 Merge pull request #1010 from bbassie/feature/plugin-browser
Plugin management through web interface
2026-10-27 21:32:51 +00:00
Cooldude2606 e36a76acc5 Merge pull request #1129 from bbassie/refactor/async-factory
Replace bootstrap methods with async factories
2026-10-27 21:31:49 +00:00
Cooldude2606 bda5c5a8c0 Merge pull request #1128 from bbassie/remove-find-unused-name
Remove lib.findUnusedName
2026-10-27 21:31:21 +00:00
Arvids Godjuks 86829d50ba In review guidance ask to look for similar existing functionality and surface to the developer if functionality is being duplicated (#1139) 2026-10-27 11:52:53 +02:00
Danielv123 903e2e0ab6 Set NODE_ENV=production when publishing to npm
The About dialog reported the environment as dev in released builds because NODE_ENV was never set when the web UI was built for publishing.
2026-10-25 21:40:08 +01:00
bbassieandClaude Opus 5.5 abb04f3258 Create FactorioServer and Instance through async factories
FactorioServer.fromOptions finds the Factorio install and generates the
RCON password before constructing, so _version and _dataDir are never
null and setTargetVersion and init are gone. checkForUpdates is now a
module function called before fromOptions.

Instance.fromConfig resolves the version, downloads Factorio if needed
and builds the server, then constructs the instance with it. It runs
before the instance is connected, so the starting and stopped statuses
and the version requests go through the host link. FactorioVersionsRequest
and LatestReleasesRequest now accept host as a source, and the host
tracks starting instances so a second start during a download is
rejected. Plugin loading stays a separate loadPlugins step after the
instance is connected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 00:11:22 +00:00
bbassieandClaude Opus 5.5 3a81cfd94b Order fromConfig arguments config first
Controller.fromConfig and Host.fromConfig now take the config, then the
plugin infos, then the logger or connector.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 00:05:32 +00:00
bbassieandClaude Opus 5.5 ee097bfa6f Test that fromConfig saves instances changed while loading
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 23:51:22 +00:00
bbassieandClaude Opus 5.5 d7fc3aaec4 Replace bootstrap methods with async factories
Controller.bootstrap and Host.bootstrap returned constructor arguments
that the caller had to spread into new Controller/Host, and
DatastoreProvider.bootstrap did the same for datastores. They are
replaced by Controller.fromConfig, Host.fromConfig and
KeyValueDatastore/SubscribableDatastore.fromProvider, which load the
data and return the finished instance.

Closes #1126

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-23 23:51:22 +00:00
bbassieandClaude Opus 5.5 5355a0170a Use the exclusive move for save rename and widen its copy fallback
Fall back to copying on any link error except EEXIST, since filesystems
without hard links report it with several codes (EISDIR on exFAT under
Windows, ENOSYS on some FUSE mounts). Rename now goes through the same
move so it also gets the fallback and the rollback. Only report a
missing source save on transfer when the source is actually missing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 00:22:17 +01:00
bbassieandClaude Opus 5.5 e16d0bc410 Address review of save transfer and instance dir naming
Instance dir and save names are tried without a retry cap, as on master
and in downloadFile, so an 11th instance named "New Instance" still gets
a folder.

Moves fall back to an exclusive copy when hard links are unavailable
(EXDEV, EPERM, ENOTSUP), and remove the new target if the source can't
be unlinked, so a failed move leaves no duplicate.

Local transfers keep appending .zip to the target name, as
findUnusedName did.

The copy auto-rename test now checks that the existing target save is
left untouched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 00:22:17 +01:00
a6d2d48e48 Remove lib.findUnusedName
The name it returns may already be taken by a concurrently running process
by the time the caller uses it. Callers now try the name and retry with a
-2, -3, ... suffix on EEXIST instead.

Save transfers previously renamed with fs.rename, which silently overwrites
a save that appears at the target, and copied without COPYFILE_EXCL. Moves
now use link + unlink like save rename does, and copies pass COPYFILE_EXCL,
so both fail with EEXIST rather than overwrite.

Supersedes #880.

Co-authored-by: Hornwitser <code@hornwitser.no>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-24 00:22:17 +01:00
290 changed files with 16126 additions and 4003 deletions

No files matched your search

+6 -33
View File
@@ -1,5 +1,5 @@
name: Install dependencies
description: Set up pnpm and Node.js, restore caches and run pnpm install
description: Set up pnpm and Node.js and run pnpm install
inputs:
node-version:
@@ -12,42 +12,15 @@ inputs:
runs:
using: composite
steps:
- uses: pnpm/action-setup@v6
- uses: actions/setup-node@v7
with:
node-version: ${{ inputs.node-version }}
package-manager-cache: false
# The lockfile is not committed, so the usual lockfile hash key does not
# apply. Instead a new store snapshot is saved whenever a package.json
# changes or a week passes, and every run restores the most recent one.
- id: store
shell: bash
run: |
echo "path=$(pnpm store path --silent)" >> "$GITHUB_OUTPUT"
echo "week=$(date -u +%G-%V)" >> "$GITHUB_OUTPUT"
- uses: actions/cache@v6
with:
path: ${{ steps.store.outputs.path }}
key: pnpm-store-${{ runner.os }}-${{ steps.store.outputs.week }}-${{ hashFiles('**/package.json', 'pnpm-workspace.yaml') }}
restore-keys: pnpm-store-${{ runner.os }}-
# The prepare scripts run tsc --build and webpack, both incremental when
# their previous output is present. Every run saves its output and the
# next run starts from the most recent one.
- if: inputs.ignore-scripts != 'true'
uses: actions/cache@v6
with:
path: |
packages/*/dist
plugins/*/dist
packages/*/node_modules/.cache
plugins/*/node_modules/.cache
key: build-${{ runner.os }}-${{ github.sha }}
restore-keys: build-${{ runner.os }}-
# pnpm/action-setup installs a bootstrap pnpm and then switches to the
# packageManager version, which takes about 10 seconds.
- shell: bash
run: npm install --global --no-audit --no-fund "$(node -p 'require("./package.json").packageManager.split("+")[0]')"
- shell: bash
run: pnpm i --no-frozen-lockfile ${{ inputs.ignore-scripts == 'true' && '--ignore-scripts' || '' }}
# Drop packages the restored store had but this install did not use, so
# the snapshot does not grow with every version bump.
- shell: bash
run: pnpm store prune
+4
View File
@@ -4,6 +4,10 @@ on:
push:
pull_request:
concurrency:
group: ${{ github.workflow }}-${{ github.ref == 'refs/heads/master' && github.sha || github.ref }}
cancel-in-progress: true
env:
EMMYLUA_CHECK_VERSION: "0.25.1"
+33 -11
View File
@@ -7,6 +7,12 @@ on:
push:
pull_request:
# Superseded runs of a branch or pull request are cancelled. Runs on master
# each get their own group so every commit there is tested.
concurrency:
group: ${{ github.workflow }}-${{ github.ref == 'refs/heads/master' && github.sha || github.ref }}
cancel-in-progress: true
env:
NODE_VERSION: 24.x
@@ -22,17 +28,33 @@ jobs:
strategy:
fail-fast: false
matrix:
# Test Node.js 24 and 26 across the Factorio matrix; Node.js 22 runs coverage below.
# To stay within the concurrent job limit, every Factorio version is
# tested on the latest Node.js, and every other supported Node.js
# version only with the latest Factorio. Coverage is performed on the
# oldest supported Node.js.
# Node.js release schedule: https://nodejs.org/en/about/releases/
node-version: [24.x, 26.x]
# Supported factorio versions are latest stable and one minor version before stable (even if from a previous major)
# Supported factorio versions are latest stable and one minor version before stable (even if from a previous major)
node-version: [26.x]
factorio-version: [1.1.110, 2.0.77, 2.1.17]
# Coverage is performed on the older of the supported node versions
# The tests run in two halves of similar length on separate runners,
# the clusterioctl, plugin system and instance integration tests and the rest.
tests: [ctl, other]
include:
- node-version: 24.x
factorio-version: 2.1.17
tests: ctl
- node-version: 24.x
factorio-version: 2.1.17
tests: other
- node-version: 22.x
factorio-version: 2.1.17
tests: ctl
coverage: true
- node-version: 22.x
factorio-version: 2.1.17
tests: other
coverage: true
steps:
@@ -45,21 +67,23 @@ jobs:
run: wget -q -O factorio.tar.gz https://www.factorio.com/get-download/${{ matrix.factorio-version }}/headless/linux64 && tar -xf factorio.tar.gz && rm factorio.tar.gz
- name: Run tests
if: ${{ !matrix.coverage }}
run: pnpm test
run: pnpm test --grep "^(Integration of Clusterio (clusterioctl|plugin system)|Clusterio Instance) " ${{ matrix.tests == 'other' && '--invert' || '' }}
- name: Run coverage
if: ${{ matrix.coverage }}
run: pnpm run ci-cover
run: pnpm run ci-cover --grep "^(Integration of Clusterio (clusterioctl|plugin system)|Clusterio Instance) " ${{ matrix.tests == 'other' && '--invert' || '' }}
- name: Upload coverage to Codecov
if: ${{ matrix.coverage }}
uses: codecov/codecov-action@v7
with:
files: ./coverage/lcov.info
# Otherwise the raw V8 coverage in coverage/tmp is uploaded as well
disable_search: true
token: ${{ secrets.CODECOV_TOKEN }}
- name: Upload test logs for debugging
if: ${{ !cancelled() }}
uses: actions/upload-artifact@v7
with:
name: logs-${{ matrix.node-version }}-${{ matrix.factorio-version }}
name: logs-${{ matrix.node-version }}-${{ matrix.factorio-version }}-${{ matrix.tests }}
path: temp/test/logs
version-compat:
@@ -87,8 +111,6 @@ jobs:
uses: ./.github/actions/pnpm-install
with:
node-version: ${{ env.NODE_VERSION }}
- name: Build
run: pnpm run build
- name: Test ${{ matrix.controller }} controller with ${{ matrix.host }} host
run: >
node scripts/version-compat-test.js
+2
View File
@@ -23,6 +23,8 @@ jobs:
registry-url: https://registry.npmjs.org
- run: pnpm install --no-frozen-lockfile
- run: pnpm -r --filter='./packages/*' --filter='./plugins/*' --fail-if-no-match publish --no-git-checks
env:
NODE_ENV: production
- uses: softprops/action-gh-release@v3
with:
files: pnpm-lock.yaml
+1 -3
View File
@@ -23,9 +23,6 @@ coverage
build/Release
dist/
# Autogenerated docs output
/docs/api
node_modules
/factorio*
@@ -35,6 +32,7 @@ node_modules
/plugin-list.json
/config-*.json
/config-*.json.lock
/npm-operation.lock*
pnpm-lock.yaml
mod_cache.json
.luarc.json
+458
View File
@@ -0,0 +1,458 @@
// Formatting rules come from @stylistic as Oxlint does not implement them.
{
"$schema": "./node_modules/oxlint/configuration_schema.json",
"plugins": [
"node"
],
"jsPlugins": [
"@stylistic/eslint-plugin"
],
"categories": {
"correctness": "off"
},
"env": {
"builtin": true,
"node": true
},
"ignorePatterns": [
"**/dist/",
"dist/"
],
"rules": {
"@stylistic/indent": [
"error",
"tab",
{
"ignoredNodes": [
"JSXFragment"
],
"ignoreComments": true,
"SwitchCase": 1
}
],
"accessor-pairs": "error",
"array-callback-return": "error",
"arrow-body-style": "error",
"block-scoped-var": "error",
"complexity": [
"error",
{
"max": 25
}
],
"curly": "error",
"default-case": "error",
"default-case-last": "error",
"eqeqeq": "error",
"func-name-matching": "error",
"func-style": [
"error",
"declaration",
{
"allowArrowFunctions": true
}
],
"grouped-accessor-pairs": "error",
"guard-for-in": "error",
"id-match": "error",
"max-depth": "error",
"max-nested-callbacks": "error",
"new-cap": [
"error",
{
"capIsNewExceptions": [
"StringEnum",
"StringKey"
],
"capIsNewExceptionPattern": "^Type\\.",
"newIsCapExceptions": [
"constructor"
]
}
],
"no-alert": "error",
"no-array-constructor": "error",
"no-caller": "error",
"no-console": "error",
"no-constant-condition": [
"error",
{
"checkLoops": false
}
],
"no-constructor-return": "error",
"no-div-regex": "error",
"no-duplicate-imports": "error",
"no-else-return": "error",
"no-eq-null": "error",
"no-eval": "error",
"no-extend-native": "error",
"no-extra-bind": "error",
"no-extra-label": "error",
"no-implicit-coercion": "error",
"no-implicit-globals": "error",
"no-implied-eval": "error",
"no-iterator": "error",
"no-label-var": "error",
"no-lone-blocks": "error",
"no-lonely-if": "error",
"no-loop-func": "error",
"no-multi-assign": "error",
"no-multi-str": "error",
"no-nested-ternary": "error",
"no-new": "error",
"no-new-func": "error",
"no-new-wrappers": "error",
"no-plusplus": [
"error",
{
"allowForLoopAfterthoughts": true
}
],
"no-proto": "error",
"no-restricted-globals": "error",
"no-restricted-imports": "error",
"no-restricted-properties": "error",
"no-return-assign": "error",
"no-script-url": "error",
"no-self-compare": "error",
"no-sequences": "error",
"no-shadow": [
"error",
{
"allow": [
"Event",
"Request",
"yargs"
]
}
],
"no-template-curly-in-string": "error",
"no-throw-literal": "error",
"no-unmodified-loop-condition": "error",
"no-unneeded-ternary": "error",
"no-unused-expressions": "error",
"no-useless-backreference": "error",
"no-useless-call": "error",
"no-useless-computed-key": "error",
"no-useless-concat": "error",
"no-useless-rename": "error",
"no-useless-return": "error",
"no-var": "error",
"no-void": [
"error",
{
"allowAsStatement": true
}
],
"prefer-arrow-callback": "error",
"prefer-exponentiation-operator": "error",
"prefer-numeric-literals": "error",
"prefer-object-spread": "error",
"prefer-promise-reject-errors": "error",
"prefer-regex-literals": "error",
"prefer-rest-params": "error",
"prefer-spread": "error",
"prefer-template": "error",
"radix": [
"error",
"always"
],
"symbol-description": "error",
"unicode-bom": [
"error",
"never"
],
"vars-on-top": "error",
"yoda": [
"error",
"never",
{
"exceptRange": true
}
],
"node/no-sync": [
"error",
{
"ignores": [
"TechnologySync",
"techsToSync"
]
}
],
"@stylistic/array-bracket-newline": "off",
"@stylistic/array-bracket-spacing": [
"error",
"never"
],
"@stylistic/array-element-newline": "off",
"@stylistic/arrow-parens": "off",
"@stylistic/arrow-spacing": [
"error",
{
"after": true,
"before": true
}
],
"@stylistic/block-spacing": "error",
"@stylistic/brace-style": [
"error",
"1tbs",
{
"allowSingleLine": true
}
],
"@stylistic/comma-dangle": [
"error",
{
"arrays": "always-multiline",
"objects": "always-multiline",
"imports": "always-multiline",
"exports": "always-multiline",
"functions": "only-multiline",
"generics": "only-multiline",
"enums": "always-multiline",
"tuples": "always-multiline"
}
],
"@stylistic/comma-spacing": "error",
"@stylistic/comma-style": [
"error",
"last"
],
"@stylistic/computed-property-spacing": [
"error",
"never"
],
"@stylistic/dot-location": [
"error",
"property"
],
"@stylistic/eol-last": "error",
"@stylistic/function-call-argument-newline": "off",
"@stylistic/function-paren-newline": "off",
"@stylistic/generator-star-spacing": [
"error",
{
"before": false,
"after": true
}
],
"@stylistic/implicit-arrow-linebreak": "error",
"@stylistic/jsx-quotes": "error",
"@stylistic/key-spacing": "error",
"@stylistic/keyword-spacing": "error",
"@stylistic/line-comment-position": "off",
"@stylistic/linebreak-style": [
"error",
"unix"
],
"@stylistic/lines-around-comment": "off",
"@stylistic/lines-between-class-members": [
"error",
"always",
{
"exceptAfterSingleLine": true
}
],
"@stylistic/max-len": [
"error",
{
"code": 120
}
],
"@stylistic/max-statements-per-line": "off",
"@stylistic/multiline-comment-style": [
"error",
"separate-lines"
],
"@stylistic/multiline-ternary": "off",
"@stylistic/new-parens": "error",
"@stylistic/newline-per-chained-call": "off",
"@stylistic/no-confusing-arrow": "error",
"@stylistic/no-extra-parens": "off",
"@stylistic/no-floating-decimal": "error",
"@stylistic/no-mixed-operators": [
"error",
{
"groups": [
[
"&",
"|",
"^",
"~",
"<<",
">>",
">>>"
],
[
"==",
"!=",
"===",
"!==",
">",
">=",
"<",
"<="
],
[
"in",
"instanceof"
]
]
}
],
"@stylistic/no-multi-spaces": "error",
"@stylistic/no-multiple-empty-lines": "error",
"@stylistic/no-tabs": "off",
"@stylistic/no-trailing-spaces": "error",
"@stylistic/no-whitespace-before-property": "error",
"@stylistic/nonblock-statement-body-position": "error",
"@stylistic/object-curly-newline": "error",
"@stylistic/object-curly-spacing": "off",
"@stylistic/one-var-declaration-per-line": "error",
"@stylistic/operator-linebreak": [
"error",
"after",
{
"overrides": {
"?": "before",
":": "before",
"||": "before",
"&&": "before",
"??": "before",
"|": "before",
"&": "before"
}
}
],
"@stylistic/padded-blocks": "off",
"@stylistic/padding-line-between-statements": "error",
"@stylistic/quote-props": "off",
"@stylistic/quotes": [
"error",
"double",
{
"avoidEscape": true
}
],
"@stylistic/rest-spread-spacing": [
"error",
"never"
],
"@stylistic/semi": "error",
"@stylistic/semi-spacing": "error",
"@stylistic/semi-style": "off",
"@stylistic/space-before-blocks": "error",
"@stylistic/space-before-function-paren": "off",
"@stylistic/space-in-parens": [
"error",
"never"
],
"@stylistic/space-infix-ops": "off",
"@stylistic/space-unary-ops": "error",
"@stylistic/spaced-comment": [
"error",
"always",
{
"line": {
"markers": [
"/"
]
}
}
],
"@stylistic/switch-colon-spacing": "error",
"@stylistic/template-curly-spacing": [
"error",
"never"
],
"@stylistic/template-tag-spacing": "error",
"@stylistic/wrap-iife": "error",
"@stylistic/wrap-regex": "off",
"@stylistic/yield-star-spacing": "error"
},
"overrides": [
{
"files": [
"{packages,plugins}/**/*.{ts,tsx}"
],
"rules": {
"no-unused-vars": "off",
"typescript/ban-ts-comment": "error",
"typescript/no-duplicate-enum-values": "error",
"typescript/no-empty-object-type": "error",
"typescript/no-explicit-any": "off",
"typescript/no-extra-non-null-assertion": "error",
"typescript/no-misused-new": "error",
"typescript/no-namespace": "error",
"typescript/no-non-null-asserted-optional-chain": "error",
"typescript/no-require-imports": "off",
"typescript/no-this-alias": "error",
"typescript/no-unnecessary-type-constraint": "error",
"typescript/no-unsafe-declaration-merging": "error",
"typescript/no-unsafe-function-type": "error",
"typescript/no-wrapper-object-types": "error",
"typescript/prefer-as-const": "error",
"typescript/prefer-namespace-keyword": "error",
"typescript/triple-slash-reference": "off",
"no-shadow": [
"error",
{
"allow": [
"Event",
"Request",
"yargs"
]
}
],
"no-useless-constructor": "error"
},
"plugins": [
"typescript"
]
},
{
"files": [
"test/**/*.js",
"plugins/*/test/**/*.js"
],
"rules": {
"prefer-arrow-callback": "off"
},
"env": {
"mocha": true
}
},
{
"files": [
"packages/web_ui/src/**/*.{ts,tsx}",
"packages/lib/browser.ts",
"packages/create/templates/*/web/**/*.{ts,js,tsx,jsx}",
"{packages,plugins}/*/web/**/*.{ts,tsx}"
],
"globals": {
"SpeechRecognitionAlternative": "readonly",
"SpeechRecognitionResult": "readonly",
"SpeechRecognitionResultList": "readonly"
},
"env": {
"browser": true
}
},
{
"files": [
"packages/create/templates/**/*.{js,cjs,ts,cts,jsx,tsx}"
],
"rules": {
"@stylistic/indent": "off",
"@stylistic/spaced-comment": "off",
"@stylistic/comma-spacing": "off",
"@stylistic/max-len": "off"
},
"jsPlugins": [
"@stylistic/eslint-plugin"
]
}
]
}
+1 -1
View File
@@ -18,7 +18,7 @@ Use the detailed guides for the affected area:
- Read the relevant source and tests before changing behavior.
If documentation and implementation disagree, report the discrepancy and distinguish observed behavior from intended policy.
Update affected documentation in the same change.
- Follow nearby code and `eslint.config.mjs`: tabs, double quotes, semicolons, and `.js` extensions in relative TypeScript imports.
- Follow nearby code and `.oxlintrc.json`: tabs, double quotes, semicolons, and `.js` extensions in relative TypeScript imports.
Import shared public APIs through `@clusterio/lib` and check Node/browser exports when changing them.
- Run commands from the root using the pinned pnpm version and the [documented Node requirements](docs/contributing.md#supported-nodejs-and-factorio-versions).
Install with `pnpm install --no-frozen-lockfile`; the lockfile is intentionally ignored.
+18 -3
View File
@@ -251,6 +251,7 @@ Do not install the [Clusterio mod](https://mods.factorio.com/mod/clusterio), thi
After completing the installation and creating your admin user, start up the controller and at least one host separately.
The installer provides the `run-controller` and `run-host` scripts to make this simple.
Once the controller process is running you can log into the web interface which is hosted by default on http://localhost:8080/ (adjust the port number if you changed it), use the admin authentication token provided from the installation to log in.
The first time a host starts it prints a link for approving it, open the link and click Approve.
The basics of setting up a Factorio server from the web interface is to create an instance, assign it to a host and then click start. You can also use the "Connect via Steam" button to launch Factorio directly and connect to the server using Steam's protocol handler.
@@ -272,9 +273,23 @@ If you have copied these files over to `/etc/systemd/system/` then you can start
## Setting up remote hosts
Run the installer as described in the installation section and choose "Host only" as the operating mode to install, it'll ask for a controller URL and an authentication token.
The URL is the same as what is needed to connect to the web interface, and the authentication token can be generated on the Hosts page in the web interface.
Once you start up the host it should show up in the Hosts list and be available for assigning and running instances on.
Run the installer as described in the installation section and choose "Host only" as the operating mode to install, it'll ask for a name for the host and a controller URL.
The URL is the same as what is needed to connect to the web interface.
Then start the host, the first time it starts it prints a code and waits for the host to be approved:
Waiting for approval of "my-host" with code ABC-DEF, open this link to approve it:
https://example.com/hosts?approve=ABC-DEF
Or run: npx clusterioctl host approve ABC-DEF
Open the link to approve it in the web interface, where you can also change its name, or click Add host on the Hosts page and approve the request with the same code, or run `npx clusterioctl host approve <code>`.
The host then saves its connection details and starts up, it should show up in the Hosts list and be available for assigning and running instances on.
Only approve a request if its code matches the one shown by the host, approving gives the host full access to the cluster.
Approving requires the `core.host.approve` permission, which the Cluster Admin role has.
If you have that permission you can skip the wait by starting the host with `npx clusteriohost run --user-token <token>` and your own authentication token, it is only used to approve the host and is not saved.
To remove a host from the cluster, click Revoke host access on the page for the host in the web interface, or run `npx clusterioctl host revoke <host>`.
This disconnects the host, its instances become unassigned and its saved connection details stop working.
To add the computer back to the cluster start the host again, it asks for approval again.
## Setting up clusterioctl
+1 -1
View File
@@ -49,7 +49,7 @@ async function removeBuildArtifacts() {
tryRemove(path.join(basePath, packageName, "dist"))
);
tasks.push(
tryRemove(path.join(basePath, packageName, "node_modules", ".cache"))
tryRemove(path.join(basePath, packageName, "node_modules", ".vite"))
);
}
}
+1
View File
@@ -43,6 +43,7 @@ Plugin classes also supply a `plugin` name.
`Link.register` registers request and event classes and compiles their payload validators.
Core classes are registered by the link module; `registerPluginMessages` registers each plugin's `messages` array.
It also registers the plugin's `errors` array with `Link.registerError`, so an error thrown by a request handler is rejected on the requester as the same class.
Incoming messages invoke handlers attached with `link.handle(MessageClass, handler)`.
Plugins attach these handlers from their entrypoint functions rather than relying on method-name conventions.
+2 -1
View File
@@ -45,6 +45,7 @@ For plugins the group name is the name of the plugin, e.g., the `level` field of
**initialValue** (optional if optional is true):
Value this config field should take in a newly initialized config.
This can also be a synchronous function returning the value to use.
The function is called with the config after its stored fields are loaded, so it can base the value on other fields.
**validator** (optional):
Function called with the value being set and the config it belongs to.
@@ -116,7 +117,7 @@ E.g.:
let value = configInstance.fields["group.field"]
The set method also takes an extra argument, which is the value to set the field to.
If the value is not valid this will throw an InvalidValue exception, if the field doesn't exist it will throw an InvalidField exception.
If the value is not valid this will throw an InvalidValueError exception, if the field doesn't exist it will throw an InvalidFieldError exception.
## Config Lifecycle
+57 -5
View File
@@ -48,6 +48,23 @@ Must be greater than 0.
Defaults to 60.
### controller.host_pairing_timeout
Time in seconds a host can wait for approval before its request expires, after which the host requests again with a new code.
Set to 0 for no limit.
Defaults to 3600.
### controller.host_pairing_max_pending
Maximum number of hosts that can wait for approval at the same time.
Hosts over the limit keep trying until there is room.
Set to 0 for no limit.
Defaults to 50.
### controller.http_port
Port to host HTTP server on.
@@ -93,7 +110,7 @@ Defaults to null meaning the header is never trusted.
### controller.public_url
Public address the controller is accessible on.
Used as the URL in configs generated with `clusteriocontroller bootstrap create-ctl-config` and `clusterioctl host create-config`, and by plugins like Player Auth that need to show the address of the web interface in-game.
Used as the URL in configs generated with `clusteriocontroller bootstrap create-ctl-config`, and by plugins like Player Auth that need to show the address of the web interface in-game.
This should be a full URL ending with a /.
Defaults to null meaning assume localhost.
@@ -103,7 +120,7 @@ Defaults to null meaning assume localhost.
Used to override where static assets such as the web bundle is loaded from.
This can be used if you want to put the static assets on a separate cdn url like `https://cdn.example.com/static/` while keeping the API hosted directly by clusterio.
Due to a limitation on how the webpack assets are built the pathname component of the url have to be `/static/`, values with any other path are rejected.
Due to a limitation on how the web assets are built the pathname component of the url have to be `/static/`, values with any other path are rejected.
See also the `clusteriocontroller copy-static` command for how to obtain the static asset files that need to be hosted.
@@ -277,6 +294,7 @@ Defaults to 1000.
Whether the `controller update` command in clusterioctl and the update button in the web interface are allowed to update the `@clusterio/controller` package with npm.
For security reasons this can only be changed locally with `clusteriocontroller config set` while the controller is stopped, and it is hidden in the web interface.
The installer sets this, controller.allow_plugin_updates and controller.allow_plugin_install from one question about remote updates and plugin installs.
See [Managing a Cluster](/docs/managing-a-cluster.md) for details on remote updates.
Defaults to true.
@@ -326,6 +344,7 @@ Defaults to "New Host".
### host.id
Immutable numeric id of the host which uniquely identifies this host in the cluster.
The controller picks the id when the host is approved and the host writes it here, the controller does not accept hosts with an id it has not approved.
Hidden in the web interface.
Defaults to a random 31 bit number.
@@ -335,6 +354,7 @@ Defaults to a random 31 bit number.
Directory to look for the Factorio server(s) in.
This can either point to a Factorio server directory, or to a directory containing multiple versions of the Factorio server.
The headless build has no graphics, so when exporting data from an instance running it the base game's icons are read from a full install of the same major and minor version in this directory instead, if there is one.
Defaults to "factorio".
@@ -365,7 +385,9 @@ Defaults to "http://localhost:8080/".
### host.controller_token
Access token used for authenticating with the controller.
You can generate an access token with `clusterioctl host generate-token --id <host-id>`, or use the `clusterioctl host create-config` to create a new host config with the correct url and token in it.
The host saves it together with host.id when it is approved, see [Setting up remote hosts](/README.md#setting-up-remote-hosts).
The token only works while the host exists on the controller, if the host is revoked it asks to be approved again the next time it starts.
Set it to `enter token here` to make the host ask to join again.
The value can only be read and changed from the host itself.
Defaults to "enter token here".
@@ -418,6 +440,7 @@ Defaults to 60.
Whether the `host update` command in clusterioctl and the update button in the web interface are allowed to update the `@clusterio/host` package with npm.
For security reasons this can only be changed locally with `clusteriohost config set` while the host is stopped, and it is hidden in the web interface.
The installer sets this, host.allow_plugin_updates and host.allow_plugin_install from one question about remote updates and plugin installs.
See [Managing a Cluster](/docs/managing-a-cluster.md) for details on remote updates.
Defaults to true.
@@ -498,6 +521,31 @@ This allows you to prevent certain instances from being started when using the b
Defaults to false.
### instance.is_template
If enabled this instance is a template other instances can inherit config values from.
Templates cannot be assigned to a host and therefore never start.
Hidden from `instance list` in clusterioctl unless `--templates` is passed, and listed separately in the web interface.
An instance cannot stop being a template while other instances inherit from it.
Hidden in the web interface, set when creating the template.
Defaults to false.
### instance.template_id
ID of the template to inherit config values from.
Inherited fields take their value from the template and follow it when it changes, setting a field on this instance stops inheriting it.
Object fields like `factorio.settings` are inherited per property, properties set on this instance override the ones from the template.
When set on an existing instance fields with the same value as the template or the default value become inherited, except the server name in `factorio.settings`.
`instance.name`, `instance.is_template`, `instance.template_id`, the port fields, `factorio.rcon_password` and hidden fields are never inherited.
A template may itself inherit from another template, but not from itself through a chain of templates.
The fields currently inherited are stored in the hidden `instance.inherited_fields` field.
In the web interface inherited values are linked and read only until they are unlinked.
Defaults to null meaning no template.
### factorio.version
Version of Factorio to use for this instance.
@@ -591,6 +639,8 @@ Most plugins require the use of save patching to function.
Turning it off makes it possible to use Clusterio to run and manage regular vanilla games and scenarios not compatible with Clusterio.
Plugins that need save patching can't be loaded on the instance while this is disabled.
Disabling it while such a plugin is loaded is rejected, and so is loading such a plugin while it is disabled.
Saves patched before are restored to their original control.lua and have the modules removed when started with this disabled.
Data the modules stored in the game is left in place.
Defaults to true.
@@ -644,6 +694,8 @@ The settings in `data/server-settings.example.json` of the Factorio installation
Changes to the following properties will be applied live if the instance is running while it is changed: `afk_autokick_interval`, `allow_commands`, `autosave_interval`, `autosave_only_on_server`, `description`, `ignore_player_limit_for_returning_players`, `max_players`, `max_upload_slots`, `max_upload_in_kilobytes_per_second`, `name`, `only_admins_can_pause_the_game`, `game_password`, `require_user_verification`, `tags`, `visibility`.
With `auto_pause` on, the plugins included with Clusterio stop their periodic commands while no players are online, so the game stays paused.
Defaults to a set of common settings filled in when the instance is created, such as `"tags": ["clusterio"]`, `"auto_pause": false` and a `name` and `description` based on the cluster and instance names.
@@ -739,9 +791,9 @@ Defaults to 5.
Whether to load the given plugin on the instance.
Note that plugins need to be loaded on the controller and the host in order for them to be loaded on instances.
Loading a plugin that requires save patching or script commands is rejected while factorio.enable_save_patching or factorio.enable_script_commands respectively is disabled on the instance.
Loading a plugin that requires save patching, script commands or Lua UDP is rejected while factorio.enable_save_patching, factorio.enable_script_commands or factorio.enable_lua_udp respectively is disabled on the instance.
Defaults to true.
Defaults to true, or false if the instance does not have the features the plugin requires.
## Ctl Configuration
+44 -9
View File
@@ -18,6 +18,7 @@ Read our [past decisions](/docs/decisions.md) for compatibility policies and des
- [Checking your code for errors](#checking-your-code-for-errors)
- [Cleaning generated and runtime files](#cleaning-generated-and-runtime-files)
- [Submitting a Pull Request](#submitting-a-pull-request)
- [Breaking Compatibility with the Last Release](#breaking-compatibility-with-the-last-release)
- [Editing your Feature Branch](#editing-your-feature-branch)
- [Resolving Merge Conflicts](#resolving-merge-conflicts)
- [Updating Your GitHub Fork](#updating-your-github-fork)
@@ -42,7 +43,7 @@ Clusterio manages distributed Factorio servers through a controller, hosts, and
| `packages/host/` | Instance lifecycle, Factorio processes, save patching, and exports. Start with `src/Host.ts`, `src/Instance.ts`, `src/server.ts`, and `src/patch.ts`. |
| `packages/ctl/` | Command-line administration and plugin commands. |
| `packages/lib/` | Shared messages, routing, config, permissions, plugin loading, logging, and utilities for Node and browser consumers. |
| `packages/web_ui/` | React/Ant Design interface and browser control connection; bundled through the controller's webpack build. |
| `packages/web_ui/` | React/Ant Design interface and browser control connection; bundled through the controller's Vite build. |
| `packages/create/` | Installer and JavaScript/TypeScript plugin templates. |
| `plugins/` | Bundled plugins, including their Node, web, Lua, and test code. |
| `external_plugins/` | Plugins developed in independent repositories and included in the local workspace. |
@@ -86,10 +87,12 @@ Install dependencies with:
This installs dependencies and builds the packages.
`pnpm-lock.yaml` is intentionally ignored; do not force-add it or assume a frozen-lockfile install is supported.
For a fresh development cluster, run the installer and select standalone mode to create `config-controller.json`, `config-host.json`, and the credentials needed to connect them:
For a fresh development cluster, run the installer and select standalone mode to create `config-controller.json` and `config-host.json`:
node packages/create --dev
The first time the host starts it prints a link for approving it in the web interface.
You can skip this step if you already have a working controller and host configuration.
To add plugins to use in the development environment use
@@ -119,13 +122,13 @@ To have TypeScript watch for changes and rebuild immediadly you may run `pnpm wa
For web development on the controller there are also the following flags:
`--dev` - Recompile the `web_ui` package on the fly providing live reloading of it.
`--dev` - Serve the `web_ui` package from source through Vite's development server with hot module replacement.
`--dev-plugin [name]` - Recompile the web module for the given plugin on the fly proving live reloading of it.
`--dev-plugin [name]` - Serve the web module for the given plugin from source the same way.
This also serves the `web_ui` package from source, as the plugin has to share its modules with it.
These options can be combined and multiple plugins can be actived for dev mode.
Keep in mind if you're working on external plugins the `--dev-plugin` option is likely to only work if the plugin is placed inside the `external_plugins` directory and its dependencies are installed with pnpm as part of the clusterio repository workspace.
Note that webpack is configured to delete the web interface build when using these options, so you will have to run `pnpm -r run prepare` before you start the controller without these flags again.
### Build outputs
@@ -136,9 +139,8 @@ Note that webpack is configured to delete the web interface build when using the
| `pnpm watch` | TypeScript compilation as files change; restart affected Node processes afterward. |
| `pnpm -r run prepare` | Compiled packages, generated validators, and production web bundles. |
| `pnpm build-mod` | Factorio mod archives in root `dist/`. |
| `pnpm docs` | Generated API documentation. |
`pnpm build` alone does not regenerate the standalone validator or webpack bundles.
`pnpm build` alone does not regenerate the standalone validator or web bundles.
After changing `packages/lib/src/data/messages_core.ts`, run `pnpm -r run prepare` to compile the schema and regenerate its validator.
@@ -228,7 +230,7 @@ Inspect `node clean.js --help` and preview the selected scope with `--dry-run` b
| Command | Removal scope |
| --- | --- |
| `node clean.js --dry-run` | Preview workspace package/plugin build output and webpack caches; omit `--dry-run` to clean, or use `pnpm clean`. |
| `node clean.js --dry-run` | Preview workspace package/plugin build output and Vite dependency caches; omit `--dry-run` to clean, or use `pnpm clean`. |
| `node clean.js tests --dry-run` | Also preview root `.nyc_output/`, `coverage/`, the whole `temp/` directory, and `static/`. |
| `node clean.js all --dry-run` | Also preview runtime databases, instances, logs, mods, local configs, `FactorioAdminToken.txt`, and `plugin-list.json`. |
@@ -251,6 +253,38 @@ If there are more changes you want to have merged or mistakes you need to fix yo
Pull request descriptions should contain a `### Changelog` section with additions for the changelog contained inside a code block, see the [CHANGELOG.md](/CHANGELOG.md) file for the formatting of changelog entries and possible sections to use.
If there are no changelog entries to add, remove the code block and start the changelog section with the text `None.`, followed by an optional reason for why it is not is needed.
### Breaking Compatibility with the Last Release
The version-compat job in CI runs the local controller with the latest published host, and the published controller with the local host.
It fails if a link message changed its src, dst, data or response schema since that release.
Control is left out of src and dst, and messages that only go between control and the controller are not checked, as the web UI is served by the controller it talks to.
If the break is intended, add an entry to [scripts/version-compat-breaks.json](../scripts/version-compat-breaks.json) in the same pull request.
Breaking changes are only accepted if the next release is intended to include them, see our decision on [breaking changes](/docs/decisions.md#breaking-changes).
The entry is keyed by the published version directly before the breaking change was introduced, which is the latest release at the time.
Every entry needs the pull request number, a reason saying what breaks, and an impact saying why end users are not or only minimally affected.
```json
{
"2.0.3": {
"messages": {
"request FactorioVersionsRequest": {
"pr": 1129,
"reason": "What breaks between the two versions.",
"impact": "Why end users are not or only minimally affected."
}
},
"connect": { "pr": 1234, "reason": "Why a host cannot connect at all.", "impact": "..." }
}
}
```
Listed messages are reported as warnings with the reason and impact instead of failing the job.
Only add `connect` when the published and local versions cannot connect at all, it skips starting the cluster.
Listing a message that did not change fails the job.
Entries only apply while their version is the latest release, after the next release they are ignored and can be removed.
Describe the break in the Breaking Changes section of the changelog.
### Editing your Feature Branch
Until your feature branch is merged you can edit it as you see fit by using the rebase tool.
@@ -308,7 +342,8 @@ The format of this log should be self explanatory, please follow it carefully. P
For development, use the latest Node.js LTS release, following the [installation instructions](../README.md#installation).
Make sure the Node.js built-in APIs and npm packages you use remain compatible with the supported versions.
The GitHub Actions tests currently run on Node.js 24 and 26 across the Factorio matrix, with Node.js 22 used for coverage; see the [CI workflow](../.github/workflows/node.js.yml) for the version matrix.
The GitHub Actions tests run every Factorio version on the latest Node.js, and every other supported Node.js version with the latest Factorio only, which keeps the number of jobs within GitHub's concurrent job limit.
Coverage is collected on the oldest supported Node.js; see the [CI workflow](../.github/workflows/node.js.yml) for the version matrix.
For Factorio, active support covers the latest stable version and the previous major version, using version lines such as 2.1, 2.0 and 1.1 as described in the [Factorio support policy](decisions.md#factorio).
Older versions from 0.17 onwards receive passive support where it does not restrict actively supported versions.
+20 -13
View File
@@ -18,7 +18,7 @@ If there is a particular issue or concern you would like the team to comment on
- [Naming Times and Durations](#naming-times-and-durations)
- [Strings](#strings)
3. Code Style (TS / JS / Web)
- [ESLint](#eslint)
- [Oxlint](#oxlint)
- [Imports Within `lib` Package](#imports-within-lib-package)
- [Name Style](#name-style-ts)
4. Code Style (Lua)
@@ -38,7 +38,7 @@ If there is a particular issue or concern you would like the team to comment on
- [React](#react)
- [Separation of Host and Controller](#separation-of-host-and-controller)
- [Typescript](#typescript)
- [Webpack](#webpack)
- [Vite](#vite)
8. Supportability
- [Breaking Changes](#breaking-changes)
- [Distribution](#distribution)
@@ -91,6 +91,11 @@ Therefore, we decided to use TSDoc comments for this purpose as it is well inter
Our specific implementation is TypeDoc because it is lightweight it and can output TSDoc comments as html.
API Extractor was also considered but not selected.
🧾 29th October 2026 | [Github PR #1147](https://github.com/clusterio/clusterio/pull/1147)
We dropped TypeDoc as the generated docs were never published and it does not support TypeScript 7.
TSDoc comments are still used for editor hover text.
## Code Style (General)
@@ -157,14 +162,15 @@ Therefore, we selected double quotes as our standard.
## Code Style (TS / JS / Web)
### ESLint
### Oxlint
🧾 Time Immemorial
🧾 20th October 2026
> Use ESLint for style checking.
> Use Oxlint for style checking, with formatting rules from `@stylistic/eslint-plugin`.
We needed to maintain a consistent style across all our files.
Therefore, we utilise the industry-standard ESLint, as it is familiar and highly configurable.
We needed to maintain a consistent style across all our files and used ESLint for it from the start.
Oxlint runs the same rules about three times faster, reads ESLint disable comments, and does not depend on the TypeScript compiler API.
The formatting rules ESLint deprecated run through `@stylistic/eslint-plugin` loaded as an Oxlint JS plugin, so the code style did not change.
### Imports Within `lib` Package
@@ -347,14 +353,15 @@ Transitioning to any other langauge would be too significant an undertaking.
Initially, we supported multiple languages for plugins, including Rust and Go; however, this approach was quickly abandoned due to its complexity.
### Webpack
### Vite
🧾 Time Immemorial
🧾 20th October 2026
> Our web stack uses Webpack and associated plugins.
> Our web stack uses Vite. Plugin web bundles are ES modules that import shared dependencies through an import map.
We required a bundler for our web build, and at the time, Webpack was the standard.
While we have considered alternatives since, transitioning to another solution would be a significant undertaking.
We used Webpack with Module Federation for loading plugins from the start, as it was the standard at the time.
Vite builds the web interface several times faster, its development server updates modules without reloading the page, and plugin bundles no longer depend on one bundler's runtime.
See [#1127](https://github.com/clusterio/clusterio/issues/1127) for the discussion.
## Supportability
@@ -370,7 +377,7 @@ We aim to be highly transparent with breaking changes and maintain compatibility
During the alpha stage every alpha was allowed to contain breaking changes, as long as they were documented and provided migration actions.
From 2.0 onwards changes which require action from people running a cluster, such as changes to configs, stored data or the command line, are held back for the next major version.
Changes to the plugin API may still be made in minor versions.
Plugin authors should check the changelog when a new minor version is released, non-trivial changes come with a migration guide such as [Migrating Plugins to 2.0](/docs/devs/plugin-migration-to-2.0.md).
Plugin authors should check the changelog when a new minor version is released, non-trivial changes come with a migration guide such as [Migrating Plugins to 2.1](/docs/devs/plugin-migration-to-2.1.md).
### Distribution
+105
View File
@@ -0,0 +1,105 @@
# Migrating Plugins to 2.1
This guide covers the changes needed to bring a plugin written for 2.0 up to 2.1.
Plugins still on an alpha should start with the [2.0 guide](plugin-migration-to-2.0.md).
## Request errors
`RequestError` is gone.
Errors thrown by request handlers are now received by the requester as the same class, see [Request errors](/docs/writing-plugins.md#request-errors).
Replace `RequestError` with the class that fits:
- Invalid input: `ValidationError`.
- Operation disabled by config: `ForbiddenError`.
- Missing instance, user, save etc.: `NotFoundError`.
- Instance running, name taken etc.: `ConflictError`.
- Host not connected, instance not running: `UnavailableError`.
- Timeouts: `TimeoutError`.
`catch` blocks that check `err instanceof RequestError` should check `ClientError` instead, or the specific class.
Checks of `err.code` for remote errors can use `instanceof`, e.g. `err instanceof lib.SessionLost`.
The config errors `lib.InvalidValue`, `lib.InvalidField` and `lib.InvalidAccess` were renamed to `lib.InvalidValueError`, `lib.InvalidFieldError` and `lib.InvalidAccessError`.
Errors of classes that are not registered, like a `TypeError` from a bug, are received as `InternalServerError`.
Plugins that define their own error classes need to list them in `errors` of the plugin declaration.
## Build the web module with Vite
The web interface is now built with Vite instead of webpack, and plugin web modules are loaded as ES modules instead of through Module Federation.
A plugin built with webpack fails to load with the error "Incompatible dist/web/manifest.json, the plugin must be rebuilt with Vite".
Delete `webpack.config.cjs` and add a `vite.config.js` next to `package.json`:
```js
import { defineConfig } from "vite";
import { clusterioPlugin } from "@clusterio/web_ui/vite.mjs";
export default defineConfig({
plugins: [clusterioPlugin()],
});
```
In `package.json` replace `webpack-cli --mode production` in the `prepare` script with `vite build`, and replace the `webpack`, `webpack-cli` and `webpack-merge` dev dependencies with `vite`.
Add `vite.config.js` to `.npmignore` if you had `webpack.config.js` there.
`clusterioPlugin` exposes `index.ts` (or `index.js`) as `"./"`, `package.json` as `"./package.json"` and `web/index.tsx` (or `.ts`, `.jsx`, `.js`) as `"./web"`.
These are the same keys the plugin template used with webpack, so `webEntrypoint: "./web"` stays as it is.
If your webpack config exposed other paths, move the code so it is reachable from `web/index.tsx`.
The web interface provides `@clusterio/lib`, `@clusterio/web_ui`, `@sinclair/typebox`, `antd`, `react`, `react-dom`, `react-router` and `react-router-dom` to plugins.
These are left as imports in the plugin bundle and resolved by the browser, there is no need to list them as shared.
Everything else is bundled into the plugin, including `@ant-design/icons`.
Import `react-dom/client` and `react/jsx-runtime` by those names, other subpaths of the shared packages are bundled as a separate copy.
Imports of `assert`, `buffer`, `events`, `os`, `path`, `process`, `stream`, `util` and `zlib` resolve to the same browser packages as before, provided by `@clusterio/web_ui`.
The `container` property of `WebPluginContext` is now the plugin's ES module namespace, `await container.get("./web")` returns the module instead of a factory function.
`lib.PluginWebpackEnvInfo` is renamed to `lib.PluginWebEnvInfo`.
## Renamed lib functions
`lib.safeOutputFile` is renamed to `lib.atomicOverwriteFile`.
It takes the same arguments.
## Host pairing
Host tokens are no longer generated by users.
A host joins the cluster by having the code it shows when it first starts approved, see [Host pairing](protocol.md#host-pairing) for the HTTP endpoints it uses.
- `HostGenerateTokenRequest` and `HostConfigCreateRequest` are removed.
Pending requests are listed with `HostPairingListRequest`, approved with `HostPairingApproveRequest` and rejected with `HostPairingRejectRequest`, and changes to them are sent to subscribers of `HostPairingUpdatesEvent`.
- `HostRevokeTokensRequest` is replaced by `HostRevokeRequest`, which deletes the host instead of only invalidating its tokens.
Instances assigned to the host become unassigned.
- The permissions `core.host.generate_token`, `core.host.revoke_token` and `core.host.create_config` are removed.
Check `core.host.approve` and `core.host.revoke` instead.
- `clusteriocontroller bootstrap generate-host-token` is removed, scripts that set up hosts can start the host with `--user-token` to approve it with a user token.
Host records are no longer created when a host connects, only by approving it, and the controller rejects hosts it has no record of.
Revoking a host deletes its record and note.
Plugins that keep data per host should remove it when the host is deleted.
On the controller the `update` event of `controller.hosts.records` gives the record with `isDeleted` set to true, and subscribers of `HostUpdatesEvent` receive the host with `isDeleted` set to true.
A host that is revoked and approved again gets a new id.
## Host manager
`controller.hosts` is now a `HostManager`, like `controller.users` is a `UserManager`.
`get`, `getMutable`, `has` and `values` work as before.
Everything else on the datastore, like `set`, `delete`, `size` and the `update` event, is on `controller.hosts.records`.
The host manager also signs and checks host tokens and creates and deletes hosts:
- `controller.generateHostToken(hostId)` is now `controller.hosts.signHostToken(hostId)`.
- `controller.verifyHostToken(token)` is now `controller.hosts.verifyHostToken(token)`.
It returns the `HostRecord` instead of the host id.
- `controller.hostPairing.createHost(name)` is now `controller.hosts.createHost(name)`.
It returns the new `HostRecord` instead of `{ hostId, token, name }`, sign a token for it with `signHostToken`.
- `controller.hosts.revokeHost(hostId, revokedBy)` deletes a host the same way `HostRevokeRequest` does.
User tokens are checked with `controller.users.verifyUserToken(token)`, which returns the user or throws if the token is invalid.
+64 -1
View File
@@ -32,6 +32,63 @@ The [send-buffer setting](../configuration.md#controllermax_send_buffer_size) co
![Connection flow](../assets/connection_flow.svg)
## Host pairing
The host id and token sent in `registerHost` are requested by the host over HTTP the first time it starts, before it connects.
Hosts are only added to the controller by approving a pairing request, and `registerHost` is rejected with close code 3000 (Unauthorized) for a host id the controller has no record of.
Revoking a host deletes its record, so its token stops working and the host has to be approved again the next time it starts, which gives it a new id.
See [Setting up remote hosts](/README.md#setting-up-remote-hosts) for how this looks to the user.
The endpoints below take and return JSON bodies and are sent with `Cache-Control: no-store`.
The `POST` endpoints return errors as `{"error": "<message>"}` and respond with 503 while the controller is in recovery mode.
A request body that is not valid JSON or is larger than 4 kB is rejected with 400 or 413 before it reaches the endpoint, without a JSON body.
`GET /api/host-pairing/check` also works in recovery mode, and responds to a token that is not valid with a plain 401 without a JSON body.
A user or host token is sent in either an `Authorization: Bearer <token>` header or an `X-Access-Token` header.
### `POST /api/host-pairing`
Creates a request for a host to join the cluster.
- `name: string` - Name of the host, at most 100 characters after trimming whitespace, without control characters.
- `version: string` (optional) - Version of the host.
Responds with 202 and `{"requestId": string, "code": string}`.
The `code` is shown to the user to approve the request with, formatted as six characters like `ABC-DEF`.
Codes are matched ignoring case and anything that is not a letter or digit, so `abcdef` matches `ABC-DEF`.
The `requestId` is a secret used to poll for the result, it is never shown to users.
If a user token is sent and the user has the `core.host.approve` permission the host is created right away and the response is 200 with the same body as an approved poll.
An invalid token gives 401 and a user without the permission gives 403, a token is never ignored to create a pending request instead.
A missing or too long name gives 400.
Too many pending requests from the same address, or in total as set by [controller.host_pairing_max_pending](/docs/configuration.md#controllerhost_pairing_max_pending), gives 429 with a `Retry-After` header in seconds.
### `POST /api/host-pairing/poll`
- `requestId: string` - Id returned when the request was created.
Responds with 202 and `{"status": "pending"}` while the request is waiting for approval.
Once approved it responds with 200 and `{"hostId": integer, "token": string, "name": string}`, where `name` is the name the host was approved with, which can differ from the one asked for.
Rejected requests give 403, and unknown or expired requests give 404.
The result of an approved or rejected request is only returned once, after which the request is gone.
A pending request expires 30 seconds after it was last polled, and after waiting for the time set by [controller.host_pairing_timeout](/docs/configuration.md#controllerhost_pairing_timeout).
The host polls every 2 seconds, and makes a new request when the previous one has expired.
### `POST /api/host-pairing/cancel`
- `requestId: string` - Id returned when the request was created.
Removes a pending request, sent by the host when it stops waiting.
Responds with 204 whether or not the request existed.
### `GET /api/host-pairing/check`
Checks a host token.
Responds with 200 and `{"valid": true}` if the token is valid and the host it was issued to still exists, and with 401 otherwise.
A host with a saved token checks it with this when it starts, and requests to join again if the token is not accepted.
## Connector flow
To manage the complexity of connecting, reconnecting, resuming or invalidating dropped sessions the WebSocket connection for a `Link` class is managed by a connector.
@@ -119,6 +176,11 @@ Host handshake for establishing a new connection session.
- `plugins: Object<string, string>` -
Object mapping plugin names to plugin version for plugins that are available on the host.
The controller closes the connection with code 3000 (Unauthorized) if the token is invalid, was issued for another id, or if there is no host with this id, see [Host pairing](#host-pairing).
The host checks its token before connecting and requests to join again if it is not accepted, so this normally only happens to a host that is already running.
A host that has been rejected exits with code 8, and the run scripts created by the installer do not restart it.
A host that is revoked while connected is disconnected, is rejected when it tries to connect again, and shuts down.
#### `registerControl`
Control handshake for enstablishing a new connection session.
@@ -223,7 +285,8 @@ Plugins may define their own requests as well.
- `src: Address` - Responding endpoint.
- `dst: Address` - Requester's address, including the original request ID.
- `data.message: string` - Human-readable error description.
- `data.code: string` (optional) - Error code.
- `data.code: string` (optional) - Registered name of the error class, `<plugin>:<class>` for plugin errors.
Missing or unknown names are received as `InternalServerError`.
- `data.stack: string` (optional) - Error stack.
+1
View File
@@ -27,6 +27,7 @@ The save patching is based on the built in [event_handler](https://github.com/wu
Mods on the other hand work without modifications.
This can be disabled with the `factorio.enable_save_patching` config on a per instance basis, however disabling it means that modules are not loaded into the game and most plugins will not function.
A save that was patched before has its modules removed again when it's started with save patching disabled.
To use the modules outside of Clusterio the `npx clusteriohost create-scenario` command creates a scenario with the modules patched into it.
Note that the IPC between the game and the host is not available outside of Clusterio, so modules that depend on it will not work as expected in such a scenario.
+77 -20
View File
@@ -16,17 +16,40 @@ The easiest way to do this is to run `npx clusteriocontroller bootstrap create-c
Lists all hosts.
### Generate token
Hosts are added to the cluster by running the installer on the computer the host runs on and approving the code it shows, see [Setting up remote hosts](/README.md#setting-up-remote-hosts).
Listing, approving and rejecting hosts waiting for approval needs the `core.host.approve` permission.
ctl> host generate-token [--id <host id>]
### List pending hosts
Generate a token for a host. `--id` is the host id.
ctl> host pending
### Create config
Lists hosts waiting for approval with the code they show, the name they asked for, the address they connected from and how long they have been waiting.
A request goes away when its host stops waiting, and after the time set by `controller.host_pairing_timeout`, an hour by default, after which the host asks again with a new code.
ctl> host create-config [--id <host id>] [--name <host name>] [--generate-token <bool>] [--output <path to output>]
### Approve host
Create a host config. `--id` id the host id. `--name` is the host name. `--generate-token` decides whether to also generate a token, defaulting to `0`. `--output` is the filename for the output of the config; Default is "config-host.json", "-" for stdout.
ctl> host approve <code> [--name <host name>]
Approves the host showing the code and prints the id of the new host.
The code is not case sensitive and the dash can be left out.
`--name` gives the host a different name than the one it asked for.
Approving gives the host full access to the cluster, check with `host pending` that the name and address match the computer you are setting up.
### Reject host
ctl> host reject <code>
Rejects the host showing the code, the host then exits with an error.
### Revoke host
ctl> host revoke <host>
Specify the host by name or ID.
Removes the host from the cluster, disconnects it and makes its token stop working.
Instances assigned to the host become unassigned, their files stay on the computer.
Requires the `core.host.revoke` permission.
To add the computer back to the cluster, start the host again and approve it.
### Update Clusterio
@@ -56,15 +79,16 @@ Marks the host as requiring a restart, the same way a plugin update does. Use it
### List Instances
ctl> instance list
ctl> instance list [--templates]
List instances.
Templates are only listed if `--templates` is passed.
### Create instance
ctl> instance create <name> [--id <instance id>]
ctl> instance create <name> [--id <instance id>] [--from <instance id>] [--template <instance>] [--as-template]
Create a new instance. `name` is the instance name. `--id` is the optional id.
Create a new instance. `name` is the instance name. `--id` is the optional id. `--from` copies the config of an existing instance. `--template` makes the new instance inherit all config values from the given template. `--as-template` creates the instance as a template, see `instance.is_template` in the [configuration docs](configuration.md). Use `--as-template --from <instance>` to make a template out of an existing instance.
### Kill instance
@@ -97,7 +121,7 @@ Most notably Clusterio records wall clock time while the save time is recorded i
ctl> instance config list <instance>
List config values for an instance. `instance` is the instance whose config values to list.
List config values for an instance. `instance` is the instance whose config values to list. Values inherited from a template are marked with `(inherited)`, object fields also list the properties set on the instance.
#### Set config value
@@ -109,11 +133,17 @@ Set field in instance config. `instance` is instance to set. `field` is field to
To be written.
#### Inherit config value
ctl> instance config inherit <instance> <field> [prop]
Make a field in the instance config take its value from the template again, undoing a previous `set`. `instance` is the instance to update, `field` is the field to inherit, `prop` limits it to one property of an object field.
#### Edit instance config
ctl> instance config edit <instance> [editor]
Edit instance config in a text editor. Once file is saved and editor has been exited, config values will be read back in. `instance` is the instance to edit, `editor` is the editor to use. If `editor` is not given, clusterioctl will read from the `EDITOR` or `VISUAL` enviroment variables.
Edit instance config in a text editor. Once file is saved and editor has been exited, changed config values will be read back in. `instance` is the instance to edit, `editor` is the editor to use. If `editor` is not given, clusterioctl will read from the `EDITOR` or `VISUAL` enviroment variables.
### Assign instance to host
@@ -270,9 +300,11 @@ To be written.
## Plugins (Remote)
By default, remote updates are enabled unless you selected "no" during installation.
Remote updates allow ctl and the web UI to update plugins on machines within the cluster.
However, remote installations are disabled by default to prevent the risk of arbitrary code execution through the installation of malicious npm packages onto the machine. See below if you wish to enable this feature.
The installer asks "Allow remote updates and plugin installs from the web interface?" and the answer sets `allow_remote_updates`, `allow_plugin_updates` and `allow_plugin_install` together on that machine, pass `--remote-npm` or `--no-remote-npm` to answer it from the command line.
Remote updates allow ctl and the web UI to update Clusterio and plugins on machines within the cluster.
Remote installs allow them to install any package from npm onto the machine, which is a risk of arbitrary code execution through malicious npm packages, so only allow them if you trust everyone with access to install plugins.
In a config created without the installer remote updates are enabled and remote installs are disabled by default.
See below for changing these settings after installing.
### List
@@ -283,10 +315,15 @@ This command lists all plugins currently known to the remote. Newly installed pl
### Update
ctl> controller plugin update <name> [--restart]
ctl> host plugin update <host> <name> [--restart]
ctl> controller plugin update <name>[@<version>] [--restart]
ctl> host plugin update <host> <name>[@<version>] [--restart]
If remote plugin updates are enabled on the remote, this will execute `npm update --save <name>`. Changes will not be applied until the remote is restarted. If the `--restart` flag is provided, the remote will automatically restart once the update is complete.
If remote plugin updates are enabled on the remote, this will execute `npm update --save <name>`, which stays within the version range saved in the remote's `package.json`.
To update to a specific version, including one outside that range, pass it as `<name>@<version>`, for example `@clusterio/plugin-example@2.1.0`.
The version must be published on npm, and the remote then executes `npm install --save <name>@<version>`.
The plugin must already be installed on the remote, and only remote plugin updates need to be enabled.
Changes will not be applied until the remote is restarted.
If the `--restart` flag is provided, the remote will automatically restart once the update is complete.
### Update All
@@ -297,10 +334,30 @@ Updates the Clusterio package and all plugins installed from npm in a single `np
### Install
ctl> controller plugin install <name> [--restart]
ctl> host plugin install <host> <name> [--restart]
ctl> controller plugin install <name>[@<version>] [--restart]
ctl> host plugin install <host> <name>[@<version>] [--restart]
If remote plugin installs are enabled on the remote, this will execute `npm install --save <name>`.
Pass `<name>@<version>` to install a specific version, for example to match the version used by the rest of the cluster.
The package, and the version if given, must be published on npm.
Changes will not be applied until the remote is restarted.
If the `--restart` flag is provided, the remote will automatically restart once the update is complete.
### Search
ctl> controller plugin search [query] [--page <n>] [--page-size <n>]
Searches the npm registry for packages tagged with the clusterio-plugin keyword.
The search is done by the controller and does not require remote installs to be enabled.
The web UI has a plugin browser on the plugins page that uses the same search.
### Concurrent Operations
Only one npm operation runs at a time on a machine.
A standalone controller and host share their working directory, so the operation holds the `npm-operation.lock` file there while npm runs.
Other installs and updates on the same machine or installation are rejected with "Another package operation is already in progress on this machine" until it finishes.
A lock file left behind by a process that is no longer running is removed automatically.
If remote plugin installs are enabled on the remote, this will execute `npm install --save <name>`. Changes will not be applied until the remote is restarted. If the `--restart` flag is provided, the remote will automatically restart once the update is complete.
### Enable/Disable Updates
+14
View File
@@ -19,6 +19,20 @@ Use [contributing](contributing.md) for setup and validation commands.
Check supported configurations and compatibility policy; a merged revision or earlier clean review is not proof of correctness.
## Surface overlapping responsibilities
- For substantial new or replaced functionality, check whether existing code or test helpers already serve the same responsibility, including outside the changed files.
Search by behavior, then verify the implementations and their callers; nearby code is not necessarily the preferred shared implementation.
- Compare contracts, callers, and constraints, not just names or similar code.
Distinguish shared behavior and general correctness requirements from differences that may justify separate implementations.
- Surface significant overlap to the developer with source locations for both implementations, the duplicated responsibility, its consequences, and any legitimate differences.
Explain maintenance or consistency risks without presenting them as reproduced runtime defects.
- Recommend an approach based on the evidence, including alternatives, tradeoffs, and affected callers or tests.
Leave the architectural decision to the developer unless already authorized; do not silently consolidate the implementations or dismiss the overlap as intentional.
- Recheck after substantial fixes, especially when shared functionality is replaced, bypassed, or overridden.
Keep unresolved design questions in the candidate ledger and final report, and record the developer's decision when provided.
## Exercise the boundaries
- Write a small matrix of relevant states and transitions before probing.
+1 -1
View File
@@ -63,7 +63,7 @@ This creates a private key file named `key.pem` and a public certificate file na
npx clusteriocontroller config set controller.tls_certificate cert.pem
npx clusteriocontroller config set controller.tls_private_key key.pem
Additionally you will need copy the `cert.pem` file to all of the computers you want to set up as hosts and configure node to trust it by setting the `NODE_EXTRA_CA_CERTS` environment variable to a path to this file before the host is started. You can do this by passing `--self-signed-cert path/to/cert` to the installer when setting up a host, or manually by editing run-host.cmd, run-host.sh and/or clusteriohost.service by adding the following line next to the one setting `NODE_OPTIONS`:
Additionally you will need copy the `cert.pem` file to all of the computers you want to set up as hosts and configure node to trust it by setting the `NODE_EXTRA_CA_CERTS` environment variable to a path to this file before the host is started. You can do this by passing `--self-signed-cert path/to/cert` to the installer when setting up a host, which also makes the installer trust the certificate when it asks the controller to approve the host, or manually by editing run-host.cmd, run-host.sh and/or clusteriohost.service by adding the following line next to the one setting `NODE_OPTIONS`:
**For run-host.cmd**
+106 -5
View File
@@ -16,6 +16,7 @@ Each entrypoint of a plugin exports a function that attaches handlers to hooks c
- [Defining Link Messages](#defining-link-messages)
- [Sending Link Messages](#sending-link-messages)
- [Handling connection events](#handling-connection-events)
- [Web Pages for a Single Resource](#web-pages-for-a-single-resource)
- [Collecting Statistics](#collecting-statistics)
- [Adding Custom Commands to clusterioctl](#adding-custom-commands-to-clusterioctl)
@@ -138,8 +139,8 @@ The following properties are recognized:
Object mapping ctl configuration fields to `FieldDefinition` objects.
**webEntrypoint**:
Optional Module Federation exposure key for the web entrypoint, such as `"./web"`.
It must match the key in the plugin's webpack `exposes` configuration; it is not a Node.js file path.
Optional key of the web entrypoint in the plugin's web bundle, this should be `"./web"`.
The `clusterioPlugin` Vite plugin exposes `web/index.tsx` (or `.ts`, `.jsx`, `.js`) under this key; it is not a Node.js file path.
**migrationEntrypoint**:
Path to a Node.js module relative to the plugin directory which exports the migration entrypoint function for this plugin.
@@ -154,6 +155,10 @@ The following properties are recognized:
Array of request and event classes for this plugin.
See guide for [defining link messages](#defining-link-messages) below.
**errors**:
Array of error classes for this plugin.
See [Request errors](#request-errors).
**permissions**:
Array of permission definitions for this plugin.
See [Plugin Permissions](#plugin-permissions)
@@ -314,6 +319,7 @@ export const plugin = {
```
Enabling the plugin on an instance that has the feature turned off then fails config validation, so the mistake is caught when the config is edited instead of when the instance starts.
Instances that have the feature turned off get the plugin disabled by default when it is added.
Other constraints on config fields are expressed with a `validator` on the field definition, which throws when the value is not acceptable.
Use `dependsOn` when the check involves other fields so that the validator runs when those change too.
@@ -501,6 +507,25 @@ Data out from Factorio does not have the same limits as data into Factorio, RCON
For `send_json` it's possible that payloads greater than 4kB are received after payloads that were sent at a later point in time.
For RCON, commands longer than 50 characters may end up being executed after shorter commands sent after it.
### Auto pause
Each RCON command makes a paused game run one tick, so commands sent on a timer keep a server with `auto_pause` on running while nobody is online.
`instance.autoPaused` is true while the instance was started with `auto_pause` on and no players are online, and the `autoPauseChanged` hook is invoked with the new value when it changes.
Skip periodic commands while it is true:
```js
instance.hooks.start.attach(plugin.name, async () => {
timer = setInterval(() => {
if (!instance.autoPaused) {
instance.sendRcon("/sc my_plugin.update()").catch(handleError);
}
}, 1000);
});
```
Commands sent in response to events from elsewhere in the cluster are fine, they cost one tick each.
A game paused with `game.tick_paused` does not run ticks for commands.
### Communicating over UDP
Instances can optionally exchange UDP packets with the Factorio server for low latency fire-and-forget communication, using the `--enable-lua-udp` option of Factorio.
@@ -641,16 +666,16 @@ Registration rejects control-origin messages with no permission declaration.
A callback receives `(user, message)`, where `message` is the raw wire envelope.
It runs on the controller before destination payload validation and deserialization, so `message.data` is untrusted and may be missing or malformed.
Check any required payload shape before inspecting fields, and use `user.checkPermission(...)` or throw `PermissionError` for a normal denial.
Check any required payload shape before inspecting fields, and use `user.checkPermission(...)` or throw a `ClientError` such as `PermissionError` for a normal denial.
For example, this checks the payload shape needed to choose a permission; the destination schema still validates the full payload:
```js
import { PermissionError } from "@clusterio/lib";
import { ValidationError } from "@clusterio/lib";
function checkFrobnicatePermission(user, message) {
const data = message.data;
if (!data || typeof data !== "object" || typeof data.special !== "boolean") {
throw new PermissionError("Expected a boolean special field");
throw new ValidationError("Expected a boolean special field");
}
user.checkPermission(data.special ? "foo_frobber.special" : "foo_frobber.frobnicate");
}
@@ -740,6 +765,54 @@ connection.sendTo({ instanceId: 123 }, message);
connection.sendTo({ controlId: 123 }, message);
```
### Request errors
An error thrown by a request handler is sent back to the requester, and the promise returned by `.sendTo()` rejects with an instance of the same class.
The error classes follow the HTTP status codes:
```
Error
├── ClientError (4xx)
│ ├── ValidationError (400)
│ │ ├── InvalidValueError
│ │ └── InvalidFieldError
│ ├── ForbiddenError (403)
│ │ ├── PermissionError
│ │ └── InvalidAccessError
│ ├── NotFoundError (404)
│ ├── TimeoutError (408)
│ └── ConflictError (409)
└── ServerError (5xx)
├── InternalServerError (500)
└── UnavailableError (503)
```
Throw a `ClientError` subclass when the request can't be done as asked, and `UnavailableError` when the host or instance that should handle it is not connected or not running, for example:
```js
controller.handle(messages.Frobnicate, async request => {
if (!frobnicators.has(request.id)) {
throw new lib.NotFoundError(`Frobnicator ${request.id} does not exist`);
}
// ...
});
```
Errors of a class that is not registered are logged as unexpected by the side that threw them and are received as `InternalServerError`.
Registered errors are not logged.
The message and stack trace of the original error are kept in both cases.
Plugins can define their own error classes by listing them in `errors` of the plugin declaration.
They need a static `plugin` property set to the plugin name and must take the message as the first constructor argument:
```js
export class FrobnicatorJammed extends lib.ConflictError {
static plugin = "foo_frobber";
}
```
A subclass that is not registered is received as its closest registered parent class.
### Handling connection events
There are a few connection related events that plugins neeed to repsond to in order to avoid data loss and connection problems.
@@ -771,6 +844,34 @@ This means that if your plugin sends a lot of events or requests, they can end u
To avoid this you should be throtteling and/or stopping your requests/events after `drop` has been raised, and continue back as normal when `resume` is raised.
## Web Pages for a Single Resource
A web page showing one resource from a subscription has more to handle than the resource itself.
It can still be loading, it might not exist, it can be deleted while the page is open, and the subscription is rejected if the user lacks the permission for it.
`useResourceState` and `ResourceState` from `@clusterio/web_ui` show these the same way as the pages for hosts, instances and the other built in resources.
```tsx
function FrobberPage() {
const id = Number(useParams().id);
const [frobbers, synced, error] = useSyncExternalStore(subscribe, () => frobberSubscriber.getSnapshot());
const frobber = frobbers.get(id);
const state = useResourceState(frobber, synced, error, id);
const nav = [{ name: "Frobbers", path: "/frobbers" }, { name: frobber?.name ?? String(id) }];
if (!frobber) {
return <ResourceState state={state} type="Frobber" id={id} nav={nav} listPath="/frobbers" error={error} />;
}
return <PageLayout nav={nav}>{frobber.name}</PageLayout>;
}
```
`useResourceState` returns `"ok"` when there is a value and otherwise `"loading"`, `"missing"`, `"deleted"` or `"error"`.
It tells deleted from missing by remembering if it has seen a value for that id.
`ResourceState` renders the whole page for the states without a value, with a link back to the page at `listPath`.
The `useHost`, `useInstance`, `useUser`, `useRole` and `useModPack` hooks return `[value, synced, error]` so they can be used the same way.
## Collecting Statistics
Clusterio comes with its own Prometheus client implementation, one part due to Not Invented Here and another part due to collectors in prom-client being difficult to get to work nicely with collecting data from plugins optionally loaded at runtime on different computers.
-504
View File
@@ -1,504 +0,0 @@
import {
defineConfig,
globalIgnores,
} from "eslint/config";
import typescriptEslint from "@typescript-eslint/eslint-plugin";
import node from "eslint-plugin-node";
import globals from "globals";
import tsParser from "@typescript-eslint/parser";
import js from "@eslint/js";
import stylistic from "@stylistic/eslint-plugin";
import {
FlatCompat,
} from "@eslint/eslintrc";
const compat = new FlatCompat({
baseDirectory: import.meta.dirname,
recommendedConfig: js.configs.recommended,
allConfig: js.configs.all,
});
export default defineConfig([{
plugins: {
"@typescript-eslint": typescriptEslint,
"@stylistic": stylistic,
node,
},
languageOptions: {
globals: {
...globals.node,
...globals.commonjs,
},
},
"rules": {
"@stylistic/indent": ["error", "tab", {
"ignoredNodes": [
"JSXFragment", // TODO: Linter crashes if this is removed
],
"ignoreComments": true,
"SwitchCase": 1, // Indent case statements one level from switch
}],
"accessor-pairs": "error",
"array-bracket-newline": "off",
"array-bracket-spacing": ["error", "never"],
"array-callback-return": "error",
"array-element-newline": "off",
"arrow-body-style": "error",
"arrow-parens": "off",
"arrow-spacing": ["error", {
"after": true,
"before": true,
}],
"block-scoped-var": "error",
"block-spacing": "error",
"brace-style": ["error", "1tbs", {
"allowSingleLine": true,
}],
"camelcase": "off",
"capitalized-comments": "off",
"class-methods-use-this": "off",
"comma-dangle": ["error", {
"arrays": "always-multiline",
"objects": "always-multiline",
"imports": "always-multiline",
"exports": "always-multiline",
"functions": "only-multiline",
}],
"comma-spacing": "error",
"comma-style": ["error", "last"],
"complexity": ["error", { "max": 25 }],
"computed-property-spacing": ["error", "never"],
"consistent-return": "error",
"consistent-this": "error",
"curly": "error",
"default-case": "error",
"default-case-last": "error",
"default-param-last": "off",
"dot-location": ["error", "property"],
"dot-notation": ["error", {
"allowPattern": "^[a-z]+(_[a-z]+)*$",
}],
"eol-last": "error",
"eqeqeq": "error",
"func-call-spacing": "error",
"func-name-matching": "error",
"func-names": "off",
"func-style": ["error", "declaration", {
"allowArrowFunctions": true,
}],
"function-call-argument-newline": "off",
"function-paren-newline": "off",
"generator-star-spacing": ["error", {
"before": false,
"after": true,
}],
"grouped-accessor-pairs": "error",
"guard-for-in": "error",
"id-blacklist": "error",
"id-length": "off",
"id-match": "error",
"implicit-arrow-linebreak": "error",
"indent-legacy": "off",
"init-declarations": "off",
"jsx-quotes": "error",
"key-spacing": "error",
"keyword-spacing": "error",
"line-comment-position": "off",
"linebreak-style": ["error", "unix"],
"lines-around-comment": "off",
"lines-around-directive": "off",
"lines-between-class-members": ["error", "always", {
"exceptAfterSingleLine": true,
}],
"max-classes-per-file": "off",
"max-depth": "error",
"max-len": ["error", {
"code": 120,
}],
"max-lines": "off",
"max-lines-per-function": "off",
"max-nested-callbacks": "error",
"max-params": "off",
"max-statements": "off",
"max-statements-per-line": "off",
"multiline-comment-style": ["error", "separate-lines"],
"multiline-ternary": "off",
"new-cap": ["error", {
"capIsNewExceptions": ["StringEnum", "StringKey"],
"capIsNewExceptionPattern": "^Type\\.",
"newIsCapExceptions": ["constructor"],
}],
"new-parens": "error",
"newline-after-var": "off",
"newline-before-return": "off",
"newline-per-chained-call": "off",
"no-alert": "error",
"no-array-constructor": "error",
"no-await-in-loop": "off",
"no-bitwise": "off",
"no-buffer-constructor": "error",
"no-caller": "error",
"no-catch-shadow": "off",
"no-confusing-arrow": "error",
"no-console": "error",
"no-constant-condition": ["error", {
"checkLoops": false,
}],
"no-constructor-return": "error",
"no-continue": "off",
"no-div-regex": "error",
"no-duplicate-imports": "error",
"no-else-return": "error",
"no-empty-function": "off",
"no-eq-null": "error",
"no-eval": "error",
"no-extend-native": "error",
"no-extra-bind": "error",
"no-extra-label": "error",
"no-extra-parens": "off",
"no-floating-decimal": "error",
"no-implicit-coercion": "error",
"no-implicit-globals": "error",
"no-implied-eval": "error",
"no-inline-comments": "off",
"no-invalid-this": "error",
"no-iterator": "error",
"no-label-var": "error",
"no-lone-blocks": "error",
"no-lonely-if": "error",
"no-loop-func": "error",
"no-magic-numbers": "off",
"no-mixed-operators": ["error", {
"groups": [
["&", "|", "^", "~", "<<", ">>", ">>>"],
["==", "!=", "===", "!==", ">", ">=", "<", "<="],
["in", "instanceof"],
],
}],
"no-multi-assign": "error",
"no-multi-spaces": "error",
"no-multi-str": "error",
"no-multiple-empty-lines": "error",
"no-native-reassign": "error",
"no-negated-condition": "off",
"no-negated-in-lhs": "error",
"no-nested-ternary": "error",
"no-new": "error",
"no-new-func": "error",
"no-new-object": "error",
"no-new-wrappers": "error",
"no-octal-escape": "error",
"no-param-reassign": "off",
"no-plusplus": ["error", {
"allowForLoopAfterthoughts": true,
}],
"no-proto": "error",
"no-restricted-exports": "error",
"no-restricted-globals": "error",
"no-restricted-imports": "error",
"no-restricted-properties": "error",
"no-restricted-syntax": "error",
"no-return-assign": "error",
"no-return-await": "off",
"no-script-url": "error",
"no-self-compare": "error",
"no-sequences": "error",
"no-shadow": ["error", {
"allow": ["Event", "Request", "yargs"],
}],
"no-spaced-func": "error",
"no-tabs": "off",
"no-template-curly-in-string": "error",
"no-ternary": "off",
"no-throw-literal": "error",
"no-trailing-spaces": "error",
"no-undef-init": "error",
"no-undefined": "off",
"no-unmodified-loop-condition": "error",
"no-unneeded-ternary": "error",
"no-unused-expressions": "error",
"no-use-before-define": "off",
"no-useless-backreference": "error",
"no-useless-call": "error",
"no-useless-computed-key": "error",
"no-useless-concat": "error",
"no-useless-constructor": "off",
"no-useless-rename": "error",
"no-useless-return": "error",
"no-var": "error",
"no-void": ["error", {
"allowAsStatement": true,
}],
"no-warning-comments": "off",
"no-whitespace-before-property": "error",
"nonblock-statement-body-position": "error",
"object-curly-newline": "error",
"object-curly-spacing": "off",
"object-shorthand": "off",
"one-var": "off",
"one-var-declaration-per-line": "error",
"operator-assignment": "off",
"operator-linebreak": ["error", "after", {
"overrides": {
"?": "before",
":": "before",
"||": "before",
"&&": "before",
"??": "before",
},
}],
"padded-blocks": "off",
"padding-line-between-statements": "error",
"prefer-arrow-callback": "error",
"prefer-const": "off",
"prefer-destructuring": "off",
"prefer-exponentiation-operator": "error",
"prefer-named-capture-group": "off",
"prefer-numeric-literals": "error",
"prefer-object-spread": "error",
"prefer-promise-reject-errors": "error",
"prefer-reflect": "off",
"prefer-regex-literals": "error",
"prefer-rest-params": "error",
"prefer-spread": "error",
"prefer-template": "error",
"quote-props": "off",
"quotes": ["error", "double", {
"avoidEscape": true,
}],
"radix": ["error", "always"],
"require-atomic-updates": "off",
"require-await": "off",
"require-jsdoc": "off",
"require-unicode-regexp": "off",
"rest-spread-spacing": ["error", "never"],
"semi": "error",
"semi-spacing": "error",
"semi-style": "off",
"sort-imports": "off",
"sort-keys": "off",
"sort-vars": "off",
"space-before-blocks": "error",
"space-before-function-paren": "off",
"space-in-parens": ["error", "never"],
"space-infix-ops": "off",
"space-unary-ops": "error",
"spaced-comment": ["error", "always", {
"line": {
"markers": ["/"],
},
}],
"switch-colon-spacing": "error",
"symbol-description": "error",
"template-curly-spacing": ["error", "never"],
"template-tag-spacing": "error",
"unicode-bom": ["error", "never"],
"vars-on-top": "error",
"wrap-iife": "error",
"wrap-regex": "off",
"yield-star-spacing": "error",
"yoda": ["error", "never", {
"exceptRange": true,
}],
"node/callback-return": "off",
"node/exports-style": "off",
"node/file-extension-in-import": "off",
"node/global-require": "off",
"node/handle-callback-err": "off",
"node/no-callback-literal": "off",
"node/no-deprecated-api": "off",
"node/no-exports-assign": "off",
"node/no-extraneous-import": "off",
"node/no-extraneous-require": "off",
"node/no-missing-import": "off",
"node/no-missing-require": "off",
"node/no-mixed-requires": "off",
"node/no-new-require": "off",
"node/no-path-concat": "off",
"node/no-process-env": "off",
"node/no-process-exit": "off",
"node/no-restricted-require": "off",
"node/no-sync": "error",
"node/no-unpublished-bin": "off",
"node/no-unpublished-import": "off",
"node/no-unpublished-require": "off",
"node/no-unsupported-features/es-builtins": "off",
"node/no-unsupported-features/es-syntax": "off",
"node/no-unsupported-features/node-builtins": "off",
"node/prefer-global/buffer": "off",
"node/prefer-global/console": "off",
"node/prefer-global/process": "off",
"node/prefer-global/text-decoder": "off",
"node/prefer-global/text-encoder": "off",
"node/prefer-global/url-search-params": "off",
"node/prefer-global/url": "off",
"node/prefer-promises/dns": "off",
"node/prefer-promises/fs": "off",
"node/process-exit-as-throw": "off",
"node/shebang": "off",
},
}, {
files: ["{packages,plugins}/**/*.ts?(x)"],
languageOptions: {
parser: tsParser,
},
extends: compat.extends("plugin:@typescript-eslint/recommended"),
"rules": {
"@typescript-eslint/no-explicit-any": "off",
"@typescript-eslint/no-useless-constructor": "error",
"@typescript-eslint/no-unused-vars": "off",
"@typescript-eslint/triple-slash-reference": "off",
"@typescript-eslint/no-shadow": ["error", {
"allow": ["Event", "Request", "yargs"],
}],
// Allow require imports where necessary (dynamic loading, webpack configs, etc.)
"@typescript-eslint/no-require-imports": "off",
"prefer-const": "off",
"no-shadow": "off",
},
}, {
files: ["test/**/*.js", "plugins/*/test/**/*.js"],
languageOptions: {
globals: {
...globals.mocha,
},
sourceType: "module",
parserOptions: {},
},
"rules": {
"prefer-arrow-callback": "off",
"no-invalid-this": "off",
"node/no-unpublished-import": "off",
"node/no-unpublished-require": "off",
},
}, {
files: [
"packages/web_ui/src/**/*.ts?(x)",
"packages/lib/browser.ts",
"packages/create/templates/*/web/**/*.{t,j}s?(x)",
"{packages,plugins}/*/web/**/*.ts?(x)",
],
languageOptions: {
globals: {
...globals.browser,
},
"sourceType": "module",
parserOptions: {
"ecmaFeatures": {
"jsx": true,
},
},
},
"rules": {
"node/callback-return": "off",
"node/exports-style": "off",
"node/file-extension-in-import": "off",
"node/global-require": "off",
"node/handle-callback-err": "off",
"node/no-callback-literal": "off",
"node/no-deprecated-api": "off",
"node/no-exports-assign": "off",
"node/no-extraneous-import": "off",
"node/no-extraneous-require": "off",
"node/no-missing-import": "off",
"node/no-missing-require": "off",
"node/no-mixed-requires": "off",
"node/no-new-require": "off",
"node/no-path-concat": "off",
"node/no-process-env": "off",
"node/no-process-exit": "off",
"node/no-restricted-require": "off",
"node/no-sync": "error",
"node/no-unpublished-bin": "off",
"node/no-unpublished-import": "off",
"node/no-unpublished-require": "off",
"node/no-unsupported-features/es-builtins": "off",
"node/no-unsupported-features/es-syntax": "off",
"node/no-unsupported-features/node-builtins": "off",
"node/prefer-global/buffer": "off",
"node/prefer-global/console": "off",
"node/prefer-global/process": "off",
"node/prefer-global/text-decoder": "off",
"node/prefer-global/text-encoder": "off",
"node/prefer-global/url-search-params": "off",
"node/prefer-global/url": "off",
"node/prefer-promises/dns": "off",
"node/prefer-promises/fs": "off",
"node/process-exit-as-throw": "off",
"node/shebang": "off",
},
}, {
files: ["packages/create/templates/**/*.?(c){t,j}s?(x)"],
"rules": {
"node/no-missing-require": "off",
"node/no-missing-import": "off",
"@stylistic/indent": "off",
"spaced-comment": "off",
"comma-spacing": "off",
"max-len": "off",
"indent": "off",
},
}, globalIgnores(["**/dist/", "dist/"])]);
+4 -14
View File
@@ -15,10 +15,9 @@
"cover": "c8 pnpm test",
"fast-cover": "FAST_TEST=y c8 pnpm test",
"ci-cover": "c8 --reporter=lcovonly pnpm test",
"lint": "eslint packages plugins test",
"lint-fix": "eslint --fix packages plugins test",
"lint": "oxlint packages plugins test",
"lint-fix": "oxlint --fix packages plugins test",
"clean": "node ./clean.js",
"docs": "typedoc",
"bundle-dependencies": "bundle-dependencies"
},
"c8": {
@@ -29,7 +28,7 @@
]
},
"engines": {
"node": ">=22"
"node": "^22.18.0 || >=24.2.0"
},
"workspaces": [
"packages/*",
@@ -43,22 +42,13 @@
"@clusterio/host": "workspace:^",
"@clusterio/lib": "workspace:^",
"@clusterio/web_ui": "workspace:^",
"@eslint/eslintrc": "^3.3.7",
"@eslint/js": "^10.0.1",
"@stylistic/eslint-plugin": "^5.10.0",
"@swc/core": "catalog:",
"@types/node": "catalog:",
"c8": "^12.0.0",
"@typescript-eslint/eslint-plugin": "^8.71.2",
"@typescript-eslint/parser": "^8.71.2",
"eslint": "^10.13.0",
"eslint-plugin-node": "^11.1.0",
"express": "catalog:",
"form-data": "^4.0.6",
"globals": "^17.13.0",
"jsonwebtoken": "catalog:",
"mocha": "^11.8.0",
"typedoc": "^0.28.20",
"oxlint": "^1.88.0",
"typescript": "catalog:",
"yargs": "catalog:",
"yauzl": "catalog:",
+1 -1
View File
@@ -1,2 +1,2 @@
webpack.config.js
vite.config.js
dist/browser
+17 -24
View File
@@ -23,13 +23,14 @@ import util from "util";
import winston from "winston";
import DailyRotateFile from "winston-daily-rotate-file";
import jwt from "jsonwebtoken";
// homebrew modules
// Reduce startup time by lazy compiling schemas.
import "./src/set_lazy_schema_compilation.js";
import * as lib from "@clusterio/lib";
import { ConsoleTransport, levels, logger } from "@clusterio/lib";
import Controller from "./src/Controller.js";
import type Controller from "./src/Controller.js";
import UserManager from "./src/UserManager.js";
import UserRecord from "./src/UserRecord.js";
import packageConfig from "@clusterio/controller/package.json" with { type: "json" };
@@ -147,15 +148,15 @@ async function handleBootstrapCommand(
path.join(databaseDirectory, "roles.json"),
);
const roles = new lib.SubscribableDatastore(...await new lib.JsonIdDatastoreProvider(
const roles = await lib.SubscribableDatastore.fromProvider(new lib.JsonIdDatastoreProvider(
path.join(databaseDirectory, "roles.json"),
lib.Role.fromJSON.bind(lib.Role),
).bootstrap());
));
const users = new lib.SubscribableDatastore(...await new lib.JsonIdDatastoreProvider(
const users = await lib.SubscribableDatastore.fromProvider(new lib.JsonIdDatastoreProvider(
path.join(databaseDirectory, "users.json"),
UserRecord.fromJSON.bind(UserRecord),
).bootstrap());
));
const userManager = new UserManager(users, roles, controllerConfig);
@@ -186,14 +187,6 @@ async function handleBootstrapCommand(
// eslint-disable-next-line no-console
console.log(userManager.signUserToken(user));
} else if (subCommand === "generate-host-token") {
// eslint-disable-next-line no-console
console.log(jwt.sign(
{ aud: "host", host: args.id },
// Auth secret is never null after startup
Buffer.from(controllerConfig.get("controller.auth_secret")!, "base64")
));
} else if (subCommand === "create-ctl-config") {
let admin = userManager.getByName(args.name);
if (!admin) {
@@ -203,6 +196,7 @@ async function handleBootstrapCommand(
}
let ctlConfig = new lib.CtlConfig("control");
const { default: Controller } = await import("./src/Controller.js");
ctlConfig.set("ctl.controller_url", Controller.calculateControllerUrl(controllerConfig));
ctlConfig.set(
"ctl.controller_token",
@@ -215,7 +209,7 @@ async function handleBootstrapCommand(
console.log(content);
} else {
logger.info(`Writing ${args.output}`);
await lib.safeOutputFile(args.output, content, { mode: 0o600 });
await lib.atomicOverwriteFile(args.output, content, { mode: 0o600 });
}
}
}
@@ -322,9 +316,6 @@ async function initialize(): Promise<InitializeParameters> {
.command("generate-user-token <name>", "Generate authentication token for the given user", yargs => {
yargs.positional("name", { describe: "Name of the user", type: "string" });
})
.command("generate-host-token <id>", "Generate authentication token for the given host", yargs => {
yargs.positional("id", { describe: "ID of the host", type: "number" });
})
.command("create-ctl-config <name>", "Create clusterioctl config for the given user", yargs => {
yargs.option("output", {
describe: "Path to output config (- for stdout)", type: "string",
@@ -400,7 +391,8 @@ async function initialize(): Promise<InitializeParameters> {
}
logger.info(`Loading available plugins from ${args.pluginList}`);
let pluginList = await lib.loadPluginList(args.pluginList);
const skippedNpmPlugins = new Map<string, string>();
let pluginList = await lib.loadPluginList(args.pluginList, true, skippedNpmPlugins);
// If the command is plugin management we don't try to load plugins
if (command === "plugin") {
@@ -415,7 +407,7 @@ async function initialize(): Promise<InitializeParameters> {
}
logger.info("Loading Plugin info");
const pluginInfos = await lib.loadPluginInfos(pluginList);
const pluginInfos = await lib.loadPluginInfos(pluginList, skippedNpmPlugins);
lib.registerPluginMessages(pluginInfos);
lib.registerPluginPermissions(pluginInfos);
lib.addPluginConfigFields(pluginInfos);
@@ -474,13 +466,14 @@ async function startup() {
return;
}
controller = new Controller(
clusterLogger,
pluginInfos,
// Loaded here as the server is not needed by the other commands
const { default: Controller } = await import("./src/Controller.js");
controller = await Controller.fromConfig(
controllerConfig,
pluginInfos,
clusterLogger,
Boolean(args.canRestart),
Boolean(args.recovery),
...await Controller.bootstrap(controllerConfig)
);
// Refuse to start if there are no users loaded
+2
View File
@@ -5,7 +5,9 @@ export { default as ControlConnection } from "./src/ControlConnection.js";
export { default as HostConnection } from "./src/HostConnection.js";
export { default as InstanceRecord } from "./src/InstanceRecord.js";
export { default as InstanceManager } from "./src/InstanceManager.js";
export { default as HostManager } from "./src/HostManager.js";
export { default as HostRecord } from "./src/HostRecord.js";
export { default as HostPairingManager, HostPairingError } from "./src/HostPairingManager.js";
export { BaseControllerPlugin, ControllerHooks, type ControllerPluginContext } from "./src/BaseControllerPlugin.js";
export { default as UserManager } from "./src/UserManager.js";
export { default as UserRecord } from "./src/UserRecord.js";
+5 -7
View File
@@ -25,14 +25,14 @@
"type": "module",
"main": "dist/node/index.js",
"scripts": {
"prepare": "tsc --build && webpack-cli --mode production",
"prepare": "tsc --build && vite build",
"test": "echo \"Error: no test specified\" && exit 1"
},
"bin": {
"clusteriocontroller": "./dist/node/controller.js"
},
"engines": {
"node": ">=22"
"node": "^22.18.0 || >=24.2.0"
},
"dependencies": {
@@ -51,7 +51,6 @@
},
"devDependencies": {
"@clusterio/web_ui": "workspace:^",
"@swc/core": "catalog:",
"@types/busboy": "^1.5.5",
"@types/compression": "^1.8.1",
"@types/express": "catalog:",
@@ -61,13 +60,12 @@
"@types/semver": "catalog:",
"@types/ws": "catalog:",
"@types/yargs": "catalog:",
"antd": "catalog:",
"react": "catalog:",
"react-dom": "catalog:",
"react-router": "catalog:",
"react-router-dom": "catalog:",
"typescript": "catalog:",
"webpack": "catalog:",
"webpack-cli": "catalog:",
"webpack-dev-middleware": "^8.3.0",
"webpack-merge": "catalog:"
"vite": "catalog:"
}
}
+5 -5
View File
@@ -63,7 +63,7 @@ export default class BaseConnection extends lib.Link {
let { id } = request;
let modPack = this._controller.modPacks.get(id);
if (!modPack) {
throw new lib.RequestError(`Mod pack with ID ${id} does not exist`);
throw new lib.NotFoundError(`Mod pack with ID ${id} does not exist`);
}
return modPack;
}
@@ -71,11 +71,11 @@ export default class BaseConnection extends lib.Link {
async handleModPackGetDefaultRequest(): Promise<lib.ModPack> {
let id = this._controller.config.get("controller.default_mod_pack_id");
if (id === null) {
throw new lib.RequestError("Default mod pack not set on controller");
throw new lib.ConflictError("Default mod pack not set on controller");
}
let modPack = this._controller.modPacks.get(id);
if (!modPack) {
throw new lib.RequestError(`Default mod pack configured (${id}) does not exist`);
throw new lib.NotFoundError(`Default mod pack configured (${id}) does not exist`);
}
return modPack;
}
@@ -84,10 +84,10 @@ export default class BaseConnection extends lib.Link {
let filename = lib.ModInfo.filename(mod.name, mod.version);
let modInfo = this._controller.modStore.files.get(filename);
if (!modInfo) {
throw new lib.RequestError(`Mod ${filename} does not exist on controller`);
throw new lib.NotFoundError(`Mod ${filename} does not exist on controller`);
}
if (mod.sha1 && mod.sha1 !== modInfo.sha1) {
throw new lib.RequestError(`Mod ${filename} checksum does not match controller's checksum`);
throw new lib.ConflictError(`Mod ${filename} checksum does not match controller's checksum`);
}
return modInfo;
}
@@ -107,7 +107,7 @@ export class ControllerHooks extends lib.AsyncHookCollection {
* @param curr - The current value of the field.
* @param prev - The previous value of the field.
*/
// eslint-disable-next-line max-len
// eslint-disable-next-line @stylistic/max-len
readonly instanceConfigFieldChanged: lib.AsyncHook<[instance: InstanceRecord, field: string, curr: unknown, prev: unknown]>;
/**
@@ -165,7 +165,7 @@ export class ControllerHooks extends lib.AsyncHookCollection {
* The connection the event occured on.
* @param event - one of connect, drop, resume and close
*/
// eslint-disable-next-line max-len
// eslint-disable-next-line @stylistic/max-len
readonly hostConnectionEvent: lib.AsyncHook<[connection: HostConnection, event: "connect" | "drop" | "resume" | "close"]>;
/**
@@ -202,7 +202,7 @@ export class ControllerHooks extends lib.AsyncHookCollection {
* The connection the event occured on.
* @param event - one of connect, drop, resume, and close.
*/
// eslint-disable-next-line max-len
// eslint-disable-next-line @stylistic/max-len
readonly controlConnectionEvent: lib.AsyncHook<[connection: ControlConnection, event: "connect" | "drop" | "resume" | "close"]>;
/**
+126 -98
View File
@@ -8,6 +8,7 @@ import { logFilter, logger } from "@clusterio/lib";
import BaseConnection from "./BaseConnection.js";
import User from "./User.js";
import UserRecord from "./UserRecord.js";
import type InstanceRecord from "./InstanceRecord.js";
import * as routes from "./routes.js";
import Controller from "./Controller.js";
@@ -81,10 +82,11 @@ export default class ControlConnection extends BaseConnection {
this.handle(lib.ControllerConfigSetRequest, this.handleControllerConfigSetRequest.bind(this));
this.handle(lib.ControllerConfigSetFieldRequest, this.handleControllerConfigSetFieldRequest.bind(this));
this.handle(lib.ControllerConfigSetPropRequest, this.handleControllerConfigSetPropRequest.bind(this));
this.handle(lib.HostRevokeTokensRequest, this.handleHostRevokeTokensRequest.bind(this));
this.handle(lib.HostRevokeRequest, this.handleHostRevokeRequest.bind(this));
this.handle(lib.HostListRequest, this.handleHostListRequest.bind(this));
this.handle(lib.HostGenerateTokenRequest, this.handleHostGenerateTokenRequest.bind(this));
this.handle(lib.HostConfigCreateRequest, this.handleHostConfigCreateRequest.bind(this));
this.handle(lib.HostPairingListRequest, this.handleHostPairingListRequest.bind(this));
this.handle(lib.HostPairingApproveRequest, this.handleHostPairingApproveRequest.bind(this));
this.handle(lib.HostPairingRejectRequest, this.handleHostPairingRejectRequest.bind(this));
this.handle(lib.InstanceDetailsGetRequest, this.handleInstanceDetailsGetRequest.bind(this));
this.handle(lib.InstanceDetailsListRequest, this.handleInstanceDetailsListRequest.bind(this));
this.handle(lib.InstanceCreateRequest, this.handleInstanceCreateRequest.bind(this));
@@ -93,6 +95,7 @@ export default class ControlConnection extends BaseConnection {
this.handle(lib.InstanceConfigSetRequest, this.handleInstanceConfigSetRequest.bind(this));
this.handle(lib.InstanceConfigSetFieldRequest, this.handleInstanceConfigSetFieldRequest.bind(this));
this.handle(lib.InstanceConfigSetPropRequest, this.handleInstanceConfigSetPropRequest.bind(this));
this.handle(lib.InstanceConfigInheritFieldRequest, this.handleInstanceConfigInheritFieldRequest.bind(this));
this.handle(lib.InstanceAssignRequest, this.handleInstanceAssignRequest.bind(this));
this.handle(lib.InstanceSaveDetailsListRequest, this.handleInstanceSaveDetailsListRequest.bind(this));
this.handle(lib.InstanceRenameSaveRequest, controller.sendRequestToHostByInstanceId.bind(controller));
@@ -135,6 +138,8 @@ export default class ControlConnection extends BaseConnection {
this.handle(lib.PluginListRequest, this.handlePluginListRequest.bind(this));
this.handle(lib.PluginUpdateRequest, this.handlePluginUpdateRequest.bind(this));
this.handle(lib.PluginInstallRequest, this.handlePluginInstallRequest.bind(this));
this.handle(lib.PluginSearchRequest, this.handlePluginSearchRequest.bind(this));
this.handle(lib.PluginLatestVersionsRequest, this.handlePluginLatestVersionsRequest.bind(this));
this.handle(lib.UpdateAllRequest, this.handleUpdateAllRequest.bind(this));
this.handle(lib.DebugDumpWsRequest, this.handleDebugDumpWsRequest.bind(this));
}
@@ -153,7 +158,7 @@ export default class ControlConnection extends BaseConnection {
try {
this.checkPermission(message, entry);
} catch (err: any) {
this.connector.sendResponseError(new lib.ResponseError(err.message, err.code), message.src);
this.connector.sendResponseError(lib.Link.errorToResponse(err, false), message.src);
logger.audit(`Permission denied for ${message.name} by ${this.user.id} from ${this.connector.dst}`);
throw err;
}
@@ -195,12 +200,15 @@ export default class ControlConnection extends BaseConnection {
async handleControllerRestartRequest() {
if (!this._controller.canRestart) {
throw new lib.RequestError("Cannot restart, controller does not have a process monitor to restart it.");
throw new lib.ConflictError("Cannot restart, controller does not have a process monitor to restart it.");
}
if (await lib.packageOperationRunning()) {
throw new lib.ConflictError("Cannot restart while a plugin install or update is running on this machine.");
}
const downgrade = await this._controller.checkRestartDowngrade();
if (downgrade) {
const { installedVersion, runningVersion } = downgrade;
throw new lib.RequestError(
throw new lib.ConflictError(
`Cannot restart controller with installed Clusterio version ${installedVersion} because it ` +
`is older than running version ${runningVersion}. ` +
"Stop the controller before starting the older version manually."
@@ -243,49 +251,25 @@ export default class ControlConnection extends BaseConnection {
this._controller.config.setProp(field as keyof lib.ControllerConfigFields, prop, value, "control");
}
async handleHostRevokeTokensRequest(request: lib.HostRevokeTokensRequest) {
const host = this._controller.hosts.getMutable(request.hostId);
if (!host) {
throw new Error(`Unknown host id (${request.hostId})`);
}
host.tokenValidAfter = Math.floor(Date.now() / 1000);
const hostConnection = this._controller.wsServer.hostConnections.get(request.hostId);
if (hostConnection) {
hostConnection.connector.terminate();
}
this._controller.hosts.set(host);
async handleHostRevokeRequest(request: lib.HostRevokeRequest) {
await this._controller.hosts.revokeHost(request.hostId, this.user.name);
}
async handleHostListRequest(): Promise<lib.HostDetails[]> {
return [...this._controller.hosts.values()].map(host => host.toHostDetails());
}
async handleHostGenerateTokenRequest(message: lib.HostGenerateTokenRequest) {
let hostId = message.hostId;
if (hostId === undefined) {
hostId = Math.random() * 2**31 | 0;
}
return this._controller.generateHostToken(hostId);
async handleHostPairingListRequest(): Promise<lib.HostPairingDetails[]> {
this._controller.hostPairing.prune();
return [...this._controller.hostPairing.records.values()];
}
async handleHostConfigCreateRequest(request: lib.HostConfigCreateRequest) {
const hostConfig = new lib.HostConfig("host");
async handleHostPairingApproveRequest(request: lib.HostPairingApproveRequest) {
return this._controller.hostPairing.approve(request.code, request.name, this.user.name);
}
hostConfig.set("host.controller_url", this._controller.getControllerUrl());
if (request.id !== undefined) {
hostConfig.set("host.id", request.id);
}
if (request.name !== undefined) {
hostConfig.set("host.name", request.name);
}
if (request.generateToken) {
this.user.checkPermission("core.host.generate_token");
hostConfig.set("host.controller_token", this._controller.generateHostToken(hostConfig.get("host.id")));
}
return hostConfig.toJSON();
async handleHostPairingRejectRequest(request: lib.HostPairingRejectRequest) {
this._controller.hostPairing.reject(request.code, this.user.name);
}
async handleInstanceDetailsGetRequest(request: lib.InstanceDetailsGetRequest) {
@@ -298,18 +282,24 @@ export default class ControlConnection extends BaseConnection {
async handleInstanceCreateRequest(request: lib.InstanceCreateRequest) {
if ((request.config["instance.assigned_host"] ?? null) !== null) {
throw new lib.RequestError("instance.assigned_host must be set through the assign-host interface");
throw new lib.ValidationError("instance.assigned_host must be set through the assign-host interface");
}
const instanceConfig = new lib.InstanceConfig("controller");
if (request.cloneFromId) {
const baseInstance = this._controller.instances.get(request.cloneFromId);
if (!baseInstance) {
throw new lib.RequestError(`Instance with ID ${request.cloneFromId} does not exist`);
throw new lib.NotFoundError(`Instance with ID ${request.cloneFromId} does not exist`);
}
instanceConfig.update(baseInstance.config.toJSON(), false);
instanceConfig.set("instance.assigned_host", null); // New instances are unassigned
}
instanceConfig.update(request.config, false, "control");
this._controller.instances.checkInheritance(
instanceConfig, instanceConfig.get("instance.is_template"), instanceConfig.get("instance.template_id")
);
if (Object.hasOwn(request.config, "instance.template_id")) {
this._controller.instances.applyTemplate(instanceConfig, Object.keys(request.config));
}
await this._controller.instances.createInstance(instanceConfig);
}
@@ -325,12 +315,12 @@ export default class ControlConnection extends BaseConnection {
private static validateInstanceConfigSetField(fieldName: string) {
if (fieldName === "instance.assigned_host") {
throw new lib.RequestError("instance.assigned_host must be set through the assign-host interface");
throw new lib.ValidationError("instance.assigned_host must be set through the assign-host interface");
}
if (fieldName === "instance.id") {
// XXX is this worth implementing? It's race condition galore.
throw new lib.RequestError("Setting instance.id is not supported");
throw new lib.ValidationError("Setting instance.id is not supported");
}
}
@@ -350,17 +340,41 @@ export default class ControlConnection extends BaseConnection {
}
}
this.checkStagedInheritance(instance);
instance.config.commitStaging();
for (const [fieldName, fieldValue] of Object.entries(request.fields)) {
const props = typeof fieldValue === "object"
? Object.keys(fieldValue).filter(prop => fieldValue[prop] !== undefined)
: undefined;
this._controller.instances.overrideField(instance, fieldName, props);
}
await this._controller.instanceConfigUpdated(instance);
} finally {
instance.config.revertStaging();
}
}
private checkStagedInheritance(instance: InstanceRecord) {
const staged = instance.config.staging;
this._controller.instances.checkInheritance(
instance.config, staged.get("instance.is_template"), staged.get("instance.template_id")
);
}
async handleInstanceConfigSetFieldRequest(request: lib.InstanceConfigSetFieldRequest) {
const instance = this._controller.instances.getForRequest(request.instanceId);
ControlConnection.validateInstanceConfigSetField(request.field);
instance.config.set(request.field as keyof lib.InstanceConfigFields, request.value, "control");
const field = request.field as keyof lib.InstanceConfigFields;
ControlConnection.validateInstanceConfigSetField(field);
if (["instance.is_template", "instance.template_id"].includes(field)) {
try {
instance.config.stage(field, request.value, "control");
this.checkStagedInheritance(instance);
} finally {
instance.config.revertStaging();
}
}
instance.config.set(field, request.value, "control");
this._controller.instances.overrideField(instance, field);
await this._controller.instanceConfigUpdated(instance);
}
@@ -369,9 +383,18 @@ export default class ControlConnection extends BaseConnection {
let { field, prop, value } = request;
ControlConnection.validateInstanceConfigSetField(field);
instance.config.setProp(field as keyof lib.InstanceConfigFields, prop, value, "control");
if (value !== undefined) {
this._controller.instances.overrideField(instance, field, [prop]);
}
await this._controller.instanceConfigUpdated(instance);
}
async handleInstanceConfigInheritFieldRequest(request: lib.InstanceConfigInheritFieldRequest) {
ControlConnection.validateInstanceConfigSetField(request.field);
this._controller.instances.inheritField(request.instanceId, request.field, request.prop);
await this._controller.instanceConfigUpdated(this._controller.instances.getForRequest(request.instanceId));
}
async handleInstanceAssignRequest(request: lib.InstanceAssignRequest) {
await this._controller.instances.assignInstance(request.instanceId, request.hostId ?? null);
}
@@ -388,7 +411,7 @@ export default class ControlConnection extends BaseConnection {
let ready = new Promise<void>((resolve, reject) => {
stream.events.on("source", resolve);
stream.events.on("timeout", () => reject(
new lib.RequestError("Timed out establishing stream from host")
new lib.TimeoutError("Timed out establishing stream from host")
));
});
ready.catch(() => {});
@@ -405,17 +428,17 @@ export default class ControlConnection extends BaseConnection {
async handleInstanceTransferSaveRequest(request: lib.InstanceTransferSaveRequest) {
if (request.sourceInstanceId === request.targetInstanceId) {
throw new lib.RequestError("Source and target instance may not be the same");
throw new lib.ValidationError("Source and target instance may not be the same");
}
let sourceInstance = this._controller.instances.getForRequest(request.sourceInstanceId);
let targetInstance = this._controller.instances.getForRequest(request.targetInstanceId);
let sourceHostId = sourceInstance.config.get("instance.assigned_host");
let targetHostId = targetInstance.config.get("instance.assigned_host");
if (sourceHostId === null) {
throw new lib.RequestError("Source instance is not assigned a host");
throw new lib.ConflictError("Source instance is not assigned a host");
}
if (targetHostId === null) {
throw new lib.RequestError("Target instance is not assigned a host");
throw new lib.ConflictError("Target instance is not assigned a host");
}
// Let host handle request if source and target is on the same host.
@@ -426,12 +449,12 @@ export default class ControlConnection extends BaseConnection {
// Check connectivity
let sourceHostConnection = this._controller.wsServer.hostConnections.get(sourceHostId);
if (!sourceHostConnection || sourceHostConnection.connector.closing) {
throw new lib.RequestError("Source host is not connected to the controller");
throw new lib.UnavailableError("Source host is not connected to the controller");
}
let targetHostConnection = this._controller.wsServer.hostConnections.get(targetHostId);
if (!targetHostConnection || targetHostConnection.connector.closing) {
throw new lib.RequestError("Target host is not connected to the controller");
throw new lib.UnavailableError("Target host is not connected to the controller");
}
// Create stream to proxy from target to source
@@ -440,7 +463,7 @@ export default class ControlConnection extends BaseConnection {
if (stream.source) {
stream.source.destroy();
}
stream.events.emit("error", new lib.RequestError("Timed out establishing transfer stream"));
stream.events.emit("error", new lib.TimeoutError("Timed out establishing transfer stream"));
});
// Ignore errors if not listening for them to avoid crash.
@@ -480,10 +503,10 @@ export default class ControlConnection extends BaseConnection {
async handleModPackCreateRequest(request: lib.ModPackCreateRequest) {
let modPack = request.modPack;
if (modPack.id === undefined) {
throw new lib.RequestError("Mod pack need an ID to be created");
throw new lib.ValidationError("Mod pack need an ID to be created");
}
if (this._controller.modPacks.has(modPack.id)) {
throw new lib.RequestError(`Mod pack with ID ${modPack.id} already exist`);
throw new lib.ConflictError(`Mod pack with ID ${modPack.id} already exist`);
}
this._controller.modPacks.set(modPack);
}
@@ -491,7 +514,7 @@ export default class ControlConnection extends BaseConnection {
async handleModPackUpdateRequest(request: lib.ModPackUpdateRequest) {
let modPack = request.modPack;
if (modPack.id === undefined || !this._controller.modPacks.has(modPack.id)) {
throw new lib.RequestError(`Mod pack with ID ${modPack.id} does not exist`);
throw new lib.NotFoundError(`Mod pack with ID ${modPack.id} does not exist`);
}
this._controller.modPacks.set(modPack);
}
@@ -500,7 +523,7 @@ export default class ControlConnection extends BaseConnection {
let { id } = request;
let modPack = this._controller.modPacks.getMutable(id);
if (!modPack) {
throw new lib.RequestError(`Mod pack with ID ${id} does not exist`);
throw new lib.NotFoundError(`Mod pack with ID ${id} does not exist`);
}
this._controller.modPacks.delete(modPack);
this._controller.deleteNote("mod_pack", id);
@@ -582,7 +605,7 @@ export default class ControlConnection extends BaseConnection {
author: (a:ModVersions, b:ModVersions) => strcmp(a.versions[0].author, b.versions[0].author),
};
if (!Object.prototype.hasOwnProperty.call(sorters, sort)) {
throw new lib.RequestError(`Invalid value for sort: ${sort}`);
throw new lib.ValidationError(`Invalid value for sort: ${sort}`);
}
resultList.sort(sorters[sort as keyof typeof sorters]);
let order = request.sortOrder;
@@ -629,7 +652,7 @@ export default class ControlConnection extends BaseConnection {
} catch (error: any) {
logger.error(`Error fetching all mods from portal (${request.factorioVersion}): ${error}`);
// Propagate a user-friendly error back to the client
throw new lib.RequestError(`Portal mod fetch failed: ${error}`);
throw new lib.InternalServerError(`Portal mod fetch failed: ${error}`);
}
}
@@ -647,7 +670,7 @@ export default class ControlConnection extends BaseConnection {
const token = this._controller.config.get("controller.factorio_token");
if (!username || username === "" || !token || token === "") {
throw new lib.RequestError("Factorio credentials (username, token) not configured on the controller.");
throw new lib.ConflictError("Factorio credentials (username, token) not configured on the controller.");
}
try {
@@ -656,18 +679,16 @@ export default class ControlConnection extends BaseConnection {
} catch (error: any) {
logger.error(`Error downloading mods from portal: ${error.message}`);
// Improve error message clarity
let errorMessage = "Mod portal download failed";
if (error instanceof lib.RequestError) {
errorMessage += `: ${error.message}`;
} else if (error.message.includes("401") || error.message.includes("Unauthorized")) {
errorMessage += ": Authentication failed. Check Factorio username/token in controller config.";
} else if (error.message.includes("404") || error.message.includes("Not Found")) {
errorMessage += ": Mod version not found on the portal.";
} else {
errorMessage += `: ${error.message}`;
const errorMessage = "Mod portal download failed";
if (error.message.includes("401") || error.message.includes("Unauthorized")) {
throw new lib.ConflictError(
`${errorMessage}: Authentication failed. Check Factorio username/token in controller config.`
);
}
throw new lib.RequestError(errorMessage);
if (error.message.includes("404") || error.message.includes("Not Found")) {
throw new lib.NotFoundError(`${errorMessage}: Mod version not found on the portal.`);
}
throw new lib.InternalServerError(`${errorMessage}: ${error.message}`);
}
}
@@ -677,7 +698,7 @@ export default class ControlConnection extends BaseConnection {
for (const mod of request.mods) {
const modInfo = installedMods.find(m => m.name === mod.name && mod.version.testVersion(m.version));
if (!modInfo) {
throw new lib.RequestError(`Failed to find mod ${mod.name} in store after download attempt.`);
throw new lib.InternalServerError(`Failed to find mod ${mod.name} in store after download attempt.`);
}
modInfos.push(modInfo);
}
@@ -711,7 +732,7 @@ export default class ControlConnection extends BaseConnection {
} catch (error: any) {
if (!error.message.includes("404") && !error.message.includes("Not Found")) {
logger.error(`Error fetching mod from portal: ${error.message}`);
throw new lib.RequestError(`Dependency resolution failed: ${error.message}`);
throw new lib.InternalServerError(`Dependency resolution failed: ${error.message}`);
} else {
releases = null;
}
@@ -951,7 +972,7 @@ export default class ControlConnection extends BaseConnection {
let { id, name, description, permissions } = request;
let role = this._controller.roles.getMutable(id);
if (!role) {
throw new lib.RequestError(`Role with ID ${id} does not exist`);
throw new lib.NotFoundError(`Role with ID ${id} does not exist`);
}
role.name = name;
@@ -963,7 +984,7 @@ export default class ControlConnection extends BaseConnection {
async handleRoleGrantDefaultPermissionsRequest(request: lib.RoleGrantDefaultPermissionsRequest) {
let role = this._controller.roles.get(request.id);
if (!role) {
throw new lib.RequestError(`Role with ID ${request.id} does not exist`);
throw new lib.NotFoundError(`Role with ID ${request.id} does not exist`);
}
role.grantDefaultPermissions();
@@ -975,7 +996,7 @@ export default class ControlConnection extends BaseConnection {
const role = this._controller.roles.get(id);
if (!role) {
throw new lib.RequestError(`Role with ID ${id} does not exist`);
throw new lib.NotFoundError(`Role with ID ${id} does not exist`);
}
this._controller.roles.delete(role);
this._controller.deleteNote("role", id);
@@ -994,7 +1015,7 @@ export default class ControlConnection extends BaseConnection {
async handleNoteSetRequest(request: lib.NoteSetRequest) {
const { resourceType, resourceId, content } = request;
if (!resourceType || !resourceId) {
throw new lib.RequestError("resourceType and resourceId must not be empty");
throw new lib.ValidationError("resourceType and resourceId must not be empty");
}
this._controller.setNote(resourceType, resourceId, content, this.user.name);
}
@@ -1003,7 +1024,7 @@ export default class ControlConnection extends BaseConnection {
let name = request.name;
let user = this._controller.users.getByName(name);
if (!user) {
throw new lib.RequestError(`User ${name} does not exist`);
throw new lib.NotFoundError(`User ${name} does not exist`);
}
return user.toUserDetails();
@@ -1020,7 +1041,7 @@ export default class ControlConnection extends BaseConnection {
async handleUserRevokeTokenRequest(request: lib.UserRevokeTokenRequest) {
let user = this._controller.users.getByName(request.name);
if (!user) {
throw new lib.RequestError(`User '${request.name}' does not exist`);
throw new lib.NotFoundError(`User '${request.name}' does not exist`);
}
if (user.id !== this.user.id) {
this.user.checkPermission("core.user.revoke_other_token");
@@ -1037,12 +1058,12 @@ export default class ControlConnection extends BaseConnection {
async handleUserUpdateRolesRequest(request: lib.UserUpdateRolesRequest) {
let user = this._controller.users.getByName(request.name);
if (!user) {
throw new lib.RequestError(`User '${request.name}' does not exist`);
throw new lib.NotFoundError(`User '${request.name}' does not exist`);
}
for (let roleId of request.roles) {
if (!this._controller.roles.has(roleId)) {
throw new lib.RequestError(`Role with ID ${roleId} does not exist`);
throw new lib.NotFoundError(`Role with ID ${roleId} does not exist`);
}
}
@@ -1058,7 +1079,7 @@ export default class ControlConnection extends BaseConnection {
this.user.checkPermission("core.user.create");
user = this._controller.users.createUser(name);
} else {
throw new lib.RequestError(`User '${name}' does not exist`);
throw new lib.NotFoundError(`User '${name}' does not exist`);
}
}
@@ -1074,7 +1095,7 @@ export default class ControlConnection extends BaseConnection {
this.user.checkPermission("core.user.create");
user = this._controller.users.createUser(name);
} else {
throw new lib.RequestError(`User '${name}' does not exist`);
throw new lib.NotFoundError(`User '${name}' does not exist`);
}
}
@@ -1092,7 +1113,7 @@ export default class ControlConnection extends BaseConnection {
this.user.checkPermission("core.user.create");
user = this._controller.users.createUser(name);
} else {
throw new lib.RequestError(`User '${name}' does not exist`);
throw new lib.NotFoundError(`User '${name}' does not exist`);
}
}
@@ -1104,7 +1125,7 @@ export default class ControlConnection extends BaseConnection {
let name = request.name;
let user = this._controller.users.getByName(name);
if (!user) {
throw new lib.RequestError(`User '${name}' does not exist`);
throw new lib.NotFoundError(`User '${name}' does not exist`);
}
this._controller.users.deleteUser(user);
@@ -1335,47 +1356,54 @@ export default class ControlConnection extends BaseConnection {
async handleControllerUpdateRequest(request: lib.ControllerUpdateRequest) {
if (!this._controller.config.get("controller.allow_remote_updates")) {
throw new lib.RequestError("Remote updates are disabled on this machine");
throw new lib.ForbiddenError("Remote updates are disabled on this machine");
}
return await lib.updatePackage("@clusterio/controller");
}
async handlePluginUpdateRequest(request: lib.PluginUpdateRequest) {
if (!this._controller.config.get("controller.allow_plugin_updates")) {
throw new lib.RequestError("Plugin updates are disabled on this machine");
throw new lib.ForbiddenError("Plugin updates are disabled on this machine");
}
return await lib.handlePluginUpdate(request.pluginPackage, this._controller.pluginInfos);
}
async handlePluginInstallRequest(request: lib.PluginInstallRequest) {
if (!this._controller.config.get("controller.allow_plugin_install")) {
throw new lib.RequestError("Plugin installs are disabled on this machine");
throw new lib.ForbiddenError("Plugin installs are disabled on this machine");
}
await lib.handlePluginInstall(request.pluginPackage);
// A new plugin is not in pluginInfos, so checkRestartRequired cannot notice it
if (!this._controller.pluginInfos.some(pluginInfo => pluginInfo.npmPackage === request.pluginPackage)) {
const npmPackage = await lib.handlePluginInstall(request.pluginPackage);
// A new plugin is not in pluginInfos, so checkRestartRequired cannot notice it. Version
// changes to loaded plugins are noticed there.
if (!this._controller.pluginInfos.some(pluginInfo => pluginInfo.npmPackage === npmPackage)) {
this._controller.config.restartRequired = true;
}
}
async handleUpdateAllRequest(request: lib.UpdateAllRequest) {
if (!this._controller.config.get("controller.allow_remote_updates")) {
throw new lib.RequestError("Remote updates are disabled on this machine");
throw new lib.ForbiddenError("Remote updates are disabled on this machine");
}
if (!this._controller.config.get("controller.allow_plugin_updates")) {
throw new lib.RequestError("Plugin updates are disabled on this machine");
throw new lib.ForbiddenError("Plugin updates are disabled on this machine");
}
return await lib.handleUpdateAll("@clusterio/controller", this._controller.pluginInfos);
}
async handlePluginListRequest(request: lib.PluginListRequest) {
return this._controller.pluginInfos.map(pluginInfo => lib.PluginDetails.fromNodeEnvInfo(
pluginInfo,
this._controller.loadedPlugins.has(pluginInfo),
this._controller.config.get(`${pluginInfo.name}.load_plugin`),
async handlePluginSearchRequest(request: lib.PluginSearchRequest) {
return await lib.searchPlugins(request.query, request.page, request.pageSize);
}
async handlePluginLatestVersionsRequest(request: lib.PluginLatestVersionsRequest) {
return new lib.PluginLatestVersionsRequest.Response(await lib.fetchLatestVersions(
request.packages, this._controller.pluginLatestVersionCache,
));
}
async handlePluginListRequest(request: lib.PluginListRequest) {
return await this._controller.listPlugins();
}
async handleDebugDumpWsRequest(request: lib.DebugDumpWsRequest) {
if (this.ws_dumper) {
return; // Already dumping
+185 -90
View File
@@ -11,7 +11,6 @@ import events from "events";
import fs from "node:fs/promises";
import http from "http";
import https from "https";
import jwt from "jsonwebtoken";
import path from "path";
import semver from "semver";
import stream from "stream";
@@ -28,7 +27,9 @@ import User from "./User.js";
import UserRecord from "./UserRecord.js";
import UserManager from "./UserManager.js";
import WsServer from "./WsServer.js";
import HostManager from "./HostManager.js";
import HostRecord from "./HostRecord.js";
import HostPairingManager from "./HostPairingManager.js";
import { BaseControllerPlugin, ControllerHooks } from "./BaseControllerPlugin.js";
import ControllerRouter from "./ControllerRouter.js";
import InstanceManager from "./InstanceManager.js";
@@ -52,7 +53,7 @@ type ControllerDebugEvents = {
type ControllerEvents = {
"stop": [],
}
};
/**
* Manages all controller related operations
@@ -75,6 +76,8 @@ export default class Controller {
hooks = new ControllerHooks(logger);
/** Plugins which are currently loaded */
loadedPlugins: Set<lib.PluginNodeEnvInfo> = new Set();
/** Latest versions of plugins on npm, as shown on the plugins page */
pluginLatestVersionCache: lib.LatestVersionCache = new Map();
/** WebSocket server */
wsServer: WsServer;
@@ -85,10 +88,14 @@ export default class Controller {
/** Event subscription controller */
subscriptions: lib.SubscriptionController;
/** Host manager for the cluster, also contains access to the host records */
hosts: HostManager;
/** User manager for the cluster, also contains access to the user records */
users: UserManager;
/** Instance manager for the cluster, also contains access to the user records */
instances: InstanceManager;
/** Pending requests from hosts to join the cluster */
hostPairing: HostPairingManager;
// Possible states are new, starting, running, stopping, stopped
private _state: string = "new";
@@ -104,7 +111,7 @@ export default class Controller {
_registeredEvents = new Map<lib.EventClass<unknown>, lib.EventHandler<unknown>>();
_snoopedEvents = new Map();
devMiddleware: any | null = null;
devServer: import("vite").ViteDevServer | null = null;
startedAtMs = Date.now();
autosaveInterval?: ReturnType<typeof setInterval>;
@@ -133,7 +140,7 @@ export default class Controller {
const cachePath = path.join(this.config.get("controller.database_directory"), "latest-releases.json");
try {
const releases = await lib.fetchLatestReleases();
await lib.safeOutputFile(cachePath, JSON.stringify(releases, null, "\t"));
await lib.atomicOverwriteFile(cachePath, JSON.stringify(releases, null, "\t"));
return releases;
} catch (err: any) {
logger.warn(`Failed to retrieve Factorio releases ${err.message}`);
@@ -148,7 +155,14 @@ export default class Controller {
}
});
static async bootstrap(config: lib.ControllerConfig) {
/** Create a controller with its datastores and mod store loaded from disk */
static async fromConfig(
config: lib.ControllerConfig,
pluginInfos: lib.PluginNodeEnvInfo[],
clusterLogger: winston.Logger,
canRestart = false,
recoveryMode = false,
) {
let databaseDirectory = config.get("controller.database_directory");
await fs.mkdir(databaseDirectory, { recursive: true });
@@ -158,23 +172,23 @@ export default class Controller {
path.join(databaseDirectory, "roles.json"),
);
const systems = new lib.SubscribableDatastore(...await new lib.JsonIdDatastoreProvider(
const systems = await lib.SubscribableDatastore.fromProvider(new lib.JsonIdDatastoreProvider(
path.join(databaseDirectory, "systems.json"),
lib.SystemInfo.fromJSON.bind(lib.SystemInfo),
this.migrateSystems,
).bootstrap());
));
const hosts = new lib.SubscribableDatastore(...await new lib.JsonIdDatastoreProvider(
const hosts = await lib.SubscribableDatastore.fromProvider(new lib.JsonIdDatastoreProvider(
path.join(databaseDirectory, "hosts.json"),
HostRecord.fromJSON.bind(HostRecord),
this.migrateHosts, this.finaliseHosts,
).bootstrap());
));
const instances = new lib.SubscribableDatastore(...await new lib.JsonIdDatastoreProvider(
const instances = await lib.SubscribableDatastore.fromProvider(new lib.JsonIdDatastoreProvider(
path.join(databaseDirectory, "instances.json"),
InstanceRecord.fromJSON.bind(InstanceRecord),
this.migrateInstances, this.finaliseInstances,
).bootstrap());
));
// Save plugin versions written while loading
for (const instance of instances.values()) {
if (instance.config.dirty) {
@@ -182,38 +196,40 @@ export default class Controller {
}
}
const saves = new lib.SubscribableDatastore(...await new lib.JsonIdDatastoreProvider(
const saves = await lib.SubscribableDatastore.fromProvider(new lib.JsonIdDatastoreProvider(
path.join(databaseDirectory, "saves.json"),
lib.SaveDetails.fromJSON.bind(lib.SaveDetails)
).bootstrap());
));
const modPacks = new lib.SubscribableDatastore(...await new lib.JsonIdDatastoreProvider(
const modPacks = await lib.SubscribableDatastore.fromProvider(new lib.JsonIdDatastoreProvider(
path.join(databaseDirectory, "mod-packs.json"),
lib.ModPack.fromJSON.bind(lib.ModPack),
this.migrateModPacks,
).bootstrap());
));
const roles = new lib.SubscribableDatastore(...await new lib.JsonIdDatastoreProvider(
const roles = await lib.SubscribableDatastore.fromProvider(new lib.JsonIdDatastoreProvider(
path.join(databaseDirectory, "roles.json"),
lib.Role.fromJSON.bind(lib.Role),
this.migrateRoles,
).bootstrap());
));
const users = new lib.SubscribableDatastore(...await new lib.JsonIdDatastoreProvider(
const users = await lib.SubscribableDatastore.fromProvider(new lib.JsonIdDatastoreProvider(
path.join(databaseDirectory, "users.json"),
UserRecord.fromJSON.bind(UserRecord),
undefined, this.finaliseUsers,
).bootstrap());
));
const notes = new lib.SubscribableDatastore(...await new lib.JsonIdDatastoreProvider(
const notes = await lib.SubscribableDatastore.fromProvider(new lib.JsonIdDatastoreProvider(
path.join(databaseDirectory, "notes.json"),
lib.Note.fromJSON.bind(lib.Note),
).bootstrap());
));
let modsDirectory = config.get("controller.mods_directory");
await fs.mkdir(modsDirectory, { recursive: true });
const modStore = await lib.ModStore.fromDirectory(modsDirectory);
await this.validateExports(modPacks, "static");
// Add default mod packs
if (modPacks.size === 0) {
modPacks.setMany(lib.ModPack.defaultModPacks);
@@ -223,7 +239,12 @@ export default class Controller {
lib.Role.ensureDefaultAdminRole(roles);
lib.Role.ensureDefaultPlayerRole(roles);
return [
return new this(
clusterLogger,
pluginInfos,
config,
canRestart,
recoveryMode,
systems,
hosts,
instances,
@@ -233,7 +254,7 @@ export default class Controller {
roles,
users,
notes,
] as const;
);
}
constructor(
@@ -253,7 +274,7 @@ export default class Controller {
public recoveryMode: boolean = false,
public systems = new lib.SubscribableDatastore<lib.SystemInfo>(),
/** Mapping of host id to host record */
public hosts = new lib.SubscribableDatastore<HostRecord>(),
hosts = new lib.SubscribableDatastore<HostRecord>(),
/** Mapping of instance id to instance record */
instances = new lib.SubscribableDatastore<InstanceRecord>(),
/** Mapping of save id to save details */
@@ -280,8 +301,10 @@ export default class Controller {
this.trustedProxies = this.parseTrustedProxies();
this.wsServer = new WsServer(this);
this.hosts = new HostManager(hosts, this);
this.users = new UserManager(users, roles, config);
this.instances = new InstanceManager(instances, this);
this.hostPairing = new HostPairingManager(this);
this.modStore.on("change", mod => {
this.modsUpdated([mod]);
@@ -291,6 +314,7 @@ export default class Controller {
this.subscriptions = new lib.SubscriptionController(config.get("controller.max_subscription_filters"));
this.subscriptions.handle(lib.SystemInfoUpdateEvent, this.handleSystemInfoSubscription.bind(this));
this.subscriptions.handle(lib.HostUpdatesEvent, this.handleHostSubscription.bind(this));
this.subscriptions.handle(lib.HostPairingUpdatesEvent, this.handleHostPairingSubscription.bind(this));
this.subscriptions.handle(lib.InstanceDetailsUpdatesEvent, this.handleInstanceDetailsSubscription.bind(this));
this.subscriptions.handle(
lib.InstanceSaveDetailsUpdatesEvent, this.handleInstanceSaveDetailsSubscription.bind(this)
@@ -303,7 +327,8 @@ export default class Controller {
// Handle updates for datastores
this.systems.on("update", this.systemsUpdated.bind(this));
this.hosts.on("update", this.hostsUpdated.bind(this));
this.hosts.records.on("update", this.hostsUpdated.bind(this));
this.hostPairing.records.on("update", this.hostPairingRequestsUpdated.bind(this));
this.instances.records.on("update", this.instanceUpdated.bind(this));
this.saves.on("update", this.savesUpdated.bind(this));
this.modPacks.on("update", this.modPacksUpdated.bind(this));
@@ -364,7 +389,7 @@ export default class Controller {
}, 600e3);
logSizeGauge.callback = async () => { logSizeGauge.set(await this.logSize()); };
// Start webpack development server if enabled
// Start Vite development server if enabled
if (args.dev || args.devPlugin) {
await this._startDevServer(args);
}
@@ -402,15 +427,21 @@ export default class Controller {
}
Controller.addAppRoutes(this.app, this.pluginInfos);
if (this.devServer) {
this.app.use(this.devServer.middlewares);
}
if (!args.dev) {
if (!this.devServer) {
let manifestPath = path.join(import.meta.dirname, "..", "..", "web", "manifest.json");
let manifest = await Controller.loadJsonObject(manifestPath);
if (!manifest["main.js"]) {
logger.error("Missing main.js entry in dist/web/manifest.json");
if (!manifest.main) {
logger.error("Missing main entry in dist/web/manifest.json");
}
this.app.locals.mainBundle = manifest["main.js"] || "no_web_build";
this.app.locals.webBundle = {
main: manifest.main || "no_web_build",
imports: manifest.imports || {},
};
}
this.app.locals.staticRoot = this.config.get("controller.static_url");
@@ -469,6 +500,7 @@ export default class Controller {
logger.info(`Listening for HTTPS on port ${(this.httpsServer.address() as AddressInfo).port}`);
}
this.hostPairing.start();
this.onAutosaveIntervalChanged();
this.onSystemMetricsIntervalChanged();
@@ -478,37 +510,44 @@ export default class Controller {
}
async _startDevServer(args: ControllerArgs) {
logger.warn("Webpack development mode enabled");
logger.warn("Vite development mode enabled");
const webpack = (await import("webpack")).default;
const webpackDevMiddleware = (await import("webpack-dev-middleware")).default;
const webpackConfigs = [];
const vite = await import("vite");
const webUiVite = await import("@clusterio/web_ui/vite.mjs");
if (args.dev) {
// @ts-expect-error this file is run from a different folder
webpackConfigs.push((await import("../../../webpack.config.cjs")).default({})); // Path outside of build
}
if (args.devPlugin) {
let devPlugins = new Map();
for (let name of args.devPlugin) {
let info = this.pluginInfos.find(i => i.name === name);
if (!info) {
throw new lib.StartupError(`No plugin named ${name}`);
}
const webpackConfig = path.posix.join(info.requirePath, "webpack.config.cjs");
let config = (await import(
path.isAbsolute(webpackConfig) ? pathToFileURL(webpackConfig).href : webpackConfig
)).default({});
devPlugins.set(name, webpackConfigs.length);
webpackConfigs.push(config);
const devPlugins = new Map<string, string>();
const pluginDirs = new Map<string, string>();
for (let name of (args.devPlugin ?? []).map(String)) {
let info = this.pluginInfos.find(i => i.name === name);
if (!info) {
throw new lib.StartupError(`No plugin named ${name}`);
}
this.app.locals.devPlugins = devPlugins;
pluginDirs.set(name, path.dirname(info.packagePath));
devPlugins.set(name, webUiVite.devPluginUrl(name));
}
this.app.locals.devPlugins = devPlugins;
const compiler = webpack(webpackConfigs);
this.devMiddleware = webpackDevMiddleware(compiler, { serverSideRender: true });
this.app.use(this.devMiddleware);
const controllerDir = path.join(import.meta.dirname, "..", "..", "..");
this.devServer = await vite.createServer({
configFile: path.join(controllerDir, "vite.config.js"),
root: controllerDir,
base: "/static/",
appType: "custom",
server: {
middlewareMode: true,
preTransformRequests: false,
},
plugins: [webUiVite.clusterioDevPlugins(pluginDirs)],
});
// Plugins served from source import the shared modules from the development server, the web
// interface has to use the same copies, so it is served from source as well with --dev-plugin.
this.app.locals.webBundle = {
main: this.devServer.config.plugins.find(p => p.name === "clusterio:app")!.api.entry,
imports: Object.fromEntries(webUiVite.sharedModules.map(
name => [name, webUiVite.devSharedUrl(name)]
)),
};
this.app.locals.devServer = this.devServer;
}
/**
@@ -555,12 +594,14 @@ export default class Controller {
this.autosaveInterval = undefined;
}
this.hostPairing.stop();
if (this.clusterLogIndex) {
await this.clusterLogIndex.save();
}
if (this.devMiddleware) {
await new Promise((resolve, reject) => { this.devMiddleware.close(resolve); });
if (this.devServer) {
await this.devServer.close();
}
await this.hooks.shutdown.invoke();
@@ -634,7 +675,9 @@ export default class Controller {
requests.push(hostConnection.send(new lib.SystemInfoRequest()));
}
const restartRequired = await this.checkRestartRequired();
requests.push(lib.gatherSystemInfo("controller", this.canRestart, restartRequired, this.startedAtMs));
requests.push(lib.gatherSystemInfo(
"controller", this.canRestart, restartRequired, this.startedAtMs, await this.listPlugins(),
));
const newMetrics = await Promise.all(requests);
for (const metric of newMetrics) {
this.systems.set(metric);
@@ -644,6 +687,15 @@ export default class Controller {
}
}
/** Plugins on the controller, including those installed or updated since it started */
async listPlugins() {
return await lib.listPluginDetails(
this.pluginInfos,
pluginInfo => this.loadedPlugins.has(pluginInfo),
pluginInfo => this.config.get(`${pluginInfo.name}.load_plugin`),
);
}
async checkRestartRequired() {
if (this.config.restartRequired) {
return true;
@@ -747,7 +799,7 @@ export default class Controller {
await Promise.all([
this.config.save(),
this.systems.save(),
this.hosts.save(),
this.hosts.records.save(),
this.instances.records.save(),
this.saves.save(),
this.modPacks.save(),
@@ -829,8 +881,43 @@ export default class Controller {
});
}
/** Remove export manifests that reference files missing from staticDirectory */
static async validateExports(modPacks: lib.SubscribableDatastore<lib.ModPack>, staticDirectory: string) {
const invalid: lib.ModPack[] = [];
for (const modPack of modPacks.valuesMutable()) {
if (!modPack.exportManifest) {
continue;
}
const missing: string[] = [];
for (const file of Object.values(modPack.exportManifest.assets)) {
const valid = path.basename(file) === file
&& await fs.stat(path.join(staticDirectory, file)).then(s => s.isFile(), () => false);
if (!valid) {
missing.push(file);
}
}
if (missing.length) {
Controller.dropExport(modPack, `missing ${missing.join(", ")}`);
invalid.push(modPack);
}
}
if (invalid.length) {
modPacks.setMany(invalid);
}
}
/** Clear the export manifest of a mutable mod pack, caller must save it */
static dropExport(modPack: lib.ModPack, reason: string) {
logger.warn(`Removing export data of mod pack ${modPack.name} (id ${modPack.id}): ${reason}`);
modPack.exportManifest = undefined;
}
static migrateRoles(rawJson: unknown[]): Static<typeof lib.Role.jsonSchema>[] {
const serialized = rawJson as Static<typeof lib.Role.jsonSchema>[];
// migrate: host token permissions were removed with host pairing in 2.1
const removedPermissions = new Set([
"core.host.generate_token", "core.host.revoke_token", "core.host.create_config",
]);
return serialized.map(json => {
json.permissions = json.permissions.map(permission => {
// migrate: core.instance.save.list.subscribe was renamed in alpha 17
@@ -838,7 +925,7 @@ export default class Controller {
return "core.instance.save.subscribe";
}
return permission;
});
}).filter(permission => !removedPermissions.has(permission));
return lib.Role.pluginMigrations.reduce(json);
});
}
@@ -933,6 +1020,7 @@ export default class Controller {
express.static(path.join(import.meta.dirname, "..", "..", "web", "static"), staticOptions)
);
app.use("/static", express.static("static", staticOptions)); // Used for data export files
app.use("/static", routes.missingExportAsset);
// Add API routes
routes.addRouteHandlers(app);
@@ -964,21 +1052,25 @@ export default class Controller {
return systems.length ? new lib.SystemInfoUpdateEvent(systems) : null;
}
/**
* Generate access token for host
*
* @returns access token for host
* @param hostId - ID of host to generate a token for.
*/
generateHostToken(hostId: number): string {
return jwt.sign({ aud: "host", host: hostId }, this.authSecret);
}
hostsUpdated(hosts: HostRecord[]) {
let updates = hosts.map(host => host.toHostDetails());
this.subscriptions.broadcast(new lib.HostUpdatesEvent(updates));
}
hostPairingRequestsUpdated(requests: lib.HostPairingDetails[]) {
this.subscriptions.broadcast(new lib.HostPairingUpdatesEvent(requests));
}
async handleHostPairingSubscription(request: lib.SubscriptionRequest) {
if (request.action === "unsubscribe") {
return null;
}
const requests = [...this.hostPairing.records.values()].filter(
pairing => pairing.updatedAtMs > request.lastRequestTimeMs,
);
return requests.length ? new lib.HostPairingUpdatesEvent(requests) : null;
}
async handleHostSubscription(request: lib.SubscriptionRequest) {
if (request.action === "unsubscribe") {
return null;
@@ -1332,26 +1424,29 @@ export default class Controller {
let depth = routeDepth + Number(req.path.slice(-1) === "/");
let webRoot = "../".repeat(depth) || "./";
let staticRoot = res.app.locals.staticRoot ?? `${webRoot}static/`;
let mainBundle: string = "";
if (res.app.locals.mainBundle) {
mainBundle = res.app.locals.mainBundle;
} else {
let stats = res.locals.webpack.devMiddleware.stats.stats[0];
mainBundle = stats.toJson().assetsByChunkName["main"][0];
}
mainBundle = routes.stripStaticPrefix(mainBundle);
const { main, imports } = res.app.locals.webBundle;
const importMap = JSON.stringify({
imports: Object.fromEntries(Object.entries(imports).map(
([name, file]) => [name, `${staticRoot}${routes.stripStaticPrefix(file)}`]
)),
}).replace(/</g, "\\u003c");
fs.readFile(
path.join(import.meta.dirname, "..", "..", "..", "web", "index.html"),
"utf8",
).then((content) => {
res.type("text/html");
res.send(content
).then(async (content) => {
content = content
.replace(/__CLUSTER_NAME__/g, res.app.locals.controller.config.get("controller.name"))
.replace(/__WEB_ROOT__/g, webRoot)
.replace(/__STATIC_ROOT__/g, staticRoot)
.replace(/__MAIN_BUNDLE__/g, mainBundle)
);
.replace(/__IMPORT_MAP__/g, () => importMap)
.replace(/__MAIN_BUNDLE__/g, () => routes.stripStaticPrefix(main))
;
if (res.app.locals.devServer) {
content = await res.app.locals.devServer.transformIndexHtml(req.originalUrl, content);
}
res.type("text/html");
res.send(content);
}).catch(err => {
next(err);
});
@@ -1440,24 +1535,24 @@ export default class Controller {
} else if (dst.type === lib.Address.control) {
connection = this.wsServer.controlConnections.get(dst.id);
if (!connection) {
throw new lib.RequestError("Target control connection does not exist.");
throw new lib.NotFoundError("Target control connection does not exist.");
}
} else if (dst.type === lib.Address.instance) {
const instance = this.instances.getForRequest(dst.id);
let hostId = instance.config.get("instance.assigned_host");
if (hostId === null) {
throw new lib.RequestError("Instance is not assigned to a host");
throw new lib.ConflictError("Instance is not assigned to a host");
}
connection = this.wsServer.hostConnections.get(hostId);
if (!connection) {
throw new lib.RequestError("Host containing instance is not connected");
throw new lib.UnavailableError("Host containing instance is not connected");
}
} else if (dst.type === lib.Address.host) {
connection = this.wsServer.hostConnections.get(dst.id);
if (!connection) {
throw new lib.RequestError("Host is not connected");
throw new lib.UnavailableError("Host is not connected");
}
} else {
@@ -1541,11 +1636,11 @@ export default class Controller {
let instance = this.instances.getForRequest(request.instanceId);
let hostId = instance.config.get("instance.assigned_host");
if (hostId === null) {
throw new lib.RequestError("Instance is not assigned to a host");
throw new lib.ConflictError("Instance is not assigned to a host");
}
let connection = this.wsServer.hostConnections.get(hostId);
if (!connection) {
throw new lib.RequestError("Host containing instance is not connected");
throw new lib.UnavailableError("Host containing instance is not connected");
}
return await connection.send(request);
}
+1 -1
View File
@@ -74,7 +74,7 @@ export default class ControllerRouter {
return false;
}
origin.connector.sendResponseError(
new lib.ResponseError(msg || "Unroutable destination"), message.src
new lib.ResponseError(msg || "Unroutable destination", "UnavailableError"), message.src
);
return true;
}
+20 -16
View File
@@ -34,20 +34,17 @@ export default class HostConnection extends BaseConnection {
this.plugins = new Map(Object.entries(registerData.plugins));
const previousHostInfo = this._controller.hosts.get(registerData.id);
this.info = new HostRecord(
registerData.id,
previousHostInfo?.name ?? "",
registerData.version,
this.plugins,
true,
remoteAddress,
previousHostInfo?.publicAddress,
previousHostInfo?.tokenValidAfter,
0,
false,
);
this._controller.hosts.set(this.info);
// Hosts are only created by approving them, WsServer rejects hosts without a record.
const record = this._controller.hosts.getMutable(registerData.id);
if (!record) {
throw new Error(`Host ${registerData.id} is not registered`);
}
record.version = registerData.version;
record.plugins = this.plugins;
record.connected = true;
record.remoteAddress = remoteAddress;
this.info = record;
this._controller.hosts.records.set(this.info);
this._checkPluginVersions();
@@ -71,7 +68,7 @@ export default class HostConnection extends BaseConnection {
this._controller.instances.records.setMany(instances);
this.info.connected = false;
this._controller.hosts.set(this.info);
this.saveRecord();
});
this.handle(lib.GetFactorioCredentialsRequest, this.handleGetFactorioCredentialsRequest.bind(this));
@@ -175,10 +172,17 @@ export default class HostConnection extends BaseConnection {
return {};
}
/** Store changes to the host record, unless the host was revoked */
saveRecord() {
if (this._controller.hosts.getMutable(this.id) === this.info) {
this._controller.hosts.records.set(this.info);
}
}
async handleHostInfoUpdateEvent(event: lib.HostInfoUpdateEvent) {
this.info.name = event.update.name;
this.info.publicAddress = event.update.publicAddress;
this._controller.hosts.set(this.info);
this.saveRecord();
}
async handleInstanceStatusChangedEvent(request: lib.InstanceStatusChangedEvent) {
+149
View File
@@ -0,0 +1,149 @@
import crypto from "node:crypto";
import jwt from "jsonwebtoken";
import * as lib from "@clusterio/lib";
import { logger } from "@clusterio/lib";
import type Controller from "./Controller.js";
import HostRecord from "./HostRecord.js";
/**
* Access layer for hosts on the controller.
*
* @alias module:controller/src/HostManager
*/
export default class HostManager {
constructor(
public readonly records: lib.SubscribableDatastore<HostRecord>,
private readonly _controller: Controller,
) { }
/* ------------------------------------------------------------------------- */
/* Basic Accessors */
/* ------------------------------------------------------------------------- */
has(id: HostRecord["id"]): boolean {
return this.records.has(id);
}
getMutable(id: HostRecord["id"]): HostRecord | undefined {
return this.records.getMutable(id);
}
get(id: HostRecord["id"]): Readonly<HostRecord> | undefined {
return this.records.get(id);
}
valuesMutable(): IterableIterator<HostRecord> {
return this.records.values();
}
values(): IterableIterator<Readonly<HostRecord>> {
return this.valuesMutable();
}
/**
* Get host by ID for a request.
*
* @param hostId - ID of host to get.
* @returns The host record.
* @throws {module:lib.NotFoundError} if the host does not exist.
*/
getForRequest(hostId: number): HostRecord {
const host = this.records.getMutable(hostId);
if (!host) {
throw new lib.NotFoundError(`Unknown host id (${hostId})`);
}
return host;
}
/* ------------------------------------------------------------------------- */
/* Lifecycle */
/* ------------------------------------------------------------------------- */
/**
* Create a host record with a new random id
*
* @param name - Name of the host.
* @returns the created host record.
*/
createHost(name: string): HostRecord {
let hostId;
do {
hostId = crypto.randomInt(1, 2 ** 31);
} while (this.records.has(hostId));
const host = new HostRecord(
hostId, name, "", new Map(), false, "", "", Math.floor(Date.now() / 1000),
);
this.records.set(host);
return host;
}
/**
* Delete a host, disconnect it and unassign its instances
*
* @param hostId - ID of host to revoke.
* @param revokedBy - Name of the user revoking it, for logging.
*/
async revokeHost(hostId: number, revokedBy: string) {
const host = this.getForRequest(hostId);
this.records.delete(host);
this._controller.deleteNote("host", hostId, revokedBy);
const system = this._controller.systems.getMutable(hostId);
if (system) {
this._controller.systems.delete(system);
}
// Disconnected first so unassigning the instances does not depend on the host responding.
const hostConnection = this._controller.wsServer.hostConnections.get(hostId);
if (hostConnection) {
hostConnection.connector.terminate();
}
const assigned = [...this._controller.instances.values()].filter(
instance => instance.config.get("instance.assigned_host") === hostId
);
for (const instance of assigned) {
await this._controller.instances.unassignInstance(instance.id);
}
logger.audit(`User ${revokedBy} revoked host ${host.name} (${host.id})`);
}
/* ------------------------------------------------------------------------- */
/* Helpers */
/* ------------------------------------------------------------------------- */
/**
* Sign access token for the given host
*
* @param hostId - ID of host to sign token for.
* @returns JWT access token for the host.
*/
signHostToken(hostId: number): string {
return jwt.sign({ aud: "host", host: hostId }, this._controller.authSecret);
}
/**
* Check a host token
*
* @param token - Host token to check.
* @returns the host the token was issued to.
* @throws if the token is invalid, its host is not registered or it was issued before tokenValidAfter.
*/
verifyHostToken(token: string): HostRecord {
// migrate: allow pre-rename tokens issued to hosts before alpha-14
const payload = jwt.verify(token, this._controller.authSecret, { audience: ["host", "slave"] });
if (typeof payload === "string") {
throw new Error("unexpected JsonWebToken type");
}
const hostId = payload.host !== undefined ? payload.host : payload.slave;
const host = this.records.getMutable(hostId);
if (!host) {
throw new Error("host is not registered, start the host again to request to join the cluster");
}
if (!payload.iat || payload.iat < host.tokenValidAfter) {
throw new Error("invalid token");
}
return host;
}
}
@@ -0,0 +1,299 @@
import type Controller from "./Controller.js";
import crypto from "crypto";
import net from "net";
import * as lib from "@clusterio/lib";
import { logger } from "@clusterio/lib";
/** Symbols used in pairing codes, avoids 0/O and 1/I/L */
const codeAlphabet = "23456789ABCDEFGHJKMNPQRSTUVWXYZ";
const codeLength = 6;
/** Time a request is kept after the last poll from the host */
export const pollTimeoutMs = 30e3;
/** Maximum number of pending requests from one address */
export const maxPendingPerAddress = 3;
/** Maximum length of a host name */
export const maxNameLength = 100;
/** Control and bidirectional override characters refused in host names */
const nameControlCharacters = /[\p{Cc}\u202a-\u202e\u2066-\u2069]/u;
/** Result of an approved request, sent to the host */
export interface HostPairingResult {
hostId: number,
token: string,
name: string,
}
/** Error with an HTTP status code to respond to the host with */
export class HostPairingError extends Error {
constructor(
message: string,
public status: number,
/** Seconds to wait before trying again, for status 429 */
public retryAfter?: number,
) {
super(message);
}
}
type PairingRequest = {
/** Secret the host polls with, never shown to users */
secret: string,
details: lib.HostPairingDetails,
/** Address with IPv6 addresses grouped by /64, used for limits */
addressKey: string,
lastPollMs: number,
} & (
{ state: "pending" | "rejected" } | { state: "approved", result: HostPairingResult }
);
/**
* Strips the IPv4 mapped prefix and groups IPv6 addresses by their /64
*
* @param address - Address to make a key for
* @returns key used to count requests from an address.
*/
export function addressKey(address: string) {
if (address.startsWith("::ffff:") && net.isIPv4(address.slice(7))) {
return address.slice(7);
}
if (!net.isIPv6(address)) {
return address;
}
const [head, tail = ""] = address.split("::");
const headParts = head ? head.split(":") : [];
const tailParts = tail ? tail.split(":") : [];
const missing = 8 - headParts.length - tailParts.length;
const parts = address.includes("::")
? [...headParts, ...new Array(missing).fill("0"), ...tailParts]
: headParts;
return `${parts.slice(0, 4).map(part => part.toLowerCase().replace(/^0+(?=.)/, "")).join(":")}::/64`;
}
/**
* Normalise a pairing code typed by a user
*
* @param code - Code as typed, may contain dashes and lower case letters.
* @returns code formatted as shown by the host.
*/
export function normaliseCode(code: string) {
const plain = code.toUpperCase().replace(/[^0-9A-Z]/g, "");
return `${plain.slice(0, 3)}-${plain.slice(3)}`;
}
function generateCode() {
let code = "";
for (let i = 0; i < codeLength; i++) {
code += codeAlphabet[crypto.randomInt(codeAlphabet.length)];
}
return normaliseCode(code);
}
/**
* Keeps track of requests from hosts to join the cluster
*
* Requests only live in memory, and only while the host keeps polling
* for the result.
*/
export default class HostPairingManager {
/** Pending requests by code, broadcast to subscribed control links */
records = new lib.SubscribableDatastore<lib.HostPairingDetails>();
private _bySecret = new Map<string, PairingRequest>();
private _byCode = new Map<string, PairingRequest>();
private _pruneInterval?: ReturnType<typeof setInterval>;
constructor(
private _controller: Controller,
) { }
start() {
this._pruneInterval = setInterval(() => this.prune(), 5e3);
this._pruneInterval.unref();
}
stop() {
clearInterval(this._pruneInterval);
this._pruneInterval = undefined;
}
/**
* Check and trim a host name given by a host or a user
*
* @param name - Name to check.
* @returns the trimmed name.
*/
static validateName(name: unknown) {
if (typeof name !== "string" || !name.trim()) {
throw new HostPairingError("Host name is required", 400);
}
const trimmed = name.trim();
if (trimmed.length > maxNameLength) {
throw new HostPairingError(`Host name must be at most ${maxNameLength} characters`, 400);
}
// Names come from unauthenticated requests and are printed to terminals and logs.
if (nameControlCharacters.test(trimmed)) {
throw new HostPairingError("Host name may not contain control characters", 400);
}
return trimmed;
}
/**
* Create a pending request
*
* @param name - Name the host asked for.
* @param version - Version of the host.
* @param remoteAddress - Address the request came from.
* @returns the secret to poll with and the code to show the user.
*/
create(name: string, version: string, remoteAddress: string) {
this.prune();
const key = addressKey(remoteAddress);
const now = Date.now();
const pendingFromAddress = [...this._bySecret.values()].filter(
request => request.addressKey === key && request.state === "pending"
).length;
if (pendingFromAddress >= maxPendingPerAddress) {
throw new HostPairingError("Too many pending requests from this address", 429, 30);
}
const maxPending = this._controller.config.get("controller.host_pairing_max_pending");
if (maxPending && this.records.size >= maxPending) {
throw new HostPairingError("Too many pending requests", 429, 30);
}
let code;
do {
code = generateCode();
} while (this._byCode.has(code));
const request: PairingRequest = {
secret: crypto.randomBytes(32).toString("base64url"),
details: new lib.HostPairingDetails(code, name, remoteAddress, version, now),
addressKey: key,
lastPollMs: now,
state: "pending",
};
this._bySecret.set(request.secret, request);
this._byCode.set(code, request);
this.records.set(request.details);
logger.verbose(`Host pairing request ${code} for "${name}" from ${remoteAddress}`);
return { requestId: request.secret, code };
}
/**
* Poll for the result of a request
*
* @param secret - Request id returned when the request was created.
* @returns the result if the request was approved, or null if still pending.
*/
poll(secret: string): HostPairingResult | null {
this.prune();
const request = this._bySecret.get(secret);
if (!request) {
throw new HostPairingError("Request not found", 404);
}
request.lastPollMs = Date.now();
if (request.state === "approved") {
this._remove(request);
return request.result;
}
if (request.state === "rejected") {
this._remove(request);
throw new HostPairingError("Request was rejected", 403);
}
return null;
}
/**
* Cancel a pending request
*
* @param secret - Request id returned when the request was created.
*/
cancel(secret: string) {
const request = this._bySecret.get(secret);
if (request && request.state === "pending") {
logger.verbose(`Host pairing request ${request.details.code} cancelled by the host`);
this._remove(request);
}
}
private _pending(code: string) {
this.prune();
const request = this._byCode.get(normaliseCode(code));
if (!request || request.state !== "pending") {
throw new lib.NotFoundError(`No pending host request with code ${code}`);
}
return request;
}
/**
* Approve a pending request, creating the host
*
* @param code - Code of the request.
* @param name - Name to give the host, defaults to the requested name.
* @param approvedBy - Name of the user approving it, for logging.
* @returns the id of the new host.
*/
approve(code: string, name: string | undefined, approvedBy: string) {
const request = this._pending(code);
let hostName;
try {
hostName = name === undefined ? request.details.name : HostPairingManager.validateName(name);
} catch (err: any) {
throw new lib.ValidationError(err.message);
}
const host = this._controller.hosts.createHost(hostName);
const result = { hostId: host.id, token: this._controller.hosts.signHostToken(host.id), name: host.name };
Object.assign(request, { state: "approved", result });
this.records.delete(request.details);
logger.audit(
`User ${approvedBy} approved host ${result.name} (${result.hostId}) with code ` +
`${request.details.code} from ${request.details.remoteAddress}`
);
return result.hostId;
}
/**
* Reject a pending request
*
* @param code - Code of the request.
* @param rejectedBy - Name of the user rejecting it, for logging.
*/
reject(code: string, rejectedBy: string) {
const request = this._pending(code);
request.state = "rejected";
this.records.delete(request.details);
logger.audit(
`User ${rejectedBy} rejected host ${request.details.name} with code ` +
`${request.details.code} from ${request.details.remoteAddress}`
);
}
private _remove(request: PairingRequest) {
this._bySecret.delete(request.secret);
this._byCode.delete(request.details.code);
this.records.delete(request.details);
}
/** Remove requests the host stopped polling for or that waited too long */
prune() {
const now = Date.now();
const timeoutMs = this._controller.config.get("controller.host_pairing_timeout") * 1000;
for (const request of this._bySecret.values()) {
const timedOut = request.state === "pending" && timeoutMs > 0
&& now - request.details.createdAtMs > timeoutMs;
if (!timedOut && now - request.lastPollMs <= pollTimeoutMs) {
continue;
}
if (request.state === "approved") {
logger.warn(
`Approved host ${request.result.name} (${request.result.hostId}) was never picked up ` +
"by the host, revoke it on the hosts page if it is not needed"
);
}
this._remove(request);
}
}
}
+300 -14
View File
@@ -1,9 +1,14 @@
import { isDeepStrictEqual } from "node:util";
import * as lib from "@clusterio/lib";
import { logger } from "@clusterio/lib";
import type Controller from "./Controller.js";
import type HostConnection from "./HostConnection.js";
import InstanceRecord from "./InstanceRecord.js";
type InheritedFields = lib.InstanceConfigFields["instance.inherited_fields"];
type InheritableField = keyof lib.InstanceConfigFields & string;
export default class InstanceManager {
private static readonly DefaultFactorioSettings = {
tags: ["clusterio"],
@@ -24,6 +29,9 @@ export default class InstanceManager {
non_blocking_saving: false,
} as const;
/** Non-zero while template values are being copied into inheriting instances */
private _inheriting = 0;
constructor(
public readonly records: lib.SubscribableDatastore<InstanceRecord>,
private readonly _controller: Controller,
@@ -65,12 +73,12 @@ export default class InstanceManager {
*
* @param instanceId - ID of instance to get.
* @returns The instance record.
* @throws {module:lib.RequestError} if the instance does not exist.
* @throws {module:lib.NotFoundError} if the instance does not exist.
*/
getForRequest(instanceId: number): InstanceRecord {
const instance = this.records.get(instanceId);
if (!instance) {
throw new lib.RequestError(`Instance with ID ${instanceId} does not exist`);
throw new lib.NotFoundError(`Instance with ID ${instanceId} does not exist`);
}
return instance;
}
@@ -106,11 +114,271 @@ export default class InstanceManager {
*/
addInstanceHooks(instance: InstanceRecord): void {
instance.config.on("fieldChanged", (field: string, curr: unknown, prev: unknown) => {
if (field === "instance.template_id") {
this._linkMatchingFields(instance);
} else if (!this._inheriting && instance.config.isInherited(field)) {
this._unlinkEditedField(instance, field, curr, prev);
}
this.records.set(instance);
this._notifyConfigFieldChanged(instance, field, curr, prev);
this._updateInheritors(instance, field, curr);
});
}
/* ------------------------------------------------------------------------- */
/* Inheritance */
/* ------------------------------------------------------------------------- */
/**
* Get the instances that inherit directly from the given template.
*
* @param templateId - ID of the template.
* @returns Instances with `instance.template_id` set to templateId.
*/
inheritorsOf(templateId: number): InstanceRecord[] {
return [...this.records.values()].filter(
instance => instance.config.get("instance.template_id") === templateId
);
}
/**
* Check that new values for `instance.is_template` and `instance.template_id` are allowed.
*
* @param config - Current config of the instance.
* @param isTemplate - New value for `instance.is_template`.
* @param templateId - New value for `instance.template_id`.
* @throws {module:lib.ValidationError} if the template is not a template or would create a cycle.
* @throws {module:lib.ConflictError} if the instance is assigned or still has inheritors.
*/
checkInheritance(config: lib.InstanceConfig, isTemplate: boolean, templateId: number | null): void {
const instanceId = config.get("instance.id");
if (isTemplate && config.get("instance.assigned_host") !== null) {
throw new lib.ConflictError("Instance must be unassigned before it can be made a template");
}
if (!isTemplate && config.get("instance.is_template")) {
const count = this.inheritorsOf(instanceId).length;
if (count) {
throw new lib.ConflictError(`Instance is the template of ${count} other instance(s)`);
}
}
if (templateId === null) {
return;
}
const template = this.records.get(templateId);
if (!template) {
throw new lib.NotFoundError(`Instance with ID ${templateId} does not exist`);
}
if (!template.config.get("instance.is_template")) {
throw new lib.ValidationError(`Instance ${template.config.get("instance.name")} is not a template`);
}
const visited = new Set<number>();
for (let id: number | null = templateId; id !== null && !visited.has(id);) {
if (id === instanceId) {
throw new lib.ValidationError("Templates cannot inherit from themselves");
}
visited.add(id);
id = this.records.get(id)?.config.get("instance.template_id") ?? null;
}
}
/**
* Copy the inheritable values from the template set in `instance.template_id`
* into a config that is not yet part of the cluster and mark them inherited.
*
* @param config - Config of the instance to be created.
* @param own - Fields set by the creator, object fields keep their properties as overrides.
*/
applyTemplate(config: lib.InstanceConfig, own: string[] = []): void {
const template = this._templateOf(config);
if (!template) {
return;
}
const inherited: InheritedFields = {};
for (const name of InstanceManager._inheritableFields()) {
const def = lib.InstanceConfig.fieldDefinitions[name];
if (def.type === "object") {
inherited[name] = own.includes(name) ? Object.keys(config.get(name) as object ?? {}) : [];
} else if (!own.includes(name)) {
inherited[name] = true;
}
}
config.set("instance.inherited_fields", inherited);
config.update(InstanceManager._inheritedValues(config, template.config, Object.keys(inherited)), false);
}
/**
* Make a field, or a property of an object field, take its value from the template again.
*
* @param instanceId - ID of the instance to update.
* @param field - Name of the field to inherit.
* @param prop - Property of an object field to inherit, inherits all properties if not given.
* @throws {module:lib.ValidationError} if the instance has no template or the field cannot be inherited.
*/
inheritField(instanceId: number, field: string, prop?: string): void {
const instance = this.getForRequest(instanceId);
if (!this._templateOf(instance.config)) {
throw new lib.ValidationError("Instance does not have a template");
}
if (!lib.InstanceConfig.isInheritable(field)) {
throw new lib.ValidationError(`Field ${field} cannot be inherited`);
}
const name = field as InheritableField;
const inherited = { ...instance.config.get("instance.inherited_fields") };
if (lib.InstanceConfig.fieldDefinitions[name].type !== "object") {
inherited[field] = true;
} else if (prop === undefined) {
inherited[field] = [];
} else {
const entry = inherited[field];
const overrides = entry instanceof Array ? entry : Object.keys(instance.config.get(name) as object ?? {});
inherited[field] = overrides.filter(p => p !== prop);
}
instance.config.set("instance.inherited_fields", inherited);
this._copyInheritedValues(instance, [field]);
}
/**
* Stop inheriting the given field or properties, used when a value is explicitly set.
*
* @param instance - Instance the field was set on.
* @param field - Name of the field set.
* @param props - Properties set on an object field.
*/
overrideField(instance: InstanceRecord, field: string, props?: string[]): void {
const entry = instance.config.get("instance.inherited_fields")[field];
if (!instance.config.isInherited(field)) {
return;
}
const inherited = { ...instance.config.get("instance.inherited_fields") };
if (entry instanceof Array && props) {
inherited[field] = [...new Set([...entry, ...props])];
} else {
delete inherited[field];
}
instance.config.set("instance.inherited_fields", inherited);
}
private _templateOf(config: lib.InstanceConfig) {
const templateId = config.get("instance.template_id");
return templateId !== null ? this.records.get(templateId) : undefined;
}
private static _inheritableFields() {
return Object.keys(lib.InstanceConfig.fieldDefinitions).filter(
name => lib.InstanceConfig.isInheritable(name)
) as InheritableField[];
}
/** Default value of a field for the given config */
private static _defaultValue(name: InheritableField, config: lib.InstanceConfig): unknown {
const def = lib.InstanceConfig.fieldDefinitions[name];
if (name === "factorio.settings") {
return InstanceManager.DefaultFactorioSettings;
}
return typeof def.initialValue === "function" ? def.initialValue(config) : def.initialValue ?? null;
}
/** Values the given inherited fields take from the template, with object overrides applied */
private static _inheritedValues(config: lib.InstanceConfig, template: lib.InstanceConfig, fields: string[]) {
const inherited = config.get("instance.inherited_fields");
const values: Record<string, any> = {};
for (const field of fields) {
const name = field as InheritableField;
const entry = inherited[field];
if (!entry) {
continue;
}
const value = structuredClone(template.get(name)) as any;
if (entry instanceof Array) {
const own = config.get(name) as Record<string, unknown> ?? {};
for (const prop of entry) {
if (Object.hasOwn(own, prop)) {
value[prop] = structuredClone(own[prop]);
}
}
}
values[field] = value;
}
return values;
}
/** Update the given inherited fields of a live instance from its template */
private _copyInheritedValues(instance: InstanceRecord, fields: string[]): void {
const template = this._templateOf(instance.config);
if (!template) {
return;
}
const values = InstanceManager._inheritedValues(instance.config, template.config, fields);
this._inheriting += 1;
try {
instance.config.update(values, true);
} finally {
this._inheriting -= 1;
}
}
/** Unlink a field, or the changed properties of an object field, edited on the instance */
private _unlinkEditedField(instance: InstanceRecord, field: string, curr: unknown, prev: unknown): void {
const entry = instance.config.get("instance.inherited_fields")[field];
if (!(entry instanceof Array)) {
this.overrideField(instance, field);
return;
}
const currObj = curr as Record<string, unknown> ?? {};
const prevObj = prev as Record<string, unknown> ?? {};
const changed = Object.keys(currObj).filter(p => !isDeepStrictEqual(currObj[p], prevObj[p]));
const removed = Object.keys(prevObj).filter(p => !Object.hasOwn(currObj, p));
const overrides = [...new Set([...entry, ...changed])].filter(p => !removed.includes(p));
const inherited = { ...instance.config.get("instance.inherited_fields"), [field]: overrides };
instance.config.set("instance.inherited_fields", inherited);
if (removed.length) {
// Removed properties fall back to the template value
this._copyInheritedValues(instance, [field]);
}
}
/** Recompute inherited fields after template_id changed, values equal to the template or default are inherited */
private _linkMatchingFields(instance: InstanceRecord): void {
const template = this._templateOf(instance.config);
const inherited: InheritedFields = {};
if (template) {
for (const name of InstanceManager._inheritableFields()) {
const value = instance.config.get(name) as any;
const templateValue = template.config.get(name) as any;
const defaultValue = InstanceManager._defaultValue(name, instance.config) as any;
if (lib.InstanceConfig.fieldDefinitions[name].type === "object") {
inherited[name] = Object.keys(value ?? {}).filter(prop => !(
isDeepStrictEqual(value[prop], templateValue?.[prop])
|| defaultValue && isDeepStrictEqual(value[prop], defaultValue[prop])
) || name === "factorio.settings" && prop === "name");
} else if (
isDeepStrictEqual(value, templateValue)
|| defaultValue !== undefined && isDeepStrictEqual(value, defaultValue)
) {
inherited[name] = true;
}
}
}
instance.config.set("instance.inherited_fields", inherited);
this._copyInheritedValues(instance, Object.keys(inherited));
}
/** Copy a changed value from a template to the instances inheriting it */
private _updateInheritors(template: InstanceRecord, field: string, value: unknown): void {
if (!lib.InstanceConfig.isInheritable(field)) {
return;
}
for (const instance of this.inheritorsOf(template.id)) {
if (!instance.config.isInherited(field)) {
continue;
}
this._copyInheritedValues(instance, [field]);
this._controller.instanceConfigUpdated(instance).catch(err => {
logger.error(`Failed to send config of instance ${instance.id} to its host:\n${err.stack}`);
});
}
}
/* ------------------------------------------------------------------------- */
/* Lifecycle */
/* ------------------------------------------------------------------------- */
@@ -125,7 +393,7 @@ export default class InstanceManager {
* @param instanceConfig - Config to base the newly created instance on.
* @param _suppressChanges - When true, side effects are suppressed, caller takes ownership.
* @returns The created instance record.
* @throws {module:lib.RequestError} if an instance with the same ID already exists.
* @throws {module:lib.ConflictError} if an instance with the same ID already exists.
*/
async createInstance(
instanceConfig: lib.InstanceConfig,
@@ -133,16 +401,27 @@ export default class InstanceManager {
): Promise<InstanceRecord> {
const instanceId = instanceConfig.get("instance.id");
if (this.records.has(instanceId)) {
throw new lib.RequestError(`Instance with ID ${instanceId} already exists`);
throw new lib.ConflictError(`Instance with ID ${instanceId} already exists`);
}
const controllerName = this._controller.config.get("controller.name");
instanceConfig.set("factorio.settings", {
name: `${controllerName} - ${instanceConfig.get("instance.name")}`,
description: `Clusterio instance for ${controllerName}`,
...InstanceManager.DefaultFactorioSettings,
...instanceConfig.get("factorio.settings"),
});
const serverName = `${controllerName} - ${instanceConfig.get("instance.name")}`;
const settings = instanceConfig.get("factorio.settings");
if (!instanceConfig.isInherited("factorio.settings")) {
instanceConfig.set("factorio.settings", {
name: serverName,
description: `Clusterio instance for ${controllerName}`,
...InstanceManager.DefaultFactorioSettings,
...settings,
});
} else if (instanceConfig.isInherited("factorio.settings", "name")) {
// Each instance gets its own server name
instanceConfig.setProp("factorio.settings", "name", serverName);
const inherited = instanceConfig.get("instance.inherited_fields");
instanceConfig.set("instance.inherited_fields", {
...inherited, "factorio.settings": [...inherited["factorio.settings"] as string[], "name"],
});
}
const instance = new InstanceRecord(instanceConfig, "unassigned");
this.addInstanceHooks(instance);
@@ -172,8 +451,8 @@ export default class InstanceManager {
* @param hostId - ID of host to assign the instance to. If undefined,
* the instance will be unassigned.
*
* @throws {module:lib.RequestError} if the instance does not exist.
* @throws {module:lib.RequestError} if the target host is not connected.
* @throws {module:lib.NotFoundError} if the instance does not exist.
* @throws {module:lib.UnavailableError} if the target host is not connected.
*/
async assignInstance(
instanceId: number,
@@ -186,12 +465,15 @@ export default class InstanceManager {
if (currentAssignedHost === hostId) {
return;
}
if (hostId !== null && instance.config.get("instance.is_template")) {
throw new lib.ValidationError("Templates cannot be assigned to a host");
}
let newHostConnection: HostConnection | undefined;
if (hostId !== null) {
newHostConnection = hostConnections.get(hostId);
if (!newHostConnection) {
throw new lib.RequestError("Target host is not connected to the controller");
throw new lib.UnavailableError("Target host is not connected to the controller");
}
}
@@ -247,11 +529,15 @@ export default class InstanceManager {
*
* @param instanceId - ID of instance to delete.
*
* @throws {module:lib.RequestError} if the instance does not exist.
* @throws {module:lib.NotFoundError} if the instance does not exist.
*/
async deleteInstance(instanceId: number): Promise<void> {
const instance = this.getForRequest(instanceId);
const hostId = instance.config.get("instance.assigned_host");
const inheritors = this.inheritorsOf(instanceId).length;
if (inheritors) {
throw new lib.ConflictError(`Instance is the template of ${inheritors} other instance(s)`);
}
if (hostId !== null) {
await this._controller.sendTo(
@@ -83,6 +83,9 @@ export default class InstanceRecord {
this.startedAtMs,
this.updatedAtMs,
this.config.get("instance.exclude_from_start_all"),
this.config.get("instance.is_template"),
this.config.get("instance.template_id") ?? undefined,
this.config.get("factorio.mod_pack_id") ?? undefined,
);
}
+30 -7
View File
@@ -40,8 +40,8 @@ export default class UserManager {
}
// We intentionally write the roles first because if it fails then the old user file will still contain them
await lib.safeOutputFile(rolesFilePath, JSON.stringify(combinedJson.roles));
await lib.safeOutputFile(usersFilePath, JSON.stringify(combinedJson.users));
await lib.atomicOverwriteFile(rolesFilePath, JSON.stringify(combinedJson.roles));
await lib.atomicOverwriteFile(usersFilePath, JSON.stringify(combinedJson.users));
} catch (err: any) {
if (err.code !== "ENOENT") {
throw err;
@@ -151,11 +151,34 @@ export default class UserManager {
* @returns JWT access token for the user.
*/
signUserToken(user: UserRecord): string {
return jwt.sign(
{ aud: "user", user: user.id },
// auth secret is never null after startup
Buffer.from(this._controllerConfig.get("controller.auth_secret")!, "base64")
);
return jwt.sign({ aud: "user", user: user.id }, this._authSecret);
}
/**
* Check a user token
*
* @param token - User token to check.
* @returns the user the token was issued to.
* @throws if the token is invalid, its user does not exist or it was issued before tokenValidAfter.
*/
verifyUserToken(token: string): User {
const payload = jwt.verify(token, this._authSecret, { audience: "user" });
if (typeof payload === "string") {
throw new Error("unexpected JsonWebToken type");
}
const user = this.getByNameMutable(payload.user);
if (!user) {
throw new Error("invalid user");
}
if (!payload.iat || payload.iat < user.tokenValidAfter) {
throw new Error("invalid token");
}
return user;
}
private get _authSecret() {
// auth secret is never null after startup
return Buffer.from(this._controllerConfig.get("controller.auth_secret")!, "base64");
}
/** Clear all player stats for specific instance */
+9 -25
View File
@@ -252,21 +252,9 @@ ${err.stack}`
}
try {
// migrate: allow pre-rename tokens issued to hosts before alpha-14
const tokenPayload = jwt.verify(data.token, this.controller.authSecret, { audience: ["host", "slave"] });
if (typeof tokenPayload === "string") {
throw new Error("unexpected JsonWebToken type");
}
// migrate: allow pre-rename tokens issued to hosts before alpha-14
if ((tokenPayload.host !== undefined ? tokenPayload.host : tokenPayload.slave) !== data.id) {
if (this.controller.hosts.verifyHostToken(data.token).id !== data.id) {
throw new Error("missmatched host id");
}
let host = this.controller.hosts.get(data.id);
if (!tokenPayload.iat || tokenPayload.iat < (host?.tokenValidAfter ?? 0)) {
throw new Error("invalid token");
}
} catch (err: any) {
logger.verbose(`WsServer | authentication failed for ${this.remoteAddr(req)}`);
wsRejectedConnectionsCounter.inc();
@@ -303,6 +291,13 @@ ${err.stack}`
);
}
// The host may have been revoked while waiting for the old connection to close.
if (!this.controller.hosts.has(data.id)) {
wsRejectedConnectionsCounter.inc();
socket.close(lib.ConnectionClosed.Unauthorized, "Authentication failed: host was revoked");
return;
}
let [connector, sessionToken] = this.createSession(new lib.Address(lib.Address.host, data.id));
// Handle close before HostConnection does. This avoids a host
@@ -325,18 +320,7 @@ ${err.stack}`
async registerControl(data: lib.RegisterControlData, socket: WebSocket, req: IncomingMessage) {
let user;
try {
const tokenPayload = jwt.verify(data.token, this.controller.authSecret, { audience: "user" });
if (typeof tokenPayload === "string") {
throw new Error("unexpected JsonWebToken type");
}
user = this.controller.users.getByNameMutable(tokenPayload.user);
if (!user) {
throw new Error("invalid user");
}
if ((tokenPayload.iat || 0) < user.tokenValidAfter) {
throw new Error("invalid token");
}
user = this.controller.users.verifyUserToken(data.token);
user.checkPermission("core.control.connect");
} catch (err: any) {
+150 -40
View File
@@ -1,11 +1,11 @@
import type { Application, Request, Response } from "express";
import express, { type Application, type Request, type Response } from "express";
import Controller from "./Controller.js";
import HostPairingManager, { HostPairingError } from "./HostPairingManager.js";
import busboy from "busboy";
import crypto from "crypto";
import events from "events";
import fs from "node:fs/promises";
import jwt from "jsonwebtoken";
import path from "path";
import nodeStream from "stream";
import util from "util";
@@ -20,9 +20,10 @@ declare global {
namespace Express {
export interface Locals {
controller: Controller,
mainBundle: string,
webBundle: { main: string, imports: Record<string, string> },
devServer?: import("vite").ViteDevServer,
staticRoot: string | null,
devPlugins: Map<string, number>,
devPlugins?: Map<string, string>,
streams: Map<string, ProxyStream>,
pendingMetrics?: Promise<string>,
}
@@ -124,7 +125,7 @@ async function getMetrics(req: Request, res: Response) {
/**
* Removes the static/ prefix webpack adds to bundle paths so they can be
* Removes the static/ prefix from bundle paths in manifest.json so they can be
* resolved relative to staticRoot.
*/
export function stripStaticPrefix(bundlePath: string) {
@@ -140,22 +141,18 @@ function getPlugins(req: Request, res: Response) {
// Note: Cast through unknown is needed because load_plugin is
// defined at runtime and no other fields in the controller config
// currently have the boolean type.
let web: { main:any, error:string|undefined } = { main: undefined, error: undefined};
let web: { main: any, error: string|undefined } = { main: undefined, error: undefined};
let devPlugins = req.app.locals.devPlugins;
if (devPlugins && devPlugins.has(name)) {
let stats = res.locals.webpack.devMiddleware.stats.stats[devPlugins.get(name)!];
web.main = stats.toJson().assetsByChunkName[name][0];
web.main = devPlugins.get(name);
} else if (pluginInfo.manifest) {
web.main = pluginInfo.manifest[`${pluginInfo.name}.js`];
web.main = pluginInfo.manifest.main;
if (!web.main) {
web.error = `Missing ${pluginInfo.name}.js entry in manifest.json`;
web.error = "Incompatible dist/web/manifest.json, the plugin must be rebuilt with Vite";
}
} else if (lib.pluginNeedsWebBuild(pluginInfo)) {
web.error = "Missing dist/web/manifest.json";
}
if (web.main === "remoteEntry.js") {
web.error = "Incompatible old remoteEntry.js entrypoint.";
}
if (web.main) {
web.main = stripStaticPrefix(web.main);
}
@@ -191,17 +188,7 @@ function validateHostToken(req: Request, res: Response, next: any) {
}
try {
const tokenPayload = jwt.verify(token, req.app.locals.controller.authSecret, { audience: "host" });
if (typeof tokenPayload === "string") {
throw new Error("unexpected JsonWebToken type");
}
let host = req.app.locals.controller.hosts.get(tokenPayload.host);
if (!host) {
throw new Error("invalid host");
}
if (!tokenPayload.iat || tokenPayload.iat < (host.tokenValidAfter ?? 0)) {
throw new Error("invalid token");
}
req.app.locals.controller.hosts.verifyHostToken(token);
} catch (err) {
res.sendStatus(401);
return;
@@ -218,19 +205,7 @@ function validateUserToken(req: Request, res: Response, next: any) {
}
try {
const tokenPayload = jwt.verify(token, req.app.locals.controller.authSecret, { audience: "user" });
if (typeof tokenPayload === "string") {
throw new Error("unexpected JsonWebToken type");
}
let user = req.app.locals.controller.users.getByName(tokenPayload.user);
if (!user) {
throw new Error("invalid user");
}
if (!tokenPayload.iat || tokenPayload.iat < user.tokenValidAfter) {
throw new Error("invalid token");
}
res.locals.user = user;
res.locals.user = req.app.locals.controller.users.verifyUserToken(token);
} catch (err) {
res.sendStatus(401);
return;
@@ -239,6 +214,105 @@ function validateUserToken(req: Request, res: Response, next: any) {
next();
}
// Responds with the status of a pairing error, returns false for other errors.
function sendHostPairingError(res: Response, err: unknown) {
if (!(err instanceof HostPairingError)) {
return false;
}
if (err.retryAfter !== undefined) {
res.set("Retry-After", String(err.retryAfter));
}
res.status(err.status).json({ error: err.message });
return true;
}
// Checks recovery mode and the request id for the pairing endpoints, returns false if responded to.
function checkHostPairingRequest(req: Request, res: Response, needsRequestId: boolean) {
if (req.app.locals.controller.recoveryMode) {
res.status(503).json({ error: "Controller is in recovery mode" });
return false;
}
if (needsRequestId && typeof req.body?.requestId !== "string") {
res.status(400).json({ error: "requestId is required" });
return false;
}
return true;
}
// Request from a host for a host to join the cluster.
function createHostPairing(req: Request, res: Response) {
if (!checkHostPairingRequest(req, res, false)) {
return;
}
const controller = req.app.locals.controller;
const remoteAddress = controller.wsServer.remoteAddr(req);
try {
const name = HostPairingManager.validateName(req.body?.name);
const version = typeof req.body?.version === "string" ? req.body.version.slice(0, 64) : "";
// A user allowed to approve hosts can skip the wait by passing their token.
const token = getAccessToken(req);
if (token !== undefined) {
let user;
try {
user = controller.users.verifyUserToken(token);
} catch (err) {
res.status(401).json({ error: "Invalid user token" });
return;
}
try {
user.checkPermission("core.host.approve");
} catch (err) {
logger.audit(
`User ${user.name} tried to approve host ${name} from ${remoteAddress} without permission`
);
res.status(403).json({ error: `User ${user.name} is not allowed to approve hosts` });
return;
}
const host = controller.hosts.createHost(name);
logger.audit(
`User ${user.name} approved host ${host.name} (${host.id}) from ${remoteAddress} with their token`
);
res.status(200).json({ hostId: host.id, token: controller.hosts.signHostToken(host.id), name: host.name });
return;
}
res.status(202).json(controller.hostPairing.create(name, version, remoteAddress));
} catch (err) {
if (!sendHostPairingError(res, err)) {
throw err;
}
}
}
// Poll from a host for the result of its request.
function pollHostPairing(req: Request, res: Response) {
if (!checkHostPairingRequest(req, res, true)) {
return;
}
try {
const result = req.app.locals.controller.hostPairing.poll(req.body.requestId);
if (result) {
res.status(200).json(result);
} else {
res.status(202).json({ status: "pending" });
}
} catch (err) {
if (!sendHostPairingError(res, err)) {
throw err;
}
}
}
// Cancel from a host that stopped waiting.
function cancelHostPairing(req: Request, res: Response) {
if (!checkHostPairingRequest(req, res, true)) {
return;
}
req.app.locals.controller.hostPairing.cancel(req.body.requestId);
res.sendStatus(204);
}
// Handle an uploaded export package.
async function uploadExport(req: Request, res: Response) {
if (req.get("Content-Type") !== "application/zip") {
@@ -293,7 +367,7 @@ async function uploadExport(req: Request, res: Response) {
let { name, ext } = path.posix.parse(filePath);
let hash = await lib.hashStream(await zip.openStream(file));
assets[name] = `${name}.${hash}${ext}`;
await lib.safeOutputFile(path.join("static", `${name}.${hash}${ext}`), content);
await lib.atomicOverwriteFile(path.join("static", `${name}.${hash}${ext}`), content);
}
modPack.exportManifest = new lib.ExportManifest(assets, new Date().toISOString());
@@ -303,12 +377,36 @@ async function uploadExport(req: Request, res: Response) {
res.sendStatus(200);
}
// Reached when no static file matched, drops exports whose files were removed
export function missingExportAsset(req: Request, res: Response, next: any) {
const file = req.path.slice(1);
if (!["GET", "HEAD"].includes(req.method) || !file) {
next();
return;
}
const modPacks = res.app.locals.controller.modPacks;
const affected = [...modPacks.valuesMutable()].filter(
modPack => modPack.exportManifest && Object.values(modPack.exportManifest.assets).includes(file)
);
if (!affected.length) {
next();
return;
}
for (const modPack of affected) {
Controller.dropExport(modPack, `${file} was removed from disk`);
}
modPacks.setMany(affected);
res.status(404).type("text/plain").send(
`Export data file ${file} is missing, run export-data on an instance to regenerate it`
);
}
type ProxyStreamEvents = {
"error": [ err: any ],
"close": [],
"timeout": [],
"source": [],
}
};
export interface ProxyStream {
id: string;
@@ -548,7 +646,7 @@ function checkModName(name: string) {
try {
lib.checkFilename(name);
} catch (err: any) {
throw new lib.RequestError(`Mod name ${err.message}`);
throw new lib.ValidationError(`Mod name ${err.message}`);
}
}
@@ -689,6 +787,18 @@ export function addRouteHandlers(app: Application) {
);
app.get("/api/plugins", getPlugins);
app.get("/api/cluster-name", getClusterName);
// Set before parsing so errors from the body parser and token check are not cached either.
const noStore = (req: Request, res: Response, next: any) => {
res.set("Cache-Control", "no-store");
next();
};
const pairingJson = express.json({ limit: "4kb" });
app.post("/api/host-pairing", noStore, pairingJson, createHostPairing);
app.post("/api/host-pairing/poll", noStore, pairingJson, pollHostPairing);
app.post("/api/host-pairing/cancel", noStore, pairingJson, cancelHostPairing);
app.get("/api/host-pairing/check", noStore, validateHostToken, (req: Request, res: Response) => {
res.json({ valid: true });
});
app.put("/api/upload-export",
validateHostToken,
(req:Request, res:Response, next:any) => uploadExport(req, res).catch(next)
@@ -0,0 +1,3 @@
// This needs to be a separate file due to ECMAScript Modules evaluating code in
// imported modules before any code in the file that imports it.
(global as any).lazySchemaCompilation = true;
-4
View File
@@ -1,4 +0,0 @@
{
"entryPoints": ["index.ts"],
"includeVersion": true
}
+6
View File
@@ -0,0 +1,6 @@
import { defineConfig } from "vite";
import { clusterioApp } from "@clusterio/web_ui/vite.mjs";
export default defineConfig({
plugins: [clusterioApp({ entry: "web/index.tsx" })],
});
+3 -3
View File
@@ -5,6 +5,7 @@
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="theme-color" content="#2a2a2a" />
<title>__CLUSTER_NAME__</title>
<script type="importmap">__IMPORT_MAP__</script>
<style>
html {
color: #fff;
@@ -26,7 +27,6 @@
<div id="root">
<div id="bootstrap-msg">Loading web interface...</div>
</div>
<script id="script-bundle" src="__STATIC_ROOT____MAIN_BUNDLE__" defer></script>
<script>
webRoot = new URL("__WEB_ROOT__", document.location).pathname;
staticRoot = new URL("__STATIC_ROOT__", document.location).href;
@@ -41,9 +41,9 @@
bootstrapMsg.textContent = `${errorMsg}\nSee web console for details.`;
}
}
const bundleScript = document.getElementById("script-bundle");
bundleScript.addEventListener("error", () => scriptError("Main bundle failed to load."));
window.addEventListener("error", event => scriptError(event.error));
</script>
<script id="script-bundle" type="module" src="__STATIC_ROOT____MAIN_BUNDLE__"
onerror="scriptError('Main bundle failed to load.')"></script>
</body>
</html>
+1
View File
@@ -1,3 +1,4 @@
// eslint-disable-next-line no-implicit-globals
declare function scriptError(err: any): void;
import("@clusterio/web_ui")
.then(webUi => webUi.bootstrap())
-31
View File
@@ -1,31 +0,0 @@
"use strict";
const path = require("path");
const webpack = require("webpack");
const { merge } = require("webpack-merge");
const common = require("@clusterio/web_ui/webpack.common");
module.exports = (env = {}, argv = {}) => merge(common(env, argv), {
context: __dirname,
entry: "./web/index.tsx",
devServer: {
contentBase: "./dist/web",
},
output: {
path: path.resolve(__dirname, "dist", "web"),
},
plugins: [
new webpack.container.ModuleFederationPlugin({
name: "controller",
shared: {
"@clusterio/lib": { singleton: true },
"@clusterio/web_ui": { singleton: true },
"antd": { singleton: true },
"react": { singleton: true },
"react-dom": { singleton: true },
"react-router": { singleton: true },
"react-router-dom": { singleton: true },
},
}),
],
});
+110
View File
@@ -4,3 +4,113 @@ Installer for Clusterio.
Install the latest [Node.js LTS release](https://nodejs.org/en/download) before running the installer.
See [the Clusterio installation guide](https://github.com/clusterio/clusterio#installation) for platform-specific instructions.
Run the installer in a new empty directory:
npm init "@clusterio"
It asks which components to install and how to set them up.
Every question can be answered with an option instead, pass options to the installer after `--`:
npm init "@clusterio" -- --mode host --host-name my-host --controller-url https://example.com/
The installer runs without prompts when all the questions for the mode are answered with options.
## Modes
- `standalone`: controller and host on this computer. The host is approved like any other host the first time it starts.
- `controller`: controller only.
- `host`: host only, joins the cluster of an existing controller, see [Setting up a host](#setting-up-a-host).
- `ctl`: the clusterioctl command line interface only.
- `plugins`: plugins only, for adding plugins to an existing installation.
- `plugin-template`: files for writing a new plugin.
## Options
| Option | Modes | Description |
| --- | --- | --- |
| `--mode <mode>` | all | Operating mode to install. |
| `--admin <name>` | controller | Name of the admin account to create. |
| `--http-port <port>` | controller | Port the controller web interface listens on, defaults to 8080. |
| `--host-name <name>` | host | Name to request for the host, defaults to the name of the computer. |
| `--controller-url <url>` | host, ctl | URL of the controller, the same as for its web interface. |
| `--controller-token <token>` | ctl | Authentication token saved for clusterioctl. |
| `--public-address <address>` | host | DNS name or IP address players connect to this computer with. |
| `--factorio-dir <path>` | host | Path to the Factorio installation. |
| `--self-signed-cert <path>` | host | Certificate to trust for a controller using a self-signed certificate. It is added to the run scripts. |
| `--remote-npm`, `--no-remote-npm` | controller, host | Allow or deny remote updates, plugin updates and plugin installs from the web interface. |
| `--plugins <package...>` | all | Plugins to install, pass `--plugins` without packages to install none. |
| `--plugin-template <template...>`, `--plugin-name <name>` | plugin-template | Templates to use and name of the new plugin. |
| `--log-level <level>` | all | Log level to print, defaults to info. |
| `--allow-install-as-root` | all | Linux only, allow installing as root (not recommended). |
| `--migrate-rename` | | Migrate an installation from before the slave/master rename. |
Options for the `controller` and `host` modes also work in the `standalone` mode, except for `--controller-url` as the host connects to the controller on the same computer.
`--host-name` defaults to `local` in the `standalone` mode.
## Setting up a host
A new host has to be approved before it can join the cluster:
1. Run the installer with the host mode on the computer for the host.
It asks for a name for the host, the controller URL, the public address, the Factorio installation, whether to allow remote updates and which plugins to install.
2. Start the host with `run-host.cmd`, `run-host.sh` or the systemd service in the `systemd` folder.
The first time it starts it prints a code and waits for the host to be approved:
Waiting for approval of "my-host" with code ABC-DEF, open this link to approve it:
https://example.com/hosts?approve=ABC-DEF
Or run: npx clusterioctl host approve ABC-DEF
3. Open the link to approve the request in the web interface, where its name can be changed first.
It can also be approved under Add host on the Hosts page, or with `npx clusterioctl host approve ABC-DEF`.
Only approve a request if its code matches the one shown by the host.
Approving requires the `core.host.approve` permission.
4. The host saves `host.id`, `host.name` and `host.controller_token` to `config-host.json` and starts up.
Later starts connect to the cluster straight away.
The host keeps waiting until the request is approved, also if the controller is restarted or can't be reached for a while.
Press Ctrl+C to stop waiting.
If the request is rejected the host exits with code 8, which the run scripts do not restart on.
Start the host with `npx clusteriohost run --user-token <token>` and the token of a user that has the `core.host.approve` permission to approve the host without waiting.
The token is only used for approving the host and is not saved.
If the controller uses a self-signed certificate pass it with `--self-signed-cert <path>`, see [setting up TLS](https://github.com/clusterio/clusterio/blob/master/docs/setting-up-tls.md#creating-and-using-a-self-signed-certificate).
The installer adds it to the run scripts.
When a host is revoked in the web interface or with `npx clusterioctl host revoke <host>` its token stops working and a running host shuts down.
Starting it again requests to join the cluster again, and the host gets a new id once approved.
To move a host to another controller, change `host.controller_url` with `npx clusteriohost config set host.controller_url <url>`.
The new controller does not accept the token of the host, so the host requests to join it the next time it starts.
## Docker
To run a host in a container install it when building the image with all answers given as options.
The host requests to join the cluster the first time the container starts.
```dockerfile
FROM node:22
# Install the Factorio headless server to /opt/factorio here
RUN mkdir /clusterio && chown node:node /clusterio
USER node
WORKDIR /clusterio
RUN npm init "@clusterio" -- --mode host --host-name my-host --controller-url https://example.com/ \
--public-address factorio.example.com --factorio-dir /opt/factorio --no-remote-npm --plugins
VOLUME /clusterio
CMD exec ./node_modules/.bin/clusteriohost run ${USER_TOKEN:+--user-token "$USER_TOKEN"}
```
docker run -d --name my-host -v my-host:/clusterio -p 34100-34199:34100-34199/udp my-host-image
docker logs my-host
- The volume holds `config-host.json`, without it every new container requests to join as a new host.
Docker copies the installation from the image into a new empty volume, later image updates do not change the installation in an existing volume.
- The code to approve is printed to the container log, or set `USER_TOKEN` to approve the host without waiting.
- Stop the container after revoking its host, otherwise a restart requests to join again.
+178 -199
View File
@@ -1,10 +1,14 @@
#!/usr/bin/env node
import checkbox from "@inquirer/checkbox";
import confirm from "@inquirer/confirm";
import input from "@inquirer/input";
import select from "@inquirer/select";
import child_process from "node:child_process";
import fs from "node:fs/promises";
import inquirer from "inquirer";
import os from "node:os";
import path from "node:path";
import readline from "node:readline";
import stream from "node:stream";
import util from "node:util";
import yargs from "yargs";
@@ -16,15 +20,13 @@ const scriptExt = process.platform === "win32" ? ".cmd" : "";
import { escapeArg } from "./escape_arg.js";
// I hate this, there is a bug with rl.close on windows that was meant to have been fixed in Node14.4
// See nodejs/node#21771 and nodejs/node#30701 for the apparent fix that was applied
// One option is to not call rl.close: SBoudrias/Inquirer.js#767
// But this comes with its own issues: SBoudrias/Inquirer.js#899
// See nodejs/node#21771, nodejs/node#30701 and nodejs/node#31762
// The solution below was adapted from aws-amplify/amplify-cli#12347
if (process.platform === "win32") {
const _inquirer_close = inquirer.ui.Prompt.prototype.close;
inquirer.ui.Prompt.prototype.close = function() {
this.rl.terminal = false;
_inquirer_close.call(this);
const close = readline.Interface.prototype.close;
readline.Interface.prototype.close = function() {
this.terminal = false;
close.call(this);
};
}
@@ -102,7 +104,7 @@ async function pathExists(filePath) {
return true;
}
async function safeOutputFile(file, data, options={}) {
async function atomicOverwriteFile(file, data, options={}) {
let { dir, name, ext } = path.parse(file);
let temporary = path.join(dir, `${name}.tmp${ext}`);
if (dir) {
@@ -161,21 +163,48 @@ async function execCtl(args) {
return await execFile(path.join("node_modules", ".bin", `clusterioctl${scriptExt}`), args);
}
function validateHostToken(token) {
let parts = token.split(".");
if (parts.length !== 3) {
throw new InstallError("Invalid token");
}
// Same characters as the controller refuses in host names.
const hostNameControlCharacters = /[\p{Cc}\u202a-\u202e\u2066-\u2069]/u;
let parsed;
function validateHostName(value) {
if (!value || !value.trim()) {
return "Host name may not be empty";
}
if (value.trim().length > 100) {
return "Host name may be at most 100 characters";
}
if (hostNameControlCharacters.test(value)) {
return "Host name may not contain control characters";
}
return true;
}
function validateControllerUrl(value) {
let url;
try {
parsed = JSON.parse(Buffer.from(parts[1], "base64"));
url = new URL(value);
} catch (err) {
throw new InstallError("Invalid token");
return "Controller URL must be a URL like https://example.com/";
}
if (!["http:", "https:"].includes(url.protocol)) {
return "Controller URL must start with http:// or https://";
}
return true;
}
if (parsed.aud !== "host" || !Number.isInteger(parsed.host)) {
throw new InstallError("Invalid token");
// The host requires the controller URL to end with a slash.
function normaliseControllerUrl(value) {
const url = new URL(value);
if (!url.pathname.endsWith("/")) {
url.pathname += "/";
}
return url.href;
}
// Remote updates, plugin updates and plugin installs are allowed or denied together.
async function setRemoteUpdates(exec, component, allow) {
for (let field of ["allow_remote_updates", "allow_plugin_updates", "allow_plugin_install"]) {
await exec(["config", "set", `${component}.${field}`, allow]);
}
}
@@ -228,7 +257,7 @@ async function migrateRename(args) {
async function migrateConfig(source, destination) {
if (await pathExists(source) && !await pathExists(destination)) {
await safeOutputFile(destination, JSON.stringify(
await atomicOverwriteFile(destination, JSON.stringify(
renameConfig(JSON.parse(await fs.readFile(source))), null, "\t"
), { mode: 0o600 });
logger.info(`Migrated ${source} to ${destination}`);
@@ -240,7 +269,7 @@ async function migrateRename(args) {
let instancesFile = path.join("database", "instances.json");
if (await pathExists(instancesFile)) {
await safeOutputFile(instancesFile, JSON.stringify(
await atomicOverwriteFile(instancesFile, JSON.stringify(
JSON.parse(await fs.readFile(instancesFile)).map(renameConfig), null, "\t"
));
logger.info(`Migrated ${instancesFile}`);
@@ -252,7 +281,7 @@ async function migrateRename(args) {
for (let role of users["roles"]) {
role["permissions"] = role["permissions"].map(rename);
}
await safeOutputFile(usersFile, JSON.stringify(users, null, "\t"));
await atomicOverwriteFile(usersFile, JSON.stringify(users, null, "\t"));
logger.info(`Migrated ${usersFile}`);
}
@@ -280,7 +309,7 @@ async function migrateRename(args) {
lines.push(inputLine);
}
}
safeOutputFile(outputFile, Buffer.concat(lines));
atomicOverwriteFile(outputFile, Buffer.concat(lines));
}
async function migrateLogsDir(inputLogs, outputLogs) {
@@ -405,7 +434,7 @@ async function groupIdToName(gid) {
async function writeScripts(mode, selfSignedCert) {
if (["standalone", "controller"].includes(mode)) {
if (process.platform === "win32") {
await safeOutputFile(
await atomicOverwriteFile(
"run-controller.cmd",
`\
@echo off
@@ -418,7 +447,7 @@ goto restart
`
);
} else {
await safeOutputFile(
await atomicOverwriteFile(
"run-controller.sh",
`\
#!/bin/bash
@@ -430,7 +459,7 @@ done
`,
{ mode: 0o755 },
);
await safeOutputFile(
await atomicOverwriteFile(
"systemd/clusteriocontroller.service",
`\
[Unit]
@@ -457,7 +486,7 @@ WantedBy=multi-user.target
if (["standalone", "host"].includes(mode)) {
if (process.platform === "win32") {
let extraCaCertsLine = selfSignedCert ? `set "NODE_EXTRA_CA_CERTS=${selfSignedCert}"\n` : "";
await safeOutputFile(
await atomicOverwriteFile(
"run-host.cmd",
`\
@echo off
@@ -471,7 +500,7 @@ goto restart
);
} else {
let extraCaCertsLine = selfSignedCert ? `export "NODE_EXTRA_CA_CERTS=${selfSignedCert}"\n` : "";
await safeOutputFile(
await atomicOverwriteFile(
"run-host.sh",
`\
#!/bin/bash
@@ -484,7 +513,7 @@ done
{ mode: 0o755 },
);
extraCaCertsLine = selfSignedCert ? `Environment=NODE_EXTRA_CA_CERTS=${selfSignedCert}\n` : "";
await safeOutputFile(
await atomicOverwriteFile(
"systemd/clusteriohost.service",
`\
[Unit]
@@ -510,96 +539,87 @@ WantedBy=multi-user.target
}
// eslint-disable-next-line complexity
async function inquirerMissingArgs(args) {
async function promptMissingArgs(args) {
let answers = {};
if (args.mode) { answers.mode = args.mode; }
if (args["plugin-template"]) { answers.mode = "plugin-template"; }
answers = await inquirer.prompt([
{
type: "list",
name: "mode",
message: "Operating mode to install",
default: "standalone",
choices: [
{ name: "Standalone (install both controller and host on this computer)", value: "standalone" },
{ name: "Controller only", value: "controller" },
{ name: "Host only", value: "host" },
{ name: "Ctl only", value: "ctl" },
{ name: "Plugins only", value: "plugins" },
{ name: "Plugin template", value: "plugin-template" },
],
},
], answers);
answers.mode ??= await select({
message: "Operating mode to install",
default: "standalone",
choices: [
{ name: "Standalone (install both controller and host on this computer)", value: "standalone" },
{ name: "Controller only", value: "controller" },
{ name: "Host only", value: "host" },
{ name: "Ctl only", value: "ctl" },
{ name: "Plugins only", value: "plugins" },
{ name: "Plugin template", value: "plugin-template" },
],
});
if (["standalone", "controller"].includes(answers.mode)) {
if (args.admin) { answers.admin = args.admin; }
answers = await inquirer.prompt([
{
type: "input",
name: "admin",
message: "Admin account name",
validate: input => {
if (!input) {
return "May not be empty";
}
return true;
},
answers.admin ??= await input({
message: "Admin account name",
validate: value => {
if (!value) {
return "May not be empty";
}
return true;
},
], answers);
});
if (args.httpPort) { answers.httpPort = args.httpPort; }
answers = await inquirer.prompt([
{
type: "input",
name: "httpPort",
message: "HTTP port to listen on",
default: 8080,
validate: input => {
if (!input) {
return "May not be empty";
}
if (!Number.isInteger(Number(input))) {
return "Must be a number";
}
return true;
},
answers.httpPort ??= await input({
message: "HTTP port to listen on",
default: "8080",
validate: value => {
if (!value) {
return "May not be empty";
}
if (!Number.isInteger(Number(value))) {
return "Must be a number";
}
return true;
},
], answers);
});
}
if (answers.mode === "standalone") {
answers.hostName = args.hostName ?? "local";
const result = validateHostName(answers.hostName);
if (result !== true) {
throw new InstallError(result);
}
}
if (answers.mode === "host") {
if (args.hostName) { answers.hostName = args.hostName; }
answers = await inquirer.prompt([
{
type: "input",
name: "hostName",
message: "Name of host",
},
], answers);
if (args.controllerUrl) { answers.controllerUrl = args.controllerUrl; }
answers.hostName ??= await input({
message: "Name of host",
default: os.hostname(),
validate: validateHostName,
});
answers.controllerUrl ??= await input({
message: "Controller URL",
validate: validateControllerUrl,
});
// The prompts do not validate answers given as arguments
for (let result of [validateHostName(answers.hostName), validateControllerUrl(answers.controllerUrl)]) {
if (result !== true) {
throw new InstallError(result);
}
}
answers.controllerUrl = normaliseControllerUrl(answers.controllerUrl);
}
if (["host", "ctl"].includes(answers.mode)) {
if (answers.mode === "ctl") {
if (args.controllerUrl) { answers.controllerUrl = args.controllerUrl; }
answers = await inquirer.prompt([
{
type: "input",
name: "controllerUrl",
message: "Controller URL",
},
], answers);
answers.controllerUrl ??= await input({ message: "Controller URL" });
if (args.controllerToken) { answers.controllerToken = args.controllerToken; }
answers = await inquirer.prompt([
{
type: "input",
name: "controllerToken",
message: "Controller authentication Token",
},
], answers);
}
if (answers.mode === "host") {
validateHostToken(answers.controllerToken);
answers.controllerToken ??= await input({ message: "Controller authentication Token" });
}
if (["standalone", "host"].includes(answers.mode)) {
@@ -612,14 +632,10 @@ async function inquirerMissingArgs(args) {
myIp = await result.text();
} catch (err) { /* ignore */ }
}
answers = await inquirer.prompt([
{
type: "input",
name: "publicAddress",
message: "Public DNS/IP address of this server",
default: myIp,
},
], answers);
answers.publicAddress ??= await input({
message: "Public DNS/IP address of this server",
default: myIp,
});
if (args.factorioDir) { answers.factorioDir = args.factorioDir; }
let locations = factorioLocations[process.platform] || [];
@@ -630,28 +646,17 @@ async function inquirerMissingArgs(args) {
}
}
answers = await inquirer.prompt([
{
type: "list",
name: "factorioDir",
message: "Path to Factorio installation",
choices: [
...foundLocations.map(location => ({ name: `${location} (auto detected)`, value: location })),
{ name: "Use local factorio directory, you must copy an installation to it", value: "factorio" },
{ name: "Provide path manually", value: null },
],
},
], answers);
answers.factorioDir ??= await select({
message: "Path to Factorio installation",
choices: [
...foundLocations.map(location => ({ name: `${location} (auto detected)`, value: location })),
{ name: "Use local factorio directory, you must copy an installation to it", value: "factorio" },
{ name: "Provide path manually", value: null },
],
});
if (answers.factorioDir === null) {
answers = await inquirer.prompt([
{
type: "input",
name: "factorioDir",
message: "Path to Factorio installation",
askAnswered: true,
},
], answers);
answers.factorioDir = await input({ message: "Path to Factorio installation" });
} else if (answers.factorioDir === "factorio") {
await fs.mkdir("factorio", { recursive: true });
}
@@ -659,14 +664,10 @@ async function inquirerMissingArgs(args) {
if (["standalone", "controller", "host"].includes(answers.mode)) {
if (args.hasOwnProperty("remoteNpm")) { answers.remoteNpm = args.remoteNpm; }
answers = await inquirer.prompt([
{
type: "confirm",
name: "remoteNpm",
message: "Allow remote updates via npm?",
default: true,
},
], answers);
answers.remoteNpm ??= await confirm({
message: "Allow remote updates and plugin installs from the web interface?",
default: true,
});
}
if (answers.mode === "plugin-template") {
@@ -677,29 +678,23 @@ async function inquirerMissingArgs(args) {
answers.pluginName = args["plugin-name"];
}
answers.plugins = [];
answers = await inquirer.prompt([
{
type: "checkbox",
name: "pluginTemplate",
message: "Plugin templates to use",
choices: [
{ name: "Controller", value: "controller" },
{ name: "Host", value: "host" },
{ name: "Instance", value: "instance" },
{ name: "Lua Module", value: "module" },
{ name: "Command Line", value: "ctl" },
{ name: "Web UI", value: "web" },
{ name: "No Typescript", value: "js" },
{ name: "No Config", value: "no_config" },
],
},
{
type: "input",
name: "pluginName",
message: "Name of your new plugin",
default: path.basename(path.resolve()),
},
], answers);
answers.pluginTemplate ??= await checkbox({
message: "Plugin templates to use",
choices: [
{ name: "Controller", value: "controller" },
{ name: "Host", value: "host" },
{ name: "Instance", value: "instance" },
{ name: "Lua Module", value: "module" },
{ name: "Command Line", value: "ctl" },
{ name: "Web UI", value: "web" },
{ name: "No Typescript", value: "js" },
{ name: "No Config", value: "no_config" },
],
});
answers.pluginName ??= await input({
message: "Name of your new plugin",
default: path.basename(path.resolve()),
});
}
if (dev) {
@@ -707,15 +702,11 @@ async function inquirerMissingArgs(args) {
} else if (args.plugins) {
answers.plugins = args.plugins;
}
answers = await inquirer.prompt([
{
type: "checkbox",
name: "plugins",
message: "Plugins to install",
choices: availablePlugins,
pageSize: 20,
},
], answers);
answers.plugins ??= await checkbox({
message: "Plugins to install",
choices: availablePlugins,
pageSize: 20,
});
return answers;
}
@@ -743,13 +734,13 @@ async function main() {
nargs: 1, describe: "HTTP port to listen on [standalone/controller]", type: "number",
})
.option("host-name", {
nargs: 1, describe: "Host name [host]", type: "string",
nargs: 1, describe: "Host name [standalone/host]", type: "string",
})
.option("controller-url", {
nargs: 1, describe: "Controller URL [host/ctl]", type: "string",
})
.option("controller-token", {
nargs: 1, describe: "Controller authentication token [host/ctl]", type: "string",
nargs: 1, describe: "Controller authentication token [ctl]", type: "string",
})
.option("public-address", {
nargs: 1, describe: "DNS/IP Address to connect to this server [standalone/host]", type: "string",
@@ -761,7 +752,8 @@ async function main() {
nargs: 1, describe: "Path to self-signed certificate of the controller [standalone/host]", type: "string",
})
.option("remote-npm", {
nargs: 0, description: "Allow remote updates via npm [standalone/controller/host]", type: "boolean",
nargs: 0, type: "boolean", description:
"Allow remote updates and plugin installs from the web interface [standalone/controller/host]",
})
.option("plugins", {
array: true, describe: "Plugins to install", type: "string",
@@ -798,7 +790,7 @@ async function main() {
return;
}
let answers = await inquirerMissingArgs(args);
let answers = await promptMissingArgs(args);
logger.verbose(JSON.stringify(answers));
if (answers.pluginTemplate) {
@@ -816,44 +808,28 @@ async function main() {
await execController(["config", "create"]);
await execController(["bootstrap", "create-admin", answers.admin]);
await execController(["config", "set", "controller.http_port", answers.httpPort]);
await execController(["config", "set", "controller.allow_remote_updates", answers.remoteNpm]);
await execController(["config", "set", "controller.allow_plugin_updates", answers.remoteNpm]);
await setRemoteUpdates(execController, "controller", answers.remoteNpm);
let result = await execController(["bootstrap", "generate-user-token", answers.admin]);
adminToken = result.stdout.split("\n").slice(-2)[0];
}
if (answers.mode === "standalone") {
if (["standalone", "host"].includes(answers.mode)) {
logger.info("Setting up host");
await execHost(["config", "create"]);
await execHost(["config", "set", "host.name", "local"]);
let result = await execHost(["config", "show", "host.id"]);
let hostId = Number.parseInt(result.stdout.split("\n").slice(-2)[0], 10);
result = await execController(["bootstrap", "generate-host-token", hostId]);
let hostToken = result.stdout.split("\n").slice(-2)[0];
// Default to localhost on correct port for host in standalone mode
await execHost(["config", "set", "host.controller_url", `http://localhost:${answers.httpPort}/`]);
await execHost(["config", "set", "host.controller_token", hostToken]);
await execHost(["config", "set", "host.public_address", answers.publicAddress]);
await execHost(["config", "set", "host.factorio_directory", answers.factorioDir]);
await execHost(["config", "set", "host.allow_remote_updates", answers.remoteNpm]);
await execHost(["config", "set", "host.allow_plugin_updates", answers.remoteNpm]);
}
if (answers.mode === "host") {
logger.info("Setting up host");
let hostId = JSON.parse(Buffer.from(answers.controllerToken.split(".")[1], "base64")).host;
await execHost(["config", "create"]);
await execHost(["config", "set", "host.id", hostId]);
if (answers.mode === "standalone") {
// Default to localhost on correct port for host in standalone mode
answers.controllerUrl = `http://localhost:${answers.httpPort}/`;
}
if (["standalone", "host"].includes(answers.mode)) {
// The host requests to join the cluster when it is started.
await execHost(["config", "set", "host.name", answers.hostName]);
await execHost(["config", "set", "host.controller_url", answers.controllerUrl]);
await execHost(["config", "set", "host.controller_token", answers.controllerToken]);
await execHost(["config", "set", "host.public_address", answers.publicAddress]);
await execHost(["config", "set", "host.factorio_directory", answers.factorioDir]);
await execHost(["config", "set", "host.allow_remote_updates", answers.remoteNpm]);
await execHost(["config", "set", "host.allow_plugin_updates", answers.remoteNpm]);
await setRemoteUpdates(execHost, "host", answers.remoteNpm);
}
if (!dev && ["standalone", "controller", "host"].includes(answers.mode)) {
@@ -878,7 +854,10 @@ async function main() {
if (import.meta.main) {
main().catch(err => {
process.exitCode = 1;
if (err instanceof InstallError) {
if (err.name === "ExitPromptError") {
// Ctrl+C at a prompt
process.exitCode = 130;
} else if (err instanceof InstallError) {
logger.error(err.message);
} else {
logger.fatal(`
+5 -2
View File
@@ -28,12 +28,15 @@
"clusteriocreate": "./create.js"
},
"engines": {
"node": ">=22"
"node": "^22.18.0 || >=24.2.0"
},
"dependencies": {
"@inquirer/checkbox": "^5.2.8",
"@inquirer/confirm": "^6.3.5",
"@inquirer/input": "^5.1.9",
"@inquirer/select": "^5.2.8",
"chalk": "catalog:",
"inquirer": "^8.2.7",
"yargs": "catalog:"
}
}
+7 -7
View File
@@ -88,7 +88,7 @@ export async function copyPluginTemplates(pluginName, templates) {
const config = !templates.includes("no_config") && templates.some(value => (
["controller", "host", "instance", "ctl"].includes(value)
));
const webpack = config || templates.includes("controller") || templates.includes("web");
const webBuild = config || templates.includes("controller") || templates.includes("web");
// A count of the number of isolated contexts the plugin runs under
let pluginContexts = 0;
@@ -113,15 +113,15 @@ export async function copyPluginTemplates(pluginName, templates) {
files.set("tsconfig.json", path.join(templatePath, "tsconfig.json"));
files.set("tsconfig.node.json", path.join(templatePath, "tsconfig.node.json"));
files.set("tsconfig.base.json", path.join(templatePath, "tsconfig.base.json"));
if (webpack) {
if (webBuild) {
files.set("tsconfig.browser.json", path.join(templatePath, "tsconfig.browser.json"));
}
}
// Files and dependences to support webpack
if (webpack) {
prepare.push("webpack-cli --mode production");
files.set("webpack.config.cjs", path.join(commonPath, "webpack.config.cjs"));
// Files and dependences to support the web build
if (webBuild) {
prepare.push("vite build");
files.set("vite.config.js", path.join(commonPath, "vite.config.js"));
if (templates.includes("web")) {
files.set(`web/index.${ext}x`, path.join(templatePath, `web/plugin.${ext}x`));
pluginContexts += 1;
@@ -185,7 +185,7 @@ export async function copyPluginTemplates(pluginName, templates) {
ctl: templates.includes("ctl"),
web: templates.includes("web"),
// Macro flags for context requirements
webpack: webpack,
web_build: webBuild,
config: config,
// String values for package json
clusterio_version: _package.version,
@@ -55,7 +55,7 @@
"devDependencies": {
//%// Typescript and type declarations
//%if typescript
"typescript": "^5.9.3",
"typescript": "^7.0.2",
"@types/node": "^24.13.4",
//%endif
//%if typescript & web
@@ -67,15 +67,13 @@
"react": "^18.2.0",
"react-dom": "^18.2.0",
//%endif
//%if webpack
"webpack": "^5.110.3",
"webpack-cli": "^7.2.3",
"webpack-merge": "^5.9.0",
//%if web_build
"vite": "^8.3.4",
//%endif
//%if controller
"@clusterio/controller": "^__clusterio_version__",
//%endif
//%if webpack
//%if web_build
"@clusterio/web_ui": "^__clusterio_version__",
//%endif
//%if host
@@ -87,7 +85,7 @@
"@clusterio/lib": "^__clusterio_version__"
},
"dependencies": {
"@sinclair/typebox": "^0.30.6"
"@sinclair/typebox": "^0.34.52"
},
"publishConfig": {
"access": "public"
@@ -1,3 +1,3 @@
webpack.config.js
vite.config.js
dist/browser
dist/web
@@ -0,0 +1,6 @@
import { defineConfig } from "vite";
import { clusterioPlugin } from "@clusterio/web_ui/vite.mjs";
export default defineConfig({
plugins: [clusterioPlugin()],
});
@@ -1,34 +0,0 @@
"use strict";
const path = require("path");
const webpack = require("webpack");
const { merge } = require("webpack-merge");
const common = require("@clusterio/web_ui/webpack.common");
module.exports = (env = {}, argv = {}) => merge(common(env, argv), {
context: __dirname,
entry: "./web/index.__ext__x",
output: {
path: path.resolve(__dirname, "dist", "web"),
},
plugins: [
new webpack.container.ModuleFederationPlugin({
name: "__plugin_name__",
library: { type: "window", name: "plugin___plugin_name__" },
exposes: {
"./": "./index.__ext__",
"./package.json": "./package.json",
"./web": "./web/index.__ext__x",
},
shared: {
"@clusterio/lib": { import: false },
"@clusterio/web_ui": { import: false },
//%if web
"antd": { import: false },
"react": { import: false },
"react-dom": { import: false },
//%endif
},
}),
],
});
@@ -2,6 +2,7 @@
"extends": "./tsconfig.base.json",
"compilerOptions": {
"lib": ["es2023"],
"types": ["node"],
"module": "node16",
"moduleResolution": "node16",
"target": "es2022",
+6 -6
View File
@@ -307,12 +307,12 @@ async function startCtl() {
logger.error(`Error running command: ${err.message}`);
process.exitCode = 1;
} else if (err instanceof lib.RequestError) {
if (err.stack) {
logger.error(`Error sending request:\n${err.stack}`);
} else {
logger.error(`Error sending request: ${err.message}`);
}
} else if (err instanceof lib.ClientError || err instanceof lib.UnavailableError) {
logger.error(`Error sending request: ${err.message}`);
process.exitCode = 1;
} else if (err instanceof lib.ServerError) {
logger.error(`Error sending request:\n${err.stack ?? err.message}`);
process.exitCode = 1;
} else {
+1 -1
View File
@@ -32,7 +32,7 @@
"clusterioctl": "./dist/node/ctl.js"
},
"engines": {
"node": ">=22"
"node": "^22.18.0 || >=24.2.0"
},
"dependencies": {
+21 -2
View File
@@ -180,7 +180,7 @@ controllerPluginCommands.add(new lib.Command({
}));
controllerPluginCommands.add(new lib.Command({
definition: ["update <plugin>", "Update a plugin on the controller", (yargs) => {
yargs.positional("plugin", { describe: "Plugin to update", type: "string" });
yargs.positional("plugin", { describe: "Plugin to update, optionally as name@version", type: "string" });
yargs.option("restart", { alias: "r", type: "boolean", description: "Restart after update" });
}],
handler: async function(args: { plugin: string, restart: boolean }, ctl: Ctl) {
@@ -192,9 +192,28 @@ controllerPluginCommands.add(new lib.Command({
}
},
}));
controllerPluginCommands.add(new lib.Command({
definition: ["search [query]", "Search npm for plugins", (yargs) => {
yargs.positional("query", { describe: "Text to search for", type: "string", default: "" });
yargs.option("page", { type: "number", description: "Page of results to show", default: 1 });
yargs.option("page-size", { type: "number", description: "Results per page", default: 20 });
}],
handler: async function(args: { query: string, page: number, pageSize: number }, ctl: Ctl) {
const response = await ctl.sendTo(
"controller", new lib.PluginSearchRequest(args.query, args.page, args.pageSize)
);
print(asTable(response.results.map(plugin => ({
name: plugin.name,
version: plugin.version,
publisher: plugin.publisher ?? "",
description: plugin.description ?? "",
}))));
print(`Showing ${response.results.length} of ${response.total} results`);
},
}));
controllerPluginCommands.add(new lib.Command({
definition: ["install <plugin>", "Install a plugin on the controller", (yargs) => {
yargs.positional("plugin", { describe: "Plugin to install", type: "string" });
yargs.positional("plugin", { describe: "Plugin to install, optionally as name@version", type: "string" });
yargs.option("restart", { alias: "r", type: "boolean", description: "Restart after update" });
}],
handler: async function(args: { plugin: string, restart: boolean }, ctl: Ctl) {
+64 -49
View File
@@ -5,7 +5,6 @@ import os from "os";
import child_process from "child_process";
import * as lib from "@clusterio/lib";
import { logger } from "@clusterio/lib";
import type { Ctl } from "../ctl.js";
import { print } from "./command_ops.js";
import { serializedConfigToString, getEditor, configToKeyVal } from "./config_ops.js";
@@ -51,62 +50,78 @@ hostCommands.add(new lib.Command({
},
}));
/** Format a duration in milliseconds as a short age like "1m 5s" */
function formatAge(ms: number) {
const seconds = Math.max(0, Math.floor(ms / 1000));
if (seconds < 60) {
return `${seconds}s`;
}
if (seconds < 3600) {
return `${Math.floor(seconds / 60)}m ${seconds % 60}s`;
}
return `${Math.floor(seconds / 3600)}h ${Math.floor(seconds / 60) % 60}m`;
}
/** Strip case and separators the same way the controller does when matching pairing codes */
function plainCode(code: string) {
return code.toUpperCase().replace(/[^0-9A-Z]/g, "");
}
hostCommands.add(new lib.Command({
definition: ["generate-token", "Generate token for a host", (yargs) => {
yargs.option("id", { type: "number", nargs: 1, describe: "Host id" });
}],
handler: async function(args: { id?: number }, ctl: Ctl) {
let hostId = typeof args.id === "number" ? args.id : undefined;
let response = await ctl.send(new lib.HostGenerateTokenRequest(hostId));
print(response);
definition: ["pending", "List host installers waiting for approval"],
handler: async function(args: object, ctl: Ctl) {
let requests = await ctl.send(new lib.HostPairingListRequest());
if (!requests.length) {
print("No pending host requests");
return;
}
let now = Date.now();
print(asTable(requests.map(request => ({
code: request.code,
name: request.name,
address: request.remoteAddress,
age: formatAge(now - request.createdAtMs),
}))));
},
}));
hostCommands.add(new lib.Command({
definition: ["revoke-token <host>", "Revoke all tokens for a host", (yargs) => {
yargs.positional("host", { describe: "Host to revoke tokens for", type: "string" });
definition: ["approve <code>", "Approve a host installer waiting for approval", (yargs) => {
yargs.positional("code", { describe: "Code shown by the installer", type: "string" });
yargs.option("name", {
type: "string", nargs: 1, describe: "Name to give the host instead of the one it asked for",
});
}],
handler: async function(args: { code: string, name?: string }, ctl: Ctl) {
let name = args.name?.trim();
if (name === undefined) {
let requests = await ctl.send(new lib.HostPairingListRequest());
name = requests.find(request => plainCode(request.code) === plainCode(args.code))?.name;
}
let hostId = await ctl.send(new lib.HostPairingApproveRequest(args.code, args.name));
print(name ? `Approved host ${name} with id ${hostId}` : `Approved host with id ${hostId}`);
},
}));
hostCommands.add(new lib.Command({
definition: ["reject <code>", "Reject a host installer waiting for approval", (yargs) => {
yargs.positional("code", { describe: "Code shown by the installer", type: "string" });
}],
handler: async function(args: { code: string }, ctl: Ctl) {
await ctl.send(new lib.HostPairingRejectRequest(args.code));
},
}));
hostCommands.add(new lib.Command({
definition: ["revoke <host>", "Remove a host from the cluster and disconnect it", (yargs) => {
yargs.positional("host", { describe: "Host to revoke", type: "string" });
}],
handler: async function(args: { host: string }, ctl: Ctl) {
let hostId = await lib.resolveHost(ctl, args.host);
await ctl.send(new lib.HostRevokeTokensRequest(hostId));
await ctl.send(new lib.HostRevokeRequest(hostId));
},
}));
hostCommands.add(new lib.Command({
definition: ["create-config", "Create host config", (yargs) => {
yargs.option("id", { type: "number", nargs: 1, describe: "Host id" });
yargs.option("name", { type: "string", nargs: 1, describe: "Host name" });
yargs.option("generate-token", {
type: "boolean", nargs: 0, describe: "Generate authentication token", default: false,
});
yargs.option("output", {
type: "string", nargs: 1, describe: "Path to output config (- for stdout)", default: "config-host.json",
});
}],
handler: async function(
args: { id?: number, name?: string, generateToken: boolean, output: string },
ctl: Ctl
) {
let config = await ctl.send(
new lib.HostConfigCreateRequest(args.id, args.name, args.generateToken)
);
let content = JSON.stringify(config, null, "\t");
if (args.output === "-") {
print(content);
} else {
logger.info(`Writing ${args.output}`);
try {
await fs.writeFile(args.output, content, { flag: "wx", mode: 0o600 });
} catch (err: any) {
if (err.code === "EEXIST") {
throw new lib.CommandError(`File ${args.output} already exists`);
}
throw err;
}
}
},
}));
hostCommands.add(new lib.Command({
definition: ["update <host>", "Update the host", (yargs) => {
yargs.positional("host", { describe: "Host to update", type: "string" });
@@ -271,14 +286,14 @@ hostPluginCommands.add(new lib.Command({
let hostId = await lib.resolveHost(ctl, args.host);
const plugins = await ctl.sendTo({ hostId }, new lib.PluginListRequest());
print(asTable(plugins.map(p => ({
title: p.title, version: p.version, loaded: p.loaded, enabled: p.enabled, npmPackage: p.npmPackage,
title: p.title, version: p.version, state: p.state, npmPackage: p.npmPackage,
}))));
},
}));
hostPluginCommands.add(new lib.Command({
definition: ["update <host> <plugin>", "Update a plugin on a host", (yargs) => {
yargs.positional("host", { describe: "Host to update on", type: "string" });
yargs.positional("plugin", { describe: "Plugin to update", type: "string" });
yargs.positional("plugin", { describe: "Plugin to update, optionally as name@version", type: "string" });
yargs.option("restart", { alias: "r", type: "boolean", description: "Restart after update" });
}],
handler: async function(args: { host: string, plugin: string, restart: boolean }, ctl: Ctl) {
@@ -294,7 +309,7 @@ hostPluginCommands.add(new lib.Command({
hostPluginCommands.add(new lib.Command({
definition: ["install <host> <plugin>", "Install a plugin on a host", (yargs) => {
yargs.positional("host", { describe: "Host to install on", type: "string" });
yargs.positional("plugin", { describe: "Plugin to install", type: "string" });
yargs.positional("plugin", { describe: "Plugin to install, optionally as name@version", type: "string" });
yargs.option("restart", { alias: "r", type: "boolean", description: "Restart after update" });
}],
handler: async function(args: { host: string, plugin: string, restart: boolean }, ctl: Ctl) {
+56 -9
View File
@@ -20,10 +20,14 @@ export const instanceCommands = new lib.CommandTree({
name: "instance", alias: ["i"], description: "Instance management",
});
instanceCommands.add(new lib.Command({
definition: [["list", "l"], "List instances known to the controller"],
handler: async function(args: object, ctl: Ctl) {
definition: [["list", "l"], "List instances known to the controller", (yargs) => {
yargs.options({
"templates": { describe: "Include templates", nargs: 0, type: "boolean", default: false },
});
}],
handler: async function(args: { templates: boolean }, ctl: Ctl) {
let list = await ctl.send(new lib.InstanceDetailsListRequest());
print(asTable(list));
print(asTable(list.filter(instance => args.templates || !instance.isTemplate)));
},
}));
@@ -34,18 +38,30 @@ instanceCommands.add(new lib.Command({
yargs.options({
"id": { type: "number", nargs: 1, describe: "Instance id" },
"from": { type: "number", nargs: 1, describe: "Clone config from instance id" },
"template": { type: "string", nargs: 1, describe: "Template to inherit config from" },
"as-template": { type: "boolean", nargs: 0, describe: "Create the instance as a template" },
});
}],
handler: async function(args: { name: string, id?: number, from?: number, }, ctl: Ctl) {
handler: async function(
args: { name: string, id?: number, from?: number, template?: string, asTemplate?: boolean },
ctl: Ctl,
) {
let instanceConfig = new lib.InstanceConfig("control");
let fields: (keyof lib.InstanceConfigFields)[] = ["instance.id", "instance.name"];
if (args.id !== undefined) {
instanceConfig.set("instance.id", args.id);
}
instanceConfig.set("instance.name", args.name);
if (args.template !== undefined) {
instanceConfig.set("instance.template_id", await lib.resolveInstance(ctl, args.template));
fields.push("instance.template_id");
}
if (args.asTemplate) {
instanceConfig.set("instance.is_template", true);
fields.push("instance.is_template");
}
await ctl.send(new lib.InstanceCreateRequest(
instanceConfig.toRemote("controller", [
"instance.id", "instance.name",
]),
instanceConfig.toRemote("controller", fields),
args.from
));
},
@@ -61,13 +77,42 @@ instanceConfigCommands.add(new lib.Command({
handler: async function(args: { instance: string }, ctl: Ctl) {
let instanceId = await lib.resolveInstance(ctl, args.instance);
let config = await ctl.send(new lib.InstanceConfigGetRequest(instanceId));
const inherited = config["instance.template_id"] !== null
? config["instance.inherited_fields"] as Record<string, true | string[]> ?? {}
: {};
for (let [name, value] of Object.entries(config)) {
print(`${name} ${JSON.stringify(value)}`);
if (name === "instance.inherited_fields") {
continue;
}
const entry = inherited[name];
let note = "";
if (entry instanceof Array && entry.length) {
note = ` (inherited except ${entry.join(", ")})`;
} else if (entry) {
note = " (inherited)";
}
print(`${name} ${JSON.stringify(value)}${note}`);
}
},
}));
instanceConfigCommands.add(new lib.Command({
definition: [
"inherit <instance> <field> [prop]",
"Make field or property in instance config use the value of its template",
(yargs) => {
yargs.positional("instance", { describe: "Instance to update config on", type: "string" });
yargs.positional("field", { describe: "Field to inherit", type: "string" });
yargs.positional("prop", { describe: "Property of object field to inherit", type: "string" });
},
],
handler: async function(args: { instance: string, field: string, prop?: string }, ctl: Ctl) {
let instanceId = await lib.resolveInstance(ctl, args.instance);
await ctl.send(new lib.InstanceConfigInheritFieldRequest(instanceId, args.field, args.prop));
},
}));
instanceConfigCommands.add(new lib.Command({
definition: ["set <instance> <field> [value]", "Set field in instance config", (yargs) => {
yargs.positional("instance", { describe: "Instance to set config on", type: "string" });
@@ -170,8 +215,10 @@ instanceConfigCommands.add(new lib.Command({
await events.once(editorSpawn, "exit");
const data = await fs.readFile(tmpFile, "utf8");
const final = await configToKeyVal(data);
const original = await configToKeyVal(allConfigElements);
for (let index in final) {
if (index in final) {
// Unchanged fields are skipped so they keep inheriting from the template
if (index in final && final[index] !== original[index]) {
try {
await ctl.send(new lib.InstanceConfigSetFieldRequest(
instanceId,
+1 -1
View File
@@ -2,7 +2,7 @@ import * as lib from "@clusterio/lib";
export async function getEditor(argsEditor: string) {
// eslint-disable-next-line
return argsEditor || process.env.EDITOR || process.env.VISUAL || undefined
return argsEditor || process.env.EDITOR || process.env.VISUAL || undefined;
// needed for the process.env statements to not be flagged by eslint
// priority for editors is CLI argument > env.EDITOR > env.VISUAL
}
-4
View File
@@ -1,4 +0,0 @@
{
"entryPoints": ["index.ts"],
"includeVersion": true
}
+4
View File
@@ -132,6 +132,10 @@ Lists up all configuration entries with their currently configured values.
### `run`
Runs the host.
A host that has not been approved prints a code and a link and waits for approval before it connects, see [Setting up a host](https://github.com/clusterio/clusterio/blob/master/packages/create/README.md#setting-up-a-host).
If the controller no longer accepts the token of the host when it starts, for example because the host was revoked, it asks to be approved again.
If its request is rejected it exits with code 8 and the run scripts created by the installer do not restart it.
A host that is revoked while it runs shuts down.
## See Also
+46 -19
View File
@@ -20,11 +20,14 @@ import winston from "winston";
import "winston-daily-rotate-file";
// internal libraries
// Reduce startup time by lazy compiling schemas.
import "./src/set_lazy_schema_compilation.js";
import * as lib from "@clusterio/lib";
import { ConsoleTransport, levels, logger } from "@clusterio/lib";
import Host from "./src/Host.js";
import type Host from "./src/Host.js";
import { createScenarioCommand, handleCreateScenarioCommand } from "./src/create_scenario.js";
import { PairingError, ensurePaired } from "./src/pairing.js";
let host: Host | undefined;
@@ -134,6 +137,10 @@ async function startHost() {
type: "boolean", nargs: 0, default: false,
describe: "Start the host in recovery mode with all plugins disabled and controller disconnected",
});
yargs.option("user-token", {
type: "string", nargs: 1,
describe: "Token of a user allowed to approve hosts, approves this host without waiting. Not saved",
});
})
.demandCommand(1, "You need to specify a command to run")
.strict()
@@ -164,7 +171,8 @@ async function startHost() {
}
logger.info(`Loading available plugins from ${args.pluginList}`);
let pluginList = await lib.loadPluginList(args.pluginList);
const skippedNpmPlugins = new Map<string, string>();
let pluginList = await lib.loadPluginList(args.pluginList, true, skippedNpmPlugins);
// If the command is plugin management we don't try to load plugins
if (command === "plugin") {
@@ -173,7 +181,7 @@ async function startHost() {
}
logger.info("Loading Plugin info");
let pluginInfos = await lib.loadPluginInfos(pluginList);
let pluginInfos = await lib.loadPluginInfos(pluginList, skippedNpmPlugins);
lib.registerPluginMessages(pluginInfos);
lib.registerPluginPermissions(pluginInfos);
lib.addPluginConfigFields(pluginInfos);
@@ -211,19 +219,6 @@ async function startHost() {
// and as the process name in ps/top on linux.
process.title = "clusteriohost";
// make sure we have the controller access token
if (hostConfig.get("host.controller_token") === "enter token here") {
logger.fatal("ERROR invalid config!");
logger.fatal(
"Controller requires an access token for socket operations. As clusterio\n"+
"hosts depends upon this, please set your token using the command npx\n"+
"clusteriohost config set host.controller_token <token>. You can generate an\n"+
"auth token using npx clusterioctl generate-host-token."
);
process.exitCode = 1;
return;
}
// make sure url ends with /
if (!hostConfig.get("host.controller_url").endsWith("/")) {
logger.fatal("ERROR invalid config!");
@@ -232,14 +227,46 @@ async function startHost() {
return;
}
if (!args.recovery) {
const abort = new AbortController();
const onSignal = () => { abort.abort(); };
process.on("SIGINT", onSignal);
process.on("SIGTERM", onSignal);
process.on("SIGHUP", onSignal);
try {
await ensurePaired(hostConfig, {
version: packageConfig.version,
userToken: args.userToken as string | undefined,
signal: abort.signal,
});
} catch (err: any) {
if (abort.signal.aborted) {
logger.info("Stopped waiting for approval");
return;
}
if (err instanceof PairingError) {
logger.fatal(`Unable to join the cluster: ${err.message}`);
// Exit code 8 stops the run scripts from restarting the host.
process.exitCode = err.status === 403 ? 8 : 1;
return;
}
throw err;
} finally {
process.off("SIGINT", onSignal);
process.off("SIGTERM", onSignal);
process.off("SIGHUP", onSignal);
}
}
// Loaded here as the server is not needed by the other commands
const { default: Host } = await import("./src/Host.js");
let hostConnector = new HostConnector(hostConfig, pluginInfos);
host = new Host(
hostConnector,
host = await Host.fromConfig(
hostConfig,
pluginInfos,
hostConnector,
Boolean(args.canRestart),
Boolean(args.recovery),
...await Host.bootstrap(hostConfig)
);
// Handle interrupts
+1
View File
@@ -4,6 +4,7 @@ export { BaseInstancePlugin, InstanceHooks, type InstancePluginContext } from ".
export { default as Host, HostRouter } from "./src/Host.js";
export { default as Instance } from "./src/Instance.js";
export { default as InstanceConnection } from "./src/InstanceConnection.js";
export { PairingError, checkHostToken, pairHost } from "./src/pairing.js";
if (import.meta.main) {
bootstrap();
+6 -2
View File
@@ -33,13 +33,17 @@
"clusteriohost": "./dist/node/host.js"
},
"engines": {
"node": ">=22"
"node": "^22.18.0 || >=24.2.0"
},
"dependencies": {
"@clusterio/lib": "workspace:^",
"@jimp/core": "^1.6.1",
"@jimp/js-png": "^1.6.1",
"@jimp/plugin-crop": "^1.6.1",
"@jimp/plugin-resize": "^1.6.1",
"@jimp/types": "^1.6.1",
"@sinclair/typebox": "catalog:",
"jimp": "^1.6.1",
"pidusage": "^4.1.0",
"rcon-client": "^4.2.5",
"semver": "catalog:",
+16
View File
@@ -23,6 +23,7 @@ export class InstanceHooks extends lib.AsyncHookCollection {
this.controllerConnectionEvent = this.newHook();
this.prepareControllerDisconnect = this.newHook();
this.playerEvent = this.newHook();
this.autoPauseChanged = this.newHook();
}
/**
@@ -147,6 +148,19 @@ export class InstanceHooks extends lib.AsyncHookCollection {
* @param event - Information about the event.
*/
readonly playerEvent: lib.AsyncHook<[event: lib.PlayerEvent]>;
/**
* Called when Factorio auto pauses or unpauses the game
*
* Invoked when the last player leaves or the first player joins an
* instance started with the auto_pause server setting on. Every RCON
* command makes a paused game run one tick, so periodic commands
* should be skipped while paused. See `Instance.autoPaused`. It is
* not invoked on start, read `autoPaused` in the start hook instead.
*
* @param paused - true if the game is now paused.
*/
readonly autoPauseChanged: lib.AsyncHook<[paused: boolean]>;
}
// eslint-disable-next-line @typescript-eslint/no-unsafe-declaration-merging
@@ -160,6 +174,7 @@ export interface BaseInstancePlugin {
onControllerConnectionEvent?(event: "connect" | "drop" | "resume" | "close"): void;
onPrepareControllerDisconnect?(connection: Instance): Promise<void>;
onPlayerEvent?(event: lib.PlayerEvent): Promise<void>;
onAutoPauseChanged?(paused: boolean): Promise<void>;
}
/**
@@ -222,6 +237,7 @@ export class BaseInstancePlugin {
attach(instance.hooks.controllerConnectionEvent, this.onControllerConnectionEvent);
attach(instance.hooks.prepareControllerDisconnect, this.onPrepareControllerDisconnect);
attach(instance.hooks.playerEvent, this.onPlayerEvent);
attach(instance.hooks.autoPauseChanged, this.onAutoPauseChanged);
}
static fromContext(context: InstancePluginContext) {
+143 -75
View File
@@ -22,7 +22,38 @@ function checkRequestSaveName(name: string) {
try {
lib.checkFilename(name);
} catch (err: any) {
throw new lib.RequestError(`Save name ${err.message}`);
throw new lib.ValidationError(`Save name ${err.message}`);
}
}
async function copySaveExclusive(sourcePath: string, targetPath: string) {
await fs.copyFile(
sourcePath,
targetPath,
fs.constants.COPYFILE_EXCL | fs.constants.COPYFILE_FICLONE
);
}
/**
* Move a file without overwriting an existing target, fails with EEXIST
* if the target exists.
*/
async function moveSaveExclusive(sourcePath: string, targetPath: string) {
try {
await fs.link(sourcePath, targetPath);
} catch (err: any) {
// Hard links fail across filesystems and on some filesystems with
// varying codes, the copy is still exclusive.
if (err.code === "EEXIST") {
throw err;
}
await copySaveExclusive(sourcePath, targetPath);
}
try {
await fs.unlink(sourcePath);
} catch (err: any) {
await fs.unlink(targetPath).catch(() => {});
throw err;
}
}
@@ -172,7 +203,7 @@ export class HostRouter {
return false;
}
origin.connector.sendResponseError(
new lib.ResponseError(msg || "Instance is not running."), message.src
new lib.ResponseError(msg || "Instance is not running.", "UnavailableError"), message.src
);
}
return true;
@@ -234,13 +265,13 @@ export class HostRouter {
let dst = message.dst;
if (nextHop) {
origin.connector.sendResponseError(
new lib.ResponseError("Instance is already running."), message.src
new lib.ResponseError("Instance is already running.", "ConflictError"), message.src
);
return;
}
if (this.host._shuttingDown) {
origin.connector.sendResponseError(
new lib.ResponseError("Host is shutting down."), message.src
new lib.ResponseError("Host is shutting down.", "UnavailableError"), message.src
);
return;
}
@@ -249,9 +280,7 @@ export class HostRouter {
instanceConnection.connector.forward(message);
}).catch(err => {
logger.error(`Error starting instance:\n${err.stack ?? err.message}`, this.host.instanceLogMeta(dst.id));
origin.connector.sendResponseError(
new lib.ResponseError(err.message, err.code, err.stack), message.src
);
origin.connector.sendResponseError(lib.Link.errorToResponse(err), message.src);
});
}
@@ -264,9 +293,7 @@ export class HostRouter {
}
} catch (err: any) {
if (message.type === "request") {
origin.connector.sendResponseError(
new lib.ResponseError(err.message, err.code, err.stack), message.src
);
origin.connector.sendResponseError(lib.Link.errorToResponse(err), message.src);
}
logger.warn(`Failed to deliver ${(message as any).name || "message"} (${message.type}): ${err.message}`);
if (err instanceof lib.InvalidMessage && err.errors) {
@@ -322,6 +349,8 @@ export default class Host extends lib.Link {
_startup = true;
_disconnecting = false;
_shuttingDown = false;
/** Instances waiting on {@link Instance.fromConfig} */
_startingInstances = new Set<number>();
startedAtMs = Date.now();
@@ -329,12 +358,18 @@ export default class Host extends lib.Link {
"_warning": "Changes to this file will be overwritten by the controller's copy.",
};
static async bootstrap(hostConfig: lib.HostConfig) {
/** Create a host with its mod store loaded from disk */
static async fromConfig(
hostConfig: lib.HostConfig,
pluginInfos: lib.PluginNodeEnvInfo[],
connector: HostConnector,
canRestart = false,
recoveryMode = false,
) {
const modsDirectory = hostConfig.get("host.mods_directory");
await fs.mkdir(modsDirectory, { recursive: true });
return [
await lib.ModStore.fromDirectory(modsDirectory),
] as const;
const modStore = await lib.ModStore.fromDirectory(modsDirectory);
return new this(connector, hostConfig, pluginInfos, canRestart, recoveryMode, modStore);
}
constructor(
@@ -498,8 +533,8 @@ export default class Host extends lib.Link {
}
let instancesDir = this.config.get("host.instances_directory");
for (let i = 0; i < 10; i++) { // Limit attempts in case this is somehow an infinite loop
let candidateDir = path.join(instancesDir, await lib.findUnusedName(instancesDir, name));
for (let i = 1; ; i++) {
let candidateDir = path.join(instancesDir, i === 1 ? name : `${name}-${i}`);
try {
await fs.mkdir(candidateDir);
} catch (err: any) {
@@ -510,7 +545,6 @@ export default class Host extends lib.Link {
}
return candidateDir;
}
throw Error("Unable to create instance dir, retry threshold reached");
}
async broadcastEventToInstance<T>(event: lib.Event<T>) {
@@ -528,12 +562,15 @@ export default class Host extends lib.Link {
async handleHostRestartRequest() {
if (!this.canRestart) {
throw new lib.RequestError("Cannot restart, host does not have a process monitor to restart it.");
throw new lib.ConflictError("Cannot restart, host does not have a process monitor to restart it.");
}
if (await lib.packageOperationRunning()) {
throw new lib.ConflictError("Cannot restart while a plugin install or update is running on this machine.");
}
const downgrade = await this.checkRestartDowngrade();
if (downgrade) {
const { installedVersion, runningVersion } = downgrade;
throw new lib.RequestError(
throw new lib.ConflictError(
`Cannot restart host with installed Clusterio version ${installedVersion} because it is older than ` +
`running version ${runningVersion}. Stop the host before starting the older version manually.`
);
@@ -554,7 +591,7 @@ export default class Host extends lib.Link {
if (fieldName === "host.id") {
// Changing host.id at runtime is not worth the effort to
// support and will break a lot of things if it is allowed.
throw new lib.RequestError("Setting 'host.id' while host is running is not supported");
throw new lib.ConflictError("Setting 'host.id' while host is running is not supported");
}
}
@@ -758,7 +795,7 @@ export default class Host extends lib.Link {
this.send(
new lib.InstanceStatusChangedEvent(
instanceId,
instanceConnection ? instanceConnection.instance.status : "stopped",
instanceConnection ? instanceConnection.instance.status : this._unconnectedStatus(instanceId),
instanceConnection ? instanceConnection.instance.server.gamePort : this.gamePort(instanceId),
instanceConnection
? instanceConnection.instance.server.version
@@ -799,7 +836,7 @@ export default class Host extends lib.Link {
getRequestInstance(instanceId: number) {
let instance = this.assignedInstances.get(instanceId);
if (!instance) {
throw new lib.RequestError(`Instance with ID ${instanceId} does not exist`);
throw new lib.NotFoundError(`Instance with ID ${instanceId} does not exist`);
}
return instance;
}
@@ -845,7 +882,7 @@ export default class Host extends lib.Link {
}
if (!availablePorts.size) {
const others = inUse.size ? ` (${inUse.size} in use by other processes)` : "";
throw new lib.RequestError(`No available port left to assign from host.factorio_port_range${others}`);
throw new lib.ConflictError(`No available port left to assign from host.factorio_port_range${others}`);
}
const [newPort] = availablePorts;
instanceToAssign.config.set("factorio.host_assigned_game_port", newPort);
@@ -874,6 +911,11 @@ export default class Host extends lib.Link {
}
}
/** Status of an instance without a connection */
_unconnectedStatus(instanceId: number) {
return this._startingInstances.has(instanceId) ? "starting" as const : "stopped" as const;
}
/**
* Initialize and connect an unloaded instance
*
@@ -882,29 +924,36 @@ export default class Host extends lib.Link {
*/
async _connectInstance(instanceId: number) {
let unloadedInstance = this.getRequestInstance(instanceId);
if (this.instanceConnections.has(instanceId)) {
throw new lib.RequestError(`Instance with ID ${instanceId} is running`);
if (this.instanceConnections.has(instanceId) || this._startingInstances.has(instanceId)) {
throw new lib.ConflictError(`Instance with ID ${instanceId} is running`);
}
let hostAddress = new lib.Address(lib.Address.host, this.config.get("host.id"));
let instanceAddress = new lib.Address(lib.Address.instance, instanceId);
let [connectionClient, connectionServer] = lib.VirtualConnector.makePair(instanceAddress, hostAddress);
let instance = new Instance(
this, connectionClient,
unloadedInstance.path, this.config.get("host.factorio_directory"),
unloadedInstance.config
);
let instance: Instance;
this._startingInstances.add(instanceId);
try {
instance = await Instance.fromConfig(
unloadedInstance.config, this, connectionClient,
unloadedInstance.path, this.config.get("host.factorio_directory"),
);
} finally {
this._startingInstances.delete(instanceId);
}
let instanceConnection = new InstanceConnection(connectionServer, this, instance);
this.instanceConnections.set(instanceId, instanceConnection);
await instance.init(this.pluginInfos);
await instance.loadPlugins(this.pluginInfos);
return instanceConnection;
}
async handleSystemInfoRequest() {
const restartRequired = await this.checkRestartRequired();
return lib.gatherSystemInfo(this.config.get("host.id"), this.canRestart, restartRequired, this.startedAtMs);
return lib.gatherSystemInfo(
this.config.get("host.id"), this.canRestart, restartRequired, this.startedAtMs, await this.listPlugins(),
);
}
async checkRestartRequired() {
@@ -1025,19 +1074,16 @@ export default class Host extends lib.Link {
checkRequestSaveName(newName);
let instance = this.getRequestInstance(instanceId);
try {
// Link save in the new location.
await fs.link(
await moveSaveExclusive(
path.join(instance.path, "saves", oldName),
path.join(instance.path, "saves", newName),
);
// Unlink save in old location.
await fs.unlink(path.join(instance.path, "saves", oldName));
} catch (err: any) {
if (err.code === "EEXIST") {
throw new lib.RequestError(`${newName} already exists`);
throw new lib.ConflictError(`${newName} already exists`);
}
if (err.code === "ENOENT") {
throw new lib.RequestError(`${oldName} does not exist`);
throw new lib.NotFoundError(`${oldName} does not exist`);
}
throw err;
}
@@ -1057,13 +1103,32 @@ export default class Host extends lib.Link {
);
} catch (err: any) {
if (err.code === "ENOENT") {
throw new lib.RequestError(`${source} does not exist`);
throw new lib.NotFoundError(`${source} does not exist`);
}
throw err;
}
await this.sendSaveListUpdate(instanceId, path.join(instance.path, "saves"));
}
async _moveOrCopySave(sourcePath: string, targetPath: string, copy: boolean) {
const { dir, name, ext } = path.parse(targetPath);
for (let i = 1; ; i++) {
targetPath = path.format({ dir, name: i === 1 ? name : `${name}-${i}`, ext });
try {
if (copy) {
await copySaveExclusive(sourcePath, targetPath);
} else {
await moveSaveExclusive(sourcePath, targetPath);
}
return path.basename(targetPath);
} catch (err: any) {
if (err.code !== "EEXIST") {
throw err;
}
}
}
}
async handleInstanceTransferSaveRequest(request: lib.InstanceTransferSaveRequest) {
let { sourceName, targetName, copy, sourceInstanceId, targetInstanceId } = request;
checkRequestSaveName(sourceName);
@@ -1071,31 +1136,27 @@ export default class Host extends lib.Link {
let sourceInstance = this.getRequestInstance(sourceInstanceId);
let targetInstance = this.getRequestInstance(targetInstanceId);
if (!targetName.endsWith(".zip")) {
targetName += ".zip";
}
// For consistency with remote transfer initiated through pullSave the
// target is renamed if it already exists.
targetName = await lib.findUnusedName(
path.join(targetInstance.path, "saves"), targetName, ".zip"
);
const sourcePath = path.join(sourceInstance.path, "saves", sourceName);
try {
if (copy) {
await fs.copyFile(
path.join(sourceInstance.path, "saves", sourceName),
path.join(targetInstance.path, "saves", targetName),
fs.constants.COPYFILE_FICLONE
);
} else {
await fs.rename(
path.join(sourceInstance.path, "saves", sourceName),
path.join(targetInstance.path, "saves", targetName),
);
}
targetName = await this._moveOrCopySave(
sourcePath,
path.join(targetInstance.path, "saves", targetName),
copy,
);
} catch (err: any) {
if (err.code === "ENOENT") {
throw new lib.RequestError(`${sourceName} does not exist`);
// ENOENT may also come from a missing target saves folder.
if (err.code === "ENOENT" && !await fs.access(sourcePath).then(() => true, () => false)) {
throw new lib.NotFoundError(`${sourceName} does not exist`);
}
throw err;
}
await this.sendSaveListUpdate(sourceInstanceId, path.join(sourceInstance.path, "saves"));
await this.sendSaveListUpdate(targetInstanceId, path.join(targetInstance.path, "saves"));
@@ -1123,7 +1184,7 @@ export default class Host extends lib.Link {
await fs.unlink(path.join(instance.path, "saves", name));
} catch (err: any) {
if (err.code === "ENOENT") {
throw new lib.RequestError(`${name} does not exist`);
throw new lib.NotFoundError(`${name} does not exist`);
}
throw err;
}
@@ -1154,7 +1215,7 @@ export default class Host extends lib.Link {
content = (await fs.open(path.join(instance.path, "saves", name))).createReadStream();
} catch (err: any) {
if (err.code === "ENOENT") {
throw new lib.RequestError(`${name} does not exist`);
throw new lib.NotFoundError(`${name} does not exist`);
}
throw err;
}
@@ -1173,12 +1234,12 @@ export default class Host extends lib.Link {
async handleInstanceDeleteInternalRequest(request: lib.InstanceDeleteInternalRequest) {
let instanceId = request.instanceId;
if (this.instanceConnections.has(instanceId)) {
throw new lib.RequestError(`Instance with ID ${instanceId} is running`);
throw new lib.ConflictError(`Instance with ID ${instanceId} is running`);
}
let instance = this.discoveredInstances.get(instanceId);
if (!instance) {
throw new lib.RequestError(`Instance with ID ${instanceId} does not exist`);
throw new lib.NotFoundError(`Instance with ID ${instanceId} does not exist`);
}
this.discoveredInstances.delete(instanceId);
@@ -1188,45 +1249,52 @@ export default class Host extends lib.Link {
async handleHostUpdateRequest(request: lib.HostUpdateRequest) {
if (!this.config.get("host.allow_remote_updates")) {
throw new lib.RequestError("Remote updates are disabled on this machine");
throw new lib.ForbiddenError("Remote updates are disabled on this machine");
}
return await lib.updatePackage("@clusterio/host");
}
async handlePluginUpdateRequest(request: lib.PluginUpdateRequest) {
if (!this.config.get("host.allow_plugin_updates")) {
throw new lib.RequestError("Plugin updates are disabled on this machine");
throw new lib.ForbiddenError("Plugin updates are disabled on this machine");
}
return await lib.handlePluginUpdate(request.pluginPackage, this.pluginInfos);
}
async handlePluginInstallRequest(request: lib.PluginInstallRequest) {
if (!this.config.get("host.allow_plugin_install")) {
throw new lib.RequestError("Plugin installs are disabled on this machine");
throw new lib.ForbiddenError("Plugin installs are disabled on this machine");
}
await lib.handlePluginInstall(request.pluginPackage);
// A new plugin is not in pluginInfos, so checkRestartRequired cannot notice it
if (!this.pluginInfos.some(pluginInfo => pluginInfo.npmPackage === request.pluginPackage)) {
const npmPackage = await lib.handlePluginInstall(request.pluginPackage);
// A new plugin is not in pluginInfos, so checkRestartRequired cannot notice it. Version
// changes to loaded plugins are noticed there.
if (!this.pluginInfos.some(pluginInfo => pluginInfo.npmPackage === npmPackage)) {
this.config.restartRequired = true;
}
}
async handleUpdateAllRequest(request: lib.UpdateAllRequest) {
if (!this.config.get("host.allow_remote_updates")) {
throw new lib.RequestError("Remote updates are disabled on this machine");
throw new lib.ForbiddenError("Remote updates are disabled on this machine");
}
if (!this.config.get("host.allow_plugin_updates")) {
throw new lib.RequestError("Plugin updates are disabled on this machine");
throw new lib.ForbiddenError("Plugin updates are disabled on this machine");
}
return await lib.handleUpdateAll("@clusterio/host", this.pluginInfos);
}
async handlePluginListRequest(request: lib.PluginListRequest) {
return this.pluginInfos.map(pluginInfo => lib.PluginDetails.fromNodeEnvInfo(
pluginInfo,
this.loadedPlugins.has(pluginInfo),
this.config.get(`${pluginInfo.name}.load_plugin`),
));
return await this.listPlugins();
}
/** Plugins on this host, including those installed or updated since it started */
async listPlugins() {
return await lib.listPluginDetails(
this.pluginInfos,
pluginInfo => this.loadedPlugins.has(pluginInfo),
pluginInfo => this.config.get(`${pluginInfo.name}.load_plugin`),
pluginInfo => Boolean(pluginInfo.hostEntrypoint || pluginInfo.instanceEntrypoint),
);
}
sendHostUpdate() {
@@ -1253,7 +1321,7 @@ export default class Host extends lib.Link {
let instanceConnection = this.instanceConnections.get(instanceId);
list.push(new lib.HostInstanceUpdate(
instance.config.toRemote("controller"),
instanceConnection ? instanceConnection.instance.status : "stopped",
instanceConnection ? instanceConnection.instance.status : this._unconnectedStatus(instanceId),
instanceConnection ? instanceConnection.instance.server.gamePort : this.gamePort(instanceId),
instanceConnection ? instanceConnection.instance.server.version : undefined,
));
+128 -66
View File
@@ -10,8 +10,8 @@ const execAsync = util.promisify(exec);
// internal libraries
import * as lib from "@clusterio/lib";
import { FactorioServer } from "./server.js";
import { loadModules, patch } from "./patch.js";
import { FactorioServer, checkForUpdates } from "./server.js";
import { loadModules, patch, unpatch } from "./patch.js";
import { exportData } from "./export.js";
import type Host from "./Host.js";
import { BaseInstancePlugin, InstanceHooks } from "./BaseInstancePlugin.js";
@@ -21,7 +21,7 @@ function checkRequestSaveName(name: string) {
try {
lib.checkFilename(name);
} catch (err: any) {
throw new lib.RequestError(`Save name ${err.message}`);
throw new lib.ValidationError(`Save name ${err.message}`);
}
}
@@ -32,6 +32,13 @@ const scriptCommands = [
"/silent-command", "/sc",
];
function instanceLogger(config: lib.InstanceConfig) {
return lib.logger.child({
instance_id: config.get("instance.id"),
instance_name: config.get("instance.name"),
});
}
// Statuses the instance passes through on its way to running or stopped.
const transitoryStatuses: ReadonlySet<Instance["_status"]> = new Set([
"starting", "stopping", "creating_save", "exporting_data",
@@ -183,6 +190,7 @@ export default class Instance extends lib.Link {
_loadedSave: string | null = null;
_playerCheckInterval: ReturnType<typeof setInterval> | undefined;
_hadPlayersOnline = false;
_autoPause = false;
_playerAutosaveSlot = 1;
_expectedUserUpdates: { action: string, name: string, reason: string }[] = [];
_settleWaiters: (() => void)[] = [];
@@ -191,8 +199,8 @@ export default class Instance extends lib.Link {
host: Host,
connector: lib.VirtualConnector,
dir: string,
factorioDir: string,
instanceConfig: lib.InstanceConfig
instanceConfig: lib.InstanceConfig,
server: FactorioServer,
) {
super(connector);
this._host = host;
@@ -202,10 +210,7 @@ export default class Instance extends lib.Link {
this.id = this.config.get("instance.id");
this.logger = lib.logger.child({
instance_id: this.id,
instance_name: this.name,
});
this.logger = instanceLogger(instanceConfig);
this.hooks = new InstanceHooks(this.logger);
@@ -225,28 +230,7 @@ export default class Instance extends lib.Link {
};
this.config.on("fieldChanged", this._configFieldChanged);
let serverOptions = {
logger: this.logger,
// TODO: factorio.version is not validated
version: this.config.get("factorio.version") as lib.TargetVersion,
executablePath: this.config.get("factorio.executable_path") ?? undefined,
gamePort: this.config.get("factorio.game_port") ?? host.assignGamePort(this.id),
rconPort: this.config.get("factorio.rcon_port") ?? undefined,
rconPassword: this.config.get("factorio.rcon_password") ?? undefined,
enableLuaUdp: this.config.get("factorio.enable_lua_udp"),
luaUdpPort: this.config.get("factorio.lua_udp_port") ?? undefined,
enableWhitelist: this.config.get("factorio.enable_whitelist"),
enableAuthserverBans: this.config.get("factorio.enable_authserver_bans"),
verboseLogging: this.config.get("factorio.verbose_logging"),
consoleLogging: this.config.get("factorio.console_logging"),
stripPaths: this.config.get("factorio.strip_paths"),
maxConcurrentCommands: this.config.get("factorio.max_concurrent_commands"),
shutdownTimeoutMs: this.config.get("factorio.shutdown_timeout") * 1000,
};
this.server = new FactorioServer(
factorioDir, this._dir, serverOptions
);
this.server = server;
this.server.on("output", (parsed, line) => {
this.logger.log("server", { message: line, instance_id: this.id, parsed });
@@ -405,10 +389,19 @@ export default class Instance extends lib.Link {
}
}
/**
* True while Factorio has paused the game because auto_pause is on
* and no players are online.
*/
get autoPaused() {
return this._autoPause && this.playersOnline.size === 0;
}
_recordPlayerJoin(name: string) {
if (this.playersOnline.has(name)) {
return;
}
const wasPaused = this.autoPaused;
this.playersOnline.add(name);
let stats = this.playerStats.get(name);
@@ -428,6 +421,9 @@ export default class Instance extends lib.Link {
stats,
};
this.sendTo("controller", new lib.InstancePlayerUpdateEvent("join", name, stats));
if (wasPaused) {
this.hooks.autoPauseChanged.invoke(false);
}
this.hooks.playerEvent.invoke(event);
}
@@ -449,6 +445,9 @@ export default class Instance extends lib.Link {
stats,
};
this.sendTo("controller", new lib.InstancePlayerUpdateEvent("leave", name, stats, reason));
if (this.autoPaused) {
this.hooks.autoPauseChanged.invoke(true);
}
this.hooks.playerEvent.invoke(event);
}
@@ -645,6 +644,7 @@ end`.replace(/\r?\n/g, " ");
notifyExit() {
this._loadedSave = null;
this._autoPause = false;
this.notifyStatus("stopped");
this.connector.emit("close");
@@ -670,20 +670,22 @@ end`.replace(/\r?\n/g, " ");
this._saveStats().catch(err => this.logger.error(`Error saving stats:\n${err.stack}`));
}
async _loadStats() {
static async _readStats(statsPath: string) {
let instanceStats;
try {
instanceStats = JSON.parse(await fs.readFile(this.path("instance-stats.json"), "utf8"));
instanceStats = JSON.parse(await fs.readFile(statsPath, "utf8"));
} catch (err: any) {
if (err.code === "ENOENT") {
return;
return { playerStats: new Map<string, lib.PlayerStats>(), playerAutosaveSlot: 1 };
}
throw err;
}
this.playerStats = new Map(instanceStats["players"].map(
([id, stats]: [number, Static<typeof lib.PlayerStats.jsonSchema>]) => [id, new lib.PlayerStats(stats)])
);
this._playerAutosaveSlot = instanceStats["player_autosave_slot"] || 1;
return {
playerStats: new Map<string, lib.PlayerStats>(instanceStats["players"].map(
([id, stats]: [string, Static<typeof lib.PlayerStats.jsonSchema>]) => [id, new lib.PlayerStats(stats)]
)),
playerAutosaveSlot: (instanceStats["player_autosave_slot"] as number) || 1,
};
}
async _saveStats() {
@@ -691,24 +693,38 @@ end`.replace(/\r?\n/g, " ");
players: [...this.playerStats],
player_autosave_slot: this._playerAutosaveSlot,
}, null, "\t");
await lib.safeOutputFile(this.path("instance-stats.json"), content);
await lib.atomicOverwriteFile(this.path("instance-stats.json"), content);
}
async init(pluginInfos: lib.PluginNodeEnvInfo[]) {
// After changing the status this function is responsible for calling notifyExit if it throws.
this.notifyStatus("starting");
/**
* Create an instance with a Factorio server, downloading Factorio if needed
*
* The host reports the starting status for the instance while this runs.
*/
static async fromConfig(
config: lib.InstanceConfig,
host: Host,
connector: lib.VirtualConnector,
dir: string,
factorioDir: string,
) {
const id = config.get("instance.id");
// TODO: factorio.version is not validated
const targetVersion = config.get("factorio.version") as lib.TargetVersion;
const gamePort = config.get("factorio.game_port") ?? await host.assignFreeGamePort(id);
host.send(new lib.InstanceStatusChangedEvent(id, "starting", gamePort, targetVersion));
let server: FactorioServer;
let stats: Awaited<ReturnType<typeof Instance._readStats>>;
try {
if (this.config.get("factorio.game_port") === null) {
this.server.gamePort = await this._host.assignFreeGamePort(this.id);
}
const factorioVersions = await this.sendTo("controller", new lib.FactorioVersionsRequest());
await this._loadStats();
const factorioVersions = await host.sendTo("controller", new lib.FactorioVersionsRequest());
stats = await Instance._readStats(path.join(dir, "instance-stats.json"));
// Resolve a release channel target (e.g. "stable") to a concrete
// version using the controller's cached latest-releases data, so the
// existing download/lookup logic can fetch and run it.
const targetVersion = this.config.get("factorio.version") as string;
let version = targetVersion;
if (lib.isReleaseChannel(targetVersion)) {
const releases = await this.sendTo("controller", new lib.LatestReleasesRequest());
const releases = await host.sendTo("controller", new lib.LatestReleasesRequest());
const resolved = lib.resolveReleaseChannel(releases, targetVersion);
if (!resolved) {
throw new Error(
@@ -716,16 +732,46 @@ end`.replace(/\r?\n/g, " ");
"latest-releases unavailable and no cached copy on the controller."
);
}
this.server.setTargetVersion(resolved);
version = resolved;
}
await this.server.checkForUpdates(factorioVersions);
await this.server.init();
const logger = instanceLogger(config);
await checkForUpdates(factorioDir, version, factorioVersions, logger);
server = await FactorioServer.fromOptions(factorioDir, dir, {
logger,
version,
executablePath: config.get("factorio.executable_path") ?? undefined,
gamePort,
rconPort: config.get("factorio.rcon_port") ?? undefined,
rconPassword: config.get("factorio.rcon_password") ?? undefined,
enableLuaUdp: config.get("factorio.enable_lua_udp"),
luaUdpPort: config.get("factorio.lua_udp_port") ?? undefined,
enableWhitelist: config.get("factorio.enable_whitelist"),
enableAuthserverBans: config.get("factorio.enable_authserver_bans"),
verboseLogging: config.get("factorio.verbose_logging"),
consoleLogging: config.get("factorio.console_logging"),
stripPaths: config.get("factorio.strip_paths"),
maxConcurrentCommands: config.get("factorio.max_concurrent_commands"),
shutdownTimeoutMs: config.get("factorio.shutdown_timeout") * 1000,
});
} catch (err) {
this.notifyExit();
await this.sendSaveListUpdate();
host.send(new lib.InstanceStatusChangedEvent(id, "stopped", gamePort, targetVersion));
await host.sendSaveListUpdate(id, path.join(dir, "saves"));
throw err;
}
const instance = new this(host, connector, dir, config, server);
instance._status = "starting";
instance.playerStats = stats.playerStats;
instance._playerAutosaveSlot = stats.playerAutosaveSlot;
return instance;
}
/**
* Load the instance plugins and report the instance as initialised
*
* Must be called once the instance is connected to the host.
*/
async loadPlugins(pluginInfos: lib.PluginNodeEnvInfo[]) {
// load plugins
const plugins: Record<string, string> = {};
for (let pluginInfo of pluginInfos) {
@@ -824,10 +870,11 @@ end`.replace(/\r?\n/g, " ");
"_comment_warning": warning,
...await this.resolveServerSettings(this.config.get("factorio.settings"), includeCredentials),
};
await lib.safeOutputFile(
await lib.atomicOverwriteFile(
this.server.writePath("server-settings.json"),
JSON.stringify(serverSettings, null, "\t")
);
return serverSettings;
}
/**
@@ -923,12 +970,12 @@ end`.replace(/\r?\n/g, " ");
* Factorio refuses to read it if it is newer than the game. It then
* loads default settings instead, which can corrupt the save.
*
* @throws {lib.RequestError} if the mod pack version is higher than the server version.
* @throws {lib.ConflictError} if the mod pack version is higher than the server version.
*/
checkModPackVersion(modPack: lib.ModPack) {
const serverVersion = this.server.version;
if (serverVersion && modPack.integerFactorioVersion > lib.integerFullVersion(serverVersion)) {
throw new lib.RequestError(
throw new lib.ConflictError(
`Mod pack ${modPack.name} is for Factorio ${modPack.factorioVersion} ` +
`which is newer than the Factorio ${serverVersion} this instance runs`
);
@@ -942,12 +989,14 @@ end`.replace(/\r?\n/g, " ");
*/
async prepare() {
this.logger.verbose("Writing server-settings.json");
await this.writeServerSettings(true);
// auto_pause is only read on startup
const serverSettings = await this.writeServerSettings(true);
this._autoPause = serverSettings.auto_pause === true;
this._expectedUserUpdates = [];
if (this.config.get("factorio.sync_adminlist") !== "disabled") {
this.logger.verbose("Writing server-adminlist.json");
await lib.safeOutputFile(
await lib.atomicOverwriteFile(
this.server.writePath("server-adminlist.json"),
JSON.stringify([...this._host.adminlist], null, "\t")
);
@@ -955,7 +1004,7 @@ end`.replace(/\r?\n/g, " ");
if (this.config.get("factorio.sync_banlist") !== "disabled") {
this.logger.verbose("Writing server-banlist.json");
await lib.safeOutputFile(
await lib.atomicOverwriteFile(
this.server.writePath("server-banlist.json"),
JSON.stringify([...this._host.banlist].map(
([username, reason]) => ({ username, reason })
@@ -965,7 +1014,7 @@ end`.replace(/\r?\n/g, " ");
if (this.config.get("factorio.sync_whitelist") !== "disabled") {
this.logger.verbose("Writing server-whitelist.json");
await lib.safeOutputFile(
await lib.atomicOverwriteFile(
this.server.writePath("server-whitelist.json"),
JSON.stringify([...this._host.whitelist], null, "\t")
);
@@ -1033,6 +1082,9 @@ end`.replace(/\r?\n/g, " ");
}
if (!this.config.get("factorio.enable_save_patching")) {
if (await unpatch(this.path("saves", saveName))) {
this.logger.info("Removed Clusterio modules from save as save patching is disabled");
}
return saveName;
}
@@ -1259,6 +1311,16 @@ end`.replace(/\r?\n/g, " ");
}
}
/** Stop the instance once a start, stop or save in progress has finished */
async _stopWhenSettled() {
// Stopping concurrently with these races them and asserts on the server state.
// Checked right before stop() as another waiter may have started stopping.
while (transitoryStatuses.has(this._status)) {
await this._waitForSettledStatus();
}
await this.stop();
}
async kill() {
if (this._status === "stopped") {
return;
@@ -1331,7 +1393,7 @@ end`.replace(/\r?\n/g, " ");
async handleInstanceLoadScenarioRequest(request: lib.InstanceLoadScenarioRequest) {
if (this.config.get("factorio.enable_save_patching")) {
this.notifyExit();
throw new lib.RequestError("Load scenario cannot be used with save patching enabled");
throw new lib.ConflictError("Load scenario cannot be used with save patching enabled");
}
try {
@@ -1393,7 +1455,7 @@ end`.replace(/\r?\n/g, " ");
async handleInstanceSaveGameRequest() {
if (this._status !== "running") {
throw new lib.RequestError("Instance is not running");
throw new lib.UnavailableError("Instance is not running");
}
const saved = new Promise<void>((resolve, reject) => {
@@ -1403,7 +1465,7 @@ end`.replace(/\r?\n/g, " ");
};
const onExit = () => {
this.server.off("save-finished", onSaved);
reject(new lib.RequestError("Instance stopped before the save finished"));
reject(new lib.UnavailableError("Instance stopped before the save finished"));
};
this.server.once("save-finished", onSaved);
this.server.once("exit", onExit);
@@ -1445,7 +1507,7 @@ end`.replace(/\r?\n/g, " ");
}
async handleInstanceRestartRequest(request: lib.InstanceRestartRequest) {
await this.stop();
await this._stopWhenSettled();
// Stopping invalidates this instance object, a new one must be created by the host
// To do this we simulate a waking an instance and sending a start request
const instanceConnection = await this._host._connectInstance(this.id);
@@ -1453,7 +1515,7 @@ end`.replace(/\r?\n/g, " ");
}
async handleInstanceStopRequest() {
await this.stop();
await this._stopWhenSettled();
}
async handleInstanceKillRequest() {
+118 -35
View File
@@ -1,12 +1,16 @@
// Export of item icons and locale
import fs from "node:fs/promises";
import path from "path";
import { Jimp, JimpMime, type JimpInstance } from "jimp";
import { createJimp } from "@jimp/core";
import png from "@jimp/js-png";
import * as crop from "@jimp/plugin-crop";
import * as resize from "@jimp/plugin-resize";
import type {} from "@jimp/types"; // needed to name jimp types in export.d.ts
import yazl from "yazl";
import * as lib from "@clusterio/lib";
import * as libBuildMod from "@clusterio/lib/dist/node/build_mod.js";
import type { FactorioServer } from "./server.js";
import { findVersion, isHeadless, type FactorioServer } from "./server.js";
import { definesPrototypesFallback } from "./export_fallback.js";
interface SimpleIconSpecification {
@@ -91,6 +95,16 @@ async function loadZip(server: FactorioServer, modVersions: Map<string, string>,
return modZip;
}
/** Where icon files are read from during an export. */
interface ExportFiles {
/** Mod zips opened during the export, closed when it ends. */
zipCache: ZipCache;
/** Data dir of a full install to fall back to for builtin mod files. */
graphicsDataDir?: string;
/** Why no full install was found for a headless server. */
graphicsError?: string;
}
function closeZipCache(zipCache: ZipCache) {
for (const { zip } of zipCache.values()) {
zip.close();
@@ -103,7 +117,7 @@ function closeZipCache(zipCache: ZipCache) {
*
* @param server - The server to load the file from.
* @param modVersions - Mapping of mod to version used.
* @param zipCache - Mod zips opened by this export.
* @param files - Where to read files from.
* @param modPath - Factorio style path to the file to load.
* @returns The content of the file or null if not found.
* @internal
@@ -111,7 +125,7 @@ function closeZipCache(zipCache: ZipCache) {
async function loadFile(
server: FactorioServer,
modVersions: Map<string, string>,
zipCache: ZipCache,
files: ExportFiles,
modPath: string,
) {
let match = /^__([^\/]+)__\/(.*)$/.exec(modPath);
@@ -124,19 +138,25 @@ async function loadFile(
const builtinModNames = ["core", ...lib.ModPack.getBuiltinModNames(server.version!)];
if (builtinModNames.includes(mod)) {
try {
return await fs.readFile(server.dataPath(mod, filePath));
} catch (err: any) {
if (err.code === "ENOENT") {
return null;
}
throw err;
const dataDirs = [server.dataPath()];
if (files.graphicsDataDir) {
dataDirs.push(files.graphicsDataDir);
}
for (const dataDir of dataDirs) {
try {
return await fs.readFile(path.join(dataDir, mod, filePath));
} catch (err: any) {
if (err.code !== "ENOENT") {
throw err;
}
}
}
return null;
}
let modZip;
try {
modZip = await loadZip(server, modVersions, zipCache, mod);
modZip = await loadZip(server, modVersions, files.zipCache, mod);
} catch (err: any) {
if (err.code === "ENOENT") {
return null;
@@ -147,12 +167,14 @@ async function loadFile(
return await modZip.zip.readFile(path.posix.join(modZip.root, filePath));
}
type Image = JimpInstance;
// Only the parts of jimp the export uses
const Jimp = createJimp({ formats: [png], plugins: [crop.methods, resize.methods] });
type Image = InstanceType<typeof Jimp>;
type IconCache = Map<string, Image | null>;
async function loadIcon(
server: FactorioServer,
modVersions: Map<string, string>,
zipCache: ZipCache,
files: ExportFiles,
iconPath: string,
iconSize: number,
iconMipmaps: number,
@@ -160,7 +182,7 @@ async function loadIcon(
) {
let icon = iconCache.get(iconPath);
if (icon === undefined) {
let fileContent = await loadFile(server, modVersions, zipCache, iconPath);
let fileContent = await loadFile(server, modVersions, files, iconPath);
if (fileContent) {
icon = await Jimp.fromBuffer(fileContent) as Image;
icon.crop({ x: 0, y: 0, w: iconSize, h: iconSize });
@@ -176,12 +198,12 @@ async function loadIcon(
async function loadSimpleIcon(
server: FactorioServer,
modVersions: Map<string, string>,
zipCache: ZipCache,
files: ExportFiles,
item: SimpleIconSpecification,
size: number,
iconCache: IconCache,
): Promise<Image | null> {
let icon = await loadIcon(server, modVersions, zipCache, item.icon, item.icon_size ?? 64, 0, iconCache);
let icon = await loadIcon(server, modVersions, files, item.icon, item.icon_size ?? 64, 0, iconCache);
if (icon) {
let iconScale = size / (item.icon_size ?? 64);
if (iconScale !== 1) {
@@ -195,11 +217,11 @@ async function loadSimpleIcon(
async function loadLayeredIcon(
server: FactorioServer,
modVersions: Map<string, string>,
zipCache: ZipCache,
files: ExportFiles,
item: LayeredIconSpecification,
size: number,
iconCache: IconCache,
): Promise<Image> {
): Promise<Image | null> {
let baseLayerSize = (item.icons[0].icon_size || item.icon_size) ?? 64;
let icon = new Jimp({ width: size, height: size });
@@ -212,11 +234,12 @@ async function loadLayeredIcon(
for (let layer of item.icons) {
let layerSize = (layer.icon_size || item.icon_size) ?? 64;
let iconLayer = await loadIcon(
server, modVersions, zipCache, layer.icon, layerSize, layer.icon_mipmaps || 0, iconCache,
server, modVersions, files, layer.icon, layerSize, layer.icon_mipmaps || 0, iconCache,
);
// A partial icon would be wrong, skip it like a missing simple icon
if (!iconLayer) {
continue;
return null;
}
iconLayer = iconLayer.clone();
@@ -331,7 +354,7 @@ async function growSheet(state: SheetState) {
async function exportIcon(
server: FactorioServer,
modVersions: Map<string, string>,
zipCache: ZipCache,
files: ExportFiles,
state: SheetState,
prototype: PrototypeWithIcon,
) {
@@ -342,9 +365,11 @@ async function exportIcon(
if (prototype.icons) {
icon = await loadLayeredIcon(
server, modVersions, zipCache, prototype, SHEET_ICON_SIZE, state.iconCache,
server, modVersions, files, prototype, SHEET_ICON_SIZE, state.iconCache,
);
iconPos = state.pos;
if (icon) {
iconPos = state.pos;
}
iconPath = prototype.icons[0].icon;
} else if (prototype.icon) {
@@ -352,7 +377,7 @@ async function exportIcon(
iconPath = prototype.icon;
if (iconPos === undefined) {
icon = await loadSimpleIcon(
server, modVersions, zipCache, prototype, SHEET_ICON_SIZE, state.iconCache,
server, modVersions, files, prototype, SHEET_ICON_SIZE, state.iconCache,
);
if (icon) {
iconPos = state.pos;
@@ -388,7 +413,7 @@ async function exportIcon(
*
* @param server - The server to load icons from.
* @param modVersions - Mapping of mod name to version.
* @param zipCache - Mod zips opened by this export.
* @param files - Where to read icon files from.
* @param prototypes - Array of prototypes with icon to export for this base type.
* @param baseName - name of base type (used in metadata).
* @param state - Shared spritesheet state (mutated in place).
@@ -398,7 +423,7 @@ async function exportIcon(
async function exportIcons(
server: FactorioServer,
modVersions: Map<string, string>,
zipCache: ZipCache,
files: ExportFiles,
prototypes: PrototypeWithIcon[],
baseName: string,
state: SheetState,
@@ -407,7 +432,13 @@ async function exportIcons(
state.metadata[baseName] ??= metadata;
let count = 0;
for (let prototype of prototypes) {
const icon = await exportIcon(server, modVersions, zipCache, state, prototype);
let icon: lib.ExportMetadataEntry["icon"];
try {
icon = await exportIcon(server, modVersions, files, state, prototype);
} catch (err: any) {
// One broken icon should not stop the rest of the export
server._logger.warn(`Skipping icon of ${prototype.type} ${prototype.name}: ${err.message}`);
}
if (icon) {
count += 1;
}
@@ -478,6 +509,7 @@ async function exportLocale(
modZip = await loadZip(server, modVersions, zipCache, mod);
} catch (err: any) {
if (err.code === "ENOENT") {
server._logger.warn(`Skipping locale of ${mod}, its mod zip was not found`);
continue;
}
throw err;
@@ -497,20 +529,67 @@ async function exportLocale(
/**
* Export locale, icons and metadata for the given factorio server
*
* If the server runs the headless build, builtin mod graphics are read
* from a full install of the same major and minor version found next to
* it, if there is one.
*
* @param server - The server to export the data from.
* @returns zip file with exported data.
*/
export async function exportData(server: FactorioServer) {
const zipCache: ZipCache = new Map();
const files: ExportFiles = { zipCache: new Map() };
if (await isHeadless(server.dataPath())) {
const majorMinor = server.version!.replace(/\.\d+$/, "") as lib.MajorMinorVersion;
try {
[files.graphicsDataDir] = await findVersion(server._factorioDir, majorMinor, false);
} catch (err: any) {
// Without it the export still has everything but the base game icons
files.graphicsError = err.message;
}
}
try {
return await buildExport(server, zipCache);
return await buildExport(server, files);
} finally {
// Free up the zip files loaded during the export.
closeZipCache(zipCache);
closeZipCache(files.zipCache);
}
}
async function buildExport(server: FactorioServer, zipCache: ZipCache) {
/**
* Warn if any builtin mod icon files could not be found
*
* @param server - The server the export ran on.
* @param files - Where icon files were read from.
* @param iconCache - Icons loaded by the export, null for missing files.
* @internal
*/
function checkMissingBuiltinIcons(server: FactorioServer, files: ExportFiles, iconCache: IconCache) {
const builtinModNames = ["core", ...lib.ModPack.getBuiltinModNames(server.version!)];
let missing = 0;
for (const [iconPath, icon] of iconCache) {
const mod = /^__([^\/]+)__\//.exec(iconPath)?.[1];
if (icon === null && mod && builtinModNames.includes(mod)) {
missing += 1;
}
}
if (!missing) {
return;
}
const dirs = [server.dataPath(), files.graphicsDataDir].filter(Boolean).join(" or ");
let message =
`${missing} icon files of the base game were not found in ${dirs}, icons using them were not exported.`;
if (files.graphicsError !== undefined) {
const majorMinor = server.version!.replace(/\.\d+$/, "");
message +=
` The headless build of Factorio has no graphics and no full install was found (${files.graphicsError}).` +
` Add a full install of Factorio ${majorMinor} to host.factorio_directory to read them from there,` +
" or export the data on a host with a full install.";
}
server._logger.warn(message);
}
async function buildExport(server: FactorioServer, files: ExportFiles) {
await generateExportMod(server);
let settings: lib.ExportSettings = {};
@@ -590,7 +669,7 @@ async function buildExport(server: FactorioServer, zipCache: ZipCache) {
}
}
let locale = await exportLocale(server, modVersions, zipCache, modOrder, "en");
let locale = await exportLocale(server, modVersions, files.zipCache, modOrder, "en");
let zip = new yazl.ZipFile();
const addJson = (name: string, value: unknown) => zip.addBuffer(Buffer.from(JSON.stringify(value)), name);
@@ -611,14 +690,15 @@ async function buildExport(server: FactorioServer, zipCache: ZipCache) {
if (prototypesWithIcons.length === 0) {
continue;
}
const count = await exportIcons(server, modVersions, zipCache, prototypesWithIcons, baseName, state);
const count = await exportIcons(server, modVersions, files, prototypesWithIcons, baseName, state);
server._logger.info(`Exported ${count} ${baseName} icons`);
}
checkMissingBuiltinIcons(server, files, state.iconCache);
// Crop sheet to actual used height and write single spritesheet + metadata.
const usedRows = Math.ceil(state.pos / (SHEET_WIDTH / SHEET_ICON_SIZE));
state.sheet.crop({ x: 0, y: 0, w: SHEET_WIDTH, h: Math.max(usedRows * SHEET_ICON_SIZE, 1) });
zip.addBuffer(await state.sheet.getBuffer(JimpMime.png), "export/spritesheet.png");
zip.addBuffer(await state.sheet.getBuffer("image/png"), "export/spritesheet.png");
addJson("export/metadata.json", state.metadata satisfies lib.ExportMetadata);
zip.end();
@@ -629,6 +709,9 @@ async function buildExport(server: FactorioServer, zipCache: ZipCache) {
// For testing only
export const _createSheetState = createSheetState;
export const _exportIcons = exportIcons;
export const _exportLocale = exportLocale;
export const _loadSimpleIcon = loadSimpleIcon;
export const _loadLayeredIcon = loadLayeredIcon;
export const _checkMissingBuiltinIcons = checkMissingBuiltinIcons;
+305
View File
@@ -0,0 +1,305 @@
import { setTimeout as sleep } from "node:timers/promises";
import * as lib from "@clusterio/lib";
import { logger } from "@clusterio/lib";
/** Time to wait for a response from the controller */
const requestTimeoutMs = 30e3;
/** Placeholder token of a host that has not been paired */
const unpairedToken = "enter token here";
/** Error responded by the controller */
export class PairingError extends Error {
constructor(
message: string,
/** HTTP status the controller responded with */
public status?: number,
) {
super(message);
this.name = "PairingError";
}
}
export interface PairingResult {
hostId: number;
token: string;
name: string;
}
export interface PairHostOptions {
/** URL of the controller, ending with a slash */
controllerUrl: string;
/** Name to request for the host */
name: string;
/** Version of the host, shown with the request */
version?: string;
/** Token of a user allowed to approve hosts, approves the host without waiting */
token?: string;
/** Interval to poll for the result at */
pollIntervalMs?: number;
/** Delay before trying again when the controller can't be reached or is busy */
retryDelayMs?: number;
/** Called with the code to approve the request with each time a request is made */
onPending?: (code: string) => void;
/** Called when the controller can't be reached or is busy and the request is retried */
onRetry?: (err: Error, delayMs: number) => void;
/** Signal to cancel pairing with */
signal?: AbortSignal;
}
interface Response {
status: number;
headers: Headers;
json: any;
}
async function request(
controllerUrl: string,
route: string,
options: { body?: object, token?: string, signal?: AbortSignal, timeoutMs?: number },
): Promise<Response> {
const headers: Record<string, string> = { "Accept": "application/json" };
if (options.body) {
headers["Content-Type"] = "application/json";
}
if (options.token) {
headers["Authorization"] = `Bearer ${options.token}`;
}
const timeout = AbortSignal.timeout(options.timeoutMs ?? requestTimeoutMs);
const response = await fetch(new URL(route, controllerUrl), {
method: options.body ? "POST" : "GET",
headers,
body: options.body ? JSON.stringify(options.body) : undefined,
signal: options.signal ? AbortSignal.any([options.signal, timeout]) : timeout,
});
let json;
try {
json = await response.json();
} catch {
json = undefined;
}
return { status: response.status, headers: response.headers, json };
}
function responseError(response: Response) {
const message = typeof response.json?.error === "string"
? response.json.error
: `Unexpected response from controller: ${response.status}`;
return new PairingError(message, response.status);
}
function pairingResult(response: Response): PairingResult {
const { hostId, token, name } = response.json ?? {};
if (!Number.isInteger(hostId) || typeof token !== "string" || typeof name !== "string") {
throw new PairingError("Invalid response from controller", response.status);
}
return { hostId, token, name };
}
/** Responses that are worth trying again later */
function isTemporary(status: number) {
return status === 429 || status >= 500;
}
function retryDelay(response: Response, defaultMs: number) {
const seconds = Number(response.headers.get("retry-after"));
return Number.isFinite(seconds) && seconds > 0 ? seconds * 1000 : defaultMs;
}
/**
* Poll a request until it is approved, rejected or no longer known by the controller
*
* @returns the result if approved, or null if the controller no longer knows the request.
*/
async function poll(
requestId: string,
options: Required<Pick<PairHostOptions, "pollIntervalMs" | "retryDelayMs">> & PairHostOptions,
) {
let delayMs = options.pollIntervalMs;
while (true) {
await sleep(delayMs, undefined, { signal: options.signal });
delayMs = options.pollIntervalMs;
let response;
try {
response = await request(options.controllerUrl, "api/host-pairing/poll", {
body: { requestId }, signal: options.signal,
});
} catch (err: any) {
if (options.signal?.aborted) {
throw err;
}
// Keep polling through network errors, the request is made again if it expired meanwhile.
delayMs = options.retryDelayMs;
options.onRetry?.(err, delayMs);
continue;
}
if (response.status === 200) {
return pairingResult(response);
}
if (response.status === 404) {
return null;
}
if (isTemporary(response.status)) {
delayMs = retryDelay(response, options.retryDelayMs);
options.onRetry?.(responseError(response), delayMs);
continue;
}
if (response.status !== 202) {
throw responseError(response);
}
}
}
/**
* Request to join the cluster and wait until the request is approved
*
* Waits through network errors and controller restarts, making a new request
* if the controller no longer knows the previous one.
*
* @param options - Pairing options.
* @returns id, token and approved name of the host.
* @throws {PairingError} if the controller refuses or rejects the request.
* @throws {DOMException} AbortError if the signal is aborted.
*/
export async function pairHost(options: PairHostOptions): Promise<PairingResult> {
const opts = { pollIntervalMs: 2000, retryDelayMs: 10e3, version: "", ...options };
let requestId: string | undefined;
try {
while (true) {
opts.signal?.throwIfAborted();
let response;
try {
response = await request(opts.controllerUrl, "api/host-pairing", {
body: { name: opts.name, version: opts.version }, token: opts.token, signal: opts.signal,
});
} catch (err: any) {
if (opts.signal?.aborted) {
throw err;
}
opts.onRetry?.(err, opts.retryDelayMs);
await sleep(opts.retryDelayMs, undefined, { signal: opts.signal });
continue;
}
if (response.status === 200) {
return pairingResult(response);
}
if (isTemporary(response.status)) {
const delayMs = retryDelay(response, opts.retryDelayMs);
opts.onRetry?.(responseError(response), delayMs);
await sleep(delayMs, undefined, { signal: opts.signal });
continue;
}
if (response.status === 404) {
throw new PairingError("The controller must be updated to a version that supports host pairing", 404);
}
if (response.status !== 202) {
throw responseError(response);
}
if (typeof response.json?.requestId !== "string" || typeof response.json.code !== "string") {
throw new PairingError("Invalid response from controller", response.status);
}
requestId = response.json.requestId as string;
opts.onPending?.(response.json.code);
const result = await poll(requestId, opts);
requestId = undefined;
if (result) {
return result;
}
}
} catch (err) {
if (!opts.signal?.aborted) {
throw err;
}
if (requestId !== undefined) {
await request(opts.controllerUrl, "api/host-pairing/cancel", {
body: { requestId }, timeoutMs: 5e3,
}).catch(() => {});
}
throw new DOMException("Pairing was cancelled", "AbortError");
}
}
/**
* Check if the controller accepts a host token
*
* @param controllerUrl - URL of the controller, ending with a slash.
* @param token - Host token to check.
* @param signal - Signal to cancel the check with.
* @returns true if the token is valid and its host is registered.
* @throws {PairingError} if the controller gives an unexpected response.
*/
export async function checkHostToken(controllerUrl: string, token: string, signal?: AbortSignal) {
const response = await request(controllerUrl, "api/host-pairing/check", { token, signal });
if (response.status === 200) {
return true;
}
if (response.status === 401) {
return false;
}
throw responseError(response);
}
/**
* Pair the host with the controller if it has not been paired, or the controller no longer accepts it
*
* Waits until the request is approved and saves the id, name and token of the host to its config.
*
* @param config - Config of the host.
* @param options - Pairing options.
* @throws {PairingError} if the controller refuses or rejects the request.
* @throws {DOMException} AbortError if the signal is aborted.
*/
export async function ensurePaired(
config: lib.HostConfig,
options: { version: string, userToken?: string, signal?: AbortSignal },
) {
const controllerUrl = config.get("host.controller_url");
const token = config.get("host.controller_token");
if (token && token !== unpairedToken) {
let valid;
try {
valid = await checkHostToken(controllerUrl, token, options.signal);
} catch (err: any) {
if (options.signal?.aborted) {
throw err;
}
// Connecting reports the problem, or authenticates if the controller predates pairing.
logger.verbose(`Unable to check if the controller accepts this host: ${err.message}`);
return;
}
if (valid) {
return;
}
logger.warn("The controller no longer accepts this host, requesting to join again");
}
const name = config.get("host.name");
logger.info(`Requesting to join the cluster at ${controllerUrl} as "${name}"`);
const result = await pairHost({
controllerUrl,
name,
version: options.version,
token: options.userToken,
signal: options.signal,
onPending: code => {
const approveUrl = new URL(`hosts?approve=${code}`, controllerUrl).href;
// Logged as warnings for the systemd service, which only prints warnings and above.
// The link is logged on its own so terminals can make it clickable without it wrapping.
logger.warn(`Waiting for approval of "${name}" with code ${code}, open this link to approve it:`);
logger.warn(approveUrl);
logger.warn(`Or run: npx clusterioctl host approve ${code}`);
},
onRetry: (err, delayMs) => {
logger.warn(`Requesting to join failed: ${err.message}, trying again in ${Math.ceil(delayMs / 1000)}s`);
},
});
config.set("host.id", result.hostId);
config.set("host.name", result.name);
config.set("host.controller_token", result.token);
await config.save();
logger.info(`Host approved as "${result.name}" with id ${result.hostId}`);
}
+130 -48
View File
@@ -227,6 +227,7 @@ export class PatchInfo {
json.patch_number,
lib.ModuleInfo.fromJSON(json.scenario),
await Promise.all(json.modules.map(m => SaveModule.fromSave(m, zip, root))),
json.version,
);
}
}
@@ -381,6 +382,72 @@ const knownScenarios: Record<string, lib.ModuleInfo> = {
"3af547d2f4db3728b75ac5d1b3a4f47830dc48e7": new lib.ModuleInfo("freeplay", "2.0.29", [], ["scenario"]),
};
/**
* Names of the zip entries added by patching for the modules in the patch info
*/
function moduleEntries(zip: lib.ZipArchive, patchInfo: PatchInfo, root: string) {
const entries = new Set<string>();
if (patchInfo.version === 0) {
const modulesPath = path.posix.join(root, "modules/");
for (const name of zip.entries.keys()) {
if (name.startsWith(modulesPath)) {
entries.add(name);
}
}
} else {
for (const module of patchInfo.modules) {
for (const filepath of module.files.keys()) {
entries.add(path.posix.join(root, filepath));
}
}
}
return entries;
}
/**
* Write a copy of the save next to it with entries removed and added
*
* Keeps the order of the entries in the original.
*
* @returns Path to the temporary file written.
*/
async function rewriteSave(
zip: lib.ZipArchive, savePath: string, removed: Set<string>, added: Map<string, Buffer>
) {
added = new Map(added);
const output = new lib.ZipWriter();
const [tempSavePath, writeStream] = await lib.createTempWriteStream(savePath);
const writing = pipeline(output.outputStream, writeStream);
// Awaited below, but a write error can reject it before then.
writing.catch(() => {});
try {
for (const entry of zip.entries.values()) {
const name = entry.fileName;
if (removed.has(name)) {
continue;
}
const contents = added.get(name);
if (contents !== undefined) {
await output.addBuffer(contents, name);
added.delete(name);
} else {
await output.addEntry(zip, entry);
}
}
for (const [name, contents] of added) {
await output.addBuffer(contents, name);
}
await output.end();
await writing;
} catch (err) {
output.outputStream.destroy();
await writing.catch(() => {});
await fs.rm(tempSavePath, { force: true });
throw err;
}
return tempSavePath;
}
/**
* Patch a save with the given modules
*
@@ -412,6 +479,16 @@ export async function patch(savePath: string, modules: SaveModule[]) {
);
}
// Unpatching put the original control.lua back in place of scenario.lua
const scenarioRequired = [...patchInfo.scenario.load, ...patchInfo.scenario.require].includes("scenario");
if (scenarioRequired && !zip.file(rootPath("scenario.lua"))) {
const control = await zip.readFile(rootPath("control.lua"));
if (control === null) {
throw new Error("Unable to patch save, missing control.lua file.");
}
files.set("scenario.lua", control);
}
// No info file present, try to detect if it's a known compatible scenario.
} else {
const controlFile = zip.file(rootPath("control.lua"));
@@ -432,22 +509,8 @@ export async function patch(savePath: string, modules: SaveModule[]) {
patchInfo.patchNumber = patchInfo.patchNumber + 1;
// Remove any existing modules from the save
const removed = new Set<string>();
if (patchInfo.version === 0) {
const modulesPath = rootPath("modules/");
for (const name of zip.entries.keys()) {
if (name.startsWith(modulesPath)) {
removed.add(name);
}
}
} else {
for (const module of patchInfo.modules) {
for (const filepath of module.files.keys()) {
removed.add(rootPath(filepath));
}
}
patchInfo.modules = [];
}
const removed = moduleEntries(zip, patchInfo, root);
patchInfo.modules = [];
reorderDependencies(modules);
@@ -463,45 +526,64 @@ export async function patch(savePath: string, modules: SaveModule[]) {
files.set("control.lua", Buffer.from(generateLoader(patchInfo), "utf8"));
files.set("clusterio.json", Buffer.from(JSON.stringify(patchInfo, null, "\t"), "utf8"));
// Write back the save, keeping the order of the entries in the original
const added = new Map([...files].map(([relativePath, contents]) => [rootPath(relativePath), contents]));
const output = new lib.ZipWriter();
let writeStream;
[tempSavePath, writeStream] = await lib.createTempWriteStream(savePath);
const writing = pipeline(output.outputStream, writeStream);
// Awaited below, but a write error can reject it before then.
writing.catch(() => {});
try {
for (const entry of zip.entries.values()) {
const name = entry.fileName;
if (removed.has(name)) {
continue;
}
const contents = added.get(name);
if (contents !== undefined) {
await output.addBuffer(contents, name);
added.delete(name);
} else {
await output.addEntry(zip, entry);
}
}
for (const [name, contents] of added) {
await output.addBuffer(contents, name);
}
await output.end();
await writing;
} catch (err) {
output.outputStream.destroy();
await writing.catch(() => {});
await fs.rm(tempSavePath, { force: true });
throw err;
}
tempSavePath = await rewriteSave(zip, savePath, removed, added);
} finally {
zip.close();
}
await fs.rename(tempSavePath, savePath);
}
/**
* Remove the modules patched into a save
*
* Moves the original control.lua that patching kept as scenario.lua back
* and removes the module files. The patch info is kept with no modules so that
* the patch number keeps counting if the save is patched again.
*
* @param savePath - Path to the Factorio save to unpatch.
* @returns true if the save was changed.
*/
export async function unpatch(savePath: string) {
const zip = await lib.ZipArchive.fromFile(savePath);
let tempSavePath: string;
try {
const root = lib.findRoot(zip);
const rootPath = (relativePath: string) => path.posix.join(root, relativePath);
const patchInfoContent = await zip.readFile(rootPath("clusterio.json"));
if (patchInfoContent === null) {
return false;
}
const patchInfo = await PatchInfo.fromSave(JSON.parse(patchInfoContent.toString("utf8")), zip, root);
if (patchInfo.version > PatchInfo.currentVersion) {
throw new Error(
`Save patch version ${patchInfo.version} is newer than the patch version this ` +
`version of Clusterio can unpatch (${PatchInfo.currentVersion})`
);
}
if (patchInfo.version !== 0 && !patchInfo.modules.length) {
return false;
}
const removed = moduleEntries(zip, patchInfo, root);
patchInfo.modules = [];
const scenario = await zip.readFile(rootPath("scenario.lua"));
if (scenario !== null) {
removed.add(rootPath("scenario.lua"));
}
// Scenarios made for Clusterio have no scenario.lua, keep loading them without modules
const control = scenario ?? Buffer.from(generateLoader(patchInfo), "utf8");
tempSavePath = await rewriteSave(zip, savePath, removed, new Map([
[rootPath("control.lua"), control],
[rootPath("clusterio.json"), Buffer.from(JSON.stringify(patchInfo, null, "\t"), "utf8")],
]));
} finally {
zip.close();
}
await fs.rename(tempSavePath, savePath);
return true;
}
// For testing only
export const _generateLoader = generateLoader;
export const _reorderDependencies = reorderDependencies;
+138 -97
View File
@@ -185,6 +185,73 @@ async function downloadAndExtractTar(url: string, targetDir: string) {
// Factorio downloads in progress by target directory, shared by every server in the process
const downloadsInProgress = new Map<string, Promise<void>>();
/**
* Download the newest Factorio version matching the target if not installed
*
* Call before {@link FactorioServer.fromOptions} for it to find the new version.
*
* @param factorioDir - Path to Factorio installation dirs.
* @param targetVersion - Version to look for, or "latest".
* @param availableVersions - Versions available for download.
* @param logger - Logger to report the download on.
*/
export async function checkForUpdates(
factorioDir: string,
targetVersion: lib.TargetVersion,
availableVersions: lib.ExternalFactorioVersion[],
logger: lib.Logger = lib.logger,
) {
const latestVersion = availableVersions
.filter(v => (
targetVersion === "latest" || v.version === targetVersion || v.version.startsWith(targetVersion)
))
.sort((v1, v2) => versionOrder(v1.version, v2.version))
[0];
if (!latestVersion) {
return;
}
// A version that is still being extracted can already look installed
const dst = path.join(factorioDir, latestVersion.version);
const installedVersions = await listFactorioVersions(factorioDir);
if (!installedVersions.versions.has(latestVersion.version) || downloadsInProgress.has(dst)) {
const v = latestVersion.version;
// Can't download if not linux or this is a direct install
if (installedVersions.direct) {
logger.info(
`A newer version of factorio is available (${v}) but cannot be downloaded automatically ` +
`because ${factorioDir} is a direct (single-version) Factorio install. Use a versioned ` +
`layout, where ${factorioDir} contains a subdirectory per version, to enable automatic ` +
"downloads."
);
return;
}
if (process.platform !== "linux") {
logger.info(
`A newer version of factorio is available (${v}) but must be manually downloaded ` +
"(automatic downloads are only supported on Linux)."
);
return;
}
// Download the newer version, or wait for another server already downloading it
let download = downloadsInProgress.get(dst);
if (download) {
logger.info(`A newer version of factorio is available (${v}), waiting for download...`);
} else {
logger.info(`A newer version of factorio is available (${v}), starting download...`);
download = downloadAndExtractTar(latestVersion.headlessUrl, dst).finally(() => {
downloadsInProgress.delete(dst);
});
downloadsInProgress.set(dst, download);
}
await download;
logger.info(`Downloaded factorio version ${v}`);
}
}
/**
* Find factorio data directory of the given version
*
@@ -195,15 +262,33 @@ const downloadsInProgress = new Map<string, Promise<void>>();
* @param targetVersion -
* Version to look for, supports the special value "latest" for the
* latest version available.
* @param headless -
* Only consider headless installs if true, or only full installs if
* false.
* @returns Array with path to data dir and version found.
*/
export async function findVersion(
factorioDir: string,
targetVersion: lib.TargetVersion,
headless?: boolean,
): Promise<[string, lib.FullVersion]> {
let kind = "";
if (headless !== undefined) {
kind = headless ? "headless " : "full ";
}
const checkInstallType = async (dir: string) => (
headless === undefined || await isHeadless(path.join(dir, "data")) === headless
);
// Check if this is a direct installation dir
const directVersion = await getFactorioVersion(factorioDir);
if (directVersion !== null) {
if (!await checkInstallType(factorioDir)) {
throw new Error(
`Unable to find a ${kind}install of Factorio ${targetVersion}: ${factorioDir} is a direct ` +
`(single-version) install of the ${headless ? "full" : "headless"} build.`
);
}
if (targetVersion === "latest" || directVersion === targetVersion || directVersion.startsWith(targetVersion)) {
return [path.join(factorioDir, "data"), directVersion];
}
@@ -220,7 +305,7 @@ export async function findVersion(
// Avoid any fancy directory detection in order to follow symlinks
const fullPath = path.join(factorioDir, name);
const version = await getFactorioVersion(fullPath);
if (version === null) {
if (version === null || !await checkInstallType(fullPath)) {
continue;
}
@@ -232,7 +317,7 @@ export async function findVersion(
}
if (!versions.size) {
throw new Error(`Unable to find any Factorio install in ${factorioDir}`);
throw new Error(`Unable to find any ${kind}Factorio install in ${factorioDir}`);
}
if (lib.isPartialVersion(targetVersion)) {
@@ -248,7 +333,26 @@ export async function findVersion(
return [path.join(factorioDir, versions.get(latest)!, "data"), latest];
}
throw new Error(`Unable to find Factorio version ${targetVersion}`);
throw new Error(`Unable to find ${kind}Factorio version ${targetVersion}`);
}
/**
* Check if a Factorio install is the headless build
*
* The headless build has no icons, which is what tells it apart.
*
* @param dataDir - Path to the data dir of the install.
*/
export async function isHeadless(dataDir: string) {
try {
await fs.access(path.join(dataDir, "base", "graphics", "icons"));
return false;
} catch (err: any) {
if (err.code === "ENOENT" || err.code === "ENOTDIR") {
return true;
}
throw err;
}
}
/**
@@ -670,9 +774,8 @@ export class FactorioServer extends events.EventEmitter<FactorioServerEvents> {
_factorioDir: string;
_writeDir: string;
// Resolved in init
_version: lib.FullVersion | null = null;
_dataDir: string | null = null;
_version: lib.FullVersion;
_dataDir: string;
// Due to inconsistencies in the factorio api, we must manually watch the whitelist
// https://forums.factorio.com/viewtopic.php?t=123673
@@ -680,8 +783,7 @@ export class FactorioServer extends events.EventEmitter<FactorioServerEvents> {
_whitelist = new Set<string>();
_logger: lib.Logger;
_targetVersion: lib.TargetVersion;
_state: "new" | "init" | "create" | "running" | "stopping" = "new";
_state: "init" | "create" | "running" | "stopping" = "init";
_server: child_process.ChildProcessWithoutNullStreams | null = null;
_rconClient: Rcon | null= null;
_rconReady = false;
@@ -697,27 +799,36 @@ export class FactorioServer extends events.EventEmitter<FactorioServerEvents> {
/**
* Create a Factorio server interface
* Create a Factorio server interface for the version found by {@link fromOptions}
*
* @param factorioDir - Directory of the Factorio install(s).
* @param writeDir - Directory to write runtime data to.
* @param options - Optional parameters.
* @param dataDir - Data directory of the Factorio version to run.
* @param version - Factorio version to run.
* @param options - Optional parameters, `version` is ignored.
*/
constructor(factorioDir: string, writeDir: string, options: FactorioServerOptions) {
constructor(
factorioDir: string,
writeDir: string,
dataDir: string,
version: lib.FullVersion,
options: FactorioServerOptions & { rconPassword: string },
) {
super();
this._factorioDir = factorioDir;
this._writeDir = writeDir;
this._dataDir = dataDir;
this._version = version;
this._logger = options.logger || lib.logger;
this._targetVersion = options.version || "latest";
this.executablePath = options.executablePath;
/** UDP port used for hosting the Factorio game server on */
this.gamePort = options.gamePort || randomDynamicPort();
/** TCP port used for RCON on the Factorio game server */
this.rconPort = options.rconPort || randomDynamicPort();
/** Password used for RCON on the Factorio game server */
this.rconPassword = options.rconPassword as string; // init will generate one if not there
this.rconPassword = options.rconPassword;
/** Enable UDP messaging with the Factorio game server */
this.enableLuaUdp = options.enableLuaUdp || false;
/** UDP port the Factorio game server receives Lua UDP messages on */
@@ -886,7 +997,7 @@ export class FactorioServer extends events.EventEmitter<FactorioServerEvents> {
// Factorio before 2.1.10 crashes when a packet is received on a headless server
_checkLuaUdpVersion() {
if (this.enableLuaUdp && lib.integerFullVersion(this._version!) < lib.integerFullVersion("2.1.10")) {
if (this.enableLuaUdp && lib.integerFullVersion(this._version) < lib.integerFullVersion("2.1.10")) {
throw new Error("Lua UDP requires Factorio 2.1.10 or later");
}
}
@@ -1075,94 +1186,25 @@ export class FactorioServer extends events.EventEmitter<FactorioServerEvents> {
}
/**
* Override the target version of Factorio to run
* Create a Factorio server interface for an installed version
*
* Used to substitute a concrete version for a release channel target
* (e.g. "stable" or "experimental") before {@link checkForUpdates} and
* {@link init} run. Must be called before the server is initialized.
*/
setTargetVersion(version: lib.TargetVersion) {
this._check(["new"]);
this._targetVersion = version;
}
/**
* Initialize class instance
*
* Must be called before instances of this class can be used.
* @param factorioDir - Directory of the Factorio install(s).
* @param writeDir - Directory to write runtime data to.
* @param options - Optional parameters.
* @throws {Error} if the requested version of Factorio was not found.
*/
async init() {
this._check(["new"]);
[this._dataDir, this._version] = await findVersion(this._factorioDir, this._targetVersion);
this.rconPassword = this.rconPassword as string | undefined ?? await generatePassword(10);
this._state = "init";
}
/**
* Check of updated versions to download
*
* Should be called before init for the new version to be detected during init
*/
async checkForUpdates(availableVersions: lib.ExternalFactorioVersion[]) {
const targetVersion = this._targetVersion;
const latestVersion = availableVersions
.filter(v => (
targetVersion === "latest" || v.version === targetVersion || v.version.startsWith(targetVersion)
))
.sort((v1, v2) => versionOrder(v1.version, v2.version))
[0];
if (!latestVersion) {
return;
}
// A version that is still being extracted can already look installed
const dst = path.join(this._factorioDir, latestVersion.version);
const installedVersions = await listFactorioVersions(this._factorioDir);
if (!installedVersions.versions.has(latestVersion.version) || downloadsInProgress.has(dst)) {
const v = latestVersion.version;
// Can't download if not linux or this is a direct install
if (installedVersions.direct) {
this._logger.info(
`A newer version of factorio is available (${v}) but cannot be downloaded automatically ` +
`because ${this._factorioDir} is a direct (single-version) Factorio install. Use a versioned ` +
`layout, where ${this._factorioDir} contains a subdirectory per version, to enable automatic ` +
"downloads."
);
return;
}
if (process.platform !== "linux") {
this._logger.info(
`A newer version of factorio is available (${v}) but must be manually downloaded ` +
"(automatic downloads are only supported on Linux)."
);
return;
}
// Download the newer version, or wait for another server already downloading it
let download = downloadsInProgress.get(dst);
if (download) {
this._logger.info(`A newer version of factorio is available (${v}), waiting for download...`);
} else {
this._logger.info(`A newer version of factorio is available (${v}), starting download...`);
download = downloadAndExtractTar(latestVersion.headlessUrl, dst).finally(() => {
downloadsInProgress.delete(dst);
});
downloadsInProgress.set(dst, download);
}
await download;
this._logger.info(`Downloaded factorio version ${v}`);
}
static async fromOptions(factorioDir: string, writeDir: string, options: FactorioServerOptions) {
const [dataDir, version] = await findVersion(factorioDir, options.version || "latest");
const rconPassword = options.rconPassword ?? await generatePassword(10);
return new this(factorioDir, writeDir, dataDir, version, { ...options, rconPassword });
}
/**
* The version of Factorio in use. This will be the actual version if
* "latest" (the default) was specified as `factorioVersion` to the constructor.
* This will be undefined before the server is initialized, or if init failed.
* "latest" (the default) was passed to {@link fromOptions}.
*/
get version(): lib.FullVersion | undefined {
return this._version ?? undefined;
get version(): lib.FullVersion {
return this._version;
}
/**
@@ -1681,14 +1723,13 @@ export class FactorioServer extends events.EventEmitter<FactorioServerEvents> {
* Return path in data directory
*
* Creates a path using path.join with the given parts that's relative to
* the data directory of the Factorio server. Not valid before init has
* been called.
* the data directory of the Factorio server.
*
* @param parts - Extra parts to add to the data path.
* @returns Data directory path.
*/
dataPath(...parts: string[]) {
return path.join(this._dataDir!, ...parts);
return path.join(this._dataDir, ...parts);
}
/**
@@ -0,0 +1,3 @@
// This needs to be a separate file due to ECMAScript Modules evaluating code in
// imported modules before any code in the file that imports it.
(global as any).lazySchemaCompilation = true;
-4
View File
@@ -1,4 +0,0 @@
{
"entryPoints": ["index.ts"],
"includeVersion": true
}
+1
View File
@@ -1,3 +1,4 @@
import "./src/browser_polyfills";
export * from "./src/api";
export * from "./src/AsyncHook";
export * from "./src/SyncHook";
+1 -1
View File
@@ -6,5 +6,5 @@ export interface BuildArgs {
build?: boolean;
outputDir: string;
sourceDir: string;
};
}
export function build(args: BuildArgs): Promise<void>;
+1
View File
@@ -35,6 +35,7 @@ export * from "./src/subscriptions.js";
export * from "./src/datastore.js";
export * from "./src/load_plugin_list.js";
export * from "./src/rce_ops.js";
export * from "./src/npm_registry.js";
export * from "./src/LockFile.js";
export * from "./src/AsyncHook.js";
export * from "./src/SyncHook.js";
+1 -1
View File
@@ -46,7 +46,7 @@
"compile_validator": "node scripts/compile_validator.js"
},
"engines": {
"node": ">=22"
"node": "^22.18.0 || >=24.2.0"
},
"dependencies": {
+144 -31
View File
@@ -1,5 +1,5 @@
import { FileHandle, open, unlink, readFile } from "node:fs/promises";
import { unlinkSync } from "node:fs";
import fs, { type FileHandle } from "node:fs/promises";
import { readFileSync, unlinkSync } from "node:fs";
import assert from "assert";
/**
@@ -24,23 +24,99 @@ export class LockFileBusyError extends Error {
}
}
/**
* Error thrown when a process exited while clearing a stale lock, which needs the recovery file removed by hand.
*/
export class LockFileRecoveryError extends Error {
constructor(
readonly recoveryPath: string
) {
super(`Lock file recovery was interrupted, remove ${recoveryPath} to allow the lock to be acquired`);
this.name = "LockFileRecoveryError";
}
}
type LockState = "idle" | "acquiring" | "acquired" | "releasing";
type LockSnapshot = { content: string, ino: bigint, mtimeNs: bigint };
/**
* Cross-platform lock file utility for Node.js.
*/
export class LockFile {
/**
* Content written to the lock file. The start time tells this process apart from an earlier one with
* the same pid, and is the same for every copy of lib loaded in this process.
*/
private static readonly content = `${process.pid} ${performance.timeOrigin}\n`;
// A lock file without a valid pid is left alone for this long, as it may still be being written
private static readonly writeGraceMs = 10e3;
private handle: FileHandle | null = null;
private state: LockState = "idle";
/**
* @param filePath - Path of the lock file
* @param handleSignals -
* Release the lock on SIGINT, SIGTERM, SIGHUP and uncaught exceptions. Listening to these
* changes how the process exits, so without it the lock is only removed on exit.
*/
constructor(
private readonly filePath: string
private readonly filePath: string,
private readonly handleSignals = true,
) {}
/** Content and identity of a lock file, read through one handle so both belong to the same file */
private static async readLock(filePath: string): Promise<LockSnapshot | undefined> {
let handle;
try {
handle = await fs.open(filePath, "r");
} catch (err: any) {
if (err.code === "ENOENT") {
return undefined;
}
throw err;
}
try {
const stat = await handle.stat({ bigint: true });
return { content: await handle.readFile("utf8"), ino: stat.ino, mtimeNs: stat.mtimeNs };
} finally {
await handle.close();
}
}
private static isSameLock(a: LockSnapshot, b: LockSnapshot) {
return a.content === b.content && a.ino === b.ino && a.mtimeNs === b.mtimeNs;
}
private static lockPid(lock: LockSnapshot) {
const pid = Number(lock.content.split(" ")[0]);
return Number.isInteger(pid) && pid > 0 ? pid : undefined;
}
/** True if the lock names a process that is still running, or may still be written to */
private static lockIsLive(lock: LockSnapshot) {
const pid = LockFile.lockPid(lock);
if (pid === undefined) {
return Date.now() - Number(lock.mtimeNs / 1000000n) < LockFile.writeGraceMs;
}
if (pid === process.pid) {
// Held by this process, or left by an earlier process with the same pid
return lock.content === LockFile.content;
}
try {
// Signal 0 only checks that the process exists
process.kill(pid, 0);
return true;
} catch (err: any) {
return err.code !== "ESRCH";
}
}
/**
* Acquire the lock file.
* Throws LockFileExistsError if already locked.
* Throws LockFileBusyError if not in idle state.
* Throws LockFileRecoveryError if clearing a stale lock was interrupted.
*/
async acquire(): Promise<void> {
if (this.state !== "idle") {
@@ -49,13 +125,10 @@ export class LockFile {
this.state = "acquiring";
if (await this.isStale()) {
await unlink(this.filePath).catch(() => {}); // ignore errors, may already be removed
}
try {
this.handle = await open(this.filePath, "wx");
await this.handle.write(`${process.pid}\n`);
await this.removeStale();
this.handle = await fs.open(this.filePath, "wx");
await this.handle.write(LockFile.content);
this.attachExitHandlers();
this.state = "acquired";
} catch (err: any) {
@@ -81,7 +154,9 @@ export class LockFile {
// Ignored: file may have already been removed by another process
try { await this.handle?.close(); } catch {}
try { await unlink(this.filePath); } catch {}
if (await this.isHeldFile()) {
await fs.unlink(this.filePath).catch(() => {});
}
this.handle = null;
this.state = "idle";
@@ -89,19 +164,20 @@ export class LockFile {
/**
* Check whether the lock is currently held by this process.
* True only if the lock is acquired and the PID in the file
* matches the current process.
* True only if the lock is acquired and the lock file was written by this process.
*/
async isHeld(): Promise<boolean> {
if (this.state !== "acquired") {
return false;
}
return await this.isHeldFile();
}
private async isHeldFile() {
try {
const pidInLock = Number(await readFile(this.filePath, "utf8"));
return pidInLock === process.pid;
return (await LockFile.readLock(this.filePath))?.content === LockFile.content;
} catch {
// Ignored: file may not exist or be unreadable, so lock is not held
// Ignored: file may be unreadable, so lock is not held
return false;
}
}
@@ -110,14 +186,45 @@ export class LockFile {
* Determine whether an existing lock is stale.
*/
async isStale(): Promise<boolean> {
const lock = await LockFile.readLock(this.filePath);
return lock === undefined || !LockFile.lockIsLive(lock);
}
/** Pid of the process holding the lock, if it has written it */
async holderPid(): Promise<number | undefined> {
const lock = await LockFile.readLock(this.filePath);
return lock && LockFile.lockPid(lock);
}
/**
* Remove a stale lock file if it is still the same file. Clearing is done while holding a separate
* recovery lock, so a lock another process took after the stale check is never removed.
*/
private async removeStale(): Promise<void> {
const stale = await LockFile.readLock(this.filePath);
if (!stale || LockFile.lockIsLive(stale)) {
return;
}
const recoveryPath = `${this.filePath}.recovery`;
try {
const pidInLock = Number(await readFile(this.filePath, "utf8"));
if (!Number.isFinite(pidInLock)) { return true; }
// signal 0 is a special case to test if a process exists
process.kill(pidInLock, 0);
return false;
} catch {
return true;
await fs.writeFile(recoveryPath, LockFile.content, { flag: "wx" });
} catch (err: any) {
if (err.code !== "EEXIST") {
throw err;
}
const holder = await LockFile.readLock(recoveryPath);
if (holder && !LockFile.lockIsLive(holder)) {
throw new LockFileRecoveryError(recoveryPath);
}
return; // Another process is clearing it
}
try {
const current = await LockFile.readLock(this.filePath);
if (current && LockFile.isSameLock(current, stale)) {
await fs.unlink(this.filePath);
}
} finally {
await fs.unlink(recoveryPath).catch(() => {});
}
}
@@ -126,10 +233,12 @@ export class LockFile {
*/
private attachExitHandlers(): void {
assert(this.state === "acquiring");
process.on("SIGINT", this.handleExit);
process.on("SIGTERM", this.handleExit);
process.on("SIGHUP", this.handleExit);
process.on("uncaughtException", this.handleExit);
if (this.handleSignals) {
process.on("SIGINT", this.handleExit);
process.on("SIGTERM", this.handleExit);
process.on("SIGHUP", this.handleExit);
process.on("uncaughtException", this.handleExit);
}
// eslint-disable-next-line node/no-sync
process.on("exit", this.handleExitSync);
}
@@ -137,7 +246,7 @@ export class LockFile {
/**
* Detach process handlers, called when transitioning to "idle"
*/
private async detachExitHandlers(): Promise<void> {
private detachExitHandlers(): void {
assert(this.state === "releasing");
process.off("SIGINT", this.handleExit);
process.off("SIGTERM", this.handleExit);
@@ -147,17 +256,21 @@ export class LockFile {
process.off("exit", this.handleExitSync);
}
// Event handler for async signals / exceptions
// Event handler for async signals / exceptions
private handleExit = async (): Promise<void> => {
await this.release();
};
// Event handler for synchronous process exit
// Event handler for synchronous process exit, readLock can't be used as it is async
private handleExitSync = (): void => {
try {
unlinkSync(this.filePath);
// eslint-disable-next-line node/no-sync
if (readFileSync(this.filePath, "utf8") === LockFile.content) {
// eslint-disable-next-line node/no-sync
unlinkSync(this.filePath);
}
} catch (err) {
// Ignored: file may have already been removed by another process
// Ignored: file may have already been removed by another process
}
};
}
+2 -2
View File
@@ -3,7 +3,7 @@ import path from "path";
import events from "events";
import { Static } from "@sinclair/typebox";
import { logger } from "./logging.js";
import { downloadFile, safeOutputFile } from "./file_ops.js";
import { downloadFile, atomicOverwriteFile } from "./file_ops.js";
import { ModPortalReleaseSchema, ModPortalDetailsSchema, ModNameVersionPair } from "./data/messages_mod.js";
import { ModInfo, ModPack, MajorMinorVersion, ModVersionEquality, SourceVersion } from "./data/index.js";
@@ -148,7 +148,7 @@ export default class ModStore extends events.EventEmitter<ModStoreEvents> {
if (files.size) {
logger.info(`Loaded info for ${files.size} mods`);
const content: ModInfoCache = [...files.values()].map(mod => mod.toJSON());
await safeOutputFile(cacheFilename, JSON.stringify(content));
await atomicOverwriteFile(cacheFilename, JSON.stringify(content));
}
return new this(modsDirectory, files);
+1 -1
View File
@@ -14,4 +14,4 @@ export type PluginWebApi = {
* package.
*/
npmPackage?: string;
}
};
+9
View File
@@ -0,0 +1,9 @@
// Node.js globals used by lib and its dependencies when running in a browser.
import process from "process";
import { Buffer } from "buffer";
// Vite's development server may already have created process.env
if (typeof globalThis.process?.nextTick !== "function") {
globalThis.process = Object.assign(process, globalThis.process);
}
globalThis.Buffer ??= Buffer as any;
+55 -84
View File
@@ -7,45 +7,14 @@ import isDeepStrictEqual from "#is_deep_strict_equal";
import { basicType } from "../helpers.js";
import * as libSchema from "../schema.js";
import { StringEnum } from "../data/composites.js";
import { safeOutputFile } from "../file_ops.js";
import { atomicOverwriteFile } from "../file_ops.js";
import * as validators from "./validators.js";
import { RequestError } from "../errors.js";
import { InvalidAccessError, InvalidFieldError, InvalidValueError } from "../errors.js";
import type { SyncHook } from "../SyncHook.js";
const ConfigLocation = StringEnum(["controller", "host", "control"]);
export type ConfigLocation = Static<typeof ConfigLocation>;
/**
* Invalid Access exception
*
* Exception class for when an access attempt is made to a field that
* does not permitt this access is done.
*
* @extends Error
*/
export class InvalidAccess extends RequestError { };
/**
* Invalid Value exception
*
* Exception class for when invalid values are attempted to be set on a
* field.
*
* @extends Error
*/
export class InvalidValue extends RequestError { };
/**
* Invalid Field exception
*
* Exception class for when a group or field that does not exist is
* attempted to be accessed.
*
* @extends Error
*/
export class InvalidField extends RequestError { };
const OldConfigGroupSchema = Type.Object({
"name": Type.String(),
"fields": Type.Record(Type.String(), Type.Unknown()),
@@ -55,7 +24,7 @@ type OldConfigGroupSchema = Static<typeof OldConfigGroupSchema>;
const OldConfigSchema = Type.Object({
"groups": Type.Array(OldConfigGroupSchema),
});
export type OldConfigSchema = Static<typeof OldConfigSchema>
export type OldConfigSchema = Static<typeof OldConfigSchema>;
export const ConfigSchema = Type.Record(Type.String(), Type.Union([
Type.Null(),
@@ -69,11 +38,11 @@ export type ConfigSchema = Static<typeof ConfigSchema>;
type FieldValue = null | boolean | string | number | object;
type FieldType<T extends FieldValue> =
T extends boolean ? "boolean" :
T extends string ? "string" :
T extends number ? "number" :
T extends object ? "object" :
never;
T extends boolean ? "boolean"
: T extends string ? "string"
: T extends number ? "number"
: T extends object ? "object"
: never;
export interface FieldDefinition<
Type extends FieldValue = any,
@@ -119,10 +88,11 @@ export interface FieldDefinition<
optional?: Type extends null ? true : false;
/**
* Value this config field should take in a newly initialized config.
* This can also be a synchronous function returning the value to use.
* This can also be a synchronous function returning the value to use,
* it is called after the stored fields are loaded so that it can read them.
* Promise results are not awaited.
*/
initialValue?: unknown | (() => unknown);
initialValue?: unknown | ((config: Config<Fields>) => unknown);
/**
* Array of all values that are acceptible. Useful for making
* enumerated values.
@@ -165,21 +135,18 @@ export type ConfigDefs<
Fields extends Record<keyof Fields, FieldValue>
> = {
[Prop in keyof Fields]: FieldDefinition<Fields[Prop], Fields>;
}
};
function defaultValue(def: FieldDefinition) {
if (def.initialValue === undefined) {
return null;
}
if (typeof def.initialValue === "function") {
return def.initialValue();
}
return def.initialValue;
}
export type ConfigEvents = {
fieldChanged: [ name: string, curr: FieldValue, prev: FieldValue ],
}
};
export enum ConfigAccess {
read = 0x1,
@@ -277,14 +244,12 @@ export class Config<
}
super();
const defs = Object.entries(this.constructor.fieldDefinitions).filter(
([name, def]) => this._checkAccess(name, def, location, ConfigAccess.read, false)
);
// Function initial values are left undefined until the stored fields are loaded
this.fields = Object.fromEntries(
Object.entries(
this.constructor.fieldDefinitions
).filter(
([name, def]) => this._checkAccess(name, def, location, ConfigAccess.read, false)
).map(
([name, def]) => [name, defaultValue(def)]
)
defs.map(([name, def]) => [name, typeof def.initialValue === "function" ? undefined : defaultValue(def)])
) as Fields;
if (fields) {
@@ -295,6 +260,12 @@ export class Config<
this.update(migrated, false, location);
}
for (const [name, def] of defs) {
if (this.fields[name as keyof Fields] === undefined) {
this.fields[name as keyof Fields] = (def.initialValue as (config: Config<Fields>) => any)(this);
}
}
for (const [pluginName, version] of this.constructor.pluginVersions) {
const name = `${pluginName}.version` as keyof Fields & string;
if (name in this.fields && this.fields[name] !== version) {
@@ -368,7 +339,7 @@ export class Config<
* @param name - Name of the field being validated.
* @param def - Definition describing the field type and constraints.
* @param value - Value to validate and coerce.
* @throws {InvalidValue} if the value cannot be coerced to the correct type
* @throws {InvalidValueError} if the value cannot be coerced to the correct type
* @returns The validated and coerced value.
* @private
*/
@@ -380,7 +351,7 @@ export class Config<
// If def is an enum then it must be a value within it
if (def.enum && !def.enum.includes(value)) {
throw new InvalidValue(`Expected one of [${def.enum.join(", ")}], not ${value}`);
throw new InvalidValueError(`Expected one of [${def.enum.join(", ")}], not ${value}`);
}
// If it is a string, then attempt to coerce it
@@ -402,7 +373,7 @@ export class Config<
try {
value = JSON.parse(value);
} catch (err: any) {
throw new InvalidValue(`Error parsing value for ${name}: ${err.message}`);
throw new InvalidValueError(`Error parsing value for ${name}: ${err.message}`);
}
}
}
@@ -410,12 +381,12 @@ export class Config<
// Check if it is allowed to be null
if (value === null) {
if (!def.optional) {
throw new InvalidValue(`Field ${name} cannot be null`);
throw new InvalidValueError(`Field ${name} cannot be null`);
}
// Check if is the correct type
} else if (basicType(value) !== def.type) {
throw new InvalidValue(`Expected type of ${name} to be ${def.type}, not ${basicType(value)}`);
throw new InvalidValueError(`Expected type of ${name} to be ${def.type}, not ${basicType(value)}`);
}
return value as FieldType<T["type"]>;
@@ -429,7 +400,7 @@ export class Config<
* @param location - location to check access for.
* @param mode - type of access to check for.
* @param error - throw on failure to pass check.
* @throws {InvalidAccess}
* @throws {InvalidAccessError}
* if access is not granted and error is true.
* @returns true if access is granted
* @private
@@ -448,7 +419,7 @@ export class Config<
)
) {
if (error) {
throw new InvalidAccess(`Field '${name}' is not accessible from ${location}`);
throw new InvalidAccessError(`Field '${name}' is not accessible from ${location}`);
} else {
return false;
}
@@ -506,7 +477,7 @@ export class Config<
if (!this.filepath) {
throw new Error("Cannot save config which has no filepath");
}
await safeOutputFile(this.filepath, JSON.stringify(this, null, "\t"), { mode: 0o600 });
await atomicOverwriteFile(this.filepath, JSON.stringify(this, null, "\t"), { mode: 0o600 });
this._dirty = false;
}
@@ -648,7 +619,7 @@ export class Config<
}
let def = this.constructor.fieldDefinitions[name];
if (!def) {
throw new InvalidField(`No field named '${name}'`);
throw new InvalidFieldError(`No field named '${name}'`);
}
return this._checkAccess(name, def, location, mode, false);
}
@@ -663,7 +634,7 @@ export class Config<
get<Field extends keyof Fields & string>(name: Field, remote = this.location) {
let def = this.constructor.fieldDefinitions[name];
if (!def) {
throw new InvalidField(`No field named '${name}'`);
throw new InvalidFieldError(`No field named '${name}'`);
}
// If remote access is requested require it to be able to read the field
if (remote !== this.location) {
@@ -675,7 +646,7 @@ export class Config<
!this._checkAccess(name, def, this.location, ConfigAccess.write, false)
&& !this._checkAccess(name, def, this.location, ConfigAccess.read, false)
) {
throw new InvalidAccess(`Field '${name}' is not accessible from ${this.location}`);
throw new InvalidAccessError(`Field '${name}' is not accessible from ${this.location}`);
}
return this.fields[name];
@@ -690,13 +661,13 @@ export class Config<
* @param name - Name of field to set.
* @param newValue - Value to set for field.
* @param remote - Location this field is being set from.
* @throws {InvalidField} if field is not defined.
* @throws {InvalidValue} if value is not allowed for the field.
* @throws {InvalidFieldError} if field is not defined.
* @throws {InvalidValueError} if value is not allowed for the field.
*/
set<Field extends keyof Fields & string>(name: Field, newValue: Fields[Field], remote = this.location) {
const def = this.constructor.fieldDefinitions[name];
if (!def) {
throw new InvalidField(`No field named '${name}'`);
throw new InvalidFieldError(`No field named '${name}'`);
}
this._checkAccess(name, def, remote, ConfigAccess.write, true);
@@ -714,7 +685,7 @@ export class Config<
try {
def.validator(value, this);
} catch (err: any) {
throw new InvalidValue(`Failed validation of ${name}: ${err.message}`);
throw new InvalidValueError(`Failed validation of ${name}: ${err.message}`);
}
}
} finally {
@@ -732,7 +703,7 @@ export class Config<
try {
dependent.def.validator(this.get(dependent.name as any, remote), this.staging);
} catch (err: any) {
throw new InvalidValue(`Failed validation of dependent ${dependent.name}: ${err.message}`);
throw new InvalidValueError(`Failed validation of dependent ${dependent.name}: ${err.message}`);
}
}
} finally {
@@ -753,8 +724,8 @@ export class Config<
* @param prop - Name of property to set on field.
* @param value - the value to set the property to.
* @param remote - Location this property is being set from
* @throws {InvalidField} if field is not defined.
* @throws {InvalidValue} if field is not an object.
* @throws {InvalidFieldError} if field is not defined.
* @throws {InvalidValueError} if field is not an object.
*/
setProp<Field extends keyof Fields & string>(
name: Field,
@@ -764,7 +735,7 @@ export class Config<
) {
let def = this.constructor.fieldDefinitions[name];
if (!def) {
throw new InvalidField(`No field named '${name}'`);
throw new InvalidFieldError(`No field named '${name}'`);
}
this._checkAccess(name, def, remote, ConfigAccess.write, true);
@@ -773,7 +744,7 @@ export class Config<
}
if (def.type !== "object") {
throw new InvalidValue(`Cannot set property on non-object field '${name}'`);
throw new InvalidValueError(`Cannot set property on non-object field '${name}'`);
}
let prev = this.fields[name] as Record<string, unknown>;
@@ -793,7 +764,7 @@ export class Config<
try {
def.validator(updated, this);
} catch (err: any) {
throw new InvalidValue(`Failed validation of ${name}: ${err.message}`);
throw new InvalidValueError(`Failed validation of ${name}: ${err.message}`);
}
}
} finally {
@@ -811,7 +782,7 @@ export class Config<
try {
dependent.def.validator(this.get(dependent.name as any, remote), this.staging);
} catch (err: any) {
throw new InvalidValue(`Failed validation of dependent ${dependent.name}: ${err.message}`);
throw new InvalidValueError(`Failed validation of dependent ${dependent.name}: ${err.message}`);
}
}
} finally {
@@ -829,8 +800,8 @@ export class Config<
* @param name - Name of field to set.
* @param newValue - Value to set for field.
* @param remote - Location this field is being set from.
* @throws {InvalidField} if field is not defined.
* @throws {InvalidValue} if value is not allowed for the field.
* @throws {InvalidFieldError} if field is not defined.
* @throws {InvalidValueError} if value is not allowed for the field.
*/
stage<Field extends keyof Fields & string>(name: Field, newValue: Fields[Field], remote = this.location) {
if (this._readonly) {
@@ -839,7 +810,7 @@ export class Config<
const def = this.constructor.fieldDefinitions[name];
if (!def) {
throw new InvalidField(`No field named '${name}'`);
throw new InvalidFieldError(`No field named '${name}'`);
}
this._checkAccess(name, def, remote, ConfigAccess.write, true);
@@ -859,8 +830,8 @@ export class Config<
* @param prop - Name of property to set on field.
* @param value - the value to set the property to.
* @param remote - Location this property is being set from
* @throws {InvalidField} if field is not defined.
* @throws {InvalidValue} if field is not an object.
* @throws {InvalidFieldError} if field is not defined.
* @throws {InvalidValueError} if field is not an object.
*/
stageProp<Field extends keyof Fields & string>(
name: Field,
@@ -874,7 +845,7 @@ export class Config<
const def = this.constructor.fieldDefinitions[name];
if (!def) {
throw new InvalidField(`No field named '${name}'`);
throw new InvalidFieldError(`No field named '${name}'`);
}
this._checkAccess(name, def, remote, ConfigAccess.write, true);
@@ -883,7 +854,7 @@ export class Config<
}
if (def.type !== "object") {
throw new InvalidValue(`Cannot set property on non-object field '${name}'`);
throw new InvalidValueError(`Cannot set property on non-object field '${name}'`);
}
const prev = this.staging.get(name) as Record<string, unknown>;
@@ -901,7 +872,7 @@ export class Config<
/**
* Commit all changes in stating after they have been validated.
*
* @throws {InvalidValue} if a value is not allowed for its field.
* @throws {InvalidValueError} if a value is not allowed for its field.
*/
commitStaging() {
if (this._readonly) {
@@ -923,7 +894,7 @@ export class Config<
try {
fieldDef.validator(config.fields[fieldName as keyof Fields], config);
} catch (err: any) {
throw new InvalidValue(`Failed validation of ${fieldName}: ${err.message}`);
throw new InvalidValueError(`Failed validation of ${fieldName}: ${err.message}`);
}
}
}
+101 -28
View File
@@ -1,7 +1,7 @@
// Core definitions for the configuration system
import * as classes from "./classes.js";
import * as validators from "./validators.js";
import type { PluginNodeEnvInfo, PluginWebpackEnvInfo } from "../plugin.js";
import type { PluginNodeEnvInfo, PluginWebEnvInfo } from "../plugin.js";
import { Static } from "@sinclair/typebox";
import { isTargetVersion } from "../data/version.js";
import { SyncHook } from "../SyncHook.js";
@@ -37,6 +37,8 @@ export interface ControllerConfigFields {
"controller.default_mod_pack_id": number | null;
"controller.default_role_id": number | null;
"controller.autosave_interval": number;
"controller.host_pairing_timeout": number;
"controller.host_pairing_max_pending": number;
"controller.mod_portal_cache_duration_minutes": number;
"controller.mod_portal_page_size": number;
"controller.allow_remote_updates": boolean;
@@ -98,6 +100,20 @@ export class ControllerConfig extends classes.Config<ControllerConfigFields> {
initialValue: 60,
validator: validators.greaterThanZero,
},
"controller.host_pairing_timeout": {
title: "Host Pairing Timeout",
description: "Time in seconds a host can wait for approval before it has to request again, 0 for no limit.",
type: "number",
initialValue: 3600,
validator: validators.greaterThanEqualZero,
},
"controller.host_pairing_max_pending": {
title: "Host Pairing Max Pending",
description: "Maximum number of hosts that can wait for approval at the same time, 0 for no limit.",
type: "number",
initialValue: 50,
validator: validators.all(validators.integer, validators.greaterThanEqualZero),
},
"controller.http_port": {
title: "HTTP Port",
description: "Port to listen for HTTP connections on, set to null to not listen for HTTP connections.",
@@ -151,7 +167,7 @@ export class ControllerConfig extends classes.Config<ControllerConfigFields> {
optional: true,
validator: (value) => {
if (value) {
// This is unfortunately a limitiation of how the webpack build is set up right now.
// This is unfortunately a limitiation of how the web build is set up right now.
const pathname = new URL(value, "http://localhost/").pathname;
if (pathname !== "/static/") {
throw new Error(`Path component of static_url must be /static/ not ${pathname}.`);
@@ -472,6 +488,9 @@ export interface InstanceConfigFields {
"instance.assigned_host": number | null;
"instance.auto_start": boolean;
"instance.exclude_from_start_all": boolean;
"instance.is_template": boolean;
"instance.template_id": number | null;
"instance.inherited_fields": Record<string, true | string[]>;
[key: `${string}.load_plugin`]: boolean;
[key: `${string}.version`]: string;
@@ -523,6 +542,38 @@ export class InstanceConfig extends classes.Config<InstanceConfigFields> {
return config;
}
/** Fields that are never inherited from a template, ports clash on a shared host and secrets stay per instance */
static uninheritableFields = [
"instance.name", "instance.is_template", "instance.template_id",
"factorio.game_port", "factorio.rcon_port", "factorio.rcon_password", "factorio.lua_udp_port",
];
/** True if the field can take its value from a template */
static isInheritable(name: string) {
const def = this.fieldDefinitions[name as keyof InstanceConfigFields];
return Boolean(def)
&& !def.hidden
&& (def.access ?? this.defaultAccess).includes("controller")
&& !this.uninheritableFields.includes(name);
}
/**
* True if the field, or the given property of an object field, takes its value from the template
*
* @param name - Name of field to check.
* @param prop - Property of an object field to check.
*/
isInherited(name: string, prop?: string) {
if (this.get("instance.template_id") === null) {
return false;
}
const entry = this.get("instance.inherited_fields")[name];
if (entry instanceof Array) {
return prop === undefined || !entry.includes(prop);
}
return Boolean(entry);
}
static fieldDefinitions: classes.ConfigDefs<InstanceConfigFields> = {
"instance.name": {
type: "string",
@@ -550,6 +601,33 @@ export class InstanceConfig extends classes.Config<InstanceConfigFields> {
type: "boolean",
initialValue: false,
},
"instance.is_template": {
description:
"Mark this instance as a template other instances can inherit config from. " +
"Templates cannot be assigned to a host or started.",
access: ["controller", "control"],
type: "boolean",
initialValue: false,
hidden: true,
},
"instance.template_id": {
title: "Template",
description: "ID of the template to inherit config values from",
access: ["controller", "control"],
inputComponent: "instance_template",
type: "number",
optional: true,
// Checked by the controller
},
"instance.inherited_fields": {
description:
"Fields whose value comes from the template, for object fields the properties set on this instance",
access: ["controller", "control"],
readonly: ["controller"],
type: "object",
initialValue: {},
hidden: true,
},
"factorio.version": {
description:
"Version of the game to run. Use 'latest' for the latest installed version, a release " +
@@ -786,7 +864,7 @@ function validateFieldNames(
* Add the config fields defined by the by a specific plugin infos
*/
export function addPluginFieldDefinitions(
pluginInfo: PluginNodeEnvInfo | PluginWebpackEnvInfo,
pluginInfo: PluginNodeEnvInfo | PluginWebEnvInfo,
kind: "controllerConfigFields" | "hostConfigFields" | "instanceConfigFields",
Config: typeof classes.Config<any>
) {
@@ -800,36 +878,31 @@ export function addPluginFieldDefinitions(
// The validator is dynamic based on the feature requirements of the plugin
if (kind === "instanceConfigFields" && pluginInfo.features) {
fieldDef.dependsOn = [];
const savePatching = pluginInfo.features.includes("SavePatching");
if (savePatching) {
fieldDef.dependsOn.push("factorio.enable_save_patching");
const requirements: [string, string][] = [];
if (pluginInfo.features.includes("SavePatching")) {
requirements.push(["factorio.enable_save_patching", "save patching"]);
}
if (pluginInfo.features.includes("ScriptCommands")) {
requirements.push(["factorio.enable_script_commands", "script commands"]);
}
if (pluginInfo.features.includes("LuaUdp")) {
requirements.push(["factorio.enable_lua_udp", "Lua UDP"]);
}
const scriptCommands = pluginInfo.features.includes("ScriptCommands");
if (scriptCommands) {
fieldDef.dependsOn.push("factorio.enable_script_commands");
}
const luaUdp = pluginInfo.features.includes("LuaUdp");
if (luaUdp) {
fieldDef.dependsOn.push("factorio.enable_lua_udp");
}
if (scriptCommands || savePatching || luaUdp) {
if (requirements.length) {
fieldDef.dependsOn = requirements.map(([field]) => field);
// Default to not loaded when the instance does not meet the requirements
fieldDef.initialValue = (config: classes.Config<any>) => requirements.every(
([field]) => config.get(field)
);
fieldDef.validator = (enabled, config) => {
if (!enabled) {
return;
}
if (savePatching && !config.get("factorio.enable_save_patching")) {
throw new Error(`Plugin ${pluginInfo.name} requires save patching`);
}
if (scriptCommands && !config.get("factorio.enable_script_commands")) {
throw new Error(`Plugin ${pluginInfo.name} requires script commands`);
}
if (luaUdp && !config.get("factorio.enable_lua_udp")) {
throw new Error(`Plugin ${pluginInfo.name} requires Lua UDP`);
for (const [field, description] of requirements) {
if (!config.get(field)) {
throw new Error(`Plugin ${pluginInfo.name} requires ${description}`);
}
}
};
}
@@ -858,7 +931,7 @@ export function addPluginFieldDefinitions(
*
* @param {Array<Object>} pluginInfos - Array of plugin info objects.
*/
export function addPluginConfigFields(pluginInfos: PluginNodeEnvInfo[] | PluginWebpackEnvInfo[]) {
export function addPluginConfigFields(pluginInfos: PluginNodeEnvInfo[] | PluginWebEnvInfo[]) {
for (let pluginInfo of pluginInfos) {
addPluginFieldDefinitions(pluginInfo, "controllerConfigFields", ControllerConfig);
if (pluginInfo.hostEntrypoint || pluginInfo.instanceEntrypoint) {
@@ -0,0 +1,73 @@
import { Type, Static } from "@sinclair/typebox";
/**
* A pending request from a host to join the cluster
*
* Requests are kept in memory on the controller only while the host
* waits for them to be approved or rejected.
*/
export default class HostPairingDetails {
constructor(
/** Code shown by the host, used to identify the request. */
public code: string,
/** Name the host asked to have. */
public name: string,
/** Address the request was made from. */
public remoteAddress: string,
/** Version of the host that made the request. */
public version = "",
/** Millisecond Unix timestamp the request was made at. */
public createdAtMs = 0,
/** Millisecond Unix timestamp this entry was last updated at. */
public updatedAtMs = 0,
/** True if this request is no longer pending. */
public isDeleted = false,
) { }
get id() {
return this.code;
}
static jsonSchema = Type.Object({
"code": Type.String(),
"name": Type.String(),
"remoteAddress": Type.String(),
"version": Type.Optional(Type.String()),
"createdAtMs": Type.Optional(Type.Number()),
"updatedAtMs": Type.Optional(Type.Number()),
"isDeleted": Type.Optional(Type.Boolean()),
});
static fromJSON(json: Static<typeof this.jsonSchema>) {
return new this(
json.code,
json.name,
json.remoteAddress,
json.version,
json.createdAtMs,
json.updatedAtMs,
json.isDeleted,
);
}
toJSON() {
const json: Static<typeof HostPairingDetails.jsonSchema> = {
code: this.code,
name: this.name,
remoteAddress: this.remoteAddress,
};
if (this.version !== "") {
json.version = this.version;
}
if (this.createdAtMs !== 0) {
json.createdAtMs = this.createdAtMs;
}
if (this.updatedAtMs !== 0) {
json.updatedAtMs = this.updatedAtMs;
}
if (this.isDeleted) {
json.isDeleted = this.isDeleted;
}
return json;
}
}
+21 -2
View File
@@ -41,7 +41,11 @@ export default class InstanceDetails {
public startedAtMs = 0,
/** Millisecond Unix timestamp this entry was last updated at */
public updatedAtMs = 0,
public excludeFromStartAll = false
public excludeFromStartAll = false,
public isTemplate = false,
public templateId?: number,
/** Value of factorio.mod_pack_id, undefined means the controller default is used */
public modPackId?: number,
) {}
get isDeleted() {
@@ -61,6 +65,9 @@ export default class InstanceDetails {
"startedAtMs": Type.Optional(Type.Number()),
"updatedAtMs": Type.Optional(Type.Number()),
"excludeFromStartAll": Type.Optional(Type.Boolean()),
"isTemplate": Type.Optional(Type.Boolean()),
"templateId": Type.Optional(Type.Integer()),
"modPackId": Type.Optional(Type.Integer()),
});
static fromJSON(json: Static<typeof this.jsonSchema>) {
@@ -73,7 +80,10 @@ export default class InstanceDetails {
json.factorioVersion,
json.startedAtMs,
json.updatedAtMs,
json.excludeFromStartAll
json.excludeFromStartAll,
json.isTemplate,
json.templateId,
json.modPackId,
);
}
@@ -102,6 +112,15 @@ export default class InstanceDetails {
if (this.excludeFromStartAll !== false) {
json.excludeFromStartAll = this.excludeFromStartAll;
}
if (this.isTemplate !== false) {
json.isTemplate = this.isTemplate;
}
if (this.templateId !== undefined) {
json.templateId = this.templateId;
}
if (this.modPackId !== undefined) {
json.modPackId = this.modPackId;
}
return json;
}
+1
View File
@@ -15,6 +15,7 @@ export { default as Role } from "./Role.js";
export { default as Note } from "./Note.js";
export { default as InstanceDetails, InstanceStatus } from "./InstanceDetails.js";
export { default as HostDetails } from "./HostDetails.js";
export { default as HostPairingDetails } from "./HostPairingDetails.js";
export { default as UserDetails } from "./UserDetails.js";
export type { IUser } from "./UserDetails.js";
export * from "./composites.js";
+70 -35
View File
@@ -1,8 +1,10 @@
import { Type, Static } from "@sinclair/typebox";
import { JsonString, StringEnum, StringKey, plainJson } from "./composites.js";
import { JsonNumber, StringEnum, StringKey, jsonArray, plainJson } from "./composites.js";
import { type LogEntry, levels } from "../logging.js";
import { ControllerConfig, HostConfig } from "../config/index.js";
import { ControllerConfig } from "../config/index.js";
import { ExternalFactorioVersionSchema, LatestReleasesSchema } from "../external/index.js";
import { PluginDetails } from "./messages_plugin.js";
import HostPairingDetails from "./HostPairingDetails.js";
const LogEntrySchema = Type.Object({
"level": StringKey(levels),
@@ -119,52 +121,81 @@ export class ControllerConfigSetPropRequest {
}
}
export class HostGenerateTokenRequest {
declare ["constructor"]: typeof HostGenerateTokenRequest;
export class HostPairingListRequest {
declare ["constructor"]: typeof HostPairingListRequest;
static type = "request" as const;
static src = "control" as const;
static dst = "controller" as const;
static permission = "core.host.generate_token" as const;
constructor(
public hostId?: number,
) { }
static jsonSchema = Type.Object({
"hostId": Type.Optional(Type.Integer()),
});
static fromJSON(json: Static<typeof this.jsonSchema>) {
return new this(json.hostId);
}
static Response = JsonString;
static permission = "core.host.approve" as const;
static Response = jsonArray(HostPairingDetails);
}
export class HostConfigCreateRequest {
declare ["constructor"]: typeof HostConfigCreateRequest;
static type = "request" as const;
static src = "control" as const;
static dst = "controller" as const;
static permission = "core.host.create_config" as const;
export class HostPairingUpdatesEvent {
declare ["constructor"]: typeof HostPairingUpdatesEvent;
static type = "event" as const;
static src = "controller" as const;
static dst = "control" as const;
static permission = "core.host.approve" as const;
constructor(
public id: number | undefined,
public name: string | undefined,
public generateToken: boolean,
public updates: HostPairingDetails[],
) { }
static jsonSchema = Type.Object({
"id": Type.Optional(Type.Integer()),
"name": Type.Optional(Type.String()),
"generateToken": Type.Boolean(),
"updates": Type.Array(HostPairingDetails.jsonSchema),
});
static fromJSON(json: Static<typeof this.jsonSchema>) {
return new this(json.id, json.name, json.generateToken);
return new this(json.updates.map(update => HostPairingDetails.fromJSON(update)));
}
}
export class HostPairingApproveRequest {
declare ["constructor"]: typeof HostPairingApproveRequest;
static type = "request" as const;
static src = "control" as const;
static dst = "controller" as const;
static permission = "core.host.approve" as const;
constructor(
/** Code of the pending request, case and dashes are ignored. */
public code: string,
/** Name to give the host instead of the one it asked for. */
public name?: string,
) { }
static jsonSchema = Type.Object({
"code": Type.String(),
"name": Type.Optional(Type.String()),
});
static fromJSON(json: Static<typeof this.jsonSchema>) {
return new this(json.code, json.name);
}
static Response = plainJson(HostConfig.jsonSchema);
/** Id of the host created. */
static Response = JsonNumber;
}
export class HostPairingRejectRequest {
declare ["constructor"]: typeof HostPairingRejectRequest;
static type = "request" as const;
static src = "control" as const;
static dst = "controller" as const;
static permission = "core.host.approve" as const;
constructor(
/** Code of the pending request, case and dashes are ignored. */
public code: string,
) { }
static jsonSchema = Type.Object({
"code": Type.String(),
});
static fromJSON(json: Static<typeof this.jsonSchema>) {
return new this(json.code);
}
}
export class GetFactorioCredentialsRequest {
@@ -298,6 +329,8 @@ export class SystemInfo {
/** Millisecond Unix timestamp this entry was last updated at */
public updatedAtMs: number,
public isDeleted: boolean,
/** Plugins on the machine when last collected, kept while it is offline */
public plugins?: PluginDetails[],
) { }
static jsonSchema = Type.Object({
@@ -318,6 +351,7 @@ export class SystemInfo {
"processStartedAtMs": Type.Number(),
"updatedAtMs": Type.Number(),
"isDeleted": Type.Boolean(),
"plugins": Type.Optional(Type.Array(PluginDetails.jsonSchema)),
});
static fromJSON(json: Static<typeof this.jsonSchema>) {
@@ -339,6 +373,7 @@ export class SystemInfo {
json.processStartedAtMs,
json.updatedAtMs,
json.isDeleted,
json.plugins?.map(plugin => PluginDetails.fromJSON(plugin)),
);
}
@@ -435,7 +470,7 @@ export class DebugWsMessageEvent {
export class FactorioVersionsRequest {
declare ["constructor"]: typeof FactorioVersionsRequest;
static type = "request" as const;
static src = ["control", "instance"] as const;
static src = ["control", "host", "instance"] as const;
static dst = "controller" as const;
static permission = "core.external.get_factorio_versions" as const;
@@ -457,7 +492,7 @@ export class FactorioVersionsRequest {
export class LatestReleasesRequest {
declare ["constructor"]: typeof LatestReleasesRequest;
static type = "request" as const;
static src = ["control", "instance"] as const;
static src = ["control", "host", "instance"] as const;
static dst = "controller" as const;
static permission = "core.external.get_latest_releases" as const;
+3 -3
View File
@@ -259,12 +259,12 @@ export class SyncUserListsEvent {
}
}
export class HostRevokeTokensRequest {
declare["constructor"]: typeof HostRevokeTokensRequest;
export class HostRevokeRequest {
declare ["constructor"]: typeof HostRevokeRequest;
static type = "request" as const;
static src = "control" as const;
static dst = "controller" as const;
static permission = "core.host.revoke_token" as const;
static permission = "core.host.revoke" as const;
constructor(
public hostId: number,
@@ -176,6 +176,30 @@ export class InstanceConfigSetPropRequest {
}
}
export class InstanceConfigInheritFieldRequest {
declare ["constructor"]: typeof InstanceConfigInheritFieldRequest;
static type = "request" as const;
static src = "control" as const;
static dst = "controller" as const;
static permission = "core.instance.update_config" as const;
constructor(
public instanceId: number,
public field: string,
public prop?: string,
) { }
static jsonSchema = Type.Object({
"instanceId": Type.Integer(),
"field": Type.String(),
"prop": Type.Optional(Type.String()),
});
static fromJSON(json: Static<typeof this.jsonSchema>) {
return new this(json.instanceId, json.field, json.prop);
}
}
export class InstanceAssignRequest {
declare ["constructor"]: typeof InstanceAssignRequest;
static type = "request" as const;
-1
View File
@@ -251,7 +251,6 @@ export class ModPortalGetAllRequest {
// Define the Response class inline
static Response = class ModPortalGetAllResponse {
declare ["constructor"]: typeof ModPortalGetAllResponse;
constructor(
public mods: Static<typeof ModPortalDetailsSchema>[],
) { }
+194 -6
View File
@@ -4,41 +4,121 @@ import type { PluginNodeEnvInfo } from "../plugin.js";
import { Address, MessageRequest } from "./messages_core.js";
import type { IUser } from "./UserDetails.js";
export const pluginStateSchema = Type.Union([
Type.Literal("loaded"),
Type.Literal("disabled"),
Type.Literal("unloadOnRestart"),
Type.Literal("loadOnRestart"),
Type.Literal("noEntrypoint"),
Type.Literal("pendingInstall"),
Type.Literal("removed"),
Type.Literal("removedLoaded"),
Type.Literal("nameTaken"),
]);
/**
* State of a plugin on a machine
*
* - loaded: Running and loaded again on the next restart
* - disabled: Not loaded as `<plugin>.load_plugin` is false
* - unloadOnRestart: Running, but disabled since startup
* - loadOnRestart: Not running, enabled since startup or skipped by recovery mode
* - noEntrypoint: Has no entrypoint for this machine, so it is never loaded here
* - pendingInstall: Installed since startup, loaded on the next restart
* - removed: Uninstalled since startup while not running
* - removedLoaded: Uninstalled since startup, running until the next restart
* - nameTaken: Not loaded as another package provides a plugin with the same name
*/
export type PluginState = Static<typeof pluginStateSchema>;
/* This is similar to other plugin definitions but with lots removed */
export class PluginDetails {
constructor(
public name: string,
public title: string,
public version: string,
public loaded: boolean,
public enabled: boolean,
public state: PluginState,
public description?: string,
public npmPackage?: string,
/** Version installed on disk, differs from version when an update is waiting for a restart */
public installedVersion?: string,
/** Name in the root package.json dependencies, differs from npmPackage for aliases */
public dependencyName?: string,
) {}
static jsonSchema = Type.Object({
name: Type.String(),
title: Type.String(),
version: Type.String(),
// Missing from 2.0.3 and older, which require loaded and enabled instead
state: Type.Optional(pluginStateSchema),
loaded: Type.Boolean(),
enabled: Type.Boolean(),
description: Type.Optional(Type.String()),
npmPackage: Type.Optional(Type.String()),
installedVersion: Type.Optional(Type.String()),
dependencyName: Type.Optional(Type.String()),
});
static fromJSON(json: Static<typeof this.jsonSchema>) {
return new this(
json.name, json.title, json.version, json.loaded, json.enabled,
json.description, json.npmPackage
json.name, json.title, json.version, json.state ?? this.loadState(json.loaded, json.enabled),
json.description, json.npmPackage, json.installedVersion, json.dependencyName,
);
}
static fromNodeEnvInfo(pluginInfo: PluginNodeEnvInfo, loaded: boolean, enabled: boolean) {
toJSON(): Static<typeof PluginDetails.jsonSchema> {
return {
name: this.name,
title: this.title,
version: this.version,
state: this.state,
loaded: this.loaded,
enabled: this.loadsOnRestart,
description: this.description,
npmPackage: this.npmPackage,
installedVersion: this.installedVersion,
dependencyName: this.dependencyName,
};
}
/** State of a plugin known since startup */
static loadState(loaded: boolean, enabled: boolean): PluginState {
if (loaded) {
return enabled ? "loaded" : "unloadOnRestart";
}
return enabled ? "loadOnRestart" : "disabled";
}
static fromNodeEnvInfo(pluginInfo: PluginNodeEnvInfo, state: PluginState) {
return new this(
pluginInfo.name, pluginInfo.title, pluginInfo.version, loaded, enabled,
pluginInfo.name, pluginInfo.title, pluginInfo.version, state,
pluginInfo.description, pluginInfo.npmPackage,
);
}
/** Plugin package installed since startup, it is loaded on the next restart */
static fromPackageJson(packageName: string, packageJson: any, npmPackage?: string, dependencyName?: string) {
const description = typeof packageJson.description === "string" ? packageJson.description : undefined;
return new this(
packageName, packageName, packageJson.version, "pendingInstall",
description, npmPackage, packageJson.version, dependencyName,
);
}
/** Running on the machine now */
get loaded() {
return ["loaded", "unloadOnRestart", "removedLoaded"].includes(this.state);
}
/** Running on the machine after the next restart */
get loadsOnRestart() {
return ["loaded", "loadOnRestart", "pendingInstall"].includes(this.state);
}
/** Uninstalled since startup */
get removed() {
return this.state === "removed" || this.state === "removedLoaded";
}
}
export class PluginListRequest {
@@ -94,6 +174,114 @@ export class PluginInstallRequest {
}
}
/* Plugin package listed on the npm registry */
export class PluginSearchResult {
constructor(
public name: string,
public version: string,
public description?: string,
public publisher?: string,
public date?: string,
public homepage?: string,
public repository?: string,
public npm?: string,
) {}
static jsonSchema = Type.Object({
name: Type.String(),
version: Type.String(),
description: Type.Optional(Type.String()),
publisher: Type.Optional(Type.String()),
date: Type.Optional(Type.String()),
homepage: Type.Optional(Type.String()),
repository: Type.Optional(Type.String()),
npm: Type.Optional(Type.String()),
});
static fromJSON(json: Static<typeof this.jsonSchema>) {
return new this(
json.name, json.version, json.description, json.publisher,
json.date, json.homepage, json.repository, json.npm,
);
}
}
export class PluginSearchRequest {
declare ["constructor"]: typeof PluginSearchRequest;
static type = "request" as const;
static src = "control" as const;
static dst = "controller" as const;
static permission = "core.plugin.search";
constructor(
public query: string = "",
public page: number = 1,
public pageSize: number = 20,
) {}
static jsonSchema = Type.Object({
query: Type.String(),
page: Type.Integer(),
pageSize: Type.Integer(),
});
static fromJSON(json: Static<typeof this.jsonSchema>) {
return new this(json.query, json.page, json.pageSize);
}
static Response = class Response {
constructor(
public total: number,
public results: PluginSearchResult[],
) {}
static jsonSchema = Type.Object({
total: Type.Integer(),
results: Type.Array(PluginSearchResult.jsonSchema),
});
static fromJSON(json: Static<typeof this.jsonSchema>) {
return new this(json.total, json.results.map(r => PluginSearchResult.fromJSON(r)));
}
};
}
/* Latest version of each package published on npm, used to offer plugin updates */
export class PluginLatestVersionsRequest {
declare ["constructor"]: typeof PluginLatestVersionsRequest;
static type = "request" as const;
static src = "control" as const;
static dst = "controller" as const;
static permission = "core.plugin.list";
constructor(
public packages: string[],
) {}
static jsonSchema = Type.Object({
packages: Type.Array(Type.String()),
});
static fromJSON(json: Static<typeof this.jsonSchema>) {
return new this(json.packages);
}
static Response = class Response {
constructor(
/** Package name to its latest version, packages not found on npm are left out */
public versions: Record<string, string>,
) {}
static jsonSchema = Type.Object({
versions: Type.Record(Type.String(), Type.String()),
});
static fromJSON(json: Static<typeof this.jsonSchema>) {
return new this(json.versions);
}
};
}
export class UpdateAllRequest {
declare ["constructor"]: typeof UpdateAllRequest;
static type = "request" as const;
+3 -3
View File
@@ -2,7 +2,7 @@ import { Type, Static } from "@sinclair/typebox";
import UserDetails, { IUser } from "./UserDetails.js";
import { StringEnum, jsonArray, plainJson } from "./composites.js";
import { MessageRequest } from "./messages_core.js";
import { PermissionError } from "../errors.js";
import { ValidationError } from "../errors.js";
export class UserGetRequest {
declare ["constructor"]: typeof UserGetRequest;
@@ -277,9 +277,9 @@ export class UserBulkImportRequest {
user.checkPermission("core.user.set_whitelisted");
break;
default:
// Unvalidated input during permission check: deny, don't throw a bare Error
// Unvalidated input during permission check: reject, don't throw a bare Error
// @ts-expect-error Unreachable
throw new PermissionError(`Unknown import / restore type: ${data.importType}`);
throw new ValidationError(`Unknown import / restore type: ${data.importType}`);
}
}
}
+2 -2
View File
@@ -79,7 +79,7 @@ export async function loadJsonAsMap(filePath: string): Promise<Map<string, unkno
*/
export async function saveMapAsJson(filePath: string, map: Map<string, unknown>) {
let obj = mapToObject(map);
await libFileOps.safeOutputFile(filePath, JSON.stringify(obj, null, "\t"));
await libFileOps.atomicOverwriteFile(filePath, JSON.stringify(obj, null, "\t"));
}
/**
@@ -139,7 +139,7 @@ export async function loadJsonArrayAsMap(filePath: string): Promise<Map<unknown,
* @throws {Error} if an error occured writing to the file.
*/
export async function saveMapAsJsonArray(filePath: string, map: Map<unknown, { id: unknown }>) {
await libFileOps.safeOutputFile(filePath, JSON.stringify([...map.values()], null, "\t"));
await libFileOps.atomicOverwriteFile(filePath, JSON.stringify([...map.values()], null, "\t"));
}
+33 -15
View File
@@ -2,7 +2,7 @@ import path from "path";
import fs from "node:fs/promises";
import { EventEmitter } from "events";
import { safeOutputFile } from "./file_ops.js";
import { atomicOverwriteFile } from "./file_ops.js";
import { SubscribableValue } from "./subscriptions.js";
import { logger } from "./logging.js";
import { ControllerConfig } from "./config/index.js";
@@ -15,13 +15,8 @@ export abstract class DatastoreProvider<
K extends DatastoreKey,
V extends DatastoreValue,
> {
abstract save(data: Map<K, V>): Promise<void>
abstract load(): Promise<Map<K, V>>
// Load data from the source and return it, this should be passed to the constructor of a datastore
async bootstrap() {
return [this, await this.load()] as const;
}
abstract save(data: Map<K, V>): Promise<void>;
abstract load(): Promise<Map<K, V>>;
}
// Store all data in memory, used as a default and should be avoided
@@ -61,7 +56,7 @@ export class JsonDatastoreProvider<
// Save the data to the json file
async save(data: Map<string, V>) {
logger.verbose(`Saving ${this.filePath}`);
await safeOutputFile(this.filePath, JSON.stringify(Object.fromEntries(data), null, "\t"));
await atomicOverwriteFile(this.filePath, JSON.stringify(Object.fromEntries(data), null, "\t"));
}
// Load the data from the json file
@@ -96,7 +91,7 @@ export class JsonDatastoreProvider<
logger.warn(
`Duplicates detected in ${this.filePath}, a backup was written to: ${backupPath}`
);
await safeOutputFile(backupPath, JSON.stringify(rawJson, null, "\t"));
await atomicOverwriteFile(backupPath, JSON.stringify(rawJson, null, "\t"));
}
return result;
@@ -120,7 +115,7 @@ export class JsonIdDatastoreProvider<
// Save the data to the json file
async save(data: Map<V["id"], V>) {
logger.verbose(`Saving ${this.filePath}`);
await safeOutputFile(this.filePath, JSON.stringify([...data.values()], null, "\t"));
await atomicOverwriteFile(this.filePath, JSON.stringify([...data.values()], null, "\t"));
}
// Load the data from the json file
@@ -155,7 +150,7 @@ export class JsonIdDatastoreProvider<
logger.warn(
`Duplicates detected in ${this.filePath}, a backup was written to: ${backupPath}`
);
await safeOutputFile(backupPath, JSON.stringify(rawJson, null, "\t"));
await atomicOverwriteFile(backupPath, JSON.stringify(rawJson, null, "\t"));
}
return result;
@@ -171,6 +166,7 @@ export abstract class Datastore<
"update": [ updates: U[] ],
}> {
private dirty = false;
private saving: Promise<void> = Promise.resolve();
constructor(
private provider: DatastoreProvider<K, V> = new MemoryDatastoreProvider(),
@@ -186,10 +182,22 @@ export abstract class Datastore<
// Save the datastore to the provider
async save() {
if (this.dirty) {
await this.provider.save(this.data);
// Saves run one at a time so an older snapshot can't be written after a newer one.
const save = this.saving.catch(() => {}).then(async () => {
if (!this.dirty) {
return;
}
// Cleared before writing so changes made while saving are saved by the next save.
this.dirty = false;
}
try {
await this.provider.save(this.data);
} catch (err) {
this.dirty = true;
throw err;
}
});
this.saving = save;
await save;
}
// Load the datastore from the provider
@@ -251,6 +259,11 @@ export class KeyValueDatastore<
K extends DatastoreKey,
V extends DatastoreValue,
> extends Datastore<K, V, [key: K, value: V, isDeleted?: boolean]> {
// Create a datastore populated with the data loaded from the provider
static async fromProvider<K extends DatastoreKey, V extends DatastoreValue>(provider: DatastoreProvider<K, V>) {
return new this(provider, await provider.load());
}
// Set the value in the datastore, be careful of race conditions if you await any functions before calling set
set(key: K, value: V) {
this.data.set(key, value);
@@ -316,6 +329,11 @@ export class SubscribableDatastore<
this.lastUpdatedAtMs = SubscribableDatastore.maxUpdatedAtMs(this.data.values());
}
// Create a datastore populated with the data loaded from the provider
static async fromProvider<V extends SubscribableValue>(provider: DatastoreProvider<V["id"], V>) {
return new this(provider, await provider.load());
}
async load() {
await super.load();
this.lastUpdatedAtMs = SubscribableDatastore.maxUpdatedAtMs(this.data.values());
+67 -25
View File
@@ -18,40 +18,82 @@ export class InstallationError extends Error {
}
/**
* Thrown from requests sent when an error occured handling it
* Base class for errors caused by the request, such as invalid input,
* missing permissions or a missing resource. Corresponds to the HTTP 4xx
* range.
*
* Link does not log these as unexpected when a request handler throws them.
*/
export class RequestError extends Error {
constructor(
message: string,
public code = "RequestError",
public stack?: string,
) {
super(message);
}
}
export class ClientError extends Error { }
/**
* Thrown when a permission check fails.
*
* Is a subclass of RequestError to prevent logging stack traces when
* requests fail due to permission denied.
*
* Thrown when the request contains invalid data (HTTP 400)
*/
export class PermissionError extends RequestError {
constructor(
message: string,
code = "PermissionError",
stack?: string,
) {
super(message, code, stack);
}
export class ValidationError extends ClientError { }
}
/**
* Thrown when a config value cannot be used for its field
*/
export class InvalidValueError extends ValidationError { }
/**
* Thrown when a config field does not exist
*/
export class InvalidFieldError extends ValidationError { }
/**
* Thrown when the request is not allowed (HTTP 403)
*/
export class ForbiddenError extends ClientError { }
/**
* Thrown when a permission check fails
*/
export class PermissionError extends ForbiddenError { }
/**
* Thrown when a config field is not accessible from the current location
*/
export class InvalidAccessError extends ForbiddenError { }
/**
* Thrown when a resource the request refers to does not exist (HTTP 404)
*/
export class NotFoundError extends ClientError { }
/**
* Thrown when an operation for a request timed out (HTTP 408)
*/
export class TimeoutError extends ClientError { }
/**
* Thrown when the request conflicts with the current state, such as an
* instance that is running or a name that is taken (HTTP 409)
*/
export class ConflictError extends ClientError { }
/**
* Base class for errors caused by the side handling the request.
* Corresponds to the HTTP 5xx range.
*/
export class ServerError extends Error { }
/**
* Received in place of an error response whose class is not registered,
* such as a TypeError thrown by a request handler (HTTP 500)
*/
export class InternalServerError extends ServerError { }
/**
* Thrown when the host or instance that should handle the request is not
* connected, not running or shutting down (HTTP 503)
*/
export class UnavailableError extends ServerError { }
/**
* Signal for messages that fail validation
*/
export class InvalidMessage extends Error {
export class InvalidMessage extends ValidationError {
code = "InvalidMessage";
constructor(
message: string,
+6 -6
View File
@@ -35,7 +35,7 @@ export type FactorioModSettingType = "startup" | "runtime-global" | "runtime-per
* Boolean mod setting prototype.
* @see [bool-setting](https://wiki.factorio.com/Tutorial:Mod_settings#bool-setting)
*/
export interface FactorioBoolSettingPrototype extends FactorioPrototypeBase{
export interface FactorioBoolSettingPrototype extends FactorioPrototypeBase {
type: "bool-setting",
setting_type?: FactorioModSettingType,
default_value: boolean,
@@ -46,7 +46,7 @@ export interface FactorioBoolSettingPrototype extends FactorioPrototypeBase{
* Integer mod setting prototype.
* @see [int-setting](https://wiki.factorio.com/Tutorial:Mod_settings#int-setting)
*/
export interface FactorioIntSettingPrototype extends FactorioPrototypeBase{
export interface FactorioIntSettingPrototype extends FactorioPrototypeBase {
type: "int-setting",
setting_type?: FactorioModSettingType,
default_value: number,
@@ -59,7 +59,7 @@ export interface FactorioIntSettingPrototype extends FactorioPrototypeBase{
* Real number mod setting prototype.
* @see [double-setting](https://wiki.factorio.com/Tutorial:Mod_settings#double-setting)
*/
export interface FactorioDoubleSettingPrototype extends FactorioPrototypeBase{
export interface FactorioDoubleSettingPrototype extends FactorioPrototypeBase {
type: "double-setting",
setting_type?: FactorioModSettingType,
default_value: number,
@@ -72,7 +72,7 @@ export interface FactorioDoubleSettingPrototype extends FactorioPrototypeBase{
* String mod setting prototype.
* @see [string-setting](https://wiki.factorio.com/Tutorial:Mod_settings#string-setting)
*/
export interface FactorioStringSettingPrototype extends FactorioPrototypeBase{
export interface FactorioStringSettingPrototype extends FactorioPrototypeBase {
type: "string-setting",
setting_type?: FactorioModSettingType,
default_value: string,
@@ -85,7 +85,7 @@ export interface FactorioStringSettingPrototype extends FactorioPrototypeBase{
* Color mod setting prototype.
* @see [color-setting](https://wiki.factorio.com/Tutorial:Mod_settings#color-setting)
*/
export interface FactorioColorSettingPrototype extends FactorioPrototypeBase{
export interface FactorioColorSettingPrototype extends FactorioPrototypeBase {
type: "color-setting",
setting_type?: FactorioModSettingType,
default_value: FactorioColor,
@@ -104,7 +104,7 @@ export type FactorioModSettingPrototype =
* A list of prototypes as it appears in Factorio's Lua API.
* @see [Data.raw](https://lua-api.factorio.com/stable/types/Data.html#raw)
*/
export type FactorioPrototypes<Prototype = FactorioPrototypeBase> = Record<string, Record<string, Prototype>>
export type FactorioPrototypes<Prototype = FactorioPrototypeBase> = Record<string, Record<string, Prototype>>;
export type FactorioDefinesValue = number | string | { [index: string]: FactorioDefinesValue };
export type FactorioDefines = {
[index: string]: FactorioDefinesValue,
+2 -2
View File
@@ -12,7 +12,7 @@ export const ExternalFactorioVersionSchema = Type.Object({
headlessUrl: Type.String(),
});
export type ExternalFactorioVersion = Static<typeof ExternalFactorioVersionSchema>
export type ExternalFactorioVersion = Static<typeof ExternalFactorioVersionSchema>;
/**
* Fetch all factorio versions that support headless
@@ -47,7 +47,7 @@ function parseFactorioVersions(html: string): ExternalFactorioVersion[] {
const versions = new Map<string, ExternalFactorioVersion>();
// Match <a class=" slot-button-inline version-button-stable " href="/download/archive/2.0.72">2.0.72</a>
// eslint-disable-next-line max-len
// eslint-disable-next-line @stylistic/max-len
const anchorRegex = /<a[^>]*class="[^"]*version-button-(?<kind>stable|experimental)[^"]*"[^>]*href="[^"]*\/download\/archive\/(?<version>\d+\.\d+.\d+)"[^>]*>/g;
let match: RegExpExecArray | null;
+1 -1
View File
@@ -15,7 +15,7 @@ export const LatestReleaseBuildsSchema = Type.Record(Type.String(), FullVersionS
*/
export const LatestReleasesSchema = Type.Record(Type.String(), LatestReleaseBuildsSchema);
export type LatestReleases = Static<typeof LatestReleasesSchema>
export type LatestReleases = Static<typeof LatestReleasesSchema>;
/**
* Fetch the latest stable and experimental factorio releases
Loaded 100 of 290 files, more files were not shown because too many files have changed in this diff. Show more