The "Add mods" card on the mod pack view mounts when shown, so autoFocus
is enough there. The Search Mod Portal modal stays mounted after the
first open, so it focuses the name input from afterOpenChange instead.
Fixes#1194
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Unpatch saves started with save patching disabled
Turning factorio.enable_save_patching off did nothing to a save that
was patched before. Its control.lua still loaded every module, so the
clusterio_api interface, player_sync's join handling and the other
module event handlers kept running with no plugin on the host side.
When patching is disabled the instance now restores the original
control.lua that patching kept as scenario.lua and removes the module
files, locale files included, as listed in clusterio.json. The patch
info stays with an empty module list so the patch number keeps
counting. Otherwise patching the save again would reuse the number
stored in the game and on_server_startup would not be raised. Data
the modules put in storage is left alone.
PatchInfo.fromSave dropped the version read from the save, so the
check against newer patch versions never fired. It now keeps it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Remove scenario.lua when unpatching a save
Unpatching moves scenario.lua back to control.lua instead of copying it, so the save only keeps clusterio.json for the patch number. Patching such a save again copies control.lua back to scenario.lua.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The mod pack page only looked up mod-setting-name.<name>, so settings
with a localised_name showed the internal name, and rich text tags in
the locale showed up as raw text.
Resolve localised_name and localised_description from the exported
settings prototypes against the exported locale, falling back to the
mod-setting-name/mod-setting-description keys and then the internal
name. The resolver handles parameters, the "" and "?" keys and
__ITEM__/__ENTITY__/__TILE__/__FLUID__ references. Rich text
formatting tags are removed and prototype icons are replaced with
their name.
Fixes#768
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Unpatching moves scenario.lua back to control.lua instead of copying it, so the save only keeps clusterio.json for the patch number. Patching such a save again copies control.lua back to scenario.lua.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The instance picker on the users page and the search box on the icon
reference page held their value in component state, so leaving the page
and coming back with the browser history reset them. Both now go through
useTableQueryState like the table filters do, as user.instance and
icon.search.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The instance works out the recipe notification delta against the cleared
list the player had when the download started, but the save applied it
after the download restored player.force. A player whose force changed
kept notifications they had cleared elsewhere, and recipes only the old
force had dropped out of the failed list and were lost on the next
upload.
The download record now keeps the cleared list it sent. The save rebuilds
the stored list from it and the delta, then merges that against the
force the player has once the data is applied, as it did before #997.
Nothing extra goes over RCON.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Add HostManager for host records and host tokens
controller.hosts is now a HostManager wrapping the hosts datastore, the
same way controller.users wraps the users datastore. It owns creating
hosts (moved from HostPairingManager), revoking them (moved from
ControlConnection) and signing and verifying host tokens (moved from
Controller, generateHostToken is renamed signHostToken to match
signUserToken).
UserManager gains verifyUserToken, replacing the copies of the user
token check in WsServer and routes. The WsServer copy accepted a token
without iat when the user had no tokenValidAfter; the shared check
always requires iat, which jsonwebtoken adds by default.
Follows Cooldude2606's comment on #1161.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Return the host record from verifyHostToken
Matches verifyUserToken returning the user. HostConnection._saveInfo is
renamed saveRecord to match User.saveRecord.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Matches verifyUserToken returning the user. HostConnection._saveInfo is
renamed saveRecord to match User.saveRecord.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
controller.hosts is now a HostManager wrapping the hosts datastore, the
same way controller.users wraps the users datastore. It owns creating
hosts (moved from HostPairingManager), revoking them (moved from
ControlConnection) and signing and verifying host tokens (moved from
Controller, generateHostToken is renamed signHostToken to match
signUserToken).
UserManager gains verifyUserToken, replacing the copies of the user
token check in WsServer and routes. The WsServer copy accepted a token
without iat when the user had no tokenValidAfter; the shared check
always requires iat, which jsonwebtoken adds by default.
Follows Cooldude2606's comment on #1161.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Settings stored in a mod pack that no setting prototype in the export
data defines are now listed in a collapsed "Unknown settings" panel
under each scope instead of under an "Unknown Fields" heading. Each one
has a remove link, and a revert link once removed, matching what
`clusterioctl mod-pack edit --remove-setting` already allows. They are
never removed automatically.
Without export data nothing is called unknown and stored settings are
listed as before, without a mod heading.
Fixes#773
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Turning factorio.enable_save_patching off did nothing to a save that
was patched before. Its control.lua still loaded every module, so the
clusterio_api interface, player_sync's join handling and the other
module event handlers kept running with no plugin on the host side.
When patching is disabled the instance now restores the original
control.lua that patching kept as scenario.lua and removes the module
files, locale files included, as listed in clusterio.json. The patch
info stays with an empty module list so the patch number keeps
counting. Otherwise patching the save again would reuse the number
stored in the game and on_server_startup would not be raised. Data
the modules put in storage is left alone.
PatchInfo.fromSave dropped the version read from the save, so the
check against newer patch versions never fired. It now keeps it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
InstanceDetails now carries factorio.mod_pack_id as modPackId, so the
instance list and pages can show it without fetching each config. The
web UI resolves an unset value to the controller default and tags it
"Default". The name links to the mod pack page.
Closes#852
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Keep a syncing spectator where its body stood
A player waiting for player_sync data is a spectator who could walk away,
so restore_position teleported them back at the end. That also undid
teleports by other mods made during the sync, such as universal_edges
placing an arriving player.
The spectator is now pulled back on every tile change, and a jump of more
than 4 tiles or to another surface counts as another mod's teleport and
moves where the body comes back. A failed acquire that takes the body
keeps its place too.
Closes#908
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Rename body place to controller location
Review asked for location and controller over place and body.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Let a syncing spectator look around
Review asked for the spectator to move freely while it waits. Only a jump
of more than 16 tiles from where it was last seen, or to another surface,
counts as a teleport and moves the restore location. The body place names
from #1156 become restore_location, keep_restore_location and
controller_taken.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Review asked for the spectator to move freely while it waits. Only a jump
of more than 16 tiles from where it was last seen, or to another surface,
counts as a teleport and moves the restore location. The body place names
from #1156 become restore_location, keep_restore_location and
controller_taken.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A player waiting for player_sync data is a spectator who could walk away,
so restore_position teleported them back at the end. That also undid
teleports by other mods made during the sync, such as universal_edges
placing an arriving player.
The spectator is now pulled back on every tile change, and a jump of more
than 4 tiles or to another surface counts as another mod's teleport and
moves where the body comes back. A failed acquire that takes the body
keeps its place too.
Closes#908
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Add base instances for instance config inheritance
Instances can be marked as a base with instance.is_base and other
instances inherit config values from it through instance.base_id.
The controller copies base values into inheriting configs and keeps
track of inherited fields in the hidden instance.inherited_fields, so
hosts receive resolved configs and need no changes. Setting a field
stops inheriting it, `instance config inherit` undoes that.
Bases cannot be assigned to a host, are hidden from `instance list`
unless --bases is given, and cannot be deleted or unmarked while
other instances inherit from them. Chains of bases are allowed and
cycles are rejected.
Part of #200
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Rename bases to templates and add per property inheritance
Review changes for #1142:
- base becomes template everywhere (is_template, template_id,
--template, --is-template, --templates, InstanceDetails.isTemplate).
- Object fields inherit per property. inherited_fields lists the
overridden properties for object fields and the template object is
merged with them.
- Linking a template also inherits fields that hold the default value.
The server name stays per instance.
- Ports are never inherited.
- InstanceDetails carries templateId.
Web UI: inherited values are locked until unlocked, overridden values
can be inherited again, templates are listed separately, the create
modal can create templates and instances from templates, and the
instance page has "Create template" which can link the source.
Part of #200
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Give templates their own table and keep rcon_password per instance
The templates table shows name, inheritor count and a create instance
action instead of the instance status and start/stop columns.
factorio.rcon_password joins the ports as never inherited.
Part of #200
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Use link and disconnect icons for template inheritance
Overridden values show a disconnect icon. Clicking it stages a link,
shown with a link icon, which is applied together with the other
changes instead of right away.
Part of #200
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Show the template value while a link is staged
Part of #200
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Address review of template inheritance
- Unlinking a field enables Apply on its own and sends the current value.
- The link button no longer toggles the checkbox of its label.
- Link and disconnect icons for all four states, pending ones highlighted.
- instance.is_template is hidden.
- ctl create takes --as-template.
- Clearer names for the InstanceManager inheritance helpers.
Part of #200
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
- Unlinking a field enables Apply on its own and sends the current value.
- The link button no longer toggles the checkbox of its label.
- Link and disconnect icons for all four states, pending ones highlighted.
- instance.is_template is hidden.
- ctl create takes --as-template.
- Clearer names for the InstanceManager inheritance helpers.
Part of #200
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Overridden values show a disconnect icon. Clicking it stages a link,
shown with a link icon, which is applied together with the other
changes instead of right away.
Part of #200
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The templates table shows name, inheritor count and a create instance
action instead of the instance status and start/stop columns.
factorio.rcon_password joins the ports as never inherited.
Part of #200
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review changes for #1142:
- base becomes template everywhere (is_template, template_id,
--template, --is-template, --templates, InstanceDetails.isTemplate).
- Object fields inherit per property. inherited_fields lists the
overridden properties for object fields and the template object is
merged with them.
- Linking a template also inherits fields that hold the default value.
The server name stays per instance.
- Ports are never inherited.
- InstanceDetails carries templateId.
Web UI: inherited values are locked until unlocked, overridden values
can be inherited again, templates are listed separately, the create
modal can create templates and instances from templates, and the
instance page has "Create template" which can link the source.
Part of #200
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Instances can be marked as a base with instance.is_base and other
instances inherit config values from it through instance.base_id.
The controller copies base values into inheriting configs and keeps
track of inherited fields in the hidden instance.inherited_fields, so
hosts receive resolved configs and need no changes. Setting a field
stops inheriting it, `instance config inherit` undoes that.
Bases cannot be assigned to a host, are hidden from `instance list`
unless --bases is given, and cannot be deleted or unmarked while
other instances inherit from them. Chains of bases are allowed and
cycles are rejected.
Part of #200
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
"safe" suggested the data being written survives a crash, it only
guarantees the old file is either kept or fully replaced. The new name
says what it does. No alias, plugin API breaks are allowed in minor
versions and the 2.1 migration guide lists the rename.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Cooldude2606 <25043174+Cooldude2606@users.noreply.github.com>
* Report missing base game icons when exporting from headless Factorio
The headless build has no graphics, so export-data found none of the
base game's icon files. Plain icons were skipped silently and layered
icons were drawn blank and counted as exported.
Skip a layered icon when any of its layers is missing, and log one
warning with the number of base game icon files that were not found.
Add host.factorio_graphics_directory to read builtin mod graphics from
a full install when the instance runs the headless build.
Fixes#1068
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Export without the graphics directory when it has no matching install
If host.factorio_graphics_directory had no install matching the major and
minor version of the instance, or did not exist, findVersion threw and the
whole export failed. Without the setting the same instance exports everything
except the base game icons, so it now logs a warning that names the setting
and exports without it.
* Find a full install in host.factorio_directory instead of a setting
Per review, drop host.factorio_graphics_directory. findVersion takes a
headless option that filters installs by build, telling them apart by
data/base/graphics/icons. When the instance runs the headless build the
export looks for a full install of the same major.minor version in the
host's factorio directory. Server startup is unchanged.
If none is found the export still runs, and the reason is included in
the missing icons warning.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Address review on findVersion and the export graphics lookup
findVersion takes headless as a plain optional argument, the helper is
checkInstallType, the warning is checkMissingBuiltinIcons and the full
install lookup is inlined into exportData.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Danielv123 <danielv@danielv.no>
Factorio encodes an empty Lua table as {}, so a player with no cleared
recipes sent {} with the download request. new Set({}) threw, the
instance dropped the recipe notifications from the download, and the
next upload replaced the stored list on the controller. An empty stored
list was dropped the same way. Decode both lists treating {} as empty.
The mod pack view gave the settings form its values through the Form's
initialValues, which rc-field-form only applies when the form is
created. Settings added by a data export while the page was open were
shown empty, and color settings black, until the page was reloaded.
Each setting field now gets its saved value as its own initialValue,
which is applied when the field is created and when the form is reset.
findVersion takes headless as a plain optional argument, the helper is
checkInstallType, the warning is checkMissingBuiltinIcons and the full
install lookup is inlined into exportData.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Per review, drop host.factorio_graphics_directory. findVersion takes a
headless option that filters installs by build, telling them apart by
data/base/graphics/icons. When the instance runs the headless build the
export looks for a full install of the same major.minor version in the
host's factorio directory. Server startup is unchanged.
If none is found the export still runs, and the reason is included in
the missing icons warning.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
If host.factorio_graphics_directory had no install matching the major and
minor version of the instance, or did not exist, findVersion threw and the
whole export failed. Without the setting the same instance exports everything
except the base game icons, so it now logs a warning that names the setting
and exports without it.
inquirer 8 pulls in 50 packages and 7.4 MB (rxjs, old chalk) for every
user running npm init @clusterio. The standalone select, input, confirm
and checkbox packages from the same project cover all prompts used and
cut the create install from 70 packages / 8.1 MB to 31 / 0.94 MB.
The Windows rl.close workaround now patches readline.Interface instead
of inquirer's Prompt class, since the new prompts create their own
readline interface. Ctrl+C at a prompt exits with code 130 instead of
0, and closed stdin no longer crashes with ERR_USE_AFTER_CLOSE.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every view page handled a resource that can't be shown its own way, and
users without the subscribe permission were left on a spinner forever.
Adds useResourceState and a ResourceState component, used by the host,
instance, mod pack, icon reference, user, role and plugin pages and
exported for plugin pages. The single item hooks now also return the
subscription error.
Resolves#1165
An icon that could not be read or decoded, such as a truncated PNG, a
JPEG or a layered icon with no layers, threw out of exportIcons and
failed the whole data export. Log a warning naming the prototype and
export it without an icon instead. Also warn when a mod's zip is
missing while exporting the locale, which was skipped silently.
With --dev-plugin and no --dev the page kept the production import map,
while the development server gave the plugin its own copies of the
shared modules. The plugin then failed to load with "Attempt to import
duplicate copy of @clusterio/lib". Use the development server's web
interface and import map whenever it runs, so the plugin and the web
interface share the same modules.
Revoking a host now unassigns its instances instead of being refused
while it has any, and removes its system metrics.
Approving and revoking no longer force a save. Hosts are saved by
autosave and on exit like everything else, which also removes the
approving state from pairing requests.
The approval code and link are logged as warnings so they show up in
the systemd service, which only prints warnings and above. The pairing
limits are now the controller.host_pairing_timeout and
controller.host_pairing_max_pending config values, 0 removes the limit.
Subscribers take an optional permitted function, used for the pairing
requests instead of overrides in a subclass.
A standalone install no longer adds its host to the controller database
by signing a token and writing hosts.json itself. The host asks to join
when it first starts and is approved once with the link it prints.
The list of waiting hosts opens the approval dialog instead of having its
own name input and buttons, so hosts are approved in one place. Removes
the limit on requests per minute, the limits on pending requests already
cap what an address can do.
Also updates text that still said the installer makes the requests.
- Saves of a datastore run one at a time, and changes made while saving
are saved by the next save. Overlapping saves from approving and
revoking hosts could write an older snapshot after a newer one.
- Map subscribers fetch everything again when a new session starts, as
the controller only sends values that exist. A host revoked while a
browser was disconnected stayed in its list.
- The host token check accepts the same pre-rename tokens as the
WebSocket, so upgrading a host with such a token does not make it ask
to join again.
* Answer the disconnect prepare in the close old connections test
The test's first connector had no Link, so the controller's disconnect
prepare went unanswered and it waited the full 10 s timeout.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Start the controller and host CLIs faster
Like ctl, the controller and host now compile schema validators on first
use instead of at import, which cost about 120 ms per invocation. The
Controller and Host classes are only imported by the commands that
start them, so config, bootstrap and the other commands no longer load
express, ws and the rest of the servers. clusteriocontroller config set
goes from 413 ms to 231 ms and clusteriohost config set from 412 ms to
225 ms.
The integration tests run the bins, like npx does, instead of the
package directories.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Splitting the test jobs doubled them to 14, over GitHub's concurrent
job limit together with the other jobs. Every Factorio version now runs
on the latest Node.js only, and the other Node.js versions only with
the latest Factorio, which is 10 test jobs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The clusterioctl, plugin system and instance integration tests take
about half of the test time, so they run on one runner and the rest of
the tests on another, each half with its own controller and host.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Take FieldData from antd instead of rc-field-form
web_ui depended on rc-field-form 1.x for a single type import, while
antd 5 uses rc-field-form 2.x. Derive the type from antd's FormProps
and drop the dependency.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Move typescript to web_ui devDependencies
web_ui only needs typescript to build its own types. The runtime
dependency is a leftover from webpack's ts-loader, Vite doesn't use it,
and it pulls in a 30 MB native compiler for anyone installing web_ui.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Use native FormData in controller route tests
Node's FormData and Blob do what the form-data package did here, and
fetch sets the multipart Content-Type header itself.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Remove leftover typedoc.json files
TypeDoc was dropped in #1147, but these configs still ship in the
published packages.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
jimp pulls in every image format and plugin, about 60 packages and
26 MB that only export.ts uses. Build the Jimp class from @jimp/core
with the PNG format and the crop and resize plugins instead. The
generated spritesheet is byte-identical to the one from jimp.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
A host waiting for approval logs a link to the Hosts page with the code
in it. Opening it shows a dialog with the code, the address of the host,
a name prefilled with the requested one and a large Approve button. A
code that is no longer waiting shows a message saying so.
The installer only writes the controller URL and host name. Running the
host requests to join the cluster if it has not been approved yet, or if
the controller no longer accepts its token, and waits until an admin
approves it. Once approved it saves its id and token and starts as
normal. A rejected host exits with code 8.
clusteriohost run --user-token approves the host right away with the
token of a user allowed to approve hosts. The token is not saved.
This moves the pairing client into the host package, where it uses the
same certificate setup as the host's connection to the controller, and
removes --no-pair and the rerun handling from the installer.
Node's FormData and Blob do what the form-data package did here, and
fetch sets the multipart Content-Type header itself.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
web_ui only needs typescript to build its own types. The runtime
dependency is a leftover from webpack's ts-loader, Vite doesn't use it,
and it pulls in a 30 MB native compiler for anyone installing web_ui.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
web_ui depended on rc-field-form 1.x for a single type import, while
antd 5 uses rc-field-form 2.x. Derive the type from antd's FormProps
and drop the dependency.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Like ctl, the controller and host now compile schema validators on first
use instead of at import, which cost about 120 ms per invocation. The
Controller and Host classes are only imported by the commands that
start them, so config, bootstrap and the other commands no longer load
express, ws and the rest of the servers. clusteriocontroller config set
goes from 413 ms to 231 ms and clusteriohost config set from 412 ms to
225 ms.
The integration tests run the bins, like npx does, instead of the
package directories.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The test's first connector had no Link, so the controller's disconnect
prepare went unanswered and it waited the full 10 s timeout.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drops tests that restate message declarations, check that removed
commands are gone, test internal helpers or repeat the same behaviour
at another layer.
Hosts join the cluster by running the installer, which shows a code and
waits until the request is approved under Hosts > Add host in the web
interface or with clusterioctl host approve. An admin can pass their
user token to the installer to approve the host right away. The token
is only used for that and is not saved.
The controller only accepts hosts that have a record, and only
approving a host creates one. Revoking a host deletes its record, so
its token can no longer be used to reconnect. Hosts that have connected
before keep working.
Removes host generate-token, revoke-token and create-config from
clusterioctl, bootstrap generate-host-token, the host token messages and
the core.host.generate_token, revoke_token and create_config
permissions, which are also stripped from stored roles. Adds
clusterioctl host pending, approve, reject and revoke, and the
core.host.approve and core.host.revoke permissions.
Standalone installs add the host to the controller database directly,
since the controller is not running during the install.
pnpm/action-setup installs a bootstrap pnpm and then switches to the
packageManager version, about 10 s per job; npm installs it in 3 s.
setup-node took 4 s probing package managers for a cache it never
used. The pnpm store cache took 4 s to restore and 5 s to save, more
than downloading the packages costs, and the build cache no longer pays
off with TypeScript 7: pnpm install is 13-23 s without either cache
against 17-31 s with them.
The Codecov action searched the workspace and uploaded the raw V8
coverage in coverage/tmp next to lcov.info, 72 MB taking 11 s.
The version compatibility job ran tsc --build again after pnpm install
had already built everything.
Cancel superseded runs of branches and pull requests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
loaded, enabled, pendingInstall, removed, noEntrypoint and nameTaken
allowed far more combinations than can happen, and every consumer had
to know which took priority. PluginDetails now has one state, with
loaded, loadsOnRestart and removed as getters over it.
2.0.3 web UIs and ctl send PluginListRequest straight to hosts and
require loaded and enabled, so those are still sent next to the
optional state. Plugins listed by 2.0.3 hosts get their state from
loaded and enabled.
The web UI planner gets the same treatment: MachinePlugin has one
state in place of disabled, noEntrypoint, dependencyTaken and
nameTaken. ctl host plugin list shows the state.
Closes#1134
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
While the host has no controller session, events between the instance
and the controller are dropped in both directions, and research_sync
only synced technologies on instance start. A technology finished during
the outage stayed unresearched on the controller and on every other
instance until the instance restarted.
On a new session the instance now runs the same technology sync as on
start, before taking the cluster queue. Contributions made while the
controller is unreachable are summed per technology and sent on connect
or resume, since progress from several instances can't be rebuilt from a
state sync that keeps the highest progress. A finish made while the
connection was dropped triggers the sync on resume too.
sync_technologies now runs while the game is running, so it skips
writing progress to a technology that finished after the dump.
Fixes#1123
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The headless build has no graphics, so export-data found none of the
base game's icon files. Plain icons were skipped silently and layered
icons were drawn blank and counted as exported.
Skip a layered icon when any of its layers is missing, and log one
warning with the number of base game icon files that were not found.
Add host.factorio_graphics_directory to read builtin mod graphics from
a full install when the instance runs the headless build.
Fixes#1068
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The user page set the role selector once and never updated it. The role
page reset its whole form on every role update, wiping unsaved edits.
Both pages now follow the live data and keep local edits on top. The role
page stores only the edited fields and drops them once they match the role
or when the viewer loses core.role.update. The user page syncs the
selector while it is not dirty and drops deleted roles from a dirty
selection. A role deleted while open now says so instead of "not found".
Fixes#775, fixes#776
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The button offered Stop while the instance was starting whenever the
start came from somewhere else, such as Load save, another tab or ctl.
It now shows the action in progress as loading until the status
settles. Kill stays available from the dropdown.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A stop request sent while the instance was starting ran alongside the
start. During save preparation it did nothing and the instance came up
running anyway. Once Factorio was launched it put the server into the
stopping state, the start then failed with "Expected state running but
state is stopping" and its cleanup killed the server mid shutdown.
Stop and restart requests now wait for any start, stop or save in
progress to finish first, then stop the instance if it is running.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Since TypeScript 6 the types option defaults to an empty list, so
@types/node was only loaded because winston and undici-types reference
it. Name it explicitly. The emitted files do not change.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TypeScript 7.1 reports TS5088 for it: the inferred type of the static
Response property references itself through the declared constructor.
The other inline Response classes do not declare it either.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TypeScript 7 is the native port of the compiler. The repository builds
with it unchanged: no new type errors, the emitted JavaScript differs
only in the order of react/jsx-runtime imports, and the declarations
only in union and property order, which TypeScript 5.9 consumers read
the same way. The plugin template moves to it too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The generated API docs were never published or built in CI, and
typedoc's peer range stops at TypeScript 6.0. TSDoc comments stay.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Entries now explain why end users are not or only minimally affected,
and the docs say which version to key by and point to the breaking
changes decision.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The version-compat job fails every pull request once master breaks
compatibility with the latest release on purpose. List those breaks in
scripts/version-compat-breaks.json keyed by the published version they
break against, each with a pr number and reason. Listed messages warn
instead of failing, connect skips the cluster test, entries for other
versions are ignored, and a listed message that did not change fails.
Add the two src changes from #1129 against 2.0.3 so master passes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Keep auto paused instances paused
Every RCON command makes an auto paused Factorio server run one tick, so
subspace_storage's 1 s inventory update and 5 s ping, plus a
statistics_exporter gather on every Prometheus scrape, kept a server with
nobody online running at about 1.3 ticks per second.
The instance now records the auto_pause server setting it was started
with and exposes autoPaused, true while that setting is on and no players
are online, along with an autoPauseChanged hook. subspace_storage skips
its timers while paused and sends a ping and inventory as soon as a player
joins. statistics_exporter gathers once after the game pauses and keeps
serving those values until it resumes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Check connection and status in subspace_storage timers
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Every RCON command makes an auto paused Factorio server run one tick, so
subspace_storage's 1 s inventory update and 5 s ping, plus a
statistics_exporter gather on every Prometheus scrape, kept a server with
nobody online running at about 1.3 ticks per second.
The instance now records the auto_pause server setting it was started
with and exposes autoPaused, true while that setting is on and no players
are online, along with an autoPauseChanged hook. subspace_storage skips
its timers while paused and sends a ping and inventory as soon as a player
joins. statistics_exporter gathers once after the game pauses and keeps
serving those values until it resumes.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Users don't need the web UI load state apart from the plugin's own state now
that the web UI reloads itself. A web load error now shows under the plugin's
versions and on its view page, only for enabled plugins.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds the last three checks from #342 to test_plugin: an InstanceEcho
request handled on a running instance, a Lua module with a remote
interface that the request calls over RCON, and log tagging checks for
the plugin field and host/instance ids on the controller, host and
instance log lines.
test_plugin now has a module, so the create_scenario and patch unit
tests that relied on it having none use global_chat instead.
Closes#342
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The template asked for TypeBox ^0.30.6 while lib uses ^0.34, so a new
plugin failed tsc on the first message class. TypeBox is now provided
to plugin web code through the import map, so the template should be on
the version the web interface ships.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Controller.viteServer becomes devServer, matching the devMiddleware
name it replaced. The static_url limitation is described as one of the
web build rather than webpack.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Vite serves any file under fs.allow through /@fs/, and the default
allow list is the workspace root. In a clusterio checkout that is also
the usual data directory, so with --dev any client that could reach the
controller could fetch config-controller.json, config-ctl.json and the
database without logging in. webpack-dev-middleware only served the
compiled bundles.
Allow the controller, web_ui, lib and dev plugin directories and the
node_modules directories Node.js would resolve their imports from.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nothing else from Vite+ is used, and Oxlint reads the same rules from
.oxlintrc.json. This drops vite-plus and with it Vitest, a second copy
of Vite and about 130 packages.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lib and plugins import Node.js modules such as events and buffer, which
the build swaps for browser packages. Vite resolves an aliased import
from the importing package, and pnpm only lets a package see its own
dependencies, so the polyfills had to be dependencies of lib.
Resolve them from web_ui with a small resolver plugin instead, also
used by the dependency optimizer. Hosts no longer install the browser
packages, and plugins get the same polyfills webpack gave them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace ESLint, typescript-eslint, eslint-plugin-node and the flat
config helpers with vite-plus, which runs Oxlint for vp lint. The rules
are carried over from eslint.config.mjs by @oxlint/migrate into the lint
section of the root vite.config.ts. The formatting rules ESLint
deprecated run as @stylistic rules through Oxlint's JS plugin support,
and disable comments naming them are updated to the @stylistic names.
@stylistic understands TypeScript syntax that the core rules skipped,
so the autofix adds missing semicolons and spaces in type literals and
aliases. Leading | and & in multiline types stay where they were.
Oxlint's node/no-sync also reports identifiers ending in Sync that are
not calls. The test helper techSync is renamed and the two other names
are ignored.
Refs #1127
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Replace webpack and Module Federation with Vite 8. The controller build
emits one entry per shared module (lib, web_ui, typebox, antd, react,
react-dom, react-router) and lists them in manifest.json. The controller
turns that into an import map in index.html, and plugins are built as
ES modules that import the shared modules by name and are loaded with
import().
The shared Vite setup lives in @clusterio/web_ui/vite.mjs. Plugins use
clusterioPlugin(), which exposes index, package.json and web/index under
the same keys the webpack template used. The --dev and --dev-plugin
flags now run the Vite dev server in middleware mode with hot module
replacement.
Browser polyfills move from web_ui to lib, which is the code that needs
them, and lib sets the process and Buffer globals itself.
Refs #1127
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Keep error classes across links
Error responses now carry the registered name of the thrown error's
class, and the requester rejects with an instance of that class instead
of a generic RequestError. Error classes are registered like message
classes, and plugins list their own in `errors`.
RequestError is replaced by a hierarchy following the HTTP status codes:
ClientError (ValidationError, ForbiddenError, NotFoundError,
TimeoutError, ConflictError) and ServerError (InternalServerError).
PermissionError is PermissionDeniedError and the config errors
InvalidValue, InvalidField and InvalidAccess gain an Error suffix.
Link logs only errors whose class is not registered. These arrive on the
other side as InternalServerError with the original message and stack,
so an error forwarded through the controller is logged once, where it
was thrown.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Add UnavailableError and keep the PermissionError name
Host not connected, host shutting down and instance not running are
now UnavailableError (503) instead of ConflictError. The request error
migration notes move to a new 2.1 guide, since this ships in 2.1.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Use the new error classes in code added on master
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Host not connected, host shutting down and instance not running are
now UnavailableError (503) instead of ConflictError. The request error
migration notes move to a new 2.1 guide, since this ships in 2.1.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Error responses now carry the registered name of the thrown error's
class, and the requester rejects with an instance of that class instead
of a generic RequestError. Error classes are registered like message
classes, and plugins list their own in `errors`.
RequestError is replaced by a hierarchy following the HTTP status codes:
ClientError (ValidationError, ForbiddenError, NotFoundError,
TimeoutError, ConflictError) and ServerError (InternalServerError).
PermissionError is PermissionDeniedError and the config errors
InvalidValue, InvalidField and InvalidAccess gain an Error suffix.
Link logs only errors whose class is not registered. These arrive on the
other side as InternalServerError with the original message and stack,
so an error forwarded through the controller is logged once, where it
was thrown.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add a LogEntry type for log entries sent over links and returned by
log queries. It names the fields every entry has (level, message,
timestamp) and leaves the rest as unknown.
The console now prefixes each line with the entry's time in the
browser's local time zone, which makes it possible to tell when an
error happened after the fact.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Notes in the web UI now render through ReactMarkdown. A mention like
@instance/778528005 or @user/cooldude2606 turns into a link to that
resource's page, labelled with its name if the client has it and the id
otherwise. The stored note keeps the mention as written. Line breaks
are kept as hard breaks so existing plain text notes look the same.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
FactorioServer.fromOptions finds the Factorio install and generates the
RCON password before constructing, so _version and _dataDir are never
null and setTargetVersion and init are gone. checkForUpdates is now a
module function called before fromOptions.
Instance.fromConfig resolves the version, downloads Factorio if needed
and builds the server, then constructs the instance with it. It runs
before the instance is connected, so the starting and stopped statuses
and the version requests go through the host link. FactorioVersionsRequest
and LatestReleasesRequest now accept host as a source, and the host
tracks starting instances so a second start during a download is
rejected. Plugin loading stays a separate loadPlugins step after the
instance is connected.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Controller.fromConfig and Host.fromConfig now take the config, then the
plugin infos, then the logger or connector.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Controller.bootstrap and Host.bootstrap returned constructor arguments
that the caller had to spread into new Controller/Host, and
DatastoreProvider.bootstrap did the same for datastores. They are
replaced by Controller.fromConfig, Host.fromConfig and
KeyValueDatastore/SubscribableDatastore.fromProvider, which load the
data and return the finished instance.
Closes#1126
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fall back to copying on any link error except EEXIST, since filesystems
without hard links report it with several codes (EISDIR on exFAT under
Windows, ENOSYS on some FUSE mounts). Rename now goes through the same
move so it also gets the fallback and the rollback. Only report a
missing source save on transfer when the source is actually missing.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Instance dir and save names are tried without a retry cap, as on master
and in downloadFile, so an 11th instance named "New Instance" still gets
a folder.
Moves fall back to an exclusive copy when hard links are unavailable
(EXDEV, EPERM, ENOTSUP), and remove the new target if the source can't
be unlinked, so a failed move leaves no duplicate.
Local transfers keep appending .zip to the target name, as
findUnusedName did.
The copy auto-rename test now checks that the existing target save is
left untouched.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The name it returns may already be taken by a concurrently running process
by the time the caller uses it. Callers now try the name and retry with a
-2, -3, ... suffix on EEXIST instead.
Save transfers previously renamed with fs.rename, which silently overwrites
a save that appears at the target, and copied without COPYFILE_EXCL. Moves
now use link + unlink like save rename does, and copies pass COPYFILE_EXCL,
so both fail with EEXIST rather than overwrite.
Supersedes #880.
Co-authored-by: Hornwitser <code@hornwitser.no>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Factorio 2.1.18 removed PollutionSettings::max_pollution_to_restore_trees,
but readPollution still read it for every version, so every map setting
after it was read 9 bytes late: enemy_attack_pollution_consumption_modifier
came out as garbage, the enemy_evolution factors shifted by one field and
enemy_expansion.enabled and both expansion distances came out null.
Read the field only for strings older than 2.1.18. Add modified_v2_1_17
and modified_v2_1_18 fixtures generated with headless 2.1.17 and 2.1.18
from the modified 2.1 settings (base game only), unit tests asserting
their full decoded output, and use the 2.1.18 string for the
load-scenario integration test on 2.1.18 and later.
Fixes#1131
Co-authored-by: ColumbusLabs <287001685+ColumbusLabs@users.noreply.github.com>
Co-authored-by: Cooldude2606 <25043174+Cooldude2606@users.noreply.github.com>
Node 26.12.1 (CVE-2026-105481) makes the http agent check a kept-alive
socket for pending data before reusing it. The queued request now sees
the server's EOF, opens a new connection to the closed server and gets
ECONNREFUSED instead of ECONNRESET. Either way the request is dropped.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The download a join forces after dropping a copy that took the body found the
player already a spectator, so it kept the spectator's place and restored the
character there. It now takes the surface, position and vehicle the dropped
copy kept.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The leave handler skipped the upload for any active download, including a
prefetched copy no join had confirmed, which has replaced nothing here. Newer
local data, such as after a failed upload whose retry the join cleared, was then
never uploaded. Only a real download skips it now; a copy whose join took the
body still uploads nothing through sync_in_progress.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A copy confirmed at an earlier join has replaced the body with a spectator.
If a later join drops it (another controller run) while the generations match,
the join kept the local state: a spectator with no body, uploaded on the next
leave. Download instead. A newer copy replacing a held one keeps where the
body stood, so the replacement still counts as having taken it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An admin import or delete bumps the stored data's epoch and can start the
generations over, so a copy held from before it must not apply. The prefetch
carries the epoch like the token, and a held copy is used only when it matches
the acquire reply's.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An instance with older data can upload the generation a save already holds, and a prefetch of it then overwrote that
save's newer state. A join now drops a prefetched copy unless the ordinary download check says the data is needed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Chunks of an earlier controller run's copy still in flight could land in a newer run's copy of the same generation. Every chunk now carries the token, and the scenario ignores a chunk whose token differs from its record's.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
If the controller crashes after sending generation N but before saving it, the destination keeps that prefetch while the controller restarts at N-1, and another instance can then upload different data as generation N. The controller now picks a random token per run, sends it with every prefetch and acquire reply, and the scenario applies a held copy only when its generation and token both match, and lets a new run's copy replace one with another token.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A prefetched copy is sent without the save's list of cleared recipes, so
its delta had only add, and a recipe cleared on the destination but not
in the stored list stayed cleared where a download would give it its
notification back. Without a current list the instance now sends the
whole stored list marked full, and the save gives the recipes it has
cleared that aren't in it their notification back, as a download's
remove does.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
player_sync.prefetch_inventory holds a copy sent ahead in active_downloads
and finished_downloads, for a player who is offline or doesn't exist on
the destination, with the components its first chunk names. Chunk 1
refuses a player who is online, a download in progress, a stream of the
same or a newer generation and a finished copy of a newer one; a finished
copy of the same generation is replaced. At the acquire reply a copy is
used only when the reply is acquired with its generation and its
components are the ones the scenario syncs now. The player is then
switched to spectator as a download does and the copy is applied, at once
or at its last chunk. Otherwise the copy is dropped and the join
continues as before.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A plugin that moves a player calls remote.call("player_sync",
"set_destination", player_name, instance_id) before connect_to_server.
It only records the destination. Every leave clears it, and a leave
that uploads with reason switching_servers adds it to the upload as
destination_instance_id. The README describes it for plugin authors.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The instance passes destination_instance_id from a leave upload on as
destinationInstanceId, and sends a PrefetchEvent's data to
player_sync.prefetch_inventory in RCON chunks, using the download's
chunk loop. As for a download, the data is stripped by the instance's
own player_sync.sync_* options and the components go with the first
chunk. A non-empty reply means the save refused or ignored the copy and
stops the transfer.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An upload can carry destinationInstanceId. Once the upload is stored,
the controller sends the stored data, the upload merged with its copy,
in a PrefetchEvent to that instance, if it is another running instance
with player_sync enabled. No lock is taken or released. A failed send is
logged and doesn't fail the upload.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A QueueEvent arriving in the same socket read as a QueueSyncRequest reply is
applied before the reply, which then wrote the older queue over it. Skip the
reply when a QueueEvent was applied while it was in flight.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The drawer actions move to its header as Export, Import and a trash
icon for Delete. Import and Delete are disabled while an instance
holds the player, since the stored data is out of date until it
uploads. The drawer also shows color and cheat mode.
Inventory tables get title case names, pagination at 10 rows, item
search, sorting and quality icons. The player list shows force with
a filter instead of generation, and the database stats use
Descriptions without the largest entry.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Imports no longer accept the cutscene controller, which needs waypoints
to restore. Logistic filters and the basic item stack fields now have the
types and limits Factorio enforces, and the inventory summary skips
malformed items and counts repeated exported items once instead of
expanding them.
Lua's JSON loses precision past 2^50, so a generation there could no
longer be counted up and uploads were discarded. Imports now start over
from generation 1 when the stored or imported generation is at that limit.
Instances holding a copy from another epoch take the stored generation
instead of keeping their own higher one when there is no data to download.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
research_queue_api is compat.version_ge("2.0.56"), named like player_sync's version checks, so it doesn't read as a
setting. Handlers check it at the top; only the three research queue events are registered behind it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Imports and uploads could carry deleted: false, which the loader took for
a tombstone on the next restart. The schemas now reject the property, and
the loader only treats deleted: true as a tombstone and strips the key from
entries already saved with it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An import with color but no chat_color or tag assigned nil to the missing
LuaPlayer properties and failed restoration.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Wait for player list replication in integration tests
* Select available RCON ports before starting Factorio
* Document zero as automatic RCON port allocation
* Address review wording and test organization
* Restrict port collision hardening to integration fixtures
The send buffer limit in controller.max_send_buffer_size is set in MiB, but
buffered messages were measured with String.length, which counts UTF-16 code
units. Messages with non-ASCII text could go over the configured size in bytes
before the session was ended.
Measure each serialised message with Buffer.byteLength when it is buffered,
keep that size with the entry, and subtract the same size when it is
acknowledged. The web UI build already provides Buffer through the buffer
polyfill.
Fixes#1108
Co-authored-by: ColumbusLabs <287001685+ColumbusLabs@users.noreply.github.com>
The raise covered the whole gap between the serialized main inventory
and the main inventory with the armor restored, but only the bonus of
the armor's equipment arrives at the end of the tick. When the
destination force has a smaller character_inventory_slots_bonus than
the source, the rest of the gap never arrived and removing the raise on
the armor event dropped those slots on the ground. The raise is now the
equipment's inventory bonus minus what the grid already applies, at
most the gap, and the slots past it overflow into a corpse as without
the raise. Factorio 1.1 has no inventory bonus equipment and gets no
raise.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
on_player_armor_inventory_changed for the restored armor is raised at
the end of the download's tick, after the inventory bonus of the armor's
equipment applied, so the raise is removed there instead of in the next
tick's on_tick. The raise is only made for armor with a grid, as the
event needs the armor to be restored.
A player leaving on the tick of a download no longer has the raise
removed from the character first, which shrank the main inventory
before the bonus applied and spilled the extra slots. The raise is
taken off the uploaded inventory bonus instead.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The inventory bonus of equipment in the armor's grid (toolbelts) is
applied by the engine on the tick after the armor is inserted. The main
inventory was restored in the same tick, so the items of every slot past
80 plus research plus the armor's own bonus did not fit and were put in
a corpse: legendary mech armor with legendary toolbelts left 2125 of
2330 slots in the corpse.
The armor is now restored first, the character's inventory bonus is
raised to fit the saved main inventory, and the raise is removed again
on the next tick, or before an upload if the player leaves first.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Document the cluster research queue in the plugin README and the root
README. Review fixes: ipairs for queue order, event annotations on the new
Lua handlers, an IpcQueue type, a test for clearing the queue, and drop the
controller import from test/missing.js.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The last event of a tick now decides which player the queue edit is attributed to.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The save module reports queue edits on a following tile, applies the cluster
queue (dropping unknown technologies), and offers a lab-window checkbox.
All new events are gated on Factorio 2.0+.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Forward queue edits and follow requests from the save to the controller,
and apply controller broadcasts and the start-up sync to the save.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- QueueUpdateEvent, QueueEvent and QueueSyncRequest messages.
- The controller keeps one queue (max 7 names) in research_queue.json, saved with the other data.
- An update that changes the queue replaces it and is broadcast as QueueEvent; an equal update is ignored.
- QueueSyncRequest returns the queue and seeds it from the request when the queue is empty.
- A finished technology (completing contribution, advancing FinishedEvent, technology sync) drops its first queue entry without a broadcast.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Logistic slots saved before 2.0 are keyed by index strings, which
json_to_table keeps as strings, so the migration branch never ran and
the 2.0 branch failed on the missing sections. Detect the old format by
the missing sections and convert the keys to numbers.
The import schema now uses the minimums Factorio enforces for character
stats, checks the filter value type and the quick bar comparator, and
allows filters without quality or comparator.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An import with `crafting_queue: {}` passed the schema and broke the
next restore in deserialize_crafting_queue. The import schema now
checks the members module/serialize.lua reads from the crafting queue,
quick bar, personal logistic slots and character stats, while still
allowing a component to be left out.
Character stats missing from the data are left as they are instead of
being set to nil.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Raise the catalog and package.json floors to the latest release within
each major, and move typebox to 0.34, react-router-dom to 7,
winston-daily-rotate-file to 5, webpack-dev-middleware to 8 and
webpack-manifest-plugin to 6. None of these need code changes.
express 5.3.0 requires proxy-addr ^2.0.8 and qs ^6.16.0, which puts a
floor under the critical proxy-addr and high qs advisories for anyone
installing the published packages. webpack-dev-middleware 8 fixes a
high advisory in the controller's dev mode, and react-router 7 fixes two
moderate ones.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Browsers have no setImmediate. The web bundle only had one because jszip
depended on the setimmediate package, which installs it as a global.
Since jszip was replaced in #1011 every component using a subscription
threw on unmount, breaking page changes in the web interface.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A host running the plugin from before epochs never sends one, so after
an import or delete every upload it made was discarded while the
instance believed it succeeded. The Lua now always sends its epoch, and
an upload without one is compared by generation against the stored
data the way it was before, taking over the stored epoch. Those hosts
can still overwrite an import with a copy they failed to upload, which
the README now says.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The tombstones lived in player_sync_deleted.json, written after
player_sync.json. A stop between the two writes lost the tombstone of a
fresh delete, and the next upload from an old copy brought the deleted
data back. Deleted players are now entries with deleted: true in
player_sync.json, so both are written in one safeOutputFile call.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An import with an unknown controller or malformed inventories replaced
the stored snapshot and bumped its epoch, then broke the player's next
join. SetPlayerDataRequest now checks the fields the scenario restores,
leaving item stacks to the Lua deserializer and allowing the partial
snapshots instances with disabled components upload.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An import, delete or release for one player that finished after another
player was selected reloaded the drawer with the first player's data
under the second player's name. The drawer body is now keyed by player,
so a late completion can only update the view it was started from.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Generations alone could not order a repair against copies held by
instances. Deleting dropped the watermark, so a stale instance could
skip downloading the new baseline. An import could land on the same
generation as an upload that never reached the controller, and the
instance would then keep its own copy.
Imports and deletes now bump a per-player epoch, which the controller
returns on acquire and checks on upload. Instances throw away a copy
from an earlier epoch, and deleted players keep their generation and
epoch in player_sync_deleted.json. A restarted download also takes the
new generation instead of the outdated one.
Generations in player data must now be non-negative integers, so an
invalid import is refused before it reaches the datastore.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
InstanceDetailsGetRequest needs core.instance.get, which the
player-sync list and show commands are not documented to require.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Factorio's table_to_json writes an empty inventory as {"i":{}}, which
made the summary throw for show and the web drawer.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A request for one player could resolve after the drawer switched to
another and show its inventory, and export it, under the wrong name.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov flagged ctl.ts at 21% patch coverage. Run each command
through a fake control link with canned responses and check what it
prints and sends.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Admins had no way to look at or fix the player data the controller
stores when a sync went wrong. The web page now lists every stored
entry with its generation, size and the instance holding the lock, and
opens a drawer per player with an inventory summary, JSON export and
import, delete and lock release. The same operations are exposed as
clusterioctl player-sync commands.
Import bumps the generation past the stored one so instances holding an
older copy download the new data on the next join. Import and delete
are refused while an instance holds the lock, since its upload on leave
would undo the change. A new player_sync.inventory.modify permission
gates the write operations.
Closes#1003
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Replace jszip with yauzl and yazl
jszip loads a whole archive into memory to read it and buffers the
output when writing. Saves and mods can be several hundred megabytes,
so read zips with yauzl, which only reads the central directory and
streams entries on demand, and write them with yazl, which streams the
output.
lib gains a ZipArchive wrapper around yauzl with the lookup helpers the
callers need, findRoot() now takes it instead of a JSZip. Saves are
patched by streaming every kept entry into a new zip, mod info.json is
read straight from the file, exports are streamed to the controller and
Factorio zip downloads go through a temp file instead of a buffer.
Patching now inflates and deflates each entry again where jszip reused
the compressed data, on a 336 MB synthetic save that is 7 s instead of
0.7 s, with peak memory going from 416 MB to 151 MB.
Closes#1002
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Copy save entries without recompressing them
yazl inflates and deflates every entry it is given, so patching a save
took ten times longer than with jszip, which reused the compressed
data. Add a ZipWriter to lib that streams entries out as they are
added and copies entries from a ZipArchive byte for byte, with ZIP64
structures when sizes or offsets need them, and use it in patch().
On a 336 MB synthetic save patching now takes 0.4 s at 143 MB peak RSS
against 0.75 s at 417 MB with jszip. Factorio loads the result.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Handle save write errors before the patch loop awaits them
A write error on the temporary save, such as a full disk, rejected the
pipeline promise while addEntry() or addBuffer() was still pending. Nothing
was listening yet, so the host's unhandledRejection handler exited the
process instead of failing the instance start.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Keep the export zip cache local to each export
The cache was module level, so when two instances on one host exported at
once, the first to finish closed archives the other was still reading.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Start writing zips before adding files from disk
yazl reads files passed to addFile() as soon as they are added. An
unreadable file emitted an error while the directory traversal was still
running, before the output pipeline existed, and the process exited on the
uncaught error. build_mod and zipDirectory now start the pipeline first and
reject instead. build_mod also removes the partial zip on failure.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
The scenario no longer sends its components with the download request. The instance strips the download by its own config and sends the components it used with the first chunk. If they differ from the ones the scenario started the download with, because the config changed in between, the scenario restarts the download.
A pending download is now marked with sync_in_progress instead of clearing dirty while spectating, and it is set for every download rather than only when inventories are synced. Uploads are skipped while it is set, so a player who leaves during a download with inventories off no longer uploads their stale state.
SyncComponent names the components in the Lua annotations.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
serialize_local_controller now respawns a player waiting to respawn for a moment to read the logistic requests hidden in the ghost, when logistics are not synced. Without this the requests were lost when the download turned the ghost into a spectator.
deserialize_player compares the physical controller with the one it restores. A player in remote view with a character was seen as changing controller, so with inventories off their crafting queue was serialized and queued a second time on the same character.
The ghost branch only sets the force when the force is synced, it errored on the stripped field otherwise.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
place_items no longer creates script inventories for what does not fit. Each serialized inventory is split at the size of its destination: the slots that fit are restored slot for slot and the rest become serialized stacks, which are inserted into the free space of the same inventory and then main, guns, ammo and armor. What is still left over is returned as a list and spill_items puts it into a corpse sized to hold it. This keeps 1.0 out of reach of game.create_inventory.
The armor-first fill order is gone, as the bonus slots from armor equipment only apply on the next tick anyway (#1106). Filters are no longer stripped for destinations, they are the same inventory type the data came from.
deserialize_inventory concatenated the filter table in its error print on 2.0, which errored out the whole restore when an inventory did not support filters.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With sync_controller on and sync_inventories off, switching to a synced
god, ghost or spectator destroyed the local character and its items.
Serialize the local items first and place them in the new controller.
Logistic requests and the flashlight live on the character, which the
download replaces when inventories are synced. Save them with the local
controller and restore them when their component is not synced.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A character without requests serialized to nil, which the controller
reads as a player with nowhere to hold them, so with inventories off
the stored requests came back. Serialize an empty table instead, clear
old slots before restoring on 1.1, and fix the set_personal_logistic_slot
typo that made any 1.1 restore fail.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The instance read its config when the controller answered, so a sync_
field changed during the download could strip a component the scenario
still expected, and the scenario treated it as empty.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Syncing the controller required syncing the inventories, and with the controller off a god or ghost kept its local items while the stored character was left alone. Cooldude pointed out there is no reason the items can't follow the player to a different controller.
The controller and inventories components are now independent. The god inventory and the hidden ghost inventory belong to the inventories component. On download the scenario first settles the controller, synced or local, and then places the stored items into whatever that controller holds. An inventory goes slot for slot into the inventory of the same name when it fits, otherwise its items are inserted into the main, guns, ammo and armor inventories, and anything left goes into a corpse below the player. A spectator holds nothing, so everything goes into the corpse. A player without a character gets the crafting queue ingredients as items.
This removes the early respawn of a ghost with a stored character and the upload of the respawned character, a ghost now gets and uploads its hidden inventory. Download spectates every synced player when the inventories are synced, as a god or ghost now holds synced items too. The controller keeps the stored logistic requests when the upload came from a player with nowhere to hold them.
Slot filters are dropped when the destination does not support them, such as a script inventory.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A player who died on an instance syncing inventories but not the controller uploaded no character, so the controller kept the stored one and the player got their items back on the next join while the corpse still held them. The scenario now respawns the ghost for a moment, as the controller synced path already does to read the hidden inventory, and uploads that character.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The controller merged uploads using the uploading instance's current config. An upload serialized under different settings, for example one sent from the save right after an admin turned on sync_inventories and restarted the instance, was then merged as if it held a character and wiped the stored one. Uploads now carry the components they were serialized with, the controller merges by those and drops the key before storing.
The scenario keeps the components a player's synced session started with and serializes their upload with them, so changing the options while a player is online takes effect the next time they join. A player whose session lacks a component that is now synced downloads again even if the generation matches. Downloads snapshot the components when they start, so a change during a download can't mix two sets.
A failed acquire unsyncs the player again regardless of the inventories setting, as before this branch.
The god and dead player inventories move from the inventories component to the controller one, matching what the scenario syncs, which removes the special case in mergePlayerData.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With the controller off, a god or ghost player was still turned into a spectator while downloading, which threw away the local state the sync is meant to leave alone. Only spectate a player without a character when the controller is synced, and skip the ghost round trip when the ghost is the local state.
A ghost on an instance that syncs inventories but not the controller never received the stored character, so after respawning it uploaded an empty one over it. Respawn such a ghost straight away when there is a stored character to give it.
Clear the saved local controller when the player picks a temporary inventory after a failed download, otherwise a later download restores the stale controller over the temporary character. An empty download only gives a character when the controller is synced or the player was spectated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds an inventory_sync.sync_<component> instance config field for the
controller, force, appearance, inventories, logistics, quick bar,
settings and recipe notifications. All default to on, which keeps the
current behaviour.
A component which is off on an instance is left alone there and its
fields are not stored from that instance. The controller merges each
upload with the stored data using the uploading instance's config, the
instance strips the fields from downloads before sending them over RCON,
and the scenario is told the enabled set on start and when it changes so
it can skip serializing them and skip the steps which replace the
character.
The controller is only synced together with the inventories, as switching
controller can destroy the character holding them. Without the controller
synced the player gets back the controller they had on the instance, only
character inventories move between instances, and a player without a
character keeps their inventory.
Closes#968
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The on_research_finished handler read ignore_research_finished from the
research_sync global, which is the table of RCON functions and never has that
field. The flag that sync_technologies and research_technology set lives on the
script data, so the check never returned and research that came from the
controller was sent back as newly finished. The handler now reads the flag from
the script data.
Co-authored-by: ILLISIS <jayson.harris01@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
send_udp and get_host_udp_port indexed script data that is only created
on the first tick after patching, so calling them from on_init stopped
the server. They now return nothing until then.
Received packets were only polled once the host port was known, which
needs script commands. Poll whenever recv_udp exists instead, it does
nothing when the server runs without --enable-lua-udp.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The listener runs from the socket callback, so a non-async handler that
threw before returning a promise reached the unhandled error handler and
took down the host.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Sync quick bar slots that hold a specific item
Quick bar slots of type item (a specific blueprint or blueprint book) were
dropped with a warning. Store the player inventory and slot that holds the item,
and after the inventories are restored point the quick bar slot at the stack in
that slot. Slots whose item is not in the inventories are dropped as before.
* Download each Factorio version only once at a time
Instances starting together each downloaded the new version into the same
.tmp file and extracted into the same version directory, corrupting the
install. Keep the downloads in progress in a map by target directory and
let later callers wait on the existing download.
Fixes#1105
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RhMv1AxB8EZdyZoSJhyNmY
* Wait for a download in progress whose version looks installed
The changelog is extracted partway through the archive, so a server
checking during the extraction saw the version as installed and could
start the half-extracted install. Treat a version with a download in
progress as not installed, so the server waits on that download.
Hold the mocked download open in the concurrency tests and wait for the
second server to reach the waiting path instead of relying on a delay.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RhMv1AxB8EZdyZoSJhyNmY
---------
Co-authored-by: ILLISIS <jayson.harris01@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Restore logistic group filters when the group is missing or empty
Named personal logistic groups were restored by name only, so a force
without a group of that name got an empty one, and the empty filters were
uploaded again on leave. Keep assigning the name only when the force's group
already has filters, otherwise also write the stored filters.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* Restore ghost player force before logistic slots
A player waiting to respawn had its personal logistic slots restored
before its force was set. The group lookup and the filter write then
used the force the player had on this server, which could write the
incoming requests into a group shared with other players.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: ILLISIS <jayson.harris01@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
serialize_item_stack did not store the spoil percent and
deserialize_item_stack did not pass it to set_stack, so spoilable items
arrived fresh after a transfer. Store it when above 0 and restore it. The
percent is used instead of spoil_tick since tick counts differ between servers.
Co-authored-by: ILLISIS <jayson.harris01@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The path for a download that comes back with no player data called
restore_position(record, player), but the function takes (player, record).
That raised an error, so the download was never marked finished and the
player waited for the timeout dialog.
Co-authored-by: ILLISIS <jayson.harris01@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Danielv123 pointed out the recv_udp crash was fixed in 2.1.10, not
2.1.12: recv_udp used an input action for synchronisation which the
headless server could not send without a player index. Verified on
downloaded headless builds, 2.1.9 crashes and 2.1.10 works.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Factorio 2.0.59 through 2.1.11 crash with "Trying to add invalid input
action to the closure" when helpers.recv_udp dispatches a received
packet, whether polled from on_tick or called from a command and
whether or not the server is paused. Reproduced on headless 2.0.77 and
2.1.8, works from 2.1.12 on. FactorioServer now refuses to start with
Lua UDP enabled on older versions, the docs say 2.1.12, and the
integration tests only enable it on 2.1.12 or later.
The to-game integration test polls recv_udp from on_tick like the
clusterio module does and sends commands to make the empty server tick.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Review feedback from Cooldude2606: UDP is most useful for arbitrary
blobs, so send_udp now takes any string and the packet is just
"channel?payload". Plugins get it as a udp-<channel> event with the
payload as a Buffer, or through handleUdp which logs handler errors
like handle does for ipc events.
The host socket is now opened when the RCON client is started and
closed where the RCON client is ended, matching how the RCON client is
handled, instead of being bound in start() and closed by _resetState.
The integration round trip moved to the FactorioServer tests, where it
sends a packet each way through a real server without the module.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Codecov flagged the sendUdp methods on Instance and BaseInstancePlugin,
the bind, socket error, malformed packet and send failure paths in
FactorioServer, and the --enable-lua-udp branch of startScenario. Add
unit tests for each and start the integration scenario test with UDP
enabled on Factorio 2.0.59+.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Add opt-in UDP messaging between the host and the Factorio server using
the --enable-lua-udp option of Factorio 2.0.59+. When
factorio.enable_lua_udp is set the instance binds a localhost UDP socket,
starts Factorio with --enable-lua-udp on factorio.lua_udp_port (random
if null, like the RCON port) and tells the clusterio module the host port
via clusterio_private.update_instance.
Data from Factorio is sent with clusterio_api.send_udp(channel, data) in
the same IPC format as send_json, so plugins receive it through the
existing ipc-<channel> events. Packets not sent from the Lua UDP port
are ignored, as that is the only unauthenticated way into the IPC
handler. Data to Factorio is sent with the new sendUdp method on
instances and instance plugins, and is dispatched in-game as
on_udp_packet_received events by polling helpers.recv_udp every tick
from the clusterio module.
Plugins requiring UDP declare the new LuaUdp feature flag, which like
SavePatching and ScriptCommands refuses to load the plugin on instances
without the option enabled.
Verified against a headless 2.1.17 server: send_udp and recv_udp are
silent no-ops when the option is off, so a stale port in a save that is
moved to an instance without UDP does no harm. The integration test
matrix now starts the instance with UDP enabled on Factorio 2.0.59+ and
round trips a player event over it.
Closes#969
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The web interface registered the hook's listener directly on the
connector, so handlers got the connector's emit arguments (the hello
data for connect, nothing for the rest) instead of the event name.
Fixes#1083
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Let plugins migrate the data core stores for them
Plugins had no way to update the config fields and permissions core stores
for them, so renaming inventory_sync needed migrations in core packages.
A plugin can now declare a migrationEntrypoint. It's loaded on the
controller and hosts before configs and roles are read, for every plugin in
the plugin list, and attaches handlers to sync hooks for the controller,
host and instance configs and roles. lib gets helpers to rename config
fields and permissions, and each config gets a hidden <plugin>.version field
set after load so migrations can tell which version saved the data.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Write plugin versions while loading configs
The staging copy made by Config.set() went through the constructor, so
plugin migrations ran a second time on the live fields. A version-gated
migration was applied twice when setPluginVersions() changed the version.
Staging now copies the fields with update() instead.
A config saved without <plugin>.version got the running version as its
default, so setPluginVersions() saw no change and nothing saved the stamp.
The constructor now writes the registered plugin versions after the
migrations run and marks the config dirty when that changes a loaded value.
Controller.bootstrap() saves the instances that came out dirty, and
setPluginVersions() is gone.
Restructure the SyncHook and plugin config tests by method, and suggest the
controller config version for role migrations in the docs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Rename inventory_sync to player_sync
The plugin syncs more than inventories now (settings, appearance, recipe
notifications), so name it for what it does. The package becomes
@clusterio/plugin-player_sync and every inventory_sync identifier follows.
A migration entrypoint renames the inventory_sync config fields, including
load_plugin, and the inventory view permission. The plugin moves
inventories.json in the database directory to player_sync.json and
storage.inventory_sync in saves to storage.player_sync itself, and closes
dialogs left open under the old GUI names on startup.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Let old saves through the player left guard
on_pre_player_left_game runs before on_server_startup when an old autosave
with connected players is loaded, so the data is still under inventory_sync.
The guard returned before get_script_data() could move it, and the recovery
upload was never sent.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
on_pre_player_left_game runs before on_server_startup when an old autosave
with connected players is loaded, so the data is still under inventory_sync.
The guard returned before get_script_data() could move it, and the recovery
upload was never sent.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The plugin syncs more than inventories now (settings, appearance, recipe
notifications), so name it for what it does. The package becomes
@clusterio/plugin-player_sync and every inventory_sync identifier follows.
A migration entrypoint renames the inventory_sync config fields, including
load_plugin, and the inventory view permission. The plugin moves
inventories.json in the database directory to player_sync.json and
storage.inventory_sync in saves to storage.player_sync itself, and closes
dialogs left open under the old GUI names on startup.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The staging copy made by Config.set() went through the constructor, so
plugin migrations ran a second time on the live fields. A version-gated
migration was applied twice when setPluginVersions() changed the version.
Staging now copies the fields with update() instead.
A config saved without <plugin>.version got the running version as its
default, so setPluginVersions() saw no change and nothing saved the stamp.
The constructor now writes the registered plugin versions after the
migrations run and marks the config dirty when that changes a loaded value.
Controller.bootstrap() saves the instances that came out dirty, and
setPluginVersions() is gone.
Restructure the SyncHook and plugin config tests by method, and suggest the
controller config version for role migrations in the docs.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Plugins had no way to update the config fields and permissions core stores
for them, so renaming inventory_sync needed migrations in core packages.
A plugin can now declare a migrationEntrypoint. It's loaded on the
controller and hosts before configs and roles are read, for every plugin in
the plugin list, and attaches handlers to sync hooks for the controller,
host and instance configs and roles. lib gets helpers to rename config
fields and permissions, and each config gets a hidden <plugin>.version field
set after load so migrations can tell which version saved the data.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The controller logs the url and query of every HTTP request. Plugins
doing OAuth get the authorization code as a query parameter on their
callback route, so it ended up in the log. redactUrl now accepts a
path, code joins the redacted parameter names, and redactQuery covers
the parsed query in the log metadata.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Fix running Clusterio from published packages
Since the move to ES modules and pnpm 12, packages installed from npm
fail to start:
- The package imports of @clusterio/lib point Node at TypeScript
sources, which Node refuses to strip types from inside node_modules.
- The controller, host and ctl import their package.json from next to
the compiled entrypoint, a copy pnpm leaves out of the package.
- ctl, host and web_ui have no .npmignore, so pnpm packs them using the
repository .gitignore and leaves out nearly all of dist/.
Resolve the package imports to the compiled files at runtime while
keeping the sources for type checking, import package.json by package
name so it resolves to the real file, and add .npmignore files so pnpm
packs the same files as npm.
* Load plugins installed from npm by package name
Plugins installed from npm are listed by package name, but the plugin
loader joined that name onto the main and entrypoint paths and imported
the result as a file, which resolves relative to the working directory.
Every npm installed plugin then failed to load. Resolve these paths from
the directory of the plugin's package.json instead.
WsServer.handleConnection now listens for errors on every socket for its
whole life, and every socket a WsServerConnector holds comes from there,
new sessions and resumed ones alike. The connector's own listener only
logged the same error a second time, so it is removed.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F7obMEi4L4PzUQ8w8q9Uyz
HostRouter.wakeInstance() logged err.stack, which RequestError leaves
undefined, so a failed start was logged as "Error starting instance:
undefined" and the reason, such as no game port left in the range, was
lost.
Claude-Session: https://claude.ai/code/session_01F7obMEi4L4PzUQ8w8q9Uyz
Co-authored-by: ILLISIS <jayson.harris01@gmail.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Plugins installed from npm are listed by package name, but the plugin
loader joined that name onto the main and entrypoint paths and imported
the result as a file, which resolves relative to the working directory.
Every npm installed plugin then failed to load. Resolve these paths from
the directory of the plugin's package.json instead.
Since the move to ES modules and pnpm 12, packages installed from npm
fail to start:
- The package imports of @clusterio/lib point Node at TypeScript
sources, which Node refuses to strip types from inside node_modules.
- The controller, host and ctl import their package.json from next to
the compiled entrypoint, a copy pnpm leaves out of the package.
- ctl, host and web_ui have no .npmignore, so pnpm packs them using the
repository .gitignore and leaves out nearly all of dist/.
Resolve the package imports to the compiled files at runtime while
keeping the sources for type checking, import package.json by package
name so it resolves to the real file, and add .npmignore files so pnpm
packs the same files as npm.
WsServer attached no error listener to a new socket. WsServerConnector adds
one, but only after the handshake. ws emits an error for an invalid frame,
for example a close frame with the reserved status code 1006, and an error
event with no listener throws. That stopped the controller.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F7obMEi4L4PzUQ8w8q9Uyz
Covers the icon_size default of 64 in the export icon loaders and the
host recreating saves and script-output when an instance is assigned.
Both tests fail when the original fix is reverted.
Closes#710
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
serializedConfigToString now asks a control-located config instance
whether each field is readable and writable instead of relying on a
hard-coded list per command. Hidden and unknown fields are skipped, and
field descriptions are written as comments above each value.
Object fields such as factorio.settings are now included, so they are
written as JSON, and configToKeyVal splits on the first = only so those
values survive the round trip.
Fixes#1051
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
EventSubscriber registers with the control's onAccountUpdate when it
has one and resends its subscription through handleAccountEvent. A lost
permission makes the request fail, a granted one syncs it again.
The web Control also reports the account on connect, so resends are
queued and sent once per tick. A failed subscription request now sets
synced to false and notifies callbacks, before the error never reached
the UI.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Add a plugin migration guide for 2.0
Covers the plugin API changes between 2.0.0-alpha.27 and 2.0.0: ESM,
entrypoint functions and hooks, loadedPlugins, permissions on the
plugin declaration, the ctl renames, staticRoot and optional peer
dependencies.
Document that from 2.0 major versions contain breaking changes for end
users while minor versions may contain breaking changes for the plugin
API, and fix stale examples and names in writing-plugins.md and
managing-a-cluster.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Show the fromContext class pattern in the migration guide
Plugins that still want a class for state use a private constructor,
an async static fromContext and a static WeakMap lookup, which also
replaces controller.plugins.get() for cross-plugin access. Check
loadedPlugins with Set.has on the info from pluginInfos.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Attach hooks in fromContext and export it as the entrypoint
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
The old name read as an unconditional unsubscribe. Log following has
its own subscription outside SubscriptionController, so check
core.log.follow on permission changes too. Move the per-connection
work into one helper and split the tests into userPermissionsUpdated
and rolesUpdated, checking the account update event is sent.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add controller.max_send_buffer_size in MiB, default 64 and 0 for no
limit. Close with 1013 instead of 1008 on overflow: clients treat 1008
as fatal, so a host would shut down and the web UI would log out. With
1013 the client resumes, the controller answers with invalidate since
the session has ended, and the client registers a new session.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Lowering the limit at runtime would leave connections holding more
filters than allowed, so read it once when the controller starts.
Move the limit tests into the subscribe and replace groups.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The controller HTTP access log and the express error log wrote the full
request headers at the http level, which both default log transports
keep. Clients send their access token in X-Access-Token or
Authorization: Bearer, so user and host tokens ended up in plain text
in controller and cluster logs, and the cluster log can be read back
over the API with core.log.query.
Mask the authorization, x-access-token, cookie and proxy-authorization
headers before logging.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Every message sent over a link is kept in the send buffer until the
other side acknowledges it with a heartbeat, so it can be resent if
the session resumes. Nothing forced a peer to acknowledge, so a client
that kept heartbeating without a sequence number while subscribed to a
busy event made the controller buffer messages until it ran out of
memory.
Store buffered messages serialised, track their total size, and close
the connection once it goes over maxSendBufferSize. The controller
sets this to 64 MiB for the connections it accepts. A client that far
behind could not resume in a reasonable time anyway.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The permission for a subscription was only checked when subscribing,
so a control connection kept receiving events after the user's roles
or role permissions changed to no longer allow it.
Add SubscriptionController.unsubscribeUnpermitted which rechecks every
subscription of a link and removes the ones the user is no longer
permitted, and call it for the affected control connections from
userPermissionsUpdated and rolesUpdated.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A subscriber could keep sending subscribe requests with new filters and
the controller would add every one of them to the subscriber's filter
set without bound, growing memory until it ran out.
Add controller.max_subscription_filters (default 10000) and reject
subscribe and replace requests that would leave a subscriber holding
more filters than that for one event.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Plugins that still want a class for state use a private constructor,
an async static fromContext and a static WeakMap lookup, which also
replaces controller.plugins.get() for cross-plugin access. Check
loadedPlugins with Set.has on the info from pluginInfos.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A handler that only updates gauges has nothing to yield. collect() already
dropped undefined results at runtime, now its type says so too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Covers the plugin API changes between 2.0.0-alpha.27 and 2.0.0: ESM,
entrypoint functions and hooks, loadedPlugins, permissions on the
plugin declaration, the ctl renames, staticRoot and optional peer
dependencies.
Document that from 2.0 major versions contain breaking changes for end
users while minor versions may contain breaking changes for the plugin
API, and fix stale examples and names in writing-plugins.md and
managing-a-cluster.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Controller.loadJsonObject(path, true) returned {} for a missing file, so
loadPlugins' catch never ran and a plugin whose web build was expected but
missing was not reported (the second half of #1046; #1047 fixed the path).
With throwOnMissing set, every error now throws; without it only a missing
file is ignored, so a malformed JSON file throws again as before #1047.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BGUULoiHNbAegwMK4PSjHJ
Below the lg breakpoint the list is not rendered and the permission
groups take the full width. At md the site sider already takes 250px,
so a 200px list next to it left too little room for the permissions.
The list keeps its sticky position but is capped at the viewport height
minus its top offset and scrolls on its own when there are more groups
than fit.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A sticky list of permission groups sits left of the permissions. Clicking
an entry scrolls to that group, and its dot is blue when any permission
in the group is granted, grey otherwise. Same layout as the ExpCluster
scenario features page.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Add a request to mark hosts and the controller as needing a restart
Not every plugin arrives through PluginInstallRequest. Plugins checked
out into external_plugins or installed by hand on the machine never
flag a restart, so the web UI shows nothing and tooling that acts on
the flag never fires.
HostMarkRestartRequiredRequest and ControllerMarkRestartRequiredRequest
set config.restartRequired, guarded by the existing restart
permissions. They are exposed as "Mark Restart Required" in the control
dropdown of the host and controller pages, hidden once the flag is set,
and as mark-restart-required ctl commands. The mark shows up on the
next system metrics poll.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Cover the mark-restart-required ctl commands
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Not every plugin arrives through PluginInstallRequest. Plugins checked
out into external_plugins or installed by hand on the machine never
flag a restart, so the web UI shows nothing and tooling that acts on
the flag never fires.
HostMarkRestartRequiredRequest and ControllerMarkRestartRequiredRequest
set config.restartRequired, guarded by the existing restart
permissions. They are exposed as "Mark Restart Required" in the control
dropdown of the host and controller pages, hidden once the flag is set,
and as mark-restart-required ctl commands. The mark shows up on the
next system metrics poll.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
On Windows this test throws "EPERM: operation not permitted, rename
<path>" presumably because Windows prevents concurrent renames. This
does not affect the test outcome, so ignore the error(s).
These are used outside of the integration tests, but including the
integration index file causes the very slow setup of the contorller
and host. Avoid the integration setup costs when running commands
such as `pnpm mocha test/lib` by moving these utility functions to
the common test functions.
Without a transport Winston will print annoying "Attempt to write logs
with no transports" warnings all over the place. Setup a transport in
tests that cause logging to suppress this.
The behaviour of creating the config file if it's missing while
convenient causes confusion and problems when the tools are run
in the wrong folder. They end up creating a shadow installation
and misdirect the user with nonsensical error messages, instead
pointing to the real problem which is that the tool can't find
the configuration needed to actually run.
Require a config file to be present at startup and refuse to run if this
file is missing. To support the odd use-case of creating the config
when missing a --create-config flag is added. To conveniently generate
a config file to edit out from the "config create" command is added.
Ctl ended up calling its main class for "Control" for historical
reasons, along with its config file and plugin entrypoint.
The plugin entrypoint was renamed to ctl in 808eda0513. This
finishes that renaming by also renaming the main class, default config
file, config-control command, and main entrypoint.
The config-control command is renamed to config to be consistent with
the host and controller's config commands.
The term "control" is still kept in the config and message locations as
a common specifier for both ctl and web_ui.
_waitForRconReadyOrExit resolved with an Error to signal the process
exited before RCON came up, which is an odd shape for a promise. It now
resolves when RCON is ready and rejects otherwise, with the error emitted
with the exit, or "RCON connection lost" when there was none.
Observable behaviour is unchanged. sendRcon rejects with the same
messages as before and stop ignores the rejection, as stopping is done
once the process is gone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The instance name on a log line is whatever the host sent, so a host can
attribute a line for one of its own instances to the wrong name. Compare
it with the name the controller has on record and replace it when they
differ.
Host connections can live for days, so the warning suppression expires
after five minutes instead of latching for the life of the connection,
and the bogus instance and misaligned name warnings share it so the two
together cannot spam the log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Waiting for exit alongside rcon-ready made sendRcon reject with the flat
"RCON connection lost" for every case, including the unexpected exits
that used to reject with the EnvironmentError saying why the server went
away, as events.once rejects on error events.
Have the paired wait capture the error emitted with the exit and hand it
back, so sendRcon throws that when there is one and falls through to the
old message when there is not, which is the kill path where no error is
emitted. stop() ignores the error as its job is done once the process is
gone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The fieldChanged handler repeated the hook invoke in each branch, and
factorio.shutdown_timeout had no invoke at all, so plugins were never
told about a change to it. Do the field specific side effect, then
invoke the hook once for every field.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Puts the contributors under a "Contributors" title on a single comma
separated line, and prints commits, merged pull requests, files changed
and line additions and deletions since the last release after them.
The compare API lists at most 300 files, so the file and line counts
come from the diff media type of the same comparison instead. To fetch
it, githubFetch now lets the caller override the Accept header and
checks the Content-Type against it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Terminate control connections of deleted users
ControlConnection keeps the User object it authenticated with and checks
permissions against it. Deleting the user removed the record but left
its sessions open, so a deleted user kept all its permissions until the
WebSocket closed.
Repro: create "victim" with role 0 and connect as it. Another admin
sends UserDeleteRequest("victim"). Victim can still send
UserCreateRequest("mallory") and UserUpdateRolesRequest("mallory", [0]).
Terminate the deleted user's connections like revoke-token does. When
a user deletes itself, defer terminating the requesting connection so
the response is sent first.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Terminate deleted user sessions immediately
Drop the special case that deferred terminating the connection issuing
the delete, so deleting yourself behaves the same as revoke-token and
every session of the deleted user is closed before the handler returns.
Set tokenValidAfter to the current time in createUser instead of 0.
Deleting a user and recreating it under the same name left tokens issued
before the deletion valid, as the check is iat < tokenValidAfter. Doing
it in createUser covers every path that makes a user. A token signed in
the same second as creation is still accepted, so create-admin and the
other production paths sign after the user exists and are unaffected.
The integration harness signed the control token at import time, before
bootstrap create-admin created the user. Sign it in the root before hook
instead and reuse it in link.js.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Handle SessionLost when deleting your own account
Deleting your own account now terminates the session immediately, so the
web UI received a SessionLost rejection and reported "Error deleting
user" for a delete that had succeeded. Use the same branch revoke does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Cooldude2606 <25043174+Cooldude2606@users.noreply.github.com>
handleDebugDumpWsRequest sets clusterio_ignore_dump on the current
socket so the WsServer send wrapper does not echo the dumper's own
frames back into debugEvents. The flag was only set on the connector
"connect" event, but WsServerConnector.continue() installs a fresh
socket and emits "resume", not "connect", so a resumed session lost
the flag.
After a resume the next outgoing frame was echoed to debugEvents,
ws_dumper sent a DebugWsMessageEvent, the wrapper echoed that too,
and so on: infinite synchronous recursion ending in a RangeError:
Invalid string length from JSON.stringify in link.ts sendEvent. That
uncaught error made handleUnhandledErrors exit the controller.
Repro: admin sends DebugDumpWsRequest, the control connection drops
and resumes (e.g. a network blip), controller crashes on resume.
Set the flag on "resume" as well as "connect".
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Cooldude2606 <25043174+Cooldude2606@users.noreply.github.com>
Log messages share the root cause with the save details updates fixed in
the previous commit: the controller trusted host supplied instance
fields. LogMessageEvent only declares level and message in its schema,
but the Ajv instance is built without removeAdditional and the schema
does not set additionalProperties, so instance_id and instance_name sent
by a host survive validation and pass through fromJSON untouched. A host
could tag a log line with an instance assigned to another host and have
the controller store and serve it as that instance's output.
Lift the assignment check out of handleInstanceSaveDetailsUpdatesEvent
into a private method and use it from both handlers.
Strip the instance fields instead of dropping the line. A host racing a
reassignment does emit instance log lines for an instance the controller
has already moved, and dropping those loses real output exactly when a
migration is going wrong. The line is still genuine output from the host,
so it is recorded as a host level message, which grants the host nothing
it could not already log directly. Warn once per connection so a
misbehaving host cannot flood the log.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
_watchServerLogActions dereferenced two regex results with ! and
threw on null. The name regex /^([^ ]+)/ returns null when a line
classified as an action starts with a space, and the BAN reason
regex /\. Reason: (.+)\.$/ returns null for a [BAN] line with no
". Reason: X." tail. The output listener runs synchronously off the
stdout LineSplitter, so the throw becomes an uncaughtException and
takes the whole host down with every instance on it.
Repro: instance with enable_save_patching off and enable_script_commands
on; a control client sends /sc print("2000-01-01 00:00:00 [BAN] evil").
The raw print reaches stdout, parses as a dated BAN action, and the
reason deref throws.
Skip the line when the name regex misses and default the ban reason
to empty when the tail is absent. Apply the same name guard to
_watchPlayerPromote. The autosave ![1] in server.ts is safe because
its filter regex already guarantees the capture.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
A permission function runs during ingress permission checking, before
the request body is validated. UserBulkImportRequest.permission read the
raw, unvalidated message.data and threw a bare Error for an unknown
importType. That error is not a PermissionError, so _processMessage
rethrew it to the connector "message" listener, which emitted "error" on
the connector. Controller-side connectors have no "error" listener, so
Node rethrew and handleUnhandledErrors exited the process.
Repro: a control client with client-side validateSent disabled sends
UserBulkImportRequest with importType "foo"; the client gets a normal
RequestError response and the controller then logs a fatal uncaught
exception and exits.
Fix both layers: _processMessage now logs and drops an unexpected error
from the permission check (control connections only, where a response
was already sent) instead of rethrowing, and UserBulkImportRequest
throws a PermissionError for an unknown importType so it is denied
cleanly. A genuine PermissionError still denies silently as before.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
sendRcon waits for the rcon-ready event when RCON has not connected yet,
which is the same root cause as the stop() hang. events.once rejects on
error events, but _watchExit skips emitting error while the state is
stopping, and kill() sets that state before sending SIGKILL. exportData
does exactly that, so an RCON call racing the export never settles and
the request stays pending until the session closes.
Wait for exit as well, which leaves the existing RCON client check to
reject the call with "RCON connection lost" instead. Both call sites now
share a helper as the paired wait is identical.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Map Settings block parsed values.mapGenSettings, so filling in only
Map Settings failed with "undefined" is not valid JSON and the modal
would not submit, while filling in both sent the map gen settings JSON
to Factorio as --map-settings.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Deleting your own account now terminates the session immediately, so the
web UI received a SessionLost rejection and reported "Error deleting
user" for a delete that had succeeded. Use the same branch revoke does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Drop the special case that deferred terminating the connection issuing
the delete, so deleting yourself behaves the same as revoke-token and
every session of the deleted user is closed before the handler returns.
Set tokenValidAfter to the current time in createUser instead of 0.
Deleting a user and recreating it under the same name left tokens issued
before the deletion valid, as the check is iat < tokenValidAfter. Doing
it in createUser covers every path that makes a user. A token signed in
the same second as creation is still accepted, so create-admin and the
other production paths sign after the user exists and are unaffected.
The integration harness signed the control token at import time, before
bootstrap create-admin created the user. Sign it in the root before hook
instead and reuse it in link.js.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ControlConnection keeps the User object it authenticated with and checks
permissions against it. Deleting the user removed the record but left
its sessions open, so a deleted user kept all its permissions until the
WebSocket closed.
Repro: create "victim" with role 0 and connect as it. Another admin
sends UserDeleteRequest("victim"). Victim can still send
UserCreateRequest("mallory") and UserUpdateRolesRequest("mallory", [0]).
Terminate the deleted user's connections like revoke-token does. When
a user deletes itself, defer terminating the requesting connection so
the response is sent first.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The config changed handler ran updateFactorioSettings and
updateFactorioWhitelist as soon as the field changed. While the instance
is starting, stopping, creating_save or exporting_data the server is not
ready for RCON, so the update threw and was only logged. The three list
update events pushed from the controller had the same problem, except
Link._processEvent swallowed the rejection and the update was lost
without a trace.
Add _waitForSettledStatus, which resolves once notifyStatus lands on
running or stopped. notifyStatus is the only place _status changes and
notifyExit always ends on stopped, so a waiter cannot hang. The settings,
whitelist and list updates now go through it.
Nothing is applied when the instance settles on stopped, because there is
no server to apply it to. factorio.settings is written to
server-settings.json and the lists to the list files on the next start,
and enableWhitelist is read from the config when the next Instance is
constructed.
The plugin hook now runs immediately for the two config fields instead of
being chained onto the update. Every other field already invokes it right
away, and waiting for the instance to settle would hold the notification
back for the whole startup.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The close handler removed the log subscription transport from the global
lib logger, but updateLogSubscriptions adds it to the controller's
clusterLogger. Every control connection that subscribed to logs stayed
attached to clusterLogger after closing, keeping the connection alive and
running every later cluster log line through its filter.
handleDebugDumpWsRequest added a new debugEvents listener on each call
and overwrote ws_dumper, so the close handler only removed the last one.
Repeated requests stacked listeners, triggered MaxListenersExceededWarning
after 11 calls, and left the earlier listeners behind. Ignore the request
if the connection is already dumping.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
handleInstanceSaveDetailsUpdatesEvent trusted the instanceId in the
event and in each SaveDetails without checking the instance was assigned
to the sending host. InstanceSaveDetailsUpdatesEvent has host in its src
list and validateIngress only checks the source host id, not the
payload, so any authenticated host could corrupt another instance's
controller-side save records.
Repro: a host owning nothing sends the event with a SaveDetails for
instance 1002 (on another host) to inject a bogus save, or sends empty
updates with instanceId 1002 to make the delete-missing sweep wipe that
instance's save list.
Guard both paths: bail if event.instanceId names an instance not
assigned to this host, and drop the whole event if any update is for an
instance not assigned here. Matches handleInstanceStatusChangedEvent's
bogus-update warning.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
stop() waits for the rcon-ready event when RCON has not connected yet,
but nothing emits it once the process is gone, so the promise never
settles and the stop request stays pending until the session closes.
Wait for exit as well and drop the other listener when either fires.
Repro: send InstanceExportDataRequest to a stopped instance, then
InstanceStopRequest while the status is exporting_data. Instance.stop()
sees the server as running and calls server.stop(), then exportData
kills the process before RCON is up. The export finishes and the
instance reports stopped, but the stop request never gets a response.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
assignInstance sets instance.assigned_host to the new host before
sending InstanceAssignInternalRequest. If the host fails that request
the controller kept the new host id while the host never added the
instance. Assigning to the same host again then returned early as a
no-op. Repro: make the host's instances directory read-only, assign
instance 3001 to host 10 (EACCES on mkdir), make it writable, assign
again (ok, nothing happens), start (instance does not exist on host).
Now a failed send leaves the instance unassigned and rethrows.
InstanceCreateRequest applied instance.assigned_host from the request
config, creating an instance assigned to a host that never heard of
it. Repro: InstanceCreateRequest({"instance.id": 4001, "instance.name":
"c1", "instance.assigned_host": 10}). Reject a non-null value with the
same error the config set requests use.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The fieldChanged handler in the Instance constructor ran
updateFactorioSettings and updateFactorioWhitelist with only a finally
attached. A rejection from either went unhandled, and the host exits on
unhandled rejections, taking every instance on it down.
Two ways to hit it against a real host:
- Start an instance and immediately toggle factorio.enable_whitelist.
updateFactorioWhitelist calls sendRcon while the server state is still
"new" and it throws "Expected state running,stopping but state is new".
- Set factorio.settings tags to [1] on a running instance. applyAsConfig
called tag.replace outside its try block and threw
"tag.replace is not a function".
Catch and log both on the instance logger before invoking the hook, and
convert tags with String() so non-string entries no longer throw.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two writes to the same file at the same time shared one temporary name, so the second write clobbered the first's temporary and the first rename left the second call with nothing to rename, failing with ENOENT. The host does exactly that when it saves an instance config while the instance saves it too.
The temporary now carries a random tag and is created exclusively, so concurrent writers never touch each other's file. Since the temporary is always new, the mode is applied on creation and the chmod for reused leftovers goes away. A failed rename removes its temporary instead of leaving it for the next write to overwrite.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
yargs, @types/yargs, semver and @types/semver are used by several packages at the same version. Also point the remaining direct references to @swc/core and react-router-dom at their existing catalog entries.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
nyc instruments through the CommonJS require hook, so since the ESM migration (#984) it has seen nothing and Codecov has reported 0%. c8 reads V8's own coverage, which covers ES modules and child processes alike, and maps it back to the TypeScript sources through the emitted source maps.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Raise every caret range to the current latest within its major, in the
root package.json, the pnpm-workspace.yaml catalog, the per-package
package.json files and the create plugin template.
Majors taken: eslint 10, @eslint/js 10, globals 17, nyc 18,
@types/express 5, @types/node 24, finalhandler 2, pidusage 4,
style-loader 4, css-loader 7, yargs 18 (all packages), chalk 6.
Majors skipped: mocha 12 (integration suite fails under it, 59 tests,
starting with a 30 s timeout in clusteriocontroller run; kept at
^11.8.0), inquirer (10+ is a rewrite on @inquirer/prompts where
ui.Prompt.prototype.close no longer exists, so the win32 readline
workaround in create.js would have to go and cannot be verified here),
react/antd/react-router-dom/@types/react, typescript 7,
@sinclair/typebox, webpack-merge (exact pin), webpack-dev-middleware,
webpack-manifest-plugin, winston-daily-rotate-file.
Code adaptations: the two /api/stream/:id handlers in routes.ts are
typed Request<{ id: string }> because @types/express 5 types
req.params values as string | string[]; eslint.config.mjs adds
newIsCapExceptions: ["constructor"] because eslint 10.10 new-cap now
reports `new this.constructor()`.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The prepare scripts (tsc --build and webpack) are most of the install time in every job, so their output is cached between runs. tsc's tsbuildinfo tracks file hashes and webpack's filesystem cache keeps its node_modules modules, so a restored cache makes both incremental.
The lint and build-mod jobs never read the compiled output: eslint is not type-aware here and build_mod.js is plain JavaScript. They now install with --ignore-scripts.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The lockfile is not committed so the usual lockfile hash key does not work. Instead the store is snapshotted whenever a package.json changes or a week passes, and every run restores the most recent snapshot through a prefix restore key. pnpm still resolves the latest versions within the ranges; the cache only saves the tarball downloads. A composite action holds the setup so the four jobs share it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2.1.17 is the current experimental release. The lint gate still reports 0 diagnostics over 28 files with emmylua_check 0.25.1.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
checkout, setup-node, upload-artifact and codecov-action to v7, pnpm/action-setup to v6 and action-gh-release to v3. All of them now run on the node24 actions runtime. No inputs changed for how the workflows use them. pnpm/action-setup v6 is also what makes trusted publishing work with pnpm 12.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
pnpm 11 removed onlyBuiltDependencies in favour of allowBuilds and stopped reading non-auth settings from .npmrc, so auto-install-peers moves to pnpm-workspace.yaml. The native publish flow in pnpm 11+ keeps npm trusted publishing working as long as pnpm/action-setup is v6 or later.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Any valid Factorio player name can be claimed by a player joining the
cluster, so the docs now create the scraper user as #prometheus.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The metrics endpoint fans out a gather to every connected host, so
anyone who could reach the controller could read cluster statistics and
put load on all hosts at will. Scraping now needs a user token holding
the new core.controller.metrics permission. The token is accepted via
Authorization: Bearer as well as X-Access-Token because Prometheus can
only send the former. Concurrent scrapes share one gather so hosts see
at most one metrics request at a time.
The three connection gauges in controller.ts now no-op before the
controller exists, like the other gauges there, so the endpoint can be
tested against the mock controller.
Closes#1009
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Resolves the ESM conflicts in patch.ts, Instance.ts and host.ts. The
module gathering in Instance now goes through patch.loadModules with
the instance's loadedPlugins, and imports use explicit extensions.
Review changes: the existing-output check only treats ENOENT as
"does not exist" and rethrows anything else, a --zip option keeps the
output as a zip file for transfer (extracting stays the default), the
--scenario option describes that it takes a path and what the default
is, and the docs note that IPC is not available outside Clusterio.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
configuration.md was missing most options added over the last few years. Add the controller, host and instance options that weren't listed, mark the internal ones, note the validation rule under each field that has one, and fix entries that had gone stale: the tls defaults, the proxy_stream_timeout name, game port assignment from host.factorio_port_range, the three sync list modes, and the player_online_autosave_slots heading.
Plugin options live in the plugin READMEs, so add the missing ones for inventory_sync, player_auth and subspace_storage there. config-system.md gets the field definition properties it didn't list yet.
Co-authored-by: bbassie <fqv4b97f@bastiaanbreemer.nl>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Rethrow errors other than ENOENT and ENOTDIR when reading the package
spec as a directory, mark the tests that run npm as slow so FAST_TEST
skips them, and use ./external_plugins/my_plugin in the README examples.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Configure the controller, host and ctl packages as optional peer
dependencies to prevent them being pulled in and installed by plugins in
environments that don't have them. This means that a host only install
with a plugin that uses both host and controller will not cause the
controller package to be installed as well.
The web_ui package is only used when building the web interface. This
only happens during development and is thus adequatly covered by the
existing entry of web_ui in the devDependencies specification.
Removing web_ui from peer dependencies prevents it from being pulled in
by npm during a regular installation. Since it's never used in a
regular installation this removed a whole lot of bloat.
Point the invalid configuration section at the features flags and
config validators instead of throwing from the entrypoint, use const in
the examples, add a metrics hook example including a returned registry,
and mark Control.plugins as deprecated.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Plugins with only host, instance or ctl code have no reason to ship a
webpack build, yet the plugin list showed "Missing dist/web/manifest.json"
as an error for them and the controller logged a warning on every start.
Decide whether a build is expected from the plugin declaration, using the
same rule the create tool uses to decide when to generate one: a web or
controller entrypoint, or any config fields. Plugins outside that rule get
no bundle path and no error, the web UI skips loading them, and the plugin
pages show them as having no web module instead of an error or a prompt to
reload.
Fixes#1004
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
The mod pack's Factorio version ends up in mod-settings.dat. Factorio
refuses to read that file when the version is newer than the game and
falls back to default settings, which can wreck the save. Check the
versions in syncMods before touching the mods folder.
Closes#1006
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* Send only recipe notification changes on inventory download
Lua includes its current cleared recipe list in the download request. The instance plugin diffs it against the stored list and sends back the names to add and remove instead of the full list. Lua rebuilds the full list from the snapshot kept in the download record and runs the existing deserializer, so the end state is unchanged.
Also stop the serializer from appending to the stored failed recipe list in place, which grew the list on every upload without a download in between.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Test the inventory sync download handler
Covers the empty response, the full list without a snapshot, the delta with a snapshot, and the IPC registration.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Apply recipe notification deltas directly in Lua
Review feedback: the download field now always carries the delta and the deserializer applies it against the current state instead of rebuilding the full list first. The previous failed recipe names are passed in so they survive the round trip. The version check moved to the call site.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Fix failed not being cleared
* Send only the local state on download init
The current state sent with the download request no longer includes the failed names from the previous deserialization. Stored names unknown on this instance always end up in the add list, so the deserializer recomputes the failed list from that on every download and needs no previous state.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* Convert inventory sync test to ES module
Master migrated to ESM after this branch was created, so the CommonJS test failed to load on the merge commit.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Cooldude2606 <25043174+Cooldude2606@users.noreply.github.com>
Config files hold the auth secret and tokens, so the default 0o644 lets
every user on the system read them. Config.save, the ctl config written
by the controller bootstrap, ctl host create-config and the config
migration in create now pass mode 0o600.
safeOutputFile also applies the mode to the temporary file it writes,
since writeFile only honours mode on creation and a leftover temporary
from a failed write would otherwise keep its old mode.
Closes#1005
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Replace invokeHook and the per plugin on* methods with AsyncHook
collections on the controller, host, instance, clusterioctl and web
control objects. Plugin entrypoints export a default function that
receives a load context and attaches handlers to the hooks it needs.
Class exports still load through fromContext with a deprecation
warning. The create templates and docs use the entrypoint function.
This is clusterio/clusterio#921 recreated on top of master after the
ESM migration and the PluginDeclaration permissions change.
Closes#730
Co-Authored-By: Cooldude2606 <25043174+Cooldude2606@users.noreply.github.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Master migrated to ESM after this branch was created, so the CommonJS test failed to load on the merge commit.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Adds a permissions array to PluginDeclaration, a type-level Permissions
registry that plugins extend with module augmentation, and
registerPluginPermissions() called from all entrypoints. checkPermission
and the web UI hasPermission helpers now take PermissionName so typos are
compile errors. Core definitions move into a corePermissions array that
the interface derives its keys from.
Closes#987
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Co-authored-by: Cooldude2606 <25043174+Cooldude2606@users.noreply.github.com>
The current state sent with the download request no longer includes the failed names from the previous deserialization. Stored names unknown on this instance always end up in the add list, so the deserializer recomputes the failed list from that on every download and needs no previous state.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Migrate the usage of CommonJS specific code and imports to ESM. Roughly
summarised this:
- Changes all require() calls to import statement or expressions.
- Changes usage of module.exports to export statements.
- Replaces require.resolve with import.meta.resolve.
- Replaces conditional imports with conditional subpath imports.
- Replaces module === require.main with import.meta.main.
- Adds the full filename with extention to all relative imports.
- Changes all usage of __dirname and __filename to their import.meta
equivalents.
- Sets "type": "module" in package.json files.
- Removes usage of "use strict".
- Adds .ts an an alias of .js for the webpack build.
Due to differences in how require and import resolves files the plugin
entrypoints will have to include the filename and extension after this
migration in order to work.
Review feedback: the download field now always carries the delta and the deserializer applies it against the current state instead of rebuilding the full list first. The previous failed recipe names are passed in so they survive the round trip. The version check moved to the call site.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
resolveHost, resolveInstance, resolveModPack and retrieveRole treated any
integer like argument as an id and never looked it up as a name. An
instance created as "10" could not be referenced by name afterwards. The
lookup now treats the argument as an id only if an entry with that id
exists, and otherwise falls back to a name match.
Closes#886
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Values in the assetsByChunkName object are arrays not strings. This
caused the dev mode to fail after 2f4e0c4418 added an additional step
filtering out the static/ prefix from what it expected to be a string.
Use a fictional restart_test mock plugin added as a before step instead
of global_chat for the .checkRestartRequired() test. Prevents the test
from failing if the integration tests are not also run.
Also fixes the fixture not accounting for setting load_plugin causing
restartRequired to be set to true due to the integration tests having
already set it to the same value.
The ESM import of yargs does not support the singleton way of using
yargs, instead we have to call the yargs function to get a parser that
can be augmented with the options.
The .argv property has also been deprecated. Replace it with a call to
.parse() instead.
Installing a plugin used to take two steps, npm install followed by plugin
add, and both silently did their work in whatever directory they were run
from. The new plugin install <package> subcommand runs npm install --save
and then discovers the plugin through the existing loadPluginList logic,
so the plugin list is updated in the same step.
The command refuses to run outside a Clusterio installation, detected by
the package.json the installer writes or one that depends on a Clusterio
package. plugin add with a bare package name warns in the same situation.
Reinstalling a plugin that is already listed reports it as such instead of
failing. Installing a package without the clusterio-plugin keyword fails.
Fixes#1008
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Adds tests for handleCreateScenarioCommand using a fake Factorio
install with the test scenario as freeplay, for loadModules skipping
plugins without a module and rejecting a name clash with a bundled
module, and an integration test running clusteriohost create-scenario.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Creates a scenario directory with the Clusterio modules patched into it
so the modules can be used outside of Clusterio. The command reuses the
save patching by zipping the scenario, running patch() on it and
extracting the result. Defaults to the freeplay scenario from the
Factorio install in the host config and all plugins enabled there.
Module loading in Instance.prepareSave is moved to patch.loadModules so
the command and the instance share it, and findVersion is exported from
server.ts so the command can find the freeplay scenario.
Zipped scenarios were tried as an output form but Factorio does not
pick them up from the scenarios folder, so only directories are written.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Updating a single plugin can leave the installation with multiple
versions of @clusterio/lib. Passing the core package and every npm
installed plugin to a single npm update call lets npm resolve them
together.
UpdateAllRequest targets the controller or a host, requires both the
core update permission and core.plugin.update, and is gated by both
allow_remote_updates and allow_plugin_updates. The web UI Updates modal
gets an Update All action and ctl gets --all on the update commands.
Closes#910
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Covers the empty response, the full list without a snapshot, the delta with a snapshot, and the IPC registration.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Lua includes its current cleared recipe list in the download request. The instance plugin diffs it against the stored list and sends back the names to add and remove instead of the full list. Lua rebuilds the full list from the snapshot kept in the download record and runs the existing deserializer, so the end state is unchanged.
Also stop the serializer from appending to the stored failed recipe list in place, which grew the list on every upload without a download in between.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Generic error handlers such as FactorioServer.handle() and the event
handler catch in Link only log err.stack, so the ajv errors attached to
an InvalidMessage were lost unless the catch site checked for the class
explicitly. Appending the errors to the stack in the constructor makes
every stack-logging handler show which field failed validation without
special casing each one.
Fixes#986
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Accounts with core.instance.assign get a host selector, defaulting to
unassigned. Picking a host offers "Start after creation", which creates a
new save on first run when no save is copied, and together with a config
to copy it offers "Copy Save". Copying a config also offers a server
name, prefilled from the source, which overwrites factorio.settings.name
so the clone does not show up under the source name in the server
browser.
useInstanceConfig now clears the config while loading and drops errors,
so an account without core.instance.get_config just does not get the
server name field.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The built-in shortcuts are not toggleable in their prototypes and
set_shortcut_toggled only affects custom Lua shortcuts, so the state
behind them is synced instead: allow_dispatching_robots on the
character, inhibit_movement_bonus on the armor grid, and every field of
game_view_settings. map_view_settings is write-only and cannot be read.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The create instance modal can now copy the active save of the instance
whose config is copied, optionally saving the running game first, and
start the new instance once the copy is done. The new instance is
assigned to the host of the source so the save can be transferred.
Saving a running game is a new InstanceSaveGameRequest handled by the
host through /server-save, which waits for the save to finish. It is
also exposed as clusterioctl instance save-game.
Closes#405
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The toggle state of the personal roboport and equipment movement bonus
shortcuts lives on the player and was not part of the serialized player
data, so it reset when switching instances.
Closes#898
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Add option in the controller config to set where static assets are
loaded from in order to support deployments where the static assets are
server via a CDN while the API is directly accessed.
This is useful because some CDNs put low limits on how large requests
bodies can be, which causes issues with uploading large mods and saves.
The codecov/patch check flagged the new routes code as uncovered. The
unit tests never hit /api/plugins, so cover the static/ stripping and the
manifest error paths directly.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
staticRoot was set to the same value as webRoot, so the static/ prefix
ended up scattered across consumers: index.html relied on the manifest
path containing it, the export stores hardcoded `${staticRoot}static/`,
and bootstrap loaded plugin bundles from webRoot instead. Now staticRoot
is `${webRoot}static/` and the controller strips the static/ prefix from
bundle paths before handing them to the page and to /api/plugins.
Plugins that fetch assets with `${staticRoot}static/...` need to drop the
static/ part.
Fixes#980
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Same setup as established in ExpCluster, emmylua_check with factorio
typedefs generated by fmtk luals-addon. The moduleMap resolves
modules/clusterio and modules/<plugin> requires to their source under
packages/host and plugins, and the mod scaffolding under
packages/host/lua to bare sibling requires.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The largest source was compat.lua returning the setmetatable call
directly, which made emmylua infer the module type from the metatable
rather than the LibCompat class, hiding every field from consumers.
The remaining fixes are nil guards on gui element and character
accesses, type annotations for research_sync's script data and
player_auth's rcon arguments, casts where the api's read/write union
types are wider than the values at runtime, and scoped diagnostic
disables for the vendored json.lua and the version fallback code that
touches apis removed from the current typedefs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The smoke test only proves the handshake and a few core requests still
work, while compat most often breaks because the serialization of a
link message changed. Dump the registered link messages of the local
and published lib in separate processes (the lib forbids loading two
copies in one) and diff their src, dst and json schemas. A changed
message fails the test, added and removed messages only produce a
warning annotation since breaking compat is unavoidable there.
Requested by Cooldude2606 in review of #978.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Runs a smoke test in both directions between the current code and the
latest release published to npm: the host connects to the controller and
gets an instance assigned. Testing only against the latest release relies
on each release being compatible with the previous one.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Support enemy_expansion.build_base_unit_dispatch_cooldown in exchange strings
Factorio 2.1.13 added build_base_unit_dispatch_cooldown as the last
field of enemy_expansion in the map exchange string, and rejects map
settings without it. Track the full version in the reader state and
read the field for 2.1.13 and later. Adds a fixture string generated
with 2.1.14 and uses it in the load-scenario integration test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Replace version flags with versionAtLeast in exchange string reader
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
useTableQueryState returns pagination=false unless a pagination config
is passed, and SavesList did not pass one after #930, so the saves table
rendered all saves on one page.
Fixes#967
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Filter dependency resolution by the mod pack's Factorio version
The dependency resolver picked the newest release satisfying the version
range regardless of which Factorio version it targeted, so a 2.0 mod pack
could end up with 2.1 dependencies. Only consider portal releases and
local mods whose factorio_version matches the mod pack.
Fixes#965
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Move dependency resolver tests to test/messages/mod.test.js
Also update the existing local mod fixtures to carry a factorio_version
and drop enable-all-feature-flags from the live expectations: the 2.0
releases of pyalternativeenergy only list it as optional, it was only
pulled in before because 2.1 releases were being selected.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
ensure_character asserted on player.create_character() returning false
without any context (#912). The most likely trigger is the player being
physically inside a space platform hub (or otherwise stuck in remote view)
on the destination server: the character in the hub is destroyed while
downloading and the god controller ends up on the platform surface where
a character can not be created.
- Explicitly exit remote view and move the god controller off platform
surfaces before create_character; restore_position re-enters the platform
- Include controller/surface/connection state in the error message
- On deserialization failure treat the player as a temporary inventory
(sync = false) so the incomplete state is not uploaded over synced data
- Release the acquisition on leave when nothing is uploaded so the player
is not left "busy" on other instances
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Add NO_EXTERNAL_TEST flag to skip tests using external APIs
Adds an externalTest(this) helper alongside slowTest that skips the test
when NO_EXTERNAL_TEST is set and raises the timeout to 60s, plus a
no-external-test script. Marks the tests hitting factorio.com, the mod
portal and GitHub with it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Disable user verification in start-all and auto_start tests
Factorio contacts auth.factorio.com on startup when
require_user_verification is enabled, which made these tests fail
offline. Match the other integration tests by turning it off.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Ignore Factorio download directory created by host tests
The live-API download test in test/host/server.js writes a full headless
Factorio install (~140 MB) to test/file/factorioDownload and does not
remove it, which then shows up as untracked files.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Download Factorio in host tests to temp/test instead of test/file
The live-API download test wrote a full headless install (~140 MB) into
test/file/factorioDownload, which is tracked test fixture territory and
showed up as untracked files. temp/ is already gitignored and is where
the other tests put generated data.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Factorio 2.1.13 added build_base_unit_dispatch_cooldown as the last
field of enemy_expansion in the map exchange string, and rejects map
settings without it. Track the full version in the reader state and
read the field for 2.1.13 and later. Adds a fixture string generated
with 2.1.14 and uses it in the load-scenario integration test.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Factorio contacts auth.factorio.com on startup when
require_user_verification is enabled, which made these tests fail
offline. Match the other integration tests by turning it off.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds an externalTest(this) helper alongside slowTest that skips the test
when NO_EXTERNAL_TEST is set and raises the timeout to 60s, plus a
no-external-test script. Marks the tests hitting factorio.com, the mod
portal and GitHub with it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Also update the existing local mod fixtures to carry a factorio_version
and drop enable-all-feature-flags from the live expectations: the 2.0
releases of pyalternativeenergy only list it as optional, it was only
pulled in before because 2.1 releases were being selected.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The dependency resolver picked the newest release satisfying the version
range regardless of which Factorio version it targeted, so a 2.0 mod pack
could end up with 2.1 dependencies. Only consider portal releases and
local mods whose factorio_version matches the mod pack.
Fixes#965
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Web UI console rendered server level entries as bare parsed Factorio
output, while every other level was prefixed with its name. Nothing in a
line distinguished what the game wrote from what Clusterio wrote about it,
which matters more now that both modes of the filter carry a mix of the
two.
Emit the [server] prefix for those entries as well, matching what
TerminalFormat already does for the host and controller logs, and style the
level the same bold the terminal colourises it with.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The filter was a Switch showing only the label of the mode it was already
in, so with the default of chat mode it read as a button labelled "Chat"
with no hint that a Log mode existed or that anything was being hidden.
Swap it for a Segmented control listing both Chat and Log, so the mode that
is not active is always visible, and add a tooltip describing what each one
covers.
Fixes#853
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The console's chat filter kept only Factorio output parsed as an action,
which dropped the server's replies to those actions. Running a console
command with a typo in it echoed the command and then appeared to print
nothing, because "Cannot execute command. Error: ..." is a plain message
rather than an [ACTION] line. Factorio's engine level errors were dropped
the same way, so a failed multiplayer connection left no trace either.
Filter by where a line came from instead. Factorio interleaves two streams
on stdout: the date stamped console log, which carries what was done to the
server and the replies to it, and the seconds stamped engine log, which is
verbose diagnostic output. Keep the console log whole and take only
warnings and errors from the engine log.
Apply the same rule to Clusterio's own entries, which were previously kept
in full and drowned the Factorio output they were mixed with. Keep those at
warn or worse and drop the routine bookkeeping. Replayed over a development
cluster's history this drops 415 informational lines and surfaces engine
errors that were never visible in either mode.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The plugin list reported "Error loading module" for anything without a
loaded package, which includes plugins the controller has picked up since
the page was opened. Nothing had been tried for those, so reporting a
failure was both wrong and unhelpful: the fix is to reload, not to
investigate.
Report those as needing a reload instead. Plugins the controller cannot
serve at all are excluded during startup before they reach the loaded
plugin map, so those are checked for first and still count as an error.
Fixes#729
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The web interface loads plugin bundles, registers their messages and
adds their config fields once in bootstrap, and has no way to undo any
of it. A controller restarted to pick up a plugin being installed,
enabled or disabled comes back running a different set, and a controller
started in recovery mode runs none of them while still accepting control
connections, so open pages carry on sending messages it no longer
handles.
The controller already states the plugins it is running in the handshake
it sends on every connection, which is what hosts track as serverPlugins
and use to decide what to forward and which instance plugins to load.
Compare that against the set the page loaded and stop rendering on a
mismatch, asking to be reloaded instead.
The comparison is against loaded rather than enabled because loaded is
what the handshake reports and what the controller is actually running;
enabled also takes load_plugin into account, which only applies from the
next restart and would otherwise latch a permanent mismatch.
Closes#804
Address review feedback: modsByName is built from annotatedMods so the
advisories array always exists, and each source mod is iterated once with
no duplicate dependencies, so the existence check can never fail. Push the
recommended_dependency advisory directly.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G8b9dEaJtpvydzz4ttM7JH
Nothing covered whether plugins are loaded at all, or whether the messages
they declare are routed. The integration tests only loaded the first party
plugins, so what coverage existed came incidentally from testing those.
Add a plugin under test/file which exists purely to be loaded by the tests.
It is plain JavaScript rather than TypeScript so that it needs no build
step, which the plugin loader is happy with as it requires the entry point
and reads the version from package.json like any other plugin.
Covered so far: the plugin loads on the controller and on the host, its
title and version are reported, the command it adds is available in
clusterioctl, a request from control reaches the controller, a request from
control reaches a host, and an event sent with broadcastEventToHosts
arrives at the hosts.
Left for a follow up, as all three need a running instance: a request to an
instance, a Lua module reached over rcon, and log statements being tagged
with the plugin and host or instance id.
Part of #342, and a step towards #721 as core no longer needs a first party
plugin to have the plugin system under test.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An event broadcast from an instance to all hosts only reached the host
running that instance. The host side of an instance connection has the
host's own address as its source, so a broadcast aimed at hosts counts as
addressed to it, and a link that a message is addressed to handles it and
stops. The event was delivered locally and never routed onwards.
A broadcast is addressed to every link of the type it targets, so arriving
at one of them does not mean it has arrived everywhere. Keep routing such a
message after handling it locally. Only links that route are affected, as
instances and controls have no router and would otherwise throw on every
broadcast they receive.
The only links whose source type matches a broadcast target and which route
are the host and its instance connections, both matching a broadcast to
hosts, so add the missing case for that to the host router. It forwards to
the controller, which already passes broadcasts on to every other host, and
stops when the message came from the controller to begin with.
Drop the exclusion this bug had in the routing test matrix, which now
covers it along with the plugin filtering variants.
Fixes#575
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
execFile was called with shell: true and an array of arguments, which Node
deprecates in DEP0190 because it concatenates the arguments by space
without escaping anything, giving the impression they have been made safe.
The warning was printed to the console on every install.
The arguments here are already escaped for the shell by escapeArg, so join
them into the command instead of handing them to execFile to concatenate.
Nothing about what ends up being run changes.
Point the escaping test at the same form so it covers what the installer
actually does, and stops emitting the warning itself. Its several hundred
round trip cases still pass, which is what confirms the escaping is
unaffected.
Fixes#851
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
forwardRequest records a request so that its response can be sent back the
way it came, but recorded it before handing the message to the connector.
When the connector threw, because the session to the next hop was gone, the
record stayed behind for a request that had never left.
Routers answer the origin themselves when forwarding throws, so the request
was already accounted for. The leftover record then had it answered a
second time when the link cleared its pending requests on disconnect,
producing a duplicate response and "Received response N without a pending
request" at the far end.
Remove the record when the forward throws. Doing it here rather than in the
routers covers both the host, which catches and responds, and the
controller router, which does not.
Fixes#650
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A subscribe arrives as a request, so the source address the subscription
controller handles it with carries the request id used to route the
response back to the caller. That address was stored verbatim as the
destination for the subscription, so every event broadcast to it, and the
replay event sent when the subscription is created, went out addressed to
a request that had long since been answered.
A request id identifies one request in flight and has no meaning on an
event. The receiving link keys response routing off it, so events carrying
one are put through paths meant for response traffic.
Strip the request id when taking the subscription's destination from the
request that created it. Add Address.withoutRequestId for that, as the
same address is used in three places here.
The subscription map is keyed on addressIndex, which already excludes the
request id, so subscriptions were never duplicated by this and only the
address events are sent to was wrong.
Fixes#847
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
downloadFile opens a write stream before fetching, but only closed the
stream and removed the file it was writing to when the server answered
with a bad status. A request that never reached the server, or a
connection dropped part way through the body, left both the temporary file
and an open file handle behind.
Mods are downloaded through this in a Promise.all over every mod in a pack,
so one bad moment on the network scatters temporary files across the mods
directory. They are not harmless: createTempWriteStream picks the first
free name, so each leftover file permanently shifts the name used for
later downloads of the same mod, as the existing "should not overwrite
existing temp files" test describes.
Move the fetch and the body write into a try block and discard the stream
and its file on the way out. Destroy rather than end the stream, as the
content is being thrown away, and skip both if the stream has already
closed so that a failure after the body was written does not wait forever
for a close event that has already fired.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The mod portal takes the Factorio account token as a query parameter on the
download URL. When a download failed, downloadFile put the whole URL into the
error message, which ModStore then logged verbatim, printing the token into the
log at error level on every failed mod download.
Add a redactUrl helper that replaces the value of credential-carrying query
parameters with REDACTED, and use it when downloadFile builds its error.
Parameter names and the rest of the URL are kept so the message stays useful
for diagnosing which request failed; username is deliberately left intact as it
is not a secret and identifies the account in use.
Fixes#795
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
commitFilter now reads the live overrides from a ref that setFilter updates
synchronously, so committing within the same event that changed a filter no
longer sees a stale render closure. antd's Input.Search fires onSearch (before
onChange) when the clear button is clicked, so onSearch now commits the cleared
state instead of closing the dropdown, which had re-committed the still-live
typed value.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The search box clear (X) button cleared the filter live but only wrote the URL
when the dropdown closed. commitFilter now accepts an explicit value so the
clear handlers can commit right away (avoiding the stale-state race of reading
the live override in the same event), for both useColumnSearch and the Users
filter.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a live filter override to useTableQueryState: setFilter updates the filter
immediately (so every table's search filters as you type) without touching the
URL, and commitFilter writes it to the URL when the filter dropdown closes.
useColumnSearch and useUserFilter now filter live via setFilter and commit on
close, so the Users table no longer rewrites the URL on every keystroke and the
other tables gain live search.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
These were the only tables without the column search filter; wire useColumnSearch
into both name columns with URL-persisted filteredValue, matching the other tables.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cooldude's GameVersion brand was disjoint from the clean version types in
both directions, forcing safeVersion/safeFactorioVersion companions and
breaking widening from FullVersion at file-name call sites.
Make GameVersion a plain string alias: the raw, un-normalised version as the
game reads it. The stricter template-literal types (FullVersion,
MajorMinorVersion, PartialVersion) now widen into it for free, while a
GameVersion still can't be used where a normalised version is required
without going through normaliseGameVersion/integerGameVersion. Runtime
validation stays in GameVersionSchema.
- Drop safeVersion/safeFactorioVersion; ModInfo.version stays raw (verbatim so
it matches the mod's file name), integerVersion normalises via the new
integerGameVersion. factorioVersion reverts to a normalised MajorMinorVersion
since it is never part of a file name.
- ModRecord.version becomes a GameVersion so mod packs round-trip the exact
file-name version.
- ModVersionEquality.testVersion / ModVersionRange.testVersion accept a raw
GameVersion and normalise it; the equality bound stays strictly validated.
- Migration only pads clean partial versions; raw versions are kept verbatim.
- Cover the leading-zeros case: version kept verbatim in the file name while
comparing equal to its normalised form.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Through the creation of this commit, it was discovered that ModInfo.version must be typed as GameVersion due to file names. This results in wide spread propagation of the unsafe type. The full impact including on ModRecord and ModVersion.testVersion have no been implemented until a path forward is decided.
The strict major.minor jsonSchema did not improve any consuming type (the
factorioVersion field is already typed) and only complicated the fromJSON
signature, so revert it (and drop the now-unused MajorMinorVersionSchema).
Add a test for the unparseable factorio_version branch in fromJSON.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The stored ModInfo always normalises factorio_version to a major.minor (see
fromJSON), so the serialised schema can use the strict MajorMinorVersionSchema
while infoJsonSchema keeps the lenient GameVersionSchema for raw info.json.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Let useTableQueryState accept a full TablePaginationConfig (only defaultPageSize
is read) so UsersTable passes its pagination prop straight through instead of
extracting and casting it. Drop the redundant pagination: false from the tables
that just want it disabled (that is already the default). Clarify the comment on
the Last Seen online-only default to mention the onlyOnline prop, and explain why
onChange only uses the first sorter.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Address review feedback: introduce a MajorMinorVersion type with
isMajorMinorVersion, rename majorMinorVersion to normaliseMajorMinorVersion,
and type ModInfo.factorioVersion as MajorMinorVersion so already-reduced
values are not re-reduced (ModPackViewPage uses mod.info.factorioVersion
directly). Reuse the single gameVersionRegExp in both GameVersionSchema and
normaliseGameVersion instead of duplicating the pattern.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the padding-top scroll headroom with a dedicated .console-spacer
element so the pin-to-bottom keeps working without padding inflating the
measured height, and trim the style.css comments.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The resize observer measured offsetHeight, which on the content-box console
includes the pin-to-bottom padding-top. Feeding that back into the height
(and thus the padding) meant the padding inflated the measured height, so
the console could be enlarged but never dragged smaller again.
Make the console border-box so its height already includes the padding and
offsetHeight matches the dragged height, breaking the feedback. The default
height is bumped to 600 so the rendered size stays the same as the previous
content-box height + padding.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The mod portal "ApiVersion" enum was validated strictly when loading a
mod's info.json, which rejected versions the game itself accepts. Versions
are now normalised the same lenient way the game reads them: leading
whitespace is allowed, the major and minor are required, the patch is
optional, any trailing content is ignored, and numbers are read as integers.
This applies to both a mod's version and its factorio_version, validated by
a regex schema rather than the fixed enum. The enum is no longer used to
validate versions at all - the mod portal is the source of truth, so its
error is now surfaced when a version is unsupported. The known-version list
kept for the web interface fallback gains 2.1.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rename the deprecated Modal `destroyOnClose` prop to `destroyOnHidden`
(antd 5.25+) across the web UI.
Skip a version-tree child equal to its major.minor group key in
InputVersion, which triggered "Same value exist in the tree" warnings when
the version list contained bare major.minor entries (the ApiVersions
fallback shown before the version fetch resolves).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Configures an instance with the "experimental" channel and starts it,
exercising channel resolution end to end across the controller and host.
The resolved version may or may not be installed depending on the host's
Factorio install, so the test asserts the instance reaches a terminal state
rather than a specific outcome.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cover fetchLatestReleases and resolveReleaseChannel, the controller's
latest-releases cache with its on-disk offline fallback, the
LatestReleasesRequest round-trip, and FactorioServer.setTargetVersion.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The instance console was locked to a fixed 300px height. It can now be
drag-resized vertically (native resize handle), and the chosen height is
persisted with useLocalStorage so it is restored on the next visit.
Both the height and the scroll-anchor padding are driven by a
--console-height CSS variable, so the pin-to-bottom behaviour keeps working
at any size.
Closes#938
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
An instance's factorio.version can now target a release channel such as
"stable" or "experimental" (any key from factorio.com/api/latest-releases)
in addition to "latest" and specific X.Y / X.Y.Z versions.
The controller proxies the latest-releases API through a ValueCache and
persists the last successful response to latest-releases.json, serving that
copy when the API is unreachable. On instance start the host resolves a
channel to the concrete version it points to and feeds it into the existing
checkForUpdates/findVersion path, so a channel auto-installs and runs like a
pinned version. The version selector in the web UI lists the available
channels alongside "latest".
Also clarifies the host log/error when a newer version cannot be downloaded
automatically because the host uses a direct (single-version) Factorio
install rather than a versioned directory.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the absolutely-positioned a::after overlay with display:block on
the link. The overlay covered sibling cell content (the restart-required
icon), swallowing its hover so the tooltip never triggered. A block link
fills the cell width without overlapping siblings.
Also skip rendering the assigned-host link when an instance has no
assigned host, avoiding a dead /hosts/undefined/view link.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Stretch the navigation link over the whole table cell (a position:relative
cell with an ::after overlay) so the entire name cell - and the Instances
Assigned Host cell - is clickable and right-click "open in new tab"-able,
not just the text. Also force color:inherit on the sidebar and account links
so the focus left behind by a middle-click no longer shows them blue.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Add cluster overview landing page
Add an Overview page at / (top of the sidebar, filling the previously
blank root route) summarising the cluster: hosts connected, instances
running, users online, aggregate CPU/memory/disk usage, controller
uptime, a recent-errors panel (LogConsole), and a link to Grafana when
configured. Each card is gated by the relevant permission.
Adds a controller.grafana_url config field, read on the page via
ControllerConfigGetRequest, for the Grafana link.
Closes#928.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Add saves card and tidy the overview grid
Add a "Saves loaded" card (loaded vs total save size) to the overview, lay
the cards out four-per-row on desktop, and add a compact date format option
to MetricRelativeDate used for the controller uptime card.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Change order of cards and fix permission check
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Cooldude2606 <25043174+Cooldude2606@users.noreply.github.com>
The instance list shows the public connect address but the instance detail
page did not. Add a Public address row (host:port) with a copy button, and
lay the details out 2x2 so the extra row stays balanced.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a "Saves loaded" card (loaded vs total save size) to the overview, lay
the cards out four-per-row on desktop, and add a compact date format option
to MetricRelativeDate used for the controller uptime card.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the satisfies-ColumnsType + .map(applyColumn) approach with inline tableState.sortOrder()/filteredValue() getters set per column, so the column arrays stay plain literals (no satisfies, no constraints; UsersTable stays any[]). Move useColumnSearch into util/ next to useTableQueryState.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make the Clusterio logo in the top bar a link to / so it lands on the overview/dashboard page.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Move the instance console chat/log toggle persistence into a generic useLocalStorage hook that defers the write so it never blocks the toggle update.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add an Overview page at / (top of the sidebar, filling the previously
blank root route) summarising the cluster: hosts connected, instances
running, users online, aggregate CPU/memory/disk usage, controller
uptime, a recent-errors panel (LogConsole), and a link to Grafana when
configured. Each card is gated by the relevant permission.
Adds a controller.grafana_url config field, read on the page via
ControllerConfigGetRequest, for the Grafana link.
Closes#928.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
UsersTable declared its columns as any[], which masked a few type
mismatches. Type them as ColumnType<lib.UserDetails>[] (matching
InstanceList) and fix what surfaced: the Roles column and onFilterUser
value types (antd keys include bigint) and useUserFilter's filterDropdown
prop type (FilterDropdownProps instead of a hand-written string[] shape).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Every render site of InstanceList and UsersTable already passed persistState,
so the prop and its uncontrolled fallback branch were dead config. Remove the
prop and the conditionals so these tables always persist their state to the
URL. No runtime behaviour change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Console "Chat/Log" filter toggle on the instance page now persists its
state to localStorage, so turning it off (to see full logs) is remembered
across reloads. It still defaults to on (chat only) when nothing is stored.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Following discussion on the PR, the sort value now packs the direction as
`<field>.<asc|desc>` (e.g. user.sort=name.desc) instead of a separate
`.order` param, page size uses `.size` instead of `.pageSize`, and the
reserved system keys are sort/page/size. `.sort` is read tolerant of repeats
so multi-column sort can be added later without changing the URL shape.
Filterable columns must not use the reserved system names. Removed the
web_ui unit test since it required importing from the built dist.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
In-app navigation was done with JS onClick -> navigate() handlers instead of
real <a href> anchors, so the browser's right-click "Open in new tab" (and
middle-click / ctrl-click) was unavailable. (closes#929)
- Sidebar nav items and the account "user" entry now render their label as the
existing Link component (a real anchor via react-router useHref), keeping the
Menu onClick for plain left-clicks; stopPropagation avoids double navigation.
- The "back to instances" alert action becomes a Link.
- Table rows (Hosts, Roles, Plugins, Instances, Users, the user instance-stats
table, Mod Packs): the name cell becomes a real Link (color inherit, so it
looks unchanged) which carries the right-click context menu, and a new
useRowNavigation hook handles the whole row — left-click navigates in place,
middle-click / ctrl-click open the target in a new tab. The hook ignores
clicks landing on a real link inside the row so the anchor handles those.
Genuine actions (download/delete/modify/logout), post-create navigation and
external links are left as-is.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Tables such as Hosts, Instances, Users, Roles, etc. now write their sort
order, column filters and pagination into the URL query string, so reload,
link-sharing and browser back/forward restore the same view (closes#927).
Every parameter is namespaced with the table name so the URL is
self-describing and two tables can share a page without colliding, e.g.
?user.sort=name&user.name=bob&user.admin=true or ?instance.name=foo.
Adds two reusable hooks, exported from @clusterio/web_ui so plugins can use
them too:
- useTableQueryState: makes an antd Table controlled and round-trips its
sort/filter/pagination through the URL via useSearchParams. The pure
serialisation helpers live in util/tableQuery.ts and are unit tested.
- useColumnSearch: a column-header text-search filter (matching the Users
table style), persisted through the same filter mechanism.
Applied to Hosts, Roles, Plugins, the instance list (standalone and on the
host detail page), Users (main table and the online-players list on the
instance detail page), instance saves, the per-instance user stats, the mod
packs and stored mods tables, and the subspace_storage storage page. The
storage page's resource search moves from a standalone box into the Resource
column header, keeping its regex matching.
The bundled user filter persists as separate readable params (user.name,
user.admin, ...) via a per-column FilterCodec rather than one JSON blob.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reintroduce a slim top bar to host the menu toggle, branding and the
account menu, which gives the hamburger a natural home now that the
header is gone. The bar is more compact than the original header and
uses the dark sider colour for better contrast.
On phone screens the sider becomes an overlay Drawer (opening below the
bar and closing on navigation), following normal hamburger conventions
instead of shifting the content across. On desktop the same toggle
collapses the sider, with its width animation disabled to avoid the
content reflow glitch while resizing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remove the top header and move its contents into the sidebar, which is
now sticky and full height: logo and version at the top, navigation in
the middle, and the account menu pinned to the bottom (opening to the
right).
The sidebar is rebuilt with antd Flex, Typography and Menu so almost all
styling comes from antd, reducing the custom css to a single sticky rule.
The nav menus also use theme="dark" to match the sider and fix the low
contrast active item.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add variables specifying the repository and branch that the changelog
script parser operates on in order to make it easier to use this script
on other projects.
These packages are obsoleted by the promises API to the built-in Node fs
module. The fs-extra package is also causing problems with broken APIs
that don't behave the same as Node's fs module, and throwing errors
without code properties on them.
The Node.js documentation's examples all use this prefix, and there are
now builtin modules in Node.js only accessible via it. Add support for
using it in lib code shared in the web_ui.
A lot of code is repeated in various different ways in order to handle
streaming content to files without overwriting them, whether it's from
web resources or local streams.
Refactor the handling of these streams to all use the same utility
function for creating a write stream, and for the places downloading
urls use the same downloadFile function.
In the future, removing exports is likely going to be a feature. Handle
the case where the export data is removed from a mod pack in the locale
and prototype metadata stores.
If the unsubscribe function is called multiple times the indexOf
function will return -1 which will end up removing random other
subscription callbacks. Prevent the incorrect callabck removial
in this case.
The handling of exported data such as locales and icons in web_ui was
based on a model where there is only one set of data exported per
cluster. This model changed to have one set of exports per mod pack
when mod packs were introduced, but the existing web_ui code was
bodged to use exports from a "default mod pack" on the controller.
Refactor the code loading exported data to take the mod pack as an
argument for selecting which exported data to load. The different kinds
of prototype metadata that can be loaded is consolidated into one
`useExportPrototypeMetadata` hook. The logic for displaying icons is
put into a `FactorioIcon` component. And the obsolete
`useExportManifest` hook is removed.
These changes allow the `/icon-reference` page to display exports on a
per mod pack basis. Which has now been linked on the bottom of the mod
pack view page.
Add script which uses GitHub's REST API to fetch issues and pull
requests that have changed since the last release, parse out the
changelog entries from merged pull requests, and print a formatted
version of the changelog suitable for copying into the release
description, the CHANGELOG.md file and the Discord announcement.
The script expects Pull Requests to have a `### Changelog` section in
them with a code block containing changelog entries. As an added bonus
the script can parse out `#<number>` references so that you do not need to
write it out in the ugly `[#id](long-url-with-id-repeated)` format.
Updates documentation and pull request template to describe the
changelog format and usage of the formatting script.
Includes improvements from review and a second look.
Use the defines.prototypes data structure from the game to find which
prototype sub-types belong in the same base type, and export icons based
on these base type.
This renames the planets category to space-location, signal to
virtual-signal and plits item up into item and fluid. And changes the
exported metadata to include prototypes with no icons, as future code
might want to use it to for example enumerate all item prototypes.
These were added to as a diagnostic in case the logic turns out to be
broken. They should not be possible to reach as any value produced by
labelsToKeys will not cause the error conditions to occur.
The a and A felt lonely being the only letters included. Add the other
letters in the ASCII character set as there was no good reason to leave
them out.
Fix incorrect suite name and attempting to test no version matches using
the "latest" version which always matches in the tests for the
FactorioServer.checkForUpdates method.
The logic for turning a key back into the labelles that defined it would
not account for the labels values possibly including a comma or equal
sign, which in turn broke the logic for handling these metrics.
Fix by properly parsing the keys.
The assumption with the subscription system is that new updates will
always increment the updatedAtMs value to something greater than what
has been seen before. This might not happen if the system clock is
adjusted, or updates are sent so fast that Date.now() returns the same
value.
Fix these edge cases so that the timestamp is always incremented.
The useEffect updates list has the major pitfall that passing an object
via for example a prop like foo={{ bar: 1 }} to it causes it to to
detect it as a change on every re-render due to the newly created object
comparing unqual to the previously passed in object.
Unpack the object to prevent the identity from changing every time the
metrics update on the Host page.
If an error occurs trying to fetch the list of Factorio versions log a
warning and then return an empty list instead of preventing the system
from operating.
The FactorioServer.checkForUpdates function would wrongfully assume that
the version running would always be listed on Wube's archive page, and
throw an error if this was not the case. Ignore version checking logic
if no suitable latest version could be found.
A circular import caused code to attempt to use symbols before they were
defined. Due to the --dev mode on the controller running with different
chunck boundaries this only appeared in the --dev mode.
* #793 - Differentiate RCON output
* -removed <div>
-simplified animation logic.
-removed box-shadow in favor of outline
* -remove Rube Goldberg machine
-removed un needed .css rule
* removed redundant null check
* added an extra line (2 spaces between imports and rest of file)
---------
Co-authored-by: Solar <solar@homelab.local>
The lib library was written with the assumption that there's only one
copy of it in the whole application and everything imports that one
copy. Unfortunately there are quite a few circumstances were package
managers end up installing multiple copies of lib and then different
parts use different versions of the library.
This causes Clusterio to break with strange and hard to track down
errors. Most notably permissions and link messages for a plugin may get
registered to one copy of lib while the controller uses a different copy
and thus does not see or register that what the plugin added exists.
Detect this condition when importing @clusterio/lib and throw an error
when it has been imported twice in order to prevent the broken
installation from causing future hard to debug issues.
The CI often fails with cryptic errors that would be much easier to
debug if the logs from Factorio was provided. Upload the cluster log
that was written during the tests which includes Factorio logs so that
these can be inspected after a test failure.
The migration from phin to fetch in #755 was hampred by the lack of an
option to set the CA when using fetch. This meant places that used phin
to talk to the controller with the tlsCa option could not be migrated to
use fetch.
Remove the tlsCa option to unblock this dependency. The replacement for
this option is to set the NODE_EXTRA_CA_CERTS environment variable to a
path to the certificate file instead, which works with fetch.
Some of our scripts assume a POSIX sh like shell to work, and writing
any sort midly complex script that works with both cmd and sh is next
to impossible.
Fortunately pnpm has an option to emulate a sh like shell on Windows.
See https://pnpm.io/cli/run#shellemulator for more info.
Ctl's commands.ts was starting to grow rather big, and future features
and refactors are only going to make it bigger. Split it up into
separate modules grouped by which category of commands it serves.
* Fix tests failing due to missmatched Factorio detection
When the configured factorio directory is full of junk then the tests
use the first directory it finds for one of its tests, while the server
code uses a more elaborate logic to figure this out.
Turns out this is complicating things more than needed. Simply trying
each entry in the directory and seeing if it works when treated as a
directory always work, regardsless of symlinks, junctions,
reparse-points or whatever else filesystem magic there is.
* Skip tests requiring Factorio if it's not present
Prevent confusing test failures by skipping all the tests that require a
working Factorio installation to sucteed, and print a warning at the end
of the test suite if tests may have been skipped due to this.
* Handle ENOTDIR when determining Factorio version
If the path to the factorio install contains a component that is not a
file then ENOTDIR may be returned on POSIX like systems. Treat this the
same as ENOENT so that detection logic works when used on a file.
---------
Co-authored-by: Hornwitser <code@hornwitser.no>
* Enhance export functionality and add icon reference page
- Expanded exported assets to include additional spritesheets and metadata for recipes, signals, technologies, planets, qualities, and entities.
- added logging metrics to game logs
- Introduced a new Icon Reference page to display available icons with filtering capabilities.
- Updated ExportManifest documentation to reflect new asset categories.
* Add support for exporting static icons and update metadata handling
* -added data export snapshot metrics
* Fix HTML attribute syntax in IconReferencePage component
* Applied PR sugestions:
-removed un needed x === "string"s
-removed isntancename / modpack from icon reference and export
-logging per line on export
-fixed variable names
-linting
* linted
* Shows "no export data available" when no export happens
* -single sprite sheet
-removed static icons with no prototype
* Fix formatting
* Rename types to allowedTypes for clarity
* -removed duplication check (redundant)
-fixed bug overwriting metadata with same name from previous categories
* Rename asset keys for consistency in export validation test
---------
Co-authored-by: Solar <solar@homelab.local>
* Refactor UserManager to use Datastore
* Rename lib.User to UserDetails also adds to/fromUserDetails
* Fix existing tests
* Add user class tests
* Add tests for finaliseUsers
* Move HostDetails to own file with tests
* Move InstanceDetails to its own file with tests
* Rename HostInfo and add tests
* Rename InstanceInfo and add tests
- Also changes the names of the host properties related to instances
- Also removes the need to specify the location of InstanceInfo
* Make json round trip test validate schema
* Add InstanceManager
* Fix lint errors
* Extra tests
* Add final test coverage
* Add one last test to InstanceManager
* Response to comments
* Use generator rather than return anonumous generator
Co-authored-by: Hornwitser <github@hornwitser.no>
* Fix lint issue with generator
---------
Co-authored-by: Hornwitser <github@hornwitser.no>
* Add ModDependency and version types
* Add warnings for mod dependencies
* Add warning for wrong factorio version
* Add fix issues button
* Add automatic dependency resolution and download
* Add abilty to check for updates
* Add extra test coverage
* One extra test
* Fix copy paste error
* Template type for load plugin config
* Add plugin update checking
* Add restart icon to instance list
* Add tests
* Log package name when failing to read package json
This means inventory updates in low-volume clusters might be delayed up to 1 second more, but ensures we always have the unix time signal available, even in clusters that don't have anything in the inventory yet. Resolves#610
* Fix rendered more hooks error when refreshing hosts page
* Fix conditional render on controllerPage
Doesn't cause any issues that I know off right now, but changes to antd internal code could change that so it feels better to be sure.
* Event types for controller
* Clean up ctl event emitters
* Event types for server
* Event types for lib
* Remove TypedEventEmitter
* Event types for webui
* Fix compile issues
* Small cleanup of connectors
* Add max level select to logs
Closes: #745
* Add log filter for server actions
Closes: #744
* Respond to comments
Change name of component, clean up onChange anon function
* Move saveJson to util file
* Add button to download ctl config
* Add icon and tooltip to describe what the button does
---------
Co-authored-by: Danielv123 <danielv@danielv.no>
Some mods, for example cubium, includes bogus folders like __MACOSX in
addition to the root folder where everything is located. This only
works by chance as Factorio will check for the folder of the first
entry in the zip file and use that.
Match Factorio's logic in findRoot so that these mods work with
Clusterio.
Preventing loading just because a plugin is missing would cause the host to become inaccessible if something went wrong when managing plugins through the remote interface
Requests of the v1 format is automatically upgraded and placed in a new section. Filters in named sections are not synchronized to avoid messing with non-player entities on servers you join. Synchronizing logistics groups falls under the scope of another plugin.
Fix whitelist remove command with tests. Tests check that set-banned and set-whitelist will send rcon commands to running instances and that it is case insensitive.
By using sync file writes it prevents interrupts between the state change and the server process starting. Also adding error catching to reset the state following an error writing to the files.
Fixes: #676
When the config system was ported to TypeScript the "instance" location
was accidentally added to the type of available locations for access
control. This has never been a valid location as it was deemed too
complicated to work with. The correct location for instances is
"controller" and "host" as these are the locations the configs are
handled.
If only the config of an instance is updated the dirty flag of the
instance is not set, which results in changes being discarded when the
controller shuts down afterwards.
Fix by always triggering the update logic when an instance field is
changed.
Provide ease of diagnosing credential issues by accurately logging if
the credentials were sourced from the instance config, the host config,
the controller config, or if no credentials were found.
Prevent browsers from automatically autofilling the password entry box
for the Factorio credentials by setting the autocomplete property to
new-password on the password input.
Add config entries to controller and host for the Factorio username and
token to use when talking to the Factorio APIs. If set an username and
token is not set in factorio.settings then these settings are used,
making it easy to configure one set of credentials for a cluster/host.
For running more public provide your own hardware clusters, there's an
option to disable controller sharing the credential with the hosts so
that each host have to set their own credentials.
Implement a credential option to config entries which is a list of
locations the config entry can be read from. When specified and the
option is a string field this gets rendered as a password field in the
config UI. If the credential is not readable from control the value of
the fields is cleared when the UI is refreshed.
Rewrite shutdown logic for Factorio servers to send a /quit command via
RCON for both Linux and Windows when stopping a server to avoid having
two confusingly intermixed code paths for Windows and Unix systems.
This makes shutdown on Windows tell clients the server shut down instead
of abruptly stoping to respond.
Newer versions of Ant Design requires either dataIndex or key to be
defined on columns for sorting of the tables to work. Add missing key
entries on columns.
Support the for (const [k, v] of datastore) syntax which is commonly
used with maps by adding a Symbol.iterator function returning the
appropriate iterator.
When passing methods around they lose their binding to this, which means
the fromJSON methods no longer has this be their class. Fix by binding
the methods to the class they belong to before passing them on to the
datastore.
Added "builtInMods" array that contains a list of built-in mod names that we check against to not show an error and remove ability to delete those mods from mod packs
Fixes#581
When renaming a file over another file the operation is only atomic if
both the source file and the target file is in the same directory. Fix
safeOutputFile writing the temporary file to a different directory than
the target file due to the path separator not being included when
creating the temporary file path.
If a factorio dir is configured in config-host.json then use that in the
tests. This removes the requirement of having a factorio installation
in ./factorio or linking to one there when running the tests.
The host.controller_url config is required to end with a / for the url
joining logic to work. Fix code added #619 setting this to a value not
ending with a /.
Remove some assumptions that prevent subscription listeners from working on an
instance. This means allowing the host to forward the event from the instance
to the controller; correctly storing subscription listeners by address rather
than by connection; and a bunch of network and event stuff to correctly clean up
a listener.
Remove the assumption that all subscriptions events contains the updates
property. This meant fixing inconsistent types; always checking if the property
existed; fixing lots of the tests that made incorrect assumptions.
Minior fixes:
- Fix double firing of listener callbacks on first connection.
- Fix synced being set to true too early on first connection.
- Fix edge case interaction between last response and last snapshot for values
with updated set as 0.
Also change the the output made during the start of tests to indicate something
is happening when start up is slow and allow for silent testing to reduce the
amount of text output to the console.
Lack of typing was resulting in admin name being serialized to storage as a number and breaking websocket communications protocol due to type mismatch: expected string, got number.
Fixes#536
The @swc/core module depends on optional dependencies being installed in
order to get the correct native binding's package installed. Remove
--no-optional flag passed to pnpm to install these.
The usage of shell: true means that no escaping is done on the arguments
passed to the commands invoked by the installer. This leads to the
installation failing if the user for example puts a space in the
username for the admin account.
Fix by escaping the arguments passed on command lines. This is
unreasonably complicated on Windows 😡.
My plugins used the antd pageheader more than base did. I did not notice that these features dissappeared when we migrated from antd to our own implementation - this brings them back
Input components allow us to customize the config field editing experience in plugins, but due to an oversight this did not include object definitions.
Currently the web interface implements configs by pulling the state directly in the components where they are used. I need to access the instance config for a plugin, which would require me to duplicate parts of the instance config retrival code.
Instead, I implement it here under lib. This allows 3rd party consumers to get immediate access if we decide to implement a subscription model for configs in the future.
Use the settings prototypes to coerce any incorrectly typed mod settings
to the correct type either by casting or by replacing the value with the
default value from the prototype.
This will fix cases where an update to a mod changes the type of ono of
its defined settings.
Due a bad order of operations when a host connection was closed the
event handler listening on "close" to remove the host from the
hostConnections mapping was added after the HostConnection's event
handler listening on "close" and invoking plugin hooks in response to
the host going offline. This in turn causes plugins that send
broadcasts in repsonse to thos hooks to throw an error due to the
broadcast sending code assuming connections that exist are valid target
to send messages to.
Fix by ordering the listener for the removal of the host connection
before the listener invoking plugin events.
Store and use a cache of the loaded ModInfo data to mod-info-cache.json
in the mods directory during startup. For simplicity this does not
cache mods added to the controller or host after startup, these get
added to the cache on the next startup.
Docement the adaptations I've made to the publishing procedure which
uses a less dangerous publish command and removes the dist folders
before publishing to avoid accidental upload of stale build files.
If a bad download or file transfer causes a mod in a host's mod folder
to become corrupt the attempt to load the mod will throw an error that's
not the file not existing. This in turn causes starting the instance to
always fail as the mod is never attempted to be re-downloaded.
Fix by ignoring errors attempting to load mods already present in the
mods folder that aren't in the mod store so that a new download is
attempted after that fails.
It's possible to set the .icons to an arbitrary key value mapping
instead of an array and the game will still accept it. Convert the
icons to an array if it's an object mapping to handle this case.
Note that this doesn't preserve the order the game reads the keys in
because JavaScript reorders numeric key indexes on objects differently
to how the game orders them.
Implement the shorthand notations that Factorio recognises for colours
where individual components may be left out or the colour specified as
an array of values instead of an object.
Build and minify the Web UI using swc instead of babel, tsc and terser.
This results in a significant speedup of the build process as it skips
checking types and uses native code to do the minification.
To prevent accidental type errors being introduced the TypeScript
compiler is still used to perform type checking on the Web UI files.
Due to peer dependency resolving shenanigans the swc/core package had to
be added to the workspace, controller and web_ui to avoid pnpm creating
a duplicate install of webpack that break the build.
Add back unqualified tsconfig.json files to each package that references
both the Node.js build and the browser build. This makes tsc --build
compile both builds when the variant to do is not specified.
Distinguish the Node.js build in the same way the browser build is with
the .node suffix to the tsconfig files it uses. This means
tsconfig.node outputs to dist/node and tsconfig.browser outputs to
dist/browser.
Use a separate folder for the TypeScript build for Node.js in the dist
folder. This makes each type of build consistently have its own output
folder instead of being interleaved with the Node.js build.
Use dist/browser as the output of the browser oriented TypeScript build
of packages instead of dist/web/build. This distinguishes it better
from the Web UI bundle which is put together with webpack.
Rename the config files for the browser build of modules to better
communicate that it's purpose is to compile code for a browser
environment rather than the Web UI in particular. This also aligns with
the "browser" option in package.json, the existing browser.ts entrypoint
in lib, and how the browser environment is typically referred to.
When the level of a technology changes either by setting level
directly or by setting researched to true, the saved progress of the
research is reset only if the technology is not the current research.
If during the initial synchronisation with the controller an instance
has progressed on a multi level research, it's the current research, and
the controller's level of that research is higher, then the level is set
to the higher level and the game keeps the progress. This in turn
results in all of the progress made on the previous level being added to
the higher level.
Fix by setting the progress to zero when changing the level or
researched status of a technology that's the active research.
Throw an error if an attempt is made to check for a permission that does
not exist in the Web UI to prevent future bad permission checks going
unnoticed.
Fix attempts to check for permissions that don't exist in the Web UI
that has gone unnoticed because the check still succeeds when the
account has the cluster admin permission.
Follow the same pattern as all the other subscribable lists by having a
separate .list and .subscribe permissions for saves. This also fixes
the save list not being visible due to a typo in the permission in the
Web UI.
Without a unique per page key set react will try to reuse the existing
state and components of the previous page and update them to match the
new page. This causes unexpected behaviour as components get turned
into into a Frankenstein abomination of the previous page and the
current one fused into one.
Fix by setting a key on the top level fragment of the page which in turn
forces React to re-render the whole page whenever the user switches to
another page.
If the value in a subscribed list is updated while the updatedAtMs
timestamp is kept the same then the caching logic of the subscription
system will cause updates to be silently dropped. Log a warning if a
subscriber catches this happening to help tracking down such cases.
Always send the instance details update when an instance is reassigned,
not just when it is unassigned. The idea for not sending an update when
assigned to a host was that the host would send an update on the status
that in turn updates the instance details. However this is not actually
the case; when a stopped instance is reassigned its status does not
change so the update is ignored.
Set the save's updatedAtMs timestamp when clearing the save list of a
reassigned or deleted instance. This fixes the Web UI not updating the
saves list to remove the saves when an instance is reassigned.
If a link is closed while the prepareDisconnect action is in progress
the link will still try to signal the disconnect ready over the socket
when the prepareDisconnect action completes, which throws due to the
socket no longer being present. This error is not caught due to the
catch happening before the finally on the promise and triggers an
unhandled rejection.
Fix by rearranging the finally to come before the catch on the promise
so that errors are always handled, and checking that the link still has
a session before attempting to send the disconnect ready signal.
When using .finally on a promise a new promise is created which calls
the handler passed to it when the the original promise is resolved or
rejected, and resolves or rejects with same value as the original
promise. This means if the rejection of the new promise that was
created by finally is not handled then an unhandled promise rejection
will be raised.
Fix by making sure the promise rejection from the finally promise
created in subspace_storage's handleUpdateStorageEvent is properly
handled.
When messages were forwarded they were sent using the low level
connector .send method which accepts a message and sends it over the
link. This method sends the message as is with no modification which
means that at no point during the forwarding logic was the message
sequence number that originated from Link A renumbered to the correct
sequence number to send out on Link B which it was forwarded on.
This in turn caused the logic which drops acked messages from the send
buffer to break which in turn meant the resume could cause duplication
of message and/or messages to be lost. With enough duplicated messages
the resume of one connection could cause other connections to drop and
also initiate resume.
Fix by renumbering the sequence on forwarded messages. In the future
the sequences sent and received should also be checked to make sure this
kind of problem does not happen again, this is not done here because
partially updating a running cluster with sequence problems to one that
checks and enforces sequences numbers will be problematic.
When host and instance id in Address was reordered in 1e66f11 the
toString method Address was updated to account for this. Fix toString
showing host addresses as being instance addresses and vice versa.
When a plugin broadcasts an event to instances there may be hosts and
instances in the cluster which does not have the plugin loaded. This
would cause an error to be logged for every instance which does not have
the plugin loaded.
Filter out broadcast events sent to hosts and instances such that
instances and hosts that does not have the event do not received them.
If during the startup of an instance the Factorio server is started up
to create a new save and the host shutdown in initiated then the
instance will be commanded to stop while the FactorioServer state is
"create". This was was not handled by Instance.stop leaving the
child_process running after the host is done shutting down.
Kill the FactorioServer on stop if the current state of it is stopping
or create to make sure the server actually stops.
When a request is forwarded from a virtual link to a WebSocket link as
is the case when an instance sends a request to the controller the
origin for the request is a virtual link. Handle this case in
_clearPendingRequests instead of throwing an assertion error.
Track the number of requests currently awaiting a response from a
request handler both from interally issued requests and requests
forwarded from other parties.
Fix installer incorrectly exiting with a conflicting options errors when
the --no-download-headless option was used in conjunction with the
--factorio-dir option.
If a plugin only has entrypoints at the controller level the config will
skip adding a "load_plugin" config option for the given plugin to avoid
making options that do nothing. If such a plugin is added to a host it
will still reach loadPlugin which in turn tries to check the load_plugin
config and throw an error on startup due the field being missing.
Skip trying to load plugins without an entrypoint at the instance or
host level in hosts before checking the load_plugin config field.
Add a note about double checking which version of Node.js is installed
to avoid accidentally updating on an outdated version, and fix migrate
arguments.
Add suffixes like Ms and Ns to variables whos time unit is not in
seconds. This is aligning with a new code style of putting units
on variables that are not in SI base units.
The logical order of clusterio's components are in most places
controller, host, instance, control. It is confusing that Address
breaks this convention and I do not remember I wrote it that way.
Reorder and renumber the host and instance members of Address to
be consistent with most other places.
The entrypoint of plugins were changed in 50f12c7 and b0f6c95 but the
installer's loading of plugin info to get the correct plugin name was
not updated for these changes. Update installer plugin loading to use
the correct entrypoint.
The place for storing mods was changed from sharedMods to mods when mod
management was implemented. Rename the old sharedMods folder to mods
when the migration script is invoked.
Change the expected exported name of plugin declarations to be "plugin"
at the main entrypoint instead of the default export. This makes the
plugin's code organisation more flexible especially when using CommonJS
JavaScript modules as they can export additional symbols from the
plugin declaration file.
Implement remotely initiated stopping and restarting of the controller
and hosts. Restarting is implemented by exiting with a non-zero exit
code and relying on a process manager to start the procress again. To
prevent accidentally trying to restart on a system that doesn't have a
process manager present the --can-restart command line option needs to
be passed to enable the restart operation.
The create.js script in development mode used to use hard coded paths to
the other cluster commands (ctl, controller etc.). This means that in
dev mode a cluster could only be created from the root directory of the
project.
By using relative paths one can also create a cluster in a subdirectory.
For example by using the following:
mkdir test-cluster
cd test-cluster
node ../packages/create --dev
Record the time a player was first seen joining an instance in
PlayerStats. For existing stats this value is set to the last
joined time when it is first loaded.
Add custom input component support to the Web UI config in order to
provide a user friendly dropdown selector for the Mod Pack that is
configured as default on the controller and overridden instances.
Provide the kernel, machine, CPU Model, hostname, and Node.js version
fields to SystemInfo and display them in the Web UI. These were added
as metrics exposed to prometheus in b46efeb.
There are more pieces of information about a system than just its CPU,
Memory and disk metrics. To avoid having to build a separate collection
and subscription for metrics and system info, rename the existing system
metrics to system info in anticipation of providing additonal static
system information such as CPU model in it.
If the controller is behind a proxy then the socket.remoteAddress will
be that of the proxy instead of the end client that connected. In this
case the X-Forwarded-For header will contain the address of the end
client, but this should only be relied upon if the party connecting is a
trusted proxy, otherwise a malicious user could spoof the IP address
that is logged by providing a forged X-Forwarded-For header.
Implement a trusted proxy list consisting of a configured list of IP
addresses and/or CIDR blocks and use the last address[1] in the
X-Forwarded-For header if the remoteAddress address of the connecting
party is in the trusted proxy list.
[1]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For
Split out host specific data from RegisterHostData that is not related
to the establishment of a connection to the controller into it's own
HostInfoUpdate event. This allows this data to be updated when the
config values they are taken from are changed.
The agent string was inspired by the HTTP User-Agent header to serve as
a way to identify what's connecting to the cluster. While there may
have been vision of other software connecting to a controller than the
Clusterio provided ones this seems highly unlikely to ever happen.
Remove redundant agent string for now.
Memory sizes are usually written using the prefixes with a base of 1024
instead of 1000. Implment formatting of the power of two based prefixes
for displaying memory sizes.
Implement simple collection of basic real time system metrics of hosts
and the controller for the purpose of displaying them in the Web UI. The
metrics are collected by the controller at a configurable interval and
pushed out to web clients currently subscribed to the metrics.
The Prometheus library has an optimisation for metrics that expose
multiple time series such as Summary that provide two values in the
time series consisting of the metric name suffixed with _sum and _count.
These are provided as a series of sufixes with corresponding samples such
that a serialised summary metric named some_value would have a samples
mapping looking something like this:
[
["_sum", ["label=\"value\"", 3]],
["_count", ["lavel=\"value\"", 2]],
]
If multiple hosts expose the same metric then these need to be merged,
which is the job of mergeSamples, which merges the samples from each
sufix into the destination.
One corner case that may occur is if the source and destination metric
that's being merged have different lists of suffixed samples. In this
case the suffixes that are present in the source but not in the
destination needs to be added verbatim to the destination. The code for
handling this case incorrectly added these samples to the source metric
instead of the destination metric.
This corner case only occrs if a metric's type differs between two
hosts connected to the same controller. For example if a Gauge metric
is turned into a Summary metric.
When ws was updated to version 8 it came with a breaking change where
message data was no longer utf8 decoded from binary to a string. The
main codepath handling messages was updated to handle this but the ws
dump path was not resulting in the controller crashing when receiving
a message that is then forwarded to the dumper without decoding.
Fix by decoding the message in the dump-ws path.
The render method for columns in Antd Table take the three arguments
value, record, index. If dataIndex is set then value is the property
given by dataIndex in record, otherwise value and record is the same.
Unfortunately only record has the proper type while value is the type
any. Change usage of `render: value =>` to `render: (_, value) =>` to
get proper typing for the render methods defined on columns.
Use a separate ES Module based build for the web version of the lib
package. This fixes the rebuild on watched files changing in lib
failing with a TS1479 error.
Add host.factorio_port_range config to hosts that specify a pool of
ports that gets automatically assigned to instances which do not have a
game_port set on them. This allows setting up a port range that's
forwarded to the host on the router and then configure that port range
on the host and have instances automatically get assigned an available
port when they start up.
Resolves: #448
Due to configs needing a location to be deserialised it's not possible
to cleanly replace RawInstanceInfo with the InstanceInfo class. Rename
it to HostInstanceUpdate to better represent its purpose.
Refactor away the config group concept and simplyfy the way config
fields are defined. Fields are defined directly as properties of the
static field fieldDefinitions on the class itself with a separate
interface to declare the types of fields.
This makes it possible to easily make accessing the config fields type
checked in a way that lets plugins extend the interfaces with their own
config options by adding additional fields to the interface via
declation merging.
Remove circular dependency between inventory_sync/message.ts and
inventory_sync/instance.ts by moving the IpcPlayerData type from
message.ts to instance.ts. This solves a problem with the web
build checking types in instance.ts that it dosen't use.
Refactor the Config and ConfigGroup classes to match the JSON data class
pattern with toJSON, toRemote and fromJSON methods replacing the load,
init, and serialize methods. Since fromJSON requires extra information
the messages structures passes around the plain JSON representation of
the configs instead of serialising instances of the config classes
automatically.
If a new save is added to the saves list of an instance the current save
selected to be loaded by default changes its state without updating the
mtime of the file on disk, which in turn means basing the updatedAt
property on mtime will not work with caching. Fix by having a separate
updatedAt property that is set by the controller whenever the save
details changes from the controller's perspective.
Noticed-By: @Cooldude2606
Avoid possible use of an unintended empty mod pack by not setting the
activeModPack on the Instance until it's loaded in syncMods. This was a
leftover from when a newly created empty mod pack was supposed to act as
the default when no mod pack was set.
Process updates of the resource EventSubscriber are subscribed to and
have it keep a mapping of id to value syncronised with event provider as
long as there are subscription callbacks registered with it. The
mapping is kept around after the last callback is unsubscribed so that
it can be efficiently resynced with the resource based on the updatedAt
timestaps of individual elements.
This allows the EventSubscriber to provide the right building blocks to
provide its values to React components via the the useSyncExternalStore
hook. This in turn greatly simplifies the web_ui models for the
subscribed resources.
Reduce complexity in the implementation of EventSubscriber by requiring
a control link be passed to its constructor. This is possible after the
change in 8c724e3 made the web interface control instance available to
plugins at init time.
Since subscriptions are handlede per link there's no need for the
eventReplay to be returned as a part of the SubscriptionRequest, it can
be sent and processed normally as an event. Simplify the logic by
sending eventReplay as an event and remove SubscriptionResponse.
Remove the reliance a reference to the Controller to get the link that a
subscription is tied to by explicitly handling the SubscriptionRequest
and passing the link to the SubscriptionController.
Notify the SubscriptionController that a connection closed from the
Connection instead of having the subscription controller set up an event
to do so. This simplifies the logic on the SubscriptionController's
side.
Entites that didn't have an updatedAt time at the time of creation will
default to having one set to 0. Since entities with an updatedAt time
greater than lastResponseTime is expected to be returned this means that
these entities would not be included when a client does its initial sync
subscribing to a resource. Change the default lastResponseTime
requested so that these entities are included.
Skip calling the subscription function that should provide an event
replay if the client is unsubscribing from the event. This avoids
having to check if the client is subscribing in every subscription
function.
Keep a copy of the instance saves details on the controller so that
saves can be queried even when the host is disconnected and the
controller can properly track and provide updates on the saves to
its clients.
Refactor HostUpdatesEvent, InstanceDetailsUpdatesEvent, ModUpdatesEvent,
ModPackUpdatesEvent and UserUpdatesEvent to have multiple updates of the
entities they support in each event. This is intended to make the events
capable of syncing the entire list that they represent.
Download and cache needed mods to run instance mod packs on instance
startup. To avoid opening too many download streams at once only one
mod is downloaded at a time.
Structure the storage of mods on the controller into into a reusable
component in preparation for the host using the same kind of mod storage
to retrieve and cache the mods it needs to run mod packs on instances.
Log errors messages to the log when setting up an instance fails in the
preparation phase, creating save phase, starting up phase, and so on.
Previously these were only sent back as a an error response to the
request that iniatied the wakeup of the instance.
Build the clusterio_lib mod from source and download and copy the
subspace_storage mod used for testing if it's not present in the mods
directory. This removes a manual step needed to run tests on a new
clone of the repository.
Add helper to serialise the calls made to an async function such that no
overlapping invocations happen to the async function by queue up calls
made while it's running to continue after it's running.
Replace PluginWebApi's requirePath member with npmPackage which is only
populated if the package is not marked as private and the requirePath
matches the name. This prevents the public /api/plugins endpoint from
leaking filesystem paths into the controller.
Use the JSON data class pattern for User, Role, and PlayerStats and move
them into lib/src/data. This means that User is purely data now and
can't reference role objects directly. To make it possible easily check
permissions attached to roles and store data only visible on the
controller on user objects a ControllerUser type is introduced which is
linked with the UserManager and can therefore reference role objects
directly.
Resolves#525.
Load hosts, instances, mod packs, mods and users before constructing the
Controller instances. This allows these resources to be correctly typed
as always available and prevents any possibility of save being initiated
on these resources before they are loaded.
Some updates for instances trigger instance details updates, while other
updates to instances are not visible on instance details. Rename
instanceUpdated to instanceDetailsUpdated to better represent what it
does and only set instancesDirty when an update is made to the
instances.
Separate out the serialisation of the invocations to saveData with an
AsyncSerialMergingCallback helper than has unit tests covering its
functionality.
Exit the Space Exploration remote view if its active to prevent the
empty remote view "character" being serialized as the inventory instead
of the real character for the player.
Original-patch-by: @Reavershark
Fixes: #513
When creating temporary files use the pattern `.tmp.ext` instead of
`.ext.tmp` to be consistent with Factorio which creates `.tmp.zip` files
during saving.
If patching fails this will leave a `.tmp.zip` file in the instance saves
directory which is ignored by the default save selector.
Resolves#486.
Add controller.autosave_interval config that specifies the frequency
which controller config, hosts, instances, users, mod packs and plugin
specific data which is held in memory should be automatically flushed to
disk if they have changed since the last time they were saved. This
prevents data loss in the event of a server crash.
Plugins can use the onSaveData hook to save its own data at the same
time as the controller does and this obsoletes making an autosave
interval in the plugin.
Resolves#523.
Catch uncaught errors and display the error message on the page itself
if bootstraping fails. This should prevent most cases of the of the web
interface being stuck on "Loading web interface..." after an error was
thrown and the loading failed.
By default webpack-dev-middleware will refuse to delete files outside of
its working directory. This becomes dist/web in the configuration
currently set up, which means that a rebuild triggered on the watched
files will fail when declaration files are output to to dist/web_build.
Fix by moving dist/web_build to dist/web/build.
Clean up the configuration for compiling TypeScript for the Web
interface to us its own tsconfig.web.json config files and reference the
TypeScript source files directly in the configuration and imports
instead of files compiled to dist.
To properly support recompliation with TypeScript's projects feature
each package's tsconfig.web.json file is set to output the build to
dist/web_build which is then ignored in .npmignore. Webpack is still
used to create the proper build in dist/web, so this is mainly to make
TypeScipt's compiler happy but it also allows the root tsconfig.json
file to include the web interface code, meaning running the build script
now also checks the typing of the web interface.
Final part of #491.
The control interface is required for adding handlers for message types
a plugin defines, but the plugins were loaded before the Control
instance was created. This meant there was no sensible way for web
plugins to register handlers for their own message types.
Move the creating of the Control instance to before plugins are loaded
and set this.control for web plugins before init is called.
Output message_validate.js to both dist/src/data and src/data. This
allows tools to directly consume the TypeScript sources in src/data
without throwing an error due to message_validate being missing.
These variables were intended to be globals declare in the global
namespace. That this ends up on the window object is an implementation
detail of JavaScript. Remove the window. prefix put in front of
staticRoot and webRoot and include the global declaration in the
TypeScript module.
Resolve circular dependency of controller depending on web_ui and web_ui
depending on controller by moving the type from controller that was used
in web_ui to lib.
Add strong typing of the invokeHook function to prevent incorrect
paramaters being passed to plugin hooks. This detected a few places
where the parameters passed were incorrect.
Move BaseControllerPlugin to controller/src/BaseControllerPlugin.ts
Move BaseInstancePlugin to host/src/BaseInstancePlugin.ts
Move BaseControlPlugin to ctl/src/BaseCtlPlugin.ts
Move BaseWebPlugin to web_ui/src/BaseWebPlugin.ts
This solves the circular package dependency problem of the base plugins
needing types that are available only in the dependencies to lib.
Add configuration for VSCode to use tabs for indent, unix style endings
and leave a newline at the last line of the file. This will make
editing from VSCode less error-prone as it will select the right
settings by default.
Be consistent with the usage of tabs as indents in all JSON files in the
project. This makes it easier to configure editors to emit the right
indents when manually editing these files.
Leave a warning in case someone opens and tries to modify the
server-settings.json file in the instance folder, as any changes
made will be lost when it's overwritten.
When defining a generic type where a type argument is never used
as a part of the type itself, TypeScript will consider two instances of
the generic type with different values for the type to be the same.
This caused the handle method to accept any handler for any
Request/Event, ignoring the event/request type completely.
Fix by specifying the constructor to return the intersection of the type
itself and the Request/Event type.
Turn on source map deduction in stack traces from thrown errors. This
feature is off by default due to potential performance degration in
certain corner cases. For the dev environment add instructions for
passing the option to node.
Keep a handle to the Instance in the connection the host has to the
instance so that methods can be called directly on the Instance. This
simplifies logic that exists outside of the link system like the host
shutting down all of its instances when it exits.
Fixes validation error thrown on host shutdown with running instances.
Remove extra whitespace, use more consistent spacing, add missing
semicolons, using dangling commas, break up long lines, consistent
multiline operator placement, add missing JSDoc strings and so on.
* allow to revoke / restore host access from web ui
* refector to what is expected
* remove core.admin check on HostViewPage.tsx
* update core.host.revoke_token description
* .jsx to .tsx
* controller: tsc --build before webpack-cli
* Type Annotation
* disabled move into the <Input/> and <Checkbox/>
* cleanup debug left over
* documentation
* default ModPack
* modPackUpdateHandlers no undefined
* PluginInfo changes proposition
* PluginInfo changes proposition miss
* ModRecord & ModInfo feedback loop at const mods = [ ?
* ["connect", "drop", "resume", "close"] as const
* .tsx to .ts when possible
Allow modules to include a module_exports.lua file in their root which
will cause the file save/modules/module_name.lua to be generated
pointing to the exports file.
Map locale files located at locale/{lang}/{file} in a module to
locale/{lang}/{module-name}-{file} in the save. This additionally
support a shorthand notation by mapping locale/{lange}.cfg to
locale/{lang}/{module-name}.cfg.
Resolves#500.
* .js to .ts
* Allow access to _items entries; getItemCount return number
* add permission field to PluginPage interface
* port plugins to typescript
* update Request classes
* fix IpcFlowType
* .js to .ts
* translate
* update paths to dist/plugin
* add "tsc --build" to prepare
* use field main in package.json de find info
* tsc --build need to append before webpack
* update test and use for plugins package.main
The unknown status happens when an instance is on a host that has
disconnected, but since alt-host is not spawned in the tests on
Windows this status is not observed.
On Windows require paths use a mix of forward and backward slashes due
to require path possibly being a package path which can't have backward
slashes, or a Windows file path which can have have either. Fix plugin
loader test failing on Windows due to not using the same combination of
forward and backward slashes for require file paths.
Fix updates to Node.js timers types breaking the TypeScript build by
deducing the correct type from the return type of the
setTimeout/setInterval functions.
Fix coverage checking not being able to deduce the original TypeScript
files due to lacking a way to link the compiled .js output files with
the .ts input files.
Messages are forwareded when received from another link, this means it's
too late to throw an error that bubbles up to the code trying to send
the incorrect message thus defeating the purpose of the validation.
Only the controller has access to the secret needed to sign user tokens
so it makes sense for the user signing code to live in the controller.
This saves about 20ms on the time ctl takes to start up due to lib no
longer needing to import jsonwebtoken.
Move all functions and classes currently available through lib/name and
libName to be under a single flat namespace in lib. This make
reorganising the lib namespace much simpler as the location of symbols
can move without breaking the consumers of lib.
The multi-entrypoint design was already problematic for the web plugin
system which couldn't effectively use it due to how webpack's shared
module system is awkward to set up with multiple entrypoints per package.
- react: Update to new root rendering method
- react-router: Replace useHistory with useNavigate
- react-router: Replace <Switch> with <Routes>
- reimplement sticky notice from scratch in ModPackViewPage
- remove isomorphic-ws as it's latest version doesn't work and only
saves one line of code.
- remove import of antd.dark.css
- fix misc css breakages.
Add type object to schemas specifying properties and minItems to schemas
specifying tuples. Disable strictTuples because it's desirable to be
able to add additional items to the tuples in the future.
Update non-web related dependencies to their latest version were
possible. It appears yargs has changed its parsing of command line
options to no longer remove a layer of quotes.
The LTS releases for v12 and v14 are already end of life and the v16 is
end of life in under two months from now on. Bump supported version up
to v18 as it does not make sense to continue to support EOL releases.
Even if we wanted to keep supporting them, our dependencies will not.
Note that Node.js v18 is currently not avaliable via the NodeSource PPA
on Ubuntu 19.10 and below or Debian 9 and below.
It makes more sense to put the address first then have the large
structure to send as the second argument to sendTo. The opposite order
was originally chosen to be similar to that of sendto in BSD sockets.
Format arguments to have nested structures start on a new line instead
of inline to make it easier to see where the structures starts and what
structure it is.
Keep track of requests that have been forwarded over a link in a similar
manner to requests that were initiated on a link in order to properly
return an error response to the sender of the request if the link goes
down while the request is still pending at the forwarded node.
Validate messages received on the controller from hosts and control
nodes before handling or forwarding them. This should prevent
performing actions that are not allowed by the role system and block
simple attacks such as spoofing the source address of a message.
The data class pattern used for request and event payloads require the
class names to be preserved as they are used for identifying the type of
the requests and events.
Split up the action of registring an event or request data class to be
decoded by the Link class and the action of registring a handler for
those requests and events. This moves the validation work from Link
instantiation time to startup initialisation which makes logic errors
fail on startup instead of failing instantiation of the link when a
connection attempt is made.
Additionally implement a partial validation logic for messages received
using the newly refactored message registration. This unfortunately
ended up merged with the registration refactor.
Reverse the direction of the API for sending messages from
messageDefinition.send(link, { content });
to
link.send(new MessageClass(content));
and convert all message definitions to use the JSON serialisable class
pattern and explicit registration of message handlers, along with a
end-to-end based routing of messages instead of link to link.
The experience I gained from the existing way messages were defined and
sent over links was that it was difficult to understand for people
unfamiliar with the system. The automatic registration of message
handlers ment that code was being called as if by magic from nowhere
with no easy way to discover it. The manual forwarding and rules for
what can be sent where and how obscured its working.
The use of pure JSON as both input and output from message handlers made
it tedious to work with. Data had to be manually converted between the
program representation and the over the wire format, which made handler
code unplesant to write. Particularly because the over the wire format
used snake_case while the program code uses camelCase for variables.
The idea behind the decision to do it this way was to make it more clear
which data came from the wire and which from the program, and while this
worked it didn't make message handlers any nicer to write.
The direction of the API and magic registration of message handlers also
meant that a future migration to TypeScript would be troublesome. While
it may be possible to implement, the degree of TypeScript magic required
would go starkly against the principle of keep it simple stupid.
By refactoring to using the JSON serialisable class pattern most of
these issues go away. There is no longer a need to distinguish the on
wire format of messages with the program representation because the only
place that deals with the wire format of messages are the toJSON and
fromJSON methods on the classes themselves. Message handlers will work
with proper instances of classes instead of raw data, something that
makes them easier to write, and trivial to type with TypeScript.
End-to-end routing also simplifies sending messages that need to do
mulitple hops to reach the destination. And removes the need for having
all messages be specified in plugin's info module, as they can be routed
without having to be parsed completely.
The explicit registration of message handlers is something that I
thought would be tedious boilerplate code that could be automated away.
But experience now shows it to be the opposite. Allthough dull
repetition, the purpose it serves is to link together the code in a way
that humans can easily find. With explicit registration searching the
name of the message handler in the editor will show the registration that
ties it together with the type of message it handles and the kind of
registration. With implicit registration searching the name of the
message handler turns up only the handler with no clue as to what calls
it.
Change the JSON seriasiable data classes to use an explicit static
fromJSON method and an optional toJSON method for turning them to and
from JSON.
This improves over the previous pattern of passing the JSON to the
constructor as it frees up the constructor to be able to take whichever
argument types suits the code instead of being beholden to the JSON
serialised form.
Add migration code to installer fixing the names used in configs,
logs, database, and packages to reflect the new controller/host names
for people updating from alpha-13.
The role of a slave as a host for Factorio servers is not well reflected
in its name and has caused confusion among users. Rename slave to host
to better reflect its role as a host of Factorio servers.
The role of the master server as the party controlling every part of the
cluster is not well reflected in its name and has caused confusion among
users on several occasions. Rename master to controller to better
reflects its role.
Check patch number in update_instance step to ensure the clusterio_api
is properly initialized if the game started tick paused and no ticks
to trigger the patch check have happened yet.
Technologies that were disabled were ignored completely by research sync
due to a missunderstanding of what disabled means. Disabled does not
mean the tech is somehow inaccessible and completely out of play, it
only means it can't be selected in the technology screen.
Ignore the enabled status of technologies in research_sync to fix a
crash that happened when a mod enables a disabled by default technology
and the player attempts to research it.
If a system crash occurs after a simple write file + rename operation
the target file that was renamed over may end up being empty if the data
for the file written had not yet been committed to disk. This is
possible because no part of the open file, write data to it, close it
and then rename it ensure the data was actually written to disk prior to
the rename happening. See this discussion on the btrfs wiki[1].
Fix this possible data loss scenario by calling fsync on the temporary
file that is written prior to calling close on it.
[1] https://btrfs.wiki.kernel.org/index.php/FAQ#What_are_the_crash_guarantees_of_overwrite-by-rename.3F
All possible Promise rejections and exception bubbling should be handled
by either the code creating the exception or the code invoking the
operation that could fail. Treat therefore uncaught exceptions and
unhandled promise rejetions as fatal errors that log a traceback and
immedately terminate the Node.js process.
The format of mod-list.json written on syncMods would incorrectly write
a base mod without an enableds key which is required, causing Factorio
to ignore the file entirerly. Furthermore the enabled status was not
written causing all mods to be loaded even if they were disabled in the
mod pack.
The logging format provides both the name and id of slaves and
instances. Use the name when formatting for the web console just like
the TerminalFormat does.
Track summary statistic of total HTTP response durations using a
middleware that applies for all requests instead of having every request
needing to update the hit counter manually. Deprecates
endpointHitCounter.
The rationale for not allowing players to list users in the cluster by
default was that this would be an information disclosure of banned users
in the cluster, but given that a player is a user that can connect to a
server, and anyone that can connect to a server get the full banlist
from the server during the connection phase this does not make much
sense. Grant permission to list users by default for players.
Check that Master/Instance/Control plugins export the correct class and
that this class is a subclass of the correct base class. This gives a
better user experience when plugins are incorrectly set up and loads a
duplicated clusterio/lib, and when accidentally using the wrong name
when developing plugins.
The directory which all files are placed under in a mod is commonly
named <mod-name>_<mod-version>, but this is not a requirement as it can
be named anything. Fix code incorrectly assuming the zipped mod's root
folder follows this pattern during data export.
Use exported settings prototypes and locale to format the settings UI to
with mod names, settings names, and select dropdowns. This is a crude
approximation of how the game presents these settings to the user.
Add missing mod settings set to their default values to a mod pack when
performing a data export based on the settings prototypes that were
extracted from the game. This mirrors the behaviour of Factorio when new
setting prototypes are added by a mod.
Updates to Node.js error reporting have caused tests to fail due to them
being hardcoded to expect a certain format that has since changed. Fix
by making Node.js create the error message for us.
Store the export manifest in ModPack to allow mulitple mod packs to have
different data exported. This is necessary as the export contains the
settings prototypes needed to create a proper UI for mod settings.
For now the useLocale and useItemMetadata interfaces uses the manifest
for a default mod pack set in the master config.
Add settings and all prototypes to the export of data done. Settings
are needed for a settings GUI to function, while prototype data would be
useful to display any sort of information about things from the game.
Mark mods in the UI and the mod-pack show ctl command that are missing
with a question mark and mark mods with a missmatched checksum with an
exclamation mark.
Validate sha1 sum matches for a mod used in ModsTable if it is present.
This prevents showing details for a zip file where the version matches
but the cheksum does not.
Wipe and reconstruct the instances' mods/ folder using the configured
ModPack for the case were the mods folder is shared with the master
server. This assumes the master server will check the mod checksums
before sending the start request.
Remove ugly hack to generate and use an insecure certificate on the fly
in the HttpCloser's https tests and replace it with the already existing
properly generated test certificate. Removes the dependency on node-forge.
Make all properties on the mod pack object except sha1 required and add
the base mod if it does not exist in order to align the JSON
representation with the proposed Mod Pack String format I published on
the Factorio Forums (https://forums.factorio.com/103578)
Enforce the filename of uploaded mods match {name}_{version}.zip with
name and version taken from the zip file's info.json entry. This allows
it to be used as a unique key for the mod and solves part of the issues
with mods having different number of leading zeros in their version.
The Factorio Mod Portal uses and provides SHA1 hashes of mods, so it
makes sense to align the ModPack and ModInfo structures to sha1 by
renaming the genericly named hash property to sha1.
Make the extra content for section headers flow under the heading when
there is not enough space to show both the heading and the extra content
in a single line.
Use overflow clip instead of hidden in the page area of the site layout
to avoid creating a formatting context that would make sticky not work
as expected inside the page area.
Prevent tables cells containing byte sizes from having a line break
inserted between the number and the byte unit when they get squished by
using a no-break space instead of a regular space.
Add ModPack data type with corresponding storage on the master, messages
to create, update and delete them, as well as commands to invoke them.
The ModPack data type contains a list of mods with versions and optional
sha1 hashes, and mod settings to be used.
Add the ability to manage a set of mods to be shared by the master
server to the rest of the cluster. This implements storage, uploads,
downloads, deletetion, and listing details of mods but does not actually
do anything with the stored mods.
A file name con.zip is also treated in the same way as the special CON
redirection and may not be used as a normal filesystem path in Windows.
This includes any extension, even with multiple dots. Fix checkFilename
and cleaneFilename not handling these cases.
Fix the installer wiping the package.json file due to a call to
outputFile that needed to target the original file being incorrectly
changed to safeOutputFile.
Be a bit more accurate in the online time stat name and call it for what
it is. It counts time spend online on the server, not necessarily
time spent playing.
Fix broken support for retrying uploads and background downloads by
making crafting queue serialisation non-destructive and only performing
the serialisation and deserialisation while the player is online.
This removes the /save-crafts and /restore-crafts commands because the
existing implementation would voids the non-saved crafting queue items
if /save-crafts is performed and then new items are added to the
crafting queue before switching server.
Fixes#419.
If only the bot was clever enough to see that node-forge is only used
for a test that has no user input it might have had the foresight to
realize that crying wolf about completely irrelevant security issues
will only lead to legitimate issues being ignored in the future. But
alas, the AI overlords are not taking over any time soon.
On Linux the installer offers to correctly download and install the
latest version of Factorio. Remove no longer needed instructions for
how to do it incorrectly.
I have seen multiple people have installed Clusterio as the root account
and then run it as root when tasked with setting up Clusterio on Linux.
Make this mistake harder to do by refusing to install as root by
default.
Since the admin and whitelist status is shown with a tag next to the
name it's no longer necessary to show the admin and whitelisted switches
when the user does not have permission to use them.
Add command that extracts play time (and players) from the currently
running save to the recorded instance statistics. This is useful for
getting correct play time statistics when an existing save game is
imported.
In the case of shutdown the Factorio server will not notify the
currently online players as leaving the game. Notify the master server
that all remaining online players quit the game in this case.
Count up to 100 empty slots past the last seen slot with data in it
before stopping to look for personal logistics slots in inventory_sync.
Allows unlimeted amount of logistics request slots to be used for
heavily modded games.
Factorio 1.1.40 added a feature to use a centralized banlist maintained
by Wube. For the dedicated server this needs to be turned on with the
command line flag --use-authserver-bans.
Void excess slots when deserializing an inventory to a target that has
fewer slots than what was serialized. This fixs inventory sync failing
when a stored inventory has trash slots in use but the player does not
have the slots unlocked.
Store exported data files with a hash in the filename to allow the new
caching stategy for static files to work with it and provide a manifest
for the web interface to find the exported files with.
Render the web interface as early as possible with a black background
and show a message to indicate whether the interface is actually loading
or if an error occured loading the main bundle.
Put all web assets including plugin assets under a shared static/ route
in the web interface that's declared as immutable and cacheable for 1
year. This allows all JavaScript bundles to be loaded from disk instead
of having to check with the web server each time and significantly
speeds up the web interface load time. It also fixes the public path
issue when using the --dev-plugin option behind a proxy.
To prevent collisions and stale data all assets now include a hash in
the filename. This means the entrypoint is now dynamic and specified in
a manifest.json file created from the web build. For plugins this means
the filename attribute needs to be removed from webpack.config.js.
Mocha's recursive search feature finds all files and then removes paths
that match the exclution filter, except that the globstar filter doesn't
match directories with a dot in them which is created by pnpm. Avoid
this madness by removing the test package.
The prepare script runs before publishing a package, after installation
of a package from source, and when running npm install without
arguments in the repository. These are the times where a build of the
web interface files are required to make the web interface work and
removes a manual step from using the package(s). The "build" script on
the other hand is never automatically invoked by npm.
Save disk space, time and sanity by replacing npm and lerna with pnpm.
This is more performant and doesn't have developer experience ruining
bugs like npm link removing all previous links before creating the
requested one.
Add ability to move or copy saves between instances. For instances on
the same slave the saves are transferred with filesystem operations, for
instances on two different slaves the proxy stream interface is used to
send the save via the master server.
Travis seems to be going away from supporting Open Source Software for
free and their proceedure to gaining free tokens is getting harder and
hander to find for each year that passes. Remove Travis CI integration
and migrate the coverage reporting previously provided by the Travis
build over to the GitHub Actions.
Allow sidebar entries to be grouped together by specifiying a name under
which they will be grouped. To avoid confusion groups are rendered at
the bottom of the list.
Convert the master metrics to void instead of storing them in a variable
as they are completely self contained, and this makes it more clear that
there are no use of references to the metrics.
There are some corner cases with streams that cause them to not emit
finish close or end events which in turn means that code liseting for
these events directly may end up hanging forever. Use stream.finished
to detect end of streams which will handle these cases correctly.
The interface provided by Winston is unusably slow for querying large
logs. Reimplement the log querying with both forwards and backwards
line reading and a simple file based index to skip files which do not
contain any lines matching the filter.
Move the default location of the log files into a configurable logs
folder and split the written logs up by UTC date to more easily allow
deleting of old logs to be performed. The log rotation is implemented
by winston-daily-rotate-file.
The old log format will automatically be migrated to the new format and
location if the target folder for the new logs don't yet exists.
The burner equipment serialization code was never tested and
consequently never worked. Wasn't noticed before now because burner
equipment isn't in the vanilla game and few mods use it.
Fixes#442.
Turns out objects in the second argument to useEffect is compared by
Object.is(), this means if the array containing the instances or slaves
to show logs for get re-created during an update the LogConsole
component would send a new request for the logs.
Fix by converting the slave_ids and instance_ids arrays into strings
before passing them on to useEffect.
Add splitter based on LineSplitter which splits lines in reverse from a
reverse stream created from the added createReverseReadStream function.
Enables line processing of files starting from the last line and going
towards the first.
Pull the LineSplitter class out of lib/factorio to its own stream module
and refactor it to be based on stream.Transform. This allows composing
with other streams by piping it around.
Instances require a number of modifications leading to it not being
able to be converted into a function. The main one is that there is a
list of config elements that cannot be set via the edit command and
either have to be set via different means or cannot be set. To stop
these fields constantly sending error messages, we disregard adding them
to the file in the first place.
Some systems may resolve localhost to ::1 while Factorio doesn't respond
to RCON connection over IPv6 loopback, this causes instances startup to
fail. Fix by using 127.0.0.1 instead of localhost.
Using an event emmiter, the fields can now be set correctly in the
master config. Null values are now handled correctly, and numbers are also set
correctly.
Also now using the system described by Hornwitser, in using
libConfig.MasterConfig.groups to get definitions. Now all that's needed
is to re-read the updated data back into clustorio.
Write new file contents to a temporary file and then rename over the
target file to avoid data loss in the unlikely event there's an error
writing the new file contents or the process is interrupted.
Resolves#416.
ctl master config edit [editor]
is the new structure; it can take an optional editor, and will prio this
editor over any other editor, such as one set by env vars. Generally,
the prio goes like this: argv supplied editor > EDITOR env var > VISUAL
env var. Things needing to be done are marked with a TODO.
Implimented framework for edit command, including ability to get a temp
file, open an editor on the file, then resume code execution once the
editor exits.
Somehow the step of automatically obtaining the right address to apply
to slave.public_address, and asking for the master url and token for ctl
installations was implemented but the code to actually set it in the
relevant config was forgotten.
The serialization and deserialization of editor mode is not supported by
the current code, and while the download correctly checked for the
player being in editor mode, the upload did not check for the player
being in editor mode causing players in editor mode to break. Fix by
adding check in upload_inventory.
Fix commands syncing researces being sent before the instance is started
up, and potential desync of research progress as a result of out of
order execution of RCON commands sent.
If the session to a slave is lost while gathering metrics the error
would propagate out to the metrics route handler and break collection of
statistics. Fix by handling errors thrown from sending the getMetrics
request to slaves.
Don't bother doing the prepareDisconnect dance if the connection is
already broken. This avoids having to wait until the connection times
out when a slave gives up on a current connection and starts a new one.
If a connection has timed out the most likely cause is the connection
going stale, in which case calling close could take up to 30 seconds
before the ws lib gives up on waiting for the close message. Terminate
the connection instead on the Node.js side. Note that Firefox does not
permit applications from closing connections with a code other than
1000, or in the range 3000-4999.
The INI format Factorio use differs from what the ini package parse and
produce. Most notably no escape mechanism exists in Factorio's parser
which causes instance paths containing among other thing the # character
to completely break. Fix by replacing the ini package with a Factorio
compatible implementation of INI parsing and serialization.
Resolves the problem in #414.
Move the save list subscription permission to be after the save list
permission as these two permissions are logically related to each other
and both need to be granted to view the save list in the web interface.
Add the roles and their permissions for the user the control connection
is connected as along with updates when these change. This enables the
web interface to know which permissions it has available to it.
Fix has_data not being checked by sync_player, dialog being stuck open if
no data is received by the master and master server refusing to server
empty data.
Add optional limit to the number of log records to return in the
log query request to avoid having the web interface instance console
download the entire log for only to show a few hundred lines from it.
To make it possible to query the end of the log instead of just the
start add an order option as well.
With the instance status unknown now meaning assigned slave is offline
the save list buttons can now easily be disabled using this status.
Disable the upload save, download save and delete save buttons when the
slave is offline or the instance is not assigned to a slave.
The system for preventing items being sent without a connection to the
master and (and thus voiding the items) is to only do so if a ping from
the plugin has been received in the last 5 seconds of in-game time.
Don't send pings without being fully connected to the master to make
sure this system actually activates during connection downtime.
Ignore inventory updates and orders while the instance is not in the
running state to avoid sending commands before the server has started
up, and also ignore orders if the connection to the master server isn't
isn't online to avoid piling up requests in the buffer during session
resume attempts.
Since the connector refactor the action of sending a message on a
connector that is in the process of connecting changed from queuing up
the message to send it to throwing an error. To avoid events forwarded
by the slave to to the master when the connection is down from being
logged as an error forwardEventToMaster has to explicitly drop them.
Prevent ordering issues with save notifications on stop by making the
save list update part of the stop action instead of some unspecificed
amount of time later.
This also fixes the issue of the connection being closed before the last
instance sends it save list on slave shutdown.
Provide a convenience method for checking if the Master-Slave link is
fully online via a .connected property. This is true if the link has
been establised, is not in the dropped state, and not in the process of
disconnecting.
The activeConnectors mapping is from session id to connector, but the
master refactor in 67219ed accidentally changed it to try deleting the
entry by the connector reference instead. Possibly mistaking it for a
Set instead of a Map.
Complete rework of the inventory synchronization to reduce the number of
times the inventory has to be downloaded and the possibilities for item
loss/duplication.
The basic concept is to acquire exclusive access to the player data
before downloading the data. This prevents the player from logging in
to two servers at the same time and getting the synced inventory twice.
The acquisition will in most cases execute faster than the player can
provide input to the game on login, and if the player data on the master
isn't what was previously uploaded then the player inventory is deleted
and the player is turned into a spectator while the updated inventory is
being transferred into the game. This model allows for the downloading
of the synced inventory to be skipped in the case the player joins the
same server they left from.
The synced data is expanded to include more than just the character and
inventory to properly handle sync of dead players. If the player leaves
dead on one server this state will be saved and when the player joins
another server they continue off as dead. The ghost, god, and spectator
controllers are supported, while the cutscene and editor controllers are
not. For players in a cutscene when joining the acquisition will be
attempted immediately, but the scenario waits until the player is out of
the cutscene before acting upon the response to the acquisition. For
players in the editor mode the scenario toggles them out of the mode
before syncing data.
Since uploads to the master can fail handling of this is also added.
The scenario will retry uploading the player data at exponentially
increasing intervals until it gets the message back that the upload
succeeded. Since it's possible for servers to go down and stay down for
a long time the exclusive access held by these servers are released after
a timeout. This means that a player could be playing and or have
uploaded newer data by the time the instance comes back online and tries
to upload the inventory, in this case the master will report the upload
succeeded but discard the data. Instances may also upload old data if
they load back to an earlier save, which will also be discarded by the
master.
Other small improvements made by this rework:
- Add a retry button to the failed download dialog
- Update the existing progress dialog instead of destroying and
recreating it every time.
- Automatically stuff the items from the temporary inventory into the
synced inventory if they fit.
- Move player back to the position they started the download from if
they were turned into a spectator for the download.
- Ensure the player has a character if the stored data is wiped.
- Add error checking of the RCON commands sent by the instance.
Refine the state flow used in the connector abstractions to better
handle close while connecting and resuming. The new logic is to close
the connection immediately in these two states instead of continuing
trying to connect and then giving up some point later.
For the connectors the handshake state is split into connecting
and resuming, new is replaced by closed, and the closing state is turned
into a boolean flag as it's possible that closing is desired while
connecting, connected or resuming. The connecting state is not present
in the server connector because there's nothing for the server to do in
that connector state.
The timeout logic is completely removed, once a connection attempt is
started for the client it will continue to attempt to connect, attempt
to resume dropped connections and attempt reconnects until either an
authentication error occurs or the client explicitly requests to close
the connection.
A new session timeout is added to control how long to attempt to resume
the connection until it's considered dead and invalidated. This allows
stale connections to give up earlier, and swallow up less data while in
the dropped but not yet invalidated state.
For the client side sending messages while it's connecting has been
changed to throw an error instead of queuing up the message and
(possibly) sending it after the connection is established. Code
shouldn't try to send messages without knowing if the connection it's
trying to send them over is active or not.
The ajv module was made shared to reduce package size, but it turned out
the package size was large because of a missunderstanding in how imports
to sub-paths works with module federation. Since that was fixed the ajv
module was no longer shared, but plugins incorrectly still specified it.
In order to import a path inside packages/lib/ in the webpack shared
module environment it has to be declared as shared in both the web_ui
webpack config and the plugin's webpack config. This is rather error
prone and easy to miss. Move the burden of specifying which entrypoints
are valid in lib to lib itself by changing all imports in the web
interface code to import from @clusterio/lib instead sub paths.
For web code this means imports is done like this from now on:
import { libConfig, libLink } from "@clusterio/lib";
For info.js which has to be loadable by both Node.js and web code the
imports are done as follows:
const { libConfig, libLink } = require("@clusterio/lib");
While the Node.js only code can either continue to import sub paths
directly or use the info.js style.
Create entries for players that are present in the save on server
startup to preserving their inventory and fix attempting to reference the
player object after it had been removed in on_player_removed.
Make sure the GUI element in the on_gui_click event is valid before
attempting to access this, as another module may have destroyed it
during its event handler.
When upgrading or downgrading the inventory_sync plugin it's possible
an different structure of the global.inventory_sync table is left behind
which could cause the code to crash when certain sub tables are not
present.
Be more defensive in the initialization of the table to make sure all
sub-tables are present on startup.
Fix setting a nullable numeric field to blank in the web UI return error
that the value cannot be a string due to the field returning null and
this being converted to the string "null".
Ignore stop if instance status is already stopped. Fixes instance status
getting stuck at stopping if Factorio exits during startup due to the
stopped status having already been sent at the time stop is invoked.
Fixes#400
Convert code indents to tabs to be consistent with the rest of the
JavaScript code. This commits covers all of the lua code, some of the
hand written JSON code and the lone HTML and CSS files that were not
using tabs.
The jsonwebtoken accept strings as input for secrets which is
nonsensical from a cryptographic standpoint as the algorithms all
operate on byte sequences. Worse still, strings are encoded into bytes
using UTF-8, which disallows certain sequences of bytes from appearing
and potentially weakening the cryptography.
For legacy reasons the master secret was encoded into base64 _and passed
as a string to jsonwebtoken_. This further weakened the crypto as the
bytes that can appear in base 64 is only 1/4 of the 256 possible values
for bytes. This could significantly weaken the cryptographic strength
of the tokens.
I have known of this issue for quite some time but has put off actually
fixing it for too long now. Since the master secret is already
documented as being in base 64, decode it as base 64 and pass it as a
byte Buffer to jsonwebtoken. This will ensure every byte value is
possible in the byte sequence passed to the jsonwebtoken crypto.
The impact of this change is unfortunately that all tokens will become
invalid and will have to be reissued.
* New method avoids all dataloss and prefectly replicates the queue
* Made proper description in readme
* Added packages to make lerna build happy
* Added reference to main readme
It's possible to put leading zeroes in the names of mod zip files which
cause the existing algorithm to find the mod zip from the mod version to
fail. Fix by listing the mods directory and fixing the version numbers
for any of the mods with leading zeros in them.
Fix incorrect serialization of control characters to \xHH escapes since
JSON only support \uHHHH escapes, and serialization of infinite and NaN
numbers.
Browser like Chrome doesn't use the application/zip mime type when
uploading .zip files. Add support for application/x-zip-compressed to
the save upload endpoint.
Provide a create package for the clusterio npm org which gathers the
necessary details to install a working cluster either as standalone or
as separate nodes.
Add a generate slave token button to the web interface and modify the
generateSlaveToken message and corresponding ctl command to have the
slave id be optional. The random id is the desired behavior when adding
a slave with the installer, as it'll ask for the token for the new slave
and deduce the id from the token.
Gracefully handle invalid IDs in useSlave and useInstance as this is
necessary when when the ID isn't known by the component using the hook,
since it can't conditionally call the hook.
When factorio starts up with with the require_user_verification setting
turn on it tries to obtain a server padlock from the matchmaking server
and will fail if this does not succeed. Turn off user verification and
public visibility which require user verification both of which are on
by default for the integration tests so that they work without internet.
Add logic to stop sending commands and wait for any pending commands to
complete during instance stop in subspace_storage. This should fix the
RCON connection lost errors printed in the console during stop.
Prevent potential data loss when starting an autosave by copying it
first. This avoids overwriting a random autosave slot when the server
shuts down after having loaded an autosave.
Add /api/upload-save endpoint that streams the data via the slave
assigned to the target instance to the instance saves directory, along
with a ctl command and web interface button to do the upload.
Add link messages for listing saves and listening for changes to the
list of saves for instances along with a table in the instance view page
showing the save list for the instance and a list-saves command to ctl.
Add options to pass map seed, map exchange string, map gen settings
and/or map settings when creating a save either through the create save
request or the load scenario request.
Check if the instance status in the instanceStatusChanged event actually
changed and ignore the event if it didn't. This prevents bogus updates
from being pushed to control clients and plugins.
Provide the control link with the ability to subscribe to updates when
instance status and properties changes. This allows the web interface
to respond in real time to instance starts and stops that are triggered
from somewhere else.
Add readMapExchangeString to lib/factorio which parses and decodes the
data in Map Exchange into a format suitable for giving to the Factorio
server command line.
The plugins are available as a property on the Control provided by the
ControlContext which makes the PluginsContext redundant. Remove it and
use ControlContext to retreive the plugin list in the app.
The Node.js specific rules for Eslint were deprecated in favour of the
node plugin for Eslint. Add node plugin for eslint and switch over to
using its rules, fixing any issues detected.
Adjust the regex for detecting startup to also work with 0.16 as
updateTick was called mapTick before 0.17. Note: This commit does not
imply 0.16 is going to be supported by Cluestio.
Notify when an instance is in the process of stopping with a separate
status for it. This makes handling shutdown gracefully a bit simpler in
plugin code.
During Travis integration tests a BrokenPipeError is emitted on stderr
after the Factorio server has quit. I have no idea what's going on here
so for now just log it and ignore it.
Move the depenedncies needed to run the tests into test/package.json and
manage it using lerna. This should make it easier to selectively
install only what's needed to run the software.
Fix path stripping logic to take into account the possibility of the
writePath being relative, and that Windows uses \ as a path separator
for some paths but not all.
Add reference to the Control instance in BaseWebPlugin for web plugin
code to use in a similar manner to the instance reference in instance
plugin code.
For master plugins a message can be received from any of the clients
corrently connected to the master. In order for plugins to be able to
handle link specific requests they need a reference to the link messages
was received from. Add link as the third argument to plugin message
handlers.
The automatic determination of the public path for plugins works
correctly when they are build for production and loaded from their
respective paths. But when the dev middelware compiles it, it doesn't
know the correct path to host it at withouth the publicPath being set.
Fix by setting publicPath explicitly when compiling with the dev
middleware.
Add the name of the account that's connected to the control link on the
web interface via an additional property on the ready payload for the
control link.
Add a button on the right of the top nav bar to access account related
functions and add a log out button there. To make it fit on the navbar
the text and version is hidden on narrow layouts.
Simplify the the way markdown files are formatted to make them easier to
author without editors that do auto line breaking, and make the diffs
from them simpler to read.
Fix ServerConnector reaching an invalid state if a continue operation is
performed on an active connection. The continue would cause the
connection established logic to be runned before the connection lost
logic of the pervious connection is perfomed, resulting in a leaked
heartbeat interval and the connection lost logic being performed on the
new connection instead of the old. Prevent this by closing both the new
and the old connection in this case.
Add authentication plugin that does a simple challenge response protocol
to verify that the player trying to log in is who they claim to be. The
process goes as follows:
1. A player code is generated for a player that wants to log in and is
displayed in-game.
2. The player code is input into the web interface and the master
generates a verification code as a challenge back along with a token.
3. The player inputs the verification code in-game and is logged in to
the web interface in doing so.
To do the actual login the web interface polls a verify endpoint every 2
seconds until the code has either been verified or expired. The
challenge comes from being unable to control the verification code and
having to input it as a player logging into one of the servers in order
to login.
Some additional steps are taken in order to increase the difficulty for
an attacker to subvert the challenge:
- The login process must be started from in-game.
- Generated codes are only valid for a limited timespan which is
configurable and 2 minutes by default.
- The verification code comes with a token that only works with one
player code and verification code combo and must be provided to pass
validation.
Change to "auto" as the publicPath to have webpack deduce the correct
path at runtime. This fixes an issue with plugins being unable to be
loaded due to trying to load bundles from the wrong path when the web
interface is hosted under a proxied sub-path.
Remove basename in favor of making webRoot absolute. The distinction
between the two were supposed to be that basename was absolute and did
not have a trailing slash while webRoot was relative and had a traling
slash. This was not how it ended up working in the code, which lead to
double forward slashes in requests sent out when webRoot was not equal
to './'.
Due to the way the imports and exports works with module fedderation
each require path needs it's own shared module in the plugin. To avoid
having to specify every possible import in the module federation plugin
config, export all useful components to plugins in the browser
entrypoint for web_ui.
Note that for some reason the module wouldn't exist for import by
plugins unless some part of the build imported it. Which is the reason
for the master index.jsx to import bootstrap via it.
Add a complementary total hit counter for all HTTP requests. The
endpointHitCounter doesn't cover all endpoints and fixing it to cover
all endpoints without making it possible for it to create an infinite
number of time seriers is not straight forward.
If data from the database is not loaded but a shutdown is triggered then
the master server will overwrite the database with empty data, causing
unexpected data loss. This occurs for example when the --dev-plugin
option throws.
Fix by setting the database entries to null before they are loaded so
that should an attempt be made to save them without having been loaded
first it will throw an error instead.
The web files for plugins were hosted at /plugin/plugin_name/ while the
view page for a plugin is hosted at /plugins/plugin_name/view. Reduce
confusion by making both urls use the plural version.
Add hidden development option to run the webpack compiller for the given
plugin(s) similar to the --dev option for the built-in web interface.
This requires publicPath to be set in the webpack config and an updated
version of webpack-dev-middleware to function correctly.
Add separate BaseWebPlugin class which plugins can extend and expose
through a webEntrypoint in a similar fasion to the existing master,
instance and control entrypoints.
To simplify the logic for the rest of the web interface code the plugins
context is changed to be a mapping of loaded WebPlugin instances. This
in turn means the plugins and plugin view pages fetches the plugin list
upon load in order to show plugins that failed to load (which
incidentally also gives them an up to date list).
During startup the plugins are loaded one by one and init is called for
each of them. But due to the already loaded plugins being discarded if
one of the plugins throw an error an unclean shutdown that might hang
on timers started by plugins occurs. This leads to the master server
starting up, printing an error, displaying the shutdown sequence and
then hang forever.
Fix by storing the plugins loaded into masterPlugins as they are loaded
instead only after all of them have been loaded.
Use relative imports in lib to avoid an issue with webpack thinking
absolute imports inside @clusterio/lib are for a separate module and
giving out warnings for it being missing in package.json.
The module sharing in Webpack works on a per import basis and therefore
doesn't trigger when the value in the config is @clusterio/lib while the
module imported is @clusterio/lib/config. This in turn caused a copy of
lib to be bundled with each plugin. Fix by making sure each lib import
is shared and specified in the configs.
Rename the config and status indicating whether to load a plugin or not
from "Enabled" to "Load Plugin". The latter better describes what it
does, and unlike the former doesn't imply plugins can be enabled or
disabled on the fly.
When setting up the tests there were issues with code that didn't
properly clean up after itself and left lingering connections, intervals
or timers that kept the event loop running after the tests where done.
To fix this the --exit flag was passed as a temporary measure but then
subsequently forgotten about. The issues with keeping the event loop
alive have long since been fixed on Linux, but on Windows there was a
problem with the control connector not exiting because the master server
was killed instead of shutten down properly.
With that issue fixed there are no longer any connections or timers
keeping the event loop alive and --exit can thus be removed.
If the connect close method is called while the connection is waiting
for a reconnection attempt then the attempt is cancelled but the close
method then erroneously waited for a close event that would never
happen. Fix by resetting the connector and rasing the close event in
the close method if it's in the waiting for reconnect state.
The removal of the automatic TLS certificate generation removed the use
of node-forge in the master server, while the addition of the HttpCloser
added it as dependency of the tests. Fix node-forge dependency being
listed in lib and master instead of as a dev dependency in package.json.
Add page listing the plugins installed on the master along with a
detailed page for each plugin with fields from their package.json.
Failure to load the module and version missmatch is also reported.
This adds ./packages.json as required part of the webpack module.
Add a React context to access the plugins loaded in the web interface.
The info, /api/plugins metadata and webpack container is provided for
each plugin.
b538960 added an error boundnary for pages not directly linked on the
sidebar, but forgot to add it for pages on the sidebar. This add the
missing error boundnary for those pages.
Log the stack of the error caught during failure to load the web
interface. This ensure the error is printed to the console instead of
"[fatal] undefined".
Provide the loaded file of the tls_ca config available to slave,
instance and ctl code. This ensures they can easily use it when
connecting to the master server.
Remove the need for installing babel and the webpack loaders used in the
webpack.common.js file separately by resolving them from the web_ui
package. This allows future changes to done to the modules used without
having to update them in each plugin that uses the web_ui common config.
Pull out the instance specific logic from InstanceConfigTree and turn it
into a general BaseConfigTree component that can be used for master and
slave configs.
Prevent the page from turning into a black screen when page rendering
fails by adding an error boundary to the display of the page content in
SiteLayout.jsx.
Add a per field access parameter to the config class which specifies
which of master, slave and control can access the given field. A
location is added to the config class to specify which party owns it and
fields which do not have an access parameter are not present in the
config and will throw if attempted to be read from or writen to.
The serialize, load, update, get, set and setProp are ammended with a
location parameter which specifiy where the data is from or destined to
and the access checks are performed to ensure remote access to the
config is still limited by the access parameter of field.
With this system in place the master config can be shared with control
and slave connections without compromising secrets that should stay
private. This is primarily the master authentication secret, but other
plugins may define secrets it does not want to have shared with slave
and/or control connections.
Raise "resume" instead of "connect" when a connection is re-established
after a connection drop. This prevents logic meant to resynchronize
state after a full reconnection from being invoked whenever the
connection was temporarily dropped.
Implement --add-perms and --remove-perms to add and remove permissions
from the role without having to list up all of the permissions in the
role again.
Rename --permissions to --set-perms to be consistent with the two new
options added.
Replace old paragraph in the developing for clusterio document about the
removed HTTP interface with a notice of the new protocol running over
WebSocket.
Remove the pluginInfo parameter from lib/plugin attachPluginMessages as
the plugin info object is available in the `.info` property of the
plugin. While a plugin instance doesn't exist for the slave side and
was the reason for these to be separate arguments, I expect slave
plugins to be implemented eventually.
Move the target folder for the built web interface to dist/web/ instead
of static/. This better follows the norm for npm packages to include
built files in the dist/ folder and also allows putting other built
files into that folder in the future.
Exit immediately if shutdown has already been called. This is due to
the possibility of it being called multiple times from combinations of
signals and fatal errors.
On windows the terminal output is asynchronous by default which leads to
log lines being lost if text is written to stdout before calling
process.exit. Fix by using setBlocking before doing the log call.
Add --stdin option to the config set commands which reads the value to
set via stdin instead of on the command line. This allows a more secure
way of entering values and avoids having to deal with multiple layers of
quoting and escapes for special characters.
Add optional arguments to Request and Event to control which properties
are required to be present in the payload. This allows gracefully
extending the protocol, reducing overhead for optional data, and leaving
properties out for additional behaviour.
Add settings of interest to server administrators that appear in
server-settings.json to the factorio.settings config. This makes it
easier to see which settings can be set there.
When parsing using the .argv interface to yargs the parser state is not
restored back to its pre-parsing state after the operation completes.
This causes a susequent reparse with argv to forget its commands and
respond with an unknown arguments error when trying to use the
control-config command to clusterioctl. Fix by switching to use the
parse() interface which does restore the parser state after completing.
The automatic creation of a self-signed certificate while nice presents
several problems. To make connections not reject the self-signed
certificate the validation of TLS certificates are wholesale disabled,
decreasing the security even when properly set up and trusted
certificates are used.
To make the created self-signed certificate pass validation it is
necessary for it to list the host names it's valid for and to copy it to
each machine that intends to connect to the server. These two steps
complicate automating the process in a reliable way and will likely lead
to confused users seeking to disable TLS certificate validation as the
easy way out to problems.
For these reasons, remove the automating creation of a broke self-signed
certificate, fix validation of certificates and document how to set up
TLS properly. This makes HTTPS recommended when accessing over the
internet, but still something which must be explicitly configured.
If the server decides to exit on its own during the stop operation to
the FactorioServer class after the operation has started but before the
signal is sent to the process then the exit handler will have wiped the
server child process and cause the kill call to fail. This in turn
would let the hang timer running which would bring down the Node.js
process due to an unhandled error for the same reason.
Fix by making sure the stop operation watches for the server exiting and
returning early if the server already quit before getting to the send
stop signal part.
When closing down the HTTP and HTTPS servers existing connections are
not closed by Node.js, and may live on forever if they are actively in
use. A common case where this happens is if Prometheus is set up with a
short interval for collections.
To close down the active connections they need to be tracked and closed
when the server is closed, add the HttpCloser class which encapsulates
the tracking and closure process behind a simple interface and use this
to close down the HTTP and HTTPS server on the master.
Fix the validation loging for sending responses to requests not
functioning due to the message data property being passed incorrectly to
the validator.
When reducing the collection interval in Prometheus it may be necessary
to to reduce the metrics gathering timeout to ensure it's less than the
interval. Add config options to adjust the timeouts used for metrics
gathering so that these can be set accordingly.
Special case PluginError printing during startup in slave.js in the same
way it's done in master.js. This makes it so that the stack of the
error thrown by a plugin is printed instead of the stack trace of the
place it was wrapped into a PluginError.
Based on docker and github actions
Builds a docker image for each commit as well as the lua lib mod
Builds the master branch of subspace_storage for use with the tests
Tests are based on latest stable factorio branch
Describe why and what has been changed by this pull request, feel free to also ask for input on decisions you are unsure about etc.
The changelog lines below will be copied directly from merged pull requests to the changelog on new releases. Changes are expected to preferentially link to the associated issues. If there are no relevant issues, link to the pull request itself.
If there are no relevant changelog entries replace the code block below with "None." followed by an optional reason for why it is not needed.
# Difftastic ignores ALL comments; retain compiler/bundler/coverage directives.
if grep -Eq '@ts-|@jsx|@internal|<reference\b|<amd-|sourceMappingURL|sourceURL|webpack|vite|[#@]__|/\*!|(istanbul|c8|v8)[[:space:]]+ignore' "$before" "$after"; then
exit 0
fi
case "$file" in
*.js|*.cjs|*.mjs|*.jsx)
# JavaScript JSDoc can contain type information.
if grep -Fq '/**' "$before" "$after"; then exit 0; fi ;;
esac
if ! difft --ignore-comments --check-only --exit-code "$before" "$after"; then
# This workflow will do a clean install of node dependencies, cache/restore them, build the source code and run tests across different versions of node
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-nodejs-with-github-actions
name:Node.js CI
on:
push:
pull_request:
# Superseded runs of a branch or pull request are cancelled. Runs on master
# each get their own group so every commit there is tested.
Clusterio manages distributed Factorio servers through a controller, hosts, and plugins.
Read [contributing](docs/contributing.md) for the project map, setup, commands, debugging, tests, and code style.
For PR reviews, follow the [review guide](docs/pr-review-guide.md), including its candidate ledger and final evidence audit.
Use the detailed guides for the affected area:
- [Communication and routing](docs/communication.md), [wire protocol](docs/devs/protocol.md), and [configuration](docs/config-system.md).
- [Plugins](docs/writing-plugins.md), [plugin migration](docs/devs/plugin-migration-to-2.0.md), and [Factorio/Lua](docs/developing-for-clusterio.md).
- [Compatibility policies](docs/decisions.md) and [releases](docs/release-workflow.md).
## Working rules
- Inspect `git status` before editing.
Preserve unrelated work and external plugin repositories; continue an existing task branch.
- Read the relevant source and tests before changing behavior.
If documentation and implementation disagree, report the discrepancy and distinguish observed behavior from intended policy.
Update affected documentation in the same change.
- Follow nearby code and `.oxlintrc.json`: tabs, double quotes, semicolons, and `.js` extensions in relative TypeScript imports.
Import shared public APIs through `@clusterio/lib` and check Node/browser exports when changing them.
- Run commands from the root using the pinned pnpm version and the [documented Node requirements](docs/contributing.md#supported-nodejs-and-factorio-versions).
Install with `pnpm install --no-frozen-lockfile`; the lockfile is intentionally ignored.
-`pnpm build` compiles TypeScript.
It does not regenerate validators or production web bundles; use the appropriate package's `prepare` script.
Edit sources, not generated outputs.
- Start validation with focused tests, then run `pnpm test` and `pnpm lint` for code changes when possible.
Root tests include integration setup even with fast/external exclusions.
The harness needs Factorio, recreates `temp/test/`, and shares ports 8880/4443; do not run competing suites.
- Report checks, failures, skips, and unverified behavior precisely.
Documentation-only changes need path, command, and formatting checks.
- Preview cleanup with `node clean.js --dry-run`; see the [cleanup scopes](docs/contributing.md#cleaning-generated-and-runtime-files).
Runtime-data deletion and release publication require explicit authorization.
Keep local configs, credentials, runtime data, and generated output out of commits.
- Follow the [PR template](.github/pull_request_template.md), including `### Changelog` (`None.` when appropriate).
Clusterio is a clustered Factorio server manager that provides the tooling for implementing cross server interactions in Factorio.
It was previously best known for implementing cross server transfer and cloud storage of items via teleporter chests.
But this functionality has been pulled out of Clusterio into its own plugin for Clusterio named [Subspace Storage](https://github.com/clusterio/subspace_storage).
By itself Clusterio doesn't change the gameplay in any way, you could
even use Clusterio to manage completely vanilla Factorio servers.
Plugins do the work of modding in the visible changes into the game, see
the [Plugins section](#plugins) for ready made plugins you can install
into a Clusterio cluster.
By itself Clusterio doesn't change the gameplay in any way, you could even use Clusterio to manage completely vanilla Factorio servers.
Plugins do the work of modding in the visible changes into the game, see the [Plugins section](#plugins) for ready-made plugins you can install into a Clusterio cluster.
## Features
- Clustered Factorio server management allowing you manage the running
of Factorio servers across a fleet of physical servers from both a web
interface and a command line interface.
- Clustered Factorio server management allowing you manage the running of Factorio servers across a fleet of physical servers from both a web interface and a command line interface.
- User list management for synchronizing in-game admins, whitelisted
users, and bans to all the servers in the cluster.
- User list management for synchronizing in-game admins, whitelisted users, and bans to all the servers in the cluster.
- Integrated support for exporting statistics for the whole cluster to
Prometheus via a single metrics endpoint.
- Integrated support for exporting statistics for the whole cluster to Prometheus via a single metrics endpoint.
- Extensive plugin support for adding your own cross server features to
Factorio using Clusterio's communication backbone.
- Extensive plugin support for adding your own cross server features to Factorio using Clusterio's communication backbone.
## Plugins
The heart of Clusterio is its plugin syste. Plugins add functionality
to Factorio servers, Clusterio itself or both. These are the plugins
supported and maintained by the Clusterio developers:
The heart of Clusterio is its plugin system.
Plugins add functionality to Factorio servers, Clusterio itself or both.
These are the plugins supported and maintained by the Clusterio developers:
- [Global Chat](/plugins/global_chat/README.md): share the in-game chat
Provide shared storage that can transport items between servers via
teleport chests.
- [Global Chat](/plugins/global_chat/README.md): share the in-game chat between servers.
- [Research Sync](/plugins/research_sync/README.md): synchronize research progress and technologies unlocked and the research queue between servers.
- [Statistics Exporter](/plugins/statistics_exporter/README.md): collect in-game statistics from all the servers and makes it available to the Prometheus endpoint on the controller.
- [Subspace Storage](https://github.com/clusterio/subspace_storage): Provide shared storage that can transport items between servers via teleport chests.
- [Player Auth](/plugins/player_auth/README.md): Provides authentication to the cluster via logging into a Factorio server.
- [Player Sync](/plugins/player_sync/README.md): Synchronizes your inventory and player settings between servers.
Want to make your own plugin? Check out the documentation on [Writing
Plugins](/docs/writing-plugins.md) for where to start.
There's also plugins developed and maintained by the community:
- [Discord Bridge](https://github.com/Hornwitser/discord_bridge) (@hornwitser/discord_bridge): Bridges chat between instances and Discord.
By Hornwitser.
- [Server Select](https://github.com/Hornwitser/server_select/tree/clusterio-2.0) (@hornwitser/server_select): In-game GUI for connecting to other servers in the cluster.
Originally by Godmave, ported to 2.0 by Hornwitser.
- [ExpCluster](https://github.com/explosivegaming/ExpCluster) (@expcluster/*): A collection of plugins and scenario features including role and group syncing, moderation tools, quality of life additions, and discord account linking.
See repo readme for plugin breakdown. Maintained by Cooldude2606.
- [Chat Relay](https://github.com/PHIDIAS0303/chat_relay) (@PHIDIAS0303/chat_relay): Chat relay of Discord and Factorio, seperate channel.
The installer offers the six plugins maintained in this repository.
Community plugins are versioned and released independently of Clusterio; check their compatibility before installing them manually.
Want to make your own plugin?
Check out the documentation on [Writing Plugins](/docs/writing-plugins.md) for where to start.
If you have a plugin written for an earlier 2.0 alpha, see [Migrating Plugins to 2.0](/docs/devs/plugin-migration-to-2.0.md).
Note that from 2.0 onwards minor versions of Clusterio may contain breaking changes for plugins, while breaking changes for users running a cluster are held back for major versions.
## Ubuntu setup
## Installation
**Warning**: These instructions are for the unstable master version and is not
recommended for use, see [the 1.2.x branch][1.2.x] for how to install the stable version.
Install the latest [Node.js LTS release](https://nodejs.org/en/download), which is recommended for new Clusterio installations.
Clusterio is distributed via npm and comes with a guided install script.
If you already have a supported Node.js version installed, you can set it up in a new directory with:
Clusterio runs on Node.js v12 and up, and v10.16.0+. Node.js itself is not
supported on EOL Ubuntu releases so make sure you're on a recent release of Ubuntu.
npm init "@clusterio"
Master and all slaves:
Otherwise see below for OS specific instructions.
Check out all our of decisions for [node, factorio, and operation support](/docs/decisions.md#supportability).
The -v flag is used to specify the instance directory. Your instances (save files etc) will be stored there.
The -v flag is used to specify the instance directory.
Your instances (save files etc) will be stored there.
-->
## Windows setup
### Windows setup
**Warning**: These instructions are for the unstable master version and is not
recommended for use, see [the 1.2.x branch][1.2.x] for how to install the stable version.
1. Download and install the latest [Node.js LTS release](https://nodejs.org/en/download).
**Requirements**
2. Create a new empty directory for the installation and navigate into it.
Open a PowerShell window in this new directory by Shift+Right clicking inside it and choosing "Open PowerShell window here" and then run the following command.
download and install nodeJS 12 from http://nodejs.org. Clusterio runs on Node.js v12 and
up, and v10.16.0+.
npm init "@clusterio"
**Master**
Make sure to note down the admin authentication token it provides at the end as you will need it later.
1. Open PowerShell or Command prompt in the directory you want to install to and run the following commands.
3. If you chose to use local factorio directory for the Factorio installation, download the Windows 64-bit zip package from https://www.factorio.com/download and extract it to the `factorio` folder in your clusterio installation folder.
### MacOS Setup
1. Install the latest [Node.js LTS release](https://nodejs.org/en/download).
If you use [nvm](https://github.com/nvm-sh/nvm#installing-and-updating), run `nvm install --lts`.
2. Open Terminal or Command prompt in the directory you want to install to and run the following commands.
2. Obtain Factorio by either of these two methods:
Make sure to note down the admin authentication token it provides at the end as you will need it later.
- Via the stand alone version on from their website
1. Create a new folder named "factorio" in the the factorioClusterio
folder.
2. Download the MS Windows (64-bit zip package) from
https://www.factorio.com/download .
3. Open the zip file and drag the folder called "Factorio_x.y.z" into
the factorio folder created in step 1.
- Via steam installation
1. Create a new folder named "factorio" in the the factorioClusterio
folder.
2. Locate the game files by right clicking the game in steam,
selecting properties, then Local Files, then Browse local files.
3. Go to the parent folder of the folder that Steam opened and copy
the Factorio folder into the factorio folder created in step 1.
3. If you chose to use local factorio directory for the Factorio installation, you will need to obtain and copy a mac version of Factorio and unpack it to the `factorio` folder in your clusterio installation folder.
## Installing Plugins
### Installing Plugins
Installing plugins to make them work with Clusterio consists of two
steps. First install the package via npm, for example
For well-known plugins you can select them during installation and no further steps are necessary to make them work.
Plugins that are not offered by the installer can be installed by the name of the npm package that provides them, for example
Then tell clusterio that this plugin exists by adding it as a plugin.
This installs the package with npm and adds it to the default `plugin-list.json` file which in the shared folder setup is loaded by controller, host and ctl.
The command must be run from the directory clusterio was installed in and will refuse to run anywhere else.
If you have hosts or ctl installed on separate computers (or directories) then you need to repeat the plugin install process for all of them.
The clusteriocontroller, clusteriohost and clusterioctl commands has the plugin sub-command so you do not need to install clusteriocontroller to add plugins, instead use the clusterio command you have available.
This adds it to the default `plugin-list.json` file which in the shared
folder setup is loaded by master, slave and ctl. If you have slaves or
ctl installed on separate computers (or directories) then you need to
repeat the plugin install process for all of them. The clusteriomaster,
clusterioslave and clusterioctl commands has the plugin sub-command
so you do not need to install clusteriomaster to add plugins.
For development purposes the `plugin add` command supports adding
plugins by the absolute path to them, or a relative path that must start
with either . or .. (which will then be resolved to an absolute path).
For development purposes the `plugin add` command supports adding plugins by the absolute path to them, or a relative path that must start with either . or .. (which will then be resolved to an absolute path).
## Configure Master Server
## Configure Controller
By default the master server will create a self signed TLS certificate
and listen on HTTPS on port 8443. You can change the port used with the
command
By default the controller will listen for HTTP on port 8080.
You can change the port used with the command
npx clusteriomaster config set master.https_port 1234
npx clusteriocontroller config set controller.http_port 1234
When changing the port you will also need to change the address hosts connect with.
For the standalone installation mode you can use
npx clusteriohost config set host.controller_url http://localhost:1234/
If you plan to make your cluster available externally set the address
that it will be accessible under with, for example
npx clusteriomaster config set master.external_address https://192.0.2.4:8443/
npx clusteriocontroller config set controller.public_url http://203.0.113.4:1234/
Change the url to reflect the IP, protocol, and port the master server
is accessible under, dns names are also supported. If proxyed under a
sub-path include the sub path as well and make sure the URL ends with a
slash.
Change the url to reflect the IP, protocol, and port the controller is accessible under, dns names are also supported.
If you're planning on making the controller accessible on the internet it's recommended to set up TLS, see the [Setting Up TLS](/docs/setting-up-tls.md) document for more details.
You can list the config of the master server with the `npx
clusteriomaster config list` command. See the [readme for
@clusterio/master](/packages/master/README.md) for more
information.
You can list the config of the controller with the `npx clusteriocontroller config list` command.
See the [readme for @clusterio/controller](/packages/controller/README.md) for more information.
### Hosting Behind Proxy
## Managing Factorio mods
If you want to host the master server behind a reverse proxy the
recommended setup is to disable HTTPS and host it on a high port
over HTTP to which you point your proxy to. For example to configure
port 8080 as the HTTP port for the master server.
Mods in Clusterio are managed as parts of mod packs.
On the Mods page of the web interface, use **Search** to browse the Factorio Mod Portal and download mods to the controller, or upload mod zip files directly.
Mod Portal downloads require the controller's [Factorio account credentials](docs/configuration.md#controllerfactorio_username) to be configured.
Use **Create** in the Mod Packs section to create a mod pack, then add the mods you want to use.
Assign the mod pack to an instance under Instance config -> Factorio -> Mod pack, or set a default mod pack for new instances in the controller settings.
npx clusteriomaster config set master.http_port 8080
The proxy can then forward requests to a sub-path like `/cluster/` to
`http://localhost:8080/`.
Mod settings can be managed under the modpack configuration page in the web UI. For mod settings to show, you first need to run an export. This is done by creating an instance with the modpack assigned, opening the instance page and pressing More -> Export data.
Additianlly WebSocket connections to api/socket relative to the sub-path
the interface is hosted under also needs to be forwarded. See below for
the specifics of setting this up on Apache.
Once this is done, the mod settings will show up under the modpack configuration page.
#### Apache Config
## Setting up shared storage
To proxy with Apache make sure that proxy_module, proxy_http_module and
proxy_wstunnel_module is loaded in the config, then add the following
directives to wherever is the approriate place for your flavour of
Apache config organization.
The chests teleporting items to and from the shared storage on the controller has been moved into the Subspace Storage mod and plugin.
To get this to work, install the Subspace Storage plugin on the controller and hosts using the [plugin installation instructions](#installing-plugins).
Download or upload the [Subspace Storage mod](https://mods.factorio.com/mod/subspace_storage) and the [Clusterio Lib mod](https://mods.factorio.com/mod/clusterio_lib) to the controller, add them to a mod pack, and assign it to each participating instance as described in [Managing Factorio mods](#managing-factorio-mods).
Hosts fetch the mods from the controller and synchronize each instance's mods folder with its assigned mod pack when the instance starts.
```apache
<Location/cluster/>
ProxyPass"http://localhost:8080/"
</location>
<Location/cluster/api/socket>
ProxyPass"ws://localhost:8080/api/socket"
</Location>
```
Pay attention to the use of trailing slashes, they are important.
### Setting up an admin account
Before you can manage the cluster you need to bootstrap an admin account
for it. Replace `<username>` with your Factorio username (do not make
The first command creates a user account with the given name and
promotes it to a cluster admin. The second one sets up a
`config-control.json` config for clusterioctl to connect to the master
server under the given user account.
Do not install the [Clusterio mod](https://mods.factorio.com/mod/clusterio), this was for the old 1.2.x version of Clusterio and is not compatible with Clusterio 2.0.
## Running Clusterio
After following the installation and master configuration instructions
you can use the following commands to run Clusterio.
After completing the installation and creating your admin user, start up the controller and at least one host separately.
The installer provides the `run-controller` and `run-host` scripts to make this simple.
Once the controller process is running you can log into the web interface which is hosted by default on http://localhost:8080/ (adjust the port number if you changed it), use the admin authentication token provided from the installation to log in.
The first time a host starts it prints a link for approving it, open the link and click Approve.
The basics of setting up a Factorio server from the web interface is to create an instance, assign it to a host and then click start. You can also use the "Connect via Steam" button to launch Factorio directly and connect to the server using Steam's protocol handler.
### Running via Systemd
The install script creates systemd service scripts for clusteriocontroller and clusteriohost (if applicable) to start up as the user than ran the installer.
If you have copied these files over to `/etc/systemd/system/` then you can startup Clusterio as a background service using:
#### To start as a service run:
sudo systemctl start clusteriocontroller # for the controller
sudo systemctl start clusteriohost # for each physical server intended to host Factorio servers
#### To automatically get it to start on boot:
sudo systemctl enable clusteriocontroller # for the controller
sudo systemctl enable clusteriohost # for each physical server intended to host Factorio servers
### Master Server
## Setting up remote hosts
It's necessary to run the master server in order for anything to work.
Once you've completed the setup run the following command to start it
up:
Run the installer as described in the installation section and choose "Host only" as the operating mode to install, it'll ask for a name for the host and a controller URL.
The URL is the same as what is needed to connect to the web interface.
Then start the host, the first time it starts it prints a code and waits for the host to be approved:
npx clusteriomaster run
Waiting for approval of "my-host" with code ABC-DEF, open this link to approve it:
https://example.com/hosts?approve=ABC-DEF
Or run: npx clusterioctl host approve ABC-DEF
Open the link to approve it in the web interface, where you can also change its name, or click Add host on the Hosts page and approve the request with the same code, or run `npx clusterioctl host approve <code>`.
The host then saves its connection details and starts up, it should show up in the Hosts list and be available for assigning and running instances on.
Only approve a request if its code matches the one shown by the host, approving gives the host full access to the cluster.
Approving requires the `core.host.approve` permission, which the Cluster Admin role has.
If you have that permission you can skip the wait by starting the host with `npx clusteriohost run --user-token <token>` and your own authentication token, it is only used to approve the host and is not saved.
To remove a host from the cluster, click Revoke host access on the page for the host in the web interface, or run `npx clusterioctl host revoke <host>`.
This disconnects the host, its instances become unassigned and its saved connection details stop working.
To add the computer back to the cluster start the host again, it asks for approval again.
### Slaves
## Setting up clusterioctl
Slaves connect to the master server and are managed remotely from the
master server. In order for slaves to connect to the master server they
need a valid authentication token, you can create a slave config with
a valid token with the following command.
There's a command line interface available for Clusterio which is installed separately with the same installer as for the controller and host.
Run the installer as described in the installation section and choose "Ctl only" as the operating mode to install, you can do this in the same directory as you have installed other Clusterio component(s) to.
The installer will ask for a controller URL and an authentication token, these are the same as you would use to connect to the web interface.
If you want to use a different user for the command line interface, you can generate an authentication token for an existing user with
npx clusterioctl slave create-config --name Local --generate-token
This will write a new `config-slave.json` file in the current directory
(you can change the location with the `--output` option) with the name,
token and url to connect to the master server with. If you are making
the config for a remote slave you will need to have set the
`master.external_address` option to the URL the master server can be
reached on.
You can list the config of a slave on the slave itself with the `npx
clusterioslave config list` command. See the [readme for
@clusterio/slave](/packages/slave/README.md) for more information.
Instances are created, managed and started from the master server. For
now the only interface available is the `clusterioctl` command line tool
included in Clusterio. You can run this tool from any slave, or the
master server without having to set up a config, if you want to manage
the cluster from somewhere else you will need to set the
`control.master_url` and `control.master_token` options with the `node
clusterioctl control-config set` command:
The basic operations to start a new instance is the following
npx clusterioctl instance create "My instance"
npx clusterioctl instance assign "My instance" "Local"
npx clusterioctl instance start "My instance"
The first line creates the instance configuration on the master server.
The second assigns the instance to a slave which creates the instance
directory and files needed to run the instance on the given slave. The
third line starts the instance, which creates a new save if there are no
save games present.
There are many more commands available with clusterioctl. See the
[Managing a Cluster](/docs/managing-a-cluster.md) document or
`npx clusterioctl --help` for a full list of them.
-`@hornwitser/server_select@2.0.1` fails to load with Clusterio 2.0 because its entrypoints omit the required `.js` extension.
`@hornwitser/discord_bridge@0.2.5` is not automatically discovered because it lacks the `clusterio-plugin` keyword.
The Clusterio 2.0.2 installer still offers these versions; leave them unchecked until compatible plugin releases are available.
If already installed, stop the controller and host, run `npm uninstall @hornwitser/server_select @hornwitser/discord_bridge` in each affected installation directory, and restart them to continue with the core plugins.
See [#1102](https://github.com/clusterio/clusterio/issues/1102).
- After updating Factorio you may get the error `Error: Expected empty response but got "Cannot execute command. Error: [string "clusterio_private.update_instance(..."]:1: attempt to index global 'clusterio_private' (a nil value)?` on instance start.
This is due to Factorio applying a migration replacing the patched content of the save with the original scenario, attempting to start the instance again should resolve the issue.
## Common problems
### EACCESS [...] LISTEN 443
Some systems don't let non root processes listen to ports below 1000. Either run with `sudo` or change config.json to use higher port numbers.
Some systems don't let non root processes listen to ports below 1000.
Either run with `sudo` or change config.json to use higher port numbers.
According to [this link](https://askubuntu.com/questions/839520/open-port-443-for-a-node-web-app) if you manually installed node.js following the above instructions, you may need to run the following command to fix this issue:
### Portforwarding doesn't work on the master server when running under WSL
### Portforwarding doesn't work on the controller when running under WSL
If you follow the ubuntu guide on WSL (Windows Subsystem for Linux, Bash on Ubuntu on Windows specifically), you will find that the website works on localhost and on your local ip, but not on the global ip. This is also true when you correctly port-forwarded the correct ports. Even when routing this server through nginx in WSL, the issue persists. Then, on a hunch, I tried to run nginx from windows itself and found that this DID work. It came to me that the only usage difference between the 2 versions of nginx is that I got a Windows Firewall popup.
If you follow the ubuntu guide on WSL (Windows Subsystem for Linux, Bash on Ubuntu on Windows specifically), you will find that the website works on localhost and on your local ip, but not on the global ip.
This is also true when you correctly port-forwarded the correct ports.
Even when routing this server through nginx in WSL, the issue persists.
Then, on a hunch, I tried to run nginx from windows itself and found that this DID work.
It came to me that the only usage difference between the 2 versions of nginx is that I got a Windows Firewall popup.
<!-- Exported from clusterio_2.0_block_diagram.drawio with draw.io Desktop 32.4.1. Export settings: SVG, light theme, white background, width 1000. Native SVG labels: disable Formatted Text and Word Wrap in an export copy, preserving the title at font size 20. -->
<descid="diagram-description">A controller and two hosts run across two physical servers. Each host manages three Factorio servers. The hosts and web or command-line clients connect to the controller over WebSocket.</desc>
<titleid="diagram-title">Clusterio 2.0 system architecture</title>
<pathd="M 600 400 L 600 320 L 190 320 L 190 252.74"fill="none"stroke="#006eaf"stroke-width="2"stroke-miterlimit="10"pointer-events="stroke"style="stroke: rgb(0, 110, 175);"/>
<pathd="M 190 242.24 L 197 256.24 L 190 252.74 L 183 256.24 Z"fill="#006eaf"stroke="#006eaf"stroke-width="2"stroke-miterlimit="10"pointer-events="all"style="fill: rgb(0, 110, 175); stroke: rgb(0, 110, 175);"/>
</g>
</g>
<gdata-cell-id="5kRB54Jy1bSgiNgllITD-37">
<g>
<pathd="M 700 550 L 640 550 L 640 460"fill="none"stroke="#c73500"stroke-width="2"stroke-miterlimit="10"pointer-events="stroke"style="stroke: rgb(199, 53, 0);"/>
<sub>This document describes the communication layer in Clusterio and
unless you're developing Clusterio it's probably not going to be very
useful for you.</sub>
Communication is facilitated by the lib/link library and is based on
three core abstractions: links, connectors and messages. Links
represents one side of a two way communication pipe, connectors deal
with connecting and establishing communication, and messages are the
content sent over the link.
Clusterio uses links, connectors, and message classes to communicate between controllers, hosts, instances, and control clients.
A link sends and receives messages through a connector, while message classes define the requests and events that can be exchanged.
See the [system overview](how-it-works.md#system-overview) for a diagram of the processes and their connections, and the [protocol reference](devs/protocol.md) for wire formats and session flow.
Link
----
## Links and routing
The Link class represents one side of a two way communication channel
where the actual communication is done by a connector associated with
the link. The link also knows what the endpoint types it's connected to
are via the source and target attributes.
[Link](../packages/lib/src/link/link.ts) handles request/response matching, event dispatch, message registration, and routing through an attached router.
Its connector identifies the local and remote endpoints through `connector.src` and `connector.dst`, both `Address` objects.
The endpoint types currently implemented are master, slave, instance,
and control. Although slave and instances run in the same Node.js
program they use virtual links to communicate between them in order to
simplify the communication architecture.
The controller's `WsServer` stores one link for each host or control connection.
On a host, the `Host` object is a link to the controller, and each `Instance` is linked to a corresponding `InstanceConnection` stored on the host.
The command-line client's `Ctl` object and the web interface's `Control` object each act as a link to the controller.
Upon creation the link will register with the connector in order to
receive and process messages from it. The messages received are
validated using the validator registered for the type of the message. A
message for which there is no validator for is considered an error.
Requests can pass through several links before reaching their destination.
Each receiving link checks the message name, allowed endpoint types, and source address before routing it.
The controller checks the user's permissions before validating the message data; permission callbacks therefore receive untrusted data.
See [message permissions](writing-plugins.md#static-permission).
At the destination, the registered schema validates the payload and `fromJSON` restores the message before its handler runs.
Routing can perform work before the destination handler runs.
In particular, the [host router](../packages/host/src/Host.ts) can start or wake an instance for messages marked as instance-starting.
Connector
---------
## Connectors and sessions
The connector used with a link is responsible for establishing and/or
maintain the connection that messages are sent over. Reconnect logic is
expected to be implemented in the connector, though this has not yet
been done.
[Connectors](../packages/lib/src/link/connectors.ts) transport messages, assign sequence numbers, and deliver incoming messages to links.
WebSocket connectors connect hosts and control clients to the controller and validate incoming message structures, including their type and addresses.
Hosts and instances share a Node.js process and communicate in memory through paired `VirtualConnector` objects.
There's also the virtual connector which lets two links in the same
program be connected to each other without having to establish a
network connection.
The protocol reference describes [connection and recovery behavior](devs/protocol.md#connection-flow), [connector states and events](devs/protocol.md#connector-flow), and [graceful disconnects](devs/protocol.md#graceful-disconnect).
For handling these events in a plugin, including queueing work and preparing for disconnection, see [connection handling](writing-plugins.md#handling-connection-events).
Message
-------
## Message definitions and registration
There's two types of messages currently implemented apart from the
Link builtin close message and that's Request and Event. The builtin
messages are defined in
[packages/lib/link/messages.js](/packages/lib/link/messages.js) and are
attached to links via the attachAllMessages function, which is usally
called in the Link subclass constructor.
Core request and event classes live under [packages/lib/src/data/](../packages/lib/src/data/).
Their static `type`, `src`, and `dst` properties specify the message type and allowed endpoint types, while their schema and permission fields define the payload and access rules.
Plugin classes also supply a `plugin` name.
The messages define which links they are valid on and only attaches
handlers for those links. There's also a forwarding mechanism for when
a request or event is to be forwarded another hop.
`Link.register` registers request and event classes and compiles their payload validators.
Core classes are registered by the link module; `registerPluginMessages` registers each plugin's `messages` array.
It also registers the plugin's `errors` array with `Link.registerError`, so an error thrown by a request handler is rejected on the requester as the same class.
Incoming messages invoke handlers attached with `link.handle(MessageClass, handler)`.
Plugins attach these handlers from their entrypoint functions rather than relying on method-name conventions.
Send a message with `link.sendTo(destination, new MessageClass(...))`.
Requests return a promise for the response; events do not return a handler result.
Use broadcast destinations only for events.
When using a request's source as a later event destination, remove its request ID with `Address.withoutRequestId()`.
See [writing plugins](writing-plugins.md#defining-link-messages) for a complete example.
When changing a wire message structure in [messages_core.ts](../packages/lib/src/data/messages_core.ts), rebuild the packages with `pnpm -r run prepare` to regenerate the standalone validator.
When changing a request or event class, keep serialization, allowed endpoints, permission checks, registration, and handlers consistent.
The [contributor guide](contributing.md#checking-your-code-for-errors) explains how to run focused tests.
<sub>This document describes the implementation and inner working of the
configuration system in Clusterio, and unless you're developing
Clusterio it's probably not going to be very useful for you.</sub>
<sub>This document describes the implementation and inner working of the configuration system in Clusterio, and unless you're developing Clusterio it's probably not going to be very useful for you.</sub>
Clusterio uses a group based configuration system to manage settings for
the master server, instances, slaves, and the clusterioctl utility.
Plugins get their own config group for the master server and instances
which they can add their own fields to. The fields are defined early in
the startup, and the groups and their fields for disabled plugins will
still be created.
Clusterio uses a field based configuration system to manage settings for the controller, instances, hosts, and the clusterioctl utility.
Plugins can add their own config fields for the controller, host, instance, and ctl configs.
The fields are defined early in the startup, and fields for disabled plugins will still be created.
The built-in config groups and their fields are defined in
while plugins can add their own config groups to the `MasterConfigGroup`
and `InstanceConfigGroup` properties of the `info.js` export using
subclasses of `PluginConfigGroup`. The plugin group has the enabled
field pre-defined on it, and it's used to enable/disable plugins.
The built-in config fields are defined in [packages/lib/src/config/definitions.ts](/packages/lib/src/config/definitions.ts) while plugins add fields through `controllerConfigFields`, `hostConfigFields`, `instanceConfigFields`, and `ctlConfigFields` on the `plugin` export.
Controller, host, and instance configs also receive `<plugin_name>.load_plugin` to enable/disable loading on startup and a hidden `<plugin_name>.version` for migrations.
Defing Fields
-------------
## Defining Fields
Fields are defined using the `define` class method on group classes
after the groupName have been set and before the class has been
finalized. The `define` method takes an object as argument with the
following properties:
Fields are defined as POJS object properties where the name of the property correspond to the name of the configuration field.
The property name is the primary way the field is accessed both in code and on the command line, as such it should follow the lower\_case\_underscore style and consist of a group name and a field name separated by a dot.
For plugins the group name is the name of the plugin, e.g., the `level` field of the `foo_frobber` plugin would be named `foo_frobber.level`.
**access**:
The locations this config field can be read and modified from.
This is an array of strings which can contain the values `"controller"`, `"host"` and `"control"`.
If a party is missing in this array then that party will not receive the value of this field when the config is shared, and will be unable to modify the field.
If not set any party can read and modify this config entry.
Note that instance config fields should have both `"controller"` and `"host"` or unexpected behaviour may occur.
**type**:
The type of config value this field will support. The supported
values are boolean, string, number, and object, and if an attept is
made to set the field to a value that is not of the right type it
will be reject unless it's a string that can be converted to the
right type.
**name**:
Name of the configuration field, this is the primary way the field
is accessed, both in code and on the command line. As such it
should follow the lower\_case\_underscore style and not contain any
spaces or dots.
The type of config value this field will support.
The supported values are boolean, string, number, and object, and if an attept is made to set the field to a value that is not of the right type it will be reject unless it's a string that can be converted to the right type.
**title** (optional):
Text used to identify the config field in user interfaces.
This should not contain the grouping used in `name`.
**description** (optional):
Text used to describe the config field in user interfaces.
**enum** (optional):
Array of acceptable values this field can have. Attempts to set a
value not in this array will be rejected, which makes it useful for
making enumareted list of choices.
Array of acceptable values this field can have.
Attempts to set a value not in this array will be rejected, which makes it useful for making enumareted list of choices.
**optional** (optional):
True if this field can be set to null. Defaults to false.
True if this field can be set to null.
Defaults to false.
**initial_value** (optional if optional is true):
**initialValue** (optional if optional is true):
Value this config field should take in a newly initialized config.
This can also be an async function returning the value to use.
This can also be asynchronous function returning the value to use.
The function is called with the config after its stored fields are loaded, so it can base the value on other fields.
If the value for a field stored in on disk somehow ends up being invalid
the field will take on the initial\_value instead when the config is
loaded.
**validator** (optional):
Function called with the value being set and the config it belongs to.
It should throw an error if the value is not acceptable, and the message of the error is passed on to whoever tried to set it.
Common validators such as `greaterThanZero`, `greaterThanEqualZero`, `integer` and `filePath` are available on `Config.validators`, along with `optional` to skip null and `all` to combine several.
The config is read only while the validator runs.
**dependsOn** (optional):
Names of other fields the validator reads the value of.
When one of those fields is changed the validator of this field is run again against the new value, and the change is rejected if it fails.
**restartRequired** (optional):
True if a restart of the controller, host or instance the config belongs to is needed for changes to take effect.
Changing such a field sets the `restartRequired` flag on the config, which the web interface uses to show that a restart is pending.
Defaults to false.
**restartRequiredProps** (optional):
For object fields, names of properties that have the opposite of `restartRequired`.
Only used by the web interface to show which properties need a restart, `factorio.settings` uses it to mark the settings that are applied live.
**credential** (optional):
Locations allowed to read the value of this field.
Other locations can still change the value if allowed by `access`, which is what you want for tokens and passwords.
**readonly** (optional):
Locations allowed to change the value of this field.
Other locations can still read the value if allowed by `access`.
**hidden** (optional):
True if the web interface should not show this field.
Used for internal values that are managed by Clusterio.
**autoComplete** (optional):
Value for the autocomplete attribute of the input in the web interface, e.g. `section-factorio new-password`.
Only used for string and number fields.
**inputComponent** (optional):
Name of the input component the web interface should render for this field instead of the default for its type.
Plugins register components with the `control.hooks.inputComponents` hook in their web entrypoint.
The hook returns an object whose keys match the field's `inputComponent` value.
For example, with `inputComponent: "foo_frobber_level"` on a field:
`LevelInput` must implement the `InputComponent` contract exported by `@clusterio/web_ui`.
If the value for a field stored in on disk somehow ends up being invalid the field will take on the initial\_value instead when the config is loaded.
Accessing Fields
----------------
## Accessing Fields
On the Config instance there's two ways to access fields: through the
set and get methods using dot notation. E.g.:
On the Config instance there's two ways to access fields: through the set and get methods.
E.g.:
let value = configInstance.get("group.field");
Or by getting the group first and then calling set or get on that.
Or by through the raw fields object (which is best avoided as it doesn't check types or whether the accessed field exists).
E.g.:
let value = configInstance.group("group").get("field")
let value = configInstance.fields["group.field"]
The set method takes an extra argument, which is the value to set the
field to. If the value is not valid this will throw an InvalidValue
exception, if the group or field doesn't exist it will throw an
InvalidField exception.
The set method also takes an extra argument, which is the value to set the field to.
If the value is not valid this will throw an InvalidValueError exception, if the field doesn't exist it will throw an InvalidFieldError exception.
Config Lifecycle
----------------
## Config Lifecycle
Values for all config fields are created when the config is initialized,
as well as for any fields with missing values when a config is loaded.
Once a field has a value it is never automatically removed, even if the
field or group it was part of no longer exists. This ensures that
plugin configuration is not lost should the server be started up without
the plugins that defined the configuration installed.
Values for all config fields are created when the config is initialized, as well as for any fields with missing values when a config is loaded.
Once a field has a value it is never automatically removed, even if the field it was part of no longer exists.
This ensures that plugin configuration is not lost should the server be started up without the plugins that defined the configuration installed.
For instance configs, they are kept on the master server and serialized
and sent over to the relevant slaves on startup. This makes it possible
to edit instance configs when the slave the instance is on is offline.
A copy is also stored in the instance directory on the slave, and this
config will be deliverd to the master server on slave startup, which
enables instances to be copied between clusters as long as their IDs are
unique.
For instance configs, they are kept on the controller and serialized and sent over to the relevant hosts on startup.
This makes it possible to edit instance configs when the host the instance is on is offline.
A copy is also stored in the instance directory on the host, and this config will be deliverd to the controller on host startup, which enables instances to be copied between clusters as long as their IDs are unique.
- [Updating Your GitHub Fork](#updating-your-github-fork)
- [Publishing releases to npm](#publishing-releases-to-npm)
- [Changelog](#changelog)
- [Supported Node.js and Factorio Version](#supported-node.js-and-factorio-version)
- [Supported Node.js and Factorio Versions](#supported-nodejs-and-factorio-versions)
- [Code Style](#code-style)
- [Naming Style](#naming-style)
- [Units](#units)
- [Indenting](#indenting)
- [Line Length](#line-length)
- [Markdown](#markdown)
Workflow
--------
## Project map
While we don't require you to follow this workflow, it makes things much
simpler for both you and us if you choose to do so. Especially when it
comes to making the commit history reflect what happened and not be a
giant ball of mess. GitHub's git workflow is less than ideal for
dealing with small feature-branches so you might see things being done
here a little diffrent than the usual.
Clusterio manages distributed Factorio servers through a controller, hosts, and plugins.
It is recommended that you use the command line git version for doing
the operations described here. GUI wrappers tend to have different
ideas of what is what and you might find it difficult to make it do the
things described here. For doing the actual commits though, your
faviorit git GUI should work just fine.
| Location | Responsibility |
| --- | --- |
| `packages/controller/` | Cluster state, users and roles, connections, HTTP API, and serving the web UI. Start with `src/Controller.ts` and `src/ControlConnection.ts`. |
| `packages/host/` | Instance lifecycle, Factorio processes, save patching, and exports. Start with `src/Host.ts`, `src/Instance.ts`, `src/server.ts`, and `src/patch.ts`. |
| `packages/ctl/` | Command-line administration and plugin commands. |
| `packages/lib/` | Shared messages, routing, config, permissions, plugin loading, logging, and utilities for Node and browser consumers. |
| `packages/web_ui/` | React/Ant Design interface and browser control connection; bundled through the controller's Vite build. |
| `packages/create/` | Installer and JavaScript/TypeScript plugin templates. |
| `plugins/` | Bundled plugins, including their Node, web, Lua, and test code. |
| `external_plugins/` | Plugins developed in independent repositories and included in the local workspace. |
| `packages/host/modules/` | Lua modules injected into saves. |
| `packages/host/lua/` | Factorio mod sources, including `clusterio_lib`. |
| `test/` | Root test suite, fixtures, helpers, integration harness, and manual tests. Plugin tests also live under `plugins/*/test/`. |
If you pressed the fork button on GitHub and used `git clone` _on your
own fork_ you can skip to the next paragraph. Otherwise you will have
to click the fork button and clone your fork. If you already cloned the
main GitHub repository you can change your clone to point at your own
fork using the `git remote` command, but it's probably easier to delete
your local clone and clone it again.
Start with [how it works](how-it-works.md), [communication](communication.md), and the [config system](config-system.md) for architecture.
See [writing plugins](writing-plugins.md) and [developing for Clusterio](developing-for-clusterio.md) for plugin and Factorio contracts.
To keep your local repository up to to date with the main repository add
After this running `git fetch upstream` will get the updated branches
from the main repository. Do not run `git pull upstream` it will mess
thing up!
While we don't require you to follow this workflow, it makes things much simpler for both you and us if you choose to do so.
Especially when it comes to making the commit history reflect what happened and not be a giant ball of mess.
GitHub's git workflow is less than ideal for dealing with small feature-branches so you might see things being done here a little diffrent than the usual.
It is recommended that you use the command line git version for doing the operations described here.
GUI wrappers tend to have different ideas of what is what and you might find it difficult to make it do the things described here.
For doing the actual commits though, your faviorit git GUI should work just fine.
If you pressed the fork button on GitHub and used `git clone`_on your own fork_ you can skip to the next paragraph.
Otherwise you will have to click the fork button and clone your fork.
If you already cloned the main GitHub repository you can change your clone to point at your own fork using the `git remote` command, but it's probably easier to delete your local clone and clone it again.
To keep your local repository up to to date with the main repository add the main repository as a remote.
By default node does not enable source maps, to get stack traces pointing to the `.ts` files you will need to pass `--enable-source-maps` to `node` or set the environment variable `NODE_OPTIONS=--enable-source-maps`.
Note that to use `clusterioctl` you will have to create a ctl config first:
Run `node --enable-source-maps packages/controller run` and `node --enable-source-maps packages/host run` in separate terminals.
When editing TypeScript files ran by Node.js you need to run `pnpm build` to compile the files to JavaScript before restarting the controller/host/ctl for changes take effect.
To have TypeScript watch for changes and rebuild immediadly you may run `pnpm watch` instead.
For web development on the controller there are also the following flags:
`--dev` - Serve the `web_ui` package from source through Vite's development server with hot module replacement.
`--dev-plugin [name]` - Serve the web module for the given plugin from source the same way.
This also serves the `web_ui` package from source, as the plugin has to share its modules with it.
These options can be combined and multiple plugins can be actived for dev mode.
Keep in mind if you're working on external plugins the `--dev-plugin` option is likely to only work if the plugin is placed inside the `external_plugins` directory and its dependencies are installed with pnpm as part of the clusterio repository workspace.
### Build outputs
| Command | Output |
| --- | --- |
| `pnpm build` | Compiled JavaScript and type declarations in each package's `dist/node/` and, where applicable, `dist/browser/`. |
| `pnpm -r run prepare` | Compiled packages, generated validators, and production web bundles. |
| `pnpm build-mod` | Factorio mod archives in root `dist/`. |
`pnpm build` alone does not regenerate the standalone validator or web bundles.
After changing `packages/lib/src/data/messages_core.ts`, run `pnpm -r run prepare` to compile the schema and regenerate its validator.
### Debugging
Establish the controller, host, instance, Factorio version, enabled plugins, and exact reproduction first.
Enable source maps and confirm the affected process is running rebuilt code.
Use the contextual logger so host, instance, and plugin metadata remain available.
Development logs are in `logs/`; integration logs are in `temp/test/logs/` and are uploaded by CI.
For a failing request or event, start with its handler and the input that triggers the failure.
Handlers are registered with `handle(MessageClass, handler)`; searching for the message class will help you find its handler.
Inspect the input and the state used by the failing operation, then try to reproduce the failure in a [focused test](#checking-your-code-for-errors).
If the handler is never reached, investigate message registration, routing, and permission checks using the [communication guide](communication.md).
For errors reported by Factorio, start with the Lua stack trace and the module or mod it identifies; the [developing-for-Clusterio guide](developing-for-clusterio.md) explains how that code communicates with the host.
For connection and recovery bugs, test disconnect, reconnect, timeout, and shutdown as applicable.
A cached controller status does not prove the current state of a host or Factorio process.
Recovery mode disables normal plugin loading; account for it when diagnosing missing plugin behavior and [Lua state migrations](developing-for-clusterio.md#initialisation-and-recovery).
Verify UI behavior in the browser: successful compilation does not verify rendering, permissions, or subscriptions.
### Starting a Feature Branch
Development is done exclusively in feature branches. This means the
first thing to do before starting any development at all is to make a
new branch starting from the master or 1.2.x branch of the main
repository. To do this check out that branch from the main repository
(substitute master for 1.2.x if you're working on that branch.)
Development is done exclusively in feature branches.
To start a new feature branch, first check out master from the main repository.
git fetch upstream
git checkout upstream/master
This will give a warning about detached HEAD which means you did it
correctly. Now you can make a feature branch based on this with.
This will give a warning about detached HEAD which means you did it correctly.
Now you can make a feature branch based on this with.
git checkout -b my-feature-branch
The name of the branch is not important, as long as it's not a name
you've used before. But it's nice to have its name be somewhat
descriptive of what it implements.
The name of the branch is not important, as long as it's not a name you've used before.
But it's nice to have its name be somewhat descriptive of what it implements.
### Checking your code for errors
There's automated tests and linting setup for the project. Please do
add tests for any code you add and run both:
Tests use Mocha and Node assertions, mostly as JavaScript ESM importing compiled packages.
Run the test suite and linter from the repository root:
npm run test
npm run lint
pnpm run test
pnpm run lint
To check that your changes pass the integration tests as well as the
ESLint rules set up for the project.
`pnpm test` builds TypeScript and runs root and bundled-plugin tests, excluding `test/manual/`.
Run tests from the root: individual package `test` scripts are often placeholders that deliberately fail.
Use `pnpm cover` for the full suite with c8 coverage.
The full test suite can take a while to run.
Use `pnpm run fast-test` to skip tests marked as slow, or `pnpm run no-external-test` to skip tests that depend on external APIs such as the Factorio mod portal or GitHub.
These correspond to setting the `FAST_TEST` and `NO_EXTERNAL_TEST` environment variables and can be combined.
For example, `FAST_TEST=y NO_EXTERNAL_TEST=y pnpm test` combines both exclusions.
These flags still run the remaining integration setup and tests; neither produces an isolated unit-test run.
When writing slow tests call `slowTest(this)`, and for external API tests call `externalTest(this)`; both are exported from `test/common.js`.
For tests that need a Factorio installation, call `requiresFactorio(this)` in the test or pass the suite to `requiresFactorio`, imported from `test/integration/index.js`.
`test/common.js` also provides JSON round-trip/schema helpers, and `test/mock.js` provides shared mocks.
For a focused test, build first and select the file explicitly instead of appending it to the full-suite script:
Replace the example file with the relevant test; use `--grep` to narrow it further.
Preserve the test TLS CA configuration for HTTPS integration tests.
Use `--require ./test/setup_logging.js` when an isolated test needs the shared logger configured.
Some tests import `test/integration/index.js`, which installs root hooks and starts a controller and host even for a targeted invocation.
The integration harness recreates its configs, database, mods, and instances under `temp/test/` and uses HTTP port 8880 and HTTPS port 4443.
Do not run competing integration suites in the same checkout or on those ports.
It locates Factorio through `host.factorio_directory` in root `config-host.json`, defaulting to `factorio/`.
The full harness currently requires a usable Factorio installation: a missing directory fails discovery, and an empty directory fails setup while selecting a version.
Some individual tests have conditional skips, but these do not make a full run without Factorio succeed.
Use the relevant production bundle, mod build, or [manual scenario](../test/manual/README.md) for changes affecting those surfaces.
For documentation changes, check that links, paths, and example commands are correct.
### Cleaning generated and runtime files
Inspect `node clean.js --help` and preview the selected scope with `--dry-run` before cleaning.
| Command | Removal scope |
| --- | --- |
| `node clean.js --dry-run` | Preview workspace package/plugin build output and Vite dependency caches; omit `--dry-run` to clean, or use `pnpm clean`. |
| `node clean.js tests --dry-run` | Also preview root `.nyc_output/`, `coverage/`, the whole `temp/` directory, and `static/`. |
| `node clean.js all --dry-run` | Also preview runtime databases, instances, logs, mods, local configs, `FactorioAdminToken.txt`, and `plugin-list.json`. |
`all` deletes cluster runtime data, including configurations and saves; back up anything you want to keep before using it.
The cleaner excludes `external_plugins/` and does not remove mod archives in root `dist/`.
Do not commit local configs, tokens, databases, saves, logs, or generated build output.
### Submitting a Pull Request
Once you've commited changes to your feature branch that you want to
have included into the main repository push it to you fork with
Once you've commited changes to your feature branch that you want to have included into the main repository push it to you fork with
git push origin my-feature-branch
and make a pull request through the GitHub web interface. You can also
pass the `--set-upstream` (or `-u`) flag to make `git push` do the same
thing for this branch. After your feature branch have been merged you
should consider it final. If there are more changes you want to have
merged or mistakes you need to fix you must start a new feature branch
and submit a new pull request
and make a pull request through the GitHub web interface.
You can also pass the `--set-upstream` (or `-u`) flag to make `git push` do the same thing for this branch.
After your feature branch have been merged you should consider it final.
If there are more changes you want to have merged or mistakes you need to fix you must start a new feature branch and submit a new pull request.
Pull request descriptions should contain a `### Changelog` section with additions for the changelog contained inside a code block, see the [CHANGELOG.md](/CHANGELOG.md) file for the formatting of changelog entries and possible sections to use.
If there are no changelog entries to add, remove the code block and start the changelog section with the text `None.`, followed by an optional reason for why it is not is needed.
### Breaking Compatibility with the Last Release
The version-compat job in CI runs the local controller with the latest published host, and the published controller with the local host.
It fails if a link message changed its src, dst, data or response schema since that release.
Control is left out of src and dst, and messages that only go between control and the controller are not checked, as the web UI is served by the controller it talks to.
If the break is intended, add an entry to [scripts/version-compat-breaks.json](../scripts/version-compat-breaks.json) in the same pull request.
Breaking changes are only accepted if the next release is intended to include them, see our decision on [breaking changes](/docs/decisions.md#breaking-changes).
The entry is keyed by the published version directly before the breaking change was introduced, which is the latest release at the time.
Every entry needs the pull request number, a reason saying what breaks, and an impact saying why end users are not or only minimally affected.
```json
{
"2.0.3":{
"messages":{
"request FactorioVersionsRequest":{
"pr":1129,
"reason":"What breaks between the two versions.",
"impact":"Why end users are not or only minimally affected."
}
},
"connect":{"pr":1234,"reason":"Why a host cannot connect at all.","impact":"..."}
}
}
```
Listed messages are reported as warnings with the reason and impact instead of failing the job.
Only add `connect` when the published and local versions cannot connect at all, it skips starting the cluster.
Listing a message that did not change fails the job.
Entries only apply while their version is the latest release, after the next release they are ignored and can be removed.
Describe the break in the Breaking Changes section of the changelog.
### Editing your Feature Branch
Until your feature branch is merged you can edit it as you see fit by
using the rebase tool. If your feature branch has been merged however
it's final and you will have to start a new feature branch and submit
that in a new pull request. The base usage of rebase assuming you are
checked out on your feature branch is the following command (substitute
master with 1.2.x if your feature branch is based on that)
Until your feature branch is merged you can edit it as you see fit by using the rebase tool.
If your feature branch has been merged however it's final and you will have to start a new feature branch and submit that in a new pull request.
The base usage of rebase assuming you are checked out on your feature branch is the following command
git rebase -i upstream/master
This will open up an editor with the commits of your feature branch and
let you reorder, edit, and remove commits as you see fit.
This will open up an editor with the commits of your feature branch and let you reorder, edit, and remove commits as you see fit.
If the main repository has been updated and you have fetched down those
updates with `git fetch upstream` this will also move your feature
branch so that it is based on the latest commit from the main
repository.
If the main repository has been updated and you have fetched down those updates with `git fetch upstream` this will also move your feature branch so that it is based on the latest commit from the main repository.
If you have already pushed your feature branch to your fork a simple
`git push origin my-feature-branch` will no longer work as the histories
are different, instead you will have to run
If you have already pushed your feature branch to your fork a simple`git push origin my-feature-branch` will no longer work as the histories are different, instead you will have to run
git push origin +my-feature-branch
The + sign signifies a force push and should be used with care. If you
have a pull request that has not yet been merged submitted for this
feature branch this will update the pull request too.
The + sign signifies a force push and should be used with care.
If you have a pull request that has not yet been merged submitted for this feature branch this will update the pull request too.
### Resolving Merge Conflicts
The best way to resolve merge conflicts in feature braches is to rebase
it on top of the updated upstream. Run the following commands
(substitute master with 1.2.x if your feature branch is based on that)
The best way to resolve merge conflicts in feature braches is to rebase it on top of the updated upstream.
Run the following commands
git fetch upstream
git rebase upstream/master
This will stop on the commit(s) that caused the merge conflict and let
youresolve them manually before continuing. Once you're done force
push your updated feature branch with
This will stop on the commit(s) that caused the merge conflict and let you resolve them manually before continuing.
Once you're done force push your updated feature branch with
git push origin +my-feature-branch
### Updating Your GitHub Fork
No. This is pointless busywork that accomplishes nothing. Fetch the
main repository directly as described above and base your feature
branches on the refs from it. GitHub has no interface or functionality
for keeping the branches in your fork up to date with the branches in
the main repository and there is no point in doing this for our feature
branch based workflow.
No.
This is pointless busywork that accomplishes nothing.
Fetch the main repository directly as described above and base your feature branches on the refs from it.
GitHub has no interface or functionality for keeping the branches in your fork up to date with the branches in the main repository and there is no point in doing this for our feature branch based workflow.
Changelog
---------
### Publishing releases to npm
There's a [Changelog](../CHANGELOG.md) in the root of the project. If
you make changes that are visible to users of this project you should
add an entry to the changelog describing what has changed. The format
of this log should be self explanatory, please follow it carefully.
For publishing new releases see [package release workflow](./release-workflow.md#packages).
## Changelog
There's a [Changelog](../CHANGELOG.md) in the root of the project.
If you make changes that are visible to users of this project you should add an entry to the changelog describing what has changed.
The format of this log should be self explanatory, please follow it carefully. Please make sure to document any [breaking changes](/docs/decisions.md#breaking-changes).
Supported Node.js and Factorio Versions
---------------------------------------
## Supported Node.js and Factorio Versions
Clusterio run on Node.js v12 and up, and v10 from v10.16.0, make sure
to check that the Node builtin API's and npm packages you use are
supported on these versions. The Travis CI tests are runned on whatever
version of Node.js v10 is the default, which at the time of writing is
v10.18.0.
For development, use the latest Node.js LTS release, following the [installation instructions](../README.md#installation).
Make sure the Node.js built-in APIs and npm packages you use remain compatible with the supported versions.
The GitHub Actions tests run every Factorio version on the latest Node.js, and every other supported Node.js version with the latest Factorio only, which keeps the number of jobs within GitHub's concurrent job limit.
Coverage is collected on the oldest supported Node.js; see the [CI workflow](../.github/workflows/node.js.yml) for the version matrix.
For Factorio Clusterio 2.0 aims to support version 0.17.69 and up,
including the latest experimental release. It's recommended that you
use the lua API reference for 0.17.69, as there's no information on what
version Factorio API's were introduced in. The Travis CI tests are
runned against latest experimental release.
For Factorio, active support covers the latest stable version and the previous major version, using version lines such as 2.1, 2.0 and 1.1 as described in the [Factorio support policy](decisions.md#factorio).
Older versions from 0.17 onwards receive passive support where it does not restrict actively supported versions.
Check the Lua API documentation for the Factorio versions your changes target and preserve compatibility with the actively supported versions.
The GitHub Actions tests use pinned Factorio releases; see the [CI workflow](../.github/workflows/node.js.yml) for the current version matrix.
Check out all our of decisions for [node, factorio, and operation support](/docs/decisions.md#supportability).
Code Style
----------
The style of the code is a bit of a mixed bag at the moment. But there
are at least a few things that have been agreed upon.
## Code Style
The style of the code is a bit of a mixed bag at the moment.
But there are at least a few things that have been agreed upon.
See our [decisions document](/docs/decisions.md) for justifcations.
### Naming style
JavaScript variables in general uses camelCase. The exception is for
classes/constructors which use PascalCase and variables destructured
from over the wire objects.
JavaScript variables in general uses camelCase.
The exception is for classes/constructors which use PascalCase and variables destructured from over the wire objects.
Fields in configs and messages sent over links uses
lowercase_underscore.
Fields in configs uses lowercase_underscore.
JavaScript files are named using lowercase_underscore for files that
export multiple items. Files containing and exporting a single class
should be named the same as the class in PascalCase.
Imported modules from the lib package are prefixed with lib to make them
easier to distinguish, as the names tend to be very generic. E.g.:
JavaScript files are named using lowercase_underscore for files that export multiple items.
Files containing and exporting a single class should be named the same as the class in PascalCase.
Use ESM imports, including `.js` extensions in TypeScript relative imports.
For namespace imports within lib, use the `lib` prefix to distinguish generic module names, for example `import * as libFileOps from "./file_ops.js";`.
Production consumers should import shared public APIs through `@clusterio/lib` rather than reaching into its internals.
When adding public shared exports, check both `packages/lib/index.ts` and `packages/lib/browser.ts` and keep Node-only dependencies out of browser code.
For lua code lowercase_underscore is used for everything.
### Units
When naming variables whos unit is not a base SI unit like seconds or metres the unit used should be suffixed in the variable name:
E.g.:
const startTimeMs = Date.now();
const endNs = process.hrtime();
This makes it easier to spot different unit bases are being mixed in the code.
### Strings
JavaScript and Lua code should use double quoted strings (`"string"`)
for all strings with the only exception that strings that contain double
quotes may use single quotes (i.e., `'String with "quotes"'`).
JavaScript and Lua code should use double quoted strings (`"string"`) for all strings with the only exception that strings that contain double quotes may use single quotes (i.e., `'String with "quotes"'`).
JavaScript template literals may also be used where appropriate.
### Indenting
JavaScript code should be indented with tabs.
JavaScript and Lua code should be indented with tabs.
Lua code should be indented with 4 spaces.
Other files types should prefer tabs for indents where reasonable with the exception of Markdown.
### Line Length
Lines are limited to 120 characters where tabs count as 4 characters.
If the argument list for a function or expressino for compound statement
gets very long break them out on their own line(s). For example:
If the argument list for a function or expressino for compound statement gets very long break them out on their own line(s).
For example:
function foo(
really, long, argument, list,
@@ -262,3 +417,25 @@ gets very long break them out on their own line(s). For example:
// code
}
})
### Markdown
Markdown files are formatted using one line per sentence to improve diff reports when making changes.
Headings have two blank lines before them and one after them; the top heading has no blank lines before it, and consecutive headings have one blank line between them.
> The use of `Time Immemorial` indicates an obsverion of a previously undocumented decision with no known start date, which has remained unchallenged.
<!--
### Template
🧾 0th Month 2000 | Source
> A short sentence describing the action to take in the context of our project.
Write a breif description which includes: the reason a decision was needed, the outcome of the discussion, and the justification of the outcome.
Write using the active voice in third person "we".
Remember to have one sentence per line.
-->
## Documentation
### Decision Log
🧾 22nd December 2024 | Discord Voice Call
> Maintain a document that includes a brief description of the outcomes and the reasoning behind our decisions, along with links to the relevant sources for further reading.
We discovered that we were beginning to ask similar questions to other maintainers that we had previously raised, which led to a waste of time for all involved.
Consequently, we decided that a log of important decisions should be kept for the benefit of ourselves and future maintainers.
> When a function or it's arguments do not have a clear purpose, a TSDoc comment should be used to describe it.
While TypeScript provides useful type infomation in function hover text, it does not allow for comments about the purpose of functions and their arguments.
Therefore, we decided to use TSDoc comments for this purpose as it is well intergrated in many text editors and IDEs (such as VSCode).
Our specific implementation is TypeDoc because it is lightweight it and can output TSDoc comments as html.
API Extractor was also considered but not selected.
🧾 29th October 2026 | [Github PR #1147](https://github.com/clusterio/clusterio/pull/1147)
We dropped TypeDoc as the generated docs were never published and it does not support TypeScript 7.
TSDoc comments are still used for editor hover text.
## Code Style (General)
### Indenting
🧾 Time Immemorial
> Use tabs for indention. Except for Markdown where spaces carry meaning.
We needed to maintain a consistent whitespace style across all our files.
Therefore, we selected tabs as our standard.
### Line Length
🧾 Time Immemorial
> Lines are limited to 120 characters where tabs count as 4 characters.
We needed to maintain a consistent max line legnth across all our files.
Therefore, we selected 120 characters as our standard.
### Naming Booleans
🧾 25th November 2024 | [Discord Discussion](https://discord.com/channels/450361298220351489/450361298220351491/1310611960508977223)
> Variable and member properties which contain booleans should be named starting with a verb unless it ends with "ed", e.g. `canRestart` `expectEmpty` `hasFallback` `connected` `enabled` `loaded` `banned`
We had inconsistencies in our naming of booleans and aimed to standardise them.
There was little preference for any particular approach, so we chose the one that would require the fewest changes to existing code.
As a result, all boolean values should begin with a verb, unless they end with "ed." For clarity, both "is" and "has" are considered verbs.
### Naming Files
🧾 Time Immemorial
> Files use lowercase_underscore if multiple values are exported. If a single class is exported then it uses the class name in PascalCase.
We needed to maintain a consistent name style across all our files.
Therefore, we selected lowercase undercore for most files, with class files using PascalCase.
### Naming Times and Durations
🧾 18th January 2024 | [Discord Discussion](https://discord.com/channels/450361298220351489/450361298220351491/1197566542548828170)
> Variable and member properties which contain times or durations should end with their SI unit of messure, e.g. `updatedAtMs`
We found verifying the units of measurement used in different locations to be a simple yet repetitive task that could lead to errors.
Our main concern was the mixing of variables containing seconds and milliseconds.
Therefore, we decided that times and durations should explicitly include their units, whereas other value types do not require any units.
### Strings
🧾 Time Immemorial
> Strings should use double quotes `"` unless the string contains them, in which case single quotes `'` can be used. For JS string templates use backticks ``` ` ``` as normal.
We needed to maintain a consistent string style across all our files.
Therefore, we selected double quotes as our standard.
## Code Style (TS / JS / Web)
### Oxlint
🧾 20th October 2026
> Use Oxlint for style checking, with formatting rules from `@stylistic/eslint-plugin`.
We needed to maintain a consistent style across all our files and used ESLint for it from the start.
Oxlint runs the same rules about three times faster, reads ESLint disable comments, and does not depend on the TypeScript compiler API.
The formatting rules ESLint deprecated run through `@stylistic/eslint-plugin` loaded as an Oxlint JS plugin, so the code style did not change.
### Imports Within `lib` Package
🧾 Time Immemorial
> When importing files within the lib package, all names should be prefixed with "lib".
The exports of the various library files are often very generic and may conflict with variable names.
Therefore, when importing these files the "lib" prefix should be used to avoid confussing.
This only applies to files within `lib` importing other files in `lib` because all other packages import the whole of `lib` as a single package.
### Name Style (TS)
🧾 Time Immemorial
> Variables and members of classes use camelCase. Classes use PascalCase. Config values use lowercase_underscore. Exceptions allowed for over the write objects.
We needed to maintain a consistent name style across all our files.
Therefore, we follow common practive for JavaScript of using camelCase and PascalCase.
Although, we have also allowed the use of lowercase_underscore for config values to distinguish them from other variables.
## Code Style (Lua)
### FMTK
🧾 14th June 2024 | [External PR](https://github.com/justarandomgeek/vscode-factoriomod-debug/pull/126)
> Use of FMTK is not required but is advised.
> To enable Clusterio support, use the CLI flag `--clusterio-modules` under `Lua.runtime.pluginArgs`.
We needed to maintain a consistent style across all our files.
There are multiple competing standards for Lua code, and given the limited amount of Lua code in this project, we chose not to adopt any automatic linting or shared luarc file.
Nevertheless, we recomend the use of [FMTK](https://github.com/justarandomgeek/vscode-factoriomod-debug) and have taken steps towords supporting it in Clusterio.
### Name Style (Lua)
🧾 Time Immemorial
> Everything uses lowercase_underscore
We needed to maintain a consistent name style across all our files.
Therefore, we follow common practive for Lua of using lowercase_underscore.
## Code Style (Other)
### Markdown
🧾 Time Immemorial
> Use one sentence per line.
> Place two lines before each heading, except for the first one or when two headings are adjacent.
We needed to maintain a consistent code style across all our markdown files.
Therefore, we selected an approach which makes diff reports more useful and which makes headings use additional space to make them stand out.
## Repository Structure
### External Plugins
🧾 Time Immemorial
> No external plugins should be included in the main repository.
> A folder will exist which external plugin repositories can be cloned into and will be given the same treatment as first party plugins.
We require a designated space for developers to create their plugins without the concern of conflicts with first-party plugins.
To address this, we have introduced a dedicated folder for external plugins.
This solution is straightforward and has since proven to be highly effective.
### First Party Plugins
🧾 Time Immemorial
> First party plugins are included in the main repository.
> Those selected to be first party are held to the same stanards which apply to the core packages.
> Each will recieve a new release in step with the core packages.
We require a designated space for first-party plugins, separate from the core packages, to clearly represent the dependency between them.
Creating a folder provided a simple solution to this.
### Core Packages
🧾 Time Immemorial
> All core packages will be contained in the packages folder of the repository.
> They are allowed to be interdependent on each other.
We require a set of core packages to represent the different applications used in a cluster.
See [Separation of Host and Controller](#separation-of-host-and-controller).
To avoid putting more files into the repository root we created a folder for the core packages.
### TypeScript Configs
🧾 26th August 2023 | [Github PR #504](https://github.com/clusterio/clusterio/pull/504)
> Each package / plugin may contain any of: `tsconfig.json` `tsconfig.node.json` and `tsconfig.web.json`.
> The main respository root will contain all three files in addition to `tsconfig.base.json` which node and web extend from.
> The configs inside packages / plugins should extend from those in the respository root.
We found that maintaining independent tsconfig files became burdensome and required a way to consolidate them.
Initially, we introduced a base configuration file; however, this proved insufficient, so we extended it to include separate configurations for Node.js and the Web.
This approach led to each package having up to three tsconfig files.
Nevertheless, we believe this to be justified, given the need to differentiate between Node.js and Web builds while maintaining a single, shared configuration.
### Workspace
🧾 Time Immemorial
> The workspace root should contain only: configuration files, scripts used to build / clean the workspace, the readme, and the changelog.
We need to maintain a clean workspace root by only including files which effect all pacakges and plugins such as configuration files.
We have allowed for build and clean scripts to exist here as long as they are not published as part of any package or plugin.
## Project Architecture
### Deprecation of Phin in favour of `fetch`
🧾 12th December 2024 | [Discord Discussion Dec 24](https://discord.com/channels/450361298220351489/450361298220351491/1316781609206419567) | [Discord Discussion May 25](https://discord.com/channels/450361298220351489/450361298220351491/1240246997496102942)
> The builtin `fetch` method should be used as a replacement of the deprecated Phin module.
We required a replacement for the deprecated Phin HTTP client, which was used for making server-side requests and as part of testing.
With `fetch` being marked as stable in Node.js v21.0.0 and having first been made available in v16.15.0, it serves as a suitable replacement without the need to add new dependencies.
It had previously not been used because it was not marked as stable in any LTS version of NodeJS.
### Minimal Dependencies
🧾 Time Immemorial
> Any new dependencies need to be well justified.
We aim to minimise our exposure to upstream changes by using a minimal number of dependencies.
This also serves to reduce the bloat of a cluster installation and improve installation times.
Consequently, the addition of any new dependencies must be thoroughly justified, with evidence of research into alternative options and comparisons between them.
### React
🧾 Time Immemorial
> Our web stack uses ReactJS
We require a reactive website for our control panel, and ReactJS was selected due to its widespread use and familiarity.
Switching away from React for any reason would represent a significant undertaking.
### Separation of Host and Controller
🧾 Time Immemorial
> The primary application locations are Control (web and cli), Controller, and Host.
> Therefore the core packages are: `ctl` `controller` `host`. There are two libaray packages `lib` and `web_ui`.
> Additionally `create` exists to provide easy creation of a cluster.
A single application to perform all the duties of a cluster would complicate the reasoning and maintenance of the project, so we decided to split it into several packages.
The primary components of a cluster include servers connected to an instance, which runs on a host that connects to the controller, whose configuration is managed via a control connection from a CLI or web portal.
Based on this structure, we found it logical to divide the packages according to their installation location, resulting in `ctl`, `controller`, and `host`, with shared code separated into `lib` and `web_ui` depending on where it is utilised.
The additional `create` package is used by `npm create` to provide a user-friendly installation process.
### TypeScript
🧾 6th August 2023 | [Github PR #491](https://github.com/clusterio/clusterio/issues/491)
> We use TypeScript as our language of choice.
We previously used JavaScript and switched to TypeScript to provide a robust type system.
Transitioning to any other langauge would be too significant an undertaking.
Initially, we supported multiple languages for plugins, including Rust and Go; however, this approach was quickly abandoned due to its complexity.
### Vite
🧾 20th October 2026
> Our web stack uses Vite. Plugin web bundles are ES modules that import shared dependencies through an import map.
We used Webpack with Module Federation for loading plugins from the start, as it was the standard at the time.
Vite builds the web interface several times faster, its development server updates modules without reloading the page, and plugin bundles no longer depend on one bundler's runtime.
See [#1127](https://github.com/clusterio/clusterio/issues/1127) for the discussion.
## Supportability
### Breaking Changes
🧾 7th October 2026
> From 2.0.0 onwards, major versions contain breaking changes for end users, while minor versions may contain breaking changes for the plugin API.
> All breaking changes are documented in the changelog along with migration actions.
We aim to be highly transparent with breaking changes and maintain compatibility with as many existing installations as possible.
During the alpha stage every alpha was allowed to contain breaking changes, as long as they were documented and provided migration actions.
From 2.0 onwards changes which require action from people running a cluster, such as changes to configs, stored data or the command line, are held back for the next major version.
Changes to the plugin API may still be made in minor versions.
Plugin authors should check the changelog when a new minor version is released, non-trivial changes come with a migration guide such as [Migrating Plugins to 2.1](/docs/devs/plugin-migration-to-2.1.md).
### Distribution
🧾 Time Immemorial
> We use `git` and `pnpm` for development distribution.
> We use `npm` and the factorio mod portal for production distribution.
We need to distribute both production builds and development source code for the project.
By necessity, we use `npm` and the Factorio Mod Portal to distribute our production builds.
For development, we rely on `git` and GitHub, which are industry standards, while using `pnpm` as a more efficient variant of `npm` for development installations.
Although bun was considered upon its release, we were not convinced it was the right choice.
### Factorio
🧾 26th October 2024 | [Discord Discussion](https://discord.com/channels/450361298220351489/450361298220351491/1299637042350653461)
> We actively provide full support for the last two major versions of Factorio.
> This approach does not follow semantic versioning; instead, it follows the "middle" versions: 2.1 > 2.0 > 1.1 > 1.0 > 0.18 > 0.17.
> We offer passive support for all versions from 0.17 onwards, where implementation is trivial and does not impose restrictions on our actively supported versions.
Following the 2.0 update of Factorio, which introduced a large number of breaking changes, we needed to reconsider which versions would be actively supported, as it was no longer trivial to support all versions.
Anticipating that future breaking changes would also be significant, and assuming the majority of players would update to the latest stable version upon release, we decided to support the latest stable version and the previous major version that introduced breaking changes.
Support for other versions will not be removed if they have already been implemented and do not impose restrictions on our ability to support newer versions.
### General Support
🧾 Time Immemorial
> We provide full support via GitHub and our Discord server for the latest published version of Clusterio, as well as assistance with migrating from the previous published version.
> However, we do not offer any guarantees with our support; we assist when time permits and may not be able to resolve every issue.
> Additionally, while we offer support in setting up development workspaces and guidance for plugin development, we do not provide support for those running development builds as production instances.
We aim to support as many users of Clusterio as possible in setting up and using the platform for a variety of purposes, including setting up production instances, development workspaces, and providing guidance on plugin development.
However, our time to work on Clusterio is limited, and even more so our ability to offer support.
As such, we cannot assist with every issue and must set reasonable restrictions to ensure expectations are aligned.
Consequently, we do not offer any guarantees for our support, but we strive to provide as much assistance as possible for those setting up or running the latest published version.
Additionally, we support users in setting up development workspaces, as the more people willing to contribute, the better.
### NodeJS
🧾 Time Immemorial
> We support all LTS versions of NodeJS in their active and maintenance phases.
Our options were: supporting all versions, supporting all active versions, supporting all LTS versions, or supporting only the latest version.
As we aim to provide a stable, production-ready tool while still limiting the number of versions we need to manage, we chose to support all LTS versions because NodeJS can be installed by the majority of linux distro package managers which will always offer at least one LTS version.
### Operating Systems
🧾 Time Immemorial
> We support all major versions of Windows, Linux, and MacOS.
This is largely inpart to our use of NodeJS and is not a guarantee from us.
Our choice of Node.js has enabled us to support all major operating systems with minimal effort on our part.
Nevertheless, there have been instances where code paths have diverged between systems, and we have attempted to maintain support for as many systems as possible.
However, this is not a priority, and such support has often been reactive or provided by external contributors.
## Testing
### Continuous Integration
🧾 Time Immemorial
> We run unit and integration tests through GitHub CI actions as well as test coverage through CodeCov.
Continuous integration actions in GitHub are a valuable tool that we use to assist with the review of pull requests from both maintainers and contributors.
While we aim to increase the number of useful checks performed by CI in the future, this is not a priority over the development of new features.
### Continuous Deployment
🧾 Time Immemorial
> We publish packages and mods when a maintainer publishes a stable GitHub release.
Changes merged into the main branch are bundled into releases before publication.
Publishing a stable GitHub release triggers the [Publish workflow](../.github/workflows/publish.yml), which publishes npm packages and Factorio mods.
Publishing a prerelease or pushing a tag alone does not trigger publication.
Running clusters are updated by their operators.
See the [release workflow](release-workflow.md) for the publication procedure.
### Mocha
🧾 Time Immemorial
> We use Mocha for testing and CodeCov for coverage.
We chose Mocha because it was a good fit when we selected it.
However, we are always looking for a better framework which is worth the effort of switching.
### Version Matrix
🧾 Time Immemorial
> We perform all tests through GitHub actions using Ubuntu latest, using all supported versions of node and all supported versions of factorio.
The [current CI workflow](../.github/workflows/node.js.yml) tests selected Node.js major versions and pinned Factorio releases.
However, we do not conduct tests on all supported operating systems, as the added demand on our testing resources would be significant, and the value it would provide for a cross-platform engine like NodeJS would be limited.
This document describes the various different types of content that can
be created for Factorio and/or Clusterio and how this is made compatible
with Clusterio.
This document describes the various different types of content that can be created for Factorio and/or Clusterio and how this is made compatible with Clusterio.
- [Communicating with Clusterio](#communicating-with-clusterio)
- [Clusterio Modules](#clusterio-modules)
- [Initialisation and recovery](#initialisation-and-recovery)
- [Clusterio Lua API](#clusterio-lua-api)
- [Compatibility library](#compatibility-library)
- [Clusterio Plugins](#clusterio-plugins)
Factorio Mods
-------------
## Factorio Mods
Factorio mods that do not interact with Clusterio should not have any
considerations that need to be taken into account for them to work with
Clusterio.
Factorio mods that do not interact with Clusterio should not have any considerations that need to be taken into account for them to work with Clusterio.
Factorio Scenarios
------------------
## Factorio Scenarios
Clusterio patches saves and because of this scenarios have a few
limitations and must explicitly mark themselves as compatible with
Clusterio to work. The main limitation is that the
lib must be used, and control.lua cannot contain any code other than
calls to event_handler loading the relevant libs of the scenario.
This is because the control.lua file will be overwritten by the save
patcher. Scenarios also cannot use the event registering functions
exposed by `script` such as `script.on_event` or `script.on_nth_tick` as
using these will overwrite the handlers registered by the
event_handler lib.
Clusterio patches saves and because of this scenarios have a few limitations and must explicitly mark themselves as compatible with Clusterio to work.
The main limitation is that the [event_handler](https://github.com/wube/factorio-data/blob/master/core/lualib/event_handler.lua) lib must be used, and control.lua cannot contain any code other than calls to event_handler loading the relevant libs of the scenario.
This is because the control.lua file will be overwritten by the save patcher.
Scenarios also cannot use the event registering functions exposed by `script` such as `script.on_event` or `script.on_nth_tick` as using these will overwrite the handlers registered by the event_handler lib.
A brief description of the usage of the event_handler library is
provided in the [event_handler interface](#event_handler-interface)
section.
A brief description of the usage of the event_handler library is provided in the [event_handler interface](#event_handler-interface) section.
Once a scenario is made using the event_handler it should have a
control.lua file that looks something like this:
Once a scenario is made using the event_handler it should have a control.lua file that looks something like this:
To make Clusterio recognize and correctly handle this scenario it
needs to include a clusterio.json file with following content:
To make Clusterio recognize and correctly handle this scenario it needs to include a clusterio.json file with following content:
```json
{
@@ -70,65 +53,48 @@ needs to include a clusterio.json file with following content:
}
```
This tells Clusterio that it should load both module_foo and module_bar
by passing the result of requiring them to the `add_lib` function when
writing a new control.lua file to the save.
This tells Clusterio that it should load both module_foo and module_bar by passing the result of requiring them to the `add_lib` function when writing a new control.lua file to the save.
### event_handler interface
<sub>**Note:** At the time of writing the event_handler lib has been
a part of Factorio since at least 0.17.4, and was moved from base to
core in 0.17.63 but is still not documented anywhere. A brief
description is provided here for this reason.</sub>
<sub>**Note:** At the time of writing the event_handler lib has been a part of Factorio since at least 0.17.4, and was moved from base to core in 0.17.63 but is still not documented anywhere.
A brief description is provided here for this reason.</sub>
The event_handler lib provides a simple interface for registering
multiple callbacks to the same events without having to be concerned
with callback chaining or overwriting callbacks defined elsewhere. It
works by taking over the task of registering the actual callbacks with
Factorio, providing its own interface for the rest of the code to use.
The event_handler lib provides a simple interface for registering multiple callbacks to the same events without having to be concerned with callback chaining or overwriting callbacks defined elsewhere.
It works by taking over the task of registering the actual callbacks with Factorio, providing its own interface for the rest of the code to use.
The sole function of interest exported by event_handler is `add_lib`
which accepts a table with event handler callbacks definitions that it
will register. The following entries are recognized by `add_lib`, all
of which are optional:
The sole function of interest exported by event_handler is `add_lib` which accepts a table with event handler callbacks definitions that it will register.
The following entries are recognized by `add_lib`, all of which are optional:
-`on_init`:
Callback called when the callback of `script.on_init` is invoked.
-`on_load`:
Callback called when the callback of `script.on_load` is invoked.
-`on_configuration_changed`:
Callback called when the callback of
`script.on_configuration_changed` is invoked.
Callback called when the callback of`script.on_configuration_changed` is invoked.
-`events`:
Table mapping event ids as defined in `defines.events` to callbacks.
For example the table `{ [defines.events.on_player_died] = foo }`
will call bar with the usual `event` table as argument every time a
player dies.
For example the table `{ [defines.events.on_player_died] = foo }` will call foo with the usual `event` table as argument every time a player dies.
-`on_nth_tick`:
Table mapping the nth tick number to a callback. For example the
table `{ [30] = bar }` will call bar every 30th tick.
Table mapping the nth tick number to a callback.
For example the table `{ [30] = bar }` will call bar every 30th tick.
-`add_remote_interface`:
Callback called before on_init/on_load callbacks<sup>[1]</sup>. It
has no special meaning and receives no arguments but should be used
for registering remote interfaces.
Callback called before on_init/on_load callbacks<sup>[1]</sup>.
It has no special meaning and receives no arguments but should be used for registering remote interfaces.
-`add_commands`:
Callback called before on_init/on_load callbacks<sup>[1]</sup>. It
has no special meaning and receives no arguments but should be used
for registering commands.
Callback called before on_init/on_load callbacks<sup>[1]</sup>.
It has no special meaning and receives no arguments but should be used for registering commands.
<sub>1: Before 0.17.69 these callbacks were called after
on_init/on_load.</sub>
<sub>1: Before 0.17.69 these callbacks were called after on_init/on_load.</sub>
The usual way to use `add_lib` is to define the table of events to
register in a separate file and return it, then load it in control.lua
via `require` before passing it to `add_lib`. For example a module
could be defined as
The usual way to use `add_lib` is to define the table of events to register in a separate file and return it, then load it in control.lua via `require` before passing it to `add_lib`.
@@ -148,41 +114,27 @@ local event_handler = require("event_handler")
event_handler.add_lib(require("example"))
```
Because the event_handler lib registers events itself you may not use
`script.on_load` or `script.on_init` at all in your code and any usage
of `script.on_configuration_changed`, `script.on_nth_tick` and/or
`script.on_event` will cause the corresponding events registered with
event_handler to break and should therefore not be used.
Because the event_handler lib registers events itself you may not use`script.on_load` or `script.on_init` at all in your code and any usage of `script.on_configuration_changed`, `script.on_nth_tick` and/or `script.on_event` will cause the corresponding events registered with event_handler to break and should therefore not be used.
There's also the `add_libraries` function exported by event_handler,
which accepts a table and calls `add_lib` for each value in the table.
There's also the `add_libraries` function exported by event_handler, which accepts a table and calls `add_lib` for each value in the table.
Communicating with Clusterio
----------------------------
## Communicating with Clusterio
There's a poorly to virtuarly undocumented HTTP interface that can be
used to communicate with Clusterio to do things like get the status of
instances, grab items from the store, run commands on instances, etc,
etc. For now you can read the source code, and maybe ask on the Discord
for some pointers if you get stuck.
Clusterio uses a homebrew protocol based on sending JSON payloads over a WebSocket connection.
See [protocol.md](devs/protocol.md) for the implementation details of it.
It's also possible to write a plugin for Clusterio that exposes a custom interface over HTTP, WebSocket or any other technology supported by Node.js.
Clusterio Modules
-----------------
## Clusterio Modules
Modules are primarily used by plugins to inject code into Factorio games
with the save patcher, though it's also possible to make stand alone
modules that are loaded into the game if you don't need the capabilites
of the plugin system. The save patcher puts modules into the `modules`
folder of the Factorio save and adds code to `control.lua` to load the
module according to the `load` and `require` options to the module.json
file.
Modules are primarily used by plugins to inject code into Factorio games with the save patcher, though it's also possible to make stand alone modules that are loaded into the game if you don't need the capabilites of the plugin system.
The save patcher puts modules into the `modules` folder of the Factorio save and adds code to `control.lua` to load the module according to the `load` and `require` options to the module.json file.
Like with scenarios for Clusterio the [event_handler interface](#event_handler-interface) has to be used for any event subscriptions in modules.
Stand alone modules are placed into the modules folder of Clusterio,
plugin modules are located in the module folder of the plugin. In
either case a `module.json` file is required and has the following
structure:
Stand alone modules are placed into the modules folder of Clusterio, plugin modules are located in the module folder of the plugin.
In either case a `module.json` file is required and has the following structure:
```json
{
@@ -200,68 +152,82 @@ structure:
The following entries are supported in the module.json file:
-`name`:
Name of the module, must match the folder the module is located in
for stand alone modules or the name of the plugin the module is
located in.
Name of the module, must match the folder the module is located in for stand alone modules or the name of the plugin the module is located in.
-`version`:
Version of the module. Must be compatible with [Semantic Versioning
2.0.0](https://semver.org/). In plugin modules this defaults to the
version of the plugin otherwise it's required.
Version of the module.
Must be compatible with [Semantic Versioning 2.0.0](https://semver.org/).
In plugin modules this defaults to the version of the plugin otherwise it's required.
-`dependencies`:
Optional mapping of modules this module depends on and their
version. In the example `>=0.4.2` means that the foo module must be
at least version 0.4.2. See the [Ranges
syntax](https://www.npmjs.com/package/semver#ranges) of the
node-semver package for a full description of what operators are
supported. The events for dependencies is invoked before the events
of the dependent, and starting an instance will fail if the
dependencies cannot be satisfied. If not specified it will default
to depending on `clusterio`. Make sure to add `clusterio` to your
dependencies if you add other dependencies and depend on the Lua API.
Optional mapping of modules this module depends on and their version.
In the example `>=0.4.2` means that the foo module must be at least version 0.4.2.
See the [Ranges syntax](https://www.npmjs.com/package/semver#ranges) of the node-semver package for a full description of what operators are supported.
The events for dependencies is invoked before the events of the dependent, and starting an instance will fail if the dependencies cannot be satisfied.
If not specified it will default to depending on `clusterio`.
Make sure to add `clusterio` to your dependencies if you add other dependencies and depend on the Lua API.
-`require`:
Lua files to require in `control.lua`. This should only really be
needed if you want to make a global function available for use in
Lua commands. The paths are relative to the module's own folder and
should be specified using forward slashes as directory sepparators
if it's located in a sub directory in the module.
Lua files to require in `control.lua`.
This should only really be needed if you want to make a global function available for use in Lua commands.
The paths are relative to the module's own folder and should be specified using forward slashes as directory sepparators if it's located in a sub directory in the module.
-`load`:
Lua files to load with the `event_handler` lib, the result of
requiring the file will be passed to the `add_lib` function of the
`event_handler` lib in `control.lua`. See the section on the
[event_handler interface](#event_handler-interface) for a detailed
description on how the `event_handler` lib works. The paths are
relative to the module's own folder and should be specified using
forward slashes as directory sepparators if it's located in a sub
directory in the module.
Lua files to load with the `event_handler` lib, the result of requiring the file will be passed to the `add_lib` function of the `event_handler` lib in `control.lua`.
See the section on the [event_handler interface](#event_handler-interface) for a detailed description on how the `event_handler` lib works.
The paths are relative to the module's own folder and should be specified using forward slashes as directory sepparators if it's located in a sub directory in the module.
It's recommended to use the new style of defining modules in Lua, as
well as avoid the use of global variables as much as possible. This
means always declearing your top level variables and functions local and
exporting only the things you need in other files. You can require
other files in your module by prefixing your require paths with
`modules/your_module_name/`. It's also possible to require files from
other modules this way too.
It's recommended to use the new style of defining modules in Lua, as well as avoid the use of global variables as much as possible.
This means always declearing your top level variables and functions local and exporting only the things you need in other files.
You can require other files in your module by prefixing your require paths with `modules/your_module_name/`.
It's also possible to require files from other modules this way too.
The global variables and data stored in the global table is shared by
all Clusterio modules as well as the scenario, so it's important that
you use unique names. It's recommended to prefix the global variables
and data in the global table that your module has with the name of your
module.
Because modules can be patched into an existing game you cannot rely on
the `on_init` callback to be called in Clusterio Modules. Nor can you
rely on the `on_configuration_changed` callback, as this is not called
when level code changes. The Clusterio Lua API provides the custom
`on_server_startup` event that can be used as a substitute, see the next
section.
Persistent state is shared by all save-patched modules and the scenario.
Use `compat.script_data` to access the version-appropriate persistent table (`global` or `storage`), and keep your state under a unique module name.
See the [compatibility library](#compatibility-library) below.
Clusterio Lua API
-----------------
### Initialisation and recovery
Clusterio provides a save patched module as well as a regular Factorio
mod for interfacing with Clusterio. This includes tools sending data to
plugins, script events to listen for, and an item serialization module.
Because modules can be patched into an existing game you cannot rely on the `on_init` callback to be called in Clusterio Modules.
Nor can you rely on the `on_configuration_changed` callback, as this is not called when level code changes.
The Clusterio Lua API provides `on_server_startup` for setup and migrations, but some Factorio events can run before it.
In particular, player-leaving, console-command, and configuration-change handlers must not assume startup has initialized their state.
Use an idempotent state accessor in handlers that can run early.
It should migrate any legacy namespace before creating fresh state, fill missing fields, and return the same state on repeated calls.
A guard that returns early because the new namespace is absent can silently discard recovery work on an older save.
See [player_sync's state accessor](../plugins/player_sync/module/get_script_data.lua) for an implementation that handles a renamed namespace and [the module template](../packages/create/templates/common/module/control.lua) for lifecycle setup.
Use `on_load` only to restore local references and other operations permitted by Factorio's load lifecycle; do not create or migrate persistent state there.
Perform state changes in startup or other events that permit them.
Recovery mode skips normal Node plugin loading, so save data may still use an older plugin namespace when Lua handlers run.
Test old and current saves, early events, repeated access, and save/load behavior on the relevant supported Factorio versions.
## Clusterio Lua API
Clusterio provides a save patched module as well as a regular Factorio mod for interfacing with Clusterio.
This includes tools sending data to plugins, script events to listen for, and an item serialization module.
### Compatibility library
Use the compatibility helpers for APIs that differ between supported Factorio versions:
```lua
-- In a save-patched module:
localcompat=require("modules/clusterio/compat")
-- In a mod, use require("__clusterio_lib__/compat") instead.
localfunctionencode_example()
localscript_data=compat.script_data
localencoded=compat.table_to_json({example=true})
localdecoded=compat.json_to_table(encoded)
end
```
`script_data` selects the persistent state table, while the JSON helpers select the available runtime API.
Access persistent state from the appropriate lifecycle callbacks, after initialisation or loading has made it available.
The JSON helpers also work before runtime; other compatibility helpers may require `game` to be available.
Legacy examples using `global` or `game.table_to_json` directly are version-specific.
See [compat.lua](../packages/host/modules/clusterio/compat.lua) and the [Factorio support policy](decisions.md#factorio) before introducing version checks.
Provides the main interface to Clusterio from within the game. When
using the mod version you must call `init` before it'll function
properly.
Provides the main interface to Clusterio from within the game.
When using the mod version you must call `init` before it'll function properly.
#### init
<sub>Mod version only</sub>
Initialize the API. This should be called from both the `on_init` and
`on_load` events in your mod, and before this is called certain features
will not be available, most notably the events table.
Initialize the API.
This should be called from both the `on_init` and `on_load` events in your mod, and before this is called certain features will not be available, most notably the events table.
#### clusterio_api.events
Table of events raised by clusterio. If the game is started outside of
Clusterio then none of these events will be raised.
Table of events raised by clusterio.
If the game is started outside of Clusterio then none of these events will be raised.
In the mod version this table does not exist before the `init` function
is called.
In the mod version this table does not exist before the `init` function is called.
##### on_instance_updated
Raised after the name and id of an instance has been updated. This may
occur even if the id and name didn't change.
Raised after the name and id of an instance has been updated.
This may occur even if the id and name didn't change.
Event data:
-`instance_id`: the id of the instance.
@@ -303,37 +266,44 @@ Event data:
##### on_server_startup
Raised when Clusterio is starting up the server. It can be used to
initialize clusterio related features in a mod, and as a stand-in for
the `on_init` and `on_configuration_changed` event for modules. It is
invoked on the first tick the server runs after the save has been
patched, before most other events.
Raised when Clusterio is starting up the server.
It can be used to initialize clusterio related features in a mod, and as a stand-in for the `on_init` and `on_configuration_changed` event for modules.
It is invoked on the first tick the server runs after the save has been patched, before most other events but not all of them.
See [initialisation and recovery](#initialisation-and-recovery) for handlers that can run earlier.
Use this event to initialize and/or migrate the data structures
you need. Keep in mind that both mods and Clusterio modules can switch
from any version to any version so it should be able to handle both
forwards and backwards migrations gracefully.
Use this event to initialize and/or migrate the data structures you need.
Keep in mind that both mods and Clusterio modules can switch from any version to any version so it should be able to handle both forwards and backwards migrations gracefully.
#### clusterio_api.send_json(channel, data)
Send JSON data to Clusterio over the given channel. The `data` argument
should be table that can be serialized with `game.table_to_json`.
Clusterio plugins can listen to channels and will receive an event with
the data sent here. See the [Communicating with Factorio
section](writing-plugins.md#communicating-with-factorio) in the Writing
Plugins document for more information.
Send JSON data to Clusterio over the given channel.
The `data` argument should be a table that can be serialized with `compat.table_to_json`.
Clusterio plugins can listen to channels and will receive an event with the data sent here.
See the [Communicating with Factorio section](writing-plugins.md#communicating-with-factorio) in the Writing Plugins document for more information.
If the game is started outside of Clusterio the data will be sent, but
since there's no code following stdout to pick it up it will be lost.
If the game is started outside of Clusterio the data will be sent, but since there's no code following stdout to pick it up it will be lost.
**Note**: Payloads greater than 4 MB will cause stuttering in the game.
**Note**: This is not a binary safe way of sending data. Strings
embedded into the tables sent must be valid UTF-8 text.
**Note**: This is not a binary safe way of sending data.
Strings embedded into the tables sent must be valid UTF-8 text.
Parameters:
-`channel`: string identifying which channel to send it on.
-`data`: table that can be converted to JSON with game.table_to_json
-`data`: table that can be converted to JSON with `compat.table_to_json`
#### clusterio_api.send_udp(channel, data)
Send data to Clusterio over the given channel using UDP instead of stdout.
Clusterio plugins receive it as `udp-channel_name` events, see the [Communicating over UDP section](writing-plugins.md#communicating-over-udp) in the Writing Plugins document.
Requires `factorio.enable_lua_udp` to be enabled on the instance and Factorio 2.1.10 or later, otherwise the data is silently dropped.
The port to send to is passed to the module with a script command on startup, so `factorio.enable_script_commands` must be enabled as well.
**Note**: Delivery is not guaranteed and payloads should be kept well below 32 kB.
Parameters:
-`channel`: string identifying which channel to send it on.
-`data`: string with the payload to send, any bytes are allowed.
To serialise and deserialise data the Clusterio codebase uses a JSON serialisable data class pattern.
Instances of these classes can be serialised to JSON by passing them directly to `JSON.serialise` and deserialised by passing the serialised JSON representation to its static `fromJSON` method.
The data class also has a static `jsonSchema` property describing the serialised JSON representation for validation purposes.
See the following example code for a hypothetical `Data` class with a single property `member` defaulting to a value of `"foo"`.
```js
// Data.js
"use strictc";
/**
* A Data class example
* @alias module:Data
*/
exportclassData{
/** @type {string} */
member="foo";
constructor(member){
if(member!==undefined){this.member=member;}
}
staticjsonSchema={
type:"object",
properties:{
"member":{type:"string"},
},
};
staticfromJSON(json){
returnnewthis(json.member);
}
// toJSON is called by JSON.stringify when serialising this class
Serialising an instance of `Data` for JSON is as simple as doing `serialised = JSON.stringify(data)` as the `toJSON` method is automatically called by `JSON.stringify` to create a suitable object.
Deserialising is done by calling the static `fromJSON` method with the serialised JSON data, like `data = Data.fromJSON(JSON.parse(serialised))`.
By structuring the classes like this they can be easily composed together.
For example a class holding a data property with an instance of the Data class from the previous example can be created as follows.
// toJSON is optional and not needed for this particular class
}
```
It's easy to create a new instance of Composed by calling `new Composed(new Data("spam"))` and serialising to and from JSON is the same as for Data.
Meaning `serialised = JSON.stringify(composed)` and `composed = Composed.fromJSON(JSON.parse(serialised))` does as expected.
## Interface description
Data classes are ES6 style class declarations with the following methods and fields:
### toJSON(): any (optional)
Method returning an object or primitive suitable for passing to JSON.stringify in order to create a JSON representation of the class instance.
The returned object may contain objects that have their own toJSON methods which JSON.stringify will then call recursively.
This method is optional if calling `JSON.stringify` on an instance without it would produce the desired serialised form, however this is rarely the case as often you'll have to provide it to convert the values fram Map/Set properites, omit default values or do similar actions.
Note that the primary use case for serialising data classes is to send them over the network, this method should therefore try to reduce the size of the resulting JSON by for example omitting properties who's values are the default value.
### static fromJSON(json: any): this
Method constructing an instance of this class from one that was previously serialised by passing an instance of it to `JSON.stringify`.
While not a requirement it is preferrable if the input is not mutated and that no references to any any structures in the input ends up in the returned object.
In other words calling fromJSON twice with the same input should produce two independent objects.
Note that directly passing the output of `toJSON` to `fromJSON` is not valid, as the output of `toJSON` may require further expansion calling `toJSON` on objects returned by it.
It has to go through `JSON.stringify` or an equivalent recursive expansion of `toJSON`.
### static jsonSchema
A [JSON schema](http://json-schema.org/) describing the format of the JSON resulting from passing an instance of this class to `JSON.stringify`.
This should be lenient towards adding additional properties that would be ignored by `fromJSON` as it makes it easier to extend these classes in the future without breaking compatibility.
This guide covers the changes needed to bring a plugin written for 2.0.0-alpha.27 up to 2.0.0.
If your plugin targets an older alpha, go through the Breaking Changes sections in the [changelog](/CHANGELOG.md) for the versions in between first.
Plugins still on alpha 13 should start with the [alpha 14 guide](plugin-migration-to-alpha-14.md).
## Versioning from 2.0
From 2.0.0 onwards a major version (3.0.0) contains breaking changes for users running a cluster, while a minor version (2.1.0) may contain breaking changes for the plugin API.
This means a plugin written for 2.0 is not guaranteed to work with 2.1 without changes.
Check the Breaking Changes section of the changelog when a new minor version is released and publish an update to your plugin if needed.
See [Breaking Changes](/docs/decisions.md#breaking-changes) in the decision log.
## Convert to ES modules
Clusterio's packages are now ES modules and plugins should be too.
Add `"type": "module"` to the plugin's `package.json` and replace `require()` and `module.exports` with `import` and `export`.
Keep `"clusterio-plugin"` in the package's `"keywords"` for automatic discovery, and keep `"main"` pointing to the plugin info module (or use the default `index.js`).
An `"exports"` map is not required for ES modules.
If you add one, expose `"./package.json": "./package.json"`; otherwise Clusterio cannot discover or load the plugin by its package name.
Clusterio currently reads the implementation path from `"main"` rather than conditional exports.
See [Plugin Structure](../writing-plugins.md#plugin-structure) for a compatible package example.
Relative imports must include the full filename with the `.js` extension.
This also applies in TypeScript, where you import `./messages.js` even though the source file is `messages.ts`.
Importing a directory no longer resolves to its `index.js`.
**Alpha 27**
```js
"use strict";
constlib=require("@clusterio/lib");
constmessages=require("./messages");
constplugin={
name:"foo_frobber",
controllerEntrypoint:"controller",
messages:[messages.Frobnicate],
};
module.exports={
plugin,
};
```
**2.0**
```js
import*aslibfrom"@clusterio/lib";
import*asmessagesfrom"./messages.js";
exportconstplugin={
name:"foo_frobber",
controllerEntrypoint:"controller.js",
messages:[messages.Frobnicate],
};
```
The entrypoint paths in the plugin declaration must include the `.js` extension as well, for TypeScript plugins this means `"./dist/node/controller.js"` instead of `"./dist/node/controller"`.
`__dirname` and `__filename` do not exist in ES modules, use `import.meta.dirname` and `import.meta.filename` instead.
JSON files are imported with `import packageJson from "./package.json" with { type: "json" };`.
A `webpack.config.js` written with `require()` stops working once the package is an ES module.
Rename it to `webpack.config.cjs`, webpack-cli finds it under that name without further changes.
If your `prepare` script passes `--config webpack.config.js` update it to the new name.
TypeScript plugins using `"module": "node16"` in their tsconfig, as the plugin templates do, emit ES modules once `"type": "module"` is set and need no further config changes.
Node.js can still import a CommonJS plugin, but it only finds the `plugin` export if it can detect it from the source, which works for `module.exports = { plugin };` and fails for other forms such as an object literal written in place.
Converting the plugin avoids the problem.
## Replace plugin classes with entrypoint functions
Entrypoints now export a default async function instead of a class deriving from `BaseControllerPlugin`, `BaseHostPlugin`, `BaseInstancePlugin`, `BaseCtlPlugin` or `BaseWebPlugin`.
The function is called once when the plugin loads and attaches handlers to the hooks it needs.
Class exports still load, but they are deprecated and a warning is logged for each one.
See [Defining the plugin entrypoint](/docs/writing-plugins.md#defining-the-plugin-entrypoint) for the details.
The code from `init()` goes into the body of the function, and state that lived on `this` becomes local variables.
The context types are `ControllerPluginContext`, `HostPluginContext`, `InstancePluginContext`, `CtlPluginContext` and `WebPluginContext` from the respective packages.
### Keeping a class for state
If the plugin has enough state that a class is still the better fit, give the class a private constructor and an async `static fromContext` that creates it from the context and attaches its hooks.
`fromContext` can then be the default export of the entrypoint.
Instances are kept in a static `WeakMap` keyed by the controller, host or instance they belong to, which lets other code look them up without holding a reference to them.
The loader calls the default export without `this`, so refer to the class by name inside `fromContext` rather than using `this`.
Don't extend the deprecated base classes for this, they attach `on*` methods to hooks themselves and log a warning.
### Method to hook names
Each `on*` method has a hook of the same name without the `on` prefix and with the first letter lowercased, so `onInstanceStatusChanged` becomes `controller.hooks.instanceStatusChanged`.
The exceptions are:
-`onSaveData` on the controller is `controller.hooks.save`.
-`addCommands` on ctl plugins is `context.hooks.addCommands`.
- The `pages`, `loginForms` and `inputComponents` properties of web plugins are the `pages`, `loginForms` and `inputComponents` hooks on `control.hooks`, with handlers that return the value.
- The `componentExtra` property of web plugins is the `control.hooks.extensionComponents` hook.
Controller hooks are on `controller.hooks`, host hooks on `host.hooks`, instance hooks on `instance.hooks`, and web hooks on `control.hooks`.
The ctl hooks are passed directly as `context.hooks`.
The class properties are on the context object passed to the entrypoint:
-`this.info` is `context.plugin`.
-`this.logger` is `context.logger`.
-`this.controller`, `this.host`, `this.instance` and `this.control` are `context.controller`, `context.host`, `context.instance` and `context.control`.
-`this.metrics` on controller plugins is `context.metrics`.
-`this.package` on web plugins is `context.package`.
The helper methods of the base classes have the following replacements:
-`this.sendRcon(message, expectEmpty)` is `instance.sendRcon(message, expectEmpty, plugin.name)`.
The plugin name is used to label the RCON metrics.
-`this.broadcastEventToHosts(event)` is `controller.sendTo("allHosts", event)`.
-`this.sendOrderedRcon(message)` has no replacement.
If your plugin needs commands to execute in order, queue them in the plugin and send the next one after the previous one resolves.
### Changes to hook behaviour
Handlers attached by different plugins to the same hook now run concurrently, where the `on*` methods were called one plugin at a time.
Don't rely on another plugin's handler having finished before yours runs.
A handler that throws or takes more than 15 seconds is logged and does not prevent the other handlers from running.
Each hook accepts one handler per name, attach it under `plugin.name`.
## Accessing other plugins
The `plugins` map on `Controller`, `Host` and `Instance` has been removed.
It's replaced with `loadedPlugins`, a set of the plugin info objects for the plugins that loaded.
The set holds info objects rather than names, so look the info up in `pluginInfos` by name first to check if a plugin is loaded on the controller or host:
In the web interface `control.loadedPlugins` is a map from plugin name to plugin info.
`control.plugins` is deprecated and only holds plugins still using the class export.
With no plugin class instances, `controller.plugins.get("other_plugin")` has no direct replacement.
If one plugin needs to call into another, have the other plugin export its class and use the static lookup from [Keeping a class for state](#keeping-a-class-for-state):
This only works if both plugins resolve `other_plugin` to the same file on disk, otherwise the lookup goes to a different copy of the class and returns undefined.
For state shared between components of a web plugin, assign it to a module level variable in the entrypoint.
The web plugin template created by `@clusterio/create` shows this for a subscriber.
## Declare permissions on the plugin
Permissions can now be declared under `permissions` in the plugin declaration instead of with `lib.definePermission()`.
`lib.definePermission()` still works, but don't use both for the same permission name.
For TypeScript plugins `user.checkPermission()`, `account.hasPermission()` and the `permission` of web pages now only accept known permission names.
Add your permission names to the `Permissions` interface or these calls will fail to compile, even if you keep using `lib.definePermission()`:
```ts
declaremodule"@clusterio/lib"{
exportinterfacePermissions{
"foo_frobber.frobnicate":never;
}
}
```
The plugin template's `tsconfig.browser.json` now includes `index.ts` so the web build also sees the declaration, add it to the `include` list if your plugin has its own tsconfig for the browser.
See [Plugin Permissions](/docs/writing-plugins.md#plugin-permissions).
## Renamed ctl classes and config
The remaining uses of "control" for clusterioctl were renamed to "ctl":
-`controlConfigFields` in the plugin declaration is `ctlConfigFields`.
Fields under the old name are silently ignored.
- The `ControlConfigFields` interface is `CtlConfigFields`.
-`lib.ControlConfig` is `lib.CtlConfig`.
- The `Control` class exported by `@clusterio/ctl` is `Ctl`.
The term "control" is still used in message `src` and `dst` and in config field locations, where it covers both ctl and the web interface.
## Static assets in the web interface
`staticRoot` now includes the `static/` prefix.
Plugins that fetch assets with `${staticRoot}static/...` should drop the `static/` part.
## Package dependencies
npm installs peer dependencies by default, so listing `@clusterio/web_ui` under `peerDependencies` installs it everywhere the plugin is installed even though it's only needed to build the web bundle.
Move `@clusterio/web_ui` to `devDependencies` and mark `@clusterio/controller`, `@clusterio/host` and `@clusterio/ctl` as optional in `peerDependenciesMeta`:
This guide covers the changes needed to bring a plugin written for 2.0 up to 2.1.
Plugins still on an alpha should start with the [2.0 guide](plugin-migration-to-2.0.md).
## Request errors
`RequestError` is gone.
Errors thrown by request handlers are now received by the requester as the same class, see [Request errors](/docs/writing-plugins.md#request-errors).
Replace `RequestError` with the class that fits:
- Invalid input: `ValidationError`.
- Operation disabled by config: `ForbiddenError`.
- Missing instance, user, save etc.: `NotFoundError`.
- Instance running, name taken etc.: `ConflictError`.
- Host not connected, instance not running: `UnavailableError`.
- Timeouts: `TimeoutError`.
`catch` blocks that check `err instanceof RequestError` should check `ClientError` instead, or the specific class.
Checks of `err.code` for remote errors can use `instanceof`, e.g. `err instanceof lib.SessionLost`.
The config errors `lib.InvalidValue`, `lib.InvalidField` and `lib.InvalidAccess` were renamed to `lib.InvalidValueError`, `lib.InvalidFieldError` and `lib.InvalidAccessError`.
Errors of classes that are not registered, like a `TypeError` from a bug, are received as `InternalServerError`.
Plugins that define their own error classes need to list them in `errors` of the plugin declaration.
## Build the web module with Vite
The web interface is now built with Vite instead of webpack, and plugin web modules are loaded as ES modules instead of through Module Federation.
A plugin built with webpack fails to load with the error "Incompatible dist/web/manifest.json, the plugin must be rebuilt with Vite".
Delete `webpack.config.cjs` and add a `vite.config.js` next to `package.json`:
In `package.json` replace `webpack-cli --mode production` in the `prepare` script with `vite build`, and replace the `webpack`, `webpack-cli` and `webpack-merge` dev dependencies with `vite`.
Add `vite.config.js` to `.npmignore` if you had `webpack.config.js` there.
`clusterioPlugin` exposes `index.ts` (or `index.js`) as `"./"`, `package.json` as `"./package.json"` and `web/index.tsx` (or `.ts`, `.jsx`, `.js`) as `"./web"`.
These are the same keys the plugin template used with webpack, so `webEntrypoint: "./web"` stays as it is.
If your webpack config exposed other paths, move the code so it is reachable from `web/index.tsx`.
The web interface provides `@clusterio/lib`, `@clusterio/web_ui`, `@sinclair/typebox`, `antd`, `react`, `react-dom`, `react-router` and `react-router-dom` to plugins.
These are left as imports in the plugin bundle and resolved by the browser, there is no need to list them as shared.
Everything else is bundled into the plugin, including `@ant-design/icons`.
Import `react-dom/client` and `react/jsx-runtime` by those names, other subpaths of the shared packages are bundled as a separate copy.
Imports of `assert`, `buffer`, `events`, `os`, `path`, `process`, `stream`, `util` and `zlib` resolve to the same browser packages as before, provided by `@clusterio/web_ui`.
The `container` property of `WebPluginContext` is now the plugin's ES module namespace, `await container.get("./web")` returns the module instead of a factory function.
`lib.PluginWebpackEnvInfo` is renamed to `lib.PluginWebEnvInfo`.
## Renamed lib functions
`lib.safeOutputFile` is renamed to `lib.atomicOverwriteFile`.
It takes the same arguments.
## Host pairing
Host tokens are no longer generated by users.
A host joins the cluster by having the code it shows when it first starts approved, see [Host pairing](protocol.md#host-pairing) for the HTTP endpoints it uses.
-`HostGenerateTokenRequest` and `HostConfigCreateRequest` are removed.
Pending requests are listed with `HostPairingListRequest`, approved with `HostPairingApproveRequest` and rejected with `HostPairingRejectRequest`, and changes to them are sent to subscribers of `HostPairingUpdatesEvent`.
-`HostRevokeTokensRequest` is replaced by `HostRevokeRequest`, which deletes the host instead of only invalidating its tokens.
Instances assigned to the host become unassigned.
- The permissions `core.host.generate_token`, `core.host.revoke_token` and `core.host.create_config` are removed.
Check `core.host.approve` and `core.host.revoke` instead.
-`clusteriocontroller bootstrap generate-host-token` is removed, scripts that set up hosts can start the host with `--user-token` to approve it with a user token.
Host records are no longer created when a host connects, only by approving it, and the controller rejects hosts it has no record of.
Revoking a host deletes its record and note.
Plugins that keep data per host should remove it when the host is deleted.
On the controller the `update` event of `controller.hosts.records` gives the record with `isDeleted` set to true, and subscribers of `HostUpdatesEvent` receive the host with `isDeleted` set to true.
A host that is revoked and approved again gets a new id.
## Host manager
`controller.hosts` is now a `HostManager`, like `controller.users` is a `UserManager`.
`get`, `getMutable`, `has` and `values` work as before.
Everything else on the datastore, like `set`, `delete`, `size` and the `update` event, is on `controller.hosts.records`.
The host manager also signs and checks host tokens and creates and deletes hosts:
-`controller.generateHostToken(hostId)` is now `controller.hosts.signHostToken(hostId)`.
-`controller.verifyHostToken(token)` is now `controller.hosts.verifyHostToken(token)`.
It returns the `HostRecord` instead of the host id.
-`controller.hostPairing.createHost(name)` is now `controller.hosts.createHost(name)`.
It returns the new `HostRecord` instead of `{ hostId, token, name }`, sign a token for it with `signHostToken`.
-`controller.hosts.revokeHost(hostId, revokedBy)` deletes a host the same way `HostRevokeRequest` does.
User tokens are checked with `controller.users.verifyUserToken(token)`, which returns the user or throws if the token is invalid.
Between version 2.0.0-alpha.13 and 2.0.0-alpha.14 the codebase underwent major refactoring and was ported to TypeScript.
In order to support alpha 14 there are several major changes that need to be done to existing plugins, these are in no particular order
## Rename master and slave
All references to slave and master in the code was renamed to controller and host respectively which means that code referencing these in plugins must be renamed.
In short this means performing a global search and replace on the codebase replacing:
-`master` with `controller`
-`Master` with `Controller`
-`slave` with `host`
-`Slave` with `Host`
in all places it occurs, even as part of longer names such as changing `BaseMasterPlugin` to `BaseControllerPlugin`.
## Change lib imports to import directly from the top level package
Previously code could import sub packages of lib such as lib/lua_tools.
These imports need to be changed to all import from lib directly.
Previously `BaseMasterPlugin` and `BaseInstancePlugin` were imported from `@clusterio/lib`, these have been moved to `@clusterio/controller` and `@clusterio/host` respectively.
Previously the info of a plugin would be loaded by requiring `./plugin_dir/info`, this has been changed to require `./plugin_dir` which in turn means that `info.js` is no longer the default entrypoint.
To solve this you can either add `"main": "info.js",` to package.json or rename `info.js` to `index.js` (the rest of this guide assumes the entrypoint was renamed to `index.js`).
The motivation behind this change was to allow TypeScript to output its build to a sub-folder of the plugin, if you're using TypeScript you'll likely have the entrypoint be located in `dist/node/index.js` instead.
The expected place where the declaration of a plugin is exported has also changed from `module.exports` to `module.exports.plugin`:
**Alpha 13**
```js
// info.js
module.exports={
name:"plugin_name",
...
}
```
**Alpha 14 (JS)**
```js
// index.js
constplugin={
name:"plugin_name",
...
}
module.exports={
plugin,
};
```
**Alpha 14 (TS)**
```ts
// index.ts
importtype*aslibfrom"@clusterio/lib";
exportconstplugin: lib.PluginDeclaration={
name:"plugin_name",
...
};
```
### Modify exposed path in webpack.config.js
The `./info` import path exposed in the ModuleFederationPlugin in the `webpack.config.js` needs to be changed to `./`.
E.g.:
**Alpha 13**
```js
// In webpack.config.js
newwebpack.container.ModuleFederationPlugin({
...
exposes:{
"./info":"./info.js",
...
},
}),
```
**Alpha 14**
```js
// In webpack.config.js
newwebpack.container.ModuleFederationPlugin({
...
exposes:{
"./":"./index.js",
...
},
}),
```
## Remove usage of ConfigGroup
The Config system was refactored and the `ConfigGroup` concept was completely removed.
Config fields are now defined directly on the exported plugin info.
// Event is an instance of the ForwardedEvent class
if(event.stringValue){
// ...
}
// src and dst describe where the message came from and who it was addressed to.
}
...
}
```
### Refactor message sending
Message are now addressed end to end instead of using the confusing forwardTo and broadcastTo properties of the message definition.
When broadcasting events the special `"allInstances"`, `"allHosts"`, and `"allControls"` shorthand addresses can be used.
There's two methods for sending messages `.send` and `.sendTo` where `.send` is a shorthand for `.sendTo` with an address filled in as the other side of the link, this is mostly only useful in control plugins where the other side of the link is the controller.
Clusterio communicates using a simple homebrew message passing protocol based on JSON over WebSocket with limited support for reliable transport of messages.
The protocol is implemented in [packages/lib/src/link](/packages/lib/src/link) and uses the `ws` library to handle the WebSocket connections on the Node.js side and the native WebSocket API on the browser side.
The wire schemas are defined in [messages_core.ts](/packages/lib/src/data/messages_core.ts); see the [communication overview](../communication.md) for how links, connectors, and routing fit together.
Two general kinds of messages are supported: events and requests.
Events are messages sent by one party and received by the other and offer no notification for the sender on whether it was actually received.
Requests are pairs of request-response messages where one side sends the requests and the other side replies with a response.
Apart from the handshake used to establish connections the protocol is directionless, and either party can send or receive both requests and events.
## Connection flow
Clients connect via WebSocket to the `/api/socket` endpoint and upon establishing connection to the server the connection is in the handshake phase and the server will send a `hello` message to the client.
If the client has no previous session with the server it sends a `registerHost` or `registerControl` message depending on what kind of client it is to the server.
If the registration is valid, the server responds with a `ready` message containing the client's address, a session token, and session parameters.
The connection has then left the handshake phase; otherwise the server closes it with a WebSocket close code indicating the kind of failure.
Upon leaving the handshake phase both parties should start sending heartbeat messages with the sequence number of the last received message.
Routable messages are stored in a send buffer until a heartbeat acknowledges an equal or higher sequence number, or the session ends.
The heartbeat's top-level `seq` acknowledges receipt by the transport, not successful completion of a message handler.
If the connection goes stale or is unexpectedly closed, the client can re-establish the same session by connecting again and sending a `resume` message after receiving `hello`.
If the session resume is successful then the server will respond with a `continue` message and the connection is active again.
The client then re-sends buffered messages newer than the `lastSeq` in `continue`.
The server likewise re-sends messages newer than the `lastSeq` supplied by the client in `resume`.
Otherwise the server responds with `invalidate`, and the client discards the old send buffer and registers a new session.
Session expiry, buffer overflow, or process failure can lose pending work; plugins must not treat transport recovery as an application-level exactly-once guarantee.
The [send-buffer setting](../configuration.md#controllermax_send_buffer_size) controls how much unacknowledged message data the controller retains for a connection.
## Host pairing
The host id and token sent in `registerHost` are requested by the host over HTTP the first time it starts, before it connects.
Hosts are only added to the controller by approving a pairing request, and `registerHost` is rejected with close code 3000 (Unauthorized) for a host id the controller has no record of.
Revoking a host deletes its record, so its token stops working and the host has to be approved again the next time it starts, which gives it a new id.
See [Setting up remote hosts](/README.md#setting-up-remote-hosts) for how this looks to the user.
The endpoints below take and return JSON bodies and are sent with `Cache-Control: no-store`.
The `POST` endpoints return errors as `{"error": "<message>"}` and respond with 503 while the controller is in recovery mode.
A request body that is not valid JSON or is larger than 4 kB is rejected with 400 or 413 before it reaches the endpoint, without a JSON body.
`GET /api/host-pairing/check` also works in recovery mode, and responds to a token that is not valid with a plain 401 without a JSON body.
A user or host token is sent in either an `Authorization: Bearer <token>` header or an `X-Access-Token` header.
### `POST /api/host-pairing`
Creates a request for a host to join the cluster.
-`name: string` - Name of the host, at most 100 characters after trimming whitespace, without control characters.
-`version: string` (optional) - Version of the host.
Responds with 202 and `{"requestId": string, "code": string}`.
The `code` is shown to the user to approve the request with, formatted as six characters like `ABC-DEF`.
Codes are matched ignoring case and anything that is not a letter or digit, so `abcdef` matches `ABC-DEF`.
The `requestId` is a secret used to poll for the result, it is never shown to users.
If a user token is sent and the user has the `core.host.approve` permission the host is created right away and the response is 200 with the same body as an approved poll.
An invalid token gives 401 and a user without the permission gives 403, a token is never ignored to create a pending request instead.
A missing or too long name gives 400.
Too many pending requests from the same address, or in total as set by [controller.host_pairing_max_pending](/docs/configuration.md#controllerhost_pairing_max_pending), gives 429 with a `Retry-After` header in seconds.
### `POST /api/host-pairing/poll`
-`requestId: string` - Id returned when the request was created.
Responds with 202 and `{"status": "pending"}` while the request is waiting for approval.
Once approved it responds with 200 and `{"hostId": integer, "token": string, "name": string}`, where `name` is the name the host was approved with, which can differ from the one asked for.
Rejected requests give 403, and unknown or expired requests give 404.
The result of an approved or rejected request is only returned once, after which the request is gone.
A pending request expires 30 seconds after it was last polled, and after waiting for the time set by [controller.host_pairing_timeout](/docs/configuration.md#controllerhost_pairing_timeout).
The host polls every 2 seconds, and makes a new request when the previous one has expired.
### `POST /api/host-pairing/cancel`
-`requestId: string` - Id returned when the request was created.
Removes a pending request, sent by the host when it stops waiting.
Responds with 204 whether or not the request existed.
### `GET /api/host-pairing/check`
Checks a host token.
Responds with 200 and `{"valid": true}` if the token is valid and the host it was issued to still exists, and with 401 otherwise.
A host with a saved token checks it with this when it starts, and requests to join again if the token is not accepted.
## Connector flow
To manage the complexity of connecting, reconnecting, resuming or invalidating dropped sessions the WebSocket connection for a `Link` class is managed by a connector.
There's three kinds of connectors defined: `WebSocketClientConnector`, `WsServerConnector` and `VirtualConnector`.
The virtual connector is for in-memory connections and doesn't have any connection state logic.
For the WebSocket connectors there's four states they can exist in:
-`closed` - The connector has just been created or has been explicitly closed.
-`connecting` (client only) - The connector is in the process of trying to connect to the server.
-`connected` - The connector is successfully connected.
-`resuming` - The connection dropped and the connector is in the process of attempting to resume it.
Along with the following events emitted when transition between states:
-`connect` - A new session is established, moving from `connecting` (client) or `closed` (server) to `connected`.
-`drop` - The transport was lost but the session is retained for resumption, moving from `connected` to `resuming`.
-`resume` - The retained session is reconnected and unacknowledged buffered messages are replayed, moving from `resuming` to `connected`.
-`invalidate` - The previous session cannot be reused, moving from `resuming` to `connecting` (client only); server session expiry emits `close` instead.
-`close` - The connector is closed, moving from any state to `closed`.
Invalidation or closure rejects pending requests with `SessionLost`.
Plugins must handle lost sessions, stale state, and application retries where required; see [connection handling](../writing-plugins.md#handling-connection-events) for the plugin hooks and queueing behavior.
The client connector, named `WebSocketClientConnector` in the source, keeps trying to establish and maintain a connection after `.connect()` is called, stopping on terminal failures such as authentication or policy rejection, or when `.close()`/`.setClosing()` is called.
For example, policy violation (1008) stops retries, while try-again-later (1013), used for send-buffer overflow, permits reconnecting.
All communication uses WebSocket text frames containing a JSON object.
Every frame has a `type` string; the other fields depend on that type.
Requests, responses, and events carry an integer transport sequence number in `seq`, plus source and destination addresses in `src` and `dst`.
Message-specific data is carried in `data` when present.
An address is `[type, id]`, optionally followed by a request ID: `[type, id, requestId]`.
Address types are 0 for controller, 1 for host, 2 for instance, 3 for control, and 4 for broadcast.
The controller's ID is 0; a broadcast address uses its ID to identify the target endpoint type.
For example, `[2, 123]` addresses instance 123, while `[4, 2]` addresses all instances.
Broadcast destinations are for events.
A request puts its request ID in `src`; the response returns that ID in `dst` so the sender can match it to the pending request.
This ID is separate from transport `seq`, which is used for acknowledgements and buffering.
### Handshake Messages
The following messages are used during the handshake phase.
Their listed fields belong inside the frame's `data` object, except for `invalidate`, which has no data.
Property names are case-sensitive.
#### `hello`
Sent by the controller when the WebSocket connection has been opened.
-`version: string` - The version of the controller, e.g. "2.0.0".
-`plugins: Object<string, string>` -
Object mapping plugin names to plugin version for plugins that are loaded on the controller.
#### `registerHost`
Host handshake for establishing a new connection session.
-`token: string` - Authentication token.
-`version: string` - The protocol version of the host, e.g. "2.0.0".
-`id: integer` - ID of the host.
-`plugins: Object<string, string>` -
Object mapping plugin names to plugin version for plugins that are available on the host.
The controller closes the connection with code 3000 (Unauthorized) if the token is invalid, was issued for another id, or if there is no host with this id, see [Host pairing](#host-pairing).
The host checks its token before connecting and requests to join again if it is not accepted, so this normally only happens to a host that is already running.
A host that has been rejected exits with code 8, and the run scripts created by the installer do not restart it.
A host that is revoked while connected is disconnected, is rejected when it tries to connect again, and shuts down.
#### `registerControl`
Control handshake for enstablishing a new connection session.
-`token: string` - Authentication token.
-`version: string` - The protocol version of the control, e.g. "2.0.0".
#### `resume`
Handshake for resuming an existing connection session where the connection had dropped unexpectedly.
-`sessionToken: string` - Session token to resume session with.
-`lastSeq: integer` (optional) - Last transport sequence number received by the client; omitted if none has been received.
#### `invalidate`
Sent by the controller in reply to a resume for a session that cannot be resumed.
The frame is `{"type":"invalidate"}`, without a `data` object.
The client should start a new session with `registerHost` or `registerControl` in response.
#### `continue`
Sent by the controller to signal resuming the session was successful.
-`lastSeq: integer` (optional) - Last transport sequence number received by the server; omitted if none has been received.
-`sessionTimeout: number` - Session timeout in seconds.
-`heartbeatInterval: number` - Interval between heartbeats in seconds.
#### `ready`
Sent by the controller to signal the connection has been established with a new session.
-`src: Address` - The client's address, used as the source of messages it sends.
-`sessionToken: string` - Token to resume the session with.
-`sessionTimeout: number` - Session timeout in seconds.
-`heartbeatInterval: number` - Interval between heartbeats in seconds.
-`account` (optional) - Account details for a control client: a `name` and a `roles` array whose entries contain `name`, `id`, and `permissions`.
### Heartbeat message
Once connected as indicated by the `ready` and `continue` messages both sides of the connection starts sending heartbeats at the inverval given in the `ready` and `continue` messages.
The heartbeat should contain the `seq` of the last received message, and receiving a heartbeat with a given sequence number means that the other party of the link has received all messages up to that seq number.
#### `heartbeat`
Sent by both the controller and clients at the interval given in the continue/ready messages.
-`seq: integer` (optional, top-level) - Last transport sequence number received over the link.
For example, `{"type":"heartbeat","seq":123}` acknowledges messages through sequence 123.
Before receiving any messages, send `{"type":"heartbeat"}`; there is no `data.last_seq` field.
### Events
Events are messages sent in one direction that the receiver acts upon without replying; use [requests](#requests) when a response is needed.
Their wire `type` is `"event"`, with a separate `name` identifying the registered event class.
Plugin names are prefixed, for example `foo_frobber:SomeEvent`.
The `data` payload, when present, contains event-specific properties.
See [packages/lib/src/data/](/packages/lib/src/data/) for core event definitions.
Plugins may define their own events as well.
#### `event`
-`seq: integer` - Transport sequence number.
-`src: Address` - Source address.
-`dst: Address` - Destination address, including broadcast destinations.
-`name: string` - Registered event name.
-`data` (optional) - Payload defined by the event class.
### Requests
Requests expect a response in return.
They function like HTTP requests except that either party of a connection can initiate one.
The wire `type` is `"request"`; a successful reply uses `"response"`, and a failed request uses `"responseError"`.
The request's `name` identifies the registered class, prefixed with the plugin name for plugin messages, such as `foo_frobber:Frobnicate`.
See [packages/lib/src/data/](/packages/lib/src/data/) for core request definitions.
Plugins may define their own requests as well.
#### `request`
-`seq: integer` - Transport sequence number.
-`src: Address` - Source address, including the request ID.
-`dst: Address` - Destination address.
-`name: string` - Registered request name.
-`data` (optional) - Payload defined by the request class.
#### `response` success
-`seq: integer` - This response's transport sequence number, not the request's sequence number.
-`src: Address` - Responding endpoint.
-`dst: Address` - Requester's address, including the original request ID.
-`data` (optional) - Payload defined by the request class's `Response` definition.
#### `responseError`
-`seq: integer` - This error response's transport sequence number.
-`src: Address` - Responding endpoint.
-`dst: Address` - Requester's address, including the original request ID.
-`data.code: string` (optional) - Registered name of the error class, `<plugin>:<class>` for plugin errors.
Missing or unknown names are received as `InternalServerError`.
-`data.stack: string` (optional) - Error stack.
### Graceful Disconnect
To reduce data loss when closing a link, ask the peer to prepare and wait for it to finish before closing the WebSocket.
Either party can initiate the disconnect and is then responsible for closing the socket.
These are connector control frames, not registered request/response classes; forced closure and process crashes can still lose work.
#### `disconnect` with `data: "prepare"`
The frame `{"type":"disconnect","data":"prepare"}` requests a graceful disconnect.
The receiving party should stop tasks that send events and requests over the link and wait for pending requests it has over the link to complete, but it should not stop responding to requests sent to it.
Only after this process is done should the response be sent.
#### `disconnect` with `data: "ready"`
The frame `{"type":"disconnect","data":"ready"}` signals that preparation is complete and the connection can be closed.
When using `WebSocketBaseConnector.disconnect()`, the initiator must stop producing new work and await its own `Link.prepareDisconnect()` before calling `disconnect()`.
The connector closes as soon as the peer sends `ready` or the timeout expires; it does not drain the initiator's pending requests itself.
There is no error-response variant; the connector also has a timeout so an unresponsive peer cannot block shutdown indefinitely.
### Example handshake
Server hello after opening the WebSocket.
```json
{
"type":"hello",
"data":{
"version":"2.0.0",
"plugins":{}
}
}
```
Host registering a new connection session.
```json
{
"type":"registerHost",
"data":{
"token":"xyzabc123",
"version":"2.0.0",
"id":123,
"plugins":{}
}
}
```
Controller reply signaling the session is established and ready.
```json
{
"type":"ready",
"data":{
"src":[1,123],
"sessionToken":"xyz.azy.bzy",
"sessionTimeout":60,
"heartbeatInterval":15
}
}
```
Host and controller can now send events and requests over the connection.
library to Factorio and requires scenarios to be coded specifically for
it. Mods on the other hand work without modifications.
This can be disabled with the `factorio.enable_save_patching` config on
a per instance basis, however disabling it means that modules are not
loaded into the game and most plugins will not function.
<sub>1: Before version 2.0 of Clusterio this was done at runtime
with a patcher called Hotpatch, but due to the difficulty in supporting
run time patching and the lack of documentation it was replace it with
save patching instead.</sub>
Clusterio runs a group of Factorio servers as a cluster, managed through a web interface or the `clusterioctl` command-line client.
The controller is the central point of contact: clients connect to it to manage the cluster, and hosts connect to it to receive commands and exchange messages.
Each host manages its local Factorio server processes, with each server belonging to an instance.
User Accounts
-------------
## System overview
The master server creates and stores accounts for all Factorio users
that log into instances under its control. These accounts are used to
store per player data and authorize management of the cluster. By
default users accounts are assigned a Player role that gives a limited
read access to the cluster resources, and admins of the cluster can be
assigned a Cluster Admin role to get access to manage it.
The diagram shows one possible layout with two physical servers, each running a host and three Factorio servers.
The controller shares a machine with one host in this example, but it can run on a separate machine.
The number of hosts and instances can vary with the needs of the cluster.
The blue arrows point from the hosts and clients to the controller they connect to; messages can travel in both directions over those WebSocket connections.
The orange lines connect each host to the Factorio processes it manages.
Open the [full-size diagram](assets/clusterio_2.0_block_diagram.svg) to zoom in, or use the [Draw.io source](assets/clusterio_2.0_block_diagram.drawio) to edit it.
For how messages are handled and routed between these components, see the [communication guide](communication.md).
## Save Patching
In order to efficiently get data in and out of running Factorio servers and to provide custom interfaces Clusterio uses save patching<sup>[1]</sup> to add code to the game.
The save patching is based on the built in [event_handler](https://github.com/wube/factorio-data/blob/master/core/lualib/event_handler.lua) library to Factorio and requires scenarios to be coded specifically for it.
Mods on the other hand work without modifications.
This can be disabled with the `factorio.enable_save_patching` config on a per instance basis, however disabling it means that modules are not loaded into the game and most plugins will not function.
A save that was patched before has its modules removed again when it's started with save patching disabled.
To use the modules outside of Clusterio the `npx clusteriohost create-scenario` command creates a scenario with the modules patched into it.
Note that the IPC between the game and the host is not available outside of Clusterio, so modules that depend on it will not work as expected in such a scenario.
<sub>1: Before version 2.0 of Clusterio this was done at runtime with a patcher called Hotpatch, but due to the difficulty in supporting run time patching and the lack of documentation it was replace it with save patching instead.</sub>
## User Accounts
The controller creates and stores accounts for all Factorio users that log into instances under its control.
These accounts are used to store per player data and authorize management of the cluster.
By default users accounts are assigned a Player role that gives a limited read access to the cluster resources, and admins of the cluster can be assigned a Cluster Admin role to get access to manage it.
Clusterio clusters are managed through the master server by using the
`clusterioctl` command line interface which is invoked by running
`npx clusterioctl <command>` in the clusterio directory. This document
uses the shorthand `ctl> foo` to indicate `npx clusterioctl foo` should
be executed. Mandatory parameters are shown in `<angles bracket>` and
optional pameters are in `[square brackets]`.
Clusterio clusters are managed through the controller by using the `clusterioctl` command line interface which is invoked by running `npx clusterioctl <command>` in the clusterio directory.
This document uses the shorthand `ctl> foo` to indicate `npx clusterioctl foo` should be executed.
Mandatory parameters are shown in `<angles bracket>` and optional pameters are in `[square brackets]`.
Before `clusterioctl` can be used it needs to be configured for the
cluster it will connect to. The easiest way to do this is to run
`npx clusteriomaster bootstrap create-ctl-config <username>` on the
master server, which creates the necessary `config-control.json` for
managing the cluster as the given user.
Before `clusterioctl` can be used it needs to be configured for the cluster it will connect to.
The easiest way to do this is to run`npx clusteriocontroller bootstrap create-ctl-config <username>` on the controller, which creates the necessary `config-ctl.json` for managing the cluster as the given user.
Slaves
------
## Hosts
### List hosts
ctl> host list
Lists all hosts.
Hosts are added to the cluster by running the installer on the computer the host runs on and approving the code it shows, see [Setting up remote hosts](/README.md#setting-up-remote-hosts).
Listing, approving and rejecting hosts waiting for approval needs the `core.host.approve` permission.
### List pending hosts
ctl> host pending
Lists hosts waiting for approval with the code they show, the name they asked for, the address they connected from and how long they have been waiting.
A request goes away when its host stops waiting, and after the time set by `controller.host_pairing_timeout`, an hour by default, after which the host asks again with a new code.
### Approve host
ctl> host approve <code> [--name <host name>]
Approves the host showing the code and prints the id of the new host.
The code is not case sensitive and the dash can be left out.
`--name` gives the host a different name than the one it asked for.
Approving gives the host full access to the cluster, check with `host pending` that the name and address match the computer you are setting up.
### Reject host
ctl> host reject <code>
Rejects the host showing the code, the host then exits with an error.
### Revoke host
ctl> host revoke <host>
Specify the host by name or ID.
Removes the host from the cluster, disconnects it and makes its token stop working.
Instances assigned to the host become unassigned, their files stay on the computer.
Requires the `core.host.revoke` permission.
To add the computer back to the cluster, start the host again and approve it.
### Update Clusterio
ctl> host update <host> [--all] [--restart]
Specify the host by name or ID.
If remote updates are enabled on the host, this will attempt to update `@clusterio/host` on the machine. Add `--all` to also update all npm-installed plugins. Changes will not be applied until the host is restarted. If the `--restart` flag is provided, the host will automatically restart once the update is complete.
Remote updates are enabled by default but can be disabled on a per-machine basis at the owner's discretion. For security reasons, this configuration setting cannot be changed remotely. To modify it, run the following command locally while the host is offline:
host> config set host.allow_remote_updates true/false
The controller also supports remote updates, following the same logic. The corresponding commands are:
ctl> controller update [--all] [--restart]
controller> config set controller.allow_remote_updates true/false
### Mark restart required
ctl> host mark-restart-required <host>
Marks the host as requiring a restart, the same way a plugin update does. Use it after installing or changing a plugin by other means than `host plugin install`, such as a plugin checked out into `external_plugins`. The mark shows up in the web interface on the next system metrics poll. The controller has the same command:
ctl> controller mark-restart-required
## Instances
### List Instances
ctl> instance list [--templates]
List instances.
Templates are only listed if `--templates` is passed.
Create a new instance. `name` is the instance name. `--id` is the optional id. `--from` copies the config of an existing instance. `--template` makes the new instance inherit all config values from the given template. `--as-template` creates the instance as a template, see `instance.is_template` in the [configuration docs](configuration.md). Use `--as-template --from <instance>` to make a template out of an existing instance.
### Kill instance
ctl> instance kill <name>
Terminates the running Factorio server without giving it a chance to save or cleanup after itself.
Note: This may cause loss of data.
### Save game
ctl> instance save-game <name>
Saves the running game of the instance to the save it was started from and waits for the save to finish.
### Extract players from instance
ctl> instance extract-players <name>
Creates a user account in the cluster for each player that has been online on the currently running save of the instance and sets the online time of that account on this instance to the online time recorded in the save.
Useful when importing a save to a cluster.
Note that the online time in a save is recorded differntly from how Clusterio records online time.
Most notably Clusterio records wall clock time while the save time is recorded in ticks and this extraction of player time assumes the game runs at 60 ticks a second.
### Config management
#### List Configuration
ctl> instance config list <instance>
List config values for an instance. `instance` is the instance whose config values to list. Values inherited from a template are marked with `(inherited)`, object fields also list the properties set on the instance.
#### Set config value
ctl> instance config set <instance> <field> [value] [--stdin]
Set field in instance config. `instance` is instance to set. `field` is field to set. `value` is value to set in field. `--stdin` signals to read value from stdin.
Make a field in the instance config take its value from the template again, undoing a previous `set`. `instance` is the instance to update, `field` is the field to inherit, `prop` limits it to one property of an object field.
#### Edit instance config
Users
-----
ctl> instance config edit <instance> [editor]
Clusterio automatically creates user accounts for all players that join
an instance when save patching is enabled. These accounts are used to
store per player data shared between instances, like if the player
should be whitelisted, admin or banned.
Edit instance config in a text editor. Once file is saved and editor has been exited, changed config values will be read back in. `instance` is the instance to edit, `editor` is the editor to use. If `editor` is not given, clusterioctl will read from the `EDITOR` or `VISUAL` enviroment variables.
### Assign instance to host
ctl> instance assign <instance> [host]
Assigns instance to host. `instance` is instance to assign, `host` is host to assign to. If `host` is not provided, instance will be unassigned.
## Users
Clusterio automatically creates user accounts for all players that join an instance when save patching is enabled.
These accounts are used to store per player data shared between instances, like if the player should be whitelisted, admin or banned.
### List users
@@ -46,40 +167,32 @@ Lists all user accounts in the cluster along with some data for each.
ctl> user create <name>
Creates a new empty user account for the given Factorio player. Note
that case matters here. This is usually not required as user accounts
are created automatically for players that join instances with save
patching enabled.
Creates a new empty user account for the given Factorio player.
Note that case matters here.
This is usually not required as user accounts are created automatically for players that join instances with save patching enabled.
### Promote User to Server Admin
ctl> user set-admin <name> [--revoke] [--create]
Promotes the user given to in-game admin on instances with the
`sync_adminlist` option enabled. If the `--revoke` switch is used the
user is removed from the adminlist.
Promotes the user given to in-game admin on instances with the`sync_adminlist` option enabled.
If the `--revoke` switch is used the user is removed from the adminlist.
Since admin status is a part of the account data the account must exist
for this to succeed, passing `--create` will create the account if it
does not exist.
Since admin status is a part of the account data the account must exist for this to succeed, passing `--create` will create the account if it does not exist.
**Note:** Being a server admin does not grant any access to manage the
cluster. See [set-roles](#set-cluster-roles) for adding roles which
grant access to managing the clusetr.
**Note:** Being a server admin does not grant any access to manage the cluster.
See [set-roles](#set-cluster-roles) for adding roles which grant access to managing the clusetr.
### Whitelist User
ctl> user set-whitelisted <name> [--remove] [--create]
Add the user given to the whitelist on instances with the
`sync_whitelist` option enabled. If the `--remove` switch is used the
user is removed from the whitelist.
Add the user given to the whitelist on instances with the`sync_whitelist` option enabled.
If the `--remove` switch is used the user is removed from the whitelist.
Since whitelisted status is a part of the account data the account must
exist for this to succeed, passing `--create` will create the account if
it does not exist.
Since whitelisted status is a part of the account data the account must exist for this to succeed, passing `--create` will create the account if it does not exist.
### Ban User
@@ -87,46 +200,201 @@ it does not exist.
ctl> user set-banned <name> [--reason <message>] [--pardon] [--create]
Ban user in-game from instances with the `sync_banlist` option enabled.
Reason is a message that will be shown to the user when attemption to
log in. If the `--pardon` switch is used it removes the ban.
Reason is a message that will be shown to the user when attemption to log in.
If the `--pardon` switch is used it removes the ban.
Since ban status is a part of the account data the account must exist
for this to succeed, passing `--create` will create the account if it
does not exist.
Since ban status is a part of the account data the account must exist for this to succeed, passing `--create` will create the account if it does not exist.
Note: This bans the user from logging in to Factorio servers in the
cluster, it does not revoke access to any cluster management they might
have, see next section on setting roles for revoking tha.
Note: This bans the user from logging in to Factorio servers in the cluster, it does not revoke access to any cluster management they might have, see next section on setting roles for revoking tha.
### Bulk importing admin, whitelist and ban lists
Whitelists and admin lists can be bulk imported with the following commands:
ctl> user whitelist-bulk-import <file> [--create]
ctl> user admin-bulk-import <file> [--create]
The file should simply be newline seperated names:
joe
bob
allan
The `--create` option will create the user if they do not exist, and then set the relevent value.
Banlists can be bulk imported with the following command:
ctl> user banned-bulk-import <file> [--create]
The file should be names, followed by a ban reason:
joe greifed copper mines
bob stole materials
allan rude language
The first string on the line is used as the username, the others are used as the ban reason.
The `--create` option will create the user if they do not exist, then ban them.
### Set Cluster Roles
ctl> user set-roles <name> [roles...]
Replaces the roles the user has in the cluster with a new list of roles.
Calling with an empty roles argument will remove all roles from the
user.
Calling with an empty roles argument will remove all roles from the user.
By default there's a roled named Cluster Admin which grants access to
everything and a role named Player which grants a limited read access to
the cluster, see [the section on roles](#roles) for more information
about setting up roles and permissions.
By default there's a roled named Cluster Admin which grants access to everything and a role named Player which grants a limited read access to the cluster, see [the section on roles](#roles) for more information about setting up roles and permissions.
### Delete user
ctl> user delete <name>
Deletes everything stored on the master server for this user.
Deletes everything stored on the controller for this user.
Note: If the player is banned from the cluster this will effectively
unban them, as the ban status is stored with the user account.
Note: If the player is banned from the cluster this will effectively unban them, as the ban status is stored with the user account.
Note: If the player joins the cluster again a new account will be made
for them automatically.
Note: If the player joins the cluster again a new account will be made for them automatically.
Roles
-----
## Roles
To be written.
## Notes
Free text notes can be attached to hosts, instances, users, roles, mod packs, mods and saves.
They are stored on the controller and shown to everyone with access to the resource in the web interface, and can for example be used to record who runs a host or what an instance is meant for.
The web interface renders notes as Markdown.
A mention of the form `@<type>/<id>` becomes a link to that resource showing its name, for example `@instance/778528005` or `@user/cooldude2606`.
This works for hosts, instances, users, roles and mod packs, and the id is shown if the name is not available.
The note itself stores the mention as written.
### List notes
ctl> note list [--type <type>]
Lists all notes, optionally limited to one resource type.
### Set note
ctl> note set <type> <id> <content>
Sets the note of a resource.
The type is one of host, instance, user, role, mod_pack, mod or save.
The id is the numeric id for hosts, instances, roles and mod packs, the lower case name for users, `<name>_<version>` for mods and `<instance id>/<file name>` for saves.
### Delete note
ctl> note delete <type> <id>
Removes the note of a resource.
Notes are also removed when the resource they are attached to is deleted.
## Plugins (Local)
To be written.
## Plugins (Remote)
The installer asks "Allow remote updates and plugin installs from the web interface?" and the answer sets `allow_remote_updates`, `allow_plugin_updates` and `allow_plugin_install` together on that machine, pass `--remote-npm` or `--no-remote-npm` to answer it from the command line.
Remote updates allow ctl and the web UI to update Clusterio and plugins on machines within the cluster.
Remote installs allow them to install any package from npm onto the machine, which is a risk of arbitrary code execution through malicious npm packages, so only allow them if you trust everyone with access to install plugins.
In a config created without the installer remote updates are enabled and remote installs are disabled by default.
See below for changing these settings after installing.
### List
ctl> controller plugin list
ctl> host plugin list <host>
This command lists all plugins currently known to the remote. Newly installed plugins will not be displayed until the remote is restarted. Plugins that are distributed as an npm package can be updated remotely.
If remote plugin updates are enabled on the remote, this will execute `npm update --save <name>`, which stays within the version range saved in the remote's `package.json`.
To update to a specific version, including one outside that range, pass it as `<name>@<version>`, for example `@clusterio/plugin-example@2.1.0`.
The version must be published on npm, and the remote then executes `npm install --save <name>@<version>`.
The plugin must already be installed on the remote, and only remote plugin updates need to be enabled.
Changes will not be applied until the remote is restarted.
If the `--restart` flag is provided, the remote will automatically restart once the update is complete.
### Update All
ctl> controller update --all [--restart]
ctl> host update <host> --all [--restart]
Updates the Clusterio package and all plugins installed from npm in a single `npm update --save` command. Updating everything at once avoids ending up with multiple versions of `@clusterio/lib` in the installation. Requires both remote updates and remote plugin updates to be enabled on the remote.
Searches the npm registry for packages tagged with the clusterio-plugin keyword.
The search is done by the controller and does not require remote installs to be enabled.
The web UI has a plugin browser on the plugins page that uses the same search.
### Concurrent Operations
Only one npm operation runs at a time on a machine.
A standalone controller and host share their working directory, so the operation holds the `npm-operation.lock` file there while npm runs.
Other installs and updates on the same machine or installation are rejected with "Another package operation is already in progress on this machine" until it finishes.
A lock file left behind by a process that is no longer running is removed automatically.
### Enable/Disable Updates
host> config set host.allow_plugin_updates true/false
controller> config set controller.allow_plugin_updates true/false
Remote updates are enabled by default but can be disabled on a per-machine basis at the owner's discretion. For security reasons, this configuration setting cannot be changed remotely. To modify it, run the above commands locally while the remote is offline.
### Enable/Disable Installs
host> config set host.allow_plugin_install true/false
controller> config set controller.allow_plugin_install true/false
Remote installs are disabled by default but can be enabled on a per-machine basis at the owner's discretion. For security reasons, this configuration setting cannot be changed remotely. To modify it, run the above commands locally while the remote is offline.
## Prometheus Metrics
The controller exports metrics for the whole cluster on the `/metrics` endpoint.
Fetching it requires a token for a user with the `core.controller.metrics` permission, sent either as an `Authorization: Bearer <token>` header or as an `X-Access-Token` header.
Requests without a valid token get a 401 response and requests for users lacking the permission get a 403 response.
Create a role with just this permission, a user to hold it, and generate a token for that user.
Give the user a name that is not a valid Factorio player name, such as one starting with `#`, so a player joining the cluster can never claim the account and its token:
ctl> role create Prometheus --permissions core.controller.metrics
Then pass the token in the scrape job for the controller in `prometheus.yml`:
scrape_configs:
- job_name: clusterio
authorization:
credentials: <token>
static_configs:
- targets: ["localhost:8080"]
Concurrent scrapes share a single round of metrics gathering from the hosts, so multiple scrapers polling the controller do not multiply the load on the hosts.
Review observable behavior across the whole PR, including callers, consumers, and lifecycle transitions.
Reading every changed file and passing tests do not establish that those behaviors were covered.
Use [contributing](contributing.md) for setup and validation commands.
## Establish the scope
- Record the PR's base and current head; preserve unrelated work with an isolated checkout when needed.
- Read the requirements and relevant documentation at the reviewed revision.
Distinguish intended policy from observed behavior when they disagree.
- Map each affected behavior from input through authorization, routing, state changes, and its consumers.
On a follow-up, verify the fixes and reassess the whole PR's affected behavior; explicitly label a deliberately narrower review.
- Compare the PR base, previously reviewed head, and current head when attributing a follow-up finding.
Distinguish a new regression, an incomplete fix, and a previously missed defect.
Existing producer behavior can become unsafe through a new consumer; reject a finding as pre-existing only after comparing the complete failing path.
- Trace the actual entrypoint and earlier guards before declaring a failure reachable.
Check supported configurations and compatibility policy; a merged revision or earlier clean review is not proof of correctness.
## Surface overlapping responsibilities
- For substantial new or replaced functionality, check whether existing code or test helpers already serve the same responsibility, including outside the changed files.
Search by behavior, then verify the implementations and their callers; nearby code is not necessarily the preferred shared implementation.
- Compare contracts, callers, and constraints, not just names or similar code.
Distinguish shared behavior and general correctness requirements from differences that may justify separate implementations.
- Surface significant overlap to the developer with source locations for both implementations, the duplicated responsibility, its consequences, and any legitimate differences.
Explain maintenance or consistency risks without presenting them as reproduced runtime defects.
- Recommend an approach based on the evidence, including alternatives, tradeoffs, and affected callers or tests.
Leave the architectural decision to the developer unless already authorized; do not silently consolidate the implementations or dismiss the overlap as intentional.
- Recheck after substantial fixes, especially when shared functionality is replaced, bypassed, or overridden.
Keep unresolved design questions in the candidate ledger and final report, and record the developer's decision when provided.
## Exercise the boundaries
- Write a small matrix of relevant states and transitions before probing.
Distinguish intended state, eligibility, and observed state, and vary the conditions that can make them differ.
Exercise transitions through the components that produce and consume the state; constructing the final state alone does not verify the transition.
- Check adverse paths as well as success: failure, retry, removal, restart, reconnect, and overlapping operations where applicable.
Select relevant cases from these recurring review gaps:
| Boundary | Questions to check |
| --- | --- |
| Data contracts | Does real serializer output satisfy every consumer, including Lua restoration and inspection UI? Check omitted, empty, partial, malformed, reserved, and numeric-limit values before durable mutation. |
| State and identity | Do delayed reads and mutation completions still belong to the selected entity? Do disabled components retain their local state? Can an identifier or generation be reused after recovery? |
| Persistence and synchronization | Do save/reload, migration, deletion, and resubscription preserve versions and tombstones? Compare full snapshots, deltas, and optimized paths with ordinary transfers. |
| Ownership and errors | Can concurrent work remove another operation's lock, handle, or cache? Check synchronous throws, early promise rejection, partial mutation, and cleanup after each failure. |
| Protocol and compatibility | What does the receiver do with the message, close code, or acknowledgement? Check early startup, mixed versions, and supported older APIs. |
| Distribution | Do generated validators, browser bundles, and installed packages behave like the source checkout? Check actual export maps and plugin discovery. |
- Use focused reproductions with concrete inputs and expected versus actual results.
Keep the logic under review real: stubbing permissions, effects, or state transitions can erase the defect being investigated.
Check that the harness reaches the intended path; compare a normal case and the base revision where practical.
Identify assumptions introduced by fixtures, caches, shared identities, or execution order, and vary those relevant to the change.
- Control concurrency with explicit gates and assertions about ownership and completion, rather than timing alone.
Check both that invalid actions are prevented and that valid work eventually makes progress.
Vary delays and completion order, including changes to an action's preconditions between initiation and execution.
## Account for every candidate
- Keep a compact ledger of candidate issues and uncovered scenarios throughout the review.
Give each candidate a disposition: reported, rejected with evidence, or unresolved with the next check needed.
- Before finishing, compare probe outputs with the final findings.
A reproduced anomaly must not disappear just because several other issues were already found.
- Carry unresolved candidates and coverage gaps into handoffs and context summaries, alongside published findings.
For each finding, identify the trigger, consequence, source location, and supporting evidence.
Reconcile the previous ledger at every follow-up; an unconfirmed candidate remains unresolved until evidence rejects it or confirms it.
Name remaining material gaps in the final report even when other findings already block merging.
## Report and follow up
- Separate source inspection, focused probes, full suites, hosted CI, browser checks, and Factorio runtime evidence.
Record failures, skips, and unavailable validation; investigate external-service failures before attributing them to the PR.
- Reproduce the original failure when checking a fix, including adjacent cases exposed by the change.
A fix for one malformed field calls for checking the remaining consumer requirements, not just that example.
Limit conclusions to the scenarios exercised, and state which relevant assumptions remain unverified.
Resolve a review thread only when its exact concern is addressed.
- When authorized to publish, refresh the head and existing comments, avoid duplicates, and anchor feedback to the reviewed code.
Read back each posted inline comment to verify its body and location.
Pull requests will usually have suggestions for changelog entries. These can be automatically parsed and formatted by running `node scripts/pr_to_changelog.mts`, though you will have to manually sort through which should go in the changelog in the repository root and which should go in the changelogs in mod directories.
If the script fails due to GitHub's 60/hour public access rate limit, you can set the `GH_TOKEN` environment variable to a fine-grained personal access token which has a much higher rate limits.
## Packages
The [Publish workflow](../.github/workflows/publish.yml) publishes npm packages and Lua mods on GitHub's `release: released` event for a stable release.
Publishing a prerelease does not trigger it; see [GitHub's release event documentation](https://docs.github.com/en/actions/reference/workflows-and-actions/events-that-trigger-workflows#release).
The npm release includes the packages under `packages/` and the six core plugins under `plugins/`.
Community plugins hosted in separate repositories have independent versions and releases.
For this workflow, first merge the changelog and package version updates into `master`, then create and push the release tag on that merged commit.
Create a draft GitHub release for the existing tag and use the GitHub Release Markdown from `scripts/pr_to_changelog.mts` as its notes.
Publishing the draft as a stable release triggers the workflow; pushing a tag alone does not.
Preparing a release PR, pushing a tag, and publishing a release or packages are separate actions; obtain authorization for the publication steps before executing them on someone else's behalf.
The steps below describe publishing npm packages manually instead.
> [!NOTE]
> You will need commit access to the repository as well publish access to the @clusterio org on npm to do this.
1. Inspect `git status` and `git remote -v`, verify which remote is the main repository, and use a clean release worktree on its latest master.
The commands below assume that remote is named `upstream`; preserve unrelated work and external plugin checkouts.
```sh
git switch master
git pull --ff-only upstream master
git status
```
2. Create a release branch and bump the version in the `package.json` files. The commands below use 2.0.1 to 2.0.2 as an example; substitute the versions for the release you are preparing.
```sh
git switch -c release/2.0.2
sed -i 's/"version": "2\.0\.1"/"version": "2.0.2"/' {packages,plugins}/*/package.json
pnpm install --no-frozen-lockfile
```
3. Review and commit the version and changelog changes, then open and merge a release pull request.
```sh
git add -p
git commit -m "Prepare 2.0.2 release"
```
4. After the release pull request is merged, tag the merged commit and push the tag.
```sh
git switch master
git pull --ff-only upstream master
git tag v2.0.2
git push upstream v2.0.2
```
5. For manual publication, perform a clean install and publish the packages. For automated publication, publish the GitHub release as described above instead.
The repository does not track `pnpm-lock.yaml`, so allow pnpm to generate or update it during installation.
2. Run `pnpm install --no-frozen-lockfile` in the repository root
3. Run `pnpm adduser` to authenticate
4. Run `pnpm publish` in the plugin/package directory
Before confirming the publish, ensure the packed size is approximately the same size as the last version (or within the expected size)
After publishing, test creating and installing the plugin on a fresh install.
## Lua mods
To publish lua mods like clusterio_lib, start by building from source:
```sh
pnpm run build-mod
```
or download the latest artifacts from the CI. When uploading the mods to the mod portal, make sure the latest version is the last version uploaded, as the game/mod portal/ingame updater isn't consistent in how it figures out the "latest version"
## Release notes
Release notes are expected to be published alongside new releases. We use the github releases system as well as posting in the #announcements channel on the discord. Major releases may warrant a ping.
By default Clusterio will listen for connections over HTTP which is fine for local networks but over the Internet it should idealy be over HTTPS as there are some credentials that could be exposed to eavesdroppers.
The recommended setup is to have an ordinary HTTPS server software like Apache or Nginx to act as a reverse proxy in front of the Clusterio controller.
The reverse proxy would then be set up with a TLS certificate from a Certificate Authoritiy like [Let's Encrypt](https://letsencrypt.org).
Alternatively you can have Clusterio listen for connections over HTTPS directly by providing it with a TLS certificate and private key.
Note that a TLS certificate signed by a recognized Certificate Authority in general requires that you own a domain.
See the last section for how to create and use a self-signed certificate for an IP address if you don't have a domain.
## Using a reverse proxy
Configure the proxy to forward requests over HTTP to the port the controller is listening to so that for example a request to `https://example.com/instances/` is forwarded to `http://localhost:8080/instances/`.
It is possible to host the interface under a sub-path, in that case a request under the sub-path should be mapped to the controller with the sub-path stripped out.
For example `https://example.com/sub/path/instances/` would be mapped to `http://localhost:8080/instances`.
Additianlly WebSocket connections to `/api/socket` also needs to be forwarded.
Typically this requires explicit configuration and support for forwarding WebSocket connections.
If the interface hosted under a sub-path the `/api/socket` address is relative to the sub-path and the sub-path also needs to be stripped out when forwarded.
You can also optionally set the controller to listen only on the loopback interface to prevent it being directly connected to from the outside.
npx clusteriocontroller config set controller.bind_address 127.0.0.1
### Apache Config
To proxy with Apache make sure that `proxy_module`, `proxy_http_module` and `proxy_wstunnel_module` is loaded in the config, then add the following directives to wherever is the approriate place for your flavour of Apache config organization.
```apache
<Location/sub/path/>
ProxyPass"http://localhost:8080/"
</location>
<Location/sub/path/api/socket>
ProxyPass"ws://localhost:8080/api/socket"
</Location>
```
Replace `/sub/path/` with the location on the proxy you want to host the Clusterio interface under.
If the whole domain is to be dedicated to the Clusterio interface then `/sub/path/` should be just `/`.
Pay attention to the use of trailing slashes, as they are important.
## Hosting HTTPS directly from Node.js
Listening on HTTPS is supported by the Clusterio controller via the `controller.https_port` option provided paths to a certificate file and a private key file is configured for the `controller.tls_certificate` and `controller.tls_private_key` options.
Both of these are expected to be unencrypted PEM files.
## Creating and using a self-signed certificate
Create a self-signed certificate with OpenSSL by running
substituting `<ip>` with the IP address the server will be accessed under, for example `IP:203.0.113.4`.
Multiple addresses can be given by supplying a comma sepparated list of `IP:address` pairs and DNS names can also be specified using the `DNS:example.com` format.
You need to supply every name you want to connect to the server under the subjectAltName field, for example to connect to the HTTPS port locally via an address like `https://localhost:4443/` you need to add `DNS:localhost`.
This creates a private key file named `key.pem` and a public certificate file named `cert.pem` and these are used for the `controller.tls_certificate` and `controller.tls_private_key` config options respectively on the controller:
npx clusteriocontroller config set controller.tls_certificate cert.pem
npx clusteriocontroller config set controller.tls_private_key key.pem
Additionally you will need copy the `cert.pem` file to all of the computers you want to set up as hosts and configure node to trust it by setting the `NODE_EXTRA_CA_CERTS` environment variable to a path to this file before the host is started. You can do this by passing `--self-signed-cert path/to/cert` to the installer when setting up a host, which also makes the installer trust the certificate when it asks the controller to approve the host, or manually by editing run-host.cmd, run-host.sh and/or clusteriohost.service by adding the following line next to the one setting `NODE_OPTIONS`:
**For run-host.cmd**
set "NODE_EXTRA_CA_CERTS=path\to\cert.pem"
**For run-host.sh**
export "NODE_EXTRA_CA_CERTS=path/to/cert.pem"
**For clusteriohost.service**
Environment=NODE_EXTRA_CA_CERTS=path/to/cert.pem
You will also need to copy the `cert.pem` file to all of the computers you want to remotely manage the cluster via clusterctl and set the `NODE_EXTRA_CA_CERTS` environment variable as a path to that file before running clusterioctl. For example:
The controller forwards data between Clusterio hosts connected to it and allows the cluster to be remotely managed through WebSocket connections to it either by using the included web interface or the Clusterio ctl command line utility.
# Usage
npx clusteriocontroller <command>
Common options:
*`--plugin-list <file>` JSON file to use for storing the list of plugins that are available to the controller.
Defaults to `plugin-list.json` and will be created if it does not exist.
See the `plugin` command for managing this list.
*`--config <file>` JSON file to use for storing configuration for the controller.
Defaults to `config-controller.json` and is required for operation.
See the `config` command for inspecting and modifying the configuration.
*`--create-config` If the config file is missing, create a new one and carry on instead of exiting with an error.
Alternative to the `config create` command.
### `plugin <command>`
Configure plugins available to be loaded by the controller.
The available plugins will be loaded unless they have been disabled in the configuration, see the config command for disabling plugins.
#### `plugin install <package>`
Install a plugin from npm and add it to the plugin list.
The package is passed to `npm install --save` so anything npm accepts works, including a version like `@clusterio/plugin-subspace_storage@2.0.0`.
This command refuses to run outside the directory Clusterio was installed in, as the plugin has to end up in that directory's node_modules to be found.
For example, installing the Subspace Storage plugin:
Since the `plugin-list.json` is shared between controller, host and ctl you usually only need to do this once per machine.
#### `plugin add <path>`
Add plugin either by require path or relative/absolute path to the plugin directory.
A relative path must start with ./ or ../ (or .\ and ..\ on Windows) otherwise it will be assumed to be a require path for an installed package in node_modules.
This is mainly useful for plugins under development, packages published on npm are better installed with `plugin install`.
For example, adding a plugin checked out in the external_plugins directory:
Since the `plugin-list.json` is shared between controller, host and ctl you usually only need to do this once per machine.
#### `plugin list`
Lists the plugins set up to be available by name followed by path.
### `config`
Manage the controller configuration offline.
This should only be used when the controller is stopped, otherwise the config read might be out of date and config changes will be overwritten when the controller shuts down.
#### `config create`
Create a new config file with the default options inserted into if it doesn't already exist.
See also the `--create-config` option.
#### `config set <config-entry> [value]`
Set a config entry to the given value.
If value is not provided the entry is set to null.
If the config entry is of type object the value must be a valid JSON serialization of an object.
See docs/configuration.md in the main repositiory for the available configuration entries.
#### `config show <config-entry>`
Shows the value for a single config entry.
#### `config list`
Lists up all configuration entries with their currently configured values.
### `bootstrap`
Commands for setting up initial admin acount and access to the cluster.
#### `bootstrap create-admin <name>`
Creates a new user account with the given name if it does not already exist in the cluster and grant it full cluster admin permission.
You should be using your own Factorio multiplayer username here.
#### `bootstrap generate-user-token <name>`
Generate an access token for logging in and possibly managing the cluster from the given user account.
This can be used both in the web interface and as the `controller_token` to connect with clusterioctl.
#### `bootstrap create-ctl-config <name>`
Creates a clusterioctl config for the given user with url and token set up for connecting to the cluster.
### `copy-static [target]`
Copies the static web assets to the given target folder (which defaults to `./static/`).
Useful in case you want to host the assets on a CDN or separate web server and want provide the files directly instead of fetching them from the controller.
Note that mod pack exports will create additional static files that also have to be hosted, these are written to the `./static/` folder in the clusterio installation directory.
See also the `controller.public_url` config option for setting where static assets are loaded from.
### `run`
Runs the controller.
## See Also
[The Clusterio repository](https://github.com/clusterio/clusterio) for instructions on how to set up a cluster.
| `--public-address <address>` | host | DNS name or IP address players connect to this computer with. |
| `--factorio-dir <path>` | host | Path to the Factorio installation. |
| `--self-signed-cert <path>` | host | Certificate to trust for a controller using a self-signed certificate. It is added to the run scripts. |
| `--remote-npm`, `--no-remote-npm` | controller, host | Allow or deny remote updates, plugin updates and plugin installs from the web interface. |
| `--plugins <package...>` | all | Plugins to install, pass `--plugins` without packages to install none. |
| `--plugin-template <template...>`, `--plugin-name <name>` | plugin-template | Templates to use and name of the new plugin. |
| `--log-level <level>` | all | Log level to print, defaults to info. |
| `--allow-install-as-root` | all | Linux only, allow installing as root (not recommended). |
| `--migrate-rename` | | Migrate an installation from before the slave/master rename. |
Options for the `controller` and `host` modes also work in the `standalone` mode, except for `--controller-url` as the host connects to the controller on the same computer.
`--host-name` defaults to `local` in the `standalone` mode.
## Setting up a host
A new host has to be approved before it can join the cluster:
1. Run the installer with the host mode on the computer for the host.
It asks for a name for the host, the controller URL, the public address, the Factorio installation, whether to allow remote updates and which plugins to install.
2. Start the host with `run-host.cmd`, `run-host.sh` or the systemd service in the `systemd` folder.
The first time it starts it prints a code and waits for the host to be approved:
Waiting for approval of "my-host" with code ABC-DEF, open this link to approve it:
https://example.com/hosts?approve=ABC-DEF
Or run: npx clusterioctl host approve ABC-DEF
3. Open the link to approve the request in the web interface, where its name can be changed first.
It can also be approved under Add host on the Hosts page, or with `npx clusterioctl host approve ABC-DEF`.
Only approve a request if its code matches the one shown by the host.
Approving requires the `core.host.approve` permission.
4. The host saves `host.id`, `host.name` and `host.controller_token` to `config-host.json` and starts up.
Later starts connect to the cluster straight away.
The host keeps waiting until the request is approved, also if the controller is restarted or can't be reached for a while.
Press Ctrl+C to stop waiting.
If the request is rejected the host exits with code 8, which the run scripts do not restart on.
Start the host with `npx clusteriohost run --user-token <token>` and the token of a user that has the `core.host.approve` permission to approve the host without waiting.
The token is only used for approving the host and is not saved.
If the controller uses a self-signed certificate pass it with `--self-signed-cert <path>`, see [setting up TLS](https://github.com/clusterio/clusterio/blob/master/docs/setting-up-tls.md#creating-and-using-a-self-signed-certificate).
The installer adds it to the run scripts.
When a host is revoked in the web interface or with `npx clusterioctl host revoke <host>` its token stops working and a running host shuts down.
Starting it again requests to join the cluster again, and the host gets a new id once approved.
To move a host to another controller, change `host.controller_url` with `npx clusteriohost config set host.controller_url <url>`.
The new controller does not accept the token of the host, so the host requests to join it the next time it starts.
## Docker
To run a host in a container install it when building the image with all answers given as options.
The host requests to join the cluster the first time the container starts.
```dockerfile
FROMnode:22
# Install the Factorio headless server to /opt/factorio here
RUN mkdir /clusterio && chown node:node /clusterio
Loaded 100 of 774 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Daniel
phidias
