Compare commits

..
Author SHA1 Message Date
Hornwitser 464d588b5e v2.0.0-alpha.18 2024-04-28 19:14:27 +02:00
Hornwitser 0ddc1899fa Merge pull request #616 from Hornwitser/fix-mod-settings-types
Handle bad mod setting types
2024-04-28 19:12:22 +02:00
Hornwitser 4e1116ae2d Gracefully handle bad mod setting types in Web UI
Make sure all cobinations of missmatched types and missing mod settings
are gracefully handled by the Web UI.

Fixes #609.
2024-04-28 16:59:03 +02:00
Hornwitser 3cf9f96233 Coerce mod settings types during data export
Use the settings prototypes to coerce any incorrectly typed mod settings
to the correct type either by casting or by replacing the value with the
default value from the prototype.

This will fix cases where an update to a mod changes the type of ono of
its defined settings.
2024-04-28 16:52:33 +02:00
Cooldude2606 c37fe0f438 Merge pull request #613 from Danielv123/Fix-patch-with-missing-file
Correctly handle module files missing from a previously patched save
2024-04-27 22:12:35 +01:00
Danielv123 8831b33c01 Filter out missing files on load 2024-04-27 22:09:16 +02:00
Hornwitser 56124574fb Add changelog for #614 2024-04-27 19:58:00 +02:00
Hornwitser 9b416583f0 Add changelog for #570 2024-04-27 19:57:28 +02:00
Hornwitser b885e37b5b Add changelog for #606 2024-04-27 19:39:57 +02:00
Hornwitser 7e7e763517 Add changelog for #600 2024-04-27 19:38:21 +02:00
Daniel 811c4f7127 Use shell:true when running non-user input .cmd files on windows (#614)
Required after CVE fix: https://nodejs.org/en/blog/vulnerability/april-2024-security-releases-2/
Resolves #612
2024-04-23 22:19:30 +01:00
Danielv123 f0fb471ea1 Correctly handle module files missing from a previously patched save
Resolves #611
2024-04-23 21:57:55 +02:00
Hornwitser 79c1a2dc99 Fix plugin hooks called before host removal (#607)
Due a bad order of operations when a host connection was closed the
event handler listening on "close" to remove the host from the
hostConnections mapping was added after the HostConnection's event
handler listening on "close" and invoking plugin hooks in response to
the host going offline.  This in turn causes plugins that send
broadcasts in repsonse to thos hooks to throw an error due to the
broadcast sending code assuming connections that exist are valid target
to send messages to.

Fix by ordering the listener for the removal of the host connection
before the listener invoking plugin events.
2024-03-10 00:43:38 +01:00
Cooldude2606 a8ce8e7cc0 Merge pull request #570 from Cooldude2606/feature/plugin-template
Add command to create a new plugin from a template
2024-03-08 23:05:40 +00:00
Cooldude2606 50687515fa Merge branch 'feature/plugin-template' of https://github.com/Cooldude2606/clusterio into feature/plugin-template 2024-03-08 22:00:55 +00:00
Cooldude2606 6f81216e7c Addressed remaining review comments 2024-03-08 22:00:38 +00:00
Cooldude2606andHornwitser 3423e0a30c Apply suggestions from code review
- Copy paste errors.
- Incorrect capitalisation.
- Incorrect exports.
- Missed find and replace.

Co-authored-by: Hornwitser <github@hornwitser.no>
2024-03-08 20:08:16 +00:00
Cooldude2606 8fbdb5dfab Merge branch 'master' into feature/plugin-template 2024-03-06 20:12:08 +00:00
Hornwitser 7625fd82e3 Cache ModInfo to reduce startup time (#606)
Store and use a cache of the loaded ModInfo data to mod-info-cache.json
in the mods directory during startup.  For simplicity this does not
cache mods added to the controller or host after startup, these get
added to the cache on the next startup.
2024-03-06 17:30:45 +00:00
Cooldude2606 5dcc66034b Fix possible undefined title error (#605) 2024-03-06 18:11:45 +01:00
Cooldude2606 a66b1af454 Extra touch ups 2024-03-05 00:56:01 +00:00
Cooldude2606 3adfcf0a29 Updated web content permission 2024-02-28 16:36:35 +00:00
Cooldude2606andHornwitser 9a4241e4cd Update flags for output file
Co-authored-by: Hornwitser <github@hornwitser.no>
2024-02-28 16:26:53 +00:00
Cooldude2606 8dcb2197d6 Use try catch for output file existing 2024-02-28 15:25:08 +00:00
Cooldude2606 a168f7db5e Add call to action for config desc 2024-02-27 23:57:13 +00:00
Cooldude2606 283aba70df Update descriptions 2024-02-27 23:53:20 +00:00
Cooldude2606 62abfa903b Change to new preprocessor style 2024-02-27 23:07:38 +00:00
Danielv123 b7fec4c700 Set codecov to informational 2024-02-27 12:09:15 -06:00
Cooldude2606 ec19d5456e Merge branch 'master' into feature/plugin-template 2024-02-27 13:07:08 +00:00
Danielv123 e5d82dcfde Don't fail CI due to project level codecov regressions 2024-02-26 22:34:41 -06:00
Daniel a68bb9c0a8 Increase libLink event emitter leak warning limit (#600) 2024-02-25 23:50:14 +01:00
Hornwitser 2075814ed1 Add changelog for #598 2024-02-23 18:09:09 +01:00
Hornwitser f07ee22969 Add changelog for #556 2024-02-23 18:06:02 +01:00
Cooldude2606 a70490648f Fix total db size calculation for inv sync (#598) 2024-02-23 16:53:15 +00:00
Hornwitser e3abf8ba33 Merge pull request #556 from Cooldude2606/feature/http-logging
Add logging for express requests and errors
2024-02-23 17:35:06 +01:00
Cooldude2606 46309ce5b8 Lint Fixes 2024-02-18 17:58:16 +00:00
Cooldude2606 f392b258f7 Add subscriptions and luacheck 2024-02-17 03:11:54 +00:00
Cooldude2606 1afee2ede5 Merge branch 'master' into feature/http-logging 2024-02-14 22:46:19 +00:00
Cooldude2606 0b2fcdcec4 Merged redundant middleware 2024-02-14 22:37:48 +00:00
DanielandHornwitser 735faf4b2c Apply suggestions from review
Co-authored-by: Hornwitser <github@hornwitser.no>
2024-02-14 09:26:13 +01:00
Danielv123 d2775e4db9 Update message section of writing-plugins document 2024-02-14 09:26:13 +01:00
Hornwitser b3fc278b2c Add template to CHANGELOG.md
Document the format and expected sections of entries as a template in
the changelog so that it's easier to add new log entries.
2024-02-12 18:21:46 +01:00
Hornwitser 5f85c0575e v2.0.0-alpha.17 2024-02-12 17:54:48 +01:00
Hornwitser cf2b3db20d Add thank you note to contributors of Alpha 17 2024-02-12 17:53:16 +01:00
Hornwitser a9c6430531 Document current publishing workflow
Docement the adaptations I've made to the publishing procedure which
uses a less dangerous publish command and removes the dist folders
before publishing to avoid accidental upload of stale build files.
2024-02-12 17:53:13 +01:00
Hornwitser 8a580a66be Add breaking change note for 535e43b 2024-02-12 17:37:27 +01:00
Hornwitser 8ae5e11021 Add changelog for #587 2024-02-12 17:33:13 +01:00
Hornwitser fac04adc6f Add known issue about the base mod 2024-02-12 17:29:51 +01:00
Hornwitser d8f1d8a3e2 Add known issue about Factorio migrations 2024-02-12 17:25:10 +01:00
Hornwitser abcbf3fddc Add pagination to saves list (#592)
Prevent the saves list from pushing the console and config off the
screen by turning on the pagination for it.
2024-02-12 17:24:00 +01:00
Cooldude2606 e5a4d51c86 Merge pull request #590 from Hornwitser/bugfixes-3
Bugfixes 3
2024-02-11 23:01:43 +00:00
Hornwitser 97e615fc21 Merge pull request #584 from Hornwitser/build-rework
Build rework
2024-02-11 22:14:35 +01:00
HornwitserandCooldude2606 e990134b70 Use suggested wording in docs/contributing.md
Co-authored-by: Cooldude2606 <25043174+Cooldude2606@users.noreply.github.com>
2024-02-11 21:40:32 +01:00
Hornwitser c2af8a2377 Merge pull request #587 from Danielv123/make-resource-references-clickable
Make resource references clickable
2024-02-11 16:58:50 +01:00
Hornwitser c9609259db Ignore error loading mod in fetchMod
If a bad download or file transfer causes a mod in a host's mod folder
to become corrupt the attempt to load the mod will throw an error that's
not the file not existing.  This in turn causes starting the instance to
always fail as the mod is never attempted to be re-downloaded.

Fix by ignoring errors attempting to load mods already present in the
mods folder that aren't in the mod store so that a new download is
attempted after that fails.
2024-02-11 15:48:45 +01:00
Danielv123 3cddd2b9f8 Export Link component for use by plugins 2024-02-10 16:33:22 +03:00
Danielv123 8879f8dd80 Replaced react Link with custom component
The custom component merges the behaviour of antd Typography.Link and the react-router.dom Link after the custom component prop was removed in v6 according to this guide: https://reactrouter.com/en/main/upgrading/v5#remove-link-component-prop
2024-02-09 09:11:53 +03:00
Danielv123 4aca5050b9 Allow sending instance management messages from controller plugins 2024-02-09 07:00:05 +03:00
Danielv123 89919f7a89 Fix inconsistent indentation 2024-02-09 06:49:17 +03:00
Danielv123 e2da9ac03d Make role tag in users table clickable
As per issue #572
2024-02-09 06:30:43 +03:00
Hornwitser e8d9eb6916 Convert prototype icons to array during export
It's possible to set the .icons to an arbitrary key value mapping
instead of an array and the game will still accept it.  Convert the
icons to an array if it's an object mapping to handle this case.

Note that this doesn't preserve the order the game reads the keys in
because JavaScript reorders numeric key indexes on objects differently
to how the game orders them.
2024-02-08 21:18:48 +01:00
Hornwitser 8865509011 Delete player statistics when deleting instances
Clear the statistics stored per user for the instance when an instance
is deleted.
2024-02-08 19:06:21 +01:00
Danielv123 bcda88710e Make host name in instance list clickable
As per issue #572
2024-02-08 20:50:14 +03:00
Danielv123 d6b0e88dda Make host name on instance page clickable 2024-02-08 20:49:33 +03:00
Hornwitser 1021e29c98 Support shorthand notation for colours
Implement the shorthand notations that Factorio recognises for colours
where individual components may be left out or the colour specified as
an array of values instead of an object.
2024-02-08 18:42:38 +01:00
Hornwitser 96082e32b7 Replace deprecated usage of Select bordered 2024-02-08 15:25:06 +01:00
Hornwitser f70b4f74b1 Add watch script to rebuild on changes 2024-02-08 05:23:48 +01:00
Hornwitser ebb93ada94 Replace babel and ts-loader with swc
Build and minify the Web UI using swc instead of babel, tsc and terser.
This results in a significant speedup of the build process as it skips
checking types and uses native code to do the minification.

To prevent accidental type errors being introduced the TypeScript
compiler is still used to perform type checking on the Web UI files.

Due to peer dependency resolving shenanigans the swc/core package had to
be added to the workspace, controller and web_ui to avoid pnpm creating
a duplicate install of webpack that break the build.
2024-02-08 04:57:19 +01:00
Hornwitser 597d080d71 Add tsconfig.json that references both builds
Add back unqualified tsconfig.json files to each package that references
both the Node.js build and the browser build.  This makes tsc --build
compile both builds when the variant to do is not specified.
2024-02-08 03:20:26 +01:00
Hornwitser e4f90b3480 Move tsconfig.json to tsconfig.node.json
Distinguish the Node.js build in the same way the browser build is with
the .node suffix to the tsconfig files it uses.  This means
tsconfig.node outputs to dist/node and tsconfig.browser outputs to
dist/browser.
2024-02-08 03:20:26 +01:00
Hornwitser 8c4d2db192 Move TypeScript Node.js build to dist/node
Use a separate folder for the TypeScript build for Node.js in the dist
folder.  This makes each type of build consistently have its own output
folder instead of being interleaved with the Node.js build.
2024-02-08 03:20:26 +01:00
Hornwitser 152eb7d454 Move TypeScript browser build to dist/browser
Use dist/browser as the output of the browser oriented TypeScript build
of packages instead of dist/web/build.  This distinguishes it better
from the Web UI bundle which is put together with webpack.
2024-02-07 18:35:05 +01:00
Hornwitser 8e5d581b9b Move tsconfig.web.json to tsconfig.browser.json
Rename the config files for the browser build of modules to better
communicate that it's purpose is to compile code for a browser
environment rather than the Web UI in particular.  This also aligns with
the "browser" option in package.json, the existing browser.ts entrypoint
in lib, and how the browser environment is typically referred to.
2024-02-07 17:42:39 +01:00
Hornwitser 99990c7a0d Fix incorrect header level in CHANGELOG.md 2024-02-07 00:05:13 +01:00
Hornwitser ffd61b961d Merge pull request #583 from Hornwitser/bugfixes-2
Bugfixes 2
2024-02-07 00:03:05 +01:00
Hornwitser dcf216ca09 Fix research sync applying progress to wrong level
When the level of a technology changes either by setting level
directly or by setting researched to true, the saved progress of the
research is reset only if the technology is not the current research.

If during the initial synchronisation with the controller an instance
has progressed on a multi level research, it's the current research, and
the controller's level of that research is higher, then the level is set
to the higher level and the game keeps the progress.  This in turn
results in all of the progress made on the previous level being added to
the higher level.

Fix by setting the progress to zero when changing the level or
researched status of a technology that's the active research.
2024-02-06 22:43:04 +01:00
Hornwitser 6bf2bf51f2 Remove deprecated showArrow prop
Ant design has deprecated the Select's showArrow prop.  It's now always
shown by default.
2024-02-06 22:43:04 +01:00
Hornwitser 44b0a8ff83 Throw error on bad permission check in Wed UI
Throw an error if an attempt is made to check for a permission that does
not exist in the Web UI to prevent future bad permission checks going
unnoticed.
2024-02-06 22:43:04 +01:00
Hornwitser d3f8c993b1 Fix bad permission checks in Web UI
Fix attempts to check for permissions that don't exist in the Web UI
that has gone unnoticed because the check still succeeds when the
account has the cluster admin permission.
2024-02-06 22:43:04 +01:00
Hornwitser 535e43b527 Rename save.list.subscribe permission
Follow the same pattern as all the other subscribable lists by having a
separate .list and .subscribe permissions for saves.  This also fixes
the save list not being visible due to a typo in the permission in the
Web UI.
2024-02-06 22:43:04 +01:00
Hornwitser 2df67e32da Set key for top level page fragment
Without a unique per page key set react will try to reuse the existing
state and components of the previous page and update them to match the
new page.  This causes unexpected behaviour as components get turned
into into a Frankenstein abomination of the previous page and the
current one fused into one.

Fix by setting a key on the top level fragment of the page which in turn
forces React to re-render the whole page whenever the user switches to
another page.
2024-02-06 22:43:04 +01:00
Hornwitser 6b85567bc6 Provide theme to top level ErrorBoundary
If the theme is not provided then the App's ErrorBoundary will render
white text on white background, and vice versa for black.
2024-02-06 22:43:04 +01:00
Hornwitser 2e25831b4d Warn on subscription updates with same timestamp
If the value in a subscribed list is updated while the updatedAtMs
timestamp is kept the same then the caching logic of the subscription
system will cause updates to be silently dropped.  Log a warning if a
subscriber catches this happening to help tracking down such cases.
2024-02-06 22:43:04 +01:00
Hornwitser 021af793fc Handle instance status updates with port change
If an instance is reassigned its port may change without the status
changing, these updates should not be ignored.
2024-02-06 22:43:04 +01:00
Hornwitser 7ba14f7433 Fix instance details not updating on reassigmnent
Always send the instance details update when an instance is reassigned,
not just when it is unassigned.  The idea for not sending an update when
assigned to a host was that the host would send an update on the status
that in turn updates the instance details.  However this is not actually
the case; when a stopped instance is reassigned its status does not
change so the update is ignored.
2024-02-06 22:43:04 +01:00
Hornwitser 40f5a6ab7c Fix timestamp not set on clearing instance saves
Set the save's updatedAtMs timestamp when clearing the save list of a
reassigned or deleted instance.  This fixes the Web UI not updating the
saves list to remove the saves when an instance is reassigned.
2024-02-06 22:43:04 +01:00
Hornwitser 48c31b4825 Fix unhandled rejection on disconnect (#582)
If a link is closed while the prepareDisconnect action is in progress
the link will still try to signal the disconnect ready over the socket
when the prepareDisconnect action completes, which throws due to the
socket no longer being present.  This error is not caught due to the
catch happening before the finally on the promise and triggers an
unhandled rejection.

Fix by rearranging the finally to come before the catch on the promise
so that errors are always handled, and checking that the link still has
a session before attempting to send the disconnect ready signal.
2024-02-05 23:36:43 +01:00
Hornwitser 520007c8d1 Fix unhandled rejection on RCON lost (#580)
When using .finally on a promise a new promise is created which calls
the handler passed to it when the the original promise is resolved or
rejected, and resolves or rejects with same value as the original
promise.  This means if the rejection of the new promise that was
created by finally is not handled then an unhandled promise rejection
will be raised.

Fix by making sure the promise rejection from the finally promise
created in subspace_storage's handleUpdateStorageEvent is properly
handled.
2024-02-05 15:51:15 +01:00
Hornwitser c2b6a6e1c0 v2.0.0-alpha.16 2024-01-26 13:04:07 +01:00
Hornwitser f5255d6f76 Fix message seq reuse on forwarding (#579)
When messages were forwarded they were sent using the low level
connector .send method which accepts a message and sends it over the
link.  This method sends the message as is with no modification which
means that at no point during the forwarding logic was the message
sequence number that originated from Link A renumbered to the correct
sequence number to send out on Link B which it was forwarded on.

This in turn caused the logic which drops acked messages from the send
buffer to break which in turn meant the resume could cause duplication
of message and/or messages to be lost.  With enough duplicated messages
the resume of one connection could cause other connections to drop and
also initiate resume.

Fix by renumbering the sequence on forwarded messages.  In the future
the sequences sent and received should also be checked to make sure this
kind of problem does not happen again, this is not done here because
partially updating a running cluster with sequence problems to one that
checks and enforces sequences numbers will be problematic.
2024-01-26 13:02:26 +01:00
Hornwitser 7d6eac6558 v2.0.0-alpha.15 2024-01-24 04:40:21 +01:00
Hornwitser 6732397061 Fix Web UI not clearing config props (#578) 2024-01-24 04:35:24 +01:00
Cooldude2606 2020024b68 Merge pull request #577 from Hornwitser/small-things-15
Small things 15
2024-01-24 03:18:07 +00:00
Hornwitser 499e973db9 Fix InstanceConfigSetPropRequest schema
Fix the value incorrectly being set as required in the schema and thus
preventing props on config field objects of the instance from being
removed.
2024-01-24 03:43:21 +01:00
Hornwitser e3cfc090d3 Fix string format of Address
When host and instance id in Address was reordered in 1e66f11 the
toString method Address was updated to account for this.  Fix toString
showing host addresses as being instance addresses and vice versa.
2024-01-24 03:43:21 +01:00
Hornwitser 38dcff2225 Use h: as short form of host in log lines
Before the rename from hosts were called slave which is why s: made
sense. Use h: instead.
2024-01-24 03:43:21 +01:00
Cooldude2606 a1109b9354 Merge pull request #576 from Hornwitser/fix-plugin-broadcast
Fix plugin broadcasts not being filtered
2024-01-24 02:42:30 +00:00
Hornwitser 2a57ddb02e Add note about send and bcast test functions 2024-01-24 03:08:24 +01:00
Hornwitser 847945cfdd Fix plugin broadcasts not being filtered
When a plugin broadcasts an event to instances there may be hosts and
instances in the cluster which does not have the plugin loaded.  This
would cause an error to be logged for every instance which does not have
the plugin loaded.

Filter out broadcast events sent to hosts and instances such that
instances and hosts that does not have the event do not received them.
2024-01-24 01:56:01 +01:00
Cooldude2606 58231cc02c Add IPC to templates and handle helper function server 2024-01-24 00:40:22 +00:00
Cooldude2606 ab88955111 Merge pull request #574 from Hornwitser/fix-auto-start
Fix instance auto start
Fix host hang on stopping during create save
2024-01-23 18:41:44 +00:00
Hornwitser fc5701f243 Fix instance auto start
Bypass the Link layer and invoke the handleInstanceStartRequest method
directly on the Instance object during the host's instance auto start
phase.
2024-01-23 16:23:09 +01:00
Hornwitser 256b455315 Fix host hang on stopping during create save
If during the startup of an instance the Factorio server is started up
to create a new save and the host shutdown in initiated then the
instance will be commanded to stop while the FactorioServer state is
"create".  This was was not handled by Instance.stop leaving the
child_process running after the host is done shutting down.

Kill the FactorioServer on stop if the current state of it is stopping
or create to make sure the server actually stops.
2024-01-23 16:18:21 +01:00
Cooldude2606 2916dcc58c Fix hang from rl.close on windows 2024-01-23 02:47:10 +00:00
Cooldude2606 b9a2d6b7bd Add current folder as default for plugin name 2024-01-23 01:00:04 +00:00
Cooldude2606 224a451bbd Fix lint for jsx template 2024-01-23 00:34:46 +00:00
Cooldude2606 b222f6dcb1 Update plugin creation docs 2024-01-23 00:24:54 +00:00
Cooldude2606 9b0293501f Add js templates and config templates 2024-01-23 00:14:08 +00:00
Cooldude2606 2733f2e3c5 Merge pull request #571 from Hornwitser/link-bugfixes
Link bugfixes
2024-01-22 21:57:50 +00:00
Hornwitser 96bd5fb1c2 Handle virtual links in Link._clearPendingRequests
When a request is forwarded from a virtual link to a WebSocket link as
is the case when an instance sends a request to the controller the
origin for the request is a virtual link.  Handle this case in
_clearPendingRequests instead of throwing an assertion error.
2024-01-22 20:48:01 +01:00
Hornwitser 123148e277 Fix pending request not deleted on response
Fix memory leak due to the recorded _pendingRequests entry for a request
not being deleted when the response for the request has been received.
2024-01-22 20:48:01 +01:00
Hornwitser d7edae99c5 Add metrics for pending request
Track the number of requests currently awaiting a response from a
request handler both from interally issued requests and requests
forwarded from other parties.
2024-01-22 20:48:01 +01:00
Cooldude2606 de00a043c3 Update lint rules for templates 2024-01-22 19:09:48 +00:00
Cooldude2606 396e2c59c7 Revert accidental edit to vscode settings 2024-01-22 18:55:34 +00:00
Cooldude2606 c3b2c846b7 Add package.json template 2024-01-22 18:42:37 +00:00
Cooldude2606 be8bba8ff8 Complete template files 2024-01-22 05:01:09 +00:00
Cooldude2606 c81d2b61a6 Add template copying 2024-01-22 02:47:34 +00:00
Cooldude2606 534dc14221 Add create command mode/args for plugin templates 2024-01-21 23:36:27 +00:00
Hornwitser 51fe6c9519 Merge pull request #569 from Hornwitser/bugfixes
Bugfixes for Alpha 14
2024-01-22 00:35:28 +01:00
Hornwitser e0161114e7 Fix --no-download-headless conflict in installer
Fix installer incorrectly exiting with a conflicting options errors when
the --no-download-headless option was used in conjunction with the
--factorio-dir option.
2024-01-21 23:49:19 +01:00
Hornwitser 2872faab98 Skip loading plugins without entrypoint on hosts
If a plugin only has entrypoints at the controller level the config will
skip adding a "load_plugin" config option for the given plugin to avoid
making options that do nothing.  If such a plugin is added to a host it
will still reach loadPlugin which in turn tries to check the load_plugin
config and throw an error on startup due the field being missing.

Skip trying to load plugins without an entrypoint at the instance or
host level in hosts before checking the load_plugin config field.
2024-01-21 23:49:19 +01:00
Hornwitser 1d1e80fbf0 Export Control and related types from ctl 2024-01-21 23:49:19 +01:00
Hornwitser 3d7c11a8b6 Add missing enable source maps for host service 2024-01-21 23:49:19 +01:00
Hornwitser 72264f0c53 Add suport for color mod setting (#568)
Implement the color-setting type in ModPack and display a color picker
for it in the Web UI. This type of setting was added in Factorio 1.1.77.
2024-01-21 23:48:26 +01:00
Hornwitser d6d6c052aa Add known issue about being kicked out of the Web UI 2024-01-21 02:17:12 +01:00
Hornwitser 901ce4e08e Add broken instance data export to known issues 2024-01-21 01:01:08 +01:00
Hornwitser 5fdc585212 Add known issues section
Some problems were not caught during testing and is not straight forward
to fix.  Add them to the known issues section for now.
2024-01-20 22:48:20 +01:00
Hornwitser 0aa330561b Update migration guide
Add a note about double checking which version of Node.js is installed
to avoid accidentally updating on an outdated version, and fix migrate
arguments.
2024-01-20 22:44:28 +01:00
Hornwitser 1be1b029a0 2.0.0-alpha.14 2024-01-20 21:58:42 +01:00
Hornwitser cafafbd540 Fix handleFetchPlayerCodeRequest returning incorrect properties 2024-01-20 21:58:42 +01:00
Hornwitser 5ae19042f1 Fix capacity typo in metrics 2024-01-20 20:37:13 +01:00
HornwitserandCooldude2606 90969bd90b Add changelog for 2.0.0-alpha.14 (#562)
Add changelog and migration guide for alpha 14.

Co-authored-by: Cooldude2606 <25043174+Cooldude2606@users.noreply.github.com>
2024-01-20 20:33:45 +01:00
Hornwitser c53781d16c Add time unit suffixes to variables (#561)
Add suffixes like Ms and Ns to variables whos time unit is not in
seconds.  This is aligning with a new code style of putting units
on variables that are not in SI base units.
2024-01-20 13:26:55 +01:00
Hornwitser 9842705eb9 Improve error reporting of plugins in the Web UI (#557)
Keep the error message and display it in the plugin page when loading
the plugin info or the plugin itself fails to load in the Web UI.
2024-01-20 05:20:24 +01:00
HornwitserandCooldude2606 f702c956a5 Document migration of plugins to alpha 14 (#558)
Co-authored-by: Cooldude2606 <25043174+Cooldude2606@users.noreply.github.com>
2024-01-20 05:13:55 +01:00
Hornwitser ca58e2f0a6 Merge pull request #560 from Hornwitser/small-things-14
Small things 14
2024-01-20 04:36:09 +01:00
Hornwitser c3c2528c30 Add missing updates to instance.updatedAt
Fix instances not properly updating their status to unassigned and
unknown due to the updatedAt field not being updated with the current
timestamp.
2024-01-20 02:44:55 +01:00
Hornwitser 1e66f11917 Reorder members of Address
The logical order of clusterio's components are in most places
controller, host, instance, control.  It is confusing that Address
breaks this convention and I do not remember I wrote it that way.
Reorder and renumber the host and instance members of Address to
be consistent with most other places.
2024-01-20 02:19:47 +01:00
Hornwitser bb1401a683 Update jsonwebtoken used by player_auth
This was missed during 1a9cbd2 which updated jsonwebtoken in other
packages.
2024-01-20 02:03:23 +01:00
Hornwitser 96f48ca102 Fix installer breaking on setting up plugins
The entrypoint of plugins were changed in 50f12c7 and b0f6c95 but the
installer's loading of plugin info to get the correct plugin name was
not updated for these changes.  Update installer plugin loading to use
the correct entrypoint.
2024-01-20 01:50:42 +01:00
Hornwitser 671ff23085 Write new run scripts during migration
If the old run-master or run-slave scripts are present write the new
run-controller and/or run-host scripts.
2024-01-20 01:30:24 +01:00
Hornwitser 5e2221796a Rename sharedMods to mods on migration
The place for storing mods was changed from sharedMods to mods when mod
management was implemented.  Rename the old sharedMods folder to mods
when the migration script is invoked.
2024-01-20 01:22:29 +01:00
Hornwitser adc2d9cb4d Fix HostInfo.name description 2024-01-19 19:17:16 +01:00
Hornwitser d5626d31e6 Fix rendering of ModDetails 2024-01-19 19:17:16 +01:00
Hornwitser b0f6c95fca Move export for plugin declaration (#555)
Change the expected exported name of plugin declarations to be "plugin"
at the main entrypoint instead of the default export.  This makes the
plugin's code organisation more flexible especially when using CommonJS
JavaScript modules as they can export additional symbols from the
plugin declaration file.
2024-01-18 20:31:52 +01:00
Cooldude2606 fcf174376d Clean up the request correctly 2024-01-18 18:50:31 +00:00
Cooldude2606 0e68596fbd Remove insecure and bloated module 2024-01-18 18:50:30 +00:00
Cooldude2606 9bcbc1e7f3 Add http logging for express 2024-01-18 18:50:29 +00:00
Hornwitser 8884116ee9 Implement stop and restart of controller/hosts (#554)
Implement remotely initiated stopping and restarting of the controller
and hosts.  Restarting is implemented by exiting with a non-zero exit
code and relying on a process manager to start the procress again.  To
prevent accidentally trying to restart on a system that doesn't have a
process manager present the --can-restart command line option needs to
be passed to enable the restart operation.
2024-01-18 16:10:42 +01:00
Lars Corbijn 885c18c0d4 Use a relative path for commands in create (#553)
The create.js script in development mode used to use hard coded paths to
the other cluster commands (ctl, controller etc.). This means that in
dev mode a cluster could only be created from the root directory of the
project.

By using relative paths one can also create a cluster in a subdirectory.
For example by using the following:

mkdir test-cluster
cd test-cluster
node ../packages/create --dev
2024-01-17 21:44:08 +01:00
Hornwitser 527c98d378 Add first join at player statistic (#551)
Record the time a player was first seen joining an instance in
PlayerStats.  For existing stats this value is set to the last
joined time when it is first loaded.
2024-01-17 20:29:30 +01:00
Hornwitser 863d90c7fb Implement custom input for mod pack config (#547)
Add custom input component support to the Web UI config in order to
provide a user friendly dropdown selector for the Mod Pack that is
configured as default on the controller and overridden instances.
2024-01-17 20:29:04 +01:00
Hornwitser 80f0c63113 Merge pull request #552 from Hornwitser/additional-system-info
Additional system info
2024-01-17 20:28:06 +01:00
HornwitserandCooldude2606 4628e0ad59 Fix incorrect test description for SystemInfo
Co-authored-by: Cooldude2606 <25043174+Cooldude2606@users.noreply.github.com>
2024-01-17 06:46:18 +01:00
Hornwitser a786f186a2 Merge pull request #546 from Hornwitser/support-reverse-proxies
Use X-Forwarded-For header on trusted proxies
2024-01-17 05:27:45 +01:00
Hornwitser b5ed0800cf Add address host connected from to the host info
Provide an easy way to see where a host is connecting from in the web UI
and clusterioctl list.
2024-01-16 21:27:02 +01:00
Hornwitser de47989527 Add system info to SystemInfo
Provide the kernel, machine, CPU Model, hostname, and Node.js version
fields to SystemInfo and display them in the Web UI.  These were added
as metrics exposed to prometheus in b46efeb.
2024-01-16 21:27:02 +01:00
Hornwitser 4e36fa1b4e Rename SystemMetrics to SystemInfo
There are more pieces of information about a system than just its CPU,
Memory and disk metrics.  To avoid having to build a separate collection
and subscription for metrics and system info, rename the existing system
metrics to system info in anticipation of providing additonal static
system information such as CPU model in it.
2024-01-16 21:27:02 +01:00
Hornwitser 5f25aad273 Use X-Forwarded-For header on trusted proxies
If the controller is behind a proxy then the socket.remoteAddress will
be that of the proxy instead of the end client that connected.  In this
case the X-Forwarded-For header will contain the address of the end
client, but this should only be relied upon if the party connecting is a
trusted proxy, otherwise a malicious user could spoof the IP address
that is logged by providing a forged X-Forwarded-For header.

Implement a trusted proxy list consisting of a configured list of IP
addresses and/or CIDR blocks and use the last address[1] in the
X-Forwarded-For header if the remoteAddress address of the connecting
party is in the trusted proxy list.

[1]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For
2024-01-16 06:21:48 +01:00
Hornwitser c2a3d4361a Merge pull request #545 from Hornwitser/remote-host-config
Remote host config
2024-01-16 04:18:20 +01:00
Hornwitser 688ddc529b Separate connection related data from host data
Split out host specific data from RegisterHostData that is not related
to the establishment of a connection to the controller into it's own
HostInfoUpdate event.  This allows this data to be updated when the
config values they are taken from are changed.
2024-01-15 22:19:38 +01:00
Hornwitser 790f880e98 Split up BaseConfigTree component
Pull out the computation of the tree data into a separate function to
reduce the complexity of the BaseConfigTree component.
2024-01-15 20:32:56 +01:00
Hornwitser 707c0bb453 Add remote configuration of hosts
Implement remote editing of the host config in a similar fasion to how
the remote editing of the controller config works.
2024-01-15 20:28:44 +01:00
Hornwitser 7dce25423c Add non_blocking_saving to instance defaults (#544) 2024-01-15 05:04:34 +01:00
Hornwitser 705fb90740 Remove agent string
The agent string was inspired by the HTTP User-Agent header to serve as
a way to identify what's connecting to the cluster.  While there may
have been vision of other software connecting to a controller than the
Clusterio provided ones this seems highly unlikely to ever happen.

Remove redundant agent string for now.
2024-01-15 03:49:41 +01:00
Hornwitser 8704a24f26 Sort hosts in AssignInstanceModal (#542) 2024-01-15 03:36:02 +01:00
Hornwitser 6f6fe54d5c Merge pull request #541 from Hornwitser/system-metrics
Real-time metrics
2024-01-15 03:34:44 +01:00
Hornwitser d2861c4457 Add tests for derived SystemMetrics 2024-01-15 03:27:37 +01:00
Hornwitser ae2c5cf98e Implement power of two size suffies for memory
Memory sizes are usually written using the prefixes with a base of 1024
instead of 1000.  Implment formatting of the power of two based prefixes
for displaying memory sizes.
2024-01-14 09:57:44 +01:00
Hornwitser d64bf6015e Show System metrics in hosts page
Implement simple collection of basic real time system metrics of hosts
and the controller for the purpose of displaying them in the Web UI. The
metrics are collected by the controller at a configurable interval and
pushed out to web clients currently subscribed to the metrics.
2024-01-14 09:57:44 +01:00
Hornwitser 69bdc8e884 Update antd
Use the latest version of antd to get support for the hidden property on
table columns.
2024-01-14 00:42:43 +01:00
Hornwitser b46efeb4e3 Add system info, CPU, memory and disk metrics
Collect basic metrics about the system the controller and hosts run on
including CPU, memory and disk capacity and usage.
2024-01-14 00:42:43 +01:00
Hornwitser 81d6369a34 Fix incorrect merging of samples from hosts
The Prometheus library has an optimisation for metrics that expose
multiple time series such as Summary that provide two values in the
time series consisting of the metric name suffixed with _sum and _count.
These are provided as a series of sufixes with corresponding samples such
that a serialised summary metric named some_value would have a samples
mapping looking something like this:

[
    ["_sum", ["label=\"value\"", 3]],
    ["_count", ["lavel=\"value\"", 2]],
]

If multiple hosts expose the same metric then these need to be merged,
which is the job of mergeSamples, which merges the samples from each
sufix into the destination.

One corner case that may occur is if the source and destination metric
that's being merged have different lists of suffixed samples.  In this
case the suffixes that are present in the source but not in the
destination needs to be added verbatim to the destination.  The code for
handling this case incorrectly added these samples to the source metric
instead of the destination metric.

This corner case only occrs if a metric's type differs between two
hosts connected to the same controller.  For example if a Gauge metric
is turned into a Summary metric.
2024-01-14 00:42:43 +01:00
Hornwitser 32d70a83d1 Fix debug dump-ws crashing the controller
When ws was updated to version 8 it came with a breaking change where
message data was no longer utf8 decoded from binary to a string.  The
main codepath handling messages was updated to handle this but the ws
dump path was not resulting in the controller crashing when receiving
a message that is then forwarded to the dumper without decoding.

Fix by decoding the message in the dump-ws path.
2024-01-14 00:42:43 +01:00
Hornwitser 1f61e49599 Merge pull request #539 from Hornwitser/small-things-13
Small things 13
2024-01-11 02:26:49 +01:00
Hornwitser b976de926b Fix unassigning instances broken in Web UI 2024-01-11 01:21:35 +01:00
Hornwitser 880216b775 Add typing to column render methods
The render method for columns in Antd Table take the three arguments
value, record, index. If dataIndex is set then value is the property
given by dataIndex in record, otherwise value and record is the same.
Unfortunately only record has the proper type while value is the type
any.  Change usage of `render: value =>` to `render: (_, value) =>` to
get proper typing for the render methods defined on columns.
2024-01-11 01:21:35 +01:00
Hornwitser e530bbc83a Replace deprecated usage of Breadcrumb.Item 2024-01-11 01:21:35 +01:00
Hornwitser a67a4a41da Use separate build for web version of lib
Use a separate ES Module based build for the web version of the lib
package.  This fixes the rebuild on watched files changing in lib
failing with a TS1479 error.
2024-01-11 01:21:35 +01:00
Hornwitser 2bdb03b0a9 Merge pull request #538 from Hornwitser/host-port-ranges
Host port ranges
2024-01-11 00:57:01 +01:00
Hornwitser 8733575255 Assign game port from host configured port ranges
Add host.factorio_port_range config to hosts that specify a pool of
ports that gets automatically assigned to instances which do not have a
game_port set on them.  This allows setting up a port range that's
forwarded to the host on the router and then configure that port range
on the host and have instances automatically get assigned an available
port when they start up.

Resolves: #448
2024-01-11 00:19:19 +01:00
Hornwitser 9742eb1645 Add parser for simple range expression
Implement parsing of ranges of the form "1, 3-5" into a set of all the
numbers it represents.
2024-01-11 00:19:19 +01:00
Hornwitser c87493af0c Merge pull request #537 from Hornwitser/refactor-config
Refactor config
2024-01-11 00:05:44 +01:00
Hornwitser d8301f3c61 Rename RawInstanceInfo to HostInstanceUpdate
Due to configs needing a location to be deserialised it's not possible
to cleanly replace RawInstanceInfo with the InstanceInfo class.  Rename
it to HostInstanceUpdate to better represent its purpose.
2024-01-10 23:52:43 +01:00
Hornwitser c38a63692d Fix docs for initialValue
Remove reference to async function support in the doc for initialValue
that was removed in 27af142.
2024-01-10 23:37:50 +01:00
Hornwitser 1f19f1bfcb Add note about config value cast through unknown 2024-01-10 23:29:32 +01:00
Hornwitser c0f65e0610 Revert mistaken removal of await
Revert mistaken removal of await from controller plugin initialization
in 27af142.
2024-01-10 23:16:29 +01:00
Hornwitser 1985abbe6c Fix web interface not loading
The config refactor accidentally introduced a circular dependency loop
which would break the web interface from loading.
2024-01-09 02:25:36 +01:00
Hornwitser e74065a493 Remove config groups
Refactor away the config group concept and simplyfy the way config
fields are defined.  Fields are defined directly as properties of the
static field fieldDefinitions on the class itself with a separate
interface to declare the types of fields.

This makes it possible to easily make accessing the config fields type
checked in a way that lets plugins extend the interfaces with their own
config options by adding additional fields to the interface via
declation merging.
2024-01-09 02:19:07 +01:00
Hornwitser 96210373f3 Move IpcPlayerData out of inventory_sync/instance.ts
Remove circular dependency between inventory_sync/message.ts and
inventory_sync/instance.ts by moving the IpcPlayerData type from
message.ts to instance.ts.  This solves a problem with the web
build checking types in instance.ts that it dosen't use.
2024-01-08 23:54:12 +01:00
Hornwitser 7bbb44d4e6 Move TypedEventEmitter into its own file 2024-01-08 23:47:24 +01:00
Hornwitser 943c0d9cb4 Merge pull request #534 from Hornwitser/refactor-subscriptions
Refactor subscriptions
2024-01-07 02:54:20 +01:00
Hornwitser 001db4befe Refactor config to JSON data class pattern
Refactor the Config and ConfigGroup classes to match the JSON data class
pattern with toJSON, toRemote and fromJSON methods replacing the load,
init, and serialize methods.  Since fromJSON requires extra information
the messages structures passes around the plain JSON representation of
the configs instead of serialising instances of the config classes
automatically.
2024-01-06 19:53:32 +01:00
Hornwitser 27af142a87 Remove silly support for async initial_value
This complicates the handling of configs for no good reason.
2024-01-06 19:49:35 +01:00
Hornwitser ede150d80f Add roundtrip test for InstanceInfo 2024-01-06 19:32:42 +01:00
Hornwitser 699ef2c3cb Separate updatedAt from mtime in SaveDetails
If a new save is added to the saves list of an instance the current save
selected to be loaded by default changes its state without updating the
mtime of the file on disk, which in turn means basing the updatedAt
property on mtime will not work with caching.  Fix by having a separate
updatedAt property that is set by the controller whenever the save
details changes from the controller's perspective.

Noticed-By: @Cooldude2606
2024-01-06 18:52:41 +01:00
Hornwitser 1ae686b3ea Don't initalize activeModPack to an empty mod pack
Avoid possible use of an unintended empty mod pack by not setting the
activeModPack on the Instance until it's loaded in syncMods.  This was a
leftover from when a newly created empty mod pack was supposed to act as
the default when no mod pack was set.
2023-12-21 00:43:12 +01:00
Hornwitser 6d840fc1e7 Remove redundant .pop() in test 2023-12-20 22:32:12 +01:00
Hornwitser c62ea697fe Fix typos 2023-12-20 22:29:44 +01:00
Hornwitser 8c17adf15b Refactor EventSubscriber to track its resource
Process updates of the resource EventSubscriber are subscribed to and
have it keep a mapping of id to value syncronised with event provider as
long as there are subscription callbacks registered with it.  The
mapping is kept around after the last callback is unsubscribed so that
it can be efficiently resynced with the resource based on the updatedAt
timestaps of individual elements.

This allows the EventSubscriber to provide the right building blocks to
provide its values to React components via the the useSyncExternalStore
hook.  This in turn greatly simplifies the web_ui models for the
subscribed resources.
2023-12-12 23:40:29 +01:00
Hornwitser 867bb0c646 Add ModInfo.id getter
Provide a simple unique identifier for mods consisting of the name
followed by an underscore and then the version.
2023-12-12 00:08:25 +01:00
Hornwitser ddba58ca99 Require control in EventSubscriber.constructor
Reduce complexity in the implementation of EventSubscriber by requiring
a control link be passed to its constructor.  This is possible after the
change in 8c724e3 made the web interface control instance available to
plugins at init time.
2023-12-11 23:14:37 +01:00
Hornwitser 6cab1fa842 Remove SubscriptionResponse
Since subscriptions are handlede per link there's no need for the
eventReplay to be returned as a part of the SubscriptionRequest, it can
be sent and processed normally as an event.  Simplify the logic by
sending eventReplay as an event and remove SubscriptionResponse.
2023-12-11 21:47:15 +01:00
Hornwitser 4439691695 Decouple SubscriptionController from Controller
Remove the reliance a reference to the Controller to get the link that a
subscription is tied to by explicitly handling the SubscriptionRequest
and passing the link to the SubscriptionController.
2023-12-11 21:47:15 +01:00
Hornwitser 1d729adbb7 Move responsibility of handling unsubscription on close
Notify the SubscriptionController that a connection closed from the
Connection instead of having the subscription controller set up an event
to do so.  This simplifies the logic on the SubscriptionController's
side.
2023-12-11 21:47:15 +01:00
Hornwitser 0df672ecca Implement subscription handlers for resources
Send the hosts, instances, saves, mods, mod packs and users that has
updated since the lastRequestTime provided in the request.
2023-12-11 20:21:43 +01:00
Hornwitser 0c4387769e Use -1 as the default lastResponseTime
Entites that didn't have an updatedAt time at the time of creation will
default to having one set to 0.  Since entities with an updatedAt time
greater than lastResponseTime is expected to be returned this means that
these entities would not be included when a client does its initial sync
subscribing to a resource.  Change the default lastResponseTime
requested so that these entities are included.
2023-12-11 20:21:43 +01:00
Hornwitser 5748f834ff Only call subscription update function on subscribe
Skip calling the subscription function that should provide an event
replay if the client is unsubscribing from the event.  This avoids
having to check if the client is subscribing in every subscription
function.
2023-12-11 20:21:43 +01:00
Hornwitser a2914bda33 Cache save details on the controller
Keep a copy of the instance saves details on the controller so that
saves can be queried even when the host is disconnected and the
controller can properly track and provide updates on the saves to
its clients.
2023-12-11 20:21:43 +01:00
Hornwitser d186f85773 Refactor InstanceInfo
Refactor to store instance status, gamePort, and updatedAt to disk for
instances.  This enables InstanceDetails to properly record when it was
updated.
2023-12-09 17:23:24 +01:00
Hornwitser c248363895 Implement HostInfo as a class
Replace POJO definition of HostInfo with a class that properly manages
its own serialisation to JSON.
2023-12-09 17:23:24 +01:00
Hornwitser 20330bc2c3 Push multiple updates in each update event
Refactor HostUpdatesEvent, InstanceDetailsUpdatesEvent, ModUpdatesEvent,
ModPackUpdatesEvent and UserUpdatesEvent to have multiple updates of the
entities they support in each event. This is intended to make the events
capable of syncing the entire list that they represent.
2023-12-08 22:01:46 +01:00
Hornwitser f12a9926eb Add isDeleted and updatedAt to subscribable types
Prepare for subscriptinos being able to be cached and resumed by adding
updatedAt and isDeleted fields to the subscribable types.
2023-12-08 18:22:34 +01:00
Hornwitser f00995a7f9 Merge pull request #533 from Hornwitser/sync-mods-with-controller
Sync mods with controller
2023-12-04 02:19:29 +01:00
Hornwitser 58914042cc Remove channels to simplify subscription
Make subscriptions either on or off to greatly simplify the logic of
subscribing to events.
2023-12-03 20:32:48 +01:00
Hornwitser a2eb98ac93 Implement hosts downloading needed mods
Download and cache needed mods to run instance mod packs on instance
startup.  To avoid opening too many download streams at once only one
mod is downloaded at a time.
2023-12-03 20:15:07 +01:00
Hornwitser 853866492b Refactor controller mod storage into ModStore
Structure the storage of mods on the controller into into a reusable
component in preparation for the host using the same kind of mod storage
to retrieve and cache the mods it needs to run mod packs on instances.
2023-12-03 19:59:14 +01:00
Hornwitser 448cfbbb51 Log errors on more instance failures conditions
Log errors messages to the log when setting up an instance fails in the
preparation phase, creating save phase, starting up phase, and so on.
Previously these were only sent back as a an error response to the
request that iniatied the wakeup of the instance.
2023-12-03 19:59:14 +01:00
Hornwitser 1095d2466d Fix path traversal on mod upload
Fix possibility of a maliciously crafted mod being written outside of
the mods directory and thus possibily overwriting unrelated files.
2023-12-03 19:59:14 +01:00
Hornwitser 94cb607930 Improve typing of various controller entities 2023-12-03 19:59:14 +01:00
Hornwitser 8b0d7a67b5 Automatically obtain required mods in test
Build the clusterio_lib mod from source and download and copy the
subspace_storage mod used for testing if it's not present in the mods
directory.  This removes a manual step needed to run tests on a new
clone of the repository.
2023-12-03 00:05:44 +01:00
Hornwitser 94d67cef71 Add AsyncSerialCallback helper
Add helper to serialise the calls made to an async function such that no
overlapping invocations happen to the async function by queue up calls
made while it's running to continue after it's running.
2023-12-03 00:01:23 +01:00
Hornwitser 07160c2fe0 Merge pull request #532 from Hornwitser/small-things-12
Small things 12
2023-12-02 14:57:58 +01:00
Hornwitser ec734c92bc Fix bad typings detected by linting
Fix badly typed expressions that were detected by turning on
`@typescript-eslint/no-unnecessary-condition` in eslint.
2023-12-02 01:47:02 +01:00
Hornwitser 90a88f7e01 Fix /api/plugins leaking filesystem paths
Replace PluginWebApi's requirePath member with npmPackage which is only
populated if the package is not marked as private and the requirePath
matches the name. This prevents the public /api/plugins endpoint from
leaking filesystem paths into the controller.
2023-12-02 01:43:28 +01:00
Hornwitser 42ae4d6897 Fix research finished not syncing properly
Fix the research finished event not being sent to the controller, which
in turn meant it would not sync properly.
2023-12-02 01:43:28 +01:00
Hornwitser 11ffee646e Add the other packages to typedoc output
Add controller, host, ctl, and web_ui packages to the typedoc output.
2023-12-02 01:43:28 +01:00
Hornwitser e453a80155 Refactor User and Role into data classes (#531)
Use the JSON data class pattern for User, Role, and PlayerStats and move
them into lib/src/data.  This means that User is purely data now and
can't reference role objects directly.  To make it possible easily check
permissions attached to roles and store data only visible on the
controller on user objects a ControllerUser type is introduced which is
linked with the UserManager and can therefore reference role objects
directly.

Resolves #525.
2023-12-02 01:41:43 +01:00
Hornwitser f802e6ba67 Merge pull request #527 from Hornwitser/autosave-data
Implement autosaving of data held in memory
2023-12-01 02:30:54 +01:00
Hornwitser ab52cc35bf Provide controller resources at construction time
Load hosts, instances, mod packs, mods and users before constructing the
Controller instances.  This allows these resources to be correctly typed
as always available and prevents any possibility of save being initiated
on these resources before they are loaded.
2023-11-30 18:31:42 +01:00
Hornwitser a0b02ee68b Separate updates of Instance and InstanceDetails
Some updates for instances trigger instance details updates, while other
updates to instances are not visible on instance details. Rename
instanceUpdated to instanceDetailsUpdated to better represent what it
does and only set instancesDirty when an update is made to the
instances.
2023-11-30 15:01:14 +01:00
Hornwitser fae6ed3297 Implement serial calls to save with a helper
Separate out the serialisation of the invocations to saveData with an
AsyncSerialMergingCallback helper than has unit tests covering its
functionality.
2023-11-30 01:57:18 +01:00
Hornwitser a0807b22a4 Fix inventory wipe on SE remote view (#530)
Exit the Space Exploration remote view if its active to prevent the
empty remote view "character" being serialized as the inventory instead
of the real character for the player.

Original-patch-by: @Reavershark
Fixes: #513
2023-11-30 01:32:13 +01:00
Hornwitser 2c0c84b191 Add the ctl counterpart to the revoke host action. (#528)
Resolves #512.
2023-11-30 01:24:23 +01:00
Hornwitser d5bcd34cdb Use .tmp before the file extension (#529)
When creating temporary files use the pattern `.tmp.ext` instead of
`.ext.tmp` to be consistent with Factorio which creates `.tmp.zip` files
during saving.

If patching fails this will leave a `.tmp.zip` file in the instance saves
directory which is ignored by the default save selector.

Resolves #486.
2023-11-30 01:21:06 +01:00
Hornwitser ffe5040644 Implement autosaving of data held in memory
Add controller.autosave_interval config that specifies the frequency
which controller config, hosts, instances, users, mod packs and plugin
specific data which is held in memory should be automatically flushed to
disk if they have changed since the last time they were saved.  This
prevents data loss in the event of a server crash.

Plugins can use the onSaveData hook to save its own data at the same
time as the controller does and this obsoletes making an autosave
interval in the plugin.

Resolves #523.
2023-11-29 20:18:50 +01:00
Hornwitser faadf873fb Merge pull request #524 from Hornwitser/typescript-plugin-web
Port plugin web code to TypeScript
2023-11-29 20:16:24 +01:00
Hornwitser 1434ede2dc Improve error reporting when web_ui fails to load
Catch uncaught errors and display the error message on the page itself
if bootstraping fails. This should prevent most cases of the of the web
interface being stuck on "Loading web interface..." after an error was
thrown and the loading failed.
2023-11-29 19:01:46 +01:00
Hornwitser f1edbd78cb Move dist/web_build inside dest/web
By default webpack-dev-middleware will refuse to delete files outside of
its working directory. This becomes dist/web in the configuration
currently set up, which means that a rebuild triggered on the watched
files will fail when declaration files are output to to dist/web_build.

Fix by moving dist/web_build to dist/web/build.
2023-11-29 18:02:27 +01:00
Hornwitser 0a4368d1c0 Rename index.css to style.css
Make the tab completion on web/in<Tab> give the TypeScript code instead
of annoyingly stopping at the extension.
2023-11-28 15:52:54 +01:00
Hornwitser 6a404d9fb3 Refactor and port plugin web code to TypeScript
Clean up the configuration for compiling TypeScript for the Web
interface to us its own tsconfig.web.json config files and reference the
TypeScript source files directly in the configuration and imports
instead of files compiled to dist.

To properly support recompliation with TypeScript's projects feature
each package's tsconfig.web.json file is set to output the build to
dist/web_build which is then ignored in .npmignore.  Webpack is still
used to create the proper build in dist/web, so this is mainly to make
TypeScipt's compiler happy but it also allows the root tsconfig.json
file to include the web interface code, meaning running the build script
now also checks the typing of the web interface.

Final part of #491.
2023-11-28 15:52:54 +01:00
Hornwitser 8c724e3bc5 Provide control at init time for web plugins
The control interface is required for adding handlers for message types
a plugin defines, but the plugins were loaded before the Control
instance was created. This meant there was no sensible way for web
plugins to register handlers for their own message types.

Move the creating of the Control instance to before plugins are loaded
and set this.control for web plugins before init is called.
2023-11-28 15:35:40 +01:00
Hornwitser 70d2b319b3 Fix types related to plugins and the web interface 2023-11-28 15:35:40 +01:00
Hornwitser 8b5c66d900 Output compiled validator to source tree as well
Output message_validate.js to both dist/src/data and src/data. This
allows tools to directly consume the TypeScript sources in src/data
without throwing an error due to message_validate being missing.
2023-11-28 15:35:40 +01:00
Hornwitser ceb7e48c99 Fix global declaration of staticRoot and webRoot
These variables were intended to be globals declare in the global
namespace. That this ends up on the window object is an implementation
detail of JavaScript. Remove the window. prefix put in front of
staticRoot and webRoot and include the global declaration in the
TypeScript module.
2023-11-28 15:35:40 +01:00
Hornwitser 9df6717364 Move PluginWebApi to lib
Resolve circular dependency of controller depending on web_ui and web_ui
depending on controller by moving the type from controller that was used
in web_ui to lib.
2023-11-28 14:48:06 +01:00
Hornwitser 47c5a727b5 Merge pull request #522 from Hornwitser/base-plugin-move
Refactor plugin code
2023-11-28 14:47:23 +01:00
Hornwitser 654df76065 Strongly type invokeHook
Add strong typing of the invokeHook function to prevent incorrect
paramaters being passed to plugin hooks. This detected a few places
where the parameters passed were incorrect.
2023-11-28 14:28:22 +01:00
Hornwitser e3e5867d60 Fix PlayerEvent being completely broken
Fix the name being passed as the type when deserialising the event, the
stats being missing, and incorrect parameters passed to plugins.
2023-11-28 14:28:22 +01:00
Hornwitser b9c3458cc2 Fix onMetrics using the wrong iteration type 2023-11-28 14:07:38 +01:00
Hornwitser ee9c200ebf Implement host plugins
Add ability to define plugin code that runs in the context of a host,
starts up with the host does, and stops when the host shuts down.
2023-11-25 19:18:33 +01:00
Hornwitser 808eda0513 Move base plugins to their respective packages
Move BaseControllerPlugin to controller/src/BaseControllerPlugin.ts
Move BaseInstancePlugin to host/src/BaseInstancePlugin.ts
Move BaseControlPlugin to ctl/src/BaseCtlPlugin.ts
Move BaseWebPlugin to web_ui/src/BaseWebPlugin.ts

This solves the circular package dependency problem of the base plugins
needing types that are available only in the dependencies to lib.
2023-11-25 18:41:43 +01:00
Hornwitser 63dc370ea0 Merge pull request #520 from Hornwitser/vs-code-config
Add VSCode workspace config
2023-11-25 18:41:23 +01:00
Hornwitser 26827ec8a7 Configure VSCode Workspace
Add configuration for VSCode to use tabs for indent, unix style endings
and leave a newline at the last line of the file.  This will make
editing from VSCode less error-prone as it will select the right
settings by default.
2023-11-25 18:36:30 +01:00
Hornwitser 87e52b804e Convert JSON file indents to tabs
Be consistent with the usage of tabs as indents in all JSON files in the
project.  This makes it easier to configure editors to emit the right
indents when manually editing these files.
2023-11-25 18:19:15 +01:00
Hornwitser 702c323610 Merge pull request #519 from Hornwitser/small-things-11
Small things 11
2023-11-25 17:53:08 +01:00
Hornwitser d47fd20926 Fix --dev broken after 7517917 2023-11-24 21:11:56 +01:00
Hornwitser 51f7430a4b Add warning to generated server-settings.json
Leave a warning in case someone opens and tries to modify the
server-settings.json file in the instance folder, as any changes
made will be lost when it's overwritten.
2023-11-24 21:07:17 +01:00
Hornwitser fc6e9d0002 Fix type checking on Request and Event types
When defining a generic type where a type argument is never used
as a part of the type itself, TypeScript will consider two instances of
the generic type with different values for the type to be the same.
This caused the handle method to accept any handler for any
Request/Event, ignoring the event/request type completely.

Fix by specifying the constructor to return the intersection of the type
itself and the Request/Event type.
2023-11-24 20:56:02 +01:00
Hornwitser f23e442c4c Ignore BOM in locale files
Ignore BOM at the start of Factorio locale files as this is a thing
Factorio also does.

Fixes #507
2023-11-24 20:53:16 +01:00
Hornwitser 6eda1f78d5 Turn on --enable-source-maps
Turn on source map deduction in stack traces from thrown errors. This
feature is off by default due to potential performance degration in
certain corner cases.  For the dev environment add instructions for
passing the option to node.
2023-11-24 20:53:16 +01:00
Hornwitser 142cf8a626 Fix log splitting not happening on UTC boundnary 2023-11-24 20:53:16 +01:00
Hornwitser 56d3997675 Refactor to keep handle to Instance in connection
Keep a handle to the Instance in the connection the host has to the
instance so that methods can be called directly on the Instance. This
simplifies logic that exists outside of the link system like the host
shutting down all of its instances when it exits.

Fixes validation error thrown on host shutdown with running instances.
2023-11-24 20:53:16 +01:00
Hornwitser 82ed0adfc3 Merge pull request #517 from Hornwitser/setup-typescript-eslint
Setup TypeScript ESLint
2023-11-20 16:04:47 +01:00
Hornwitser 9f31394318 Name a type for event data
Use a named type for the EventData used in subscriptions.  This was done
as a workaround for a bug in the linter.
2023-11-20 14:43:03 +01:00
Hornwitser 2a1821993c Remove .js extension from import
This confuses the node/no-missing-import linter rule.
2023-11-20 14:03:52 +01:00
Hornwitser 3cc9ec3c68 Ignore complexity in ConfigGroup.define
This method will most likely be rewritten in a future config refactor.
Ignore linter complexity for now.
2023-11-20 13:50:20 +01:00
Hornwitser 52d3bba6f6 Move instance buttons out of InstanceViewPage
Break out the button row at the top of the instance page into its own
component to reduce the complexity of InstanceViewPage.
2023-11-20 13:49:23 +01:00
Hornwitser 0de70df472 Move dev server logic out of _startInternal
Break out the logic for setting up the webpack dev server into its own
method to reduce the complexity of _startInternal.
2023-11-20 13:46:51 +01:00
Hornwitser 7517917832 Move controller's TypeScript output to dist
Be consistent with the other packages and set the controller's
TypeScript compilation outDir to dist instead of dist/controller.
2023-11-20 13:46:25 +01:00
Hornwitser af9ed19af0 Setup typescript-eslint
Configure eslint to work with TypeScript via the typescript-eslint
plugin.
2023-11-19 04:50:07 +01:00
Hornwitser b78ca31b09 Add typing for serialized collector type
Setup TypeScript tyings and JSON schema for serializeResult and
deserializeResult and use these in the metrics related requests.
2023-11-19 04:50:07 +01:00
Hornwitser b66138587b Move exposition content type to its own var
Remove anti-pattern of setting properties on Function object that is
difficult to document.
2023-11-19 04:50:07 +01:00
Hornwitser 80e1904175 Fix bad typings left over from the migration 2023-11-19 04:50:07 +01:00
Hornwitser 7314dc99c8 Fix code style errors
Remove extra whitespace, use more consistent spacing, add missing
semicolons, using dangling commas, break up long lines, consistent
multiline operator placement, add missing JSDoc strings and so on.
2023-11-19 04:48:38 +01:00
Hornwitser 97e05c0610 Update GitHub actions 2023-11-18 22:14:29 +01:00
Hornwitser 2c9b07b8d6 Add subscription handlers in constructor
Move the calls to add subscrition handlers into the constructor of
Controller to reduce the amount of code in the start logic.
2023-11-18 22:12:14 +01:00
Hornwitser 60f09b11f9 Merge pull request #510 from Cooldude2606/refactor/subscriptions
Refactor/subscriptions - Create a generalized event subscription class
2023-10-13 14:45:44 +02:00
Cooldude2606andHornwitser 54cb483a31 Convert overloads to generic function
Co-authored-by: Hornwitser <github@hornwitser.no>
2023-10-13 11:21:55 +01:00
Hornwitser d58602f1a1 Fix merge conflicts 2023-10-12 17:21:56 +02:00
Hornwitser d84445baa1 Fix line endings and indents 2023-10-12 17:07:07 +02:00
Jonas Meeuws d73f26c83f Fix subspace_storage IpcItems type and UpdateStorageEvent.type (#516) 2023-10-12 16:20:27 +02:00
Cooldude2606 23ba5d612a Fixed failing tests for unsubscribe 2023-10-06 18:56:27 +01:00
Cooldude2606 b15b1a6007 Fixed bugs found during pre-review 2023-10-06 18:30:15 +01:00
Luciole e2dea35e02 allow to revoke host tokens from web ui (#511)
* allow to revoke / restore host access from web ui

* refector to what is expected

* remove core.admin check on HostViewPage.tsx

* update core.host.revoke_token description
2023-09-29 19:50:14 +02:00
Cooldude2606 11d820b085 Increase patch coverage 2023-09-07 22:31:09 +01:00
Cooldude2606 4986a06cdf Fixed web_ui websocket type error 2023-09-07 21:58:10 +01:00
Cooldude2606 621f850804 Some cleanup 2023-09-07 21:36:27 +01:00
Cooldude2606 2afddafe4d All tests passing 2023-09-07 20:54:21 +01:00
Cooldude2606 2c9050824e Test suite mostly ready 2023-09-07 03:36:38 +01:00
Cooldude2606 cccb27c733 Add test skeleton 2023-09-06 21:16:31 +01:00
Cooldude2606 71166af75b Fixed TestControl subscription requests 2023-09-06 01:09:02 +01:00
Cooldude2606 da25196830 Refactor to use subsribers 2023-09-05 01:46:45 +01:00
Cooldude2606 17943c48c3 A few renames and comments 2023-09-04 22:26:18 +01:00
Cooldude2606 62e69e3dbf Add SubscriptionController and EventSubscriber 2023-09-04 21:46:28 +01:00
Luciole 838170a9ff Port packages/web_ui to TypeScript (#508)
* .jsx to .tsx

* controller: tsc --build before webpack-cli

* Type Annotation

* disabled move into the <Input/> and <Checkbox/>

* cleanup debug left over

* documentation

* default ModPack

* modPackUpdateHandlers no undefined

* PluginInfo changes proposition

* PluginInfo changes proposition miss

* ModRecord & ModInfo feedback loop at const mods = [ ?

* ["connect", "drop", "resume", "close"] as const

* .tsx to .ts when possible
2023-09-02 14:51:35 +02:00
Cooldude2606 a4893751e9 Add module export detection (#505)
Allow modules to include a module_exports.lua file in their root which
will cause the file save/modules/module_name.lua to be generated
pointing to the exports file.
2023-08-27 17:47:37 +02:00
Hornwitser ee105e0e28 Merge pull request #504 from Hornwitser/tsconfig-cleanup
TSConfig cleanup
2023-08-27 17:33:49 +02:00
Hornwitser 55da662344 Rebuild TypeScript files on test
Prevent testing the wrong files by making sure the TypeScript build is
up to date before running the tests.
2023-08-26 17:11:47 +02:00
Hornwitser 425daac6b6 Set up references in TSConfig
Link the projects together properly so that builds don't use out of date
files and editor "go to definition" function give the right file.
2023-08-26 17:11:47 +02:00
Hornwitser c3d78a8cb8 Organise TSConfig files using inheritance
Removed duplicated TSConfig options by using a shared base config which
all plugins inherit shared settings from.
2023-08-26 17:11:46 +02:00
Luciole 1afbbe04e1 Fix issue with research_sync plugin (#502)
* fix type issues on loading technologies and broadcasting progress

* Distinguish Technology and TechnologySync type
2023-08-26 16:35:39 +02:00
Hornwitser 36ced0d3ba Merge pull request #503 from Luciole0x/inventory_sync-fix
fix inventory_sync plugin
2023-08-26 14:52:25 +02:00
Luciole 318a4301c0 Define responses types, DownloadResponse.player_data > playerData 2023-08-26 00:16:31 +02:00
Luciole 29eccb27ed Allow pluggins to get custom config field 2023-08-26 00:08:56 +02:00
Hornwitser 06f28eced9 Merge pull request #501 from Hornwitser/module-locale
Module locale
2023-08-25 22:07:11 +02:00
Hornwitser cc8ec36e5a Map locale from modules into the save's locale
Map locale files located at locale/{lang}/{file} in a module to
locale/{lang}/{module-name}-{file} in the save. This additionally
support a shorthand notation by mapping locale/{lange}.cfg to
locale/{lang}/{module-name}.cfg.

Resolves #500.
2023-08-25 16:46:13 +02:00
Hornwitser a1148c7ec9 Refactor module patching logic
Break out the module info and patch info into data classes and pack all
the files for a module into one more easily modified structure.
2023-08-25 16:33:02 +02:00
Hornwitser b1a6628bfa Remove left over load entries from 7b8127f
Remove save_crafts.lua and restore_crafts.lua entries in the module.json
for inventory_sync which was left over from the removal in 7b8127f.
2023-08-25 15:26:44 +02:00
Hornwitser 6d1ff4c178 Fix web interface broken by TypeScript porting 2023-08-25 15:25:43 +02:00
Luciole 0b3550b472 Port remaining plugins to Typescript (#499)
* .js to .ts

* Allow access to _items entries; getItemCount return number

* add permission field to PluginPage interface

* port plugins to typescript

* update Request classes

* fix IpcFlowType
2023-08-22 16:17:16 +02:00
Luciole 50f12c719b Port global_chat plugin to typescript (#497)
* .js to .ts

* translate

* update paths to dist/plugin

* add "tsc --build" to prepare

* use field main in package.json de find info

* tsc --build need to append before webpack

* update test and use for plugins package.main
2023-08-18 00:06:49 +02:00
Hornwitser 477b600b1b Merge pull request #498 from Hornwitser/typescript-host
Port host package to TypeScript
2023-08-17 19:48:22 +02:00
Hornwitser 946c6e93c4 Port host package to TypeScript 2023-08-17 16:47:09 +02:00
Hornwitser 880fc77b88 Fix logger.child returning incorrect type 2023-08-17 16:47:09 +02:00
Hornwitser 94fbb8a92f Rename MessageSrcDst to MessageRoutable 2023-08-17 16:47:09 +02:00
Hornwitser d088219d8d Merge pull request #496 from Hornwitser/fix-tests-on-windows
Fix tests on windows
2023-08-17 16:46:38 +02:00
Hornwitser 47375ec075 Skip unknown instance status check on Windows
The unknown status happens when an instance is on a host that has
disconnected, but since alt-host is not spawned in the tests on
Windows this status is not observed.
2023-08-15 18:44:53 +02:00
Hornwitser 115cb3fd5d Fix path slash issue in plugin loader test
On Windows require paths use a mix of forward and backward slashes due
to require path possibly being a package path which can't have backward
slashes, or a Windows file path which can have have either.  Fix plugin
loader test failing on Windows due to not using the same combination of
forward and backward slashes for require file paths.
2023-08-15 18:44:53 +02:00
Hornwitser 8bf5296fe6 Gracefully handle error linking mods on Windows 2023-08-15 18:44:53 +02:00
Hornwitser 9c9adf7c8e Use return type deduction for timers
Fix updates to Node.js timers types breaking the TypeScript build by
deducing the correct type from the return type of the
setTimeout/setInterval functions.
2023-08-15 18:01:27 +02:00
Hornwitser e51a644788 Merge pull request #495 from Luciole0x/master
Transition controller package to TypeScript.
2023-08-15 17:11:26 +02:00
Luciole e59027f557 don't cast filename to string 2023-08-14 22:27:35 +02:00
Luciole 4024969be8 no default for routes 2023-08-14 18:54:49 +02:00
Luciole f2736bae40 tsconfig update; fix function signature; test: add .default 2023-08-14 14:34:56 +02:00
Luciole a26dcf5075 defaultRoleId: explicitly check for null 2023-08-14 13:31:55 +02:00
Luciole 8108fcac81 update package.json to fix test running and import version 2023-08-14 11:56:41 +02:00
Luciole 8256f06a6a update QueryLogFilter: make maxLevel optional 2023-08-14 11:29:24 +02:00
Luciole 25e65910dd Controller: back to hosts, instances, modPacks, mods init to null 2023-08-14 11:26:48 +02:00
Luciole ef32f6f466 .closing > .connector.closing 2023-08-14 11:03:44 +02:00
Luciole 23089aa1af createUser: if no default_role_id, assign no role 2023-08-14 11:00:11 +02:00
Luciole 3036507baa clarify null / undefined / 0
remove useless type import and signature
export directly
cleanup
2023-08-14 10:54:30 +02:00
Luciole c96a13ef30 fix removed Buffer from base64 auth_secret 2023-08-14 10:03:31 +02:00
Luciole 3fac5b00d3 non-null assertion on response.socket 2023-08-14 09:48:38 +02:00
Luciole 3e4acaaf4e use instance.game_port 2023-08-14 09:47:50 +02:00
Luciole 799cfe7ff2 test: import controller/* from dist folder 2023-08-13 19:21:51 +02:00
Luciole 6104843858 transition of controller package to typescript 2023-08-13 18:12:12 +02:00
Luciole e011625a2d expose and/or modify signatures from lib package 2023-08-13 16:53:03 +02:00
Hornwitser b72fa25d3f Merge pull request #494 from Hornwitser/typescript-ctl
Port ctl package to Typescript
2023-08-12 20:52:14 +02:00
Hornwitser ec0fded0cc Port ctl package to TypeScript 2023-08-12 20:36:33 +02:00
Hornwitser 5ec7c9f99c Fix bad typings in lib
These were missed or broken in the in the inital porting of lib to
TypeScript.
2023-08-12 19:59:44 +02:00
Hornwitser 6d523096ce Merge pull request #493 from Hornwitser/typescript-lib
Port lib package to TypeScript
2023-08-12 14:44:59 +02:00
Hornwitser f875843c5c Turn on source maps for .ts files
Fix coverage checking not being able to deduce the original TypeScript
files due to lacking a way to link the compiled .js output files with
the .ts input files.
2023-08-12 14:34:36 +02:00
Hornwitser 954bc9b314 Use composite build
Make TypeScript build faster for a multi package setup by allowing the
compiller to quickly determine if a package is already built.
2023-08-12 02:31:17 +02:00
Hornwitser a66e930997 Use strict type checking 2023-08-12 02:31:17 +02:00
Hornwitser 2ca8e69025 Remove message_validate.js ignore
This file was relocated to the dist folder.
2023-08-11 01:16:12 +02:00
Hornwitser 679c00527f Port remaining lib/data modules to TypeScript 2023-08-11 01:13:38 +02:00
Hornwitser 2a52a82fb7 Port message types to TypeScript 2023-08-11 00:24:00 +02:00
Hornwitser 15ea9a91dc Port basic lib sub-modules to TypeScript 2023-08-06 18:38:20 +02:00
Hornwitser e0c197b9bc Port basic lib modules to TypeScript 2023-08-06 01:03:13 +02:00
Hornwitser 2988cd962c Port stream.ts to TypeScript 2023-08-03 15:51:58 +02:00
Hornwitser dedb42447c Port helpers.js to TypeScript 2023-08-03 15:03:52 +02:00
Hornwitser 74e9ed8257 Replace jsdoc with typedoc
Use TypeDoc instead of JSDoc because it has better support for
documenting TypeScript APIs.
2023-08-03 15:02:31 +02:00
Hornwitser 3a321b39e1 Remove incorrect validation of forwared messages
Messages are forwareded when received from another link, this means it's
too late to throw an error that bubbles up to the code trying to send
the incorrect message thus defeating the purpose of the validation.
2023-08-03 15:01:34 +02:00
Hornwitser 90798205e0 Set up TypeScript for lib
Add a TypeScript build suitable for incremental migration to TypeScript
to the lib package.
2023-08-03 15:01:34 +02:00
Hornwitser a1ee147c46 Merge pull request #490 from Hornwitser/speedup-ctl
Speedup ctl
2023-07-30 02:27:19 +02:00
Hornwitser 5eb91be6fb Pre-compile Message validator
Use a script to pre-compile the validator for the Message class to
reduce the loading time for ctl by about 40ms.
2023-07-30 01:54:59 +02:00
Hornwitser 83695e3e4c Merge pull request #488 from Hornwitser/refactor-lib
Refactor lib
2023-07-30 00:10:25 +02:00
Hornwitser ae542e9501 Lazy compile schema validators for ctl
Speed up the startup time for ctl by delaying schemas compilation until
the validator is used. Saves approximately 120ms on startup time.
2023-07-28 21:36:43 +02:00
Hornwitser a5c7eb3a27 Move user token signing to UserManager
Only the controller has access to the secret needed to sign user tokens
so it makes sense for the user signing code to live in the controller.
This saves about 20ms on the time ctl takes to start up due to lib no
longer needing to import jsonwebtoken.
2023-07-28 16:03:29 +02:00
Hornwitser 6af25e085c Move Factorio server implementation to host
Move Factorio related code in lib that's only used by the host into the
host package.
2023-07-28 16:03:29 +02:00
Hornwitser bc574ee1cb Reorganise packages/lib
Move the majority of the script files into a src directory to be
consistent with the organisation of the other packages.
2023-07-28 16:03:29 +02:00
Hornwitser bfe119eaa3 Flatten lib namespace
Move all functions and classes currently available through lib/name and
libName to be under a single flat namespace in lib.  This make
reorganising the lib namespace much simpler as the location of symbols
can move without breaking the consumers of lib.

The multi-entrypoint design was already problematic for the web plugin
system which couldn't effectively use it due to how webpack's shared
module system is awkward to set up with multiple entrypoints per package.
2023-07-28 16:03:29 +02:00
Hornwitser bb58670597 Merge pull request #487 from Hornwitser/node-js-16
Bump supported Node.js to v18 and update dependencies
2023-07-27 23:17:09 +02:00
Hornwitser c2fa873d85 Update version of pnpm used on CI 2023-07-27 21:04:27 +02:00
Hornwitser d9e94648d3 Fix wrong Node.js version in contributing docs. 2023-07-27 17:55:34 +02:00
Hornwitser fd709fd98c Update web dependencies
- react: Update to new root rendering method
- react-router: Replace useHistory with useNavigate
- react-router: Replace <Switch> with <Routes>
- reimplement sticky notice from scratch in ModPackViewPage
- remove isomorphic-ws as it's latest version doesn't work and only
  saves one line of code.
- remove import of antd.dark.css
- fix misc css breakages.
2023-07-27 17:50:14 +02:00
Hornwitser 8d98eaf3a5 Reimplement PageHeader
Implement our own version ofwhe PageHeader component, due to it being
removed from ant design v5.
2023-07-27 17:37:27 +02:00
Hornwitser 789c02b5c7 Fix missing return in applyModPackChanges 2023-07-27 13:43:32 +02:00
Hornwitser a1c9bc5f9b Migrate deprecated ant design APIs
Rename visible to open and use menu instead of overlay in Dropdown as
both of these have been removed in the next major version of Antd.
2023-07-27 13:42:35 +02:00
Hornwitser 2de029b96e Fix strict mode warnings from Ajv
Add type object to schemas specifying properties and minItems to schemas
specifying tuples. Disable strictTuples because it's desirable to be
able to add additional items to the tuples in the future.
2023-07-27 00:35:26 +02:00
Hornwitser 1a9cbd26b3 Update non-web dependencies
Update non-web related dependencies to their latest version were
possible. It appears yargs has changed its parsing of command line
options to no longer remove a layer of quotes.
2023-07-27 00:35:26 +02:00
Hornwitser a96a8a6d25 Bump minimum supported Node.js version to v18
The LTS releases for v12 and v14 are already end of life and the v16 is
end of life in under two months from now on.  Bump supported version up
to v18 as it does not make sense to continue to support EOL releases.
Even if we wanted to keep supporting them, our dependencies will not.

Note that Node.js v18 is currently not avaliable via the NodeSource PPA
on Ubuntu 19.10 and below or Debian 9 and below.
2023-07-27 00:35:26 +02:00
Hornwitser 93b19110df Merge pull request #485 from Hornwitser/data-class-refactor
Data class refactor
2023-07-27 00:34:57 +02:00
Hornwitser 295cc7543a Reverse argument order for sendTo
It makes more sense to put the address first then have the large
structure to send as the second argument to sendTo.  The opposite order
was originally chosen to be similar to that of sendto in BSD sockets.
2023-07-27 00:24:24 +02:00
Hornwitser 183d2921a8 Up timeout of mod-pack roundtrip test 2023-07-26 14:36:01 +02:00
Hornwitser 156d517976 Prefer putting nested structures on new lines
Format arguments to have nested structures start on a new line instead
of inline to make it easier to see where the structures starts and what
structure it is.
2023-07-26 14:27:49 +02:00
Hornwitser cb21a63ac2 Track forwarded requests to notify if link dies
Keep track of requests that have been forwarded over a link in a similar
manner to requests that were initiated on a link in order to properly
return an error response to the sender of the request if the link goes
down while the request is still pending at the forwarded node.
2023-07-25 21:01:48 +02:00
Hornwitser 0f9224a4ef Add lint-fix script to auto fix linter errors 2023-07-25 17:26:12 +02:00
Hornwitser adb9ebafe1 Fix broken tests under fast-test 2023-07-25 17:25:39 +02:00
Hornwitser 87172dfb8d Implement basic ingress filtering on controller
Validate messages received on the controller from hosts and control
nodes before handling or forwarding them.  This should prevent
performing actions that are not allowed by the role system and block
simple attacks such as spoofing the source address of a message.
2023-07-25 17:21:44 +02:00
Hornwitser 7a9aec74b5 Optimize ModPackViewPage performance
Avoid re-rendering everything by using useCallback and memo to skip
rendering of form elements that haven't changed.
2023-06-11 17:30:29 +02:00
Hornwitser 790b03bc03 Fix accidentally duplicated imports
Import lib instead of lib/data in the messages files to avoid these
getting packed with the web build.
2023-06-11 17:30:24 +02:00
Hornwitser 04b46540fb Fix minimized web_ui build stripping class names
The data class pattern used for request and event payloads require the
class names to be preserved as they are used for identifying the type of
the requests and events.
2023-06-11 17:30:11 +02:00
Hornwitser 7f8a700716 Refactor message registration and validation
Split up the action of registring an event or request data class to be
decoded by the Link class and the action of registring a handler for
those requests and events.  This moves the validation work from Link
instantiation time to startup initialisation which makes logic errors
fail on startup instead of failing instantiation of the link when a
connection attempt is made.

Additionally implement a partial validation logic for messages received
using the newly refactored message registration.  This unfortunately
ended up merged with the registration refactor.
2023-06-03 16:47:47 +02:00
Hornwitser bcd94709d2 Refactor link system to JSON serialisable class pattern
Reverse the direction of the API for sending messages from

    messageDefinition.send(link, { content });

to

    link.send(new MessageClass(content));

and convert all message definitions to use the JSON serialisable class
pattern and explicit registration of message handlers, along with a
end-to-end based routing of messages instead of link to link.

The experience I gained from the existing way messages were defined and
sent over links was that it was difficult to understand for people
unfamiliar with the system.  The automatic registration of message
handlers ment that code was being called as if by magic from nowhere
with no easy way to discover it.  The manual forwarding and rules for
what can be sent where and how obscured its working.

The use of pure JSON as both input and output from message handlers made
it tedious to work with.  Data had to be manually converted between the
program representation and the over the wire format, which made handler
code unplesant to write.  Particularly because the over the wire format
used snake_case while the program code uses camelCase for variables.
The idea behind the decision to do it this way was to make it more clear
which data came from the wire and which from the program, and while this
worked it didn't make message handlers any nicer to write.

The direction of the API and magic registration of message handlers also
meant that a future migration to TypeScript would be troublesome.  While
it may be possible to implement, the degree of TypeScript magic required
would go starkly against the principle of keep it simple stupid.

By refactoring to using the JSON serialisable class pattern most of
these issues go away.  There is no longer a need to distinguish the on
wire format of messages with the program representation because the only
place that deals with the wire format of messages are the toJSON and
fromJSON methods on the classes themselves.  Message handlers will work
with proper instances of classes instead of raw data, something that
makes them easier to write, and trivial to type with TypeScript.

End-to-end routing also simplifies sending messages that need to do
mulitple hops to reach the destination.  And removes the need for having
all messages be specified in plugin's info module, as they can be routed
without having to be parsed completely.

The explicit registration of message handlers is something that I
thought would be tedious boilerplate code that could be automated away.
But experience now shows it to be the opposite.  Allthough dull
repetition, the purpose it serves is to link together the code in a way
that humans can easily find.  With explicit registration searching the
name of the message handler in the editor will show the registration that
ties it together with the type of message it handles and the kind of
registration.  With implicit registration searching the name of the
message handler turns up only the handler with no clue as to what calls
it.
2023-04-23 13:19:46 +02:00
Hornwitser 9a967a5779 Refactor JSON serialiable class pattern
Change the JSON seriasiable data classes to use an explicit static
fromJSON method and an optional toJSON method for turning them to and
from JSON.

This improves over the previous pattern of passing the JSON to the
constructor as it frees up the constructor to be able to take whichever
argument types suits the code instead of being beholden to the JSON
serialised form.
2023-02-25 00:02:12 +01:00
Hornwitser a448d87e3a Fix instanceCreate example 2023-02-24 23:29:06 +01:00
Hornwitser 538159dd71 Remove time dependency in HttpCloser tests (#484)
Attempt to fix the HttpCloser tests failing due to timing issues by
replacing dodgy time based logic with explicit synchronisation.
2023-02-24 15:41:17 +01:00
Hornwitser a8af39e675 Merge pull request #483 from Hornwitser/control-connection-refactor
Control connection refactor
2023-02-24 15:40:59 +01:00
Hornwitser 05e9230e9e Add .id shorthand to InstanceInfo
Turn instance.config.get("instance.id") into instance.id.
2023-02-24 02:00:12 +01:00
Hornwitser f662437b3f Refactor actions out of ControlConnection
Move implementation of instance related actions out of ControlConnection
and into Controller to allow plugins to easily make use of these methods.
2023-02-24 02:00:12 +01:00
Hornwitser 496ebd18a5 Put instance runtime state into InstanceInfo class
Use the data class pattern for storing the instance data on the
controller.  Improves documentation for plugins trying to use it.
2023-02-24 02:00:12 +01:00
Hornwitser 6a2aaf3d26 Merge pull request #481 from Hornwitser/rename-master-slave
Rename master/slave terminology to controller/host
2023-02-23 19:53:42 +01:00
Hornwitser 56106d41eb Add migration mode to installer
Add migration code to installer fixing the names used in configs,
logs, database, and packages to reflect the new controller/host names
for people updating from alpha-13.
2023-02-23 00:12:48 +01:00
Hornwitser a32fb81273 Bump LogIndex version
The rename means that the log index is no longer correct. Bump the
version so that its content is ignored and a new index is generated.
2023-02-23 00:12:48 +01:00
Hornwitser 10528fbb90 Allow pre-rename host tokens
Ease migration to the new controller/host naming by allowing tokens
issued to the slave naming to be authenticated by the controller.
2023-02-23 00:12:48 +01:00
Hornwitser b4259f59cf Match process title with script name
The clusteriocontroller and clusteriohost scripts don't use camelCase so
the process title set by these scripts shouldn't use it either.
2023-02-22 15:50:02 +01:00
Hornwitser 116eceaac4 Remove migration of alpha-10 single file logs
This code is no longer functional after the master/slave rename and has
served its purpose to migrate the exsting logs to the new format.
2023-02-22 15:47:19 +01:00
Hornwitser 6fc3abafd5 Rename Slave to Host
The role of a slave as a host for Factorio servers is not well reflected
in its name and has caused confusion among users.  Rename slave to host
to better reflect its role as a host of Factorio servers.
2023-02-22 15:30:51 +01:00
Hornwitser 70ad295a89 Rename Master to Controller
The role of the master server as the party controlling every part of the
cluster is not well reflected in its name and has caused confusion among
users on several occasions.  Rename master to controller to better
reflects its role.
2023-02-22 15:30:51 +01:00
Hornwitser 6ae4c8ef46 Merge pull request #480 from Hornwitser/small-things-10
Small things 10
2023-02-22 15:28:34 +01:00
Hornwitser 8662243ed1 Show which config entries requires a restart
Add a label in the web UI to config entries that require a restart to
take effect as this is not obvious to end users.
2023-02-21 22:39:23 +01:00
Hornwitser 0af1a25c93 Fix bad identifier in copy save code 2023-02-21 22:39:23 +01:00
Hornwitser 57c7e04f04 Use Summary metric for query log duration 2023-02-21 22:39:23 +01:00
Hornwitser 2960db2cca Add revoke user token action
Implement revoking of user tokens to allow lost credentials to be
invalidated.
2023-02-21 22:39:23 +01:00
Hornwitser 5426d9b2bd Log warning on mismatched slave or plugin versions
Log warning to the master log if a slave connects with a missmatched
version, missmatched plugins or missmatched plugin versions.
2023-02-21 19:35:35 +01:00
Hornwitser 17e4b088f1 Fix crash when game starts tick_paused
Check patch number in update_instance step to ensure the clusterio_api
is properly initialized if the game started tick paused and no ticks
to trigger the patch check have happened yet.
2023-02-20 17:45:10 +01:00
Hornwitser 475182efd0 Ignore techonolgy enabled status for sync
Technologies that were disabled were ignored completely by research sync
due to a missunderstanding of what disabled means.  Disabled does not
mean the tech is somehow inaccessible and completely out of play, it
only means it can't be selected in the technology screen.

Ignore the enabled status of technologies in research_sync to fix a
crash that happened when a mod enables a disabled by default technology
and the player attempts to research it.
2023-02-20 04:45:18 +01:00
Hornwitser c60abdc349 Call fsync on patching saves
Similar to 1b2a20c call fsync on the newly written patched save before
renaming it to prevent data loss in the event of a system crash.
2023-02-17 23:52:31 +01:00
Hornwitser 43e6dd60b3 Removed duplicated rejection handling
Remove code accidentally left out by the change in 50703f8.
2023-02-17 23:52:31 +01:00
Hornwitser b7b7aa737a Merge pull request #479 from Hornwitser/small-things-9
Small things 9
2023-02-16 20:15:26 +01:00
Hornwitser 1b2a20c395 Call fsync on safeOutputFile
If a system crash occurs after a simple write file + rename operation
the target file that was renamed over may end up being empty if the data
for the file written had not yet been committed to disk.  This is
possible because no part of the open file, write data to it, close it
and then rename it ensure the data was actually written to disk prior to
the rename happening. See this discussion on the btrfs wiki[1].

Fix this possible data loss scenario by calling fsync on the temporary
file that is written prior to calling close on it.

[1] https://btrfs.wiki.kernel.org/index.php/FAQ#What_are_the_crash_guarantees_of_overwrite-by-rename.3F
2023-02-16 05:19:04 +01:00
Hornwitser 50703f8ae6 Treat unhandled errors as fatal
All possible Promise rejections and exception bubbling should be handled
by either the code creating the exception or the code invoking the
operation that could fail.  Treat therefore uncaught exceptions and
unhandled promise rejetions as fatal errors that log a traceback and
immedately terminate the Node.js process.
2023-02-16 01:52:42 +01:00
Hornwitser 27633d51b6 Fix log filtering in Web UI
Remove broken alternative logFilter implementation in
web_ui/src/websocket in favour of having one proper
function in lib/logging
2023-02-16 01:22:30 +01:00
Hornwitser 4f6452c90c Fix Web UI not removing log subscriptions 2023-02-16 01:07:42 +01:00
Hornwitser 5bb10ef970 Attribute instance logged errors
Fix errors during startup and from plugins not being correctly
attributed to the instance they originate from.
2023-02-16 01:06:03 +01:00
Hornwitser 59eca88e91 Filter out instance logs from slave console
Exclude instance related logs from the slave console as these are not
significant for the running of the slave and just add noise.
2023-02-16 00:57:54 +01:00
Hornwitser bfe523ac45 Fix disabled mods still being loaded
The format of mod-list.json written on syncMods would incorrectly write
a base mod without an enableds key which is required, causing Factorio
to ignore the file entirerly.  Furthermore the enabled status was not
written causing all mods to be loaded even if they were disabled in the
mod pack.
2023-02-15 20:31:02 +01:00
Hornwitser fda2d7f574 Fix error applying factorio.settings log location
Log to the instance instead of the generic logger when an error occurs
applying a factorio.settings config entry.
2023-02-15 20:31:02 +01:00
Hornwitser 1ba091fd0e Fix WebConsoleFormat not using names
The logging format provides both the name and id of slaves and
instances.  Use the name when formatting for the web console just like
the TerminalFormat does.
2023-02-15 20:31:02 +01:00
Hornwitser 5e01f427af Fix slave crashing when Factorio server is not executable
Handle errors events emitted from the child process created by
lib/factorio/server and translate this error if it's EACCES.
2023-02-15 20:31:02 +01:00
Hornwitser 0341d41165 Fix debug dump-ws command broken by 246ce73 2023-02-15 20:31:02 +01:00
Hornwitser 082d07d718 Fix no way to unassign instances in Web UI 2023-02-15 20:31:02 +01:00
Hornwitser 2d8b35291d Fix incorrect JSDoc async return annotations
The return type of an async function should be documented as a Promise
of what the function returns as this is what JSDoc expects.
2023-02-15 20:31:02 +01:00
Hornwitser a7dedcd3b7 Trim whitespace from token login in Web UI
Prevent accidentally copying spaces before and/or after the token when
pasting it into the login form from failing to log in.
2023-02-15 20:31:02 +01:00
Hornwitser d39b525b07 Add clusterio_master_log_bytes metric
Export a metric for total size of logs stored on the master server.
2023-02-15 20:31:02 +01:00
Hornwitser 351fcba4f8 Print version on startup
Help determining what is running by printing the version that is running
on master and slave startup.
2023-02-15 20:31:02 +01:00
Hornwitser 3d186a24ff Fix incorrect handling of 0 ids
Fix tests of slave and instance ids that incorrectly return the wrong
result when the id is 0.
2023-02-15 20:31:02 +01:00
Hornwitser 5ad89f072f Replace endpointHitCounter
Track summary statistic of total HTTP response durations using a
middleware that applies for all requests instead of having every request
needing to update the hit counter manually.  Deprecates
endpointHitCounter.
2023-02-15 20:31:02 +01:00
Hornwitser 98dc0174d6 Implement basic Summary metric
Provide the basic version of Summary that does not report quantiles in
the prometheus library.  Useful for geting averages of observed values.
2023-02-15 20:31:02 +01:00
Hornwitser d29715e9ad Allow players to list users by default
The rationale for not allowing players to list users in the cluster by
default was that this would be an information disclosure of banned users
in the cluster, but given that a player is a user that can connect to a
server, and anyone that can connect to a server get the full banlist
from the server during the connection phase this does not make much
sense.  Grant permission to list users by default for players.
2023-02-15 20:31:02 +01:00
Hornwitser 58e5d96e1d Annotate error when loading module.json fails
If a plugin has an invalid module/module.json file this should also give
a readable error.
2023-02-15 20:31:02 +01:00
Hornwitser 033de0e20a Validate plugin entrypoint
Check that Master/Instance/Control plugins export the correct class and
that this class is a subclass of the correct base class.  This gives a
better user experience when plugins are incorrectly set up and loads a
duplicated clusterio/lib, and when accidentally using the wrong name
when developing plugins.
2023-02-15 20:31:02 +01:00
Hornwitser 6d7a2d46d6 Fix sorting of last seen column 2023-02-15 20:31:02 +01:00
Hornwitser bf92dde28d Merge pull request #478 from Hornwitser/mod-management
Add mod pack oriented mod management
2023-02-15 18:32:37 +01:00
Hornwitser 0eabdf011b Fix mod loading tests 2023-02-02 01:07:33 +01:00
Hornwitser 91af66e2db Fix export when default mod pack is used 2023-02-02 01:07:08 +01:00
Hornwitser 8db1b48691 Fix export for mods with uncommon root directory
The directory which all files are placed under in a mod is commonly
named <mod-name>_<mod-version>, but this is not a requirement as it can
be named anything.  Fix code incorrectly assuming the zipped mod's root
folder follows this pattern during data export.
2023-02-02 00:28:58 +01:00
Hornwitser 3f6a7e4336 Format mod settings with prototypes and locale in web UI
Use exported settings prototypes and locale to format the settings UI to
with mod names, settings names, and select dropdowns.  This is a crude
approximation of how the game presents these settings to the user.
2023-02-02 00:28:58 +01:00
Hornwitser ddc548f590 Set default values for mod settings on export
Add missing mod settings set to their default values to a mod pack when
performing a data export based on the settings prototypes that were
extracted from the game. This mirrors the behaviour of Factorio when new
setting prototypes are added by a mod.
2023-02-02 00:28:58 +01:00
Hornwitser 39ed098a64 Fix error messages changing with Node.js version
Updates to Node.js error reporting have caused tests to fail due to them
being hardcoded to expect a certain format that has since changed.  Fix
by making Node.js create the error message for us.
2023-02-02 00:28:58 +01:00
Hornwitser 13e12f7335 Tie data exports to mod packs
Store the export manifest in ModPack to allow mulitple mod packs to have
different data exported. This is necessary as the export contains the
settings prototypes needed to create a proper UI for mod settings.

For now the useLocale and useItemMetadata interfaces uses the manifest
for a default mod pack set in the master config.
2023-02-02 00:28:58 +01:00
Hornwitser 7c6aaeb05d Export settings and all prototypes
Add settings and all prototypes to the export of data done.  Settings
are needed for a settings GUI to function, while prototype data would be
useful to display any sort of information about things from the game.
2023-02-01 15:29:23 +01:00
Hornwitser fb75e47eda Mark missing or missmatched checksum for mods in mod packs
Mark mods in the UI and the mod-pack show ctl command that are missing
with a question mark and mark mods with a missmatched checksum with an
exclamation mark.
2022-11-10 00:28:50 +01:00
Hornwitser e6add2f7e6 Fix handling of numeric settings in ModPackViewPage
Due to a mistake in the logic numeric settings would end up getting
converted to string settings when edited.
2022-11-10 00:28:50 +01:00
Hornwitser 32cf9321cc Make sure checksum matches in ModsTable
Validate sha1 sum matches for a mod used in ModsTable if it is present.
This prevents showing details for a zip file where the version matches
but the cheksum does not.
2022-11-10 00:28:50 +01:00
Hornwitser dfadc62720 Prety print database/mod-packs.json 2022-11-10 00:28:50 +01:00
Hornwitser fb4ffaffed Implement mod pack loading for shared mods
Wipe and reconstruct the instances' mods/ folder using the configured
ModPack for the case were the mods folder is shared with the master
server.  This assumes the master server will check the mod checksums
before sending the start request.
2022-11-10 00:28:50 +01:00
Hornwitser 0ebb2f5858 Add import and export mod-pack commands 2022-11-10 00:28:50 +01:00
Hornwitser 87598b24ee Use pre-generated certificate for HttpCloser test (#477)
Remove ugly hack to generate and use an insecure certificate on the fly
in the HttpCloser's https tests and replace it with the already existing
properly generated test certificate. Removes the dependency on node-forge.
2022-11-09 20:18:24 +01:00
Hornwitser 4e9f3ee173 Merge pull request #475 from Danielv123/slaves-generate-random-token
Generate a new random slave token on opening modal
2022-11-09 20:17:54 +01:00
Daniel 4eb4bb9623 Fix slave token modal form validation 2022-10-17 08:23:43 +00:00
Daniel 7bb33554f4 Generate a new random slave token on opening modal 2022-10-16 18:49:48 +00:00
Hornwitser 2ae8270a1d Don't hypenate mod-pack 2022-10-15 18:01:54 +02:00
Hornwitser 20e620993b Use Factorio's version component names
Minor nitpick: The coponent names of the Factorio version is called
main, major, minor.  Change integerFactorioVersion to match.
2022-10-15 18:01:54 +02:00
Hornwitser 660d0f08a1 Fully specify mod pack Factorio version
Remove support for major only versions of Factorio being specified in
mod packs.
2022-10-15 18:01:54 +02:00
Hornwitser 53a5d7ea6f Add ModPack.toModSettingsDat
Add utility function to convert the settings representation of a ModPack
into the mod-settings.dat format which Factorio reads.
2022-10-15 18:01:54 +02:00
Hornwitser fafdac21fb Follow mod pack string proposal
Make all properties on the mod pack object except sha1 required and add
the base mod if it does not exist in order to align the JSON
representation with the proposed Mod Pack String format I published on
the Factorio Forums (https://forums.factorio.com/103578)
2022-10-15 18:01:54 +02:00
Hornwitser f863aee3c2 Add enabled field for mods in mod packs
Make it possible to disable built in mods like base by providing an
enabled field that can be set to false in mod packs.
2022-10-15 18:01:54 +02:00
Hornwitser 2384c48749 Document and expose data members under lib/data
Make members of lib/data exposed under lib/data instead of
lib/data/<member>.
2022-09-26 16:54:19 +02:00
Hornwitser 7072b94355 Derive mod filename from name and version
Enforce the filename of uploaded mods match {name}_{version}.zip with
name and version taken from the zip file's info.json entry.  This allows
it to be used as a unique key for the mod and solves part of the issues
with mods having different number of leading zeros in their version.
2022-09-26 00:34:25 +02:00
Hornwitser 7d34c60dd1 Break out version to integer functions
Move the multiple implementation of version to integers calculation to
dedicated functions in one place.
2022-09-25 23:33:22 +02:00
Hornwitser af79e2abbd Remove includeHash option to toModPackString
Since the Mod Portal provides SHA1 hashes it makes sense to always
include the hashes with mod pack strings if they are available.
2022-09-24 21:49:41 +02:00
Hornwitser 9b4ca86650 Rename mod hash field to sha1
The Factorio Mod Portal uses and provides SHA1 hashes of mods, so it
makes sense to align the ModPack and ModInfo structures to sha1 by
renaming the genericly named hash property to sha1.
2022-09-24 21:43:07 +02:00
Hornwitser bddde30be4 Add web UI for mod packs
Implement create, import, show, export, update, and delete UI for mod
packs on the master server.
2022-09-24 21:17:18 +02:00
Hornwitser 3eb63db313 Add search mods request
Add paged search mods request suitable for showing a real time list of
results for a user provided search expression.
2022-09-24 21:10:38 +02:00
Hornwitser 6039715a53 Wrap section header extra content under the header
Make the extra content for section headers flow under the heading when
there is not enough space to show both the heading and the extra content
in a single line.
2022-09-24 00:49:56 +02:00
Hornwitser 85cec761e1 Use overflow clip in site layout
Use overflow clip instead of hidden in the page area of the site layout
to avoid creating a formatting context that would make sticky not work
as expected inside the page area.
2022-09-24 00:46:38 +02:00
Hornwitser 476f460f75 Add react wrapper for navigator.clipboard
Add a thin wrapper of the navigator.clipboard interface suitable for
use with React.
2022-09-24 00:44:13 +02:00
Hornwitser 40a0b929d0 Break mod details out into a component 2022-09-24 00:39:16 +02:00
Hornwitser 82c6448149 Use no-break space in formatBytes
Prevent tables cells containing byte sizes from having a line break
inserted between the number and the byte unit when they get squished by
using a no-break space instead of a regular space.
2022-09-24 00:30:18 +02:00
Hornwitser 9fb3d6d8db Implement mod pack storage
Add ModPack data type with corresponding storage on the master, messages
to create, update and delete them, as well as commands to invoke them.
The ModPack data type contains a list of mods with versions and optional
sha1 hashes, and mod settings to be used.
2022-09-16 02:44:06 +02:00
Hornwitser dfcbc559b8 Add parseSearchString to helpers
Implement parsing of a search string into terms which can be used to
match results with them.
2022-09-16 02:44:06 +02:00
Hornwitser 3b57c1e0ae Add shared mods management
Add the ability to manage a set of mods to be shared by the master
server to the rest of the cluster.  This implements storage, uploads,
downloads, deletetion, and listing details of mods but does not actually
do anything with the stored mods.
2022-09-13 19:15:46 +02:00
Hornwitser 2ab915cc00 Move formatBytes to libHelpers 2022-08-24 15:42:26 +02:00
Hornwitser 921ebad8fb Allow importing libHash to succeed in browser
Alsthough this does not make libHash work in the browser it allows other
modules to require libHash while still being exported to the browser.
2022-08-24 15:42:26 +02:00
Hornwitser a40fefc8d7 Pull findRoot out into libZipOps
The findRoot function useful for managing mod zip files on the master.
2022-08-24 15:42:26 +02:00
Hornwitser 9a1c7b2ecd Bad Windows names also apply with any extension
A file name con.zip is also treated in the same way as the special CON
redirection and may not be used as a normal filesystem path in Windows.
This includes any extension, even with multiple dots.  Fix checkFilename
and cleaneFilename not handling these cases.
2022-08-24 15:42:26 +02:00
Hornwitser 14934a0d3a Move checkFilename and cleanFilename to libFileOps
These are also useful for managing mod zip files on the master.
2022-08-24 15:42:26 +02:00
Arvids Godjuks e82914c4e1 Merge pull request #473 from rstrom1763/master
Corrected assorted spelling and grammar in README
2022-07-30 23:38:28 +03:00
Ryan 190db2f98b Corrected assorted spelling and grammar in README
Corrected assorted spelling and grammar in the README.md file.
2022-07-30 15:02:26 -05:00
Hornwitser 5570d2da38 Update instructions for publishing
Manual workflow for now.
2022-07-20 15:27:37 +02:00
Hornwitser 949ba708ea Hotfix installer wiping package.json
Fix the installer wiping the package.json file due to a call to
outputFile that needed to target the original file being incorrectly
changed to safeOutputFile.
2022-07-20 14:40:41 +02:00
Hornwitser bcb2365c9a 2.0.0-alpha.13 2022-07-20 13:30:35 +02:00
Hornwitser 259d18eb30 Fix Web UI not displaying date for last seen 2022-07-20 13:30:35 +02:00
Hornwitser 04852196ad Rename play time to online time
Be a bit more accurate in the online time stat name and call it for what
it is.  It counts time spend online on the server, not necessarily
time spent playing.
2022-07-20 13:18:47 +02:00
Hornwitser fc1daa7e71 Bump path traversal test timeout (#472)
Double timeout on these two tests as they sometimes time out on GitHub
Actions.
2022-07-19 21:24:44 +02:00
Hornwitser e234d3db36 Fix disconnect_reason missing before Factorio 1.1.3 (#471)
Provide "quit" as the reason for disconnecting in versions of Factorio
before 1.1.3 which added reason to the on_player_left_game event.
2022-07-19 21:24:26 +02:00
Hornwitser 7b8127fef8 Refactor inventory_sync serialisation (#470)
Fix broken support for retrying uploads and background downloads by
making crafting queue serialisation non-destructive and only performing
the serialisation and deserialisation while the player is online.

This removes the /save-crafts and /restore-crafts commands because the
existing implementation would voids the non-saved crafting queue items
if /save-crafts is performed and then new items are added to the
crafting queue before switching server.

Fixes #419.
2022-07-19 21:24:12 +02:00
Hornwitser db84b5a6ce Shut up Dependabot (#469)
If only the bot was clever enough to see that node-forge is only used
for a test that has no user input it might have had the foresight to
realize that crying wolf about completely irrelevant security issues
will only lead to legitimate issues being ignored in the future. But
alas, the AI overlords are not taking over any time soon.
2022-07-19 21:23:33 +02:00
Hornwitser c79ced28dd Merge pull request #468 from Hornwitser/installer-fixes
Installer fixes
2022-07-19 21:23:09 +02:00
Hornwitser 2408dc8985 Remove obsolete Factorio download instructions
On Linux the installer offers to correctly download and install the
latest version of Factorio.  Remove no longer needed instructions for
how to do it incorrectly.
2022-07-18 20:10:08 +02:00
Hornwitser 2953eb17d6 Refuse installing Clusterio as root
I have seen multiple people have installed Clusterio as the root account
and then run it as root when tasked with setting up Clusterio on Linux.
Make this mistake harder to do by refusing to install as root by
default.
2022-07-18 20:05:37 +02:00
Hornwitser c9619397b6 Use current user's group for systemd files
Generate systemd files using the current user's main group name instead
of nogroup as the latter is not present in all systems.
2022-07-18 19:39:01 +02:00
Daniel c8a6b8094e Merge pull request #452 from Danielv123/slave-guided-setup
Added setup guide for slaves from web interface
2022-07-16 16:51:41 +02:00
danielv 7ee9c60e38 Add syntax highlighting to codeblock 2022-07-16 15:59:57 +02:00
danielv 4b0a33174d Added setup guide for slaves from web interface 2022-07-16 15:59:27 +02:00
Hornwitser db27650b60 Merge pull request #467 from Hornwitser/user-ui-update
User UI update
2022-07-16 15:11:21 +02:00
Hornwitser aadf24db99 Add test for sending user ban_reason 2022-07-16 14:30:06 +02:00
Hornwitser 2496e756f1 Fix display of stats in Web UI when empty 2022-07-15 16:14:04 +02:00
Hornwitser 5ba9f94b7e Only show admin/whitelist switches if usable
Since the admin and whitelist status is shown with a tag next to the
name it's no longer necessary to show the admin and whitelisted switches
when the user does not have permission to use them.
2022-07-15 16:14:04 +02:00
Hornwitser 9fa41e37f3 Show ban reason in Web UI
Show and update live the reason given for someone being banned in the
Web UI.
2022-07-15 15:57:45 +02:00
Hornwitser d184b9db81 Merge pull request #466 from Hornwitser/player-tracking
Per player statistics
2022-07-13 22:49:33 +02:00
Hornwitser 08e7434d24 Add command to extract play time from save
Add command that extracts play time (and players) from the currently
running save to the recorded instance statistics.  This is useful for
getting correct play time statistics when an existing save game is
imported.
2022-07-13 00:31:02 +02:00
Hornwitser 0245ccd16c Share player stats with master server
Send the per player stats with the player event updates to the master
server and expose it to the Web UI and ctl interfaces.
2022-07-12 23:03:50 +02:00
Hornwitser deb6842a4d Record all players as leaving on server exit
In the case of shutdown the Factorio server will not notify the
currently online players as leaving the game.  Notify the master server
that all remaining online players quit the game in this case.
2022-07-12 23:03:50 +02:00
Hornwitser ac62f66aa1 Move PlayerStats to separate data class 2022-07-12 23:03:50 +02:00
Hornwitser 097827b515 Update linter for ES2022 syntax
Allow modern JavaScript syntax including class property declarations
since Node.js v12 supports this.
2022-07-12 17:29:22 +02:00
Hornwitser 9a3edc8c11 Fix web build broken by assert and buffer imports
Not sure what made it break, but it seems like it shouldn't have worked
to begin with.
2022-07-08 01:00:54 +02:00
Hornwitser 096f338277 Add live updating of users
Provied a subscription for user updates and send updates whenever the
details of a user account is changed.
2022-07-08 01:00:54 +02:00
Hornwitser db562ce6a0 Provide reason for player leaving
Expose the reason for why a player is leaving a server in playerEvent.
2022-07-08 01:00:54 +02:00
Hornwitser 8017497bdd Merge pull request #464 from Hornwitser/player-time-autosave
Player online autosave pool
2022-06-30 14:33:19 +02:00
Hornwitser 3cb9843f23 Add player online autosave pool
Add separate autosave pool for autosaves that have had players online
since the previous autosave.
2022-06-30 01:03:29 +02:00
Hornwitser a063cf2db5 Track player stats on instances
Track which players are online and record basic stats for each player on
each instance including when save patching is disabled.
2022-06-29 21:35:56 +02:00
Hornwitser 72357dfd23 Fix wrong action in load scenario error message 2022-06-29 21:25:25 +02:00
Hornwitser 0df6701326 Use consistent path for saves in Instance 2022-06-29 21:25:25 +02:00
Hornwitser f5e558cb2f Sneaky tab removal 2022-06-29 21:25:25 +02:00
Hornwitser 995f0934ed Merge pull request #463 from Hornwitser/player-time-autosave-bugfixes
Player time autosave bugfixes
2022-06-24 13:08:35 +02:00
Hornwitser ef74436cde Fix jsdoc for #462
Fix references and missing files from the jsdoc output after the
refactor in #462.
2022-06-23 16:57:52 +02:00
Hornwitser 51c025c79b Fix Log out button not logging out 2022-06-23 16:57:52 +02:00
Hornwitser 12767fd58c Merge pull request #462 from Hornwitser/minor-refactor
Minor refactor
2022-06-23 16:40:14 +02:00
Hornwitser 826d89c7dc Fix race condition on creating instance directory 2022-06-23 00:09:28 +02:00
Hornwitser 393652c1d9 Allow instance names to be any string
Replace bad characters with underscore instead of rejecting bad instance
names.
2022-06-23 00:08:23 +02:00
Hornwitser 246ce73d94 Break commands out of ctl.js 2022-06-22 22:33:54 +02:00
Hornwitser 951bad4134 Refactor slave code
Move classes from slave.js monolit into their own source files.
2022-06-22 22:16:58 +02:00
Hornwitser ae17b00b6f Hide start button based on permissions
Hide the start/stop buttin in instance view if the current used does not
have permission to use it.
2022-06-22 21:26:41 +02:00
Hornwitser 36808b34de Merge pull request #461 from Hornwitser/small-things-8
Small things 8
2022-06-20 13:26:14 +02:00
Hornwitser 105f99ced3 Preserve entered vehicle on inventory sync
Save vehicle the player is driving before downloading the inventory and
restore the player position to this vehicle if possible.
2022-06-19 19:51:22 +02:00
Hornwitser 430a64722e Use empty slots as end for logistics requests
Count up to 100 empty slots past the last seen slot with data in it
before stopping to look for personal logistics slots in inventory_sync.
Allows unlimeted amount of logistics request slots to be used for
heavily modded games.
2022-06-19 17:03:08 +02:00
Hornwitser 7e9604a254 Fix LogConsole scroll anchor trasmutation
Fix react mutating the scroll anchor element in LogConsole into a log
line and breaking scroll anchoring to the bottom as a result.
2022-06-19 16:38:02 +02:00
Hornwitser 5369f283d7 Merge pull request #460 from Hornwitser/cache-web
Cache static web files
2022-06-19 15:34:19 +02:00
Hornwitser 654465e4eb Fix LogConsole using duplicated keys
Fix past lines loaded from the log containing keys that clash with new
items logged.
2022-06-09 00:53:12 +02:00
Hornwitser c5a0c0a873 Migrate to new <Menu> syntax
The use of <Menu.Item> children to <Menu> is deprecated, convert
existing code to use the new items={...} syntax.
2022-06-09 00:52:27 +02:00
Hornwitser 16a466aed0 Merge pull request #458 from Hornwitser/replace-npm
Replace npm and lerna with pnm
2022-06-09 00:11:24 +02:00
Hornwitser 3e836c1dd3 Fix web interface breaking when disabling plugins
Fix incorrect logic to set up plugins in the web interface causing an
emptry plugin to be atempted to be loaded when a plugin was disabled.
2022-06-08 23:26:33 +02:00
Hornwitser ddf15f9b92 Add option to turn on authentication server bans
Factorio 1.1.40 added a feature to use a centralized banlist maintained
by Wube.  For the dedicated server this needs to be turned on with the
command line flag --use-authserver-bans.
2022-06-08 22:43:58 +02:00
Hornwitser 54897c1447 Fix deserializing to a smaller inventory
Void excess slots when deserializing an inventory to a target that has
fewer slots than what was serialized.  This fixs inventory sync failing
when a stored inventory has trash slots in use but the player does not
have the slots unlocked.
2022-06-08 22:26:08 +02:00
Hornwitser 2a221456d9 Fix exported data loading in web interface
Store exported data files with a hash in the filename to allow the new
caching stategy for static files to work with it and provide a manifest
for the web interface to find the exported files with.
2022-06-08 21:01:36 +02:00
Hornwitser 48a57bf16c Use name of the cluster as web title
Set the title attribute of the web interface page to the name of the
cluster.
2022-06-08 21:01:36 +02:00
Hornwitser 8742b4127c Improve pre-loaded feedback of website
Render the web interface as early as possible with a black background
and show a message to indicate whether the interface is actually loading
or if an error occured loading the main bundle.
2022-06-08 21:01:36 +02:00
Hornwitser e32637503d Move all web assets to shared cached space
Put all web assets including plugin assets under a shared static/ route
in the web interface that's declared as immutable and cacheable for 1
year.  This allows all JavaScript bundles to be loaded from disk instead
of having to check with the web server each time and significantly
speeds up the web interface load time.  It also fixes the public path
issue when using the --dev-plugin option behind a proxy.

To prevent collisions and stale data all assets now include a hash in
the filename.  This means the entrypoint is now dynamic and specified in
a manifest.json file created from the web build.  For plugins this means
the filename attribute needs to be removed from webpack.config.js.
2022-06-08 21:01:36 +02:00
Hornwitser 2401fe25fa Remove test package
Mocha's recursive search feature finds all files and then removes paths
that match the exclution filter, except that the globstar filter doesn't
match directories with a dot in them which is created by pnpm.  Avoid
this madness by removing the test package.
2022-06-08 16:48:40 +02:00
Hornwitser f0b60119cb Add pnpm to GitHub Action
Based on https://pnpm.io/continuous-integration#github-actions
2022-06-05 17:00:21 +02:00
Hornwitser 4d5597a1da Use prepare instead of build as script
The prepare script runs before publishing a package, after installation
of a package from source, and when running npm install without
arguments in the repository.  These are the times where a build of the
web interface files are required to make the web interface work and
removes a manual step from using the package(s). The "build" script on
the other hand is never automatically invoked by npm.
2022-06-05 16:13:53 +02:00
Hornwitser 2973485321 Replace npm and lerna with pnpm
Save disk space, time and sanity by replacing npm and lerna with pnpm.
This is more performant and doesn't have developer experience ruining
bugs like npm link removing all previous links before creating the
requested one.
2022-06-05 16:13:53 +02:00
Hornwitser 1e4292876c v2.0.0-alpha.12 2022-04-22 12:36:00 +02:00
Hornwitser ca6464baeb Merge pull request #456 from Hornwitser/save-transfer
Save transfer
2022-03-30 22:51:04 +02:00
Hornwitser c2cb58d088 Comment out command logging in tests
Don't log commands executed while running tests as these spam the output
more than they are useful right now.
2022-03-29 22:55:13 +02:00
Hornwitser 38c249ec70 Implement save transfer
Add ability to move or copy saves between instances.  For instances on
the same slave the saves are transferred with filesystem operations, for
instances on two different slaves the proxy stream interface is used to
send the save via the master server.
2022-03-28 20:42:08 +02:00
Hornwitser 2e1c1fd3c8 Remove duplicated instance check
Deduplicate instance existence check for requests in master.
2022-03-28 20:42:08 +02:00
Hornwitser 4edc7a2949 Implement rename and copy save
Add ability to rename and copy existing saves on an instance.
2022-03-27 19:59:29 +02:00
Hornwitser 9b5c6f8554 Remove duplicated request checks
Move duplicated checking of save filenames and instance existing from
requests into separate functions.
2022-03-27 18:50:15 +02:00
Hornwitser 0de964ec02 Reorganise instance save commands
Move create, list, upload and dalete save management commands to its own
'instance save' namespace in the command line interface.
2022-03-27 15:50:42 +02:00
Hornwitser 7db091d427 Simplify wording of save permissions
Use shorter and more consise titles for the save management related
permissions.
2022-03-27 15:39:40 +02:00
Hornwitser 9cf7dea579 Add mods for subspace_storage in GitHub Actions (#455)
The GitHub Actions workflow was missing the mods needed to do the tests
for subspace_storage, causing coverage to be lower than it should be.
2022-03-27 13:16:12 +02:00
Hornwitser 9465a488cc Migrate coverage reporting to GitHub Actions (#454)
Travis seems to be going away from supporting Open Source Software for
free and their proceedure to gaining free tokens is getting harder and
hander to find for each year that passes.  Remove Travis CI integration
and migrate the coverage reporting previously provided by the Travis
build over to the GitHub Actions.
2022-03-26 21:23:19 +01:00
Hornwitser 5e94198216 Merge pull request #453 from Hornwitser/small-things-7
Small things 7
2022-03-26 21:22:35 +01:00
Hornwitser 854d2b3c79 Suggested grammar fix 2022-03-26 20:27:59 +01:00
Hornwitser 946847c95d Send save list update after manual save
Trigger a save list update in the event an admin manually saves the game
with the /server-save command.
2022-03-25 23:15:07 +01:00
Hornwitser ce95f868fd Fix curently typo 2022-03-25 23:15:07 +01:00
Hornwitser e5a63a3508 Add start-stop button to instance list 2022-03-25 23:15:07 +01:00
Hornwitser 1e4d709a06 Implement kill instance
Add ability to kill running instances in case it is desirable to stop
the server without saving over the current save file.
2022-03-25 20:52:31 +01:00
Hornwitser a93f048814 Add support for sidebar groups
Allow sidebar entries to be grouped together by specifiying a name under
which they will be grouped.  To avoid confusion groups are rendered at
the bottom of the list.
2022-02-10 14:51:51 +01:00
Hornwitser 5319ff3907 Add hook to add extra content to Web UI pages
Allow plugins to extend Web UI pages provided by Clusterio by
adding components in the componentExtra property of the plugin.
2022-02-10 14:51:51 +01:00
Hornwitser 68b8ac6b8c Add onMasterConfigFieldChanged hook
Add hook for master plugin code to react to the master config changing.
2022-02-10 10:29:47 +01:00
Arvids Godjuks a979c0cfac Merge pull request #446 from jake-87/ctl-docs
Added more documentation for managing-a-cluster.md
2022-01-19 13:39:45 +02:00
J87 3da8c30a64 Remove unnecessary information 2022-01-11 19:55:18 +11:00
J87 f8c72a1e0a Fixed wording issue 2022-01-10 12:37:02 +11:00
Hornwitser 4a059efe18 Merge pull request #447 from Hornwitser/lgtm-alerts
Fix issues detected by lgtm.com
2022-01-09 19:06:38 +01:00
Hornwitser dec5467538 Use anonymous master metrics
Convert the master metrics to void instead of storing them in a variable
as they are completely self contained, and this makes it more clear that
there are no use of references to the metrics.
2022-01-05 19:45:24 +01:00
Hornwitser 61cd5bd76c Fix issues detected by lgtm.com 2022-01-03 19:45:46 +01:00
jake-87 1b1524e787 Fix poor wording 2022-01-02 19:23:54 +11:00
jake-87 488d81f424 Updated wording in docs
Also added more information on the banned-bulk-import option.
2022-01-02 19:16:23 +11:00
Hornwitser 1ff1e3e5f8 v2.0.0-alpha.11 2021-12-19 15:57:08 +01:00
Hornwitser decee0ecc2 Merge pull request #445 from Hornwitser/log-fix
Fix logs
2021-12-19 15:14:52 +01:00
Hornwitser 170a3d3cf7 Use stream.finished for detecting end of streams
There are some corner cases with streams that cause them to not emit
finish close or end events which in turn means that code liseting for
these events directly may end up hanging forever.  Use stream.finished
to detect end of streams which will handle these cases correctly.
2021-12-19 03:21:00 +01:00
Hornwitser 601ebba919 Replace log query implementation
The interface provided by Winston is unusably slow for querying large
logs.  Reimplement the log querying with both forwards and backwards
line reading and a simple file based index to skip files which do not
contain any lines matching the filter.
2021-12-19 02:06:59 +01:00
Hornwitser fd1f3329bd Split logs by date
Move the default location of the log files into a configurable logs
folder and split the written logs up by UTC date to more easily allow
deleting of old logs to be performed.  The log rotation is implemented
by winston-daily-rotate-file.

The old log format will automatically be migrated to the new format and
location if the target folder for the new logs don't yet exists.
2021-12-19 01:42:34 +01:00
Hornwitser fd72cd4fd5 Fix serialization of burner equipment (#443)
The burner equipment serialization code was never tested and
consequently never worked.  Wasn't noticed before now because burner
equipment isn't in the vanilla game and few mods use it.

Fixes #442.
2021-12-15 16:22:19 +01:00
Hornwitser 200ccf8e06 Fix LogConsole re-requesting logs
Turns out objects in the second argument to useEffect is compared by
Object.is(), this means if the array containing the instances or slaves
to show logs for get re-created during an update the LogConsole
component would send a new request for the logs.

Fix by converting the slave_ids and instance_ids arrays into strings
before passing them on to useEffect.
2021-12-15 00:19:47 +01:00
Hornwitser 285d71e13d Add ReverseLineSplitter
Add splitter based on LineSplitter which splits lines in reverse from a
reverse stream created from the added createReverseReadStream function.
Enables line processing of files starting from the last line and going
towards the first.
2021-12-14 18:44:19 +01:00
Hornwitser 8de76b5cfe Refactor LineSplitter
Pull the LineSplitter class out of lib/factorio to its own stream module
and refactor it to be based on stream.Transform.  This allows composing
with other streams by piping it around.
2021-12-11 21:31:15 +01:00
jake-87 8ecc6d1c83 Added instance and user documentation 2021-12-10 13:01:03 +11:00
Hornwitser 6da7b84757 v2.0.0-alpha.10.c hotfix for installer 2021-12-09 00:14:19 +01:00
Arvids Godjuks 072e95046a Merge pull request #436 from jake-87/patch-1
Wrong capitalisation on `process.env`
2021-12-09 01:04:43 +02:00
Hornwitser 5aa4d99ea3 Fix installer ignoring --plugins argument (#440)
Fix issue passing --plugins to installer which was discovered in #438.
2021-12-09 00:03:45 +01:00
Hornwitser f1ac22881c Use strict parsing in installer (#439)
Bail out if arguments that aren't understood are passed to the
installer script instead of silently ignoring them.
2021-12-08 23:59:40 +01:00
J87 fb85869d09 Fix spelling mistake 2021-12-07 22:13:28 +11:00
J87 c69672ec60 Variable misspelling 2021-12-07 21:42:58 +11:00
J87 3e1a41393a Fix incorrrent spelling of variable
Honestly not sure how this one slipped through
2021-12-07 21:42:10 +11:00
J87 e9225a4b2c Wrong capitalisation on process.env 2021-12-07 21:38:46 +11:00
Arvids Godjuks df80effbdf Merge pull request #434 from Danielv123/small-changes-for-plugins
Small changes for plugins
2021-11-25 01:37:28 +02:00
Arvids Godjuks 35a60e798e Merge pull request #431 from jake-87/user-editable-config
Implementing user editable config for CLI editors
2021-11-25 01:35:14 +02:00
J87 dfd0cb2d36 Fix linter issue 2021-11-25 07:45:56 +11:00
danielv da941a299c Start all and stop all instance buttons 2021-11-24 21:33:55 +01:00
J87 81e3c9e19b Removed unnecessary double negation 2021-11-24 14:31:42 +11:00
J87 b9e8ffa3d1 Fix linting error 2021-11-24 09:33:08 +11:00
J87 62efc63830 Fixed variable definition 2021-11-23 22:03:36 +11:00
J87 e6e5805a49 Refactored serializedConfigToString into function
I'm unable to test right now, so this is currently not yet tested. Will test asap.
2021-11-23 13:12:49 +11:00
Danielv123 a3d98595e2 Fix linter errors 2021-11-22 13:28:29 +01:00
Danielv123 6af34dc3a6 Update BaseConfigTree when props.id changes 2021-11-22 13:17:39 +01:00
Danielv123 1cf3e39d72 Avoid savesProvider throwing errors on missing instanceId 2021-11-22 12:44:48 +01:00
Danielv123 b192c82dcc Export induvidual components from the web interface for plugin use 2021-11-22 10:20:44 +01:00
Danielv123 9138efca6f Clear log console when changing props 2021-11-22 10:20:06 +01:00
Danielv123 52238de9b3 Expose react-router-dom to allow plugins to use useHistory()
See https://githubmemory.com/repo/module-federation/module-federation-examples/issues/716
2021-11-22 10:19:43 +01:00
J87 cc064c2fd1 Fix mistake with Instance descriptions
Also added warning for description not found in master.
2021-11-22 09:36:20 +11:00
J87 baf7dbfe0d Fixed linter issue 2021-11-21 20:26:17 +11:00
J87 0bc6cf1178 Minified getEditor function. 2021-11-21 20:25:33 +11:00
Daniel 8638632600 Merge pull request #425 from Danielv123/fix-invokeHook
Add timeout to invokeHook on plugins
2021-11-21 10:03:30 +01:00
jake87 6d5d9ced47 Refactored editor selection to function 2021-11-21 13:39:23 +11:00
jake87 e825f643e2 Refactored string processing to function
Previously string processing was done the exact same way for both master
and instance; this has now been refactored into a function.
2021-11-21 13:31:12 +11:00
jake87 c4f656996e Fixed linting issues 2021-11-21 12:24:00 +11:00
J87 103c0e3820 Fix missing brace from conflict merge 2021-11-21 12:15:46 +11:00
J87 6da349c888 Fix commented out catch bracket 2021-11-21 12:14:49 +11:00
J87 15c3bb81a6 Fix "emitter" spelling & rename to be more descriptive 2021-11-21 12:12:54 +11:00
J87 4a628df108 Remove unneeded backslash 2021-11-21 12:11:18 +11:00
J87 25ae9b9a0d Add example to "no editor" error 2021-11-21 12:10:24 +11:00
J87 6b10f2ab7d Resolve function style 2021-11-21 12:07:30 +11:00
danielv 770e6a41d4 Add timeout to invokeHook on plugins 2021-11-20 12:44:46 +01:00
J87 375f591fe5 Remove outdated comments 2021-11-20 22:28:21 +11:00
Daniel fcdbf59f7a Merge pull request #433 from Danielv123/fix-node12-support
Support node12 by removing optional chaining
2021-11-20 12:17:34 +01:00
Danielv123 6c80227103 Support node12 by removing optional chaining 2021-11-20 12:03:46 +01:00
Daniel 0016c6271a Merge pull request #427 from Danielv123/gh-actions
Gh actions
2021-11-20 11:50:24 +01:00
Daniel c557b7d6b2 Merge pull request #428 from Hornwitser/fix-localhost-resolve
Fix error when localhost resolves to ::1
2021-11-20 11:49:56 +01:00
J87 64c328c53d Merge branch 'master' into user-editable-config 2021-11-20 16:37:49 +11:00
J87 e5994287fb Remove outdated comments 2021-11-20 16:33:47 +11:00
jake87 7126014619 Added editing for instances
Instances require a number of modifications leading to it not being
able to be converted into a function. The main one is that there is a
list of config elements that cannot be set via the edit command and
either have to be set via different means or cannot be set. To stop
these fields constantly sending error messages, we disregard adding them
to the file in the first place.
2021-11-20 16:23:21 +11:00
Arvids Godjuks 3b8e89ea00 Merge pull request #424 from bbassie/slave-instance-ui-updates
Slave instance UI updates
2021-11-19 21:48:52 +02:00
Danielv123 e72ee3ecaf Added linter to CI action 2021-11-19 17:58:54 +01:00
jake87 31cfc45e58 Fix null processing ( again ) 2021-11-17 21:16:22 +11:00
jake87 834fc8cf98 Fixed null value parsing 2021-11-17 18:16:24 +11:00
jake87 eb3b7df390 Fix to be more consistant with the rest of clustorio
Also made sure "null" is entered as empty as to be consistant with the
rest of clustorio.
2021-11-17 17:45:54 +11:00
jake87 76882ad1c4 Fix styling to be more consistant with the rest of clustorio 2021-11-17 17:42:39 +11:00
bbassie 15381ac260 Final running status color update 2021-11-16 23:50:13 +01:00
bbassie b7e6acb24e Added instance deleted alert and updated not found 2021-11-16 23:44:07 +01:00
bbassie b86305db8c Updated colors of the instance tag 2021-11-16 23:42:02 +01:00
bbassie fe803a99ca Instance status tag color changes 2021-11-16 21:50:21 +01:00
jake87 4612826664 spelling mistake 2021-11-16 21:43:59 +11:00
Hornwitser a5a99e2ab1 Fix error when localhost resolves to ::1
Some systems may resolve localhost to ::1 while Factorio doesn't respond
to RCON connection over IPv6 loopback, this causes instances startup to
fail.  Fix by using 127.0.0.1 instead of localhost.
2021-11-15 21:29:51 +01:00
Danielv123 56584a25c8 Remove old pipeline 2021-11-15 10:26:32 +01:00
Danielv123 03ba7cb68c Added new testing pipeline 2021-11-15 10:20:18 +01:00
jake87 cc605250db removed final debugging helpers 2021-11-14 20:55:16 +11:00
jake87 ae34a86cbf Removed debugging prints, and added removing of tmpFile 2021-11-14 20:54:27 +11:00
jake87 86de5ae151 Fixed config editing, fixed child spawning, fixed config setting
Using an event emmiter, the fields can now be set correctly in the
master config. Null values are now handled correctly, and numbers are also set
correctly.
2021-11-14 20:51:06 +11:00
jake87 bcc25047f2 Readded config file editing, working on config file value setting 2021-11-13 22:35:15 +11:00
bbassie f5ce0a8a8c Missing import 2021-11-07 22:00:27 +01:00
bbassie 55bbc10986 Added slave status indicator tag to slave details page
Co-authored-by: Danielv123 <danielv@live.no>
2021-11-07 21:54:10 +01:00
bbassie 4fef2d8143 Fix message null issues 2021-11-07 21:35:48 +01:00
bbassie 4432a689bb Fix instance list 2021-11-07 21:31:56 +01:00
jake87 6013fcd528 Started rework of file loading system 2021-11-06 17:04:34 +11:00
jake87 71fb8541d5 Added more comments, added quotes around strings 2021-11-06 14:32:13 +11:00
jake87 4398d33fea Fixed definitions to be consistant with bullet points. 2021-11-05 23:32:18 +11:00
jake87 92814aa9b3 Reworked to use config list's grabber system
Also now using the system described by Hornwitser, in using
libConfig.MasterConfig.groups to get definitions. Now all that's needed
is to re-read the updated data back into clustorio.
2021-11-05 23:24:13 +11:00
bbassie 69d4a883ad Add copy public address button 2021-11-04 22:25:53 +01:00
bbassie 2e5617c4b5 Fix instance port and status update
Master was not recieving instance status after restart. Instead only a status from the slave but the slave doesn't store the full instance status.
2021-11-04 22:25:36 +01:00
bbassie f930ed36a0 Changed to using the real factorio port 2021-11-04 21:29:53 +01:00
bbassie 83b720eade Added public_address field to lists
- InstanceList
- SlavesPage
Added public_address field to messages
- slaveProperties
- instanceProperties
2021-11-04 01:39:28 +01:00
bbassie c3a1b420ea Added InstanceStatusTag react component
- Modified InstanceList & InstanceViewPage to use it
2021-11-04 01:36:56 +01:00
Hornwitser c889d8f325 Ignore SessionLost from updateInstances (#422)
Don't shutdown the slave if the connection session is lost during
updateInstances.  Resolves #418.
2021-11-03 16:27:21 +01:00
Hornwitser e23d07e965 Use safe file writing pattern (#421)
Write new file contents to a temporary file and then rename over the
target file to avoid data loss in the unlikely event there's an error
writing the new file contents or the process is interrupted.

Resolves #416.
2021-11-03 16:14:18 +01:00
jake87 08a8182173 Reworked to use throw error, added system for getting config groups
Now using `throw error` to exit out, instead of exit(1). Reworked config
get system to use the map of configs in libConfig.MasterConfig.groups .
2021-11-01 20:40:44 +11:00
Hornwitser 9e7a7a602f Merge pull request #420 from Hornwitser/slave-page
Slave page
2021-10-31 16:53:34 +01:00
Hornwitser 6e5e5f0aeb Show log console for master and slaves in web UI
Rename InstanceConsole to LogConsole and use it to show logs for the
master server and slaves as well.
2021-10-31 10:41:25 +01:00
Hornwitser 7769c1f89e Add SlaveViewPage
Add slave overview page displaying instances assigned and some basic
properties.
2021-10-31 10:41:25 +01:00
Hornwitser a2a5cf6d4c Warn on instances with duplicated ids
It's possible that someone copies an instance folder and don't change
the instance id in it.  Emit a warning in case this happens.
2021-10-31 07:26:56 +01:00
jake87 77e9b929dc Testing why config not being returned correctly 2021-10-30 12:31:20 +11:00
jake87 b6306031dc Reworked some of editor logic to work with async functions
Now async functions can be used after the editor returns, as before they
couldnt be used due to .on being non-async.
2021-10-29 22:21:46 +11:00
jake87 0564eee400 Experementing with the master config
no important work, commit for my own sake
2021-10-29 22:09:01 +11:00
jake87 f548ac2ecc Added more logic into ctl.js
Now prints out a basic approximation of the final look of the config
file; with comments on the function of each part.
2021-10-29 20:42:32 +11:00
jake87 c9fd99589b Added more selection options for the edit
ctl master config edit [editor]
is the new structure; it can take an optional editor, and will prio this
editor over any other editor, such as one set by env vars. Generally,
the prio goes like this: argv supplied editor > EDITOR env var > VISUAL
env var. Things needing to be done are marked with a TODO.
2021-10-27 20:16:37 +11:00
jake87 23d8a819af Added basic framework for edit command
Implimented framework for edit command, including ability to get a temp
file, open an editor on the file, then resume code execution once the
editor exits.
2021-10-26 21:23:23 +11:00
jake87 14b3331c8e Added basic logic for config edit command
Added groundwork for the ctl master config edit command,
including temporary file grabbing and JSON config fetching.
2021-10-26 20:46:24 +11:00
jake87 f266c441db Fixed async logic within lib/file_ops.js 2021-10-26 20:46:24 +11:00
jake87 ed73e3ad1e Fixed variable naming issue with file_ops.getTempFile 2021-10-26 20:46:05 +11:00
jake87 4f9a8fb45c Rewrote temp function to be safer, using findUnusedName 2021-10-26 17:50:39 +11:00
jake87 6910737374 Added function in lib/file_ops.js to return a temp filename with salt 2021-10-25 21:58:56 +11:00
Hornwitser bdfe8e630e v2.0.0-alpha.10.b hotfix for installer 2021-10-05 20:19:58 +02:00
Hornwitser 66719375af Fix installer not setting address and ctl config (#417)
Somehow the step of automatically obtaining the right address to apply
to slave.public_address, and asking for the master url and token for ctl
installations was implemented but the code to actually set it in the
relevant config was forgotten.
2021-10-05 20:05:44 +02:00
508 changed files with 43298 additions and 21974 deletions

No files matched your search

+1 -2
View File
@@ -13,8 +13,7 @@ factorio
factorio.tar.gz
# We build internal mods during build, mods from local test env should not be applied
sharedMods
sharedModsDisabled
mods
# These files should never be required during the build
.git
+82 -26
View File
@@ -1,32 +1,52 @@
"use strict";
module.exports = {
"plugins": [
"@typescript-eslint",
"node",
],
"root": true,
"env": {
"node": true,
"commonjs": true,
"es6": true,
},
"parserOptions": {
"ecmaVersion": 11,
"es2022": true,
},
"overrides": [
{
"files": ["{packages,plugins}/**/*.ts?(x)"],
"parser": "@typescript-eslint/parser",
"extends": [
"plugin:@typescript-eslint/recommended",
],
"rules": {
"@typescript-eslint/no-explicit-any": "off",
"@typescript-eslint/no-useless-constructor": "error",
"@typescript-eslint/no-unused-vars": "off",
"@typescript-eslint/triple-slash-reference": "off",
"prefer-const": "off",
},
},
{
"files": ["test/**/*.js", "plugins/*/test/**/*.js"],
"env": {
"mocha": "true",
},
"parserOptions": { sourceType: "commonjs" },
"rules": {
"prefer-arrow-callback": "off",
"no-invalid-this": "off",
"valid-jsdoc": "off",
"node/no-unpublished-import": "off",
"node/no-unpublished-require": "off",
},
},
{
"files": ["packages/web_ui/src/**/*.jsx", "packages/lib/browser.js", "plugins/*/web/**/*.jsx"],
"files": [
"packages/web_ui/src/**/*.ts?(x)",
"packages/lib/browser.ts",
"packages/create/templates/*/web/**/*.{t,j}s?(x)",
"{packages,plugins}/*/web/**/*.ts?(x)",
],
"env": {
"browser": true,
},
@@ -75,6 +95,19 @@ module.exports = {
"node/shebang": "off",
},
},
{
"files": [
"packages/create/templates/**/*.{t,j}s?(x)",
],
"rules": {
"node/no-missing-require": "off",
"node/no-missing-import": "off",
"spaced-comment": "off",
"comma-spacing": "off",
"max-len": "off",
"indent": "off",
},
},
],
"rules": {
@@ -111,7 +144,7 @@ module.exports = {
"curly": "error",
"default-case": "error",
"default-case-last": "error",
"default-param-last": "error",
"default-param-last": "off",
"dot-location": ["error", "property"],
"dot-notation": ["error", { "allowPattern": "^[a-z]+(_[a-z]+)*$" }],
"eol-last": "error",
@@ -119,7 +152,11 @@ module.exports = {
"func-call-spacing": "error",
"func-name-matching": "error",
"func-names": "off",
"func-style": ["error", "declaration"],
"func-style": ["error", "declaration", {
// Allow arrow functions as it better expresses referencing this
// of the containing scope, which is unweilding in plain functions.
"allowArrowFunctions": true,
}],
"function-call-argument-newline": "off",
"function-paren-newline": "off",
"generator-star-spacing": ["error", { "before": false, "after": true }],
@@ -139,7 +176,7 @@ module.exports = {
"linebreak-style": ["error", "unix"],
"lines-around-comment": "off",
"lines-around-directive": "off",
"lines-between-class-members": ["error", "always"],
"lines-between-class-members": ["error", "always", { "exceptAfterSingleLine": true }],
"max-classes-per-file": "off",
"max-depth": "error",
"max-len": ["error", { "code": 120 }],
@@ -151,7 +188,10 @@ module.exports = {
"max-statements-per-line": "off",
"multiline-comment-style": ["error", "separate-lines"],
"multiline-ternary": "off",
"new-cap": "error",
"new-cap": ["error", {
"capIsNewExceptions": ["StringEnum"],
"capIsNewExceptionPattern": "^Type\\.",
}],
"new-parens": "error",
"newline-after-var": "off",
"newline-before-return": "off",
@@ -162,7 +202,7 @@ module.exports = {
"no-bitwise": "off",
"no-buffer-constructor": "error",
"no-caller": "error",
"no-catch-shadow": "error",
"no-catch-shadow": "off",
"no-confusing-arrow": "error",
"no-console": "error",
"no-constant-condition": ["error", { "checkLoops": false }],
@@ -226,7 +266,7 @@ module.exports = {
"no-script-url": "error",
"no-self-compare": "error",
"no-sequences": "error",
"no-shadow": ["error", { "allow": ["yargs"] }],
"no-shadow": ["error", { "allow": ["Event", "Request", "yargs"] }],
"no-spaced-func": "error",
"no-tabs": ["error", { "allowIndentationTabs": true }],
"no-template-curly-in-string": "error",
@@ -238,12 +278,12 @@ module.exports = {
"no-unmodified-loop-condition": "error",
"no-unneeded-ternary": "error",
"no-unused-expressions": "error",
"no-use-before-define": "error",
"no-use-before-define": "off",
"no-useless-backreference": "error",
"no-useless-call": "error",
"no-useless-computed-key": "error",
"no-useless-concat": "error",
"no-useless-constructor": "error",
"no-useless-constructor": "off",
"no-useless-rename": "error",
"no-useless-return": "error",
"no-var": "error",
@@ -263,6 +303,7 @@ module.exports = {
":": "before",
"||": "before",
"&&": "before",
"??": "before",
},
}],
"padded-blocks": "off",
@@ -293,20 +334,28 @@ module.exports = {
"semi-style": "off",
"sort-imports": "off",
"sort-keys": "off",
"sort-vars": "error",
"sort-vars": "off",
"space-before-blocks": "error",
"space-before-function-paren": "off",
"space-in-parens": ["error", "never"],
"space-infix-ops": "off",
"space-unary-ops": "error",
"spaced-comment": ["error", "always"],
"spaced-comment": ["error", "always", {
"line": {
"markers": ["/"],
},
}],
"strict": "error",
"switch-colon-spacing": "error",
"symbol-description": "error",
"template-curly-spacing": ["error", "never"],
"template-tag-spacing": "error",
"unicode-bom": ["error", "never"],
"valid-jsdoc": ["error", { "requireReturn": false }],
"valid-jsdoc": ["error", {
"requireReturn": false,
"requireReturnType": false,
"requireParamType": false,
}],
"vars-on-top": "error",
"wrap-iife": "error",
"wrap-regex": "off",
@@ -316,14 +365,17 @@ module.exports = {
"node/callback-return": "off",
"node/exports-style": "off",
"node/file-extension-in-import": "off",
"node/global-require": "error",
"node/handle-callback-err": "error",
"node/no-callback-literal": "error",
"node/global-require": "off",
// Node.js callback passing style of (err, value) is not used in this project
"node/handle-callback-err": "off",
"node/no-callback-literal": "off",
"node/no-deprecated-api": "error",
"node/no-exports-assign": "error",
"node/no-extraneous-import": "error",
"node/no-extraneous-require": "error",
"node/no-missing-import": "error",
"node/no-extraneous-import": "off",
"node/no-extraneous-require": "off",
"node/no-missing-import": ["error", {
"tryExtensions": [".js", ".ts"],
}],
"node/no-missing-require": "error",
"node/no-mixed-requires": "error",
"node/no-new-require": "error",
@@ -338,9 +390,9 @@ module.exports = {
"error",
{ "allowModules": ["webpack", "webpack-merge", "webpack-dev-middleware"] },
],
"node/no-unsupported-features/es-builtins": "error",
"node/no-unsupported-features/es-syntax": "error",
"node/no-unsupported-features/node-builtins": "error",
"node/no-unsupported-features/es-builtins": "off",
"node/no-unsupported-features/es-syntax": "off",
"node/no-unsupported-features/node-builtins": "off",
"node/prefer-global/buffer": "error",
"node/prefer-global/console": "error",
"node/prefer-global/process": "error",
@@ -351,6 +403,10 @@ module.exports = {
"node/prefer-promises/dns": "off",
"node/prefer-promises/fs": "off",
"node/process-exit-as-throw": "error",
"node/shebang": "error",
"node/shebang": ["error", {
"convertPath": {
"*.ts": ["^(.+)\\.ts$", "dist/node/$1.js"],
},
}],
},
};
-159
View File
@@ -1,159 +0,0 @@
name: Docker
on:
push:
# Publish `master` as Docker `latest` image.
branches:
- '**'
# Publish `v1.2.3` tags as releases.
tags:
- v*
# Run tests for any PRs.
pull_request:
release:
types: [released, prereleased]
env:
# TODO: Change variable to your image's name.
IMAGE_NAME: clusterio
jobs:
# Build dependencies and clusterio image
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Build docker image
run: DOCKER_BUILDKIT=1 docker build . -t $IMAGE_NAME --target clusterio_final
- name: Export docker image
run: docker save -o clusterio_docker_image $IMAGE_NAME
- name: Upload docker image
uses: actions/upload-artifact@master
with:
name: clusterio_docker_image
path: clusterio_docker_image
# Build dependencies and clusterio image with test tooling
build_testing:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Build docker image
run: DOCKER_BUILDKIT=1 docker build . -t $IMAGE_NAME --target clusterio_testing
- name: Export docker image
run: docker save -o clusterio_docker_image_testing $IMAGE_NAME
- name: Upload docker image
uses: actions/upload-artifact@master
with:
name: clusterio_docker_image_testing
path: clusterio_docker_image_testing
# Upload clusterio_lib and subspace_storage versions
upload_mods:
needs: build
runs-on: ubuntu-latest
steps:
- name: Download docker image
uses: actions/download-artifact@master
with:
name: clusterio_docker_image
path: .
- name: Load docker image
run: docker load -i clusterio_docker_image
- name: Create container
run: docker create --name clusterio_container $IMAGE_NAME
- name: Move mods out of container
run: docker cp clusterio_container:/clusterio/sharedMods ./mods
- name: Remove container
run: docker rm clusterio_container
- name: List extracted mods
run: ls mods
- name: Upload mods
uses: actions/upload-artifact@master
with:
name: mods
path: mods
# Run tests.
# See also https://docs.docker.com/docker-hub/builds/automated-testing/
test:
needs: build_testing
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Download docker image
uses: actions/download-artifact@master
with:
name: clusterio_docker_image_testing
path: .
- name: ls
run: ls && ls clusterio_docker_image_testing && ls /home/runner/work/clusterio && ls /home/runner/work/clusterio/clusterio
- name: Load docker image
run: docker load -i clusterio_docker_image_testing
- name: Run tests
run: |
if [ -f docker-compose.test.yml ]; then
docker-compose --file docker-compose.test.yml run unit_test
else
docker build . --file Dockerfile
fi
# Push image to GitHub Packages.
# See also https://docs.docker.com/docker-hub/builds/
push:
# Ensure test job passes before pushing image.
needs: [test, build]
runs-on: ubuntu-latest
if: github.event_name == 'push'
steps:
- uses: actions/checkout@v2
- name: Download artifact
uses: actions/download-artifact@master
with:
name: clusterio_docker_image
path: .
- name: Load docker image
run: docker load -i clusterio_docker_image
- name: Log into registry
run: echo "${{ secrets.CR_PAT }}" | docker login ghcr.io -u ${{ github.actor }} --password-stdin
- name: Push image
run: |
IMAGE_ID=ghcr.io/${{ github.actor }}/$IMAGE_NAME
# Change all uppercase to lowercase
IMAGE_ID=$(echo $IMAGE_ID | tr '[A-Z]' '[a-z]')
# Strip git ref prefix from version
VERSION=$(echo "${{ github.ref }}" | sed -e 's,.*/\(.*\),\1,')
# Strip "v" prefix from tag name
[[ "${{ github.ref }}" == "refs/tags/"* ]] && VERSION=$(echo $VERSION | sed -e 's/^v//')
# Use Docker `latest` tag convention
[ "$VERSION" == "master" ] && VERSION=latest
echo IMAGE_ID=$IMAGE_ID
echo VERSION=$VERSION
docker tag $IMAGE_NAME $IMAGE_ID:$VERSION
docker push $IMAGE_ID:$VERSION
+52
View File
@@ -0,0 +1,52 @@
# This workflow will do a clean install of node dependencies, cache/restore them, build the source code and run tests across different versions of node
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-nodejs-with-github-actions
name: Node.js CI
on:
push:
pull_request:
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [18.x, 20.x]
# See supported Node.js release schedule at https://nodejs.org/en/about/releases/
steps:
- uses: actions/checkout@v3
- uses: pnpm/action-setup@v2.4.0
with:
version: 8
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node-version }}
- run: pnpm i --no-optional --no-frozen-lockfile
- run: wget -q -O factorio.tar.gz https://www.factorio.com/get-download/latest/headless/linux64 && tar -xf factorio.tar.gz && rm factorio.tar.gz
- name: Run tests
if: ${{ matrix.node-version != '18.x' }}
run: pnpm test
- name: Run coverage
if: ${{ matrix.node-version == '18.x' }}
run: pnpm run ci-cover
- name : Upload coverage to Codecov
if: ${{ matrix.node-version == '18.x' }}
uses: codecov/codecov-action@v2
with:
files: ./coverage/lcov.info
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: pnpm/action-setup@v2.4.0
with:
version: 8
- uses: actions/setup-node@v3
with:
node-version: 18.x
- run: pnpm i --no-optional --no-frozen-lockfile
- run: pnpm run lint
+6 -7
View File
@@ -1,7 +1,6 @@
# Logs
logs
*.log
npm-debug.log*
*debug.log*
# Runtime data
pids
@@ -33,15 +32,15 @@ node_modules
Factorio*
/factorio
/instances
/sharedMods
/mods
/database
/plugin-list.json
/config-master.json
/config-controller.json
/config-control.json
/config-slave.json
package-lock.json
/config-host.json
pnpm-lock.yaml
mod_cache.json
static/export
/static/
saves
/temp/
+1
View File
@@ -0,0 +1 @@
auto-install-peers=false
-15
View File
@@ -1,15 +0,0 @@
language: node_js
node_js:
- 12
before_install:
- curl -o factorio.tar.gz -L https://www.factorio.com/get-download/latest/headless/linux64
- tar -xf factorio.tar.gz
- npm install -g codecov
script:
- npx lerna bootstrap --hoist --no-ci
- node packages/lib/build_mod --source-dir packages/slave/lua/clusterio_lib --output-dir temp/test/sharedMods
- curl -o temp/test/sharedMods/subspace_storage_1.99.8.zip -L https://github.com/clusterio/subspace_storage/releases/download/1.99.6/subspace_storage_1.99.8.zip
- npx lerna run build
- npm run-script ci-cover
- codecov -f coverage/lcov.info
- npm run-script lint
+3
View File
@@ -0,0 +1,3 @@
*
!.gitignore
!settings.json
+11
View File
@@ -0,0 +1,11 @@
{
"editor.detectIndentation": false,
"editor.insertSpaces": false,
"files.eol": "\n",
"files.insertFinalNewline": true,
"files.trimFinalNewlines": true,
"[markdown]": {
"editor.indentSize": 4,
"editor.insertSpaces": true,
},
}
+208 -23
View File
@@ -1,5 +1,192 @@
# Changelog
<!-- Template for entries. Add sections that apply as changes are made.
## Version 2.0.0-alpha.xx
### Major Features
- Major additions and/or major improvements to managing clusters.
### Features
- New features added.
### Changes
- Bugfixes and small changes to existing behaviour.
### Breaking Changes
- Changes which break backwards compatibility with older hosts or plugins code.
Many thanks to the following for contributing to this release:
[@username](https://github.com/username)
-->
## Version 2.0.0-alpha.18
### Features
- Added logging of HTTP requests and errors on the controller [#556](https://github.com/clusterio/clusterio/pull/556).
- Added caching of data parsed from mods to speed up startup time [#606](https://github.com/clusterio/clusterio/pull/606).
- Added command to installer to generate the code for making a new plugin [#570](https://github.com/clusterio/clusterio/pull/570).
### Changes
- Fixed Inventory Sync reporting a negative database size [#598](https://github.com/clusterio/clusterio/pull/598).
- Fixed plugin events being invoked while a host connection was in an invalid state.
- Suppresed bogus warning about event listener leak in the browser's console [#600](https://github.com/clusterio/clusterio/pull/600).
- Fixed installer breaking on Windows after Node.js released a security fix [#614](https://github.com/clusterio/clusterio/pull/614).
- During a data export the mod settings in mod packs will now be corrected to the type of the settings prototype if the type is incorrect.
- Fixed missing color setting causing the Mod Pack view in the Web UI to show an error [#609](https://github.com/clusterio/clusterio/issues/609).
Many thanks to the following for contributing to this release:
[@Cooldude2606](https://github.com/Cooldude2606)
[@Danielv123](https://github.com/Danielv123)
[@Hornwitser](https://github.com/Hornwitser)
## Version 2.0.0-alpha.17
### Features
- Made the host an instance is assigned to link to the host page for that host in the Web UI [#587](https://github.com/clusterio/clusterio/pull/587).
- Added pagination to the saves list.
### Changes
- Fixed an unhandled error in Subspace Storage crashing the host.
- Fixed an unhandled rejection in disconnect logic crashing the host.
- Fixed instance screen in Web UI not updating when an instance was reassigned.
- Fixed an error thrown in the rendering of one page in the Web UI causing all pages to become inaccessible.
- Fixed Web UI not showing instance saves when the permission to list and subscribe the saves is granted but the user is not a cluster admin.
- Similarly fixed Web UI not showing the mod pack create, delete, list and host revoke token actions if the user is not a cluster admin.
- Fixed Research Sync applying progress of a previous techology level to the level on the controller when it's the active research during instance startup.
- Fixed data export with a mod using shorthand notation for the default value of a color-setting causing the ModPack data to be set to invalid data.
- Fixed per user per instance player stats not being deleted when the instance it's for is deleted.
- Fixed mods setting the item prototype icons property to an object mapping instead of an array breaking the data export.
- Fixed a bad mod in the host's mods folder causing instance start to fail instead of the host re-downloading the mod.
### Breaking Changes
- Renamed the permission `core.instance.save.list.subscribe` to `core.instance.save.subscribe`.
Many thanks to the following for contributing to this release:
[@Cooldude2606](https://github.com/Cooldude2606)
[@Danielv123](https://github.com/Danielv123)
[@Hornwitser](https://github.com/Hornwitser)
## Version 2.0.0-alpha.16
### Changes
- Fixed message sequences not being renumbered when forwarded between links causing reconnects to send duplicated messages.
## Version 2.0.0-alpha.15
### Features
- Added support for `color-settings` Mod settings [#568](https://github.com/clusterio/clusterio/issues/568).
- Added `clusterio_controller_pending_requests` and `clusterio_host_pending_requests` metrics.
### Changes
- Fixed generated systemd file for clusteriohost not enabling source maps.
- Fixed ctl not exporting types used for adding commands to ctl plugins.
- Fixed controller only plugins throwing an error about missing a config field on startup out when added to hosts.
- Fixed installer incorrectly claiming the options --factorio-dir and --no-download-headless conflicts with each other.
- Fixed a memory leak on every request sent over the WebSocket.
- Fixed assertion error on closing or invalidating a link with messages forwarded from a virtual link.
- Fixed host hanging if stopped while an instance is in the process of creating a save during startup.
- Fixed `instance.auto_start` option.
- Fixed host log spammed with errors when plugins broadcast messages to instances with the plugin disabled.
- Fixed properties of object config fields not being possible to remove.
## Version 2.0.0-alpha.14
This release does not retain backwards compatibility with 2.0.0-alpha.13 meaning all parts of the cluster needs to updated at the same time.
See the [migration guide](/docs/migrating-from-2.0.0-alpha.13.md) for how to update.
### Major Features
- Added mod management. Mods can be uploaded to the controller and included in mod packs that can be assigned to instances and run.
- Added host port ranges which the instance ports will be automatically assigned a port from if factorio.game_port is not set.
- Added system info such as CPU model, hostname and kernel along with basic metrics for CPU usage, memory usage and disk usage for the controller and hosts.
- Added ability to remotely view and modify the configuration of hosts.
### Features
- Added logging of Clusterio version on startup
- Added `clusterio_controller_log_bytes` metric counting the size of the log files on the controller.
- Added warning logged when plugins on a connecting host does not match the ones installed on the controller.
- Added ability to revoke user tokens.
- Added restart required indicator to config fields in the Web UI.
- Added ability to revoke host tokens.
- Added support for plugins to run in the context of a host.
- Added more feedback when the web UI and or its plugins fails to load.
- Added autosaving of data held in memory on the controller such as configs, instances, users, mods to prevent large data loss on crashing.
- Added `system_info` metric with kernel, instruction set, CPU Model, hostname and Node.js version of the controller.
- Added `clusterio_host_system_info` metric with kernel, instruction set, CPU Model, hostname and Node.js version of hosts.
- Added `system_cpu_seconds_total`, `system_cpu_idle_seconds_total`, `system_memory_capacity_bytes`, `system_memory_available_bytes`, `system_disk_capacity_bytes`, `system_disk_available_bytes` metrics for CPU, memory and disk usage on the controller.
- Added `clusterio_host_system_cpu_seconds_total`, `clusterio_host_system_cpu_idle_seconds_total`, `clusterio_host_system_memory_capacity_bytes`, `clusterio_host_system_memory_available_bytes`, `clusterio_host_system_disk_capacity_bytes`, `clusterio_host_system_disk_available_bytes` metrics for CPU, memory and disk usage on hosts.
- Added support for reading the client's IP address from the `X-Forwarded-For` header when the source address is in the `controller.trusted_proxies` config option.
- Added field showing the IP address a host is connecting from.
- Added per player statistic tracking when they were first seen joining a particular server.
- Added ability to remotely restart and/or stop the controller and hosts.
### Changes
- Fixed filenames with extensions that are invalid on Windows such as "con.zip" not being treated as invalid.
- Fixed generate host token modal not using a random id when the optional id field is unset.
- Exported Factorio data is now tied to mod packs instead of being per cluster.
- Fixed export failing if a mod uses a different root folder mod_version in its zip file.
- Fixed sorting of user last seen column in the Web UI.
- The list users permission is now granted by default to players.
- Fixed incorrect handling of hosts and instance with an id of 0 causing them to not update properly.
- Fixed unable to unassign instances in the Web UI.
- Fixed host crashing if the Factorio server is not marked executable.
- Fixed errors logged from instance not being attributed to the instance in certain situations.
- Hosts no longer print messages from instances into their console.
- Clusterio now treat unhandled Node.js errors and promise rejections as fatal and exit immediately.
- Fixed a potential for data loss after a hard system crash due to fsync not being called on files written.
- Fixed Factorio server crashing in the research_sync plugin when a mod enables a previously disabled technology and a player attempts to research it.
- Fixed Factorio server crashing when the game starts with tick_paused = true
- Removed migration of alpha 10's single file log format.
- Log errors linking mods on Windows instead of preventing startup.
- Added locale support for modules patched into to the game save.
- Added support for defining a module_exports.lua file for modules patched into the game save.
- Fixed controller logs being split on server's local time day boundary instead of UTC day boundary.
- Fixed Unicode BOM in mod locale files breaking instance data export.
- Added a warning to the server-setting.json that is automatically written to instance folders stating changes will be overwritten.
- Changed save patching to use a `.tmp.zip` extension for the temporary file created instead of `zip.tmp`.
- Fixed Inventory Sync wiping the player's inventory when logging out while in the Space Exploration remote view mode.
- Fixed the public /api/plugins endponit leaking filesystem paths on the controller.
- Refactored list subscriptions in the Web UI to cache and only fetch updated entries when navigating to previously seen list.
- Added caching of save lists on the controller, enabling the saves of instances on offline hosts to still be seen.
- Hosts in the assign instance dropdown is now sorted by name.
- Added `non_blocking_saving` as one of the default settings added to `factorio.settings` on new instances.
### Breaking Changes
- Renamed usage of the terms master and slave to controller and host in all places they occur.
- Mods located in `sharedMods` are no longer copied to instances' mods folder.
- The instances' mods folder is now managed through mod packs, mods not part of the mod pack will be automatically deleted.
- Removed ability to import sub packages like `@clusterio/lib/link` from.
- Replaced `clusterio_controller_http_hits_total` metric with `clusterio_controller_http_endpoint_duration_seconds`.
- Replaced `clusterio_controller_last_query_log_duration_seconds` metric with `clusterio_controller_query_log_duration_seconds` summary.
- Reworked the network protocol to be simpler to work and reason with.
- Bumped minimum supported Node.js version to v18.
- Ported the codebase to TypeScript
- Changed the config format stored on disk to be much simpler.
### Contributors
Many thanks to the following for contributing code in this release:
[@Hornwitser](https://github.com/Hornwitser) [@Luciole0x](https://github.com/Luciole0x) [@Cooldude2606](https://github.com/Cooldude2606) [@Danielv123](https://github.com/Danielv123) [@Reavershark](https://github.com/Reavershark) [@Laar](https://github.com/Laar) [@rstrom1763](https://github.com/rstrom1763)
## Version 2.0.0
### Major Features
@@ -8,25 +195,25 @@
Removed the Hotpatch scenario and the depency on it for getting code into the game.
Added a save patcher than runs before starting up Factorio that patches in lua modules based on the event_loader lib into the savegame.
Regular freeplay games can now be used with Clusterio and will be compatible without having to convert them to Hotpatch.
- Daemonized slaves.
Slaves now have the ability to run multiple Factorio instances, starting and stopping them individually.
To manage this the local command line interface has been replaced with a remote interface on the master server that can be accessed through the clusterioctl cli tool.
- Rewritten the communication between slaves and the master from scratch.
The new system is based on a WebSocket connection between the slaves and the master server and provides efficient validated bi-directonal communication.
- Daemonized hosts.
Hosts now have the ability to run multiple Factorio instances, starting and stopping them individually.
To manage this the local command line interface has been replaced with a remote interface on the controller that can be accessed through the clusterioctl cli tool.
- Rewritten the communication between hosts and the controller from scratch.
The new system is based on a WebSocket connection between the hosts and the controller and provides efficient validated bi-directonal communication.
- Rewritten the plugin system from scratch.
Plugins now inherit from a base class and use the same WebSocket connection Clusterio uses to communicate.
- New configuration system with support for initializing configs when needed, modifying config entries using built-in commands, and updating instance configuration remotely.
- Packaged and uploaded to npm again, greatly simplifying distribution and installation of both Clusterio and plugins.
- Added centralized logging for the cluster.
All logs from both Clusterio and Factorio is stored in a shared log which can be inspected and queried.
- Added remote save management with the ability to list, create, upload, download, delete and load saves for instances.
- Added remote save management with the ability to list, create, rename, copy, upload, download, transfer, delete and load saves for instances.
### Features
- Added export of pollution statitics.
- Reconnection logic that esures no data is dropped talking to the master server provided the session can be resumed.
- Reconnection logic that esures no data is dropped talking to the controller provided the session can be resumed.
- Added support for having multiple Factorio installs and selecting which version to use on a per instance basis.
- Added config to specify which path the master interface is accessed under, allowing it to be proxied behind web-server.
- Added config to specify which path the controller interface is accessed under, allowing it to be proxied behind web-server.
- Added WebSocket usage statistics.
- Added commands sent statistic.
- Added Factorio server CPU and memory usage statistic.
@@ -37,18 +224,18 @@
Previously this was handled by the playerManager plugin.
- Added stripping of long paths in the Factorio server log.
- Added option to configure maximum number of commands to send in parallel.
- Added option to auto start instances on slave startup.
- Added metric mapping of slave and instance ids to their names, allowing them to be displayed by name in queries that join with the mapping.
- Added option to auto start instances on host startup.
- Added metric mapping of host and instance ids to their names, allowing them to be displayed by name in queries that join with the mapping.
- Added option to configure timeout for exported Prometheus metrics.
- Added option to set address to bind HTTP(S) port to.
- Added an overview of the installed plugins to the web interface.
- Added clusterioctl command to list plugins on the master server.
- Added clusterioctl command to list plugins on the controller.
- Added Player Auth plugin for logging in to the web interface by proving the ability to log in to a server as a given user.
- Added Node.js based installer to simplify setting up a cluster.
### Changes
- Added error handling during master startup.
- Added error handling during controller startup.
- Factorio game and rcon port now defaults to a random port above 49151.
- Removed unimplemented mods update command.
- Fixed rcon password being generated with Math.random().
@@ -69,14 +256,14 @@
- Removed factorio_version from config.
The version installed is auto detected and used instead.
- Removed the playerManager specific command CLI tools/delete_player.js.
- Creating an instance, assigning it to a slave, creating a save for an instance and starting an instance is now four separate commands.
- Removed oddball limits to slaves.json size.
- Moved slave specific and instance specific configuration into their own configuration files.
- Creating an instance, assigning it to a host, creating a save for an instance and starting an instance is now four separate commands.
- Removed oddball limits to hosts.json size.
- Moved host specific and instance specific configuration into their own configuration files.
- Removed unused binary option from plugin config.
- Removed info and shout command from globalChat plugin - Removed mirrorAllChat and enableCrossServerShout configuration options for globalChat plugin.
- Removed UPSdisplay plugin.
UPS statistics is exported by the statistics exporter plugin.
- Master server now defaults to hosting on https on port 8443.
- Controller now defaults to hosting on https on port 8443.
- Renamed client to slave.
- Removed rotation of factorio-current.log files.
@@ -86,7 +273,6 @@
Breaks playerManager.
- The masterIP and masterPort config entries has been merged into masterURL.
Breaks discordChat, playerManager, serverSelect, trainTeleports and clusterioModel.
- Instance names can no longer be invalid Windows file names.
- Removed config management from the command line and the server manager.
- Moved ejs templates into views folder and changed their extension to .ejs.
Breaks playerManager.
@@ -101,7 +287,7 @@
- Removed the remoteCommands plugin and the old runCommand interface.
Breaks playerManager and external tools sending commands.
- Removed broken serverManager plugin.
- Removed fields info, time, rconPort, rconPassword, serverPort, unique, mods, instanceName, playerCount, mac, and meta from the slaves in the slave database.
- Removed fields info, time, rconPort, rconPassword, serverPort, unique, mods, instanceName, playerCount, mac, and meta from the hosts in the host database.
- Removed getInstanceName and lib/clusterTools.
Breaks playerManager, and discordChat.
- Removed the /api/rconPasswords, and /api/slaves endpoints.
@@ -110,7 +296,7 @@
Breaks researchSync, and UPSdisplay.
- Removed the hello event from the socket connection handshake.
Breaks playerManager, trainTeleports, serverSelect, and discordChat
- Changed the format of database/slaves.json.
- Changed the format of database/hosts.json.
- Removed the output file subscription system.
Breaks inventoryImports, playerManager, trainTeleports, serverSelect and researchSync.
- Removed the factorioOutput hook from instance plugins.
@@ -135,17 +321,16 @@
- Removed msBetweenCommands config option.
The RCON is instead limited to 5 concurrent commands.
- Removed allowRemoteCommandExecution config option.
Remote commands are always allowed with the move to master managing slaves/instances.
- Removed `--databaseDirectory`, `--masterPort`, and `--sslPort` command line arguments from the master server.
Remote commands are always allowed with the move to controller managing hosts/instances.
- Removed `--databaseDirectory`, `--controllerPort`, and `--sslPort` command line arguments from the controller.
- Implemented a new config system that replaces the old.
Breaks all plugins.
- Removed usage of socket.io entirely in favor of a plain WebSocket connection.
- Renamed clusterioMod plugin to subspace_storage.
- Master no longer creates secret-api-token.txt.
- Controller no longer creates secret-api-token.txt.
- Removed automatic creation of self-signed TLS certificate.
- Node.js versions below 12 are no longer supported.
## Version 1.2.4
- Removed broken remote combinator signaling.
+13 -18
View File
@@ -1,4 +1,4 @@
FROM node:12 as subspace_storage_builder
FROM node:20 as subspace_storage_builder
RUN apt update && apt install -y git
WORKDIR /
RUN git clone https://github.com/clusterio/subspace_storage.git
@@ -6,7 +6,7 @@ WORKDIR /subspace_storage
RUN npm install \
&& node build
FROM node:12 as clusterio_builder
FROM node:20 as clusterio_builder
RUN apt update \
&& apt install -y wget \
&& mkdir /clusterio
@@ -15,25 +15,22 @@ RUN wget -q -O factorio.tar.gz https://www.factorio.com/get-download/latest/head
# Copy project files into the container
COPY . .
RUN npm install \
&& npx lerna bootstrap --hoist
RUN pnpm install
# Install plugins. This is intended as a reasonable default, enabling plugins to make for fun gameplay.
# If you want a different set of plugins, consider using this as the base image for your own.
#RUN npm install @clusterio/plugin-subspace_storage
#RUN npx clusteriomaster plugin add @clusterio/plugin-subspace_storage
#RUN pnpm install @clusterio/plugin-subspace_storage
#RUN npx clusteriocontroller plugin add @clusterio/plugin-subspace_storage
COPY --from=subspace_storage_builder /subspace_storage/dist/ /clusterio/sharedMods/
COPY --from=subspace_storage_builder /subspace_storage/dist/ /clusterio/mods/
# Build submodules (web UI, libraries, plugins etc)
RUN npx lerna run build
# Build Lua library mod
RUN node packages/lib/build_mod --build --source-dir packages/slave/lua/clusterio_lib \
&& mv dist/* sharedMods/ \
RUN node packages/lib/build_mod --build --source-dir packages/host/lua/clusterio_lib \
&& mv dist/* mods/ \
&& mkdir temp \
&& mkdir temp/test \
&& cp sharedMods/ temp/test/ -r \
&& ls sharedMods
&& cp mods/ temp/test/ -r \
&& ls mods
# Remove node_modules
RUN find . -name 'node_modules' -type d -prune -print -exec rm -rf '{}' \;
@@ -46,8 +43,7 @@ COPY --from=clusterio_builder /clusterio /clusterio
WORKDIR /clusterio
# Install runtime dependencies
RUN npm install --production
RUN npx lerna bootstrap -- --production --no-optional
RUN pnpm install --production
LABEL maintainer "danielv@danielv.no"
FROM frolvlad/alpine-glibc AS clusterio_testing
@@ -58,6 +54,5 @@ COPY --from=clusterio_builder /clusterio /clusterio
WORKDIR /clusterio
# Install runtime dependencies
RUN npm install
RUN npx lerna bootstrap
RUN npm install chalk semver
RUN pnpm install
RUN pnpm install chalk semver
+77 -65
View File
@@ -12,7 +12,7 @@ Discord for development/support/play: https://discord.gg/5XuDkje
Clusterio 2.0 is still in alpha, however the previous stable has been abandoned and is no longer supported.
Despite being alpha it's reasonably stable now and there's no major breakages expected before a stable release of 2.0.
If you are starting a new cluster it's highly recommended to use the 2.0 alpha.
If you are starting a new cluster, it's highly recommended to use the 2.0 alpha.
### Ways to support me/the project:
@@ -33,12 +33,13 @@ If you are starting a new cluster it's highly recommended to use the 2.0 alpha.
* [Windows setup](#windows-setup)
* [MacOS setup](#macos-setup)
* [Installing Plugins](#installing-plugins)
* [Configure Master Server](#configure-master-server)
* [Configure Controller](#configure-controller)
* [Managing Factorio mods](#managing-factorio-mods)
* [Setting up shared storage](#setting-up-shared-storage)
* [Running Clusterio](#running-clusterio)
* [Setting up remote slaves](#setting-up-remote-slaves)
* [Setting up remote hosts](#setting-up-remote-hosts)
* [Setting up clusterioctl](#setting-up-clusterioctl)
* [Known issues](#known-issues)
* [Common problems](#Common-problems)
@@ -49,7 +50,7 @@ It was previously best known for implementing cross server transfer and cloud st
But this functionality has been pulled out of Clusterio into its own plugin for Clusterio named [Subspace Storage](https://github.com/clusterio/subspace_storage).
By itself Clusterio doesn't change the gameplay in any way, you could even use Clusterio to manage completely vanilla Factorio servers.
Plugins do the work of modding in the visible changes into the game, see the [Plugins section](#plugins) for ready made plugins you can install into a Clusterio cluster.
Plugins do the work of modding in the visible changes into the game, see the [Plugins section](#plugins) for ready-made plugins you can install into a Clusterio cluster.
## Features
@@ -71,7 +72,7 @@ These are the plugins supported and maintained by the Clusterio developers:
- [Global Chat](/plugins/global_chat/README.md): share the in-game chat between servers.
- [Research Sync](/plugins/research_sync/README.md): synchronize research progress and technologies unlocked between servers.
- [Statistics Exporter](/plugins/statistics_exporter/README.md): collect in-game statistics from all the servers and makes it available to the Prometheus endpoint on the master server.
- [Statistics Exporter](/plugins/statistics_exporter/README.md): collect in-game statistics from all the servers and makes it available to the Prometheus endpoint on the controller.
- [Subspace Storage](https://github.com/clusterio/subspace_storage): Provide shared storage that can transport items between servers via teleport chests.
- [Player Auth](/plugins/player_auth/README.md): Provides authentication to the cluster via logging into a Factorio server.
- [Inventory Sync](/plugins/inventory_sync/README.md): Synchronizes your inventory between servers.
@@ -80,7 +81,7 @@ There's also plugins developed and maintained by the community:
- [Discord Bridge](https://github.com/Hornwitser/discord_bridge) (@hornwitser/discord_bridge): Bridges chat between instances and Discord.
By Hornwitser.
- [Server Select](https://github.com/Hornwitser/server_select/tree/clusterio-2.0) (@hornwitser/server_select): In-game GUI for connecting to other server in the cluster.
- [Server Select](https://github.com/Hornwitser/server_select/tree/clusterio-2.0) (@hornwitser/server_select): In-game GUI for connecting to other servers in the cluster.
Originally by Godmave, ported to 2.0 by Hornwitser.
Want to make your own plugin?
@@ -89,8 +90,8 @@ Check out the documentation on [Writing Plugins](/docs/writing-plugins.md) for w
## Installation
Clusterio runs on Node.js v12 and up, it's distributed via npm and comes with a guided install script.
If you alread have a recent Node.js installed you can set it up in a new directory with.
Clusterio runs on Node.js v18 and up, it's distributed via npm and comes with a guided install script.
If you already have a recent Node.js installed, you can set it up in a new directory with:
npm init "@clusterio"
@@ -99,13 +100,13 @@ Otherwise see below for OS specific instructions.
### Ubuntu setup
1. Install Node.js v12 or higher.
For 20.04 LTS or below the version of Node.js provided by the Ubuntu repos are too old and you will have to use the nodesource PPA, otherwise you may skip the first line.
1. Install Node.js v18 or higher.
Note the version of Node.js provided by the Ubuntu/debian repos are too old and you will have to use the nodesource PPA
wget -qO - https://deb.nodesource.com/setup_14.x | sudo -E bash -
sudo apt install nodejs
curl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash - &&\
sudo apt-get install -y nodejs
2. Create a new directory and run the Clusterio installer:
2. Log in as the user you wish to install Clusterio as (do not use root), and then create a new directory and run the Clusterio installer:
mkdir clusterio
cd clusterio
@@ -113,19 +114,12 @@ Otherwise see below for OS specific instructions.
Make sure to note down the admin authentication token it provides at the end as you will need it later.
3. If you chose to use local factorio directory for the Factorio installation then download the headless build of Factorio and unpack it:
wget -O factorio.tar.xz https://www.factorio.com/get-download/latest/headless/linux64
tar -xf factorio.tar.xz
To specify a version of Factorio to download replace "latest" in the URL with a version number like "1.0.0".
4. Optionally copy the generated systemd service files in `systemd` folder to `/etc/systemd/system/`.
3. Optionally copy the generated systemd service files in `systemd` folder to `/etc/systemd/system/`.
**Ubuntu with Docker**
The Docker support for Clusterio is curently broken.
The Docker support for Clusterio is currently broken.
If you need it open an issue about it.
<!--
@@ -133,9 +127,9 @@ Clusterio has *very* limited support for using docker.
sudo docker build -t clusterio --no-cache --force-rm clusterio
sudo docker run --name master -e MODE=master -p 1234:8080 -d -it --restart=unless-stopped danielvestol/clusterio
sudo docker run --name controller -e MODE=controller -p 1234:8080 -d -it --restart=unless-stopped danielvestol/clusterio
sudo docker run --name slave -e MODE=slave -e INSTANCE=world1 -v /srv/clusterio/instances:/clusterio/instances -p 1235:34167 -it --restart=unless-stopped danielvestol/clusterio
sudo docker run --name host -e MODE=host -e INSTANCE=world1 -v /srv/clusterio/instances:/clusterio/instances -p 1235:34167 -it --restart=unless-stopped danielvestol/clusterio
The -v flag is used to specify the instance directory.
Your instances (save files etc) will be stored there.
@@ -152,7 +146,7 @@ Your instances (save files etc) will be stored there.
Make sure to note down the admin authentication token it provides at the end as you will need it later.
3. If you chose to use local factorio directory for the Factorio installation then download the Windows 64-bit zip package from https://www.factorio.com/download and extract it to the `factorio` folder in your clusterio installation folder.
3. If you chose to use local factorio directory for the Factorio installation, download the Windows 64-bit zip package from https://www.factorio.com/download and extract it to the `factorio` folder in your clusterio installation folder.
### MacOS Setup
@@ -167,12 +161,12 @@ Your instances (save files etc) will be stored there.
Make sure to note down the admin authentication token it provides at the end as you will need it later.
3. If you chose to use local factorio directory for the Factorio installation you will need to obtain and copy a mac version of Factorio and unpack it to to the `factorio` folder in your clusterio installation folder.
3. If you chose to use local factorio directory for the Factorio installation, you will need to obtain and copy a mac version of Factorio and unpack it to the `factorio` folder in your clusterio installation folder.
### Installing Plugins
For well known plugins you can select them during installation and no further steps are necessary to make them work.
For well-known plugins you can select them during installation and no further steps are necessary to make them work.
Installing plugins that are not offered by the installer consists of two steps.
First install the package the plugin is provided by via npm, for example
@@ -180,96 +174,114 @@ First install the package the plugin is provided by via npm, for example
Then tell clusterio that this plugin exists by adding it as a plugin.
npx clusteriomaster plugin add @clusterio/plugin-subspace_storage
npx clusteriocontroller plugin add @clusterio/plugin-subspace_storage
This adds it to the default `plugin-list.json` file which in the shared folder setup is loaded by master, slave and ctl.
If you have slaves or ctl installed on separate computers (or directories) then you need to repeat the plugin install process for all of them.
The clusteriomaster, clusterioslave and clusterioctl commands has the plugin sub-command so you do not need to install clusteriomaster to add plugins, instead use the clusterio command you have available.
This adds it to the default `plugin-list.json` file which in the shared folder setup is loaded by controller, host and ctl.
If you have hosts or ctl installed on separate computers (or directories) then you need to repeat the plugin install process for all of them.
The clusteriocontroller, clusteriohost and clusterioctl commands has the plugin sub-command so you do not need to install clusteriocontroller to add plugins, instead use the clusterio command you have available.
For development purposes the `plugin add` command supports adding plugins by the absolute path to them, or a relative path that must start with either . or .. (which will then be resolved to an absolute path).
## Configure Master Server
## Configure Controller
By default the master server will listen for HTTP on port 8080.
By default the controller will listen for HTTP on port 8080.
You can change the port used with the command
npx clusteriomaster config set master.http_port 1234
npx clusteriocontroller config set controller.http_port 1234
When changing the port you will also need to change the address slaves connect with.
When changing the port you will also need to change the address hosts connect with.
For the standalone installation mode you can use
npx clusterioslave config set slave.master_url http://localhost:1234/
npx clusteriohost config set host.controller_url http://localhost:1234/
If you plan to make your cluster available externally set the address
that it will be accessible under with, for example
npx clusteriomaster config set master.external_address http://203.0.113.4:1234/
npx clusteriocontroller config set controller.external_address http://203.0.113.4:1234/
Change the url to reflect the IP, protocol, and port the master server is accessible under, dns names are also supported.
If you're planning on making the master server accessible on the internet it's recommended to set up TLS, see the [Setting Up TLS](/docs/setting-up-tls.md) document for more details.
Change the url to reflect the IP, protocol, and port the controller is accessible under, dns names are also supported.
If you're planning on making the controller accessible on the internet it's recommended to set up TLS, see the [Setting Up TLS](/docs/setting-up-tls.md) document for more details.
You can list the config of the master server with the `npx clusteriomaster config list` command.
See the [readme for @clusterio/master](/packages/master/README.md) for more information.
You can list the config of the controller with the `npx clusteriocontroller config list` command.
See the [readme for @clusterio/controller](/packages/controller/README.md) for more information.
## Managing Factorio mods
Clusterio currently only provides a manual method for running mods on servers by copying them either to each instance's individual `mods` folder, or by copying them into a slave installation's `sharedMods` folder which will link/copy them to the `mods` folder of each instance on that slave if the file doesn't already exist in the `mods` folder for that instance.
Clusterio currently only provides a manual method for running mods on servers by copying them either to each instance's individual `mods` folder, or by copying them into a host installation's `sharedMods` folder which will link/copy them to the `mods` folder of each instance on that host if the file doesn't already exist in the `mods` folder for that instance.
Mods are copied from `sharedMods` on Windows and symlinked on Linux, this means that when mods are removed from `sharedMods` the mods will also be removed from the instance's `mods` folder on Linux if it got there by being linked by Clusterio, but not on Windows.
For managing the mod settings the `mod-settings.dat` and `mod-list.json` files will also be linked/copied over if present in `sharedMods`, but since Factorio creates these files automatically it might be necessary to delete them from each instance's `mods` folder for changes to them in `sharedMods` to take effect.
For managing the mod settings, the `mod-settings.dat` and `mod-list.json` files will also be linked/copied over if present in `sharedMods`, but since Factorio creates these files automatically it might be necessary to delete them from each instance's `mods` folder for changes to them in `sharedMods` to take effect.
Better remote management of mods is a planned feature.
## Setting up shared storage
The chests teleporting items to and from the shared storage on the master server has been moved into the Subspace Storage mod and plugin.
To get this to work you will need to install the Subspace Storage plugin and copy the [Subspace Storage mod](https://mods.factorio.com/mod/subspace_storage) and the [Clusterio Lib mod](https://mods.factorio.com/mod/clusterio_lib) into the `sharedMods` folder on each slave installation.
The chests teleporting items to and from the shared storage on the controller has been moved into the Subspace Storage mod and plugin.
To get this to work you will need to install the Subspace Storage plugin and copy the [Subspace Storage mod](https://mods.factorio.com/mod/subspace_storage) and the [Clusterio Lib mod](https://mods.factorio.com/mod/clusterio_lib) into the `sharedMods` folder on each host installation.
Do not install the [Clusterio mod](https://mods.factorio.com/mod/clusterio), this was for the old 1.2.x version of Clusterio and is not compatible with Clusterio 2.0.
## Running Clusterio
After completing the installation start up the master server and at least one slave separately.
The installer provides the `run-master` and `run-slave` scripts to make this simple.
Once the master process is running you can log into the web interface which is hosted by default on http://localhost:8080/ (adjust the port number if you changed it), use the admin authentication token provided from the installation to log in.
After completing the installation start up the controller and at least one host separately.
The installer provides the `run-controller` and `run-host` scripts to make this simple.
Once the controller process is running you can log into the web interface which is hosted by default on http://localhost:8080/ (adjust the port number if you changed it), use the admin authentication token provided from the installation to log in.
The basics of setting up a Factorio server from the web interface is to create an instance, assign it to a slave and then click start.
The basics of setting up a Factorio server from the web interface is to create an instance, assign it to a host and then click start.
### Running via Systemd
The install script creates systemd service scripts for clusteriomaster and clusterioslave (if applicable) to start up as the user than ran the installer.
If you have copied these files over to `/etc/systemd/system/` then you can startup clusterio as a background service using:
The install script creates systemd service scripts for clusteriocontroller and clusteriohost (if applicable) to start up as the user than ran the installer.
If you have copied these files over to `/etc/systemd/system/` then you can startup Clusterio as a background service using:
#### To start as a service run:
sudo systemctl start clusteriomaster # for the master server
sudo systemctl start clusterioslave # for each server hosting a slave
sudo systemctl start clusteriocontroller # for the controller
sudo systemctl start clusteriohost # for each physical server intended to host Factorio servers
#### To automatically get it to start on boot:
sudo systemctl enable clusteriomaster # for the master server
sudo systemctl enable clusterioslave # for each server hosting a slave
sudo systemctl enable clusteriocontroller # for the controller
sudo systemctl enable clusteriohost # for each physical server intended to host Factorio servers
## Setting up remote slaves
## Setting up remote hosts
Run the installer as described in the installation section and choose "Slave only" as the operating mode to install, it'll ask for a master URL and an authentication token.
The URL is the same as what is needed to connect to the web interface, and the athentication token can be generated on the Slaves page in the web interface.
Once you start up the slave it should show up in the Slaves list and be available for assigning and running instances on.
Run the installer as described in the installation section and choose "Host only" as the operating mode to install, it'll ask for a controller URL and an authentication token.
The URL is the same as what is needed to connect to the web interface, and the authentication token can be generated on the Hosts page in the web interface.
Once you start up the host it should show up in the Hosts list and be available for assigning and running instances on.
## Setting up clusterioctl
There's a command line interface available for Clusterio which is installed separately with the same installer as for the master and slave.
Run the installer as described in the installation section and choose "Ctl only" as the operating mode to install, you can do this in the same directory as you have installed other clusterio component(s) to.
The installer will ask for a master URL and an authentication token, these are the same as you would use to connect to the web interface.
If you want to use a different user for the command line interface you can generate an authentication token for an existing user with
There's a command line interface available for Clusterio which is installed separately with the same installer as for the controller and host.
Run the installer as described in the installation section and choose "Ctl only" as the operating mode to install, you can do this in the same directory as you have installed other Clusterio component(s) to.
The installer will ask for a controller URL and an authentication token, these are the same as you would use to connect to the web interface.
If you want to use a different user for the command line interface, you can generate an authentication token for an existing user with
npx clusteriomaster bootstrap generate-user-token <username>
npx clusteriocontroller bootstrap generate-user-token <username>
## Known Issues
- When the `server_select` plugin is installed you may see `[error] Unhandled event server_select:UpdateInstancesEvent` logged when starting an instance.
This should be ignored as it does not indicate an error occured.
- After updating Factorio you may get the error `Error: Expected empty response but got "Cannot execute command. Error: [string "clusterio_private.update_instance(..."]:1: attempt to index global 'clusterio_private' (a nil value)?` on instance start.
This is due to Factorio applying a migration replacing the patched content of the save with the original scenario, attempting to start the instance again should resolve the issue.
- In the mod pack view of the web interface the "base" mod is displayed with an error complaining about the mod being missing from the controller storage.
This is due to the "base" mod currently not being connectly handled as a built-in mod to Factorio in the web interface, this error message should be ignored.
- After migrating from Alpha 13 to Alpha 14 running an instance data export fails with the error `Upload failed: 401 Unauthorized` due to the host token being issued to a slave.
To workaround this find the id of the host in the host config and generate a new host token with the same id.
Stop the host and replace the value of the `host.controller_token` field in `config-host.json` with the new host token before starting it up again.
- After migrating from Alpha 13 to Alpha 14 you get kicked out from the web interface due to the master to controller rename.
If you don't have the authentication token readily available you can manually migrate the token by opening the Inspect/Developer Tools on the page and running the following piece of JavaScript in the Console:
```
localStorage.controller_token = localStorage.master_token
```
## Common problems
@@ -283,7 +295,7 @@ According to [this link](https://askubuntu.com/questions/839520/open-port-443-fo
sudo setcap 'cap_net_bind_service=+ep' $(readlink -f $(which node))
### Portforwarding doesn't work on the master server when running under WSL
### Port forwarding doesn't work on the controller when running under WSL
If you follow the ubuntu guide on WSL (Windows Subsystem for Linux, Bash on Ubuntu on Windows specifically), you will find that the website works on localhost and on your local ip, but not on the global ip.
This is also true when you correctly port-forwarded the correct ports.
+6
View File
@@ -0,0 +1,6 @@
coverage:
status:
project:
default:
if_ci_failed: success
informational: true
+4 -4
View File
@@ -1,9 +1,9 @@
version: '3'
services:
master:
controller:
build: .
command: npx clusteriomaster run
slave:
command: npx clusteriocontroller run
host:
build: .
command: npx clusterioslave run
command: npx clusteriohost run
+2 -2
View File
@@ -11,8 +11,8 @@ Links represents one side of a two way communication pipe, connectors deal with
The Link class represents one side of a two way communication channel where the actual communication is done by a connector associated with the link.
The link also knows what the endpoint types it's connected to are via the source and target attributes.
The endpoint types currently implemented are master, slave, instance, and control.
Although slave and instances run in the same Node.js program they use virtual links to communicate between them in order to simplify the communication architecture.
The endpoint types currently implemented are controller, host, instance, and control.
Although host and instances run in the same Node.js program they use virtual links to communicate between them in order to simplify the communication architecture.
Upon creation the link will register with the connector in order to receive and process messages from it.
The messages received are validated using the validator registered for the type of the message.
+22 -24
View File
@@ -2,36 +2,34 @@
<sub>This document describes the implementation and inner working of the configuration system in Clusterio, and unless you're developing Clusterio it's probably not going to be very useful for you.</sub>
Clusterio uses a group based configuration system to manage settings for the master server, instances, slaves, and the clusterioctl utility.
Plugins get their own config group for the master server and instances which they can add their own fields to.
The fields are defined early in the startup, and the groups and their fields for disabled plugins will still be created.
Clusterio uses a field based configuration system to manage settings for the controller, instances, hosts, and the clusterioctl utility.
Plugins can add their own config fields for the controller and instances configs.
The fields are defined early in the startup, and fields for disabled plugins will still be created.
The built-in config groups and their fields are defined in [packages/lib/config/definitions.js](/packages/lib/config/definitions.js) while plugins can add their own config groups to the `MasterConfigGroup` and `InstanceConfigGroup` properties of the `info.js` export using subclasses of `PluginConfigGroup`.
The plugin group has the enabled field pre-defined on it, and it's used to enable/disable plugins.
The built-in config fields are defined in [packages/lib/config/definitions.js](/packages/lib/config/definitions.js) while plugins can add their own config field definitions to the `controllerConfigFields` and `instanceConfigFields` properties of the `plugin` export.
A `<plugin_name>.load_plugin` field is automatically created for each plugin and it's used to enable/disable the loading plugins on startup.
## Defing Fields
Fields are defined using the `define` class method on group classes after the groupName have been set and before the class has been finalized.
The `define` method takes an object as argument with the following properties:
Fields are defined as POJS object properties where the name of the property correspond to the name of the configuration field.
The property name is the primary way the field is accessed both in code and on the command line, as such it should follow the lower\_case\_underscore style and consist of a group name and a field name separated by a dot.
For plugins the group name is the name of the plugin, e.g., the `level` field of the `foo_frobber` plugin would be named `foo_frobber.lavel`.
**access**:
The locations this config field can be read and modified from.
This is an array of strings which can contain the values `"master"`, `"slave"` and `"control"`.
This is an array of strings which can contain the values `"controller"`, `"host"` and `"control"`.
If a party is missing in this array then that party will not receive the value of this field when the config is shared, and will be unable to modify the field.
Takes the value of `defaultAccess` set on the config group class if not passed.
Note that instance config groups should have both `"master"` and `"slave"` for all of its fields or unexpected behaviour may occur.
If not set any party can read and modify this config entry.
Note that instance config fields should have both `"controller"` and `"host"` or unexpected behaviour may occur.
**type**:
The type of config value this field will support.
The supported values are boolean, string, number, and object, and if an attept is made to set the field to a value that is not of the right type it will be reject unless it's a string that can be converted to the right type.
**name**:
Name of the configuration field, this is the primary way the field is accessed, both in code and on the command line.
As such it should follow the lower\_case\_underscore style and not contain any spaces or dots.
**title** (optional):
Text used to identify the config field in user interfaces.
This should not contain the grouping used in `name`.
**description** (optional):
Text used to describe the config field in user interfaces.
@@ -44,7 +42,7 @@ The `define` method takes an object as argument with the following properties:
True if this field can be set to null.
Defaults to false.
**initial_value** (optional if optional is true):
**initialValue** (optional if optional is true):
Value this config field should take in a newly initialized config.
This can also be an async function returning the value to use.
@@ -53,26 +51,26 @@ If the value for a field stored in on disk somehow ends up being invalid the fie
## Accessing Fields
On the Config instance there's two ways to access fields: through the set and get methods using dot notation.
On the Config instance there's two ways to access fields: through the set and get methods.
E.g.:
let value = configInstance.get("group.field");
Or by getting the group first and then calling set or get on that.
Or by through the raw fields object (which is best avoided as it doesn't check types or whether the accessed field exists).
E.g.:
let value = configInstance.group("group").get("field")
let value = configInstance.fields["group.field"]
The set method takes an extra argument, which is the value to set the field to.
If the value is not valid this will throw an InvalidValue exception, if the group or field doesn't exist it will throw an InvalidField exception.
The set method also takes an extra argument, which is the value to set the field to.
If the value is not valid this will throw an InvalidValue exception, if the field doesn't exist it will throw an InvalidField exception.
## Config Lifecycle
Values for all config fields are created when the config is initialized, as well as for any fields with missing values when a config is loaded.
Once a field has a value it is never automatically removed, even if the field or group it was part of no longer exists.
Once a field has a value it is never automatically removed, even if the field it was part of no longer exists.
This ensures that plugin configuration is not lost should the server be started up without the plugins that defined the configuration installed.
For instance configs, they are kept on the master server and serialized and sent over to the relevant slaves on startup.
This makes it possible to edit instance configs when the slave the instance is on is offline.
A copy is also stored in the instance directory on the slave, and this config will be deliverd to the master server on slave startup, which enables instances to be copied between clusters as long as their IDs are unique.
For instance configs, they are kept on the controller and serialized and sent over to the relevant hosts on startup.
This makes it possible to edit instance configs when the host the instance is on is offline.
A copy is also stored in the instance directory on the host, and this config will be deliverd to the controller on host startup, which enables instances to be copied between clusters as long as their IDs are unique.
+80 -67
View File
@@ -1,9 +1,9 @@
# Configuration
## Master Configuration
## Controller Configuration
### master.name
### controller.name
Name of the cluster.
Used to distinguish it from other clusters and is prepended to the name of servers in the server list.
@@ -11,37 +11,37 @@ Used to distinguish it from other clusters and is prepended to the name of serve
Defaluts to "Your Cluster".
### master.database_directory
### controller.database_directory
Directory used to store cluster data to.
Needs to be writeable by the master server.
Needs to be writeable by the controller.
This is also used by plugins to store its data.
Defaults to "database".
### master.http_port
### controller.http_port
Port to host HTTP server on.
If set to null no HTTP server will be exposed.
At least one of this and master.https_port needs to be set to a port.
At least one of this and controller.https_port needs to be set to a port.
Defaults to 8080.
### master.https_port
### controller.https_port
Port to host HTTPS server on.
Uses master.tls_certificate and master.tls_private_key as the certificate and private key for the HTTPS server.
Uses controller.tls_certificate and controller.tls_private_key as the certificate and private key for the HTTPS server.
If set to null no HTTPS server will be exposed.
At least one of this and master.http_port needs to be set to a port.
At least one of this and controller.http_port needs to be set to a port.
See the [Setting up TLS](/docs/setting-up-tls.md) document for a guide to setting up HTTPS with Clusterio.
Defaults to null.
### master.bind_address
### controller.bind_address
IP address to bind the HTTP and HTTPS ports on.
Useful to limit which interface to accept connections from.
@@ -51,39 +51,39 @@ interfaces.
Defaults to null.
### master.external_address
### controller.external_address
External address the master server is accessible on.
Currently only used for `clusteriomaster bootstrap create-ctl-config` in order to give the right url to connect to.
External address the controller is accessible on.
Currently only used for `clusteriocontroller bootstrap create-ctl-config` in order to give the right url to connect to.
This should be a full URL ending with a /.
Defaults to null meaning assume localhost.
### master.tls_certificate
### controller.tls_certificate
Path to TLS certificate to use for the HTTPS server when master.https_port is configured.
Path to TLS certificate to use for the HTTPS server when controller.https_port is configured.
Defaults to "database/certificates/cert.crt".
### master.tls_private_key
### controller.tls_private_key
Path to TLS private key to use for the HTTPS server when master.https_port is configured.
Path to TLS private key to use for the HTTPS server when controller.https_port is configured.
Defaults to "database/certificates/cert.key".
### master.auth_secret
### controller.auth_secret
Base64 encoded authentication secret used to verify tokens issued to slaves, users, and sessions.
Base64 encoded authentication secret used to verify tokens issued to hosts, users, and sessions.
This should contain 256 bytes of random data, and if changed will cause all tokens become invalid.
Keep this secret if leaked an attacker could easily compromise the cluster.
Defaults to null which means generate a secure secret on startup.
### master.heartbeat_interval
### controller.heartbeat_interval
Interval in seconds heartbeats are sent out at for WebSocket connections.
If a WebSocket connection hasn't received a heartbeat in 2 times the heartbeat interval it will be considered stale and closed.
@@ -92,33 +92,33 @@ A lower value means less time between connections going stale and them being clo
Defaults to 15.
### master.session_timeout
### controller.session_timeout
Time in seconds from the connection is closed following a heartbeat timeout until the session is invalidated and data loss occurs.
A higher timeout gives more time for a client to recconect and resume an active session over a connection that was dropped, but also leads to stale connections taking longer to clear out.
In the case of the master or client crashing the data loss is unavoidable.
In the case of the controller or client crashing the data loss is unavoidable.
Value should not be less than the configured `max_reconnect_delay` of the clients and it should be greater than 2 times `master.heartbeat_interval`.
Value should not be less than the configured `max_reconnect_delay` of the clients and it should be greater than 2 times `controller.heartbeat_interval`.
Defaults to 60.
### master.metrics_timeout
### controller.metrics_timeout
Timeout in seconds before a call to gather metrics from a slave times out.
Timeout in seconds before a call to gather metrics from a host times out.
This should be less than both the timeout and collection interval configured for the collection job in Prometheus.
Defaults to 8.
### master.stream_timeout
### controller.stream_timeout
Timeout in seconds for proxy streams through the master that has been created but is not yet sending data to be automatically closed.
Timeout in seconds for proxy streams through the controller that has been created but is not yet sending data to be automatically closed.
Streams that have started sending data before this timeout is not affected.
Defaults to 15.
### master.default_role_id
### controller.default_role_id
Role to automatically grant to new users.
If null no role is granted.
@@ -128,30 +128,30 @@ Defaults to 1 which correspond to the default Player role.
### <plugin_name>.load_plugin
Whether to load the plugin on the master server.
Plugins not loaded on the master server will not be loaded on instances.
Whether to load the plugin on the controller.
Plugins not loaded on the controller will not be loaded on instances.
Defaults to true.
## Slave Configuration
## Host Configuration
### slave.name
### host.name
Name of the slave.
Shows up in slave lists and is used to reference this slave in the clusterioctl command line interface.
Name of the host.
Shows up in host lists and is used to reference this host in the clusterioctl command line interface.
Defaults to "New Slave".
Defaults to "New Host".
### slave.id
### host.id
Immutable numeric id of the slave which uniquely identifies this slave in the cluster.
Immutable numeric id of the host which uniquely identifies this host in the cluster.
Defaults to a random 31 bit number.
### slave.factorio_directory
### host.factorio_directory
Directory to look for the Factorio server(s) in.
This can either point to a Factorio server directory, or to a directory containing multiple versions of the Factorio server.
@@ -159,48 +159,48 @@ This can either point to a Factorio server directory, or to a directory containi
Defaults to "factorio".
### slave.instances_directory
### host.instances_directory
Directory to store instances in.
One sub-directory will be created in it for each instance assigned to this slave and these sub-directories will contain logs, saves, and various auto generated configuration files for the instances.
One sub-directory will be created in it for each instance assigned to this host and these sub-directories will contain logs, saves, and various auto generated configuration files for the instances.
Defaults to "instances".
### slave.master_url
### host.controller_url
URL to connect to the master server to.
URL to connect to the controller to.
Defaults to "http://localhost:8080/".
### slave.master_token
### host.controller_token
Access token used for authenticating with the master server.
You can generate an access token with `clusterioctl slave generate-token --id <slave-id>`, or use the `clusterioctl slave create-config` to create a new slave config with the correct url and token in it.
Access token used for authenticating with the controller.
You can generate an access token with `clusterioctl host generate-token --id <host-id>`, or use the `clusterioctl host create-config` to create a new host config with the correct url and token in it.
Defaults to "enter token here".
### slave.tls_ca
### host.tls_ca
Path to certificate in PEM format to use for validating a TLS connection to the master server.
If you have a self signed certificate on the master server you will need to copy the certificate to your slaves and set it as the certificate authority with this option.
Path to certificate in PEM format to use for validating a TLS connection to the controller.
If you have a self signed certificate on the controller you will need to copy the certificate to your hosts and set it as the certificate authority with this option.
Defaults to null meaning use Node.js's default set of trusted certificate authorities.
### slave.public_address
### host.public_address
External address instances hosted on this slave can be accessed on.
External address instances hosted on this host can be accessed on.
This is used by plugins like Server Select to give the correct address to switch between instances in-game.
Defaults to "localhost".
### slave.max_reconnect_delay
### host.max_reconnect_delay
Maximum delay in seconds to wait after connection to the master server is dropped before attempting to reconnect to it.
Maximum delay in seconds to wait after connection to the controller is dropped before attempting to reconnect to it.
The actual delay on each reconnect will be a random number between 0 and this configured value to avoid all clients trying to reconnect at the same time.
Defaults to 60.
@@ -222,19 +222,19 @@ Imutable numeric id of the instance which uniquely identifies this instance in t
Defaluts to a random 31 bit integer.
### instance.assigned_slave
### instance.assigned_host
Slave this instance is assigned to.
Host this instance is assigned to.
To change this you need to use the separate assign instance interface.
After being assigned to a slave the directory for the instance is created on the slave.
After being assigned to a host the directory for the instance is created on the host.
Defaults to null meaning not assigned.
### instance.auto_start
If enabled start this instance when the slave it is assigned to is started up.
Does not affect start or stop of the instance while the slave is running.
If enabled start this instance when the host it is assigned to is started up.
Does not affect start or stop of the instance while the host is running.
Defaults to false.
@@ -242,7 +242,7 @@ Defaults to false.
### factorio.version
Version of Factorio to use for this instance.
Can be a version like "1.0.0" or special string "latest" meaning the latest version of Factorio found on the slave.
Can be a version like "1.0.0" or special string "latest" meaning the latest version of Factorio found on the host.
Defaults to "latest".
@@ -271,6 +271,19 @@ If null a random secure password is generated and used each time the instance st
Defaults to null.
### player_online_autosave_slots
Keeps a separate pool for autosaves where at least on player has been online since the previous autosave was done.
This requires regular autosaving to be enabled and will rename autosaves to `_autosave_poN.zip` (where N is a slot number) whenever a player has been online for that save.
The slot numbers start at 1, are incremented by 1 after each autosave that is renamed and reset back to 1 whenever this value is exceeded.
Note that this value should be set to a higher value than the value of `autosave_slots` in `factorio.settings`.
Useful for preventing the loss of all progress when nobody is around on always online servers with auto pause disabled.
If 0 no autosave renaming takes place.
Defaults to 5.
### factorio.enable_save_patching
Whether to use save patching or not.
@@ -370,39 +383,39 @@ Defaults to 5.
### <plugin_name>.load_plugin
Whether to load the given plugin on the instance.
Note that plugins need to be loaded on the master server in order for them to be loaded on instances.
Note that plugins need to be loaded on the controller in order for them to be loaded on instances.
Defaults to true.
## Control Configuration
### control.master_url
### control.controller_url
URL to connect to the master server to.
URL to connect to the controller to.
Defaults to null meaning complain about it not being set and exit.
### control.master_token
### control.controller_token
Access token used for authenticating with the master server.
You can generate an access token with `clusteriomaster bootstrap generate-user-token <username>`, or use the `clusteriomaster bootstrap create-ctl-config <username>` to create a new ctl config with the correct url and token in it.
Access token used for authenticating with the controller.
You can generate an access token with `clusteriocontroller bootstrap generate-user-token <username>`, or use the `clusteriocontroller bootstrap create-ctl-config <username>` to create a new ctl config with the correct url and token in it.
Defaults to null meaning complain about it not being set and exit.
### control.tls_ca
Path to certificate in PEM format to use for validating a TLS connection to the master server.
If you have a self signed certificate on the master server you will need to copy the certificate to the computer you run clusterctl from and set it as the certificate authority with this option.
Path to certificate in PEM format to use for validating a TLS connection to the controller.
If you have a self signed certificate on the controller you will need to copy the certificate to the computer you run clusterctl from and set it as the certificate authority with this option.
Defaults to null meaning use Node.js's default set of trusted certificate authorities.
### control.max_reconnect_delay
Max duration in seconds to wait before attempting to reconnect with the master server after the connection is dropped.
Max duration in seconds to wait before attempting to reconnect with the controller after the connection is dropped.
The actual delay on each reconnect will be a random number between 0 and this configured value to avoid all clients trying to reconnect at the same time.
Defaults to 60.
+49 -29
View File
@@ -19,6 +19,7 @@ Though we have some conventions and workflows that we kindly ask you to follow.
- [Supported Node.js and Factorio Version](#supported-node.js-and-factorio-version)
- [Code Style](#code-style)
- [Naming Style](#naming-style)
- [Units](#units)
- [Indenting](#indenting)
- [Line Length](#line-length)
- [Markdown](#markdown)
@@ -49,12 +50,10 @@ Do not run `git pull upstream` it will mess thing up!
### Setting up the Development Environment
The project is organized into several packages that are all managed through Lerna.
In order to get the development environment up and running you will need to run the following commands:
The project is organized into several packages that are all tied together using pnpm's workspace feature.
In order to get the development environment up and running you will need to run the following command:
npm install
npx lerna bootstrap --hoist
npx lerna run build
pnpm install
This installs dependencies needed by tests and links the packages up so they work from the git work tree.
To start a cluster from the repo you will first need to initialize it, this can be done using the installer:
@@ -70,24 +69,30 @@ The `plugins/` directory contains the core plugins for the project and can be ad
The documentation uses the binaries provided by the packages, but for the development environment you will need to call the packages directly:
npx clusteriomaster -> node packages/master
npx clusterioslave -> node packages/slave
npx clusteriocontroller -> node packages/controller
npx clusteriohost -> node packages/host
npx clusterioctl -> node packages/ctl
By default node does not enable source maps, to get stack traces pointing to the `.ts` files you will need to pass `--enable-source-maps` to `node` or set the environment variable `NODE_OPTIONS=--enable-source-maps`.
Note that to use `clusterioctl` you will have to create a control config first:
node packages/master bootstrap create-ctl-config [admin-user]
node packages/controller bootstrap create-ctl-config [admin-user]
Once you've set up the cluster you can use `node packages/master run` to run the master server and `node packages/slave run` to run the slave which connects to the master server.
Once you've set up the cluster you can use `node packages/controller run` to run the controller and `node packages/host run` to run the host which connects to the controller.
For web development on the master there are also the following flags.
When editing TypeScript files ran by Node.js you need to run `pnpm build` to compile the files to JavaScript before restarting the controller/host/ctl for changes take effect.
To have TypeScript watch for changes and rebuild immediadly you may run `pnpm watch` instead.
For web development on the controller there are also the following flags:
`--dev` - Recompile the `web_ui` package on the fly providing live reloading of it.
`--dev-plugin [name]` - Recompile the web module for the given plugin on the fly proving live reloading of it.
These options can be combined and multiple plugins can be actived for dev mode.
Note that webpack is configured to delete the web interface build when using these options, so you will have to run `npx lerna run build` before you start the master without these flags again.
Keep in mind if you're working on external plugins the `--dev-plugin` option is likely to only work if the plugin is placed inside the `external_plugins` directory and its dependencies are installed with pnpm as part of the clusterio repository workspace.
Note that webpack is configured to delete the web interface build when using these options, so you will have to run `pnpm run -r --if-present prepare` before you start the controller without these flags again.
### Starting a Feature Branch
@@ -113,8 +118,8 @@ But it's nice to have its name be somewhat descriptive of what it implements.
There's automated tests and linting setup for the project.
Please do add tests for any code you add and run both:
npm run test
npm run lint
pnpm run test
pnpm run lint
To check that your changes pass the integration tests as well as the ESLint rules set up for the project.
@@ -182,12 +187,12 @@ The format of this log should be self explanatory, please follow it carefully.
## Supported Node.js and Factorio Versions
Clusterio run on Node.js v12 and up, make sure to check that the Node builtin API's and npm packages you use are supported on these versions.
The Travis CI tests are runned on whatever version of Node.js v12 is the default.
Clusterio run on Node.js v18 and up, make sure to check that the Node builtin API's and npm packages you use are supported on these versions.
The GitHub Actions tests are run on whatever version of Node.js v18 and v20 is the default.
For Factorio Clusterio 2.0 aims to support version 0.17.69 and up, including the latest experimental release.
It's recommended that you use the lua API reference for 0.17.69, as there's no information on what version Factorio API's were introduced in.
The Travis CI tests are runned against latest experimental release.
The GitHub Actions tests are run against the latest experimental release.
## Code Style
@@ -201,20 +206,31 @@ But there are at least a few things that have been agreed upon.
JavaScript variables in general uses camelCase.
The exception is for classes/constructors which use PascalCase and variables destructured from over the wire objects.
Fields in configs and messages sent over links uses lowercase_underscore.
Fields in configs uses lowercase_underscore.
JavaScript files are named using lowercase_underscore for files that export multiple items.
Files containing and exporting a single class should be named the same as the class in PascalCase.
Imported modules from the lib package are prefixed with lib to make them easier to distinguish, as the names tend to be very generic.
Relative imports in lib package are prefixed with lib to make them easier to distinguish, as the names tend to be very generic.
E.g.:
const libLink = require("@clusterio/lib/link");
const libLuaTools = require("@clusterio/lib/lua_tools");
const libLink = require("./link");
const libLuaTools = require("./lua_tools");
Note that no sub-modules of the lib package is imported directly from the outside of it with the exception of `build_mod`.
For lua code lowercase_underscore is used for everything.
### Units
When naming variables whos unit is not a base SI unit like seconds or metres the unit used should be suffixed in the variable name:
E.g.:
const startTimeMs = date.now();
const endNs = process.hrtime();
This makes it easier to spot different unit bases are being mixed in the code.
### Strings
@@ -272,7 +288,7 @@ Headings use the `# heading` format.
## Pushing releases to npm
The releases are done by creating a new release tag with lerna and then uploading that with lerna.
The releases are done by updating packages versions and then publishing them with pnpm.
You will need commit access to the repository as well publish access to the @clusterio org on npm to do this.
1. To do a release switch to the master branch and make sure it's up to date with the repository before proceeding.
@@ -282,23 +298,27 @@ You will need commit access to the repository as well publish access to the @clu
git pull --ff-only upstream master
```
Check and make sure there are no untracked files with `git status`, otherwise they will be auto commited!
Check and make sure there are no untracked files with `git status`.
2. Run a lerna version to create and tag a new version.
This will push a commit to the master branch.
2. Update the version in package files.
Make sure to update the command for the right version jump.
```sh
npx lerna version prerelease --git-remote upstream --no-granular-pathspec
sed -i 's/2\.0\.0-alpha\.12/2\.0\.0-alpha\.13/' {packages,plugins}/*/package.json
```
3. Run a full build in every package to update the dist/ files.
3. Inspect, commit and tag the changes.
Again, make sure to be tagging the right version.
```sh
npx lerna run build
git add -p
git commit -m v2.0.0-alpha.13
git tag v2.0.0-alpha.13
```
4. Publish the packages with lerna.
4. Publish the packages.
```sh
npx lerna publish from-package
rm -r ./{plugins,packages}/*/dist
pnpm -r --filter='./{plugins,packages}/*' publish
```
+119
View File
@@ -0,0 +1,119 @@
# JSON Serialisable Classes
To serialise and deserialise data the Clusterio codebase uses a JSON serialisable data class pattern.
Instances of these classes can be serialised to JSON by passing them directly to `JSON.serialise` and deserialised by passing the serialised JSON representation to its static `fromJSON` method.
The data class also has a static `jsonSchema` property describing the serialised JSON representation for validation purposes.
See the following example code for a hypothetical `Data` class with a single property `member` defaulting to a value of `"foo"`.
```js
// Data.js
"use strictc";
/**
* A Data class example
* @alias module:Data
*/
class Data {
/** @type {string} */
member = "foo";
constructor(member) {
if (member !== undefined) { this.member = member; }
}
static jsonSchema = {
type: "object",
properties: {
"member": { type: "string" },
},
};
static fromJSON(json) {
return new this(json.member);
}
// toJSON is called by JSON.stringify when serialising this class
toJSON() {
let json = {};
// Omit property if it's set to the default value
if (this.member !== "foo") { json["member"] = this.member; }
return json;
}
}
module.exports = Data;
```
Serialising an instance of `Data` for JSON is as simple as doing `serialised = JSON.stringify(data)` as the `toJSON` method is automatically called by `JSON.stringify` to create a suitable object.
Deserialising is done by calling the static `fromJSON` method with the serialised JSON data, like `data = Data.fromJSON(JSON.parse(serialised))`.
By structuring the classes like this they can be easily composed together.
For example a class holding a data property with an instance of the Data class from the previous example can be created as follows.
```js
// Composed.js
"use strict";
const Data = require("./Data");
/**
* Composed data class example
* @alias module:Data
*/
class Composed {
/** @type {module:Data|undefined} */
data;
constructor(data) {
if (data) { this.data = data; }
}
static jsonSchema = {
type: "object",
properties: {
"data": Data.jsonSchema,
},
};
static fromJSON(json) {
let data;
if (json["data"]) { data = Data.fromJSON(json["data"]); }
return new this(data);
}
// toJSON is optional and not needed for this particular class
}
module.exports = Composed;
```
It's easy to create a new instance of Composed by calling `new Composed(new Data("spam"))` and serialising to and from JSON is the same as for Data.
Meaning `serialised = JSON.stringify(composed)` and `composed = Composed.fromJSON(JSON.parse(serialised))` does as expected.
## Interface description
Data classes are ES6 style class declarations with the following methods and fields:
### toJSON(): any (optional)
Method returning an object or primitive suitable for passing to JSON.stringify in order to create a JSON representation of the class instance.
The returned object may contain objects that have their own toJSON methods which JSON.stringify will then call recursively.
This method is optional if calling `JSON.stringify` on an instance without it would produce the desired serialised form, however this is rarely the case as often you'll have to provide it to convert the values fram Map/Set properites, omit default values or do similar actions.
Note that the primary use case for serialising data classes is to send them over the network, this method should therefore try to reduce the size of the resulting JSON by for example omitting properties who's values are the default value.
### static fromJSON(json: any): this
Method constructing an instance of this class from one that was previously serialised by passing an instance of it to `JSON.stringify`.
While not a requirement it is preferrable if the input is not mutated and that no references to any any structures in the input ends up in the returned object.
In other words calling fromJSON twice with the same input should produce two independent objects.
Note that directly passing the output of `toJSON` to `fromJSON` is not valid, as the output of `toJSON` may require further expansion calling `toJSON` on objects returned by it.
It has to go through `JSON.stringify` or an equivalent recursive expansion of `toJSON`.
### static jsonSchema
A [JSON schema](http://json-schema.org/) describing the format of the JSON resulting from passing an instance of this class to `JSON.stringify`.
This should be lenient towards adding additional properties that would be ignored by `fromJSON` as it makes it easier to extend these classes in the future without breaking compatibility.
+523
View File
@@ -0,0 +1,523 @@
# Migrating Plugins to Alpha 14
Between version 2.0.0-alpha.13 and 2.0.0-alpha.14 the codebase underwent major refactoring and was ported to TypeScript.
In order to support alpha 14 there are several major changes that need to be done to existing plugins, these are in no particular order
## Rename master and slave
All references to slave and master in the code was renamed to controller and host respectively which means that code referencing these in plugins must be renamed.
In short this means performing a global search and replace on the codebase replacing:
- `master` with `controller`
- `Master` with `Controller`
- `slave` with `host`
- `Slave` with `Host`
in all places it occurs, even as part of longer names such as changing `BaseMasterPlugin` to `BaseControllerPlugin`.
## Change lib imports to import directly from the top level package
Previously code could import sub packages of lib such as lib/lua_tools.
These imports need to be changed to all import from lib directly.
**Alpha 13**
```js
const libLuaTools = require("@clusterio/lib/lua_tools");
const libErrors = require("@clusterio/lib/errors");
// Using imports
new libErrors.RequestError(...)
libLuaTools.escapeString(...)
```
**Alpha 14 (JS)**
```js
const lib = require("@clusterio/lib");
// Using library components
new lib.RequestError(...)
lib.escapeString(...)
```
**Alpha 14 (TS)**
```ts
import * as lib from "@clusterio/lib";
// Using library components
new lib.RequestError(...)
lib.escapeString(...)
```
## Change import of base plugins
Previously `BaseMasterPlugin` and `BaseInstancePlugin` were imported from `@clusterio/lib`, these have been moved to `@clusterio/controller` and `@clusterio/host` respectively.
**Alpha 13**
```js
const { BaseMasterPlugin } = require("@clusterio/lib");
const { BaseInstancePlugin } = require("@clusterio/lib");
```
**Alpha 14 (JS)**
```js
const { BaseControllerPlugin } = require("@clusterio/controller");
const { BaseInstancePlugin } = require("@clusterio/host");
```
**Alpha 14 (TS)**
```ts
import { BaseControllerPlugin } from "@clusterio/controller";
import { BaseInstancePlugin } from "@clusterio/host";
```
## Change main entrypoint of plugin
Previously the info of a plugin would be loaded by requiring `./plugin_dir/info`, this has been changed to require `./plugin_dir` which in turn means that `info.js` is no longer the default entrypoint.
To solve this you can either add `"main": "info.js",` to package.json or rename `info.js` to `index.js` (the rest of this guide assumes the entrypoint was renamed to `index.js`).
The motivation behind this change was to allow TypeScript to output its build to a sub-folder of the plugin, if you're using TypeScript you'll likely have the entrypoint be located in `dist/node/index.js` instead.
The expected place where the declaration of a plugin is exported has also changed from `module.exports` to `module.exports.plugin`:
**Alpha 13**
```js
// info.js
module.exports = {
name: "plugin_name",
...
}
```
**Alpha 14 (JS)**
```js
// index.js
const plugin = {
name: "plugin_name",
...
}
module.exports = {
plugin,
};
```
**Alpha 14 (TS)**
```ts
// index.ts
import type * as lib from "@clusterio/lib";
export const plugin: lib.PluginDeclaration = {
name: "plugin_name",
...
};
```
### Modify exposed path in webpack.config.js
The `./info` import path exposed in the ModuleFederationPlugin in the `webpack.config.js` needs to be changed to `./`.
E.g.:
**Alpha 13**
```js
// In webpack.config.js
new webpack.container.ModuleFederationPlugin({
...
exposes: {
"./info": "./info.js",
...
},
}),
```
**Alpha 14**
```js
// In webpack.config.js
new webpack.container.ModuleFederationPlugin({
...
exposes: {
"./": "./index.js",
...
},
}),
```
## Remove usage of ConfigGroup
The Config system was refactored and the `ConfigGroup` concept was completely removed.
Config fields are now defined directly on the exported plugin info.
**Alpha 13**
```js
const lib = require("@clusterio/lib");
class MasterConfigGroup extends lib.PluginConfigGroup { }
MasterConfigGroup.groupName = "plugin_name";
MasterConfigGroup.defaultAccess = ["master", "control"];
MasterConfigGroup.define({
name: "option",
title: "Option",
description: "A config option.",
type: "string",
optional: true,
});
module.exports = {
name: "plugin_name",
MasterConfigGroup,
...
}
```
**Alpha 14 (JS)**
```js
const plugin = {
name: "plugin_name",
controllerConfigFields: {
"plugin_name.option": {
title: "Option",
description: "A config option.",
type: "string",
initial_value: "foo",
optional: true,
},
},
...
}
module.exports = {
plugin,
};
```
**Alpha 14 (TS)**
```ts
import type * as lib from "@clusterio/lib";
// Declare the type of the field to make the type of controller.config.get(field) correctly deduced.
declare module "@clusterio/lib" {
export interface ControllerConfigFields {
"plugin_name.option": null | string;
}
}
export const plugin: lib.PluginDeclaration = {
name: "plugin_name",
controllerConfigFields: {
"plugin_name.option": {
title: "Option",
description: "A config option.",
type: "string",
initialValue: "foo",
optional: true,
},
},
...
};
```
### Use onControllerConfigFieldChanged instead of listering to the fieldChanged event
**Alpha 13**
```js
this.controller.config.on("fieldChanged", (group, field, prev) => {
if (group.name === "plugin_name" && field === "option") {
const curr = group.get(field);
// Do stuff with curr and or prev
}
}
```
**Alpha 14 (JS)**
```js
async onControllerConfigFieldChanged(field, curr, prev) {
if (field === "plugin_name.option") {
// Do stuff with curr and or prev
}
}
```
**Alpha 14 (TS)**
```ts
async onControllerConfigFieldChanged(field: string, curr: unknown, prev: unknown) {
if (field === "plugin_name.option") {
// Do stuff with curr and or prev
}
}
```
## Redefine link messages as classes
The link layer has been majorly refactored and Request and Events are now defined as [JSON Serialisable Classes](./json-serialisable-classes.md).
**Alpha 13**
```js
module.exports = {
name: "plugin_name",
messages: {
forwarded: new lib.Event({
type: "plugin_name:forwarded",
links: ["instance-slave", "slave-master"],
forwardTo: "master",
eventProperties: {
"string_value": { type: "string" },
"number_value": { type: "number" },
},
}),
simple: new lib.Request({
type: "plugin_name:simple",
links: ["master-slave", "slave-instance"],
forwardTo: "instance",
requestProperties: {
"instance_id": { type: "number" },
"string_value": { type: "string" },
},
responseProperties: {
"number_value": { type: "number" },
},
}),
},
};
```
**Alpha 14 (JS)**
```js
const lib = require("@clusterio/lib");
class ForwardedEvent {
static type = "event";
static src = "instance";
static dst = "controller";
static plugin = "plugin_name";
constructor(stringValue, numberValue) {
this.stringValue = stringValue;
this.numberValue = numberValue;
}
static jsonSchema = {
type: "object",
required: ["stringValue", "numberValue"],
properties: {
"stringValue": { type: "string" },
"numberValue": { type: "number" },
},
};
static fromJSON(json) {
return new this(json.stringValue, json.numberValue);
}
}
class SimpleRequest {
static type = "request";
static src = "controller";
static dst = "instance";
static plugin = "plugin_name";
constructor(stringValue) {
this.stringValue = stringValue;
}
static jsonSchema = {
type: "object",
required: ["stringValue"],
properties: {
"stringValue": { type: "string" },
},
};
static Response = lib.JsonNumber
}
const plugin = {
name: "plugin_name",
messages: [
ForwardedEvent,
SimpleRequest,
],
...
};
module.exports = {
plugin,
ForwardedEvent,
SimpleRequest,
}
```
**Alpha 14 (TS)**
```ts
import * as lib from "@clusterio/lib";
import { Static, Type } from "@sinclair/typebox";
export class ForwardedEvent {
declare ["constructor"]: typeof ForwardedEvent;
static type = "event" as const;
static src = "instance" as const;
static dst = "controller" as const;
static plugin = "plugin_name" as const;
constructor(
public stringValue: string,
public numberValue: number,
) { }
static jsonSchema = Type.Object({
"stringValue": Type.String(),
"numberValue": Type.Number(),
});
static fromJSON(json: Static<typeof this.jsonSchema>) {
return new this(json.stringValue, json.numberValue);
}
}
export class SimpleRequest {
declare ["constructor"]: typeof SimpleRequest;
static type = "request" as const;
static src = "controller" as const;
static dst = "instance" as const;
static plugin = "plugin_name" as const;
constructor(
public stringValue: string,
) { }
static jsonSchema = Type.Object({
"stringValue": Type.String(),
});
static fromJSON(json: Static<typeof this.jsonSchema>) {
return new this(json.stringValue);
}
static Response = lib.JsonNumber
}
export const plugin: lib.PluginDeclaration = {
name: "plugin_name",
messages: [
ForwardedEvent,
SimpleRequest,
],
...
}
```
### Refactor message sending
Message handlers are now explicitly registered instead of relying on magic name lookups that don't make sense unless you know them.
**Alpha 13**
```js
const lib = require("@clusterio/lib");
class ControllerPlugin extends lib.BaseControllerPlugin {
// event name is magically created by concatenating EventHandler or
// RequestHandler to the name in the messages export in in info.js.
async forwardedEventHandler(message) {
// message.data contains the raw JSON data sent.
if (message.data.string_value) {
// ...
}
}
}
```
**Alpha 14 (JS)**
```js
const { BaseControllerPlugin } = require("@clusterio/lib");
const { ForwardedEvent } = require("./index.js");
class ControllerPlugin extends BaseControllerPlugin {
async init() {
// Explicit binding between message class and the handler for it
this.controller.handle(ForwardedEvent, this.handleForwardedEvent.bind(this));
// Note: On an instance plugin you would use this.instance.handle
}
async handleForwardedEvent(event, src, dst) {
// Event is an instance of the ForwardedEvent class
if (event.stringValue) {
// ...
}
// src and dst are instances of lib.Address and describe where
// the message came from and who it was addressed to.
}
...
}
module.exports = {
ControllerPlugin,
}
```
**Alpha 14 (TS)**
```ts
import * as lib from "@clusterio/lib";
import { BaseControllerPlugin } from "@clusterio/controller";
import { ForwardedEvent } from "./index.js";
export class ControllerPlugin extends BaseControllerPlugin {
async init() {
// When using TypeScript this call is type checked to make sure the handler accepts the right class.
this.controller.handle(ForwardedEvent, this.handleForwardedEvent.bind(this));
// Note: On an instance plugin you would use this.instance.handle
}
async handleForwardedEvent(event: ForwardedEvent, src: lib.Address, dst: lib.Address) {
// Event is an instance of the ForwardedEvent class
if (event.stringValue) {
// ...
}
// src and dst describe where the message came from and who it was addressed to.
}
...
}
```
### Refactor message sending
Message are now addressed end to end instead of using the confusing forwardTo and broadcastTo properties of the message definition.
When broadcasting events the special `"allInstances"`, `"allHosts"`, and `"allControls"` shorthand addresses can be used.
There's two methods for sending messages `.send` and `.sendTo` where `.send` is a shorthand for `.sendTo` with an address filled in as the other side of the link, this is mostly only useful in control plugins where the other side of the link is the controller.
**Alpha 13**
```js
// from instance plugin
this.info.messages.forwarded.send(this.instance, {
string_value: "foo",
number_value: 123,
});
// from master plugin
const instance_id = ...
const response = await this.forwardRequestToInstance(this.info.messages.simple, {
instance_id,
string_value: "foo",
});
const value = response.number_value;
```
**Alpha 14**
```js
const { ForwardedEvent } = require("./info.js");
// from instance plugin
this.instance.sendTo("controller", new ForwardedEvent("foo", 123));
// from controller plugin
const instanceId = ...
const value = await this.controller.sendTo({ instanceId }, new SimpleRequest("foo"));
// if using TypeScript then the type of value will be inferred as number here
```
+21 -26
View File
@@ -12,9 +12,9 @@ Events and requests are defined as instances of the `Request` and `Event` classe
All messages sent and received are validated with JSON schemas to catch potential errors in sending messages and avoid having to write lots of validation code in message handlers.
Messages are sent by calling the `.send()` method of the message instance and passing a `Link` instance to send it over and the data to send.
For the master server there's a `Link` instance for each slave and control connection stored in the `WsServer` class.
For slaves the `Slave` instance is a link to the master server, and each `Instance` instance is an in-memory link to the `Slave` instance and has a corresponding `InstanceConnection` link stored on the `Slave` instance.
For ctl and web_ui there's a `Control` instance that acts as the link to the master server.
For the controller there's a `Link` instance for each host and control connection stored in the `WsServer` class.
For hosts the `Host` instance is a link to the controller, and each `Instance` instance is an in-memory link to the `Host` instance and has a corresponding `InstanceConnection` link stored on the `Host` instance.
For ctl and web_ui there's a `Control` instance that acts as the link to the controller.
Messages received invokes message handlers that were registered when the link was brought up.
These handlers are named after the name of the property the message is stored under with `EventHandler` appended for events and `RequestHandler` appended for requests.
@@ -24,7 +24,7 @@ The handlers are defined on the link class itself for core messages, and on the
## Connection flow
Clients connect via WebSocket to the `/api/socket` endpoint and upon establishing connection to the server the connection is in the handshake phase and the server will send a `hello` message to the client.
If the client has no previous session with the server it sends a `register_slave` or `register_control` message depending on what kind of client it is to the server.
If the client has no previous session with the server it sends a `register_host` or `register_control` message depending on what kind of client it is to the server.
If the register is valid then the server will respond with a `ready` message with a session token and the connection has left the handshake phase, otherwise the server will close the connection with a WebSocket close code indicating the kind of failure.
Upon leaving the handshake phase both parties should start sending heartbeat messages with the sequence number of the last received message.
@@ -33,7 +33,7 @@ Messages sent over the link should be stored in a send buffer and only cleared f
If the connection goes stale or is unexpectedly closed the client can reastablish the same session by connecting to the server again and sending a `resume` message after received the `hello` from the server.
If the session resume is successful then the server will respond with a `continue` message and the connection is active again.
The client should then re-send all messages it has in its send buffer that is greater than the `last_seq` number in the `continue` message.
Otherwise the server will respond with `invalidate` and the client is expected to drop its send buffer start a new session with `register_slave` or `register_control`.
Otherwise the server will respond with `invalidate` and the client is expected to drop its send buffer start a new session with `register_host` or `register_control`.
![Connection flow](../assets/connection_flow.svg)
@@ -96,30 +96,27 @@ The following messages are the only valid messages that can be sent during the h
#### `hello`
Sent by the master server when the WebSocket connection has been opened.
Sent by the controller when the WebSocket connection has been opened.
- `version: string` - The version of the master, e.g. "2.0.0".
- `version: string` - The version of the controller, e.g. "2.0.0".
- `plugins: Object<string, string>` -
Object mapping plugin names to plugin version for plugins that are loaded on the master server.
Object mapping plugin names to plugin version for plugins that are loaded on the controller.
#### `register_slave`
#### `register_host`
Slave handshake for establishing a new connection session.
Host handshake for establishing a new connection session.
- `token: string` - Authentication token.
- `agent: string` - Human readable string of the software connecting.
- `version: string` - The protocol version of the slave, e.g. "2.0.0".
- `name: string` - Name of the slave.
- `id: integer` - ID of the slave.
- `version: string` - The protocol version of the host, e.g. "2.0.0".
- `id: integer` - ID of the host.
- `plugins: Object<string, string>` -
Object mapping plugin names to plugin version for plugins that are available on the slave.
Object mapping plugin names to plugin version for plugins that are available on the host.
#### `register_control`
Control handshake for enstablishing a new connection session.
- `token: string` - Authentication token.
- `agent: string` - Human readable string of the software connecting.
- `version: string` - The protocol version of the control, e.g. "2.0.0".
#### `resume`
@@ -131,12 +128,12 @@ Handshake for resuming an existing connection session where the connection had d
#### `invalidate`
Sent by the master server in reply to a resume for a session that cannot be resumed.
Sent by the controller in reply to a resume for a session that cannot be resumed.
The client should start a new session with a register\_\* message in response.
#### `continue`
Sent by the master server to signal resuming the session was successful.
Sent by the controller to signal resuming the session was successful.
- `last_seq: integer | null` - Last message seq received by the server.
- `session_timeout: integer` - Session timeout in seconds.
@@ -144,7 +141,7 @@ Sent by the master server to signal resuming the session was successful.
#### `ready`
Sent by the master server to signal the connection has been established with a new session.
Sent by the controller to signal the connection has been established with a new session.
- `session_token: string` - Token to resume the session with.
- `session_timeout: integer` - Session timeout in seconds.
@@ -159,7 +156,7 @@ The heartbeat should contain the `seq` of the last received message, and receivi
#### `heartbeat`
Sent by both the master server and clients at the interval given in the continue/ready messages.
Sent by both the controller and clients at the interval given in the continue/ready messages.
- `last_seq: integer` - Last message seq received over the link.
@@ -235,23 +232,21 @@ Server hello after opening the WebSocket.
}
```
Slave registering a new connection session.
Host registering a new connection session.
```json
{
"seq": null,
"type": "register_slave",
"type": "register_host",
"data": {
"token": "xyzabc123",
"agent": "Clusterio Slave",
"version": "2.0.0",
"name": "Foo",
"id": 123
}
}
```
Master server reply signaling the session is established and ready.
Controller reply signaling the session is established and ready.
```json
{
@@ -265,4 +260,4 @@ Master server reply signaling the session is established and ready.
}
```
Slave and master can now send events and requests over the connection.
Host and controller can now send events and requests over the connection.
+1 -1
View File
@@ -11,6 +11,6 @@ This can be disabled with the `factorio.enable_save_patching` config on a per in
## User Accounts
The master server creates and stores accounts for all Factorio users that log into instances under its control.
The controller creates and stores accounts for all Factorio users that log into instances under its control.
These accounts are used to store per player data and authorize management of the cluster.
By default users accounts are assigned a Player role that gives a limited read access to the cluster resources, and admins of the cluster can be assigned a Cluster Admin role to get access to manage it.
-23
View File
@@ -1,23 +0,0 @@
{
"plugins": [
"plugins/markdown"
],
"tags": {
"allowUnknownTags": false
},
"opts": {
"destination": "docs/api",
"recurse": true
},
"source": {
"include": [
"packages/master/src/",
"packages/master/master.js",
"packages/slave/slave.js",
"packages/ctl/ctl.js",
"packages/lib/",
"plugins/"
],
"excludePattern": "node_modules"
}
}
+105 -5
View File
@@ -1,22 +1,95 @@
# Managing a Cluster
Clusterio clusters are managed through the master server by using the `clusterioctl` command line interface which is invoked by running `npx clusterioctl <command>` in the clusterio directory.
Clusterio clusters are managed through the controller by using the `clusterioctl` command line interface which is invoked by running `npx clusterioctl <command>` in the clusterio directory.
This document uses the shorthand `ctl> foo` to indicate `npx clusterioctl foo` should be executed.
Mandatory parameters are shown in `<angles bracket>` and optional pameters are in `[square brackets]`.
Before `clusterioctl` can be used it needs to be configured for the cluster it will connect to.
The easiest way to do this is to run `npx clusteriomaster bootstrap create-ctl-config <username>` on the master server, which creates the necessary `config-control.json` for managing the cluster as the given user.
The easiest way to do this is to run `npx clusteriocontroller bootstrap create-ctl-config <username>` on the controller, which creates the necessary `config-control.json` for managing the cluster as the given user.
## Slaves
## Hosts
To be written.
### List hosts
ctl> host list
Lists all hosts.
### Generate token
ctl> host generate-token [--id <host id>]
Generate a token for a host. `--id` is the host id.
### Create config
ctl> host create-config [--id <host id>] [--name <host name>] [--generate-token <bool>] [--output <path to output>]
Create a host config. `--id` id the host id. `--name` is the host name. `--generate-token` decides whether to also generate a token, defaulting to `0`. `--output` is the filename for the output of the config; Default is "config-host.json", "-" for stdout.
## Instances
### List Instances
ctl> instance list
List instances.
### Create instance
ctl> instance create <name> [--id <instance id>]
Create a new instance. `name` is the instance name. `--id` is the optional id.
### Kill instance
ctl> instance kill <name>
Terminates the running Factorio server without giving it a chance to save or cleanup after itself.
Note: This may cause loss of data.
### Extract players from instance
ctl> instance extract-players <name>
Creates a user account in the cluster for each player that has been online on the currently running save of the instance and sets the online time of that account on this instance to the online time recorded in the save.
Useful when importing a save to a cluster.
Note that the online time in a save is recorded differntly from how Clusterio records online time.
Most notably Clusterio records wall clock time while the save time is recorded in ticks and this extraction of player time assumes the game runs at 60 ticks a second.
### Config management
#### List Configuration
ctl> instance config list <instance>
List config values for an instance. `instance` is the instance whose config values to list.
#### Set config value
ctl> instance config set <instance> <field> [value] [--stdin]
Set field in instance config. `instance` is instance to set. `field` is field to set. `value` is value to set in field. `--stdin` signals to read value from stdin.
#### Set config property of field
To be written.
#### Edit instance config
ctl> instance config edit <instance> [editor]
Edit instance config in a text editor. Once file is saved and editor has been exited, config values will be read back in. `instance` is the instance to edit, `editor` is the editor to use. If `editor` is not given, clusterioctl will read from the `EDITOR` or `VISUAL` enviroment variables.
### Assign instance to host
ctl> instace assign <instance> [host]
Assigns instance to host. `instance` is instance to assign, `host` is host to assign to. If `host` is not provided, instance will be unassigned.
## Users
@@ -74,6 +147,33 @@ Since ban status is a part of the account data the account must exist for this t
Note: This bans the user from logging in to Factorio servers in the cluster, it does not revoke access to any cluster management they might have, see next section on setting roles for revoking tha.
### Bulk importing admin, whitelist and ban lists
Whitelists and admin lists can be bulk imported with the following commands:
ctl> user whitelist-bulk-import <file> [--create]
ctl> user admin-bulk-import <file> [--create]
The file should simply be newline seperated names:
joe
bob
allan
The `--create` option will create the user if they do not exist, and then set the relevent value.
Banlists can be bulk imported with the following command:
ctl> user banned-bulk-import <file> [--create]
The file should be names, followed by a ban reason:
joe greifed copper mines
bob stole materials
allan rude language
The first string on the line is used as the username, the others are used as the ban reason.
The `--create` option will create the user if they do not exist, then ban them.
### Set Cluster Roles
@@ -89,7 +189,7 @@ By default there's a roled named Cluster Admin which grants access to everything
ctl> user delete <name>
Deletes everything stored on the master server for this user.
Deletes everything stored on the controller for this user.
Note: If the player is banned from the cluster this will effectively unban them, as the ban status is stored with the user account.
+42
View File
@@ -0,0 +1,42 @@
# Migrating from 2.0.0-alpha.13 to 2.0.0-alpha.14
Alpha 14 comes with significant breaking changes for existing clusters, most notably the rename of master and slave to controller and host.
Please study the entire migration guide carefully before attempting to update an existing cluster.
## Before updating
If you're using mods you need to create mod pack string for the mod configurations you're currently running.
You can do this by running the follow RCON command on an instance that is currently running the mod configuration you want to use (you can also run this command in single player and find the string printed in Factorio's [Log file](https://wiki.factorio.com/Log_file)):
/sc local m={name="pack",description="",factorio_version=game.active_mods.base,mods={},settings={startup={},["runtime-global"]={},["runtime-per-user"]={}}} for n,v in pairs(game.active_mods) do table.insert(m.mods,{name=n,enabled=true,version=v}) end local function set(s,c) for n,v in pairs(s) do m.settings[c][n]=v end end set(settings.startup,"startup") set(settings.global,"runtime-global") if game.player then set(game.player.mod_settings,"runtime-per-user") else set(settings.player,"runtime-per-user") end local e=game.encode_string(game.table_to_json(m)) log(e) rcon.print(e)
This should produce a [Mod Pack String](https://forums.factorio.com/viewtopic.php?f=96&t=103578) which you need to copy and paste somewhere to keep it for later.
Mod Pack strings looks something like this:
eNqFketuwyAMhd/Fv0OUVpW25lWmqSLESdAIRNh0l4h3n9mlitZt5QdCx/7so8MKXs8ILSzaPEEFPZKJdmEbvIgiDNpwiDaczhjpU93Vu/r+Tmpz6Anah/V7RqcJRUavO4c9tBwTVnAF5upCGJeIsYx3tvsPbQTdb0lKHYlnPJHY0+ONvcdjvW8gP1ZAyGz9KL5XINaR0yLPXEFMnu2ManSh0+6j/GOFitqXG4mjNSUidVm5wlm7JMYG7Qhl3BX8FjyqZ9vztOk+NM2fvRPaceLbzbN+UejQFFOWXzfArvk6v2HWD9ZLv5JfxA1UwsubPBaMKhHGklLO76vAwfQ=
If you have multiple mod configuration create a mod pack string for each and note them down.
Double check that you are on a supported version of Node.js by running `node --version`, the version should be v18 or later, if not you will need to update Node.js before continuing.
## Migrate installations
**Note:** Check that plugins you are using have updates avaialble for alpha 14 before attempting to update, if they do not either wait for the plugin to be updated or uninstall the plugin from all places it is installed on first.
Shutdown all slaves and the master server and then migrate each installations by running the following command:
npm exec @clusterio/create -y -- --migrate-rename --log-level=verbose
This will convert the existing installation's usage of the master and slave names in packages, configs, database and logs to the new controller and host terms and update the installed packages.
If you're using systemd then copy the generated clusteriocontroller.service and/or clusteriohost.service files from the systemd folder to /etc/systemd/system/ and remove the old clusteriomaster.service and clusterioslave.service files.
Once the installations have been migrated start up the controller and hosts again.
## Post installation
Import the Mod Pack strings that was created beforehand on the controller and upload the mods needed by those mod packs to the controller.
You can assign a default mod pack in the controller config which instances use by default or set the mod pack on the per instance basis in the instance config.
Once you have assigned a mod pack to an instance run a data export on the instance in order to properly populate the mod settings in the mod and get item icons and item names resolved in the web interface.
+11 -11
View File
@@ -1,7 +1,7 @@
# Setting up TLS
By default Clusterio will listen for connections over HTTP which is fine for local networks but over the Internet it should idealy be over HTTPS as there are some credentials that could be exposed to eavesdroppers.
The recommended setup is to have an ordinary HTTPS server software like Apache or Nginx to act as a reverse proxy in front of the Clusterio master server.
The recommended setup is to have an ordinary HTTPS server software like Apache or Nginx to act as a reverse proxy in front of the Clusterio controller.
The reverse proxy would then be set up with a TLS certificate from a Certificate Authoritiy like [Let's Encrypt](https://letsencrypt.org).
Alternatively you can have Clusterio listen for connections over HTTPS directly by providing it with a TLS certificate and private key.
@@ -11,17 +11,17 @@ See the last section for how to create and use a self-signed certificate for an
## Using a reverse proxy
Configure the proxy to forward requests over HTTP to the port the master server is listening to so that for example a request to `https://example.com/instances/` is forwarded to `http://localhost:8080/instances/`.
It is possible to host the interface under a sub-path, in that case a request under the sub-path should be mapped to the master server with the sub-path stripped out.
Configure the proxy to forward requests over HTTP to the port the controller is listening to so that for example a request to `https://example.com/instances/` is forwarded to `http://localhost:8080/instances/`.
It is possible to host the interface under a sub-path, in that case a request under the sub-path should be mapped to the controller with the sub-path stripped out.
For example `https://example.com/sub/path/instances/` would be mapped to `http://localhost:8080/instances`.
Additianlly WebSocket connections to `/api/socket` also needs to be forwarded.
Typically this requires explicit configuration and support for forwarding WebSocket connections.
If the interface hosted under a sub-path the `/api/socket` address is relative to the sub-path and the sub-path also needs to be stripped out when forwarded.
You can also optionally set the master server to listen only on the loopback interface to prevent it being directly connected to from the outside.
You can also optionally set the controller to listen only on the loopback interface to prevent it being directly connected to from the outside.
npx clusteriomaster config set master.bind_address 127.0.0.1
npx clusteriocontroller config set controller.bind_address 127.0.0.1
### Apache Config
@@ -44,7 +44,7 @@ Pay attention to the use of trailing slashes, as they are important.
## Hosting HTTPS directly from Node.js
Listening on HTTPS is supported by the Clusterio master server via the `master.https_port` option provided paths to a certificate file and a private key file is configured for the `master.tls_certificate` and `master.tls_private_key` options.
Listening on HTTPS is supported by the Clusterio controller via the `controller.https_port` option provided paths to a certificate file and a private key file is configured for the `controller.tls_certificate` and `controller.tls_private_key` options.
Both of these are expected to be unencrypted PEM files.
@@ -58,14 +58,14 @@ substituting `<ip>` with the IP address the server will be accessed under, for e
Multiple addresses can be given by supplying a comma sepparated list of `IP:address` pairs and DNS names can also be specified using the `DNS:example.com` format.
You need to supply every name you want to connect to the server under the subjectAltName field, for example to connect to the HTTPS port locally via an address like `https://localhost:4443/` you need to add `DNS:localhost`.
This creates a private key file named `key.pem` and a public certificate file named `cert.pem` and these are used for the `master.tls_certificate` and `master.tls_private_key` config options respectively on the master server:
This creates a private key file named `key.pem` and a public certificate file named `cert.pem` and these are used for the `controller.tls_certificate` and `controller.tls_private_key` config options respectively on the controller:
npx clusteriomaster config set master.tls_certificate cert.pem
npx clusteriomaster config set master.tls_private_key key.pem
npx clusteriocontroller config set controller.tls_certificate cert.pem
npx clusteriocontroller config set controller.tls_private_key key.pem
Additionally you will need copy the `cert.pem` file to all of the computers you want to set up as slaves and configure it as the `slave.tls_ca` option:
Additionally you will need copy the `cert.pem` file to all of the computers you want to set up as hosts and configure it as the `host.tls_ca` option:
npx clusterioslave config set slave.tls_ca cert.pem
npx clusteriohost config set host.tls_ca cert.pem
You will also need to copy the `cert.pem` file to all of the computers you want to remotely manage the cluster via clusterctl and configure it as `control.tls_ca` option:
+190 -214
View File
@@ -12,8 +12,6 @@ The plugin classes have pre-defined hooks that are called during various stages
- [Plugin Configuration](#plugin-configuration)
- [Communicating with Factorio](#communicating-with-factorio)
- [Defining Link Messages](#defining-link-messages)
- [Defining Events](#defining-events)
- [Defining Requests](#defining-requests)
- [Sending Link Messages](#sending-link-messages)
- [Handling connection events](#handling-connection-events)
- [Collecting Statistics](#collecting-statistics)
@@ -25,9 +23,9 @@ The plugin classes have pre-defined hooks that are called during various stages
The basic file structure of a plugin is the following.
plugin_name/
+- info.js
+- index.js
+- package.json
+- master.js
+- controller.js
+- instance.js
+- control.js
+- test/
@@ -41,22 +39,21 @@ The usual guides for creating such packages apply.
At minimum the `package.json` file must contain a version entry.
A possible workflow for developing plugins is to place the plugin in a sub-directory of where clusterio has been installed, and rely on Node.js searching up the folder heirarchy for it to find `@clusterio/lib`.
To add it to `plugin-list.json` so that it gets loaded use the `plugin add <path>` sub-command to either clusteriomaster, clusterioslave or clusterioctl.
After creating a directory for your plugin, the boilerplate can be generated automaticlly from a set of tempaltes using `npm exec @clusterio/create -y -- --plugin-template` after which you will be asked which areas your plugin will interact with and your plugin's name.
To add it to `plugin-list.json` so that it gets loaded use the `plugin add <path>` sub-command to either clusteriocontroller, clusteriohost or clusterioctl.
Note that it's important that the path starts with ./ or ../ (use .\ or ..\ on Windows).
For a plugin the most important file is the `info.js` file.
Without it the plugin will not recognized by Clusterio.
Here's an example of it:
For a plugin to be recognised by Clusterio it needs to export an entry named `plugin` from its main entrypoint.
By default the main entrypoint is the `index.js` file, but this may be changed by setting the `"main"` entry to a different file in `package.json`.
Here's an example of `index.js`:
```js
const { libLink } = require("@clusterio/lib"); // For messages
module.exports = {
module.exports.plugin = {
name: "foo_frobber",
title: "Foo Frobber",
description: "Does advanced frobnication",
instanceEntrypoint: "instance",
masterEntrypoint: "master",
controllerEntrypoint: "controller",
messages: {
/* See below */
},
@@ -80,23 +77,23 @@ The following properties are recognized:
**instanceEntrypoint**:
Path to a Node.js module relative to the plugin directory which contains the InstancePlugin class definition for this plugin.
This is an optional paramater.
A plugin may have code only for instances but it must still be loaded on the master in order for it to be possible to load it on an instance.
A plugin may have code only for instances but it must still be loaded on the controller in order for it to be possible to load it on an instance.
**InstanceConfigGroup**:
Subclass of `PluginConfigGroup` for defining the per instance configuration fields for this plugin.
**instanceConfigFields**:
Object mapping instance configuration fields to `FieldDefinitions`for this plugin.
See [Plugin Configuration](#plugin-configuration)
**masterEntrypoint**:
Path to a Node.js module relative to the plugin directory which contains the MasterPlugin class definiton for this plugin.
**controllerEntrypoint**:
Path to a Node.js module relative to the plugin directory which contains the ControllerPlugin class definiton for this plugin.
This is an optional parameter.
A plugin can be made that only runs on the master server.
A plugin can be made that only runs on the controller.
**MasterConfigGroup**:
Subclass of `PluginConfigGroup` for defining the master server configuration fields for this plugin.
**ControllerConfigGroup**:
Object mapping controller configuration fields to `FieldDefinitions`for this plugin.
See [Plugin Configuration](#plugin-configuration)
**controlEntrypoint**:
Path to a Node.js module relative to the plugin directory which contains the ControlPlugin class definition for this plugin.
**ctlEntrypoint**:
Path to a Node.js module relative to the plugin directory which contains the CtlPlugin class definition for this plugin.
This is an optional paramater.
A plugin can be made that only runs on the clusterioctl side.
@@ -108,21 +105,21 @@ The optional module folder contains a Clusterio module that will be patched into
See the section on [Clusterio Modules](developing-for-clusterio.md) in the Developing for Clusterio document.
The only restriction imposed on modules embedded into plugins is that they must be named the same as the plugin.
While there is no standard for how to organize a plugin it's recommended to put the MasterPlugin class definition into master.js and the InstancePlugin class definition into instance.js.
While there is no standard for how to organize a plugin it's recommended to put the ControllerPlugin class definition into controller.js and the InstancePlugin class definition into instance.js.
You can put them into whatever file you want (even the same one for both).
For both instanceEntrypoint and masterEntrypoint the path should not end with .js and it should use forward slashes for directory separators if any.
For both instanceEntrypoint and controllerEntrypoint the path should not end with .js and it should use forward slashes for directory separators if any.
## Defining the plugin class
The plugin class should derive from its respective base class defined in `lib/plugin`.
For example, to define a MasterPlugin class the following code can be used:
For example, to define a ControllerPlugin class the following code can be used:
```js
const libPlugin = require("@clusterio/lib/plugin");
const { BaseControllerPlugin } = require("@clusterio/controller");
class MasterPlugin extends libPlugin.BaseMasterPlugin {
class ControllerPlugin extends BaseControllerPlugin {
async init() {
this.foo = 42;
await this.startFrobnication();
@@ -132,12 +129,12 @@ class MasterPlugin extends libPlugin.BaseMasterPlugin {
}
module.exports = {
MasterPlugin,
ControllerPlugin,
}
```
For the instance plugin it's exactly the same except "Master" is replaced with "Instance", and for the clusterioctl plugin "Control" is used.
The available hooks that you can override are documented in the base class [in lib/plugin.js](/packages/lib/plugin.js).
For the instance plugin it's exactly the same except "Controller" is replaced with "Instance", for host plugins "Host" is used and for the clusterioctl plugin "Ctl" is used.
The available hooks that you can override are documented in the base classes.
It's best to avoid defining a constructor, but if you insist on defining one, forward all arguments to the base class.
E.g.:
@@ -157,7 +154,7 @@ The async init method is always called immediatly after the constructor, so ther
## Logging Messages
The base plugin classes provide a winston logger for logging messages to the shared cluster log.
For instances a copy of the log is also stored on the slave the instance is on.
For instances a copy of the log is also stored on the host the instance is on.
To use it, pass a string to one of the log levels functions, for example:
```js
@@ -202,39 +199,42 @@ For plugin hooks expections thrown are automatically catched and logged, but for
## Plugin Configuration
Clusterio provides a configuration system that handles storing, distributing, editing and validating config fields for you.
You can take advantage of it by subclassing `PluginConfigGroup`, setting `defaultAccess` to where config entries can be accessed from, setting the `groupName` to your plugin name, defining fields on it, finalizing it, and passing it as either `MasterConfigGroup` or `InstanceConfigGroup` in the `info.js` export.
For example in info.js:
You can take advantage of it by declaring config fields under either `controllerConfigFields` or `instanceConfigFields` in the `plugin` export.
For example in index.js:
```js
const libConfig = require("@clusterio/lib/config");
```ts
import type * as lib from "@clusterio/lib";
class MasterConfigGroup extends libConfig.PluginConfigGroup { }
MasterConfigGroup.defaultAccess = ["master", "slave", "control"];
MasterConfigGroup.groupName = "foo_frobber";
MasterConfigGroup.define({
name: "level",
description: "Level of frobnication done",
type: "number",
initial_value: 2,
});
MasterConfigGroup.finalize();
declare module "@clusterio/lib" {
// Extend the interface defining available fields so that instance.config.get(...)
// will have our own options.
export interface ControllerConfigFields {
"foo_frobber.level": number, // If optional is true then this should have | null in addition.
}
}
module.exports = {
export default {
...
MasterConfigGroup: MasterConfigGroup,
};
controllerConfigFields: {
"foo_frobber.level": {
description: "Level of frobnication done",
type: "number",
initialValue: 2,
},
}
} satisfies lib.PluginDeclaration;
```
Code inside the `MasterPlugin` class will then be able to access the level config field through the `Config` object at `this.master.config`, for example in the MasterPluginClass:
Code inside the `ControllerPlugin` class will then be able to access the level config field through the `Config` object at `this.controller.config`, for example in the ControllerPluginClass:
```js
```ts
async init() {
let level = this.master.config.get("foo_frobber.level");
let level = this.controller.config.get("foo_frobber.level");
this.logger.info(`I got a frobnication level of ${level}`);
}
```
The same applies for instance configs, replace "master" with "instance" where appropriate.
The same applies for instance configs, replace "controller" with "instance" where appropriate.
See [Configuration System](config-system.md) for more details on how this system works.
@@ -301,6 +301,20 @@ local clusterio_api = require("modules/clusterio/api")
clusterio_api.send_json("my_plugin_foo", { data = 123 })
```
For convience a helper function is exposed on server specifically for handling IPC events.
```js
async init() {
this.instance.server.handle("my_plugin_foo", this.handleFoo.bind(this));
}
async handleFoo(content) {
// Do stuff with content
}
```
**Note:** This only handles IPC events, all other events must use `on`. You should not include the `ipc-` prefix when using this method. Error logging is handled for you, if you want custom error handling use `on`.
It's recommended to either use the plugin name as the channel name or to prefix the channel name with the name of the plugin if you need multiple channels.
It's also important to catch any errors that might occur as they will otherwise be propogated to the instance code and kill the server.
@@ -313,8 +327,8 @@ For RCON, commands longer than 50 characters may end up being executed after sho
## Defining Link Messages
You will most likely have to communicate with the master or other instances in your plugin for it to do anything useful.
For this there's a WebSocket communication channel established between the slaves and the master server that plugins can define their own messages to send over it.
You will most likely have to communicate with the controller or other instances in your plugin for it to do anything useful.
For this there's a WebSocket communication channel established between the hosts and the controller that plugins can define their own messages to send over it.
This channel is bi-directional and all messages sent over it are validated with a JSON schema (see [this guide][guide] for an introduction to writing JSON schema).
[guide]: https://json-schema.org/learn/getting-started-step-by-step.html
@@ -324,199 +338,161 @@ Events are simple one-way notifications that invoke a handler on the target it's
Requests are pairs of request and response messages where the request is sent to the target and the response is the reply back from the target.
The requests are similar to HTTP requests, except both parties of a link may initiate one.
### Defining Events
Events are defined as properties of the messages object exported by `info.js` that map to instances of the `Event` class from `lib/link`.
The name of the property correspond to the handler invoked on the plugin class.
The Event constructor takes an object of properties that define the event, for example the following could be defined in `info.js`:
Messages are defined as items of the `plugin.messages` array exported by `index.js`.
For example, the following could be defined in `plugin`:
```js
messages: {
startFrobnication: new libLink.Event({
type: "foo_frobber:start_frobnication",
links: ["master-slave", "slave-instance"],
forwardTo: "instance",
eventRequired: ["frobnication_type"],
eventProperties: {
"frobnication_type": { type: "string" },
"urgent": { type: "boolean" },
},
}),
},
```
This specifies an event that can be sent from the master to a slave, and from a slave to an instance.
It also specifies that the event must contain the property `frobnication_type`, with a string value in the data payload and that it may optionally contain a boolean `urgent` property.
It will also be forwarded by slaves to a specific instance.
The following properties are recognized by the Event constructor:
#### type
The message type sent over the wire.
This must start with the name of the plugin followed by colon and and be unique for the plugin.
The type of the message sent over the socket will have the suffix `_event` appended to it.
#### links
An array of strings describing which links this event can be sent over.
Direction matters, `"master-slave"` means the event can be sent from the master to the slave, but can't be sent back the other way, unless `"slave-master"` is also present in the links array.
The available endpoints are `master`, `slave`, `instance`, and `control`.
Master talks with slave and control, and slave talks to instance.
The full chain must be specified as the individual links in order for a message to travers multiple hops, (i.e., for a message to go from master to instance it must have both `"master-slave"` and `"slave-instance"` in the links array).
See `forwardTo` and `broadcastTo` for ways to forward an event to the next link in a chain.
#### forwardTo
Target to forward an event to.
Can either be `"master"`, to indicate a slave should forward it to the master server, or `"instance"`, to indicate it should be forwarded to the instances specified by the `instance_id` event property.
This works by using a default handler for the event at the links that forward it.
#### broadcastTo
Target to broadcast this message towards.
A value of "instance" means the event will be broadcast to all instances downstream of the target it's sent to, but not back from where it came from.
Currently, only "instance" is supported.
This means that sending the event to a slave from an instance will cause it to be broadcast to all instances of that slave except for the instance it came from.
#### eventRequired
By default all properties defined in `eventProperties` are required to be present in the payload for the event.
This can be overriden by specifying an array of properties which are required here.
This is equivalent to using the `required` keyword in JSON schema.
#### eventProperties
Object with properties mapping to a JSON schema of that property that specifies what's valid to send in the event.
This is equivalent to using the `properties` keyword in JSON schema, except that the properties specified are by default required and additional properties are not allowed.
See [this guide][guide] for an introduction to writing JSON schemas.
The forwardTo and broadcastTo can be combined such that specifying `"master"` as the forwardTo value and `"instance"` as the broadcastTo value will cause the event to be broadcast to all instances in the cluster.
For this to work, you will need to specify `instance-slave`, `slave-master`, `master-slave`, and `slave-instance` as the links.
Keep in mind when forwarding events that if the target an event is being forwarded to is not online, the event will be dropped.
Use a request if you need a confirmation that the message was received.
### Definining Requests
Requests are defined as properties of the messages object exported by `info.js` that map to instances of the `Request` class from `lib/link`.
The name of the property corresponds to the handler invoked on the plugin class.
The Request constructor takes an object of properties that define the event.
For example, the following could be defined in `info.js`:
```js
messages: {
reportFrobnication: new libLink.Request({
type: "foo_frobber:report_frobnication",
links: ["master-slave", "slave-instance"],
forwardTo: "instance",
requestRequired: ["verbosity"],
requestProperties: {
"verbosity": { type: "integer" },
"special": { type: "boolean" },
},
responseProperties: {
"report": {
type: "array",
items: { type: "string" },
messages: [
class Frobnicate {
static plugin = "foo_frobber"; // Plugin name
static type = "request";
static src = "controller";
static dst = "instance";
static jsonSchema = {
type: "object",
properties: {
"verbosity": { type: "integer" },
"special": { type: "boolean" },
},
},
}),
},
required: ["verbosity"],
additionalProperties: false,
};
constructor(json) {
this.verbosity = json.verbosity;
this.special = json.special;
}
static fromJSON(json) {
return new Frobnicate(json);
}
toJSON() {
return {
verbosity: this.verbosity,
special: this.special,
};
}
static Response = {
jsonSchema: {
type: "object",
properties: {
"report": {
type: "array",
items: { type: "string" },
},
},
},
fromJSON(json) {
return json
}
}
}
]
```
This specifies a request that can be sent from the master to a slave, and from a slave to an instance.
The request data must contain the property `verbosity` with an integer number as the value, as well as the `instance_id` property (implied by `forwardTo: "instance"`) and it may also contain a boolean `special` property.
It also defines that the response sent must contain a `report` property mapping to an array of strings.
When received by a slave, it will also be forwarded to the instance specified by `instance_id`.
This specifies a request that can be sent from the controller to an instance, being routed through the correct host behind the scenes.
The request data must contain the property `verbosity` with an integer number as the value, and it may also contain a boolean `special` property.
It also defines that the response sent may contain a `report` property mapping to an array of strings.
The following properties are recognized by the Request constructor:
The following properties are recognized as part of a message class:
#### type
#### static plugin
The message type sent over the wire.
This must start with the name of the plugin followed by colon and and be unique for the plugin.
The type of the message sent over the socket will have the suffix `_request` appended to it for the request and `_response` appended to it for the response.
Must contain the name of the plugin this message is defined by.
This property is not used in core messages.
#### links
#### static type
An array of strings describing which links this request can be sent over.
Direction matters; `"master-slave"` means the request can be sent from the master to the slave and the slave can reply to the master, but the slave can't send a request to the master unless `"slave-master"` is also present in the links array.
A string describing the message type sent over the wire.
This can be either `"request"` or `"event"`.
The primary difference is whether they allow for a response.
The available endpoints are `master`, `slave`, `instance`, and `control`.
Master talks with slave and control, and slave talks to instance.
The full chain must be specified as the individual links in order for a message to travers multiple hops, (i.e., for a message to go from master to instance it must have both `"master-slave"` and `"slave-instance"` in the links array).
See `forwardTo` for ways to forward a request to the next link in a chain.
#### static src
#### forwardTo
A string or an array of strings describing where this message is allowed to originate from.
The available endpoints are `"controller"`, `"host"`, `"instance"`, or `"control"`.
Messages sent from a different source will be rejected.
Target to forward the request to.
Can either be `"master"` to indicate a slave should forward it to the master server when receiving it from an instance, or `"instance"` to indicate it should be forwarded to the instances specified by the `instance_id` request property.
This works by using a default handler for the request by the links that forward it.
#### static dst
#### requestRequired
A string or array of strings describing where this message may be addressed to.
The available endpoints are `"controller"`, `"host"`, `"instance"`, or `"control"`.
If the target is multiple hops away it the message will be forwarded towards its destination by the intermediaries.
For example a message sent from a control connection an instance will be forwarded by the controller to the relevant host and from the host to the correct instance.
Messages sent to a different destination will be rejected.
By default all properties defined in `requestProperties` are required to be present in the payload for the request.
This can be overriden by specifying an array of properties which are required here.
This is equivalent to using the `required` keyword in JSON schema.
#### static jsonSchema
#### requestProperties
A JSON schema that specifies what's valid to send in the message.
Object with properties mapping to a JSON schema of that property that specifies what's valid to send in the request.
This is equivalent to using the `properties` keyword in JSON schema, except that the properties specified are by default required and additional properties are not allowed.
See [this guide][guide] for an introduction to writing JSON schemas
#### toJSON
#### responseRequired
Optional method to serialize an instance of the class to a JSON object, see the [documentation for `JSON.stringify()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify#description) for the semantics of this method.
This should return an object that's valid according to `jsonSchema` and accepted by `fromJSON`, and may be omitted if the default serialisation by `JSON.stringify()` already does this.
Same as the requestRequired only for the response sent back by the target.
#### static fromJSON
#### responseProperties
Required function to restore the class from a JSON object.
This function is called when the message reaches its destination and is expected to return an identical instance of the class.
I.e. the result of `fromJSON(JSON.parse(JSON.stringify(instance)))` should be a complete copy of the message.
Same as the requestProperties only for the response sent back by the target.
#### static Response
Optional [JSON sterilisable class](/docs/devs/json-serialisable-classes.md) that specifies the response to the request.
Required properties are `jsonSchema` and `fromJSON`.
## Sending Link Messages
Link messages are sent by calling the `.send()` method on the Event/Request instance with the link you want to send it over and the data to send.
For `InstancePlugin` code the link to the slave is the `instance` itself, which is accessible through the `.instance` property of the `InstancePlugin`.
The `.info` property of the plugin class exposes the data exported from the plugin's `info.js` module.
In other words:
Link messages are sent by calling the `.sendTo()` method on a connection object with the destination and the data you want to send, where the data is an instance of one of the defined message classes in plugin.messages.
For example:
```js
// In an InstancePlugin class
async frobnicate() {
this.info.messages.exampleEvent.send(this.instance, { foo: "bar" });
const response = await this.instance.sendTo("controller", new messages.Frobnicate({
verbosity: 2,
special: false,
}));
console.log(response); // { report: [...] }
}
```
For the Request class the send method is async and returns the response data received from the target it was sent to, or throws an error if the request failed.
For classes with `static type = "request"` the send method is async and returns a promise that resolves to the response data received from the target it was sent to, or rejects with an error if the request failed.
The destination specification can either be the ID of a particular control, host or instance, or one of the keywords used to send to multiple targets at once.
Example destinations:
```js
sendTo("controller", message);
sendTo("allHosts", message);
sendTo("allControls", message);
sendTo("allInstances", message);
sendTo({ hostId: 123 }, message);
sendTo({ instanceId: 123 }, message);
sendTo({ controlId: 123 }, message);
```
### Handling connection events
There are a few connection related events that plugins neeed to repsond to in order to avoid data loss and connection problems.
The most important is the prepare disconnect for the link between master and slave.
This is signaled to `MasterPlugin` classes via the `onPrepareSlaveDisconnect` hook and to `InstancePlugin` classes via the `onPrepareMasterDisconnect` hook.
The most important is the prepare disconnect for the link between controller and host.
This is signaled to `ControllerPlugin` classes via the `onPrepareHostDisconnect` hook and to `InstancePlugin` classes via the `onPrepareControllerDisconnect` hook.
After the prepare disconnect the connection will be closed, which will result in pending requests and events being dropped.
Plugins must respond to the prepare disconnect by stopping any processess it does that send events or requests over the link in question.
This can be accomplished either through listening for the prepare disconnect hook, or by checking the `connected` property of the `SlaveConnection` class and `Slave` class on the master and slave respectively.
This can be accomplished either through listening for the prepare disconnect hook, or by checking the `connected` property of the `HostConnection` class and `Host` class on the controller and host respectively.
For example the sending of an event from an `InstancePlugin` class can be stopped while the connection is not connected, not in the dropped state, and not in the process of discunnecting by using the following code:
```js
if (this.slave.connected) {
this.info.messages.frobnicate.send(this.instance, { foo: "bar" });
if (this.host.connected) {
this.instance.sendTo("controller", new messages.Frobnicate({ foo: "bar" }));
}
```
If the event or request needs to be sent to the master it can be put into a queue stored on the plugin instance and sent out when the connection is established again.
The re-establishement of the connection is notified to plugins via the `connect` event to the `onMasterConnectionEvent` and `onSlaveConnectionEvent` hooks.
If the event or request needs to be sent to the controller it can be put into a queue stored on the plugin instance and sent out when the connection is established again.
The re-establishement of the connection is notified to plugins via the `connect` event to the `onControllerConnectionEvent` and `onHostConnectionEvent` hooks.
The second connection event which is of lesser importance to respond to is the `drop` connection event served through `onMasterConnectionEvent` for `InstancePlugin` classes and through `onSlaveConnectionEvent` for `MasterPlugin` classes.
This is raised when the connection between the master and slave in question is lost, most likely due to networking issues.
When in the dropped state the slave will keep trying to reconnect to the master server in order to re-establish it, and if successful no events or requests will be lost.
The second connection event which is of lesser importance to respond to is the `drop` connection event served through `onControllerConnectionEvent` for `InstancePlugin` classes and through `onHostConnectionEvent` for `ControllerPlugin` classes.
This is raised when the connection between the controller and host in question is lost, most likely due to networking issues.
When in the dropped state the host will keep trying to reconnect to the controller in order to re-establish it, and if successful no events or requests will be lost.
However while in the dropped state any requests and events sent gets queued up in memory until the connection is either re-established or the session times out.
This means that if your plugin sends a lot of events or requests, they can end up being queued up in a buffer and sent out all at once the connection is re-estabished.
To avoid this you should be throtteling and/or stopping your requests/events after `drop` has been raised, and continue back as normal when `resume` is raised.
@@ -530,23 +506,23 @@ In its simplest form collecting data from plugins consists of defining the metri
For example:
```js
const { Counter } = require("@clusterio/lib/prometheus");
const { Counter } = require("@clusterio/lib");
const fooMetric = new Counter(
"clusterio_foo_frobber_foo_metric", "Measures the level of foo",
);
// Somewhere in the master plugin code
// Somewhere in the controller plugin code
fooMetric.inc();
```
This works for master plugins, and the metric will be automatically available through the /metric HTTP endpoint.
This works for controller plugins, and the metric will be automatically available through the /metric HTTP endpoint.
It's recommended that plugin metrics follow `clusterio_<plugin_name>_<metric_name>` as the naming scheme.
For metrics that are per-instance, you must define an `instance_id` label and set it accordingly, for example:
```js
const { Counter } = require("@clusterio/lib/prometheus");
const { Counter } = require("@clusterio/lib");
const barMetric = new Gauge(
"clusterio_foo_frobber_bar_metric", "Bar instance level",
@@ -557,11 +533,11 @@ const barMetric = new Gauge(
barMetric.labels(String(this.instance.id)).set(someValue);
```
Metrics are automatically registered to the default registry, and this default registry is automatically polled by the master server on slaves.
This means that it's important that you place the definition of the metric at module level so that it's not created more than once over the lifetime of a slave.
Metrics are automatically registered to the default registry, and this default registry is automatically polled by the controller on hosts.
This means that it's important that you place the definition of the metric at module level so that it's not created more than once over the lifetime of a host.
Since the metrics remember their values and would continue to be exported after an instance is shutdown, there's code at instance shutdown that removes all the values where the `instance_id` label matches the id of the instance shut down.
For statistics you need to update on collection there's an `onMetrics` hook on both master and instance plugins that is run before the metrics in the default registry are collected.
For statistics you need to update on collection there's an `onMetrics` hook on both controller and instance plugins that is run before the metrics in the default registry are collected.
## Adding Custom Commands to clusterioctl
@@ -570,7 +546,7 @@ The control entrypoint for plugins allows you to extend clustectl with your own
The creation of custom commands typically starts with defining a command tree for the plugin:
```js
const { Command, CommandTree } = require("@clusterio/lib/command");
const { Command, CommandTree } = require("@clusterio/lib");
const fooFrobberCommands = new CommandTree({
name: "foo-frobber", description: "Foo Frobber Plugin commands"
});
@@ -600,20 +576,20 @@ For a command the `definition` is the arguments to pass to [yargs.command](http:
The `handler` is an async function that's invoked when the command is executed and it's passed the parsed command line arguments and a reference to the `Control` class of clusterioctl.
It's possible to optain a reference to the plugin class with `control.plugins.get(info.name)`.
Note that messages sent from clusterioctl needs to have `"control-master"` as a part of the links array for it to be accepted by the master server, see [Defining Link Messages](#defining-link-messages) for how to define the messages that can be sent to the master.
Note that messages sent from clusterioctl needs to have `"control-controller"` as a part of the links array for it to be accepted by the controller, see [Defining Link Messages](#defining-link-messages) for how to define the messages that can be sent to the controller.
To have the command tree become part of clusterioctl it needs to be added to the rootCommand tree in `addCommands` callback of the Control plugin:
To have the command tree become part of clusterioctl it needs to be added to the rootCommand tree in `addCommands` callback of the Ctl plugin:
```js
const libPlugin = require("@clusterio/lib/plugin");
const { BaseCtlPlugin } = require("@clusterio/ctl");
class ControlPlugin extends libPlugin.BaseControlPlugin {
class CtlPlugin extends BaseCtlPlugin {
async addCommands(rootCommand) {
rootCommand.add(fooFrobberCommands);
}
}
module.exports = {
ControlPlugin,
CtlPlugin,
}
```
```
+3
View File
@@ -0,0 +1,3 @@
*
!.gitignore
!README.md
+4
View File
@@ -0,0 +1,4 @@
# External plugins directory
Plugins placed here will have their npm dependencies incorporated in the shared pnpm storage for this repository when `pnpm install` is invoked.
Use this to enable `--dev-plugin` to work with plugins developed outside of the Clusterio repository.
-8
View File
@@ -1,8 +0,0 @@
{
"packages": [
"packages/*",
"plugins/*",
"test"
],
"version": "2.0.0-alpha.10"
}
+45 -29
View File
@@ -1,31 +1,47 @@
{
"name": "root",
"private": true,
"scripts": {
"test": "mocha --check-leaks -R spec --exclude \"**/node_modules/**\" --recursive test plugins/global_chat/test plugins/research_sync/test plugins/statistics_exporter/test plugins/subspace_storage/test plugins/player_auth/test",
"fast-test": "FAST_TEST=y npm test",
"cover": "nyc npm test",
"fast-cover": "FAST_TEST=y nyc npm test",
"ci-cover": "nyc --reporter=lcovonly npm run-script test",
"lint": "eslint \"*.js\" packages plugins test",
"docs": "jsdoc --configure docs/jsdoc.json",
"bundle-dependencies": "bundle-dependencies"
},
"nyc": {
"exclude": [
"test/**",
"plugins/*/test/**"
]
},
"engines": {
"node": ">=12"
},
"devDependencies": {
"eslint": "^7.25.0",
"eslint-plugin-node": "^11.1.0",
"jsdoc": "^3.6.6",
"lerna": "^4.0.0",
"mocha": "^8.4.0",
"nyc": "^15.1.0"
}
"name": "root",
"private": true,
"scripts": {
"build": "tsc --build",
"watch": "tsc --build --watch",
"test": "pnpm run build && mocha --check-leaks -R spec --recursive test \"plugins/*/test/**\" --exclude \"test/manual/**\"",
"fast-test": "FAST_TEST=y pnpm test",
"cover": "nyc pnpm test",
"fast-cover": "FAST_TEST=y nyc pnpm test",
"ci-cover": "nyc --reporter=lcovonly pnpm run-script test",
"lint": "eslint packages plugins test",
"lint-fix": "eslint --fix packages plugins test",
"docs": "typedoc",
"bundle-dependencies": "bundle-dependencies"
},
"nyc": {
"exclude": [
"test/**",
"plugins/*/test/**"
]
},
"engines": {
"node": ">=18"
},
"devDependencies": {
"@clusterio/controller": "workspace:*",
"@clusterio/ctl": "workspace:*",
"@clusterio/host": "workspace:*",
"@clusterio/lib": "workspace:*",
"@swc/core": "^1.4.0",
"@typescript-eslint/eslint-plugin": "^6.4.1",
"@typescript-eslint/parser": "^6.4.1",
"eslint": "^8.45.0",
"eslint-plugin-node": "^11.1.0",
"express": "^4.18.2",
"form-data": "^4.0.0",
"fs-extra": "^11.1.1",
"jsonwebtoken": "^9.0.1",
"jszip": "^3.10.0",
"mocha": "^10.2.0",
"nyc": "^15.1.0",
"phin": "^3.7.0",
"typedoc": "^0.24.8",
"typescript": "^5.1.6"
}
}
@@ -1 +1,2 @@
webpack.config.js
dist/browser
@@ -1,27 +1,27 @@
# Clusterio Master Server
# Clusterio Controller
Communication hub for Clusterio clusters.
The master server forwards data between Clusterio slaves connected to it and allows the cluster to be remotely managed through WebSocket connections to it either by using the included web interface or the Clusterio ctl command line utility.
The controller forwards data between Clusterio hosts connected to it and allows the cluster to be remotely managed through WebSocket connections to it either by using the included web interface or the Clusterio ctl command line utility.
# Usage
npx clusteriomaster <command>
npx clusteriocontroller <command>
Common options:
* `--plugin-list <file>` JSON file to use for storing the list of plugins that are available to the master server.
* `--plugin-list <file>` JSON file to use for storing the list of plugins that are available to the controller.
Defaults to `plugin-list.json` and will be created if it does not exist.
See the `plugin` command for managing this list.
* `--config <file>` JSON file to use for storing configuration for the master server.
Defaults to `config-master.json` and will be created if it does not exist.
* `--config <file>` JSON file to use for storing configuration for the controller.
Defaults to `config-controller.json` and will be created if it does not exist.
See the `config` command for inspecting and modifying the configuration.
### `plugin <command>`
Configure plugins available to be loaded by the master server.
Configure plugins available to be loaded by the controller.
The available plugins will be loaded unless they have been disabled in the configuration, see the config command for disabling plugins.
@@ -33,9 +33,9 @@ A relative path must start with ./ or ../ (or .\ and ..\ on Windows) otherwise i
For example, installing the Subspace Storage plugin:
npm install @clusterio/plugin-subspace_storage
npx clusteriomaster plugin add @clusterio/plugin-subspace_storage
npx clusteriocontroller plugin add @clusterio/plugin-subspace_storage
Since the `plugin-list.json` is shared between master, slave and ctl you usually only need to do this once per machine.
Since the `plugin-list.json` is shared between controller, host and ctl you usually only need to do this once per machine.
#### `plugin remove <name>`
@@ -46,10 +46,10 @@ Removing and unistalling a plugin is usually not neccessary as the functions pro
For example, uninstalling the Subspace Storage plugin:
npx clusteriomaster plugin remove subspace_storage
npx clusteriocontroller plugin remove subspace_storage
npm uninstall @clusterio/plugin-subspace_storage
Since the `plugin-list.json` is shared between master, slave and ctl you usually only need to do this once per machine.
Since the `plugin-list.json` is shared between controller, host and ctl you usually only need to do this once per machine.
#### `plugin list`
@@ -59,8 +59,8 @@ Lists the plugins set up to be available by name followed by path.
### `config`
Manage the master server configuration offline.
This should only be used when the master server is stopped, otherwise the config read might be out of date and config changes will be overwritten when the master server shuts down.
Manage the controller configuration offline.
This should only be used when the controller is stopped, otherwise the config read might be out of date and config changes will be overwritten when the controller shuts down.
#### `config set <config-entry> [value]`
@@ -96,7 +96,7 @@ You should be using your own Factorio multiplayer username here.
#### `bootstrap generate-user-token <name>`
Generate an access token for logging in and possibly managing the cluster from the given user account.
This can be used both in the web interface and as the `master_token` to connect with clusterioctl.
This can be used both in the web interface and as the `controller_token` to connect with clusterioctl.
#### `bootstrap create-ctl-config <name>`
@@ -106,7 +106,7 @@ Creates a clusterioctl config for the given user with url and token set up for c
### `run`
Runs the master server.
Runs the controller.
## See Also
+490
View File
@@ -0,0 +1,490 @@
#!/usr/bin/env node
/**
* Clusterio controller
*
* Facilitates communication between hosts and control of the cluster
* through WebSocet connections, and hosts a webserver for browser
* interfaces and Prometheus statistics export. It is remotely controlled
* by {@link module:ctl/ctl}.
*
* @module controller/controller
* @author Danielv123, Hornwitser
* @example
* npx clusteriocontroller run
*/
import path from "path";
import fs from "fs-extra";
import crypto from "crypto";
import setBlocking from "set-blocking";
import yargs from "yargs";
import util from "util";
import winston from "winston";
import DailyRotateFile from "winston-daily-rotate-file";
import jwt from "jsonwebtoken";
// homebrew modules
import * as lib from "@clusterio/lib";
const { ConsoleTransport, levels, logger } = lib;
import Controller from "./src/Controller";
import UserManager from "./src/UserManager";
import { version } from "./package.json";
// globals
let controller: Controller;
void new lib.Gauge(
"clusterio_controller_pending_requests",
"Count of pending link requests currently waiting in memory on the controller.",
{
callback: (gauge) => {
let count = 0;
if (controller) {
for (const host of controller.wsServer.hostConnections.values()) {
count += host.pendingRequestCount;
}
for (const control of controller.wsServer.controlConnections.values()) {
count += control.pendingRequestCount;
}
}
gauge.set(count);
},
}
);
void new lib.Gauge(
"clusterio_controller_host_mapping",
"Mapping of Host ID to name",
{
labels: ["host_id", "host_name"],
callback: function(gauge:lib.Gauge) {
gauge.clear();
if (!controller || !controller.hosts) {
return;
}
for (let [id, host] of controller.hosts) {
gauge.labels({
host_id: String(id),
host_name: host.name,
}).set(1);
};
},
}
);
void new lib.Gauge(
"clusterio_controller_instance_mapping",
"Mapping of Instance ID to name and host",
{
labels: ["instance_id", "instance_name", "host_id"],
callback: function(gauge:lib.Gauge) {
gauge.clear();
if (!controller || !controller.instances) {
return;
}
for (let [id, instance] of controller.instances) {
gauge.labels({
instance_id: String(id),
instance_name: String(instance.config.get("instance.name")),
host_id: String(instance.config.get("instance.assigned_host")),
}).set(1);
}
},
}
);
void new lib.Gauge(
"clusterio_controller_websocket_active_connections",
"How many WebSocket connections are currently open to the controller",
{ callback: function(gauge:lib.Gauge) {
gauge.set(controller.wsServer.activeConnectors.size);
}}
);
void new lib.Gauge(
"clusterio_controller_active_hosts",
"How many hosts are currently connected to the controller",
{ callback: function(gauge:lib.Gauge) {
gauge.set(controller.wsServer.hostConnections.size);
}}
);
void new lib.Gauge(
"clusterio_controller_connected_clients_count",
"How many clients are currently connected to this controller",
{
labels: ["type"], callback: async function(gauge:lib.Gauge) {
gauge.labels("host").set(controller.wsServer.hostConnections.size);
gauge.labels("control").set(controller.wsServer.controlConnections.size);
},
},
);
async function handleBootstrapCommand(
args: any,
controllerConfig: lib.ControllerConfig
): Promise<void> {
let subCommand = args._[1];
let userManager = new UserManager(controllerConfig);
await userManager.load(path.join(controllerConfig.get("controller.database_directory"), "users.json"));
if (subCommand === "create-admin") {
if (!args.name) {
logger.error("name cannot be blank");
process.exitCode = 1;
return;
}
let admin = userManager.users.get(args.name);
if (!admin) {
admin = userManager.createUser(args.name);
}
let adminRole = lib.ensureDefaultAdminRole(userManager.roles);
admin.roleIds.add(adminRole.id);
admin.isAdmin = true;
await userManager.save(path.join(controllerConfig.get("controller.database_directory"), "users.json"));
} else if (subCommand === "generate-user-token") {
let user = userManager.users.get(args.name);
if (!user) {
logger.error(`No user named '${args.name}'`);
process.exitCode = 1;
return;
}
// eslint-disable-next-line no-console
console.log(userManager.signUserToken(user.name));
} else if (subCommand === "generate-host-token") {
// eslint-disable-next-line no-console
console.log(jwt.sign(
{ aud: "host", host: args.id },
Buffer.from(controllerConfig.get("controller.auth_secret"), "base64")
));
} else if (subCommand === "create-ctl-config") {
let admin = userManager.users.get(args.name);
if (!admin) {
logger.error(`No user named '${args.name}'`);
process.exitCode = 1;
return;
}
let controlConfig = new lib.ControlConfig("control");
controlConfig.set("control.controller_url", Controller.calculateControllerUrl(controllerConfig));
controlConfig.set(
"control.controller_token",
userManager.signUserToken(admin.name),
);
let content = JSON.stringify(controlConfig, null, "\t");
if (args.output === "-") {
// eslint-disable-next-line no-console
console.log(content);
} else {
logger.info(`Writing ${args.output}`);
await lib.safeOutputFile(args.output, content);
}
}
}
export interface ControllerArgs {
[x: string]: unknown;
logLevel: string;
logDirectory: string;
pluginList: string;
devPlugin?: (string | number)[] | undefined;
dev?: boolean;
config: string;
_: (string | number)[];
$0: string;
}
interface InitializeParameters {
args: ControllerArgs;
shouldRun: boolean;
clusterLogger: winston.Logger;
pluginInfos: lib.PluginNodeEnvInfo[] | null;
controllerConfigPath: string | null;
controllerConfig: lib.ControllerConfig | null;
}
async function initialize(): Promise<InitializeParameters> {
// argument parsing
const args = await yargs
.scriptName("controller")
.usage("$0 <command> [options]")
.option("log-level", {
nargs: 1,
describe: "Log level to print to stdout",
default: "info",
choices: ["none"].concat(Object.keys(levels)),
type: "string",
})
.option("log-directory", {
nargs: 1,
describe: "Directory to place logs in",
default: "logs",
type: "string",
})
.option("config", {
nargs: 1,
describe: "controller config file to use",
default: "config-controller.json",
type: "string",
})
.option("plugin-list", {
nargs: 1,
describe: "File containing list of plugins available with their install path",
default: "plugin-list.json",
type: "string",
})
.command("plugin", "Manage available plugins", lib.pluginCommand)
.command("config", "Manage Controller config", lib.configCommand)
.command("bootstrap", "Bootstrap access to cluster", yargs => {
yargs
.command("create-admin <name>", "Create a cluster admin")
.command("generate-user-token <name>", "Generate authentication token for the given user")
.command("generate-host-token <id>", "Generate authentication token for the given host", yargs => {
yargs.positional("id", { describe: "ID of the host", type: "number" });
})
.command("create-ctl-config <name>", "Create clusterioctl config for the given user", yargs => {
yargs.option("output", {
describe: "Path to output config (- for stdout)", type: "string",
nargs: 1, default: "config-control.json",
});
})
.demandCommand(1, "You need to specify a command to run");
})
.command("run", "Run controller", yargs => {
yargs.option("can-restart", {
type: "boolean", nargs: 0, default: false,
describe: "Indicate that a process monitor will restart the controller on failure",
});
yargs.option("dev", { hidden: true, type: "boolean", nargs: 0 });
yargs.option("dev-plugin", { hidden: true, type: "array" });
})
.demandCommand(1, "You need to specify a command to run")
.strict()
.argv
;
// Combined log stream of the whole cluster.
const clusterLogger = winston.createLogger({
format: winston.format.json(),
level: "verbose",
levels,
});
clusterLogger.add(new DailyRotateFile({
format: winston.format.json(),
filename: "cluster-%DATE%.log",
utc: true,
dirname: path.join(args.logDirectory, "cluster"),
}));
// Log stream for the controller.
logger.add(new DailyRotateFile({
format: winston.format.json(),
filename: "controller-%DATE%.log",
utc: true,
dirname: path.join(args.logDirectory, "controller"),
}));
logger.add(new winston.transports.Stream({
stream: clusterLogger,
}));
if (args.logLevel !== "none") {
logger.add(new ConsoleTransport({
level: args.logLevel,
format: new lib.TerminalFormat(),
}));
}
lib.handleUnhandledErrors();
let command = args._[0];
let shouldRun = false;
if (command === "run") {
logger.info(`Starting Clusterio controller ${version}`);
shouldRun = true;
}
logger.info(`Loading available plugins from ${args.pluginList}`);
let pluginList = new Map();
try {
pluginList = new Map(JSON.parse(await fs.readFile(args.pluginList, { encoding: "utf8" })));
} catch (err: any) {
if (err.code !== "ENOENT") {
throw err;
}
}
// If the command is plugin management we don't try to load plugins
if (command === "plugin") {
await lib.handlePluginCommand(args, pluginList, args.pluginList);
return {
args,
shouldRun,
clusterLogger,
pluginInfos: null,
controllerConfigPath: null,
controllerConfig: null,
};
}
logger.info("Loading Plugin info");
const pluginInfos = await lib.loadPluginInfos(pluginList);
lib.registerPluginMessages(pluginInfos);
lib.addPluginConfigFields(pluginInfos);
const controllerConfigPath = args.config;
logger.info(`Loading config from ${controllerConfigPath}`);
let controllerConfig = new lib.ControllerConfig("controller");
try {
let jsonConfig = JSON.parse(await fs.readFile(controllerConfigPath, { encoding: "utf8" }));
controllerConfig = lib.ControllerConfig.fromJSON(jsonConfig, "controller");
} catch (err: any) {
if (err.code === "ENOENT") {
logger.info("Config not found, initializing new config");
controllerConfig = new lib.ControllerConfig("controller");
} else {
throw new lib.StartupError(`Failed to load ${controllerConfigPath}: ${err.message}`);
}
}
if (!controllerConfig.get("controller.auth_secret")) {
logger.info("Generating new controller authentication secret");
let asyncRandomBytes = util.promisify(crypto.randomBytes);
let bytes = await asyncRandomBytes(256);
controllerConfig.set("controller.auth_secret", bytes.toString("base64"));
await lib.safeOutputFile(
controllerConfigPath, JSON.stringify(controllerConfig, null, "\t")
);
}
if (command === "config") {
await lib.handleConfigCommand(
args, controllerConfig, controllerConfigPath
);
} else if (command === "bootstrap") {
await handleBootstrapCommand(args, controllerConfig);
}
return {
args,
clusterLogger,
pluginInfos,
controllerConfigPath,
controllerConfig,
shouldRun,
};
}
async function startup() {
// Set the process title, shows up as the title of the CMD window on windows
// and as the process name in ps/top on linux.
process.title = "clusteriocontroller";
let {
args,
shouldRun,
clusterLogger,
pluginInfos,
controllerConfigPath,
controllerConfig,
} = await initialize();
if (!shouldRun || !pluginInfos || !controllerConfigPath || !controllerConfig) {
return;
}
controller = new Controller(
clusterLogger,
pluginInfos,
controllerConfigPath,
controllerConfig,
Boolean(args.canRestart),
...await Controller.bootstrap(controllerConfig)
);
let secondSigint = false;
process.on("SIGINT", () => {
if (secondSigint) {
setBlocking(true);
logger.fatal("Caught second interrupt, terminating immediately");
// eslint-disable-next-line node/no-process-exit
process.exit(1);
}
secondSigint = true;
logger.info("Caught interrupt signal, shutting down");
controller.stop();
});
let secondSigterm = false;
process.on("SIGTERM", () => {
if (secondSigterm) {
setBlocking(true);
logger.fatal("Caught second termination, terminating immediately");
// eslint-disable-next-line node/no-process-exit
process.exit(1);
}
secondSigterm = true;
logger.info("Caught termination signal, shutting down");
controller.stop();
});
process.on("SIGHUP", () => {
logger.info("Terminal closed, shutting down");
controller.stop();
});
await controller.start(args);
}
export function bootstrap() {
// eslint-disable-next-line no-console
console.warn(`
+==========================================================+
I WARNING: This is the development branch for the 2.0 I
I version of clusterio. Expect things to break. I
+==========================================================+
`
);
startup().catch(err => {
if (err instanceof lib.StartupError) {
logger.fatal(`
+-------------------------------+
| Unable to to start controller |
+-------------------------------+
${err.stack}`
);
} else if (err instanceof lib.PluginError) {
logger.fatal(`
${err.pluginName} plugin threw an unexpected error
during startup, please report it to the plugin author.
------------------------------------------------------
${err.original.stack}`
);
} else {
logger.fatal(`
+------------------------------------------------------------+
| Unexpected error occured while starting controller, please |
| report it to https://github.com/clusterio/clusterio/issues |
+------------------------------------------------------------+
${err.stack}`
);
}
process.exitCode = 1;
});
}
if (module === require.main) {
bootstrap();
}
+14
View File
@@ -0,0 +1,14 @@
import { bootstrap } from "./controller";
export { default as Controller } from "./src/Controller";
export { default as ControllerUser } from "./src/ControllerUser";
export { default as ControllerRouter } from "./src/ControllerRouter";
export { default as ControlConnection } from "./src/ControlConnection";
export { default as HostConnection } from "./src/HostConnection";
export { default as InstanceInfo } from "./src/InstanceInfo";
export { default as BaseControllerPlugin } from "./src/BaseControllerPlugin";
export { default as UserManager } from "./src/UserManager";
export { default as WsServer } from "./src/WsServer";
if (module === require.main) {
bootstrap();
}
+62
View File
@@ -0,0 +1,62 @@
{
"name": "@clusterio/controller",
"description": "Implementation of Clusterio controller",
"author": "Hornwitser",
"license": "MIT",
"version": "2.0.0-alpha.18",
"main": "dist/node/index.js",
"scripts": {
"prepare": "tsc --build && webpack-cli --env production",
"test": "echo \"Error: no test specified\" && exit 1"
},
"bin": {
"clusteriocontroller": "./dist/node/controller.js"
},
"keywords": [
"factorio"
],
"engines": {
"node": ">=18"
},
"dependencies": {
"@clusterio/lib": "^2.0.0-alpha.18",
"@sinclair/typebox": "^0.30.4",
"busboy": "^1.6.0",
"compression": "^1.7.4",
"express": "^4.18.2",
"finalhandler": "^1.2.0",
"fs-extra": "^11.1.1",
"jsonwebtoken": "^9.0.1",
"jszip": "^3.10.1",
"set-blocking": "^2.0.0",
"webpack-cli": "^5.1.4",
"winston": "^3.10.0",
"winston-daily-rotate-file": "^4.7.1",
"ws": "^8.13.0",
"yargs": "^17.7.2"
},
"devDependencies": {
"@clusterio/web_ui": "^2.0.0-alpha.18",
"@swc/core": "^1.4.0",
"@types/busboy": "^1.5.0",
"@types/compression": "^1.7.2",
"@types/express": "^4.17.17",
"@types/finalhandler": "^1.2.0",
"@types/fs-extra": "^11.0.1",
"@types/jsonwebtoken": "^9.0.2",
"@types/node": "^20.4.5",
"@types/set-blocking": "^2.0.0",
"@types/ws": "^8.5.5",
"@types/yargs": "^17.0.24",
"react": "^18.2.0",
"react-dom": "^18.2.0",
"react-router-dom": "^6.4.14",
"typescript": "^5.1.6",
"webpack": "^5.88.2",
"webpack-dev-middleware": "^6.1.1",
"webpack-merge": "^5.9.0"
},
"publishConfig": {
"access": "public"
}
}
+109
View File
@@ -0,0 +1,109 @@
import path from "path";
import fs from "fs-extra";
import type Controller from "./Controller";
import type WsServerConnector from "./WsServerConnector";
import * as lib from "@clusterio/lib";
const { logger } = lib;
import * as routes from "./routes";
/**
* Base class for controller connections
*
* @extends module:lib.Link
* @alias module:controller/src/BaseConnection
*/
export default class BaseConnection extends lib.Link {
declare ["connector"]: WsServerConnector;
constructor(
connector: lib.WebSocketBaseConnector,
public _controller: Controller
) {
super(connector);
this.router = _controller.router;
for (let [Request, handler] of this._controller._registeredRequests) { this.handle(Request, handler); }
for (let [Request, handler] of this._controller._fallbackedRequests) { this.fallbackRequest(Request, handler); }
for (let [Event, handler] of this._controller._registeredEvents) { this.handle(Event, handler); }
for (let [Event, handler] of this._controller._snoopedEvents) { this.snoopEvent(Event, handler); }
this.handle(lib.ModPackGetRequest, this.handleModPackGetRequest.bind(this));
this.handle(lib.ModPackGetDefaultRequest, this.handleModPackGetDefaultRequest.bind(this));
this.handle(lib.ModDownloadRequest, this.handleModDownloadRequest.bind(this));
this.handle(lib.SubscriptionRequest, this.handleSubscriptionRequest.bind(this));
this.connector.on("close", () => {
this._controller.subscriptions.unsubscribe(this);
});
}
async disconnect(code: number, reason: string) {
try {
await this.connector.disconnect();
} catch (err: any) {
if (!(err instanceof lib.SessionLost)) {
logger.error(`"Unexpected error preparing disconnect:\n${err.stack}`);
}
}
}
/**
* True if the link is connected, not in the dropped state and not in
* the process of disconnecting.
*/
get connected(): boolean {
return this.connector.connected;
}
async handleModPackGetRequest(request: lib.ModPackGetRequest): Promise<lib.ModPack> {
let { id } = request;
let modPack = this._controller.modPacks.get(id);
if (!modPack) {
throw new lib.RequestError(`Mod pack with ID ${id} does not exist`);
}
return modPack;
}
async handleModPackGetDefaultRequest(): Promise<lib.ModPack> {
let id = this._controller.config.get("controller.default_mod_pack_id");
if (id === null) {
throw new lib.RequestError("Default mod pack not set on controller");
}
let modPack = this._controller.modPacks.get(id);
if (!modPack) {
throw new lib.RequestError(`Default mod pack configured (${id}) does not exist`);
}
return modPack;
}
getMod(mod: { name: string, version: string, sha1?: string }) {
let filename = lib.ModInfo.filename(mod.name, mod.version);
let modInfo = this._controller.modStore.files.get(filename);
if (!modInfo) {
throw new lib.RequestError(`Mod ${filename} does not exist on controller`);
}
if (mod.sha1 && mod.sha1 !== modInfo.sha1) {
throw new lib.RequestError(`Mod ${filename} checksum does not match controller's checksum`);
}
return modInfo;
}
async handleModDownloadRequest(request: lib.ModDownloadRequest) {
let mod = this.getMod(request);
let modPath = path.join(this._controller.config.get("controller.mods_directory"), mod.filename);
let stream = await routes.createProxyStream(this._controller.app);
stream.filename = mod.filename;
stream.source = fs.createReadStream(modPath);
stream.mime = "application/zip";
stream.size = String(mod.size);
return stream.id;
}
async handleSubscriptionRequest(request: lib.SubscriptionRequest, src: lib.Address, dst: lib.Address) {
return await this._controller.subscriptions.handleRequest(this, request, src, dst);
}
}
@@ -0,0 +1,285 @@
import type {
CollectorResult, Event, InstanceStatus, Logger,
ModPack, ModInfo, PlayerEvent, PluginNodeEnvInfo,
} from "@clusterio/lib";
import type Controller from "./Controller";
import type InstanceInfo from "./InstanceInfo";
import type ControlConnection from "./ControlConnection";
import type HostConnection from "./HostConnection";
/**
* Base class for controller plugins
*
* Controller plugins are subclasses of this class which get instantiated by
* the controller on startup when the plugin is enabled in the config.
* To be discovered the class must be exported under the name `ControllerPlugin`
* in the module specified by the `controllerEntrypoint` in the plugin's
* `plugin` export.
*/
export default class BaseControllerPlugin {
/**
* Logger for this plugin
*
* Instance of winston Logger for sending log messages from this
* plugin. Supported methods and their corresponding log levels are
* `error`, `warn`, `audit`, `info` and `verbose`.
*/
logger: Logger;
constructor(
public info: PluginNodeEnvInfo,
public controller: Controller,
public metrics: any,
logger: Logger
) {
this.logger = logger.child({ plugin: this.info.name }) as unknown as Logger;
}
/**
* Called immediately after the class is instantiated
*/
async init() { }
/**
* Called when the controller saves data in memory to disk
*
* Invoked on the configured controller.autosave_interval by the
* controller and intended to be used to flush any in memory data that
* has changed to disk.
*
* This will also be called during graceful shutdown after {@link
* BaseControllerPlugin.onShutdown} have been invoked and all links have
* been disconnected.
*/
async onSaveData() { }
/**
* Called when the status of an instance changes
*
* Invoked when the controller has changed the status of an instance
* or received notice from a host that the status of an instance has
* changed. The possible statuses that can be notified about are:
* - `unassigned:`: Instance is no longer asssigned to a host.
* - `unknown`: Host assigned to instance is offline.
* - `stopped`: Instance is no longer running or was just assigned to a
* host.
* - `starting`: Instance is in the process of starting up.
* - `running`: Instance startup completed and is now running.
* - `stopping`: Instance is in the processing stopping.
* - `creating_save`: Instance is in the process of creating a save.
* - `exporting_data`: Instance is in the process of exporting item
* icons and locale data.
* - `deleted`: Instance was deleted.
*
* On controller startup all known instances gets the status `unknown` if
* they are assigned to a host, when the host then connects to the
* controller and informs of the current status of its instances this hook
* is invoked for all those instances. You can detect this situation by
* checking if prev equals `unknown`. If the host disconnects the
* instances will again get the `unknown` status.
*
* When instances are created on the controller they will notify of a status
* change with prev set to null. While the status of new instances is
* in most cases `unassigned` it's possible for the created instance to
* start with any state in some host connection corner cases.
*
* Note that it's possible for status change notification to get lost in
* the case of network outages if reconnect fails to re-establish the
* session between the controller and the host.
*
* @param instance -
* The instance that changed.
* @param prev - the previous status of the instance.
*/
async onInstanceStatusChanged(instance: InstanceInfo, prev?: InstanceStatus) { }
/**
* Called when the value of a controller config field changed.
*
* Invoked after the value of the config field given by `field` has
* changed on the controller.
*
* @param field - Name of the field that changed.
* @param curr - The current value of the field.
* @param prev - The previous value of the field.
*/
async onControllerConfigFieldChanged(field: string, curr: unknown, prev: unknown) { }
/**
* Called when the value of an instance config field changed.
*
* Invoked after the value of the config field given by `field` has
* changed on an instance.
*
* @param instance -
* The instance the config changed on.
* @param field - Name of the field that changed.
* @param curr - The current value of the field.
* @param prev - The previous value of the field.
*/
async onInstanceConfigFieldChanged(instance: InstanceInfo, field: string, curr: unknown, prev: unknown) { }
/**
* Called before collecting Prometheus metrics
*
* Invoked before the default metrics of prometheus is collected. Note
* that since this is done while replying to /metrics this should not
* take a long time, otherwise the requst will time out.
*
* Although you can return the results of collecting your prometheus
* collectors directly, all gauges in the default registry is already
* automatically collected. It's therefore recommended to only update
* the Gauges/Counters/etc in onMetric and let Clusterio deal will
* collecting the values.
*
* @returns an async iterator of prometheus metric results or undefined.
*/
async onMetrics(): Promise<void | AsyncIterable<CollectorResult>> { }
/**
* Called when the controller is shutting down
*/
async onShutdown() { }
/**
* Called when an event on a host connection happens
*
* The event param may be one of connect, drop, resume and close and has
* the following meaning:
*
* ##### connect
*
* Invoked when a new connection to the host has been established.
*
* ##### drop
*
* Invoked when a connection loss is detected between the controller and a
* host. Plugins should respond to this event by throtteling messages
* it is sending to the given host connection to an absolute minimum.
*
* Messages sent over a dropped host connection will get queued up in
* memory on the controller and sent all in one go when the connection is
* re-established again.
*
* ##### resume
*
* Invoked when a connection that had previously been dropped is
* re-established
*
* ##### close
*
* Invoked when the connection to the host has been closed.
*
* @param connection -
* The connection the event occured on.
* @param event - one of connect, drop, resume and close
*/
onHostConnectionEvent(connection: HostConnection, event: "connect" | "drop" | "resume" | "close") { }
/**
* Called when an avent on a control connection happens
*
* The event param may be one of connect, drop, resume and close and has
* the following meaning:
*
* ##### connect
*
* Invoked when a new connection to the control has been established.
*
* ##### drop
*
* Invoked when a connection loss is detected between the controller and a
* control. Plugins should respond to this event by throtteling
* messages it is sending to the given control connection to an absolute
* minimum.
*
* Messages sent over a dropped control connection will get queued up in
* memory on the controller and sent all in one go when the connection is
* re-established again.
*
* ##### resume
*
* Invoked when a connection that had previously been dropped is
* re-established.
*
* ##### close
*
* Invoked when the connection to the control has been closed.
*
* @param connection -
* The connection the event occured on.
* @param event - one of connect, drop, resume, and close.
*/
onControlConnectionEvent(connection: ControlConnection, event: "connect" | "drop" | "resume" | "close") { }
/**
* Called when a host is preparing to disconnect from the controller
*
* Invoked when a host has requested the disconnection from the controller.
* This typically happens when it is in the process of shutting down.
*
* Plugins must stop sending messages to the host in question after the
* prepare disconnect has been handled.
*
* @param connection -
* The connection to the host preparing to disconnect.
*/
async onPrepareHostDisconnect(connection: HostConnection) { }
/**
* Called when one or more mod packs are updated
*
* Invoked when one or more mod pack have been added, updated or deleted
* in the cluster.
*
* If a mod pack has been deleted its `.isDeleted` property will be
* true.
*
* @param modPacks - Mod packs that updated.
*/
async onModPacksUpdated(modPacks: ModPack[]) { }
/**
* Called when one or more mod stored on the controller are updated
*
* Invoked when one or more mods have been added, updated or deleted
* from the pool of shared mods stored on the cluster.
*
* If a mod has been deleted its `.isDeleted` property will be true.
*
* @param mods - Mods that updated.
*/
async onModsUpdated(mods: ModInfo[]) { }
/**
* Called when a player joins or leaves an instance
*
* Invoked when a player either joins or leaves an instance in the
* cluster.
*
* @param instance -
* The instance it occured on.
* @param event - Information about the event.
*/
async onPlayerEvent(instance: InstanceInfo, event: PlayerEvent) { }
/**
* Broadcast event to all connected hosts
*
* Sends the given event to all hosts connected to the controller.
* This does not include hosts that are in the process of closing the
* connection, which typically happens when they are shutting down.
*
* @param event - Event to send
*/
broadcastEventToHosts<T>(event: Event<T>) {
for (let hostConnection of this.controller.wsServer.hostConnections.values()) {
if (
!hostConnection.connector.closing
&& (!event.constructor.plugin || hostConnection.plugins.has(event.constructor.plugin))
) {
hostConnection.send(event);
}
}
}
}
@@ -0,0 +1,770 @@
import type WsServerConnector from "./WsServerConnector";
import events from "events";
import * as lib from "@clusterio/lib";
const { logFilter, logger } = lib;
import BaseConnection from "./BaseConnection";
import ControllerUser from "./ControllerUser";
import * as routes from "./routes";
import Controller from "./Controller";
const strcmp = new Intl.Collator(undefined, { numeric: true, sensitivity: "base" }).compare;
const queryLogTime = new lib.Summary(
"clusterio_controller_query_log_duration_seconds",
"Time in seconds log queries took to execute."
);
/**
* Represents the connection to a control link
*
* @extends module:controller/src/BaseConnection
* @alias module:controller/src/ControlConnection
*/
export default class ControlConnection extends BaseConnection {
private _version: string;
logTransport: lib.LinkTransport | null = null;
logSubscriptions = {
all: false,
controller: false,
hostIds: [] as number[],
instanceIds: [] as number[],
};
ws_dumper: ((...args: any[]) => void) | null = null;
declare connector: WsServerConnector;
constructor(
registerData: { version: string },
connector: WsServerConnector,
controller: Controller,
public user: ControllerUser, // The user making this connection.
public id: number
) {
super(connector, controller);
this._version = registerData.version;
this.connector.on("connect", () => {
this.connector._socket!.clusterio_ignore_dump = Boolean(this.ws_dumper);
});
this.connector.on("close", () => {
if (this.logTransport) {
logger.remove(this.logTransport);
this.logTransport = null;
}
if (this.ws_dumper) {
this._controller.debugEvents.off("message", this.ws_dumper);
}
});
for (let event of ["connect", "drop", "resume", "close"] as const) {
this.connector.on(event, () => {
for (let controllerPlugin of this._controller.plugins.values()) {
controllerPlugin.onControlConnectionEvent(this, event);
}
});
}
this.handle(lib.ControllerStopRequest, this.handleControllerStopRequest.bind(this));
this.handle(lib.ControllerRestartRequest, this.handleControllerRestartRequest.bind(this));
this.handle(lib.ControllerConfigGetRequest, this.handleControllerConfigGetRequest.bind(this));
this.handle(lib.ControllerConfigSetFieldRequest, this.handleControllerConfigSetFieldRequest.bind(this));
this.handle(lib.ControllerConfigSetPropRequest, this.handleControllerConfigSetPropRequest.bind(this));
this.handle(lib.HostRevokeTokensRequest, this.handleHostRevokeTokensRequest.bind(this));
this.handle(lib.HostListRequest, this.handleHostListRequest.bind(this));
this.handle(lib.HostGenerateTokenRequest, this.handleHostGenerateTokenRequest.bind(this));
this.handle(lib.HostConfigCreateRequest, this.handleHostConfigCreateRequest.bind(this));
this.handle(lib.InstanceDetailsGetRequest, this.handleInstanceDetailsGetRequest.bind(this));
this.handle(lib.InstanceDetailsListRequest, this.handleInstanceDetailsListRequest.bind(this));
this.handle(lib.InstanceCreateRequest, this.handleInstanceCreateRequest.bind(this));
this.handle(lib.InstanceDeleteRequest, this.handleInstanceDeleteRequest.bind(this));
this.handle(lib.InstanceConfigGetRequest, this.handleInstanceConfigGetRequest.bind(this));
this.handle(lib.InstanceConfigSetFieldRequest, this.handleInstanceConfigSetFieldRequest.bind(this));
this.handle(lib.InstanceConfigSetPropRequest, this.handleInstanceConfigSetPropRequest.bind(this));
this.handle(lib.InstanceAssignRequest, this.handleInstanceAssignRequest.bind(this));
this.handle(lib.InstanceSaveDetailsListRequest, this.handleInstanceSaveDetailsListRequest.bind(this));
this.handle(lib.InstanceRenameSaveRequest, controller.sendRequestToHostByInstanceId.bind(controller));
this.handle(lib.InstanceCopySaveRequest, controller.sendRequestToHostByInstanceId.bind(controller));
this.handle(lib.InstanceDeleteSaveRequest, controller.sendRequestToHostByInstanceId.bind(controller));
this.handle(lib.InstanceDownloadSaveRequest, this.handleInstanceDownloadSaveRequest.bind(this));
this.handle(lib.InstanceTransferSaveRequest, this.handleInstanceTransferSaveRequest.bind(this));
this.handle(lib.ModPackListRequest, this.handleModPackListRequest.bind(this));
this.handle(lib.ModPackCreateRequest, this.handleModPackCreateRequest.bind(this));
this.handle(lib.ModPackUpdateRequest, this.handleModPackUpdateRequest.bind(this));
this.handle(lib.ModPackDeleteRequest, this.handleModPackDeleteRequest.bind(this));
this.handle(lib.ModGetRequest, this.handleModGetRequest.bind(this));
this.handle(lib.ModListRequest, this.handleModListRequest.bind(this));
this.handle(lib.ModSearchRequest, this.handleModSearchRequest.bind(this));
this.handle(lib.ModDeleteRequest, this.handleModDeleteRequest.bind(this));
this.handle(lib.LogSetSubscriptionsRequest, this.handleLogSetSubscriptionsRequest.bind(this));
this.handle(lib.LogQueryRequest, this.handleLogQueryRequest.bind(this));
this.handle(lib.PermissionListRequest, this.handlePermissionListRequest.bind(this));
this.handle(lib.RoleListRequest, this.handleRoleListRequest.bind(this));
this.handle(lib.RoleCreateRequest, this.handleRoleCreateRequest.bind(this));
this.handle(lib.RoleUpdateRequest, this.handleRoleUpdateRequest.bind(this));
this.handle(lib.RoleGrantDefaultPermissionsRequest, this.handleRoleGrantDefaultPermissionsRequest.bind(this));
this.handle(lib.RoleDeleteRequest, this.handleRoleDeleteRequest.bind(this));
this.handle(lib.UserGetRequest, this.handleUserGetRequest.bind(this));
this.handle(lib.UserListRequest, this.handleUserListRequest.bind(this));
this.handle(lib.UserCreateRequest, this.handleUserCreateRequest.bind(this));
this.handle(lib.UserRevokeTokenRequest, this.handleUserRevokeTokenRequest.bind(this));
this.handle(lib.UserUpdateRolesRequest, this.handleUserUpdateRolesRequest.bind(this));
this.handle(lib.UserSetAdminRequest, this.handleUserSetAdminRequest.bind(this));
this.handle(lib.UserSetBannedRequest, this.handleUserSetBannedRequest.bind(this));
this.handle(lib.UserSetWhitelistedRequest, this.handleUserSetWhitelistedRequest.bind(this));
this.handle(lib.UserDeleteRequest, this.handleUserDeleteRequest.bind(this));
this.handle(lib.DebugDumpWsRequest, this.handleDebugDumpWsRequest.bind(this));
}
validateIngress(message: lib.MessageRequest | lib.MessageEvent) {
let origin = this.connector.dst;
if (origin.type !== message.src.type || origin.id !== message.src.id) {
throw new lib.InvalidMessage(`Received message with invalid src ${message.src} from ${origin}`);
}
}
validatePermission(
message: lib.MessageRequest | lib.MessageEvent,
entry: lib.RequestEntry | lib.EventEntry
) {
try {
this.checkPermission(message, entry);
} catch (err: any) {
this.connector.sendResponseError(new lib.ResponseError(err.message, err.code), message.src);
logger.audit(`Permission denied for ${message.name} by ${this.user.name} from ${this.connector.dst}`);
throw err;
}
}
checkPermission(
message: lib.MessageRequest | lib.MessageEvent,
entry: lib.RequestEntry | lib.EventEntry
) {
let permission;
if (message.type === "request") {
permission = (entry as lib.RequestEntry).Request.permission;
} else if (message.type === "event") {
permission = (entry as lib.EventEntry).Event.permission;
} else {
return;
}
if (permission === null) {
return;
}
if (typeof permission === "string") {
this.user.checkPermission(permission);
return;
}
if (typeof permission === "function") {
permission(this.user, message);
return;
}
throw new Error("Should be unreachable");
}
async handleControllerStopRequest() {
this._controller.stop();
}
async handleControllerRestartRequest() {
if (!this._controller.canRestart) {
throw new lib.RequestError("Cannot restart, controller does not have a process monitor to restart it.");
}
this._controller.shouldRestart = true;
this._controller.stop();
}
async handleControllerConfigGetRequest() {
return this._controller.config.toRemote("control");
}
async handleControllerConfigSetFieldRequest(request: lib.ControllerConfigSetFieldRequest) {
this._controller.config.set(request.field as keyof lib.ControllerConfigFields, request.value, "control");
}
async handleControllerConfigSetPropRequest(request: lib.ControllerConfigSetPropRequest) {
let { field, prop, value } = request;
this._controller.config.setProp(field as keyof lib.ControllerConfigFields, prop, value, "control");
}
async handleHostRevokeTokensRequest(request: lib.HostRevokeTokensRequest) {
const host = this._controller.hosts.get(request.hostId);
if (!host) {
throw new Error(`Unknown host id (${request.hostId})`);
}
host.tokenValidAfter = Math.floor(Date.now() / 1000);
const hostConnection = this._controller.wsServer.hostConnections.get(request.hostId);
if (hostConnection) {
hostConnection.connector.terminate();
}
this._controller.hostsUpdated([host]);
}
async handleHostListRequest(): Promise<lib.HostDetails[]> {
return [...this._controller.hosts.values()].map(host => host.toHostDetails());
}
async handleHostGenerateTokenRequest(message: lib.HostGenerateTokenRequest) {
let hostId = message.hostId;
if (hostId === undefined) {
hostId = Math.random() * 2**31 | 0;
}
return this._controller.generateHostToken(hostId);
}
async handleHostConfigCreateRequest(request: lib.HostConfigCreateRequest) {
const hostConfig = new lib.HostConfig("host");
hostConfig.set("host.controller_url", this._controller.getControllerUrl());
if (request.id !== undefined) {
hostConfig.set("host.id", request.id);
}
if (request.name !== undefined) {
hostConfig.set("host.name", request.name);
}
if (request.generateToken) {
this.user.checkPermission("core.host.generate_token");
hostConfig.set("host.controller_token", this._controller.generateHostToken(hostConfig.get("host.id")));
}
return hostConfig.toJSON();
}
async handleInstanceDetailsGetRequest(request: lib.InstanceDetailsGetRequest) {
return this._controller.getRequestInstance(request.instanceId).toInstanceDetails();
}
async handleInstanceDetailsListRequest() {
return [...this._controller.instances.values()].map(instance => instance.toInstanceDetails());
}
async handleInstanceCreateRequest(request: lib.InstanceCreateRequest) {
const instanceConfig = new lib.InstanceConfig("controller");
instanceConfig.update(request.config, false, "control");
await this._controller.instanceCreate(instanceConfig);
}
async handleInstanceDeleteRequest(request: lib.InstanceDeleteRequest) {
await this._controller.instanceDelete(request.instanceId);
}
async handleInstanceConfigGetRequest(request: lib.InstanceConfigGetRequest) {
let instance = this._controller.getRequestInstance(request.instanceId);
return instance.config.toRemote("control");
}
async handleInstanceConfigSetFieldRequest(request: lib.InstanceConfigSetFieldRequest) {
let instance = this._controller.getRequestInstance(request.instanceId);
if (request.field === "instance.assigned_host") {
throw new lib.RequestError("instance.assigned_host must be set through the assign-host interface");
}
if (request.field === "instance.id") {
// XXX is this worth implementing? It's race condition galore.
throw new lib.RequestError("Setting instance.id is not supported");
}
instance.config.set(request.field as keyof lib.InstanceConfigFields, request.value, "control");
await this._controller.instanceConfigUpdated(instance);
}
async handleInstanceConfigSetPropRequest(request: lib.InstanceConfigSetPropRequest) {
let instance = this._controller.getRequestInstance(request.instanceId);
let { field, prop, value } = request;
instance.config.setProp(field as keyof lib.InstanceConfigFields, prop, value, "control");
await this._controller.instanceConfigUpdated(instance);
}
async handleInstanceAssignRequest(request: lib.InstanceAssignRequest) {
await this._controller.instanceAssign(request.instanceId, request.hostId);
}
async handleInstanceSaveDetailsListRequest() {
return [...this._controller.saves.values()];
}
async handleInstanceDownloadSaveRequest(request: lib.InstanceDownloadSaveRequest) {
let { instanceId, name } = request;
let stream = await routes.createProxyStream(this._controller.app);
stream.filename = name;
let ready = new Promise((resolve, reject) => {
stream.events.on("source", resolve);
stream.events.on("timeout", () => reject(
new lib.RequestError("Timed out establishing stream from host")
));
});
ready.catch(() => {});
await this._controller.sendToHostByInstanceId(new lib.InstancePushSaveRequest(
instanceId,
stream.id,
name,
));
await ready;
return stream.id;
}
async handleInstanceTransferSaveRequest(request: lib.InstanceTransferSaveRequest) {
if (request.sourceInstanceId === request.targetInstanceId) {
throw new lib.RequestError("Source and target instance may not be the same");
}
let sourceInstance = this._controller.getRequestInstance(request.sourceInstanceId);
let targetInstance = this._controller.getRequestInstance(request.targetInstanceId);
let sourceHostId = sourceInstance.config.get("instance.assigned_host");
let targetHostId = targetInstance.config.get("instance.assigned_host");
if (sourceHostId === null) {
throw new lib.RequestError("Source instance is not assigned a host");
}
if (targetHostId === null) {
throw new lib.RequestError("Target instance is not assigned a host");
}
// Let host handle request if source and target is on the same host.
if (sourceHostId === targetHostId) {
return await this._controller.sendTo({ hostId: sourceHostId }, request);
}
// Check connectivity
let sourceHostConnection = this._controller.wsServer.hostConnections.get(sourceHostId);
if (!sourceHostConnection || sourceHostConnection.connector.closing) {
throw new lib.RequestError("Source host is not connected to the controller");
}
let targetHostConnection = this._controller.wsServer.hostConnections.get(targetHostId);
if (!targetHostConnection || targetHostConnection.connector.closing) {
throw new lib.RequestError("Target host is not connected to the controller");
}
// Create stream to proxy from target to source
let stream = await routes.createProxyStream(this._controller.app);
stream.events.on("timeout", () => {
if (stream.source) {
stream.source.destroy();
}
stream.events.emit("error", new lib.RequestError("Timed out establishing transfer stream"));
});
// Ignore errors if not listening for them to avoid crash.
stream.events.on("error", () => { /* ignore */ });
// Establish push from source host to stream, this is done first to
// ensure the file size is known prior to the target host pull.
await Promise.all([
this._controller.sendTo(
{ hostId: sourceHostId },
new lib.InstancePushSaveRequest(request.sourceInstanceId, stream.id, request.sourceName),
),
events.once(stream.events, "source"),
]);
// Establish pull from target host to stream and wait for completion.
let storedName = await this._controller.sendTo(
{ hostId: targetHostId },
new lib.InstancePullSaveRequest(request.targetInstanceId, stream.id, request.targetName),
);
// Delete source save if this is not a copy
if (!request.copy) {
await this._controller.sendTo(
{ hostId: sourceHostId },
new lib.InstanceDeleteSaveRequest(request.sourceInstanceId, request.sourceName),
);
}
return storedName;
}
async handleModPackListRequest() {
return [...this._controller.modPacks.values()];
}
async handleModPackCreateRequest(request: lib.ModPackCreateRequest) {
let modPack = request.modPack;
if (modPack.id === undefined) {
throw new lib.RequestError("Mod pack need an ID to be created");
}
if (this._controller.modPacks.has(modPack.id)) {
throw new lib.RequestError(`Mod pack with ID ${modPack.id} already exist`);
}
this._controller.modPacks.set(modPack.id, modPack);
this._controller.modPacksUpdated([modPack]);
}
async handleModPackUpdateRequest(request: lib.ModPackUpdateRequest) {
let modPack = request.modPack;
if (modPack.id === undefined || !this._controller.modPacks.has(modPack.id)) {
throw new lib.RequestError(`Mod pack with ID ${modPack.id} does not exist`);
}
this._controller.modPacks.set(modPack.id, modPack);
this._controller.modPacksUpdated([modPack]);
}
async handleModPackDeleteRequest(request: lib.ModPackDeleteRequest) {
let { id } = request;
let modPack = this._controller.modPacks.get(id);
if (!modPack) {
throw new lib.RequestError(`Mod pack with ID ${id} does not exist`);
}
modPack.isDeleted = true;
this._controller.modPacks.delete(id);
this._controller.modPacksUpdated([modPack]);
}
async handleModGetRequest(request: lib.ModGetRequest) {
return this.getMod(request);
}
async handleModListRequest() {
return [...this._controller.modStore.mods()];
}
static termsMatchesMod(terms: lib.ParsedTerm[], mod: lib.ModInfo) {
for (let term of terms) {
if (term.type === "word") {
if (!lib.wordMatches(term,
mod.name, mod.version, mod.title, mod.author, mod.contact,
mod.homepage, mod.description, mod.filename
)) {
return false;
}
} else if (term.type === "attribute") {
if (!lib.wordMatches(term.value, mod[term.name as keyof lib.ModInfo] as string)) {
return false;
}
}
}
return true;
}
async handleModSearchRequest(request: lib.ModSearchRequest) {
let query = lib.parseSearchString(request.query, {
name: "word",
// version
title: "word",
author: "word",
contact: "word",
homepage: "word",
description: "word",
// factorioVersion
// dependencies
filename: "word",
// size
sha1: "word",
});
let factorioVersion = request.factorioVersion;
type ModVersions = { name: string, versions: lib.ModInfo[] };
let results: Map<string, ModVersions> = new Map();
for (let mod of this._controller.modStore.mods()) {
if (
mod.factorioVersion !== factorioVersion
|| !ControlConnection.termsMatchesMod(query.terms, mod)
) {
continue;
}
let result = results.get(mod.name);
if (!result) {
result = {
name: mod.name,
versions: [],
};
results.set(mod.name, result);
}
result.versions.push(mod);
}
for (let result of results.values()) {
result.versions.sort((a, b) => b.integerVersion - a.integerVersion);
result.versions.map(e => e.toJSON());
}
let resultList = [...results.values()];
const sort = request.sort;
if (sort) {
const sorters = {
name: (a:ModVersions, b:ModVersions) => strcmp(a.versions[0].name, b.versions[0].name),
title: (a:ModVersions, b:ModVersions) => strcmp(a.versions[0].title, b.versions[0].title),
author: (a:ModVersions, b:ModVersions) => strcmp(a.versions[0].author, b.versions[0].author),
};
if (!Object.prototype.hasOwnProperty.call(sorters, sort)) {
throw new lib.RequestError(`Invalid value for sort: ${sort}`);
}
resultList.sort(sorters[sort as keyof typeof sorters]);
let order = request.sortOrder;
if (order === "desc") {
resultList.reverse();
}
}
const page = request.page;
const pageSize = request.pageSize || 10;
resultList = resultList.slice((page - 1) * pageSize, page * pageSize);
return {
queryIssues: query.issues,
pageCount: Math.ceil(results.size / pageSize),
resultCount: results.size,
results: resultList,
};
}
async handleModDeleteRequest(request: lib.ModDeleteRequest) {
await this._controller.modStore.deleteMod(request.name, request.version);
}
async handleLogSetSubscriptionsRequest(request: lib.LogSetSubscriptionsRequest) {
this.logSubscriptions = {
all: request.all || false,
controller: request.controller || false,
hostIds: request.hostIds || [],
instanceIds: request.instanceIds || [],
};
this.updateLogSubscriptions();
}
updateLogSubscriptions() {
let { all, controller, hostIds, instanceIds } = this.logSubscriptions;
if (all || controller || hostIds.length || instanceIds.length) {
if (!this.logTransport) {
this.logTransport = new lib.LinkTransport({ link: this });
this._controller.clusterLogger.add(this.logTransport);
}
this.logTransport.filter = logFilter(this.logSubscriptions);
} else if (this.logTransport) {
this._controller.clusterLogger.remove(this.logTransport);
this.logTransport = null;
}
}
async handleLogQueryRequest(request: lib.LogQueryRequest) {
let _request = {
limit: request.limit,
order: request.order,
maxLevel: request.maxLevel,
all: request.all,
controller: request.controller,
hostIds: request.hostIds,
instanceIds: request.instanceIds,
};
let observeDuration = queryLogTime.startTimer();
let { all, controller, hostIds, instanceIds } = request;
let log;
if (!all && controller && !hostIds.length && !instanceIds.length) {
log = await this._controller.queryControllerLog(_request);
} else {
log = await this._controller.queryClusterLog(_request);
}
observeDuration();
return { log };
}
async handlePermissionListRequest() {
return [...lib.permissions.values()];
}
async handleRoleListRequest() {
return [...this._controller.userManager.roles.values()];
}
async handleRoleCreateRequest(request: lib.RoleCreateRequest) {
let lastId = Math.max.apply(null, [...this._controller.userManager.roles.keys()]);
// Start at 5 to leave space for future default roles
let id = Math.max(5, lastId+1);
this._controller.userManager.roles.set(id, lib.Role.fromJSON({ id, ...request }));
this._controller.userManager.dirty = true;
return id;
}
async handleRoleUpdateRequest(request: lib.RoleUpdateRequest) {
let { id, name, description, permissions } = request;
let role = this._controller.userManager.roles.get(id);
if (!role) {
throw new lib.RequestError(`Role with ID ${id} does not exist`);
}
role.name = name;
role.description = description;
role.permissions = new Set(permissions);
this._controller.rolePermissionsUpdated(role);
}
async handleRoleGrantDefaultPermissionsRequest(request: lib.RoleGrantDefaultPermissionsRequest) {
let role = this._controller.userManager.roles.get(request.id);
if (!role) {
throw new lib.RequestError(`Role with ID ${request.id} does not exist`);
}
role.grantDefaultPermissions();
this._controller.rolePermissionsUpdated(role);
}
async handleRoleDeleteRequest(request: lib.RoleDeleteRequest) {
let id = request.id;
if (!this._controller.userManager.roles.delete(id)) {
throw new lib.RequestError(`Role with ID ${id} does not exist`);
}
this._controller.userManager.dirty = true;
for (let user of this._controller.userManager.users.values()) {
if (user.roleIds.delete(id)) {
this._controller.userPermissionsUpdated(user);
}
}
}
async handleUserGetRequest(request: lib.UserGetRequest): Promise<lib.User> {
let name = request.name;
let user = this._controller.userManager.users.get(name);
if (!user) {
throw new lib.RequestError(`User ${name} does not exist`);
}
return user;
}
async handleUserListRequest(): Promise<lib.User[]> {
return [...this._controller.userManager.users.values()];
}
async handleUserCreateRequest(request: lib.UserCreateRequest) {
let user = this._controller.userManager.createUser(request.name);
this._controller.usersUpdated([user]);
}
async handleUserRevokeTokenRequest(request: lib.UserRevokeTokenRequest) {
let user = this._controller.userManager.users.get(request.name);
if (!user) {
throw new lib.RequestError(`User '${request.name}' does not exist`);
}
if (user.name !== this.user.name) {
this.user.checkPermission("core.user.revoke_other_token");
}
user.invalidateToken();
for (let controlConnection of this._controller.wsServer.controlConnections.values()) {
if (controlConnection.user.name === user.name) {
controlConnection.connector.terminate();
}
}
this._controller.usersUpdated([user]);
}
async handleUserUpdateRolesRequest(request: lib.UserUpdateRolesRequest) {
let user = this._controller.userManager.users.get(request.name);
if (!user) {
throw new lib.RequestError(`User '${request.name}' does not exist`);
}
for (let roleId of request.roles) {
if (!this._controller.userManager.roles.has(roleId)) {
throw new lib.RequestError(`Role with ID ${roleId} does not exist`);
}
}
user.roleIds = new Set(request.roles);
this._controller.userPermissionsUpdated(user);
this._controller.usersUpdated([user]);
}
async handleUserSetAdminRequest(request: lib.UserSetAdminRequest) {
let { name, create, admin } = request;
let user = this._controller.userManager.users.get(name);
if (!user) {
if (create) {
this.user.checkPermission("core.user.create");
user = this._controller.userManager.createUser(name);
} else {
throw new lib.RequestError(`User '${name}' does not exist`);
}
}
user.isAdmin = admin;
this._controller.usersUpdated([user]);
this._controller.sendTo("allInstances", new lib.InstanceAdminlistUpdateEvent(name, admin));
}
async handleUserSetBannedRequest(request: lib.UserSetBannedRequest) {
let { name, create, banned, reason } = request;
let user = this._controller.userManager.users.get(name);
if (!user) {
if (create) {
this.user.checkPermission("core.user.create");
user = this._controller.userManager.createUser(name);
} else {
throw new lib.RequestError(`User '${name}' does not exist`);
}
}
user.isBanned = banned;
user.banReason = reason;
this._controller.usersUpdated([user]);
this._controller.sendTo("allInstances", new lib.InstanceBanlistUpdateEvent(name, banned, reason));
}
async handleUserSetWhitelistedRequest(request: lib.UserSetWhitelistedRequest) {
let { name, create, whitelisted } = request;
let user = this._controller.userManager.users.get(name);
if (!user) {
if (create) {
this.user.checkPermission("core.user.create");
user = this._controller.userManager.createUser(name);
} else {
throw new lib.RequestError(`User '${name}' does not exist`);
}
}
user.isWhitelisted = whitelisted;
this._controller.usersUpdated([user]);
this._controller.sendTo("allInstances", new lib.InstanceWhitelistUpdateEvent(name, whitelisted));
}
async handleUserDeleteRequest(request: lib.UserDeleteRequest) {
let name = request.name;
let user = this._controller.userManager.users.get(name);
if (!user) {
throw new lib.RequestError(`User '${name}' does not exist`);
}
user.isDeleted = true;
this._controller.userManager.users.delete(name);
this._controller.usersUpdated([user]);
if (user.isAdmin) {
this._controller.sendTo("allInstances", new lib.InstanceAdminlistUpdateEvent(name, false));
}
if (user.isWhitelisted) {
this._controller.sendTo("allInstances", new lib.InstanceWhitelistUpdateEvent(name, false));
}
if (user.isBanned) {
this._controller.sendTo("allInstances", new lib.InstanceBanlistUpdateEvent(name, false, ""));
}
}
async handleDebugDumpWsRequest(request: lib.DebugDumpWsRequest) {
this.ws_dumper = data => {
if (this.connector.connected) {
this.send(new lib.DebugWsMessageEvent(data.direction, data.content));
}
};
this.connector._socket!.clusterio_ignore_dump = true;
this._controller.debugEvents.on("message", this.ws_dumper);
}
}
module.exports = ControlConnection;
File diff suppressed because it is too large. Load diff
+132
View File
@@ -0,0 +1,132 @@
import type Controller from "./Controller";
import type BaseConnection from "./BaseConnection";
import * as lib from "@clusterio/lib";
import HostConnection from "./HostConnection";
import ControlConnection from "./ControlConnection";
const { logger } = lib;
export default class ControllerRouter {
constructor(
public controller: Controller
) {}
/**
* @param origin - Source link of the message.
* @param message - Link the message originated from.
* @param entry - message entry if message is a request or an event.
* @param hasFallback - true if fallback handling is available.
* @returns true if the message was handled
*/
forwardMessage(
origin: ControlConnection | HostConnection,
message: lib.MessageRoutable,
entry: lib.RequestEntry | lib.EventEntry | undefined,
hasFallback: boolean
): boolean {
if (!["request", "response", "responseError", "event"].includes(message.type)) {
throw new Error(`Message type ${message.type} can't be forwarded`);
}
let dst: lib.Address = message.dst;
let nextHop: ControlConnection | HostConnection | undefined;
let msg: string = "";
if (dst.type === lib.Address.broadcast) {
this.broadcastMessage(origin, message, entry as lib.EventEntry);
return true;
} else if (dst.type === lib.Address.host) {
nextHop = this.controller.wsServer.hostConnections.get(dst.id);
if (!nextHop) {
msg = `Host ${dst.id} is offline`;
}
} else if (dst.type === lib.Address.instance) {
let instance = this.controller.instances.get(dst.id);
if (!instance) {
msg = `Instance ${dst.id} does not exist`;
} else {
let assignedHost = instance.config.get("instance.assigned_host");
if (assignedHost === null) {
msg = `Instance ${dst.id} is not assigned a host`;
} else {
nextHop = this.controller.wsServer.hostConnections.get(assignedHost);
if (!nextHop) {
msg = `Assigned host for instance ${dst.id} is offline`;
}
}
}
} else if (dst.type === lib.Address.control) {
nextHop = this.controller.wsServer.controlConnections.get(dst.id);
if (!nextHop) {
msg = "Control connection does not exist";
}
} else if (dst.type === lib.Address.controller) {
msg = `Unexpected message forwarded to ${dst}`;
}
if (nextHop === origin) {
msg = `Message would return back to sender ${origin.connector.dst}.`;
nextHop = undefined;
}
if (message.type === "request") {
if (!nextHop) {
if (hasFallback) {
return false;
}
origin.connector.sendResponseError(
new lib.ResponseError(msg || "Unroutable destination"), message.src
);
return true;
}
}
if (nextHop) {
if (message.type === "request") {
nextHop.forwardRequest(message as lib.MessageRequest, origin);
} else {
nextHop.connector.forward(message);
}
} else {
this.warnUnrouted(message, msg);
}
return true;
}
broadcastMessage(
origin: lib.Link,
message: lib.MessageRoutable,
entry: lib.EventEntry,
) {
let dst = message.dst;
if (message.type !== "event") {
this.warnUnrouted(message, `Unexpected broadcast of ${message.type}`);
return;
}
const plugin = entry.Event.plugin;
if (dst.id === lib.Address.host || dst.id === lib.Address.instance) {
for (let hostConnection of this.controller.wsServer.hostConnections.values()) {
if (hostConnection !== origin && (!plugin || hostConnection.plugins.has(plugin))) {
hostConnection.connector.forward(message);
}
}
} else if (dst.id === lib.Address.control) {
for (let controlConnection of this.controller.wsServer.controlConnections.values()) {
if (controlConnection !== origin) {
controlConnection.connector.forward(message);
}
}
} else {
this.warnUnrouted(message, `Unexpected broacdast target ${dst.id}`);
}
}
warnUnrouted(
message: lib.MessageRoutable,
msg: string
) {
let dst = message.dst;
let baseMsg = `No destination for ${message.constructor.name} routed from ${message.src} to ${dst}`;
logger.warn(msg ? `${baseMsg}: ${msg}.` : `${baseMsg}.`);
}
}
+93
View File
@@ -0,0 +1,93 @@
import { Static, Type } from "@sinclair/typebox";
import { IControllerUser, PermissionError, PlayerStats, Role, User, permissions } from "@clusterio/lib";
import type UserManager from "./UserManager";
export default class ControllerUser extends User implements IControllerUser {
constructor(
public userManager: UserManager,
/** Unix time in seconds the user token must be issued after to be valid. */
public tokenValidAfter = 0,
...args: ConstructorParameters<typeof User>
) {
super(...args);
}
static jsonSchema = Type.Object({
...User.jsonSchema.properties,
token_valid_after: Type.Optional(Type.Number()),
});
static fromJSON(json: Static<typeof this.jsonSchema>, userManager: UserManager) {
const roleIds = new Set<number>(json.roles ?? []);
const instanceStats = new Map(
(json.instance_stats ?? []).map(
([id, stats]) => [id, PlayerStats.fromJSON(stats)]
)
);
const playerStats = User._calculatePlayerStats(instanceStats);
return new this(
userManager,
json.token_valid_after,
json.name,
roleIds,
new Set(json.instances),
json.is_admin,
json.is_banned,
json.is_whitelisted,
json.ban_reason,
json.updated_at_ms,
json.is_deleted,
playerStats,
instanceStats,
);
}
toJSON(controller?: boolean): Static<typeof ControllerUser.jsonSchema> {
if (!controller) {
return super.toJSON();
}
const json: Static<typeof ControllerUser.jsonSchema> = super.toJSON();
if (this.tokenValidAfter) {
json.token_valid_after = this.tokenValidAfter;
}
return json;
}
get roles(): ReadonlySet<Readonly<Role>> {
return new Set([...this.roleIds].map(
id => this.userManager.roles.get(id)
).filter(
(r): r is Role => Boolean(r)
));
}
/**
* Invalidate current tokens for the user
*
* Sets the tokenValidAfter property to the current time, which causes
* all currently issued tokens for the user to become invalid.
*/
invalidateToken() {
this.tokenValidAfter = Math.floor(Date.now() / 1000);
}
checkPermission(permission: string) {
if (!permissions.has(permission)) {
throw new Error(`permission ${permission} does not exist`);
}
for (let roleId of this.roleIds) {
let role = this.userManager.roles.get(roleId);
if (!role) {
continue;
}
if (role.permissions.has("core.admin") || role.permissions.has(permission)) {
return;
}
}
throw new PermissionError("Permission denied");
}
}
+345
View File
@@ -0,0 +1,345 @@
import type Controller from "./Controller";
import type WsServerConnector from "./WsServerConnector";
import * as lib from "@clusterio/lib";
const { logger } = lib;
import BaseConnection from "./BaseConnection";
import InstanceInfo from "./InstanceInfo";
import HostInfo from "./HostInfo";
/**
* Represents the connection to a host
*
* @extends module:controller/src/BaseConnection
* @alias module:controller/src/HostConnection
*/
export default class HostConnection extends BaseConnection {
private info: HostInfo;
plugins: Map<string, string>;
constructor(
registerData: lib.RegisterHostData,
connector: WsServerConnector,
controller: Controller,
remoteAddress: string,
) {
super(connector, controller);
this.plugins = new Map(Object.entries(registerData.plugins));
const previousHostInfo = this._controller.hosts.get(registerData.id);
this.info = new HostInfo(
registerData.id,
previousHostInfo?.name ?? "",
registerData.version,
this.plugins,
true,
remoteAddress,
previousHostInfo?.publicAddress,
previousHostInfo?.tokenValidAfter,
0,
false,
);
this._controller.hosts.set(this.id, this.info);
this._controller.hostsUpdated([this.info]);
this._checkPluginVersions();
for (let event of ["connect", "drop", "resume", "close"] as const) {
// eslint-disable-next-line no-loop-func
this.connector.on(event, () => {
for (let plugin of this._controller.plugins.values()) {
plugin.onHostConnectionEvent(this, event);
}
});
}
this.connector.on("close", () => {
const now = Date.now();
// Update status to unknown for instances on this host.
const instances: InstanceInfo[] = [];
for (let instance of this._controller.instances.values()) {
if (instance.config.get("instance.assigned_host") !== this.id) {
continue;
}
instances.push(instance);
let prev = instance.status;
instance.status = "unknown";
instance.updatedAtMs = now;
lib.invokeHook(this._controller.plugins, "onInstanceStatusChanged", instance, prev);
}
this._controller.instanceDetailsUpdated(instances);
this.info.connected = false;
this._controller.hostsUpdated([this.info]);
});
this.handle(lib.HostInfoUpdateEvent, this.handleHostInfoUpdateEvent.bind(this));
this.handle(lib.InstanceStatusChangedEvent, this.handleInstanceStatusChangedEvent.bind(this));
this.handle(lib.InstancesUpdateRequest, this.handleInstancesUpdateRequest.bind(this));
this.handle(lib.InstanceSaveDetailsUpdatesEvent, this.handleInstanceSaveDetailsUpdatesEvent.bind(this));
this.handle(lib.LogMessageEvent, this.handleLogMessageEvent.bind(this));
this.handle(lib.InstancePlayerUpdateEvent, this.handleInstancePlayerUpdateEvent.bind(this));
}
validateIngress(message: lib.MessageRoutable) {
let origin = this.connector.dst;
switch (message.src.type) {
case lib.Address.control:
case lib.Address.controller:
throw new lib.InvalidMessage(`Received message with invalid src ${message.src} from ${origin}`);
case lib.Address.host:
if (message.src.id !== origin.id) {
throw new lib.InvalidMessage(
`Received message with invalid src ${message.src} from ${origin}`
);
}
break;
case lib.Address.instance:
let instance = this._controller.instances.get(message.src.id);
if (!instance || instance.config.get("instance.assigned_host") !== this.id) {
throw new lib.InvalidMessage(
`Received message with invalid src ${message.src} from ${origin}`
);
}
break;
default:
throw new Error("Should be unreachable");
}
}
_checkPluginVersions() {
let pluginInfos = new Map(this._controller.pluginInfos.map(i => [i.name, i]));
for (let [name, version] of this.plugins) {
let info = pluginInfos.get(name);
if (!info) {
logger.warn(
`Host ${this.info.name} (${this.id}) has plugin ${name} ${version} which the ` +
"controller does not have",
);
continue;
}
if (info.version !== version) {
logger.warn(
`Host ${this.info.name} (${this.id}) has plugin ${name} ${version} which does not match ` +
`the version of this plugin on the controller (${info.version})`
);
}
}
for (let [name, info] of pluginInfos) {
if (!this.plugins.has(name)) {
logger.warn(`Host ${this.info.name} (${this.id}) is missing plugin ${name} ${info.version}`);
}
}
}
async prepareDisconnect() {
await lib.invokeHook(this._controller.plugins, "onPrepareHostDisconnect", this);
return await super.prepareDisconnect();
}
/**
* ID of the host this connection is connected to
*
* @returns {number} host ID.
*/
get id() {
return this.info.id;
}
async handleHostInfoUpdateEvent(event: lib.HostInfoUpdateEvent) {
this.info.name = event.update.name;
this.info.publicAddress = event.update.publicAddress;
this._controller.hostsUpdated([this.info]);
}
async handleInstanceStatusChangedEvent(request: lib.InstanceStatusChangedEvent) {
let instance = this._controller.instances.get(request.instanceId);
// It's possible to get updates from an instance that does not exist
// or is not assigned to the host it originated from if it was
// reassigned or deleted while the connection to the host it was
// originally on was down at the time.
if (!instance || instance.config.get("instance.assigned_host") !== this.id) {
logger.warn(`Got bogus update for instance id ${request.instanceId}`);
return;
}
// We may receive status changed where the status hasn't changed
// from our perspective if the connection was down at the time it
// changed. Hosts also send status updates on assignInstance which
// for hacky reason is also used to push config changes and
// restablish status after a connection loss.
if (
instance.status === request.status
&& instance.gamePort === request.gamePort
) {
return;
}
let prev = instance.status;
instance.status = request.status;
instance.gamePort = request.gamePort;
instance.updatedAtMs = Date.now();
logger.verbose(`Instance ${instance.config.get("instance.name")} State: ${instance.status}`);
this._controller.instanceDetailsUpdated([instance]);
await lib.invokeHook(this._controller.plugins, "onInstanceStatusChanged", instance, prev);
}
async handleInstancesUpdateRequest(request: lib.InstancesUpdateRequest) {
// Push updated instance configs
for (let instance of this._controller.instances.values()) {
if (instance.config.get("instance.assigned_host") === this.id) {
await this.send(
new lib.InstanceAssignInternalRequest(instance.id, instance.config.toRemote("host"))
);
}
}
// Assign instances the host has but controller does not
const instanceUpdates: InstanceInfo[] = [];
for (let instanceData of request.instances) {
const instanceConfig = new lib.InstanceConfig("controller");
instanceConfig.update(instanceData.config, false, "host");
let controllerInstance = this._controller.instances.get(instanceConfig.get("instance.id"));
if (controllerInstance) {
// Check if this instance is assigned somewhere else.
if (controllerInstance.config.get("instance.assigned_host") !== this.id) {
await this.send(
new lib.InstanceUnassignInternalRequest(instanceConfig.get("instance.id"))
);
continue;
}
// Already have this instance, update state instead
if (controllerInstance.status !== instanceData.status) {
let prev = controllerInstance.status;
controllerInstance.status = instanceData.status;
controllerInstance.gamePort = instanceData.gamePort;
controllerInstance.updatedAtMs = Date.now();
logger.verbose(`Instance ${instanceConfig.get("instance.name")} State: ${instanceData.status}`);
instanceUpdates.push(controllerInstance);
await lib.invokeHook(
this._controller.plugins, "onInstanceStatusChanged", controllerInstance, prev
);
}
continue;
}
instanceConfig.set("instance.assigned_host", this.id);
let newInstance = new InstanceInfo(
instanceConfig,
instanceData.status,
instanceData.gamePort,
Date.now(),
);
this._controller.instances.set(instanceConfig.get("instance.id"), newInstance);
this._controller.instancesDirty = true;
this._controller.addInstanceHooks(newInstance);
await this.send(
new lib.InstanceAssignInternalRequest(
instanceConfig.get("instance.id"), instanceConfig.toRemote("host")
)
);
instanceUpdates.push(newInstance);
await lib.invokeHook(this._controller.plugins, "onInstanceStatusChanged", newInstance);
}
if (instanceUpdates.length) {
this._controller.instanceDetailsUpdated(instanceUpdates);
}
// Push lists to make sure they are in sync.
let adminlist: Set<string> = new Set();
let banlist: Map<string, string> = new Map();
let whitelist: Set<string> = new Set();
for (let user of this._controller.userManager.users.values()) {
if (user.isAdmin) {
adminlist.add(user.name);
}
if (user.isBanned) {
banlist.set(user.name, user.banReason);
}
if (user.isWhitelisted) {
whitelist.add(user.name);
}
}
await this.send(new lib.SyncUserListsEvent(adminlist, banlist, whitelist));
}
async handleInstanceSaveDetailsUpdatesEvent(event: lib.InstanceSaveDetailsUpdatesEvent) {
const now = Date.now();
const updates: lib.SaveDetails[] = [];
for (const save of event.updates) {
const existingSave = this._controller.saves.get(save.id);
if (existingSave && save.equals(existingSave)) {
continue;
}
save.updatedAtMs = now;
this._controller.saves.set(save.id, save);
updates.push(save);
}
if (event.instanceId !== undefined) {
const updatedSaves = new Set(event.updates.map(s => s.id));
for (const [id, save] of this._controller.saves) {
if (save.instanceId === event.instanceId && !updatedSaves.has(id)) {
save.isDeleted = true;
save.updatedAtMs = now;
updates.push(save);
this._controller.saves.delete(id);
}
}
}
// Hosts eagerly send updates, which means we may get an update
// where nothing actualy changed.
if (updates.length) {
this._controller.savesUpdated(updates);
}
}
async handleLogMessageEvent(event: lib.LogMessageEvent) {
this._controller.clusterLogger.log({
...event.info,
host_id: this.id,
host_name: this.info.name,
});
}
async handleInstancePlayerUpdateEvent(event: lib.InstancePlayerUpdateEvent, src: lib.Address) {
let instanceId = src.id;
let user = this._controller.userManager.users.get(event.name);
if (!user) {
user = this._controller.userManager.createUser(event.name);
}
if (event.type === "join") {
this._controller.userManager.notifyJoin(user, instanceId);
} else if (event.type === "leave") {
this._controller.userManager.notifyLeave(user, instanceId);
}
user.instanceStats.set(instanceId, event.stats);
user.recalculatePlayerStats();
this._controller.usersUpdated([user]);
let instance = this._controller.instances.get(instanceId)!;
await lib.invokeHook(this._controller.plugins, "onPlayerEvent", instance, {
type: event.type,
name: event.name,
reason: event.reason,
stats: event.stats,
});
}
}
+88
View File
@@ -0,0 +1,88 @@
import { Type, Static } from "@sinclair/typebox";
import { HostDetails } from "@clusterio/lib";
/**
* Info about a host known to the controller.
*/
export default class HostInfo {
constructor(
/** Id of this host */
public id: number,
/** Name of this host */
public name: string,
/** Version this host last connected with */
public version: string,
/** Plugins this host last connected with */
public plugins: Map<string, string>,
/** True if this host is currently connected to controller */
public connected: boolean = false,
/** IP this host last connected from */
public remoteAddress: string = "",
/** Value of host.public_address configured for this host */
public publicAddress: string = "",
/** Unix timestamp in seconds host token must be issued after to be valid */
public tokenValidAfter: number = 0,
/** Millisecond Unix timestamp this entry was last updated at */
public updatedAtMs: number = 0,
/** True if this host has been deleted */
public isDeleted: boolean = false,
) { }
static jsonSchema = Type.Object({
"id": Type.Number(),
"name": Type.String(),
"version": Type.String(),
"plugins": Type.Record(Type.String(), Type.String()),
"connected": Type.Boolean(),
"remote_address": Type.String(),
"public_address": Type.String(),
"token_valid_after": Type.Number(),
"updated_at_ms": Type.Number(),
"is_deleted": Type.Boolean(),
});
static fromJSON(json: Static<typeof this.jsonSchema>) {
return new this(
json.id,
json.name,
json.version,
new Map(Object.entries(json.plugins)),
json.connected,
json.remote_address,
json.public_address,
json.token_valid_after,
json.updated_at_ms,
json.is_deleted,
);
}
toJSON(): Static<typeof HostInfo.jsonSchema> {
return {
id: this.id,
name: this.name,
version: this.version,
plugins: Object.fromEntries(this.plugins),
connected: this.connected,
remote_address: this.remoteAddress,
public_address: this.publicAddress,
token_valid_after: this.tokenValidAfter,
updated_at_ms: this.updatedAtMs,
is_deleted: this.isDeleted,
};
}
toHostDetails() {
return new HostDetails(
this.version,
this.name,
this.id,
this.connected,
this.remoteAddress,
this.publicAddress,
this.tokenValidAfter,
this.updatedAtMs,
this.isDeleted,
);
}
}
@@ -1,7 +1,7 @@
"use strict";
const http = require("http");
const https = require("https");
const util = require("util");
import type { Socket } from "net";
import http from "http";
import https from "https";
/**
* Gracefully close a Node.js HTTP(S) server
@@ -18,36 +18,40 @@ const util = require("util");
* // in the shutdown logic
* await serverCloser.close();
*
* @alias module:master/src/HttpCloser
* @alias module:controller/src/HttpCloser
*/
class HttpCloser {
export default class HttpCloser {
private _server: http.Server | https.Server;
private _sockets: Set<Socket>;
private _responses: Set<http.ServerResponse>;
/**
* Server to attach this closer to.
* @param {https.Server | http.Server} server - HTTP(S) to attach to
* Create HttpCloser attached to a server.
* @param server - Node.js HTTP(S) server to attach to.
*/
constructor(server) {
constructor(server: http.Server | https.Server) {
this._server = server;
this._sockets = new Set();
this._responses = new Set();
if (this._server instanceof http.Server) {
this._server.on("connection", this._addSocket.bind(this));
} else if (this._server instanceof https.Server) {
if (this._server instanceof https.Server) {
this._server.on("secureConnection", this._addSocket.bind(this));
} else if (this._server instanceof http.Server) {
this._server.on("connection", this._addSocket.bind(this));
} else {
throw Error("server must be an instance of either http.Server or https.Server.");
}
this._server.on("request", this._addRequest.bind(this));
}
_addSocket(socket) {
_addSocket(socket:Socket) {
this._sockets.add(socket);
socket.once("close", () => {
this._sockets.delete(socket);
});
}
_addRequest(request, response) {
_addRequest(request: http.IncomingMessage, response: http.ServerResponse) {
this._responses.add(response);
response.once("close", () => this._responses.delete(response));
response.once("finish", () => this._responses.delete(response));
@@ -63,16 +67,16 @@ class HttpCloser {
* If there's requests in progress that still haven't been completed
* after the timeout value passed they will be aborted.
*
* @param {number} timeout -
* @param timeoutMs -
* Time in ms to wait for connections to close.
*/
async close(timeout = 5000) {
async close(timeoutMs: number = 5000): Promise<void> {
let activeSockets = new Set();
for (let response of this._responses) {
if (!response.headersSent) {
response.setHeader("Connection", "close");
} else {
let socket = response.socket;
let socket = response.socket!;
response.once("finish", () => {
socket.end();
});
@@ -93,7 +97,7 @@ class HttpCloser {
// Node.js, see nodejs/node#27428.
socket.destroy();
}
}, timeout);
}, timeoutMs);
this._server.close(err => {
clearTimeout(timer);
if (err) {
@@ -105,5 +109,3 @@ class HttpCloser {
});
}
};
module.exports = HttpCloser;
+55
View File
@@ -0,0 +1,55 @@
import * as lib from "@clusterio/lib";
import { Static, Type } from "@sinclair/typebox";
/**
* Runtime status of an instance on the controller
* @alias module:controller/src/InstanceInfo
*/
export default class InstanceInfo {
constructor(
public config: lib.InstanceConfig,
public status: lib.InstanceStatus,
public gamePort?: number,
public updatedAtMs = 0,
) {
this.config = config;
this.status = status;
}
static jsonSchema = Type.Object({
"config": lib.InstanceConfig.jsonSchema,
"status": lib.InstanceStatus,
"gamePort": Type.Optional(Type.Number()),
"updatedAtMs": Type.Optional(Type.Number()),
});
static fromJSON(json: Static<typeof this.jsonSchema>, location: lib.ConfigLocation) {
return new this(
lib.InstanceConfig.fromJSON(json.config, location),
json.status,
json.gamePort,
json.updatedAtMs,
);
}
toInstanceDetails() {
return new lib.InstanceDetails(
this.config.get("instance.name"),
this.id,
this.config.get("instance.assigned_host") ?? undefined,
this.gamePort,
this.status,
this.updatedAtMs,
);
}
get isDeleted() {
return this.status === "deleted";
}
/** Shorthand for `instance.config.get("instance.id")` */
get id():number {
return this.config.get("instance.id");
}
}
+125
View File
@@ -0,0 +1,125 @@
import fs from "fs-extra";
import jwt from "jsonwebtoken";
import * as lib from "@clusterio/lib";
import ControllerUser from "./ControllerUser";
/**
* Manages users and roles
* @alias module:controller/src/UserManager
*/
export default class UserManager {
roles: Map<number, lib.Role> = new Map();
users: Map<string, ControllerUser> = new Map();
dirty = false;
/**
* Set of users currently online in the cluster.
*/
onlineUsers = new Set<ControllerUser>();
constructor(
private _config: lib.ControllerConfig
) {
}
async load(filePath: string): Promise<void> {
try {
let content = JSON.parse(await fs.readFile(filePath, { encoding: "utf8" }));
for (let serializedRole of content.roles) {
let role = lib.Role.fromJSON(serializedRole);
this.roles.set(role.id, role);
}
for (let serializedUser of content.users) {
let user = ControllerUser.fromJSON(serializedUser, this);
this.users.set(user.name, user);
}
} catch (err: any) {
if (err.code !== "ENOENT") {
throw err;
}
// Create default roles if loading failed
lib.ensureDefaultAdminRole(this.roles);
lib.ensureDefaultPlayerRole(this.roles);
}
}
async save(filePath:string): Promise<void> {
let serializedRoles = [];
for (let role of this.roles.values()) {
serializedRoles.push(role.toJSON());
}
let serializedUsers = [];
for (let user of this.users.values()) {
serializedUsers.push(user.toJSON(true));
}
let serialized = {
users: serializedUsers,
roles: serializedRoles,
};
this.dirty = false;
await lib.safeOutputFile(filePath, JSON.stringify(serialized, null, "\t"));
}
/**
* Creates a new user and add it to the user database.
* @param name - Name of the user to create.
* @returns The created user.
*/
createUser(name:string): ControllerUser {
if (this.users.has(name)) {
throw new Error(`User '${name}' already exists`);
}
let roles = new Set<number>();
let defaultRoleId = this._config.get("controller.default_role_id");
if (defaultRoleId !== null && this.roles.has(defaultRoleId)) {
roles.add(defaultRoleId);
}
let user = new ControllerUser(this, 0, name, roles);
this.users.set(name, user);
this.dirty = true;
return user;
}
/**
* Sign access token for the given user name
*
* @param name - user name to sign token for
* @returns JWT access token for the user.
*/
signUserToken(name: string): string {
return jwt.sign(
{ aud: "user", user: name },
Buffer.from(this._config.get("controller.auth_secret"), "base64")
);
}
notifyJoin(user: ControllerUser, instance_id: number) {
user.instances.add(instance_id);
this.onlineUsers.add(user);
}
notifyLeave(user: ControllerUser, instance_id: number) {
user.instances.delete(instance_id);
if (!user.instances.size) {
this.onlineUsers.delete(user);
}
}
clearStatsOfInstance(instanceId: number) {
for (const user of this.users.values()) {
this.notifyLeave(user, instanceId);
user.instanceStats.delete(instanceId);
user.recalculatePlayerStats();
}
this.dirty = true;
}
}
+382
View File
@@ -0,0 +1,382 @@
import type { IncomingMessage } from "http";
import type { Duplex } from "stream";
import type Controller from "./Controller";
import jwt from "jsonwebtoken";
import WebSocket, { WebSocketServer } from "ws";
import * as lib from "@clusterio/lib";
const { logger } = lib;
import ControlConnection from "./ControlConnection";
import HostConnection from "./HostConnection";
import WsServerConnector from "./WsServerConnector";
import { version as packageVersion } from "../package.json";
const wsMessageCounter = new lib.Counter(
"clusterio_controller_websocket_message_total",
"How many messages have been received over WebSocket on the controller",
{ labels: ["direction"] }
);
const wsConnectionsCounter = new lib.Counter(
"clusterio_controller_websocket_connections_total",
"How many WebSocket connections have been initiated on the controller"
);
const wsRejectedConnectionsCounter = new lib.Counter(
"clusterio_controller_websocket_rejected_connections_total",
"How many WebSocket connections have been rejected during the handshake on the controller"
);
/**
* WebSocket server
* @alias module:controller/src/WsServer
*/
export default class WsServer {
controller: Controller;
stopAcceptingNewSessions: boolean = false;
controlConnections: Map<number, ControlConnection> = new Map();
hostConnections: Map<number, HostConnection> = new Map();
activeConnectors: Map<number, WsServerConnector> = new Map();
pendingSockets: Set<WebSocket> = new Set();
// Unique string for the session token audience
sessionAud: string = `session-${Date.now()}`;
nextSessionId: number = 1;
nextControlId: number = 1;
wss: WebSocket.Server;
constructor(controller: Controller) {
this.controller = controller;
this.wss = new WebSocket.Server({
noServer: true,
path: "/api/socket",
});
this.wss.on("connection",
(socket: lib.WebSocketClusterio, req: IncomingMessage) => this.handleConnection(socket, req),
);
}
remoteAddr(req: IncomingMessage) {
const remote = req.socket.remoteAddress ?? "";
if (
remote && req.headers["x-forwarded-for"]
&& (
(req.socket.remoteFamily === "IPv4" && this.controller.trustedProxies.check(remote, "ipv4"))
|| (req.socket.remoteFamily === "IPv6" && this.controller.trustedProxies.check(remote, "ipv6"))
)
) {
return (req.headers["x-forwarded-for"] as string).split(",").slice(-1)[0].trim();
}
return remote;
}
async stop(): Promise<void> {
this.stopAcceptingNewSessions = true;
let disconnectTasks = [];
for (let controlConnection of this.controlConnections.values()) {
disconnectTasks.push(controlConnection.disconnect(1001, "Server Quit"));
}
for (let hostConnection of this.hostConnections.values()) {
disconnectTasks.push(hostConnection.disconnect(1001, "Server Quit"));
}
logger.info(`WsServer | Waiting for ${disconnectTasks.length} connectors to close`);
for (let task of disconnectTasks) {
try {
await task;
} catch (err: any) {
if (!(err instanceof lib.SessionLost)) {
logger.warn(`Unexpected error disconnecting connector:\n${err.stack}`);
}
}
}
for (let socket of this.pendingSockets) {
socket.close(1001, "Server Quit");
}
}
handleUpgrade(
req: IncomingMessage,
socket: Duplex,
head: Buffer
) {
// For reasons that defy common sense, the connection event has
// to be emitted explictly when using noServer.
this.wss.handleUpgrade(req, socket, head, (ws) => {
this.wss.emit("connection", ws, req);
});
}
handleConnection(socket: lib.WebSocketClusterio, req: IncomingMessage) {
logger.verbose(`WsServer | new connection from ${this.remoteAddr(req)}`);
// Track statistics
wsConnectionsCounter.inc();
socket.on("message", (message: WebSocket.RawData) => {
wsMessageCounter.labels("in").inc();
if (!socket.clusterio_ignore_dump) {
this.controller.debugEvents.emit("message", { direction: "in", content: message.toString("utf8") });
}
});
let originalSend = socket.send;
socket.send = (...args: [string, any]) => {
wsMessageCounter.labels("out").inc();
if (typeof args[0] === "string" && !socket.clusterio_ignore_dump) {
this.controller.debugEvents.emit("message", { direction: "out", content: args[0] });
}
return originalSend.call(socket, ...args);
};
// Start connection handshake.
let loadedPlugins: {[key:string]: string} = {};
for (let [name, plugin] of this.controller.plugins) {
loadedPlugins[name] = plugin.info.version;
}
socket.send(JSON.stringify(new lib.MessageHello(new lib.HelloData(packageVersion, loadedPlugins))));
this.attachHandler(socket, req);
}
attachHandler(
socket: WebSocket,
req: IncomingMessage
) {
this.pendingSockets.add(socket);
let timeoutId = setTimeout(() => {
logger.verbose(`WsServer | closing ${this.remoteAddr(req)} after timing out on handshake`);
wsRejectedConnectionsCounter.inc();
socket.terminate();
this.pendingSockets.delete(socket);
}, 30*1000);
socket.once("message", (message: string) => {
clearTimeout(timeoutId);
this.pendingSockets.delete(socket);
this.handleHandshake(
message, socket, req
).catch(err => {
logger.error(`
+------------------------------------------------------------+
| Unexpected error occured in WebSocket handshake, please |
| report it to https://github.com/clusterio/clusterio/issues |
+------------------------------------------------------------+
${err.stack}`
);
wsRejectedConnectionsCounter.inc();
socket.close(1011, "Unexpected error");
});
});
}
async handleHandshake(rawMessage: string, socket: WebSocket, req: IncomingMessage) {
let message;
try {
const json = JSON.parse(rawMessage);
lib.Message.validate(json);
message = lib.Message.fromJSON(json);
} catch (err: any) {
logger.verbose(`WsServer | closing ${this.remoteAddr(req)} after receiving invalid message`);
wsRejectedConnectionsCounter.inc();
socket.close(1002, `Invalid message: ${err.message}`);
return;
}
if (message.type === "resume") {
this.resumeSession((message as lib.MessageResume).data, socket, req);
return;
}
if (message.type !== "registerHost" && message.type !== "registerControl") {
logger.verbose(`WsServer | closing ${this.remoteAddr(req)} after receiving invalid handshake`);
wsRejectedConnectionsCounter.inc();
socket.close(1002, "Bad handshake");
return;
}
if (this.stopAcceptingNewSessions) {
logger.verbose(`WsServer | closing ${this.remoteAddr(req)}, server is shutting down`);
wsRejectedConnectionsCounter.inc();
socket.close(1001, "Shutting down");
return;
}
if (message.type === "registerHost") {
await this.registerHost((message as lib.MessageRegisterHost).data, socket, req);
return;
}
if (message.type === "registerControl") {
await this.registerControl((message as lib.MessageRegisterControl).data, socket, req);
return;
}
throw new Error("This should be unreachable");
}
createSession(dst: lib.Address): [ WsServerConnector, string ] {
let sessionId = this.nextSessionId;
this.nextSessionId += 1;
let sessionToken = jwt.sign(
{ aud: this.sessionAud, sid: sessionId },
Buffer.from(this.controller.config.get("controller.auth_secret"), "base64"),
);
let sessionTimeout = this.controller.config.get("controller.session_timeout");
let heartbeatInterval = this.controller.config.get("controller.heartbeat_interval");
let connector = new WsServerConnector(dst, sessionId, sessionTimeout, heartbeatInterval);
this.activeConnectors.set(sessionId, connector);
connector.on("close", () => {
this.activeConnectors.delete(sessionId);
});
return [connector, sessionToken];
}
async registerHost(data: lib.RegisterHostData, socket: WebSocket, req: IncomingMessage) {
try {
let tokenPayload = jwt.verify(
data.token,
Buffer.from(this.controller.config.get("controller.auth_secret"), "base64"),
// migrate: allow pre-rename tokens issued to hosts before alpha-14
{ audience: ["host", "slave"] }
);
if (typeof tokenPayload === "string") {
throw new Error("unexpected JsonWebToken type");
}
// migrate: allow pre-rename tokens issued to hosts before alpha-14
if ((tokenPayload.host !== undefined ? tokenPayload.host : tokenPayload.slave) !== data.id) {
throw new Error("missmatched host id");
}
let host = this.controller.hosts.get(data.id);
if (!tokenPayload.iat || tokenPayload.iat < (host?.tokenValidAfter ?? 0)) {
throw new Error("invalid token");
}
} catch (err: any) {
logger.verbose(`WsServer | authentication failed for ${this.remoteAddr(req)}`);
wsRejectedConnectionsCounter.inc();
socket.close(4003, `Authentication failed: ${err.message}`);
return;
}
let [connector, sessionToken] = this.createSession(new lib.Address(lib.Address.host, data.id));
let connection = this.hostConnections.get(data.id);
if (connection) {
logger.verbose(`WsServer | disconnecting existing connection for host ${data.id}`);
await connection.disconnect(1008, "Registered from another connection");
}
logger.info(
`WsServer | registered host ${data.id} using version ${data.version}`
);
if (data.version !== packageVersion) {
logger.warn(
`Host ${data.id} connected using version ${data.version} which does not match the ` +
`version of the controller is currently running (${packageVersion}). It may not work as expected.`
);
}
// Handle close before HostConnection does. This avoids a host
// connection event happening between the connection breaking and
// it being removed from the host connections list.
connector.on("close", () => {
if (this.hostConnections.get(data.id) === connection) {
this.hostConnections.delete(data.id);
} else {
logger.warn("Unlisted HostConnection closed");
}
});
connection = new HostConnection(data, connector, this.controller, this.remoteAddr(req));
this.hostConnections.set(data.id, connection);
let src = new lib.Address(lib.Address.host, data.id);
connector.ready(socket, src, sessionToken, undefined);
}
async registerControl(data: lib.RegisterControlData, socket: WebSocket, req: IncomingMessage) {
let user;
try {
let tokenPayload = jwt.verify(
data.token,
Buffer.from(this.controller.config.get("controller.auth_secret"), "base64"),
{ audience: "user" }
);
if (typeof tokenPayload === "string") {
throw new Error("unexpected JsonWebToken type");
}
user = this.controller.userManager.users.get(tokenPayload.user);
if (!user) {
throw new Error("invalid user");
}
if ((tokenPayload.iat || 0) < user.tokenValidAfter) {
throw new Error("invalid token");
}
user.checkPermission("core.control.connect");
} catch (err: any) {
logger.verbose(`WsServer | authentication failed for ${this.remoteAddr(req)}`);
wsRejectedConnectionsCounter.inc();
socket.close(4003, `Authentication failed: ${err.message}`);
return;
}
let id = this.nextControlId;
this.nextControlId += 1;
let [connector, sessionToken] = this.createSession(new lib.Address(lib.Address.control, id));
logger.verbose(`WsServer | registered control ${id} from ${this.remoteAddr(req)}`);
connector.on("close", () => {
this.controlConnections.delete(id);
});
let connection = new ControlConnection(data, connector, this.controller, user, id);
this.controlConnections.set(id, connection);
let account = new lib.AccountDetails(
user.name,
[...user.roles].map(r => ({
name: r.name,
id: r.id,
permissions: [...r.permissions],
}))
);
let src = new lib.Address(lib.Address.control, id);
connector.ready(socket, src, sessionToken, account);
}
resumeSession(data: lib.ResumeData, socket: WebSocket, req: IncomingMessage) {
let connector;
try {
let payload = jwt.verify(
data.sessionToken,
Buffer.from(this.controller.config.get("controller.auth_secret"), "base64"),
{ audience: this.sessionAud }
);
if (typeof payload === "string") {
throw new Error("unexpected JsonWebToken type");
}
connector = this.activeConnectors.get(payload.sid);
if (!connector) {
throw new Error();
}
} catch (err) {
socket.send(JSON.stringify(new lib.MessageInvalidate()));
this.attachHandler(socket, req);
return;
}
connector.continue(socket, data.lastSeq as number);
}
}
@@ -1,24 +1,27 @@
"use strict";
const events = require("events");
import type WebSocket from "ws";
const libLink = require("@clusterio/lib/link");
const { logger } = require("@clusterio/lib/logging");
import events from "events";
import * as lib from "@clusterio/lib";
const { logger } = lib;
/**
* Connector for master server connections
* Connector for controller connections
*
* @extends module:lib/link.WebSocketBaseConnector
* @alias module:master/src/WsServerConnector
* @extends module:lib.WebSocketBaseConnector
* @alias module:controller/src/WsServerConnector
*/
class WsServerConnector extends libLink.WebSocketBaseConnector {
constructor(sessionId, sessionTimeout, heartbeatInterval) {
super(sessionTimeout);
export default class WsServerConnector extends lib.WebSocketBaseConnector {
_timeoutId: ReturnType<typeof setTimeout> | null = null;
this._socket = null;
this._sessionId = sessionId;
this._heartbeatInterval = heartbeatInterval;
this._timeoutId = null;
constructor(
dst: lib.Address,
private _sessionId: number,
private _sessionTimeout: number,
public _heartbeatInterval: number,
) {
super(new lib.Address(lib.Address.controller, 0), dst);
// The following states are used in the server connector
// closed: Connection is closed
@@ -27,8 +30,10 @@ class WsServerConnector extends libLink.WebSocketBaseConnector {
}
_reset() {
clearTimeout(this._timeoutId);
this._timeoutId = null;
if (this._timeoutId) {
clearTimeout(this._timeoutId);
this._timeoutId = null;
}
super._reset();
}
@@ -37,28 +42,31 @@ class WsServerConnector extends libLink.WebSocketBaseConnector {
*
* Sends the ready message over the socket to initiate the session.
*
* @param {Object} socket - WebSocket connection to client.
* @param {string} sessionToken -
* the session token to send to the client.
* @param {Object=} additionalData -
* extra properties to send along the ready message.
* @param socket - WebSocket connection to client.
* @param src - Source address for this link.
* @param sessionToken - the session token to send to the client.
* @param account - account data to provide to control connection
*/
ready(socket, sessionToken, additionalData) {
ready(
socket: WebSocket.WebSocket,
src: lib.Address,
sessionToken: string,
account: lib.AccountDetails | undefined
) {
this._socket = socket;
this._socket.send(JSON.stringify({
seq: null,
type: "ready",
data: {
session_token: sessionToken,
session_timeout: this._sessionTimeout,
heartbeat_interval: this._heartbeatInterval,
...additionalData,
},
}));
this._sendInternal(new lib.MessageReady(
new lib.ReadyData(
src,
sessionToken,
this._sessionTimeout,
this._heartbeatInterval,
account,
),
));
this._state = "connected";
this._attachSocketHandlers();
this.emit("connect");
this.emit("connect", { src: this.src });
}
/**
@@ -67,14 +75,17 @@ class WsServerConnector extends libLink.WebSocketBaseConnector {
* Terminates the current socket and contiunes the session over the
* socket given from the message sequence given.
*
* @param {module:net.Socket} socket - New socket to continue on.
* @param {number} lastSeq - The last message the client received.
* @param socket - New socket to continue on.
* @param lastSeq - The last message the client received.
*/
continue(socket, lastSeq) {
continue(
socket: WebSocket.WebSocket,
lastSeq: number
) {
// It's possible the previous connection hasn't closed yet due to a
// stale connection. Terminate it if so.
if (this._state === "connected") {
if (this._state === "connected" && this._socket !== null) {
this._socket.terminate();
this._socket.once("close", () => this.continue(socket, lastSeq));
return;
@@ -87,15 +98,9 @@ class WsServerConnector extends libLink.WebSocketBaseConnector {
this._timeoutId = null;
}
this._socket.send(JSON.stringify({
seq: null,
type: "continue",
data: {
last_seq: this._lastReceivedSeq,
session_timeout: this._sessionTimeout,
heartbeat_interval: this._heartbeatInterval,
},
}));
this._sendInternal(new lib.MessageContinue(
new lib.ContinueData(this._sessionTimeout, this._heartbeatInterval, this._lastReceivedSeq)
));
this._state = "connected";
this._attachSocketHandlers();
@@ -113,11 +118,16 @@ class WsServerConnector extends libLink.WebSocketBaseConnector {
}
_attachSocketHandlers() {
if (this._socket === null) {
throw new Error("this._socket is null");
}
this.startHeartbeat();
this._socket.on("close", (code, reason) => {
this._socket.on("close", (code: number, reason: string) => {
logger.verbose(`Connector | Close (code: ${code}, reason: ${reason})`);
this.stopHeartbeat();
this._socket = null;
if (this._closing) {
this._reset();
@@ -128,10 +138,9 @@ class WsServerConnector extends libLink.WebSocketBaseConnector {
this.emit("drop");
this._timeoutId = setTimeout(() => { this._timedOut(); }, this._sessionTimeout * 1000);
}
});
this._socket.on("error", err => {
this._socket.on("error", (err: any) => {
// It's assumed that close is always called by ws
logger.verbose("Connector | Error:", err);
});
@@ -139,27 +148,21 @@ class WsServerConnector extends libLink.WebSocketBaseConnector {
this._socket.on("open", () => {
logger.verbose("Connector | Open");
});
this._socket.on("ping", data => {
this._socket.on("ping", (data: string) => {
logger.verbose(`Connector | Ping (data: ${data}`);
});
this._socket.on("pong", data => {
this._socket.on("pong", (data: string) => {
logger.verbose(`Connector | Pong (data: ${data}`);
});
// Handle messages
this._socket.on("message", data => {
let message = JSON.parse(data);
if (["connected", "closing"].includes(this._state)) {
if (message.seq !== null) {
this._lastReceivedSeq = message.seq;
}
if (message.type === "heartbeat") {
this._processHeartbeat(message);
} else {
this.emit("message", message);
}
this._socket.on("message", (data: string) => {
let message = this._parseMessage(data);
if (!message) {
return;
}
if (this._state === "connected") {
this._processMessage(message);
} else {
throw new Error(`Received message in unexpected state ${this._state}`);
@@ -172,10 +175,13 @@ class WsServerConnector extends libLink.WebSocketBaseConnector {
*
* Sends a close frame and disconnects the connector.
*
* @param {number} code - WebSocket close code.
* @param {string} reason - WebSocket close reason.
* @param code - WebSocket close code.
* @param reason - WebSocket close reason.
*/
async close(code, reason) {
async close(
code: number,
reason: string
): Promise<void> {
if (this._state === "closed") {
return;
}
@@ -187,10 +193,26 @@ class WsServerConnector extends libLink.WebSocketBaseConnector {
}
this._closing = true;
this._socket.close(code, reason);
if (this._socket !== null) {
this._socket.close(code, reason);
}
await events.once(this, "close");
}
/**
* Forcefully close the connection immediately
*
* This should only be used if there's a security implication from
* letting the connection carry on with a normal close proceedure.
*/
terminate() {
this.setClosing();
if (this._socket) {
this._socket.terminate();
}
}
/**
* Notify the connection is expected to close
*
@@ -209,5 +231,3 @@ class WsServerConnector extends libLink.WebSocketBaseConnector {
this._closing = true;
}
}
module.exports = WsServerConnector;
+13
View File
@@ -0,0 +1,13 @@
import * as lib from "@clusterio/lib";
import util from "util";
export const endpointHitCounter = new lib.Counter(
"clusterio_controller_http_endpoint_hits_total",
"How many requests a particular HTTP endpoint has gotten",
{ labels: ["route"], register: false }
);
endpointHitCounter.labels = util.deprecate(
endpointHitCounter.labels,
"incrementing endpointHitCounter is no longer needed and has no effect"
);
+693
View File
@@ -0,0 +1,693 @@
import type { Application, Request, Response } from "express";
import type Controller from "./Controller";
import busboy from "busboy";
import crypto from "crypto";
import events from "events";
import fs from "fs-extra";
import JSZip from "jszip";
import jwt from "jsonwebtoken";
import path from "path";
import nodeStream from "stream";
import util from "util";
import * as lib from "@clusterio/lib";
const { logger } = lib;
const finished = util.promisify(nodeStream.finished);
declare global {
// eslint-disable-next-line @typescript-eslint/no-namespace
namespace Express {
export interface Locals {
controller: Controller,
mainBundle: string,
devPlugins: Map<string, number>,
streams: Map<string, ProxyStream>
}
}
}
// Merges samples from sourceResult to destinationResult
function mergeSamples(destinationResult: lib.CollectorResultSerialized, sourceResult: lib.CollectorResultSerialized) {
let receivedSamples = new Map(sourceResult.samples);
for (let [suffix, suffixSamples] of destinationResult.samples) {
if (receivedSamples.has(suffix)) {
suffixSamples.push(...receivedSamples.get(suffix) as any[]);
receivedSamples.delete(suffix);
}
}
for (let entry of receivedSamples) {
destinationResult.samples.push(entry);
}
}
// Prometheus polling endpoint
async function getMetrics(req: Request, res: Response, next: any) {
const controller: Controller = req.app.locals.controller;
let results: lib.CollectorResult[] = [];
let pluginResults = await lib.invokeHook(controller.plugins, "onMetrics");
for (let metricIterator of pluginResults) {
for await (let metric of metricIterator) {
results.push(metric);
}
}
let requests: Promise<InstanceType<typeof lib.HostMetricsRequest["Response"]> | null>[] = [];
let timeoutMs = controller.config.get("controller.metrics_timeout") * 1000;
for (let [hostId, hostConnection] of controller.wsServer.hostConnections) {
if (!hostConnection.connected) {
continue;
}
requests.push(lib.timeout(
hostConnection.send(new lib.HostMetricsRequest()).catch((err: any) => {
if (!(err instanceof lib.SessionLost)) {
logger.error(`Unexpected error gathering metrics from host:\n${err.stack}`);
}
return null;
}),
timeoutMs, null
));
}
for await (let result of lib.defaultRegistry.collect()) {
results.push(result);
}
let resultMap: Map<string, lib.CollectorResultSerialized> = new Map();
for (let response of await Promise.all(requests)) {
if (!response) {
// TODO: Log timeout occured?
continue;
}
for (let result of response.results) {
if (!resultMap.has(result.metric.name)) {
resultMap.set(result.metric.name, result);
} else {
// Merge metrics received by multiple hosts
mergeSamples(resultMap.get(result.metric.name)!, result);
}
}
}
for (let result of resultMap.values()) {
results.push(lib.deserializeResult(result));
}
let text = await lib.exposition(results);
res.set("Content-Type", lib.expositionContentType);
res.send(text);
}
function getPlugins(req: Request, res: Response) {
let plugins: lib.PluginWebApi[] = [];
for (let pluginInfo of req.app.locals.controller.pluginInfos) {
let name = pluginInfo.name;
let loaded = req.app.locals.controller.plugins.has(name);
let enabled = loaded && req.app.locals.controller.config.get(
`${pluginInfo.name}.load_plugin` as keyof lib.ControllerConfigFields,
) as unknown as boolean;
// Note: Cast through unknown is needed because load_plugin is
// defined at runtime and no other fields in the controller config
// currently have the boolean type.
let web: { main:any, error:string|undefined } = { main: undefined, error: undefined};
let devPlugins = req.app.locals.devPlugins;
if (devPlugins && devPlugins.has(name)) {
let stats = res.locals.webpack.devMiddleware.stats.stats[devPlugins.get(name)!];
web.main = stats.toJson().assetsByChunkName[name];
} else if (pluginInfo.manifest) {
web.main = pluginInfo.manifest[`${pluginInfo.name}.js`];
if (!web.main) {
web.error = `Missing ${pluginInfo.name}.js entry in manifest.json`;
}
} else {
web.error = "Missing dist/web/manifest.json";
}
if (web.main === "remoteEntry.js") {
web.error = "Incompatible old remoteEntry.js entrypoint.";
}
plugins.push({ name, version: pluginInfo.version, enabled, loaded, web, npmPackage: pluginInfo.npmPackage });
}
res.send(plugins);
}
function validateHostToken(req: Request, res: Response, next: any) {
let token = req.header("x-access-token");
if (!token) {
res.sendStatus(401);
return;
}
try {
const tokenPayload = jwt.verify(
token,
Buffer.from(req.app.locals.controller.config.get("controller.auth_secret"), "base64"),
{ audience: "host" }
);
if (typeof tokenPayload === "string") {
throw new Error("unexpected JsonWebToken type");
}
let host = req.app.locals.controller.hosts.get(tokenPayload.host);
if (!host) {
throw new Error("invalid host");
}
if (!tokenPayload.iat || tokenPayload.iat < (host.tokenValidAfter ?? 0)) {
throw new Error("invalid token");
}
} catch (err) {
res.sendStatus(401);
return;
}
next();
}
function validateUserToken(req: Request, res: Response, next: any) {
let token = req.header("x-access-token");
if (!token) {
res.sendStatus(401);
return;
}
try {
let tokenPayload = jwt.verify(
token,
Buffer.from(req.app.locals.controller.config.get("controller.auth_secret"), "base64"),
{ audience: "user" }
);
if (typeof tokenPayload === "string") {
throw new Error("unexpected JsonWebToken type");
}
let user = req.app.locals.controller.userManager.users.get(tokenPayload.user);
if (!user) {
throw new Error("invalid user");
}
if (!tokenPayload.iat || tokenPayload.iat < user.tokenValidAfter) {
throw new Error("invalid token");
}
res.locals.user = user;
} catch (err) {
res.sendStatus(401);
return;
}
next();
}
// Handle an uploaded export package.
async function uploadExport(req: Request, res: Response) {
if (req.get("Content-Type") !== "application/zip") {
res.sendStatus(415);
return;
}
if (typeof req.query.mod_pack_id !== "string") {
res.sendStatus(400);
return;
}
let modPackId = Number.parseInt(req.query.mod_pack_id, 10);
if (!Number.isInteger(modPackId)) {
res.sendStatus(400);
return;
}
let modPack = res.app.locals.controller.modPacks.get(modPackId);
if (!modPack) {
res.sendStatus(400);
return;
}
let data = [];
for await (let chunk of req) {
data.push(chunk);
}
let buffer: Buffer | null = Buffer.concat(data);
let zip = await JSZip.loadAsync(buffer);
buffer = null;
// This is hardcoded to prevent path expansion attacks
let exportFiles = [
"export/settings.json",
"export/prototypes.json",
"export/item-spritesheet.png",
"export/item-metadata.json",
"export/locale.json",
];
let assets: any = {};
let settingPrototypes = {};
for (let filePath of exportFiles) {
let file = zip.file(filePath);
if (!file) {
continue;
}
if (filePath === "export/settings.json") {
settingPrototypes = JSON.parse(await file.async("text"));
}
let { name, ext } = path.posix.parse(filePath);
let hash = await lib.hashStream(file.nodeStream());
assets[name] = `${name}.${hash}${ext}`;
await lib.safeOutputFile(path.join("static", `${name}.${hash}${ext}`), await file.async("nodebuffer"));
}
modPack.exportManifest = new lib.ExportManifest(assets);
modPack.fillDefaultSettings(settingPrototypes, logger);
res.app.locals.controller.modPacksUpdated([modPack]);
res.sendStatus(200);
}
export interface ProxyStream {
id: string;
source: nodeStream.Readable | null;
flowing: boolean;
size?: string;
mime?: string;
filename: string | null;
events: events.EventEmitter;
timeout: ReturnType<typeof setTimeout>;
}
export async function createProxyStream(app: Application): Promise<ProxyStream> {
let asyncRandomBytes = util.promisify(crypto.randomBytes);
let id = (await asyncRandomBytes(8)).toString("hex");
let stream: ProxyStream = {
id,
source: null,
flowing: false,
size: undefined,
mime: undefined,
filename: null,
events: new events.EventEmitter(),
timeout: setTimeout(() => {
stream.events.emit("timeout");
}, app.locals.controller.config.get("controller.proxy_stream_timeout") * 1000),
};
stream.events.on("close", () => {
clearTimeout(stream.timeout);
app.locals.streams.delete(id);
});
stream.events.on("timeout", () => {
stream.events.emit("close");
});
app.locals.streams.set(id, stream);
return stream;
}
async function putStream(req: Request, res: Response) {
const stream = req.app.locals.streams.get(req.params.id);
if (!stream || stream.source) {
res.sendStatus(404);
return;
}
stream.source = req;
stream.mime = req.get("Content-Type");
stream.size = req.get("Content-Length");
stream.events.emit("source");
stream.events.on("close", () => {
if (!stream.flowing) {
req.resume();
res.sendStatus(500);
} else {
res.sendStatus(200);
}
});
}
function startStream(res: Response, stream: ProxyStream) {
res.append("Content-Type", stream.mime);
if (stream.size) {
res.append("Content-Length", stream.size);
}
if (stream.filename) {
res.append("Content-Disposition", `attachment; filename="${stream.filename}"`);
} else {
res.append("Content-Disposition", "attachment");
}
stream.source!.pipe(res);
stream.flowing = true;
res.on("close", () => {
stream.events.emit("close");
});
clearTimeout(stream.timeout);
}
async function getStream(req: Request, res: Response) {
const stream = req.app.locals.streams.get(req.params.id);
if (!stream || stream.flowing) {
res.sendStatus(404);
return;
}
if (stream.source) {
startStream(res, stream);
} else {
stream.events.on("source", () => startStream(res, stream));
stream.events.on("timeout", () => {
res.sendStatus(500);
});
}
}
const zipMimes = [
"application/zip",
"application/x-zip-compressed",
];
const contentTypeRegExp = /^([!#$%&'*+\-.^_`|~0-9A-Za-z]+\/[!#$%&'*+\-.^_`|~0-9A-Za-z]+)/;
async function uploadSave(req: Request, res: Response) {
try {
res.locals.user.checkPermission("core.instance.save.upload");
} catch (err: any) {
res.status(403).json({ request_errors: [err.message] });
return;
}
let contentType = req.get("Content-Type") || "";
let match = contentTypeRegExp.exec(contentType);
if (!match) {
res.status(415).json({ request_errors: ["invalid Content-Type"] });
return;
}
let contentMime = match[1].toLowerCase();
let tasks = [];
let errors: string[] = [];
let requestErrors = [];
let saves: string[] = [];
async function handleFile(instanceId:number, stream: nodeStream.Readable, filename: string, streamMime: string) {
let proxyStream = await createProxyStream(req.app);
proxyStream.source = stream;
proxyStream.mime = streamMime;
let timeout = new Promise<never>((_, reject) => {
proxyStream.events.on("timeout", () => {
stream.resume();
reject(new Error("Timed out establishing stream to host"));
});
});
try {
let storedName: string = await Promise.race([
req.app.locals.controller.sendToHostByInstanceId(
new lib.InstancePullSaveRequest(
instanceId,
proxyStream.id,
filename,
)
),
timeout,
]);
saves.push(storedName);
} catch (err: any) {
proxyStream.events.emit("close");
logger.error(`Error uploading save: ${err.message}`);
errors.push(err.message);
stream.resume();
}
}
if (contentMime === "multipart/form-data") {
await new Promise(resolve => {
let instanceId: number | undefined;
function fileHandler(
name: string,
stream: nodeStream.Readable,
{ filename, mimeType }: { filename: string, mimeType: string }
) {
if (instanceId === undefined) {
requestErrors.push("instance_id must come before files uploaded");
}
if (!zipMimes.includes(mimeType)) {
requestErrors.push("invalid file Content-Type");
}
if (!filename.endsWith(".zip")) {
requestErrors.push("filename must end with .zip");
}
if (errors.length || requestErrors.length) {
stream.resume();
return;
}
tasks.push(handleFile(instanceId as number, stream, filename, mimeType));
}
let parser = busboy({ headers: req.headers });
parser.on("file", fileHandler);
parser.on("field", (name, value, info) => {
if (name === "instance_id") {
instanceId = Number.parseInt(value, 10);
if (Number.isNaN(instanceId)) {
requestErrors.push("invalid instance_id");
}
}
});
parser.on("close", resolve);
parser.on("error", (err: any) => {
logger.error(`Error parsing multipart request in upload-save:\n${err.stack}`);
errors.push(err.message);
});
req.pipe(parser);
});
} else if (zipMimes.includes(contentMime)) {
let filename = req.query.filename;
if (typeof filename !== "string") {
requestErrors.push("Missing or invalid filename parameter");
} else if (!filename.endsWith(".zip")) {
requestErrors.push("filename must end with .zip");
}
let instanceId = Number.parseInt(String(req.query.instance_id), 10);
if (Number.isNaN(instanceId)) {
requestErrors.push("Missing or invalid instance_id parameter");
}
if (errors.length || requestErrors.length) {
req.resume();
} else {
tasks.push(handleFile(instanceId, req, filename as string, contentMime));
}
} else {
res.status(415).json({ request_errors: ["invalid Content-Type"] });
return;
}
await Promise.all(tasks);
if (errors.length) {
res.status(500);
res.json({ errors, request_errors: requestErrors });
return;
}
if (requestErrors.length) {
res.status(400);
res.json({ request_errors: requestErrors });
return;
}
res.json({ saves });
}
function checkModName(name: string) {
try {
lib.checkFilename(name);
} catch (err: any) {
throw new lib.RequestError(`Mod name ${err.message}`);
}
}
async function uploadMod(req: Request, res: Response) {
try {
res.locals.user.checkPermission("core.mod.upload");
} catch (err: any) {
res.status(403).json({ request_errors: [err.message] });
return;
}
let contentType = req.get("Content-Type")!;
let match = contentTypeRegExp.exec(contentType);
if (!match) {
res.status(415).json({ request_errors: ["invalid Content-Type"] });
return;
}
let contentMime = match[1].toLowerCase();
let tasks = [];
let errors: string[] = [];
let requestErrors = [];
let mods: any[] = [];
async function handleFile(stream: nodeStream.Readable, filename: string) {
try {
checkModName(filename);
} catch (err: any) {
logger.error(`Error uploading mod: ${err.message}`);
errors.push(err.message);
stream.resume();
return;
}
const modsDirectory = req.app.locals.controller.config.get("controller.mods_directory");
let tempFilename = filename.replace(/(\.zip)?$/, ".tmp.zip");
let writeStream;
try {
while (true) {
try {
writeStream = fs.createWriteStream(path.join(modsDirectory, tempFilename), { flags: "wx" });
await events.once(writeStream, "open");
break;
} catch (err: any) {
if (err.code === "EEXIST") {
tempFilename = await lib.findUnusedName(modsDirectory, tempFilename, ".tmp.zip");
} else {
throw err;
}
}
}
stream.pipe(writeStream);
await finished(writeStream);
const modInfo = await lib.ModInfo.fromModFile(path.join(modsDirectory, tempFilename));
checkModName(modInfo.filename);
await fs.rename(path.join(modsDirectory, tempFilename), path.join(modsDirectory, modInfo.filename));
req.app.locals.controller.modStore.addMod(modInfo);
mods.push(modInfo.toJSON());
} catch (err: any) {
logger.error(`Error uploading mod: ${err.message}`);
errors.push(err.message);
stream.resume();
// Attempt to clean up.
if (writeStream) {
writeStream.destroy();
}
try {
await fs.unlink(path.join(modsDirectory, tempFilename));
} catch (unlinkErr: any) {
if (unlinkErr.code !== "ENOENT") {
logger.error(`Error removing ${tempFilename}: ${unlinkErr.message}`);
}
}
}
}
if (contentMime === "multipart/form-data") {
await new Promise(resolve => {
function fileHandler(
name: string,
stream: nodeStream.Readable,
{ filename, mimeType }: { filename: string, mimeType: string }
) {
if (!zipMimes.includes(mimeType)) {
requestErrors.push("invalid file Content-Type");
}
if (!filename.endsWith(".zip")) {
requestErrors.push("filename must end with .zip");
}
if (errors.length || requestErrors.length) {
stream.resume();
return;
}
tasks.push(handleFile(stream, filename));
}
let parser = busboy({ headers: req.headers });
parser.on("file", fileHandler);
parser.on("close", resolve);
parser.on("error", (err: any) => {
logger.error(`Error parsing multipart request in upload-mod:\n${err.stack}`);
errors.push(err.message);
});
req.pipe(parser);
});
} else if (zipMimes.includes(contentMime)) {
let filename = req.query.filename;
if (typeof filename !== "string") {
requestErrors.push("Missing or invalid filename parameter");
} else if (!filename.endsWith(".zip")) {
requestErrors.push("filename must end with .zip");
}
if (errors.length || requestErrors.length) {
req.resume();
} else {
tasks.push(handleFile(req, filename as string));
}
} else {
res.status(415).json({ request_errors: ["invalid Content-Type"] });
return;
}
await Promise.all(tasks);
if (errors.length) {
res.status(500);
res.json({ errors, request_errors: requestErrors });
return;
}
if (requestErrors.length) {
res.status(400);
res.json({ request_errors: requestErrors });
return;
}
res.json({ mods });
}
export function addRouteHandlers(app: Application) {
app.get("/metrics", (req:Request, res:Response, next:any) => getMetrics(req, res, next).catch(next));
app.get("/api/plugins", getPlugins);
app.put("/api/upload-export",
validateHostToken,
(req:Request, res:Response, next:any) => uploadExport(req, res).catch(next)
);
app.put("/api/stream/:id", (req:Request, res:Response, next:any) => putStream(req, res).catch(next));
app.get("/api/stream/:id", (req:Request, res:Response, next:any) => getStream(req, res).catch(next));
app.post("/api/upload-save",
validateUserToken,
(req:Request, res:Response, next:any) => uploadSave(req, res).catch(next)
);
app.post("/api/upload-mod",
validateUserToken,
(req:Request, res:Response, next:any) => uploadMod(req, res).catch(next)
);
}
// Routes used in the web interface and served by the controller
export const webRoutes = [
"/",
"/controller",
"/hosts",
"/hosts/:id/view",
"/instances",
"/instances/:id/view",
"/mods",
"/mods/mod-packs/:id/view",
"/users",
"/users/:id/view",
"/roles",
"/roles/:id/view",
"/plugins",
"/plugins/:name/view",
];
+12
View File
@@ -0,0 +1,12 @@
{
"extends": "../../tsconfig.browser.json",
"references": [
{ "path": "../lib/tsconfig.browser.json" },
{ "path": "../web_ui/tsconfig.browser.json" },
],
"include": [ "web/**/*.tsx", "web/**/*.ts", "package.json" ],
"compilerOptions": {
"rootDir": "..",
"outDir": "dist/browser",
}
}
+7
View File
@@ -0,0 +1,7 @@
{
"files": [],
"references": [
{ "path": "./tsconfig.node.json" },
{ "path": "./tsconfig.browser.json" },
],
}
+10
View File
@@ -0,0 +1,10 @@
{
"extends": "../../tsconfig.node.json",
"references": [
{ "path": "../lib/tsconfig.node.json" },
],
"include": ["index.ts", "controller.ts", "package.json", "src/**/*.ts"],
"compilerOptions": {
"outDir": "dist/node",
},
}
+4
View File
@@ -0,0 +1,4 @@
{
"entryPoints": ["index.ts"],
"includeVersion": true
}
+49
View File
@@ -0,0 +1,49 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="theme-color" content="#2a2a2a" />
<title>__CLUSTER_NAME__</title>
<style>
html {
color: #fff;
background-color: #000;
}
#bootstrap-msg {
font: 20px serif;
color: #aaa;
margin: auto;
margin-top: 1em;
width: fit-content;
white-space: pre;
}
</style>
</head>
<body>
<noscript>You need to enable JavaScript to run this app.</noscript>
<div id="root">
<div id="bootstrap-msg">Loading web interface...</div>
</div>
<script id="script-bundle" src="__STATIC_ROOT____MAIN_BUNDLE__" defer></script>
<script>
webRoot = new URL("__WEB_ROOT__", document.location).pathname;
staticRoot = new URL("__STATIC_ROOT__", document.location).pathname;
function scriptError(err) {
console.error(err);
let bootstrapMsg = document.getElementById("bootstrap-msg");
if (bootstrapMsg) {
let errorMsg = err;
if (err?.message) {
errorMsg = `${err?.constructor?.name ?? "Error"}: ${err.message}`;
}
bootstrapMsg.textContent = `${errorMsg}\nSee web console for details.`;
}
}
const bundleScript = document.getElementById("script-bundle");
bundleScript.addEventListener("error", () => scriptError("Main bundle failed to load."));
window.addEventListener("error", event => scriptError(event.error));
</script>
</body>
</html>
+5
View File
@@ -0,0 +1,5 @@
declare function scriptError(err: any): void;
import("@clusterio/web_ui")
.then(webUi => webUi.bootstrap())
.catch(scriptError)
;
File renamed without changes.

Before

Width:  |  Height:  |  Size: 20 KiB

After

Width:  |  Height:  |  Size: 20 KiB

@@ -7,24 +7,24 @@ const common = require("@clusterio/web_ui/webpack.common");
module.exports = (env = {}) => merge(common(env), {
context: __dirname,
entry: "./web/index.jsx",
entry: "./web/index.tsx",
devServer: {
contentBase: "./dist/web",
},
output: {
publicPath: "auto",
filename: "bundle.js",
path: path.resolve(__dirname, "dist", "web"),
},
plugins: [
new webpack.container.ModuleFederationPlugin({
name: "master",
name: "controller",
shared: {
"@clusterio/lib": { singleton: true },
"@clusterio/web_ui": { singleton: true },
"antd": { singleton: true },
"react": { singleton: true },
"react-dom": { singleton: true },
"react-router": { singleton: true },
"react-router-dom": { singleton: true },
},
}),
],
+414 -104
View File
@@ -2,7 +2,6 @@
"use strict";
const child_process = require("child_process");
const events = require("events");
const fs = require("fs-extra");
const inquirer = require("inquirer");
const os = require("os");
@@ -13,9 +12,23 @@ const util = require("util");
const yargs = require("yargs");
const { levels, logger, setLogLevel } = require("./logging");
const { copyPluginTemplates } = require("./template");
let dev = false;
const scriptExt = process.platform === "win32" ? ".cmd" : "";
const finished = util.promisify(stream.finished);
// I hate this, there is a bug with rl.close on windows that was meant to have been fixed in Node14.4
// See nodejs/node#21771 and nodejs/node#30701 for the apparent fix that was applied
// One option is to not call rl.close: SBoudrias/Inquirer.js#767
// But this comes with its own issues: SBoudrias/Inquirer.js#899
// The solution below was adapted from aws-amplify/amplify-cli#12347
if (process.platform === "win32") {
const _inquirer_close = inquirer.ui.Prompt.prototype.close;
inquirer.ui.Prompt.prototype.close = function() {
this.rl.terminal = false;
_inquirer_close.call(this);
};
}
const availablePlugins = [
{ name: "Global Chat", value: "@clusterio/plugin-global_chat" },
@@ -85,11 +98,17 @@ class LineSplitter extends stream.Transform {
class InstallError extends Error { }
async function safeOutputFile(file, data, options={}) {
let { dir, name, ext } = path.parse(file);
let temporary = path.join(dir, `${name}.tmp${ext}`);
await fs.outputFile(temporary, data, options);
await fs.rename(temporary, file);
}
async function execFile(cmd, args) {
logger.verbose(`executing ${cmd} ${args.join(" ")}`);
const asyncExec = util.promisify(child_process.execFile);
return new Promise((resolve, reject) => {
let child = child_process.execFile(cmd, args, (err, stdout, stderr) => {
let child = child_process.execFile(cmd, args, { shell: true }, (err, stdout, stderr) => {
if (err) {
reject(err);
} else {
@@ -105,21 +124,28 @@ async function execFile(cmd, args) {
});
}
async function execMaster(args) {
async function execController(args) {
if (dev) {
return await execFile("node", [path.join("packages", "master"), ...args]);
return await execFile("node", [path.join(__dirname, "..", "controller"), ...args]);
}
return await execFile(path.join("node_modules", ".bin", `clusteriomaster${scriptExt}`), args);
return await execFile(path.join("node_modules", ".bin", `clusteriocontroller${scriptExt}`), args);
}
async function execSlave(args) {
async function execHost(args) {
if (dev) {
return await execFile("node", [path.join("packages", "slave"), ...args]);
return await execFile("node", [path.join(__dirname, "..", "host"), ...args]);
}
return await execFile(path.join("node_modules", ".bin", `clusterioslave${scriptExt}`), args);
return await execFile(path.join("node_modules", ".bin", `clusteriohost${scriptExt}`), args);
}
function validateSlaveToken(token) {
async function execCtl(args) {
if (dev) {
return await execFile("node", [path.join(__dirname, "..", "ctl"), ...args]);
}
return await execFile(path.join("node_modules", ".bin", `clusterioctl${scriptExt}`), args);
}
function validateHostToken(token) {
let parts = token.split(".");
if (parts.length !== 3) {
throw new InstallError("Invalid token");
@@ -132,7 +158,7 @@ function validateSlaveToken(token) {
throw new InstallError("Invalid token");
}
if (parsed.aud !== "slave" || !Number.isInteger(parsed.slave)) {
if (parsed.aud !== "host" || !Number.isInteger(parsed.host)) {
throw new InstallError("Invalid token");
}
}
@@ -156,9 +182,169 @@ async function validateInstallDir() {
}
}
async function validateNotRoot(args) {
if (args.allowInstallAsRoot) {
return;
}
if (os.userInfo().uid === 0) {
throw new InstallError(
"Refusing to install as root. Create a separate user account for clusterio " +
"and then use su/sudo to switch to it before invoking the installer."
);
}
}
async function migrateRename(args) {
function rename(string) {
return string.replace(/master/g, "controller").replace(/slave/g, "host");
}
function renameConfig(config) {
for (let group of config["groups"]) {
group["name"] = rename(group["name"]);
group["fields"] = Object.fromEntries(
Object.entries(group["fields"]).map(([k, v]) => [rename(k), v])
);
}
return config;
}
async function migrateConfig(source, destination) {
if (await fs.pathExists(source) && !await fs.pathExists(destination)) {
await safeOutputFile(destination, JSON.stringify(
renameConfig(JSON.parse(await fs.readFile(source))), null, "\t"
));
logger.info(`Migrated ${source} to ${destination}`);
}
}
await migrateConfig("config-master.json", "config-controller.json");
await migrateConfig("config-slave.json", "config-host.json");
let instancesFile = path.join("database", "instances.json");
if (await fs.pathExists(instancesFile)) {
await safeOutputFile(instancesFile, JSON.stringify(
JSON.parse(await fs.readFile(instancesFile)).map(renameConfig), null, "\t"
));
logger.info(`Migrated ${instancesFile}`);
}
let usersFile = path.join("database", "users.json");
if (await fs.pathExists(usersFile)) {
let users = JSON.parse(await fs.readFile(usersFile));
for (let role of users["roles"]) {
role["permissions"] = role["permissions"].map(rename);
}
await safeOutputFile(usersFile, JSON.stringify(users, null, "\t"));
logger.info(`Migrated ${usersFile}`);
}
function renameLogLine(info) {
return Object.fromEntries(
Object.entries(info).map(([k, v]) => [rename(k), v])
);
}
async function migrateLog(inputFile, outputFile) {
let lineStream = new LineSplitter({ readableObjectMode: true });
let fileStream = fs.createReadStream(inputFile);
fileStream.pipe(lineStream);
let lines = [];
for await (let inputLine of lineStream) {
try {
lines.push(Buffer.from(JSON.stringify(renameLogLine(JSON.parse(inputLine)))));
lines.push(Buffer.from("\n"));
} catch (err) {
if (!(err instanceof SyntaxError)) {
throw err;
}
logger.warn(`Invalid log line: ${inputLine.toString()}`);
lines.push(inputLine);
}
}
safeOutputFile(outputFile, Buffer.concat(lines));
}
async function migrateLogsDir(inputLogs, outputLogs) {
if (await fs.pathExists(inputLogs)) {
logger.info(`Migrating ${inputLogs} to ${outputLogs}`);
for (let logFile of await fs.readdir(inputLogs)) {
if (!logFile.endsWith(".log")) {
continue;
}
let inputFile = path.join(inputLogs, logFile);
let outputFile = path.join(outputLogs, rename(logFile));
if (!await fs.pathExists(outputFile)) {
await migrateLog(inputFile, outputFile);
logger.verbose(`Migrated ${inputFile} to ${outputFile}`);
}
}
}
}
await migrateLogsDir(path.join("logs", "master"), path.join("logs", "controller"));
await migrateLogsDir(path.join("logs", "slave"), path.join("logs", "host"));
if (
await fs.pathExists(path.join("logs", "cluster"))
&& !await fs.pathExists(path.join("logs", "cluster-prerename"))
) {
await fs.rename(path.join("logs", "cluster"), path.join("logs", "cluster-prerename"));
await migrateLogsDir(path.join("logs", "cluster-prerename"), path.join("logs", "cluster"));
}
if (
await fs.pathExists("sharedMods")
&& !await fs.pathExists("mods")
) {
logger.info("Moving sharedMods/ to mods/");
await fs.rename("sharedMods", "mods");
}
if (!args.dev) {
let pkg = JSON.parse(await fs.readFile("package.json"));
if (pkg.dependencies) {
let convert = ["@clusterio/master", "@clusterio/slave"];
let uninstall = convert.filter(c => pkg.dependencies[c] !== undefined);
let install = uninstall.map(rename);
if (uninstall.length) {
logger.info(`Replacing ${uninstall.join(" and ")}`);
await execFile(`npm${scriptExt}`, ["install", ...install]);
await execFile(`npm${scriptExt}`, ["uninstall", ...uninstall]);
}
}
logger.info("Updating packages");
await execFile(`npm${scriptExt}`, ["update"]);
}
const hasRunMaster = await fs.pathExists("run-master.sh") || await fs.pathExists("run-master.cmd");
const hasRunSlave = await fs.pathExists("run-slave.sh") || await fs.pathExists("run-slave.cmd");
if (hasRunMaster || hasRunSlave) {
logger.info("Writing run scripts");
let mode = "standalone";
if (!hasRunSlave) { mode = "controller"; }
if (!hasRunMaster) { mode = "host"; }
await writeScripts(mode);
}
logger.info(
"Migration complete, you may now delete the following left over files and directories (if present):" +
"\n- config-master.json" +
"\n- config-slave.json" +
"\n- logs/master" +
"\n- logs/slave" +
"\n- logs/cluster-prerename" +
"\n- systemd/clusteriomaster.service" +
"\n- systemd/clusterioslave.service" +
"\n- run-master.sh / run-master.cmd" +
"\n- run-slave.sh / run-slave.cmd"
);
}
async function installClusterio(mode, plugins) {
try {
await fs.writeFile("package.json", JSON.stringify({
await fs.outputFile("package.json", JSON.stringify({
name: "clusterio-install",
private: true,
}, null, 2), { flag: "wx" });
@@ -169,11 +355,11 @@ async function installClusterio(mode, plugins) {
}
let components = [];
if (["standalone", "master"].includes(mode)) {
components.push("@clusterio/master");
if (["standalone", "controller"].includes(mode)) {
components.push("@clusterio/controller");
}
if (["standalone", "slave"].includes(mode)) {
components.push("@clusterio/slave");
if (["standalone", "host"].includes(mode)) {
components.push("@clusterio/host");
}
if (mode === "ctl") {
components.push("@clusterio/ctl");
@@ -201,78 +387,129 @@ async function installClusterio(mode, plugins) {
for (let plugin of plugins) {
if (!pluginList.has(plugin)) {
// eslint-disable-next-line node/global-require
let pluginInfo = require(require.resolve(path.posix.join(plugin, "info"), { paths: [process.cwd()] }));
let pluginInfo = require(require.resolve(plugin, { paths: [process.cwd()] })).plugin;
pluginList.set(pluginInfo.name, plugin);
}
}
try {
await fs.writeFile("plugin-list.json", JSON.stringify([...pluginList], null, 4));
await safeOutputFile("plugin-list.json", JSON.stringify([...pluginList], null, "\t"));
} catch (err) {
throw new InstallError(`Error writing plugin-list.json: ${err.message}`);
}
}
}
async function groupIdToName(gid) {
try {
let exec = util.promisify(child_process.exec);
let { stdout } = await exec(`getent group ${gid}`);
return stdout.split(":")[0];
} catch (err) {
logger.warn(`getent group ${gid} failed: ${err.message}`);
return gid;
}
}
async function writeScripts(mode) {
if (["standalone", "master"].includes(mode)) {
if (["standalone", "controller"].includes(mode)) {
if (process.platform === "win32") {
await fs.writeFile(
"run-master.cmd",
"@echo off\n.\\node_modules\\.bin\\clusteriomaster.cmd run\n"
await safeOutputFile(
"run-controller.cmd",
`\
@echo off
set "NODE_OPTIONS=--enable-source-maps %NODE_OPTIONS%"
:restart
call .\\node_modules\\.bin\\clusteriocontroller.cmd run --can-restart
if %errorlevel% equ 0 exit /b
if %errorlevel% equ 8 exit /b
goto restart
`
);
} else {
await fs.writeFile(
"run-master.sh",
"#!/bin/sh\nexec ./node_modules/.bin/clusteriomaster run\n",
await safeOutputFile(
"run-controller.sh",
`\
#!/bin/bash
export "NODE_OPTIONS=--enable-source-maps $NODE_OPTIONS"
while true; do
./node_modules/.bin/clusteriocontroller run --can-restart
if [[ $? -eq 0 || $? -eq 8 ]]; then exit $?; fi
done
`,
{ mode: 0o755 },
);
await fs.outputFile(
"systemd/clusteriomaster.service",
`[Unit]
Description=Clusterio Master
await safeOutputFile(
"systemd/clusteriocontroller.service",
`\
[Unit]
Description=Clusterio Controller
[Service]
User=${os.userInfo().username}
Group=nogroup
Group=${await groupIdToName(os.userInfo().gid)}
WorkingDirectory=${process.cwd()}
KillMode=mixed
KillSignal=SIGINT
ExecStart=${process.cwd()}/node_modules/.bin/clusteriomaster run --log-level=warn
Environment=NODE_OPTIONS=--enable-source-maps
ExecStart=${process.cwd()}/node_modules/.bin/clusteriocontroller run --log-level=warn --can-restart
Restart=on-failure
RestartPreventExitStatus=8
[Install]
WantedBy=multi-user.target
`);
`
);
}
}
if (["standalone", "slave"].includes(mode)) {
if (["standalone", "host"].includes(mode)) {
if (process.platform === "win32") {
await fs.writeFile(
"run-slave.cmd",
"@echo off\n.\\node_modules\\.bin\\clusterioslave.cmd run\n"
await safeOutputFile(
"run-host.cmd",
`\
@echo off
set "NODE_OPTIONS=--enable-source-maps %NODE_OPTIONS%"
:restart
call .\\node_modules\\.bin\\clusteriohost.cmd run --can-restart
if %errorlevel% equ 0 exit /b
if %errorlevel% equ 8 exit /b
goto restart
`
);
} else {
await fs.writeFile(
"run-slave.sh",
"#!/bin/sh\nexec ./node_modules/.bin/clusterioslave run\n",
await safeOutputFile(
"run-host.sh",
`\
#!/bin/bash
export "NODE_OPTIONS=--enable-source-maps $NODE_OPTIONS"
while true; do
./node_modules/.bin/clusteriohost run --can-restart
if [[ $? -eq 0 || $? -eq 8 ]]; then exit $?; fi
done
`,
{ mode: 0o755 },
);
await fs.outputFile(
"systemd/clusterioslave.service",
`[Unit]
Description=Clusterio Slave
await safeOutputFile(
"systemd/clusteriohost.service",
`\
[Unit]
Description=Clusterio Host
[Service]
User=${os.userInfo().username}
Group=nogroup
Group=${await groupIdToName(os.userInfo().gid)}
WorkingDirectory=${process.cwd()}
KillMode=mixed
KillSignal=SIGINT
ExecStart=${process.cwd()}/node_modules/.bin/clusterioslave run --log-level=warn
Environment=NODE_OPTIONS=--enable-source-maps
ExecStart=${process.cwd()}/node_modules/.bin/clusteriohost run --log-level=warn --can-restart
Restart=on-failure
RestartPreventExitStatus=8
[Install]
WantedBy=multi-user.target
`);
`
);
}
}
}
@@ -281,6 +518,7 @@ WantedBy=multi-user.target
async function inquirerMissingArgs(args) {
let answers = {};
if (args.mode) { answers.mode = args.mode; }
if (args["plugin-template"]) { answers.mode = "plugin-template"; }
answers = await inquirer.prompt([
{
type: "list",
@@ -288,16 +526,17 @@ async function inquirerMissingArgs(args) {
message: "Operating mode to install",
default: "standalone",
choices: [
{ name: "Standalone (install both master and slave on this computer)", value: "standalone" },
{ name: "Master only", value: "master" },
{ name: "Slave only", value: "slave" },
{ name: "Standalone (install both controller and host on this computer)", value: "standalone" },
{ name: "Controller only", value: "controller" },
{ name: "Host only", value: "host" },
{ name: "Ctl only", value: "ctl" },
{ name: "Plugins only", value: "plugins" },
{ name: "Plugin template", value: "plugin-template" },
],
},
], answers);
if (["standalone", "master"].includes(answers.mode)) {
if (["standalone", "controller"].includes(answers.mode)) {
if (args.admin) { answers.admin = args.admin; }
answers = await inquirer.prompt([
{
@@ -314,43 +553,43 @@ async function inquirerMissingArgs(args) {
], answers);
}
if (answers.mode === "slave") {
if (args.slaveName) { answers.slaveName = args.slaveName; }
if (answers.mode === "host") {
if (args.hostName) { answers.hostName = args.hostName; }
answers = await inquirer.prompt([
{
type: "input",
name: "slaveName",
message: "Name of slave",
name: "hostName",
message: "Name of host",
},
], answers);
}
if (["slave", "ctl"].includes(answers.mode)) {
if (args.masterUrl) { answers.masterUrl = args.masterUrl; }
if (["host", "ctl"].includes(answers.mode)) {
if (args.controllerUrl) { answers.controllerUrl = args.controllerUrl; }
answers = await inquirer.prompt([
{
type: "input",
name: "masterUrl",
message: "Master server URL",
name: "controllerUrl",
message: "Controller URL",
},
], answers);
if (args.masterToken) { answers.masterToken = args.masterToken; }
if (args.controllerToken) { answers.controllerToken = args.controllerToken; }
answers = await inquirer.prompt([
{
type: "input",
name: "masterToken",
message: "Master authentication Token",
name: "controllerToken",
message: "Controller authentication Token",
},
], answers);
}
if (answers.mode === "slave") {
validateSlaveToken(answers.masterToken);
if (answers.mode === "host") {
validateHostToken(answers.controllerToken);
}
if (["standalone", "slave"].includes(answers.mode)) {
let myIp;
if (["standalone", "host"].includes(answers.mode)) {
let myIp = "localhost";
if (args.publicAddress) {
answers.publicAddress = args.publicAddress;
} else {
@@ -388,7 +627,7 @@ async function inquirerMissingArgs(args) {
},
], answers);
if (answers.downloadHeadless) {
if (!answers.factorioDir && answers.downloadHeadless) {
answers.factorioDir = "factorio";
}
}
@@ -420,7 +659,43 @@ async function inquirerMissingArgs(args) {
}
}
if (dev) { answers.plugins = []; }
if (answers.mode === "plugin-template") {
if (args["plugin-template"] && args["plugin-template"].length > 0) {
answers.pluginTemplate = args["plugin-template"];
}
if (args["plugin-name"]) {
answers.pluginName = args["plugin-name"];
}
answers.plugins = [];
answers = await inquirer.prompt([
{
type: "checkbox",
name: "pluginTemplate",
message: "Plugin templates to use",
choices: [
{ name: "Controller", value: "controller" },
{ name: "Host", value: "host" },
{ name: "Instance", value: "instance" },
{ name: "Lua Module", value: "module" },
{ name: "Command Line", value: "ctl" },
{ name: "Web UI", value: "web" },
{ name: "No Typescript", value: "js" },
],
},
{
type: "input",
name: "pluginName",
message: "Name of your new plugin",
default: path.basename(path.resolve()),
},
], answers);
}
if (dev) {
answers.plugins = [];
} else if (args.plugins) {
answers.plugins = args.plugins;
}
answers = await inquirer.prompt([
{
type: "checkbox",
@@ -463,7 +738,7 @@ async function downloadLinuxServer() {
const writeStream = fs.createWriteStream(tmpArchivePath);
res.pipe(writeStream);
await events.once(res, "end");
await finished(writeStream);
await fs.rename(tmpArchivePath, archivePath);
try {
@@ -492,39 +767,53 @@ async function main() {
})
.option("mode", {
nargs: 1, describe: "Operating mode to install",
choices: ["standalone", "master", "slave", "ctl", "plugins"],
choices: ["standalone", "controller", "host", "ctl", "plugins", "plugin-template"],
})
.option("migrate-rename", {
nargs: 0, describe: "Migrate from before slave/master rename", default: false, type: "boolean",
})
.option("admin", {
nargs: 1, describe: "Admin account name [standalone/master]", type: "string",
nargs: 1, describe: "Admin account name [standalone/controller]", type: "string",
})
.option("slave-name", {
nargs: 1, describe: "Slave name [slave]", type: "string",
.option("host-name", {
nargs: 1, describe: "Host name [host]", type: "string",
})
.option("master-url", {
nargs: 1, describe: "Master URL [slave/ctl]", type: "string",
.option("controller-url", {
nargs: 1, describe: "Controller URL [host/ctl]", type: "string",
})
.option("master-token", {
nargs: 1, describe: "Master authentication token [slave/ctl]", type: "string",
.option("controller-token", {
nargs: 1, describe: "Controller authentication token [host/ctl]", type: "string",
})
.option("public-address", {
nargs: 1, describe: "DNS/IP Address to connect to this server [standalone/slave]", type: "string",
nargs: 1, describe: "DNS/IP Address to connect to this server [standalone/host]", type: "string",
})
.option("factorio-dir", {
nargs: 1, describe: "Path to Factorio installation [standalone/slave]", type: "string",
nargs: 1, describe: "Path to Factorio installation [standalone/host]", type: "string",
})
.option("plugins", {
array: true, describe: "Plugins to install", type: "string",
});
})
.option("plugin-template", {
array: true, describe: "Plugin template to use [plugin-template]", type: "string",
})
.option("plugin-name", {
nargs: 1, describe: "Name of your new plugin [plugin-template]", type: "string",
})
.strict()
;
if (process.platform === "linux") {
args = args
.option("allow-install-as-root", {
nargs: 0, describe: "(Linux only) Allow installing as root (not recommended)", type: "boolean",
})
.option("download-headless", {
nargs: 0,
describe: "(Linux only) Automatically download and unpack the latest factorio release. " +
"Can be set to false using --no-download-headless.",
type: "boolean",
})
.conflicts("factorio-dir", "download-headless");
;
}
args = args.argv;
@@ -534,56 +823,77 @@ async function main() {
if (!dev) {
await validateInstallDir();
await validateNotRoot(args);
}
if (args.migrateRename) {
await migrateRename(args);
return;
}
let answers = await inquirerMissingArgs(args);
if (answers.downloadHeadless) {
await downloadLinuxServer();
logger.verbose(JSON.stringify(answers));
if (answers.pluginTemplate) {
await copyPluginTemplates(answers.pluginName, answers.pluginTemplate);
return;
}
logger.verbose(JSON.stringify(answers));
if (answers.downloadHeadless) {
if (answers.factorioDir !== "factorio") {
throw new InstallError("--download-headless option requires --factorio-dir to be set to factorio");
}
await downloadLinuxServer();
}
if (!dev) {
await installClusterio(answers.mode, answers.plugins);
}
let adminToken = null;
if (["standalone", "master"].includes(answers.mode)) {
logger.info("Setting up master");
await execMaster(["bootstrap", "create-admin", answers.admin]);
let result = await execMaster(["bootstrap", "generate-user-token", answers.admin]);
if (["standalone", "controller"].includes(answers.mode)) {
logger.info("Setting up controller");
await execController(["bootstrap", "create-admin", answers.admin]);
let result = await execController(["bootstrap", "generate-user-token", answers.admin]);
adminToken = result.stdout.split("\n").slice(-2)[0];
}
if (answers.mode === "standalone") {
logger.info("Setting up slave");
await execSlave(["config", "set", "slave.name", "local"]);
logger.info("Setting up host");
await execHost(["config", "set", "host.name", "local"]);
let result = await execSlave(["config", "show", "slave.id"]);
let slaveId = Number.parseInt(result.stdout.split("\n").slice(-2)[0], 10);
let result = await execHost(["config", "show", "host.id"]);
let hostId = Number.parseInt(result.stdout.split("\n").slice(-2)[0], 10);
result = await execMaster(["bootstrap", "generate-slave-token", slaveId]);
let slaveToken = result.stdout.split("\n").slice(-2)[0];
result = await execController(["bootstrap", "generate-host-token", hostId]);
let hostToken = result.stdout.split("\n").slice(-2)[0];
await execSlave(["config", "set", "slave.master_token", slaveToken]);
await execSlave(["config", "set", "slave.factorio_directory", answers.factorioDir]);
await execHost(["config", "set", "host.controller_token", hostToken]);
await execHost(["config", "set", "host.public_address", answers.publicAddress]);
await execHost(["config", "set", "host.factorio_directory", answers.factorioDir]);
}
if (answers.mode === "slave") {
logger.info("Setting up slave");
let slaveId = JSON.parse(Buffer.from(answers.masterToken.split(".")[1], "base64")).slave;
await execSlave(["config", "set", "slave.id", slaveId]);
await execSlave(["config", "set", "slave.name", answers.slaveName]);
await execSlave(["config", "set", "slave.master_url", answers.masterUrl]);
await execSlave(["config", "set", "slave.master_token", answers.masterToken]);
await execSlave(["config", "set", "slave.factorio_directory", answers.factorioDir]);
if (answers.mode === "host") {
logger.info("Setting up host");
let hostId = JSON.parse(Buffer.from(answers.controllerToken.split(".")[1], "base64")).host;
await execHost(["config", "set", "host.id", hostId]);
await execHost(["config", "set", "host.name", answers.hostName]);
await execHost(["config", "set", "host.controller_url", answers.controllerUrl]);
await execHost(["config", "set", "host.controller_token", answers.controllerToken]);
await execHost(["config", "set", "host.public_address", answers.publicAddress]);
await execHost(["config", "set", "host.factorio_directory", answers.factorioDir]);
}
if (!dev && ["standalone", "master", "slave"].includes(answers.mode)) {
if (!dev && ["standalone", "controller", "host"].includes(answers.mode)) {
logger.info("Writing run scripts");
await writeScripts(answers.mode);
}
if (answers.mode === "ctl") {
await execCtl(["control-config", "set", "control.controller_url", answers.controllerUrl]);
await execCtl(["control-config", "set", "control.controller_token", answers.controllerToken]);
}
/* eslint-disable no-console */
console.log(`Successfully installed ${answers.mode}`);
if (adminToken) {
+26 -26
View File
@@ -1,28 +1,28 @@
{
"name": "@clusterio/create",
"description": "Installer for clusterio",
"author": "Hornwitser",
"license": "MIT",
"version": "2.0.0-alpha.9",
"main": "create.js",
"bin": {
"clusteriocreate": "./create.js"
},
"keywords": [
"factorio",
"clusterio"
],
"engines": {
"node": ">=12"
},
"dependencies": {
"chalk": "^4.1.0",
"fs-extra": "^8.1.0",
"inquirer": "^8.1.0",
"phin": "^3.5.1",
"yargs": "^14.2.0"
},
"publishConfig": {
"access": "public"
}
"name": "@clusterio/create",
"description": "Installer for clusterio",
"author": "Hornwitser",
"license": "MIT",
"version": "2.0.0-alpha.18",
"main": "create.js",
"bin": {
"clusteriocreate": "./create.js"
},
"keywords": [
"factorio",
"clusterio"
],
"engines": {
"node": ">=18"
},
"dependencies": {
"chalk": "^4.1.2",
"fs-extra": "^11.1.1",
"inquirer": "^8.2.4",
"phin": "^3.7.0",
"yargs": "^17.7.2"
},
"publishConfig": {
"access": "public"
}
}
+171
View File
@@ -0,0 +1,171 @@
"use strict";
const { logger } = require("./logging");
const fs = require("fs-extra");
const path = require("path");
async function copyTemplateFile(src, dst, properties) {
logger.verbose(`Writing ${dst} from template ${src}`);
// Closure, get the property value or throw
function getProperty(property) {
if (!properties.hasOwnProperty(property)) {
throw new Error(`Unknown property (${property}) found in template file ${src}`);
}
return properties[property];
}
// Closure, Get the boolean conjunction of a set of properties
function propertyConjunction(props) {
for (const property of props) {
if (property.startsWith("!")) {
if (Boolean(getProperty(property.slice(1)))) {
return false;
}
} else if (!Boolean(getProperty(property))) {
return false;
}
}
return true;
}
// Process the the template file
const badChars = /[\(\)-+/*<>=]/;
const templateData = await fs.readFile(src, "utf8");
const outputData = templateData
// match preprocessor comments: //%//
.replace(/^\/\/%\/\/.*\n/gm, "")
// match preprocessor insert: __property__
.replace(/__([a-zA-Z][a-zA-Z0-9_]*?[a-zA-Z0-9]?)__/gm, (_, property) => getProperty(property))
// match preprocessor conditionals: //%if <CONDITION> [// COMMENT]\n <CONTENT> \n//%endif [COMMENT]\n
.replace(/^\/\/%if ([^\n\/]*)(?:\/\/.*?)?\n((?:.|\n)*?)\/\/%endif.*\n/gm, (_, condition, content) => {
let include = false;
const conjunctions = condition.split("|");
for (const conjunct of conjunctions) {
if (badChars.test(conjunct)) {
throw new Error("Preprocessor condition can only contain logical operators" +
"such as & | or ! it can not have arithmetic, comparisons or parenthesis"
);
}
const property_names = conjunct.split("&").map(s => s.trim());
if (propertyConjunction(property_names)) {
include = true;
break;
}
}
return include ? content : "";
});
// Attempt to write the output file, warn if it already exists
try {
await fs.outputFile(dst, outputData, { flag: "wx" });
} catch (err) {
if (err.code === "EEXIST") {
logger.warn(`Could not create file ${dst} because it already exists`);
} else {
throw err;
}
}
}
async function copyPluginTemplates(pluginName, templates) {
logger.info(`Please wait, coping templates for ${templates.join(", ")}`);
const files = new Map();
const prepare = [];
const javascriptOnly = templates.includes("js");
const webpack = templates.includes("controller") || templates.includes("web");
// Get the file extension and path to the templates
const ext = javascriptOnly ? "js" : "ts";
const templatePath = path.resolve(__dirname, javascriptOnly ? "./templates/plugin-js" : "./templates/plugin-ts");
const commonPath = path.resolve(__dirname, "./templates/common");
// Files included in all templates
files.set(".gitignore", path.join(commonPath, ".gitignore"));
files.set(".npmignore", path.join(commonPath, ".npmignore"));
files.set("package.json", path.join(commonPath, "package.json"));
files.set(`messages.${ext}`, path.join(templatePath, `messages.${ext}`));
files.set(`index.${ext}`, path.join(templatePath, `index.${ext}`));
// Files and dependencies to support typescript
if (!javascriptOnly) {
prepare.push("tsc --build");
files.set("tsconfig.json", path.join(templatePath, "tsconfig.json"));
files.set("tsconfig.node.json", path.join(templatePath, "tsconfig.node.json"));
if (webpack) {
files.set("tsconfig.browser.json", path.join(templatePath, "tsconfig.browser.json"));
}
}
// Files and dependences to support webpack
if (webpack) {
prepare.push("webpack-cli --env production");
files.set("webpack.config.js", path.join(commonPath, "webpack.config.js"));
files.set(`web/index.${ext}x`, path.join(templatePath,
templates.includes("web") ? `web/plugin.${ext}x` : `web/no_plugin.${ext}x`
));
}
// Files for the controller
if (templates.includes("controller")) {
files.set(`controller.${ext}`, path.join(templatePath, `controller.${ext}`));
}
// Files for hosts
if (templates.includes("host")) {
files.set(`host.${ext}`, path.join(templatePath, `host.${ext}`));
}
// Files for instances
if (templates.includes("instance")) {
files.set(`instance.${ext}`, path.join(templatePath, `instance.${ext}`));
}
// Files for lua modules
if (templates.includes("module")) {
files.set(".luacheckrc", path.join(commonPath, ".luacheckrc"));
files.set("module/module.json", path.join(commonPath, "module/module.json"));
files.set("module/control.lua", path.join(commonPath, "module/control.lua"));
files.set("module/module_exports.lua", path.join(commonPath, "module/module_exports.lua"));
if (templates.includes("instance")) {
files.set("module/globals.lua", path.join(commonPath, "module/globals.lua"));
}
}
// Files for the control tool
if (templates.includes("ctl")) {
files.set(`ctl.${ext}`, path.join(templatePath, `ctl.${ext}`));
}
// Properties that will control the replacements in the templates
const properties = {
typescript: !javascriptOnly,
controller: templates.includes("controller"),
host: templates.includes("host"),
instance: templates.includes("instance"),
module: templates.includes("module"),
ctl: templates.includes("ctl"),
web: templates.includes("web"),
plugin_name: pluginName,
prepare: prepare.join(" && "),
ext: ext,
};
// Write all the template files
const writes = [];
for (let [dst, src] of files) {
writes.push(copyTemplateFile(src, dst, properties));
}
// Wait for writes to complete
await Promise.all(writes);
logger.info("Successfully wrote all template files");
}
module.exports = {
copyPluginTemplates,
};
@@ -0,0 +1,3 @@
dist/
node_modules/
package-lock.json
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,3 @@
webpack.config.js
dist/web/build
dist/browser
@@ -0,0 +1,54 @@
local clusterio_api = require("modules/clusterio/api")
--- Top level module table, contains event handlers and public methods
local MyModule = {
events = {},
on_nth_tick = {},
}
--- global is 'synced' between players, you should use your plugin name to avoid conflicts
-- setupGlobalData should either be removed or called during clusterio_api.events.on_server_startup
local globalData = {}
local function setupGlobalData()
if global["__plugin_name__"] == nil then
global["__plugin_name__"] = {
-- starting values go here
}
end
globalData = global["__plugin_name__"]
end
--- Public methods should be available though your top level module table
function MyModule.foo()
game.print("foo")
end
--- Private methods should be local to the file, this will prevent others from calling it
local function bar()
game.print("bar")
end
--- Clusterio provides a few custom events, on_server_startup is the most useful and should be used in place of on_load
MyModule.events[clusterio_api.events.on_server_startup] = function(event)
setupGlobalData()
game.print(game.table_to_json(event))
end
--- Factorio events are accessible through defines.events, you can have one handler per event per module
MyModule.events[defines.events.on_player_crafted_item] = function(event)
game.print(game.table_to_json(event))
//%if instance
clusterio_api.send_json("__plugin_name__-plugin_example_ipc", {
tick = game.tick, player_name = game.get_player(event.player_index).name
})
//%endif
end
--- Nth tick is a special case that requires its own table, the index represents the time period between calls in ticks
MyModule.on_nth_tick[300] = function()
game.print(game.tick)
bar()
end
--- Always return the top level module table from control, this is how clusterio will access your event handlers
return MyModule
@@ -0,0 +1,3 @@
--- Expose any globals here, typically just your control file if you want to access it through rcon or commands
-- luacheck: globals __plugin_name__
__plugin_name__ = require("modules/__plugin_name__/control")
@@ -0,0 +1,14 @@
{
"name": "__plugin_name__",
"load": [
"control.lua"
],
"require": [
//%if instance
"globals.lua"
//%endif
],
"dependencies": {
"clusterio": "*"
}
}
@@ -0,0 +1,3 @@
--- Can contain anything you want to allow other plugins to have access to, this example exposes the control file
-- Access the exports from other modules using require("modules/__plugin_name__")
return require("modules/__plugin_name__/control")
@@ -0,0 +1,53 @@
{
"name": "__plugin_name__",
"version": "0.1.0",
"description": "Example Description. Package. Change me in package.json",
//%if typescript
"main": "dist/node/index.js",
//%endif
//%if !typescript
"main": "index.js",
//%endif
"scripts": {
"prepare": "__prepare__"
},
"engines": {
"node": ">=18"
},
"peerDependencies": {
"@clusterio/lib": "^2.0.0-alpha.17"
},
"devDependencies": {
//%// Typescript and type declarations
//%if typescript
"typescript": "^5.1.6",
"@types/node": "^20.4.5",
//%endif
//%if typescript & web
"@types/react": "^18.2.21",
//%endif
//%// Everything else
//%if web
"antd": "^5.13.0",
"react": "^18.2.0",
"react-dom": "^18.2.0",
//%endif
//%if web | controller // The controller uses webpack even when there is no plugin code
"webpack": "^5.88.2",
"webpack-cli": "^5.1.4",
"webpack-merge": "^5.9.0",
"@clusterio/web_ui": "^2.0.0-alpha.17",
//%endif
"@clusterio/lib": "^2.0.0-alpha.17"
},
"dependencies": {
"@sinclair/typebox": "^0.30.4"
},
"publishConfig": {
"access": "public"
},
"keywords": [
"clusterio",
"factorio"
]
}
@@ -0,0 +1,36 @@
"use strict";
const path = require("path");
const webpack = require("webpack");
const { merge } = require("webpack-merge");
const common = require("@clusterio/web_ui/webpack.common");
module.exports = (env = {}) => merge(common(env), {
context: __dirname,
entry: "./web/index.__ext__x",
output: {
path: path.resolve(__dirname, "dist", "web"),
},
plugins: [
new webpack.container.ModuleFederationPlugin({
name: "__plugin_name__",
library: { type: "window", name: "plugin___plugin_name__" },
exposes: {
"./": "./index.__ext__",
"./package.json": "./package.json",
//%if web
"./web": "./web/index.__ext__x",
//%endif
},
shared: {
"@clusterio/lib": { import: false },
"@clusterio/web_ui": { import: false },
//%if web
"antd": { import: false },
"react": { import: false },
"react-dom": { import: false },
//%endif
},
}),
],
});
@@ -0,0 +1,73 @@
"use strict";
const lib = require("@clusterio/lib");
const { BaseControllerPlugin } = require("@clusterio/controller");
const {
PluginExampleEvent, PluginExampleRequest,
//%if controller & web // Subscribing requires web content and the controller
ExampleSubscribableUpdate, ExampleSubscribableValue,
//%endif
} = require("./messages");
class ControllerPlugin extends BaseControllerPlugin {
//%if controller & web // Subscribing requires web content and the controller
exampleDatabase;
storageDirty = false;
//%endif
async init() {
this.controller.handle(PluginExampleEvent, this.handlePluginExampleEvent.bind(this));
this.controller.handle(PluginExampleRequest, this.handlePluginExampleRequest.bind(this));
//%if controller & web // Subscribing requires web content and the controller
this.controller.subscriptions.handle(ExampleSubscribableUpdate, this.handleExampleSubscription.bind(this));
this.exampleDatabase = new Map(); // If needed, replace with loading from database file
//%endif
}
async onControllerConfigFieldChanged(field, curr, prev) {
this.logger.info(`controller::onControllerConfigFieldChanged ${field}`);
}
//%if instance
async onInstanceConfigFieldChanged(instance, field, curr, prev) {
this.logger.info(`controller::onInstanceConfigFieldChanged ${instance.id} ${field}`);
}
//%endif
async onSaveData() {
this.logger.info("controller::onSaveData");
}
async onShutdown() {
this.logger.info("controller::onShutdown");
}
async onPlayerEvent(instance, event) {
this.logger.info(`controller::onPlayerEvent ${instance.id} ${JSON.stringify(event)}`);
}
async handlePluginExampleEvent(event) {
this.logger.info(JSON.stringify(event));
}
async handlePluginExampleRequest(request) {
this.logger.info(JSON.stringify(request));
return {
myResponseString: request.myString,
myResponseNumbers: request.myNumberArray,
};
}
//%if controller & web // Subscribing requires web content and the controller
async handleExampleSubscription(request) {
const values = [...this.exampleDatabase.values()].filter(
value => value.updatedAtMs > request.lastRequestTimeMs,
);
return values.length ? new ExampleSubscribableUpdate(values) : null;
}
//%endif
}
module.exports = {
ControllerPlugin,
};
@@ -0,0 +1,31 @@
"use strict";
const { BaseCtlPlugin } = require("@clusterio/ctl");
const { CommandTree, Command } = require("@clusterio/lib");
const { PluginExampleEvent, PluginExampleRequest } = require("./messages");
/* eslint-disable no-console */
const pluginCommands = new CommandTree({
name: "__plugin_name__", description: "The default description I forgot to change",
});
pluginCommands.add(new Command({
definition: ["my-command <foo>", "My command description that I really should change", (yargs) => {
yargs.positional("foo", { describe: "foo is foo!", type: "string" });
}],
handler: async function(args, control) {
const response = await control.sendTo("controller", new PluginExampleRequest(args.foo, [1, 2, 3]));
console.log(response);
console.log(args);
},
}));
/* eslint-enable no-console */
class CtlPlugin extends BaseCtlPlugin {
async addCommands(rootCommand) {
rootCommand.add(pluginCommands);
}
}
module.exports = {
CtlPlugin,
};
@@ -0,0 +1,35 @@
"use strict";
const lib = require("@clusterio/lib");
const { BaseHostPlugin } = require("@clusterio/host");
const { PluginExampleEvent, PluginExampleRequest } = require("./messages");
class HostPlugin extends BaseHostPlugin {
async init() {
this.host.handle(PluginExampleEvent, this.handlePluginExampleEvent.bind(this));
this.host.handle(PluginExampleRequest, this.handlePluginExampleRequest.bind(this));
}
async onHostConfigFieldChanged(field, curr, prev) {
this.logger.info(`host::onInstanceConfigFieldChanged ${field}`);
}
async onShutdown() {
this.logger.info("host::onShutdown");
}
async handlePluginExampleEvent(event) {
this.logger.info(JSON.stringify(event));
}
async handlePluginExampleRequest(request) {
this.logger.info(JSON.stringify(request));
return {
myResponseString: request.myString,
myResponseNumbers: request.myNumberArray,
};
}
}
module.exports = {
HostPlugin,
};
@@ -0,0 +1,109 @@
"use strict";
const lib = require("@clusterio/lib");
const Messages = require("./messages");
lib.definePermission({
name: "__plugin_name__.example.permission.event",
title: "Example permission event",
description: "Example Description. Event. Change me in index.js",
});
lib.definePermission({
name: "__plugin_name__.example.permission.request",
title: "Example permission request",
description: "Example Description. Request. Change me in index.js",
});
//%if controller & web // Subscribing requires web content and the controller
lib.definePermission({
name: "__plugin_name__.example.permission.subscribe",
title: "Example permission subscribe",
description: "Example Description. Subscribe. Change me in index.js",
});
//%endif
//%if web
lib.definePermission({
name: "__plugin_name__.page.view",
title: "Example page view permission",
description: "Example Description. View. Change me in index.js",
});
//%endif
const plugin = {
name: "__plugin_name__",
title: "__plugin_name__",
description: "Example Description. Plugin. Change me in index.js",
//%if controller
controllerEntrypoint: "./controller",
controllerConfigFields: {
"__plugin_name__.myControllerField": {
title: "My Controller Field",
description: "This should be removed from index.js",
type: "string",
initialValue: "Remove Me",
},
},
//%endif
//%if host
hostEntrypoint: "./host",
hostConfigFields: {
"__plugin_name__.myHostField": {
title: "My Host Field",
description: "This should be removed from index.js",
type: "string",
initialValue: "Remove Me",
},
},
//%endif
//%if instance
instanceEntrypoint: "./instance",
instanceConfigFields: {
"__plugin_name__.myInstanceField": {
title: "My Instance Field",
description: "This should be removed from index.js",
type: "string",
initialValue: "Remove Me",
},
},
//%endif
//%if ctl
ctlEntrypoint: "./ctl",
controlConfigFields: {
"__plugin_name__.myControlField": {
title: "My Control Field",
description: "This should be removed from index.js",
type: "string",
initialValue: "Remove Me",
},
},
//%endif
messages: [
Messages.PluginExampleEvent,
Messages.PluginExampleRequest,
//%if controller & web // Subscribing requires web content and the controller
Messages.ExampleSubscribableUpdate,
//%endif
],
//%if web // Web content template has an example route which is the plugin name
webEntrypoint: "./web",
routes: [
"/__plugin_name__",
],
//%endif
//%if controller & !web // The controller always includes web entry even if there is no content
webEntrypoint: "./web",
routes: [],
//%endif
};
module.exports = {
plugin,
};
@@ -0,0 +1,55 @@
"use strict";
const lib = require("@clusterio/lib");
const { BaseInstancePlugin } = require("@clusterio/host");
const { PluginExampleEvent, PluginExampleRequest } = require("./messages");
class InstancePlugin extends BaseInstancePlugin {
async init() {
this.instance.handle(PluginExampleEvent, this.handlePluginExampleEvent.bind(this));
this.instance.handle(PluginExampleRequest, this.handlePluginExampleRequest.bind(this));
//%if module
this.instance.server.handle("__plugin_name__-plugin_example_ipc", this.handlePluginExampleIPC.bind(this));
//%endif
}
async onInstanceConfigFieldChanged(field, curr, prev) {
this.logger.info(`instance::onInstanceConfigFieldChanged ${field}`);
}
async onStart() {
this.logger.info("instance::onStart");
}
async onStop() {
this.logger.info("instance::onStop");
}
async onPlayerEvent(event) {
this.logger.info(`onPlayerEvent::onPlayerEvent ${JSON.stringify(event)}`);
}
async handlePluginExampleEvent(event) {
this.logger.info(JSON.stringify(event));
//%if module
this.sendRcon("/sc __plugin_name__.foo()");
//%endif
}
async handlePluginExampleRequest(request) {
this.logger.info(JSON.stringify(request));
return {
myResponseString: request.myString,
myResponseNumbers: request.myNumberArray,
};
}
//%if module
async handlePluginExampleIPC(event) {
this.logger.info(JSON.stringify(event));
}
//%endif
}
module.exports = {
InstancePlugin,
};
@@ -0,0 +1,101 @@
"use strict";
const { plainJson, jsonArray, JsonBoolean, JsonNumber, JsonString, StringEnum } = require("@clusterio/lib");
const { Type } = require("@sinclair/typebox");
class PluginExampleEvent {
static type = "event";
static src = ["host", "control"];
static dst = ["controller", "host", "instance"];
static plugin = "__plugin_name__";
static permission = "__plugin_name__.example.permission.event";
constructor(myString, myNumberArray) {
this.myString = myString;
this.myNumberArray = myNumberArray;
}
static jsonSchema = Type.Object({
"myString": Type.String(),
"myNumberArray": Type.Array(Type.Number()),
});
static fromJSON(json) {
return new PluginExampleEvent(json.myString, json.myNumberArray);
}
}
class PluginExampleRequest {
static type = "request";
static src = ["host", "control"];
static dst = ["controller", "host", "instance"];
static plugin = "__plugin_name__";
static permission = "__plugin_name__.example.permission.request";
constructor(myString, myNumberArray) {
this.myString = myString;
this.myNumberArray = myNumberArray;
}
static jsonSchema = Type.Object({
"myString": Type.String(),
"myNumberArray": Type.Array(Type.Number()),
});
static fromJSON(json) {
return new PluginExampleRequest(json.myString, json.myNumberArray);
}
static Response = plainJson(Type.Object({
"myResponseString": Type.String(),
"myResponseNumbers": Type.Array(Type.Number()),
}));
}
//%if controller & web // Subscribing requires web content and the controller
class ExampleSubscribableValue {
constructor(id, updatedAtMs, isDeleted) {
this.id = id;
this.updatedAtMs = updatedAtMs;
this.isDeleted = isDeleted;
}
static jsonSchema = Type.Object({
id: Type.String(),
updatedAtMs: Type.Number(),
isDeleted: Type.Boolean(),
});
static fromJSON(json) {
return new this(json.id, json.updatedAtMs, json.isDeleted);
}
}
class ExampleSubscribableUpdate {
static type = "event";
static src = "controller";
static dst = "control";
static plugin = "__plugin_name__";
static permission = "__plugin_name__.example.permission.subscribe";
constructor(updates) {
this.updates = updates;
}
static jsonSchema = Type.Object({
"updates": Type.Array(ExampleSubscribableValue.jsonSchema),
});
static fromJSON(json) {
return new this(json.updates.map(update => ExampleSubscribableValue.fromJSON(update)));
}
}
//%endif
module.exports = {
PluginExampleEvent,
PluginExampleRequest,
//%if controller & web // Subscribing requires web content and the controller
ExampleSubscribableValue,
ExampleSubscribableUpdate,
//%endif
};
@@ -0,0 +1,78 @@
import React, {
useContext, useEffect, useState,
//%if controller & web // Subscribing requires web content and the controller
useCallback, useSyncExternalStore,
//%endif
} from "react";
// import {
//
// } from "antd";
import {
BaseWebPlugin, PageLayout, Control, ControlContext, notifyErrorHandler,
} from "@clusterio/web_ui";
import {
PluginExampleEvent, PluginExampleRequest,
//%if controller & web // Subscribing requires web content and the controller
ExampleSubscribableUpdate,
//%endif
} from "../messages";
import * as lib from "@clusterio/lib";
function MyTemplatePage() {
let control = useContext(ControlContext);
//%if controller & web // Subscribing requires web content and the controller
const plugin = control.plugins.get("__plugin_name__");
const [subscribableData, synced] = plugin.useSubscribableData();
//%endif
return <PageLayout nav={[{ name: "__plugin_name__" }]}>
<h2>__plugin_name__</h2>
//%if controller & web // Subscribing requires web content and the controller
Synced: {String(synced)} Data: {JSON.stringify([...subscribableData.values()])}
//%endif
</PageLayout>;
}
export class WebPlugin extends BaseWebPlugin {
//%if controller & web // Subscribing requires web content and the controller
subscribableData = new lib.EventSubscriber(ExampleSubscribableUpdate, this.control);
//%endif
async init() {
this.pages = [
{
path: "/__plugin_name__",
sidebarName: "__plugin_name__",
permission: "__plugin_name__.page.view",
content: <MyTemplatePage/>,
},
];
this.control.handle(PluginExampleEvent, this.handlePluginExampleEvent.bind(this));
this.control.handle(PluginExampleRequest, this.handlePluginExampleRequest.bind(this));
}
//%if controller & web // Subscribing requires web content and the controller
useSubscribableData() {
const control = useContext(ControlContext);
const subscribe = useCallback((callback) => this.subscribableData.subscribe(callback), [control]);
return useSyncExternalStore(subscribe, () => this.subscribableData.getSnapshot());
}
//%endif
async handlePluginExampleEvent(event) {
this.logger.info(JSON.stringify(event));
}
async handlePluginExampleRequest(request) {
this.logger.info(JSON.stringify(request));
return {
myResponseString: request.myString,
myResponseNumbers: request.myNumberArray,
};
}
}
@@ -0,0 +1,68 @@
import * as lib from "@clusterio/lib";
import { BaseControllerPlugin, InstanceInfo } from "@clusterio/controller";
import {
PluginExampleEvent, PluginExampleRequest,
//%if controller & web // Subscribing requires web content and the controller
ExampleSubscribableUpdate, ExampleSubscribableValue,
//%endif
} from "./messages";
export class ControllerPlugin extends BaseControllerPlugin {
//%if controller & web // Subscribing requires web content and the controller
exampleDatabase!: Map<string, ExampleSubscribableValue>;
storageDirty = false;
//%endif
async init() {
this.controller.handle(PluginExampleEvent, this.handlePluginExampleEvent.bind(this));
this.controller.handle(PluginExampleRequest, this.handlePluginExampleRequest.bind(this));
//%if controller & web // Subscribing requires web content and the controller
this.controller.subscriptions.handle(ExampleSubscribableUpdate, this.handleExampleSubscription.bind(this));
this.exampleDatabase = new Map(); // If needed, replace with loading from database file
//%endif
}
async onControllerConfigFieldChanged(field: string, curr: unknown, prev: unknown) {
this.logger.info(`controller::onControllerConfigFieldChanged ${field}`);
}
//%if instance
async onInstanceConfigFieldChanged(instance: InstanceInfo, field: string, curr: unknown, prev: unknown) {
this.logger.info(`controller::onInstanceConfigFieldChanged ${instance.id} ${field}`);
}
//%endif
async onSaveData() {
this.logger.info("controller::onSaveData");
}
async onShutdown() {
this.logger.info("controller::onShutdown");
}
async onPlayerEvent(instance: InstanceInfo, event: lib.PlayerEvent) {
this.logger.info(`controller::onPlayerEvent ${instance.id} ${JSON.stringify(event)}`);
}
async handlePluginExampleEvent(event: PluginExampleEvent) {
this.logger.info(JSON.stringify(event));
}
async handlePluginExampleRequest(request: PluginExampleRequest) {
this.logger.info(JSON.stringify(request));
return {
myResponseString: request.myString,
myResponseNumbers: request.myNumberArray,
};
}
//%if controller & web // Subscribing requires web content and the controller
async handleExampleSubscription(request: lib.SubscriptionRequest) {
const values = [...this.exampleDatabase.values()].filter(
value => value.updatedAtMs > request.lastRequestTimeMs,
);
return values.length ? new ExampleSubscribableUpdate(values) : null;
}
//%endif
}
@@ -0,0 +1,27 @@
type Control = any; // import type { Control } from "@clusterio/ctl";
import { BaseCtlPlugin } from "@clusterio/ctl";
import { CommandTree, Command } from "@clusterio/lib";
import { PluginExampleEvent, PluginExampleRequest } from "./messages";
/* eslint-disable no-console */
const pluginCommands = new CommandTree({
name: "__plugin_name__", description: "The default description I forgot to change",
});
pluginCommands.add(new Command({
definition: ["my-command <foo>", "My command description that I really should change", (yargs) => {
yargs.positional("foo", { describe: "foo is foo!", type: "string" });
}],
handler: async function(args: { foo: string }, control: Control) {
const response = await control.sendTo("controller", new PluginExampleRequest(args.foo, [1, 2, 3]));
console.log(response);
console.log(args);
},
}));
/* eslint-enable no-console */
export class CtlPlugin extends BaseCtlPlugin {
async addCommands(rootCommand: CommandTree) {
rootCommand.add(pluginCommands);
}
}
@@ -0,0 +1,30 @@
import * as lib from "@clusterio/lib";
import { BaseHostPlugin } from "@clusterio/host";
import { PluginExampleEvent, PluginExampleRequest } from "./messages";
export class HostPlugin extends BaseHostPlugin {
async init() {
this.host.handle(PluginExampleEvent, this.handlePluginExampleEvent.bind(this));
this.host.handle(PluginExampleRequest, this.handlePluginExampleRequest.bind(this));
}
async onHostConfigFieldChanged(field: string, curr: unknown, prev: unknown) {
this.logger.info(`host::onInstanceConfigFieldChanged ${field}`);
}
async onShutdown() {
this.logger.info("host::onShutdown");
}
async handlePluginExampleEvent(event: PluginExampleEvent) {
this.logger.info(JSON.stringify(event));
}
async handlePluginExampleRequest(request: PluginExampleRequest) {
this.logger.info(JSON.stringify(request));
return {
myResponseString: request.myString,
myResponseNumbers: request.myNumberArray,
};
}
}
@@ -0,0 +1,127 @@
import * as lib from "@clusterio/lib";
import * as Messages from "./messages";
lib.definePermission({
name: "__plugin_name__.example.permission.event",
title: "Example permission event",
description: "Example Description. Event. Change me in index.ts",
});
lib.definePermission({
name: "__plugin_name__.example.permission.request",
title: "Example permission request",
description: "Example Description. Request. Change me in index.ts",
});
//%if controller & web // Subscribing requires web content and the controller
lib.definePermission({
name: "__plugin_name__.example.permission.subscribe",
title: "Example permission subscribe",
description: "Example Description. Subscribe. Change me in index.ts",
});
//%endif
//%if web
lib.definePermission({
name: "__plugin_name__.page.view",
title: "Example page view permission",
description: "Example Description. View. Change me in index.ts",
});
//%endif
declare module "@clusterio/lib" {
//%if controller
export interface ControllerConfigFields {
"__plugin_name__.myControllerField": string;
}
//%endif
//%if host
export interface HostConfigFields {
"__plugin_name__.myHostField": string;
}
//%endif
//%if instance
export interface InstanceConfigFields {
"__plugin_name__.myInstanceField": string;
}
//%endif
//%if ctl
export interface ControlConfigFields {
"__plugin_name__.myControlField": string;
}
//%endif
}
export const plugin: lib.PluginDeclaration = {
name: "__plugin_name__",
title: "__plugin_name__",
description: "Example Description. Plugin. Change me in index.ts",
//%if controller
controllerEntrypoint: "./dist/node/controller",
controllerConfigFields: {
"__plugin_name__.myControllerField": {
title: "My Controller Field",
description: "This should be removed from index.js",
type: "string",
initialValue: "Remove Me",
},
},
//%endif
//%if host
hostEntrypoint: "./dist/node/host",
hostConfigFields: {
"__plugin_name__.myHostField": {
title: "My Host Field",
description: "This should be removed from index.js",
type: "string",
initialValue: "Remove Me",
},
},
//%endif
//%if instance
instanceEntrypoint: "./dist/node/instance",
instanceConfigFields: {
"__plugin_name__.myInstanceField": {
title: "My Instance Field",
description: "This should be removed from index.js",
type: "string",
initialValue: "Remove Me",
},
},
//%endif
//%if ctl
ctlEntrypoint: "./dist/node/ctl",
controlConfigFields: {
"__plugin_name__.myControlField": {
title: "My Control Field",
description: "This should be removed from index.js",
type: "string",
initialValue: "Remove Me",
},
},
//%endif
messages: [
Messages.PluginExampleEvent,
Messages.PluginExampleRequest,
//%if controller & web // Subscribing requires web content and the controller
Messages.ExampleSubscribableUpdate,
//%endif
],
//%if web // Web content template has an example route which is the plugin name
webEntrypoint: "./web",
routes: [
"/__plugin_name__",
],
//%endif
//%if controller & !web // The controller always includes web entry even if there is no content
webEntrypoint: "./web",
routes: [],
//%endif
};
@@ -0,0 +1,57 @@
import * as lib from "@clusterio/lib";
import { BaseInstancePlugin } from "@clusterio/host";
import { PluginExampleEvent, PluginExampleRequest } from "./messages";
//%if module
type PuginExampleIPC = {
tick: number,
player_name: string,
};
//%endif
export class InstancePlugin extends BaseInstancePlugin {
async init() {
this.instance.handle(PluginExampleEvent, this.handlePluginExampleEvent.bind(this));
this.instance.handle(PluginExampleRequest, this.handlePluginExampleRequest.bind(this));
//%if module
this.instance.server.handle("__plugin_name__-plugin_example_ipc", this.handlePluginExampleIPC.bind(this));
//%endif
}
async onInstanceConfigFieldChanged(field: string, curr: unknown, prev: unknown) {
this.logger.info(`instance::onInstanceConfigFieldChanged ${field}`);
}
async onStart() {
this.logger.info("instance::onStart");
}
async onStop() {
this.logger.info("instance::onStop");
}
async onPlayerEvent(event: lib.PlayerEvent) {
this.logger.info(`onPlayerEvent::onPlayerEvent ${JSON.stringify(event)}`);
//%if module
this.sendRcon("/sc __plugin_name__.foo()");
//%endif
}
async handlePluginExampleEvent(event: PluginExampleEvent) {
this.logger.info(JSON.stringify(event));
}
async handlePluginExampleRequest(request: PluginExampleRequest) {
this.logger.info(JSON.stringify(request));
return {
myResponseString: request.myString,
myResponseNumbers: request.myNumberArray,
};
}
//%if module
async handlePluginExampleIPC(event: PuginExampleIPC) {
this.logger.info(JSON.stringify(event));
}
//%endif
}
@@ -0,0 +1,97 @@
import { plainJson, jsonArray, JsonBoolean, JsonNumber, JsonString, StringEnum } from "@clusterio/lib";
import { Type, Static } from "@sinclair/typebox";
export class PluginExampleEvent {
declare ["constructor"]: typeof PluginExampleEvent;
static type = "event" as const;
static src = ["host", "control"] as const;
static dst = ["controller", "host", "instance"] as const;
static plugin = "__plugin_name__" as const;
static permission = "__plugin_name__.example.permission.event";
constructor(
public myString: string,
public myNumberArray: number[],
) {
}
static jsonSchema = Type.Object({
"myString": Type.String(),
"myNumberArray": Type.Array(Type.Number()),
});
static fromJSON(json: Static<typeof PluginExampleEvent.jsonSchema>) {
return new PluginExampleEvent(json.myString, json.myNumberArray);
}
}
export class PluginExampleRequest {
declare ["constructor"]: typeof PluginExampleRequest;
static type = "request" as const;
static src = ["host", "control"] as const;
static dst = ["controller", "host", "instance"] as const;
static plugin = "__plugin_name__" as const;
static permission = "__plugin_name__.example.permission.request";
constructor(
public myString: string,
public myNumberArray: number[],
) {
}
static jsonSchema = Type.Object({
"myString": Type.String(),
"myNumberArray": Type.Array(Type.Number()),
});
static fromJSON(json: Static<typeof PluginExampleRequest.jsonSchema>) {
return new PluginExampleRequest(json.myString, json.myNumberArray);
}
static Response = plainJson(Type.Object({
"myResponseString": Type.String(),
"myResponseNumbers": Type.Array(Type.Number()),
}));
}
//%if controller & web // Subscribing requires web content and the controller
export class ExampleSubscribableValue {
constructor(
public id: string,
public updatedAtMs: number,
public isDeleted: boolean,
) {
}
static jsonSchema = Type.Object({
id: Type.String(),
updatedAtMs: Type.Number(),
isDeleted: Type.Boolean(),
});
static fromJSON(json: Static<typeof this.jsonSchema>) {
return new this(json.id, json.updatedAtMs, json.isDeleted);
}
}
export class ExampleSubscribableUpdate {
declare ["constructor"]: typeof ExampleSubscribableUpdate;
static type = "event" as const;
static src = "controller" as const;
static dst = "control" as const;
static plugin = "__plugin_name__" as const;
static permission = "__plugin_name__.example.permission.subscribe";
constructor(
public updates: ExampleSubscribableValue[],
) { }
static jsonSchema = Type.Object({
"updates": Type.Array(ExampleSubscribableValue.jsonSchema),
});
static fromJSON(json: Static<typeof this.jsonSchema>) {
return new this(json.updates.map(update => ExampleSubscribableValue.fromJSON(update)));
}
}
//%endif
@@ -0,0 +1,11 @@
{
"extends": "../../tsconfig.browser.json",
"references": [
{ "path": "../../packages/lib/tsconfig.browser.json" },
{ "path": "../../packages/web_ui/tsconfig.browser.json" },
],
"include": [ "web/**/*.tsx", "web/**/*.ts", "messages.ts", "package.json" ],
"compilerOptions": {
"outDir": "dist/browser",
}
}
@@ -0,0 +1,9 @@
{
"files": [],
"references": [
//%if web
{ "path": "./tsconfig.browser.json" },
//%endif
{ "path": "./tsconfig.node.json" }
]
}
@@ -0,0 +1,11 @@
{
"extends": "../../tsconfig.node.json",
"references": [
{ "path": "../../packages/lib/tsconfig.node.json" },
],
"include": ["./**/*.ts"],
"exclude": ["test/*", "./dist/*"],
"compilerOptions": {
"outDir": "dist/node",
},
}
@@ -0,0 +1,85 @@
import React, {
useContext, useEffect, useState,
//%if controller & web // Subscribing requires web content and the controller
useCallback, useSyncExternalStore,
//%endif
} from "react";
// import {
//
// } from "antd";
import {
BaseWebPlugin, PageLayout, Control, ControlContext, notifyErrorHandler,
} from "@clusterio/web_ui";
import {
PluginExampleEvent, PluginExampleRequest,
//%if controller & web // Subscribing requires web content and the controller
ExampleSubscribableUpdate,
//%endif
} from "../messages";
import * as lib from "@clusterio/lib";
function MyTemplatePage() {
const control = useContext(ControlContext);
//%if controller & web // Subscribing requires web content and the controller
const plugin = control.plugins.get("__plugin_name__") as WebPlugin;
const [subscribableData, synced] = plugin.useSubscribableData();
//%endif
return <PageLayout nav={[{ name: "__plugin_name__" }]}>
<h2>__plugin_name__</h2>
//%if controller & web // Subscribing requires web content and the controller
Synced: {String(synced)} Data: {JSON.stringify([...subscribableData.values()])}
//%endif
</PageLayout>;
}
export class WebPlugin extends BaseWebPlugin {
//%if controller & web // Subscribing requires web content and the controller
subscribableData = new lib.EventSubscriber(ExampleSubscribableUpdate, this.control);
//%endif
async init() {
this.pages = [
{
path: "/__plugin_name__",
sidebarName: "__plugin_name__",
// This permission is client side only, so it must match the permission string of a resource request to be secure
// An undefined value means that the page will always be visible
//%if controller & web // Subscribing requires web content and the controller
permission: "__plugin_name__.example.permission.subscribe",
//%endif
//%if !controller | !web
permission: undefined, // "__plugin_name__.example.permission.request",
//%endif
content: <MyTemplatePage/>,
},
];
this.control.handle(PluginExampleEvent, this.handlePluginExampleEvent.bind(this));
this.control.handle(PluginExampleRequest, this.handlePluginExampleRequest.bind(this));
}
//%if controller & web // Subscribing requires web content and the controller
useSubscribableData() {
const control = useContext(ControlContext);
const subscribe = useCallback((callback: () => void) => this.subscribableData.subscribe(callback), [control]);
return useSyncExternalStore(subscribe, () => this.subscribableData.getSnapshot());
}
//%endif
async handlePluginExampleEvent(event: PluginExampleEvent) {
this.logger.info(JSON.stringify(event));
}
async handlePluginExampleRequest(request: PluginExampleRequest) {
this.logger.info(JSON.stringify(request));
return {
myResponseString: request.myString,
myResponseNumbers: request.myNumberArray,
};
}
}
+114
View File
@@ -0,0 +1,114 @@
# Templates
Create has the ability to create new plugins from templates to avoid needing to manually copy lots of boiler plate. This folder contains all those templates for both Javascript and Typescript plugins. These templates have custom preprocessor instructions that can be used to alter a template based on developer input, this syntax is described below.
- [The Basics](#the-basics)
- [Insert Instruction](#insert-instruction) (`__variable__`)
- [Conditional Instruction](#conditional-instruction) (`//%if` `//%endif`)
- [Available Variables](#available-variables)
## The Basics
All files can contain preprocessor instructions whether json, js, lua, or anything else. All instructions use the same syntax although it has been designed to work well with js and ts files.
Except for the insert instruction, all preprocessors use a `//%` prefix and must be on their own line. These lines are never included in the output. Parsing is stopped when `//` is encountered to allow for comments on preprocessors. Eg. `//%if foo // this is a comment`.
Empty instructions, aka just comments, can be inserted into templates using `//%//` and are not included in the output file.
`templates/plugin-js` contains all the files which will be used by a javascript only plugin.
`templates/plugin-ts` contains all the files which will be used by a plugin that supports typescript.
`templates/common` contains files used by both types of plugins, this is mostly lua code or root files like `package.json`
Files are not copied via wildcards and instead are explicitly listed within `template.js` so any new templates will need to be added there.
## Insert Instruction
There are a number of template variables available to be inserted into templates. All variables are raw string values, that is they are inserted directly into the output file without any modifications. To signify a place to insert a variable you need only surround its identifier with double underscores, examples below.
Example Variables:
```sh
my_number=1
my_string=foo
my_property=bar: "baz"
```
Example Template:
```ts
const __my_string__: number = __my_number__;
console.log(__my_string__);
console.log({
foo: "Hello, __my_string__!",
__my_property__,
my_property: __my_string__,
});
```
Example Output:
```ts
const foo: number = 1;
console.log(foo);
console.log({
foo: "Hello, foo!",
bar: "baz",
my_property: foo,
});
```
## Conditional Instruction
Conditionals allow for code to be included/omitted based on the value of template variables. Currently only boolean logic in [disjunctive normal form](https://en.wikipedia.org/wiki/Disjunctive_normal_form) is supported and any variables that are not booleans are evaluated with the [Boolean constructor](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Boolean).
Variables used in condition statements do not require double underscores. Their values can be inverted using `!`. Arithmetic and comparison operations are not supported. Parentheses are also not supported, conditions must be in DNF.
Conversion to DNF:
`A & (B | !C)` -> `A & B | A & !C` evaluated as `(A and B) or (A and not C)`
Example Variables:
```sh
my_string=foo
say_hello=true
say_done=true
verbose=false
```
Example Template:
```ts
//%if verbose
console.log("Starting...");
//%endif
//%if say_hello & my_string
console.log("Hello, __my_string__!");
//%endif
//%if say_done | verbose
console.log("Done.");
//%endif
```
Example Output:
```ts
console.log("Hello, foo!");
console.log("Done.");
```
## Available Variables
| Identifier | Example | Description |
| - | - | - |
| typescript | `true` / `false` | `true` if typescript templates are included |
| controller | `true` / `false` | `true` if controller templates are included |
| host | `true` / `false` | `true` if host templates are included |
| instance | `true` / `false` | `true` if instance templates are included |
| module | `true` / `false` | `true` if module templates are included |
| ctl | `true` / `false` | `true` if control templates are included |
| web | `true` / `false` | `true` if web templates are included |
| plugin_name | `my_plugin` | name of the plugin |
| prepare | `tsc --build` | commands to run for the prepare script |
| ext | `ts` / `js` | file extension of source files |
+3 -3
View File
@@ -37,7 +37,7 @@ For example, installing the Subspace Storage plugin:
npm install @clusterio/plugin-subspace_storage
npx clusterioctl plugin add @clusterio/plugin-subspace_storage
Since the `plugin-list.json` is shared between master, slave and ctl you usually only need to do this once per machine.
Since the `plugin-list.json` is shared between controller, host and ctl you usually only need to do this once per machine.
#### `plugin remove <name>`
@@ -51,7 +51,7 @@ For example, uninstalling the Subspace Storage plugin:
npx clusterioctl plugin remove subspace_storage
npm uninstall @clusterio/plugin-subspace_storage
Since the `plugin-list.json` is shared between master, slave and ctl you usually only need to do this once per machine.
Since the `plugin-list.json` is shared between controller, host and ctl you usually only need to do this once per machine.
#### `plugin list`
@@ -62,7 +62,7 @@ Lists the plugins set up to be available by name followed by path.
### `config-control`
Manage the ctl configuration.
This allows setting the url and token to the master server to manage.
This allows setting the url and token to the controller to manage.
#### `config-control set <config-entry> [value]`
-1213
View File
File diff suppressed because it is too large. Load diff
+357
View File
@@ -0,0 +1,357 @@
#!/usr/bin/env node
/**
* Command line interface for controlling a Clusterio cluster
* @module ctl/ctl
*/
import fs from "fs-extra";
import yargs from "yargs";
import path from "path";
import { version } from "./package.json";
import setBlocking from "set-blocking";
import { strict as assert } from "assert";
// Reduce startup time by lazy compiling schemas.
(global as any).lazySchemaCompilation = true;
import * as lib from "@clusterio/lib";
import { ConsoleTransport, levels, logger } from "@clusterio/lib";
import * as commands from "./src/commands";
import BaseCtlPlugin from "./src/BaseCtlPlugin";
/**
* Connector for control connection to controller
* @private
*/
class ControlConnector extends lib.WebSocketClientConnector {
private _token: string;
constructor(url: string, maxReconnectDelay: number, tlsCa: string | undefined, token: string) {
super(url, maxReconnectDelay, tlsCa);
this._token = token;
}
register() {
logger.verbose("Connector | registering control");
this.sendHandshake(
new lib.MessageRegisterControl(
new lib.RegisterControlData(
this._token,
version,
)
)
);
}
}
/**
* Handles running the control
*
* Connects to the controller over WebSocket and sends commands to it.
* @static
*/
export class Control extends lib.Link {
/** Control config used for connecting to the controller. */
config: lib.ControlConfig;
/** Certificate authority used to validate TLS connections to the controller. */
tlsCa?: string;
/** Mapping of plugin names to their instance for loaded plugins. */
plugins: Map<string, BaseCtlPlugin>;
/** Keep the control connection alive after the command completes. */
keepOpen = false;
constructor(
connector: ControlConnector,
controlConfig: lib.ControlConfig,
tlsCa: string | undefined,
ctlPlugins: Map<string, BaseCtlPlugin>
) {
super(connector);
this.config = controlConfig;
this.tlsCa = tlsCa;
this.plugins = ctlPlugins;
this.handle(lib.LogMessageEvent, this.handleLogMessageEvent.bind(this));
this.handle(lib.DebugWsMessageEvent, this.handleDebugWsMessageEvent.bind(this));
}
async setLogSubscriptions({
all = false,
controller = false,
hostIds = [] as number[],
instanceIds = [] as number[],
maxLevel = undefined as keyof typeof levels | undefined,
}) {
await this.send(
new lib.LogSetSubscriptionsRequest(
all, controller, hostIds, instanceIds, maxLevel,
)
);
}
async handleLogMessageEvent(event: lib.LogMessageEvent) {
logger.log(event.info as any);
}
async handleDebugWsMessageEvent(event: lib.DebugWsMessageEvent) {
// eslint-disable-next-line no-console
console.log("WS", event.direction, event.content);
}
async shutdown() {
try {
await (this.connector as ControlConnector).disconnect();
} catch (err) {
if (!(err instanceof lib.SessionLost)) {
throw err;
}
}
}
}
async function loadPlugins(pluginList: Map<string, string>) {
let pluginInfos = await lib.loadPluginInfos(pluginList);
lib.registerPluginMessages(pluginInfos);
lib.addPluginConfigFields(pluginInfos);
let ctlPlugins = new Map<string, BaseCtlPlugin>();
for (let pluginInfo of pluginInfos) {
if (!pluginInfo.ctlEntrypoint) {
continue;
}
let CtlPlugin = await lib.loadPluginClass(
pluginInfo.name,
path.posix.join(pluginInfo.requirePath, pluginInfo.ctlEntrypoint),
"CtlPlugin",
BaseCtlPlugin,
);
let ctlPlugin = new CtlPlugin(pluginInfo, logger);
ctlPlugins.set(pluginInfo.name, ctlPlugin);
await ctlPlugin.init();
}
return ctlPlugins;
}
interface CtlArguments {
[index: string]: unknown;
$0: string,
_: (string | number)[],
logLevel: keyof typeof levels,
config: string,
pluginList: string,
}
async function startControl() {
yargs
.scriptName("clusterioctl")
.usage("$0 <command> [options]")
.option("log-level", {
nargs: 1,
describe: "Log level to print to stderr",
default: "server",
choices: Object.keys(levels),
type: "string",
})
.option("config", {
nargs: 1,
describe: "config file to get credentails from",
default: "config-control.json",
defaultDescription: "auto",
type: "string",
})
.option("plugin-list", {
nargs: 1,
describe: "File containing list of plugins available with their install path",
default: "plugin-list.json",
type: "string",
})
.command("plugin", "Manage available plugins", lib.pluginCommand)
.command("control-config", "Manage Control config", lib.configCommand)
.wrap(yargs.terminalWidth())
.help(false) // Disable help to avoid triggering it on the first parse.
;
// Parse the args first to get the configured plugin list.
// eslint-disable-next-line node/no-sync
let args = yargs.parseSync() as CtlArguments;
// Log stream for the ctl session.
logger.add(
new ConsoleTransport({
errorLevels: Object.keys(levels),
level: args.logLevel,
format: new lib.TerminalFormat(),
})
);
lib.handleUnhandledErrors();
logger.verbose(`Loading available plugins from ${args.pluginList}`);
let pluginList = new Map();
try {
pluginList = new Map(JSON.parse(await fs.readFile(args.pluginList, "utf8")));
} catch (err: any) {
if (err.code !== "ENOENT") {
throw err;
}
}
// If the command is plugin management we don't try to load plugins
if (args._[0] === "plugin") {
await lib.handlePluginCommand(args, pluginList, args.pluginList);
return;
}
logger.verbose("Loading Plugins");
let ctlPlugins = await loadPlugins(pluginList);
// Add all cluster management commands including ones from plugins
let rootCommands = await commands.registerCommands(ctlPlugins, yargs);
// Reparse after commands have been added with help and strict checking.
args = yargs
.help()
.strict()
.parse() as CtlArguments
;
logger.verbose(`Loading config from ${args.config}`);
let controlConfig;
try {
const jsonConfig = JSON.parse(await fs.readFile(args.config, "utf8"));
controlConfig = lib.ControlConfig.fromJSON(jsonConfig, "control");
} catch (err: any) {
if (err.code === "ENOENT") {
logger.verbose("Config not found, initializing new config");
controlConfig = new lib.ControlConfig("control");
} else {
throw new lib.StartupError(`Failed to load ${args.config}: ${err.message}`);
}
}
if (args._.length === 0) {
yargs.showHelp();
yargs.exit(1, undefined as unknown as Error); // Type definition file is wrong.
}
// Handle the control-config command before trying to connect.
if (args._[0] === "control-config") {
await lib.handleConfigCommand(args, controlConfig, args.config);
return;
}
// Determine which command is being executed.
let commandPath = [...args._] as string[];
let targetCommand: lib.CommandTree | lib.Command = rootCommands;
while (commandPath.length && targetCommand instanceof lib.CommandTree) {
targetCommand = targetCommand.get(commandPath.shift()!)!;
}
assert(targetCommand instanceof lib.Command);
// The remaining commands require connecting to the controller.
if (!controlConfig.get("control.controller_url") || !controlConfig.get("control.controller_token")) {
logger.error("Missing URL and/or token to connect with. See README.md for setting up access.");
process.exitCode = 1;
return;
}
let tlsCa: string | undefined;
let tlsCaPath = controlConfig.get("control.tls_ca");
if (tlsCaPath) {
tlsCa = await fs.readFile(tlsCaPath, "utf8");
}
let controlConnector = new ControlConnector(
controlConfig.get("control.controller_url")!,
controlConfig.get("control.max_reconnect_delay"),
tlsCa,
controlConfig.get("control.controller_token")!,
);
let control = new Control(controlConnector, controlConfig, tlsCa, ctlPlugins);
try {
await controlConnector.connect();
} catch (err) {
if (err instanceof lib.AuthenticationFailed) {
throw new lib.StartupError(err.message);
}
throw err;
}
process.on("SIGINT", () => {
logger.info("Caught interrupt signal, closing connection");
control.shutdown().catch(err => {
setBlocking(true);
logger.error(err.stack);
// eslint-disable-next-line node/no-process-exit
process.exit(1);
});
});
try {
await targetCommand.run(args, control);
} catch (err) {
control.keepOpen = false;
if (err instanceof lib.CommandError) {
logger.error(`Error running command: ${err.message}`);
process.exitCode = 1;
} else if (err instanceof lib.RequestError) {
if (err.stack) {
logger.error(`Error sending request:\n${err.stack}`);
} else {
logger.error(`Error sending request: ${err.message}`);
}
process.exitCode = 1;
} else {
throw err;
}
} finally {
if (!control.keepOpen) {
await control.shutdown();
}
}
}
export function bootstrap() {
// eslint-disable-next-line no-console
console.warn(`
+==========================================================+
I WARNING: This is the development branch for the 2.0 I
I version of clusterio. Expect things to break. I
+==========================================================+
`
);
startControl().catch(err => {
if (err.errors) {
logger.fatal(JSON.stringify(err.errors, null, "\t"));
}
if (!(err instanceof lib.StartupError)) {
logger.fatal(`
+------------------------------------------------------------+
| Unexpected error occured while starting control, please |
| report it to https://github.com/clusterio/clusterio/issues |
+------------------------------------------------------------+
${err.stack}`
);
} else {
logger.error(`
+---------------------------------+
| Unable to to start clusterioctl |
+---------------------------------+
${err.stack}`
);
}
process.exitCode = 1;
});
}
if (module === require.main) {
bootstrap();
}
+7
View File
@@ -0,0 +1,7 @@
import { bootstrap } from "./ctl";
export * from "./ctl";
export { default as BaseCtlPlugin } from "./src/BaseCtlPlugin";
if (module === require.main) {
bootstrap();
}
+38 -30
View File
@@ -1,32 +1,40 @@
{
"name": "@clusterio/ctl",
"description": "Clusterio command line management tool",
"author": "Hornwitser",
"license": "MIT",
"version": "2.0.0-alpha.10",
"main": "ctl.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"bin": {
"clusterioctl": "./ctl.js"
},
"keywords": [
"factorio"
],
"engines": {
"node": ">=12"
},
"dependencies": {
"@clusterio/lib": "^2.0.0-alpha.10",
"as-table": "^1.0.31",
"fs-extra": "^8.1.0",
"phin": "^3.5.1",
"set-blocking": "^2.0.0",
"winston": "^3.3.3",
"yargs": "^14.2.0"
},
"publishConfig": {
"access": "public"
}
"name": "@clusterio/ctl",
"description": "Clusterio command line management tool",
"author": "Hornwitser",
"license": "MIT",
"version": "2.0.0-alpha.18",
"main": "dist/node/index.js",
"scripts": {
"prepare": "tsc --build",
"test": "echo \"Error: no test specified\" && exit 1"
},
"bin": {
"clusterioctl": "./dist/node/ctl.js"
},
"keywords": [
"factorio"
],
"engines": {
"node": ">=18"
},
"dependencies": {
"@clusterio/lib": "^2.0.0-alpha.18",
"as-table": "^1.0.55",
"fs-extra": "^11.1.1",
"phin": "^3.7.0",
"set-blocking": "^2.0.0",
"winston": "^3.10.0",
"yargs": "^17.7.2"
},
"publishConfig": {
"access": "public"
},
"devDependencies": {
"@types/fs-extra": "^11.0.1",
"@types/node": "^20.4.5",
"@types/set-blocking": "^2.0.0",
"@types/yargs": "^17.0.24",
"typescript": "^5.1.6"
}
}
+47
View File
@@ -0,0 +1,47 @@
import type { CommandTree, Logger, PluginNodeEnvInfo } from "@clusterio/lib";
/**
* Base class for clusterioctl plugins
*
* Ctl plugins are subclasses of this class which get instantiated by
* clusterioctl in order to extend its functionallity. To be discovered the
* class must be exported under the name `CtlPlugin` in the module
* specified by the `ctlEntrypoint` in the plugin's `plugin` export.
*/
export default class BaseCtlPlugin {
/**
* Logger for this plugin
*
* Instance of winston Logger for sending log messages from this
* plugin. Supported methods and their corresponding log levels are
* `error`, `warn`, `audit`, `info` and `verbose`.
*/
logger: Logger;
constructor(
/**
* The plugin's own info module
*/
public info: PluginNodeEnvInfo,
logger: Logger,
) {
this.logger = logger.child({ plugin: this.info.name }) as unknown as Logger;
}
/**
* Called immediately after the class is instantiated
*/
async init() { }
/**
* Called to add commands to the command line interface.
*
* Invoked by clusterioctl to let plugins add commands. `rootCommand` is
* the top level command node which the plugin should add its own {@link
* CommandTree} to.
*
* @param rootCommand -
* Root of the clusterioctl command tree.
*/
async addCommands(rootCommand: CommandTree) { }
}
File diff suppressed because it is too large. Load diff
+6
View File
@@ -0,0 +1,6 @@
{
"files": [],
"references": [
{ "path": "./tsconfig.node.json" },
],
}
+10
View File
@@ -0,0 +1,10 @@
{
"extends": "../../tsconfig.node.json",
"references": [
{ "path": "../lib/tsconfig.node.json" },
],
"include": ["index.ts", "package.json", "ctl.ts", "src/**/*.ts"],
"compilerOptions": {
"outDir": "dist/node",
},
}
Loaded 100 of 508 files, more files were not shown because too many files have changed in this diff. Show more