Merge remote-tracking branch 'upstream/master' into feature/wait-for-plugin

This commit is contained in:
bbassie committed 2026-11-20 21:13:53 +00:00
commit 7f688d2ed0
77 files changed
+2006 -693

No files matched your search

+38 -2
View File
@@ -31,11 +31,47 @@ Many thanks to the following for contributing to this release:
[@username](https://github.com/username)
-->
## Unreleased
## Version 2.1.1
### Features
- Hosts on ARM64 Linux download and run the ARM64 headless build of Factorio, available from Factorio 2.1.18. [#1228](<https://github.com/clusterio/clusterio/pull/1228>)
- Added `requiredMods` to plugin declarations. Instances refuse to start when their mod pack lacks a mod a loaded plugin requires, and the controller downloads missing required mods and their dependencies from the mod portal on start. Subspace Storage requires the subspace_storage mod. [#535](<https://github.com/clusterio/clusterio/issues/535>)
- Added `--output json` and `--output wire` to clusterioctl for machine readable results and errors. [#1230](<https://github.com/clusterio/clusterio/pull/1230>)
### Fixes
- Fixed cramped plugin rows and wrapping header branding on mobile screens.
- Fixed a newly paired host reporting that it needs a restart. [#1204](<https://github.com/clusterio/clusterio/issues/1204>)
- Fixed the installer's run-controller and run-host scripts exiting with 0 when the controller or host exited with 8 to stop the restart loop. [#1208](<https://github.com/clusterio/clusterio/issues/1208>)
- Fixed clusterioctl user list, user show and role list printing [object Set] for roles, instances and permissions. [#1205](<https://github.com/clusterio/clusterio/issues/1205>)
- Fixed typo in the delete instance confirmation. [#1209](<https://github.com/clusterio/clusterio/issues/1209>)
- The Search Mod Portal dialog now opens on the default mod pack's Factorio version instead of always 2.0. [#1209](<https://github.com/clusterio/clusterio/issues/1209>)
- Fixed the mod portal cache key logging "undefined" when hide_deprecated was not sent. [#1209](<https://github.com/clusterio/clusterio/issues/1209>)
- Fixed player_sync losing quick bar slots of blueprint library records when a player joins an instance for the first time. [#1219](<https://github.com/clusterio/clusterio/issues/1219>)
- Fixed cramped plugin rows and wrapping header branding on mobile screens. [#1223](<https://github.com/clusterio/clusterio/pull/1223>)
- Fixed player_sync keeping an instance's old copy of a player after the controller refused its upload, which let an instance restarted after a crash overwrite newer player data. [#1218](<https://github.com/clusterio/clusterio/issues/1218>)
### Changes
- Restarting the controller or a host from the web interface now asks for confirmation, and for hosts lists the running instances that will be stopped. [#1207](<https://github.com/clusterio/clusterio/issues/1207>)
- New clusters default controller.default_mod_pack_id to the newest Base Game mod pack, so instances can create saves and start without setting it first. [#1203](<https://github.com/clusterio/clusterio/issues/1203>)
- Output from npm during `plugin install` now goes through the logger. [#1224](<https://github.com/clusterio/clusterio/pull/1224>)
- The controller collects system info, including restart required, right after it starts and when a host connects instead of waiting for the first system metrics interval. [#1227](<https://github.com/clusterio/clusterio/pull/1227>)
- clusterioctl prints all tables with the same formatter and prints command results to stdout instead of the log. [#1230](<https://github.com/clusterio/clusterio/pull/1230>)
- clusterioctl prints "Command ran successfully" for commands that have no other output. [#1230](<https://github.com/clusterio/clusterio/pull/1230>)
### Meta
- Added a CI check that pull request descriptions have a valid changelog section. [#1226](<https://github.com/clusterio/clusterio/pull/1226>)
### Contributors
Many thanks to the following for contributing to this release:
[@bbassie](<https://github.com/bbassie>), [@ILLISIS](<https://github.com/ILLISIS>), [@Danielv123](<https://github.com/Danielv123>)
### Statistics
Commits: 44 Pull Requests: 19 Files Changed: 81 Additions: +2,676 Deletions: -739
## Version 2.1.0
+18
View File
@@ -8,6 +8,24 @@ Before `clusterioctl` can be used it needs to be configured for the cluster it w
The easiest way to do this is to run `npx clusteriocontroller bootstrap create-ctl-config <username>` on the controller, which creates the necessary `config-ctl.json` for managing the cluster as the given user.
## Output Format
ctl> --output json <command>
Print the result of the command as JSON on stdout instead of text meant for reading.
Lists are printed as an array of objects, other results as an object, and commands that have no result print `{"success": true}`.
If the command fails it prints `{"error": "<ClassName>", "message": "<message>"}` and exits with a non-zero exit code.
Log messages are written to stderr in all formats.
`--output wire` prints the same results in the format they are sent between the controller, hosts and clusterioctl, which uses snake_case names like `is_admin`.
The `json` format uses the property names shown in the tables of the human format.
`log follow` and `debug dump-ws` keep running and print one JSON object per line as entries arrive.
`instance start-all` and `instance stop-all` print an array with an entry per instance, which has an `error` property if that instance failed to start or stop.
The default is `--output human`, which prints `Command ran successfully` for commands that have no result.
## Hosts
### List hosts
+37
View File
@@ -151,6 +151,10 @@ The following properties are recognized:
Array of features the plugin needs from the instances it is loaded on, any of `"SavePatching"`, `"ScriptCommands"` and `"LuaUdp"`.
See [Handling Invalid Configuration](#handling-invalid-configuration)
**requiredMods**:
Array of Factorio mods the plugin needs in the mod pack of the instances it is loaded on, written as Factorio dependency strings.
See [Handling Invalid Configuration](#handling-invalid-configuration)
**messages**:
Array of request and event classes for this plugin.
See guide for [defining link messages](#defining-link-messages) below.
@@ -347,6 +351,25 @@ export const plugin = {
Enabling the plugin on an instance that has the feature turned off then fails config validation, so the mistake is caught when the config is edited instead of when the instance starts.
Instances that have the feature turned off get the plugin disabled by default when it is added.
If the plugin talks to a Factorio mod, list the mod under `requiredMods` using the same format as `dependencies` in a mod's info.json, without a prefix.
List a mod twice to give a version range:
```js
export const plugin = {
name: "foo_frobber",
// ...
requiredMods: ["frobber >= 1.2.0", "frobber < 2.0.0"],
};
```
An instance with the plugin loaded refuses to start if its mod pack is missing one of the mods, has it disabled, or has a version outside the range.
Leave out the dependencies of the required mods. Factorio checks those itself.
When the controller starts it downloads required mods it does not have, and their dependencies, from the mod portal for the Factorio version of the default mod pack.
This needs `controller.factorio_username` and `controller.factorio_token` to be set.
The mods still have to be added to the mod packs by hand.
Forks of a required mod are not supported.
Other constraints on config fields are expressed with a `validator` on the field definition, which throws when the value is not acceptable.
Use `dependsOn` when the check involves other fields so that the validator runs when those change too.
See [Configuration System](config-system.md) for details.
@@ -998,6 +1021,20 @@ For a command the `definition` is the arguments to pass to [yargs.command](http:
The `handler` is an async function that's invoked when the command is executed and it's passed the parsed command line arguments and a reference to the `Ctl` class of clusterioctl.
Note that messages sent from clusterioctl needs to have `"control"` as a part of `src` for it to be accepted by the controller, see [Defining Link Messages](#defining-link-messages) for how to define the messages that can be sent to the controller.
Commands print their results through `ctl.output` so that `--output json` works for them:
```js
handler: async function(args, ctl) {
const result = await ctl.sendTo("controller", new messages.FrobnicateRequest(args.type));
ctl.output.result(result, () => console.log(`Frobnicated ${result.count} foos`));
},
```
`ctl.output.result(data, human)` prints `data` as JSON in json and wire mode and calls `human` otherwise.
`ctl.output.table(rows, map)` prints an array of objects as a JSON array, or as a table in human mode with `map` applied to each row first.
`ctl.output.human` is true when the output is meant for reading.
Commands that print nothing get `Command ran successfully` or `{"success": true}` printed for them, and errors thrown from the handler are printed as `{"error": "<ClassName>", "message": "<message>"}` in json and wire mode.
To have the command tree become part of clusterioctl it needs to be added to the rootCommand tree in the `addCommands` hook of the ctl entrypoint:
```js
+4 -7
View File
@@ -21,7 +21,6 @@ import crypto from "crypto";
import yargs from "yargs";
import util from "util";
import winston from "winston";
import DailyRotateFile from "winston-daily-rotate-file";
// homebrew modules
@@ -355,19 +354,17 @@ async function initialize(): Promise<InitializeParameters> {
level: "verbose",
levels,
});
clusterLogger.add(new DailyRotateFile({
clusterLogger.add(new lib.DailyFileTransport({
format: winston.format.json(),
filename: "cluster-%DATE%.log",
utc: true,
dirname: path.join(args.logDirectory, "cluster"),
prefix: "cluster",
}));
// Log stream for the controller.
logger.add(new DailyRotateFile({
logger.add(new lib.DailyFileTransport({
format: winston.format.json(),
filename: "controller-%DATE%.log",
utc: true,
dirname: path.join(args.logDirectory, "controller"),
prefix: "controller",
}));
logger.add(new winston.transports.Stream({
stream: clusterLogger,
+1 -2
View File
@@ -1,6 +1,6 @@
{
"name": "@clusterio/controller",
"version": "2.1.0",
"version": "2.1.1",
"description": "Implementation of Clusterio controller",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Hornwitser <github@hornwitser.no>",
@@ -45,7 +45,6 @@
"jsonwebtoken": "catalog:",
"semver": "catalog:",
"winston": "catalog:",
"winston-daily-rotate-file": "catalog:",
"ws": "catalog:",
"yargs": "catalog:"
},
+3 -206
View File
@@ -662,36 +662,7 @@ export default class ControlConnection extends BaseConnection {
* @param request - Request object with mod name, version, and factorioVersion.
*/
async handleModPortalDownloadRequest(request: lib.ModPortalDownloadRequest) {
const factorioVersion = request.factorioVersion;
const toDownload = this._controller.modStore.filterInstalled(request.mods, false);
if (toDownload.length > 0) {
// Get Factorio credentials from config
const username = this._controller.config.get("controller.factorio_username");
const token = this._controller.config.get("controller.factorio_token");
if (!username || username === "" || !token || token === "") {
throw new lib.ConflictError("Factorio credentials (username, token) not configured on the controller.");
}
try {
logger.info(`Downloading ${toDownload.length} mods for Factorio ${factorioVersion} from portal.`);
await this._controller.modStore.downloadMods(toDownload, username, token, factorioVersion);
} catch (error: any) {
logger.error(`Error downloading mods from portal: ${error.message}`);
const errorMessage = "Mod portal download failed";
if (error.message.includes("401") || error.message.includes("Unauthorized")) {
throw new lib.ConflictError(
`${errorMessage}: Authentication failed. Check Factorio username/token in controller config.`
);
}
if (error.message.includes("404") || error.message.includes("Not Found")) {
throw new lib.NotFoundError(`${errorMessage}: Mod version not found on the portal.`);
}
throw new lib.InternalServerError(`${errorMessage}: ${error.message}`);
}
}
await this._controller.downloadModsFromPortal(request.mods, request.factorioVersion);
// Verify all mods are present
const modInfos = [];
@@ -707,183 +678,9 @@ export default class ControlConnection extends BaseConnection {
return modInfos;
}
// eslint-disable-next-line complexity
async handleModDependencyResolveRequest(request: lib.ModDependencyResolveRequest) {
const {factorioVersion, checkForUpdates} = request;
const searchQueue = [...request.mods];
const localInfos = [...this._controller.modStore.mods()];
const builtInModNames = lib.ModPack.getBuiltinModNames(factorioVersion);
const skipSearch = new Set(builtInModNames);
type ModPortalModType = InstanceType<typeof lib.ModPortalGetAllRequest.Response>["mods"][number];
type ModPortalReleaseType = NonNullable<ModPortalModType["releases"]>[number];
const dependencyRequirements = new Map<string, lib.ModVersionRange>();
const recommendedRequirements = new Map<string, lib.ModVersionRange>();
const optionalRequirements = new Map<string, lib.ModVersionRange>();
const candidateReleases = new Map<string, lib.ModInfo>();
const errors = new Map<string, lib.ModDependencyResolveErrors>();
const _modPortalReleases = new Map<string, ModPortalModType | null>();
async function fetchModReleases(modName: string) {
let releases = _modPortalReleases.get(modName);
if (releases === undefined) {
try {
releases = await lib.ModStore.fetchModReleases(modName);
} catch (error: any) {
if (!error.message.includes("404") && !error.message.includes("Not Found")) {
logger.error(`Error fetching mod from portal: ${error.message}`);
throw new lib.InternalServerError(`Dependency resolution failed: ${error.message}`);
} else {
releases = null;
}
}
}
_modPortalReleases.set(modName, releases);
return releases;
}
function selectModRelease(releases: ModPortalModType | null, versionRange: lib.ModVersionRange) {
if (!releases || !releases.releases) {
return null;
}
// Select the latest matching version for the mod pack's Factorio version
const release = releases.releases
.filter(info => (
versionRange.testVersion(info.version)
&& lib.normaliseMajorMinorVersion(
lib.normaliseSourceVersion(info.info_json.factorio_version)
) === factorioVersion
))
.reduce<ModPortalReleaseType | undefined>((max, cur) => (
max && lib.integerSourceVersion(max.version) > lib.integerSourceVersion(cur.version) ? max : cur
), undefined);
if (!release) {
return null;
}
// Create the mod info class from the selected
try {
return lib.ModInfo.fromJSON({
...release.info_json, // includes dependencies
sha1: release.sha1,
version: release.version,
name: releases.name,
author: releases.owner,
title: releases.title,
});
} catch {
// Can error if there are invalid dependencies
errors.set(releases.name, "invalidDependency");
return null;
}
}
while (searchQueue.length > 0) {
// Dequeue the next mod, skip if incompatible
const mod = searchQueue.shift()!;
if (mod.incompatible) {
errors.set(mod.name, "incompatible");
continue;
}
// Get the current version range (dependencies can be promoted to a stricter requirement)
const versionRange =
dependencyRequirements.get(mod.name)
?? recommendedRequirements.get(mod.name)
?? optionalRequirements.get(mod.name)
?? new lib.ModVersionRange();
if (mod.required) {
// Add to required, remove from recommended and optional
dependencyRequirements.set(mod.name, versionRange);
recommendedRequirements.delete(mod.name);
optionalRequirements.delete(mod.name);
} else if (mod.recommended) {
// If not required, add to recommended and remove from optional
if (!dependencyRequirements.has(mod.name)) {
recommendedRequirements.set(mod.name, versionRange);
optionalRequirements.delete(mod.name);
}
} else if (
!dependencyRequirements.has(mod.name)
&& !recommendedRequirements.has(mod.name)
) {
// If not required or recommended, add to optional
optionalRequirements.set(mod.name, versionRange);
}
// Update the version range if needed
if (mod.version) {
versionRange.combineVersion(mod.version);
if (!versionRange.valid) {
candidateReleases.delete(mod.name);
errors.set(mod.name, "unsatisfiable");
continue;
}
}
// Do not add dependencies for errors, skips, optionals, or valid candidates
const existingCandidate = candidateReleases.get(mod.name);
if (errors.has(mod.name) || skipSearch.has(mod.name) || optionalRequirements.has(mod.name) || (
existingCandidate && versionRange.testVersion(existingCandidate.version)
)) {
continue;
}
// Check if a local version fits the requirements
let candidate = localInfos
.filter(info => (
info.name === mod.name
&& info.factorioVersion === factorioVersion
&& versionRange.testVersion(info.version)
))
.reduce<lib.ModInfo | undefined>((max, cur) => (
max && max.integerVersion > cur.integerVersion ? max : cur
), undefined);
// Check if a mod poral version fits the requirements
if (!candidate || checkForUpdates) {
const releases = await fetchModReleases(mod.name);
const release = selectModRelease(releases, versionRange);
// We only select the mod portal release if it is more recent than the local candidate
if (release && (!candidate || release.integerVersion > candidate.integerVersion)) {
candidate = release;
}
}
// Queue the dependencies of the new candidate
if (candidate) {
candidateReleases.set(mod.name, candidate);
for (const dep of candidate.dependencies) {
searchQueue.push(dep);
}
} else {
candidateReleases.delete(mod.name);
skipSearch.add(mod.name);
}
}
// Add the not found dependencies as errors
for (const modName of dependencyRequirements.keys()) {
if (!candidateReleases.has(modName) && !errors.has(modName)) {
errors.set(modName, "notFound");
}
}
// Remove incompatible and unsatisfiable dependencies if they aren't required by others
for (const [modName, error] of errors) {
if (!dependencyRequirements.has(modName) && (error === "incompatible" || error === "unsatisfiable")) {
errors.delete(modName);
}
}
return new lib.ModDependencyResolveRequest.Response(
[...candidateReleases.values()], errors,
return await this._controller.resolveModDependencies(
request.mods, request.factorioVersion, request.checkForUpdates,
);
}
+262
View File
@@ -513,6 +513,9 @@ export default class Controller {
logger.info("Started controller");
this.startedAtMs = Date.now();
this.downloadRequiredMods().catch(
err => logger.warn(`Unable to download mods required by plugins: ${err.message}`)
);
this.onAutosaveIntervalChanged();
this.onSystemMetricsIntervalChanged();
this._state = "running";
@@ -1170,6 +1173,265 @@ export default class Controller {
}
/**
* Download mods from the Factorio mod portal to the mod store
*
* Mods already in the mod store are skipped.
*
* @throws {lib.ConflictError} if the Factorio credentials are not configured.
*/
async downloadModsFromPortal(mods: lib.ModNameVersionPair[], factorioVersion: lib.MajorMinorVersion) {
const toDownload = this.modStore.filterInstalled(mods, false);
if (toDownload.length > 0) {
// Get Factorio credentials from config
const username = this.config.get("controller.factorio_username");
const token = this.config.get("controller.factorio_token");
if (!username || username === "" || !token || token === "") {
throw new lib.ConflictError("Factorio credentials (username, token) not configured on the controller.");
}
try {
logger.info(`Downloading ${toDownload.length} mods for Factorio ${factorioVersion} from portal.`);
await this.modStore.downloadMods(toDownload, username, token, factorioVersion);
} catch (error: any) {
logger.error(`Error downloading mods from portal: ${error.message}`);
const errorMessage = "Mod portal download failed";
if (error.message.includes("401") || error.message.includes("Unauthorized")) {
throw new lib.ConflictError(
`${errorMessage}: Authentication failed. Check Factorio username/token in controller config.`
);
}
if (error.message.includes("404") || error.message.includes("Not Found")) {
throw new lib.NotFoundError(`${errorMessage}: Mod version not found on the portal.`);
}
throw new lib.InternalServerError(`${errorMessage}: ${error.message}`);
}
}
}
/**
* Resolve mods and their dependencies against the mod store and the Factorio mod portal
*/
// eslint-disable-next-line complexity
async resolveModDependencies(
mods: lib.ModDependency[],
factorioVersion: lib.MajorMinorVersion,
checkForUpdates = false,
) {
const searchQueue = [...mods];
const localInfos = [...this.modStore.mods()];
const builtInModNames = lib.ModPack.getBuiltinModNames(factorioVersion);
const skipSearch = new Set(builtInModNames);
type ModPortalModType = InstanceType<typeof lib.ModPortalGetAllRequest.Response>["mods"][number];
type ModPortalReleaseType = NonNullable<ModPortalModType["releases"]>[number];
const dependencyRequirements = new Map<string, lib.ModVersionRange>();
const recommendedRequirements = new Map<string, lib.ModVersionRange>();
const optionalRequirements = new Map<string, lib.ModVersionRange>();
const candidateReleases = new Map<string, lib.ModInfo>();
const errors = new Map<string, lib.ModDependencyResolveErrors>();
const _modPortalReleases = new Map<string, ModPortalModType | null>();
async function fetchModReleases(modName: string) {
let releases = _modPortalReleases.get(modName);
if (releases === undefined) {
try {
releases = await lib.ModStore.fetchModReleases(modName);
} catch (error: any) {
if (!error.message.includes("404") && !error.message.includes("Not Found")) {
logger.error(`Error fetching mod from portal: ${error.message}`);
throw new lib.InternalServerError(`Dependency resolution failed: ${error.message}`);
} else {
releases = null;
}
}
}
_modPortalReleases.set(modName, releases);
return releases;
}
function selectModRelease(releases: ModPortalModType | null, versionRange: lib.ModVersionRange) {
if (!releases || !releases.releases) {
return null;
}
// Select the latest matching version for the mod pack's Factorio version
const release = releases.releases
.filter(info => (
versionRange.testVersion(info.version)
&& lib.normaliseMajorMinorVersion(
lib.normaliseSourceVersion(info.info_json.factorio_version)
) === factorioVersion
))
.reduce<ModPortalReleaseType | undefined>((max, cur) => (
max && lib.integerSourceVersion(max.version) > lib.integerSourceVersion(cur.version) ? max : cur
), undefined);
if (!release) {
return null;
}
// Create the mod info class from the selected
try {
return lib.ModInfo.fromJSON({
...release.info_json, // includes dependencies
sha1: release.sha1,
version: release.version,
name: releases.name,
author: releases.owner,
title: releases.title,
});
} catch {
// Can error if there are invalid dependencies
errors.set(releases.name, "invalidDependency");
return null;
}
}
while (searchQueue.length > 0) {
// Dequeue the next mod, skip if incompatible
const mod = searchQueue.shift()!;
if (mod.incompatible) {
errors.set(mod.name, "incompatible");
continue;
}
// Get the current version range (dependencies can be promoted to a stricter requirement)
const versionRange =
dependencyRequirements.get(mod.name)
?? recommendedRequirements.get(mod.name)
?? optionalRequirements.get(mod.name)
?? new lib.ModVersionRange();
if (mod.required) {
// Add to required, remove from recommended and optional
dependencyRequirements.set(mod.name, versionRange);
recommendedRequirements.delete(mod.name);
optionalRequirements.delete(mod.name);
} else if (mod.recommended) {
// If not required, add to recommended and remove from optional
if (!dependencyRequirements.has(mod.name)) {
recommendedRequirements.set(mod.name, versionRange);
optionalRequirements.delete(mod.name);
}
} else if (
!dependencyRequirements.has(mod.name)
&& !recommendedRequirements.has(mod.name)
) {
// If not required or recommended, add to optional
optionalRequirements.set(mod.name, versionRange);
}
// Update the version range if needed
if (mod.version) {
versionRange.combineVersion(mod.version);
if (!versionRange.valid) {
candidateReleases.delete(mod.name);
errors.set(mod.name, "unsatisfiable");
continue;
}
}
// Do not add dependencies for errors, skips, optionals, or valid candidates
const existingCandidate = candidateReleases.get(mod.name);
if (errors.has(mod.name) || skipSearch.has(mod.name) || optionalRequirements.has(mod.name) || (
existingCandidate && versionRange.testVersion(existingCandidate.version)
)) {
continue;
}
// Check if a local version fits the requirements
let candidate = localInfos
.filter(info => (
info.name === mod.name
&& info.factorioVersion === factorioVersion
&& versionRange.testVersion(info.version)
))
.reduce<lib.ModInfo | undefined>((max, cur) => (
max && max.integerVersion > cur.integerVersion ? max : cur
), undefined);
// Check if a mod poral version fits the requirements
if (!candidate || checkForUpdates) {
const releases = await fetchModReleases(mod.name);
const release = selectModRelease(releases, versionRange);
// We only select the mod portal release if it is more recent than the local candidate
if (release && (!candidate || release.integerVersion > candidate.integerVersion)) {
candidate = release;
}
}
// Queue the dependencies of the new candidate
if (candidate) {
candidateReleases.set(mod.name, candidate);
for (const dep of candidate.dependencies) {
searchQueue.push(dep);
}
} else {
candidateReleases.delete(mod.name);
skipSearch.add(mod.name);
}
}
// Add the not found dependencies as errors
for (const modName of dependencyRequirements.keys()) {
if (!candidateReleases.has(modName) && !errors.has(modName)) {
errors.set(modName, "notFound");
}
}
// Remove incompatible and unsatisfiable dependencies if they aren't required by others
for (const [modName, error] of errors) {
if (!dependencyRequirements.has(modName) && (error === "incompatible" || error === "unsatisfiable")) {
errors.delete(modName);
}
}
return new lib.ModDependencyResolveRequest.Response(
[...candidateReleases.values()], errors,
);
}
/**
* Download the mods required by plugins that are missing from the mod store
*
* Resolves for the Factorio version of the default mod pack. Dependencies
* of the required mods are downloaded too.
*/
async downloadRequiredMods() {
const required = this.pluginInfos.flatMap(pluginInfo => lib.pluginRequiredMods(pluginInfo));
const modPackId = this.config.get("controller.default_mod_pack_id");
const modPack = modPackId !== null ? this.modPacks.get(modPackId) : undefined;
if (!required.length || !modPack) {
return;
}
const factorioVersion = lib.normaliseMajorMinorVersion(modPack.factorioVersion);
const { dependencies, errors } = await this.resolveModDependencies(required, factorioVersion);
const builtInMods = new Set(lib.ModPack.getBuiltinModNames(factorioVersion));
for (const [name, error] of errors) {
if (!builtInMods.has(name)) {
logger.warn(`Unable to resolve mod ${name} required by plugins: ${error}`);
}
}
const toDownload = this.modStore.filterInstalled(dependencies.map(
info => ({ name: info.name, version: new lib.ModVersionEquality("=", info.normalisedVersion) })
), false);
if (!toDownload.length) {
return;
}
const names = toDownload.map(mod => `${mod.name} ${mod.version.version}`).join(", ");
logger.info(`Downloading mods required by plugins: ${names}`);
await this.downloadModsFromPortal(toDownload, factorioVersion);
}
modPacksUpdated(modPacks: lib.ModPack[]) {
this.subscriptions.broadcast(new lib.ModPackUpdatesEvent(modPacks));
this.hooks.modPacksUpdated.invoke(modPacks);
+1 -1
View File
@@ -828,7 +828,7 @@ async function uploadMod(req: Request, res: Response) {
// Pipe does not pass on an upload that was cut short.
await Promise.all([finished(file.stream), finished(writeStream)]);
const modInfo = await lib.ModInfo.fromModFile(tempFilePath);
const modInfo = await lib.ModStore.readModInfo(tempFilePath);
checkModName(modInfo.filename);
await fs.rename(tempFilePath, path.join(modsDirectory, modInfo.filename));
req.app.locals.controller.modStore.addMod(modInfo);
+2 -5
View File
@@ -1,5 +1,5 @@
/* eslint-disable no-console */
import chalk from "chalk";
import { styleText } from "node:util";
export const levels = Object.freeze({
fatal: 0,
@@ -27,10 +27,7 @@ for (let [name, level] of Object.entries(levels)) {
if (level > maxLogLevel) {
return;
}
let style = chalk;
for (let part of colors[name].split(" ")) {
style = style[part];
}
const style = text => styleText(colors[name].split(" "), text);
if (["fatal", "error"].includes(name)) {
console.error(`${style(name)} ${message}`);
} else if (["warn"].includes(name)) {
+1 -2
View File
@@ -1,6 +1,6 @@
{
"name": "@clusterio/create",
"version": "2.1.0",
"version": "2.1.1",
"description": "Installer for clusterio",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Hornwitser <github@hornwitser.no>",
@@ -36,7 +36,6 @@
"@inquirer/confirm": "^6.3.5",
"@inquirer/input": "^5.1.9",
"@inquirer/select": "^5.2.8",
"chalk": "catalog:",
"yargs": "catalog:"
}
}
+5
View File
@@ -23,6 +23,11 @@ Common options:
* `--create-config` If the config file is missing, create a new one and carry on instead of exiting with an error.
Alternative to the `config create` command.
* `--output <human|json|wire>` Format to print the results of cluster commands in.
Defaults to `human`.
See [Output format](/docs/managing-a-cluster.md#output-format) for what `json` and `wire` print.
The `plugin` and `config` commands described below always print human readable text.
### `plugin <command>`
+57 -17
View File
@@ -17,6 +17,7 @@ import { ConsoleTransport, levels, logger } from "@clusterio/lib";
import * as commands from "./src/commands.js";
import { BaseCtlPlugin, CtlHooks } from "./src/BaseCtlPlugin.js";
import { CommandOutput, errorToJson, outputFormats, print, type OutputFormat } from "./src/command_ops.js";
/**
@@ -55,6 +56,8 @@ export class Ctl extends lib.Link {
config: lib.CtlConfig;
/** Keep the control connection alive after the command completes. */
keepOpen = false;
/** Output of the command being run. */
output = new CommandOutput();
constructor(
connector: CtlConnector,
@@ -82,12 +85,20 @@ export class Ctl extends lib.Link {
}
async handleLogMessageEvent(event: lib.LogMessageEvent) {
logger.log(event.info as any);
// Only log following commands print log events as results, the rest log them to stderr
if (this.output.streamingLog) {
this.output.result(event.info, () => logger.log(event.info as any));
} else {
logger.log(event.info as any);
}
}
async handleDebugWsMessageEvent(event: lib.DebugWsMessageEvent) {
// eslint-disable-next-line no-console
console.log("WS", event.direction, event.content);
this.output.result(
{ direction: event.direction, content: event.content },
// eslint-disable-next-line no-console
() => console.log("WS", event.direction, event.content),
);
}
async shutdown() {
@@ -126,6 +137,7 @@ interface CtlArguments {
$0: string,
_: (string | number)[],
logLevel: keyof typeof levels,
output: OutputFormat,
config: string,
pluginList: string,
bypassLockFile: boolean,
@@ -165,6 +177,13 @@ export async function initialize(
choices: Object.keys(levels),
type: "string",
})
.option("output", {
nargs: 1,
describe: "Format to print command results in",
default: "human",
choices: outputFormats,
type: "string",
})
.option("config", {
nargs: 1,
describe: "config file to get credentails from",
@@ -260,6 +279,20 @@ export function selectTargetCommand(args: CtlArguments, rootCommands: lib.Comman
return targetCommand;
}
/**
* Run the command selected by args, printing results in the format given by args.output
*/
export async function runCommand(
args: CtlArguments,
rootCommands: lib.CommandTree,
link: lib.Link & { output?: CommandOutput },
) {
link.output = new CommandOutput(args.output);
const targetCommand = selectTargetCommand(args, rootCommands);
await targetCommand.run(args, link);
link.output.finish();
}
async function startCtl() {
const {
args,
@@ -271,10 +304,19 @@ async function startCtl() {
return;
}
try {
await connectAndRun(args, rootCommands, ctlConfig);
} catch (err) {
if (args.output !== "human") {
print(JSON.stringify(errorToJson(err)));
}
throw err;
}
}
async function connectAndRun(args: CtlArguments, rootCommands: lib.CommandTree, ctlConfig: lib.CtlConfig) {
if (!ctlConfig.get("ctl.controller_url") || !ctlConfig.get("ctl.controller_token")) {
logger.error("Missing URL and/or token to connect with. See README.md for setting up access.");
process.exitCode = 1;
return;
throw new lib.StartupError("Missing URL and/or token to connect with. See README.md for setting up access.");
}
let controlConnector = new CtlConnector(
@@ -298,26 +340,25 @@ async function startCtl() {
process.on("SIGTERM", lib.createShutdownGuard(logger, "termination", ctl.shutdown.bind(ctl)));
try {
const targetCommand = selectTargetCommand(args, rootCommands);
await targetCommand.run(args, ctl);
await runCommand(args, rootCommands, ctl);
} catch (err) {
ctl.keepOpen = false;
let message: string;
if (err instanceof lib.CommandError) {
logger.error(`Error running command: ${err.message}`);
process.exitCode = 1;
message = `Error running command: ${err.message}`;
} else if (err instanceof lib.ClientError || err instanceof lib.UnavailableError) {
logger.error(`Error sending request: ${err.message}`);
process.exitCode = 1;
message = `Error sending request: ${err.message}`;
} else if (err instanceof lib.ServerError) {
logger.error(`Error sending request:\n${err.stack ?? err.message}`);
process.exitCode = 1;
message = `Error sending request:\n${err.stack ?? err.message}`;
} else {
throw err;
}
ctl.output.error(err, () => logger.error(message));
process.exitCode = 1;
} finally {
if (!ctl.keepOpen) {
@@ -328,6 +369,7 @@ async function startCtl() {
export function bootstrap() {
startCtl().catch(err => {
process.exitCode = 1;
if (err.errors) {
logger.fatal(JSON.stringify(err.errors, null, "\t"));
}
@@ -340,15 +382,13 @@ export function bootstrap() {
${err.stack}`
);
} else {
logger.error(`
logger.fatal(`
+---------------------------------+
| Unable to to start clusterioctl |
+---------------------------------+
${err.stack}`
);
}
process.exitCode = 1;
});
}
+1
View File
@@ -1,5 +1,6 @@
import { bootstrap } from "./ctl.js";
export * from "./ctl.js";
export { CommandOutput, type OutputFormat } from "./src/command_ops.js";
export { BaseCtlPlugin, CtlHooks, type CtlPluginContext } from "./src/BaseCtlPlugin.js";
if (import.meta.main) {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@clusterio/ctl",
"version": "2.1.0",
"version": "2.1.1",
"description": "Clusterio command line management tool",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Hornwitser <github@hornwitser.no>",
+117
View File
@@ -13,3 +13,120 @@ function formatCell(value: unknown) {
}
export const asTable = asTableModule.configure({ delimiter: " | ", print: formatCell });
/** Copy of object with only the given fields, or all of them if fields includes "all" */
export function pickFields(object: object, fields: string[]) {
if (fields.includes("all")) {
return object;
}
return Object.fromEntries(Object.entries(object).filter(([field]) => fields.includes(field)));
}
/** Convert class instances, Sets and Maps into plain JSON values, ignoring wire format toJSON methods */
export function toPlain(value: unknown): unknown {
if (value instanceof Set || value instanceof Array) {
return [...value].map(toPlain);
}
if (value instanceof Map) {
return Object.fromEntries([...value].map(([key, entry]) => [key, toPlain(entry)]));
}
if (value instanceof Date) {
return value.toISOString();
}
if (typeof value === "object" && value !== null) {
return Object.fromEntries(Object.entries(value).map(([key, entry]) => [key, toPlain(entry)]));
}
return value;
}
/** Describe an error as `{ error: ClassName, message: string }` */
export function errorToJson(err: unknown) {
if (err instanceof Error) {
return { error: err.constructor.name, message: err.message };
}
return { error: "Error", message: String(err) };
}
/** JSON.stringify replacer for Sets and Maps that have no toJSON method */
function wireReplacer(key: string, value: unknown) {
if (value instanceof Set) {
return [...value];
}
if (value instanceof Map) {
return Object.fromEntries(value);
}
return value;
}
export const outputFormats = ["human", "json", "wire"] as const;
export type OutputFormat = typeof outputFormats[number];
/**
* Prints the results of a ctl command in the format chosen with --output
*
* In json and wire mode each call to result prints one line of JSON to stdout. json uses the
* property names of the objects, wire uses their toJSON methods to give the format sent over the link.
*/
export class CommandOutput {
/** True once a result has been printed */
written = false;
/** Command prints log events as results while it runs and has no final result */
streamingLog = false;
constructor(
public format: OutputFormat = "human",
) { }
get human() {
return this.format === "human";
}
/**
* Print the result of a command
*
* @param data - Printed as JSON in json and wire mode.
* @param human - Prints the result in human mode, defaults to printing data as is.
*/
result(data: unknown, human: () => void = () => print(data)) {
this.written = true;
if (this.format === "json") {
print(JSON.stringify(toPlain(data)));
} else if (this.format === "wire") {
print(JSON.stringify(data, wireReplacer));
} else {
human();
}
}
/**
* Print a list of objects as a table, or as an array in json and wire mode
*
* @param rows - Printed as is in json and wire mode.
* @param map - Converts each row into the table row printed in human mode.
*/
table<T extends object>(rows: T[], map: (row: T) => object = row => row) {
this.result(rows, () => print(asTable(rows.map(map))));
}
/**
* Print an error as `{ "error": ClassName, "message": string }` in json and wire mode
*
* @param err - Error to print.
* @param human - Reports the error in human mode.
*/
error(err: unknown, human: () => void) {
this.written = true;
if (this.human) {
human();
} else {
print(JSON.stringify(errorToJson(err)));
}
}
/** Print a success result for commands that finished without printing one */
finish() {
if (!this.written && !this.streamingLog) {
this.result({ success: true }, () => print("Command ran successfully"));
}
}
}
+25 -26
View File
@@ -1,15 +1,15 @@
import fs from "node:fs/promises";
import asTableModule from "as-table";
import events from "events";
import os from "os";
import child_process from "child_process";
import * as lib from "@clusterio/lib";
import { logger } from "@clusterio/lib";
import type { Ctl } from "../ctl.js";
import { print } from "./command_ops.js";
import { serializedConfigToString, getEditor, configToKeyVal } from "./config_ops.js";
const asTable = asTableModule.configure({ delimiter: " | " });
import { asTable, print } from "./command_ops.js";
import {
serializedConfigToString, getEditor, configToKeyVal, printConfig, printRestartRequired,
} from "./config_ops.js";
export const controllerCommands = new lib.CommandTree({ name: "controller", description: "Controller management" });
controllerCommands.add(new lib.Command({
@@ -44,9 +44,8 @@ controllerCommands.add(new lib.Command({
}
if (args.restart) {
await ctl.send(new lib.ControllerRestartRequest());
} else {
print("Controller updated; a restart is required to apply the changes.");
}
printRestartRequired(ctl, !args.restart, "Controller updated");
},
}));
@@ -57,11 +56,7 @@ const controllerConfigCommands = new lib.CommandTree({
controllerConfigCommands.add(new lib.Command({
definition: ["list", "List controller configuration"],
handler: async function(args: object, ctl: Ctl) {
let config = await ctl.send(new lib.ControllerConfigGetRequest());
for (let [name, value] of Object.entries(config)) {
print(`${name} ${JSON.stringify(value)}`);
}
printConfig(ctl, await ctl.send(new lib.ControllerConfigGetRequest()));
},
}));
@@ -146,19 +141,22 @@ controllerConfigCommands.add(new lib.Command({
index,
final[index],
));
} catch (err) {
// eslint-disable-next-line
print(`Attempt to set ${index} to ${final[index] || String(null)} failed; set back to previous value.`);
print(err);
} catch (err: any) {
// If the string is empty, it's better to just print "" instead of nothing
logger.error(
`Attempt to set ${index} to ${final[index] || String(null)} failed; ` +
`set back to previous value: ${err.message}`
);
}
}
}
try {
await fs.unlink(tmpFile);
} catch (err) {
print("err: temporary file", tmpFile, "could not be deleted.");
print("This is not fatal, but they may build up over time if the issue persists.");
logger.warn(
`Temporary file ${tmpFile} could not be deleted. ` +
"This is not fatal, but they may build up over time if the issue persists."
);
}
},
}));
@@ -175,7 +173,7 @@ controllerPluginCommands.add(new lib.Command({
let url = new URL(ctl.config.get("ctl.controller_url")!);
url.pathname += "api/plugins";
let response = await fetch(url);
print(asTable(await response.json() as []));
ctl.output.table(await response.json() as object[]);
},
}));
controllerPluginCommands.add(new lib.Command({
@@ -187,9 +185,8 @@ controllerPluginCommands.add(new lib.Command({
await ctl.sendTo("controller", new lib.PluginUpdateRequest(args.plugin));
if (args.restart) {
await ctl.send(new lib.ControllerRestartRequest());
} else {
print("Plugin updated; a restart is required to apply the changes.");
}
printRestartRequired(ctl, !args.restart, "Plugin updated");
},
}));
controllerPluginCommands.add(new lib.Command({
@@ -202,13 +199,16 @@ controllerPluginCommands.add(new lib.Command({
const response = await ctl.sendTo(
"controller", new lib.PluginSearchRequest(args.query, args.page, args.pageSize)
);
print(asTable(response.results.map(plugin => ({
const results = response.results.map(plugin => ({
name: plugin.name,
version: plugin.version,
publisher: plugin.publisher ?? "",
description: plugin.description ?? "",
}))));
print(`Showing ${response.results.length} of ${response.total} results`);
}));
ctl.output.result({ total: response.total, results }, () => {
print(asTable(results));
print(`Showing ${results.length} of ${response.total} results`);
});
},
}));
controllerPluginCommands.add(new lib.Command({
@@ -220,9 +220,8 @@ controllerPluginCommands.add(new lib.Command({
await ctl.sendTo("controller", new lib.PluginInstallRequest(args.plugin));
if (args.restart) {
await ctl.send(new lib.ControllerRestartRequest());
} else {
print("Plugin installed; a restart is required to apply the changes.");
}
printRestartRequired(ctl, !args.restart, "Plugin installed");
},
}));
+3 -7
View File
@@ -1,5 +1,6 @@
import * as lib from "@clusterio/lib";
import type { Ctl } from "../ctl.js";
import { print } from "./command_ops.js";
export const debugCommands = new lib.CommandTree({ name: "debug", description: "Debugging utilities" });
debugCommands.add(new lib.Command({
@@ -7,6 +8,7 @@ debugCommands.add(new lib.Command({
handler: async function(args: object, ctl: Ctl) {
await ctl.send(new lib.DebugDumpWsRequest());
ctl.keepOpen = true;
ctl.output.streamingLog = true;
},
}));
@@ -20,12 +22,6 @@ debugCommands.add(new lib.Command({
}],
handler: async function(args: { exchangeString: string, pretty: boolean }, ctl: Ctl) {
const result = lib.readMapExchangeString(args.exchangeString);
if (args.pretty) {
// eslint-disable-next-line no-console
console.log(JSON.stringify(result, null, 2));
} else {
// eslint-disable-next-line no-console
console.log(JSON.stringify(result));
}
ctl.output.result(result, () => print(JSON.stringify(result, null, args.pretty ? 2 : undefined)));
},
}));
+38 -38
View File
@@ -1,15 +1,15 @@
import fs from "node:fs/promises";
import asTableModule from "as-table";
import events from "events";
import os from "os";
import child_process from "child_process";
import * as lib from "@clusterio/lib";
import { logger } from "@clusterio/lib";
import type { Ctl } from "../ctl.js";
import { print } from "./command_ops.js";
import { serializedConfigToString, getEditor, configToKeyVal } from "./config_ops.js";
const asTable = asTableModule.configure({ delimiter: " | " });
import { asTable, print } from "./command_ops.js";
import {
serializedConfigToString, getEditor, configToKeyVal, printConfig, printRestartRequired,
} from "./config_ops.js";
export const hostCommands = new lib.CommandTree({ name: "host", description: "Host management" });
hostCommands.add(new lib.Command({
@@ -45,8 +45,7 @@ hostCommands.add(new lib.Command({
hostCommands.add(new lib.Command({
definition: [["list", "l"], "List hosts connected to the controller"],
handler: async function(args: object, ctl: Ctl) {
let hosts = await ctl.send(new lib.HostListRequest());
print(asTable(hosts));
ctl.output.table(await ctl.send(new lib.HostListRequest()));
},
}));
@@ -71,17 +70,19 @@ hostCommands.add(new lib.Command({
definition: ["pending", "List host installers waiting for approval"],
handler: async function(args: object, ctl: Ctl) {
let requests = await ctl.send(new lib.HostPairingListRequest());
if (!requests.length) {
print("No pending host requests");
return;
}
let now = Date.now();
print(asTable(requests.map(request => ({
code: request.code,
name: request.name,
address: request.remoteAddress,
age: formatAge(now - request.createdAtMs),
}))));
ctl.output.result(requests, () => {
if (!requests.length) {
print("No pending host requests");
return;
}
let now = Date.now();
print(asTable(requests.map(request => ({
code: request.code,
name: request.name,
address: request.remoteAddress,
age: formatAge(now - request.createdAtMs),
}))));
});
},
}));
@@ -99,7 +100,10 @@ hostCommands.add(new lib.Command({
name = requests.find(request => plainCode(request.code) === plainCode(args.code))?.name;
}
let hostId = await ctl.send(new lib.HostPairingApproveRequest(args.code, args.name));
print(name ? `Approved host ${name} with id ${hostId}` : `Approved host with id ${hostId}`);
ctl.output.result(
{ id: hostId, name },
() => print(name ? `Approved host ${name} with id ${hostId}` : `Approved host with id ${hostId}`),
);
},
}));
@@ -137,9 +141,8 @@ hostCommands.add(new lib.Command({
}
if (args.restart) {
await ctl.sendTo({ hostId }, new lib.HostRestartRequest());
} else {
print("Host updated; a restart is required to apply the changes.");
}
printRestartRequired(ctl, !args.restart, "Host updated");
},
}));
@@ -153,11 +156,7 @@ hostConfigCommands.add(new lib.Command({
}],
handler: async function(args: { host: string }, ctl: Ctl) {
let hostId = await lib.resolveHost(ctl, args.host);
let config = await ctl.sendTo({ hostId }, new lib.HostConfigGetRequest());
for (let [name, value] of Object.entries(config)) {
print(`${name} ${JSON.stringify(value)}`);
}
printConfig(ctl, await ctl.sendTo({ hostId }, new lib.HostConfigGetRequest()));
},
}));
@@ -255,19 +254,22 @@ hostConfigCommands.add(new lib.Command({
index,
final[index],
));
} catch (err) {
// eslint-disable-next-line
print(`Attempt to set ${index} to ${final[index] || String(null)} failed; set back to previous value.`);
} catch (err: any) {
// If the string is empty, it's better to just print "" instead of nothing
print(err);
logger.error(
`Attempt to set ${index} to ${final[index] || String(null)} failed; ` +
`set back to previous value: ${err.message}`
);
}
}
}
try {
await fs.unlink(tmpFile);
} catch (err) {
print("err: temporary file", tmpFile, "could not be deleted.");
print("This is not fatal, but they may build up over time if the issue persists.");
logger.warn(
`Temporary file ${tmpFile} could not be deleted. ` +
"This is not fatal, but they may build up over time if the issue persists."
);
}
},
}));
@@ -285,9 +287,9 @@ hostPluginCommands.add(new lib.Command({
handler: async function(args: { host: string }, ctl: Ctl) {
let hostId = await lib.resolveHost(ctl, args.host);
const plugins = await ctl.sendTo({ hostId }, new lib.PluginListRequest());
print(asTable(plugins.map(p => ({
ctl.output.table(plugins, p => ({
title: p.title, version: p.version, state: p.state, npmPackage: p.npmPackage,
}))));
}));
},
}));
hostPluginCommands.add(new lib.Command({
@@ -301,9 +303,8 @@ hostPluginCommands.add(new lib.Command({
await ctl.sendTo({ hostId }, new lib.PluginUpdateRequest(args.plugin));
if (args.restart) {
await ctl.sendTo({ hostId }, new lib.HostRestartRequest());
} else {
print("Plugin updated; a restart is required to apply the changes.");
}
printRestartRequired(ctl, !args.restart, "Plugin updated");
},
}));
hostPluginCommands.add(new lib.Command({
@@ -317,9 +318,8 @@ hostPluginCommands.add(new lib.Command({
await ctl.sendTo({ hostId }, new lib.PluginInstallRequest(args.plugin));
if (args.restart) {
await ctl.sendTo({ hostId }, new lib.HostRestartRequest());
} else {
print("Plugin installed; a restart is required to apply the changes.");
}
printRestartRequired(ctl, !args.restart, "Plugin installed");
},
}));
hostCommands.add(hostPluginCommands);
+70 -53
View File
@@ -1,5 +1,4 @@
import fs from "node:fs/promises";
import asTableModule from "as-table";
import events from "events";
import path from "path";
import os from "os";
@@ -10,10 +9,9 @@ import util from "util";
import * as lib from "@clusterio/lib";
import { logger } from "@clusterio/lib";
import type { Ctl } from "../ctl.js";
import { print } from "./command_ops.js";
import { serializedConfigToString, getEditor, configToKeyVal } from "./config_ops.js";
import { errorToJson, print } from "./command_ops.js";
import { serializedConfigToString, getEditor, configToKeyVal, printConfig } from "./config_ops.js";
const asTable = asTableModule.configure({ delimiter: " | " });
const finished = util.promisify(stream.finished);
export const instanceCommands = new lib.CommandTree({
@@ -27,7 +25,7 @@ instanceCommands.add(new lib.Command({
}],
handler: async function(args: { templates: boolean }, ctl: Ctl) {
let list = await ctl.send(new lib.InstanceDetailsListRequest());
print(asTable(list.filter(instance => args.templates || !instance.isTemplate)));
ctl.output.table(list.filter(instance => args.templates || !instance.isTemplate));
},
}));
@@ -64,6 +62,7 @@ instanceCommands.add(new lib.Command({
instanceConfig.toRemote("controller", fields),
args.from
));
ctl.output.result({ id: instanceConfig.get("instance.id") }, () => {});
},
}));
@@ -81,19 +80,13 @@ instanceConfigCommands.add(new lib.Command({
? config["instance.inherited_fields"] as Record<string, true | string[]> ?? {}
: {};
for (let [name, value] of Object.entries(config)) {
if (name === "instance.inherited_fields") {
continue;
}
printConfig(ctl, config, name => {
const entry = inherited[name];
let note = "";
if (entry instanceof Array && entry.length) {
note = ` (inherited except ${entry.join(", ")})`;
} else if (entry) {
note = " (inherited)";
return ` (inherited except ${entry.join(", ")})`;
}
print(`${name} ${JSON.stringify(value)}${note}`);
}
return entry ? " (inherited)" : "";
});
},
}));
@@ -225,19 +218,22 @@ instanceConfigCommands.add(new lib.Command({
index,
final[index],
));
} catch (err) {
// eslint-disable-next-line
print(`Attempt to set ${index} to ${final[index] || String(null)} failed; set back to previous value.`);
} catch (err: any) {
// If the string is empty, it's better to just print "" instead of nothing
print(err);
logger.error(
`Attempt to set ${index} to ${final[index] || String(null)} failed; ` +
`set back to previous value: ${err.message}`
);
}
}
}
try {
await fs.unlink(tmpFile);
} catch (err) {
print("err: temporary file", tmpFile, "could not be deleted.");
print("This is not fatal, but they may build up over time if the issue persists.");
logger.warn(
`Temporary file ${tmpFile} could not be deleted. ` +
"This is not fatal, but they may build up over time if the issue persists."
);
}
},
}));
@@ -291,9 +287,7 @@ instanceSaveCommands.add(new lib.Command({
let instanceId = await lib.resolveInstance(ctl, args.instance);
let saves = await ctl.sendTo("controller", new lib.InstanceSaveDetailsListRequest());
saves = saves.filter(save => save.instanceId === instanceId);
print(asTable(saves.map(
({ mtimeMs, ...rest }) => ({ ...rest, mtime: new Date(mtimeMs).toLocaleString() })
)));
ctl.output.table(saves, ({ mtimeMs, ...rest }) => ({ ...rest, mtime: new Date(mtimeMs).toLocaleString() }));
},
}));
@@ -394,13 +388,16 @@ instanceSaveCommands.add(new lib.Command({
logger.error(requestError);
}
if (result.saves && result.saves.length) {
logger.info(`Successfully uploaded as ${result.saves[0]}`);
}
if ((result.errors || []).length || (result.request_errors || []).length) {
throw new lib.CommandError("Uploading save failed");
}
const name = result.saves?.[0];
ctl.output.result({ name }, () => {
if (name) {
print(`Successfully uploaded as ${name}`);
}
});
},
}));
@@ -439,7 +436,10 @@ instanceSaveCommands.add(new lib.Command({
args.copy
)
);
print(`Transferred as ${storedName} to ${args.targetInstance}.`);
ctl.output.result(
{ name: storedName, instanceId: targetInstanceId },
() => print(`Transferred as ${storedName} to ${args.targetInstance}.`),
);
},
}));
@@ -456,7 +456,10 @@ instanceSaveCommands.add(new lib.Command({
url.pathname += `api/stream/${streamId}`;
const filename = await lib.downloadFile(url, args.save, "rename");
logger.info(`Downloaded ${args.save}${args.save === filename ? "" : ` as ${filename}`}`);
ctl.output.result(
{ filename },
() => print(`Downloaded ${args.save}${args.save === filename ? "" : ` as ${filename}`}`),
);
},
}));
@@ -566,27 +569,34 @@ instanceCommands.add(new lib.Command({
});
if (instancesToStart.length === 0) {
print("No instances to start.");
ctl.output.result([], () => print("No instances to start."));
return;
}
print(`Starting ${instancesToStart.length} instance(s):`);
for (const instance of instancesToStart) {
print(` - ${instance.name} (ID: ${instance.id})`);
const human = ctl.output.human;
if (human) {
print(`Starting ${instancesToStart.length} instance(s):`);
for (const instance of instancesToStart) {
print(` - ${instance.name} (ID: ${instance.id})`);
}
}
// Start all filtered instances
const startPromises = instancesToStart.map(async (instance: lib.InstanceDetails) => {
const results = await Promise.all(instancesToStart.map(async (instance: lib.InstanceDetails) => {
try {
await ctl.sendTo({ instanceId: instance.id }, new lib.InstanceStartRequest());
print(`✓ Started ${instance.name}`);
if (human) {
print(`✓ Started ${instance.name}`);
}
return { id: instance.id, name: instance.name };
} catch (error) {
print(`✗ Failed to start ${instance.name}: ${error}`);
if (human) {
print(`✗ Failed to start ${instance.name}: ${error}`);
}
return { id: instance.id, name: instance.name, error: errorToJson(error) };
}
});
await Promise.all(startPromises);
print("Start all operation completed.");
}));
ctl.output.result(results, () => print("Start all operation completed."));
},
}));
@@ -646,27 +656,34 @@ instanceCommands.add(new lib.Command({
.filter((instance: lib.InstanceDetails) => ["starting", "running"].includes(instance.status));
if (instancesToStop.length === 0) {
print("No instances to stop.");
ctl.output.result([], () => print("No instances to stop."));
return;
}
print(`Stopping ${instancesToStop.length} instance(s):`);
for (const instance of instancesToStop) {
print(` - ${instance.name} (ID: ${instance.id})`);
const human = ctl.output.human;
if (human) {
print(`Stopping ${instancesToStop.length} instance(s):`);
for (const instance of instancesToStop) {
print(` - ${instance.name} (ID: ${instance.id})`);
}
}
// Stop all filtered instances
const stopPromises = instancesToStop.map(async (instance: lib.InstanceDetails) => {
const results = await Promise.all(instancesToStop.map(async (instance: lib.InstanceDetails) => {
try {
await ctl.sendTo({ instanceId: instance.id }, new lib.InstanceStopRequest());
print(`✓ Stopped ${instance.name}`);
if (human) {
print(`✓ Stopped ${instance.name}`);
}
return { id: instance.id, name: instance.name };
} catch (error) {
print(`✗ Failed to stop ${instance.name}: ${error}`);
if (human) {
print(`✗ Failed to stop ${instance.name}: ${error}`);
}
return { id: instance.id, name: instance.name, error: errorToJson(error) };
}
});
await Promise.all(stopPromises);
print("Stop all operation completed.");
}));
ctl.output.result(results, () => print("Stop all operation completed."));
},
}));
@@ -703,6 +720,6 @@ instanceCommands.add(new lib.Command({
);
// Factorio includes a newline in its response output.
process.stdout.write(result);
ctl.output.result({ result }, () => process.stdout.write(result));
},
}));
+13 -10
View File
@@ -27,6 +27,7 @@ logCommands.add(new lib.Command({
let hostIds = args.host ? [await lib.resolveHost(ctl, args.host)] : [];
await ctl.setLogSubscriptions({ all: args.all, controller: args.controller, hostIds, instanceIds });
ctl.keepOpen = true;
ctl.output.streamingLog = true;
},
}));
@@ -75,16 +76,18 @@ logCommands.add(new lib.Command({
result.log.reverse();
}
let stdoutLogger = winston.createLogger({
level: "verbose",
levels,
format: new lib.TerminalFormat({ showTimestamp: true }),
transports: [
new ConsoleTransport({ errorLevels: [], warnLevels: [] }),
],
ctl.output.result(result.log, () => {
let stdoutLogger = winston.createLogger({
level: "verbose",
levels,
format: new lib.TerminalFormat({ showTimestamp: true }),
transports: [
new ConsoleTransport({ errorLevels: [], warnLevels: [] }),
],
});
for (let info of result.log) {
stdoutLogger.log(info as any);
}
});
for (let info of result.log) {
stdoutLogger.log(info as any);
}
},
}));
+32 -43
View File
@@ -1,5 +1,4 @@
import fs from "node:fs/promises";
import asTableModule from "as-table";
import events from "events";
import path from "path";
import stream from "stream";
@@ -8,9 +7,7 @@ import util from "util";
import * as lib from "@clusterio/lib";
import { logger } from "@clusterio/lib";
import type { Ctl } from "../ctl.js";
import { print } from "./command_ops.js";
const asTable = asTableModule.configure({ delimiter: " | " });
import { asTable, pickFields, print } from "./command_ops.js";
const finished = util.promisify(stream.finished);
export const modCommands = new lib.CommandTree({ name: "mod", description: "Manage uploaded mods" });
@@ -24,16 +21,18 @@ modCommands.add(new lib.Command({
throw new lib.CommandError("mod-version must match format digit.digit.digit");
}
let modInfo = await ctl.send(new lib.ModGetRequest(args.name, args.modVersion));
for (let [field, value] of Object.entries(modInfo)) {
if (value instanceof Array) {
print(`${field}:`);
for (let entry of value) {
print(` ${entry}`);
ctl.output.result(modInfo, () => {
for (let [field, value] of Object.entries(modInfo)) {
if (value instanceof Array) {
print(`${field}:`);
for (let entry of value) {
print(` ${entry}`);
}
} else {
print(`${field}: ${value}`);
}
} else {
print(`${field}: ${value}`);
}
}
});
},
}));
@@ -50,17 +49,7 @@ modCommands.add(new lib.Command({
}],
handler: async function(args: { fields: string[] }, ctl: Ctl) {
let mods = await ctl.send(new lib.ModListRequest());
if (!args.fields.includes("all")) {
for (let entry of mods) {
for (let field of Object.keys(entry)) {
if (!args.fields.includes(field)) {
// @ts-expect-error terrible hack
delete entry[field];
}
}
}
}
print(asTable(mods));
ctl.output.table(mods, mod => pickFields(mod, args.fields));
},
}));
@@ -121,21 +110,19 @@ modCommands.add(new lib.Command({
args.sortOrder
));
let results = response.results.flatMap(result => result.versions);
if (!args.fields.includes("all")) {
for (let entry of results) {
for (let field of Object.keys(entry)) {
if (!args.fields.includes(field)) {
// @ts-expect-error terrible hack
delete entry[field];
}
}
ctl.output.result({
page: args.page,
pageCount: response.pageCount,
resultCount: response.resultCount,
queryIssues: response.queryIssues,
results,
}, () => {
for (let issue of response.queryIssues) {
print(issue);
}
}
for (let issue of response.queryIssues) {
print(issue);
}
print(`page ${args.page} of ${response.pageCount} (${response.resultCount} results)`);
print(asTable(results));
print(`page ${args.page} of ${response.pageCount} (${response.resultCount} results)`);
print(asTable(results.map(mod => pickFields(mod, args.fields))));
});
},
}));
@@ -174,14 +161,16 @@ modCommands.add(new lib.Command({
logger.error(requestError);
}
if (result.mods && result.mods.length) {
const mod = lib.ModInfo.fromJSON(result.mods[0]);
logger.info(`Successfully uploaded ${mod.filename}`);
}
if ((result.errors || []).length || (result.request_errors || []).length) {
throw new lib.CommandError("Uploading mod failed");
}
const uploaded = result.mods?.length ? lib.ModInfo.fromJSON(result.mods[0]).filename : undefined;
ctl.output.result({ filename: uploaded }, () => {
if (uploaded) {
print(`Successfully uploaded ${uploaded}`);
}
});
},
}));
@@ -201,7 +190,7 @@ modCommands.add(new lib.Command({
let filename = `${args.name}_${args.modVersion}.zip`;
await lib.downloadFile(url, filename, "overwrite");
logger.info(`Downloaded ${filename}`);
ctl.output.result({ filename }, () => print(`Downloaded ${filename}`));
},
}));
+49 -51
View File
@@ -1,11 +1,7 @@
import asTableModule from "as-table";
import * as lib from "@clusterio/lib";
import { logger } from "@clusterio/lib";
import type { Ctl } from "../ctl.js";
import { print } from "./command_ops.js";
const asTable = asTableModule.configure({ delimiter: " | " });
import { pickFields, print } from "./command_ops.js";
export const modPackCommands = new lib.CommandTree({ name: "mod-pack", description: "Mod Pack" });
modPackCommands.add(new lib.Command({
@@ -19,56 +15,50 @@ modPackCommands.add(new lib.Command({
let modPack = await ctl.send(
new lib.ModPackGetRequest(await lib.resolveModPack(ctl, args.modPack))
);
for (let [field, value] of Object.entries(modPack)) {
if (field === "mods") {
print(`${field}:`);
for (let [name, entry] of value) {
const mod = mods.get(`${name}_${entry.version}`);
const missing = !mod && "? ";
const badChecksum = mod && entry.sha1 && entry.sha1 !== mod.sha1 && "! ";
const warn = missing || badChecksum || "";
const enabled = entry.enabled ? "" : "(disabled) ";
print(` ${warn}${enabled}${name} ${entry.version}${entry.sha1 ? ` (${entry.sha1})` : ""}`);
}
} else if (field === "settings") {
print(`${field}:`);
for (let [scope, settings] of Object.entries(value)) {
print(` ${scope}:`);
for (let [setting, settingValue] of settings as any) {
print(` ${setting}: ${JSON.stringify(settingValue.value)}`);
}
}
} else if (field === "exportManifest") {
print(`${field}:`);
if (value && value.assets) {
print(" assets:");
for (let [name, fileName] of Object.entries(value.assets)) {
print(` ${name}: ${fileName}`);
}
}
} else {
print(`${field}: ${value}`);
}
}
ctl.output.result(modPack, () => printModPack(modPack, mods));
},
}));
function printModPack(modPack: lib.ModPack, mods: Map<string, lib.ModInfo>) {
for (let [field, value] of Object.entries(modPack)) {
if (field === "mods") {
print(`${field}:`);
for (let [name, entry] of value) {
const mod = mods.get(`${name}_${entry.version}`);
const missing = !mod && "? ";
const badChecksum = mod && entry.sha1 && entry.sha1 !== mod.sha1 && "! ";
const warn = missing || badChecksum || "";
const enabled = entry.enabled ? "" : "(disabled) ";
print(` ${warn}${enabled}${name} ${entry.version}${entry.sha1 ? ` (${entry.sha1})` : ""}`);
}
} else if (field === "settings") {
print(`${field}:`);
for (let [scope, settings] of Object.entries(value)) {
print(` ${scope}:`);
for (let [setting, settingValue] of settings as any) {
print(` ${setting}: ${JSON.stringify(settingValue.value)}`);
}
}
} else if (field === "exportManifest") {
print(`${field}:`);
if (value && value.assets) {
print(" assets:");
for (let [name, fileName] of Object.entries(value.assets)) {
print(` ${name}: ${fileName}`);
}
}
} else {
print(`${field}: ${value}`);
}
}
}
modPackCommands.add(new lib.Command({
definition: [["list", "l"], "List mod packs in the cluster"],
handler: async function(args: object, ctl: Ctl) {
let modPacks = await ctl.send(new lib.ModPackListRequest());
let fields = ["id", "name", "factorioVersion"];
for (let entry of modPacks) {
for (let field of Object.keys(entry)) {
if (!fields.includes(field)) {
// @ts-expect-error terrible hack
delete entry[field];
}
}
}
print(asTable(modPacks));
ctl.output.table(modPacks, modPack => pickFields(modPack, ["id", "name", "factorioVersion"]));
},
}));
@@ -194,10 +184,17 @@ modPackCommands.add(new lib.Command({
setModPackModsEnabled(modPack, args.disabledMods, false);
setModPackSettings(modPack, args);
await ctl.send(new lib.ModPackCreateRequest(modPack));
print(`Created mod pack ${modPack.name} (${modPack.id})`);
printCreated(ctl, modPack);
},
}));
function printCreated(ctl: Ctl, modPack: lib.ModPack) {
ctl.output.result(
{ id: modPack.id, name: modPack.name },
() => print(`Created mod pack ${modPack.name} (${modPack.id})`),
);
}
modPackCommands.add(new lib.Command({
definition: ["import <string>", "Import mod pack string", (yargs) => {
yargs.positional("string", { describe: "Mod pack string to import", type: "string" });
@@ -205,7 +202,7 @@ modPackCommands.add(new lib.Command({
handler: async function(args: { string: string }, ctl: Ctl) {
const modPack = lib.ModPack.fromModPackString(args.string);
await ctl.send(new lib.ModPackCreateRequest(modPack));
print(`Created mod pack ${modPack.name} (${modPack.id})`);
printCreated(ctl, modPack);
},
}));
@@ -217,7 +214,8 @@ modPackCommands.add(new lib.Command({
const modPack = await ctl.send(
new lib.ModPackGetRequest(await lib.resolveModPack(ctl, args.modPack))
);
print(modPack.toModPackString());
const modPackString = modPack.toModPackString();
ctl.output.result({ modPackString }, () => print(modPackString));
},
}));
+3 -7
View File
@@ -1,11 +1,7 @@
import asTableModule from "as-table";
import * as lib from "@clusterio/lib";
import type { Ctl } from "../ctl.js";
import { print } from "./command_ops.js";
const asTable = asTableModule.configure({ delimiter: " | " });
export const noteCommands = new lib.CommandTree({
name: "note", description: "Notes attached to hosts, instances, users and other resources",
});
@@ -20,12 +16,12 @@ noteCommands.add(new lib.Command({
if (args.type !== undefined) {
notes = notes.filter(note => note.resourceType === args.type);
}
print(asTable(notes.map(note => ({
ctl.output.table(notes, note => ({
type: note.resourceType,
id: note.resourceId,
updatedBy: note.updatedBy,
content: note.content,
}))));
}));
},
}));
@@ -42,7 +38,7 @@ noteCommands.add(new lib.Command({
if (!note) {
throw new lib.CommandError(`No note for ${args.type} ${args.id}`);
}
print(note.content);
ctl.output.result(note, () => print(note.content));
},
}));
+1 -7
View File
@@ -1,16 +1,10 @@
import asTableModule from "as-table";
import * as lib from "@clusterio/lib";
import type { Ctl } from "../ctl.js";
import { print } from "./command_ops.js";
const asTable = asTableModule.configure({ delimiter: " | " });
export const permissionCommands = new lib.CommandTree({ name: "permission", description: "Permission inspection" });
permissionCommands.add(new lib.Command({
definition: [["list", "l"], "List permissions in the cluster"],
handler: async function(args: object, ctl: Ctl) {
let permissions = await ctl.send(new lib.PermissionListRequest());
print(asTable(permissions));
ctl.output.table(await ctl.send(new lib.PermissionListRequest()));
},
}));
+3 -5
View File
@@ -1,15 +1,13 @@
import * as lib from "@clusterio/lib";
import { logger } from "@clusterio/lib";
import type { Ctl } from "../ctl.js";
import { asTable, print } from "./command_ops.js";
import { print } from "./command_ops.js";
export const roleCommands = new lib.CommandTree({ name: "role", description: "Role management" });
roleCommands.add(new lib.Command({
definition: [["list", "l"], "List roles in the cluster"],
handler: async function(args: object, ctl: Ctl) {
let roles = await ctl.send(new lib.RoleListRequest());
print(asTable(roles));
ctl.output.table(await ctl.send(new lib.RoleListRequest()));
},
}));
@@ -27,7 +25,7 @@ roleCommands.add(new lib.Command({
args.description,
args.permissions,
));
logger.info(`Created role ID ${id}`);
ctl.output.result({ id }, () => print(`Created role ID ${id}`));
},
}));
+42 -48
View File
@@ -5,6 +5,10 @@ import * as lib from "@clusterio/lib";
import type { Ctl } from "../ctl.js";
import { asTable, print } from "./command_ops.js";
function printProperties(object: object) {
print(asTable(Object.entries(object).map(([property, value]) => ({ property, value }))));
}
export const userCommands = new lib.CommandTree({ name: "user", alias: ["u"], description: "User management" });
userCommands.add(new lib.Command({
definition: ["show <name>", "Show details for one user", (yargs) => {
@@ -14,30 +18,23 @@ userCommands.add(new lib.Command({
});
}],
handler: async function(args: { name: string, instanceStats: boolean }, ctl: Ctl) {
let user = await ctl.send(new lib.UserGetRequest(args.name));
Object.assign(user, user.playerStats);
// @ts-expect-error Terrible hack
delete user.playerStats;
let instanceStats = user.instanceStats;
// @ts-expect-error Terrible hack
delete user.instanceStats;
print(asTable(Object.entries(user).map(([property, value]) => ({ property, value }))));
if (args.instanceStats) {
let instances = await ctl.send(new lib.InstanceDetailsListRequest());
function instanceName(id: number) {
let instance = instances.find(i => i.id === id);
if (instance) {
return instance.name;
}
return "<deleted>";
const user = await ctl.send(new lib.UserGetRequest(args.name));
const { playerStats, instanceStats, ...properties } = user;
const instances = args.instanceStats && ctl.output.human
? await ctl.send(new lib.InstanceDetailsListRequest())
: [];
ctl.output.result(user, () => {
printProperties({ ...properties, ...playerStats });
if (!args.instanceStats) {
return;
}
for (let [id, playerInstanceStats] of instanceStats || []) {
for (let [id, playerInstanceStats] of instanceStats) {
const name = instances.find(i => i.id === id)?.name ?? "<deleted>";
print();
print(`Instance ${instanceName(id)} (${id}):`);
print(asTable(Object.entries(playerInstanceStats).map(([property, value]) => ({ property, value }))));
print(`Instance ${name} (${id}):`);
printProperties(playerInstanceStats);
}
}
});
},
}));
@@ -49,29 +46,17 @@ userCommands.add(new lib.Command({
});
}],
handler: async function(args: { stats: boolean, attributes: boolean }, ctl: Ctl) {
let users = await ctl.send(new lib.UserListRequest());
for (let user of users) {
if (args.stats) {
Object.assign(user, user.playerStats);
}
// @ts-expect-error Terrible hack
delete user.playerStats;
// @ts-expect-error Terrible hack
delete user.isDeleted;
// @ts-expect-error Terrible hack
delete user.banReason;
// @ts-expect-error Terrible hack
delete user.instanceStats;
if (!args.attributes) {
// @ts-expect-error Terrible hack
delete user.isAdmin;
// @ts-expect-error Terrible hack
delete user.isWhitelisted;
// @ts-expect-error Terrible hack
delete user.isBanned;
}
}
print(asTable(users));
const users = await ctl.send(new lib.UserListRequest());
ctl.output.table(users, user => {
const {
playerStats, isDeleted, banReason, instanceStats, isAdmin, isWhitelisted, isBanned, ...row
} = user;
return {
...row,
...args.attributes && { isAdmin, isWhitelisted, isBanned },
...args.stats && playerStats,
};
});
},
}));
@@ -219,11 +204,17 @@ async function handleImportOrRestore(args: {
new lib.UserBulkImportRequest(importType, importType === "users" ? data.users : data, restore)
);
print(`${restore ? "Restored" : "Imported"} ${importType} from ${args.filepath}`);
let backupFile: string | undefined;
if (restore && backup && !args.noBackup) {
await fs.writeFile(`${importType}-backup.json`, JSON.stringify(backup, undefined, "\t"));
print(`Wrote backup to ${importType}-backup.json`);
backupFile = `${importType}-backup.json`;
await fs.writeFile(backupFile, JSON.stringify(backup, undefined, "\t"));
}
ctl.output.result({ type: importType, backupFile }, () => {
print(`${restore ? "Restored" : "Imported"} ${importType} from ${args.filepath}`);
if (backupFile) {
print(`Wrote backup to ${backupFile}`);
}
});
}
userCommands.add(new lib.Command({
@@ -296,6 +287,9 @@ userCommands.add(new lib.Command({
await fs.writeFile(args.filepath, JSON.stringify(await ctl.send(
new lib.UserBulkExportRequest(exportType)
), undefined, "\t"));
print(`Exported ${exportType} to ${args.filepath}`);
ctl.output.result(
{ type: exportType, filepath: args.filepath },
() => print(`Exported ${exportType} to ${args.filepath}`),
);
},
}));
+23
View File
@@ -1,4 +1,6 @@
import * as lib from "@clusterio/lib";
import type { Ctl } from "../ctl.js";
import { print } from "./command_ops.js";
export async function getEditor(argsEditor: string) {
// eslint-disable-next-line
@@ -53,3 +55,24 @@ export function serializedConfigToString(
}
return allConfigElements;
}
/** Print config fields as `name value` lines, or as an object in json mode */
export function printConfig(ctl: Ctl, config: lib.ConfigSchema, note = (name: string) => "") {
ctl.output.result(config, () => {
for (let [name, value] of Object.entries(config)) {
// Shown as an (inherited) note on each field by instance config list instead
if (name !== "instance.inherited_fields") {
print(`${name} ${JSON.stringify(value)}${note(name)}`);
}
}
});
}
/** Print whether a restart is still needed after an update or install */
export function printRestartRequired(ctl: Ctl, restartRequired: boolean, action: string) {
ctl.output.result({ restartRequired }, () => {
if (restartRequired) {
print(`${action}; a restart is required to apply the changes.`);
}
});
}
+2 -4
View File
@@ -17,7 +17,6 @@ import path from "path";
import yargs from "yargs";
import packageConfig from "@clusterio/host/package.json" with { type: "json" };
import winston from "winston";
import "winston-daily-rotate-file";
// internal libraries
// Reduce startup time by lazy compiling schemas.
@@ -147,11 +146,10 @@ async function startHost() {
.parseSync()
;
logger.add(new winston.transports.DailyRotateFile({
logger.add(new lib.DailyFileTransport({
format: winston.format.json(),
filename: "host-%DATE%.log",
utc: true,
dirname: path.join(args.logDirectory, "host"),
prefix: "host",
}));
if (args.logLevel !== "none") {
logger.add(new ConsoleTransport({
+1 -2
View File
@@ -1,6 +1,6 @@
{
"name": "@clusterio/host",
"version": "2.1.0",
"version": "2.1.1",
"description": "Implementation of Clusterio host server",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Hornwitser <github@hornwitser.no>",
@@ -49,7 +49,6 @@
"semver": "catalog:",
"set-blocking": "catalog:",
"winston": "catalog:",
"winston-daily-rotate-file": "catalog:",
"yargs": "catalog:",
"yazl": "catalog:"
},
+33
View File
@@ -1054,6 +1054,39 @@ end`.replace(/\r?\n/g, " ");
}
await this.syncMods();
this.checkRequiredMods(this.activeModPack);
}
/**
* Check that the mod pack has the mods required by the loaded plugins
*
* @throws {lib.ConflictError} if a required mod is missing, disabled or the wrong version.
*/
checkRequiredMods(modPack: lib.ModPack) {
const problems = [];
for (const pluginInfo of this.loadedPlugins) {
const unmet = [];
for (const dependency of lib.pluginRequiredMods(pluginInfo)) {
const mod = modPack.mods.get(dependency.name);
if (!mod) {
unmet.push(`${dependency.spec} (missing)`);
} else if (!mod.enabled) {
unmet.push(`${dependency.spec} (disabled)`);
} else if (dependency.version && !dependency.version.testVersion(mod.version)) {
unmet.push(`${dependency.spec} (found ${mod.version})`);
}
}
if (unmet.length) {
problems.push(`Plugin ${pluginInfo.name} requires ${unmet.join(", ")}.`);
}
}
if (problems.length) {
throw new lib.ConflictError(
`Mod pack ${modPack.name} does not have the mods required by the loaded plugins. ` +
`${problems.join(" ")} ` +
"Add the mods to the mod pack or set <plugin>.load_plugin to false for this instance."
);
}
}
/**
+11 -4
View File
@@ -1,6 +1,6 @@
{
"name": "@clusterio/lib",
"version": "2.1.0",
"version": "2.1.1",
"description": "Shared library for Clusterio",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Hornwitser <github@hornwitser.no>",
@@ -31,6 +31,13 @@
},
"import": "./src/link/WebSocket.browser.ts"
},
"#zlib_sync": {
"node": {
"types": "./src/zlib_sync.node.ts",
"default": "./dist/node/src/zlib_sync.node.js"
},
"import": "./src/zlib_sync.browser.ts"
},
"#is_deep_strict_equal": {
"node": {
"types": "./src/is_deep_strict_equal.node.ts",
@@ -51,11 +58,9 @@
"dependencies": {
"@sinclair/typebox": "catalog:",
"@types/yauzl": "catalog:",
"@types/yazl": "catalog:",
"ajv": "^8.20.0",
"chalk": "catalog:",
"fast-deep-equal": "^3.1.3",
"fflate": "catalog:",
"set-blocking": "catalog:",
"triple-beam": "^1.4.1",
"winston": "catalog:",
@@ -71,6 +76,8 @@
"@types/triple-beam": "^1.3.5",
"@types/ws": "catalog:",
"@types/yargs": "catalog:",
"@types/yauzl": "catalog:",
"@types/yazl": "catalog:",
"typescript": "catalog:"
}
}
+39 -1
View File
@@ -4,6 +4,8 @@ import events from "events";
import { Static } from "@sinclair/typebox";
import { logger } from "./logging.js";
import { downloadFile, atomicOverwriteFile } from "./file_ops.js";
import { hashFile } from "./hash.js";
import { ZipArchive, findRoot } from "./zip_ops.js";
import { ModPortalReleaseSchema, ModPortalDetailsSchema, ModNameVersionPair } from "./data/messages_mod.js";
import { ModInfo, ModPack, MajorMinorVersion, ModVersionEquality, SourceVersion } from "./data/index.js";
@@ -54,8 +56,44 @@ export default class ModStore extends events.EventEmitter<ModStoreEvents> {
mods() { return this.files.values(); }
/**
* Read the info.json of a mod zip file along with its size, mtime and hash
*
* @param modPath - Path to the mod zip file.
* @returns Mod info for the file.
*/
static async readModInfo(modPath: string) {
let modInfo: Static<typeof ModInfo.jsonSchema>;
let zip = await ZipArchive.fromFile(modPath);
try {
let content = await zip.readFile(path.posix.join(findRoot(zip), "info.json"));
if (content === null) {
throw new Error("Mod contains no info.json file");
}
modInfo = JSON.parse(content.toString("utf8"));
} finally {
zip.close();
}
let valid = ModInfo.validateInfo(modInfo);
if (!valid) {
throw new Error("Mod's info.json is not valid");
}
const stat = await fs.stat(modPath);
return ModInfo.fromJSON({
...modInfo,
size: stat.size,
sha1: await hashFile(modPath),
mtime_ms: stat.mtimeMs,
updated_at_ms: stat.mtimeMs,
is_deleted: false,
});
}
static async loadFile(modsDirectory: string, file: string) {
let modInfo = await ModInfo.fromModFile(path.join(modsDirectory, file));
let modInfo = await ModStore.readModInfo(path.join(modsDirectory, file));
if (modInfo.filename !== file) {
throw new Error(`filename does not match the expected name ${modInfo.filename}.`);
}
-34
View File
@@ -1,10 +1,6 @@
import fs from "node:fs/promises";
import path from "path";
import { Type, Static } from "@sinclair/typebox";
import * as libHash from "../hash.js";
import * as libSchema from "../schema.js";
import { ZipArchive, findRoot } from "../zip_ops.js";
import { ModRecord } from "./ModPack.js";
import {
@@ -349,34 +345,4 @@ export default class ModInfo {
if (this.isDeleted) { json.is_deleted = this.isDeleted; }
return json;
}
static async fromModFile(modPath: string) {
let modInfo: Static<typeof ModInfo.jsonSchema>;
let zip = await ZipArchive.fromFile(modPath);
try {
let content = await zip.readFile(path.posix.join(findRoot(zip), "info.json"));
if (content === null) {
throw new Error("Mod contains no info.json file");
}
modInfo = JSON.parse(content.toString("utf8"));
} finally {
zip.close();
}
let valid = this.validateInfo(modInfo);
if (!valid) {
throw new Error("Mod's info.json is not valid");
}
const stat = await fs.stat(modPath);
return this.fromJSON({
...modInfo,
size: stat.size,
sha1: await libHash.hashFile(modPath),
mtime_ms: stat.mtimeMs,
updated_at_ms: stat.mtimeMs,
is_deleted: false,
});
}
}
+2 -2
View File
@@ -1,4 +1,4 @@
import zlib from "zlib";
import * as zlib from "#zlib_sync";
import { Type, Static } from "@sinclair/typebox";
import * as libSchema from "../schema.js";
@@ -306,7 +306,7 @@ export default class ModPack {
// eslint-disable-next-line node/no-sync
let buf = zlib.deflateSync(JSON.stringify(json));
return buf.toString("base64");
return Buffer.from(buf).toString("base64");
}
static fromModPackString(modPackString: string) {
+1 -1
View File
@@ -1,4 +1,4 @@
import zlib from "zlib";
import * as zlib from "#zlib_sync";
class MapReaderState {
pos = 0;
+6
View File
@@ -0,0 +1,6 @@
/** Throws if condition is falsy, node:assert is not available in the browser. */
export function assert(condition: unknown): asserts condition {
if (!condition) {
throw new Error("Assertion failed");
}
}
+1 -1
View File
@@ -1,5 +1,4 @@
// Connection adapters for Links
import { strict as assert } from "assert";
import events from "events";
import { WebSocket } from "#WebSocket";
@@ -7,6 +6,7 @@ import * as libData from "../data/index.js";
import * as libErrors from "../errors.js";
import { logger } from "../logging.js";
import ExponentialBackoff from "../ExponentialBackoff.js";
import { assert } from "./assert.js";
import type { Request, Event } from "./link.js";
/**
+1 -1
View File
@@ -5,7 +5,7 @@ import { logger } from "../logging.js";
import * as libSchema from "../schema.js";
import { dataClasses, errorClasses } from "./messages.js";
import { BaseConnector, WebSocketBaseConnector } from "./connectors.js";
import { strict as assert } from "assert";
import { assert } from "./assert.js";
import type { PluginNodeEnvInfo, PluginWebEnvInfo } from "../plugin.js";
import type {
AddressType, JSONDeserialisable, MessageRoutable, MessageRequest, MessageEvent, IUser,
+61 -12
View File
@@ -5,14 +5,14 @@
* @module lib/logging_utils
*/
import fs from "node:fs/promises";
import type { ReadStream } from "node:fs";
import { type ReadStream, type WriteStream, createWriteStream, mkdirSync } from "node:fs";
import os from "node:os";
import path from "path";
import stream from "stream";
import util from "util";
import util, { styleText } from "util";
import winston from "winston";
import Transport from "winston-transport";
import { LEVEL, MESSAGE } from "triple-beam";
import chalk from "chalk";
import * as libData from "./data/index.js";
import * as libErrors from "./errors.js";
@@ -108,30 +108,30 @@ export type ParsedFactorioOutput =
function formatServerOutput(parsed: ParsedFactorioOutput) {
let time = "";
if (parsed.format === "seconds") {
time = `${chalk.yellow(parsed.time.padStart(8))} `;
time = `${styleText("yellow", parsed.time.padStart(8))} `;
} else if (parsed.format === "date") {
time = `${chalk.yellow(parsed.time)} `;
time = `${styleText("yellow", parsed.time)} `;
}
let info = "";
if (parsed.type === "log") {
let level = parsed.level;
if (level === "Script") {
level = chalk.bold.greenBright(level);
level = styleText(["bold", "greenBright"], level);
} else if (level === "Verbose") {
level = chalk.bold.gray(level);
level = styleText(["bold", "gray"], level);
} else if (level === "Info") {
level = chalk.bold.blueBright(level);
level = styleText(["bold", "blueBright"], level);
} else if (parsed.level === "Warning") {
level = chalk.bold.yellowBright(level);
level = styleText(["bold", "yellowBright"], level);
} else if (parsed.level === "Error") {
level = chalk.bold.redBright(level);
level = styleText(["bold", "redBright"], level);
}
info = `${level} ${chalk.gray(parsed.file)}: `;
info = `${level} ${styleText("gray", parsed.file)}: `;
} else if (parsed.type === "action") {
info = `[${chalk.yellow(parsed.action)}] `;
info = `[${styleText("yellow", parsed.action)}] `;
}
return time + info + parsed.message;
@@ -226,6 +226,55 @@ export class LinkTransport extends Transport {
}
}
/**
* Writes log entries as lines to dirname/prefix-YYYY-MM-DD.log, opening
* a new file when the UTC date changes.
*/
export class DailyFileTransport extends Transport {
dirname: string;
prefix: string;
now: () => number;
date?: string;
stream?: WriteStream;
constructor(
options: Transport.TransportStreamOptions & {
dirname: string,
prefix: string,
now?: () => number,
}
) {
super(options);
this.dirname = options.dirname;
this.prefix = options.prefix;
this.now = options.now ?? Date.now;
// eslint-disable-next-line node/no-sync
mkdirSync(this.dirname, { recursive: true });
}
log(info: any, callback: () => void) {
const date = new Date(this.now()).toISOString().slice(0, 10);
if (!this.stream || date !== this.date) {
this.stream?.end();
this.date = date;
this.stream = createWriteStream(path.join(this.dirname, `${this.prefix}-${date}.log`), { flags: "a" });
this.stream.on("error", err => this.emit("error", err));
}
this.stream.write(info[MESSAGE] + os.EOL);
return callback();
}
close(callback?: () => void) {
const fileStream = this.stream;
this.stream = undefined;
if (fileStream) {
fileStream.end(callback);
} else {
callback?.();
}
}
}
const logFileGlob = /^[a-z]+-(\d{4}-\d{2}-\d{2})\.log$/;
const logIndexVersion = 2;
+23
View File
@@ -9,6 +9,7 @@ import type { PermissionDefinition } from "./permissions.js";
import type { PlayerStats, Role } from "./data/index.js";
import type { SyncHook } from "./SyncHook.js";
import type { ErrorClass } from "./link/index.js";
import { ModDependency } from "./data/ModInfo.js";
export const PluginFeatureFlags = [
/** The plugin requires module code to be patched into the save */
@@ -41,6 +42,11 @@ export type PluginDeclaration = {
migrationEntrypoint?: string;
features?: (typeof PluginFeatureFlags)[number][];
/**
* Factorio mods the plugin needs in the instance's mod pack, as Factorio
* dependency strings like `"subspace_storage >= 2.3.0"`.
*/
requiredMods?: string[];
messages?: any[];
errors?: ErrorClass[];
@@ -65,6 +71,23 @@ export function pluginNeedsWebBuild(info: PluginDeclaration) {
);
}
/**
* Parse the requiredMods of a plugin
*
* A mod listed more than once must satisfy every entry, which gives a range.
*
* @throws Error if an entry is invalid or has a dependency prefix.
*/
export function pluginRequiredMods(pluginInfo: PluginDeclaration) {
return (pluginInfo.requiredMods ?? []).map(spec => {
const dependency = new ModDependency(spec);
if (dependency.type !== "required") {
throw new Error(`requiredMods entry "${spec}" can not have a dependency prefix`);
}
return dependency;
});
}
export type PluginNodeEnvInfo = PluginDeclaration & {
/**
* Path to the folder with the static files that should be hosted on the web
+6
View File
@@ -84,6 +84,12 @@ export async function loadPluginInfos(pluginList: Map<string, string>, skippedNp
);
}
try {
libPlugin.pluginRequiredMods(pluginInfo);
} catch (err: any) {
throw new libErrors.PluginError(pluginName, err);
}
// migrate: ignore incompatible old plugins
if (pluginInfo.messages && !(pluginInfo.messages instanceof Array)) {
logger.warn(`Ignoring incompatible pre alpha.14 plugin ${pluginName}`);
+42 -10
View File
@@ -7,10 +7,31 @@ import { PassThrough, type Readable } from "stream";
import util from "util";
import zlib from "zlib";
import yauzl from "yauzl";
import type yazl from "yazl";
const deflateRaw = util.promisify(zlib.deflateRaw);
/**
* Entry in a ZipArchive
*
* Local subset of yauzl.Entry so the published declarations do not import
* yauzl, which would make @types/yauzl a runtime dependency of lib.
*/
export interface ZipEntry {
fileName: string;
fileComment: string;
versionMadeBy: number;
versionNeededToExtract: number;
generalPurposeBitFlag: number;
compressionMethod: number;
lastModFileTime: number;
lastModFileDate: number;
crc32: number;
compressedSize: number;
uncompressedSize: number;
externalFileAttributes: number;
getLastModDate(options?: { timezone?: "local" | "UTC" | null, forceDosFormat?: boolean }): Date;
}
/**
* Zip archive opened for random access reading
@@ -21,10 +42,9 @@ const deflateRaw = util.promisify(zlib.deflateRaw);
*/
export class ZipArchive {
private constructor(
/** Underlying yauzl zip file. */
public zipfile: yauzl.ZipFile,
private zipfile: yauzl.ZipFile,
/** Entries by file name in the order they appear in the central directory. */
public entries: Map<string, yauzl.Entry>,
public entries: Map<string, ZipEntry>,
) { }
/**
@@ -77,8 +97,17 @@ export class ZipArchive {
*
* @param entry - Entry to read, must belong to this archive.
*/
openStream(entry: yauzl.Entry): Promise<Readable> {
return this.zipfile.openReadStreamPromise(entry);
openStream(entry: ZipEntry): Promise<Readable> {
return this.zipfile.openReadStreamPromise(entry as yauzl.Entry);
}
/**
* Open a stream of the content of an entry as stored in the archive
*
* @param entry - Entry to read, must belong to this archive.
*/
openRawStream(entry: ZipEntry): Promise<Readable> {
return this.zipfile.openReadStreamPromise(entry as yauzl.Entry, { decodeFileData: false });
}
/**
@@ -86,7 +115,7 @@ export class ZipArchive {
*
* @param entry - Entry to read, must belong to this archive.
*/
async readEntry(entry: yauzl.Entry) {
async readEntry(entry: ZipEntry) {
const chunks: Buffer[] = [];
for await (const chunk of await this.openStream(entry)) {
chunks.push(chunk);
@@ -153,7 +182,10 @@ export function findRoot(zip: ZipArchive) {
*
* @param zipFile - zip file being written.
*/
export function zipOutputStream(zipFile: yazl.ZipFile) {
export function zipOutputStream(zipFile: {
outputStream: NodeJS.ReadableStream,
on(event: "error", listener: (err: Error) => void): unknown,
}) {
const output = zipFile.outputStream as Readable;
zipFile.on("error", err => output.destroy(err));
return output;
@@ -224,7 +256,7 @@ export class ZipWriter {
* @param zip - Archive to copy from.
* @param entry - Entry to copy, must belong to zip.
*/
async addEntry(zip: ZipArchive, entry: yauzl.Entry) {
async addEntry(zip: ZipArchive, entry: ZipEntry) {
const writerEntry: WriterEntry = {
fileName: Buffer.from(entry.fileName, "utf8"),
fileComment: Buffer.from(entry.fileComment, "utf8"),
@@ -242,7 +274,7 @@ export class ZipWriter {
offset: this.offset,
};
await this.write(this.localFileHeader(writerEntry));
const stream = await zip.zipfile.openReadStreamPromise(entry, { decodeFileData: false });
const stream = await zip.openRawStream(entry);
for await (const chunk of stream) {
await this.write(chunk);
}
+15
View File
@@ -0,0 +1,15 @@
/* eslint-disable node/no-sync */
import { strToU8, unzlibSync, zlibSync } from "fflate";
export function inflateSync(data: Uint8Array): Uint8Array {
try {
return unzlibSync(data);
} catch (err: any) {
// Match the error code node:zlib gives for invalid data.
throw Object.assign(new Error(err.message), { code: "Z_DATA_ERROR" });
}
}
export function deflateSync(data: Uint8Array | string): Uint8Array {
return zlibSync(typeof data === "string" ? strToU8(data) : data);
}
+11
View File
@@ -0,0 +1,11 @@
import zlib from "zlib";
export function inflateSync(data: Uint8Array): Uint8Array {
// eslint-disable-next-line node/no-sync
return zlib.inflateSync(data);
}
export function deflateSync(data: Uint8Array | string): Uint8Array {
// eslint-disable-next-line node/no-sync
return zlib.deflateSync(data);
}
+1
View File
@@ -19,6 +19,7 @@
"./src/hash.ts",
"./src/ini.ts",
"./src/is_deep_strict_equal.browser.ts",
"./src/zlib_sync.browser.ts",
"./src/link/*.ts",
"./src/lua_tools.ts",
"./src/logging.ts",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@clusterio/web_ui",
"version": "2.1.0",
"version": "2.1.1",
"description": "Clusterio web interface implementation",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Hornwitser",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@clusterio/plugin-global_chat",
"version": "2.1.0",
"version": "2.1.1",
"description": "Clusterio plugin forwarding between Factorio servers",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Hornwitser <github@hornwitser.no>",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@clusterio/plugin-player_auth",
"version": "2.1.0",
"version": "2.1.1",
"description": "Clusterio plugin authenticating logged in players to the web interface",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Hornwitser <github@hornwitser.no>",
+1 -1
View File
@@ -6,7 +6,7 @@ As a player, it mostly just works. Most important to know is that factorio data
A note on crashing servers:
When a server crashes while you are online there won't be time to immediately upload the inventory. Instead, the inventory is uploaded as soon as the server restarts. This means if you have been playing on a different server from the one that crashed and gathered/consumed items but are offline at the time of the first server coming online again your inventory will reset to the autosave.
When a server crashes while you are online there won't be time to immediately upload the inventory. Instead, the inventory is uploaded as soon as the server restarts. If you have been playing on a different server in the meantime the controller keeps what that server uploaded, and the crashed server downloads it the next time you join it.
## Installation
+1
View File
@@ -258,6 +258,7 @@ export default async function(context: ControllerPluginContext) {
prefetch(instanceId, request.destinationInstanceId, merged);
}
}
return new msg.UploadRequest.Response(store);
});
controller.handle(msg.DownloadRequest, async (request: msg.DownloadRequest) => {
+51 -26
View File
@@ -30,14 +30,20 @@ playerSyncCommands.add(new lib.Command({
handler: async function(args: object, ctl: Ctl) {
let players = await ctl.send(new msg.ListPlayersRequest());
players.sort((a, b) => a.name.localeCompare(b.name));
print("name | generation | size | acquired by");
for (let player of players) {
print(
`${player.name} | ${player.generation ?? "-"} | ` +
`${player.size === undefined ? "-" : lib.formatBytes(player.size)} | ` +
`${await instanceName(ctl, player.instanceId)}`
);
if (!ctl.output.human) {
ctl.output.result(players);
return;
}
let rows = [];
for (let player of players) {
rows.push({
name: player.name,
generation: player.generation ?? "-",
size: player.size === undefined ? "-" : lib.formatBytes(player.size),
"acquired by": await instanceName(ctl, player.instanceId),
});
}
ctl.output.table(rows);
},
}));
@@ -47,24 +53,37 @@ playerSyncCommands.add(new lib.Command({
}],
handler: async function(args: { player: string }, ctl: Ctl) {
let response = await ctl.send(new msg.GetPlayerDataRequest(args.player));
print(`acquired by: ${await instanceName(ctl, response.instanceId)}`);
let playerData = response.playerData;
if (!playerData) {
print(`No player data stored for ${args.player}`);
return;
}
print(`generation: ${playerData.generation}`);
print(`size: ${lib.formatBytes(JSON.stringify(playerData).length)}`);
print(`controller: ${playerData.controller}`);
print(`force: ${playerData.force}`);
for (let inventory of summarizePlayerInventories(playerData)) {
print(`${inventory.name}:`);
for (let item of inventory.items) {
let quality = item.quality ? ` (${item.quality})` : "";
let size = item.exportSize ? ` ${lib.formatBytes(item.exportSize)}` : "";
print(` ${item.count} ${item.name}${quality}${size}`);
const summary = {
instanceId: response.instanceId,
...playerData && {
generation: playerData.generation,
size: JSON.stringify(playerData).length,
controller: playerData.controller,
force: playerData.force,
inventories: summarizePlayerInventories(playerData),
},
};
const acquiredBy = ctl.output.human ? await instanceName(ctl, response.instanceId) : "";
ctl.output.result(summary, () => {
print(`acquired by: ${acquiredBy}`);
if (!playerData) {
print(`No player data stored for ${args.player}`);
return;
}
}
print(`generation: ${summary.generation}`);
print(`size: ${lib.formatBytes(summary.size!)}`);
print(`controller: ${summary.controller}`);
print(`force: ${summary.force}`);
for (let inventory of summary.inventories!) {
print(`${inventory.name}:`);
for (let item of inventory.items) {
let quality = item.quality ? ` (${item.quality})` : "";
let size = item.exportSize ? ` ${lib.formatBytes(item.exportSize)}` : "";
print(` ${item.count} ${item.name}${quality}${size}`);
}
}
});
},
}));
@@ -81,9 +100,12 @@ playerSyncCommands.add(new lib.Command({
let content = JSON.stringify(response.playerData, null, "\t");
if (args.file) {
await fs.writeFile(args.file, content);
print(`Wrote player data for ${args.player} to ${args.file}`);
ctl.output.result(
{ file: args.file },
() => print(`Wrote player data for ${args.player} to ${args.file}`),
);
} else {
print(content);
ctl.output.result(response.playerData, () => print(content));
}
},
}));
@@ -101,7 +123,10 @@ playerSyncCommands.add(new lib.Command({
playerData.name = args.player;
playerData.generation ??= 0;
let response = await ctl.send(new msg.SetPlayerDataRequest(args.player, playerData));
print(`Stored player data for ${args.player} as generation ${response.generation}`);
ctl.output.result(
{ generation: response.generation },
() => print(`Stored player data for ${args.player} as generation ${response.generation}`),
);
},
}));
+6 -3
View File
@@ -200,11 +200,12 @@ export default async function(context: InstancePluginContext) {
delete player_data.destination_instance_id;
logger.verbose(`Uploading ${player_data.name} (${JSON.stringify(player_data).length / 1000}kB)`);
let stored: boolean;
try {
await instance.sendTo(
({ stored } = await instance.sendTo(
"controller",
new UploadRequest(instance.id, player_data.name, player_data, destinationInstanceId),
);
));
} catch (err: any) {
if (!(err instanceof lib.SessionLost)) {
@@ -213,8 +214,10 @@ export default async function(context: InstancePluginContext) {
return;
}
// A copy the controller did not store must not pass for the stored one: the save downloads at the next join
const reply = stored ? "confirm_upload" : "reject_upload";
await instance.sendRcon(
`/sc player_sync.confirm_upload("${player_data.name}", ${player_data.generation})`, true, plugin.name
`/sc player_sync.${reply}("${player_data.name}", ${player_data.generation})`, true, plugin.name
);
}
+18
View File
@@ -262,12 +262,30 @@ export class ReleaseRequest {
}
}
export class UploadResponse {
constructor(
/** False when the controller kept what it has: another copy is newer, locked or from another epoch */
public stored: boolean,
) {
}
// No type: a controller from before this response replies with nothing, which counts as stored
static jsonSchema = Type.Unsafe<{ stored: boolean } | undefined>({
properties: { "stored": { type: "boolean" } },
});
static fromJSON(json: Static<typeof UploadResponse.jsonSchema>) {
return new this(json?.stored ?? true);
}
}
export class UploadRequest {
declare ["constructor"]: typeof UploadRequest;
static type = "request" as const;
static src = "instance" as const;
static dst = "controller" as const;
static plugin = "player_sync" as const;
static Response = UploadResponse;
constructor(
public instanceId: number,
+32 -1
View File
@@ -384,12 +384,14 @@ function player_sync.check_players_in_cutscene()
end
-- Whether a join should take the stored data: there is some, and this save hasn't synced the player in that epoch at
-- that generation or newer with every component synced here now. A player without a record here has nothing to keep.
-- that generation or newer with every component synced here now, or the controller did not store its last upload
-- (reject_upload). A player without a record here has nothing to keep.
local function download_needed(player_record, acquire_response, script_data)
return acquire_response.has_data
and (
not player_record
or not player_record.sync
or player_record.outdated
or (player_record.epoch or 0) ~= (acquire_response.epoch or 0)
or acquire_response.generation > player_record.generation
or serialize.newly_synced(player_record.components, script_data.components)
@@ -543,6 +545,8 @@ function player_sync.sync_player(acquire_response)
start_sync(player_record, script_data.components)
player_record.dirty = true
player_record.sync_in_progress = false
-- Nothing stored to download: the copy here is all there is
player_record.outdated = nil
-- Deleted data keeps its generation and epoch so the next upload is accepted as the new baseline
player_record.epoch = epoch
-- A copy from another epoch has nothing to do with the stored generation
@@ -573,6 +577,7 @@ function player_sync.finish_download(player, finished_record)
player_record.dirty = true
player_record.sync_in_progress = false
player_record.outdated = nil
start_sync(player_record, finished_record.components)
player_record.generation = finished_record.generation
player_record.epoch = finished_record.epoch
@@ -725,6 +730,32 @@ function player_sync.confirm_upload(player_name, generation)
player_record.dirty = false
end
-- Invoked by the instance code when the controller did not store the uploaded data: it holds another copy that is
-- newer, locked by another instance or from another epoch. This save's copy lost, even at the same generation, so it
-- is not uploaded again and the next join downloads. A retry of an upload the controller already stored ends up here
-- too, and then downloads what this save sent.
function player_sync.reject_upload(player_name, generation)
local player = game.get_player(player_name)
if not player then
return
end
local script_data = get_script_data()
-- A player who is back already still waits for the acquire reply, which the controller sent after this one, and
-- that join then downloads. Past that reply the join has decided and this is not about the copy here any more.
if player.connected and not script_data.players_waiting_for_acquire[player_name] then
return
end
local player_record = script_data.players[player.name]
if not player_record or player_record.generation ~= generation then
return
end
log("Upload of " .. player_name .. " was not stored, the next join downloads")
script_data.active_uploads[player_name] = nil
player_record.dirty = false
player_record.outdated = true
end
function player_sync.initiate_inventory_download(player, player_record, generation, epoch)
player.print("Initiating inventory download...")
local record = {
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@clusterio/plugin-player_sync",
"version": "2.1.0",
"version": "2.1.1",
"description": "Clusterio plugin forwarding between Factorio servers",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Danielv <danielv@danielv.no>",
+12
View File
@@ -77,6 +77,18 @@ describe("player_sync controller", function() {
.playerData;
}
it("should reply whether the upload was stored", async function() {
const handler = mock.getHandler(controller, msg.UploadRequest);
const send = (instanceId, generation) => handler(
{ instanceId, playerName: "test", playerData: { generation, name: "test" } }
);
assert.deepEqual(await send(1, 2), new msg.UploadRequest.Response(true));
// A crashed instance restarting with the generation another instance already uploaded
assert.deepEqual(await send(1, 2), new msg.UploadRequest.Response(false));
await mock.getHandler(controller, msg.AcquireRequest)(new msg.AcquireRequest(1, "test"));
controller.instances.set(2, controller.instances.get(1));
assert.deepEqual(await send(2, 3), new msg.UploadRequest.Response(false));
});
it("should store the upload as is when it has no components", async function() {
const playerData = { generation: 2, name: "test", force: "enemy", tag: "uploaded" };
assert.deepEqual(await upload(playerData), playerData);
+20 -1
View File
@@ -5,7 +5,7 @@ import * as lib from "@clusterio/lib";
import * as mock from "../../../test/mock.js";
import loadInstancePlugin, { recipeNotificationDelta, applyRecipeNotificationDelta } from "../dist/node/instance.js";
import { AcquireResponse, DownloadResponse, PrefetchEvent } from "../dist/node/messages.js";
import { AcquireResponse, DownloadResponse, PrefetchEvent, UploadResponse } from "../dist/node/messages.js";
import { components } from "../dist/node/components.js";
import { plugin as info } from "../dist/node/index.js";
@@ -275,6 +275,25 @@ describe("player_sync", function() {
const request = await upload({ name: "test", generation: 2 });
assert.equal(request.destinationInstanceId, undefined);
});
async function rconAfterReply(stored) {
instance.server.reset();
instance.connector.once("send", message => {
instance.connector.emit("message", new lib.MessageResponse(
1, message.dst, message.src, new UploadResponse(stored)
));
});
instance.server.emit("ipc-player_sync_upload", { name: "test", generation: 2 });
for (let i = 0; i < 100 && !instance.server.rconCommands.length; i++) {
await lib.wait(1);
}
return instance.server.rconCommands;
}
it("should confirm an upload the controller stored", async function() {
assert.deepEqual(await rconAfterReply(true), ['/sc player_sync.confirm_upload("test", 2)']);
});
it("should reject an upload the controller did not store", async function() {
assert.deepEqual(await rconAfterReply(false), ['/sc player_sync.reject_upload("test", 2)']);
});
});
describe("PrefetchEvent", function() {
+35 -6
View File
@@ -4,7 +4,7 @@ import os from "node:os";
import path from "node:path";
import * as lib from "@clusterio/lib";
import { CtlHooks } from "@clusterio/ctl";
import { CommandOutput, CtlHooks } from "@clusterio/ctl";
import * as mock from "../../../test/mock.js";
import { testRoundTripJsonSerialisable, testMatrix } from "../../../test/common.js";
@@ -60,6 +60,14 @@ describe("player_sync plugin", function() {
assert(!validate({ instanceId: 1, playerName: "player", playerData: data }), `accepted ${deleted}`);
}
});
it("should take the empty reply of an older controller as a stored upload", function() {
const validate = lib.compile(msg.UploadResponse.jsonSchema);
assert(validate(undefined));
assert.equal(msg.UploadResponse.fromJSON(undefined).stored, true);
assert(validate({ stored: false }));
assert.equal(msg.UploadResponse.fromJSON({ stored: false }).stored, false);
assert(!validate({ stored: "no" }));
});
describe("SetPlayerDataRequest schema", function() {
const validate = lib.compile(msg.SetPlayerDataRequest.jsonSchema);
// Shaped like module/serialize.lua output
@@ -295,6 +303,7 @@ describe("player_sync plugin", function() {
let printed;
let responses;
const ctl = {
output: new CommandOutput(),
async send(message) {
sent.push(message);
const handler = responses.get(message.constructor);
@@ -323,6 +332,7 @@ describe("player_sync plugin", function() {
beforeEach(function() {
sent = [];
printed = [];
ctl.output = new CommandOutput();
responses = new Map([
[lib.InstanceDetailsGetRequest, request => ({ id: request.instanceId, name: "Test" })],
[msg.ListPlayersRequest, () => [
@@ -352,10 +362,12 @@ describe("player_sync plugin", function() {
it("list should print sorted entries", async function() {
await run("list");
assert.deepEqual(printed, [
"name | generation | size | acquired by",
`alice | 2 | ${lib.formatBytes(100)} | -`,
"bob | - | - | Test (1)",
const lines = printed.join("\n").split("\n").map(line => line.trimEnd());
assert.deepEqual(lines, [
"name | generation | size | acquired by",
"--------------------------------------------",
`alice | 2 | ${lib.formatBytes(100)} | -`,
"bob | - | - | Test (1)",
]);
});
it("show should print a summary", async function() {
@@ -376,7 +388,7 @@ describe("player_sync plugin", function() {
throw new lib.PermissionError("Permission denied");
});
await run("list");
assert.equal(printed[2], "bob | - | - | 1");
assert.match(printed[0], /bob +\| - +\| - +\| 1/);
printed = [];
await run("show", { player: "alice" });
assert.equal(printed[0], "acquired by: 1");
@@ -413,6 +425,23 @@ describe("player_sync plugin", function() {
new lib.CommandError("Player data must be a JSON object"),
);
});
it("should print JSON in json mode", async function() {
ctl.output = new CommandOutput("json");
await run("list");
assert.deepEqual(JSON.parse(printed[0]), [
{ name: "alice", generation: 2, size: 100 },
{ name: "bob", instanceId: 1 },
]);
printed = [];
await run("show", { player: "alice" });
const shown = JSON.parse(printed[0]);
assert.equal(shown.instanceId, 1);
assert.equal(shown.generation, 2);
assert.deepEqual(shown.inventories[0].items[0], { name: "wood", count: 5, quality: "rare" });
printed = [];
await run("export", { player: "alice" });
assert.deepEqual(JSON.parse(printed[0]), storedData);
});
it("delete and release should send requests", async function() {
await run("delete", { player: "alice" });
await run("release", { player: "alice" });
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@clusterio/plugin-research_sync",
"version": "2.1.0",
"version": "2.1.1",
"description": "Clusterio plugin for synchronizing research between Factorio servers",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Hornwitser <github@hornwitser.no>",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@clusterio/plugin-statistics_exporter",
"version": "2.1.0",
"version": "2.1.1",
"description": "Clusterio plugin for gathering statistics from Factorio servers",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Hornwitser <github@hornwitser.no>",
+4
View File
@@ -12,6 +12,10 @@ Run the following commands in the folder Clusterio is installed to:
Substitute clusteriocontroller with clusteriohost or clusterioctl if this a dedicate host or ctl installation respectively.
The plugin needs the [Subspace Storage mod](https://mods.factorio.com/mod/subspace_storage) in the mod pack of every instance it is loaded on.
The controller downloads it from the mod portal on start if the Factorio credentials are configured, but you have to add it to the mod packs yourself.
Instances with the plugin loaded refuse to start without it. Set `subspace_storage.load_plugin` to false on instances that should not use it.
## Controller Configuration
+1
View File
@@ -19,6 +19,7 @@ export const plugin: lib.PluginDeclaration = {
title: "Subspace Storage",
description: "Provides shared storage across instances for the Subspace Storage mod",
instanceEntrypoint: "dist/node/instance.js",
requiredMods: ["subspace_storage >= 1.99.0"],
instanceConfigFields: {
"subspace_storage.log_item_transfers": {
title: "Log Item Transfers",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@clusterio/plugin-subspace_storage",
"version": "2.1.0",
"version": "2.1.1",
"description": "Clusterio plugin for sharing storage between Factorio servers",
"homepage": "https://github.com/clusterio/clusterio#readme",
"author": "Hornwitser <github@hornwitser.no>",
+1 -2
View File
@@ -21,8 +21,8 @@ catalog:
'@types/yazl': ^3.3.1
'@vitejs/plugin-react': ^6.1.2
antd: ^5.29.3
chalk: ^6.0.1
express: ^5.3.0
fflate: ^0.8.3
jsonwebtoken: ^9.0.3
react: ^18.3.1
react-dom: ^18.3.1
@@ -34,7 +34,6 @@ catalog:
typescript: ^7.0.2
vite: ^8.3.4
winston: ^3.19.0
winston-daily-rotate-file: ^5.0.0
winston-transport: ^4.9.0
ws: ^8.22.0
yargs: ^18.2.0
+44
View File
@@ -365,6 +365,50 @@ describe("controller/src/Controller", function() {
await assert.rejects(Controller.loadJsonObject(path.join(dir, "invalid.json")), SyntaxError);
});
});
describe(".downloadRequiredMods()", function() {
let ctrl, resolveCalls, downloadCalls, resolved;
function modInfo(name, version) {
return lib.ModInfo.fromJSON({ name, version, factorio_version: "2.0" });
}
beforeEach(function() {
const config = new ControllerConfig("controller", { "controller.version": controllerVersion });
ctrl = new Controller({}, [
{ name: "foo", title: "Foo", requiredMods: ["foo_mod >= 1.0.0"] },
{ name: "bar", title: "Bar" },
], config);
const modPack = ModPack.fromJSON({ id: 7, name: "default", factorio_version: "2.0.10" });
ctrl.modPacks.set(modPack);
config.set("controller.default_mod_pack_id", 7);
resolveCalls = [];
downloadCalls = [];
resolved = [modInfo("foo_mod", "1.2.0"), modInfo("dep_mod", "0.1.0")];
ctrl.resolveModDependencies = async (mods, factorioVersion) => {
resolveCalls.push([mods.map(mod => mod.spec), factorioVersion]);
return new lib.ModDependencyResolveRequest.Response(resolved, new Map());
};
ctrl.downloadModsFromPortal = async (mods, factorioVersion) => {
downloadCalls.push([mods.map(mod => `${mod.name} ${mod.version}`), factorioVersion]);
};
});
it("should download required mods and their dependencies for the default mod pack", async function() {
await ctrl.downloadRequiredMods();
assert.deepEqual(resolveCalls, [[["foo_mod >= 1.0.0"], "2.0"]]);
assert.deepEqual(downloadCalls, [[["foo_mod = 1.2.0", "dep_mod = 0.1.0"], "2.0"]]);
});
it("should skip mods already in the mod store", async function() {
ctrl.modStore = new lib.ModStore("mods", new Map(resolved.map(info => [info.filename, info])));
await ctrl.downloadRequiredMods();
assert.deepEqual(downloadCalls, []);
});
it("should do nothing when no plugin requires mods", async function() {
ctrl.pluginInfos = [{ name: "bar", title: "Bar" }];
await ctrl.downloadRequiredMods();
assert.deepEqual(resolveCalls, []);
});
});
describe(".loggableHeaders()", function() {
it("should redact credentials and keep other headers", function() {
const headers = {
+95
View File
@@ -0,0 +1,95 @@
import assert from "node:assert/strict";
import * as lib from "@clusterio/lib";
import { CommandOutput, errorToJson, pickFields, toPlain } from "@clusterio/ctl/dist/node/src/command_ops.js";
/** Run fn and return the lines it printed with console.log */
function captureLog(fn) {
const lines = [];
const originalLog = console.log; // eslint-disable-line no-console
console.log = (...args) => lines.push(args.join(" ")); // eslint-disable-line no-console
try {
fn();
} finally {
console.log = originalLog; // eslint-disable-line no-console
}
return lines;
}
describe("ctl/command_ops", function() {
describe("toPlain()", function() {
it("converts Sets, Maps and Dates", function() {
assert.deepEqual(
toPlain({ set: new Set([1, 2]), map: new Map([[3, { a: new Set(["b"]) }]]), date: new Date(0) }),
{ set: [1, 2], map: { 3: { a: ["b"] } }, date: "1970-01-01T00:00:00.000Z" },
);
});
it("uses own properties instead of toJSON", function() {
const role = new lib.Role(1, "name", "desc", new Set(["core.admin"]));
assert.deepEqual(toPlain(role).permissions, ["core.admin"]);
});
});
describe("pickFields()", function() {
it("keeps only the given fields unless all is given", function() {
assert.deepEqual(pickFields({ a: 1, b: 2 }, ["a"]), { a: 1 });
assert.deepEqual(pickFields({ a: 1, b: 2 }, ["all"]), { a: 1, b: 2 });
});
});
describe("errorToJson()", function() {
it("uses the class name of the error", function() {
assert.deepEqual(errorToJson(new lib.NotFoundError("gone")), { error: "NotFoundError", message: "gone" });
assert.deepEqual(errorToJson("text"), { error: "Error", message: "text" });
});
});
describe("class CommandOutput", function() {
it("calls the human formatter in human mode", function() {
const output = new CommandOutput("human");
const human = () => console.log("human"); // eslint-disable-line no-console
assert.deepEqual(captureLog(() => output.result({ a: 1 }, human)), ["human"]);
assert.deepEqual(captureLog(() => output.finish()), []);
});
it("prints a success line in human mode for commands without a result", function() {
assert.deepEqual(captureLog(() => new CommandOutput("human").finish()), ["Command ran successfully"]);
});
it("prints one line of JSON per result in json mode", function() {
const output = new CommandOutput("json");
const lines = captureLog(() => {
output.result({ a: new Set([1]) }, () => assert.fail("called human formatter"));
output.table([{ b: 2 }]);
output.finish();
});
assert.deepEqual(lines, ['{"a":[1]}', '[{"b":2}]']);
});
it("prints the toJSON format in wire mode", function() {
const output = new CommandOutput("wire");
const role = new lib.Role(1, "name", "desc", new Set(["core.admin"]));
const lines = captureLog(() => output.result({ role, set: new Set([1]), map: new Map([["a", 2]]) }));
assert.deepEqual(JSON.parse(lines[0]), { role: JSON.parse(JSON.stringify(role)), set: [1], map: { a: 2 } });
});
it("prints success for commands without a result", function() {
assert.deepEqual(captureLog(() => new CommandOutput("json").finish()), ['{"success":true}']);
const streaming = new CommandOutput("json");
streaming.streamingLog = true;
assert.deepEqual(captureLog(() => streaming.finish()), []);
});
it("prints errors as JSON", function() {
const output = new CommandOutput("json");
const lines = captureLog(() => {
output.error(new lib.CommandError("failed"), () => assert.fail("called human formatter"));
output.finish();
});
assert.deepEqual(lines, ['{"error":"CommandError","message":"failed"}']);
});
it("calls the human error reporter in human mode", function() {
const output = new CommandOutput("human");
const human = () => console.log("failed"); // eslint-disable-line no-console
const lines = captureLog(() => {
output.error(new lib.CommandError("failed"), human);
output.finish();
});
assert.deepEqual(lines, ["failed"]);
});
});
});
+88
View File
@@ -0,0 +1,88 @@
import assert from "node:assert/strict";
import * as lib from "@clusterio/lib";
import { modCommands } from "@clusterio/ctl/dist/node/src/commands_mod.js";
import { modPackCommands } from "@clusterio/ctl/dist/node/src/commands_mod_pack.js";
import { userCommands } from "@clusterio/ctl/dist/node/src/commands_user.js";
import { CommandOutput } from "@clusterio/ctl/dist/node/src/command_ops.js";
/** Run fn and return the lines it printed with console.log */
async function captureLog(fn) {
const lines = [];
const originalLog = console.log; // eslint-disable-line no-console
console.log = (...args) => lines.push(args.join(" ")); // eslint-disable-line no-console
try {
await fn();
} finally {
console.log = originalLog; // eslint-disable-line no-console
}
return lines;
}
/** Mock ctl answering requests by their class */
function mockCtl(responses) {
return {
output: new CommandOutput(),
async send(message) {
return responses.get(message.constructor)(message);
},
async sendTo(destination, message) {
return this.send(message);
},
};
}
describe("ctl/commands", function() {
describe("mod show", function() {
const show = modCommands.get("show");
it("prints list fields one entry per line", async function() {
const ctl = mockCtl(new Map([
[lib.ModGetRequest, () => ({ name: "a_mod", dependencies: ["base", "? other"] })],
]));
const lines = await captureLog(() => show.run({ name: "a_mod", modVersion: "1.0.0" }, ctl));
assert.deepEqual(lines, ["name: a_mod", "dependencies:", " base", " ? other"]);
});
it("rejects an invalid version", async function() {
await assert.rejects(
show.run({ name: "a_mod", modVersion: "latest" }, mockCtl(new Map())),
new lib.CommandError("mod-version must match format digit.digit.digit"),
);
});
});
describe("mod-pack show", function() {
it("marks missing mods and checksum mismatches and lists export assets", async function() {
const modPack = lib.ModPack.fromJSON({ id: 7, name: "pack" });
modPack.mods.set("gone", { name: "gone", enabled: true, version: "1.0.0" });
modPack.mods.set("changed", { name: "changed", enabled: false, version: "1.0.0", sha1: "aaa" });
modPack.exportManifest = { assets: { settings: "settings.json" } };
const ctl = mockCtl(new Map([
[lib.ModListRequest, () => [{ id: "changed_1.0.0", sha1: "bbb" }]],
[lib.ModPackListRequest, () => [modPack]],
[lib.ModPackGetRequest, () => modPack],
]));
const lines = await captureLog(() => modPackCommands.get("show").run({ modPack: "pack" }, ctl));
assert(lines.includes(" ? gone 1.0.0"), lines.join("\n"));
assert(lines.includes(" ! (disabled) changed 1.0.0 (aaa)"), lines.join("\n"));
assert(lines.includes(" settings: settings.json"), lines.join("\n"));
});
});
describe("user show", function() {
const show = userCommands.get("show");
it("prints stats per instance", async function() {
const stats = new lib.PlayerStats();
stats.joinCount = 2;
const user = new lib.UserDetails("alice");
user.instanceStats = new Map([[1, stats], [2, stats]]);
user.recalculatePlayerStats();
const ctl = mockCtl(new Map([
[lib.UserGetRequest, () => user],
[lib.InstanceDetailsListRequest, () => [{ id: 1, name: "one" }]],
]));
const lines = await captureLog(() => show.run({ name: "alice", instanceStats: true }, ctl));
assert(lines.includes("Instance one (1):"), lines.join("\n"));
assert(lines.includes("Instance <deleted> (2):"), lines.join("\n"));
assert(lines.some(line => /joinCount +\| 4/.test(line)), lines.join("\n"));
});
});
});
+91
View File
@@ -0,0 +1,91 @@
import assert from "node:assert/strict";
import * as lib from "@clusterio/lib";
import { instanceCommands } from "@clusterio/ctl/dist/node/src/commands_instance.js";
import { CommandOutput } from "@clusterio/ctl/dist/node/src/command_ops.js";
/** Run fn and return the lines it printed with console.log */
async function captureLog(fn) {
const lines = [];
const originalLog = console.log; // eslint-disable-line no-console
console.log = (...args) => lines.push(args.join(" ")); // eslint-disable-line no-console
try {
await fn();
} finally {
console.log = originalLog; // eslint-disable-line no-console
}
return lines;
}
describe("ctl/commands_instance", function() {
let instances;
let sentTo;
const ctl = {
output: new CommandOutput(),
async send(message) {
assert(message instanceof lib.InstanceDetailsListRequest);
return instances;
},
async sendTo(destination, message) {
sentTo.push([destination.instanceId, message.constructor.name]);
if (destination.instanceId === 2) {
throw new lib.CommandError("broken");
}
},
};
beforeEach(function() {
sentTo = [];
instances = [
{ id: 1, name: "one", status: "stopped", excludeFromStartAll: false },
{ id: 2, name: "two", status: "stopped", excludeFromStartAll: false },
{ id: 3, name: "three", status: "stopped", excludeFromStartAll: true },
{ id: 4, name: "four", status: "running", excludeFromStartAll: false },
];
});
describe("start-all", function() {
const startAll = instanceCommands.get("start-all");
it("starts stopped instances and reports each one", async function() {
ctl.output = new CommandOutput("human");
const lines = await captureLog(() => startAll.run({ force: false }, ctl));
assert.deepEqual(sentTo, [[1, "InstanceStartRequest"], [2, "InstanceStartRequest"]]);
assert(lines.includes("✓ Started one"), lines.join("\n"));
assert(lines.some(line => line.startsWith("✗ Failed to start two")), lines.join("\n"));
});
it("prints an entry per instance in json mode", async function() {
ctl.output = new CommandOutput("json");
const lines = await captureLog(() => startAll.run({ force: true }, ctl));
assert.deepEqual(JSON.parse(lines[0]), [
{ id: 1, name: "one" },
{ id: 2, name: "two", error: { error: "CommandError", message: "broken" } },
{ id: 3, name: "three" },
]);
});
it("reports when there is nothing to start", async function() {
instances = [];
ctl.output = new CommandOutput("human");
assert.deepEqual(await captureLog(() => startAll.run({ force: false }, ctl)), ["No instances to start."]);
});
});
describe("stop-all", function() {
const stopAll = instanceCommands.get("stop-all");
it("stops running instances and reports each one", async function() {
instances[1].status = "running";
ctl.output = new CommandOutput("human");
const lines = await captureLog(() => stopAll.run({}, ctl));
assert.deepEqual(sentTo, [[2, "InstanceStopRequest"], [4, "InstanceStopRequest"]]);
assert(lines.includes("✓ Stopped four"), lines.join("\n"));
assert(lines.some(line => line.startsWith("✗ Failed to stop two")), lines.join("\n"));
});
it("prints an entry per instance in json mode", async function() {
ctl.output = new CommandOutput("json");
const lines = await captureLog(() => stopAll.run({}, ctl));
assert.deepEqual(JSON.parse(lines[0]), [{ id: 4, name: "four" }]);
});
it("reports when there is nothing to stop", async function() {
instances = [];
ctl.output = new CommandOutput("human");
assert.deepEqual(await captureLog(() => stopAll.run({}, ctl)), ["No instances to stop."]);
});
});
});
+56
View File
@@ -0,0 +1,56 @@
import assert from "node:assert/strict";
import * as lib from "@clusterio/lib";
import { Ctl } from "@clusterio/ctl/dist/node/ctl.js";
import { CommandOutput } from "@clusterio/ctl/dist/node/src/command_ops.js";
/** Run fn and return the lines it printed with console.log */
async function captureLog(fn) {
const lines = [];
const originalLog = console.log; // eslint-disable-line no-console
console.log = (...args) => lines.push(args.join(" ")); // eslint-disable-line no-console
try {
await fn();
} finally {
console.log = originalLog; // eslint-disable-line no-console
}
return lines;
}
describe("ctl/ctl", function() {
describe("class Ctl", function() {
let ctl;
beforeEach(function() {
const connector = new lib.VirtualConnector(
lib.Address.fromShorthand({ controlId: 1 }),
lib.Address.fromShorthand("controller"),
);
ctl = new Ctl(connector, new lib.CtlConfig("control"));
});
it("prints log events as JSON lines only when streaming the log", async function() {
const info = { level: "info", message: "hello" };
ctl.output = new CommandOutput("json");
const logged = await captureLog(() => ctl.handleLogMessageEvent(new lib.LogMessageEvent({ ...info })));
assert(logged.every(line => !line.startsWith("{")), "log event printed as a result");
ctl.output.streamingLog = true;
assert.deepEqual(
await captureLog(() => ctl.handleLogMessageEvent(new lib.LogMessageEvent({ ...info, timestamp: 1 }))),
['{"level":"info","message":"hello","timestamp":1}'],
);
});
it("logs log events in human mode", async function() {
ctl.output.streamingLog = true;
const info = { level: "verbose", message: "hello" };
assert.deepEqual(await captureLog(() => ctl.handleLogMessageEvent(new lib.LogMessageEvent(info))), []);
});
it("prints debug WebSocket messages", async function() {
const event = new lib.DebugWsMessageEvent("in", "content");
assert.deepEqual(await captureLog(() => ctl.handleDebugWsMessageEvent(event)), ["WS in content"]);
ctl.output = new CommandOutput("json");
assert.deepEqual(
await captureLog(() => ctl.handleDebugWsMessageEvent(event)),
['{"direction":"in","content":"content"}'],
);
});
});
});
+43
View File
@@ -336,6 +336,49 @@ describe("class Instance", function() {
});
});
describe(".checkRequiredMods()", function() {
beforeEach(function() {
instance.loadedPlugins = new Set([
{ name: "foo", requiredMods: ["foo_mod", "bar_mod >= 1.1.0", "bar_mod < 2.0.0"] },
{ name: "bar" },
]);
});
function modPack(mods) {
return lib.ModPack.fromJSON({ name: "pack", mods });
}
it("should accept a mod pack with the required mods", function() {
instance.checkRequiredMods(modPack([
{ name: "foo_mod", enabled: true, version: "1.0.0" },
{ name: "bar_mod", enabled: true, version: "1.1.0" },
]));
});
it("should reject missing, disabled and out of range mods", function() {
const expected = (unmet) => new lib.ConflictError(
"Mod pack pack does not have the mods required by the loaded plugins. " +
`Plugin foo requires ${unmet}. ` +
"Add the mods to the mod pack or set <plugin>.load_plugin to false for this instance."
);
assert.throws(
() => instance.checkRequiredMods(modPack([{ name: "bar_mod", enabled: true, version: "1.0.0" }])),
expected("foo_mod (missing), bar_mod >= 1.1.0 (found 1.0.0)"),
);
assert.throws(
() => instance.checkRequiredMods(modPack([
{ name: "foo_mod", enabled: false, version: "1.0.0" },
{ name: "bar_mod", enabled: true, version: "2.0.0" },
])),
expected("foo_mod (disabled), bar_mod < 2.0.0 (found 2.0.0)"),
);
});
it("should ignore plugins that are not loaded", function() {
instance.loadedPlugins = new Set([{ name: "bar" }]);
instance.checkRequiredMods(modPack([]));
});
});
describe("save name checks", function() {
it("should reject a start save outside the saves directory", async function() {
await assert.rejects(
+124
View File
@@ -2064,6 +2064,9 @@ describe("Integration of Clusterio", function() {
const roundtrip = lib.ModPack.fromModPackString(result.stdout.trim());
roundtrip.id = reference.id;
assert.deepEqual(roundtrip, reference);
const shown = await execCtlOutput("mod-pack show imported-pack");
assert.match(shown, /empty_mod 1\.0\.0/);
assert.match(shown, /MyDouble: 12\.25/);
});
});
@@ -2119,6 +2122,7 @@ describe("Integration of Clusterio", function() {
describe("mod show", function() {
it("gives details of a mod", async function() {
assert.match(await execCtlOutput("mod show empty_mod 1.0.0"), /name: empty_mod/);
let result = await execCtlProcess("mod show empty_mod 1.0.0");
let hash = await lib.hashFile(path.join("temp", "test", "mods", "empty_mod_1.0.0.zip"));
let stat = await fs.stat(path.join("temp", "test", "mods", "empty_mod_1.0.0.zip"));
@@ -2317,6 +2321,126 @@ describe("Integration of Clusterio", function() {
});
});
describe("config edit", function() {
/** Write an editor script that replaces the edited file with content */
async function writeEditor(content) {
const file = path.resolve("temp", "test", "editor.sh");
await fs.writeFile(file, `#!/bin/sh\ncat > "$1" <<'EOF'\n${content}\nEOF\n`, { mode: 0o755 });
return file;
}
before(function() {
if (process.platform === "win32") {
this.skip();
}
});
it("sets controller config fields and reports failed ones", async function() {
const config = await getControl().send(new lib.ControllerConfigGetRequest());
const name = config["controller.name"];
const editor = await writeEditor("controller.name = Edited-Cluster\ncontroller.no_such_field = 1");
await execCtl(`controller config edit ${editor}`);
const edited = await getControl().send(new lib.ControllerConfigGetRequest());
assert.equal(edited["controller.name"], "Edited-Cluster");
await execCtl(`controller config set controller.name ${name}`);
});
it("sets host config fields and reports failed ones", async function() {
const config = await getControl().sendTo({ hostId }, new lib.HostConfigGetRequest());
const editor = await writeEditor(`host.name = ${config["host.name"]}\nhost.no_such_field = 1`);
await execCtl(`host config edit ${hostId} ${editor}`);
});
it("sets instance config fields and reports failed ones", async function() {
const { id } = JSON.parse(await execCtlOutput("--output json instance create config-edit"));
try {
const editor = await writeEditor("instance.auto_start = true\ninstance.no_such_field = 1");
await execCtl(`instance config edit ${id} ${editor}`);
const config = await getControl().send(new lib.InstanceConfigGetRequest(id));
assert.equal(config["instance.auto_start"], true);
} finally {
await execCtl(`instance delete ${id}`);
}
});
});
describe("--output json", function() {
it("prints lists as an array of objects", async function() {
const roles = JSON.parse(await execCtlOutput("--output json role list"));
assert.deepEqual(roles.find(role => role.id === 0).permissions, ["core.admin"]);
});
it("prints single results as an object", async function() {
const { id } = JSON.parse(await execCtlOutput("--output json role create json-role"));
const roles = await getControl().send(new lib.RoleListRequest());
assert.equal(roles.find(role => role.id === id)?.name, "json-role");
assert.equal(await execCtlOutput(`--output json role delete ${id}`), '{"success":true}');
});
it("prints the toJSON format in wire mode", async function() {
const [jsonUser] = JSON.parse(await execCtlOutput("--output json user list"));
const [wireUser] = JSON.parse(await execCtlOutput("--output wire user list"));
assert("roleIds" in jsonUser);
assert(!("roleIds" in wireUser));
assert.equal(wireUser.name, jsonUser.name);
});
it("prints a success line for commands without a result", async function() {
const { id } = JSON.parse(await execCtlOutput("--output json role create human-role"));
assert.equal(await execCtlOutput(`role delete ${id}`), "Command ran successfully");
});
it("prints whole objects for lists that hide fields in human mode", async function() {
const [mod] = JSON.parse(await execCtlOutput("--output json mod list"));
assert("sha1" in mod);
const [wireMod] = JSON.parse(await execCtlOutput("--output wire mod list"));
assert("factorio_version" in wireMod);
const [modPack] = JSON.parse(await execCtlOutput("--output wire mod-pack list"));
assert("mods" in modPack);
const user = JSON.parse(await execCtlOutput("--output wire user show test --instance-stats"));
assert.equal(user.name, "test");
});
it("prints the user and instance stats in human mode", async function() {
const output = await execCtlOutput("user show test --instance-stats");
assert.match(output, /name +\| test/);
});
it("prints the result of human mode commands with more options", async function() {
assert.match(await execCtlOutput("user list --stats --attributes"), /isAdmin/);
const exchange = JSON.parse(await execCtlOutput(
`--output json debug parse-exchange-string ${testStrings.default.replace(/\s/g, "")}`
));
assert.equal(typeof exchange, "object");
});
it("follows the log without a final result", async function() {
try {
assert.equal(await execCtlOutput("--output json log follow --controller"), "");
} finally {
getControl().keepOpen = false;
}
});
it("prints log queries in human mode", async function() {
assert.notEqual(await execCtlOutput("log query --controller --limit 1"), "");
});
it("prints pending host requests in human mode", async function() {
assert.notEqual(await execCtlOutput("host pending"), "");
});
it("prints log queries as an array", async function() {
const log = JSON.parse(await execCtlOutput("--output json log query --controller --limit 1"));
assert.equal(log.length, 1);
assert.equal(typeof log[0].message, "string");
});
it("prints errors as JSON with a non-zero exit code", async function() {
await assert.rejects(
execCtlProcess("--output json instance stop no-such-instance"),
err => {
assert.equal(err.code, 1);
assert.deepEqual(
JSON.parse(err.stdout),
{ error: "CommandError", message: "No instance named no-such-instance" },
);
return true;
},
);
await assert.rejects(
execCtlProcess("--output json user set-admin no-such-user"),
err => JSON.parse(err.stdout).error === "NotFoundError",
);
});
});
describe("user list", function() {
it("prints roles and instances as lists", async function() {
const output = await execCtlOutput("user list");
+21 -3
View File
@@ -10,7 +10,7 @@ import events from "node:events";
import * as lib from "@clusterio/lib";
import { LineSplitter, logger } from "@clusterio/lib";
import { selectTargetCommand, initialize as initializeCtl } from "@clusterio/ctl";
import { runCommand, initialize as initializeCtl } from "@clusterio/ctl";
import { _listFactorioVersions } from "@clusterio/host/dist/node/src/server.js";
import { pairHost } from "@clusterio/host";
@@ -242,8 +242,7 @@ export async function execCtl(command) {
}
control.config = initArgs.ctlConfig;
const targetCommand = selectTargetCommand(initArgs.args, initArgs.rootCommands);
await targetCommand.run(initArgs.args, control);
await runCommand(initArgs.args, initArgs.rootCommands, control);
} finally {
process.chdir("../..");
inExecCtl = false;
@@ -353,6 +352,22 @@ before(async function() {
sourceDir: "packages/host/lua/clusterio_lib",
outputDir: modsDir,
});
// Stand-in for the mod required by the subspace_storage plugin
const subspaceStorageDir = path.join("temp", "test", "subspace_storage");
await fs.rm(subspaceStorageDir, { force: true, recursive: true });
await fs.mkdir(subspaceStorageDir, { recursive: true });
await fs.writeFile(path.join(subspaceStorageDir, "info.json"), JSON.stringify({
name: "subspace_storage",
variants: [
{ version: "2.3.11", factorio_version: "1.1" },
{ version: "2.3.20", factorio_version: "2.0" },
{ version: "2.3.21", factorio_version: "2.1" },
],
title: "Subspace Storage test stub",
author: "Clusterio Team",
dependencies: ["base", "clusterio_lib"],
}));
await lib.build({ build: true, pack: true, sourceDir: subspaceStorageDir, outputDir: modsDir });
console.log("Setting Controller Config");
await execController("config create");
@@ -410,6 +425,9 @@ before(async function() {
const [major, minor] = lib.normaliseMajorMinorVersion(latestFactorioVersion).split(".");
testPack.mods.set("clusterio_lib", { name: "clusterio_lib", enabled: true, version: `2.1.${major}${minor}` });
testPack.mods.set(
"subspace_storage", { name: "subspace_storage", enabled: true, version: `2.3.${major}${minor}` }
);
await control.sendTo("controller", new lib.ModPackCreateRequest(testPack));
await control.sendTo(
+34 -1
View File
@@ -6,7 +6,8 @@ import stream from "node:stream";
// Capture native fetch before any potential mocks are applied
const nativeFetch = global.fetch;
import { ModStore, ModInfo, ModVersionEquality, zipOutputStream } from "@clusterio/lib";
import { ModStore, ModInfo, ModVersionEquality, zipOutputStream, hashFile } from "@clusterio/lib";
import * as libBuildMod from "@clusterio/lib/build_mod.js";
import { externalTest, createZip, createZipBuffer } from "../common.js";
import "../setup_logging.js";
@@ -706,6 +707,38 @@ describe("lib/ModStore", function () {
});
// === Instance Methods ===
describe("readModInfo (static)", function () {
it("should load from a mod zip file", async function () {
await libBuildMod.build({
build: true,
pack: true,
sourceDir: path.join("test", "file", "empty_mod"),
outputDir: path.join("temp", "test"),
});
const filePath = path.join("temp", "test", "empty_mod_1.0.0.zip");
const hash = await hashFile(filePath);
const stat = await fs.stat(filePath);
const mod = await ModStore.readModInfo(filePath);
assert.deepEqual(
mod,
ModInfo.fromJSON({
name: "empty_mod",
version: "1.0.0",
title: "An Empty Mod",
author: "Me",
description: "An empty mod for testing",
factorio_version: "1.1",
dependencies: [],
filename: "empty_mod_1.0.0.zip",
size: stat.size,
mtime_ms: stat.mtimeMs,
updated_at_ms: stat.mtimeMs,
sha1: hash,
})
);
});
});
describe("Instance Methods", function () {
let modStore;
let testMod1Name = "test-mod-1";
-32
View File
@@ -1,9 +1,6 @@
import assert from "node:assert/strict";
import fs from "node:fs/promises";
import path from "node:path";
import * as lib from "@clusterio/lib";
import * as libBuildMod from "@clusterio/lib/build_mod.js";
import { ModInfo, ModDependency } from "@clusterio/lib";
import { testMatrix, testRoundTripJsonSerialisable } from "../../common.js";
@@ -413,35 +410,6 @@ describe("lib/data/ModInfo", function() {
["UltraMod", "SuperLib >= 1.00", "! bad-mod"].map(dep => new ModDependency(dep))
);
});
it("should load from a mod zip file", async function() {
await libBuildMod.build({
build: true,
pack: true,
sourceDir: path.join("test", "file", "empty_mod"),
outputDir: path.join("temp", "test"),
});
const filePath = path.join("temp", "test", "empty_mod_1.0.0.zip");
const hash = await lib.hashFile(filePath);
const stat = await fs.stat(filePath);
const mod = await ModInfo.fromModFile(filePath);
assert.deepEqual(
mod,
ModInfo.fromJSON({
name: "empty_mod",
version: "1.0.0",
title: "An Empty Mod",
author: "Me",
description: "An empty mod for testing",
factorio_version: "1.1",
dependencies: [],
filename: "empty_mod_1.0.0.zip",
size: stat.size,
mtime_ms: stat.mtimeMs,
updated_at_ms: stat.mtimeMs,
sha1: hash,
})
);
});
describe("checkDependencySatisfaction()", function() {
it("should return undefined when satisfied", function() {
const mod = ModInfo.fromJSON({ dependencies: ["! a", "b = 1.0.0", "? c = 2.0.0"] });
+72
View File
@@ -1,5 +1,6 @@
import assert from "node:assert/strict";
import fs from "node:fs/promises";
import os from "node:os";
import path from "node:path";
import { testLines } from "./factorio/lines.js";
@@ -66,6 +67,77 @@ describe("lib/logging_utils.js", function() {
});
});
describe("class DailyFileTransport", function() {
let tempDir = path.join("temp", "test", "daily_file");
beforeEach(async function() {
await fs.rm(tempDir, { recursive: true, force: true });
});
function create(now, dirname = tempDir) {
return new lib.DailyFileTransport({ dirname, prefix: "test", now });
}
function write(transport, message) {
transport.log({ [Symbol.for("message")]: JSON.stringify({ message }) }, () => {});
}
async function close(transport) {
await new Promise(resolve => transport.close(resolve));
}
async function read(file) {
let content = await fs.readFile(path.join(tempDir, file), "utf8");
assert(content.endsWith(os.EOL));
return content.split(os.EOL).slice(0, -1).map(line => JSON.parse(line).message);
}
it("should create the directory and write lines to the file for the UTC date", async function() {
let transport = create(() => Date.parse("2026-11-19T12:00:00Z"), path.join(tempDir, "nested"));
write(transport, "first");
write(transport, "second");
await close(transport);
assert.deepEqual(await read(path.join("nested", "test-2026-11-19.log")), ["first", "second"]);
});
it("should append to an existing file", async function() {
await fs.mkdir(tempDir, { recursive: true });
await fs.writeFile(path.join(tempDir, "test-2026-11-19.log"), `{"message":"old"}${os.EOL}`);
let transport = create(() => Date.parse("2026-11-19T12:00:00Z"));
write(transport, "new");
await close(transport);
assert.deepEqual(await read("test-2026-11-19.log"), ["old", "new"]);
});
it("should switch files when the UTC date changes", async function() {
let time = Date.parse("2026-11-19T23:59:59.999Z");
let transport = create(() => time);
write(transport, "before");
time += 1;
write(transport, "after");
await close(transport);
assert.deepEqual((await fs.readdir(tempDir)).sort(), ["test-2026-11-19.log", "test-2026-11-20.log"]);
assert.deepEqual(await read("test-2026-11-19.log"), ["before"]);
assert.deepEqual(await read("test-2026-11-20.log"), ["after"]);
});
it("should use the UTC date rather than the local one", async function() {
let transport = create(() => Date.parse("2026-11-20T00:30:00+02:00"));
write(transport, "message");
await close(transport);
assert.deepEqual(await fs.readdir(tempDir), ["test-2026-11-19.log"]);
});
it("should emit write errors", async function() {
await fs.mkdir(path.join(tempDir, "test-2026-11-19.log"), { recursive: true });
let transport = create(() => Date.parse("2026-11-19T12:00:00Z"));
let error = new Promise(resolve => transport.once("error", resolve));
write(transport, "message");
assert.equal((await error).code, "EISDIR");
});
it("should close when nothing was written", async function() {
await close(create(Date.now));
assert.deepEqual(await fs.readdir(tempDir), []);
});
});
describe("deleteOldLogs", function() {
let tempDir = path.join("temp", "test", "log_retention");
let files = [
+18
View File
@@ -24,4 +24,22 @@ describe("lib/plugin", function() {
}
});
});
describe("pluginRequiredMods()", function() {
it("should parse dependency strings", function() {
const mods = lib.pluginRequiredMods({
name: "foo", title: "Foo", requiredMods: ["bar", "baz >= 1.2.0", "baz < 2.0.0"],
});
assert.deepEqual(mods.map(mod => [mod.name, mod.version?.toString()]), [
["bar", undefined], ["baz", ">= 1.2.0"], ["baz", "< 2.0.0"],
]);
});
it("should return an empty list without requiredMods", function() {
assert.deepEqual(lib.pluginRequiredMods({ name: "foo", title: "Foo" }), []);
});
it("should reject dependency prefixes and invalid versions", function() {
for (const spec of ["? bar", "! bar", "~ bar", "bar >= 1..2"]) {
assert.throws(() => lib.pluginRequiredMods({ name: "foo", title: "Foo", requiredMods: [spec] }));
}
});
});
});
+25
View File
@@ -0,0 +1,25 @@
/* eslint-disable node/no-sync */
import assert from "node:assert/strict";
import zlib from "node:zlib";
import * as browserZlib from "../../packages/lib/src/zlib_sync.browser.ts";
describe("lib/zlib_sync.browser", function() {
const text = JSON.stringify({ name: "pack", mods: [{ name: "base", version: "2.0.0" }] });
it("should inflate data deflated by node:zlib", function() {
const inflated = browserZlib.inflateSync(zlib.deflateSync(text));
assert.equal(Buffer.from(inflated).toString(), text);
});
it("should deflate data node:zlib can inflate", function() {
assert.equal(zlib.inflateSync(browserZlib.deflateSync(text)).toString(), text);
const bytes = Buffer.from(text);
assert.equal(zlib.inflateSync(browserZlib.deflateSync(bytes)).toString(), text);
});
it("should throw with a Z_ code on invalid data", function() {
assert.throws(
() => browserZlib.inflateSync(Buffer.from("not zlib data")),
err => err.code === "Z_DATA_ERROR"
);
});
});