Merge pull request #282 from Hornwitser/small-things

Small things
This commit is contained in:
Hornwitser authored and GitHub committed 2020-04-06 14:50:55 +02:00
commit 112cad8a49
62 files changed
+591 -342

No files matched your search

+4 -1
View File
@@ -25,6 +25,9 @@ coverage
build/Release
dist/
# Autogenerated docs output
/docs/api
node_modules/*
Factorio*
@@ -42,7 +45,7 @@ static/pictures/spritesheet.js
static/pictures/spritesheet.png
static/external
saves
/test/temp/
/temp/
# weird pattern to ignore sharedPlugins except not ignore some core modules
!sharedPlugins/
+3 -1
View File
@@ -6,6 +6,8 @@ before_install:
- tar -xf factorio.tar.gz
- npm install -g codecov
script:
- node build-mod --bump-patch --factorio-version 0.18 --output-dir sharedMods
- curl -o sharedMods/subspace_storage_1.99.1.zip -L https://github.com/clusterio/factorioClusterioMod/releases/download/1.99.1/subspace_storage_1.99.1.zip
- npm run-script ci-cover
- cat test/temp/integration/log.txt
- cat temp/test/integration/log.txt
- codecov -f coverage/lcov.info
+21 -4
View File
@@ -31,6 +31,12 @@ Version 2.0.0
- Added export of pollution statitics.
- Reconnection logic that esures no data is dropped talking to the master
server provided the session can be resumed.
- Added support for having multiple Factorio installs and selecting which
version to use on a per instance basis.
- Added config to specify which path the master interface is accessed under,
allowing it to be proxied behind web-server.
- Added WebSocket usage statistics.
- Added commands sent statistic.
### Changes
@@ -79,6 +85,7 @@ Version 2.0.0
- Removed UPSdisplay plugin. UPS statistics is exported by the statistics
exporter plugin.
- Master server now defaults to hosting on https on port 8443.
- Renamed client to slave.
### Breaking Changes
@@ -130,11 +137,20 @@ Version 2.0.0
socket_io_transmit_bytes,
- Removed clusterio_connected_instaces_gauge and added
clusterio_master_connected_clients_count in its place.
- Renamed clusterio_player_count_gauge to clusterio_instance_player_count
- Removed clusterio_UPS_gauge and added clusterio_instance_game_ticks_total in
its place.
- Renamed clusterio_player_count_gauge to
clusterio_statistics_exporter_instance_player_count
- Removed clusterio_UPS_gauge and added
clusterio_statistics_exporter_instance_game_ticks_total in its place.
- Renamed clusterio_endpoint_hit_gauge to clusterio_http_enpoint_hits_total
- Renamed clusterio_statistics_gauge to clusterio_instance_force_flows
- Renamed clusterio_statistics_gauge to
clusterio_statistics_exporter_instance_force_flows
- Renamed clusterio_nn_dole_gauge to clusterio_subspace_storage_nn_dole_gauge.
- Renamed clusterio_dole_factor_gauge to
clusterio_subspace_storage_dole_factor_gauge.
- Renamed clusterio_import_gauge to clusterio_subspace_storage_import_total.
- Renamed clusterio_export_gauge to clusterio_subspace_storage_export_total.
- Renamed clusterio_master_inventory_gauge to
clusterio_subspace_storage_master_inventory.
- Removed clusterioMod plugin specific config options logItemTransfers,
disableFairItemDistribution useNeuralNetDoleDivider, autosaveInterval, and
disableImportsOfEverythingExceptElectricity from the master config.
@@ -146,6 +162,7 @@ Version 2.0.0
arguments from the master server.
- Implemented a new config system that replaces the old. Breaks all plugins.
- Removed usage of socket.io entirely in favor of a plain WebSocket connection.
- Renamed clusterioMod plugin to subspace_storage.
Version 1.2.4
+19 -13
View File
@@ -128,7 +128,7 @@ Clusterio has *very* limited support for using docker.
sudo docker run --name master -e MODE=master -p 1234:8080 -d -it --restart=unless-stopped danielvestol/clusterio
sudo docker run --name slave -e MODE=client -e INSTANCE=world1 -v /srv/clusterio/instances:/factorioClusterio/instances -p 1235:34167 -it --restart=unless-stopped danielvestol/clusterio
sudo docker run --name slave -e MODE=slave -e INSTANCE=world1 -v /srv/clusterio/instances:/factorioClusterio/instances -p 1235:34167 -it --restart=unless-stopped danielvestol/clusterio
The -v flag is used to specify the instance directory. Your instances (save files etc) will be stored there.
-->
@@ -142,7 +142,7 @@ Clusterio is built up of multiple parts. Here is a quick guide:
Master = master.js
Server host (Slave) = client.js + factorio server
Server host (Slave) = slave.js + factorio server
Game Client = The people connecting to the server
@@ -167,19 +167,25 @@ reboot when you are done, then proceed to the next steps. *reboots matter*
- Via the stand alone version on from their website
1. Download the MS Windows (64-bit zip package) from https://www.factorio.com/download
1. Create a new folder named "factorio" in the the factorioClusterio
folder.
2. Open the zip file and drag the folder called "Factorio_0.17.x" into the factorioClusterio folder
2. Download the MS Windows (64-bit zip package) from
https://www.factorio.com/download .
3. Rename the folder to "factorio"
3. Open the zip file and drag the folder called "Factorio_0.x.y" into
the factorio folder created in step 1.
- Via steam installation
1. Locate the game files by right clicking the game in steam, selecting properties, then Local Files, then Browse local files.
1. Create a new folder named "factorio" in the the factorioClusterio
folder.
2. Go to the parent folder of the folder that Steam opened and copy the Factorio folder into the factorioClusterio folder
2. Locate the game files by right clicking the game in steam,
selecting properties, then Local Files, then Browse local files.
3. Rename the folder to "factorio"
3. Go to the parent folder of the folder that Steam opened and copy
the Factorio folder into the factorio folder created in step 1.
## Running Clusterio
@@ -207,20 +213,20 @@ up.
To set up the configuration for a new local slave run the following.
node client config set slave.name "Local"
node client config set slave.master_token "<token>"
node slave config set slave.name "Local"
node slave config set slave.master_token "<token>"
This will write a new `config-slave.json` file in the current directory
(you can change the location with the `--config` option) with the name
and token provided. If you are connecting to a remote master server you
will also need to set the `slave.master_url` option to that url.
You can list the config of a slave with the `node client config list`
command. Use `node client config --help` for more information.
You can list the config of a slave with the `node slave config list`
command. Use `node slave config --help` for more information.
Once the config is set up run the slave with
node client start
node slave run
### Instances
+16
View File
@@ -23,12 +23,24 @@ async function main() {
'build': { describe: "Build mod", type: 'boolean', default: true },
'pack': { describe: "Pack into zip file", type: 'boolean', default: true },
'output-dir': { describe: "Path to output built mod", nargs: 1, type: 'string', default: "dist" },
"bump-patch": { describe: "Increment patch number of build", type: "boolean", default: false },
"factorio-version": { describe: "Override factorio_version", type: "string" },
})
.argv
;
let info = JSON.parse(await fs.readFile(path.join(sourceDir, "info.json")));
if (args.bumpPatch) {
let [major, minor, patch] = info.version.split(".");
patch = String(Number.parseInt(patch, 10) + 1);
info.version = [major, minor, patch].join(".");
}
if (args.factorioVersion) {
info.factorio_version = args.factorioVersion;
}
if (args.clean) {
let splitter = /^(.*)_(\d+\.\d+\.\d+)(\.zip)?$/
for (let entry of await fs.readdir(args.outputDir)) {
@@ -67,6 +79,8 @@ async function main() {
zip.file(path.posix.join(modName, path.basename(filePath)), fs.createReadStream(filePath));
}
zip.file(path.posix.join(modName, "info.json"), JSON.stringify(info, null, 4));
let modPath = path.join(args.outputDir, `${modName}.zip`);
console.log(`Writing ${modPath}`);
let writeStream = zip.generateNodeStream().pipe(fs.createWriteStream(modPath));
@@ -83,6 +97,8 @@ async function main() {
for (let filePath of additionalFiles) {
await fs.copy(filePath, path.join(modDir, path.basename(filePath)));
}
await fs.writeFile(path.join(modDir, "info.json"), JSON.stringify(info, null, 4));
}
}
}
-32
View File
@@ -1,32 +0,0 @@
{
"__comment": "Where factorio folder. Should be in the same folder as this config file.",
"factorioDirectory": "./factorio",
"databaseDirectory": "./database",
"__commentMasterPort": "HTTP port for the master to listen to",
"masterPort": 8080,
"__commentsslPort": "HTTPS port for the master to listen to. Set to 0 or an empty string to not listen for HTTPS.",
"sslPort": 443,
"sslCert": "database/certificates/cert.crt",
"sslPrivKey": "database/certificates/cert.key",
"__commentAuthSecret": "This is a long random string used for generating authentication tokens. Do not share. Regenerates when secret-api-token.txt is deleted.",
"masterAuthSecret": "",
"__comment3": "Your factorio.com username for factorio MP services, also username will be admin on new instances",
"username": "",
"__comment4": "Token from your factorio.com profile page. Required for MP and mod manager services.",
"token": "",
"__comment5": "General user settings",
"verify_user_identity": true,
"auto_pause": false,
"allow_commands": "admins-only",
"game_password": "",
"description": "This server runs Clusterio",
"visibility": {
"public": true,
"lan": true
},
}
+13 -1
View File
@@ -176,7 +176,19 @@ Code Style
----------
The style of the code is a bit of a mixed bag at the moment. But there
are at least two things that have been agreed upon.
are at least a few things that have been agreed upon.
### Strings
JavaScript and Lua code should use double quoted strings (`"string"`)
for all strings with the only exception that strings that contain double
quotes may use single quotes (i.e., `'String with "quotes"'`.) For
JavaScript template literals may also be used where appropriate.
There are places in the codebase that are using `'single'` quoted
strings. Should you be modifying such code use double quotes on new and
modified lines and leave the rest untouched.
### Indenting
+23
View File
@@ -0,0 +1,23 @@
{
"plugins": [
"plugins/markdown"
],
"tags": {
"allowUnknownTags": false
},
"opts": {
"destination": "docs/api",
"recurse": true
},
"source": {
"include": [
"master.js",
"slave.js",
"clusterctl.js",
"lib/",
"plugins/",
"routes/",
"routes.js"
]
}
}
+8 -5
View File
@@ -459,21 +459,24 @@ the metric and updating it somewhere in the plugin code. For example:
const { Counter } = require('lib/prometheus');
const fooMetric = new Counter(
'clusterio_foo_metric', "Measures the level of foo",
'clusterio_foo_frobber_foo_metric', "Measures the level of foo",
);
// Somewhere in the master plugin code
fooMetric.inc();
This works for master plugins, and the metric will be automatically
available through the /metric HTTP endpoint. For metrics that are
per-instance you must define an `instance_id` label and set it
accordingly, for example:
available through the /metric HTTP endpoint. It's recommended that
plugin metrics follow `clusterio_<plugin_name>_<metric_name>` as the
naming scheme.
For metrics that are per-instance you must define an `instance_id` label and
set it accordingly, for example:
const { Counter } = require('lib/prometheus');
const barMetric = new Gauge(
'clusterio_bar_metric', "Bar instance level",
'clusterio_foo_frobber_bar_metric', "Bar instance level",
{ labels: ['instance_id'] }
);
+2 -15
View File
@@ -4,6 +4,8 @@
*/
"use strict";
const { basicType } = require("lib/helpers");
/**
* Invalid Value exception
@@ -39,21 +41,6 @@ function splitOn(separator, string) {
return [string.slice(0, index), string.slice(index + separator.length)];
}
/**
* Return a string describing the type of the value passed
*
* Works the same as typeof, excpet that null and array types get their
* own string.
*
* @param value - value to return the type of.
* @returns {string} basic type of the value passed.
*/
function basicType(value) {
if (value === null) { return "null"; }
if (value instanceof Array) { return "array"; }
return typeof value;
}
/**
* Collection of config groups
*
+13
View File
@@ -42,6 +42,13 @@ MasterGroup.define({
optional: true,
initial_value: 8443,
});
MasterGroup.define({
name: "web_root",
title: "Web Root",
description: "Sub-path to where the web interface is hosted. Needed when proxying the web interface.",
type: "string",
initial_value: "/",
});
MasterGroup.define({
name: "tls_certificate",
title: "TLS Certificate",
@@ -165,6 +172,12 @@ InstanceGroup.finalize();
class FactorioGroup extends classes.ConfigGroup { }
FactorioGroup.groupName = "factorio";
FactorioGroup.define({
name: "version",
description: "Version of the game to run, use latest to run the latest installed version",
type: "string",
initial_value: "latest",
});
FactorioGroup.define({
name: "game_port",
description: "UDP port to run game on, uses a random port if null",
+7 -21
View File
@@ -8,6 +8,8 @@
"use strict";
const fs = require("fs-extra");
const { basicType } = require("lib/helpers");
/**
* Converts a Map with only string keys to a JavaScript object.
@@ -46,16 +48,8 @@ async function loadJsonAsMap(filePath) {
let content = await fs.readFile(filePath);
let parsed = JSON.parse(content);
if (parsed instanceof Array) {
throw new Error(`Expected object but got array`);
}
if (parsed === null) {
throw new Error(`Expected object but got null`);
}
if (typeof parsed !== "object") {
throw new Error(`Expected object but got ${typeof parsed}`);
if (basicType(parsed) !== "object") {
throw new Error(`Expected object but got ${basicType(parsed)}`);
}
return new Map(Object.entries(parsed));
@@ -104,21 +98,13 @@ async function loadJsonArrayAsMap(filePath) {
let content = await fs.readFile(filePath);
let parsed = JSON.parse(content);
if (parsed === null) {
throw new Error(`Expected array but got null`);
}
if (typeof parsed !== "object") {
throw new Error(`Expected array but got ${typeof parsed}`);
}
if (!(parsed instanceof Array)) {
throw new Error(`Expected array but got object`);
if (basicType(parsed) !== "array") {
throw new Error(`Expected array but got ${basicType(parsed)}`);
}
let map = new Map();
for (let element of parsed) {
if (element === null || element instanceof Array || typeof element !== "object") {
if (basicType(element) !== "object") {
throw new Error(`Expected all elements to be objects`);
}
+104 -11
View File
@@ -22,7 +22,16 @@ const errors = require("lib/errors");
* reading the changelog.txt in it.
*/
async function getVersion(changelogPath) {
let changelog = await fs.readFile(changelogPath, "utf-8");
let changelog;
try {
changelog = await fs.readFile(changelogPath, "utf-8");
} catch (err) {
if (err.code === "ENOENT") {
return null;
}
throw err;
}
for (let line of changelog.split(/[\r\n]+/)) {
let index = line.indexOf(":");
if (index !== -1) {
@@ -33,7 +42,80 @@ async function getVersion(changelogPath) {
}
}
throw new Error("Unable to determine the version of Factorio");
return null;
}
/**
* Comparison function for sorting Factorio version strings
*/
function versionOrder(a, b) {
let aParts = a.split(".").map(s => Number.parseInt(s, 10));
let bParts = b.split(".").map(s => Number.parseInt(s, 10));
for (let i = 0; i < aParts.length; i++) {
if (aParts[i] === bParts[i]) {
continue;
} else if (aParts[i] < bParts[i]) {
return 1;
} else {
return -1;
}
}
return 0;
}
/**
* Find factorio data directory of the given version
*
* Searches the given factorio dir for an installation of Factorio of
* the target version.
*/
async function findVersion(factorioDir, targetVersion) {
// There are two supported setups: having the factorio dir be the actual
// install directory, and having the factorio dir be a folder containing
// multiple install directories
let simpleVersion = await getVersion(path.join(factorioDir, "data", "changelog.txt"));
if (simpleVersion !== null) {
if (simpleVersion === targetVersion || targetVersion === "latest") {
return [path.join(factorioDir, "data"), simpleVersion];
}
throw new Error(
`Factorio version ${targetVersion} was requested, but install directory contains ${simpleVersion}`
);
}
let versions = new Map();
for (let entry of await fs.readdir(factorioDir, { withFileTypes: true })) {
if (!entry.isDirectory()) {
continue;
}
let version = await getVersion(path.join(factorioDir, entry.name, "data", "changelog.txt"));
if (version === null) {
continue;
}
if (version === targetVersion) {
return [path.join(factorioDir, entry.name, "data"), version];
}
versions.set(version, entry.name);
}
if (!versions.size) {
throw new Error(`Unable to find any Factorio install in ${factorioDir}`);
}
if (targetVersion === "latest") {
let latest = [...versions.keys()].sort(versionOrder)[0];
return [path.join(factorioDir, versions.get(latest), "data"), latest];
}
throw new Error(`Unable to find Factorio version ${targetVersion}`);
}
/**
@@ -337,22 +419,27 @@ const outputHeuristics = [
* output - invoked with parsed output from Factorio
* rcon-ready - invoked when the RCON client has connected
* game-ready - invoked when the server is finished starting up
* stopped - invoked when the sterver has exited
* exit - invoked when the sterver has exited
*/
class FactorioServer extends events.EventEmitter {
/**
* Create a Factorio server interface
*
* @param {string} dataDir - Data directory of the Factorio install.
* @param {string} factorioDir - Directory of the Factorio install(s).
* @param {string} writeDir - Directory to write runtime data to.
* @param {object} options - Optional parameters.
*/
constructor(dataDir, writeDir, options) {
constructor(factorioDir, writeDir, options) {
super();
this._dataDir = dataDir;
this._factorioDir = factorioDir;
this._writeDir = writeDir;
// Resolved in init
this._version = null;
this._dataDir = null;
this._targetVersion = options.version || "latest";
this.gamePort = options.gamePort || randomDynamicPort();
this.rconPort = options.rconPort || randomDynamicPort();
this.rconPassword = options.rconPassword;
@@ -418,7 +505,9 @@ class FactorioServer extends events.EventEmitter {
throw new Error(`Unknown IPC type '${type}'`);
}
this.emit(`ipc-${channel}`, content);
if (!this.emit(`ipc-${channel}`, content)) {
console.warn(`Warning: Unhandled ipc-${channel}`, content);
}
}
_handleOutput(line, source) {
@@ -489,7 +578,7 @@ class FactorioServer extends events.EventEmitter {
*/
async init() {
this._check(["new"]);
this._version = await getVersion(this.dataPath("changelog.txt"));
[this._dataDir, this._version] = await findVersion(this._factorioDir, this._targetVersion);
this.rconPassword = this.rconPassword || await generatePassword(10);
this._state = "init";
}
@@ -547,7 +636,7 @@ class FactorioServer extends events.EventEmitter {
this._gameReady = false;
this._unexpected = [];
this.emit('stopped');
this.emit("exit");
});
}
@@ -589,6 +678,7 @@ class FactorioServer extends events.EventEmitter {
);
this._attachStdio();
this._server.once("exit", () => this.emit("exit"));
try {
let [code, signal] = await events.once(this._server, 'exit');
@@ -788,7 +878,8 @@ class FactorioServer extends events.EventEmitter {
* Return path in data directory
*
* Creates a path using path.join with the given parts that's relative to
* the data directory of the Factorio server.
* the data directory of the Factorio server. Not valid before init has
* been called.
*/
dataPath(...parts) {
return path.join(this._dataDir, ...parts);
@@ -811,7 +902,7 @@ class FactorioServer extends events.EventEmitter {
*
* @returns {object} the parsed server-settings.
*/
async exampleSettings(dataDir) {
async exampleSettings() {
return JSON.parse(await fs.readFile(this.dataPath("server-settings.example.json"), "utf-8"));
}
@@ -832,6 +923,8 @@ module.exports = {
// For testing only
_getVersion: getVersion,
_versionOrder: versionOrder,
_findVersion: findVersion,
_randomDynamicPort: randomDynamicPort,
_generatePassword: generatePassword,
_LineSplitter: LineSplitter,
+23
View File
@@ -0,0 +1,23 @@
/**
* Collection of small utilites that are useful in multiple places.
* @module
*/
/**
* Return a string describing the type of the value passed
*
* Works the same as typeof, excpet that null and array types get their
* own string.
*
* @param value - value to return the type of.
* @returns {string} basic type of the value passed.
*/
function basicType(value) {
if (value === null) { return "null"; }
if (value instanceof Array) { return "array"; }
return typeof value;
}
module.exports = {
basicType,
}
-1
View File
@@ -1 +0,0 @@
module.exports = require("./isFactorioCommand");
@@ -1,23 +0,0 @@
const luamin = require("luamin");
function isFactorioCommand(command){
if(typeof command != "string"){
return false;
} else if(!command.includes("/silent-command ") && !command.includes("/c ")){
return false;
} else if(command[0] != "/"){
return false;
} else {
// check that what follows /c is valid LUA
let x = command.replace("/silent-command ", "").replace("/c ","");
let y
try{
y = luamin.minify(x);
} catch (e) {
return false;
}
// by this point it is surely a LUA command
return true;
}
}
module.exports = isFactorioCommand;
+5
View File
@@ -484,6 +484,11 @@ class Event {
}
}
messages.instanceStopped = new Event({
type: 'instance_stopped',
links: ['instance-slave'],
});
messages.updateInstances = new Event({
type: 'update_instances',
links: ['slave-master'],
+2 -2
View File
@@ -1,6 +1,6 @@
/**
Provides tools to manage mods in a clusterio install
@module modManager
@module
*/
const asTable = require("as-table").configure({delimiter: ' | '});
@@ -36,7 +36,7 @@ function formatBytes(size, decimals) {
/**
Lists mods for an instance as a table. Warning, handles printing internally.
@param {Instance} instance - Instance object, see client.js
@param {Instance} instance - Instance object, see slave.js
@example
var modManager = require("modManager");
+1 -1
View File
@@ -1,6 +1,6 @@
/**
Helper functions related to objects
@module objectOps
@module
*/
/**
+25 -3
View File
@@ -1,3 +1,7 @@
/**
* Plugin interfaces and utilities.
* @module
*/
"use strict";
const fs = require("fs-extra");
const path = require("path");
@@ -9,9 +13,27 @@ const errors = require("lib/errors");
* Base class for instance plugins
*/
class BaseInstancePlugin {
constructor(info, instance) {
constructor(info, instance, slave) {
/**
* The plugin's own info module
*/
this.info = info;
/**
* Instance the plugin started for
* @type {Instance}
*/
this.instance = instance;
/**
* Slave running the instance
*
* With the exepction of accessing the slave's config you should
* avoid ineracting with the slave object directly.
*
* @type {Slave}
*/
this.slave = slave;
}
/**
@@ -50,9 +72,9 @@ class BaseInstancePlugin {
async onStop() { }
/**
* Called when the instance is shutting down
* Called when the Factorio server exited
*/
async onExit() { }
onExit() { }
/**
* Called when the Factorio outputs a line
-11
View File
@@ -1,11 +0,0 @@
/**
An item with a count specified
@typedef {object} itemStack
@property {string} name The name of the item in the stack
@property {(number|string)} count The amount of items in the stack
@example
itemStack = {name:"iron-plate", count:"12"}
itemStack = {name:"copper-plate", count:972}
*/
+61 -7
View File
@@ -2,7 +2,7 @@
Clusterio master server. Facilitates communication between slaves through
a webserver, storing data related to slaves like production graphs.
@module clusterioMaster
@module
@author Danielv123
@example
@@ -72,6 +72,7 @@ app.set('views', ['views', 'plugins']);
// give ejs access to some interesting information
app.use(function(req, res, next){
res.locals.root = masterConfig.get("master.web_root");
res.locals.res = res;
res.locals.req = req;
res.locals.masterPlugins = masterPlugins;
@@ -85,11 +86,38 @@ require("./routes/api/getPictures")(app);
// Set folder to serve static content from (the website)
app.use(express.static('static'));
const endpointHitCounter = new prometheus.Gauge(
'clusterio_master_http_endpoint_hits_total', "How many requests a particular endpoint has gotten",
const endpointHitCounter = new prometheus.Counter(
"clusterio_master_http_endpoint_hits_total",
"How many requests a particular HTTP endpoint has gotten",
{ labels: ['route'] }
);
const wsMessageCounter = new prometheus.Counter(
"clusterio_master_websocket_message_total",
"How many messages have been received over WebSocket on the master server",
{ labels: ["direction"] }
);
const wsConnectionsCounter = new prometheus.Counter(
"clusterio_master_websocket_connections_total",
"How many WebSocket connections have been initiated on the master server"
);
const wsRejectedConnectionsCounter = new prometheus.Counter(
"clusterio_master_websocket_rejected_connections_total",
"How many WebSocket connections have been rejected during the handshake on the master server"
);
const wsActiveConnectionsGauge = new prometheus.Gauge(
"clusterio_master_websocket_active_connections",
"How many WebSocket connections are currently open to the master server"
);
const wsActiveSlavesGauge = new prometheus.Gauge(
"clusterio_master_active_slaves",
"How many slaves are currently connected to the master"
);
// Prometheus polling endpoint
async function getMetrics(req, res, next) {
endpointHitCounter.labels(req.route.path).inc();
@@ -132,6 +160,10 @@ async function getMetrics(req, res, next) {
results.push(prometheus.deserializeResult(result));
}
wsActiveConnectionsGauge.set(activeConnectors.size);
wsActiveSlavesGauge.set(slaveConnections.size);
let text = await prometheus.exposition(results);
res.set('Content-Type', prometheus.exposition.contentType);
res.send(text);
@@ -411,6 +443,18 @@ class ControlConnection extends BaseConnection {
db.instances.set(instanceId, instanceConfig);
}
async deleteInstanceRequestHandler(message, request) {
let instanceConfig = db.instances.get(message.data.instance_id);
if (!instanceConfig) {
throw new errors.RequestError(`Instance with ID ${message.data.instance_id} does not exist`);
}
if (instanceConfig.get('instance.assigned_slave') !== null) {
await this.forwardRequestToInstance(message, request);
}
db.instances.delete(message.data.instance_id);
}
async getInstanceConfigRequestHandler(message) {
let instanceConfig = db.instances.get(message.data.instance_id);
if (!instanceConfig) {
@@ -500,10 +544,6 @@ class SlaveConnection extends BaseConnection {
version: this._version,
});
this.connector.on('message', () => {
// XXX prometheusWsUsageCounter.labels('message', "other").inc();
});
this.connector.on("close", () => {
if (slaveConnections.get(this._id) === this) {
slaveConnections.delete(this._id);
@@ -731,6 +771,15 @@ let activeConnectors = new Map();
wss.on("connection", function (socket, req) {
console.log(`SOCKET | new connection from ${req.socket.remoteAddress}`);
// Track statistics
wsConnectionsCounter.inc();
socket.on("message", () => { wsMessageCounter.labels("in").inc(); });
let originalSend = socket.send;
socket.send = (...args) => {
wsMessageCounter.labels("out").inc();
return originalSend.call(socket, ...args);
};
// Start connection handshake.
socket.send(JSON.stringify({ seq: null, type: "hello", data: { version }}));
@@ -739,6 +788,7 @@ wss.on("connection", function (socket, req) {
let timeoutId = setTimeout(() => {
console.log(`SOCKET | closing ${req.socket.remoteAddress} after timing out on handshake`);
wsRejectedConnectionsCounter.inc();
socket.close(1008, "Handshake timeout");
pendingSockets.delete(socket);
}, 30*1000);
@@ -756,6 +806,7 @@ wss.on("connection", function (socket, req) {
+----------------------------------------------------------------+`
);
console.error(err);
wsRejectedConnectionsCounter.inc();
socket.close(1011, "Unexpected error");
});
});
@@ -775,12 +826,14 @@ async function handleHandshake(message, socket, req, attachHandler) {
message = JSON.parse(message);
} catch (err) {
console.log(`SOCKET | closing ${req.socket.remoteAddress} after receiving invalid JSON`);
wsRejectedConnectionsCounter.inc();
socket.close(1001, "Invalid JSON");
return;
}
if (!schema.clientHandshake(message)) {
console.log(`SOCKET | closing ${req.socket.remoteAddress} after receiving invalid handshake:`, message);
wsRejectedConnectionsCounter.inc();
socket.close(1001, "Bad handshake");
return;
}
@@ -815,6 +868,7 @@ async function handleHandshake(message, socket, req, attachHandler) {
if (!result.ok) {
console.error(`SOCKET | authentication failed for ${req.socket.remoteAddress}`);
wsRejectedConnectionsCounter.inc();
socket.close(4003, "Authentication failed");
return;
}
+4
View File
@@ -1,4 +1,8 @@
---------------------------------------------------------------------------------------------------
Version: 0.1.1
Features:
- Added support for Factorio 0.18.x
---------------------------------------------------------------------------------------------------
Version: 0.1.0
Major Features:
- API for mods to interact with Clusterio
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "clusterio_lib",
"version": "0.1.0",
"version": "0.1.1",
"title": "Clusterio Library (Alpha)",
"author": "Clusterio Team",
"homepage": "https://github.com/clusterio/factorioClusterio",
+16 -4
View File
@@ -2,6 +2,18 @@
-- Based on code from playerManager and trainTeleports
local serialize = {}
-- 0.17 compatibility
local supports_bar, get_bar, has_bar
if (pcall(function() local mods = script.active_mods end)) then
supports_bar = "supports_bar"
get_bar = "get_bar"
set_bar = "set_bar"
else
supports_bar = "hasbar"
get_bar = "getbar"
set_bar = "setbar"
end
-- Equipment Grids are serialized into an array of equipment entries
-- where ench entry is a table with the following fields:
-- n: name
@@ -181,8 +193,8 @@ end
-- It's also possible that the slot is empty but has a slot filter.
function serialize.serialize_inventory(inventory)
local serialized = {}
if inventory.hasbar() and inventory.getbar() <= #inventory then
serialized.b = inventory.getbar()
if inventory[supports_bar]() and inventory[get_bar]() <= #inventory then
serialized.b = inventory[get_bar]()
end
serialized.i = {}
@@ -227,8 +239,8 @@ function serialize.serialize_inventory(inventory)
end
function serialize.deserialize_inventory(inventory, serialized)
if serialized.b and inventory.hasbar() then
inventory.setbar(serialized.b)
if serialized.b and inventory[supports_bar]() then
inventory[set_bar](serialized.b)
end
local last_slot_index = 0
+2 -1
View File
@@ -10,7 +10,7 @@
"cover": "nyc npm test",
"fast-cover": "FAST_TEST=y nyc npm test",
"ci-cover": "nyc --reporter=lcovonly npm run-script test",
"start": "echo Run node client start [instance] to start slaves && node master",
"docs": "jsdoc --configure docs/jsdoc.json",
"install": "node lib/npmPostinstall",
"bundle-dependencies": "bundle-dependencies"
},
@@ -68,6 +68,7 @@
},
"devDependencies": {
"html5-validator": "^1.2.1",
"jsdoc": "^3.6.3",
"klaw": "^3.0.0",
"mocha": "^6.2.0",
"mocha-sinon": "^2.1.0",
+4 -4
View File
@@ -3,19 +3,19 @@ const { Gauge } = require('lib/prometheus');
const instancePlayerCount = new Gauge(
'clusterio_instance_player_count', "Amount of players connected to this cluster",
"clusterio_statistics_exporter_instance_player_count", "Amount of players connected to this cluster",
{ labels: ['instance_id'] }
);
const instanceGameTicksTotal = new Gauge(
'clusterio_instance_game_ticks_total', "Game tick an instance has progressed to",
"clusterio_statistics_exporter_instance_game_ticks_total", "Game tick an instance has progressed to",
{ labels: ['instance_id'] }
);
const instanceForceFlowStatistics = new Gauge(
'clusterio_instance_force_flow_statistics', "Items/fluids/enemies/buildings produced/built/killed by a force",
"clusterio_statistics_exporter_instance_force_flow_statistics", "Items/fluids/enemies/buildings produced/built/killed by a force",
{ labels: ['instance_id', 'force', 'statistic', 'direction', 'name'] },
);
const instanceGameFlowStatistics = new Gauge(
'clusterio_instance_game_flow_statistics', "Pollution produced/consumed in the game",
"clusterio_statistics_exporter_instance_game_flow_statistics", "Pollution produced/consumed in the game",
{ labels: ['instance_id', 'statistic', 'direction', 'name'] },
);
+2 -2
View File
@@ -4,12 +4,12 @@ const prometheus = require("lib/prometheus");
const prometheusNNDoleGauge = new prometheus.Gauge(
"clusterio_nn_dole_gauge",
"clusterio_subspace_storage_nn_dole_gauge",
"Current demand being supplied by Neural Network ; 1 means all demand covered, 0.5 means about half of each supply is covered, 0 means no items are given",
{ labels: ["resource"] }
);
const prometheusDoleFactorGauge = new prometheus.Gauge(
"clusterio_dole_factor_gauge",
"clusterio_subspace_storage_dole_factor_gauge",
"The current dole division factor for this resource",
{ labels: ["resource"] }
);
+6 -5
View File
@@ -22,16 +22,17 @@ class InstancePlugin extends plugin.BaseInstancePlugin {
async onStart() {
this.pingId = setInterval(() => {
this.instance.server.sendRcon(
"/sc __subspace_storage__ global.ticksSinceMasterPinged = 0"
"/sc __subspace_storage__ global.ticksSinceMasterPinged = 0", true
).catch(unexpectedError);
}, 5000);
let response = await this.info.messages.getStorage.send(this.instance);
// TODO Diff with dump of invdata produce minimal command to sync
let itemsJson = luaTools.escapeString(JSON.stringify(response.items));
await this.instance.server.sendRcon(`/sc __subspace_storage__ UpdateInvData("${itemsJson}", true)`);
await this.instance.server.sendRcon(`/sc __subspace_storage__ UpdateInvData("${itemsJson}", true)`, true);
}
async onStop() {
onExit() {
clearInterval(this.pingId);
}
@@ -66,7 +67,7 @@ class InstancePlugin extends plugin.BaseInstancePlugin {
}
let itemsJson = luaTools.escapeString(JSON.stringify(response.items));
await this.instance.server.sendRcon(`/sc __subspace_storage__ Import("${itemsJson}")`);
await this.instance.server.sendRcon(`/sc __subspace_storage__ Import("${itemsJson}")`, true);
}
// combinator signals ---------------------------------------------------------
@@ -85,7 +86,7 @@ class InstancePlugin extends plugin.BaseInstancePlugin {
items.push(["signal-unixtime", Math.floor(Date.now()/1000)]);
let itemsJson = luaTools.escapeString(JSON.stringify(items));
await this.instance.server.sendRcon(`/sc __subspace_storage__ UpdateInvData("${itemsJson}")`);
await this.instance.server.sendRcon(`/sc __subspace_storage__ UpdateInvData("${itemsJson}")`, true);
}
}
+14 -13
View File
@@ -10,32 +10,33 @@ const routes = require("./routes");
const dole = require("./dole");
const prometheusExportGauge = new prometheus.Gauge(
"clusterio_export_gauge",
const exportCounter = new prometheus.Counter(
"clusterio_subspace_storage_export_total",
"Resources exported by instance",
{ labels: ["instance_id", "resource"] }
);
const prometheusImportGauge = new prometheus.Gauge(
"clusterio_import_gauge",
const importCounter = new prometheus.Counter(
"clusterio_subspace_storage_import_total",
"Resources imported by instance",
{ labels: ["instance_id", "resource"] }
);
const masterInventoryGauge = new prometheus.Gauge(
"clusterio_subspace_storage_master_inventory",
"Amount of resources stored on master",
{ labels: ["resource"] }
);
class MasterPlugin extends plugin.BaseMasterPlugin {
async init() {
this.prometheusMasterInventoryGauge = new prometheus.Gauge(
"clusterio_master_inventory_gauge",
"Amount of resources stored on master",
{ labels: ["resource"] }
);
let root = this.master.config.get("master.web_root");
this.ui = {
sidebar: [
{
getHtml: () => `
<div class="nav-item mr-1">
<a class="nav-link align-middle" href="/subspace_storage/storage">Storage</a>
<a class="nav-link align-middle" href="${root}subspace_storage/storage">Storage</a>
</div>`,
},
],
@@ -91,7 +92,7 @@ class MasterPlugin extends plugin.BaseMasterPlugin {
for (let item of message.data.items) {
this.items.addItem(item[0], item[1]);
prometheusExportGauge.labels(String(instanceId), item[0]).inc(item[1]);
exportCounter.labels(String(instanceId), item[0]).inc(item[1]);
}
this.updateStorage();
@@ -152,7 +153,7 @@ class MasterPlugin extends plugin.BaseMasterPlugin {
if (itemsRemoved.length) {
for (let item of itemsRemoved) {
prometheusImportGauge.labels(String(instanceId), item[0]).inc(item[1]);
importCounter.labels(String(instanceId), item[0]).inc(item[1]);
}
this.updateStorage();
@@ -171,7 +172,7 @@ class MasterPlugin extends plugin.BaseMasterPlugin {
onMetrics() {
if (this.items) {
for (let [key, count] of this.items._items) {
this.prometheusMasterInventoryGauge.labels(key).set(Number(count) || 0);
masterInventoryGauge.labels(key).set(Number(count) || 0);
}
}
}
+1 -1
View File
@@ -62,7 +62,7 @@ function updateInventory() {
drawcontents(data);
}
}
xmlhttp.open("GET", "/api/inventory", true);
xmlhttp.open("GET", `${root}api/inventory`, true);
xmlhttp.send();
}
if(JSON.parse(localStorage.settings)["Periodically update storage screen"]) {
+1 -1
View File
@@ -1,7 +1,7 @@
<!doctype HTML>
<head>
<%- include("head") %>
<script defer src="/subspace_storage/storage.js"></script>
<script defer src="<%= locals.root %>subspace_storage/storage.js"></script>
</head>
<body>
+50 -41
View File
@@ -16,6 +16,12 @@ const luaTools = require('lib/luaTools');
const config = require('lib/config');
const instanceRconCommandsCounter = new prometheus.Counter(
"clusterio_instance_rcon_commands_total",
"How many commands have been sent to the instance",
{ labels: ["instance_id"] }
);
/**
* Keeps track of the runtime parameters of an instance
*/
@@ -29,15 +35,22 @@ class Instance extends link.Link{
this.config = instanceConfig;
let serverOptions = {
version: this.config.get("factorio.version"),
gamePort: this.config.get('factorio.game_port'),
rconPort: this.config.get('factorio.rcon_port'),
rconPassword: this.config.get('factorio.rcon_password'),
};
this.server = new factorio.FactorioServer(
path.join(factorioDir, "data"), this._dir, serverOptions
factorioDir, this._dir, serverOptions
);
let originalSendRcon = this.server.sendRcon;
this.server.sendRcon = (...args) => {
instanceRconCommandsCounter.labels(String(this.config.get("instance.id"))).inc();
return originalSendRcon.call(this.server, ...args);
}
this.server.on('output', (output) => {
link.messages.instanceOutput.send(this, { instance_id: this.config.get("instance.id"), output })
@@ -47,9 +60,22 @@ class Instance extends link.Link{
});
}
});
this.server.on("error", err => {
console.log(`Error in instance ${this.name}:`, err);
});
this.server.on("exit", err => {
link.messages.instanceStopped.send(this);
// Notify plugins of exit
for (let pluginInstance of this.plugins.values()) {
pluginInstance.onExit();
}
});
}
async init(pluginInfos) {
async init(pluginInfos, slave) {
await this.server.init();
// load plugins
@@ -61,7 +87,7 @@ class Instance extends link.Link{
// require plugin class and initialize it
let pluginLoadStarted = Date.now();
let { InstancePlugin } = require(`./plugins/${pluginInfo.name}/${pluginInfo.instanceEntrypoint}`);
let instancePlugin = new InstancePlugin(pluginInfo, this);
let instancePlugin = new InstancePlugin(pluginInfo, this, slave);
await instancePlugin.init();
this.plugins.set(pluginInfo.name, instancePlugin);
plugin.attachPluginMessages(this, pluginInfo, instancePlugin);
@@ -237,11 +263,6 @@ class Instance extends link.Link{
collector.removeAll({ instance_id: String(this.config.get("instance.id")) });
}
}
// Notify plugins of exit
for (let pluginInstance of this.plugins.values()) {
await pluginInstance.onExit();
}
}
async getMetricsRequestHandler() {
@@ -277,14 +298,10 @@ class Instance extends link.Link{
await this.writeServerSettings();
console.log("Creating save .....");
await symlinkMods(this, "sharedMods", console);
await this.server.create("world");
console.log("Clusterio | Successfully created save");
// Notify plugins of exit
for (let pluginInstance of this.plugins.values()) {
await pluginInstance.onExit();
}
}
async stopInstanceRequestHandler() {
@@ -359,9 +376,10 @@ async function discoverInstances(instancesDir, logger) {
}
class InstanceConnection extends link.Link {
constructor(connector, slave) {
constructor(connector, slave, instanceId) {
super('slave', 'instance', connector);
this.slave = slave;
this.instanceId = instanceId;
link.attachAllMessages(this);
for (let pluginInfo of slave.pluginInfos) {
@@ -399,6 +417,10 @@ class InstanceConnection extends link.Link {
event.send(instanceConnection, message.data);
}
}
async instanceStoppedEventHandler(message) {
this.slave.instanceConnections.delete(this.instanceId);
}
}
class SlaveConnector extends link.WebSocketClientConnector {
@@ -538,11 +560,11 @@ class Slave extends link.Link {
}
let [connectionClient, connectionServer] = link.VirtualConnector.makePair();
let instanceConnection = new InstanceConnection(connectionServer, this);
let instanceConnection = new InstanceConnection(connectionServer, this, instanceId);
let instance = new Instance(
connectionClient, instanceInfo.path, this.config.get("slave.factorio_directory"), instanceInfo.config
);
await instance.init(this.pluginInfos);
await instance.init(this.pluginInfos, this);
// XXX: race condition on multiple simultanious calls
this.instanceConnections.set(instanceId, instanceConnection);
@@ -590,25 +612,12 @@ class Slave extends link.Link {
async createSaveRequestHandler(message, request) {
let instanceId = message.data.instance_id;
let instanceConnection = await this._connectInstance(instanceId);
try {
await request.send(instanceConnection, message.data);
this.instanceConnections.delete(instanceId);
} catch (err) {
await this.stopInstance(instanceId);
throw err;
}
await request.send(instanceConnection, message.data);
}
async stopInstance(instanceId) {
let instanceConnection = this.instanceConnections.get(instanceId);
await link.messages.stopInstance.send(instanceConnection, { instance_id: instanceId });
this.instanceConnections.delete(instanceId);
}
async stopInstanceRequestHandler(message, request) {
await this.forwardRequestToInstance(message, request);
this.instanceConnections.delete(message.data.instance_id);
}
async deleteInstanceRequestHandler(message) {
@@ -760,13 +769,13 @@ async function symlinkMods(instance, sharedMods, logger) {
}
}
async function startClient() {
async function startSlave() {
// add better stack traces on promise rejection
process.on('unhandledRejection', r => console.log(r));
// argument parsing
const args = yargs
.scriptName("client")
.scriptName("slave")
.usage("$0 <command> [options]")
.option('config', {
nargs: 1,
@@ -775,7 +784,7 @@ async function startClient() {
type: 'string',
})
.command("config", "Manage Slave config", config.configCommand)
.command('start', "Start slave")
.command("run", "Run slave")
.demandCommand(1, "You need to specify a command to run")
.strict()
.argv
@@ -807,7 +816,7 @@ async function startClient() {
return;
}
// If we get here the command was start
// If we get here the command was run
await fs.ensureDir(slaveConfig.get("slave.instances_directory"));
await fs.ensureDir("sharedMods");
@@ -815,14 +824,14 @@ async function startClient() {
// Set the process title, shows up as the title of the CMD window on windows
// and as the process name in ps/top on linux.
process.title = "clusterioClient";
process.title = "clusterioSlave";
// make sure we have the master access token
if (slaveConfig.get("slave.master_token") === "enter token here") {
console.error("ERROR invalid config!");
console.error(
"Master server now needs an access token for write operations. As clusterio\n"+
"slaves depends upon this, please set your token using the command node client\n"+
"slaves depends upon this, please set your token using the command node slave\n"+
"config set slave.master_token <token>. You can retrieve your auth token from\n"+
"the master in secret-api-token.txt after running it once."
);
@@ -915,12 +924,12 @@ I version of clusterio. Expect things to break. I
+==========================================================+
`
);
startClient().catch(err => {
startSlave().catch(err => {
console.error(`
+---------------------------------------------------------------+
| Unexpected error occured while starting client, please report |
| it to https://github.com/clusterio/factorioClusterio/issues |
+---------------------------------------------------------------+`
+--------------------------------------------------------------+
| Unexpected error occured while starting slave, please report |
| it to https://github.com/clusterio/factorioClusterio/issues |
+--------------------------------------------------------------+`
);
console.error(err);
+2 -3
View File
@@ -1,6 +1,5 @@
(function (){
ensureSetting("Display offline slaves", "boolean", false);
ensureSetting("Periodically update storage screen", "boolean", true);
}())
@@ -199,7 +198,7 @@ function getImageFromName(name){
return "https://wiki.factorio.com/images/" + capitalizeFirstLetter(replaceAll(factorioLocale["fluid-name"][name], ' ', '_').toLowerCase()) + ".png";
} else if(factorioLocale["equipment-name"] && factorioLocale["equipment-name"][name]){
return "https://wiki.factorio.com/images/" + capitalizeFirstLetter(replaceAll(factorioLocale["equipment-name"][name], ' ', '_').toLowerCase()) + ".png";
} else return "/pictures/unknown-item.png";
} else return `${root}pictures/unknown-item.png`;
}
window.factorioLocale = {};
$.getJSON("/api/getFactorioLocale", (locale) => {
@@ -253,6 +252,6 @@ var imagelinks = {
"battery-mk2-equipment": "/pictures/Battery_MK2.png",
"personal-roboport-mk2-equipment": "/pictures/Personal_roboport_MK2.png",
}
var populateImageLinks = async () => (await getJSON("/api/getPictures")).forEach(img => imagelinks[img.name] = img.path)
var populateImageLinks = async () => (await getJSON(`${root}api/getPictures`)).forEach(img => imagelinks[img.name] = img.path)
populateImageLinks()
-26
View File
@@ -1,26 +0,0 @@
// ask master for slaves and render them nicely on a page with production graphs
setTimeout(function() {
var xmlhttp = new XMLHttpRequest();
xmlhttp.onreadystatechange = function() {
if(xmlhttp.readyState == 4 && xmlhttp.status == 200) {
var slaveData = JSON.parse(xmlhttp.responseText);
var HTML = "<p>Current connections:</p>"
for(i=0;i<Object.keys(slaveData).length;i++){
var key = Object.keys(slaveData)[i]
// Only show slaves connected last 2 minutes (depending on setting now)
if(JSON.parse(localStorage.settings)["Display offline slaves"]) var displayOffline = true;
if(Date.now() - slaveData[key].time < 120000 || displayOffline) {
let seenDate = moment(Number(slaveData[key].time)).format('DD.MM.YY, hh:mm:ss')
HTML += "<div class='slaveBox'>";
HTML += "<a href='nodeDetails?instanceID="+slaveData[key].unique+"'><h2>" + slaveData[key].instanceName + "</h2></a><p>ID: " + slaveData[key].unique + "</p><p>Last seen: "+seenDate+"</p><p>Online players: "+slaveData[key].playerCount+"</p><p>IP: "+slaveData[key].publicIP +":"+ slaveData[key].serverPort+"</p>"
HTML += "<p>Host: "+slaveData[key].mac+"</p>";
HTML += "</div>";
}
}
document.querySelector("#slaves > #display").innerHTML = HTML
}
}
xmlhttp.open("GET", "api/slaves", true);
xmlhttp.send();
}, 0)
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
+11 -11
View File
@@ -32,11 +32,11 @@ let slaveProcess;
let url = "https://localhost:4443/";
let token = jwt.sign({ id: "api" }, "TestSecretDoNotUse");
let instancesDir = path.join("test", "temp", "instances");
let databaseDir = path.join("test", "temp", "databse");
let masterConfigPath = path.join("test", "temp", "master-integration.json");
let slaveConfigPath = path.join("test", "temp", "slave-integration.json");
let controlConfigPath = path.join("test", "temp", "control-integration.json");
let instancesDir = path.join("temp", "test", "instances");
let databaseDir = path.join("temp", "test", "databse");
let masterConfigPath = path.join("temp", "test", "master-integration.json");
let slaveConfigPath = path.join("temp", "test", "slave-integration.json");
let controlConfigPath = path.join("temp", "test", "control-integration.json");
async function exec(...args) {
console.log(args[0]);
@@ -79,17 +79,17 @@ before(async function() {
await exec(`node master --config ${masterConfigPath} config set master.https_port 4443`);
await exec(`node master --config ${masterConfigPath} config set master.heartbeat_interval 0.25`);
await exec(`node client --config ${slaveConfigPath} config set slave.id 4`);
await exec(`node client --config ${slaveConfigPath} config set slave.name slave`);
await exec(`node client --config ${slaveConfigPath} config set slave.instances_directory ${instancesDir}`);
await exec(`node client --config ${slaveConfigPath} config set slave.master_url "${url}"`);
await exec(`node client --config ${slaveConfigPath} config set slave.master_token "${token}"`);
await exec(`node slave --config ${slaveConfigPath} config set slave.id 4`);
await exec(`node slave --config ${slaveConfigPath} config set slave.name slave`);
await exec(`node slave --config ${slaveConfigPath} config set slave.instances_directory ${instancesDir}`);
await exec(`node slave --config ${slaveConfigPath} config set slave.master_url "${url}"`);
await exec(`node slave --config ${slaveConfigPath} config set slave.master_token "${token}"`);
await exec(`node clusterctl --config ${controlConfigPath} control-config set control.master_url "https://localhost:4443/"`);
await exec(`node clusterctl --config ${controlConfigPath} control-config set control.master_token "${token}"`);
masterProcess = await spawn("master:", `node master --config ${masterConfigPath} run`, "All plugins loaded");
slaveProcess = await spawn("slave:", `node client --config ${slaveConfigPath} start`, "SOCKET | received ready from master");
slaveProcess = await spawn("slave:", `node slave --config ${slaveConfigPath} run`, "SOCKET | received ready from master");
});
after(async function() {
+1 -1
View File
@@ -13,7 +13,7 @@ require("./server")
describe("Integration of lib/factorio/patch", function() {
describe("patch()", function() {
let savePath = path.join("test", "temp", "integration", "saves", "test.zip");
let savePath = path.join("temp", "test", "integration", "saves", "test.zip");
it("should patch a freeplay game", async function() {
slowTest(this);
await factorio.patch(savePath, [{
+2 -2
View File
@@ -20,8 +20,8 @@ describe("Integration of lib/factorio/server", function() {
});
describe("class FactorioServer", function() {
let writePath = path.join("test", "temp", "integration");
let server = new factorio.FactorioServer(path.join("factorio", "data"), writePath, {});
let writePath = path.join("temp", "test", "integration");
let server = new factorio.FactorioServer("factorio", writePath, {});
let logFile;
before(async function() {
+13 -12
View File
@@ -1,5 +1,6 @@
const assert = require("assert").strict;
const fs = require("fs-extra");
const path = require("path");
const database = require("lib/database");
@@ -30,7 +31,7 @@ describe("lib/database", function() {
for (let type of badTypes) {
it(`should reject on ${type} JSON`, async function() {
await assert.rejects(
database.loadJsonAsMap(`test/json/${type}.json`),
database.loadJsonAsMap(path.join("test", "file", "json", `${type}.json`)),
new Error(`Expected object but got ${type}`)
);
});
@@ -38,21 +39,21 @@ describe("lib/database", function() {
it("should work on empty object JSON", async function() {
assert.deepEqual(
await database.loadJsonAsMap('test/json/object.json'),
await database.loadJsonAsMap(path.join("test", "file", "json", "object.json")),
new Map()
);
});
it("should work on object JSON", async function() {
assert.deepEqual(
await database.loadJsonAsMap('test/json/load_map.json'),
await database.loadJsonAsMap(path.join("test", "file", "json", "load_map.json")),
new Map([['a', 1], ['b', true]])
);
});
it("should give an empty Map for non-existant file", async function() {
assert.deepEqual(
await database.loadJsonAsMap('test/json/does-not-exist'),
await database.loadJsonAsMap(path.join("test", "file", "json", "does-not-exist")),
new Map()
);
});
@@ -60,7 +61,7 @@ describe("lib/database", function() {
describe("saveMapAsJson()", function() {
it("should save a mapping as JSON", async function() {
let testFile = 'test/temp/save_map.json';
let testFile = path.join("temp", "test", "save_map.json");
async function deleteTestFile() {
try {
await fs.unlink(testFile);
@@ -90,7 +91,7 @@ describe("lib/database", function() {
for (let type of badTypes) {
it(`should reject on ${type} JSON`, async function() {
await assert.rejects(
database.loadJsonArrayAsMap(`test/json/${type}.json`),
database.loadJsonArrayAsMap(path.join("test", "file", "json", `${type}.json`)),
new Error(`Expected array but got ${type}`)
);
});
@@ -98,35 +99,35 @@ describe("lib/database", function() {
it("should reject on null element", async function() {
await assert.rejects(
database.loadJsonArrayAsMap('test/json/array_null.json'),
database.loadJsonArrayAsMap(path.join("test", "file", "json", "array_null.json")),
new Error("Expected all elements to be objects")
);
});
it("should reject on element missing id", async function() {
await assert.rejects(
database.loadJsonArrayAsMap('test/json/array_object.json'),
database.loadJsonArrayAsMap(path.join("test", "file", "json", "array_object.json")),
new Error("Expected all elements to have an id property")
);
});
it("should work on empty array JSON", async function() {
assert.deepEqual(
await database.loadJsonArrayAsMap('test/json/array.json'),
await database.loadJsonArrayAsMap(path.join("test", "file", "json", "array.json")),
new Map()
);
});
it("should work on array JSON", async function() {
assert.deepEqual(
await database.loadJsonArrayAsMap('test/json/load_array_map.json'),
await database.loadJsonArrayAsMap(path.join("test", "file", "json", "load_array_map.json")),
new Map([['a', {id: 'a'}], ['b', {id: 'b'}]])
);
});
it("should give an empty Map for non-existant file", async function() {
assert.deepEqual(
await database.loadJsonArrayAsMap('test/json/does-not-exist'),
await database.loadJsonArrayAsMap(path.join("test", "file", "json", "does-not-exist")),
new Map()
);
});
@@ -134,7 +135,7 @@ describe("lib/database", function() {
describe("saveMapAsJsonArray()", function() {
it("should save a mapping as JSON", async function() {
let testFile = 'test/temp/save_array_map.json';
let testFile = path.join("temp", "test", "save_array_map.json");
async function deleteTestFile() {
try {
await fs.unlink(testFile);
+1 -1
View File
@@ -212,7 +212,7 @@ describe("lib/factorio", function() {
it("should throw on unknown scenario", async function() {
let zip = new jszip();
zip.file("world/control.lua", "-- unknown\n");
let zipPath = path.join("test", "temp", "patch.zip");
let zipPath = path.join("temp", "test", "patch.zip");
await fs.outputFile(zipPath, await zip.generateAsync({type: "nodebuffer"}))
await assert.rejects(
+66 -5
View File
@@ -13,10 +13,71 @@ describe("lib/factorio/server", function() {
let version = await factorio._getVersion(path.join("test", "file", "changelog-test.txt"));
assert.equal(version, "0.1.1");
});
it("should throw if unable to find the version", async function() {
it("should return null if unable to find the version", async function() {
let version = await factorio._getVersion(path.join("test", "file", "changelog-bad.txt"));
assert.equal(version, null);
});
it("should return null if file does not exist", async function() {
let version = await factorio._getVersion(path.join("test", "file", "does-not-exist.txt"));
assert.equal(version, null);
});
});
describe("_versionOrder()", function() {
it("should sort an array of versions", function() {
let versions = ["1.2.3", "0.1.4", "0.1.2", "1.2.3", "0.1.5", "1.10.2"];
versions.sort(factorio._versionOrder);
assert.deepEqual(
versions,
["1.10.2", "1.2.3", "1.2.3", "0.1.5", "0.1.4", "0.1.2"]
);
});
});
describe("_findVersion()", function() {
it("should find a given install dir with latest as target", async function() {
let installDir = path.join("test", "file", "factorio");
let [dir, version] = await factorio._findVersion(installDir, "latest");
assert.equal(dir, path.join(installDir, "data"));
assert.equal(version, "0.1.1");
});
it("should find a given install dir with correct version as target", async function() {
let installDir = path.join("test", "file", "factorio");
let [dir, version] = await factorio._findVersion(installDir, "0.1.1");
assert.equal(dir, path.join(installDir, "data"));
assert.equal(version, "0.1.1");
});
it("should reject if the install dir version does not match target version", async function() {
let installDir = path.join("test", "file", "factorio");
await assert.rejects(
factorio._getVersion(path.join("test", "file", "changelog-bad.txt")),
new Error("Unable to determine the version of Factorio")
factorio._findVersion(installDir, "0.1.2"),
new Error("Factorio version 0.1.2 was requested, but install directory contains 0.1.1")
);
});
it("should search given directory for latest Factorio install", async function() {
let installDir = path.join("test", "file");
let [dir, version] = await factorio._findVersion(installDir, "latest");
assert.equal(dir, path.join(installDir, "factorio", "data"));
assert.equal(version, "0.1.1");
});
it("should search given directory for given Factorio install", async function() {
let installDir = path.join("test", "file");
let [dir, version] = await factorio._findVersion(installDir, "0.1.1");
assert.equal(dir, path.join(installDir, "factorio", "data"));
assert.equal(version, "0.1.1");
});
it("should reject if no factorio install with the given version was found", async function() {
let installDir = path.join("test", "file");
await assert.rejects(
factorio._findVersion(installDir, "0.1.2"),
new Error("Unable to find Factorio version 0.1.2")
);
});
it("should reject if no factorio install was found", async function() {
let installDir = path.join("test", "file", "instances");
await assert.rejects(
factorio._findVersion(installDir, "latest"),
new Error(`Unable to find any Factorio install in ${installDir}`)
);
});
});
@@ -105,8 +166,8 @@ describe("lib/factorio/server", function() {
});
describe("class FactorioServer", function() {
let writePath = path.join("test", "temp", "should_not_exist");
let server = new factorio.FactorioServer(path.join("test", "file", "factorio", "data"), writePath, {});
let writePath = path.join("temp", "test", "server");
let server = new factorio.FactorioServer(path.join("test", "file", "factorio"), writePath, {});
describe(".init()", function() {
it("should not throw on first call", async function() {
+1 -1
View File
@@ -5,7 +5,7 @@ const path = require('path');
const fileOps = require('lib/fileOps');
describe("fileOps.js", function(){
let baseDir = path.join("test", "temp", "fileOps");
let baseDir = path.join("temp", "test", "fileOps");
async function setupTestingEnv() {
await fs.ensureDir(path.join(baseDir, "test", "folder"));
await fs.ensureDir(path.join(baseDir, "test", "another folder"));
+1 -1
View File
@@ -5,7 +5,7 @@ const path = require("path");
const generateSSLcert = require("lib/generateSSLcert")
describe("generateSSLcert.js(options)", ()=>{
let testDir = "test/temp/certTest";
let testDir = path.join("temp", "test", "certTest");
before(async function() {
await fs.remove(testDir);
});
+17
View File
@@ -0,0 +1,17 @@
const assert = require("assert").strict;
const helpers = require("lib/helpers");
describe("lib/helpers", function() {
describe("basicType()", function() {
it("should return the expected values", function() {
assert.equal(helpers.basicType("s"), "string");
assert.equal(helpers.basicType(null), "null");
assert.equal(helpers.basicType(undefined), "undefined");
assert.equal(helpers.basicType([1, 2]), "array");
assert.equal(helpers.basicType(7), "number");
assert.equal(helpers.basicType({ a: 1 }), "object");
});
});
});
@@ -1,18 +0,0 @@
const isFactorioCommand = require("lib/isFactorioCommand");
const assert = require("assert");
describe("inventoryImports/isFactorioCommand.js()", function(){
it("Returns false if input is not a factorio command", function(){
assert(isFactorioCommand({}) == false, "This is not even a string so should return false");
assert(isFactorioCommand("Howdy boys!") == false, "This is not a command so should return false");
assert(isFactorioCommand("/c this ain't a command because its invalid LUA") == false, "This is not valid LUA so it should return false");
assert(isFactorioCommand("Trying to trick the system /c game.print('Hello world!')") == false, "this is still not a command");
});
it("Returns true if it is a valid command", function(){
assert(isFactorioCommand("/c game.print('hello world')"), "This is a valid command, should output true not false")
});
it("Fails to understand that leading spaces are valid", function(){
assert(isFactorioCommand(" /c game.print('Hello world!')") == false, "This should be true but often returns false unintentionally. If you see this you probably fixed the bug so change this test, OK?")
});
});
+2 -2
View File
@@ -18,7 +18,7 @@ describe("lib/plugin", function() {
it("should define defaults for hooks", async function() {
await instancePlugin.onStart();
await instancePlugin.onStop();
await instancePlugin.onExit();
instancePlugin.onExit();
await instancePlugin.onOutput({});
})
});
@@ -35,7 +35,7 @@ describe("lib/plugin", function() {
});
describe("loadPluginInfos()", function() {
let baseDir = path.join('test', 'temp', 'plugin');
let baseDir = path.join("temp", "test", "plugin");
let emptyDir = path.join(baseDir, 'emptyDir');
let emptyPlugin = path.join(baseDir, 'emptyPlugin');
let testPlugin = path.join(baseDir, 'testPlugin');
-1
View File
@@ -3,4 +3,3 @@
require('lib/manager/modManager');
require('lib/npmPostinstall.js');
require('lib/typedef');
+12 -12
View File
@@ -4,9 +4,9 @@ const fs = require('fs-extra');
const link = require("lib/link");
const config = require("lib/config");
const client = require("../client");
const slave = require("../slave");
describe("Client testing", function() {
describe("Slave testing", function() {
before(function() {
config.InstanceConfig.finalize();
});
@@ -17,7 +17,7 @@ describe("Client testing", function() {
let instanceConfig = new config.InstanceConfig();
await instanceConfig.init();
instanceConfig.set("instance.name", "foo");
instance = new client._Instance(new link.VirtualConnector(), "dir", "factorioDir", instanceConfig);
instance = new slave._Instance(new link.VirtualConnector(), "dir", "factorioDir", instanceConfig);
});
describe(".name", function() {
@@ -38,11 +38,11 @@ describe("Client testing", function() {
describe("checkFilename()", function() {
it("should allow a basic name", function() {
client._checkFilename("file");
slave._checkFilename("file");
});
function check(item, msg) {
assert.throws(() => client._checkFilename(item), new Error(msg));
assert.throws(() => slave._checkFilename(item), new Error(msg));
}
it("should throw on non-string", function() {
@@ -89,7 +89,7 @@ describe("Client testing", function() {
});
describe("symlinkMods()", function() {
let testDir = path.join("test", "temp", "symlink");
let testDir = path.join("temp", "test", "symlink");
let discardingLogger = {
warning: function() { },
log: function() { },
@@ -108,14 +108,14 @@ describe("Client testing", function() {
let instanceConfig = new config.InstanceConfig();
await instanceConfig.init();
instanceConfig.set("instance.name", "test");
instance = new client._Instance(
instance = new slave._Instance(
new link.VirtualConnector(), path.join(testDir, "instance"), "factorioDir", instanceConfig
);
await fs.outputFile(instance.path("mods", "mod_i.zip"), "i");
});
it("should link mods and data files", async function() {
await client._symlinkMods(instance, path.join(testDir, "shared"), discardingLogger);
await slave._symlinkMods(instance, path.join(testDir, "shared"), discardingLogger);
assert.equal(await fs.readFile(instance.path("mods", "mod_a.zip"), "utf-8"), "a");
assert.equal(await fs.readFile(instance.path("mods", "mod_b.zip"), "utf-8"), "b");
@@ -125,14 +125,14 @@ describe("Client testing", function() {
it("should ignore directories", async function() {
await fs.ensureDir(path.join(testDir, "shared", "dir"));
await client._symlinkMods(instance, path.join(testDir, "shared"), discardingLogger);
await slave._symlinkMods(instance, path.join(testDir, "shared"), discardingLogger);
assert(!await fs.exists(instance.path("mods", "dir"), "utf-8"), "dir was unxpectedly linked");
});
it("should ignore files", async function() {
await fs.outputFile(path.join(testDir, "shared", "file"), "f");
await client._symlinkMods(instance, path.join(testDir, "shared"), discardingLogger);
await slave._symlinkMods(instance, path.join(testDir, "shared"), discardingLogger);
assert(!await fs.exists(instance.path("mods", "file"), "utf-8"), "dir was unxpectedly linked");
});
@@ -144,7 +144,7 @@ describe("Client testing", function() {
}
await fs.unlink(path.join(testDir, "shared", "mod_a.zip"));
await client._symlinkMods(instance, path.join(testDir, "shared"), discardingLogger);
await slave._symlinkMods(instance, path.join(testDir, "shared"), discardingLogger);
await assert.rejects(fs.lstat(instance.path("mods", "mod_a.zip")), { code: "ENOENT" });
assert.equal(await fs.readFile(instance.path("mods", "mod_b.zip"), "utf-8"), "b");
@@ -157,7 +157,7 @@ describe("Client testing", function() {
it("should discover test instance", async function() {
let logger = { log: () => {}, error: () => {} };
let instancePath = path.join("test", "file", "instances");
let instanceInfos = await client._discoverInstances(instancePath, logger);
let instanceInfos = await slave._discoverInstances(instancePath, logger);
let referenceConfig = new config.InstanceConfig();
await referenceConfig.init();
+8 -17
View File
@@ -1,12 +1,13 @@
<title>Clusterio</title>
<link rel="stylesheet" type="text/css" href="/style.css">
<script src="/external/jquery.js"></script>
<script src="/external/jquery-ui.js"></script>
<script src="/external/moment.min.js"></script>
<script src="/external/mousetrap.min.js"></script>
<script src="/main.js"></script>
<link href="/bootstrap.min.css" rel="stylesheet">
<link href="/external/css/Roboto.css" rel="stylesheet">
<script>root = "<%= locals.root %>";</script>
<script src="<%= locals.root %>external/jquery.js"></script>
<script src="<%= locals.root %>external/jquery-ui.js"></script>
<script src="<%= locals.root %>external/moment.min.js"></script>
<script src="<%= locals.root %>external/mousetrap.min.js"></script>
<script src="<%= locals.root %>main.js"></script>
<link href="<%= locals.root %>bootstrap.min.css" rel="stylesheet">
<link href="<%= locals.root %>external/css/Roboto.css" rel="stylesheet">
<link href="https://fonts.googleapis.com/css?family=Orbitron" rel="stylesheet">
<link rel="stylesheet" href="https://use.fontawesome.com/releases/v5.6.0/css/all.css" integrity="sha384-aOkxzJ5uQz7WBObEZcHvV5JvRW3TUc2rNPA7pe3AwnsUohiw1Vj2Rgx2KSOkF5+h" crossorigin="anonymous">
<link rel="apple-touch-icon" sizes="57x57" href="/favicon/apple-icon-57x57.png">
@@ -26,13 +27,3 @@
<meta name="msapplication-TileColor" content="#ffffff">
<meta name="msapplication-TileImage" content="/ms-icon-144x144.png">
<meta name="theme-color" content="#ffffff">
<!-- Global site tag (gtag.js) - Google Analytics -->
<script async src="https://www.googletagmanager.com/gtag/js?id=UA-124032859-1"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'UA-124032859-1');
gtag('config', 'AW-1000520291');
</script>
+3 -3
View File
@@ -1,12 +1,12 @@
<nav class="navbar nav-pills nav-fill">
<div class="nav-item col-3">
<a class="navbar-brand border-0" href="/">
<img src="/pictures/logo.png" width="50" height="50" class="d-inline-block align-middle " alt=""/>
<img src="<%= locals.root %>pictures/logo.png" width="50" height="50" class="d-inline-block align-middle " alt=""/>
<h3 class=" ml-3 nav-title">Clusterio</h3>
</a>
</div>
<div class="nav-item mr-1">
<a class="nav-link align-middle" href="/nodes">Slaves</a>
<a class="nav-link align-middle" href="<%= locals.root %>nodes">Slaves</a>
</div><%
for (let plug of locals.masterPlugins.values()) {
if (
@@ -24,6 +24,6 @@ for (let plug of locals.masterPlugins.values()) {
}
} %>
<div class="nav-item">
<a class="nav-link align-middle" href="/settings">Settings</a>
<a class="nav-link align-middle" href="<%= locals.root %>settings">Settings</a>
</div>
</nav>
-1
View File
@@ -1,7 +1,6 @@
<!doctype HTML>
<head>
<%- include("head") %>
<!-- <script src="slaveDisplay.js"></script> -->
</head>
<body>