Commit Graph
209 Commits
Author SHA1 Message Date
Hornwitser 75067e992d Add WebSocket usage statistics
Add Prometheus metrics for messages sent, connections made, connections
rejected, active connections, and active slaves.
2020-04-04 12:16:57 +02:00
Hornwitser 8ebf0fc754 Add master server Web Root config
Add web_root option to the master server and use it create URLs for
resources hosted by it.  This permits hosting the master server web
interface behind reverse proxy under a dedicated path.
2020-04-04 10:05:37 +02:00
Hornwitser 3343d53d75 Allow deleting unassigned instances
Handle delete instance requests for instances that are not assigned to a
slave.
2020-04-03 22:22:23 +02:00
Hornwitser 5c16809b5b JSDoc use autodetected module name
Change @module <module name> entries to just @module as JSDoc is able to
automatically determine the module name.
2020-04-03 20:51:28 +02:00
Hornwitser 4c26fcee3a Fix connection shutdown logic
Fix the WebSocket link not being properly teared down on the master
side causing the handshake to deadlock when the slave tried to connect
again.
2020-04-03 20:44:20 +02:00
Hornwitser d4e072f376 Rewrite reconnection logic
Implement proper session tracking and resume on reconnect for the
WebSocket server.
2020-04-02 21:58:44 +02:00
Hornwitser 677512512e Replace socket.io with plain WebSocket
Replace socket.io with plain WebSocket using the ws library as it
doesn't really provide any useful tools over plain WebSocket when it
comes to implementing reliable communication that works across
reconnects.  The widespread browser support for WebSocket also means
it has little value to add for the web interface over WebSocket.
2020-03-22 02:10:54 +01:00
Hornwitser d9fe408c42 Fix master web interface
Fix the breaking changes to the plugin struture and remove slave
information that is no longer present.
2020-03-13 22:46:17 +01:00
Hornwitser 090fbd6dff Expose master interfaces to plugin
Expose Express.js app, master config, instance and slave database, slave
connections and metrics to the master plugins.
2020-03-13 22:46:17 +01:00
Hornwitser cb1d744060 Handle errors during master plugin initialization
Catch errors thrown when constructing or initializing master pluigns and
wrap them in PluginError to ensure that they are properly reported as
plugin errors.
2020-03-13 22:46:17 +01:00
Hornwitser 5e5149dc93 Implement new configuration system
Replace current configuration mess with a trimmed down version of the
original vision of #200.  Configs are collection of config groups that
have fields that can be set.  These groups allow organizing the fields
as well as opertunity for plugins to add their own groups.  By default
plugins have an auto generated group with the enabled field.
2020-03-09 20:48:13 +01:00
Hornwitser 26eefb0476 Rewrite statistics export
Add automatic export of metrics from slaves to the master server.  And
rewrite the statistics export plugin to use this new system.
2019-12-20 21:26:29 +01:00
Hornwitser 1f6574a89c Improve master shutdown
Close active connections and http servers during master shutdown and let
node exit on its own.  If this fails with an error set output streams to
blocking and report it then shutdown with process.exit.

Also add double SIGINT handling to immediately shutdown.
2019-12-20 02:37:27 +01:00
Hornwitser bcceae687f Reimplement plugin system
Implement a new plugin system based subclassing pre-defined base clases
and communicating with pre-defined link messages.
2019-12-14 17:48:56 +01:00
Hornwitser 513651393b Implement forwarding to instances for events
Implement forward to instance for Event messages.
2019-12-14 17:31:02 +01:00
Hornwitser e5115df28b Implement broadcast for events
Add broadcastTo property to events that broadcasts them to all entities
of the specified target that's downstream to the connection it was sent
to.
2019-12-13 22:37:58 +01:00
Hornwitser 21d9fdff7b Remove old plugin system
Remove the old client/instance fused plugin system along with
instanceManagement in client.js and getPlugins in master.js.
2019-12-13 22:37:49 +01:00
Hornwitser 5d8fa4e4ce Refactor instance communication
Change Instance class to be a subclass of Link and refactor the
communication to and establisment of instances to use a virtual
connection between the slave and the instance.  This is simplifies the
implementation of plugin communication for instances.

Replace InstanceRequest with a forwardTo option to Request which does
the task of forwarding in a much cleaner way.
2019-12-13 21:57:05 +01:00
Hornwitser 62afcaadab Use shared StartupError class
Remove separate StartupError class definition in master.js in favor of
the shared StartupError class in lib/error.js.
2019-12-13 21:39:54 +01:00
Hornwitser 3be264c9a9 Add integration testing of commands
Add testing of the clusterctl commands against the master and slave
code.
2019-11-30 00:08:10 +01:00
Hornwitser 3bccbac496 Refactor to a connector based concept in lib/link
Remove connection and client subclasses of Link and put the connection
logic into a "connector" class.  The connector deals with sending
and receiving messages, handling the connection and reconnection of lost
connections and giving sequence numbers to messages.

This move the logic of the actual connection away from the link
sub-classes, which in turn means the connection should be a lot easier
to mock in tests.
2019-11-30 00:08:10 +01:00
Hornwitser 039b41d3e6 Move events and requests into messages
Merge the events and requests categories in lib/link into messages and
use the class name of the message to determine which handler to call.

Also swaps the order of the arguments for forwardInstanceRequest to be
consistent with the rest of the lib/link code and fixes up the docs for
requests and events.
2019-11-30 00:08:10 +01:00
Hornwitser 1dea5fddbe Forward Factorio logs to clusterctl
Add instance output event that is forwarded by the master server to
control connections that are subscribing to them.  Add code subscribing
to the relevant instance output when dealing with instances in
clusterctl.
2019-11-23 15:18:20 +01:00
Hornwitser 7a2abaa5db Daemonize slave
Rework client.js to support running multiple instances at once and be
managed remotely and implement the neccessary communication
infrastructure for this.

Due to circular dependencies on the implementation of this the master
branch is significantly broken after this.
2019-11-23 15:16:47 +01:00
Hornwitser 27fc68da0f Remove remoteCommands plugin and interface
Remove the half baked remote rcon command interface in preparation for a
better version based on a new socket.io communication layer.
2019-11-20 22:37:25 +01:00
Hornwitser d38e9a53c4 Remove broken combinator signaling
Remove badly implemented and broken remote combinator signaling.
2019-11-20 22:37:25 +01:00
Hornwitser f3171c2feb Remove support for uploading mods to master
Remove the uploadMod and checkMod APIs as these have become more or less
obsolete after Factorio implementd syncing mods to the server in game.
2019-11-20 22:37:23 +01:00
Hornwitser 7359f9075e Port increase limit of Json payloads to master server
The default limit of 100kb to JSON payloads of bodyParser can be
exceeded when running modded games with researchSync.  Increase the
limit to 10mb as a bandaid to fix this issue.

This is a port of 25eb5e5.
2019-11-15 15:38:56 +01:00
Hornwitser e4aae88614 Move clusterioMod specific code into the plugin
Move item database, tests, endpoint definitions for /api/place,
/api/remove, /api/inventory, /api/inventoryAsObject, and the web
interface view for the storage page into the clusterioMod plugin.
2019-10-28 19:46:22 +01:00
Hornwitser ff3fc3b94b Remove .js suffix in require paths
Remove the .js suffix in require paths to make requires consistent
across the project.
2019-10-28 19:45:34 +01:00
Hornwitser 6bde8a2219 Remove .js suffix from node invocations
Remove the .js suffix from help texts of node invocations as well as
actual invocations as this is not necessary.  Removing it opens up the
possibility of restructuring the code into a directory with the same
name.
2019-10-28 19:45:32 +01:00
Hornwitser 93480c8b29 Refactor getFactorioLocale
Move getFactorioLocale to the new lib/factorio namespace and refactor it
to use async functions instead of callbacks.  Split tests into
integration with factorio and testing of the function itself.

Also removes the cacheing from the function itself as this breaks the
isolated testing of it.
2019-10-14 18:56:46 +02:00
Hornwitser a62e8d17a3 Move ejs templates to views/
Move the ejs templates to the default Express.js views/ folder and
change their extension to .ejs.  Reconfigure Express.js to look for
templates in the views folders as well as the plugins folder.  This
enables plugins to render templates using paths of following format
"pluginName/path/to/page-or-view".
2019-09-18 16:54:11 +02:00
Hornwitser dd865b42b5 Fix incorrect handling of instance ids
After 651b09b the instance ID would be treated differently depending on
wheteher it was a string or a number in the master server.  This caused
researchSync to break as it would send the ID as a number while the
master server would have it stored as a string.

Fix by coercing instanceID to a string in the master HTTP API.
2019-09-16 16:04:08 +02:00
Hornwitser 500c9584eb Remove commented out itemStats code
Remove old commented out code dealing with item statistics that is no
longer relevant.
2019-09-14 00:56:22 +02:00
Hornwitser 8aa66e9e68 Port authentication of socket.io server
Authenticate connections made to the socket.io server via a token query
parameter that's validated in a similar manner to the x-access-token for
HTTP requests.

This is a port of 0c45e57.
2019-09-11 16:59:11 +02:00
AlienXAXS ea5c0d5aa3 Fixes editSlaveMeta and db AddItem errors (#228)
Fixed editSlaveMeta to pass a number to db.addItem method
2019-09-10 17:52:09 +02:00
TheAIBot e14f7c22fd Merge pull request #222 from Hornwitser/config-part-1
Config Rewrite Part 1
2019-09-05 11:33:43 +02:00
Hornwitser a31778c7ff Remove unused config parameter to pluginManager
Remove the unused library initialization parameter config to
pluginManager and reorganize pluginManager to no longer be defined by a
function.
2019-09-04 20:43:13 +02:00
Hornwitser 35546b4b18 Move loading of config in master to startServer
Move the loading of the config file in the master server to the
startServer function.  Remove config.json.test as it's no longer needed.
2019-09-04 20:43:13 +02:00
Hornwitser 651b09b613 Refactor database and item retreival
Change the ugly global slaves object to be a member of the db object and
implement the items database as a separate integrety verifying class in
lib/database.
2019-09-04 20:43:13 +02:00
Hornwitser 23ce86aef6 Fix master socket.io broken by a3fc1dd
Add missing attach calls to the master socket.io object that was
accidentally removed by the refactoring in a3fc1dd.  Unfortunately I've
been unable to determine if socket.io actually supports listening on
multiple servers at the same time, this needs further investigation.
2019-09-04 17:39:49 +02:00
Hornwitser 5d6f6de379 Export app as a separate entry from master.js
Change the module exports from master.js to allow additional objects to
be passed to allow better unit testing.
2019-09-04 17:36:06 +02:00
Hornwitser af2471b427 Remove config dependency in lib/authenticate
Restructure lib/authenticate to set the authentication secret at a later
point in time via setAuthSecret.  This breaks playerManager.
2019-09-04 17:36:06 +02:00
Hornwitser 703423ae54 Add development branch warning at startup
Add a big banner based warning telling this is the development branch
for clusterio to the startup of client.js and master.js.
2019-09-01 23:34:52 +02:00
Hornwitser 84f48e36f4 Improve error reporting at startup
Add StartupError that differentiates unhandled errors from errors with
known causes and use it to only print the "please report" banner when an
unhandled error occurs.

Also makes the error reporting more visible in the console.
2019-09-01 23:24:32 +02:00
Hornwitser b73b202513 Remove remanent remoteMap code
Remove remoteMap specific code left behind after the removal of the
remoteMap plugin in 6415784.  The plugin interface should be versatile
enough to allow all of this code to reside inside the plugin itself.
2019-08-29 11:38:57 +02:00
Hornwitser ab2a377ca5 Remove Prometheus pushgateway code
Remove code pushing metrics to a hardcoded Prometheus pushgateway that
is no longer in operation.  There's no point supporting the use of a
pushgateway as there's already a full client implemented and since the
master server is not ephemeral or batch job it shouldn't be using a
pushgateway in the first place.
2019-08-24 11:25:40 +02:00
Hornwitser ea2f291025 Remove reference to old authentication scheme
Remove long comment and commented out code about an old sanity check in
/api/place.
2019-08-24 11:25:40 +02:00
Hornwitser 3e80e40223 Move database and masterMods folder creation
Move the code creating the database and masterMods folders into the
startServer function in master.js.
2019-08-24 11:25:39 +02:00