Commit Graph
223 Commits
Author SHA1 Message Date
Hornwitser 260f976fa3 Implement export of item icons and locale
Add mod and command for exporting the Factorio locale including mods,
and item icons assemled into a sprite sheet including mods.
2020-05-26 01:11:49 +02:00
Hornwitser 3211fbe985 Notify master about instance status update
Send updates to the master server when the running status of instances
change and add onInstanceStatusChanged hook to master plugins.
2020-05-11 18:10:11 +02:00
Hornwitser a8e29fdc56 Separate instance from config in master
Separate the instance object and the config that belongs to the instance
object in the instances db held in memory on the master.
2020-05-11 18:07:41 +02:00
Hornwitser a194508f52 Filter plugin messages forwarded to links
Drop events forwarded to a slave or instance that does not have the
plugin the event is defined in.  Reject requests for the same reason.
2020-05-11 18:04:03 +02:00
Hornwitser ca2839594e Timeout quikly when disconnecting existing slave
If a slave that is already connected attempts connecting the existing
connection for that slave is disconnected.  The most common scenario for
this is that the existing connection is stale and the slave is
attempting to connect agan.  However if the connection is stale then the
disconnection will take 15 minutes to time out and the slave will appear
unresponsive during this time.  Fix this problem by setting the timeout
to 15 seconds before disconnecting the existing connection.
2020-05-09 16:36:33 +02:00
Hornwitser c35b6de70e Notify on connection events
Implement notification to plugins for disconnect request, connect, drop,
and close.  This allows plugins to properly handle slaves and the master
server shutting down without sending messages that will get lost.
2020-05-08 23:49:28 +02:00
Hornwitser b151dd8469 Fix documentation
Fix missing module declarations, broken lings, broken formating and
incorrect scopes in the documentation.  For modules split over several
files it appears the only viable option is to use the memberof directive
on each documented member and only declare the index file as a module.
2020-05-08 17:28:06 +02:00
Hornwitser 5c1214ad38 Rename connected and dropped events
Rename the connector events connected and dropped to connect and drop
respectively to be consistent with common Node.js names for events like
close.
2020-05-07 12:36:43 +02:00
Hornwitser 1629eaf225 Track plugins loaded by master and slaves
Send a mapping of the plugins the master server has loaded to slaves
vice versa.  Limit the plugins that may be loaded by instances to only
those that the master server has loaded.
2020-05-06 20:14:40 +02:00
Hornwitser 4f463a8180 Reduce number of bits in generated test cert
Add config entry for the number of bits to use in the auto generate TLS
certificate for the master server and reduce it to 512 for the
integration test.  This reduces the time needed to generate the
certificate in the integration test.
2020-04-09 20:02:29 +02:00
Hornwitser 2c855a5846 Add debug-dump-ws command
Add debug command to dump the WebSocket messages being sent and received
by the master server to aid in finding chatty events.
2020-04-09 19:50:01 +02:00
Hornwitser 61d92a6910 Rename master plugin onExit hook to onShutdown
Rename the onExit hook for master plugins to onShutdown to better
reflect when it is invoked.
2020-04-09 19:50:01 +02:00
Hornwitser 9507608e59 Rework disconnect logic for connectors
Rework the way disconnection at shutdown is done to properly close
the connections when the master server is shut down.
2020-04-09 19:50:01 +02:00
Hornwitser 47ea8f5477 Rename connection shutdown to prepare disconnect
Rename the shutdownConnection request to prepareDisconnect to better
reflect its actual purpose.
2020-04-09 16:17:38 +02:00
Hornwitser 75067e992d Add WebSocket usage statistics
Add Prometheus metrics for messages sent, connections made, connections
rejected, active connections, and active slaves.
2020-04-04 12:16:57 +02:00
Hornwitser 8ebf0fc754 Add master server Web Root config
Add web_root option to the master server and use it create URLs for
resources hosted by it.  This permits hosting the master server web
interface behind reverse proxy under a dedicated path.
2020-04-04 10:05:37 +02:00
Hornwitser 3343d53d75 Allow deleting unassigned instances
Handle delete instance requests for instances that are not assigned to a
slave.
2020-04-03 22:22:23 +02:00
Hornwitser 5c16809b5b JSDoc use autodetected module name
Change @module <module name> entries to just @module as JSDoc is able to
automatically determine the module name.
2020-04-03 20:51:28 +02:00
Hornwitser 4c26fcee3a Fix connection shutdown logic
Fix the WebSocket link not being properly teared down on the master
side causing the handshake to deadlock when the slave tried to connect
again.
2020-04-03 20:44:20 +02:00
Hornwitser d4e072f376 Rewrite reconnection logic
Implement proper session tracking and resume on reconnect for the
WebSocket server.
2020-04-02 21:58:44 +02:00
Hornwitser 677512512e Replace socket.io with plain WebSocket
Replace socket.io with plain WebSocket using the ws library as it
doesn't really provide any useful tools over plain WebSocket when it
comes to implementing reliable communication that works across
reconnects.  The widespread browser support for WebSocket also means
it has little value to add for the web interface over WebSocket.
2020-03-22 02:10:54 +01:00
Hornwitser d9fe408c42 Fix master web interface
Fix the breaking changes to the plugin struture and remove slave
information that is no longer present.
2020-03-13 22:46:17 +01:00
Hornwitser 090fbd6dff Expose master interfaces to plugin
Expose Express.js app, master config, instance and slave database, slave
connections and metrics to the master plugins.
2020-03-13 22:46:17 +01:00
Hornwitser cb1d744060 Handle errors during master plugin initialization
Catch errors thrown when constructing or initializing master pluigns and
wrap them in PluginError to ensure that they are properly reported as
plugin errors.
2020-03-13 22:46:17 +01:00
Hornwitser 5e5149dc93 Implement new configuration system
Replace current configuration mess with a trimmed down version of the
original vision of #200.  Configs are collection of config groups that
have fields that can be set.  These groups allow organizing the fields
as well as opertunity for plugins to add their own groups.  By default
plugins have an auto generated group with the enabled field.
2020-03-09 20:48:13 +01:00
Hornwitser 26eefb0476 Rewrite statistics export
Add automatic export of metrics from slaves to the master server.  And
rewrite the statistics export plugin to use this new system.
2019-12-20 21:26:29 +01:00
Hornwitser 1f6574a89c Improve master shutdown
Close active connections and http servers during master shutdown and let
node exit on its own.  If this fails with an error set output streams to
blocking and report it then shutdown with process.exit.

Also add double SIGINT handling to immediately shutdown.
2019-12-20 02:37:27 +01:00
Hornwitser bcceae687f Reimplement plugin system
Implement a new plugin system based subclassing pre-defined base clases
and communicating with pre-defined link messages.
2019-12-14 17:48:56 +01:00
Hornwitser 513651393b Implement forwarding to instances for events
Implement forward to instance for Event messages.
2019-12-14 17:31:02 +01:00
Hornwitser e5115df28b Implement broadcast for events
Add broadcastTo property to events that broadcasts them to all entities
of the specified target that's downstream to the connection it was sent
to.
2019-12-13 22:37:58 +01:00
Hornwitser 21d9fdff7b Remove old plugin system
Remove the old client/instance fused plugin system along with
instanceManagement in client.js and getPlugins in master.js.
2019-12-13 22:37:49 +01:00
Hornwitser 5d8fa4e4ce Refactor instance communication
Change Instance class to be a subclass of Link and refactor the
communication to and establisment of instances to use a virtual
connection between the slave and the instance.  This is simplifies the
implementation of plugin communication for instances.

Replace InstanceRequest with a forwardTo option to Request which does
the task of forwarding in a much cleaner way.
2019-12-13 21:57:05 +01:00
Hornwitser 62afcaadab Use shared StartupError class
Remove separate StartupError class definition in master.js in favor of
the shared StartupError class in lib/error.js.
2019-12-13 21:39:54 +01:00
Hornwitser 3be264c9a9 Add integration testing of commands
Add testing of the clusterctl commands against the master and slave
code.
2019-11-30 00:08:10 +01:00
Hornwitser 3bccbac496 Refactor to a connector based concept in lib/link
Remove connection and client subclasses of Link and put the connection
logic into a "connector" class.  The connector deals with sending
and receiving messages, handling the connection and reconnection of lost
connections and giving sequence numbers to messages.

This move the logic of the actual connection away from the link
sub-classes, which in turn means the connection should be a lot easier
to mock in tests.
2019-11-30 00:08:10 +01:00
Hornwitser 039b41d3e6 Move events and requests into messages
Merge the events and requests categories in lib/link into messages and
use the class name of the message to determine which handler to call.

Also swaps the order of the arguments for forwardInstanceRequest to be
consistent with the rest of the lib/link code and fixes up the docs for
requests and events.
2019-11-30 00:08:10 +01:00
Hornwitser 1dea5fddbe Forward Factorio logs to clusterctl
Add instance output event that is forwarded by the master server to
control connections that are subscribing to them.  Add code subscribing
to the relevant instance output when dealing with instances in
clusterctl.
2019-11-23 15:18:20 +01:00
Hornwitser 7a2abaa5db Daemonize slave
Rework client.js to support running multiple instances at once and be
managed remotely and implement the neccessary communication
infrastructure for this.

Due to circular dependencies on the implementation of this the master
branch is significantly broken after this.
2019-11-23 15:16:47 +01:00
Hornwitser 27fc68da0f Remove remoteCommands plugin and interface
Remove the half baked remote rcon command interface in preparation for a
better version based on a new socket.io communication layer.
2019-11-20 22:37:25 +01:00
Hornwitser d38e9a53c4 Remove broken combinator signaling
Remove badly implemented and broken remote combinator signaling.
2019-11-20 22:37:25 +01:00
Hornwitser f3171c2feb Remove support for uploading mods to master
Remove the uploadMod and checkMod APIs as these have become more or less
obsolete after Factorio implementd syncing mods to the server in game.
2019-11-20 22:37:23 +01:00
Hornwitser 7359f9075e Port increase limit of Json payloads to master server
The default limit of 100kb to JSON payloads of bodyParser can be
exceeded when running modded games with researchSync.  Increase the
limit to 10mb as a bandaid to fix this issue.

This is a port of 25eb5e5.
2019-11-15 15:38:56 +01:00
Hornwitser e4aae88614 Move clusterioMod specific code into the plugin
Move item database, tests, endpoint definitions for /api/place,
/api/remove, /api/inventory, /api/inventoryAsObject, and the web
interface view for the storage page into the clusterioMod plugin.
2019-10-28 19:46:22 +01:00
Hornwitser ff3fc3b94b Remove .js suffix in require paths
Remove the .js suffix in require paths to make requires consistent
across the project.
2019-10-28 19:45:34 +01:00
Hornwitser 6bde8a2219 Remove .js suffix from node invocations
Remove the .js suffix from help texts of node invocations as well as
actual invocations as this is not necessary.  Removing it opens up the
possibility of restructuring the code into a directory with the same
name.
2019-10-28 19:45:32 +01:00
Hornwitser 93480c8b29 Refactor getFactorioLocale
Move getFactorioLocale to the new lib/factorio namespace and refactor it
to use async functions instead of callbacks.  Split tests into
integration with factorio and testing of the function itself.

Also removes the cacheing from the function itself as this breaks the
isolated testing of it.
2019-10-14 18:56:46 +02:00
Hornwitser a62e8d17a3 Move ejs templates to views/
Move the ejs templates to the default Express.js views/ folder and
change their extension to .ejs.  Reconfigure Express.js to look for
templates in the views folders as well as the plugins folder.  This
enables plugins to render templates using paths of following format
"pluginName/path/to/page-or-view".
2019-09-18 16:54:11 +02:00
Hornwitser dd865b42b5 Fix incorrect handling of instance ids
After 651b09b the instance ID would be treated differently depending on
wheteher it was a string or a number in the master server.  This caused
researchSync to break as it would send the ID as a number while the
master server would have it stored as a string.

Fix by coercing instanceID to a string in the master HTTP API.
2019-09-16 16:04:08 +02:00
Hornwitser 500c9584eb Remove commented out itemStats code
Remove old commented out code dealing with item statistics that is no
longer relevant.
2019-09-14 00:56:22 +02:00
Hornwitser 8aa66e9e68 Port authentication of socket.io server
Authenticate connections made to the socket.io server via a token query
parameter that's validated in a similar manner to the x-access-token for
HTTP requests.

This is a port of 0c45e57.
2019-09-11 16:59:11 +02:00