Commit Graph
202 Commits
Author SHA1 Message Date
Hornwitser d9fe408c42 Fix master web interface
Fix the breaking changes to the plugin struture and remove slave
information that is no longer present.
2020-03-13 22:46:17 +01:00
Hornwitser 090fbd6dff Expose master interfaces to plugin
Expose Express.js app, master config, instance and slave database, slave
connections and metrics to the master plugins.
2020-03-13 22:46:17 +01:00
Hornwitser cb1d744060 Handle errors during master plugin initialization
Catch errors thrown when constructing or initializing master pluigns and
wrap them in PluginError to ensure that they are properly reported as
plugin errors.
2020-03-13 22:46:17 +01:00
Hornwitser 5e5149dc93 Implement new configuration system
Replace current configuration mess with a trimmed down version of the
original vision of #200.  Configs are collection of config groups that
have fields that can be set.  These groups allow organizing the fields
as well as opertunity for plugins to add their own groups.  By default
plugins have an auto generated group with the enabled field.
2020-03-09 20:48:13 +01:00
Hornwitser 26eefb0476 Rewrite statistics export
Add automatic export of metrics from slaves to the master server.  And
rewrite the statistics export plugin to use this new system.
2019-12-20 21:26:29 +01:00
Hornwitser 1f6574a89c Improve master shutdown
Close active connections and http servers during master shutdown and let
node exit on its own.  If this fails with an error set output streams to
blocking and report it then shutdown with process.exit.

Also add double SIGINT handling to immediately shutdown.
2019-12-20 02:37:27 +01:00
Hornwitser bcceae687f Reimplement plugin system
Implement a new plugin system based subclassing pre-defined base clases
and communicating with pre-defined link messages.
2019-12-14 17:48:56 +01:00
Hornwitser 513651393b Implement forwarding to instances for events
Implement forward to instance for Event messages.
2019-12-14 17:31:02 +01:00
Hornwitser e5115df28b Implement broadcast for events
Add broadcastTo property to events that broadcasts them to all entities
of the specified target that's downstream to the connection it was sent
to.
2019-12-13 22:37:58 +01:00
Hornwitser 21d9fdff7b Remove old plugin system
Remove the old client/instance fused plugin system along with
instanceManagement in client.js and getPlugins in master.js.
2019-12-13 22:37:49 +01:00
Hornwitser 5d8fa4e4ce Refactor instance communication
Change Instance class to be a subclass of Link and refactor the
communication to and establisment of instances to use a virtual
connection between the slave and the instance.  This is simplifies the
implementation of plugin communication for instances.

Replace InstanceRequest with a forwardTo option to Request which does
the task of forwarding in a much cleaner way.
2019-12-13 21:57:05 +01:00
Hornwitser 62afcaadab Use shared StartupError class
Remove separate StartupError class definition in master.js in favor of
the shared StartupError class in lib/error.js.
2019-12-13 21:39:54 +01:00
Hornwitser 3be264c9a9 Add integration testing of commands
Add testing of the clusterctl commands against the master and slave
code.
2019-11-30 00:08:10 +01:00
Hornwitser 3bccbac496 Refactor to a connector based concept in lib/link
Remove connection and client subclasses of Link and put the connection
logic into a "connector" class.  The connector deals with sending
and receiving messages, handling the connection and reconnection of lost
connections and giving sequence numbers to messages.

This move the logic of the actual connection away from the link
sub-classes, which in turn means the connection should be a lot easier
to mock in tests.
2019-11-30 00:08:10 +01:00
Hornwitser 039b41d3e6 Move events and requests into messages
Merge the events and requests categories in lib/link into messages and
use the class name of the message to determine which handler to call.

Also swaps the order of the arguments for forwardInstanceRequest to be
consistent with the rest of the lib/link code and fixes up the docs for
requests and events.
2019-11-30 00:08:10 +01:00
Hornwitser 1dea5fddbe Forward Factorio logs to clusterctl
Add instance output event that is forwarded by the master server to
control connections that are subscribing to them.  Add code subscribing
to the relevant instance output when dealing with instances in
clusterctl.
2019-11-23 15:18:20 +01:00
Hornwitser 7a2abaa5db Daemonize slave
Rework client.js to support running multiple instances at once and be
managed remotely and implement the neccessary communication
infrastructure for this.

Due to circular dependencies on the implementation of this the master
branch is significantly broken after this.
2019-11-23 15:16:47 +01:00
Hornwitser 27fc68da0f Remove remoteCommands plugin and interface
Remove the half baked remote rcon command interface in preparation for a
better version based on a new socket.io communication layer.
2019-11-20 22:37:25 +01:00
Hornwitser d38e9a53c4 Remove broken combinator signaling
Remove badly implemented and broken remote combinator signaling.
2019-11-20 22:37:25 +01:00
Hornwitser f3171c2feb Remove support for uploading mods to master
Remove the uploadMod and checkMod APIs as these have become more or less
obsolete after Factorio implementd syncing mods to the server in game.
2019-11-20 22:37:23 +01:00
Hornwitser 7359f9075e Port increase limit of Json payloads to master server
The default limit of 100kb to JSON payloads of bodyParser can be
exceeded when running modded games with researchSync.  Increase the
limit to 10mb as a bandaid to fix this issue.

This is a port of 25eb5e5.
2019-11-15 15:38:56 +01:00
Hornwitser e4aae88614 Move clusterioMod specific code into the plugin
Move item database, tests, endpoint definitions for /api/place,
/api/remove, /api/inventory, /api/inventoryAsObject, and the web
interface view for the storage page into the clusterioMod plugin.
2019-10-28 19:46:22 +01:00
Hornwitser ff3fc3b94b Remove .js suffix in require paths
Remove the .js suffix in require paths to make requires consistent
across the project.
2019-10-28 19:45:34 +01:00
Hornwitser 6bde8a2219 Remove .js suffix from node invocations
Remove the .js suffix from help texts of node invocations as well as
actual invocations as this is not necessary.  Removing it opens up the
possibility of restructuring the code into a directory with the same
name.
2019-10-28 19:45:32 +01:00
Hornwitser 93480c8b29 Refactor getFactorioLocale
Move getFactorioLocale to the new lib/factorio namespace and refactor it
to use async functions instead of callbacks.  Split tests into
integration with factorio and testing of the function itself.

Also removes the cacheing from the function itself as this breaks the
isolated testing of it.
2019-10-14 18:56:46 +02:00
Hornwitser a62e8d17a3 Move ejs templates to views/
Move the ejs templates to the default Express.js views/ folder and
change their extension to .ejs.  Reconfigure Express.js to look for
templates in the views folders as well as the plugins folder.  This
enables plugins to render templates using paths of following format
"pluginName/path/to/page-or-view".
2019-09-18 16:54:11 +02:00
Hornwitser dd865b42b5 Fix incorrect handling of instance ids
After 651b09b the instance ID would be treated differently depending on
wheteher it was a string or a number in the master server.  This caused
researchSync to break as it would send the ID as a number while the
master server would have it stored as a string.

Fix by coercing instanceID to a string in the master HTTP API.
2019-09-16 16:04:08 +02:00
Hornwitser 500c9584eb Remove commented out itemStats code
Remove old commented out code dealing with item statistics that is no
longer relevant.
2019-09-14 00:56:22 +02:00
Hornwitser 8aa66e9e68 Port authentication of socket.io server
Authenticate connections made to the socket.io server via a token query
parameter that's validated in a similar manner to the x-access-token for
HTTP requests.

This is a port of 0c45e57.
2019-09-11 16:59:11 +02:00
AlienXAXS ea5c0d5aa3 Fixes editSlaveMeta and db AddItem errors (#228)
Fixed editSlaveMeta to pass a number to db.addItem method
2019-09-10 17:52:09 +02:00
TheAIBot e14f7c22fd Merge pull request #222 from Hornwitser/config-part-1
Config Rewrite Part 1
2019-09-05 11:33:43 +02:00
Hornwitser a31778c7ff Remove unused config parameter to pluginManager
Remove the unused library initialization parameter config to
pluginManager and reorganize pluginManager to no longer be defined by a
function.
2019-09-04 20:43:13 +02:00
Hornwitser 35546b4b18 Move loading of config in master to startServer
Move the loading of the config file in the master server to the
startServer function.  Remove config.json.test as it's no longer needed.
2019-09-04 20:43:13 +02:00
Hornwitser 651b09b613 Refactor database and item retreival
Change the ugly global slaves object to be a member of the db object and
implement the items database as a separate integrety verifying class in
lib/database.
2019-09-04 20:43:13 +02:00
Hornwitser 23ce86aef6 Fix master socket.io broken by a3fc1dd
Add missing attach calls to the master socket.io object that was
accidentally removed by the refactoring in a3fc1dd.  Unfortunately I've
been unable to determine if socket.io actually supports listening on
multiple servers at the same time, this needs further investigation.
2019-09-04 17:39:49 +02:00
Hornwitser 5d6f6de379 Export app as a separate entry from master.js
Change the module exports from master.js to allow additional objects to
be passed to allow better unit testing.
2019-09-04 17:36:06 +02:00
Hornwitser af2471b427 Remove config dependency in lib/authenticate
Restructure lib/authenticate to set the authentication secret at a later
point in time via setAuthSecret.  This breaks playerManager.
2019-09-04 17:36:06 +02:00
Hornwitser 703423ae54 Add development branch warning at startup
Add a big banner based warning telling this is the development branch
for clusterio to the startup of client.js and master.js.
2019-09-01 23:34:52 +02:00
Hornwitser 84f48e36f4 Improve error reporting at startup
Add StartupError that differentiates unhandled errors from errors with
known causes and use it to only print the "please report" banner when an
unhandled error occurs.

Also makes the error reporting more visible in the console.
2019-09-01 23:24:32 +02:00
Hornwitser b73b202513 Remove remanent remoteMap code
Remove remoteMap specific code left behind after the removal of the
remoteMap plugin in 6415784.  The plugin interface should be versatile
enough to allow all of this code to reside inside the plugin itself.
2019-08-29 11:38:57 +02:00
Hornwitser ab2a377ca5 Remove Prometheus pushgateway code
Remove code pushing metrics to a hardcoded Prometheus pushgateway that
is no longer in operation.  There's no point supporting the use of a
pushgateway as there's already a full client implemented and since the
master server is not ephemeral or batch job it shouldn't be using a
pushgateway in the first place.
2019-08-24 11:25:40 +02:00
Hornwitser ea2f291025 Remove reference to old authentication scheme
Remove long comment and commented out code about an old sanity check in
/api/place.
2019-08-24 11:25:40 +02:00
Hornwitser 3e80e40223 Move database and masterMods folder creation
Move the code creating the database and masterMods folders into the
startServer function in master.js.
2019-08-24 11:25:39 +02:00
Hornwitser ed54db0db9 Handle errors during master startup
Print a message urging unhandled exceptions thrown by startup to be
reported to the GitHub issue tracker and initiate shutdown.
2019-08-24 11:09:01 +02:00
Hornwitser 0398bd10f8 Remove debug code
Remove debug switch and console.log guarded by it to log every post
request to /api/getId as it's not needed.
2019-08-24 11:02:50 +02:00
Hornwitser 44b7d0aa19 Refactor SSL certificate creation
Rewrite the generateSSLcert lib to be async, use the sslCert and
sslPrivKey config entries and invoke it only if needed during master
server startup instead of in the post install script.  Also add note to
the changelog covering both this and the fix in c645fac.
2019-08-23 17:24:39 +02:00
TheAIBot c645fac3de Replace hardcoded paths with paths from the config 2019-08-23 15:08:20 +02:00
Hornwitser a3fc1dda8c Refactor master.js
Move server initialization code to startServer and call it only when
master.js is being runned directly.  Also refactor the server listen
step to work properly when https is configured and only listen after
plugins have completed loading.

Remove the async function wrapper around the master.js code.  This is
just dumb and makes the code untestable.
2019-08-21 12:43:09 +02:00
Hornwitser 442472e412 Actually fix masterPlugins visibility
The masterPlugins array should not be a global.  The ejs templates
already have it provided through locals.masterPlugins and plugins get it
through onLoadFinish.  Make masterPlugins local to master.js again and
fix the ejs templates to use locals.masterPlugins.
2019-08-21 12:43:09 +02:00
Hornwitser ca0281ae58 Remove pointless micro-optimization
This saved 30ms of the startup on my machine.  Even if it saved a second
it's still pretty pointless as we're not depending on the master server
startup being super fast.
2019-08-07 10:18:14 +02:00