Add a per field access parameter to the config class which specifies
which of master, slave and control can access the given field. A
location is added to the config class to specify which party owns it and
fields which do not have an access parameter are not present in the
config and will throw if attempted to be read from or writen to.
The serialize, load, update, get, set and setProp are ammended with a
location parameter which specifiy where the data is from or destined to
and the access checks are performed to ensure remote access to the
config is still limited by the access parameter of field.
With this system in place the master config can be shared with control
and slave connections without compromising secrets that should stay
private. This is primarily the master authentication secret, but other
plugins may define secrets it does not want to have shared with slave
and/or control connections.
Raise "resume" instead of "connect" when a connection is re-established
after a connection drop. This prevents logic meant to resynchronize
state after a full reconnection from being invoked whenever the
connection was temporarily dropped.
Remove the pluginInfo parameter from lib/plugin attachPluginMessages as
the plugin info object is available in the `.info` property of the
plugin. While a plugin instance doesn't exist for the slave side and
was the reason for these to be separate arguments, I expect slave
plugins to be implemented eventually.
Move the target folder for the built web interface to dist/web/ instead
of static/. This better follows the norm for npm packages to include
built files in the dist/ folder and also allows putting other built
files into that folder in the future.
Exit immediately if shutdown has already been called. This is due to
the possibility of it being called multiple times from combinations of
signals and fatal errors.
On windows the terminal output is asynchronous by default which leads to
log lines being lost if text is written to stdout before calling
process.exit. Fix by using setBlocking before doing the log call.
Add settings of interest to server administrators that appear in
server-settings.json to the factorio.settings config. This makes it
easier to see which settings can be set there.
The automatic creation of a self-signed certificate while nice presents
several problems. To make connections not reject the self-signed
certificate the validation of TLS certificates are wholesale disabled,
decreasing the security even when properly set up and trusted
certificates are used.
To make the created self-signed certificate pass validation it is
necessary for it to list the host names it's valid for and to copy it to
each machine that intends to connect to the server. These two steps
complicate automating the process in a reliable way and will likely lead
to confused users seeking to disable TLS certificate validation as the
easy way out to problems.
For these reasons, remove the automating creation of a broke self-signed
certificate, fix validation of certificates and document how to set up
TLS properly. This makes HTTPS recommended when accessing over the
internet, but still something which must be explicitly configured.
When closing down the HTTP and HTTPS servers existing connections are
not closed by Node.js, and may live on forever if they are actively in
use. A common case where this happens is if Prometheus is set up with a
short interval for collections.
To close down the active connections they need to be tracked and closed
when the server is closed, add the HttpCloser class which encapsulates
the tracking and closure process behind a simple interface and use this
to close down the HTTP and HTTPS server on the master.
When reducing the collection interval in Prometheus it may be necessary
to to reduce the metrics gathering timeout to ensure it's less than the
interval. Add config options to adjust the timeouts used for metrics
gathering so that these can be set accordingly.
Add exported metrics that map instance id and slave id to their
respective names. This allows getting the instance name of a metric by
doing a left join using query of the form 'metric * on(instance_id)
group_left(instance_name) clusterio_master_instance_mapping'.
Add log query and log follow commands which allows inspection of the
combined cluster log. Past entries can be queried via the new queryLog
request. The log is stored into a single flat file for now, though the
performance of this may good enough.
Add winston based logger for master, slave, ctl, lib, and web_ui code.
Logging done from slaves and instances are sent to the master server and
tagged with the instance and/or slave it originated from. This allows
for example error messages to propagate back to the source that iniated
the action. The master server also stores the log to allow it to be
queried at a later point. For slaves a copy of the slave's log is also
stored locally.
Move common functionality between labeled collectors out of
ValueCollector and into a new LabeledCollector class, and use this as
the basis for implementing a Histogram collector.
Under the current system there's no distinction made on the slave side
between instance config files that are present in the filesystem and
instances actually assigned to the slave. Which could lead to problems
with determing if an instance is actually assigned to the slave.
Resolve this issue by separating discovered instances with assigned
instances. Upon startup discovered instances are put into the
discoveredInstanceInfos mapping on the Slave object and all of these are
sent to the master server. The master server in turn sends assignment
requests for all the instances which are assigned to the slave. These
are put into the instanceInfos mapping of the Slave object which can
then be relied up to include only assigned instances.
When the slave metrics request takes more than 10 seconds the response
to the /metrics endpoint ends up being discarded by Prometheus which
defaults to a timeout of 10 seconds. Fix by ignoring slaves that take
longer than 8 seconds to respond to the metrics request.
Add command to generate a user token for connecting to the master server
web interface. This servers as an alternative to generating a control
config for use with clusterioctl.
Move the definition and implementation of the config command to
lib/shared_comhmands.js, this removes the need for lib config to depend
on the fs module, which is not accessible in web build.
Move the code for loading plugins infos from Node.js packages into a
separate module in order to break it out from lib/plugin.js which is
also used on the web interface side. This resolves a warning of using
an expression with require on the webpack build.
Fix the issue with paths resolving incorrectly for the webpack config
when running the master server with the --dev option from the project
root folder through either npx clusteriomaster or node packages/master
Mave the ignore entry for packages/master/static out of
packages/master/.gitignore to avoid npm using it and causing the web
build to be excluded from the package.
Use lib* prefix for the modules impored from the lib package to help
distinguish them more from ordinary variables. Fix up the naming
convention for files to be consistently lowercase_underscore for
JavaScript files.
Add a separate plugin-list.json file for storing which plugins are
available and the path to pass to require in order to load it. This
allows plugins to be published and installed as npm packages. And also
replaces the old method of installing plugins by putting them in the
plugins folder.
Move the code generating the master authentication secret from the
inital value function of the master.auth_secret config entry to the
master startup. Having the authentication secret generated in the
config library poses issues with using the config library in browser.
Send the status of the instance in the repsonse to the listInstances
request and use this status in the integration tests to check if the
instance was started/stopped.
Add the statuses unknown and unassigned for when the status has not yet
been set by a slave and replace the initialized status with starting,
creating_save and exporting_data.
Hornwitser