The installer only writes the controller URL and host name. Running the
host requests to join the cluster if it has not been approved yet, or if
the controller no longer accepts its token, and waits until an admin
approves it. Once approved it saves its id and token and starts as
normal. A rejected host exits with code 8.
clusteriohost run --user-token approves the host right away with the
token of a user allowed to approve hosts. The token is not saved.
This moves the pairing client into the host package, where it uses the
same certificate setup as the host's connection to the controller, and
removes --no-pair and the rerun handling from the installer.
The test's first connector had no Link, so the controller's disconnect
prepare went unanswered and it waited the full 10 s timeout.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Hosts join the cluster by running the installer, which shows a code and
waits until the request is approved under Hosts > Add host in the web
interface or with clusterioctl host approve. An admin can pass their
user token to the installer to approve the host right away. The token
is only used for that and is not saved.
The controller only accepts hosts that have a record, and only
approving a host creates one. Revoking a host deletes its record, so
its token can no longer be used to reconnect. Hosts that have connected
before keep working.
Removes host generate-token, revoke-token and create-config from
clusterioctl, bootstrap generate-host-token, the host token messages and
the core.host.generate_token, revoke_token and create_config
permissions, which are also stripped from stored roles. Adds
clusterioctl host pending, approve, reject and revoke, and the
core.host.approve and core.host.revoke permissions.
Standalone installs add the host to the controller database directly,
since the controller is not running during the install.
Add controller.max_send_buffer_size in MiB, default 64 and 0 for no
limit. Close with 1013 instead of 1008 on overflow: clients treat 1008
as fatal, so a host would shut down and the web UI would log out. With
1013 the client resumes, the controller answers with invalidate since
the session has ended, and the client registers a new session.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
These are used outside of the integration tests, but including the
integration index file causes the very slow setup of the contorller
and host. Avoid the integration setup costs when running commands
such as `pnpm mocha test/lib` by moving these utility functions to
the common test functions.
Drop the special case that deferred terminating the connection issuing
the delete, so deleting yourself behaves the same as revoke-token and
every session of the deleted user is closed before the handler returns.
Set tokenValidAfter to the current time in createUser instead of 0.
Deleting a user and recreating it under the same name left tokens issued
before the deletion valid, as the check is iat < tokenValidAfter. Doing
it in createUser covers every path that makes a user. A token signed in
the same second as creation is still accepted, so create-admin and the
other production paths sign after the user exists and are unaffected.
The integration harness signed the control token at import time, before
bootstrap create-admin created the user. Sign it in the root before hook
instead and reuse it in link.js.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Migrate the usage of CommonJS specific code and imports to ESM. Roughly
summarised this:
- Changes all require() calls to import statement or expressions.
- Changes usage of module.exports to export statements.
- Replaces require.resolve with import.meta.resolve.
- Replaces conditional imports with conditional subpath imports.
- Replaces module === require.main with import.meta.main.
- Adds the full filename with extention to all relative imports.
- Changes all usage of __dirname and __filename to their import.meta
equivalents.
- Sets "type": "module" in package.json files.
- Removes usage of "use strict".
- Adds .ts an an alias of .js for the webpack build.
Due to differences in how require and import resolves files the plugin
entrypoints will have to include the filename and extension after this
migration in order to work.
These packages are obsoleted by the promises API to the built-in Node fs
module. The fs-extra package is also causing problems with broken APIs
that don't behave the same as Node's fs module, and throwing errors
without code properties on them.
The migration from phin to fetch in #755 was hampred by the lack of an
option to set the CA when using fetch. This meant places that used phin
to talk to the controller with the tlsCa option could not be migrated to
use fetch.
Remove the tlsCa option to unblock this dependency. The replacement for
this option is to set the NODE_EXTRA_CA_CERTS environment variable to a
path to the certificate file instead, which works with fetch.
Reverse the direction of the API for sending messages from
messageDefinition.send(link, { content });
to
link.send(new MessageClass(content));
and convert all message definitions to use the JSON serialisable class
pattern and explicit registration of message handlers, along with a
end-to-end based routing of messages instead of link to link.
The experience I gained from the existing way messages were defined and
sent over links was that it was difficult to understand for people
unfamiliar with the system. The automatic registration of message
handlers ment that code was being called as if by magic from nowhere
with no easy way to discover it. The manual forwarding and rules for
what can be sent where and how obscured its working.
The use of pure JSON as both input and output from message handlers made
it tedious to work with. Data had to be manually converted between the
program representation and the over the wire format, which made handler
code unplesant to write. Particularly because the over the wire format
used snake_case while the program code uses camelCase for variables.
The idea behind the decision to do it this way was to make it more clear
which data came from the wire and which from the program, and while this
worked it didn't make message handlers any nicer to write.
The direction of the API and magic registration of message handlers also
meant that a future migration to TypeScript would be troublesome. While
it may be possible to implement, the degree of TypeScript magic required
would go starkly against the principle of keep it simple stupid.
By refactoring to using the JSON serialisable class pattern most of
these issues go away. There is no longer a need to distinguish the on
wire format of messages with the program representation because the only
place that deals with the wire format of messages are the toJSON and
fromJSON methods on the classes themselves. Message handlers will work
with proper instances of classes instead of raw data, something that
makes them easier to write, and trivial to type with TypeScript.
End-to-end routing also simplifies sending messages that need to do
mulitple hops to reach the destination. And removes the need for having
all messages be specified in plugin's info module, as they can be routed
without having to be parsed completely.
The explicit registration of message handlers is something that I
thought would be tedious boilerplate code that could be automated away.
But experience now shows it to be the opposite. Allthough dull
repetition, the purpose it serves is to link together the code in a way
that humans can easily find. With explicit registration searching the
name of the message handler in the editor will show the registration that
ties it together with the type of message it handles and the kind of
registration. With implicit registration searching the name of the
message handler turns up only the handler with no clue as to what calls
it.
The jsonwebtoken accept strings as input for secrets which is
nonsensical from a cryptographic standpoint as the algorithms all
operate on byte sequences. Worse still, strings are encoded into bytes
using UTF-8, which disallows certain sequences of bytes from appearing
and potentially weakening the cryptography.
For legacy reasons the master secret was encoded into base64 _and passed
as a string to jsonwebtoken_. This further weakened the crypto as the
bytes that can appear in base 64 is only 1/4 of the 256 possible values
for bytes. This could significantly weaken the cryptographic strength
of the tokens.
I have known of this issue for quite some time but has put off actually
fixing it for too long now. Since the master secret is already
documented as being in base 64, decode it as base 64 and pass it as a
byte Buffer to jsonwebtoken. This will ensure every byte value is
possible in the byte sequence passed to the jsonwebtoken crypto.
The impact of this change is unfortunately that all tokens will become
invalid and will have to be reissued.
If the connect close method is called while the connection is waiting
for a reconnection attempt then the attempt is cancelled but the close
method then erroneously waited for a close event that would never
happen. Fix by resetting the connector and rasing the close event in
the close method if it's in the waiting for reconnect state.
Raise "resume" instead of "connect" when a connection is re-established
after a connection drop. This prevents logic meant to resynchronize
state after a full reconnection from being invoked whenever the
connection was temporarily dropped.
The automatic creation of a self-signed certificate while nice presents
several problems. To make connections not reject the self-signed
certificate the validation of TLS certificates are wholesale disabled,
decreasing the security even when properly set up and trusted
certificates are used.
To make the created self-signed certificate pass validation it is
necessary for it to list the host names it's valid for and to copy it to
each machine that intends to connect to the server. These two steps
complicate automating the process in a reliable way and will likely lead
to confused users seeking to disable TLS certificate validation as the
easy way out to problems.
For these reasons, remove the automating creation of a broke self-signed
certificate, fix validation of certificates and document how to set up
TLS properly. This makes HTTPS recommended when accessing over the
internet, but still something which must be explicitly configured.
Add users, roles and permissions and rework the authentication to use
separate user and slave tokens for control and slave connections
respectively.
Add commands for listing permissions, roles and users, and creating,
editing and deleting roles and users.
Danielv123