Datastore core

Core Module - Datastore - A module used to store data in the global table with the option to have it sync to an external source.

Usage

-- Types of Datastore
-- This datastore will not save data externally and can be used to watch for updates on values within it
-- A common use might be to store data for a gui and only update the gui when a value changes
local LocalDatastore = Datastore.connect('LocalDatastore')

-- This datastore will allow you to use the save and request method, this allows you to have persistent data
-- Should be used over auto save as it creates less save requests, but this means you need to tell the data to be saved
-- We use this type for player data as we know the data only needs to be saved when the player leaves
local PersistentDatastore = Datastore.connect('PersistentDatastore', true) -- save_to_disk

-- This datastore is the same as above but the save method will be called automatically when ever you change a value
-- An auto save datastore should be used if the data does not change often, this can be global settings and things of that sort
-- If it is at all possible to setup events to unload and/or save the data then this is preferable
local AutosaveDatastore = Datastore.connect('AutosaveDatastore', true, true) -- save_to_disk, auto_save

-- Finally you can have a datastore that propagates its changes to all other connected servers, this means request does not need to be used
-- This should be used when you might have data conflicts while saving, this is done by pushing the saved value to all active servers
-- The request method has little use after server start as any external changes to the value will be pushed automatically
-- Auto save can also be used with this type and you should follow the same guidelines above for when this should be avoided
local PropagateDatastore = Datastore.connect('PropagateDatastore', true, false, true) -- save_to_disk, propagate_changes

-- Using Datastores Locally
-- Once you have your datastore connection setup, any further requests with connect will return the same datastore
-- This is important to know because the settings passed as parameters you have an effect when it is first created

-- One useful thing that you might want to set up before runtime is a serializer, this will convert non string keys into strings
-- This serializer will allow use to pass a player object and still have it serialized to the players name
local ExampleData = Datastore.connect('ExampleData')
ExampleData:set_serializer(function(rawKey)
    return rawKey.name
end)

-- If we want to get data from the datastore we can use get or get_all
local value = ExampleData:get(player, defaultValue)
local values = ExampleData:get_all()

-- If we want to set data then we can use set, increment, update, or update_all
ExampleData:set(player, 10)
ExampleData:increment(player)
ExampleData:update(player, function(player_name, value)
    return value * 2
end)
ExampleData:update_all(function(player_name, value)
    return value * 2
end)

-- If we want to remove data then we use remove
ExampleData:remove(player)

-- We can also listen for updates to a value done by any of the above methods with on_update
ExampleData:on_update(function(player_name, value)
    game.print(player_name..' has had their example data updated to '..tostring(value))
end)

-- Using Datastore Externally
-- If save_to_disk is used then this opens up the option for persistent data which you can request, save, and remove
-- All of the local methods are still usable put now there is the option for extra events
-- In order for this to work there must be an external script to read datastore.pipe and inject with Datastore.ingest

-- To request data you would use request and the on_load event, this event can be used to modify data before it is used
ExampleData:request(player)
ExampleData:on_load(function(player_name, value)
    game.print('Loaded example data for '..player_name)
    -- A value can be returned here to overwrite the received value
end)

-- To save data you would use save and the on_save event, this event can be used to modify data before it is saved
ExampleData:save(player)
ExampleData:on_save(function(player_name, value)
    game.print('Saved example data for '..player_name)
    -- A value can be returned here to overwrite the value which is saved
end)

-- To remove data locally but not externally, like if a player logs off, you would use unload and on_unload
ExampleData:unload(player)
ExampleData:on_unload(function(player_name, value)
    game.print('Unloaded example data for '..player_name)
    -- Any return is ignored, this is event is for cleaning up other data
end)

-- Using Datastore Messaging
-- The message action can be used regardless of save_to_disk being set as no data is saved, but an external script is still required
-- These messages can be used to send data to other servers which doesnt need to be saved such as shouts or commands
-- Using messages is quite simple only using message and on_message
ExampleData:message(key, message)
ExampleData:on_message(function(key, message)
    game.print('Received message '..message)
end)

-- Combined Datastores
-- A combined datastore is a datastore which stores its data inside of another datastore
-- This means that the data is stored more efficiently in the external database and less requests need to be made
-- To understand how combined datastores work think of each key in the parent as a table where the sub datastore is a key in that table
-- Player data is the most used version of the combined datastore, below is how the player data module is setup
local PlayerData = Datastore.connect('PlayerData', true) -- saveToDisk
PlayerData:set_serializer(Datastore.name_serializer) -- use player name as key
PlayerData:combine('Statistics')
PlayerData:combine('Settings')
PlayerData:combine('Required')

-- You can then further combine datastores to any depth, below we add some possible settings and statistics that we might use
-- Although we dont in this example, each of these functions returns the datastore object which you should use as a local value
PlayerData.Settings:combine('Color')
PlayerData.Settings:combine('Quickbar')
PlayerData.Settings:combine('JoinMessage')
PlayerData.Statistics:combine('Playtime')
PlayerData.Statistics:combine('JoinCount')

-- Because sub datastore work just like a normal datastore you dont need any special code, using get and set will still return as if it wasnt a sub datastore
-- Things like the serializer and the datastore settings are always the same as the parent so you dont need to worry about setting up the serializer each time
-- And because save, request, and unload methods all point to the root datastore you are able to request and save your data as normal

-- If you used get_all on PlayerData this is what you would get:
{
    Cooldude2606 = {
        Settings = {
            Color = 'ColorValue',
            Quickbar = 'QuickbarValue',
            JoinMessage = 'JoinMessageValue'
        },
        Statistics = {
            Playtime = 'PlaytimeValue',
            JoinCount = 'JoinCountValue'
        }
    }
}

-- If you used get_all on PlayerData.Settings this is what you would get:
{
    Cooldude2606 = {
        Color = 'ColorValue',
        Quickbar = 'QuickbarValue',
        JoinMessage = 'JoinMessageValue'
    }
}

-- If you used get_all on PlayerData.Settings.Color this is what you would get:
{
    Cooldude2606 = 'ColorValue'
}

Dependencies

utils.event

Fields

global.datastores	Save datastores in the global table

Datastore Manager

metatable	Metatable used on datastores
connect(datastoreName[, saveToDisk=false][, autoSave=false][, propagateChanges=false])	Make a new datastore connection, if a connection already exists then it is returned
combine(datastoreName, subDatastoreName)	Make a new datastore that stores its data inside of another one
ingest(action, datastoreName, key, valueJson)	Ingest the result from a request, this is used through a rcon interface to sync data
debug([datastoreName])	Debug, Use to get all datastores, or return debug info on a datastore
name_serializer(rawKey)	Commonly used serializer, returns the name of the object

Datastore Internal

debug()	Debug, Get the debug info for this datastore
raw_get(key[, fromChild=false])	Internal, Get data following combine logic
raw_set(key, value)	Internal, Set data following combine logic
serialize(rawKey)	Internal, Return the serialized key
write_action(action, key, value)	Internal, Writes an event to the output file to be saved and/or propagated

Datastore Local

combine(subDatastoreName)	Create a new datastore which is stores its data inside of this datastore
set_serializer(callback)	Set a callback that will be used to serialize keys which aren't strings
set_metadata(tags)	Set metadata tags on this datastore which can be accessed by other scripts
get(key[, default])	Get a value from local storage, option to have a default value
set(key, value)	Set a value in local storage, will trigger on_update then on_save, save_to_disk and auto_save is required for on_save
increment(key[, delta=1])	Increment the value in local storage, only works for number values, will trigger on_update then on_save, save_to_disk and auto_save is required for on_save
update(key, callback)	Use a function to update the value locally, will trigger on_update then on_save, save_to_disk and auto_save is required for on_save
remove(key)	Remove a value locally and on the external source, works regardless of propagateChanges
get_all([callback])	Get all keys in this datastore, optional filter callback
update_all(callback)	Update all keys in this datastore using the same update function

Datastore External

request(key)	Request a value from an external source, will trigger on_load when data is received
save(key)	Save a value to an external source, will trigger on_save before data is saved, save_to_disk must be set to true
unload(key)	Save a value to an external source and remove locally, will trigger on_unload then on_save, save_to_disk is not required for on_unload
message(key, message)	Use to send a message over the connection, works regardless of saveToDisk and propagateChanges
save_all([callback])	Save all the keys in the datastore, optional filter callback
unload_all([callback])	Unload all the keys in the datastore, optional filter callback

Events

raise_event(event_name, key[, value][, source])	Internal, Raise an event on this datastore
on_load	Register a callback that triggers when data is loaded from an external source, returned value is saved locally
on_save	Register a callback that triggers before data is saved, returned value is saved externally
on_unload	Register a callback that triggers before data is unloaded, returned value is ignored
on_message	Register a callback that triggers when a message is received, returned value is ignored
on_update	Register a callback that triggers any time a value is changed, returned value is ignored

Dependencies

# utils.event

Fields

# global.datastores

Save datastores in the global table

Datastore Manager

# metatable

Metatable used on datastores

Fields:

• __index
• __newidnex
• __call

# connect(datastoreName[, saveToDisk=false][, autoSave=false][, propagateChanges=false])

Make a new datastore connection, if a connection already exists then it is returned

Parameters:

• datastoreName : (string) The name that you want the new datastore to have, this can not have any whitespace
• saveToDisk : (boolean) When set to true, using the save method with write the data to datastore.pipe (default: false)
• autoSave : (boolean) When set to true, using any method which modifies data will cause the data to be saved (default: false)
• propagateChanges : (boolean) When set to true, using the save method will send the data to all other connected servers (default: false)

Returns:

• (table) The new datastore connection that can be used to access and modify data in the datastore

Usage:

-- Connecting to the test datastore which will allow saving to disk
local ExampleData = Datastore.connect('ExampleData', true) -- saveToDisk

# combine(datastoreName, subDatastoreName)

Make a new datastore that stores its data inside of another one

Parameters:

• datastoreName : (string) The name of the datastore that will contain the data for the new datastore
• subDatastoreName : (string) The name of the new datastore, this name will also be used as the key inside the parent datastore

Returns:

• (table) The new datastore connection that can be used to access and modify data in the datastore

Usage:

-- Setting up a datastore which stores its data inside of another datastore
local BarData = Datastore.combine('ExampleData', 'Bar')

# ingest(action, datastoreName, key, valueJson)

Ingest the result from a request, this is used through a rcon interface to sync data

Parameters:

• action : (string) The action that should be done, can be: remove, message, propagate, or request
• datastoreName : (string) The name of the datastore that should have the action done to it
• key : (string) The key of that datastore that is having the action done to it
• valueJson : (string) The json string for the value being ingested, remove does not require a value

Usage:

-- Replying to a data request
Datastore.ingest('request', 'ExampleData', 'TestKey', 'Foo')

# debug([datastoreName])

Debug, Use to get all datastores, or return debug info on a datastore

Parameters:

• datastoreName : (string) The name of the datastore to get the debug info of (optional)

Usage:

-- Get all the datastores
local datastores = Datastore.debug()

-- Getting the debug info for a datastore
local debug_info = Datastore.debug('ExampleData')

# name_serializer(rawKey)

Commonly used serializer, returns the name of the object

Parameters:

• rawKey : (any) The raw key that will be serialized, this can be things like player, force, surface, etc

Returns:

• (string) The name of the object that was passed

Usage:

-- Using the name serializer for your datastore
local ExampleData = Datastore.connect('ExampleData')
ExampleData:set_serializer(Datastore.name_serializer)

Datastore Internal

# debug()

Debug, Get the debug info for this datastore

Returns:

• (table) The debug info for this datastore, contains stuff like parent, settings, children, etc

Usage:

-- Get the debug info for a datastore
local ExampleData = Datastore.connect('ExampleData')
local debug_info = ExampleData:debug()

# raw_get(key[, fromChild=false])

Internal, Get data following combine logic

Parameters:

• key : (string) The key to get the value of from this datastore
• fromChild : (boolean) If the get request came from a child of this datastore (default: false)

Returns:

• (any) The value that was stored at this key in this datastore

Usage:

-- Internal, Get the data from a datastore
local value = self:raw_get('TestKey')

# raw_set(key, value)

Internal, Set data following combine logic

Parameters:

• key : (string) The key to set the value of in this datastore
• value : (any) The value that will be set at this key

Usage:

-- Internal, Set the value in a datastore
self:raw_set('TestKey', 'Foo')

# serialize(rawKey)

Internal, Return the serialized key

Parameters:

• rawKey : (any) The key that needs to be serialized, if it is already a string then it is returned

Returns:

• (string) The key after it has been serialized

Usage:

-- Internal, Ensure that the key is a string
key = self:serialize(key)

# write_action(action, key, value)

Internal, Writes an event to the output file to be saved and/or propagated

Parameters:

• action : (string) The action that should be wrote to datastore.pipe, can be request, remove, message, save, propagate
• key : (string) The key that the action is being preformed on
• value : (any) The value that should be used with the action

Usage:

-- Write a data request to datastore.pipe
self:write_action('request', 'TestKey')

-- Write a data save to datastore.pipe
self:write_action('save', 'TestKey', 'Foo')

Datastore Local

# combine(subDatastoreName)

Create a new datastore which is stores its data inside of this datastore

Parameters:

• subDatastoreName : (string) The name of the datastore that will have its data stored in this datastore

Returns:

• (table) The new datastore that was created inside of this datastore

Usage:

-- Add a new sub datastore
local ExampleData = Datastore.connect('ExampleData')
local BarData = ExampleData:combine('Bar')

# set_serializer(callback)

Set a callback that will be used to serialize keys which aren't strings

Parameters:

• callback : (function) The function that will be used to serialize non string keys passed as an argument

Usage:

-- Set a custom serializer, this would be the same as Datastore.name_serializer
local ExampleData = Datastore.connect('ExampleData')
ExampleData:set_serializer(function(rawKey)
    return rawKey.name
end)

# set_metadata(tags)

Set metadata tags on this datastore which can be accessed by other scripts

Parameters:

• tags : (table) A table of tags that you want to set in the metadata for this datastore

Usage:

-- Adding metadata that could be used by a gui to help understand the stored data
local ExampleData = Datastore.connect('ExampleData')
ExampleData:set_metadata{
    caption = 'Test Data',
    tooltip = 'Data used for testing datastores',
    type = 'table'
}

# get(key[, default])

Get a value from local storage, option to have a default value

Parameters:

• key : (any) The key that you want to get the value of, must be a string unless a serializer is set
• default : (any) The default value that will be returned if no value is found in the datastore (optional)

Usage:

-- Get a key from the datastore, the default will be deep copied if no value exists in the datastore
local ExampleData = Datastore.connect('ExampleData')
local value = ExampleData:get('TestKey')

# set(key, value)

Set a value in local storage, will trigger on_update then on_save, save_to_disk and auto_save is required for on_save

Parameters:

• key : (any) The key that you want to set the value of, must be a string unless a serializer is set
• value : (any) The value that you want to set for this key

Usage:

-- Set a value in the datastore, this will trigger on_update, if auto_save is true then will trigger save
local ExampleData = Datastore.connect('ExampleData')
ExampleData:set('TestKey', 'Foo')

# increment(key[, delta=1])

Increment the value in local storage, only works for number values, will trigger on_update then on_save, save_to_disk and auto_save is required for on_save

Parameters:

• key : (any) The key that you want to increment the value of, must be a string unless a serializer is set
• delta : (number) The amount that you want to increment the value by, can be negative or a decimal (default: 1)

Usage:

-- Increment a value in a datastore, the value must be a number or nil, if nil 0 is used as the start value
local ExampleData = Datastore.connect('ExampleData')
ExampleData:increment('TestNumber')

# update(key, callback)

Use a function to update the value locally, will trigger on_update then on_save, save_to_disk and auto_save is required for on_save

Parameters:

• key : (any) The key that you want to apply the update to, must be a string unless a serializer is set
• callback : (function) The function that will be used to update the value at this key

Usage:

-- Using a function to update a value, if a value is returned then this will be the new value
local ExampleData = Datastore.connect('ExampleData')
ExampleData:increment('TestKey', function(key, value)
    return value..value
end)

# remove(key)

Remove a value locally and on the external source, works regardless of propagateChanges

Parameters:

• key : (any) The key that you want to remove locally and externally, must be a string unless a serializer is set

Usage:

-- Remove a key locally and externally
local ExampleData = Datastore.connect('ExampleData')
ExampleData:remove('TestKey')

# get_all([callback])

Get all keys in this datastore, optional filter callback

Parameters:

• callback : (function) The filter function that can be used to filter the results returned (optional)

Returns:

• (table) All the data that is in this datastore, filtered if a filter was provided

Usage:

-- Get all the data in this datastore
local ExampleData = Datastore.connect('ExampleData')
local data = ExampleData:get_all()

-- Get all the data in this datastore, with a filter
local ExampleData = Datastore.connect('ExampleData')
local data = ExampleData:get_all(function(key, value)
    return type(value) == 'string'
end)

# update_all(callback)

Update all keys in this datastore using the same update function

Parameters:

• callback : (function) The update function that will be applied to each key

Usage:

-- Get all the data in this datastore, with a filter
local ExampleData = Datastore.connect('ExampleData')
ExampleData:update_all(function(key, value)
    return value..value
end)

Datastore External

# request(key)

Request a value from an external source, will trigger on_load when data is received

Parameters:

• key : (any) The key that you want to request from an external source, must be a string unless a serializer is set

Usage:

-- Request a key from an external source, on_load is triggered when data is received
local ExampleData = Datastore.connect('ExampleData')
ExampleData:request('TestKey')

# save(key)

Save a value to an external source, will trigger on_save before data is saved, save_to_disk must be set to true

Parameters:

• key : (any) The key that you want to save to an external source, must be a string unless a serializer is set

Usage:

-- Save a key to an external source, save_to_disk must be set to true for there to be any effect
local ExampleData = Datastore.connect('ExampleData')
ExampleData:save('TestKey')

# unload(key)

Save a value to an external source and remove locally, will trigger on_unload then on_save, save_to_disk is not required for on_unload

Parameters:

• key : (any) The key that you want to unload from the datastore, must be a string unless a serializer is set

Usage:

-- Unload a key from the datastore, get will now return nil and value will be saved externally if save_to_disk is set to true
local ExampleData = Datastore.connect('ExampleData')
ExampleData:unload('TestKey')

# message(key, message)

Use to send a message over the connection, works regardless of saveToDisk and propagateChanges

Parameters:

• key : (any) The key that you want to send a message over, must be a string unless a serializer is set
• message : (any) The message that you want to send to other connected servers, or external source

Usage:

-- Send a message to other servers on this key, can listen for messages with on_message
local ExampleData = Datastore.connect('ExampleData')
ExampleData:message('TestKey', 'Foo')

# save_all([callback])

Save all the keys in the datastore, optional filter callback

Parameters:

• callback : (function) The filter function that can be used to filter the keys saved (optional)

Usage:

-- Save all the data in this datastore
local ExampleData = Datastore.connect('ExampleData')
local data = ExampleData:save_all()

-- Save all the data in this datastore, with a filter
local ExampleData = Datastore.connect('ExampleData')
ExampleData:save_all(function(key, value)
    return type(value) == 'string'
end)

# unload_all([callback])

Unload all the keys in the datastore, optional filter callback

Parameters:

• callback : (function) The filter function that can be used to filter the keys unloaded (optional)

Usage:

-- Unload all the data in this datastore
local ExampleData = Datastore.connect('ExampleData')
ExampleData:unload_all()

-- Unload all the data in this datastore, with a filter
local ExampleData = Datastore.connect('ExampleData')
ExampleData:unload_all(function(key, value)
    return type(value) == 'string'
end)

Events

# raise_event(event_name, key[, value][, source])

Internal, Raise an event on this datastore

Parameters:

• event_name : (string) The name of the event to raise for this datastore
• key : (string) The key that this event is being raised for
• value : (any) The current value that this key has, might be a deep copy of the value (optional)
• source : (string) Where this call came from, used to do event recursion so can be parent or child (optional)

Returns:

• (any) The value that is left after being passed through all the event handlers

Usage:

-- Internal, Getting the value that should be saved
value = self:raise_event('on_save', key, value)

# on_load

Register a callback that triggers when data is loaded from an external source, returned value is saved locally

• callback : (function) The handler that will be registered to the on_load event

Usage:

-- Adding a handler to on_load, returned value will be saved locally, can be used to deserialize the value beyond a normal json
local ExampleData = Datastore.connect('ExampleData')
ExampleData:on_load(function(key, value)
    game.print('Test data loaded for: '..key)
end)

# on_save

Register a callback that triggers before data is saved, returned value is saved externally

• callback : (function) The handler that will be registered to the on_load event

Usage:

-- Adding a handler to on_save, returned value will be saved externally, can be used to serialize the value beyond a normal json
local ExampleData = Datastore.connect('ExampleData')
ExampleData:on_save(function(key, value)
    game.print('Test data saved for: '..key)
end)

# on_unload

Register a callback that triggers before data is unloaded, returned value is ignored

• callback : (function) The handler that will be registered to the on_load event

Usage:

-- Adding a handler to on_unload, returned value is ignored, can be used to clean up guis or local values related to this data
local ExampleData = Datastore.connect('ExampleData')
ExampleData:on_load(function(key, value)
    game.print('Test data unloaded for: '..key)
end)

# on_message

Register a callback that triggers when a message is received, returned value is ignored

• callback : (function) The handler that will be registered to the on_load event

Usage:

-- Adding a handler to on_message, returned value is ignored, can be used to receive messages from other connected servers without saving data
local ExampleData = Datastore.connect('ExampleData')
ExampleData:on_message(function(key, value)
    game.print('Test data message for: '..key)
end)

# on_update

Register a callback that triggers any time a value is changed, returned value is ignored

• callback : (function) The handler that will be registered to the on_load event

Usage:

-- Adding a handler to on_update, returned value is ignored, can be used to update guis or send messages when data is changed
local ExampleData = Datastore.connect('ExampleData')
ExampleData:on_update(function(key, value)
    game.print('Test data updated for: '..key)
end)