6.8.2 Backing up Databases

In a certain way, any distributed blockchain-like system may be defined as a database that stores a certain state. Performing database backups is a normal system operation. However, distributed nature may make this operation less than efficient:

- The operating platform is in constant ledger synchronization mode. Therefore, the loss of data by one node does not affect the others. When the lost node is restored, it receives an updated portion of data.

- Writing new data into the ledger requires going through a consensus procedure (see 3.4.1.2), so that “offline” changes, conflicting data, and false entries will be rejected by the system’s nodes.

Despite these remarks, undergoing backup and then restoring the database may become useful in some special cases:

- Testing and modelling of having the network switch between various initial data (tests on different initial states of the network)

- Relaunching the network (hard fork) when all nodes agree to roll back to a certain state preceding the one they are in now

- Preventative work for a private distributed network, when all nodes are moving to a new hardware platform, but it will be necessary to restore the current state.

To perform a database backup, follow the appropriate procedures:

  • Explore the platform’s physical storage architecture:

- Master data is stored in the LMDB database which is a simplified NoSQL database (key-value store) based on the . The database provides significant speed through memory mapping. It is incorrect to compare this database with well-known relational databases or even large client-server solutions, since LMDB only implements a storage layer (scheme free) and does not use write-ahead transaction logs. Thus, LMDB does not have many additional mechanisms that allow you to run SQL queries or connect data through ODBC.

- One of the advantages of using the LMDB is its lightness, speed, and efficient handling of blockages. The figure below presents the basic idea of how this solution compares to full storage implementations.

- When initializing under Docker, the DGT platform creates a specialized catalog for storing files of the LMDB’s database in the «DGT-Matagami/dgt_clust/[Cluster_Num]/[Node_Num]/data» system directory, for instance «DGT-Matagami/dgt_clust/c1/dgt1/data». Basic data:

#

LMDB File

Description

1

*-lock

Files created by the system when accessing the database. In a normal situation, when the system is stopped, such files should not be present, however, they may remain during an unexpected server overload or incorrect shutdown

2

block-0*.lmdb

Main block storage, contains KV pair: block_id + block-information

3

merkle-0*.lmdb

Merkle tree storage that ensures state integrity (root hash/global state), also represented as a pair of KV.

4

txn-receipts-0*.lmdb

This database stores receipts for successfully accepted transactions, as well as other events associated with transactions

5

pbft_consensus*.lmdb

Database of votes in the process of reaching F-BFT consensus

6

*.lmdb

Other repositories related to application solutions

Reserving data directly related to the platform core requires the correct saving of all specified data.

- Receiving ledger data in the form of a set of records does not require reservation procedures and can be performed through the API (see 3.9) in the form of uploading a list of transactions, batches, and / or blocks. The Dashboard component (see 6.4) also gives you the ability to view records interactively.

- The backup procedure can be performed in the form of dumping (complete unloading of the relevant data with the possibility of its subsequent loading) or in the form of copying (saving) files of the database.

- Here and below, the discussion will concern only cold backups that are performed in a stopped state. The hot backup procedure is not applicable to distributed systems such as blockchain.

- Additional solution components, such as Grafana (see 6.8.4) may have their own databases, the back process of which is discussed in the individual component sections.

Some components, such as Oracles, may access data outside of the blockchain solution by keeping only references in the ledger. It is not recommended to store confidential data, personal data, or data subject to the risk of attacks by quantum computers in the ledger. These components and services may use their own secure copy and restore procedures.

· To carry out further operations, stop the platform as described in 6.8.1 and reboot the server.

· To perform dumping or current databases:

  • Install the LMDB utilities if you have not already done so:

sudo  apt update
sudo apt install lmdb-utils

  • Perform the following operation for each database [DB_NAME] in the «data» directory (DGT-Matagami/dgt_clust/c1/dgt1/data) 

sudo mdb_dump -n /path/to/[DB_NAME] > /backup-path/to/[DB_NAME].dump

For example, the dump command from the «data» directory:

sudo mdb_dump -n merkle-01.lmdb >  ~/merkle-01.lmdb.dump

  • To restore a database from a dump, run the following command, making sure the Docker services are stopped:

cd /path/to
mdb_load -n -f /backup-path/to/[DB_NAME].dump

Example:

Change to the “data” directory and run the following command:

sudo mdb_load -n -f ~/merkle-01.lmdb.dump merkle-01.lmdb

· To copy a database while the Docker services are stopped:

- Stop containers – see 6.8.1

- Back up data files:

sudo cp –sparse-always /path/to/[DB_NAME] /backup-path/to/[DB_NAME].back

  • To restore the file, stop the containers and perform a reverse copy. Start the system according to instructions 6.8.1

Previous6.8.1 Stopping and Restarting the PlatformNext6.8.3 Network Performance

Last updated