Part 10: Syncing Cluster Nodes

January 20, 2020 - Reading time: 47 minutes

Admin Node

We’ve created three identical cluster nodes, each with nginx, and bludit.  Furthermore, each node is part of a load balancing pool for https://makerflare.com.  However, we need to now create a fourth cluster node which will serve as our authoring node.  We will use a subdomain called admin.makerflare.com for this node.  

 

Visitors continue to use https://makerflare.com to access your content (served by 3 nodes).  However, https://admin.makerflare.com will be used to author content (using our 4th node).  Once saved, the content (blog pages, images, etc) will be automatically synced to the other three nodes.  

 

The installation process for admin.makerflare.com will not be covered here.  The only change you need is to change the URL in your Argo Tunnel config.yml file to reflect the subdomain.  Once done, bludit will need to be installed by first browsing to https://admin.makerflare.com.  This will ensure the site.php file reflects the correct URL.  You can also of course edit this URL manually if needed.  

 

We are now ready to install Syncthing, a real-time file sync application that will keep content created on admin.makerflare.com in sync with all our cluster nodes.  Thankfully this process is far less painful than having to listen to Insync!

 

Syncthing

We will use an application called Syncthing, and configure our dev.makerflare.com node to be the master, which sends data to the other three nodes.  For this exercise, we will configure a new remote node called remote_node_1 which will reside at a different geographic location from the rest of our cluster.  We will install Syncthing, and then use this to sync existing content from our admin node to this new node.  

 

First, let’s install Syncthing as follows:

 

sudo apt-get install syncthing

 

Note that the application does not install itself as a service.  We will do that later on.  Once installed, you can start the application by simply typing syncthing.  Doing so will also launch a browser and connect to http://127.0.0.1:8384.

 

 

We can now configure our remote node to receive data from our admin node.  First, we set a name for our node, and then configure the web GUI to receive remote connections.  We do this by changing the localhost IP from 127.0.0.1 to 0.0.0.0.  We can then set a username/password for the GUI.  


Syncthing represents each device with an ID string.  We simply need to copy this string and enter it into any remote device with which we want to establish a connection.  

 

We can now copy the ID string from admin.makerflare.com, and establish a connection to our remote node.  

 

Back on our admin node, you will be prompted to accept the remote connection we created.  Now, edit the connection, and you will get the option to add the folders you previously allowed to be shared on the admin node.

 

Let’s now go through the sharing configuration on the admin node.  We are going to share three folders (databases, pages, and uploads) in bludit’s “bl-content” folder.  This is where all newly created content is stored.  However, we also need to exclude certain files because these files should be unique to each node.  You can do this by clicking on the “Ignore Patterns” tab.

 

For example, the databases folder has some files such as “site.php” which we don’t want to copy over.  However, there are other files such as “pages.php” which we do want to sync.  Pages.php has an index of all content pages in our blog, and this is written to each time new content is created.  It is therefore logical to want to sync this file across all four nodes.  

 

For the databases folder, we will now state to NOT ignore certain files which we want synced, and then ignore the rest.  

 

!include.txt

!tags.php

!categories.php

!pages.php

*

 

The above tells syncthing to NOT ignore (or to include) four files, and then ignore everything else (using the wildcard *).

 

Finally, we set the option so that Syncthing will consider this folder (on the admin node) as the master, and will only send data from this folder.  This means that if the “databases” folder on any other node changes, these changes will NOT propagate back to the admin node.  The folders should start syncing once the changes have been saved.  Note, it can take up to five min for the sync process to start.

 

 

Back on our remote node, we need to accept the folder sync from our admin node.  You will also need to select the destination folder locally.  In our case, the /var/www/html/bl-content/databases folder (and also pages, and uploads).  The Syncthing GUI will automatically present the option to add these folders.

 

You should also configure these local folders to only RECEIVE data.  This is similar to the configuration you did previously where you set the folders on the admin node to only SEND data.  Finally, we do not want changes in permissions to trigger a sync session.  Check on the option to ignore permissions.  

 

You can now repeat this process for each of the folders we want to sync from the admin node.  The sync process will take about 3-5 minutes to initialise.

 

 

The Syncthing GUI shows us that the files have been synced.  We can now access the site and can see that all the blog entries have been synced.  This means we now have yet another node active.  

 

 

We can now configure Syncthing as a service, and to start when our Raspberry Pi is powered on.  First, download the [email protected] and syncthing-resume.service files here:

 

https://github.com/syncthing/syncthing/tree/master/etc/linux-systemd/system

 

Now copy the files to /etc/systemd/system.  Finally enable and start the service:

 

sudo systemctl enable [email protected]

sudo systemctl start [email protected]

 

Note that you must replace “username” with an appropriate user in your environment.

 

Crontab

We had previously selected the option to ignore changes in permissions.  This is because Syncthing does not sync ownership.  We lose this information each time a folder/file is synced.  This causes problems since we need a certain group to have ownership on the files/folders in order for nginx to serve content correctly for visitors.

 

We can use crontab (scheduling application in Raspbian and all unix operating systems) to do this as follows:

 

1.     Create a text file and save as sync_permissions using nano.  The contents are as follows:

sudo chown -R www-data:pi /var/www/html
sudo chmod -R 770 /var/www/html

2.     Make this file an executable by typing the following command:

sudo chmod u+x sync_permissions

3.     Edit the crontab config file for our root user:  

sudo crontab -e

4.     Add the following line at the bottom:

*/5 * * * * /home/pi/Documents/sync_permissions > /home/pi/Documents/cronlog 2>%1

This tells crontab to run the sync_permissions script every 5 minutes, and log the result to a file called cronolog.  The script itself resets ownership and permissions to the user group which as the right permissions to access our content.  This content can then be displayed correctly for our visitors.  

 

Recap: 

We have created a separate node for authoring content.  This node can be accessed by using a subdomain of makerflare (admin.makerflare.com).  We were able to set this up by modifying the config file for Argo Tunnel.  

 

We then set up real-time file syncing between nodes so that any created or modified content is synced instantly between all our cluster nodes.  Remember, that Cloudflare’s load balancer is able to route requests automatically across all our nodes. 

 

Finally, we have also shown how easy it is to spin up new cluster nodes which can be deployed wherever we want.  

 

 

About

A playground of creativity that combines Cloudflare technology with hobbyists.