Comprehensive GlusterFS Installation and Configuration Guide

Introduction

GlusterFS is a distributed file system that allows you to scale storage across multiple servers. This guide covers a step-by-step installation, configuration, and verification for a replicated GlusterFS setup with two servers. Note: You can use GlusterFs for distributed storage as well, but here I am covering only the replication.

Main Concepts

• Bricks: Actual storage locations (physical directories on each node).** Never touch the physical data !!!**
• Mount Points: Where clients and servers access the GlusterFS volume and edit files.
• Replication: Ensures data consistency across multiple nodes.

Pre-requisites

- Two or more Linux servers
- Each server should have at least two storage devices:
   -  One for the operating system.
    - One for GlusterFS bricks (data storage).
- RAID 0 or RAID 1 setup (if needed for additional performance/redundancy).
- Firewall rules allowing traffic on ports 24007, 24008, and 49152-49251.
- Static IP addresses.

Steps to create replicated Volumes using Glusterfs:

Install GlusterFS on Both Servers

sudo apt update -y && sudo apt install -y glusterfs-server

Start and Enable GlusterFS Service

sudo systemctl enable --now glusterd sudo systemctl status glusterd

Check the Installed Version

gluster --version

Configure the GlusterFS Cluster Peer Connection (Run on server1 to add server2):

sudo gluster peer probe server2

Verify the peer status:

sudo gluster peer status

Creating a Replicated Volume Create Brick Directories on Both Servers Ensure RAID 0 (or another storage system) is mounted at /mnt/storage:

sudo mkdir -p /mnt/storage/dont_touch

sudo chown -R gluster:gluster /mnt/storage/dont_touch

sudo chmod -R 775 /mnt/storage/dont_touch

Create and Start a Replicated Volume Run this command on one of the servers (server1):

sudo gluster volume create myvol replica 2 transport tcp server1:/mnt/storage/dont_touch server2:/mnt/storage/dont_touch

Start the volume:

sudo gluster volume start myvol

Check volume info:

sudo gluster volume info myvol

Mount the GlusterFS Volume on Servers to be able to edit files (never touch the actual data or Bricks)

Create a Mount Point

On both servers:

sudo mkdir -p /home/glusterfs-mount

Enable Auto-Mount Edit /etc/fstab:

sudo vim /etc/fstab

Add this line:

localhost:/myvol /home/glusterfs-mount glusterfs defaults,_netdev 0 0

Reread the fstab to mount it

mount -a

Verify Replication & Functionality

Check Volume Status

sudo gluster volume status myvol

Test Replication On Server 1 (server1):

echo "Hello from Server 1" | sudo tee /mnt/glusterfs-mount/server1.txt

On Server 2 (server2):

ls -lah /mnt/glusterfs-mount/ cat /mnt/glusterfs-mount/server1.txt

✅ If you see the file on Server 2, replication is working.

GlusterFS Cheat Sheet

sudo gluster peer probe server2	Add a new GlusterFS node
sudo gluster peer status	Check peer status
sudo gluster volume create myvol replica 2 transport tcp server1:/mnt/storage/my_vol/glusterfs-data server2:/mnt/storage/my_vol/glusterfs-data force	Create a replicated volume
sudo gluster volume start myvol	Start a volume
sudo gluster volume status myvol	Check volume status
sudo gluster volume info myvol	Show volume details
sudo gluster volume heal myvol info	Check if self-healing is needed
sudo gluster volume heal myvol full	Force self-healing
df -h grep gluster	verification
sudo mount -t glusterfs localhost:/myvol /mnt/glusterfs-mount	Manually mount the GlusterFS volume
sudo nano /etc/fstab	Configure auto-mounting after reboot

🚀Conclusion

**Bricks** store actual data (/mnt/storage/dont_touch/ on both servers). **Never !!!** touch the actual data and ** Only ** edit data using mounting points
**Mount Points** allow access to the GlusterFS volume (/home/glusterfs-mount).
**Replication ensures file consistency between servers.**
**Note!!!** I say it again, Always use /home/glusterfs-mount/ to access files, NOT the brick directory.
**Enable auto-mounting via /etc/fstab to persist mounts after reboot.**

✅ Now, your GlusterFS setup is fully operational! 🎉

Optional: Adding this glusterfs to other servers (mounting) ** 1. Install glusterfs **

sudo apt update -y && sudo apt install -y glusterfs-server

2. create mount point directory

mkdir /home/glusterfs_mount/10_0_1_4

3. Add mount point to fstab

vim /etc/fstab

add this code at the end of file.

10.0.1.4,10.0.1.5:/myvol /home/glusterfs_mount/10_0_1_4 glusterfs defaults,_netdev 0 0

4. Apply it

mount -a

5. Verification

df -h | grep gluster

You can also check the directory and see your files in the mount point.