Backing up users’ home directories in a Linux installation that uses systemd

January 15, 2018 4 Comments

This is to explain how I configured Lubuntu 17.10 on my family’s PC (single seat; multiple user accounts) to backup all users’ files to a permanently-connected external USB HDD. I wanted a basic solution that would backup automatically every user’s home directory at shutdown but not when rebooting, and would do this whichever user happens to be using the PC. We have no need to archive older versions of the same file, so overwriting a file on the external USB HDD with a newer version of that file from the PC’s HDD is fine. However, we do not want files on the external USB HDD to be deleted if the corresponding files have been deleted from the PC’s HDD. Additionally, I want a time-stamped record of the backup process to be logged to a file in my home directory so I can check periodically what has been happening. As explained in my previous post, two files are used to achieve all these things:

  • a Bash script to a) determine whether the system is being rebooted, b) remount the external USB HDD on a suitable mount point, c) copy the files to the external USB HDD, and d) write to the log file;
  • a systemd unit file to launch the Bash script.

The two files are described below.

By the way, I also tried using systemd-inhibit in this case, and it did not work. Looking at the manual page for systemd-inhibit, it could be used to execute a program (the example given is ‘systemd-inhibit wodim foobar.iso‘), however I found that systemd-inhibit did not prevent systemd from shutting down the machine before the backup Bash script had run to completion.

Bash script

I created a file /usr/local/sbin/backup_home_directories.sh owned by the root user, with the following permissions and contents:

fitzcarraldo@aspirexc600:~$ ls -la /usr/local/sbin/backup_home_directories.sh
-rwxr-xr-x 1 root root 2570 Jan  8 20:37 /usr/local/sbin/backup_home_directories.sh

#!/bin/bash

# This script backs up to an external USB HDD (NTFS) labelled "FREECOM HDD" the contents of the home directories
# of the users of this Lubuntu 17.10 installation if the system is shutting down but not rebooting.
# It is launched by a systemd service /etc/systemd/system/backup-to-usb-hdd.service.

# Find out if the system is rebooting (as opposed to being shut down):
REBOOT=$( systemctl list-jobs | egrep -q 'reboot.target.*start' && echo "rebooting" || echo "not_rebooting" )
if [ $REBOOT = "not_rebooting" ]; then
# Only execute the following steps if the system is shutting down but not rebooting:
    # Clean up if the backup did not complete last time:
    umount /media/usbhdd 2>/dev/null # Make sure you enter this line correctly.
    rm -rf /media/usbhdd/* # Make sure you enter this line correctly.
    # Unmount the external USB HDD if mounted by udisks2 with the logged-in username in the path:
    umount /media/*/FREECOM\ HDD 2>/dev/null
    # Find out the USB HDD device:
    DEVICE=$( blkid | grep "FREECOM\ HDD" | cut -d ":" -f1 )
    # Create a suitable mount point if it does not already exist, and mount the device on it: 
    mkdir /media/usbhdd 2>/dev/null
    mount -t ntfs-3g $DEVICE /media/usbhdd 2>/dev/null
    # Create the backup directory on the USB HDD if it does not already exist:
    mkdir /media/usbhdd/Lubuntu_home_folders_backup 2>/dev/null
    # Backup recursively all the home directories of all the users, and add a time-stamped summary to the log file: 
    echo "********** Backing up Acer Aspire XC600 users' home directories **********" >> /home/fitzcarraldo/backup.log
    date >> /home/fitzcarraldo/backup.log
    # Log username of user shutting down the PC (may not be this user if Switch User was used):
    echo -ne "User who shutdown PC (may not be this user if Switch User has been used): " >> /home/fitzcarraldo/backup.log
    last | cut -d " " -f1 | head -1 >> /home/fitzcarraldo/backup.log
    sleep 2s
    # To backup the directories and files I prefer to use the following cp command rather than rsync:
    cp --recursive --update --preserve=all --no-dereference --force /home/ /media/usbhdd/Lubuntu_home_folders_backup 2>> /home/fitzcarraldo/backup.log
    echo "Copying completed" >> /home/fitzcarraldo/backup.log
    date >> /home/fitzcarraldo/backup.log
    echo "********** Backup completed **********" >> /home/fitzcarraldo/backup.log
    cp /home/fitzcarraldo/backup.log /media/usbhdd/Lubuntu_home_folders_backup/home/fitzcarraldo/
    # Unmount the USB HDD so that udisks2 can subsequently re-mount it with the user's username in the path:  
    umount /media/usbhdd
fi
exit 0

The Bash script unmounts the external USB HDD and mounts it to /media/usbhdd before performing a backup. This is done because the Udisks daemon (udisksd) automounts the HDD at login to a mountpoint /media/<username>/FREECOM HDD  (e.g. ‘/media/fitzcarraldo/FREECOM HDD‘ when I log in, ‘/media/claudia/FREECOM HDD‘ when user claudia logs in, and so on).

systemd unit file

I created a file /etc/systemd/system/backup-to-usb-hdd.service owned by the root user, with the following permissions and contents:

fitzcarraldo@aspirexc600:~$ ls -la /etc/systemd/system/backup-to-usb-hdd.service 
-rw-r--r-- 1 root root 274 Jan  8 17:51 /etc/systemd/system/backup-to-usb-hdd.service

[Unit]
Description=Backup home directories of all users to USB HDD
DefaultDependencies=no
Before=shutdown.target halt.target
RequiresMountsFor=/home

[Service]
Type=oneshot
ExecStart=/usr/local/sbin/backup_home_directories.sh

[Install]
WantedBy=halt.target shutdown.target

Then I enabled the unit file as follows:

fitzcarraldo@aspirexc600:~$ sudo systemctl enable backup-to-usb-hdd.service
Created symlink /etc/systemd/system/halt.target.wants/backup-to-usb-hdd.service → /etc/systemd/system/backup-to-usb-hdd.service.
Created symlink /etc/systemd/system/shutdown.target.wants/backup-to-usb-hdd.service → /etc/systemd/system/backup-to-usb-hdd.service.
fitzcarraldo@aspirexc600:~$

Reboot (or shutdown and restart) to launch it.

The log file

The log file /home/fitzcarraldo/backup.log looked like the following after shutting down a couple of times:

********** Backing up Acer Aspire XC600 users' home directories **********
Mon  8 Jan 21:07:05 GMT 2018
User who shutdown PC (may not be this user if Switch User has been used): fitzcarraldo
Copying completed
Mon  8 Jan 21:07:32 GMT 2018
********** Backup completed **********
********** Backing up Acer Aspire XC600 users' home directories **********
Mon  8 Jan 21:15:31 GMT 2018
User who shutdown PC (may not be this user if Switch User has been used): fitzcarraldo
Copying completed
Mon  8 Jan 21:15:48 GMT 2018
********** Backup completed **********

Whenever the system is rebooted by any user, nothing is appended to the log file. As desired, entries are only appended to the log file when the system is shutdown by any user.

Manually initiated backups

An added benefit is that any user in the sudo group can run the Bash script from the command line at any time to backup without having to shutdown the PC, as illustrated below:

claudia@aspirexc600:~$ cat /home/fitzcarraldo/backup.log 
********** Backing up Acer Aspire XC600 users' home directories **********
Mon  8 Jan 21:07:05 GMT 2018
User who shutdown PC (may not be this user if Switch User has been used): fitzcarraldo
Copying completed
Mon  8 Jan 21:07:32 GMT 2018
********** Backup completed **********
********** Backing up Acer Aspire XC600 users' home directories **********
Mon  8 Jan 21:15:31 GMT 2018
User who shutdown PC (may not be this user if Switch User has been used): fitzcarraldo
Copying completed
Mon  8 Jan 21:15:48 GMT 2018
********** Backup completed **********
claudia@aspirexc600:~$ date && sudo backup_home_directories.sh
Tue  9 Jan 01:57:50 GMT 2018
[sudo] password for claudia:
claudia@aspirexc600:~$ cat /home/fitzcarraldo/backup.log 
********** Backing up Acer Aspire XC600 users' home directories **********
Mon  8 Jan 21:07:05 GMT 2018
User who shutdown PC (may not be this user if Switch User has been used): fitzcarraldo
Copying completed
Mon  8 Jan 21:07:32 GMT 2018
********** Backup completed **********
********** Backing up Acer Aspire XC600 users' home directories **********
Mon  8 Jan 21:15:31 GMT 2018
User who shutdown PC (may not be this user if Switch User has been used): fitzcarraldo
Copying completed
Mon  8 Jan 21:15:48 GMT 2018
********** Backup completed **********
********** Backing up Acer Aspire XC600 users' home directories **********
Tue  9 Jan 01:57:54 GMT 2018
User who shutdown PC (may not be this user if Switch User has been used): claudia
Copying completed
Tue  9 Jan 01:58:10 GMT 2018
********** Backup completed **********
claudia@aspirexc600:~$

Filed under Linux, Lubuntu Linux Tagged with ,

About Fitzcarraldo
A Linux user with an interest in all things technical.

4 Responses to Backing up users’ home directories in a Linux installation that uses systemd

  1. reloaded says:
    January 16, 2018 at 18:10

    Being hopeless at programming I steal these bash scrips to aid learning and for possible future use however I urge you to take a long look at Syncthing which will backup or synchronise files across several computers in real time. Until recently I used it with two laptops and a server to keep a core data set on all three. https://syncthing.net/ it is well documented and still developing.

    Reply

  2. Pingback: Getting the lock screen to work properly when resuming from Suspend-to-RAM with multiple sessions in Lubuntu 17.10 | Fitzcarraldo's Blog

  3. Fitzcarraldo says:
    April 29, 2019 at 11:19

    If you have trouble with the cp command in the script due to buffer overflow when copying files to an external USB HDD, you can use rsync instead of cp, as you can limit the throughput like so:

    #
rsync --recursive --times --perms --links --protect-args --bwlimit=22500 /home/ /media/usbhdd/Lubuntu_home_folders_backup 2>> /home/fitzcarraldo/backup.log
#

    If you want to exclude certain sub-directories from being copied, let’s say the Web browser’s cache directory, you can add options to the rsync command like so:

    #
rsync --recursive --times --perms --links --protect-args --exclude '/*/.cache/mozilla' --exclude '/*/.cache/google-chrome' --exclude '/*/.cache/chromium' --delete-excluded --bwlimit=22500 /home/ /media/usbhdd/Lubuntu_home_folders_backup 2>> /home/fitzcarraldo/backup.log
#

    See man rsync for details:

    --bwlimit=RATE
       This option allows you to specify the maximum transfer rate for the data sent over the socket,  specified  in
       units  per  second.  The RATE value can be suffixed with a string to indicate a size multiplier, and may be a
       fractional value (e.g.  "--bwlimit=1.5m").  If no suffix is specified, the value will be  assumed  to  be  in
       units  of  1024 bytes (as if "K" or "KiB" had been appended).  See the --max-size option for a description of
       all the available suffixes. A value of zero specifies no limit.

       For backward-compatibility reasons, the rate limit will be rounded to  the  nearest  KiB  unit,  so  no  rate
       smaller than 1024 bytes per second is possible.

       Rsync  writes  data  over the socket in blocks, and this option both limits the size of the blocks that rsync
       writes, and tries to keep the average transfer rate at the requested limit.  Some "burstiness"  may  be  seen
       where rsync writes out a block of data and then sleeps to bring the average rate into compliance.

       Due  to  the  internal buffering of data, the --progress option may not be an accurate reflection on how fast
       the data is being sent.  This is because some files can show up as  being  rapidly  sent  when  the  data  is
       quickly  buffered,  while other can show up as very slow when the flushing of the output buffer occurs.  This
       may be fixed in a future version.
    Reply

  4. Pingback: Moving from Lubuntu 18.04 to 20.10 | Fitzcarraldo's Blog

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.