HOWTO Encrypted Backup System Using Bacula and GnuPG
From Gentoo Linux Wiki
Contents |
[edit] General Overview
[edit] Why do it this way?
On January 4, 2007 Bacula project released version 2.0.0 which includes data encryption using public/private RSA certificates and AES128 encryption. Encrypting data with GnuPG will still work without any changes to the process.
[edit] Intended Audience
- Network Administrators who need to have their backups secured via encryption
- Existing users of Bacula who need to have backup volumes encrypted
[edit] Why use this setup
- Works now and has been tested
- Frees up clients' resources by moving encryption process to the server
- Uses established and tested mechanisms to secure data
- Writes encrypted data to a single file.
- Provides extra layer of security by having Private Key protected by a password
[edit] Why NOT use this setup
- Extra overhead on the server to complete backup and restore
- Does not use Bacula's native tape-writing mechanisms to store data
- Use any other tape writing utility
- Transfer file over the network
- Director sees unencrypted files before encryption & deletion
[edit] General Overview of the data flow
The Director will do FULL backup every day for the ease of restoration. Each day it will create a new backup volume.
During the backup process Bacula server (called the Director) requests the files from the client (called File-Daemon or FD for short). During the request Director passes arguments to the FD - Director authentication password, files & folder list, request for compression and MD5/SHA1 sum. The received files being written to the hard drive volume (one large file) and cataloged to the database.
Once the Director is done with all clients, the director calls a script, which will encrypt the backup volume with GnuPG and delete the unencrypted volume. After the encryption is completed, the file will be transferred via FTP or CIFS/SMB for offsite storage. (Optionally it can be written to a tape with a 3rd party utility)
[edit] Hardware Configuration
Any sufficiently fast server will do. The requirements vary greatly between different environments. To give you a starting point, here is one example Dell PowerEdge 2800
- Single Xeon 2.8 GHz
- 1 Gig RAM
- 36 Gig RAID1 array for the /
- 169 Gig RAID5 array for the /backups
- 10/100 switched LAN
This setup backs up and encrypts data for about 50 clients, in my organization it equates to about 40 Gig compressed & encrypted data.
[edit] Software Configuration
[edit] GnuPG
GnuPG is an OpenPGP compliant utility that can use symmetric or asymmetric methods of encrypting data.
[edit] GnuPG Installation
emerge gnupg
[edit] Public / Private Key creation
For a good overview on what GnuPG is all about, please visit our own Gentoo GnuPG Guide You may or may not follow the suggested values for the key in the guide, but this is a safe setup for a backup system
- DSA and ElGamal key pair
- Key Size - 4096
- Public Key Expiration - 5 years (you will still be able to decrypt after 5 years)
- Private Key password or passphrase as complex as you can stand
Please note - you do not need and should NOT store the private key on the backup server. It is a security risk. The best way is to generate the priv/pub key pairs on a secured workstation. Export the public key only as a text file and import public key only into the backup server. (see Gentoo GnuPG Guide guide on how to do export/import)
Store the following items in a safe place
- CD or Flash Drive with Public key, Private key, Passphrase and Revocation Certificate
- Paper Printouts with Public key, Private key, Passphrase and Revocation Certificate
If you loose your private key or passphrase - bye-bye data.
[edit] Bacula
[edit] Server Installation
The latest version of Bacula in the portage is outdated (bacula-1.36.3-r3) where the latest version as of writing is 1.38.11. Below is example on how to install Bacula from source.
- 1. Download latest bacula source from SourceForge.net
- 2 Extract the bacula source
tar -xvzf bacula-x.xx.xx.tar.gz
- 3 Enter the source folder
cd bacula-x.xx.xx
- 4 Run the configure command
./configure \ --prefix=/usr \ --sbindir=/usr/sbin \ --sysconfdir=/etc/bacula \ --with-scriptdir=/etc/bacula \ --enable-smartalloc \ --with-mysql \ --with-working-dir=/var/bacula \ --with-pid-dir=/var/run \ --with-subsys-dir=/var/lock/subsys \ --enable-conio \ --with-openssl \ --enable-largefile \ --enable-wx-console \ --with-python
- 5 Compile and Install
make make install
- 6 Setup Database (assuming you have MySQL installed and working)
/etc/bacula/create_mysql_database /etc/bacula/make_mysql_tables /etc/bacula/grant_mysql_privileges
- 7 Add bacula to the /etc/conf.d/local.start
echo /etc/bacula/bacula start >> /etc/conf.d/local.start
- 8 add bacula to the /etc/conf.d/local.stop
echo /etc/bacula/bacula stop >> /etc/conf.d/local.stop
[edit] Server (Director) Configuration (bacula-dir.conf)
This is the most confusing part of the whole process. In the following example, just substitute $VARIABLE with your own value.
| File: /etc/bacula/bacula-dir.conf |
################### DEFINE DIRECTOR #############################
Director
{
Name = backup-dir #or whatever name you want
Description = "Bacula Director - $MY_LOCATION" # "Bacula Director - Main Office"
DIRport = 9101
QueryFile = "/etc/bacula/query.sql"
WorkingDirectory = "/var/bacula"
PidDirectory = "/var/run"
Maximum Concurrent Jobs = 2
FDConnectTimeout = 1min
SDConnectTimeout = 1min
Password = "$CONSOLE_PASSWORD" # "gsdfgfdsg44"
Messages = Daemon
}
################### DEFINE CLIENTS ##############################
Client {
Name = $SOME_IMPORTANT_CLIENT-fd # accountant-fd
Address = $FQDN_OR_IP # accountant.mydomain.local
FDPort = 9102
Catalog = GiantCatalog
Password = "$CLIENT_ACCESS_PASSWORD" # "strong_password"
File Retention = 6 days
Job Retention = 6 days
AutoPrune = yes
Maximum Concurrent Jobs = 2
}
Client {
Name = $LAST_CLIENT-fd # mainserver-fd
Address = $FQDN_OR_IP # mainserver.mydomain.local
FDPort = 9102
Catalog = GiantCatalog
Password = "$LAST_CLIENT_ACCESS_PASSWORD" # "strong_password2"
File Retention = 6 days
Job Retention = 6 days
AutoPrune = yes
Maximum Concurrent Jobs = 2
}
Client {
Name = $BACKUP_SERVER-fd # the bacula server itself backup-fd
Address = $FQDN_OR_IP # backup.mydomain.lan
FDPort = 9102
Catalog = GiantCatalog
Password = "$BACKUP_SERVER_ACCESS_PASSWORD" # "strong_password2"
File Retention = 6 days
Job Retention = 6 days
AutoPrune = yes
Maximum Concurrent Jobs = 2
}
############### DEFINE DEFAULT JOB PARAMETERS #####################
JobDefs
{
Name = "DefaultBackupJob"
Type = Backup
Level = Full
Client = backup-fd
Schedule = "WeeklyCycle"
Storage = File
Pool = Default
Messages = Standard
Priority = 10
Max Start Delay = 18h
}
####################### DEFINE JOBS ############################
Job
{
Name = "$SOME_IMPORTANT_CLIENT_JOB" # "Accountant Backup Job"
Client = $SOME_IMPORTANT_CLIENT-fd # accountant-fd
JobDefs = "DefaultBackupJob"
FileSet = "Generic User Fileset" #it can be reused for many clients
Write Bootstrap = "/backups/bootstraps/$SOME_IMPORTANT_CLIENT.bsr"
Maximum Concurrent Jobs = 2
}
#Backup the catalog database (after the nightly save)
Job {
Name = "BackupCatalog"
JobDefs = "DefaultBackupJob"
Client = BACKUP_SERVER-fd
FileSet="Catalog"
# This creates an ASCII copy of the catalog
RunBeforeJob = "/etc/bacula/make_catalog_backup bacula bacula"
# This deletes the copy of the catalog
RunAfterJob = "/etc/bacula/delete_catalog_backup"
Write Bootstrap = "/backups/bootstraps/BackupCatalog.bsr"
Priority = 10
}
Job
{
Name = "$LAST_CLIENT_JOB" # "Mainserver Backup Job"
Client = $LAST_CLIENT-fd # mainserver-fd
JobDefs = "DefaultBackupJob"
FileSet = "Server Fileset" #we can define a separate FileSet
Write Bootstrap = "/backups/bootstraps/$LAST_CLIENT.bsr"
Maximum Concurrent Jobs = 2
RunAfterJob = "/backups/encrypted/scripts/EncryptVolume.sh"
## NOTE - we will run the script EncryptVolume.sh after this job completes. ##
## It will encrypt volume, delete unencrypted volume and transfer it via FTP ##
}
################## Define which files to backup #####################
FileSet
{
Name = "Generic User Fileset"
Enable VSS = yes
Include
{
Options
{
signature = MD5 #Create MD5 Signature
compression=GZIP5 #Compress Incoming Data on the client
Exclude = yes
IgnoreCase = yes
# Exclude Mozilla-based programs' file caches
WildDir = "[A-Z]:/Documents and Settings/*/Application Data/*/Profiles/*/*/Cache"
WildDir = "[A-Z]:/Documents and Settings/*/Application Data/*/Profiles/*/*/Cache.Trash"
# Exclude directories full of lots and lots of useless little files
WildDir = "[A-Z]:/Documents and Settings/*/Cookies"
WildDir = "[A-Z]:/Documents and Settings/*/Recent"
WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/History"
WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/Temp"
WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/Temporary Internet Files"
# These are always open and unable to be backed up
WildFile = "[A-Z]:/Documents and Settings/All Users/Application Data/Microsoft/Network/Downloader/qmgr[01].dat"
# Temporary directories & files
WildDir = "[A-Z]:/WINNT/Temp"
WildDir = "[A-Z]:/temp"
WildFile = "*.tmp"
# Recycle bins
WildDir = "[A-Z]:/RECYCLER"
# Swap files
WildFile = "[A-Z]:/pagefile.sys"
}
File = "c:/Documents and Settings" #Define Folders to back up
File = "d:/SomePath/To Folder/Or/File.txt"
}
Exclude
{
File = "c:/test" #Define Folders to exclude
}
}
FileSet
{
Name = "Server Fileset"
Enable VSS = yes
Include
{
Options
{
signature = MD5 #Create MD5 Signature
compression=GZIP5 #Compress Incoming Data on the client
Exclude = yes
IgnoreCase = yes
# Swap files
WildFile = "[A-Z]:/pagefile.sys"
}
File = "d:/" #Back Everything for this SPECIAL_USER
}
Exclude
{
File = "c:/test" #Define Folders to exclude
}
}
################### DEFINE SCHEDULE ################################
Schedule {
Name = "WeeklyCycle"
Run = Level=Full Pool=MondayPool Monday at 12:05 #PM That is
Run = Level=Full Pool=TuesdayPool Tuesday at 02:12 #AM that is
Run = Level=Full Pool=WednesdayPool Wednesday at 12:05
Run = Level=Full Pool=ThursdayPool Thursday at 12:05
Run = Level=Full Pool=FridayPool Friday at 12:05
Run = Level=Full Pool=SaturdayPool Saturday at 16:00
Run = Level=Full Pool=SundayPool Sunday at 16:00
}
################# DEFINE STORAGE (file, tape, CDR..) ###################
Storage {
Name = File
Address = FQDN_OR_IP_OF_DIRECTOR
SDPort = 9103
Password = "$STORAGEPASS"
Device = FileStorage
Media Type = File
}
############################ DEFINE CATALOG #############################
Catalog {
Name = GiantCatalog
dbname = bacula; user = bacula; password = ""
}
########################## DEFINE MESSAGES #############################
Messages {
Name = Standard
console = all, !skipped, !saved
append = "/var/bacula/log" = all, !skipped
}
Messages {
Name = Daemon
console = all, !skipped, !saved
append = "/var/bacula/log" = all, !skipped
}
####################### DEFINE POOLS ################################
Pool {
Name = MondayPool
Pool Type = Backup
AutoPrune = yes # Prune expired volumes
Volume Retention = 6 days
Accept Any Volume = yes # write on any volume in the pool
Volume Use Duration = 6 days
Recycle = yes
RecycleOldestVolume = yes
LabelFormat = "BackupMondayVolume"
}
Pool {
Recycle = yes
Name = TuesdayPool
Pool Type = Backup
AutoPrune = yes
Volume Retention = 6 days
Accept Any Volume = yes
Volume Use Duration = 6 days
Recycle = yes
RecycleOldestVolume = yes
LabelFormat= "BackupTuesdayVolume"
}
Pool {
Recycle = yes
Name = WednesdayPool
Pool Type = Backup
AutoPrune = yes
Volume Retention = 6 days
Accept Any Volume = yes
Volume Use Duration = 6 days
Recycle = yes
RecycleOldestVolume = yes
LabelFormat="BackupWednesdayVolume"
}
Pool {
Recycle = yes
Name = ThursdayPool
Pool Type = Backup
AutoPrune = yes
Volume Retention = 6 days
Accept Any Volume = yes
Volume Use Duration = 6 days
Recycle = yes
RecycleOldestVolume = yes
LabelFormat="BackupThursdayVolume"
}
Pool {
Recycle = yes
Name = FridayPool
Pool Type = Backup
AutoPrune = yes
Volume Retention = 6 days
Accept Any Volume = yes
Volume Use Duration = 6 days
Recycle = yes
RecycleOldestVolume = yes
LabelFormat="BackupFridayVolume"
}
Pool {
Recycle = yes
Name = SaturdayPool
Pool Type = Backup
AutoPrune = yes
Volume Retention = 6 days
Accept Any Volume = yes
Volume Use Duration = 6 days
Recycle = yes
RecycleOldestVolume = yes
LabelFormat="BackupSaturdayVolume"
}
Pool {
Recycle = yes
Name = SundayPool
Pool Type = Backup
AutoPrune = yes
Volume Retention = 6 days
Accept Any Volume = yes
Volume Use Duration = 6 days
Recycle = yes
RecycleOldestVolume = yes
LabelFormat="BackupSundayVolume"
}
# Default pool definition
Pool {
Name = Default
Pool Type = Backup
}
# Restricted console used by tray-monitor to get the status of the director
Console {
Name = backup-mon
Password = "$RESTRICTED_CONSOLE_PASSWORD"
CommandACL = status, .status
}
|
[edit] Define Storage Daemon (SD)
| File: /etc/bacula/bacula-sd.conf |
Storage { # definition of myself
Name = backup-sd
SDPort = 9103
WorkingDirectory = "/var/bacula"
Pid Directory = "/var/run"
Maximum Concurrent Jobs = 20
}
# List Directors who are permitted to contact Storage daemon
Director {
Name = backup-dir
Password = "$STORAGEPASS"
}
#
# Restricted Director, used by tray-monitor to get the
# status of the storage daemon
#
Director {
Name = backup-mon
Password = "$RESTRICTED_DIR_PASS"
Monitor = yes
}
Device {
Name = FileStorage
Media Type = File
Archive Device = /backups
LabelMedia = yes; # lets Bacula label unlabeled media
Random Access = Yes;
AutomaticMount = yes; # when device opened, read it
RemovableMedia = no;
AlwaysOpen = no;
}
Messages {
Name = Standard
director = backup-dir = all
}
|
[edit] Define Console
| File: /etc/bacula/bconsole.conf |
Director {
Name = backup-dir #mainofficebackup-dir
DIRport = 9101
address = $FQDN_OR_IP
Password = "$DIRECTOR_CONSOLE_PASSWORD"
}
|
[edit] Create the backup script
This is the script that will encrypt the backup volume and upload it to a different server This script is to be executed via Bacula's RunAfterJob or Cron. It will encrypt a backup volume with GnuPG using Twofish algorithm. If encryption is to be changed use only encryption with 128-bit BLOCK size. Currently only Twofish, AES, MARS and Serpent have 128bit block.
Ciphers with 64-bit block (3DES, CAST5, BLOWFISH...) are not suitable for encrypting large files. Couple Gig volume is Ok, but as you will approach 64 Gigs, you will increase the risk of leaking decryption information (see http://en.wikipedia.org/wiki/Birthday_attack).
| File: /backups/encrypted/scripts/EncryptVolume.sh |
# Remove Yesterday’s encrypted backup from HDD cd /backups/encrypted/ rm -f *.gpg # encrypt the file with GPG using Public Key. # use only --cipher-algo aes|aes192|aes256|twofish gpg --encrypt --batch -r backup@giantmarkets.com --cipher-algo twofish \ --bzip2-compress-level 6 --output /backups/encrypted/BakupVolume.gpg /backups/Backup* # Remove unencrypted backup volume, we encrypted and saved # encrypted copy in the above step. cd /backups/ rm Backup* # transfer encrypted volume via ftp to the server with tape cd /backups/encrypted/ ftp -ni $FQDN_OR_IP <<SCRIPT user Anonymous backup_job@backup cd $SOME_FOLDER binary mdelete *.gpg mput *.gpg quit |
Please note. If the script works manually, but fails to encrypt automatically (trust errors), copy the .gnupg folder in the home directory for the working user (root for example) into the root directory
cp -r /root/.gnupg/ /
[edit] Win32 Client Installation and Configuration
Download and install the winbacula.x.x.x.exe Edit the bacula-fd.conf to let your director to connect to the client
| File: c:\bacula\bin\bacula-fd.conf |
Director {
Name = backup-dir #Director's name as defined on the server
Password = "$CLIENT_ACCESS_PASSWORD" #same as in client definition on server
}
FileDaemon {
Name = $SOME_IMPORTANT_CLIENT-fd
FDport = 9102
WorkingDirectory = "C:\\Documents and Settings\\All Users\\Application Data\\Bacula\\Work"
Pid Directory = "C:\\Documents and Settings\\All Users\\Application Data\\Bacula\\Work"
Maximum Concurrent Jobs = 2
}
# Send all messages except skipped files back to Director
Messages {
Name = Standard
director = backup-dir = all, !skipped
}
|
(re)Start the Bacula service and you are ready.
[edit] *NIX Client Installation and Configuration
- 1. Download latest bacula source from SourceForge.net
- 2 Extract the bacula source
tar -xvzf bacula-x.xx.xx.tar.gz
- 3 Enter the source folder
cd bacula-x.xx.xx
- 4 Run the configure command
./configure \ --prefix=/usr \ --sbindir=/usr/sbin \ --sysconfdir=/etc/bacula \ --with-scriptdir=/etc/bacula \ --enable-smartalloc \ --with-mysql \ --with-working-dir=/var/bacula \ --with-pid-dir=/var/run \ --with-subsys-dir=/var/lock/subsys \ --enable-conio \ --with-openssl \ --enable-largefile \ --enable-wx-console \ --with-python \ –-enable-client-only
- 5 Compile and Install
make make install
- 6 Add bacula to the /etc/conf.d/local.start
echo /etc/bacula/bacula start >> /etc/conf.d/local.start
- 7 add bacula to the /etc/conf.d/local.stop
echo /etc/bacula/bacula stop >> /etc/conf.d/local.stop
8: Edit the bacula-fd.conf
| File: c:\bacula\bin\bacula-fd.conf |
Director {
Name = backup-dir #Director's name as defined on the server
Password = "$LAST_CLIENT_ACCESS_PASSWORD" #same as in client definition on server
}
FileDaemon {
Name = $LAST_CLIENT-fd
FDport = 9102
WorkingDirectory = /bacula/working
Pid Directory = /bacula/working
}
# Send all messages except skipped files back to Director
Messages {
Name = Standard
director = backup-dir = all, !skipped
}
|
(re)Start the Bacula service and you are ready.
[edit] Restoration Process
1. Decrypt the file
gpg BackupVolume.gpg
(Assuming that gpg has your private key it will ask for the password)
2. Move the decrypted file to the bacula server (/backups directory in this example)
3a. If Bacula database is intact Follow the restoration procedures in the Bacula handbook http://bacula.org/dev-manual/Restore_Command.html
OR
3b. If Bacula database is not intact (new server for example) Use the bscan utility to recreate the database from your volume.
cd /etc/bacula/ bscan -V RestoreVolume -v -s -m -c bacula-sd.conf FileStorage
and then follow the restoration procedures in the Bacula handbook http://bacula.org/dev-manual/Restore_Command.html
