Proxmox Crashplan
Last revised 24 July 2016.
Contents
Introduction
One of the most valuable plug-ins I was running under FreeNAS was Crashplan which was responsible for ensuring I had an additional off site backup of my primary data sets. Under FreeNAS you have to run Crashplan in the officially unsupported ‘headless mode’ which until recently has been reliable. Recent changes to FreeNAS and Crashplan have contributed to the point where the stability and reliability of this critical service no longer met my requirements and was a major contributor to my decision to move away from FreeNAS jails altogether.
This guide will go through the steps to create a Linux virtual machine in Proxmox and install & configure Crashplan for reliable operation. Although we will be running a Linux desktop to enable us to interact with the Crashplan client, we will keep the resource requirements down to a reasonable minimum to reduce the burden on the host machine.
I’ll also include how to adopt an existing backup from Crashplan to prevent having to re-upload all your data.
This guide will build on my Proxmox 4.2 baseline installation which ensures the underlying host configuration is configured correctly to support reliable usage including UPS initiated shut-downs and email reporting etc.
Create Crashplan VM
Moving away from the headless setup required a desktop to support the Crashplan client. I wanted to keep resource usage down and therefore required a lightweight Linux desktop. I settled on CrunchBangPlusPlus which is an open-source, lightweight Linux desktop built upon Debian Jessie and configured out of the box with Openbox window manager and a number of other useful additions. I looked at more extreme pre-configured desktops such as TinyCore, however the additional effort of configuring TinyCore wasn’t worth the additional effort in terms of additional resource saving.
Download CrunchBangPlusPlus iso
Download the CrunchBangPlusPlus ISO image from here.
Upload ISO to Proxmox
Before we can create a VM based upon CrunchBangPlusPlus, we need to upload the ISO file to Proxmox.
- Navigate to StorageView > DataCenter > Proxmox > Local (Proxmox), select Content and then click Upload.
- Select the CrunchBangPlusPlus.ISO file you just downloaded and click Upload to transfer it to the Proxmox host.
Create the VM
- Navigate to ServerView > DataCenter > Proxmox
- Click ‘Create VM’ in the topmost menu bar.
Setup VM
- Node = proxmox
- VM ID = 103 (your ID will likely be different as system creates sequentially from 100)
- Name = crashplan
- Resource pool = blank
- Click Next
Setup the VM type
- Select Linux 4.X/3.X/2.6 Kernel
- Click Next
Setup the VM image source
- Select Use CD/DVD disc image (iso)
- Storage = local
- ISO image = Highlight the the CrunchBangPlusPlus.iso you just downloaded.
- Click Next
Setup the VM hard disk type and capacity
- Bus = VirtIO, Device = 0
- Storage = local-lvm
- Disk Size (GB) = 8
- Cache - Default (No cache)
- No Backup = [ ]
- Discard = [ ]
- IO thread = [ ]
- Click Next
Setup the VM CPU
You can use as many cores as your machine offers, because Crashplan has low CPU overheads because I don’t compress or dedupe my uploads, I’ve allocated 2 cores which is plenty. If you make use of compression, it may be worth increasing the core count.
- Sockets = 1
- Cores = 2
- Enable NUMA = [ ]
- Type = Default (kvm64)
- Total Cores = 2
- Click Next
Setup the VM memory configuration
Crashplan is fairly memory intensive, even without dedupe enabled. I’ve allocated 4GB which is enough to manage my 2TB archive. Monitor the Proxmox dashboard summary to see if there is a need to increase the memory made available to the VM.
- Used Fixed Size Memory
- Memory (MB) = 4096
- Click Next
Setup the VM network configuration
As per my baseline installation guide, I use a Intel x520 10gig network card for all of my VM hosts to communicate over and hence why I select vmbr1 here. If you are making use of a single connection, this should be set to vmbr0.
- Bridged Mode
- VLAN Tag = No VLAN
- Bridge = vmbr1
- Firewall = [ ]
- Model = VirtIO (paravirtualized)
- MAC Address = auto
- Rate limit (MB/s) = unlimited
- Multiqueues = blank
- Disconnect = [ ]
- Click Next
Confirm the VM configuration is correct
Confirm details and Click Finish to proceed.
After a few seconds your virtual machine should have been created. Navigate to ServerView > proxmox > 103 (crashplan) to see a Summary of your new VM.
Install CrunchBangPlusPlus
In the VM header, click Start to start the VM.
You can then click on Console and you should be presented with another window with the Debian installer boot menu.
Install Debian
CrunchBangPlusPlus is built on top of Debian, we’ll run through the baseline install of Debian.
Select Graphical Install
Select a language
Select your preferred installation language and click Continue
Select your location
Select your location and click Continue
Select keymap
Select your keymap and click Continue
Some files will now be copied to your boot disk and hardware will be detected and configured…
Set hostname
Enter a hostname, in this case I used ‘crashplan’. Click Continue to proceed.
Setup user
Enter full name of the user, in my case I use ‘cp’ as the system user and click Continue
Set username
Select username for the user we just created, again I’ll use ‘cp’. Click Continue to proceed.
Set password
Select a strong password and enter it twice. Click Continue to proceed.
Set timezone
Set desired timezone and click Continue to proceed.
Partition disks
The system disk will now be detected and we can commence with partitioning. Given this is a simple Linux install only running the Crashplan service, there’s no need for anything fancy. Given we only allocated 8GB to this disk we can go ahead and use all of the disk. A simple guided install will take care of creating the primary partition and a small swap space. if you start to notice swap being used it is likely worth increasing the amount of system ram to reduce this.
Select Guided - use entire disk and click Continue to proceed
Select disk
Our system only has one disk so go ahead and select ‘vda’ and click Continue to proceed.
Select partitioning scheme
We’ll go ahead and install all the files into a single partition. There’s no need to complicate this setup given its limited use. Select ‘All files in one partition’ and click Continue to proceed.
Overview confirmation
Check you are happy with the partitioning information and if so, click Continue to proceed. You can see the system has provided a small 401.6MB swap space and the rest of the disk space is assigned to the primary partition.
Highlight ‘Finish partitioning and write changes to disk’ and click Continue to proceed.
A final confirmation before the disk contents are destroyed, select ‘Yes’ and click Continue to proceed.
Partitioning will now proceed… After a short wait we can move n to configure the package manager.
Configure the package manager
We’ll need to keep our system updated and install some programs so configure the package manager by highlighting an area and click Continue to proceed.
and again, select the Debian archive mirror close to you.
Click Continue to proceed.
Install grub
In order for our VM to boot, we need to install a grub boot loader on the primary system disk. As this is the only OS, we can go ahead and install it to the Master Boot Record. Select ‘Yes’ and click Continue to proceed.
Highlight our only drive, ‘/dev/vda’ and click Continue to proceed to write the grub loader to our drive.
Install complete!
Congratulations, you now have a basic working Linux VM. Click Continue to reboot, CrunchBangPlusPlus should load through a login screen where you can enter your username and password to login to the desktop environment.
Configure CrunchBangPlusPlus
At this stage we have a functioning Linux desktop. Feel free to play around with it to familiarise yourself with it. You can see in the system info display on the background CrunchBangPlusPlus impressively runs in less than 200MB of RAM and takes 2.2GB of disk space. It is possible ot shrink it further but its so lightweight I didn’t think it was worth the effort.
Close the cpbb-welcome window, or enter q to quit it. The configurator offers some neat additional configuration steps but largely they are superfluous for our Crashplan needs. The important steps are already captured in this guide.
Prepare CBPP for Crashplan
Right click on the CBPP desktop and open a terminal prompt. We’ll do the majority of our configuration in the terminal.
Lets start off by updating our Debian base install with the following commands.
sudo -apt-get update
sudo apt-get dist-upgrade
Configure NTP
Now lets configure our server so it pulls the correct time from our local NTP server, in my case my pfSense router.
In the terminal window, enter sudo nano /etc/ntp.conf and edit the section which details the NTP servers. Comment out the default servers by adding a hash in front of them and add your local source, in my case this is my pfSense router which is at IP address 192.168.20.1.
# pool.ntp.org maps to about 1000 low-stratum NTP servers. Your server will
# pick a different set every time it starts up. Please consider joining the
# pool:
# server 0.debian.pool.ntp.org iburst dynamic
# server 1.debian.pool.ntp.org iburst dynamic
# server 2.debian.pool.ntp.org iburst dynamic
# server 3.debian.pool.ntp.org iburst dynamic
server 192.168.20.1 iburst dynamic
Start the NTP service with sudo /etc/init.d/ntp start and verify it started correctly with sudo /etc/init.d/ntp status.
Verify your server is set correctly by entering ntpq -p, for example
$ ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
*192.168.20.1 198.60.22.240 2 u 10 256 377 0.213 1.114 0.127
Configure automated shutdown (UPS)
The Promxmox host needs the ability to shutdown virtual machines correctly and safely. This is accomplished via ACPI commands. Without ACPI being installed and configured Proxmox cant initiate a safe shutdown of the VM, either automatically from a UPS event, or from user interacting with the Proxmox interface. Without ACPI correctly configured you will see errors in the VM task list such as
TASK ERROR: command '/bin/nc6 -l -p 5900 -w 10 -e '/usr/sbin/qm vncproxy 103 2>/dev/null'' failed: exit code 1
Install prerequisite software
Enter the command sudo apt-get install acpi-support to install ACPI support.
Edit shutdown script
We will edit the power button event to initiate the shutdown command. Enter sudo nano /etc/acpi/events/powerbtn-acpi-support and edit the file so it contains.
event=button[ /]power
action=/sbin/shutdown -h -P now
then start the service by entering
$ sudo /etc/init.d/acpid start
[ ok ] Starting acpid (via systemctl): acpid.service.
you can check it started correctly by
$ sudo /etc/init.d/acpid status
● acpid.service - ACPI event daemon
Loaded: loaded (/lib/systemd/system/acpid.service; disabled)
Active: active (running) since Sat 2016-07-23 18:01:55 EDT; 50s ago
Main PID: 443 (acpid)
CGroup: /system.slice/acpid.service
└─443 /usr/sbin/acpid
Jul 23 18:01:55 crashplan acpid[443]: starting up with netlink and the input layer
Jul 23 18:01:55 crashplan acpid[443]: 27 rules loaded
Jul 23 18:01:55 crashplan acpid[443]: waiting for events: event logging is off
Verify shutdown script
You should try to initiate a VM client shutdown from the Proxmox user interface and verify everything works correctly. Hopefully the VM will shutdown promptly and cleanly reporting ‘OK’ in the event log rather than the previous ‘Error, VM quit/powerdown fail’.
Due to the importance of proper shutdown procedures in time of power outage, I recommend verifying the full UPS shutdown stack.
Configure automated share mounting (CIFS)
I originally mounted my shares via fstab however this caused me some issues. In my experience, it appears fstab runs before the network is completely up and ready to accept connection attempts and therefore some folders may not mount correctly. Assuming you have created the root folder for the mount to attach to, Crashplan views this as an empty folder and politely removes all the associated data from the cloud assuming you have deleted it locally!
I now use autofs to mount drives which has been a lot more reliable.
I have a number of FreeNAS folders shared which I back up daily to Crashplan’s cloud. I’ll focus on one specific folder called ‘Data’ in this guide but it should be simple to extrapolate to all the folders you need to backup.
Install prerequisites
Install autofs, CIFS and smbclient packages with the following command.
sudo apt-get install autofs smbclient cifs-utils
I like to verify SMB by running smbclient -L <hostname> where hostname is the name of your server hosting the shares, for example.
$ smbclient -L freenas
Domain=[WORKGROUP] OS=[Windows 6.1] Server=[Samba 4.3.6-GIT-UNKNOWN]
Sharename Type Comment
--------- ---- -------
Data Disk
IPC$ IPC IPC Service (FreeNAS Server)
Domain=[WORKGROUP] OS=[Windows 6.1] Server=[Samba 4.3.6-GIT-UNKNOWN]
Server Comment
--------- -------
FREENAS FreeNAS Server
NGUVUWIN8 Win8
Workgroup Master
--------- -------
WORKGROUP FREENAS
Configure autofs
Edit autofs master configuration file by entering sudo nano /etc/auto.master
Add the following to the bottom of the file, /mnt /etc/auto.smb-shares
so the file looks like this
# Sample auto.master file
# This is an automounter map and it has the following format
# key [ -mount-options-separated-by-comma ] location
# For details of the format look at autofs(5).
#
#/misc /etc/auto.misc
#
# NOTE: mounts done from a hosts map will be mounted with the
# "nosuid" and "nodev" options unless the "suid" and "dev"
# options are explicitly given.
#
#/net -hosts
#
# Include /etc/auto.master.d/*.autofs
#
+dir:/etc/auto.master.d
#
# Include central master map if it can be found using
# nsswitch sources.
#
# Note that if there are entries for /net or /misc (as
# above) in the included master map any keys that are the
# same will not be seen as the first read key seen takes
# precedence.
#
+auto.master
/mnt /etc/auto.smb-shares
Credentials file
To avoid storing the username and passwords of the shared folders in the configuration file, we’ll create a credential file by entering sudo nano ~/.smbcredentials
this file should be populated only with the username and password used for accessing the shared folder, no spaces either side of the ‘=’ sign. For example
username=nguvu
password=mystrongpassword
Reduce the access rights to this file by entering sudo chmod 600 ~/.smbcredentials.
You will need a credential file for each share you access which doesn’t share the same username and password.
Configure shares
We’ll now create the configuration file that specifies the folders we want autofs to mount for us by entering sudo nano /etc/auto.smb-shares
Enter the following line into this file
Data -fstype=cifs,ro,credentials=/home/cp/..smbcredentials ://freenas/Data
Data is the name of the folder the line refers to.
fstype is the share filesystem type, in our case, CIFS
ro mounts the share as read-only
credentials specifies the file containing the username and password to use for authorisation.
://freenas/Data is the network location of the share
It should be simple enough to extrapolate form that line into the other shares you would like to access.
Start and verify autofs
Start autofs by entering sudo /etc/init.d/autofs start and verify its status with sudo /etc/init.d/autofs status
$ sudo /etc/init.d/autofs start
[ ok ] Starting autofs (via systemctl): autofs.service.
$ sudo /etc/init.d/autofs status
● autofs.service - LSB: Automounts filesystems on demand
Loaded: loaded (/etc/init.d/autofs)
Active: active (running) since Sun 2016-07-24 00:04:54 EDT; 31min ago
CGroup: /system.slice/autofs.service
└─1998 /usr/sbin/automount --pid-file /var/run/autofs.pid
Jul 24 00:04:54 crashplan autofs[1993]: Starting automount....
Now ensure we can access the directory we setup,
$ cd /mnt/Data
$ ls -l
total 0
drwxrwxrwx 283 root root 0 Nov 15 2015 apps
drwxrwxrwx 2 root root 0 Jul 17 19:43 FreeNAS
drwxrwxrwx 7 root root 0 Oct 18 2014 Linux
drwxrwxrwx 122 root root 0 Jul 17 19:41 Macintosh Apps
drwxrwxrwx 4 root root 0 Jul 17 19:45 pfSense
drwxrwxrwx 3 2001 root 0 Jul 17 19:33 Proxmox
The /mnt/Data folder will unmount itself after a period of activity but will remount whenever it is required by a user or an application.
Debugging
If you need to have further visibility into what’s going on with autofs, you can turn verbose logging on by editing the /etc/default/autofs file and setting the LOGGING line to VERBOSE OR DEBUG.
Install Crashplan
From Proxmox, Navigate to our Crashplan VM via ServerView > DataCener > Proxmox > 103 (crashplan)
Click on Console to start the VM, enter the username and password to login to the desktop.
Download Crashplan installer
Right click on the desktop and open a Web Browser.
Navigate to www.crashplan.com and go go downloads.
Click on ‘Download Crashplan’ which will download the Linux tar file to your downloads folder.
Right click on the desktop and open Terminal. Follow these commands to unpack the archive and commence installation.
$ cd ~/downloads
$ ls -l
-rw-r--r-- 1 cp cp 47470950 Jul 24 07:43 CrashPlan_4.7.0_Linux.tgz
Unzip the tgz file with tar -xzvf <filename>, for example
$ tar -xvzf CrashPlan_4.7.0_Linux.tgz
crashplan-install/install.sh
crashplan-install/uninstall.sh
crashplan-install/
crashplan-install/scripts/
crashplan-install/CrashPlan_4.7.0.cpi
crashplan-install/EULA.txt
crashplan-install/INSTALL
crashplan-install/README
crashplan-install/install.defaults
crashplan-install/scripts/CrashPlan.desktop
crashplan-install/scripts/CrashPlanDesktop
crashplan-install/scripts/CrashPlanEngine
crashplan-install/scripts/crashplan
crashplan-install/scripts/run.conf
cd into the extracted file folder and execute the install script.
$ cd crashplan-install
$ ./install.sh
Installation will now commence. The default options are ok and summarised below for completeness.
Would you like to switch users and install as root? (y/n) Y
Read the EULA…
Do you accept and agree to be bound by the EULA? (yes/no) yes
What parent directory do you wish to install CrashPlan into? [/usr/local] return to accept defaults
/usr/local/crashplan does not exist. Create /usr/local/crashplan? (y/n) y
What directory do you wish to link the CrashPlan executable to? [/usr/local/bin] return to accept defaults
What directory do you wish to store incoming backup data? [/usr/local/var/crashplan] return to accept defaults
/usr/local/var/crashplan does not exist. Create /usr/local/var/crashplan? (y/n) y
What directory contains your SYSV init scripts? [/etc/init.d] return to accept defaults
What directory contains your runlevel init links? [/etc/rc5.d] return to accept defaults
Verify selections
Is this correct? (y/n) y
Crashplan will now install files, configure Java and start the Crashplan engine.
Adopt Crashplan backup
Back in the CBPP desktop, open the file manager and navigate to /usr/local/crashplan/bin. Double click on the CrashplanDesktop icon.
As we are going to deal with adopting an existing Crashplan backup, enter your existing username and password and log into the Crashplan client.
Click on ‘adopt’ at the top of the page and select your archive when presented with the option.
Click on archive to adopt the backup. At this stage you will be logged out and you will have to log back in again. Assuming you followed best practice and set a encryption key to encrypt any data uploaded to Crashplan’s cloud, you’ll be notified that the backup isn’t able to run not knowing this code.
Enter your 448bit passphrase and Click Save. Crashplan will now verify the local files with those in the Crashplan cloud.
Configure Crashplan
One of the most common complaints about Crashplan is slow uploading of larger datasets. This I believe is largely affected by the deduplication of data. Thanks to NetworkRockstar it became widely understood what the impact of this deduplication process was and also how to disable it. Crashplan address the topic here themselves and it is worth reading before you consider disabling it. I’ve been running without deduplication and compression for 18 months now without any adverse effects.
To disable deduplication, nano /usr/local/crashplan/conf/my.service.xml and edit the dataDeDupAutoMaxFileSizeForWan setting to 1 as follows
<dataDeDupAutoMaxFileSizeForWan>1</dataDeDupAutoMaxFileSizeForWan>
Restart Crashplan and you should see improved upload speeds. My most recent backup ran at around 100mbps (my upload cap is 150mbps) for a significant proportion of the 2TB uploaded.