nguvu

Proxmox Crashplan

Last revised 24 July 2016.

Contents

Introduction

One of the most valuable plug-ins I was running under FreeNAS was Crashplan which was responsible for ensuring I had an additional off site backup of my primary data sets. Under FreeNAS you have to run Crashplan in the officially unsupported ‘headless mode’ which until recently has been reliable. Recent changes to FreeNAS and Crashplan have contributed to the point where the stability and reliability of this critical service no longer met my requirements and was a major contributor to my decision to move away from FreeNAS jails altogether.

This guide will go through the steps to create a Linux virtual machine in Proxmox and install & configure Crashplan for reliable operation. Although we will be running a Linux desktop to enable us to interact with the Crashplan client, we will keep the resource requirements down to a reasonable minimum to reduce the burden on the host machine.

I’ll also include how to adopt an existing backup from Crashplan to prevent having to re-upload all your data.

This guide will build on my Proxmox 4.2 baseline installation which ensures the underlying host configuration is configured correctly to support reliable usage including UPS initiated shut-downs and email reporting etc.

Create Crashplan VM

Moving away from the headless setup required a desktop to support the Crashplan client. I wanted to keep resource usage down and therefore required a lightweight Linux desktop. I settled on CrunchBangPlusPlus which is an open-source, lightweight Linux desktop built upon Debian Jessie and configured out of the box with Openbox window manager and a number of other useful additions. I looked at more extreme pre-configured desktops such as TinyCore, however the additional effort of configuring TinyCore wasn’t worth the additional effort in terms of additional resource saving.

Download CrunchBangPlusPlus iso

Download the CrunchBangPlusPlus ISO image from here.

Upload ISO to Proxmox

Before we can create a VM based upon CrunchBangPlusPlus, we need to upload the ISO file to Proxmox.

Create the VM

Setup VM

Create: Virtual Machine, General

Setup the VM type

Create: Virtual Machine, OS

Setup the VM image source

Create: Virtual Machine, CD/DVD

Setup the VM hard disk type and capacity

Create: Virtual Machine, Hard Disk

Setup the VM CPU

Create: Virtual Machine, CPU

You can use as many cores as your machine offers, because Crashplan has low CPU overheads because I don’t compress or dedupe my uploads, I’ve allocated 2 cores which is plenty. If you make use of compression, it may be worth increasing the core count.

Setup the VM memory configuration

Crashplan is fairly memory intensive, even without dedupe enabled. I’ve allocated 4GB which is enough to manage my 2TB archive. Monitor the Proxmox dashboard summary to see if there is a need to increase the memory made available to the VM.

Create: Virtual Machine, Memory

Setup the VM network configuration

As per my baseline installation guide, I use a Intel x520 10gig network card for all of my VM hosts to communicate over and hence why I select vmbr1 here. If you are making use of a single connection, this should be set to vmbr0.

Create: Virtual Machine, Network

Confirm the VM configuration is correct

Create: Virtual Machine, Confirm

Confirm details and Click Finish to proceed.

Create: Virtual Machine, Done

After a few seconds your virtual machine should have been created. Navigate to ServerView > proxmox > 103 (crashplan) to see a Summary of your new VM.

Install CrunchBangPlusPlus

In the VM header, click Start to start the VM.
You can then click on Console and you should be presented with another window with the Debian installer boot menu.

Install Debian

CrunchBangPlusPlus is built on top of Debian, we’ll run through the baseline install of Debian.

Install CBPP, Install

Select Graphical Install

Select a language

Install CBPP, Select installation language

Select your preferred installation language and click Continue

Select your location

Install CBPP, Select locale

Select your location and click Continue

Select keymap

Install CBPP, Select keymap

Select your keymap and click Continue

Some files will now be copied to your boot disk and hardware will be detected and configured…

Set hostname

Install CBPP, Set hostname

Enter a hostname, in this case I used ‘crashplan’. Click Continue to proceed.

Setup user

Install CBPP, Set user name

Enter full name of the user, in my case I use ‘cp’ as the system user and click Continue

Set username

Install CBPP, Set username

Select username for the user we just created, again I’ll use ‘cp’. Click Continue to proceed.

Set password

Install CBPP, Set password

Select a strong password and enter it twice. Click Continue to proceed.

Set timezone

Install CBPP, Set timezone

Set desired timezone and click Continue to proceed.

Partition disks

The system disk will now be detected and we can commence with partitioning. Given this is a simple Linux install only running the Crashplan service, there’s no need for anything fancy. Given we only allocated 8GB to this disk we can go ahead and use all of the disk. A simple guided install will take care of creating the primary partition and a small swap space. if you start to notice swap being used it is likely worth increasing the amount of system ram to reduce this.

Install CBPP, Set partitioning method

Select Guided - use entire disk and click Continue to proceed

Select disk

Our system only has one disk so go ahead and select ‘vda’ and click Continue to proceed.

Install CBPP, Partition Disk

Select partitioning scheme

We’ll go ahead and install all the files into a single partition. There’s no need to complicate this setup given its limited use. Select ‘All files in one partition’ and click Continue to proceed.

Install CBPP, Partition Disk

Overview confirmation

Check you are happy with the partitioning information and if so, click Continue to proceed. You can see the system has provided a small 401.6MB swap space and the rest of the disk space is assigned to the primary partition.

Install CBPP, Partition Disk

Highlight ‘Finish partitioning and write changes to disk’ and click Continue to proceed.

A final confirmation before the disk contents are destroyed, select ‘Yes’ and click Continue to proceed.

Install CBPP, Partition Disk

Partitioning will now proceed… After a short wait we can move n to configure the package manager.

Configure the package manager

We’ll need to keep our system updated and install some programs so configure the package manager by highlighting an area and click Continue to proceed.

Install CBPP, Select package mirror

and again, select the Debian archive mirror close to you.

Install CBPP, Select package manager

Click Continue to proceed.

Install grub

In order for our VM to boot, we need to install a grub boot loader on the primary system disk. As this is the only OS, we can go ahead and install it to the Master Boot Record. Select ‘Yes’ and click Continue to proceed.

Install CBPP, Install grub

Highlight our only drive, ‘/dev/vda’ and click Continue to proceed to write the grub loader to our drive.

Install CBPP, Install grub

Install complete!

Install CBPP, Install complete

Congratulations, you now have a basic working Linux VM. Click Continue to reboot, CrunchBangPlusPlus should load through a login screen where you can enter your username and password to login to the desktop environment.

CBPP, Login

Configure CrunchBangPlusPlus

At this stage we have a functioning Linux desktop. Feel free to play around with it to familiarise yourself with it. You can see in the system info display on the background CrunchBangPlusPlus impressively runs in less than 200MB of RAM and takes 2.2GB of disk space. It is possible ot shrink it further but its so lightweight I didn’t think it was worth the effort.

Close the cpbb-welcome window, or enter q to quit it. The configurator offers some neat additional configuration steps but largely they are superfluous for our Crashplan needs. The important steps are already captured in this guide.

CBPP, Setup

Prepare CBPP for Crashplan

Right click on the CBPP desktop and open a terminal prompt. We’ll do the majority of our configuration in the terminal.

Lets start off by updating our Debian base install with the following commands.

sudo -apt-get update
sudo apt-get dist-upgrade

Configure NTP

Now lets configure our server so it pulls the correct time from our local NTP server, in my case my pfSense router.

In the terminal window, enter sudo nano /etc/ntp.conf and edit the section which details the NTP servers. Comment out the default servers by adding a hash in front of them and add your local source, in my case this is my pfSense router which is at IP address 192.168.20.1.

# pool.ntp.org maps to about 1000 low-stratum NTP servers.  Your server will
# pick a different set every time it starts up.  Please consider joining the
# pool: 
# server 0.debian.pool.ntp.org iburst dynamic
# server 1.debian.pool.ntp.org iburst dynamic
# server 2.debian.pool.ntp.org iburst dynamic
# server 3.debian.pool.ntp.org iburst dynamic
server 192.168.20.1 iburst dynamic

Start the NTP service with sudo /etc/init.d/ntp start and verify it started correctly with sudo /etc/init.d/ntp status.

Verify your server is set correctly by entering ntpq -p, for example

$ ntpq -p

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
*192.168.20.1    198.60.22.240    2 u   10  256  377    0.213    1.114   0.127

Configure automated shutdown (UPS)

The Promxmox host needs the ability to shutdown virtual machines correctly and safely. This is accomplished via ACPI commands. Without ACPI being installed and configured Proxmox cant initiate a safe shutdown of the VM, either automatically from a UPS event, or from user interacting with the Proxmox interface. Without ACPI correctly configured you will see errors in the VM task list such as

TASK ERROR: command '/bin/nc6 -l -p 5900 -w 10 -e '/usr/sbin/qm vncproxy 103 2>/dev/null'' failed: exit code 1

Install prerequisite software

Enter the command sudo apt-get install acpi-support to install ACPI support.

Edit shutdown script

We will edit the power button event to initiate the shutdown command. Enter sudo nano /etc/acpi/events/powerbtn-acpi-support and edit the file so it contains.

event=button[ /]power
action=/sbin/shutdown -h -P now

then start the service by entering

$ sudo /etc/init.d/acpid start
[ ok ] Starting acpid (via systemctl): acpid.service.

you can check it started correctly by

$ sudo /etc/init.d/acpid status

● acpid.service - ACPI event daemon
   Loaded: loaded (/lib/systemd/system/acpid.service; disabled)
   Active: active (running) since Sat 2016-07-23 18:01:55 EDT; 50s ago
 Main PID: 443 (acpid)
   CGroup: /system.slice/acpid.service
           └─443 /usr/sbin/acpid

Jul 23 18:01:55 crashplan acpid[443]: starting up with netlink and the input layer
Jul 23 18:01:55 crashplan acpid[443]: 27 rules loaded
Jul 23 18:01:55 crashplan acpid[443]: waiting for events: event logging is off

Verify shutdown script

You should try to initiate a VM client shutdown from the Proxmox user interface and verify everything works correctly. Hopefully the VM will shutdown promptly and cleanly reporting ‘OK’ in the event log rather than the previous ‘Error, VM quit/powerdown fail’.

Due to the importance of proper shutdown procedures in time of power outage, I recommend verifying the full UPS shutdown stack.

Configure automated share mounting (CIFS)

I originally mounted my shares via fstab however this caused me some issues. In my experience, it appears fstab runs before the network is completely up and ready to accept connection attempts and therefore some folders may not mount correctly. Assuming you have created the root folder for the mount to attach to, Crashplan views this as an empty folder and politely removes all the associated data from the cloud assuming you have deleted it locally!
I now use autofs to mount drives which has been a lot more reliable.

I have a number of FreeNAS folders shared which I back up daily to Crashplan’s cloud. I’ll focus on one specific folder called ‘Data’ in this guide but it should be simple to extrapolate to all the folders you need to backup.

Install prerequisites

Install autofs, CIFS and smbclient packages with the following command.

sudo apt-get install autofs smbclient cifs-utils

I like to verify SMB by running smbclient -L <hostname> where hostname is the name of your server hosting the shares, for example.

$ smbclient -L freenas
Domain=[WORKGROUP] OS=[Windows 6.1] Server=[Samba 4.3.6-GIT-UNKNOWN]

  Sharename       Type      Comment
  ---------       ----      -------
  Data            Disk
  IPC$            IPC       IPC Service (FreeNAS Server)
Domain=[WORKGROUP] OS=[Windows 6.1] Server=[Samba 4.3.6-GIT-UNKNOWN]

  Server               Comment
  ---------            -------
  FREENAS              FreeNAS Server
  NGUVUWIN8            Win8

  Workgroup            Master
  ---------            -------
  WORKGROUP            FREENAS

Configure autofs

Edit autofs master configuration file by entering sudo nano /etc/auto.master

Add the following to the bottom of the file, /mnt /etc/auto.smb-shares

so the file looks like this

# Sample auto.master file
# This is an automounter map and it has the following format
# key [ -mount-options-separated-by-comma ] location
# For details of the format look at autofs(5).
#
#/misc  /etc/auto.misc
#
# NOTE: mounts done from a hosts map will be mounted with the
# "nosuid" and "nodev" options unless the "suid" and "dev"
# options are explicitly given.
#
#/net -hosts
#
# Include /etc/auto.master.d/*.autofs
#
+dir:/etc/auto.master.d
#
# Include central master map if it can be found using
# nsswitch sources.
#
# Note that if there are entries for /net or /misc (as
# above) in the included master map any keys that are the
# same will not be seen as the first read key seen takes
# precedence.
#
+auto.master

/mnt /etc/auto.smb-shares

Credentials file

To avoid storing the username and passwords of the shared folders in the configuration file, we’ll create a credential file by entering sudo nano ~/.smbcredentials

this file should be populated only with the username and password used for accessing the shared folder, no spaces either side of the ‘=’ sign. For example

username=nguvu
password=mystrongpassword

Reduce the access rights to this file by entering sudo chmod 600 ~/.smbcredentials.

You will need a credential file for each share you access which doesn’t share the same username and password.

Configure shares

We’ll now create the configuration file that specifies the folders we want autofs to mount for us by entering sudo nano /etc/auto.smb-shares

Enter the following line into this file

Data    -fstype=cifs,ro,credentials=/home/cp/..smbcredentials ://freenas/Data

Data is the name of the folder the line refers to.
fstype is the share filesystem type, in our case, CIFS
ro mounts the share as read-only
credentials specifies the file containing the username and password to use for authorisation.
://freenas/Data is the network location of the share

It should be simple enough to extrapolate form that line into the other shares you would like to access.

Start and verify autofs

Start autofs by entering sudo /etc/init.d/autofs start and verify its status with sudo /etc/init.d/autofs status

$ sudo /etc/init.d/autofs start
[ ok ] Starting autofs (via systemctl): autofs.service.

$ sudo /etc/init.d/autofs status
● autofs.service - LSB: Automounts filesystems on demand
   Loaded: loaded (/etc/init.d/autofs)
   Active: active (running) since Sun 2016-07-24 00:04:54 EDT; 31min ago
   CGroup: /system.slice/autofs.service
           └─1998 /usr/sbin/automount --pid-file /var/run/autofs.pid

Jul 24 00:04:54 crashplan autofs[1993]: Starting automount....

Now ensure we can access the directory we setup,

$ cd /mnt/Data
$ ls -l
total 0
drwxrwxrwx 283 root root 0 Nov 15  2015 apps
drwxrwxrwx   2 root root 0 Jul 17 19:43 FreeNAS
drwxrwxrwx   7 root root 0 Oct 18  2014 Linux
drwxrwxrwx 122 root root 0 Jul 17 19:41 Macintosh Apps
drwxrwxrwx   4 root root 0 Jul 17 19:45 pfSense
drwxrwxrwx   3 2001 root 0 Jul 17 19:33 Proxmox

The /mnt/Data folder will unmount itself after a period of activity but will remount whenever it is required by a user or an application.

Debugging

If you need to have further visibility into what’s going on with autofs, you can turn verbose logging on by editing the /etc/default/autofs file and setting the LOGGING line to VERBOSE OR DEBUG.

Install Crashplan

From Proxmox, Navigate to our Crashplan VM via ServerView > DataCener > Proxmox > 103 (crashplan)

Click on Console to start the VM, enter the username and password to login to the desktop.

Download Crashplan installer

Right click on the desktop and open a Web Browser.
Navigate to www.crashplan.com and go go downloads.

Click on ‘Download Crashplan’ which will download the Linux tar file to your downloads folder.
Right click on the desktop and open Terminal. Follow these commands to unpack the archive and commence installation.

$ cd ~/downloads
$ ls -l
-rw-r--r-- 1 cp cp 47470950 Jul 24 07:43 CrashPlan_4.7.0_Linux.tgz

Unzip the tgz file with tar -xzvf <filename>, for example

$ tar -xvzf CrashPlan_4.7.0_Linux.tgz
crashplan-install/install.sh
crashplan-install/uninstall.sh
crashplan-install/
crashplan-install/scripts/
crashplan-install/CrashPlan_4.7.0.cpi
crashplan-install/EULA.txt
crashplan-install/INSTALL
crashplan-install/README
crashplan-install/install.defaults
crashplan-install/scripts/CrashPlan.desktop
crashplan-install/scripts/CrashPlanDesktop
crashplan-install/scripts/CrashPlanEngine
crashplan-install/scripts/crashplan
crashplan-install/scripts/run.conf

cd into the extracted file folder and execute the install script.

$ cd crashplan-install

$ ./install.sh

Installation will now commence. The default options are ok and summarised below for completeness.

Would you like to switch users and install as root? (y/n) Y
Read the EULA…
Do you accept and agree to be bound by the EULA? (yes/no) yes
What parent directory do you wish to install CrashPlan into? [/usr/local] return to accept defaults
/usr/local/crashplan does not exist. Create /usr/local/crashplan? (y/n) y
What directory do you wish to link the CrashPlan executable to? [/usr/local/bin] return to accept defaults
What directory do you wish to store incoming backup data? [/usr/local/var/crashplan] return to accept defaults
/usr/local/var/crashplan does not exist. Create /usr/local/var/crashplan? (y/n) y
What directory contains your SYSV init scripts? [/etc/init.d] return to accept defaults
What directory contains your runlevel init links? [/etc/rc5.d] return to accept defaults
Verify selections
Is this correct? (y/n) y

Crashplan will now install files, configure Java and start the Crashplan engine.

Adopt Crashplan backup

Back in the CBPP desktop, open the file manager and navigate to /usr/local/crashplan/bin. Double click on the CrashplanDesktop icon.

Run Crashplan

As we are going to deal with adopting an existing Crashplan backup, enter your existing username and password and log into the Crashplan client.

Adopt Crashplan cloud backup

Click on ‘adopt’ at the top of the page and select your archive when presented with the option.

Adopt Crashplan archive

Click on archive to adopt the backup. At this stage you will be logged out and you will have to log back in again. Assuming you followed best practice and set a encryption key to encrypt any data uploaded to Crashplan’s cloud, you’ll be notified that the backup isn’t able to run not knowing this code.

Adoption fail

Enter your 448bit passphrase and Click Save. Crashplan will now verify the local files with those in the Crashplan cloud.

Configure Crashplan

One of the most common complaints about Crashplan is slow uploading of larger datasets. This I believe is largely affected by the deduplication of data. Thanks to NetworkRockstar it became widely understood what the impact of this deduplication process was and also how to disable it. Crashplan address the topic here themselves and it is worth reading before you consider disabling it. I’ve been running without deduplication and compression for 18 months now without any adverse effects.

To disable deduplication, nano /usr/local/crashplan/conf/my.service.xml and edit the dataDeDupAutoMaxFileSizeForWan setting to 1 as follows

<dataDeDupAutoMaxFileSizeForWan>1</dataDeDupAutoMaxFileSizeForWan>

Restart Crashplan and you should see improved upload speeds. My most recent backup ran at around 100mbps (my upload cap is 150mbps) for a significant proportion of the 2TB uploaded.

References

Debian GNU/Linux Installation Guide