Crashplan Peer-To-Peer Alternation: Duplicati Setup

26 09 2017

Duplicati Installation / Setup Notes

The only thing left to do is install Duplicati and set it up to backup to the the Minio service.  I’m not going to walk through that in as much detail but some notes:

  • In destination choose S3 Compatible (custom URL)
  • in the custom URL line enter IPADDRESS:9000 or HOSTNAME:9000
  • Bucket Name: Enter what you made above (when you save it will as you to append your username to it.  You can say no. This is Amazon S3 specific setting)
  • Enter your AWS/Access Key and the AWS/Secret Key
  • Click Advanced Settings
  • Select “s3-ext-forcepathstyle” option
    • Check the checkbox that appears to make it ‘True’
    • This has to do with the way amazon works vs a self hosted. More information here
  • Hit test and see what happens. It should say succeed.

That hopefully is enough to get some folks started.  I’ll try to remember to update this post with any new information or results if I get the time.

Advertisements




Crashplan Peer-to-Peer Alternative: Configuring Raspberry Pi

26 09 2017
MinioRpi

The goods. One 2 TB 2.5″ USB Drive & One Raspberry Pi 1 B+: Pen for scale.

DietPi Setup

  • Download and flash DietPi to and SD card. I used a 2 GB and it works just fine.
  • Boot into DietPi. Login either with SSH or at the console using…
    • Username: root
    • Password: dietpi
  • Let it update and expand the card.
  • At the initial config screen…
    • install noip client (directions are copied from the dietpi site that I  can’t seem to link to the noip article specifically)
      • Select…
      •  DeitPi-Config

      • Goto Networking Options: NAS/misc and select NoIp from the menu.
        • nb: If NoIp is not yet installed, confirm the installation and reselect NoIp from the menu once completed.
      • Type in your username and password details at the prompt
      • Type in 5 for the interval option
    • Then select Install and hit enter
      • This will warn you that you are making a bare bones system and that is what we want
    •  Let it do its thing until it is back at the login screen

Using the menus/tools to…

  • Launch the dietpi main menu
    • ./dietpi-launcher

  • Format my external drive in ext4 via dietpi-drive manager
    • Mount using GUID method
    • Run via DietPi-Drive_Manager

  • Change swap file size
    • Mine was unreasonably large so I change it to 256 MB
    • via the DietPi-Config Menu Option-> Advanced Options -> Swapfile Size

  • Change root user pw
    • via DietPi-Config Menu Option -> Security Options -> Change Root Password

  • Change Hostname if you like
    • via DietPi-Config Menu Option -> Security Options -> Change Hostname

All that was left was to ‘install’ minio.  Minio has a repository of Init scripts including systemd (which DietPi/Raspbian/Debain use).  Looking through the minio.service script and its documentation I was able to make an install that was as ‘easy as possible’.

I’m looking into scripting this install into the pre-packaded DietPi software installation option.  This would make a rebuild EVEN easier and anyone else could use it.

****EDIT: I submitted the code to auto-install minio via the DietPi config menu to the DietPi project.  To see if it has been accepted and merged into the distro check on it here.  I’ll make a new post if/when it gets merged. ***

Minio Installation Setup

Here are the installation steps:

  1. Copy Minio to /usr/local/bin
    1. cd /usr/local/bin

    2. wget https://dl.minio.io/server/minio/release/linux-arm/minio

    3.  chmod +x minio

      1. This makes it executable
  2. Copy systemd init script to /etc/systemd/system
    1. cd /etc/systemd/system

    2. wget https://github.com/minio/minio-service/raw/master/linux-systemd/minio.service

  3. Create User to run it under
    1. adduser –system –group minio-user

  4. Create config file /etc/default/minio
    1. cd /etc/default

    2. nano

    3. Enter the following information.
      1. # Local export path.
        MINIO_VOLUMES=”/PATH/TO/YOUR/STORAGE/DESTINATION”
        # Use if you want to run Minio on a custom port.
        # MINIO_OPTS=”–address :9199″
        # Access Key of the server.
        # MINIO_ACCESS_KEY=Server-Access-Key
        # Secret key of the server.
        # MINIO_SECRET_KEY=Server-Secret-Key

      2. The only mandatory entry is the volume(s) to store data on.
        1. I used DietPi’s default mounting so it was /mnt/UUID-OF-DISK/minio-data
        2. It can be any paths or in fact multiple paths just make sure user has write access to it
        3. I made my minio-user owner of the folder
          1. chown minio-user \mnt\PATH\FOLDER
      3. Options include:
        1. Non-Default Port: remove # symbol to activate the line and change the port to your choice
        2. Custom Access Key and Server Secrets — This will be auto-generated for you if you leave them blank
    4. Ctrl + W to save file. Name it “minio” (with no quotes)

    5. Ctrl + X to exit

  5. Enable the service to start on boot
    1.  systemctl enable minio.service

  6. Reboot
  7. Login to DietPi on the console and get the Access Key / Secret key by opening the generated config file (write these down you will need them to connect)
    1. nano /home/minio-user/.minio/config.json

  8. Log off the console
  9. In a brower on your network go to http://IP:9000 and you should see a login page
  10. Use the Access Key and Secret Key you copied down to login
  11. On the bottom right there is a red plus sign. Click this to create a ‘bucket’. Name it as you wish.
  12. Minio is now configured as a service running in no-login service account

DietPi Configuration Backup

  • Backup the entire installation onto a USB Key (after I made all my configs)
    • This is done after Minio and everything is installed
    • Use the System Only “Backup Mode” or else it will try to back up all your saved files to Minio
    • run the below at the bash prompt
    • ./dietpi-backup

    • Dietpi directions / details (scroll down)

I realize this looks like a lot of steps but compared to most installs it is nothing and fairly straight forward.  To the point I’m considering writing the auto-install script for DietPi myself.

That hopefully is enough to get some folks started.  I’ll try to remember to update this post with any new information or results if I get the time.

 





Crashplan Peer-to-Peer Alternative: My personal solution Minio & Duplicati

26 09 2017

An Old Dream: Small, low-power, backup repository

On thing I kept ‘wanting’ Crashplan to do that it didn’t do well/easily/at all is to run on an ARM processor. Or more precisely a raspberry pi.  I dreamed of slapping an external 2.5 usb drive to a raspberry pi and dropping it at a friends house to backup to. I decided it was as a good of time every to  consider this option again — not the least of which is my backups could be physically closer in case I need to deal with it.

My Needs

After quite a bit of research and playing I’ve settled on a plan.  My needs are as follows:

  • ARM capable
  • Raspberry Pi supported
  • Easily installed / reproduced (days long of configs, etc are fun for the initial project but if it ever goes down rebuilding is often more of a ‘need’ than a ‘project’ and is less fun to do it again over a week)
  • Small, lightweight and fast footprint
  • Preferably open source / free
  • Dynamic DNS update ability

My Proposed Plan

In a nutshell:

Server: Raspberry Pi B+ with DietPi OS running Minio

Backup Software: Duplicati

There is a lot of research and decisions and personal dead ends in this plan.  It started with Duplicati that came up often in the forums.  It looks great and is able to backup to a large number of types of destinations. It is open source, actively maintained/improved, and does block level backups, compression, and client side encryption. All features I wanted. It also has good reviews overall.

That lead me down the road of what server back end to use.  I played with Seafile which I think is really amazing and if I wanted to run a true dropbox scenario I would entirely consider.  The only downside is getting it installed is complex so I went down the road of using Docker to run it in a test-for-production mode.  It worked great except I couldn’t get the WebDAV server to work.  Which was the exact feature I wanted.  Time to move on.  One bonus of this process is I played with RancherOS which is a small (22 mb) OS that I could in theory install natively onto a Raspberry Pi (v2 and 3 only — so that is another strike against it as I have a RPi B+ already).  I really liked RancherOS but it wasn’t for this project.

Next, I searched for and found a small and easy to install WebDAV server: FuguHub. This was slick.  It is again more of a DropBox replacement but it would still do nicely.  I went OS hunting after finding this and discovered Alpine Linux. A wickedly small linux distro at unpacked and ‘installed’ around 130 mb.  I never got Alpine to run FuguHub but I think that is because I was downloading the ARM version of FuguHub rather than the x86 (I was in virtual box at the time).  That I’m pretty sure I could have gotten working. (I bailed before trying).

At the same time I came across Minio: a single executable S3 compatible storage server written in Go.  Whoa. This is as slick/even a bit slicker than FuguHub was.  Minio ran no problem on my Alpine Linux installation. Great! There were to be other challenges on Alpine to overcome: Dynamic DNS — the application they recommend only supports a single DDNS services that is still working but it isn’t free. I figured out how to make my GoDaddy DNS become a dynamic service but it would have taken some work.  Startup / Init scripts would have to be created. All things I was willing to do for a small and slick setup.  The nail in Alpine Linux coffin for me came when I realized Alpine doesn’t support a full installation to SD Card on Raspberry Pi.  They do in VM and other builds but not RPi.  I could have figured it all out but I went searching for another OS.

Enter: DietPi

Now DietPi isn’t a small at Alpine but it is 1/3rd the size of Raspbian Lite at 500 mb on disk.  Small but not tiny. Ok.  But what really sold me on DietPi was the liteweight menu’s built into the system.  I can get around in bash no problem but I don’t do it often so I have to look up commands a lot.  For initial setup this isn’t a huge deal but for break/fix scenarios using menus is way easier.  Not only that but DietPi includes an easy way to install/setup noip.com DDNS service. BAM!

The menus made it easy to…

  • Format my external drive in ext4
  • Setup/Run NoIP.com updater
  • Change swap size
  • Monitor resource usage
  • Quick and Dirty launch at startup of Minio as a test/proof of concept (runs as root so if exposing to the Internet not a great idea)
    • This system is also great for any one time actions to take on boot. No Init details just a single .sh file DietPi creates to add your command to.  One hint is if you want the autostart.sh script to end while your command still runs (ie – Minio server) put a & at the end of the line
  • It has a framework to autoinstall / config certain applications (I didn’t use this… yet)
  • Backup the entire installation onto a USB Key (after I made all my configs)

All that was left was to ‘install’ minio.  Minio has a repository of Init scripts including systemd (which DietPi/Raspbian/Debain use).  Looking through the minio.service script and its documentation I was able to make an install that was as ‘easy as possible’.

In my next post I’ll go over my configuration.

 





Crashplan Peer-to-Peer Alternative: Parents Solution

26 09 2017

 

Parents Needs /  Proposed Solution

My parents needs are:

  • Local to usb backups
  • Remote backups
  • One software to do both
  • Windows compatible
  • Not terribly expensive
  • Easy to monitor / manage

I came down to two contenders: Syncrify and BackupCow

At the end Syncify won out.  My main reason for choosing it over BackupCow is that it does block level backups and compression.  These are two very nice to haves – almost needs.

Configuration Concerns

Application Installation: Syncrify uses a client-server configuration which means that there are 2 different software installs to do depending on the role of the computer.  If the computer is the one with files to backup (ie – source) then you install the client. If the computer is the one that is receiving the files and storing the backups (ie – destination) this is the server.  One quirk is that to do local backups you need to install both the server and the client on the same machine and ‘act like’ you are backing up to a remote host.  I’ve read about it but have yet to test how well it works.  TBD. This is the ‘weakest’ part of Syncrify for the needs I’m scoping out.

Cost

Server: Each server costs exactly $0.  It appears you can install as many destinations as you like for no cost.  This is a good thing.

Client: Each computer you want to back up (ie – source / client) costs $49.

This isn’t free but for 1 computer with 1 destination $49 isn’t bad at all.  I might in fact buy a license myself and send subsets of my data to them for that cost.

I’ll post an update once I actually start testing this plan.

 





Crashplan Peer-to-Peer Alternative Plan(s): Introduction

12 09 2017

Abandonment

Well it happened. Crashplan pushed us out into the cold and now I am a Crashplan backup orphan.  Less than fun.  Luckily, I’m only using their peer-to-peer and local backup options so I have the full number of months to find a replacement.  Here are my 3 scenarios I need to find replacements for:

  1. My Parents – 2 machines that only do local to usb backups
  2. My Parents – 1 machine that backups to another computer off-site
  3. My Self – 1 computer that backups to another computer off site

Challenges

My parents live 2000 miles away.  I’ve been managing their backups to their server while also sending my backups to the same server.  The biggest challenge with this is managing their server from 2000 miles away.  Making sure Crashplan is working and running, the firewall is open and port forwarded correctly, the actual hardware hasn’t failed or is full.  Some of these are easier than others for remote management — I was happy to find I could install teamviewer as a service on the server so I can just pop in.  But still when the server is impacted in anyway (like a physical move or a power unplugged, etc) my backups stop.  Which is frustrating and sometimes they don’t come back up until I head home again two times a year.

Changes

So I decided to consider breaking up the backup schemes. Finding one solution for them and at least implementing a different copy for me or using an entirely different plan. In th next series of posts I’ll detail out my plans.

 





Bat Phone – The Phone Itself

25 10 2012

Well I got the device working (not sure if I updated that yet or not) still some more tweaking to do but the autodialing feature etc is working. So now what is left is the ‘cool-ness’ factor phone.  I’ve looked around and not found anything to my liking until today….

This is a TARDIS phone the problem is I can only find it in the UK and isn’t cheap and the reviews aren’t stellar.  So I kept poking around and found this:

Yes this is a cookie jar but since all I need is on / off hook switch and a handset I am going to try and transform this into a phone! It has a removable liner that hopefully I can cut off some of the top and allow a small space below the liner and make that a pressure activate switch for the phone circuit.  The idea would be for the phone while in the jar to register as on hook and when out of the jar to register as off hook.  Right now I think I’m going to try and steal old parts from other phones…

We will see how this goes or if it just gets deployed with a normal phone. That would be sad but better than not being deployed at all.





Step 4 – SSH Access

23 10 2012

I’ve been using the serial access since I got it working but that involves RDPing to another machine that has the serial port and is connected to the BeagleBoard-xm. Entirely doable but not ideal.

I tried to SSH (using PuTTY) to the IP of the Beagleboard-xm on port 22 and it failed. Well today I figured out that SSH is running on port 222. So there you have it.  Now I can use SSH for most of the config from my main laptop.

That is nice.