Jesse Harris IT

A blog about IT and other things I find interesting

BTRFS on Raspberry Pi

March 25, 2017 — Jesse Harris

This weekend sees a change to my 2TB attached storage on the Raspberry Pi. I've been using this very Pi your reading this page from as Media server, Local file share, Web and email server so that I can turn off my main PC to save power when not in use. To store media, its had a 2TB ext4 filesystem and with the types of data, a lot of duplication. At work, I use Windows 2012 R2 and Server 2016 and often make use of the data deduplication feature which performs a scheduled offline deduplication. Offline Dedup has the benefit of not requiring much RAM but is best for static data. This is because if the volume changes alot, the dedup process can end up never finishing in maintenance windows.

But, I digress. On Linux there are a few solutions, but ZFS dedup is online and requires extraordinary amounts of RAM. BTRFS is fast becoming a favorite filesystem and replacing ext4 and it is far more feature rich.

The Raspbian kernal has btrfs support out of the box. You can check this on any Linux system by typing : cat /proc/filesystems | grep btrfs btrfs

The conversion

At first I read up on the documentation of a btrfs-progs utility called btrfs-convert. This tool is supposed to safely in-place convert an ext2/3/4 volume to btrfs. I first tried running this from the Raspberry Pi, but monitored the process using top and iotop. After running for 8+ hours I gave up and ctrl-C'd the process. Luckily this is safe to do and the ext4 metadata is still intact.

I then plugged the USB 3 volume into my big honking Fedora box and ran the con- version. It was much quicker here but it errored out with libc errors. Forum searches on this issue reveal that this has apparently not worked on recent ker- nels.

Conversion failed. Backup and Restore

Method 3, I now scraped together enough storage elsewhere to backup the data , reformat as btrfs and then reload the data. Here is the command I used to format: mkfs.btrfs /dev/sdd

Tags: btrfs, raspberry-pi, linux, filesystems

Comments? Tweet  

Raspberry Pi Hosted Site!

August 16, 2016 — Jesse Harris

I remember the old days of the internet when creating your own site was fun. You had get a domain, sort out some DNS hosting, setup a mail server MX records, apache html. There was some real meat to the task. Those days are over for most people as free hosting on wordpress sites and blogging engines are so easy, the old way is not worth the time.

However, some things about the internet are lost to that era. Remeber navigating a site without 6 bajillion ads trying to get in your face? Remember when a page load was only a couple of hundred kilobytes at most? In 2014 gigaom reported the average website was around 2mb and I'm sure some of the main offenders are way off the charts.

With all this in mind I wanted to bring my blogging back to a simpler time and see if I could have a nice site, hosted on the cheap and stuff.

Here are some of the attributes of the site:

  • It's all hosted on a single Raspberry Pi 2
  • Uses static web pages generated manually using a bash script
  • Writen in plain text markdown converted using a perl script to html.
  • Hosted using the light on resources ngnix web server
  • Encased in an old codrel pillbox
  • Sitting under the router in the kitchen!

Tags: raspberry-pi, linux, markdown

Comments? Tweet