WebDAV vs S3 vs SFTP

hosting-services
#1

I have some spare boxes floating around so I’m working on configuring some “CrashPlan Home” like destinations that I can distribute to a few family and friends to be able to continue to be able to backup offsite to each other.

Currently I’m reviewing:

I’m trying to decide benefts of each in terms of the following (keeping in mind they’ll be remote):

  • ease of configuration / maintenance
  • performance
  • usability for other tasks

So far it seems to me that:

  • WebDAV (via NextCloud) is the easiest to set up in a server type environment AND is most usable for other tasks (goodbye Dropbox for file sharing?)
  • S3 (vai Minio) is the easiest to set up in a desktop environment

Does anybody have any experience with any of these destinations in “the real world”?

1 Like

Best/Worst back end protocol?
#2

I researched both S3 (Minio on Windows 10) vs SFTP (Bitvise WinSSHD on Windows 10). In the end I ended up going with SFTP for a handful of reasons but I don’t know that it is necessarily better. Here were the pros that I saw:

  • I’m more familiar with it
  • Very straightforward setup. Default options seemed solid, but if you have time there are lots of configuration options to tweak it just the way you want it. Each option is well documented in the UI via tool tips.
  • It’s inherently secure out of the box (can probably make it insecure if you try hard enough) without any fuss.
  • Seems more flexible overall and in my mind a bit more secure. Can configure key based auth, create IP restrictions per user, set up IP blacklisting on too many failed logon attempts etc.

I’ve been very happy with it so far and haven’t had any major issues. The biggest problem is it seems a little slower than I’d like. It takes about 20-25 minutes for each backup to “Verify backend files”. I’m currently backing up about 2.5TB which is around 90,000-100,000 files on the destination. and it takes that long just to do a directory listing via SFTP. I haven’t had time yet to see if there is something I can do to speed it up. It doesn’t seem to be any specific resource bottleneck, but my destination is a very low powered very old netbook so it might just be what it is.

1 Like

Hetzner Storage Box as Duplicati backend
#3

I went down some of these roads. My use case is I wanted to slap a raspberry pi to a usb hard drive and be able to drop it off at a friends house for low energy consumption mini-server. That way I isolated my backups from his hardware.

Anyway, minio is easy to get up and running but if you want to make it auto-start and/or use https it takes a bit more effort but honestly not that much. I’ve been using minio on my raspberry pi for a few weeks now and so far I’m happy as a clam with it. I’m running it on a raspberry pi 1 B+ model and using 50 mb block size I see the cpu spike to 100% for about 20 seconds per block on my local lan than it drops down 5% again for 30 seconds or more before it cranks up again.

I’m running my backups on an older Atom machine with only 2 GB of RAM and even on the local network the pi+minio are not the bottleneck of running backups. That leads me to believe that standard internet transfer rates are going to my biggest limiting factor in that regard.

It is true that Minio does one thing and one thing well. It isn’t a great dropbox replacement though you could use it for that. My only goal was for a backup target. I made different buckets for each backup task.

I’m running it on Raspberry Pi 1 B+ and DietPi distro. DietPi is a striped down version of raspbian that has some really nice menu system/tools that make it easy to get it up and running. One thing it has is a place to install applications with a proverbial one click and optimized for DietPi. I recently contributed code to make Minio a one click install as well as the code to make it easy to add an SSL cert to minio. All automatically – including configuration and autostart. It isn’t in the distro yet but hopefully it will be sooner than later. Added Minio Software by techdabbler · Pull Request #1152 · Fourdee/DietPi · GitHub

So I think minio is pretty easy to maintain and if the code gets merged into the distro then it will be even easier. I’d say a complete OS / applicataion front end rebuild (assuming the data drive is fine) would take me about 30 minutes tops?

On the WebDAV side I went down the road of Seafile on docker but could never get WebDAV to work easily so I bailed (but oh was it pretty!). The other WebDAV backend that was a lead contender for me before I found minio is something called fuguhub. It is more like NextCloud though not as pretty but small, lightweight, and easy to install. There is a box plugin that makes permissions easier as well.

Hope some/any of that helps.

2 Likes

One or more buckets in Backblaze B2?
#4

If you run ownCloud, beware that you really need to update it, they have a history of security exploits:
https://www.cvedetails.com/vulnerability-list/vendor_id-11929/Owncloud.htm

IMO this is because they insist on using Apache + PHP and have an “all-in-one” solution. There are so many things to keep straight in both the underlying libraries as well as their own code.

SSH is generally considered rock solid, especially if you only allow key-based authentication. My own setup uses a non-standard port and that removed virtually all failed login attempts.

Minio is newer, so it does not have a track record, but it uses the Go HTTP library, which is also considered secure and it “does one thing”.

If you want WebDAV, you can do it on top of Nginx:
https://opensource.ncsa.illinois.edu/confluence/display/ERGO/Creating+a+WebDAV+repository+server+with+NGINX

0 Likes

#5

Thanks for the input - it sounds like for my use I’ll probably stick with NextCloud simply because I’ve got beefier hardware and want to provide a DropBox alternative for my family. (Well, that and I haven’t gotten SFTP working reliably on my unRAID setup for non-root users yet).

But I’m very intrigued by @techdabbler’s solution - it sounds like for about $100 USD one indeed have a tissue-box sized setup to drop off at somebody’s house for 1TB of offsite backup.

How would you feel about trying to put together a #howto on that?

0 Likes

#6

Thanks for the heads up (and link to the CVE Details site)! I’m running NextCloud which seems to be a bit lower on the security issue scale.
https://www.cvedetails.com/vendor/15913/Nextcloud.html

But I generally keep most of my bits and pieces up-to-date when possible.

0 Likes

#7

I’ve already started a howto on my blog. Here is the relevant page in regards to getting a minio +rpi headless mini server up and running:

I still need to add the page on getting SSL working but that will be up soon.

2 Likes