One of the things that keeps us awake at nights is the fear that a ransomware attack will take out one or more of our clients. They don’t seem quite as worried as us, maybe because they have us to worry?
But really the only sure fire way of having your backups safe from an encryption program that will attack your server is to have the backups not connected to your server, or not writable.
This means that we have a strategy for some customers which involves connecting and disconnecting backup drives to the server, but that’s not desirable for a number of reasons
- People don’t bother changing the drives
- Unless we set it up, they often aren’t encrypted
- The drives take quite a beating
So in looking for an offsite solution we considered a whole bunch of storage providers- that’s enough to send you crazy just trying to compare features and prices…..
We settled on Backblaze B2. Did we tell you how much we love Backblaze? It’s ultra cheap, and Backblaze have an amazing USP- they open source all of their designs, and rely on being good to get ahead of the competition. We love that.
Anyway Backblaze B2 has some unique requirements, so we then needed to find a piece of Mac OS software that supports it.
It needed to be
- Cheap
- Scriptable (ie. automatic backups)
- Supports B2
- Can be operated by a client if needed
In comes Cloudberry Backup for Mac from Cloudberry Labs. so we bought a copy and tested it over a few weeks. Everything went well, it seemed almost too good to be true- usually you can have cheap or good but not both……
So we recommended it to clients and asked them to buy the software, and provide their credit card for billing with Backblaze. Although we would be managing it, one of the drawbacks with Backblaze is that they aren’t really set up for the whole thing to be managed by an MSP, so why not let clients save a bit of money?
So we set it up and started it backing up for 2 clients on the first day. The next morning getting 2 alerts that both servers were nearly full! A bit of investigation revealed that Cloudberry was filling up it’s own /Logs folder with thousands of tiny log files that weren’t being purged.
One server eventually had over 400,000 individual log files taking several hundreds of gigabytes. Have you ever tried deleting that from the GUI?
We had to disable it almost immediately and after over 2 weeks of going back and forth with the manufacturer, they finally admitted defeat and refunded both clients.
We are still very cranky, because our reputation is under threat in situations like this, and we depend on software houses to be able to at least support their own products. The support was appalling, they appeared to have no idea what was going on even after being sent logs, never gave suggestions for mitigation, seemed unwilling to solve the problem.
We really loved the idea of Cloudberry because it runs on just about anything, but unfortunately the reality was far from perfect.
OK, next time we try something not so cheap. But the reason for wanting cheap in this case is because Apple are deprecating their Server.app product. We’ve been preparing for this for more than a year. This means that any investment in Mac software to run on a server will be useless when that server gets replaced, so we wanted to minimise the financial damage.
Sigh, we don’t always choose the cheap option, perhaps we deserved that one!
UPDATE 7 December 2018- We’ve been contacted by Gleb Bokhan from Cloudberry, who assures us that this bug is fixed in v2.5 of the software. Can’t verify that but to be fair I’ve made this update. It doesn’t really explain why they were so uninterested in helping or fixing the issue, but everyone has bad days.