It is in no way "pilot error". If the post you replied to is correct, the dropbox software performed an action it should never perform - deleting files the user did not tell the OS to delete. What's more, every dropbox plan I'm aware of keeps deleted files for 30 days, so if it wasn't possible to recover some of the files, dropbox failed not once but twice.
It's all very well blaming the user for not having multiple redundant backups, but that doesn't change the fact that deleting user data without explicitly being told to do so is one of the most egregious sins software can commit. It's not "pilot error" when the software performs an action that should never occur.
The "pilot error" is in not testing the solution before trusting it and in making an assumption that the data would be safe in an untested external system.
Let me put it into a financial context. I did a project that added up to some $800K in cost. Two developers over about a year. Do you think that for even a microsecond I would trust Dropbox as a backup mechanism without extensive qualification and testing? And, would I have that as my sole approach to backup?
Nope.
And this isn't me coming down on Dropbox or suggesting that they are unreliable. I use Dropbox and I am very happy with the service. However, based on experience gotten the hard way, I do not use it for anything that is mission critical. The important stuff is backed-up locally, sometimes with redundant backups.
I don't have any sympathy for an engineer who uses a service and relies on it without the proper testing and qualification phase. Behaving that way is most-definitely, as I said, to be kind, "pilot error".
It is completely unfair to blame Dropbox for anything. This is like the idea of cutting-and-pasting code from the Internet and trusting it with a mission-critical aspect of your application without dissecting the code, testing it and fully qualifying it for what it is you need it to do.
There are tons of examples of this behavior, perhaps the most common one in the web development community is email validity verification. How many careless developers copy a regex like this to validate their email and don't think twice about it:
^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+$
These things are all over the web. And most of them suck. Even some of the most impressive ones will lead you astray.
How many engineers (and, as a result, websites) fail to test and research and end-up with questionable solutions?
Would you blame the regex writer for this or the engineer who chose to use it without doing any testing at all?
It doesn't matter what the regex author claims or how authoritative the website featuring the regex might be, you have to test it before deploying it! If you don't test, deploy and then loose customers because it is a bad regex it is YOUR fault, not the author's. Blaming them is nonsensical.
Not only is it well documented, the author offers test vectors and all of the relevant code and references so that you can take the time to qualify the solution.
So, yes, to be redundant, if an engineer does not do his job I am perfectly happy calling it "pilot error" and even going as far as calling it "incompetence".
How did you even get to talking about engineers and testing/qualification and $800K projects?
Dropbox is a consumer service, also useful for businesses. Nobody's trusting it with the nuclear launch codes. But no matter what, it should never, ever delete your files, unless you do it yourself. Period. And if it's ever unsure, it should ask you.
It's meant to be used with the primary copies of your files, not relegated to a special drive. If it deletes your files wrongly, the fault is clearly with Dropbox. It's marketed as an easy-to-use product you just install and use.
Well, let's just agree to disagree. Get a few more years under your belt and loose some valuable work for relying on such services without local backup and, yes, testing and qualification and then we can talk. Perhaps then you'll understand where I am coming from. There's nothing more painful than such a loss to make it clear that YOU are responsible for your data and nobody else is, no matter what the claims might say.
I cannot blame Dropbox if I loose data. Maybe the service has issues, but loosing my data is MY ISSUE, not theirs. The minute you trust service X with your data without full qualification and testing is the minute you lost your data. Don't blame them for it.
> Dropbox is a consumer service, also useful for businesses.
Same point. Same issues. If you put all of your documents and financial data on Dropbox (or service X) and don't take the steps to have local (and possibly redundant) backups you could easily be confused for a moron. Blaming service X for it is a cop-out. They had nothing to do with the total loss. Yes, they had everything to do with the partial loss.
It is my contention that it is your responsibility to fully qualify such services rather than using them blindly.
I use Dropbox extensively and have ZERO concerns about data loss.
Maybe experience has made me less trusting with what is important to me than a typical user? I could not imagine saving my kids pictures on service X without any other backups. That would be insane. And, if for any reason, service X looses my pictures I was the moron who allowed that to happen. They might suck, but it was my fault, not theirs that I lost EVERYTHING.
Maybe that's the real point I am trying to make: Loosing EVERYTHING is YOUR fault. Loosing WHAT IS STORED IN DROPBOX could be their fault. If you don't have backups for what you store in Dropbox (or service X) you are responsible for the TOTAL LOSS not service X.
> There's nothing more painful than such a loss to make it clear that YOU are responsible for your data and nobody else is, no matter what the claims might say.
These are the kind of lessons that for some reason don't transfer person-to-person like they should be only sink in once they've happened to you (at a guess, that's why you're being downvoted). Whether it is your corporate data worth $800K or your photographs of your first child hours after being born through the point that you realize that it is all gone data is priceless.
In the end it all boils down to the same thing: you can't outsource your responsibility on this one, and a single service holding your data still counts as a single point of failure.
Personal responsibility and the acceptance thereof is a major item on the checklist, anybody downvoting robomartin in this thread still has some learning to do on that front. Don't shoot the messenger if you don't like the message. Dropbox is not a backup, no matter what their marketing literature says and no matter how safe you think they are. In the end you and you alone are responsible for your data and its safety.
A back-up that you have not tried to restore might as well not be there.
>Personal responsibility and the acceptance thereof is a major item on the checklist
There are two separate things going on in this discussion.
You have a personal responsibility to keep your data safe. I don't think anyone is disputing that.
Independently, softwares and services which handle user-created files should never, ever delete them without a direct instruction from the user. Regardless of your backup procedures, it is completely unacceptable for this to happen. Even if I had a copy of my data in a fire-proof safe on every continent, I would be appalled if I found that a file syncing and backup service like Dropbox had deleted some of my files and was unable to restore them, because i) it's an unacceptable thing for any software to do, and ii) it completely contradicts all of their company messaging.
> Independently, softwares and services which handle user-created files should never, ever delete them without a direct instruction from the user.
Show me a business that is willing to offer you that guarantee and I'll show you a business that is out of business.
What if there's a fire? Or malicious destruction? Or massive hardware failure?
Anyhow, the only people who would down-vote some of the comments I am making are those who have never suffered the pain and agony of data loss. If you've ever been in those shoes you know, without an iota of doubt, that what I am saying is 100% on point.
There's nothing wrong with Dropbox. The problem is in how naively people are choosing to use the service.
Could they be better? Of course. Would I still have my own redundant backups and recommend that Dropbox users do the same and fully understand the service. Absolutely!
Personal responsibility is a bitch. It's much easier to blame someone else. Guess what? Blaming someone else for your failings will not make things right or bring back your data.
Look at bug fixing. There's a list of bug severity.
Crashes machine neeeding reboot
Crashes program but not machine (loses data, possibly saved data)
Mangles saved data
Feature not working
part of that feature not working properly
cosmetic stuff.
> There's nothing wrong with Dropbox.
There very much is something wrong with dropbox if it deletes user data without that user expecting it; ideally the user should give clear explicit permission for data to be deleted.
> Independently, softwares and services which handle user-created files should never, ever delete them without a direct instruction from the user.
We're 100% in agreement on that. But since you can never be sure that the software that you're using is bug free (and in the case of dropbox there is now at least one more datapoint that confirms that their software contained at least one bug) you have to more or less count on losing all your data.
Dropbox messaging is of course not going to highlight the fact that their software may contain bugs, it simply isn't in their best interest (and none of their competitors do so either).
Maybe they have a section to that effect in their fine print but even if they don't on hacker news we really should all know better. Unless there is someone here that only writes bug free software... In that case please send me your resume.
> Don't shoot the messenger if you don't like the message.
What if the message is liked but the messenger is behaving like a silly sausage?
Everyone knows that multiple backups are important. If I'd had multiple backups, and Dropbox had deleted items that I hadn't asked it to delete, I'd still be able to comment about that because data deletion is serious business.
This thread isn't (at least, shouldn't) be about data being deleted; it's about unpredictable behaviour from software. When that unpredictable behaviour includes deleting data that that's a valid serious concern even if everyone has great backup strategy.
> but the messenger is behaving like a silly sausage?
I think that's mostly a format issue, the underlying message is solid.
> Everyone knows that multiple backups are important.
My experience and your experience are apparently not the same in this respect. And I look at a lot of companies every year. The number of times the question 'do you do trial restores of your back-ups' is answered with either 'what backups' or 'no' is way larger than what you'd expect.
And those are companies, not individuals where I'd expect the situation to be much worse.
> If I'd had multiple backups, and Dropbox had deleted items that I hadn't asked it to delete, I'd still be able to comment about that because data deletion is serious business.
Yes, absolutely it definitely is. Dropbox dropped the ball here. The problem is that you can pretty much expect them to drop the ball on occasion, no large service has every been 100% without data loss. Amazon, google, dropbox, microsoft, they've all lost some customer data at some point.
It's the rule, not the exception. When you're dealing with data data loss is the thing you're working hard to prevent but it'll never be 100% perfect. It can't be. There will always be edge cases and the simpler you try to make it on the outside the more complex the actual software becomes. Complexity leads to bugs, bugs (can) lead to data loss.
> This thread isn't (at least, shouldn't) be about data being deleted; it's about unpredictable behaviour from software.
All software that I'm aware of contains bugs. The discipline and experience required to write bug-free software is universally claimed to be present in one industry only: aerospace/aviation. And even there they have bugs, just fewer of them at massive expense. Bugs are the norm, not the exception.
> When that unpredictable behaviour includes deleting data that that's a valid serious concern even if everyone has great backup strategy.
Yes, it is a valid concern. And the way to mitigate that concern is by looking it in the eye and saying 'I don't trust software'. Any software. Including dropbox.
I'm bummed out that so many people down-voted you when you put so much effort into your post. I think you inadvertently brought out the shitty, immature side of the HN crowd. :)
He put in a huge amount of effort to derail the discussion. Pretend that where it says "wasn't able to recover everything" the post said "wasn't able to recover everything from Dropbox, and had to resort to getting external backups mailed to him". Same terrible problem on Dropbox's end, suddenly no need to waste pages arguing about 'responsibility'.
He brought out the part of the HN crowd that wants things to be on-topic and/or useful instead of user-blaming. :)
I'll quote the most relevant and definitely constructive/useful parts here you don't have to bother with the links if you don't want to. If you do, please slow down and read it all. Then think about the idea of loosing, I don't know, a year's worth of work and blaming someone else for it.
Here it is (reformatted because HN does not have a much-needed block-quote markup):
My approach for Dropbox on both Windows and OSX is to have the
Dropbox folder on a drive that will never see development work,
even if that means creating a separate partition for it.
In most cases the system drive works fine (you do have separate
system and data drives on your machines, right?).
This forces a COPY operation rather than a file MOVE if you
drag-and-drop files into any Dropbox folder. Which means that
everything in the Dropbox folder could be trashed tomorrow and
nothing whatsoever would be lost.
Do this TODAY and unless you have massive local loss of data you don't have to worry about Dropbox loosing any of your data at all. Not one bit. If that isn't constructive I don't know what it.
Then I also said:
FEATURE REQUEST: It would be very nice if Dropbox client software
could have an option to auto-magically COPY files in and out of
Dropbox rather than allowing any files to be moved.
The above-mentioned hack works fine but it'd be nice to not have to
use it. Also, I'd venture to guess that most Dropbox users don't do
this, which opens them to unintended loss.
In other words, if Dropbox could implement the suggested approach EVERYONE, engineer or not, would be spared of total data loss due to something going on at Dropbox. Local data loss, of course, is still your responsibility.
And then, trying to understand further, I asked this:
So, I'll say that overall I've been pretty constructive. If anyone adopts my approach of storing their Dropbox folder in an unused drive they are almost guaranteed that Dropbox can't loose their data. That is worth money.
Please point me to a link or blog post with your immediate solutions to this problem. I'd love to compare notes. Maybe there's a better approach. Always interested in learning. I am sure there are lots of people who, tonight, might be interested in adopting your approach to ensuring that accidental data loss by a remote service of any kind does not mean total data loss locally.
I read your posts. The constructive parts are minor compared to the derail, and you easily could have avoided the enormous argument. Also, do you really see no downside to the fact that you have basically reimplemented manual sync on top of dropbox? It throws out most of Dropbox's safety features just because they aren't perfect, and throws out half the ease of use.
It's OK. My data is safe. I suspect yours is as well.
Mark Twain: A man holding a cat by the tail learns something he can learn in no other way.
I love done that. I've held a number of cats by the tail over the years. And, he is right, sometimes you don't learn until you have claws painfully sunk into your skin, no-matter what anyone around you might say.
And, I love Dropbox. It's a great service. As far as I am concerned, there's nothing wrong with it. If you own your data Dropbox could go out of business tomorrow and you would not care.
> I don't have any sympathy for an engineer who uses a service and relies on it without the proper testing and qualification phase. Behaving that way is most-definitely, as I said, to be kind, "pilot error".
How the heck do you effectively test a service like Dropbox fully, without making it your full-time job (IT dept). Take the other comment on this article, where someone fired up an old machine and it wiped out his files. What test would have caught that?
> How the heck do you effectively test a service like Dropbox fully, without making it your full-time job (IT dept). Take the other comment on this article, where someone fired up an old machine and it wiped out his files. What test would have caught that?
PRECISELY!
And, if you can't reasonably expect to fully test it and verify it's reliability (of any service) then you DO NOT rely on it for critical data you cannot loose. Choosing to do so and then crying foul when you do loose data isn't engineering, it's, at best, wishful thinking.
Again, this does not mean that Dropbox --or any other service for that matter-- is bad. Not at all. It just means that you have to understand what you are walking into and, if you can't, or don't, then take steps to safeguard from the potential for external data loss, because you just don't know.
It is in no way "pilot error". If the post you replied to is correct, the dropbox software performed an action it should never perform - deleting files the user did not tell the OS to delete. What's more, every dropbox plan I'm aware of keeps deleted files for 30 days, so if it wasn't possible to recover some of the files, dropbox failed not once but twice.
It's all very well blaming the user for not having multiple redundant backups, but that doesn't change the fact that deleting user data without explicitly being told to do so is one of the most egregious sins software can commit. It's not "pilot error" when the software performs an action that should never occur.