Makes me wish we didn't stop advancing optical media technology to where we have cheap and reliable archival quality 1TB discs for a few bucks each. I guess LTO is the best option for personally controlled archival.
We haven’t, but sadly the technology is locked to big tech.
Microsoft has demoed some cool technology where they store data in glass, Project Silica. Sadly, it seems unlikely this will ever be available to consumers. One neat aspect of the design is that writing data is significantly higher power than reading. So you can keep your writing devices physically separated from the readers and have no fear that malicious code could ever overwrite existing data plates.
Some blurbs
Project Silica is developing the world’s first storage technology designed and built from the media up to address humanity’s need for a long-term, sustainable storage technology. We store data in quartz glass: a low-cost, durable WORM media that is EMF-proof, and offers lifetimes of tens to hundreds of thousands of years. This has huge consequences for sustainability, as it means we can leave data in situ, and eliminate the costly cycle of periodically copying data to a new media generation.
We’re re-thinking how large-scale storage systems are built in order to fully exploit the properties of the glass media and create a sustainable and secure storage system to support archival storage for decades to come! We are co-designing the hardware and software stacks from scratch, from the media all the way up to the cloud user API. This includes a novel, low-power design for the media library that challenges what the robotics and mechanics of archival storage systems look like.
Why would they sell it directly? Works better if they can advertise their one of a kind, super stable, cloud specific data archival solution that nobody else can replicate. Or not even advertise it, but maintain lower storage costs per byte relative to AWS or Google.
As far as I know, the technology behind Amazon Glacier has never been shared. Glass disks could eventually be backing the Microsoft equivalent.
I doubt the decisions on the product came down along that logic.
Surely they could make more money by selling it in some form or another. If the economics actually gave them a storage cost advantage over AWS/GCP, then profitability must be possible.
In reality it's probably incredibly expensive, and the ROI could not be obtained without even further investment to drive the costs down.
>Why would they sell it directly? Works better if they can advertise their one of a kind, super stable, cloud specific data archival solution that nobody else can replicate.
Because network speeds aren't high enough to back up terabytes of data remotely on a regular basis. This would only work if you already store all your data with this vendor, which is probably a stupid move.
If network speeds aren't enough, there's Azure Data box, which is the equivalent to AWS Snowball, where they mail you a hard drive and you ship it back to them and they put it in their cloud.
Optical media is neat, but has a number of drawbacks when it comes to large scale operations.
What you're talking about already sort of exists, albeit media hadn't reached "cheap" yet, because the manufacturing scale wasn't there. People weren't interested enough in it. Archival Disc was a standard that Sony and Panasonic produced, https://en.wikipedia.org/wiki/Archival_Disc. Before the standard was retired you could by gen3 ones with 5.5TB of capacity, https://pro.sony/ue_US/products/optical-disc-archive-cartrid...
LTO tape was already at 15TB by the time their 300GB Discs came out, and reached 45TB capacity 3 years ago. Tape is still leaps and bounds ahead of anything achievable in optical media and isn't write-once. (https://en.wikipedia.org/wiki/Linear_Tape-Open)
Part of the problem is you can't just store and forget, you have to carry out fixity checks on a regular basis (https://blogs.loc.gov/thesignal/2014/02/check-yourself-how-a...). Same thing as with your backups, backups that don't have restores tested aren't really backups, they're just bitrot. You want to know that when you go to get something archived, it's actually there. That means you're having to load and validate every bit of media on a very regular basis, because you have to catch degradation before it's an issue. That's probably fine when you're talking a handful of discs, but it doesn't scale that well at all.
The amount of space that it takes for the drives to read the optical disc, the machinery to handle the physical automation of shuffling discs around etc. combined with the costs of it, just make no sense compared to the pre-existing solutions in the space.
You don't get the effective data density (GB/sq meter) you'd need to make it make sense, nor do the drives come at any kind of a price point that could possibly overcome those costs.
To top it all off, the storage environment conditions of optical media isn't really any different from Tape, except maybe slightly less sensitive to magnetic interference.
>LTO tape was already at 15TB by the time their 300GB Discs came out, and reached 45TB capacity 3 years ago.
No, they didn't. The largest LTO tape is only 18TB; your numbers are bogus. Those are BS advertised numbers with compression. If you're storing a bunch of movies or photos, for instance, you can't compress that data any further. The actual amount of data that the medium can physically store is the only useful number when discussing data storage media.
That's fair. The LTO capacity was already still significantly larger than archive disc at any stage in archive disc's life cycle.
Both Sony and Panasonic completely failed to demonstrate actual value from the format. Smaller capacity, for the same kinds of environmental constraints, similar size drives etc. There was just no reason to actually use it.
Yeah, it's really too bad someone hasn't made a reasonably-priced archival format that consumers and small businesses can use, because LTO isn't it. The closest they have is MDISC, but the storage capacity is small, and from what I'm reading, discs advertised with this aren't necessarily all that long-lived anyway (if they're using dye).
What we need is a cheap, write-once format that can hold at least 1TB, similar to how we used to use CD-Rs 20-25 years ago, but without organic dye like those discs and with a far longer shelf life.
I wonder if someone could mass-produce a BD-R type media, but the size of a laser-disc, and resistant to almost all scratching. Maybe put it in a case like the old 3.5" floppy disks had?
It's also quite expensive, at $6/TB/month. If you have a lot of data, that adds up quickly. Just for my 4TB backup drive, it's much cheaper to just use HDDs and rotate them myself.
Unfortunately recordable optical is on it's way out. Sony recently slashed the staff at the Japan plant that makes BD-R's (BD-R XL's). Still CMC makes CDR, DVDR, BDR though.
According to their video ( https://vimeo.com/502475794/ffbfb82b15 ), the company patented bit plane image storage. What the heck? That is so obvious and shouldn't be patentable.
On a side note, they keep touting how robust their data archival solution is. But I have my doubts. For example, if an image has a big patch of 0 or 1 bits, then it might be impossible to accurately align the bit positions ("reclocking"); this is the same issue with QR codes and why they have a masking (scrambling) technique. Another problem is that their format doesn't seem to mention error correction codes; adding Reed-Solomon ECC is an essential technique in many, many popular formats already.
