Currently using Docker Swarm with bind mounts instead of volumes, because they're a bit easier to work with, backup-wise.
Infrastructure wise, there are a few VPSes from a local provider (https://www.time4vps.com/?affid=5294 affiliate link), though I've also used DigitalOcean, Hetzner, Scaleway and others in the past, AWS, GCP and Azure would work as well, though would be much more expensive. For executing CI tasks and backing up the data, i have a few servers on my desk (though regular towers with passive cooling, instead of the rack mounted kind), since those are more cost effective, especially storage wise.
The actual files are backed up with BackupPC over the network automatically, at a set interval, to another server: https://backuppc.github.io/backuppc/ Then, of course, the backup server data is occasionally sent to another server in my local network, just to spread it around more, in case of disk failures or one of the servers shorting out.
Admittedly, it's a lot like using regular rsync with cron, although the benefits of BackupPC are being able to really easily restore files, as well as having incremental backups with compression.
Alternatively, one might also achieve the same result with something like Bacula, Kubernetes with either hostPath or maybe even just a NFS mount. Other folks might prefer something like GlusterFS or a similar distributed storage solution, or something like Longhorn for Kubernetes in particular.
Thanks for this, it's interesting to see how others are hosting these kinds of setups. Been wanting to move to it for a while but kube (which I know best) is quite large and the idea of ensuring really solid data backups often has me on edge a bit when not using something like AWS (I have a small server at home I'd use).
> Been wanting to move to it for a while but kube (which I know best) is quite large and the idea of ensuring really solid data backups often has me on edge a bit when not using something like AWS (I have a small server at home I'd use).
One option would be opting for a more lightweight Kubernetes distro, such as K3s https://k3s.io/ or k0s https://k0sproject.io/ both of which have comparatively smaller resource requirements whilst still being certified and compliant.
Personally, I'd also use something like Portainer, Rancher or Lens, but that's just because I like UIs alongside my CLIs that are nice to look at.
Infrastructure wise, there are a few VPSes from a local provider (https://www.time4vps.com/?affid=5294 affiliate link), though I've also used DigitalOcean, Hetzner, Scaleway and others in the past, AWS, GCP and Azure would work as well, though would be much more expensive. For executing CI tasks and backing up the data, i have a few servers on my desk (though regular towers with passive cooling, instead of the rack mounted kind), since those are more cost effective, especially storage wise.
The actual files are backed up with BackupPC over the network automatically, at a set interval, to another server: https://backuppc.github.io/backuppc/ Then, of course, the backup server data is occasionally sent to another server in my local network, just to spread it around more, in case of disk failures or one of the servers shorting out.
Admittedly, it's a lot like using regular rsync with cron, although the benefits of BackupPC are being able to really easily restore files, as well as having incremental backups with compression.
It is a bit rudimentary as far as handling data goes, but sometimes that's all that you need: https://blog.kronis.dev/tutorials/simple-ways-to-do-backups
Alternatively, one might also achieve the same result with something like Bacula, Kubernetes with either hostPath or maybe even just a NFS mount. Other folks might prefer something like GlusterFS or a similar distributed storage solution, or something like Longhorn for Kubernetes in particular.