> ZFS doesn't support addition of disks unless you're happy to put a RAID0 on top of your RAID5/6 and it doesn't support removal of disks at all when parity is involved. It is possible to migrate by putting giant sparse files on the existing storage, filling the filesystem, removing a sparse file, removing a disk from the original FS and "replacing" the sparse file with the actual disk but this is somewhat risky.
It may be "RAID0 on top of your RAID5/6" but it's all integrated and works smoothly. What I do is have two raidz2 vdevs of 4 disks each, one twice the size of the other, and alternate which one I upgrade (i.e. I started with 4x250gb disks and 4x500gb disks, a few years later replaced the 250gb ones with 1tb ones, then the 500gb ones with 2tb ones, and most recently the 1tb ones with 4tb ones). But yeah I am now stuck with at least 8 disks and if I wanted to migrate off them I'd have to do so all in one go.
> It may be "RAID0 on top of your RAID5/6" but it's all integrated and works smoothly.
I know that it does but I don't like the way you lose your entire pool if a single vdev fails. I'd have less of a problem with it if a pool could distribute data such that a vdev failure results in the loss of only what was on that vdev.
Shrug. I figure losing a random half of my files is pretty much as bad as losing all of them (if I wanted to make an explicit split into two halves I could use two separate pools).
It may be "RAID0 on top of your RAID5/6" but it's all integrated and works smoothly. What I do is have two raidz2 vdevs of 4 disks each, one twice the size of the other, and alternate which one I upgrade (i.e. I started with 4x250gb disks and 4x500gb disks, a few years later replaced the 250gb ones with 1tb ones, then the 500gb ones with 2tb ones, and most recently the 1tb ones with 4tb ones). But yeah I am now stuck with at least 8 disks and if I wanted to migrate off them I'd have to do so all in one go.