Today's humanity is collectively irrelevant to the universe as a whole. But, there's no obvious thing standing in the way of von Neumann probes eating Mercury into a K1 civ, using that to send a wave of colonisation VN probes to every reachable galaxy at the same time, and only then spreading out to each star within each galaxy, then star-lifting each star, and in cosmologically short timescales every star is a red dwarf surrounded by a K1 Dyson swarm.
I doubt Dyson swarms can avoid being ground into dust over a "mere" million years, so that's a very different and very dark (literally as well as metaphorically) possible future.
Micro-meteors; frictional wear and tear from normal use; proton ablation from the solar wind[0].
I'd assume we couldn't even get to that scale without solving vandalism, war, and insanity, but if not, then over the scale of a million years there will be twenty thousand space-Victorians and space-Taliban having space-Jihads against space-Buddha- and space-Baphomet-statues. I dread to think what the K2 version of the deliberate destruction and death of WW1 and WW2 would be like.
Likewise industrial accidents (space Chernobyl?), but if the big ones aren't solved there's a significant chance of a Kessler cascade rather than just, say, small incidents destroying 1% of the habitats every millennia[1].
[0] it doesn't get very deep on geological scales, but Dyson swarms aren't capable of being very deep on geological scales either.
[1] Completely arbitrary percentage of course, but that percentage would destroy half of what remained every 69-ish millennia.
Today's humanity is collectively irrelevant to the universe as a whole. But, there's no obvious thing standing in the way of von Neumann probes eating Mercury into a K1 civ, using that to send a wave of colonisation VN probes to every reachable galaxy at the same time, and only then spreading out to each star within each galaxy, then star-lifting each star, and in cosmologically short timescales every star is a red dwarf surrounded by a K1 Dyson swarm.
I doubt Dyson swarms can avoid being ground into dust over a "mere" million years, so that's a very different and very dark (literally as well as metaphorically) possible future.