The wormhole sends you back in time, but not far enough back to arrive at your point of origin before you left.
If you could treat the wormhole like a magic door that takes you across space, then two people looking at each other through the door wouldn't see anything unusual. One would be in the distant future, by the galaxy's clock, but not by the wormhole's.
So going through in one direction takes you into the future, and the other takes you into the past, but it's a constant offset. The size of the offset depends on how you moved the other end to where it is.
It's true that with multiple wormholes you could try to create a loop, though. That would probably fail due to virtual particular loops.
If you could treat the wormhole like a magic door that takes you across space, then two people looking at each other through the door wouldn't see anything unusual. One would be in the distant future, by the galaxy's clock, but not by the wormhole's.
So going through in one direction takes you into the future, and the other takes you into the past, but it's a constant offset. The size of the offset depends on how you moved the other end to where it is.
It's true that with multiple wormholes you could try to create a loop, though. That would probably fail due to virtual particular loops.