They could be. Absent other forces acting on them, they would move along geodesics, like everything else. Near an ordinary massive body, geodesics are orbits.

Popular example: the wormhole in "Interstellar" was orbiting Jupiter.