Absolutely it has for me. For 5 years I worked on a video store website as the sole developer, every few months having the two founders drop by to look over my code and tell me how terrible it was. Living in a remote suburb of the city, I didn't have the ability to go to meet-ups or build a network (outside of LinkedIn, which doesn't really count). So now I'm out of a job for the past 9 months, and don't have a network either. No idea how to rebuild my network at this point, just kind of out of ideas.
It sounds like most of your issue is not with the fact that you worked remote but that you worked remote and lived in a remote area. It is certainly possible to work remote and live close to where the action is. Example... you may live in Boston and work remote for a company in Silicon Valley. I know people who work remote but go to meetups at night all the time.
You don't need to network in person to network. I've never been to a meet-up in my life and manage to interact with hundreds of people on GitHub through side projects / open source work. If you put yourself out there constantly, good things will happen. Don't fool yourself into thinking moving to a city is a magic bullet. You can be just as isolated in the middle of a city.
This was also my experience. When the time has come to find a new jobs, I found the new position through people I met working on an open source project.
That's why I moved just outside of a major city. While I plan to work remotely for the rest of my life--and optimized for that--I'm still close enough to a city with plenty of on-site opportunities and meetups should the need arise. To me, settling down far from a city, even with a remote job, simply doesn't make any sense. There's no way I'm going to be working at the same place for the next thirty years, and while remote is optional, there should be other opportunities in the area in case remote doesn't work out.
Yes, it is easy to fall into this pit. I did this too and it was a huge mistake. I'm 1 year into looking for a job since I too have no network. Cold applications are not working :(