Another thing about Amtrak is, they do not own their tracks. Instead they use tracks from other companies and have to yield priority to them. The American train system will always be much slower than necessary because of these system delays.
There is a very good youtube video on this subject.
The Northeast Corridor[1], where the DC-NYC route that was previously mentioned is part of, is owned primarily by Amtrak -- so they get priority on the rails and can run as often (and as fast) as they please. However, they share the rails with other passenger rail lines so both entities cooperate on scheduling and maintenance.
Not to mention that there are sections that are designated as high speed sections, consist of 4 tracks (1 local and 1 express track in each direction), and is electrified. Plus, there are regional transportation agencies (and Amtrak) that operate locomotives capable of 150mph+.
Yes, I know this speed pales in comparison to Europe, but as far as I'm aware, it's still faster than any other rail system in the US.
That's why when Northeast US people rave about Amtrak service (between Boston and DC), and everyone else doesn't, both sides think the other side is crazy due to a myopic view.
What suffers in the Northeast is delivery of freight, so almost all of it needs to go by truck, which contributes to traffic congestion, and drives even more people to mass transit for long distances (which makes it easier to make a convincing case for expanding passenger rail operations). Or to relocate to more densely-populated cities that have rail access.
One interesting by-product of this, is diesel engines are banned in New York City rail tunnels, so the only way to get freight from New Jersey into New York city, is via a 140-mile detour known as the the Selkirk Hurdle[2].
In other parts of the US, the rails are owned by freight companies (BNSF, CSX, etc), so their customers' cargo gets priority, and Amtrak suffers because they don't automatically win the "battle" like they do in the Northeast.
Additionally, in these areas, due to the lack of electrification, Amtrak has to rely on an aging fleet of diesel locomotives[3] that break down often, and when they do, everything gets backed up. Couple this with the fact that the only section of Amtrak's network that's profitable is the Northeast Corridor and there's less incentive to throw good money after bad, and also leads to politicians calling for the defunding or even eradication of Amtrak.
If you ever happen to be in Europe (specifically, Switzerland or France) or Asia (specifically, Japan), try their high-speed trains. Pure perfection. Even German, Italian, Spanish, etc trains are still a lot better than the US train system.
While Switzerland has a decent rail system (high frequency, somewhat dense), it doesn't have high speed trains. Germany, Spain, Italy, Belgium, Netherlands all of at least some high speed lines.
I took a Chinese high-speed train a week ago, I concur it was very pleasant and affordable. $60 roundtrip for approx the distance between Seattle and SF, 3.5 hrs journey.
Taking trains between any of Washington, Baltimore, Philadelphia, Newark, NYC, and Boston is not really comparable to doing so in the rest of the country, (because those are on the Acela Express line which, while not as nice as high-speed rail in Europe or Asia, is a reasonably acceptable substitute).