GCE has some major limitations. They just introduced static IP addresses 2 months ago..and I had to re-create all my servers to make use of the static IP.
Then, you either assign a public IP to each machine, or build yourself a solution...luke a NAT gateway. Google does not offer a service like this.
The machine that are part of the same subnet cannot talk to each other directly.. they have to go via the google gateway. This reduces the posibility of implementing High Avalability solutions that are based on broadcast or Virtual IP (MS SQL, Windows failover, vrrp...etc).
Labels that you apply to VM's or firewall rules are free text without the possibility to select from a labels list/cloud or type a new one...accidents can happen.
The MySQL solution is only reachable via INTERNET!!!. They seems to have a beta for using a local socket if you install a sql proxy on each server where you need sql db access...but not in production.
You'll note that AWS Nat gateway are only available since less than 6 months.
Any company who started before had to implement their own NAT gateway (we did). And with the shitty networking (150 Mbps top on large instances), that was the bottleneck for the whole subnet/VPC.
GCE has a lot better networking and 1Gbps on their instances which may them way less sucky than AWS for anything routing related.
Anyway. I migrated part of our cloud infra (one hundred instances, few subnets) to the new NAT gateway recently. It's working well so far but a few notes come to mind.
You gotta use the latest terraform (or cloudformation, tip: TERRAFORM IS BETTER). Eventually, play with the Release Candidates if you started early.
Ansible doesn't support NAT Gateway at all. (And probably that puppet/chef/salt do not either.) I doubt they will before next year.
Just my 2 cents about NAT Gateway. Let's not be too quick to judge google when AWS just released the feature a few months ago and it's only being available in tools about now.
Another odd limitation: No support for internal load balancers; all LBs must be public. They have it as an alpha release available on request, so it's coming, but it's a serious deficiency right now.
AWS ELB limitations: No static IP.
Only accessible through a domain name like "elb-role-12345.eu-west.aws.amazon.com" and the underlying IPs can change every 60 seconds.
The feature has been missing for years, it doesn't have any workaround, it's not being worked on anytime soon.
By VPN I mean something that can be used individually by developers from their own machine (is there a proper technical term for this type?), not static point to point from a site such as an office (which is what CloudVPN seems made for).
We ended up setting up OpenVPN, which shouldn't be necessary. SSH is great and all, but when you add Kubernetes and other sensitive things like dashboards, you really want to virtualize everything and hide every internal service behind a proper, solid VPN gateway.
I think you're saying you want something like our BeyondCorp setup: moving from the office to a coffee shop doesn't change that you're still the authenticated user that we trust, you're just on a sketchy network now.
That's feasible but only things that do certificate exchange (https etc is easy enough). For what it's worth, part of the reason for Cloud Shell (the little shell within the web console) is so that you can interact with your infrastructure from inside the GCP network. That doesn't solve all your needs (like dashboards or deploying code from your laptop) but is often quite handy.
To the point on MS-SQL, tbh, if you need MS-SQL, Azure for SQL proper, or Azure SQL will both be far less expensive than on any other cloud platform, offsetting any pricing difference for your application VMs even.
I'll be testing some latency between DO and Azure just so I can use some of Azure's services while keeping my applications on DO. Also considering Database Labs (databaselabs.io) for PostgreSQL on DO. Though I do like some of Azure's other offerings... Azure Tables are pretty nice if your needs fit the paradigm, even wrote some node.js wrappers to make access a lot easier a couple years ago, if I go with Azure will likely update them.
Then, you either assign a public IP to each machine, or build yourself a solution...luke a NAT gateway. Google does not offer a service like this.
The machine that are part of the same subnet cannot talk to each other directly.. they have to go via the google gateway. This reduces the posibility of implementing High Avalability solutions that are based on broadcast or Virtual IP (MS SQL, Windows failover, vrrp...etc).
Labels that you apply to VM's or firewall rules are free text without the possibility to select from a labels list/cloud or type a new one...accidents can happen.
The MySQL solution is only reachable via INTERNET!!!. They seems to have a beta for using a local socket if you install a sql proxy on each server where you need sql db access...but not in production.
No support for MS SQL.