We're all assuming content comes in constantly, but that's not the case. I imagine they spike heavily at peak times, perhaps 5-6 fold (judging by my own site statistics). Serverless might be good for them because they can handle small loads with small costs and large loads with no effort.
> We're all assuming content comes in constantly, but that's not the case. I imagine they spike heavily at peak times, perhaps 5-6 fold (judging by my own site statistics). Serverless might be good for them because they can handle small loads with small costs and large loads with no effort.
No. I've pushed 1000 r/s (approx) on a price tracking tool at work for keeping track of competitors on one of those boxes. Their largest spike (even if you assume its 10x their average) would be at most 15% of that.
Wow, I wonder if it's location based. Mine was heavily situated in one location, whereas yours was perhaps global? So it was always prime time for someone? Here is a graph of my usage for the past week (note lows of 30-40 and highs of 650 odd):
> Wow, I wonder if it's location based. Mine was heavily situated in one location, whereas yours was perhaps global? So it was always prime time for someone? Here is a graph of my usage for the past week (note lows of 30-40 and highs of 650 odd):
A) I'm not sure "one location" is a city, country, or province.
B) Yes, it was global as we had offices from China to the EU.
C) Given the OP is handling 15 r/s, 150 r/s is not going to be pushing 500 mbps unless something is very poorly designed.
D) Even at spikes of 40x, it would still only be 600 r/s which is still quite doable with a commodity server.
The only serious difference is it was very large batches of requests (20 * SKUs) so people were fine with waiting a few minutes.
As an aside (and a noob question), how many requests can a single server handle? I know frameworks such as Node can do handle many more connections when compared to something like Django but what would be a reasonable ballpark?
As a concrete example, say you have an ecommerce app. Read requests take 80ms while write takes 200ms. Any good article/book I can read to get more insight regarding this?
I've personally handled hundreds of requests per second using Django behind Nginx on a m3.medium EC2 instance. The trick to scaling like this is to take full advantage of caching, and the cores available to you. With Django (and Node), this means spinning up multiple instances on a single server (one per available core).
With even moderate caching (say, 1m), you can saturate your VM's bandwidth before running out of CPU or memory.
Ultimately, you will always have a bottleneck. All you can do is identify the bottleneck (via load testing) and address it. If it's network (and you're in Amazon), you have to scale vertically until they remove that limit, or horizontally if you're already on unrestricted instances. If it's the CPU, you'll need to be more aggressive at caching completed computations, or work in a language with less overhead. If it's memory, start evaluating the tradeoffs between CPU and memory. If it's your DB, start investigating why it's slow (it's probably a poor index or configuration; DBs on decent hardware can handle millions of requests per second before degrading).
I don't have any book recommendations; I just recommend creating something and find its limits with a simple load test tool. Then figure out your bottleneck and fix that. Rinse and repeat, and learn.
Its too application centric to be honest to give good generics unless you've built something similar before.
> As a concrete example, say you have an ecommerce app. Read requests take 80ms while write takes 200ms.
An ecommerce site can be nearly 100% cached except for cart/session/search data. This gives it a massive advantage over everything else (since you can load the non-cached data via js and its relatively tiny).
I think the last time I benchmarked one of our front end servers it was ~500 r/s on 2 cores for our search page.
You really aren't going to know until you try it and benchmark it yourself tho.
$150 (at most) dedicated server for instance.