A lot of frameworks that use variants of "mark and sweep" garbage collection instead of automatic reference counting are built with the assumption that RAM is cheap and CPU cycles aren't, so they are highly optimized CPU-wise, but otherwise are RAM inefficient.
I wonder if frameworks like dotnet or JVM will introduce reference counting as a way to lower the RAM footprint?
Reference counting in multithreaded systems is much more expensive than it sounds because of the synchronization overhead. I don't see it coming back. I don't think it saves massive amounts of memory, either, especially given my observation with vmmap upthread that in many cases the code itself is a dominant part of the (virtual) memory usage.
If you use an ownership/lifetime system under the hood you only pay that synchronization overhead when ownership truly changes, i.e. when a reference is added or removed that might actually impact the object's lifecycle. That's a rare case with most uses of reference counting; most of the time you're creating a "sub"-reference and its lifetime is strictly bounded by some existing owning reference.
There are 2 unavoidable atomic updates for RC, the allocation and the free event. That alone will significantly increase the amount of traffic per thread back to main memory.
A lifetime system could possibly eliminate those, but it'd be hard to add to the JVM at this point. The JVM sort of has it in terms of escape analysis, but that's notoriously easy to defeat with pretty typical java code.
> Why would an allocation require an atomic write for a reference count?
It won't always require it, but it usually will because you have to ensure the memory containing the reference count is correctly set before handing off a pointer to the item. This has to be done almost first thing in the construction of the item.
It's not impossible that a smart compiler could see and remove that initialization and destruction if it can determine that the item never escapes the current scope. But if it does escape it by, for example, being added to a list or returned from a function, then those two atomic writes are required.
Incrementing or decrementing a shared counter is done with an atomic instruction, not with a locked critical section.
This has negligible overhead in most cases. For instance, if the shared counter is already in some cache memory the overhead is smaller than a normal non-atomic access to the main memory. The intrinsic overhead of an atomic instruction is typically about the same as that of a simple memory access to data that is stored in the L3 cache memory, e.g. of the order of 10 nanoseconds at most.
Moreover, many memory allocators use separate per-core memory heaps, so they avoid any accesses to shared memory that need atomic instructions or locking, except in the rare occasions when they interact with the operating system.
Atomic operations, especially RMW operations are very expensive, though. Not as expensive as a syscall, of course, but still a lot more expensive than non-atomic ones. Exactly because they break things like caches
Not only that, they write back to main memory. There's limited bandwidth between the CPU and main memory and with multithreading you are looking at pretty significantly increasing the amount of data transferred between the CPU and memory.
This is such a problem that the JVM gives threads their own allocation pools to write to before flushing back to the main heap. All to reduce the number of atomic writes to the pointer tracking memory in the heap.
Unlikely. Maybe I'm overly optimistic, but I think it's fairly likely that the RAM situation will have sorted itself out in a few years. Adding reference counting to the JVM and .NET would also take considerable time.
It makes more sense for application developers to think about the unnecessary complexity that they add to software.
That's not strictly true. Mark and sweep is tunable in ways ARC is not. You can increase frequency, reducing memory at the cost of increased compute, for example.
M&S also doesn't necessitate having a moving and compacting GC. That's the thing that actually makes the JVM's heap greedy.
Go also does M&S and yet uses less memory. Why? Because go isn't compacting, it's instead calling malloc and free based on the results of each GC. This means that go has slower allocation and a bigger risk of memory fragmentation, but also it keeps the go memory usage reduced compared to the JVM.
Compacting reduces memory usage - that's why it's called compacting.
The JVM uses a lot of memory a) because it's tuned for servers and not for low memory usage and b) because Java is a poorly designed language without value types.
No, it reduces memory fragmentation, which is why it's called compacting and not compression.
I do agree that the lack of value types is a big contributor to why Java uses so much memory. But it's not a server tuning thing that makes the JVM lean memory heavy.
The JVM uses moving collectors and that is the big reason why it prefers having so much memory available. Requesting and freeing memory blocks from the OS is an expensive operation which the JVM avoids by grabbing very large blocks of memory all at once. If you have a JVM with 75% old gen and 25% new gen, half that new gen will always be empty because the JVM during collection moves live data from one side of the new gen to the next. And while it does that, it slowly fills up old gen with data.
Even more modern collectors like G1 prefer a large set of empty space because it's moving portions of old gen to empty regions while it does young collection.
As I mentioned, the difference here between the JVM and python or go is that python and go do no moving. They rely heavily on the malloc implementation to handle grabbing right sized blocks from the OS and combating memory fragmentation. But, because they aren't doing any sort of moving, they can get away with having more "right sized" heaps.
I wonder if frameworks like dotnet or JVM will introduce reference counting as a way to lower the RAM footprint?