Color coding OOM (order-of-magnitude) is smart, but the colors chosen seem quite hard to distinguish. Consider picking different default colors; also consider adding a redundant representation of scale, like an integer representing OOM scale.
It is strange that a thoughtful design around OOM would also choose keep so many digits. If the goal is to summarize, then please throw those extra digits away. (e.g. 10.3992s is specifying the time down to the millisecond, but those extra ms are not meaningful at the second time scale.)
What is this about the runtime polling tasks? Does that happen in Tokio? Why? I am only familiar with 2 async runtimes (node, vert.x) and I was under the impression that neither of these "poll their threads" in any meaningful way. Threads (or process or fiber or whatever you want to call it in this context) never initiate action on their own. They are ALWAYS waiting for something to happen to them, and that includes timeouts, which would be triggered by passing a thread a clock tick. The runtime's job is to centrally manage resources that must persist between thread behavior, so at most it is going to be polling external resources, and not it's own threads.
Or maybe I don't understand what polling means here? I would interpret it as meaning "keep checking a well-known place for changes, and then do something if a change is found". But since async threads can't initiate any change on their own, this is nonsensical.
(primary author of the console here) you're right that there are too many digits of precision right now...the reason for that is that it's actually _not_ a thoughtful design at all, though I appreciate you saying that it is; I just picked an arbitrary number when I was writing the format string and didn't really think about it. We probably don't want to display that much precision --- for smaller units, we probably don't want any fractional digits, for larger units like seconds, we probably want two digits of precision maximum.
Regarding the color scheme, glad you like the idea. Because it's a terminal application, the choice of the colors was constrained a bit by the ANSI 256 color palette (https://www.ditig.com/256-colors-cheat-sheet); I wanted it to be obviously a gradient, so I just picked colors that were immediately adjacent to each other in the ANSI palette. It might be better to pick colors that are one step apart from each other, instead, so they're more distinguishable visually...but there's kind of a balancing act between distinguishability and having a clear gradient. We'll keep playing with it!
I'm a little bit of a color freak. Allow me to leave some suggestions :)
- Picking from the 256 color pallete will likely give you colors with different brightness. This may hurt readability of darker colors on a dark background, and may make some color stand out unintentionally. Consider using something like HSLuv [1] to pick colors with the same lightness, then convert to the closest Xterm color [2].
- To make it obvious there is a gradient, I'd pick one lightness (assuming HSLuv) and one saturation (I usually stick to 100%), then pick a distance in hue for each step. For example if I expect to see a maximum of 7 steps on the screen at once, one way is to start at 0, then 30, then 60, etc. You may choose to go over 180, but keep in mind 360 will be the same as 0 so maybe stop at 240. Note how by picking adjacent colors from the table you are still picking a distance, but the distance is too small so it's hard to see.
- You may want to choose a different starting point than 0, and maybe different direction for the steps, depending on whether you want the colors to "mean" anything. For example red is commonly associated with warning, so you can arrange to have the top of the range aligned with red. Or arrange to avoid the red region if you don't want that association.
> we probably want two digits of precision maximum
I think it depends a lot on jitter in the system. Sigfigs are one kind of error bar, one where I often have to haul my coworkers or myself out of trying to read things into the data that aren't there.
There are times where a 10ms change in a 2 second response actually matter to me, because that's half a percent and not all improvements which are easy are also straightforward. Sometimes you're scrambling for 3% here and 2.2% there.
But if the noise in the system is ±50ms then people declaring that they've shaved 15ms off of response time are likely deluding themselves and then deluding the rest of us.
I know how to do some of these things by hand, I'm not sure how you automate them, or in the case of a dashboard, typeset them.
> We probably don't want to display that much precision --- for smaller units, we probably don't want any fractional digits, for larger units like seconds, we probably want two digits of precision maximum.
I really like the way Haskell's Criterion library formats numbers. It always displays four digits total and selects the SI prefix appropriately. I've ported the algorithm to C here, feel free to use it as an inspiration: https://gist.github.com/pkkm/629a66d47ecd16aa89e8b67ba5abd77...
Quite a lot of terminal emulators support 24bit TrueColor escape codes, so it might be worth using that along with a color space designed for visual intensity corrolation (there's a lot of study in this for e.g. heatgraphs for maps)
Yeah, currently, the console knows how to detect TrueColor, but in this case, I just used the ANSI 256 palette rather than picking a better one when TrueColor is available...we should probably fix that!
Side note, it turns out that detecting what color palettes a terminal supports 24-bit colors is surprisingly fraught. There are a couple env variables that may be set...but not every terminal emulator will set them. And then you can use `tput`...but the terminal may not have correct data in the tput database. So that was fun to learn about!
Consider going for one of the Colorcet[0] maps if you detect you can display them. They are really useful to have a neutral view of the subject. I suggest log-scale before applying the colormap for order-of-magnitude visualization.
There's a Rust crate for these (I forgot the name).
I'll answer the polling question. The Tokio runtime (and async rust in general) works a bit differently than other async runtimes like node. With node, callbacks are provided and executed when an OS event is received. With Tokio, there are no callbacks. Instead, async logic is organized in terms of tasks (kind of like async green threads). When the task is blocked on external events, it goes into a waiting state. When external OS events are received, the task is scheduled by the runtime and eventually the runtime "polls" it. Because the poll happens in response to an OS event, most times, the poll results in the task making progress. Sometimes there are false positive polls.
This page goes into a bit more depth and shows an example of how one would implement a (very simple) runtime/executor: https://tokio.rs/tokio/tutorial/async
I read your link earlier today, and have been thinking about it; I particularly like the pedagogy of rebuilding it "in the small" with your MiniTokio example.
I don't know Rust. If I had to guess, this means that you've reused Rust's threads for tasks, such that they may not be done computing when a resource is available? In any event, I want to circle back to the OP, and note that runtime visualizations like this are awesome, and conversations like this is why. I personally don't think anyone spends enough time dwelling in their runtime(s), and certainly no async runtime has good visualizations, so its pretty cool to me that tokio-console is taking the lead here. I've been bullish on Rust for 5 years; maybe it's time to try it out for reals.
The purpose of async is mostly to avoid OS threads, and rust decided not to go down the route of implementing user space threads.
Instead, for async, rust implemented the ability to basically encode a functions stack frame and instruction pointer into a "normal" (but opaque) struct. What an async runtime like tokio does is (through a few levels of useful indirection that I won't talk about) store a list of these structs, and decide when it's a good idea to "call" one of them. When called, the structs either return a final value, or return a value saying "call me again later", in which case the runtime presumably puts it back into it's list of structs and calls it again sometime later.
Figuring out when to call it is left up to the runtime, but the useful ones will do things like record what operation it's waiting for and call it when that operation is ready.
> rust decided not to go down the route of implementing user space threads
Rust had (optional) user-space threads a long time ago, but that was removed in the pre-1.0 days as it added a lot of complexity and had some unavoidable performance loss even when opting for native threads (it forced dynamic dispatch on anything related to threading or I/O). There was a lot of discussion here but eventually it was declared that the OS thread scheduler was in fact perfectly capable of handling large numbers of threads and that virtual memory mapping meant the stack space allocation for each thread wasn’t a big deal and so green threads were removed.
I sometimes wonder what is the fundamental distinction between a callback-based API and this wakeup-based task API. I guess the main difference is that in a callback you generally provide the result as an argument whereas with a wakeup-based API you just wake the task and it has to look for the result in some stored state somewhere.
But ultimately both of them take the "rest of the computation"/continuation and store it somewhere (i.e. on some sleeping task/callback list) to be awakened/invoked later.
It is fairly subtle and mostly an implementation detail. In Rust, the concept of "polling tasks" was very exposed before the async/await keywords were introduced, so the lingo kind of stuck. There is an argument that we should move away from that lingo now that it is mostly hidden as an implementation detail, but we haven't yet.
@tijsvd mentioned that the callback model usually requires more allocation, which Rust is eager to avoid. I'll add that the wakeup/polling model plays much more nicely with Rust's ownership and borrowing rules. Callbacks usually need to hold pointers to the objects that they capture. In a GC'd language, this usually isn't a big deal, other than sometimes causing some surprising leaks. But in Rust, where the compiler wants to keep track of how long pointers live and which objects are aliased, it gets real ugly real fast. The wakeup/polling model sidesteps this nicely, because the no one besides the task itself holds any pointers to the objects that a task owns.
A waker-based API can fall back to waking all paused tasks in a background process to recover from lost events (epoll overflow or such), while a callback-based API can't "just" do so without (allocation?) cost on the happy path.
Their inherent resilience to spurious wakeups is quite useful in that regard.
They also work with exotic FD's, as long as those can still be registered with epoll. For example, pidfd can be polled for readability (despite any read(2) call failing with EINVAL), triggering when the corresponding process terminated.
I guess the benefit is that at least on Linux pre-io_uring, the async syscall way of doing things was via poll/select/epoll to notice when an fd unblocked, followed by waking whatever corouting/statemachine was interested in that event. It composes quite well.
The callback is really hard to implement without allocating memory for each wakeup. The poll mechanism can simply leave the task in place. I suspect the poll thing is also easier to generate.
Generally the way async Rust works is that you have a Future trait with a poll method, and if you call it the future will attempt to make progress if it can — e.g. if the task is a timer it will check the time and complete the task if so and otherwise return Pending.
However, async Rust includes an additional concept: Wakers. When your runtime (Tokio) calls poll on a future, it gives the future a waker object, and when the future is ready to continue work, something needs to call wake on the waker. Once this happens, Tokio will poll the task again soon, and Tokio wont poll tasks that have not been woken.
For example, for timers, Tokio includes a timer wheel (a sort of priority queue) with all the registered timers, and the timer wheel calls wake on the appropriate waker whenever a timer expires. Similarly with a message passing channel, the sender calls wake on the receiver's waker when a message is sent.
Color coding OOM (order-of-magnitude) is smart, but the colors chosen seem quite hard to distinguish. Consider picking different default colors; also consider adding a redundant representation of scale, like an integer representing OOM scale.
It is strange that a thoughtful design around OOM would also choose keep so many digits. If the goal is to summarize, then please throw those extra digits away. (e.g. 10.3992s is specifying the time down to the millisecond, but those extra ms are not meaningful at the second time scale.)
What is this about the runtime polling tasks? Does that happen in Tokio? Why? I am only familiar with 2 async runtimes (node, vert.x) and I was under the impression that neither of these "poll their threads" in any meaningful way. Threads (or process or fiber or whatever you want to call it in this context) never initiate action on their own. They are ALWAYS waiting for something to happen to them, and that includes timeouts, which would be triggered by passing a thread a clock tick. The runtime's job is to centrally manage resources that must persist between thread behavior, so at most it is going to be polling external resources, and not it's own threads.
Or maybe I don't understand what polling means here? I would interpret it as meaning "keep checking a well-known place for changes, and then do something if a change is found". But since async threads can't initiate any change on their own, this is nonsensical.