You really should consider watching the youtube talk that was linked in some sibling comments, it explains it in more detail than I could in just one post. For me personally, the real bad issues are things like the core protocol being synchronous, the coordinates being limited to 16-bit, the inherent raciness and insecurity of various things like window properties and server grabs... There is a lot of legacy functionality there too like colormaps, window borders, bitmap fonts, all the core drawing primitives, all the core input stuff.... Newer applications are not using any of that, and often with that legacy stuff the the only specified behavior for edge cases that clients expects is "do whatever Xorg does" which makes a rewrite pretty impractical. It would be interesting to see a secure rewrite of the X server in Rust or some newer language like that, but doing that would probably take many years for little benefit, I'd advise against it.
Side note, I don't get the hate for dbus, it's a rather simplistic message bus, orders of magnitude smaller than the X server. It would be much easier to implement your own dbus daemon for example.
From what I got from the video, the main complaint is that the X.org reference implementation has gotten really crusty and hard to maintain. If so, maybe the solution there would be to do some housekeeping and start deprecating features no one uses. Maybe that could include factoring legacy/obscure protocols and code-paths into separate code modules, off the server's "happy paths," which these protocols' downstream consumers can take over maintaining (if they really still need them).
This doesn't call for ditching X11 in my mind.
> For me personally, the real bad issues are things like the core protocol being synchronous, the coordinates being limited to 16-bit, the inherent raciness and insecurity of various things like window properties and server grabs...
Why can't an extension offer a way for clients to establish asynchronous communication channels to the X server? Why can't an extension offer 64-bit coordinates? Why can't an extension offer a way for an authorized program to take care of guarding and serializing access to window properties and orchestrating server grabs? Why do we need to break the world to have these things?!? These may not be trivial undertakings, but I doubt they would take anywhere close to the amount of work required to upgrade every graphical program and toolkit in UNIX-land to use a wholly-different _suite_ of input/video multiplexing systems (which on a given day will be only 95% compatible with one another in expectation).
Also, it's not like Wayland is destined to be less crufty than X11. I wouldn't be surprised at all if all of the complexity in X.org today returns to Wayland compositors by way of a bunch of all-but-required Wayland extensions that get shoe-horned in over the years. So if we're going to be shoe-horning new features into existing systems, we might as well do it on the devil we all know (or perhaps we should solve this once and for all by creating an "X12" protocol in a way that shoe-horning is painless and won't lead us to cruftiness again).
> Side note, I don't get the hate for dbus, it's a rather simplistic message bus, orders of magnitude smaller than the X server. It would be much easier to implement your own dbus daemon for example.
It's not simplistic for what it does, and its developers have a horribly-misguided "put-it-in-the-kernel-because-performance" development ideology that belies a profound lack of understanding of why or how dbus isn't fast enough for their purposes. My biggest turn-off is the fact that it doesn't do anything that I can't already do faster and cheaper with a RAM filesystem of named pipes and UNIX domain sockets.
* Want user/system namespaced paths? Create a directory for each users' endpoints that's separate from other users' endpoints, and have a distinct system/ directory that only authorized users can explore. Leverage filesystem hierarchies and permissions to communicate which endpoints belong to the same service, and to control who can access them.
* Want to register a service endpoint for suspending/shutting-down your laptop? Create a directory under system/ whose group ID is the group of users who are authorized suspend/shutdown, put two "suspend" and "shut-down" named pipes in them, and have a suspend/shutdown daemon just do blocking reads from them. Once a byte arrives on the "suspend" pipe, execute suspend-to-RAM. Once a byte arrives on the "shut-down" pipe, execute shutdown.
* Want to register a service endpoint for sending desktop notifications? Make a "notifications" directory in the user's service endpoints directory, and put a UNIX domain socket in it. Have the notification daemon listen on this socket, and simply pop up a window whenever another program connects to it and sends a properly-structured message (note that that other program must have permission to traverse the service directory to access this UNIX domain socket to do so).
* Want introspection on how to form that message? Have the daemon that implements the endpoint write out a symlink to its documentation in its service directory, which you can just `cat` or `more` to figure out how to talk to the service.
* Want something really elaborate, like sending a video stream? Transfer a file descriptor to the service provider via the UDS and then pipe the audio/video data in that way.
So, yeah -- dbus doesn't need to exist in order for us to have the things it offers.
What you're saying is mostly what has happened already except the work has just been done outside the X server. The features in X that people don't use are already considered deprecated, and factoring the legacy parts off into a separate code module is essentially what XWayland is anyway.
If you added all those things as X extensions, it would essentially be the same thing as Wayland, because clients that wouldn't use them would still be broken, and every graphical program and toolkit would still be need to be updated to use them.
Your suggestions for dbus would work for some applications but would not really work for other things that a message bus handles like multicast, global message ordering, and resource accounting. Plus GNOME and KDE adopted dbus specifically so they could get away from having to pass around random sockets in folders everywhere. I assume by "put-it-in-the-kernel-because-performance development ideology" you're referring to kdbus, which was an alternate implementation not made by the original dbus developers, and is now a dead project and is not really a thing anymore. Please don't get those things confused. Of course the reason they could do that is because dbus is also just another protocol with a reference implementation, and you could make another implementation that works closer to what you describe and maybe gets 80-90% of the way there depending on some changes in the kernel, for example I saw a hacky dbus fuse filesystem a while ago: https://github.com/sidorares/dbusfs
> If you added all those things as X extensions, it would essentially be the same thing as Wayland, because clients that wouldn't use them would still be broken, and every graphical program and toolkit would still be need to be updated to use them.
There's a massive difference between extending the X server and having each window manager implement all the trappings of an X server as a library. Namely:
* X remains the "narrow waist" for video/input multiplexing. GUI "policy" infrastructure -- window managers, panels, notification services, and so on -- remain separate programs, with separate maintainers, to be mixed and matched downstream as needed. Moreover, all these programs keep working.
* By remaining a separate X server program, we keep mechanism and policy cleanly separated. GUI "policy" infrastructure can't impose itself systemically on other GUI "policy" infrastructure, which is a good thing because all these GUI "policy" infrastructure authors tend to think their way is the best way and how dare anyone question it or resist it (see also GNOME). X keeps me and mine safe from their idiocy.
* The barrier-to-entry for creating new "policy" infrastructure remains low, since you can run these programs without coupling them to a particular compositor.
* Changes to X's rendering infrastructure get incrementally deployed. No change in any workflow is required; toolkits and programs opt-in to the new rendering infrastructure as they need to. Programs that don't opt-in keep working until the old code paths get dropped.
* Non-display services of X get preserved, like xprops, xinput, etc. All xclients keep working. If desired, these can be policed through a separate opt-in extension. Existing IPC conventions like ICCCM and NetWM keep working, so all the downstream tools that use them keep working.
> Your suggestions for dbus would work for some applications but would not really work for other things that a message bus handles like multicast, global message ordering, and resource accounting.
Nonsense.
You can multicast messages from one process to many processes via a UNIX domain socket trivially -- just send the damn message to each recipient! It's not like you're going to have 10 million clients, so copying the data isn't going to be that bad (and, the service endpoint can always throttle clients). But, sure, let's suppose the message you're trying to send is gigantic, and you do need to send it to lots of clients. You can just store it as a file (you're doing this anyway if the message is truly that big) and send each client a read-only file descriptor to go and consume it at their own pace. If you're using a file at least, all your clients will hit the same cached pages in the kernel, so you're no longer making N different copies of the data (the kernel will take care of implementing the right caching strategy for you). If you're streaming data, you could simply buffer it to a file and treat the file as a ring-buffer, and still hand out read-only file descriptors to it to downstream clients.
Global message ordering and message dependencies is also easily solved without dbus -- just implement an "ordering" service adapter. The adapter writes its own UDS to the place where its upstream services' UDSs live, and it takes care of marshaling requests and replies to and from the upstream services according to some ordering principle you require. For example, if you have a service for shutdown/suspend, and a service for logout, you could implement a small ordering adapter that prevents messages to shutdown/suspend from being delivered if the user is in the process of logging out. I'd imagine that for a DE, you could simply have a singleton ordering service adapter that determines what services get to be accessed under which circumstances (thereby cleanly separating the task of systems integration from the task of providing the individual service).
Resource accounting is similarly straightforward. Just like the "ordering" service adapter pattern, you can also create a "resource usage" service adapter pattern. For example, you can ensure that the volume increment or decrement requests to your sound daemon arrive at a fixed rate, no matter how many requests come in. As another example, if the service is streaming data, you can use a service adapter to monitor how quickly clients are consuming versus the service producing, and induce back-pressure on the service to hint that it should down-sample if clients are too slow.
Because everything is represented as files, I can do those last two things trivially with shell scripts. No need to take over the init process (cough systemd-logind cough), no need to implement a whole wire format and marshaling library and stub-compiler, no need to create language bindings, etc. Files, directories, named pipes, UNIX domain sockets, and a humble script to set desktop-wide policies on inter-service interactions are more than adequate. But noooooo, we had to build dbus and all of dbus's infrastructure.
I honestly believe the authors of dbus simply lack imagination. Like, we have all this wonderful battle-tested POSIX IPC infrastructure sitting around waiting to be used that they don't even have to maintain, and the kernel makes a fine I/O multiplexer and request broker. Why not use it to its fullest potential? It'll save time and effort, and you won't need any specialized SDKs or tooling to interact with services.
I don't want to say that I think the dbus authors are, well, stupid. If there's something that dbus does that well and truly cannot be done as described above, I'd love to know what it is, and why it justifies all the complexity of re-implementing POSIX IPC analogues in a bespoke system. But I've been writing software for over 20 years, and I've been around the block plenty of times, and this entire project smells like something someone would have written if they simply were not familiar with what their runtime environment could already offer them.
> Plus GNOME and KDE adopted dbus specifically so they could get away from having to pass around random sockets in folders everywhere.
So instead we should just implement worse-performing analogues of most of the POSIX IPC primitives in userspace and pass around service endpoints instead? Come on now.
> you're referring to kdbus, which was an alternate implementation not made by the original dbus developers, and is now a dead project and is not really a thing anymore. Please don't get those things confused.
Thanks for correcting me. I wouldn't want to hate on people for the wrong reasons ;)
------
Anyway, we've been going back and forth for a while. I'm convinced now that Wayland is just an instance of CADT and doesn't solve anything that couldn't have been solved with a less-glamorous but less-effort X extension. But whatever -- the X.org and fd.o developers are free to do whatever they want, etc. etc.
I actually like the X11 model, and wouldn't mind taking a crack at writing a Wayland compositor that simply back-ported all the non-graphical aspects of X as a Wayland extension. Then everything I'm using today could, ostensibly, keep working (and I don't have to care nearly as much what the fd.o folks do going forward).
Look, you're a smart and accomplished person and you have some developed ideas of how thing should be done, please don't hate on other open source developers or accuse them of being "idiots" or "CADT" when you yourself acknowledge that you don't fully understand their work. If you have an idea you think is better then you can just do it, you don't need to trash talk other people's work and use insults like "attention deficit teenager" to get your point across. If you want your X programs to continue working, you don't need to write a Wayland compositor, you can just keep using X. The only reason to write a Wayland compositor would be if you wanted to use Wayland clients, which would not have access to any of the X protocol features anyway.
If you don't care about policies then all those things in X can be a good thing, but if you do care about policies then Wayland could allow for a better design, at least it seems that's what GNOME and KDE are aiming for anyway since their policies are very well established at this point, and they don't really seem to care about breaking ICCCM and other such things.
As for dbus, your solutions would work for some things, but would not have exactly the same semantics as dbus and would come with their own set of issues, and requires building several more infrastructure pieces, some of which you just described. You could build those but it likely wouldn't fit the same use cases as dbus. If you're sending messages that you expect other clients to parse then you still need to agree on a wire format and marshaling library, you can't get around that. If you ask me dbus itself doesn't require much infrastructure at all, you should consider reading the source code for the dbus reference implementation at some point because it's actually pretty small and stable. And I don't understand what you mean by re-implement POSIX IPC analogues, dbus is essentially just a wire format for Unix domain sockets and a message bus that routes the messages, it doesn't re-implement anything. If you want to use dbus from shell scripts, you can use tools like dbus-send and busctl, or you can try to use something like that dbus fuse filesystem -- the nature of dbus makes it map pretty well to that, there's no reason you can't have both a message bus and an easy interface to access from shell scripts.
(Also just another nitpick here, the systemd developers are not the dbus developers, and systemd-logind doesn't take over the init process, that is its own smaller daemon)
If you want to read more, see some comments from the original dbus author:
Can you think of even a single thing dbus can do that my approach cannot do? Emphasis on cannot here -- if you do reply to this, I expect you to prove that the thing cannot be done by any simpler means. If not, then why does dbus need to exist? Better question -- why are people who insist on writing software that doesn't need to exist given decision-making powers in fd.o? Software is like a form of pollution -- more code means more bugs and more security holes (and dbus isn't immune [1]). Any greenhorn developer can write lots and lots of code; it takes wisdom and experience to avoid writing code. So if people who don't grok this are running fd.o, why should I trust anything fd.o produces?
Before you try and tone-police the above, you should know that it is fd.o that needs to convince me to venerate their software artifacts. People writing more code isn't by itself praiseworthy -- code is a goddamn liability, so it had better have a good reason to exist and (in dbus's case) have a very good reason to be widely depended-on. Just because you happen to like or use someone's code doesn't mean that it is any good.
> And I don't understand what you mean by re-implement POSIX IPC analogues, dbus is essentially just a wire format for Unix domain sockets and a message bus that routes the messages, it doesn't re-implement anything.
I guess if you didn't understand POSIX IPC, you wouldn't see how this sentence is an oxymoron. The kernel itself gives you all the trappings of a message bus for free. You don't need a wholly-separate daemon and wire format spec.
Also, the only people who seem to use dbus's wire format are dbus clients. Even when dbus was new, there were already widely-used and well-understood formats for representing structured data (e.g. ASN.1, typed netstrings, S-expressions) that could have been leveraged to make interacting with the service that much more straightforward. But then again, we're talking about people who wanted to re-invent POSIX IPC, so I guess I shouldn't be surprised they also wanted to impose their own wire format on the world.
> If you don't care about policies then all those things in X can be a good thing
I know better than anyone else on Earth what graphical policies are good for me, so I'm going to take this as your affirmation that X is indeed the right tool for the job for people like me who know what they want out of their computers. I stopped using DEs years ago because I got tired of having to fight them all the time to get them to do the things I needed.
I don't understand what you are saying about freedesktop.org. That is just another volunteer run open source organization that hosts projects that are loosely related to open source desktops, you can start contributing to that if you want, or you can not use any of it if you don't find it useful. I'm just here for an interesting conversation, I'm not trying to convince you of anything, and I would rather not continue this discussion if you're going to start throwing around insults and making it personal and accusing other developers of being ignorant or having bad intentions. Please don't do any more of that, it's not interesting conversation and it's against the rules here. You're better than that. If that's tone policing then I'm sorry but my point is we ultimately can't have a conversation if your goal is to attack other people who aren't even here and tear them down, that just isn't my goal.
I also still don't see what you mean about dbus, the Linux kernel itself doesn't specify a wire format for arbitrary messages, and doesn't specify all the things that you need to get the complete functionality of a message bus. Maybe you could get that with another operating system that is based around message passing but Linux is not that. The methods you describe could technically be done without a daemon, but they still require a lot of additional code to set up a bunch of files and sockets and enforce ordering, security, etc, which could also contain bugs. You could tell the applications to implement all that themselves or you could put it all in a daemon which is mostly what dbus does anyway, and by doing it in one daemon it totally eliminates a certain class of race conditions and synchronization issues. Again please refer to the comments by the dbus developer that I showed, this conversation is not new and already happened years ago. If you want to store ASN.1 or S-expressions in a d-bus message you can do that pretty easily. And if you really believe that your solution could work then I would encourage you to develop a dbus implementation that works like you describe and then test to see if it works exactly the same and doesn't break existing setups. But I don't think this would really work, you wouldn't really be saving many lines of code, and in particular multicast and service activation would be pretty hard to do in the way dbus does it without a central message bus.
If you don't agree with GNOME or KDE's policies and you want to implement your own IPC then that's great, I support you doing what you need to do, however they chose dbus a long time ago, and currently it's looking like X is not the right tool for them anymore, so you may just have to accept your differences and move on.
I see that you didn't take me up on my challenge to prove that dbus does anything we couldn't easily do with bread-and-butter POSIX IPC. Just regurgitating something I could have just read on the dbus homepage isn't fooling anyone.
Look, I have very strong opinions on what software I consider worth running on my computer, because I've been doing this for a long, long time. I also have very strong opinions on how to go about solving the problems that dbus, udev, systemd, logind, PulseAudo, and the rest of the fd.o middleware purport to solve. I also happen to strongly disagree with how they go about doing it. But, please don't misconstrue this as me believing that their authors don't have a right to create whatever software they see fit. I put my money where my mouth is on this and write code to do the things I want if I can't bear to use the code they publish -- in fact, I have my own binary-compatible udev replacement waiting on stand-by but ready-to-go in case udev comes to hard-depend on systemd [1].
I normally don't share my opinions publicly like this because it causes people like yourself to crawl out of the woodwork and throw well-meaning but unsolicited advice and github links at me to wrappers and adapters for projects that depend on the very software and the very architectural paradigms I'm trying to avoid. When I try to explain this is not what I asked for, it falls on deaf ears. It's a supremely annoying, frustrating experience.
I don't know if you remember, but the only thing I was trying to find out in this entire comment thread is whether or not Wayland solves a problem that was truly impossible to solve with an X extension. I've already got my answer: no, it does not. So, I think I'm done here.
Side note, I don't get the hate for dbus, it's a rather simplistic message bus, orders of magnitude smaller than the X server. It would be much easier to implement your own dbus daemon for example.