Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

sshd is started by systemd.

systemd has several ways of starting programs and waiting until they're "ready" before starting other programs that depend on them: Type=oneshot, simple, exec, forking, dbus, notify, ...

A while back, several distro maintainers found problems with using Type=exec (?) and chose Type=notify instead. When sshd is ready, it notifies systemd. How do you notify systemd? You send a datagram to systemd's unix domain socket. That's about 10 lines of C code. But to make life even simpler, systemd's developers also provided the one-line sd_notify() call, which is in libsystemd.so. This library is so other programmers can easily integrate with systemd.

So the distro maintainers patched sshd to use the sd_notify() function from libsystemd.so

What else is in libsystemd.so? That's right, systemd also does logging. All the logging functions are in there, so user programs can do logging the systemd way. You can even _read_ logs, using the functions in libsystemd.so. For example, sd_journal_open_files().

By the way... systemd supports the environment variable SYSTEMD_JOURNAL_COMPRESS which can be LZ4, XZ or ZSTD, to allow systemd log files to be compressed.

So, if you're a client program, that needs to read systemd logs, you'll call sd_journal_open_files() in libsystemd.so, which may then need liblz4, liblzma or libzstd functions.

These compression libraries could be dynamically loaded, should sd_journal_open_files() need them - which is what https://github.com/systemd/systemd/pull/31550 submitted on the 29th February this year did. But clearly that's not in common use. No, right now, most libsystemd.so libraries have headers saying "you'll need to load liblz4.so, liblzma.so and libzstd before you can load me!", so liblzma.so gets loaded for the logging functions that sshd doesn't use, so the distro maintainers of sshd can add 1 line instead of 10 to notify systemd that sshd is ready.



This seems like a clear case of premature optimization. During the three decades sshd has existed I have never seen a real world situation where the equivalent of Type=exec was not enough.

The time window where sshd is started and not yet ready to receive connections is short, and clients will have a connection timeout orders of magnitude larger. The notify functionality is more relevant for things like Java middleware processes and clients that lack the functionality to poll and wait. Under most situations none of this is relevant for sshd. These patches solve a problem very few people have.

If you really have this problem, the systemd readiness is far from enough to solve the problem. The readiness is sent too early and there could still be permission problems that would cause sshd to be ready but the connection to fail. Even more relevant is local firewall rules that are completely out of scope for a readiness check!

Polling for readiness is the only robust way.


> The readiness is sent too early

What makes you say that? The readiness notification is sent after the sshd has opened the listen socket, it literally is accepting connections at that point.


Accepting connections, yes, but for a client that is dependent on being able to establish ssh connections that is not enough. They would want to be notified on the ability to make successful connections.

Keep in mind that this is only a problem in special situations, almost no regular Linux servers carry any services that care about being able to establish ssh connections, that cannot reconnect and use appropriate socket options. For other services than ssh, such as databases, this is much more common. And for those, it is not enough that the server has opened a listening socket.

When building distributed systems, this is something you need to think about. Not so such with local systems. But the same principles apply. And the only robust way is to poll for readiness. Signalling readiness is both complex, when the dependency chains are non-trivial, and prone to error, when readiness signals arrive out of order or are dropped or fail for some reason. This could be because of operations failure but make for hard to debug cases. Dependency chains that mysteriously stop because of permission problems with out of band traffic is both classic and unnecessary problem.

All of this complexity go away when polling. This is why you should adopt this design in the somewhat unlikely case you have clients that depend on being able to make ssh connections.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: