Getting to know Arch, noticed some inconsistencies

Hi guys,

couple of days ago I decided to try out Arch Linux for the first time.

After installing a system, and adding a regular user I discovered I couldn't log in as a regular user from the virtual terminals, while ssh logins worked fine. After some digging, I discovered that the problem was the user's shell -- I assumed, incorrectly, that typing `which bash` would yield a valid path to bash to give to useradd(8), but it returns /usr/bin/bash, while /etc/shells lists only /bin/bash. Ok, I fixed that small problem quite easily. However, after these first impressions, questions popped up in my mind about some of the inconsistencies I noticed in the system defaults:

1) Why is GNU bash installed as /usr/bin/bash, then symlinked as /bin/bash, while zsh gets installed directly as /bin/zsh?

3) Why is some of the default /etc/pam.d/* configuration structured, but most is not? For example, what's the purpose of a config file named /etc/pam.d/system-remote-login in the base install? sshd config does not include it -- actually, no PAM config file in the base includes it.

Re: Getting to know Arch, noticed some inconsistencies

Lone_Wolf wrote:

1) likely has to do with the upcoming switch from /bin to /usr/bin . GNU bash is ready for that, zsh doesn't seem to be.

Thanks for that insight, Lone_Wolf. Is there a plan of this switch posted somewhere? If that is indeed the case, and all binaries are being moved to /usr/bin, so GNU bash is now installed by default as /usr/bin/bash, it would make sense to me, that /etc/shells and adduser defaults configuration should also be updated with the new path.

I asked about this on IRC first, the guys there pointed me to the bug tracker, but my ticket regarding bash and /etc/shells (https://bugs.archlinux.org/task/33677) got closed promptly, and the developer wouldn't provide the rationale for having the /bin/bash symlink, and for ssh logins not being checked against /etc/shells, saying that these are "general questions" (although the questions were rather specific), and that I should "seek help" on forums and IRC.

I sent him an email asking if he would be so kind to provide an explanation in this forum, which I sincerely hope he will.

Re: Getting to know Arch, noticed some inconsistencies

Agreed, the ticket touched on some points that probably should have been split in to multiple tickets, but I expected the Arch devs to be less dismissive and more into open discourse.

So, bash has been moved to /usr/bin/bash, two weeks ago or so, but devs intend to keep referencing it indirectly through the /bin/bash symlink by default. The question the devs seem to be avoiding to answer here is -- why? What's the gain?

Also, none of the devs have yet addressed the (somewhat related) questions about PAM config. The "philosophy" of Arch is, ostensibly, to keep things "simple", "consistent", "minimal", "without unnecessary additions" (those are the phrases I've seen used heavily in the "The Arch Way" wiki pages). Looking at Arch's /etc/pam.d/* I don't see consistence in the configuration layout (some files use the hierarchical approach and others use the flat approach), and some config files, like system-remote-login, indeed seem like unnecessary additions, if they get installed but never used.

I could raise another ticket regarding PAM config, but if the devs are unwilling to communicate and explain their decisions to the users, which is now my first impression of the Arch community, it just might be a waste of effort.

Re: Getting to know Arch, noticed some inconsistencies

vv wrote:

I expected the Arch devs to be less dismissive and more into open discourse.

I did too when I first started using Arch.

From their perspective, they have to deal with an unending wave of people who fail to understand technical issues or the philosophy of the distro. In general such situations lead to ingroup/outgroup psychology and all that it entails. They simply do not have the time and inclination to consider everything that comes their way, and, while it would be better it, is not reasonable to expect that they should. They are all volunteers who contribute considerable amounts of time and effort to the community.

Keep in mind as well that "the devs" are individuals. Some of them are much more approachable than others. Don't let the less-than-friendly ones discourage you.

If you have identified a real issue, pursue it with calm and rational arguments (and patches... devs LOVE patches) until it gets the attention it deserves or gets dismissed by the majority. In the latter case, fix the package or whatever yourself and make it available to others. If it really solves a problem then the community will adopt it and eventually the devs will reconsider it.

Of course, if nobody cares at all, just use the fix yourself and let it go.

Re: Getting to know Arch, noticed some inconsistencies

I agree with all your points, and I completely understand the devs' perspective.

From my perspective, on the other hand -- I'm evaluating a Linux distribution. Arch claims simplicity and consistence, but I take a quick first look at /etc and I'm not impressed at all. It doesn't seem streamlined, it doesn't seem consistent, it doesn't follow the "principle of least astonishment", and it doesn't seem secure by default. So the whole story about "The Arch Way" seems, on first impression, a bit of a hollow boast. So I say, ok, this is an open source project, let's try to help out, go to IRC, go to forums, file bug reports, the usual stuff. It turns out that it's very hard to get involved, very hard to find a dev with communication skills, and it's very hard to get questions answered on either forums, IRC or bug tracker. And these are questions about the Arch base system (the "filesystem" package), not about upstream, or some add-on package from AUR.

So, on first look, the project does not deliver on their bold claims, and it's hard for outsiders to get involved. How does one get enticed to continue using it?

Re: Getting to know Arch, noticed some inconsistencies

vv wrote:

So, on first look, the project does not deliver on their bold claims, and it's hard for outsiders to get involved. How does one get enticed to continue using it?

Because, for many of us, Arch is a delight to use and maintain I don't even need to add "compared to other Linux distros/OSs" because one -Syu a day is all it takes to have a sweet, day to day desktop OS.

Re: Getting to know Arch, noticed some inconsistencies

vv wrote:

From my perspective, on the other hand -- I'm evaluating a Linux distribution. Arch claims simplicity and consistence, but I take a quick first look at /etc and I'm not impressed at all. It doesn't seem streamlined, it doesn't seem consistent, it doesn't follow the "principle of least astonishment",

Ok so you are even more OCD-ish than most of us here, good.

and it doesn't seem secure by default.

Where was it claimed to be secure?

So the whole story about "The Arch Way" seems, on first impression, a bit of a hollow boast.

A philosophy is just that, a philosophy. It doesn't make any claims about the quality of the project, which you seem to think.

So, on first look, the project does not deliver on their bold claims, and it's hard for outsiders to get involved. How does one get enticed to continue using it?

Xyne answered your questions already, there is no need to keep reiterating your frustrations or you may be perceived as a potential troll.

Re: Getting to know Arch, noticed some inconsistencies

mariusmeyer, you will agree that configuring user login and authentication is part of system maintenance. Can you perhaps enlighten me by _technically_ explaining why user authentication (PAM) config is "delightful" to maintain in Arch? See my question (points 2 and 3) at the top of the thread for details.

Re: Getting to know Arch, noticed some inconsistencies

Re: Getting to know Arch, noticed some inconsistencies

litemotiv, Meyithi, I'm not trying to troll. I had three questions. Xyne provided assistance with one of them, and for his valuable insights, and his initiative I kindly thank him. Waiting for someone to address my other two questions (about PAM config), I did wonder a little bit off-topic. Can you, perhaps, offer some insight on the unanswered questions?

Re: Getting to know Arch, noticed some inconsistencies

Ok, since I got no answers to this, I decided to try and answer my own question. What puzzled me most was why is this file called /etc/pam.d/system-remote-login even there?

Try googling for this file name. Besides Arch, you'll find that Gentoo also has a configuration file of the same name. Gentoo's /etc/pam.d/system-remote-login looks like this:

auth include system-login
account include system-login
password include system-login
session include system-login

Feel free to can compare that with your /etc/pam.d/system-remote-login.

I'll try and tell you what I think happened. Devs can feel free to can fill in any mistakes I made.

There was, let's assume, a dev, I'll call him Dev1, who created the pambase Arch package. Let's assume, without meaning to insult anyone, that this developer was just a little bit lazy, or he was short on time, and copied over files from Gentoo's pambase package as a starting point. Nothing bad in not starting from scratch, this is an open source project after all.

And Gentoo's pambase is okay-ish, I guess. It has this neat hierarchy, where, for example, you have a config file for login which adds some login(1)-specific modules, and then includes system-local-login for all four management groups that PAM provides (auth, account, password and session). system-local-login in turn, as we have seen, includes system-login for all four mgmt groups, which in turn includes system-auth, etc. Other configured local login configs, such as for logging via a DM, also include system-local-login for all mgmt groups. This design makes it easier to deploy authentication changes across all configuration -- if we would want to add a new login module for all login types, we could simply add it to system-login. A mechanism for having easy separation of local and remote login config is provided, via system-{local,remote}-login files, but it is not yet used, both files are exactly the same.

If you take a look at Arch's /etc/pam.d/, you'll see that it uses this same design.

Let's further assume that Dev1 did not provide /etc/pam.d/sshd, and that this file is a part of openssh package. Let's call the openssh package maintainer Dev2.

Here's Gentoo's /etc/pam.d/sshd:

auth include system-remote-login
account include system-remote-login
password include system-remote-login
session include system-remote-login

It again follows this simple pattern -- it just includes system-remote-login, which includes system-login. So, Gentoo has no PAM modules that are specific to ssh sessions, but if such need arises, they could be simply added to this file.

Ouch. So much redundant stuff here! This file is not integrated at all with the rest of the PAM config from pambase! No includes, just flat listing of modules. Not nice at all. Not "streamlined" by a long shot.

Let's assume that Dev2, who owns the openssh package and provides the /etc/pam.d/sshd could not consult Dev1 (who owns pambase) because by that time Dev1 has left the project. Dev2 didn't bother to integrate his configuration with the rest of PAM config, he just dropped a big monolitic config file for sshd into /etc/pam.d/. And he forgot to include the PAM module that checks if the user's shell is allowed via /etc/shells. He wouldn't have made this omission if he integrated his config with pambase properly, because the common config in system-login (see above) covers that.

This had some consequences. For example, when the switch to systemd came along, devs would have had to open two separate tickets for it's pam configuration, since now that change touches both pambase and openssh packages. Furthermore, on all users' systems both packages had to be updated! This scenario would apply to any PAM config change that also affects ssh logins. In practice, for every PAM config change now you have to update the openssh package, too.

Coincidently, Dev2 was the dev who closed my ticket regading /etc/shells. He didn't want to respond to my question about sshd logins not getting checking against /etc/shells, which is configured as a part of the package he maintains. I sent him an email, asking him to provide the answer in this open forum. He didn't answer my email. He also didn't show up on this topic, yet.

You can draw your own conclusions about the quality of this work. But what frustrates me really is this kind of attitude. Someone asks a guy a perfectly reasonable question, regarding his own work, and he just acts as if he didn't hear it. I know the devs are volunteers so I will not comment on professionalism, but is this even adult behaviour?

If covering dev's own ass by avoiding questions about his work is more important to him than getting shit done right for everyone's sake, then I'm not sure if he is doing the community a favor here.

Now, let's fix this.

First, let's remove from sshd PAM config all the modules that would eventually get loaded if system-remote-login config is included, simply by eyeballing the files that get included down the chain from system-remote-login. We narrow the list to 5 modules:

Actually these are 4 modules, one is commented out. But let's say that those are sshd-specific. And now let's reference our old friend from the beginning of the story, that little file called system-remote-login to pick up the rest, and to give it, for the first time in his life on Arch repositories, and on many Arch users' hard drives, a purpose! So we get this new /etc/pam.d/sshd:

Restart sshd, can I login? Yes indeed I can. Can I login after removing my shell from /etc/shells? Indeed I can not, which is the desired behavior. Say, that took about 10 minutes, all in all. (It would be nice if anyone else would be so kind to test).

Now I wouldn't call this "streamlined" either, but it's better. It's clearer. It's more consistent. And global PAM config changes would not require updates to openssh package anymore.

Now let's diff this against the current sshd config, call that a patch (because, as Xyne said, devs LOVE patches) and file a new ticket, this time against the openssh package. And see what happens.

Re: Getting to know Arch, noticed some inconsistencies

I couldn't tell if that was sarcastic so I'll just add that OCD + understanding of system security = usually a good thing. Whether the LHS is fully met or not remains to be seen.

Meyithi wrote:

Obvious troll is boring...

Posting just to call someone a troll is itself trollish. I hope the irony is not lost on you.

@vvExplaining the technical points of the problem and providing a patch is the right way to go. Griping will only create friction and work against you. I understand the frustration but I see where this is headed if you don't let it go. I'd rather see you stick around and provide patches to improve the distro.

Re: Getting to know Arch, noticed some inconsistencies

I couldn't tell if that was sarcastic so I'll just add that OCD + understanding of system security = usually a good thing. Whether the LHS is fully met or not remains to be seen.

Yes that's what i meant, it's a good thing if you use it in a productive manner. Arch was built on OCD (and coffee), but in a social context it's something to be careful with.

Xyne wrote:

Explaining the technical points of the problem and providing a patch is the right way to go. Griping will only create friction and work against you. I understand the frustration but I see where this is headed if you don't let it go. I'd rather see you stick around and provide patches to improve the distro.