Activity

Historically, Jenkins allowed anyone read access, to the top URL wasn't the 403. The wiki is descriptive, open to edit by anyone, and not kept well up to date, so take its content with a grain of salt.

Daniel Beck
added a comment - 2017-07-20 08:04 Historically, Jenkins allowed anyone read access, to the top URL wasn't the 403. The wiki is descriptive, open to edit by anyone, and not kept well up to date, so take its content with a grain of salt.
The better maintained Jenkins handbook specifically mentions the login URL: https://jenkins.io/doc/book/managing/cli/

Daniel Beck, sorry to say but this is clearly a bug and even one that makes the CLI usage very hard. The fact that `X-SSH-Endpoint` is not reported for unauthorized requests makes impossible to use SSH cli due to:

This means that's impossible to use SSH port without first authentication to Jenkins via HTTP.

The fact that the other ports are exposed is another reason to consider this a bug.

Do I really have to modify our NGINX to add this missing headers in order to make the CLI via SSH usable?

And correction: the header is missing even from instances that allow anonymous access. For example we are using Kerberos based authentication and we allow read-only anonymous access, the header is missing unless we make requests with BASIC-auth (tokens),... which cripples the user experience.

I even tries the nginx approach, validated that it does add the header but jenkins-cli.jar still reports the header as missing, I am wondering what kind of request it does really doing,... even if log level = FINEST it still does not tell much. This start to sound like a call for wireshark... on SSL connections.

Sorin Sbarnea
added a comment - 2017-08-24 11:47 I even tries the nginx approach, validated that it does add the header but jenkins-cli.jar still reports the header as missing, I am wondering what kind of request it does really doing,... even if log level = FINEST it still does not tell much. This start to sound like a call for wireshark... on SSL connections.
add_header 'X-SSH-Endpoint' 'jenkins.example.com:50022' ;

Daniel Beck Thanks for the hint. Still, here are the results after using the /login url and no credentials, still not sending the SSH part.

One interesting remark is that it seems that in this case nginx didn't add the header, but this is my problem which I am going to investigate.

I hope that in the end we will succeed making Jenkins inform the client about the SSH port, one solution that would not need a lot of debugging and tuning at different places. Somehow, the other 3 -Port header do seem to work correctly, so I see no reason why this one would not work.... hopefully out of the box.

Sorin Sbarnea
added a comment - 2017-08-24 13:13 Daniel Beck Thanks for the hint. Still, here are the results after using the /login url and no credentials, still not sending the SSH part.
One interesting remark is that it seems that in this case nginx didn't add the header, but this is my problem which I am going to investigate.
I hope that in the end we will succeed making Jenkins inform the client about the SSH port, one solution that would not need a lot of debugging and tuning at different places. Somehow, the other 3 -Port header do seem to work correctly, so I see no reason why this one would not work.... hopefully out of the box.
+ curl -sD - -o /dev/ null -X GET https: //jenkins.example.com/login
HTTP/1.1 401 Unauthorized
Server: nginx/1.10.2
Date: Thu, 24 Aug 2017 13:05:28 GMT
Content-Length: 0
Connection: keep-alive
X-Content-Type-Options: nosniff
Set-Cookie: JSESSIONID.54ae6=95esznpbhm36f5fbd74;Path=/;Secure;HttpOnly
Expires: Thu, 01 Jan 1970 00:00:00 GMT
X-Hudson: 1.395
X-Jenkins: 2.60.3
X-Jenkins-Session: b2a74a15
X-Hudson-CLI-Port: 50000
X-Jenkins-CLI-Port: 50000
X-Jenkins-CLI2-Port: 50000
WWW-Authenticate: Negotiate
WWW-Authenticate: Basic realm= "EXAMPLE.COM"

I was able to convince jenkins-cli.jar to connect only after manually adding this to my nginx proxy:

add_header 'X-SSH-Endpoint''jenkins.example.com:50022' always;

The always part on that line is key because without it nginx will add the header only on some HTTP return codes which do NOT include 401 and 403 (the ones likely to be reported by /loging url requests).

I think that this is a confirmation that the backend doesn't work as expected and that it should always include the header regardless the response being given.

Sorin Sbarnea
added a comment - 2017-08-24 13:27 I was able to convince jenkins-cli.jar to connect only after manually adding this to my nginx proxy:
add_header 'X-SSH-Endpoint' 'jenkins.example.com:50022' always;
The always part on that line is key because without it nginx will add the header only on some HTTP return codes which do NOT include 401 and 403 (the ones likely to be reported by /loging url requests).
I think that this is a confirmation that the backend doesn't work as expected and that it should always include the header regardless the response being given.

Krzysztof Malinowski
added a comment - 2017-08-24 13:48 While I understand Daniel Beck 's arguments about this being documented, I still find it inconsistent that Jenkins announces CLI port always, but SSH port only under certain response codes.

Sorin Sbarnea
added a comment - 2017-08-24 15:44 - edited Indeed, I suppose that one of the authentication plugins is creating this behaviour because otherwise we would have a bigger number of people reporting this issue.
Yes, sshd is working and I succesfully used it using ssh client. See the output of list-plugins at
https://gist.github.com/ssbarnea/49bbe45d1c3e816605701bd69bb14fcd

But I have some interesting news! It seems that Jenkins reports X-SSH-Endpoint if you ask for ROOT (with or without ending slash). If you ask for /login or /cli it will NOT return it. So your recommendation did mislead me.

What I still do not understand is how it comes that the jenkins-cli.jar reports the X-SSH-Endpoint as missing. This make me think that the test done by the java client is not using the root url and is using the /login one which does not contain the header.

So I have reasons to believe that the broken code is inside jenkins-cli.jar.

Sorin Sbarnea
added a comment - 2017-08-24 15:51 But I have some interesting news! It seems that Jenkins reports X-SSH-Endpoint if you ask for ROOT (with or without ending slash). If you ask for /login or /cli it will NOT return it. So your recommendation did mislead me.
What I still do not understand is how it comes that the jenkins-cli.jar reports the X-SSH-Endpoint as missing. This make me think that the test done by the java client is not using the root url and is using the /login one which does not contain the header.
So I have reasons to believe that the broken code is inside jenkins-cli.jar.

Daniel Beck
added a comment - 2017-08-24 15:53 Right, the expectation is that login is always available.
https://github.com/jenkinsci/jenkins/blob/862de8612966f0acb64e6cf1205f2bc967679547/cli/src/main/java/hudson/cli/SSHCLI.java#L64
If it works for the root URL, then probably only because anonymous Overall/Read is enabled.

From my point of view I find the use of HTTP headers to document the port a bit questionable. The industry standard for this is based on DNS SRV entries

dig _jssh._tcp.subdomain.example.com SRV

See that I didn't use the default `ssh` name because that's already used by ssh protocol and many ssh clients are already supporting it. Because Jenkins SSH is not really ssh, it may deserve its own service name, like "jssh".

Sorin Sbarnea
added a comment - 2017-08-24 16:48 So what should we do now as we know what happens?
From my point of view I find the use of HTTP headers to document the port a bit questionable. The industry standard for this is based on DNS SRV entries
dig _jssh._tcp.subdomain.example.com SRV
See that I didn't use the default `ssh` name because that's already used by ssh protocol and many ssh clients are already supporting it. Because Jenkins SSH is not really ssh, it may deserve its own service name, like "jssh".

Daniel Beck
added a comment - 2017-08-24 17:51 So what should we do now as we know what happens?
Look into whether the SSO plugin does something sane. It might not be. Not the easiest thing to do given Jenkins' authentication process complexity.

This makes me think that this issue has nothing to do with the plugin and that's a core limitation/bug that prevents the SSHD plugin from adding a header, the response being send back before reaching the plugin code.

Sorin Sbarnea
added a comment - 2017-08-25 08:29 I tried to look for all the places where X-SSH-Endpoint is used and found only 3 places and none of them look like something interesting https://github.com/search?utf8=%E2%9C%93&q=org%3Ajenkinsci+X-SSH-Endpoint&type=Code
This makes me think that this issue has nothing to do with the plugin and that's a core limitation/bug that prevents the SSHD plugin from adding a header, the response being send back before reaching the plugin code.
The only headers that seem to be working in this case are those like X-Jenkins-CLI2-Port which are added inside jenkins code, not by a plugin.

Daniel Beck
added a comment - 2017-08-25 09:07 It's implemented as a page decorator, which only works for successfully served pages. Hence the problem with your auth plugin not having a 200 response for /login.

We are talking here about the ldap-plugin not some esoteric one and TBH I am not even sure that getting a 401 response from `/login` page is wrong. In fact that even more HTTP compliant than returning a 200 answer.

Somehow I have the impression that's another bug that feels between the jenkins core and its plugin. Everyone agrees thats a bug but nobody agrees which code is to blame/fix: core, ldap-plugin or sshd-plugin? ... based on the status of the ticket one could even assume that is "Not A Defect".

Sorin Sbarnea
added a comment - 2017-08-25 14:41 We are talking here about the ldap-plugin not some esoteric one and TBH I am not even sure that getting a 401 response from `/login` page is wrong. In fact that even more HTTP compliant than returning a 200 answer.
Somehow I have the impression that's another bug that feels between the jenkins core and its plugin. Everyone agrees thats a bug but nobody agrees which code is to blame/fix: core, ldap-plugin or sshd-plugin? ... based on the status of the ticket one could even assume that is "Not A Defect".

Weird, ci.jenkins.io also uses LDAP and returns 200 for /login (which could in theory be due to Overall/Read being granted, but local tests with internal user DB and without anon Overall/Read also return 200). Are you sure the reverse proxy isn't messing with that?

Daniel Beck
added a comment - 2017-08-25 15:34 - edited Weird, ci.jenkins.io also uses LDAP and returns 200 for /login (which could in theory be due to Overall/Read being granted, but local tests with internal user DB and without anon Overall/Read also return 200). Are you sure the reverse proxy isn't messing with that?

Ruben Stein
added a comment - 2017-09-07 15:03 - edited When I updated one of our machines to 2.60.2 or 2.60.3 my puppet client's CLI authentication through SSH key does not work anymore.
root@testmaster:/var/lib/jenkins# java -jar /usr/share/jenkins/jenkins-cli.jar -s https://testmaster:8443/login -i /etc/puppet/ssl_cli/testmaster-nonuser -ssh -user testmaster-nonuser help
returns code 255 and:
Sep 07, 2017 5:01:13 PM hudson.cli.SSHCLI sshConnection
WARNING: No header 'X-SSH-Endpoint' returned by Jenkins
Regarding a workaround I read adding that header via reverse proxy config. What to do about this if there is no reverse proxy in place? Is this really fixed like this?
Edit: I am using the matrix-auth plugin together with active-directory to authenticate against an AD server.

in one of the older comments (Sorin Sbarnea - 2017-08-24 15:27 comment-311573) it is mentioned that the header was manually added via an nginx in front of jenkins. He mentions that after this the cli was able to connect properly.

Ruben Stein
added a comment - 2018-02-21 08:51 Hey Jayesh,
in one of the older comments (Sorin Sbarnea - 2017-08-24 15:27 comment-311573 ) it is mentioned that the header was manually added via an nginx in front of jenkins. He mentions that after this the cli was able to connect properly.