- Generate the certificates using Lets Encrypt
- Convert the Lets Encrypt certificates to jenkins format
- Start jenkins
- Client p12 certificate file
- Concepts
- Connect jenkins to internal self-signed certificate servers and configure ssl
- Credentials
- Installing the plugin
- Kubernetes server certificate key
- Local cluster
- Pcks certificate without passphrase
- Remote cluster
- Starting a slave
- Tl;dr;
- Conclusion
Generate the certificates using Lets Encrypt
Given:
When you do:
Then:
Convert the Lets Encrypt certificates to jenkins format
Given:
When you do:
Then:
privkey-rsa.pemwill be generated. This is in rsa private key format
Start jenkins
Given:
- you are in the directory containing
jenkins.war fullchain.pemandprivkey-rsa.pemare in this directory
When you do:
Then:
Client p12 certificate file
Using all three files we need to generate client certificate file in PKCS12 format
openssl pkcs12 -export -out cert.pfx -inkey client.key -in client.crt -certfile ca.crt
NOTE: It is important that you provide a passphrase (as you will see later)
At this point, you are ready to add a new Kubernetes client certificate to Jenkins.
Click Add -> Jenkins
Make sure Kind is set to Certificate
Select Upload PKCS#12 certificate and then hit Upload Certificate.
You should see a certificate file selector:
Navigate to the client.pfx file you generated and hit Upload.
Note: You will still see the message which you can ignore:
Enter the password you used for client.pfx . If you provided the correct password you should see the above error message (‘No certificate uploaded’) changed to a warning (‘Could retrieve key “1”. You may need to provide a password’). You can ignore this warning as well.
Concepts
- since Let’s Encrypt installs a bunch of stuff, rather than hacking around with trying to control this, I simply do
everything from Docker. Then I dont have to think about what things it’s changing on my system.
Connect jenkins to internal self-signed certificate servers and configure ssl
Our Jenkins CI server will have to connect to our internal servers such as ldaps, JIRA or IRC etc. These servers will have a self-signed certificate and we will look at how to import these certificates so that Jenkins could connect to them successfully.
We need to import the .cer or .pem file generated by our internal server into the cacerts keystore under jdk/jre/lib/security. To locate where the current jdk is located, you need to follow the symlinks of the java executable. Here is a snippet of how I located my jdk directory. So, my jdk was located under /usr/java/jdk1.7.0_25/.
-bash-4.1$ whereis java java: /usr/bin/java /etc/java /usr/lib64/java /usr/share/java /usr/share/man/man1/java.1 -bash-4.1$ cd /usr/bin/ -bash-4.1$ ls -ltr java lrwxrwxrwx 1 root root 22 Jan 8 07:01 java -> /etc/alternatives/java -bash-4.1$ cd /etc/alternatives/ -bash-4.1$ ls -ltr java lrwxrwxrwx 1 root root 30 Jan 8 07:01 java -> /usr/java/jdk1.7.0_25/bin/java -bash-4.1$
To import the certificate into the java keystore, we will be using the keytool command. Keytool is a key and certificate management utility. Run the following command from a terminal window. You will be prompted for your keystore password. The default password is “changeit” if you haven’t changed it. The tool will display the certificate’s contents and will ask you if you want to accept the certificate. Press “y” and you should have your certificate imported into your java keystore.
-bash-4.1$ sudo keytool -import -trustcacerts -alias alias-for-your-internal-server-certificate -keystore /path/to/your/jdk/jre/lib/security/cacerts -file /path/to/your-certificate-.cer-or-.pem file
Configuring LDAPS:
Once you are logged onto the Jenkins CI UI console, click on Manage Jenkins –> Configure Global Security menu from the left tree. Change the ldap protocol from ldap to ldaps. If your ldap server’s ssl port is different from 636, you need to specify the port in the url itself, i.e., ldaps://your-ldap-server-host:port. Restarting the jenkins server will access the ldap server over ldaps protocol.
Configuring SSL access for Jenkins CI server:
- Under the JENKINS_HOME directory, create a hidden directory .ssl (mkdir .ssl).
- To access our CI server over https, we will need a SSL certificate. We can get an official CA certificate from a CA Authority or we could use the keytool utility to generate a self-signed certificate. The following command will store the certificate in the path specified under the -keystore switch.
- The keytool command will then prompt you to enter a passphrase for the keystore. Re-enter a secure passphrase again to confirm. The tool will then prompt you to enter some questions about your organization
- Confirm if the above mentioned information are correct. The keytool will prompt you to Enter passphrase for key (Press Enter if the same as the keystore password). Press carriage return to complete the certificate generation.
- Make the following changes to /etc/sysconfig/jenkins file. You need to have sudo access to do so.
-bash-4.1$ keytool -genkey -alias jenkins-ssl-cert -keyalg RSA -keystore $JENKINS_HOME/.ssl/.keystore -validity 365
Restarting the jenkins server will allow you to access your CI server using https:// protocol. I have smudged out all the details related to my certificate. You should see all the details you had entered during the certificate creation when you access the CI server in your browser.
Credentials
After you either provided the server certificate (or skipped the SSL check altogether), testing the connection may return following access error:
Installing the plugin
Jenkins Kubernetes Plugin (at the time of this writing) is at v0.12, and is available via Jenkins update site plugins. Installation is straight-forward and no different from other Jenkins plugins.
Kubernetes server certificate key
Grab the ‘cluster: certificate-authority-data’ value from your ~/.kube/config file
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: LSuperLongBase64EncodedString==
Local cluster
If your Jenkins Master is hosted on the same Kubernetes Cluster then all you need is to provide the Kubernetes URL for your local cluster as:
Click ‘Test Connection’ to verify the successful connection.
Pcks certificate without passphrase
If you set up the PCKS client certificate without a passphrase, Jenkins will not complain and will accept the certificate. However, using this certificate will result in a somewhat obscure error message:
Other tell-tell signs that your certificate wasn’t “successfully” accepted are:
Remote cluster
If you are not hosting Jenkins on the same Kubernetes cluster (or not hosting it on Kubernetes at all), then you need to perform a few extra steps to configure the access to your Kubernetes cluster.
Starting a slave
- Convert the cert.pem, from above, to cert.der:
openssl x509 -outform der -in cert.pem -out cert.der
- create keystore, containing this cert:
- transfer this file to the slave computer somehow (eg via /var/www/html, and download from slave)
- launch slave
- as for normal slave launch, but add `-Djavax.net.trustStore=cacerts
=> will work ok 🙂
Tl;dr;
Th Jenkins Kubernetes Plugin is a great tool to dynamically provision Jenkins slaves as pods on a Kubernetes Cluster(s). All you need to do is add and configure Kubernetes Cloud as part of the Jenkins configuration. While configuring Jenkins hosted on the Kubernetes cloud is very straight-forward in terms of credentials and accessibility, it may require additional steps if you are not running the Jenkins master on Kubernetes or would like to configure it for external Kubernetes cluster(s).
Conclusion
Jenkins Kubernetes Plugin provides additional credentials mechanisms to authenticate against the Kubernetes cluster(s) like a Kubernetes service account
However, at this time I was able to configure Kubernetes Cloud credentials using client certificates only. That is not to say that Kubernetes service accounts don’t work, just that I didn’t figure how to get it going.
I hope you find the steps above helpful in configuring your Jenkins against Kubernetes cluster(s). Let me know if you find any inaccuracies or have any questions, comments or suggestions!