Updating your alternate internal SLC Hub SSL certificates before they expire
Altair EmployeeSSL certificates need to be rotated on expiry and your company automations will hopefully take care of this.
However, if you need to do a manual replacement of SSL certificates then follow these steps.
Note that similar steps are required for each worker and for the hub control node too.
PART A - PREPARATION
No downtime is needed for Part A.
1. On each node, Make a backup of the 'ca' folder containing the existing KEY, CSR, and CERT files (in PEM format) which are found here:
/opt/altair/slchub/var/ca/public/slchub/Server.cert.pem
/opt/altair/slchub/var/ca/public/slchub/Server.csr.pem
/opt/altair/slchub/var/ca/public/slchub/Server.key.pem
/opt/altair/slchub/var/ca/public/nomad/ServerClient.cert.pem
/opt/altair/slchub/var/ca/public/nomad/ServerClient.csr.pem
/opt/altair/slchub/var/ca/public/nomad/ServerClient.key.pem
2. Send the CSRs files to your certificate provider again, two per node. You should receive back new CERT files, two per node.
(n.b. If you prefer to create new CSRs and KEYs then remove the 'ca' folder and run "hubctl generate csr" to generate the new CSRs and KEYs. And if you now need to keep the Hub running until new certificates are available for installation, then rename 'ca' to 'ca.new' and copy your backup of 'ca' back to its place in the 'var' directory.)
If the certificates are returned in various alternative formats then select the PEM format with the full chain of intermediate keys.
If necessary, manually edit the files to put the intermediate keys into reverse order.
PART B
When the new certificates are ready, schedule some downtime for the hub. 30-60 minutes should be more than enough.
3. Stop the services on all workers and on the hub control node using 'hubctl service stop'
4. On each node, put the relevant two .cert.pem files into their places in /opt/altair/slchub/var/ca/public/slchub and /opt/altair/slchub/var/ca/public/nomad.
5. Make sure that the .PEM files have got suitable read permissions. See https://hubdoc.worldprogramming.com/2023.2-2.3.3.1/configuration/features/alternateca/installcertificates/#create-the-certificates.
Steps to fix ownership and permissions:
cd /opt/altair/slchub/var/ca/public
chown slchub-casvc:slchub slchub/Server.cert.pem
chown slchub-casvc:slchub slchub/Server.key.pem
chown slchub-casvc:slchub nomad/ServerClient.cert.pem
chown slchub-casvc:slchub nomad/ServerClient.key.pem
chmod 640 slchub/Server.cert.pem # control node only
chmod 644 slchub/Server.cert.pem # worker nodes only
chmod 640 slchub/Server.key.pem # all nodes
chmod 640 nomad/ServerClient.cert.pem # all nodes
chmod 640 nomad/ServerClient.key.pem # all nodes
6. If in any doubt, you can use the openssl MD5 option to check that CSR, KEY, and CERT are matching:
cd /opt/altair/slchub/var/ca/public
openssl x509 -noout -modulus -in slchub/Server.cert.pem | openssl md5
openssl req -noout -modulus -in slchub/Server.csr.pem | openssl md5
openssl rsa -noout -modulus -in slchub/Server.key.pem | openssl md5
openssl x509 -noout -modulus -in nomad/ServerClient.cert.pem | openssl md5
openssl req -noout -modulus -in nomad/ServerClient.csr.pem | openssl md5
openssl rsa -noout -modulus -in nomad/ServerClient.key.pem | openssl md5
7. Restart the hub services with 'hubctl service start' on hub control node and then hub worker nodes.
Check services have started as expected (hubctl service status).