TL;DR: In this article, you will learn how to leverage the Ambassador API Gateway to secure the apps running in your Kubernetes clusters with TLS certificates. The approach that the article describes will enable you to use Let's Encrypt to issue certificates for free. With that, you will be able to enhance the security of your clusters (and its apps) for free, even if you are hosting multiple domains on them. You can use this GitHub repository as a reference if you need to double check the Kubernetes resources presented here.
“Learn how Ambassador, cert-manager, and Let's Encrypt can work together to issue TLS certificates to your Kubernetes cluster's domains for free!”
Tweet This
Prerequisites
To follow along with this article, you will need some previous experience with Kubernetes. If you are new to this platform, check out the "Step by Step Introduction to Basic Kubernetes Concepts" tutorial. There, you will learn everything you need to follow the instructions here.
Besides that, you will need
kubectl
, a Command-Line Interface (CLI) tool that will enable you to control your cluster from a terminal. If you don't have this tool, check the instructions on the "Installing Kube Control (kubectl)" section available on the article mentioned above.Another thing you will need is a couple of domains to test the steps described here. If you already have some domains hanging around, you can use them (or you can use subdomains under one that you own, for example:
kubernetes-tutorial.yourdomain.com
). If you don't have any domains you can use, you can get some for free on Freenom.Lastly, you will also need a Kubernetes cluster. If you already have a cluster running, you can skip the next section and jump into action. If you don't, keep reading as you will learn how to use DigitalOcean to spin up a cluster for free.
Spinning up a DO Kubernetes cluster
For starters, you can use this DO referral link to get some credit to avoid paying, while you read this article. If you don't use a referral link, you will end up paying for your cluster from the very beginning. This might not be a big problem because DigitalOcean prices are quite fair and you will need to keep the cluster running for less than an hour. That is, it won't cost more than a few cents but, well, not paying is better than paying, right?
After using this link to create your account on DigitalOcean, you will get an email confirmation. Use the link sent to you to confirm your email address. Confirming your address will make DigitalOcean ask you for a credit card. Don't worry about this. If you don't spend more than $100 USD, they won't charge a dime.
After inputting a valid credit card, you can use the next screen to create a project, or you can use this link to skip this unnecessary step and to head to DO's Managed Kubernetes dashboard.
From the dashboard, you can hit the Create a Kubernetes cluster button (you might have to click on Enable Limited Access first). Then, DigitalOcean will show you a new page with a form that you can fill in as follows:
- Select a Kubernetes version: The instructions on this article were tested with the
version. If you feel like testing other versions, feel free to do so. Just let us know how it went.1.13.5-do.1
- Choose a datacenter region: Feel free to choose whatever region you prefer.
- Choose cluster capacity: Make sure you have just one node pool, that you choose the
option, and that you have at least three nodes.$10/Month per node
- Add Tags: Don't worry about tagging anything.
- Choose a name: You can name your cluster whatever you want (e.g., "kubernetes-tutorial"). Just make sure DigitalOcean accepts the name (e.g., names can't contain spaces).
After filling in this form, you can click on the Create Cluster button. It will take a few minutes before DigitalOcean finishes creating your cluster for you. However, you can already download the cluster's config file now.
This file contains the credentials needed for you to act as the admin of the cluster, and you can find it on the cluster's dashboard. After you clicked on the Create Cluster button, DigitalOcean redirected you to your cluster's dashboard. From there, if you scroll to the bottom, you will see a button called Download Config File. Click on this button to download the file.
When you finish downloading this file, open a terminal and move the file to the
.kube
directory in your home dir (you might have to create this directory first):# make sure .kube exists mkdir ~/.kube # move the config file to it mv ~/Downloads/kubernetes-tutorial-kubeconfig.yaml ~/.kube
If needed, adjust the last command with the correct path of the downloaded file.
The
~/.kube
directory is a good place to keep your Kubernetes credentials. By default, kubectl
will use a file named config
(if it finds one inside the .kube
dir) to communicate with clusters. To use a different file, you have three alternatives:- First, you can specify another file by using the
flag in your--kubeconfig
commands, but this is too cumbersome.kubectl
- Second, you can define the
environment variable to avoid having to typeKUBECONFIG
all the time.--kubeconfig
- Third, you can merge contexts in the same
file and then you can switch contexts.config
The second option (setting the
KUBECONFIG
environment variable) is the easiest one, but feel free to choose another approach if you prefer. To set this environment, you can issue the following command:export KUBECONFIG=~/.kube/kubernetes-tutorial-kubeconfig.yaml
Note: Your file path might be different. Make sure the command above contains the right path.
Keep in mind that this command will set this environment only on this terminal's session. If you open a new terminal, you will have to execute this command again.
What is Ambassador? The API Gateway
If you don't know Ambassador, the Kubernetes-native API gateway that you will use to manage TLS certificates, don't worry. Ambassador is built on a declarative configuration model that exposes a bunch of really neat features in Envoy Proxy. This tool is an open-source project that helps you "secure, scale, and ship your microservices" and that is really easy to use. You don't really need to learn about it before diving into this article. But after you finish reading the instructions here, make sure you check out their documentation so you can see how it works.
Note: Before moving to the next section, make sure your Kubernetes cluster is up and running. To do so, execute
and check if all nodes showkubectl get nodes
on theReady
column.STATUS
Deploying Ambassador
After these introductory steps, it's time to see some action. For starters, you will need to install Ambassador in your Kubernetes cluster. To do so, execute the following code:
# install Ambassador kubectl apply -f https://getambassador.io/yaml/ambassador/ambassador-rbac.yaml
Note: If you are not using DigitalOcean, you might need to issue a few different commands. In this case, please, check the instructions on the Deploying Ambassador section of this resource.
After installing Ambassador, you will need to create a Kubernetes service to expose a
LoadBalancer
that is integrated with the gateway. To do so, you will need to create a YAML file. However, before doing this, as you will need to create a few different YAML files throughout this article, you will be better off creating and using a directory to group these files. So, in your preferred location, create a directory called ambassador-tls
; then move into it:# create a directory to add YAML files mkdir ambassador-tls # move into it cd ambassador-tls
After that, create a file called
ambassador-service.yaml
inside this directory and add the following code to it:--- apiVersion: v1 kind: Service metadata: name: ambassador spec: type: LoadBalancer externalTrafficPolicy: Local ports: - name: http protocol: TCP port: 80 targetPort: 8080 selector: service: ambassador
As mentioned, this file creates a service (
kind: Service
) that exposes your cluster through a LoadBalancer
that is attached to Ambassador (service: ambassador
). Now, execute the following command to deploy this service to your cluster:kubectl apply -f ambassador-service.yaml
This command will finish executing promptly. However, the load balancer that this service creates is an external element controlled by the Kubernetes vendor (e.g., DigitalOcean) that needs a couple of minutes to be up.
To check if your load balancer is already available, issue the following command:
kubectl get svc ambassador
If the
EXTERNAL-IP
column that this command presents is showing <pending>
, then your load balancer is not ready yet. Nevertheless, you can continue with the instructions here until further notice.As you will need an application up and running to see the whole configuration in action, you will deploy a sample application. To do so, create a file called
deployment.yaml
and add the following code to it:--- apiVersion: apps/v1 kind: Deployment metadata: name: kubernetes-tutorial-deployment spec: replicas: 1 selector: matchLabels: app: kubernetes-tutorial-deployment template: metadata: labels: app: kubernetes-tutorial-deployment spec: containers: - name: kubernetes-tutorial-application image: auth0blog/kubernetes-tutorial ports: - containerPort: 3000
When applied, this file will deploy a simple application that emulates a much simpler Twitter application where users can share their thoughts. However, note that this app doesn't even handle user authentication (by the way, if you read the Kubernetes introduction, you will notice that this is the same application).
After creating this file, execute the following command to apply the deployment:
kubectl apply -f deployment.yaml
Then, you will need to create another service. This time the goal is to expose the deployment inside your cluster. So, create a file called
service.yaml
and add the following code to it:--- apiVersion: v1 kind: Service metadata: name: kubernetes-tutorial-service annotations: getambassador.io/config: | --- apiVersion: ambassador/v1 kind: Mapping name: kubernetes-tutorial-mapping prefix: / service: kubernetes-tutorial-service host_rewrite: kubernetes-tutorial-service spec: ports: - port: 80 targetPort: 3000 name: http protocol: TCP selector: app: kubernetes-tutorial-deployment
This file is doing two important things: first, it is creating a Kubernetes service to represent your deployment and its pods; second, it is leveraging Ambassador to create a mapping (
kind: Mapping
) to define that this service must handle requests to the root path (prefix: /
).To apply this service, execute the following command:
kubectl apply -f service.yaml
Then, execute
kubectl get svc ambassador
once more and copy the external IP address of your load balancer. Now, if you use this IP address in a browser, you will be able to see the sample application running.Issuing a TLS Certificate with Ambassador
After deploying the sample application on Kubernetes and confirming that Ambassador is working, it is time to install and configure your first TLS certificate. This task will involve three steps:
- You will configure one or more domains to point to your cluster.
- You will install a tool called
to help Ambassador manage certificates.cert-manager
- You will use
and Ambassador to issue a TLS certificate.cert-manager
Pointing domains to Kubernetes' load balancer
Since Let's Encrypt will issue certificates to specific domains, you will have to configure the DNS settings of one or more domains to point to your new load balancer.
For example, if you are using Freenom, you can go to the My Domains section of your dashboard and click on the Manage Domain button. After that, move to the Manage Freenom DNS tab and add an
A
record that points to the IP address of the load balancer; then, hit Save Changes to persist the new setting.Note: After adding a new record to your domain, the DNS servers that support the internet will need a few minutes to sync. That is, you won't be able to open
in a web browser instantaneously. However, after these minutes, you will see that if you request this domain in your browser, the sample application you deployed earlier will load.http://yourdomain.com
Installing cert-manager
Having configured your domain to point to your cluster, the next thing you will do is to install
. This tool is a Kubernetes add-on that facilitates management and issuance of TLS certificates from various issuing sources. Combining cert-manager
cert-manager
with Ambassador will make TLS certificate management an easy task.To install it, you will need to create a Kubernetes namespace where you will deploy the resources that belong to
cert-manager
:# create a namespace for cert-manager's resources kubectl create namespace cert-manager
After that, you will need to disable resource validation on the namespace so the installation doesn't end up on a deadlock:
# disable resource validation on the cert-manager namespace kubectl label namespace cert-manager certmanager.k8s.io/disable-validation=true
Then, you can install
cert-manager
:kubectl apply -f https://raw.githubusercontent.com/jetstack/cert-manager/release-0.7/deploy/manifests/cert-manager.yaml
Issue certificates with Ambassador and cert-manager
Now that your domain is pointing to your cluster and that you installed
cert-manager
, you can issue a TLS certificate to your domain. To do so, you will need to create and deploy two Kubernetes resources: a ClusterIssuer
and a Certificate
.So, create a file called
certificate-issuer.yaml
and add the following resources to it:--- apiVersion: certmanager.k8s.io/v1alpha1 kind: ClusterIssuer metadata: name: kubernetes-tutorial-tk-issuer spec: acme: email: bruno.krebs@auth0.com server: https://acme-v02.api.letsencrypt.org/directory privateKeySecretRef: name: kubernetes-tutorial-tk-issuer http01: {} --- apiVersion: certmanager.k8s.io/v1alpha1 kind: Certificate metadata: name: kubernetes-tutorial-tk-certificate namespace: default spec: secretName: kubernetes-tutorial-tk-cert-secret issuerRef: name: kubernetes-tutorial-tk-issuer kind: ClusterIssuer dnsNames: - kubernetes-tutorial.tk acme: config: - http01: ingressClass: nginx domains: - kubernetes-tutorial.tk
You will have to replace a few properties on this file. For example, you will need to replace
bruno.krebs@auth0.com
with your email address. Besides that, you will have to replace the two places that use kubernetes-tutorial.tk
with the domain you are using. Lastly, you can replace the kubernetes-tutorial-tk-
prefix that this file uses to something more meaningful to you (e.g., your-domain-com-
). This last step is optional but will help you identify the resources later.Then, you can issue the following command to apply these resources to your cluster:
# create cluster issuer and certificate kubectl apply -f certificate-issuer.yaml
Now, if you check the
Certificate
status:# check the status of the certificate kubectl describe certificates kubernetes-tutorial-tk-certificate
You will notice that the last entry on
Events
is OrderCreated
. No matter how long you wait, cert-manager
will never be able to fulfill this order. The problem is that cert-manager
uses an Ingress
resource to issue the challenge to /.well-known/acme-challenge
. However, since Ambassador is not an Ingress
, you will need to create a Mapping
so cert-manager
can reach the temporary pod.To do so, first, you will need to find the
cert-manager
pod:# find cert-manager name kubectl get pods -n cert-manager
This command will fetch all pods on the
cert-manager
namespace. More specifically, this command will return three pods: one that starts with cert-manager-webhook-
; one that starts with cert-manager-cainjector-
; and one that starts with cert-manager-
only. You are interested in the latter. Copy its full name (it will be something like cert-manager-5c54bbcc4d-2dpzg
), and use it on the following command:# get cert-manager logs kubectl logs -n cert-manager cert-manager-5c54bbcc4d-2dpzg | grep acme-http-domain
This command will fetch the logs of the pod; then it will use
grep
to find lines that contain acme-http-domain
. The lines that this command returns will show two properties separated by a comma. The properties will be similar to this:- certmanager.k8s.io/acme-http-domain=
;1745815783
- and certmanager.k8s.io/acme-http-token=
744820138
You will need to use the values after the equals sign (i.e., after
acme-http-domain=
and after acme-http-token=
) in the next file.Now, create a file called
challenge-service.yaml
, and add the following code to it:--- apiVersion: v1 kind: Service metadata: name: acme-challenge-service annotations: getambassador.io/config: | --- apiVersion: ambassador/v1 kind: Mapping name: acme-challenge-mapping prefix: /.well-known/acme-challenge rewrite: "" service: acme-challenge-service spec: ports: - port: 80 targetPort: 8089 selector: certmanager.k8s.io/acme-http-domain: "1745815783" certmanager.k8s.io/acme-http-token: "744820138"
This file is using an Ambassador mapping to expose the
/.well-known/acme-challenge
route (under your domain) to enable Let's Encrypt to confirm that your cluster is indeed "attached" to the domain (or, in other words, that you own the domain you are using to issue a certificate). Before proceeding, make sure you replace the values passed to certmanager.k8s.io/acme-http-domain
and certmanager.k8s.io/acme-http-token
with the ones you fetched with grep
; then apply the service to your cluster:kubectl apply -f challenge-service.yaml
After a couple of minutes, Let's Encrypt and
cert-manager
will have accomplished their goal and will have issued a valid certificate to your domain. To confirm that, issue the following command again:# check the status of the certificate kubectl describe certificates kubernetes-tutorial-tk-certificate
Now, you will see that the last event is
CertIssued
and that the message says "Certificate issued successfully". To wrap this section, open the ambassador-service.yaml
and replace its code with this:--- apiVersion: v1 kind: Service metadata: name: ambassador annotations: getambassador.io/config: | --- apiVersion: ambassador/v1 kind: Module name: tls config: server: enabled: True redirect_cleartext_from: 8080 secret: kubernetes-tutorial-tk-cert-secret spec: type: LoadBalancer externalTrafficPolicy: Local ports: - name: http protocol: TCP port: 80 targetPort: 8080 - name: https protocol: TCP port: 443 targetPort: 8443 selector: service: ambassador
Note: If you changed the name of the secret while creating the
file, you will need to fix the code above to reference the correct one (i.e., you will have to replacecertificate-issuer.yaml
to whatever you chose).kubernetes-tutorial-tk-cert-secret
The new version of this file is going to make three changes to Ambassador. First, it will configure Ambassador's service to use the TLS
Module
(note: name: tls
is case sensitive, don't change it). Second, it will make Ambassador listen to port 443
, the default one used on secure connections (HTTPS). Third, it will redirect requests from port 80
(unsecured) to port 443
(redirect_cleartext_from: 8080
).After changing this file, use the following command to apply the changes:
kubectl apply -f ambassador-service.yaml
If everything works as expected, you will be able to load the sample app by using the
https:
protocol plus your domain (e.g., https://kubernetes-tutorial.tk
) in a web browser. Also, you will notice that, no matter how hard you try, even if you ask for an insecure version of your site first (i.e., http://
instead of https://
), your browser will load the secured version (you can thank redirect_cleartext_from
for that).“I just used Ambassador, cert-manager, and Let's Encrypt to issue TLS certificates to my Kubernetes cluster's domain for free!”
Tweet This
Managing Multiple Domains with Ambassador
Another great feature that ships with Ambassador is the support to Server Name Indication (SNI). This feature lets you supply separate TLS certificates for different domains instead of using a single TLS certificate for all domains.
To see this in action, you will need another domain, and you will need to point this domain's DNS to your load balancer's IP address. If you don't have another domain to use, you can get one for free at Freenom.
Then, you will have to go to the dashboard where you manage your domain's DNS and add an
A
record that points to the IP address of the load balancer. As a reminder, you can get the external IP address of your Kubernetes cluster by issuing kubectl get svc ambassador
.After configuring the DNS, the first thing you will have to do is to deactivate the
tls
module on your Ambassador installation. If you don't do this, Let's Encrypt won't be able to issue a certificate to your second domain. This happens because, for the moment, your cluster is configured to use the certificate issued for the first domain. If Let's Encrypt (or anyone for that matter) tries to issue a request to the second domain, they will get an error saying that the certificate is not valid for that domain.So, replace the contents of the
ambassador-service.yaml
file with this:--- apiVersion: v1 kind: Service metadata: name: ambassador spec: type: LoadBalancer externalTrafficPolicy: Local ports: - name: http protocol: TCP port: 80 targetPort: 8080 selector: service: ambassador
Then, apply this to your cluster by issuing
kubectl apply -f ambassador-service.yaml
. This will reconfigure the Ambassador service to allow unsecured (plain HTTP) requests again.After that, you will have to define the same resources you defined for your first domain: a
ClusterIssuer
and a Certificate
. So, create a file called certificate-issuer-second-domain.yaml
and add the following to it:--- apiVersion: certmanager.k8s.io/v1alpha1 kind: ClusterIssuer metadata: name: second-domain-tk-issuer spec: acme: email: bruno.krebs@auth0.com server: https://acme-v02.api.letsencrypt.org/directory privateKeySecretRef: name: second-domain-tk-issuer http01: {} --- apiVersion: certmanager.k8s.io/v1alpha1 kind: Certificate metadata: name: second-domain-tk-certificate namespace: default spec: secretName: second-domain-tk-cert-secret issuerRef: name: second-domain-tk-issuer kind: ClusterIssuer dnsNames: - second-domain.tk acme: config: - http01: ingressClass: nginx domains: - second-domain.tk
Just like before, you will have to replace a few properties on this file. For example, you will need to replace
bruno.krebs@auth0.com
with your email address. Besides that, you will have to replace the two places that use second-domain.tk
with the domain you own. Lastly, you can replace the second-domain-tk-
prefix that this file uses to something more meaningful to you (e.g., your-domain-com-
). This last step is optional but will help you identify the resources later.Then, you can issue the following command to apply these resources to your cluster:
# create cluster issuer and certificate kubectl apply -f certificate-issuer-second-domain.yaml
If you recall, the first time you were issuing a TLS certificate, you had to create a service to make
.well-known/acme-challenge
a valid path under your domain. This is necessary so Let's Encrypt can confirm the ownership of the domain. So, repeating the steps you executed before, issue the following command to get the name of the cert-manager
pod:# find cert-manager name kubectl get pods -n cert-manager
Then, use it to get two properties:
# get cert-manager logs kubectl logs -n cert-manager cert-manager-5c54bbcc4d-2dpzg | grep acme-http-domain
Note: You will have to replace the pod name in this last command.
After running this command, you will see an output that shows the following properties:
- certmanager.k8s.io/acme-http-domain=
;1745815783
- and certmanager.k8s.io/acme-http-token=
.744820138
You will need to use the values after the equals sign (i.e., after
acme-http-domain=
and after acme-http-token=
) in the challenge service. So, create a file called second-domain-challenge-service.yaml
, and add the following code to it:--- apiVersion: v1 kind: Service metadata: name: second-domain-acme-challenge-service annotations: getambassador.io/config: | --- apiVersion: ambassador/v1 kind: Mapping name: second-domain-acme-challenge-mapping prefix: /.well-known/acme-challenge rewrite: "" service: second-domain-acme-challenge-service spec: ports: - port: 80 targetPort: 8089 selector: certmanager.k8s.io/acme-http-domain: "894240303" certmanager.k8s.io/acme-http-token: "1107562110"
Note: Replace the values passed to
andacme-http-domain
with the values you copied from theacme-http-token
pod. Also, feel free to replace thecert-manager
prefix with something more meaningful.second-domain-
Now, apply this service to your cluster:
kubectl apply -f second-domain-challenge-service.yaml
After a couple of minutes, Let's Encrypt and
cert-manager
will have issued the certificate for your second domain. To confirm that, issue the following command:# check the status of the certificate kubectl describe certificates second-domain-tk-certificate
If the last event is
CertIssued
and the message says "Certificate issued successfully", then you are good to go. If the last event still is OrderCreated
, you will have to wait a couple more minutes before proceeding.Lastly, you will have to reconfigure your Ambassador service to use the two certificates. So, replace the contents of the
ambassador-service.yaml
file with this:--- apiVersion: v1 kind: Service metadata: name: ambassador annotations: getambassador.io/config: | --- apiVersion: ambassador/v1 kind: Module name: tls config: server: enabled: True redirect_cleartext_from: 8080 --- apiVersion: ambassador/v1 kind: TLSContext name: kubernetes-tutorial-tls-context hosts: - kubernetes-tutorial.tk secret: kubernetes-tutorial-tk-cert-secret --- apiVersion: ambassador/v1 kind: TLSContext name: second-domain-tls-context hosts: - second-domain.tk secret: second-domain-tk-cert-secret spec: type: LoadBalancer externalTrafficPolicy: Local ports: - name: http protocol: TCP port: 80 targetPort: 8080 - name: https protocol: TCP port: 443 targetPort: 8443 selector: service: ambassador
Then, if you apply this service (
kubectl apply -f ambassador-service.yaml
), you will be reconfiguring your Ambassador service to use both domains with separate certificates. Just note that you will have to fix the contents of this file to reflect your domains (i.e., you will have to replace kubernetes-tutorial.tk
and second-domain.tk
).Now, if you open both domains on a browser, you will see that both will load securely. Also, if you click on the
“Using Ambassador to handle multiple domains on a Kubernetes cluster is easy. And each domain can use its own TLS certificate!”
Tweet This
Recap
In this article, you learned how to take advantage of Ambassador,
cert-manager
, and Let's Encrypt to manage TLS certificates for free. With this, you will enhance the security of the apps you run in your cluster and will have more confidence that your data will not be tampered in motion (for example, by a MITM attack).What do you think about this approach? Did you know that Ambassador also supports SNI (Server Name Indication) to be able to manage different TLS certificates to different domains? If so, would you be interested in learning how to use this feature? Let us know in the comment box.
About the author
Bruno Krebs
R&D Content Architect