 |
Using istio gateways, gardener dns and kyma dashboard components | This brief is to demonstrate how SAP BTP, Kyma runtime makes it easy to set up an istio gateway with a wildcard public custom domain. Pre-requisistes:
|
Putting it all together
For the sake of this brief I have picked two different public domains registrars, namely Google Domains and Gandi.net and two different external DNS providers, namely: Google Cloud DNS and Azure DNS.
When you provision a kyma cluster it gets assigned a secure wildcard public domain name, for instance:
.kyma.ondemand.com, which is entirely managed by SAP.The built-in
kyma-system/kyma-gateway is configured with this default domain.Likewise, your kyma cluster's API server, namely:
https://api.<shoot-name>.kyma.ondemand.com, is bound to that domain as well. Thus, a cluster domain cannot be changed once a cluster has been provisioned.Q. But what if you wanted to have your own domain with a kyma cluster gateway? In other words, how to set up a domain for workloads across a kyma cluster?
A. Here you go. You will need to accomplish these three tasks, namely:
Task A. Bring your own domain (BYOD)
Task B. Set up the domain with an external DNS provider (Google CloudDNS and Azure DNS)
Task C. Configure SAP BTP, Kyma runtime (SKR) cluster DNS resources automation.
|
I shall endeavour to walk you step by step throughout each of these three tasks. However, if you are network-fluent and have complied with the Task A and Task B requirements, you may jump directly to Task C.
Task A. Bring Your Own Domain (BYOD).
The BYOD refresher:
|
1a. Custom domains with Google Domains
Google Domains is a domain registrar. For instance, you may look up your registered domain names at:
https://domains.google.com/registrar/However, Google Domains is not a DNS (domain name service) provider per se. Thus you will need to bring in the DNS servers names from your established external DNS provider.
| Steps | Google Domains registered domain's custom DNS settings |
Goto DNS settings and switch to Custom name servers. Please make sure the domain, you are about to modify the DNS settings of, is not used for your websites or email. However, If that were the case, you might want to register a different domain dedicated to your kyma workloads |  |
The custom name servers will come from either the Google Cloud DNS or Azure DNS provider. As Google Domains does not support APIs you will have apply them manually. |  |
| After saving the new DNS settings these will be effective after a short while. |  |
1b. Custom domains with Gandi.net
Gandi.net is a popular domain registrar and a DNS provider. However, Gandi is not a supported external DNS provider, thus I had to bring in the DNS name servers from an AzureDNS zone manually as well as depicted below:
| Gandi.net domains registrar |
Task B. Set up an external DNS provider. Create a DNS zone.
In order to be able to use one of your registered domains with SAP BTP, Kyma runtime, the first thing you need to do with either Google CloudDNS or AzureDNS is to create a DNS zone for your domain and then copy the DNS servers names to your domains registrar.
| What is a DNS zone? |
|
1a. Create Google Cloud DNS zone within your GCP project
In order to be able to use one your Google Domains registered domains with SKR you will need to set up a DNS zone to host the DNS records for a chosen domain name with Google Cloud DNS as follows:
Quickstart: Set up DNS records for a domain name with Cloud DNS | Google Cloud Docs
| Steps | Google Network Service/Cloud DNS |
| Goto Network Service/Cloud DNS within your project. |  |
Create a DNS zone for keeping your Google Domains domain's DNS records. |  |
| The default SOA and NS records got created.However, additional DNS records for a chosen domain name will be created when configuring the SKR cluster DNS resources. Thus no need to create the records manually. |  |
| Copy the following name servers to your registrar domain's custom DNS settings (via a click on the Registrar setup button or via Equivalent REST shown below). |  |
{
"cloudLoggingConfig": {
"enableLogging": false
},
"creationTime": "<creationTime>",
"description": "quovadis-btp",
"dnsName": "quovadis-btp.com.",
"fingerprint": "<fingerprint>",
"id": "<id>",
"location": "global",
"name": "quovadis-btp",
"nameServers": [
"ns-cloud-**.googledomains.com.",
"ns-cloud-**.googledomains.com.",
"ns-cloud-**.googledomains.com.",
"ns-cloud-**.googledomains.com."
],
"visibility": "PUBLIC"
}1b. Create Azure DNS zone within your Azure subscription resource group.
In order to be able create a DNS zone you need to have a valid Azure subscription and have created a resource group as depicted below:
 |
Azure DNS zone creation steps:
 |  |
 |  |
{
"id": "/subscriptions/<subscriptionId>/resourceGroups/quovadis/providers/Microsoft.Network/dnszones/quovadis-anywhere.com",
"name": "quovadis-anywhere.com",
"type": "Microsoft.Network/dnszones",
"etag": "<etag>",
"location": "global",
"tags": {},
"properties": {
"maxNumberOfRecordSets": 10000,
"maxNumberOfRecordsPerRecordSet": null,
"nameServers": [
"ns1-**.azure-dns.com.",
"ns2-**.azure-dns.net.",
"ns3-**.azure-dns.org.",
"ns4-**.azure-dns.info."
],
"numberOfRecordSets": 2
}
}2. Create an API access to a DNS zone
The kyma cluster will need an application level (API) access to a DNS zone. That's a pre-requisite for enabling the DNS provisioning automation on kyma side.
- With Google CloudDNS you will need to create a service account with a service key.
- With AzureDNS you will have to create an Azure AD application and service principal that can access AzureDNS zone
2a. Create a GooglePlatform Service Account and a service key
A service key is required for the kyma cluster to get access to the DNS records of the CloudDNS managed domain. The service key must allow for DNS records modifications.
| Steps | Google IAM and admin/Service Accounts |
| In your Google Cloud project goto Google IAM and admin/Service Accounts and create a service account |  |
| The service account has been created |  |
Create a service key with the service account credentials. The service key will need to passed to the kyma cluster namespace as a secret |  |
2b. create an Azure AD application and service principal
The main tutorial that explains how to do it (especially if you do not have access to a Windows machine) is here:
For instance, here go AzureDNS API credentials, namely azure.json:
{
"tenantId": "<tenantId>",
"subscriptionId": "<subscriptionId>",
"resourceGroup": "quovadis",
"aadClientId": "8e4ed817-c1b0-4cab-89a0-************",
"aadClientSecret": "c91z*************************"
}and this is how the secret should look like in kyma dashboard:
kind: Secret
apiVersion: v1
metadata:
name: quovadis-azure-dns-2
namespace: azure-dns
data:
clientID: <clientID>
clientSecret: <clientSecret>
resourceGroup: cXVvdmFkaXM=
subscriptionID: <subscriptionID>
tenantID: <tenantID>
type: OpaqueGood to know:
- in case of any error message the error message will be written into the k8s resource; thus the easiest to look it up in plain English is to open a resource in its editor and goto the yaml tab.
Task C. Configure kyma cluster DNS resources
As usual the best is to refer to the DNS providers source documentation from the Gardener project.
The External-DNS-Management project provides examples with more details forDNSProviders(30-provider-<provider-name>.yaml) and credentialSecrets(20-secret-<provider-name>.yaml) for all supported provider types.
All the DNS resources are namespaced. Thus, it's probably a good idea to have a dedicated namespace per DNS provider in a kyma cluster.
0. Create a DNS automation namespace
Let's call this namespace:
conn-proxy. Goto namespace configuration to set up the following DNS resources.1. Create an external DNS provider secret
The DNS provider secret is required for DNS resources automation.
There are two ways to create secrets. Either using the kyma dashboard secret presets or with kubectl as shown below:
$ kubectl create secret generic quovadis-btp-dns --namespace=conn-proxy--from-file=serviceaccount.json=<project name>-11111111111.json
secret/quovadis-btp-dns created2. Create a DNS provider and a DNS entry
DNS Provider:
 |
apiVersion: dns.gardener.cloud/v1alpha1
kind: DNSProvider
metadata:
name: quovadis-bpt-dns-provider
namespace: conn-proxy
labels:
app.kubernetes.io/name: quovadis-bpt-dns-provider
annotations:
dns.gardener.cloud/class: garden
spec:
domains:
include:
- quovadis-btp.com
- '*.quovadis-btp.com'
exclude: []
secretRef:
name: quovadis-btp-dns
namespace: conn-proxy
type: google-clouddns
DNS Entry:
Good to know:
- The below UI has changed and it no longer populates the istio ingress gateway as a target choice.
- Thus if you want to use the external ip address of the istio ingress gateway as a target you may need to retrieve it using the following command:
kubectl get svc istio-ingressgateway -n istio-system |
apiVersion: dns.gardener.cloud/v1alpha1
kind: DNSEntry
metadata:
name: quovadis-bpt-dns-entry
namespace: conn-proxy
labels:
app.kubernetes.io/name: quovadis-bpt-dns-entry
annotations:
dns.gardener.cloud/class: garden
spec:
dnsName: '*.quovadis-btp.com'
targets:
- **.**.**.**
text: []
ttl: 6003. Create an Issuer with ACME (Automated Certificate Management Environment)
The issuer resource is required to sign the certificate for the domain.
apiVersion: cert.gardener.cloud/v1alpha1
kind: Issuer
metadata:
name: quovadis-btp-ca-issuer
namespace: conn-proxy
labels:
app.kubernetes.io/name: quovadis-btp-ca-issuer
spec:
requestsPerDayQuota: 0
acme:
server: 'https://acme-v02.api.letsencrypt.org/directory'
email: <email address>
autoRegistration: true
privateKeySecretRef:
name: ''
namespace: conn-proxy
domains:
include:
- quovadis-btp.com
- '*.quovadis-btp.com'
4. Create a certificate resource
This has to be done in the
istio-system namespace as follows: |
apiVersion: cert.gardener.cloud/v1alpha1
kind: Certificate
metadata:
labels:
app.kubernetes.io/name: quovadis-btp-cert
name: quovadis-btp-cert
namespace: istio-system
spec:
commonName: '*.quovadis-btp.com'
issuerRef:
name: quovadis-btp-ca-issuer
namespace: conn-proxy
secretRef:
name: tls-quovadis-btp-cert
namespace: istio-system5. Create an istio gateway resource
We are back to the DNS automation namespace where we shall create an istio gateway for our own wildcard domain. Here goes the resource deployment script:
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
labels:
app.kubernetes.io/name: quovadis-gateway
name: quovadis-gateway
namespace: conn-proxy
spec:
selector:
istio: ingressgateway
servers:
- hosts:
- '*.quovadis-btp.com'
port:
name: quovadis-https
number: 443
protocol: HTTPS
tls:
credentialName: tls-quovadis-btp-cert
mode: SIMPLE
- hosts:
- '*.quovadis-btp.com'
port:
name: quovadis-http
number: 80
protocol: HTTP
tls:
httpsRedirect: true |
6. Create an API rule for a workload using the custom domain
Last but not least we can now decide to direct the traffic through this new gateway. Let's create an API rule for a function that retrieves data from a remote SAP S/4HANA system.
apiVersion: gateway.kyma-project.io/v1alpha1
kind: APIRule
metadata:
labels:
app.kubernetes.io/name: quovadis-s4
name: quovadis-s4
namespace: conn-proxy
spec:
gateway: quovadis-gateway.conn-proxy.svc.cluster.local
rules:
- accessStrategies:
- config:
jwks_urls:
- >-
https://<zone>.authentication.<region>.hana.ondemand.com/token_keys
trusted_issuers:
- >-
https://<zone>.authentication.<region>.hana.ondemand.com/oauth/token
handler: jwt
methods:
- PUT
- POST
- PATCH
- HEAD
- GET
- DELETE
path: /.*
service:
host: ***.quovadis-btp.com
name: test-s4
port: 80and voila:
 |
Good to know:
- The gateway, namely:
quovadis-gateway.conn-proxy.svc.cluster.local, can be used from any namespace across your kyma cluster
Conclusion
I hope you have enjoyed reading this blog post.
You have seen how to expose your Kyma cluster workloads with istio gateways over a custom domain.
Whether this has been easy or not I shall leave it to your judgement. The good news is that this is a one-time effort and that a one single wildcard custom domain can be used for any workload across the SKR cluster.
Last but not least, to stay up to date with the latest news on SAP BTP, Kyma runtime make sure to bookmark our Kyma topic page.
Appendix
ACME Directory URL: https://acme-v02.api.letsencrypt.org/directory
{
"ddLcOs-N9Uo": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417",
"keyChange": "https://acme-v02.api.letsencrypt.org/acme/key-change",
"meta": {
"caaIdentities": [
"letsencrypt.org"
],
"termsOfService": "https://letsencrypt.org/documents/LE-SA-v1.3-September-21-2022.pdf",
"website": "https://letsencrypt.org"
},
"newAccount": "https://acme-v02.api.letsencrypt.org/acme/new-acct",
"newNonce": "https://acme-v02.api.letsencrypt.org/acme/new-nonce",
"newOrder": "https://acme-v02.api.letsencrypt.org/acme/new-order",
"revokeCert": "https://acme-v02.api.letsencrypt.org/acme/revoke-cert"
}ACME Certificate issuer:
- https://docs.bitnami.com/kubernetes/infrastructure/cert-manager/configuration/generate-acme-certific...
- https://cert-manager.io/docs/configuration/acme/
- https://certbot-dns-google.readthedocs.io/en/stable/
- https://smallstep.com/docs/tutorials/acme-protocol-acme-clients
Additional resources
1. A hands-on tutorial leveraging Cloud-fare DNS | SAP Samples
2. New with SAP BTP, Kyma Runtime: Seamlessly connect your website or microservice to a custom domain u...
3. a video tutorial from SAP HANA Academy
https://www.youtube.com/watch?v=AvhDgMFD76s&amp;list=PLkzo92owKnVw3l4fqcLoQalyFi9K4-UdY
4. Beyond the Gardener project there are other public resources that describe the external DNS providers configurations and settings.
5. https://practicaldatascience.co.uk/data-engineering/how-to-create-a-google-service-account-client-se...
6. https://stackoverflow.com/questions/70774865/add-a-service-account-json-to-kubectl-secrets
7. https://stackexchange.github.io/dnscontrol/providers/azure_dns
SAP Community: https://community.sap.com/
SAP Community Topic Page link: https://community.sap.com/topics/kyma
SAP Community Q&A Tags:
Kyma Open Source: https://answers.sap.com/tags/2936b97d-6a90-4cd8-b635-0e51441611eb
SAP BTP, Kyma runtime: https://answers.sap.com/tags/73554900100800003012
Follow me in SAP Community: piotr.tesny
- SAP Managed Tags:
6 Comments
| User | Count |
|---|---|
| 17 | |
| 14 | |
| 11 | |
| 10 | |
| 10 | |
| 9 | |
| 8 | |
| 7 | |
| 7 | |
| 5 |
