How to configure a fresh Kubernetes Cluster (upcloud) to host websites

Add helm repos:

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo add jetstack https://charts.jetstack.io
helm repo update

Create namespace:

kubectl create namespace poc

Install ingress:

helm upgrade --install ingress-nginx ingress-nginx/ingress-nginx \  
  --namespace ingress-nginx \  
  --create-namespace
# Wait for external IP (load-balancer domain)
kubectl get svc -n ingress-nginx ingress-nginx-controller

I’ll use sub-sub-domains for PoC websites.

Create DNS record (CNAME):

CNAME  host=*.poc  target=lb-xxxxx.upcloudlb.com

Then install cert-manager:

helm upgrade --install cert-manager jetstack/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --set crds.enabled=true

Create cluster issuer:

cat <<'EOF' | kubectl apply -f -
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-http
spec:
  acme:
    email: xxx@gmail.com
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: letsencrypt-http-account-key
    solvers:
    - http01:
        ingress:
          class: nginx
EOF

Check cluster issuer:

kubectl get clusterissuer

# returns:
# NAME               READY   AGE
# letsencrypt-http   True    5s

Create a test app:

kubectl create deployment hello --image=nginx -n poc
kubectl expose deployment hello --port=80 --target-port=80 -n poc

Create ingress:

cat <<'EOF' | kubectl apply -f -
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: hello
  namespace: poc
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-http
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - hello.poc.jorvik.io
      secretName: hello-poc-tls
  rules:
    - host: hello.poc.jorvik.io
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: hello
                port:
                  number: 80
EOF

Check:

kubectl get ingress -n poc
kubectl get certificate -n poc
kubectl get challenge -n poc

Restart:

kubectl rollout restart deployment ingress-nginx-controller -n ingress-nginx
kubectl rollout status deployment ingress-nginx-controller -n ingress-nginx

Didn’t work, so:

cat > ingress-nginx-values.yaml <<'EOF'
controller:
  service:
    annotations:
      service.beta.kubernetes.io/upcloud-load-balancer-config: |
        {
          "frontends": [
            { "name": "http", "mode": "tcp" },
            { "name": "https", "mode": "tcp" }
          ],
          "backends": [
            { "name": "http", "properties": { "outbound_proxy_protocol": "v2" } },
            { "name": "https", "properties": { "outbound_proxy_protocol": "v2" } }
          ]
        }
  config:
    use-forwarded-headers: "true"
    compute-full-forwarded-for: "true"
    use-proxy-protocol: "true"
    real-ip-header: "proxy_protocol"
EOF

Followed by:

helm upgrade ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx \
  -f ingress-nginx-values.yaml

kubectl rollout status deployment ingress-nginx-controller -n ingress-nginx