Ingress for tunnels¶

Info

Inlets Uplink is designed to connect customer services to a remote Kubernetes cluster for command and control as part of a SaaS product.

Any tunnelled service can be accessed directly from within the cluster and does not need to be exposed to the public Internet for access.

Beware: by following these instructions, you are exposing one or more of those tunnels to the public Internet.

Make inlets uplink HTTP tunnels publicly accessible by setting up ingress for the data plane.

The instructions assume that you want to expose two HTTP tunnels. We will configure ingress for the first tunnel, called grafana, on the domain grafana.example.com. The second tunnel, called openfaas, will use the domain openfaas.example.com.

Both tunnels can be created with kubectl or the inlets-pro cli. See create tunnels for more info:

kubectlcli

$ cat <<EOF | kubectl apply -f - 
apiVersion: uplink.inlets.dev/v1alpha1
kind: Tunnel
metadata:
  name: grafana
  namespace: tunnels
spec:
  licenseRef:
    name: inlets-uplink-license
    namespace: tunnels
---
apiVersion: uplink.inlets.dev/v1alpha1
kind: Tunnel
metadata:
  name: openfaas
  namespace: tunnels
spec:
  licenseRef:
    name: inlets-uplink-license
    namespace: tunnels
EOF

$ inlets-pro tunnel create grafana
Created tunnel openfaas. OK.

$ inlets-pro tunnel create openfaas
Created tunnel openfaas. OK.

Follow the instruction for Kubernetes Ingress or Istio depending on how you deployed inlets uplink.

Setup tunnel ingress¶

Create a new certificate Issuer for tunnels:

export EMAIL="you@example.com"

cat > tunnel-issuer-prod.yaml <<EOF
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: tunnels-letsencrypt-prod
  namespace: inlets
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: $EMAIL
    privateKeySecretRef:
    name: tunnels-letsencrypt-prod
    solvers:
    - http01:
        ingress:
          class: "nginx"
EOF

Create an ingress resource for the tunnel:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: grafana-tunnel-ingress
  namespace: inlets
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/issuer: tunnels-letsencrypt-prod
spec:
  rules:
  - host: grafana.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: grafana.tunnels
            port:
              number: 8000
  tls:
  - hosts:
    - grafana.example.com
    secretName: grafana-cert

Note that the annotation cert-manager.io/issuer is used to reference the certificate issuer created in the first step.

To setup ingress for multiple tunnels simply define multiple ingress resources. For example apply a second ingress resource for the openfaas tunnel:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: openfaas-tunnel-ingress
  namespace: inlets
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/issuer: tunnels-letsencrypt-prod
spec:
  rules:
  - host: openfaas.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: openfaas.tunnels
            port:
              number: 8000
  tls:
  - hosts:
    - openfaas.example.com
    secretName: openfaas-cert

Setup tunnel ingress with an Istio Ingress gateway¶

Create a new certificate Issuer for tunnels:

export EMAIL="you@example.com"

cat > tunnel-issuer-prod.yaml <<EOF
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: tunnels-letsencrypt-prod
  namespace: istio-system
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: $EMAIL
    privateKeySecretRef:
      name: tunnels-letsencrypt-prod
    solvers:
    - http01:
        ingress:
          class: "istio"
EOF

We are using the Let's Encrypt production server which has strict limits on the API. A staging server is also available at https://acme-staging-v02.api.letsencrypt.org/directory. If you are creating a lot of certificates while testing it would be better to use the staging server.

Create a new certificate resource. In this case we want to expose two tunnels on their own domain, grafana.example.com and openfaas.example.com. This will require two certificates, one for each domain:

apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: grafana-cert
  namespace: istio-system
spec:
  secretName: grafana-cert
  commonName: grafana.example.com
  dnsNames:
  - grafana.example.com
  issuerRef:
    name: tunnels-letsencrypt-prod
    kind: Issuer

---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: openfaas-cert
  namespace: istio-system
spec:
  secretName: openfaas-cert
  commonName: openfaas.example.com
  dnsNames:
  - openfaas.example.com
  issuerRef:
    name: tunnels-letsencrypt-prod
    kind: Issuer

Note that both the certificates and issuer are created in the istio-system namespace.

Configure the ingress gateway for both tunnels. In this case we create a single resource for both hosts but you could also split the configuration into multiple Gateway resources.

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: tunnel-gateway
  namespace: inlets
spec:
  selector:
      istio: ingressgateway # use Istio default gateway implementation
  servers:
  - port:
      number: 443
      name: https
      protocol: HTTPS  
    tls:
      mode: SIMPLE
      credentialName: grafana-cert
    hosts:
    - grafana.example.com
  - port:
      number: 443
      name: https
      protocol: HTTPS
    tls:
      mode: SIMPLE
      credentialName: openfaas-cert
    hosts:
    - openfaas.example.com

Note that the credentialsName references the secrets for the certificates created in the previous step.

Configure the gateway's traffic routes by defining corresponding virtual services:

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: grafana
  namespace: inlets
spec:
  hosts:
  - grafana.example.com
  gateways:
  - tunnel-gateway
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: grafana.tunnels.svc.cluster.local
        port:
          number: 8000
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: openfaas
  namespace: inlets
spec:
  hosts:
  - openfaas.example.com
  gateways:
  - tunnel-gateway
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        host: openfaas.tunnels.svc.cluster.local
        port:
          number: 8000

After applying these resources you should be able to access the data plane for both tunnels on their custom domain.

Wildcard Ingress with the data-router¶

As an alternative to creating individual sets of Ingress records, DNS A/CNAME entries and TLS certificates for each tunnel, you can use the data-router to route traffic to the correct tunnel based on the hostname. This approach uses a wildcard DNS entry and a single TLS certificate for all tunnels.

The following example is adapted from the cert-manager documentation to use DigitalOcean's DNS servers, however you can find instructions for issuers such as AWS Route53, Cloudflare, and Google Cloud DNS listed.

DNS01 challenges require a secret to be created containing the credentials for the DNS provider. The secret is referenced by the issuer resource.

kubectl create secret generic \
  -n inlets digitalocean-dns \
  --from-file access-token=$HOME/do-access-token

Create a separate Issuer, assuming a domain of t.example.com, where each tunnel would be i.e. prometheus.t.example.com or api.t.example.com:

export NS="inlets"
export ISSUER_NAME="inlets-wildcard"
export DOMAIN="t.example.com"

cat <<EOF | kubectl apply -f -
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: $ISSUER_NAME
  namespace: $NS
spec:
  acme:
    email: webmaster@$DOMAIN
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: $ISSUER_NAME
    solvers:
    - dns01:
        digitalocean:
            tokenSecretRef:
              name: digitalocean-dns
              key: access-token
EOF

Update values.yaml to enable the dataRouter and to specify the wildcard domain:

## The dataRouter is an option component to enable easy Ingress to connected tunnels.
## Learn more under "Ingress for Tunnels" in the docs: https://docs.inlets.dev/
dataRouter:
  enabled: true

  # Leave out the asterix i.e. *.t.example.com would be: t.example.com
  wildcardDomain: "t.example.com"

  tls:
    issuerName: "inlets-wildcard"

    ingress:
      enabled: true
      annotations:
        # Apply basic rate limiting.
        nginx.ingress.kubernetes.io/limit-connections: "300"
        nginx.ingress.kubernetes.io/limit-rpm: "1000"

Apply the updated values:

helm upgrade --install inlets-uplink \
  oci://ghcr.io/openfaasltd/inlets-uplink-provider \
  --namespace inlets \
  --values ./values.yaml

Create a tunnel with an Ingress Domain specified in the .Spec field:

export TUNNEL_NS="tunnels"
export DOMAIN="t.example.com"

cat <<EOF | kubectl apply -f -
apiVersion: uplink.inlets.dev/v1alpha1
kind: Tunnel
metadata:
  name: fileshare
  namespace: $TUNNEL_NS
spec:
  licenseRef:
    name: inlets-uplink-license
    namespace: $TUNNEL_NS
  ingressDomains:
    - fileshare.$DOMAIN
EOF

On a private computer, create a new directory, a file to serve and then run the built-in HTTP server:

cd /tmp
mkdir -p ./share
cd ./share
echo "Hello from inlets" > index.html

inlets-pro fileserver --port 8080 --allow-browsing --webroot ./

Get the instructions to connect to the tunnel.

The --domain flag here is for your uplink control-plane, where tunnels connect, not the data-plane where ingress is served. This is usually i.e. uplink.example.com.

export TUNNEL_NS="tunnels"
export UPLINK_DOMAIN="uplink.example.com"

inlets-pro tunnel connect fileshare \
  --namespace $TUNNEL_NS \
  --domain $UPLINK_DOMAIN

Add the --upstream fileshare.t.example.com=fileshare flag to the command you were given, then run it.

The command below is sample output, do not copy it directly.

inlets-pro uplink client \
  --url=wss://uplink.example.com/tunnels/fileshare \
  --token=REDACTED \
  --upstream fileshare.t.example.com=http://127.0.0.1:8080

Now, access the tunneled service via the wildcard domain i.e. https://fileshare.t.example.com.

You should see: "Hello from inlets" printed in your browser.

Finally, you can view the logs of the data-router, to see it resolving internal tunnel service names for various hostnames:

kubectl logs -n inlets deploy/data-router

2024-01-24T11:29:16.965Z        info    data-router/main.go:51  Inlets (tm) Uplink - data-router: 

2024-01-24T11:29:16.970Z        info    data-router/main.go:90  Listening on: 8080      Tunnel namespace: (all) Kubernetes version: v1.27.4+k3s1

I0124 11:29:58.858772       1 main.go:151] Host: fileshares.t.example.com    Path: /
I0124 11:29:58.858877       1 roundtripper.go:48] "No ingress found" hostname="fileshares.t.example.com" path="/"

I0124 11:30:03.588993       1 main.go:151] Host: fileshare.t.example.com     Path: /
I0124 11:30:03.589051       1 roundtripper.go:56] "Resolved" hostname="fileshare.t.example.com" path="/" tunnel="fileshare.tunnels:8000"