lost/rskio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

expand k3s documentation (#1)

Browse Source 
traefik
clusterissuer
longhorn
webhook
This commit is contained in:
rskntroot
2025-06-20 23:42:36 -06:00
committed by GitHub
parent 526683319b
commit f73b9dfe86
5 changed files with 611 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files

123
mkdocs/docs/projects/k3s/clusterissuer.md Normal file
View File

@@ -0,0 +1,123 @@
# ClusterIssuer
Allows certificate requests from an ACME provider. This is used to enable HTTPS TLS for services you stand up.
## Setup
see [cert-manager kubectl install](https://cert-manager.io/docs/installation/kubectl/) for more info
=== "v1.18"
``` bash
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.18.0/cert-manager.yaml
```
create at least one of the `clusterissuers` types below
### External
uses LetsEncrypt and public DNS records to sign https for your sites
``` yaml title="letsencrypt/clusterissuer.yml"
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
namespace: default
spec:
acme:
server: https://acme-v02.api.letsencrypt.org/directory
email: ${EMAIL}
privateKeySecretRef:
name: letsencrypt-prod
solvers:
- selector: {}
http01:
ingress:
class: traefik
```
### Internal
pointed at an internal ACME provider to generate certs for an intranet
``` yaml title="internal/clusterissuer.yml"
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: internal-issuer
spec:
acme:
email: ${EMAIL}
server: ${ACME_URL}
privateKeySecretRef:
name: interal-issuer-account-key
caBundle: ${CA_BUNDLE_BASE64} # ca bundle that was used to generate the tls cert for the acme site
solvers:
- selector: {}
http01:
ingress:
class: traefik
```
## Certificate
### Example
create a `certificate.yml` file for a traefik `IngressRoute`
=== "Certificate"
``` yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: io-rsk-docs-tls
spec:
secretName: io-rsk-docs-tls
issuerRef:
name: dev-step-issuer
kind: ClusterIssuer
commonName: docs.dev.rsk.io
dnsNames:
- docs.dev.rsk.io
privateKey:
algorithm: RSA
encoding: PKCS1
size: 2048
usages:
- server auth
- client auth
duration: 2160h # 90 days
renewBefore: 360h # 15 days
secretTemplate:
annotations:
kubeseal-secret: "true"
labels:
domain: docs-dev-rsk-io
```
=== "IngressRoute"
``` yaml
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: rskio-docs
spec:
entryPoints:
- web
- websecure
routes:
- match: Host(`docs.dev.rsk.io`)
kind: Rule
services:
- name: rskio-docs
port: 80
tls:
secretName: io-rsk-docs-tls
```
After applying this `Certifcate` a `Secret` is created containing the `.crt` and `.key` files.
These are loaded by the traefik.io `IngressRoute` under `spec.tls.secretName`.
This enables usage of the tls cert for https client reachability.

110
mkdocs/docs/projects/k3s/longhorn.md Normal file
View File

@@ -0,0 +1,110 @@
# Longhorn
Provides distributed storage for the cluster.
We will only be editing the nodes as many of the defaults are sufficient.
## Requirements
All cluster nodes need these packages installed:
``` bash
sudo apt install open-iscsi nfs-common -y
```
see [longhorn os-specific requirements](https://longhorn.io/docs/1.9.0/deploy/install/#osdistro-specific-configuration) for more information.
## Setup
=== "v1.9.0"
``` bash
kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/v1.9.0/deploy/longhorn.yaml
```
see [longhorn installation](https://longhorn.io/docs/1.9.0/deploy/install/install-with-kubectl/#installing-longhorn) for more information.
## Dashboard
### Service
create and apply `longhorn/service.yml`
``` bash
apiVersion: v1
kind: Service
metadata:
labels:
app: longhorn-ui
name: longhorn-dashboard
namespace: longhorn-system
spec:
ports:
- port: 8000
protocol: TCP
targetPort: 8000
name: web
selector:
app: longhorn-ui
```
### Ingress
create and apply `longhorn/ingress.yml`
``` bash
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: longhorn-dashboard
namespace: longhorn-system
spec:
entryPoints:
- websecure
routes:
- match: Host(`storage.${DOMAIN_NAME}`)
kind: Rule
services:
- name: longhorn-dashboard
port: 8000
```
After creating a `ClusterIssuer` be sure to create a `Certificate` and apply it with `spec.tls.secretName`.
With Traefik you can also use certResolver, though clusterissuer certs allow for more fine-grain control.
## StorageClass
create and apply `longhorn/storageclass.yml`
``` bash
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: longhorn-data
provisioner: driver.longhorn.io
allowVolumeExpansion: true
reclaimPolicy: Delete
volumeBindingMode: Immediate
parameters:
numberOfReplicas: "3"
staleReplicaTimeout: "300"
fromBackup: ""
fsType: "ext4"
```
## PVC
create and apply `some-app/pvc.yml`
``` yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: some-app-pvc
spec:
accessModes:
- ReadWriteMany
resources:
requests:
storage: 500M
storageClassName: longhorn-data
```

163
mkdocs/docs/projects/k3s/traefik.md Normal file
View File

@@ -0,0 +1,163 @@
# Traefik
## Brief
Enabling traefik access to dashboard and metrics for traefik ingress controller in k3s kubernetes cluster
- by `rskntroot` on `2024-07-01`
## Assumptions
``` bash
$ k3s --version
k3s version v1.32.5+k3s1 (8e8f2a47)
go version go1.23.8
```
``` bash
$ kubectl version
Client Version: v1.32.5+k3s1
Kustomize Version: v5.5.0
Server Version: v1.32.5+k3s1
```
## Dashboards
K3S comes packaged with `Traefik Dashboard` enabled by default, but not exposed.
### Preparation
#### DNS
=== "DNS"
Set DNS record `traefik.your.domain.com`
=== "Hosts File"
Alternatively, you can just edit your `hosts` file.
``` title="/etc/hosts"
10.0.0.1 traefik.your.domain.com
```
!!! warning "This example does not include authentication. Exposing these dashboards is a security risk. Recommend enabling mTLS."
#### Middlewares
On host with `kubectl` access.
``` yaml title="middlewares.yml"
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: redirect-https
namespace: default
spec:
redirectScheme:
scheme: https
permanent: true
port: "443"
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: redirect-dashboard
namespace: default
spec:
redirectRegex:
regex: "^https?://([^/]+)/?$"
replacement: "https://${1}/dashboard/"
permanent: true
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: ratelimit
namespace: default
spec:
rateLimit:
average: 100
burst: 50
---
apiVersion: traefik.io/v1alpha1
kind: Middleware
metadata:
name: compress
namespace: default
spec:
compress: {}
```
``` bash
kubectl apply -f middlewares.yml
```
### Setup IngressRoute
create `ingress.yml` and update `"edge.rskio.com"` with your domain name
``` yaml title="ingress.yml"
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: traefik-dashboard
spec:
entryPoints:
- web
- websecure
routes:
- match: Host(`edge.rskio.com`) # Update with your domain name
kind: Rule
services:
- name: api@internal
kind: TraefikService
middlewares:
- name: redirect-https
- name: redirect-dashboard
- name: ratelimit
- name: compress
```
``` bash
kubectl apply -f ingress.yml
```
## Access Dashboards
You should now be able to access the Traefik Ingress Controller Dashboard and metrics remotely.
From web browser go to the domain you specified in the ingress.
=== "Traefik Dashboard"
```
https://edge.your.domain.com
```
will follow `redirect-https` and get you to
```
https://edge.your.domain.com/dashboard/#/
```
### Disable Dashboards
=== "Bash"
``` bash
kubectl delete -f ingress.yml
```
=== "Example"
``` bash
$ kubectl delete -f traefik/ingress.yml
ingressroute.traefik.io "traefik-ingress" deleted
```
## References
- [https://docs.k3s.io](https://docs.k3s.io)
- [https://doc.traefik.io/traefik/](https://doc.traefik.io/traefik/)

372
mkdocs/docs/projects/k3s/webhook.md Normal file
View File

@@ -0,0 +1,372 @@
# Webhooks
Continuous integration on easy mode.
Webhooks allow for a ton of functionality,
but we are going to use it to kick off a kubernetes job.
Effectivitely automating reloading content on a static website.
## Background
This docs website is a static site that is hosted inside an nginx container.
The storage for these redundant pods is a longhorn rwx pvc that gets stood up.
To initialize the storage a kubernetes job is run. This job does the following:
- git clones the `rskntroot/rskio` repo containing the artifacts required to render the site
- executes the `mkdocs` command to render the static site
So what if when we push to github, we setup a webhook that tells kubernetes to kick off that job?
Well, we achieve some form of automation.
So how do we do this?
## Setup
### RBAC
=== "ServiceAccount"
``` yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: webhook-job-trigger
```
=== "Dev Roles"
``` yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: job-creator
namespace: dev
rules:
- apiGroups: ["batch"]
resources: ["jobs"]
verbs: ["create"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: job-creator-binding
namespace: dev
subjects:
- kind: ServiceAccount
name: webhook-job-trigger
namespace: default
roleRef:
kind: Role
name: job-creator
apiGroup: rbac.authorization.k8s.io
```
=== "Prod Roles"
``` yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: job-creator
namespace: prod
rules:
- apiGroups: ["batch"]
resources: ["jobs"]
verbs: ["create"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: job-creator-binding
namespace: prod
subjects:
- kind: ServiceAccount
name: webhook-job-trigger
namespace: default
roleRef:
kind: Role
name: job-creator
apiGroup: rbac.authorization.k8s.io
```
### ConfigMap
We will create a config map from a directory including the following files.
#### ConvertJob
We are going to be using curl to call the kubernetes API directly,
so we need to convert our job from yaml to json.
Convert the job to JSON and save to `etc/mkdocs-dev.json`
=== "Job"
``` yaml
apiVersion: batch/v1
kind: Job
metadata:
generateName: mkdocs-builder-
namespace: dev
spec:
ttlSecondsAfterFinished: 600
template:
spec:
containers:
- name: mkdocs
image: squidfunk/mkdocs-material
command: ["/bin/sh", "-c"]
args:
- |
git clone --single-branch -b dev https://github.com/rskntroot/rskio.git --depth 1 /docs
cd /docs/mkdocs
mkdocs build --site-dir /output
volumeMounts:
- name: mkdocs-storage
mountPath: /output
restartPolicy: Never
volumes:
- name: mkdocs-storage
persistentVolumeClaim:
claimName: mkdocs-pvc
```
=== "Convert"
``` bash
mkdir etc
cat job.yml | yq -e -j | jq > etc/mkdocs-dev.json
```
The following docs we will assume that you also created `etc/mkdocs-main.json`.
#### Hooks
create `etc/hooks.yaml`
=== "etc/hooks.yaml"
``` yaml
- id: rskio-mkdocs
execute-command: /etc/webhook/reload.sh
command-working-directory: /etc/webhook
response-message: payload received
response-headers:
- name: Access-Control-Allow-Origin
value: "*"
pass-arguments-to-command:
- source: payload
name: ref
- source: payload
name: repository.full_name
trigger-rule:
and:
- match:
type: value
value: push
parameter:
source: header
name: X-GitHub-Event
- match:
type: value
value: rskntroot/rskio
parameter:
source: payload
name: repository.full_name
```
=== "Secret"
after testing come back to implement secrets
``` yaml
trigger-rule:
and:
- match:
type: payload-hmac-sha1
secret: mysecret
parameter:
source: header
name: X-Hub-Signature
```
apply the `configmap` and rollout restart the webhook deployment
#### Command
``` bash title="etc/reload.sh"
#!/bin/sh
REF=$1
REPO=$2
dispatch() {
NS=$1
JOB_JSON=$2
SA_PATH="/var/run/secrets/kubernetes.io/serviceaccount"
curl https://kubernetes.default.svc/apis/batch/v1/namespaces/${NS}/jobs \
-X POST \
-H "Authorization: Bearer $(cat ${SA_PATH}/token)" \
-H "Content-Type: application/json" \
--cacert "${SA_PATH}/ca.crt" \
-d "@${JOB_JSON}"
}
docs(){
case ${REF} in
refs/heads/dev)
dispatch dev "/etc/webhook/mkdocs-dev.json"
;;
refs/heads/main)
dispatch prod "/etc/webhook/mkdocs-main.json"
;;
*)
echo "skipping push to unsupported ref ${REF}"
exit 0
;;
esac
}
case ${REPO} in
rskntroot/rskio)
docs
;;
*)
echo "skipping push to unsupported repo ${REPO}"
;;
esac
```
#### Create
once all resources in `etc` are created run the following command:
``` bash
kubectl create configmap webhook-etc --from-file=etc
```
if you need to update anything run the following:
``` bash
kubectl delete configmap webhook-etc
kubectl create configmap webhook-etc --from-file=etc
```
### Resources
The following resources will complete the work
=== "Deployment"
``` yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: webhook-docs
spec:
replicas: 1
selector:
matchLabels:
app: webhook-docs
template:
metadata:
labels:
app: webhook-docs
spec:
serviceAccountName: webhook-job-trigger
containers:
- name: webhook-docs
image: ghcr.io/linuxserver-labs/webhook:latest
command: ["/app/webhook"]
args:
- -hooks=/etc/webhook/hooks.yaml
- -hotreload
- -verbose
volumeMounts:
- name: webhook-etc
mountPath: /etc/webhook
volumes:
- name: webhook-etc
configMap:
name: webhook-etc
defaultMode: 493 # 0755
```
=== "Service"
``` yaml
apiVersion: v1
kind: Service
metadata:
name: webhook
spec:
selector:
app: webhook-docs
ports:
- protocol: TCP
port: 9000
targetPort: 9000
type: ClusterIP
```
=== "Certificate"
``` yaml
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: io-rsk-dev-hooks-tls
spec:
secretName: io-rsk-dev-hooks-tls
issuerRef:
name: dev-step-issuer
kind: ClusterIssuer
commonName: hooks.dev.rsk.io
dnsNames:
- hooks.dev.rsk.io
privateKey:
algorithm: RSA
encoding: PKCS1
size: 2048
usages:
- server auth
- client auth
duration: 2160h # 90 days
renewBefore: 360h # 15 days
secretTemplate:
annotations:
kubeseal-secret: "true"
labels:
domain: hooks-dev-rsk-io
```
=== "Ingress"
``` yaml
apiVersion: traefik.io/v1alpha1
kind: IngressRoute
metadata:
name: webhook
spec:
entryPoints:
- websecure
routes:
- match: Host(`hooks.dev.rsk.io`)
kind: Rule
services:
- name: webhook
port: 9000
middlewares:
- name: ratelimit
tls:
secretName: io-rsk-hooks-tls
```
## Testing
``` bash
curl -X POST https://hooks.dev.rsk.io/hooks/rskio-mkdocs \
-H 'X-Github-Event: push' \
-H 'Content-type: application-json' \
-d '{"ref": "refs/heads/dev","repository": {"full_name":"rskntroot/rskio"}}'
```
!!! note "Github needs access to a public domain for this to work."