lost/rskio
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor from sphinx to mkdocs for added functionality and ease of use

Browse Source
This commit is contained in:
rskntroot
2024-06-09 06:09:09 +00:00
parent 317e362366
commit 45034f48c3
15 changed files with 619 additions and 475 deletions
Download Patch File Download Diff File Expand all files Collapse all files

34
mkdocs/docs/index.md Normal file
View File

@@ -0,0 +1,34 @@
# Home
## Rskio
This site (a project of mine in and of itself) is meant to catalog my efforts. Through the years, I have "spun my wheels" in order to learn, get something working, or even try something interesting--only for it to be lost to time. You could boil this site down to my notes. However, I fully intend for it to be much more than that. This site exists for me and the possibilty that it might help you too.
## About
Its been what, now? Around 10 years since I started working in computing proffessionally (right after dropping out of a Computer Science program, oops). Oh and about 14 since I'd learned my first programming language. What?! I'm in my thirties?! Guess I can't go saying I'm just some random kid on the internet anymore.
Just like everyone else my age, I got into computing because my family didn't know how to get the wifi working when we finally got broadband internet. Somehow that got me into robotics and programming in middle school and it snowballed from there. After a multi-year stint with the challenges of the cyber security field, I found peace by letting other people deal with it. These days I'm a full-time computer engineer that designs and deploys network infrastructure for a Tier-1 cloud provider. In my spare time, I work on projects, or dream of a financial freedom and getting able to work on whatever projects I want full-time, or both.
## What does Rskio Mean?
Prounciation:
=== "English"
`R-S-K-I-O`
=== "IPA"
`ɑːr-ɛs-keɪ-aɪ-əʊ`
Nothing. Just like QUIC and the G in GNU. As a "programmer" from a young age, I too struggle with naming variables and cache invalidation. It's sort of a mix of a nickname I had when I was young and Input/Outout or IO. After trying to think of something clever that pays homage to Nihei's TOHA Heavy Industries, I inevtiably gave up and fought with Route53 until it provided me a 5-letter .com that made sense to me.
The same goes for "Rskntroot", it's a mix of that same nickname and the classic "root" term.
## Projects
Currently, this is an unorganized list of things I have spent many of what corporate America refers to as "cycles" on.
## Storage
Also projects, but those dedicated to storage technology.

369
mkdocs/docs/projects/k3s_traefik_setup.md Normal file
View File

@@ -0,0 +1,369 @@
# K3S Traefik Setup
## Brief
Enabling internal access to dashboard and metrics for traefik ingress controller in k3s kubernetes cluster
- by `rskntroot` on `2024-07-01`
## Assumptions
``` bash
$ k3s --version
k3s version v1.29.5+k3s1 (4e53a323)
go version go1.21.9
```
``` bash
$ kubectl version
Client Version: v1.29.5+k3s1
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.5+k3s1
```
## Traefik Dashboards
`Traefik Dashboards` refers to both traefik dashboard and prometheus metrics for traefik which are readily available, however, disabled by default in K3S.
### Preparation
Enable `internal`+`.your.domain.com` in non-public DNS
- (alt) edit the `hosts` file on your admin to point the desired k3s host IP
On host with `kubectl` access:
``` bash
export DOMAIN=your.domain.com
```
!!! warning "This example does not include authentication. Exposing these dashboards is a security risk."
### Update Manifest
Add the following to `spec.valuesContent` in:
``` bash
vim /var/lib/rancher/k3s/server/manifests/traefik.yaml
```
=== "Yaml"
``` yaml
dashboard:
enabled: true
metrics:
prometheus: true
```
=== "Example"
``` yaml
spec:
chart: https://%{KUBERNETES_API}%/static/charts/traefik-25.0.3+up25.0.0.tgz
set:
global.systemDefaultRegistry: ""
valuesContent: |-
deployment:
podAnnotations:
prometheus.io/port: "8082"
prometheus.io/scrape: "true"
dashboard:
enabled: true
metrics:
prometheus: true
```
### Restart Ingress Controller
=== "Bash"
``` bash
kubectl -n kube-system scale deployment traefik --replicas=0
# wait a few seconds
kubectl -n kube-system get deployment traefik
kubectl -n kube-system scale deployment traefik --replicas=1
```
=== "Example"
``` bash
$ kubectls scale deployment traefik --replicas=0
deployment.apps/traefik scaled
$ kubectls get deployment traefik
NAME READY UP-TO-DATE AVAILABLE AGE
traefik 0/0 0 0 3d1h
$ kubectls scale deployment traefik --replicas=1
deployment.apps/traefik scaled
```
### Create Resource Definition YAML
Save the following to `traefik-dashboard.yml` in your workspace.
=== "Traefik Dashboard"
``` yaml
apiVersion: v1
kind: Service
metadata:
name: traefik-dashboard
namespace: kube-system
labels:
app.kubernetes.io/instance: traefik
app.kubernetes.io/name: traefik-dashboard
spec:
type: ClusterIP
ports:
- name: traefik
port: 9000
targetPort: 9000
protocol: TCP
selector:
app.kubernetes.io/instance: traefik-kube-system
app.kubernetes.io/name: traefik
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: traefik-ingress
namespace: kube-system
annotations:
spec.ingressClassName: traefik
spec:
rules:
- host: internal.${DOMAIN}
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: traefik-dashboard
port:
number: 9000
```
=== "Promethus Only"
``` yaml
apiVersion: v1
kind: Service
metadata:
name: traefik-metrics
namespace: kube-system
labels:
app.kubernetes.io/instance: traefik
app.kubernetes.io/name: traefik-metrics
spec:
type: ClusterIP
ports:
- name: traefik
port: 9100
targetPort: 9100
protocol: TCP
selector:
app.kubernetes.io/instance: traefik-kube-system
app.kubernetes.io/name: traefik
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: traefik-ingress
namespace: kube-system
annotations:
spec.ingressClassName: traefik
spec:
rules:
- host: internal.${DOMAIN}
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: traefik-dashboard
port:
number: 9000
- path: /metrics
pathType: Prefix
backend:
service:
name: traefik-metrics
port:
number: 9100
```
=== "Both"
``` yaml
apiVersion: v1
kind: Service
metadata:
name: traefik-dashboard
namespace: kube-system
labels:
app.kubernetes.io/instance: traefik
app.kubernetes.io/name: traefik-dashboard
spec:
type: ClusterIP
ports:
- name: traefik
port: 9000
targetPort: 9000
protocol: TCP
selector:
app.kubernetes.io/instance: traefik-kube-system
app.kubernetes.io/name: traefik
---
apiVersion: v1
kind: Service
metadata:
name: traefik-metrics
namespace: kube-system
labels:
app.kubernetes.io/instance: traefik
app.kubernetes.io/name: traefik-metrics
spec:
type: ClusterIP
ports:
- name: traefik
port: 9100
targetPort: 9100
protocol: TCP
selector:
app.kubernetes.io/instance: traefik-kube-system
app.kubernetes.io/name: traefik
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: traefik-ingress
namespace: kube-system
annotations:
spec.ingressClassName: traefik
spec:
rules:
- host: internal.${DOMAIN}
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: traefik-dashboard
port:
number: 9000
- path: /metrics
pathType: Prefix
backend:
service:
name: traefik-metrics
port:
number: 9100
```
### Create Service & Ingress Resources
[envsubst](https://www.gnu.org/software/gettext/manual/html_node/envsubst-Invocation.html) - enables code-reuse by providing environment variable substituion as demonstrated below.
=== "Bash"
``` bash
envsubst < traefik-dashboard.yml | kubectl apply -f -
```
=== "Example"
``` bash
$ envsubst < traefik-dashboards.yml | kubectl apply -f -
service/traefik-dashboard created
service/traefik-metrics created
ingress.networking.k8s.io/traefik-ingress created
$ kubectls get svc | grep traefik-
traefik-dashboard ClusterIP 10.43.157.54 <none> 9000/TCP 25s
traefik-metrics ClusterIP 10.43.189.128 <none> 9100/TCP 25s
```
### Access Dashboards
That's it. You should now be able to access the Traefik Ingress Controller Dashboard and metrics remotely.
Don't forget to include the appropriate uri paths:
=== "Traefik Dashboard"
```
https://internal.your.domain.com/dashboard/
```
!!! tip "When navigating to the traefik dashboard the `/` at the end is necessary. `/dashboard` will not work. "
=== "Promethus Metrics"
```
https://internal.your.domain.com/metrics
```
### Disable Dashboards
=== "Bash"
``` bash
envsubst < traefik-dashboard.yml | kubectl delete -f -
```
=== "Example"
``` bash
$ envsubst < traefik-dashboards.yml | kubectl delete -f -
service "traefik-dashboard" deleted
service "traefik-metrics" deleted
ingress.networking.k8s.io "traefik-ingress" deleted
```
## Shortcuts
### alias kubectls
!!! tip "When using an `alias` to substitute `kubectl` command completion will not work."
=== "Bash"
``` bash
echo 'alias kubectls="kubectl -n kube-system"' >> ~/.bashrc
source ~/.bashrc
```
=== "Example"
``` bash
$ echo 'alias kubectls="kubectl -n kube-system"' >> ~/.bashrc
$ source ~/.bashrc
$ kubectls get deployments
NAME READY UP-TO-DATE AVAILABLE AGE
coredns 1/1 1 1 3d2h
local-path-provisioner 1/1 1 1 3d2h
metrics-server 1/1 1 1 3d2h
traefik 1/1 1 1 3d2h
```
#### Alternatives
- `skubectl` means you can hit `[up-arrow]` `[ctrl]+[a]` `[s]` `[enter]` when you inevitably forget to include `-n kube-system`
- `kubectls` just adds `[alt]+[right-arrow]` into the above before `[s]`
- `kubesctl` makes sense because all of these are really kube-system-ctl, but that adds 4x `[right-arrow]`, ewww.
## References
- [https://docs.k3s.io](https://docs.k3s.io)
- [https://k3s.rocks/traefik-dashboard/](https://k3s.rocks/traefik-dashboard/)
- [https://doc.traefik.io/traefik/v2.10/](https://doc.traefik.io/traefik/v2.10/)

69
mkdocs/docs/projects/system_on_chip.md Normal file
View File

@@ -0,0 +1,69 @@
# System on Chip
## Brief
Employing containerized applications via clustering modern ARM-based System on Chip (SoC)
- by `rskntroot` on `2024-05-30`
## Background
With over 10 years of industry experience in computing technologies, I have owned a handful of RaspBerryPi through the years. After learning basic circuitry and microcontroller fundamentals thanks to Arduino back in 2010 or so, I watched rPi quickly captivate the industry. As a hobbist, I spent many hours trying to come to terms with the limitations in RaspbianOS; followed by even more time bridging the knowledge gap between what I knew from networking, embedded-systems, virtualization platforms, and Enterprise Linux and this fledgling ARM-based ecosystem.
As someone who has spent far more time building and managing Enterprise networks, servers, and services; all while trying to convice tenured Microsoft-indoctrinated IT admins that containerization in Linux was the future; I certainly didn't spend enough time completing weekend circuitry projects. It likely comes as no surprise that my fascination with ARM-based microcontrollers is in their use as an extremely lightweight replacement for rack-mounted servers--given you are NOT running computationally or storage-heavy workloads. In this regard, a power-efficent SoC that can run containerized services has quite the appeal for personal-use applications. With the cost of power these days I am baffled at the thought of running the previous generations discounted servers at home.
A few years ago, with the ever raging popularity of RaspberryPis, I like many others found myself looking for an alternative "PiLike" SoCs. After finding LibreComputers offerings, I was delighted to find the "Le Potato". For under $40 USD, the [AML-S905X-CC-V1](https://libre.computer/products/aml-s905x-cc/) could reliably run Ubuntu-Server (headless) with support for containers?! Usage of a mainstream Linux-disto on SoC was a game-changer for me. So I bought a couple and worked through the ever so "minor challenges" of getting started with a new platform. To my suprise it was rather straightforward.
Fast forward to 2024, I stumbled across the somewhat quiet release of the 2023 [AML-S905X-CC-V2](https://libre.computer/products/aml-s905x-cc-v2/). Man how time flies. For just around $35 USD, this release sports USB-C power-in. But even better is the PoE support with the addition of a $15 USD [PoE hat](https://www.loverpi.com/products/loverpi-poe-hat-with-pwm-fan-controller-for-aml-s905x-cc-v2-sweet-potato?_pos=1&_psq=poe&_ss=e&_v=1.0), you can deploy these with a single cable, given you have the infrastructure for it.
## Recommendation
Efficient, featureful, and cheap; the [Sweet Potato AML-S905X-CC-V2](https://libre.computer/products/aml-s905x-cc-v2/) is just what the doctor ordered. Seriously, check out the specs. One could always ask for more cores, RAM, eMMC. However, this little guy is packing just enough for a streamlined CI/CD pipeline to host some tolerant webservices across the edge. (For obvious reasons, "the edge" is just my family and friend's houses across the United States.)
If you need more RAM, USB3.0, or AI Acceleration is mandatory, checkout LibreComputer's [Alta](https://libre.computer/products/aml-a311d-cc/) or [Solitude](https://libre.computer/products/aml-s905d3-cc/) Models which can quickly double your investment.
## Projects
This website is hosted on these bad bois, the sweet potato that is.
> sphinx-docs w/ markdown generated website hosted on nginx alpine-linux
>
> `github` [rskntroot/rskio](https://github.com/rskntroot/rskio)
> traefik fe proxy w/ sticky loadbalancing for docker swarm
>
> `github` [rskntroot/traefik](https://github.com/rskntroot/traefik/)
### LLM on ARM
For my purposes, a handful of Sweet Potatos will do the trick, but if I start playing around with any [Large Language Models](https://en.wikipedia.org/wiki/Large_language_model)... like I keep telling myself I will, a few Altas in a cluster sounds like a fun experiment.
Maybe just biting the bullet and buying into in the [Nvidia Jerson](https://developer.nvidia.com/embedded-computing) offerings may be the play.
Or even just use that 4000-series Nvidia GPU for something other than filling up space in your handbuilt PC like the washed-up gamer you are (but that's not ARM!).
## Notes
### PoE Setup
Using Power over Ethernet (PoE) to run your SoCs is just awesome! You only need 1 cable?! Be sure to get yourself some good cables and a solid PoE switch.
I have personnally been using these:
- [CAT8 Ethernet cables](https://www.amazon.com/Ethernet-Internet-Network-Professional-Shielded/dp/B08PL1P53C/ref=sr_1_4?crid=1ELCUKBT7V3YB&dib=eyJ2IjoiMSJ9.W2iSbQd5bQGYHCf-9Vt3AovS1xEhC0zzsheMJG1QFnYut2JmmRCQzZmwq60K1uSUoGiTU8RUzjOTwybWF9lFZrGDx4abFxPwgCaCQTvL9Fvqa6UQ4Qu6o6JWEZRkWwNUWK34Izz1HPf1r54hVQ2NrN6f1r6PDUk2NDEab2zld8MVx2zRT4-s-8-jgwi8ng6wccQVAiJu-kTGeN_fkNohbUcpUMkmE-ARnhBrV05qIZg.OigmTuqYLiyRtsbLu-nXFx2nluGZo0e0IwjWTcXuHgg&dib_tag=se&keywords=cat8%2Bethernet%2Bcable&qid=1717047302&sprefix=cat8%2Caps%2C134&sr=8-4&th=1) by [VABOGU](https://www.amazon.com/stores/VABOGU/page/20815F77-3E58-4871-A2EB-1772920695D9?ref_=ast_bln)
- Ive used countless Ethernet Cables and fashioning hundreds of my own, I can confirm these are premium.
- [1G PoE+ 8-port Switch](https://www.amazon.com/dp/B08FCQ8BRC?ref=nb_sb_ss_w_as-reorder_k1_1_9&amp=&crid=1OZ5HYTQCXGAQ&amp=&sprefix=poe+switc) by [AMCREST](https://www.amazon.com/stores/Amcrest/page/2404E471-79FC-4D18-B767-8777D048264F?ref_=ast_bln)
- Unmanaged switch that I can recommend. Works like a charm.
### CAT8 Real?
Telco Data [article](https://www.telco-data.com/blog/cat-cables/):
"Category 8 is the official successor to Cat6A cabling. It is officially recognized by the IEEE and EIA and parts and pieces are standardized across manufacturers. The primary benefit of Cat8 cabling is faster throughput over short distances: 40 Gbps up to 78’ and 25 Gbps up to 100’. From 100’ to 328’, Cat8 provides the same 10Gbps throughput as Cat6A cabling."
ANSI/TIA [TIA Press Release](https://standards.tiaonline.org/tia-issues-new-balanced-twisted-pair-telecommunications-cabling-and-components-standard-addendum-1):
"TIA-568-C.2-1 - This addendum specifies minimum requirements for shielded category 8 balanced twisted-pair telecommunications cabling (e.g. channels and permanent links) and components (e.g. cable,connectors, connecting hardware, and equipment cords) that are used up to and including the equipment outlet/connector in data centers, equipment rooms, and other spaces that need high speed applications. This addendum also specifies field test procedures and applicable laboratory reference measurement procedures for all transmission parameters."

144
mkdocs/docs/storage/network_attached_storage.md Normal file
View File

@@ -0,0 +1,144 @@
# Network Attached Storage
## Brief
Scoping an 80 terabyte onsite Network Attached Storage (NAS) solution for media group usage
- by `rskntroot` on `2024-06-01`
## Assumptions
At a minimum, the storage solution must provide redundant copies of the data. The solution must function in the event of a single drive. Individual storage drives used must be enterprise-grade, warrantied, with the expectation of 1 million hours MTBF.
Given standard network connectivity is at or under 1Gbps speeds, 10Gb/s internal networking is not expected. However, the storage solution should provide future expansion for 10Gb/s networking.
Concerns regarding redudant power or battery backup for the solution are outside the scope of this document and would be handled by the customer. Recommened best practice for storage solutions is to keep on a battery backup to allow for graceful shutdown.
## Hardware
> `$8121 USD | hardware-only estimate*`
!!! warning "* Not included: shipping, taxes, build & validation services, installation support"
### Chassis
=== "QNAP TVS-H874T"
8-bay NAS: [qnap product page](https://www.qnap.com/en-us/product/tvs-h874t/specs/hardware)
- Thunderbolt 4 support
- EZCOPY USB3.2
Available Upgrades:
- 10/25 Gb/s networking (upgrade)
> `$2899 USD | chassis`
### Drives
=== "Seagate Exos X24 24TB"
Enterprise 24TB HDD: [seagate product page](https://www.seagate.com/products/enterprise-drives/exos-x/x24/)
- 5-year limited warranty
- 180MB/s Read (7200RPM)
- 2.5M MTBF
- Compatibility Check [ [PASS](https://www.qnap.com/en-us/compatibility/?model=758&category=1&filter[type]=1&filter[brand]=Seagate&filter[capacity]=24000) ]
> `$4792 USD | $599 USD (8 ea)`
=== "Solidgram P44 Pro"
2TB M.2 NVME: [solidigm product page]( https://www.solidigm.com/products/client/pro-series/p44.html#form=M.2%202280&cap=2%20TB)
- 5-year limited warranty
- 7000 MB/s Read
- 1.6M MTBF
- Compatibility Check [ [PASS](https://www.qnap.com/en-us/compatibility/?model=758&category=1&filter[type]=1&filter[brand]=Seagate&filter[capacity]=24000) ]
> `$398 USD | $199 USD (2 ea)`
### Misc
=== "Ethernet Cables"
CAT8 Ethernet Cables: [vabugo amazon store](https://www.amazon.com/stores/VABOGU/page/20815F77-3E58-4871-A2EB-1772920695D9?ref_=ast_bln)
- 1/2.5/10/25 Gb/s support
- 10' length (standard)
> `$32 USD | $16 USD (2 ea)`
## Deployment Options
### Hard Drives
=== "8 Drives"
This maxes out the hardware with a capacity of `~80TB` with read speeds maxing out around `~1440MB/s`.
=== "6 Drives"
Using 6 drives will bring the total capcacity to `~60TB` and access speed maxing out at `~1080MB/s`. This saturates a 10Gbps connection.
Initial Cost down `$1198 USD | ($599 * 8) - ($599 * 4)`
=== "4 Drives"
The minimum capacity of 4 drives provides a capacity of `~40TB` and access speed maxing out at `~720MB/s` read. Bringing the cost down
Initial Cost down `$2396 USD | ($599 * 8) - ($599 * 4)`
=== "Future Expansion"
Can we add more drives? Yes, this option is available, but requires additional hardware.
If it is expected to house more than 80TB in the near future, there are other options for hardware available.
!!! note "*2 Drives is not possible, as the requirements for 'fully redundant copies of the data' and 'functionality in the event of a single drive failure' mandate the use of RAID10. Where the minimum drives in a RAID10 deployment is 4 drives.*"
### M.2 NVME Drives
=== "Crucial P3 Plus"
4TB M.2 NVME [crucial product page](https://www.crucial.com/ssd/p3-plus/ct4000p3pssd8?_gl=1*13sw6no*_up*MQ..*_ga*MTMwNzgxNzQwNC4xNzE3Mjk2NjA3*_ga_6H4RYWV7QY*MTcxNzI5NjYwNy4xLjEuMTcxNzI5NjYxNC4wLjAuMjIzMjAyNjQ2&gclid=CjwKCAjwjeuyBhBuEiwAJ3vuoWUcasHvAhPD74JFCo8NZTNCemm5DIvp9fUT5ZnK-EjCZ97iyNceMhoC1d8QAvD_BwE&gclsrc=aw.ds)
- 5-year limited warranty
- 4,800 MB/s Read
- 1.5m MTBF
> `$718 USD | $359 USD (2 ea)`
### Chassis
=== "QNAP TVS-H874"
8-bay NAS: [qnap product page](https://www.qnap.com/en-us/product/tvs-h874/specs/hardware)
- EZCOPY USB3.2
Available Upgrades:
- 10/25 Gb/s networking card
- Thunderbolt 4 card
> `$2452 USD | chassis`
=== "QNAP TS-873A"
8-bay NAS: [qnap product page](https://www.qnap.com/en-us/product/ts-873a/specs/hardware)
- EZCOPY USB3.2
Requires:
- 32GB Kit (2x16GB) DDR4-3200 SODIMM: [crucial product page](https://www.crucial.com/memory/ddr4/ct2k16g4sfra32a?_gl=1*mq22r2*_up*MQ..*_ga*MTYxODEzNzExLjE3MTcyOTgzMTc.*_ga_6H4RYWV7QY*MTcxNzI5ODMxNy4xLjEuMTcxNzI5ODMyNS4wLjAuMTgxMDAzNjg0MQ..&gclid=CjwKCAjwjeuyBhBuEiwAJ3vuoQXDrg49snbi2juKZJSCftPl9NjUEollcaynaS-faIsCAvWBOMN4yxoCBooQAvD_BwE&gclsrc=aw.ds)
Available Upgrades:
- 10Gb/s networking card
- Thunderbolt 3 card
> `$1180 USD | chassis + memory`