Occasional blog posts from a random systems engineer

Blog - MattBits

last update:

Homelab storage Over the past 15 years, my homelab has seen a biiiig variety of iterations and solutions, in somewhat of an order: Random desktop machine with a hard drive First actual servers, with local hard drives Home-built NAS with iSCSI for VMs (ESX!) Random array of laptops with iPXE and NFS root drives (running out of money!) Real datacenter with iSCSI to NEXSAN SATABOY! Back home to iSCSI to Synology NAS Then, the latest upgrade about 6 years ago:

In my work life, I’ve moved away from working in a team and into an IC role. As a result, not only do I have a much longer list of tasks, but the scope and variety of those tasks has expanded significantly—more context switching, more stakeholders, more blockers, and more background work to keep track of. I’ve tried using various tools to manage this—Microsoft To Do, Jira boards, etc.—but they’re either slow and painful to update or just don’t feel like a true “go-to” system.

TL;DR: Profiling a Terraform provider revealed a performance bottleneck caused by repeated d.Get("task") calls inside a DiffSuppressFunc. The fix, implemented in PR #44543, introduces a singleTask flag to avoid repeated expensive calls, reducing plan time from ~15 minutes to seconds. A colleague told me she had issues deploying aws_appflow_flow, a Terraform resource for AWS. Every time she ran it, Terraform indicated that all task blocks were being modified, even though nothing had changed.

History Whilst at school and college, I loved running servers and bits and pieces. During this time, to host anything, I needed a domain name. Since I was also having fun minimicing services I had been working with during college (whilst working part-time at my old school), I decided to create more a brand-name rather than a personal-esque domain name. After various iterations (including maplehat, mhsolutions and others), as I left college, I had settled on dockstudios.

To kick this off, I’ve not had a good way to host binaries generated during CI/CD. At the moment, I have: Gitlab itself - I avoid using this for any binaries/OCI images mostly because the thing is bloaty enough and I do try to have complete backups without added hundreds of GBs of binaries to it. Sonartype Nexus - This is used for apt/yum caching and some scripted public binary mirroring Minio - currently used for Terraform state, Terraform provider binary mirroring (Terrashine) and some public certificates Given I’ve written Gitlab off, I’m left with bulky Nexus and now-unsupported Minio, so I wanted to try some different technologies and see what I could come up with

For a while, I’ve been trying to get away from my previous VPN - OpenVPN. It was clunky and, somewhat, unreliable. I’d start it up, give it a username, password, OTP and it would connect after about 30 seconds. Then I’d leave my laptop for 2 seconds, the screen would go blank and as soon as I shake the mouse, OpenVPN would have disconnected, removed my routes and be asking for re-authentication.

Draft: GRE tunnels

History In my homelab setup, the way services are run are not the simplest of approaches and I often find this bites me - but every challenge is an opening for learning, which is always great (unless I just want something to work). The challenge I’m talking about today is client identification with NAT outside of HTTP. What I mean is, in the world of HTTP, load balancers are often used.

The Problem! I’ve ocassionally found Terraform providers that take an attribute and manipulate it in a way that.. isn’t favourable. This means that, following runs see a change in the attribute (compared) to the attribute passed and want to modify it. For example!… The Docker Terraform provider (by kreuzwerker :D https://registry.terraform.io/providers/kreuzwerker/docker/latest/docs) manipulates the “image” attribute to the SHA digest of the image… This means, that if I create: resource "docker_container" "my_important_container" { .

Dymamic Terraform Provider I wanted to create a proof-of-concept simple method of storing and retrieving data. User X --push--> Database --read--> User Y This would be used as a sort of dictionary for users to query information being published by others. The exact nature and source of the data made Terraform an obvious choice for this. I explored the most basic option: A basic boiler-plated http call: data "http" "example" { url = "https://my-db.

For Christmas last year, I wanted to make a present for my Mum. My parents had recently had solar panels installed with a battery. The system worked, so I was told, that the solar power would: primarily power the house; then charge the battery; once filled heat water for the hot water tank She wanted to be more energy efficient, but knowing when to use the dishwasher and when to have a shower could be complex.