Blog - MattBits

Before I get into GitLab functions and components, I wanted to cover how I manage build and deployment artifacts, as it’s a dependency for a lot of what comes later. I generally block internet access across my homelab. Aside from specific external APIs, everything is either pre-downloaded or accessed through a caching proxy. I use Nexus for PyPI, apt, go etc. package pull-through caches. For binaries, however, I’ve moved a couple of times from a basic machine (called Vault - unfortunately this 15 year old machine pre-dated me starting to use Hashicorp Vault), through various technologies and now use Nexus.

Homelab history I’ve been running some form of home infrastructure for a long time now. Looking back through old notes and photos, it’s interesting seeing how much it’s changed over the years. Some of it was planned, a lot of it wasn’t, and there were more than a few dead ends along the way. Like most long-running homelabs, it grew one project at a time rather than from any grand plan.

Over the past couple of days, I’ve had a couple of things that have been annoyed me and have resulted in a learning and a success! Dependency OOMs I’ve been doing a bit more work on my virtual machine agent and had been trying to build it and began getting OOM errors. I followed some of the failures (it was a strong assumption that the library it was failing to build was the culprit, rather than the straw that broke the camel’s back).

Having gotten an AMD AI 395+-based machine this month, I’ve been using Lemonade to run models, which takes care of bootstrapping RocM, I now wanted to bring this into a workflow I had been tested a couple of months ago which was toying around with making a game. My goal was to create a point-and-click game, but given that I’m awful at graphics, I had been taking photos and using AI to generate the style of image and to make tweaks in preparation to build into a game.

This is a summary of how I built and tuned a local OpenCode-style coding setup on AMD ROCm using Lemonade + llama.cpp, moving through multiple models, context issues, and performance tuning before settling on Gamma 26B A4B Instruct (IT). 1. Base Setup Started with Lemonade running llama.cpp on ROCm via Docker: docker run -d \ --name lemonade-server \ -p 13305:13305 \ -v lemonade-cache:/root/.cache/huggingface \ -v lemonade-llama:/opt/lemonade/llama \ ghcr.

The beginning I’ve been running a Hashicorp stack for a while now - Vault, Consul and Nomad across a small cluster of machines. Over time, I’d built up a pretty reasonable setup, but it was always a bit… scattered. Terraform here, manual config there, some docker-compose files scattered about. The thing is, I’m running all of this on self-hosted hardware. There’s no AWS ECS or Google Cloud Run to fall back on.

This is a pledge to myself and for any one else interested in following the same journey. I don’t know jack about Kubernetes. I know about docker, integrated with it’s API, cgroups etc. I’ve used nomad and docker swarm heavily and rancher. History I have taken steps in the past to master technologies in various levels, for example Terraform, nomad Terraform Used Terraform for about a decade, managed and supporting hundreds of projects using it Working knowledge to create agnostic re-usable modules Created and fixed bugs in Terraform providers Created a Terraform module and provider registry Created a working PoC of an open source Terraform cloud alternative: implementing the Terraform cloud APIs, reverse engineered the official Terraform cloud agent APIs to a fully functional state implemented missing features (such as environments) working implementation of a custom state Vault state backend into OpenTofu (https://github.

My homelab and work life have followed very different trajectories, though often influencing one another. I like to try out interesting thought experiments at home, see how they work out to determine whether they’re worth investing in. Right now, my homelab consists of a load of VMs and my initial goal was to find a new way of monitoring (often these small tasks lead to a big spray of different tasks).

What is this? I work in close proximity to a lot of AI/ML/big data engineers and have friends who are of the same variety. I somewhat understand the basic concepts of a neural network and I have some knowledge of vectors etc. but I’m tired of being the one left confused when having conversations… So, my plan is to get stuck in a little.. at least to train something and to work my way into a problem enough that I get confused.

Homelab storage Over the past 15 years, my homelab has seen a biiiig variety of iterations and solutions, in somewhat of an order: Random desktop machine with a hard drive First actual servers, with local hard drives Home-built NAS with iSCSI for VMs (ESX!) Random array of laptops with iPXE and NFS root drives (running out of money!) Real datacenter with iSCSI to NEXSAN SATABOY! Back home to iSCSI to Synology NAS Then, the latest upgrade about 6 years ago: