This commit is contained in:
Mike McFetridge
2026-07-20 09:23:17 -04:00
parent c1315882da
commit 72272e4006
3179 changed files with 562960 additions and 14 deletions
@@ -0,0 +1,73 @@
---
tags:
- GIT
- CI/CD
- CI-CD
---
If your `docker compose up -d` command encounters broken network routing, bad volume mappings, or unparseable image tags, the deployment step will fail. By utilizing standard shell behaviors and Gitea's `if: failure()` conditions, you can instruct your runner to automatically execute a fallback deployment command using the previous stable version.
1. Setup the Backup Directory
To safely fall back to a working state, the runner needs access to the _previous_ commit's configuration. The cleanest way to do this without messing up your live directories is to copy your files into a temporary backup workspace on the live machine before running the fresh upgrade.
2. The Complete Automated Rollback Workflow File
Here is the deployment and rollback code pattern. Replace your existing deployment job block with this comprehensive configuration:
deploy-with-rollback:
runs-on: ubuntu-latest
needs: build-and-push # Ensures we only deploy if the container image successfully builds
steps:
- name: Checkout Repository Code
uses: actions/checkout@v4
- name: Set up SSH Routing
run: |
mkdir -p ~/.ssh
echo "${{ secrets.DEPLOY_SSH_KEY }}" > ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
ssh-keyscan -H ${{ secrets.DEPLOY_HOST }} >> ~/.ssh/known_hosts
- name: Create Remote Docker Context
run: |
docker context create homelab-target \
--docker "host=ssh://${{ secrets.DEPLOY_USER }}@${{ secrets.DEPLOY_HOST }}"
docker context use homelab-target
# --- STEP A: BACKUP PREVIOUS LIVE ENVIRONMENT STATE ---
- name: Stash Current Working Configuration
run: |
# Use SSH to quickly copy the current running directory configuration to a backup spot
ssh ${{ secrets.DEPLOY_USER }}@${{ secrets.DEPLOY_HOST }} \
"mkdir -p /tmp/homelab_backup && cp -r ~/homelab_stack/* /tmp/homelab_backup/ || true"
# --- STEP B: FRESH ATTEMPTED DEPLOYMENT ---
- name: Sync New Config Files and Deploy
id: deploy_attempt
run: |
# Copy your freshly fetched git workspace configs to your server deployment path
scp -r ./* ${{ secrets.DEPLOY_USER }}@${{ secrets.DEPLOY_HOST }}:~/homelab_stack/
# Attempt the live container stack upgrade
docker compose --project-directory ~/homelab_stack up -d --remove-orphans
# --- STEP C: AUTOMATIC FALLBACK ROLLBACK (Triggers only if Step B crashes) ---
- name: Execute Emergency Rollback
if: failure() && steps.deploy_attempt.outcome == 'failure'
run: |
echo "⚠️ Deployment crashed! Restoring last known good stack..."
# Restore the stashed backup file configuration over the broken files
ssh ${{ secrets.DEPLOY_USER }}@${{ secrets.DEPLOY_HOST }} \
"cp -r /tmp/homelab_backup/* ~/homelab_stack/ && rm -rf /tmp/homelab_backup"
# Rerun Docker Compose using the restored working configurations
docker compose --project-directory ~/homelab_stack up -d --remove-orphans
echo "✅ Rollback completed successfully."
🧠 Advanced Breakdown for Homelabs
- **`steps.deploy_attempt.outcome == 'failure'`**: This condition specifically targets the core deployment run. If your build or your SSH setups break, the rollback logic ignores it. It _only_ swings into action if the `docker compose up` execution itself returns an error code on your target server.
- **Why Separate Config & Images?**: Because you are using the Gitea Container Registry, images are tagged with the specific commit hash (`:${{ gitea.sha }}`). If you need to roll back, moving back to the old `docker-compose.yml` config will point Docker back to the _previous_ commit's image tag. This safely pulls and starts the old container version even if you pushed a broken image to `latest`.
@@ -0,0 +1,81 @@
---
tags:
- GIT
- CI/CD
- CI-CD
---
🚀 Part 2: Automatically Deploy to a Live Server
The safest, cleanest way to deploy to a remote homelab server from a runner is using **SSH and Docker Contexts**. This allows your Gitea Actions runner to securely command your live server's Docker daemon across your local network without needing Git installed on the destination server.
1. Generate a Deployment SSH Key [[1](https://ecostack.dev/posts/automated-docker-compose-deployment-github-actions/)]
On your local machine or server, generate a dedicated SSH key pair for Gitea Actions. **Do not use a passphrase.**
ssh-keygen -t ed25519 -f ~/.ssh/gitea_deploy_key -N ""
- Append the contents of `gitea_deploy_key.pub` to the `~/.ssh/authorized_keys` file on your **live deployment server**.
- Copy the contents of the private key `gitea_deploy_key`. Save it in your Gitea Repository Secrets as **`DEPLOY_SSH_KEY`**.
- Save your live server's local IP address or local hostname in Gitea Secrets as **`DEPLOY_HOST`**.
- Save your live server's SSH username in Gitea Secrets as **`DEPLOY_USER`**.
2. Create the Complete Production Workflow
Create or update your `.gitea/workflows/deploy.yaml` file. This workflow will validate your code first, and then deploy it only if the branch is `main`.
name: Validate and Deploy Homelab
on:
push:
branches: [ "main" ]
jobs:
test-and-deploy:
runs-on: ubuntu-latest
steps:
- name: Checkout Repository Code
uses: actions/checkout@v4
- name: Verify Compose File Validity
env:
DB_ROOT_PASSWORD: ${{ secrets.MYSQL_ROOT_PASSWORD }}
DB_PASSWORD: ${{ secrets.MYSQL_PASSWORD }}
run: docker compose config
# --- DEPLOYMENT PHASE ---
- name: Set up SSH Private Key
run: |
mkdir -p ~/.ssh
echo "${{ secrets.DEPLOY_SSH_KEY }}" > ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
# Scan the host key to prevent SSH hanging on a manual confirmation prompt
ssh-keyscan -H ${{ secrets.DEPLOY_HOST }} >> ~/.ssh/known_hosts
- name: Create Remote Docker Context
run: |
# Define a remote endpoint pointing to your live server over SSH
docker context create homelab-target \
--docker "host=ssh://${{ secrets.DEPLOY_USER }}@${{ secrets.DEPLOY_HOST }}"
- name: Deploy Containers to Live Server
env:
# Pass your production secrets to the live deployment step
DB_ROOT_PASSWORD: ${{ secrets.MYSQL_ROOT_PASSWORD }}
DB_PASSWORD: ${{ secrets.MYSQL_PASSWORD }}
run: |
echo "Switching Docker context to live server..."
docker context use homelab-target
echo "Pulling latest images and starting services remotely..."
# The --project-directory . flag ensures it reads the docker-compose file fetched by Git
docker compose --project-directory . up -d --remove-orphans
💡 Why This Setup Wins for Homelabs
- **Zero Overhead:** You do not need to install complex deployment tools or agents on your production server. It only requires standard SSH and Docker.
- **Security:** If you ever need to revoke the runner's deployment access, you simply delete the single public key from your live server's `authorized_keys` file.
- **Atomic Changes:** The `--remove-orphans` flag cleans up old containers that you removed from your Compose file since the last Git commit, keeping your server immaculate.
@@ -0,0 +1,56 @@
---
tags:
- GIT
- CI/CD
- CI-CD
---
Building images and storing them inside Gitea's built-in registry (`://example.com`) ensures that you can decouple your actual build step from the live deployment server. It also provides a snapshot history of your homelab services.
1. Update your `docker-compose.yml` to point to Gitea [[1](https://forum.gitea.com/t/strange-behavior-when-upgrading-to-1-22-1-from-1-15-9/9664)]
Change your local container configuration so that instead of building in place, it pulls a tagged image hosted right inside your Gitea instance:
services:
custom-app:
# Point the image path directly to your self-hosted Gitea registry URL
image: gitea.homelab.local/${{ gitea.repository_owner }}/custom-app:latest
ports:
- "8080:80"
2. Create the Build & Push Job Step
Add this block right before your deployment phase. Gitea Actions automatically exposes a temporary validation token (`${{ gitea.token }}`) for every run, which means you don't need to manually configure registry usernames or passwords in your secrets dashboard! [[1](https://docs.gitlab.com/user/packages/container_registry/authenticate_with_container_registry/)]
build-and-push:
runs-on: ubuntu-latest
steps:
- name: Checkout Code
uses: actions/checkout@v4
# Log directly into your self-hosted Gitea Container Registry
- name: Log in to Gitea Registry
uses: https://github.com
with:
registry: gitea.homelab.local # Replace with your real homelab Gitea domain or local IP
username: ${{ gitea.actor }}
password: ${{ gitea.token }}
# Extract metadata to properly tag your image with the Git commit hash and "latest"
- name: Extract Docker Metadata
id: meta
uses: https://github.com
with:
images: gitea.homelab.local/${{ gitea.repository }}
tags: |
type=raw,value=latest
type=sha,value=${{ gitea.sha }}
# Build the Dockerfile and push it natively to Gitea
- name: Build and Push Docker Image
uses: https://github.com
with:
context: ./app_directory # Location of your Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
@@ -0,0 +1,65 @@
---
tags:
- GIT
- CI/CD
- CI-CD
---
In a homelab, your production server usually has unique storage mounts (like `/mnt/user/appdata/nginx` or `/mnt/SSD/data`), but your Gitea runner is isolated and lacks access to those specific paths. If you try to run a verification command like `docker compose config` inside the runner, it will often fail or generate unwanted folders.
Here are the two cleanest ways to solve this:
Solution 1: Use Environment Variables for Paths (Recommended)
Instead of hardcoding absolute paths into your `docker-compose.yml`, replace them with environment variables.
Update your `docker-compose.yml` to use a variable:
services:
nginx:
image: nginx:alpine
volumes:
# Use a variable with a safe local fallback directory for testing
- ${DATA_PATH_NGINX:-./nginx_test_data}:/etc/nginx
**Map the variables differently in your workflow steps:**
When **validating** on the runner, let it fall back to the safe relative directory. When **deploying** to production, explicitly inject your real server paths.
# Step A: Validate (Uses the relative path fallback automatically)
- name: Verify Compose File Validity
run: docker compose config
# Step B: Deploy (Injects your actual live server paths)
- name: Deploy Containers to Live Server
env:
DATA_PATH_NGINX: "/mnt/user/appdata/nginx" # Real production path
run: |
docker context use homelab-target
docker compose --project-directory . up -d --remove-orphans
Solution 2: Store Production Paths in Gitea Secrets
If you want to keep your specific homelab storage architecture hidden from your Git repository code entirely, you can save the base storage directory directly as a Gitea Secret (e.g., `BASE_DATA_PATH = /mnt/user/appdata`).
1. **Reference the secret variable in your `docker-compose.yml`:**
services:
plex:
image: plexinc/pms
volumes:
- ${BASE_DATA_PATH}/plex:/config
Provide a dummy path during validation, and the real secret during deployment:
- name: Verify Compose File Validity
env:
BASE_DATA_PATH: "./test_env" # Dummy folder for the runner
run: docker compose config
- name: Deploy Containers to Live Server
env:
BASE_DATA_PATH: ${{ secrets.BASE_DATA_PATH }} # Real server path
run: |
docker context use homelab-target
docker compose --project-directory . up -d --remove-orphans
💡 Putting It Together
By combining these two methods, your workflow will safely parse your configuration using fake/relative folders on the runner service, seamlessly deploy using your live production hard drive arrays over SSH, and instantly ping your phone via Discord or Telegram with the results.
@@ -0,0 +1,41 @@
---
tags:
- GIT
- CI/CD
- CI-CD
---
🔒 Part 1: Inject Gitea Secrets into Compose Files Safely
Hardcoding sensitive data like passwords, API keys, or database credentials into your Git repository is a major security risk. Instead, you should store them in Gitea and pass them dynamically at runtime. [[1](https://www.softrams.com/post/deploying-kong-gateway-in-db-less-mode-on-kubernetes), [2](https://medium.com/@saumya07013/safely-managing-secrets-in-containers-best-practices-and-strategies-1b71e49b1525), [3](https://www.howtogeek.com/devops/how-to-protect-sensitive-secrets-and-credentials-in-your-git-repository/), [4](https://glmdev.medium.com/code-freedom-with-gitea-drone-ci-part-i-4b9dfbce1514)]
1. Add Secrets to Gitea
2. Go to your repository in Gitea.
3. Navigate to **Settings** ➡️ **Actions** ➡️ **Secrets**.
4. Click **貯/Add Secret**.
5. Add your variables (e.g., Key: `MYSQL_PASSWORD`, Value: `super_secret_password_123`). [[1](https://blog.devgenius.io/setting-up-a-simple-ci-cd-with-django-gitea-jenkins-and-of-course-docker-ab883972efb5), [2](https://testautomationu.applitools.com/observability-for-test-automation/chapter11.html), [3](https://medium.com/google-developer-experts/how-to-store-sensitive-data-on-gcp-d96e4e545224)]
6. Update Your `docker-compose.yml`
Configure your Compose file to look for standard environment variables. Do not include default values here.
services:
database:
image: mysql:8.0
environment:
- MYSQL_ROOT_PASSWORD=${DB_ROOT_PASSWORD}
- MYSQL_PASSWORD=${DB_PASSWORD}
3. Update Your Gitea Workflow File
You can map Gitea Secrets into an environment block right inside your workflow step. When `docker compose` runs, it will read those variables from the runner's system environment and inject them into the container definition. [[1](https://www.dataquest.io/blog/intro-to-docker-compose/)]
- name: Verify Compose File Validity
env:
# Map Gitea Secrets to the environment variables expected by your compose file
DB_ROOT_PASSWORD: ${{ secrets.MYSQL_ROOT_PASSWORD }}
DB_PASSWORD: ${{ secrets.MYSQL_PASSWORD }}
run: |
echo "Injecting secrets and testing configuration..."
docker compose config
@@ -0,0 +1,46 @@
---
tags:
- GIT
- CI/CD
- CI-CD
---
Here is how to add instant notifications to your setup and solve the common homelab issue of mismatched storage paths between your testing runner and your live production server.
---
🔔 Part 1: Send a Discord or Telegram Notification
You can easily append a notification step to the very end of your workflow. By utilizing Gitea's conditional blocks (`if: always()`), you can ensure you get notified whether the deployment succeeds or crashes.
Option A: Sending to Discord
1. Open your Discord server, go to your target channel settings, click **Integrations**, and create a **Webhook**. Copy the URL.
2. Store that URL in your Gitea Repository Secrets as `DISCORD_WEBHOOK`.
- name: Send Discord Status Notification
if: always() # Ensures this runs even if the deployment failed
uses: https://github.com
with:
verify: false
url: ${{ secrets.DISCORD_WEBHOOK }}
# Evaluates the final outcome of the job to change the message dynamically
title: "Homelab Deployment: ${{ job.status == 'success' && '✅ SUCCESS' || '❌ FAILED' }}"
description: "The commit '${{ gitea.event.head_commit.message }}' by ${{ gitea.actor }} has finished processing."
color: ${{ job.status == 'success' && '0x00FF00' || '0xFF0000' }}
Option B: Sending to Telegram
1. Message `@BotFather` on Telegram to create a new bot and copy your **Bot Token**.
2. Message `@userinfobot` to find your personal **Chat ID**.
3. Save these in Gitea Secrets as `TELEGRAM_TOKEN` and `TELEGRAM_TO`.
- `name: Send Telegram Status Notification`
`if: always()`
`uses: https://github.com`
`with:`
`token: ${{ secrets.TELEGRAM_TOKEN }}`
`to: ${{ secrets.TELEGRAM_TO }}`
`message: |`
`Homelab Deploy Status: ${{ job.status }}`
`Actor: ${{ gitea.actor }}`
`Commit: ${{ gitea.event.head_commit.message }}`
@@ -0,0 +1,67 @@
---
tags:
- GIT
- CI/CD
- CI-CD
---
To test and validate your `docker-compose` infrastructure configurations right inside your homelab, you can use Gitea Actions to automatically check your YAML syntax and test your container builds every time you make an edit. [[1](https://mortylen.hashnode.dev/gitea-self-hosted-workflow-action-for-ci), [2](https://ramnode.com/guides/series/gitea-devops/gitea-setup)]
Here is how to set up an automated testing pipeline specifically for a homelab Docker Compose repository.
🛠️ Step 1: Set Up the Workflow Directory
In your homelab Git repository containing your `docker-compose.yml` files, create the specific workflow path: [[1](https://www.stackhero.io/en-CA/services/Docker/documentations/Deploy-with-GitHub-Actions)]
- **File path:** `.gitea/workflows/docker-test.yaml`
📝 Step 2: Paste the Homelab Validation Template
Copy and paste this template into your new file. It is optimized to perform two critical steps: lint your YAML formatting for typos, and dry-run your Docker Compose setup to ensure your container configurations are valid. [[1](https://www.reddit.com/r/docker/comments/1i32i3z/one_large_compose_file/), [2](https://www.docker.com/blog/compose-to-kubernetes-to-cloud-kanvas/)]
`name: Docker Compose Validator`
`run-name: Homelab configuration test by ${{ gitea.actor }} 🛠️`
`on:`
`push:`
`branches: [ "main", "master" ]`
`pull_request:`
`branches: [ "main", "master" ]`
`jobs:`
`validate-and-test:`
`runs-on: ubuntu-latest`
`steps:`
`# 1. Pull down your homelab repository code`
`- name: Checkout Repository Code`
`uses: actions/checkout@v4`
`# 2. Check all YAML files for syntax, indentation, and formatting errors`
`- name: Lint YAML Syntax`
`uses: https://github.com`
`with:`
`file_or_dir: "."`
`# 3. Verify that the docker-compose syntax is structurally sound`
`- name: Verify Compose File Validity`
`run: |`
`echo "Checking docker-compose configuration..."`
`docker compose config`
`# 4. Dry-run the build sequence to ensure images and contexts are reachable`
`- name: Test Build Sequence`
`run: |`
`echo "Simulating container build sequence..."`
`docker compose build`
🔍 Breaking Down How This Works for Homelabbers
- **YAML Linting:** Homelab configurations frequently break due to an accidental extra space or missing indentation. The `action-yamllint` step scans your files and alerts you to formatting errors before they hit your live system.
- **`docker compose config`:** This command acts as a built-in validator. It reads your `docker-compose.yml` file, processes any variables, and outputs an error if you have typos in your keys, volume definitions, or network syntax. [[1](https://labex.io/tutorials/docker-how-to-use-docker-compose-config-command-to-validate-and-view-compose-files-555074), [2](https://www.reddit.com/r/docker/comments/13vupxt/switching_from_portainer_to_dockercompose_and/), [3](https://medium.com/@adriansyah1230/mastering-docker-compose-a-practical-guide-to-multi-container-applications-c76811010131)]
- **`docker compose build`:** If your compose files use the `build:` block to compile local Dockerfiles instead of just pulling pre-made images, this step ensures your Dockerfiles successfully build without crashing.
⚠️ Crucial Homelab Gotchas to Keep in Mind
1. **Environment Variables (`.env`):** If your `docker-compose.yml` relies on an external `.env` file containing local paths, secret tokens, or personal domain names, `docker compose config` will fail in Gitea Actions because that file is missing. You should either add a `.env.example` file to your Git repository and rename it during the workflow run, or map those variables into **Gitea Secrets**.
2. **Local Volume Paths:** If your compose files map to absolute local storage paths (e.g., `/mnt/user/appdata/nginx`), the Gitea Act Runner environment won't have access to those actual directories. For testing purposes, it is safest to use relative paths (e.g., `./nginx/config`) or let Docker generate named volumes during the validation phase.
3. **Runner Capabilities:** Ensure that your local `act_runner` setup has access to a Docker daemon. If you installed your runner using Docker-in-Docker (DinD), it will easily execute the `docker compose` commands embedded in this workflow. [[1](https://versich.com/blog/a-practical-guide-to-building-docker-image-in-gitlab/)]