Compose-Files/Backups/Miker/Learning CI-CD/Automatic Deployment Rollback.md

tags

tags
GIT CI/CD CI-CD

If your docker compose up -d command encounters broken network routing, bad volume mappings, or unparseable image tags, the deployment step will fail. By utilizing standard shell behaviors and Gitea's if: failure() conditions, you can instruct your runner to automatically execute a fallback deployment command using the previous stable version.

1. Setup the Backup Directory

To safely fall back to a working state, the runner needs access to the previous commit's configuration. The cleanest way to do this without messing up your live directories is to copy your files into a temporary backup workspace on the live machine before running the fresh upgrade.

2. The Complete Automated Rollback Workflow File

Here is the deployment and rollback code pattern. Replace your existing deployment job block with this comprehensive configuration:

deploy-with-rollback: runs-on: ubuntu-latest needs: build-and-push # Ensures we only deploy if the container image successfully builds

steps:
  - name: Checkout Repository Code
    uses: actions/checkout@v4

  - name: Set up SSH Routing
    run: |
      mkdir -p ~/.ssh
      echo "${{ secrets.DEPLOY_SSH_KEY }}" > ~/.ssh/id_ed25519
      chmod 600 ~/.ssh/id_ed25519
      ssh-keyscan -H ${{ secrets.DEPLOY_HOST }} >> ~/.ssh/known_hosts

  - name: Create Remote Docker Context
    run: |
      docker context create homelab-target \
        --docker "host=ssh://${{ secrets.DEPLOY_USER }}@${{ secrets.DEPLOY_HOST }}"
      docker context use homelab-target

  # --- STEP A: BACKUP PREVIOUS LIVE ENVIRONMENT STATE ---
  - name: Stash Current Working Configuration
    run: |
      # Use SSH to quickly copy the current running directory configuration to a backup spot
      ssh ${{ secrets.DEPLOY_USER }}@${{ secrets.DEPLOY_HOST }} \
        "mkdir -p /tmp/homelab_backup && cp -r ~/homelab_stack/* /tmp/homelab_backup/ || true"

  # --- STEP B: FRESH ATTEMPTED DEPLOYMENT ---
  - name: Sync New Config Files and Deploy
    id: deploy_attempt
    run: |
      # Copy your freshly fetched git workspace configs to your server deployment path
      scp -r ./* ${{ secrets.DEPLOY_USER }}@${{ secrets.DEPLOY_HOST }}:~/homelab_stack/
      
      # Attempt the live container stack upgrade
      docker compose --project-directory ~/homelab_stack up -d --remove-orphans

  # --- STEP C: AUTOMATIC FALLBACK ROLLBACK (Triggers only if Step B crashes) ---
  - name: Execute Emergency Rollback
    if: failure() && steps.deploy_attempt.outcome == 'failure'
    run: |
      echo "⚠️ Deployment crashed! Restoring last known good stack..."
      
      # Restore the stashed backup file configuration over the broken files
      ssh ${{ secrets.DEPLOY_USER }}@${{ secrets.DEPLOY_HOST }} \
        "cp -r /tmp/homelab_backup/* ~/homelab_stack/ && rm -rf /tmp/homelab_backup"
        
      # Rerun Docker Compose using the restored working configurations
      docker compose --project-directory ~/homelab_stack up -d --remove-orphans
      echo "✅ Rollback completed successfully."

🧠 Advanced Breakdown for Homelabs

• steps.deploy_attempt.outcome == 'failure': This condition specifically targets the core deployment run. If your build or your SSH setups break, the rollback logic ignores it. It only swings into action if the docker compose up execution itself returns an error code on your target server.
• Why Separate Config & Images?: Because you are using the Gitea Container Registry, images are tagged with the specific commit hash (:${{ gitea.sha }}). If you need to roll back, moving back to the old docker-compose.yml config will point Docker back to the previous commit's image tag. This safely pulls and starts the old container version even if you pushed a broken image to latest.