Continuous Testing: Automação Total no Pipeline

---

O Que É Continuous Testing?

Continuous Testing é a prática de executar testes automaticamente em cada mudança de código, proporcionando feedback rápido e contínuo sobre a qualidade do software.

Princípios

1. Automação completa – Sem intervenção manual
2. Feedback rápido – Resultados em minutos
3. Shift-Left – Testar mais cedo
4. Quality Gates – Critérios para avançar
5. Cobertura ampla – Múltiplos tipos de teste

---

Pipeline de Continuous Testing

Code → Commit → Build → Unit Tests → Integration → E2E → Security → Deploy
          ↓        ↓        ↓           ↓            ↓       ↓        ↓
      Pre-commit Hook   Coverage    Contract     Playwright  SAST     Staging
                       Gate        Tests        Tests     DAST

---

Quality Gates

Definição

# Quality Gates Configuration
quality_gates:
  - name: "Code Coverage"
    type: coverage
    minimum: 80  # {6727d158e4474b0847515bd23a8ee4bebd5f1aac7a18bc968e51d8985dccb442}
    blocking: true

- name: "Security Scan"
    type: security
    max_critical: 0
    max_high: 0
    blocking: true

- name: "Test Pass Rate"
    type: test
    minimum: 95  # {6727d158e4474b0847515bd23a8ee4bebd5f1aac7a18bc968e51d8985dccb442}
    blocking: true

- name: "Performance"
    type: performance
    max_p95_ms: 500
    blocking: false

- name: "Code Review"
    type: approval
    required_approvals: 1
    blocking: true

Implementação

#!/bin/bash
# quality-gate.sh

check_coverage() {
    COVERAGE=$(cat coverage.xml | grep -oP 'line-covered="K[0-9]+')
    TOTAL=$(cat coverage.xml | grep -oP 'line-count="K[0-9]+')
    PERCENTAGE=$((COVERAGE * 100 / TOTAL))

if [ $PERCENTAGE -lt 80 ]; then
        echo "❌ Coverage $PERCENTAGE{6727d158e4474b0847515bd23a8ee4bebd5f1aac7a18bc968e51d8985dccb442} below threshold (80{6727d158e4474b0847515bd23a8ee4bebd5f1aac7a18bc968e51d8985dccb442})"
        return 1
    fi
    echo "✅ Coverage $PERCENTAGE{6727d158e4474b0847515bd23a8ee4bebd5f1aac7a18bc968e51d8985dccb442}"
    return 0
}

check_security() {
    CRITICAL=$(grep -c "critical" snyk-report.json)
    if [ $CRITICAL -gt 0 ]; then
        echo "❌ $CRITICAL critical vulnerabilities found"
        return 1
    fi
    echo "✅ No critical vulnerabilities"
    return 0
}

check_tests() {
    PASS_RATE=$(python -c "print($passed / $total * 100)")
    if (( $(echo "$PASS_RATE < 95" | bc -l) )); then
        echo "❌ Pass rate $PASS_RATE{6727d158e4474b0847515bd23a8ee4bebd5f1aac7a18bc968e51d8985dccb442} below threshold (95{6727d158e4474b0847515bd23a8ee4bebd5f1aac7a18bc968e51d8985dccb442})"
        return 1
    fi
    echo "✅ Pass rate $PASS_RATE{6727d158e4474b0847515bd23a8ee4bebd5f1aac7a18bc968e51d8985dccb442}"
    return 0
}

# Run all gates
check_coverage && check_security && check_tests
exit $?

---

Shift-Left Testing

Traditional vs Shift-Left

TRADITIONAL:
Requirements → Design → Development → Testing → Production
                          ↓
                    (Late discovery of bugs)

SHIFT-LEFT:
Requirements → Testing → Development → Testing → Production
     ↓
(Early discovery of bugs)

Implementação

# .github/workflows/shift-left.yml
name: Shift-Left Pipeline

on: [push, pull_request]

jobs:
  # Stage 1: Pre-commit (Developer Machine)
  pre-commit-checks:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Pre-commit hooks
        run: |
          pip install pre-commit
          pre-commit run --all-files

# Stage 2: Build
  build:
    needs: pre-commit-checks
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Build
        run: docker build -t app:${{ github.sha }} .

# Stage 3: Unit Tests (Fast)
  unit-tests:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - run: pytest tests/unit -v --cov

# Stage 4: Security (SAST)
  security-sast:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: SonarCloud
        uses: SonarSource/sonarcloud-github-action@master

- name: Semgrep
        uses: returntocorp/semgrep-action@v1

# Stage 5: Integration Tests
  integration-tests:
    needs: [unit-tests, security-sast]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run integration tests
        run: pytest tests/integration -v

# Stage 6: E2E Tests
  e2e-tests:
    needs: integration-tests
    runs-on: ubuntu-latest
    if: github.event_name == 'push'
    steps:
      - uses: actions/checkout@v4
      - name: Playwright tests
        run: npx playwright test

# Stage 7: Deploy to Staging
  deploy-staging:
    needs: e2e-tests
    runs-on: ubuntu-latest
    environment: staging
    steps:
      - deploy to staging

---

Test Selection Strategy

# test_selector.py
import subprocess
from typing import List

class TestSelector:
    def __init__(self, changed_files):
        self.changed_files = changed_files

def select_tests(self) -> List[str]:
        tests = set()

# Unit tests (always run)
        tests.update(self.get_unit_tests())

# Integration tests (based on changes)
        tests.update(self.get_integration_tests())

# E2E tests (only for critical paths or major changes)
        if self.has_critical_changes():
            tests.update(self.get_critical_e2e_tests())

return list(tests)

def get_unit_tests(self):
        # All unit tests
        return self.get_tests_matching("tests/unit/*/.py")

def get_integration_tests(self):
        tests = set()
        for file in self.changed_files:
            if "database" in file:
                tests.add("tests/integration/test_database.py")
            if "api" in file or "service" in file:
                tests.add("tests/integration/test_api.py")
        return tests

def has_critical_changes(self):
        critical_patterns = ["checkout", "payment", "auth", "core"]
        return any(
            any(p in f for p in critical_patterns) 
            for f in self.changed_files
        )

def get_critical_e2e_tests(self):
        return [
            "tests/e2e/test_checkout.py",
            "tests/e2e/test_login.py",
            "tests/e2e/test_payment.py"
        ]

---

Feedback Loop

# Slack notification
name: Notify on Failure
  if: failure()
  uses: slackapi/slack-github-action@v1
  with:
    channel-id: 'qa-alerts'
    payload: |
      {
        "text": "❌ Build #${GITHUB_RUN_NUMBER} failed",
        "blocks": [{
          "type": "section",
          "text": {
            "type": "mrkdwn",
            "text": "Build Failed :x:n${GITHUB_REPOSITORY}n${GITHUB_RUN_URL}"
          }
        }]
      }

---

Métricas de Continuous Testing

def continuous_testing_metrics():
    return {
        "test_cycle_time": {
            "description": "Tempo do commit ao deploy",
            "current": "45 min",
            "target": "30 min",
            "trend": "improving"
        },
        "test_automation_rate": {
            "description": "{6727d158e4474b0847515bd23a8ee4bebd5f1aac7a18bc968e51d8985dccb442} de testes automatizados",
            "current": "85{6727d158e4474b0847515bd23a8ee4bebd5f1aac7a18bc968e51d8985dccb442}",
            "target": "90{6727d158e4474b0847515bd23a8ee4bebd5f1aac7a18bc968e51d8985dccb442}"
        },
        "quality_gate_pass_rate": {
            "description": "{6727d158e4474b0847515bd23a8ee4bebd5f1aac7a18bc968e51d8985dccb442} de builds passando em quality gates",
            "current": "92{6727d158e4474b0847515bd23a8ee4bebd5f1aac7a18bc968e51d8985dccb442}",
            "target": "95{6727d158e4474b0847515bd23a8ee4bebd5f1aac7a18bc968e51d8985dccb442}"
        },
        "defect_leakage_rate": {
            "description": "{6727d158e4474b0847515bd23a8ee4bebd5f1aac7a18bc968e51d8985dccb442} de defeitos encontrados em produção",
            "current": "8{6727d158e4474b0847515bd23a8ee4bebd5f1aac7a18bc968e51d8985dccb442}",
            "target": "<5{6727d158e4474b0847515bd23a8ee4bebd5f1aac7a18bc968e51d8985dccb442}"
        }
    }

---

Conclusão

Continuous Testing é essencial para DevOps. As chaves são:

1. Automação total – Sem intervenção manual
2. Feedback rápido – Minutos, não horas
3. Quality gates – Critérios claros
4. Shift-Left – Testar o mais cedo possível
5. Seleção inteligente – Não rodar tudo sempre