Skip to main content

Validation Best Practices

📍 AgentMap → Guides → Development → Validation Best Practices

This guide provides comprehensive best practices for integrating AgentMap's validation system into your development workflow. Following these practices will improve code quality, accelerate development cycles, and prevent deployment issues.

Development Workflow Integration​

Early and Frequent Validation​

Validate at Key Development Milestones:

# After creating a new workflow
agentmap validate csv --csv new_workflow.csv

# After making structural changes
agentmap validate all

# Before committing to version control
agentmap validate all --fail-on-warnings

IDE Integration Strategy:

  • Set up file watchers for automatic validation on save
  • Configure validation commands as IDE tasks
  • Use validation results for inline error highlighting

Incremental Development Approach​

Small, Validatable Changes:

# 1. Add basic workflow structure
GraphName,Node,AgentType
customer_flow,start,GPTAgent

# Validate immediately
agentmap validate csv --csv customer_flow.csv

# 2. Add routing logic incrementally
GraphName,Node,AgentType,Edge
customer_flow,start,GPTAgent,process_data
customer_flow,process_data,GPTAgent,end

# Validate after each logical addition
agentmap validate csv --csv customer_flow.csv

Benefits of Incremental Validation:

  • Early error detection
  • Easier debugging and troubleshooting
  • Faster cache-enabled validation
  • Reduced context switching

Cache-Optimized Workflow​

Leverage Caching for Speed:

# Let cache work during development
agentmap validate csv --csv workflow.csv        # ~250ms (first time)
agentmap validate csv --csv workflow.csv        # ~15ms (cached)

# Only bypass cache when necessary
agentmap validate csv --csv workflow.csv --no-cache  # Force fresh validation

Cache Management Best Practices:

  • Monitor cache hit rates: agentmap validate cache --stats
  • Clean up periodically: agentmap validate cache --cleanup
  • Clear cache after system updates: agentmap validate cache --clear

File Organization and Naming​

Consistent File Structure​

Recommended Directory Structure:

project/
├── workflows/
│   ├── main_customer_flow.csv
│   ├── admin_approval_flow.csv
│   └── error_handling_flow.csv
├── configs/
│   ├── development.yaml
│   ├── staging.yaml
│   └── production.yaml
├── custom_agents/
└── functions/

Descriptive Naming Conventions​

CSV Workflow Files:

# ✅ Good: Descriptive and consistent
customer_onboarding_flow.csv
payment_processing_workflow.csv
admin_approval_process.csv

# ❌ Avoid: Vague or inconsistent
flow1.csv
temp_workflow.csv
new_process_final_v2.csv

Node Naming Best Practices:

# ✅ Good: Clear, descriptive node names
GraphName,Node,AgentType
customer_flow,validate_email_format,ValidationAgent
customer_flow,send_welcome_email,EmailAgent
customer_flow,create_user_account,DatabaseAgent

# ❌ Avoid: Unclear or generic names
customer_flow,step1,GPTAgent
customer_flow,process,Agent
customer_flow,thing,SomeAgent

Error Handling and Resolution​

Systematic Error Resolution​

Priority-Based Approach:

  1. Fix Errors First: Address blocking errors before warnings
  2. Review Warnings: Don't ignore warnings - they indicate potential issues
  3. Optimize Info: Use informational messages for workflow optimization

Error Resolution Workflow:

# 1. Run full validation
agentmap validate all

# 2. Address errors systematically
# Fix structural issues first (missing columns, invalid syntax)
# Then fix logical issues (invalid references, routing problems)
# Finally address configuration issues (API keys, paths)

# 3. Re-validate after each fix
agentmap validate all

# 4. Address warnings after all errors are resolved
# 5. Final validation before deployment
agentmap validate all --fail-on-warnings

Common Error Patterns and Solutions​

CSV Structure Errors:

# ❌ Problem: Missing required column
GraphName,AgentType,Prompt
workflow1,GPTAgent,"Process data"

# ✅ Solution: Add required Node column
GraphName,Node,AgentType,Prompt
workflow1,process_step,GPTAgent,"Process data"

Node Reference Errors:

# ❌ Problem: Invalid node reference
GraphName,Node,Edge
workflow1,start,proces_data  # Typo in target node
workflow1,process_data,end

# ✅ Solution: Fix reference
GraphName,Node,Edge  
workflow1,start,process_data  # Correct reference
workflow1,process_data,end

Configuration Errors:

# ❌ Problem: Placeholder values
llm:
  openai:
    api_key: "your_api_key_here"

# ✅ Solution: Use environment variables
llm:
  openai:
    # api_key loaded from OPENAI_API_KEY environment variable
    model: "gpt-4"

Team Development Guidelines​

Version Control Integration​

Pre-Commit Validation:

# Add to .git/hooks/pre-commit or use pre-commit framework
#!/bin/bash
echo "Running AgentMap validation..."
agentmap validate all --fail-on-warnings

if [ $? -ne 0 ]; then
    echo "❌ Validation failed. Commit blocked."
    echo "Run 'agentmap validate all' to see details."
    exit 1
fi

echo "✅ Validation passed. Commit allowed."

Branch Protection Rules:

  • Require validation to pass before merging
  • Block merges with validation warnings in production branches
  • Allow warnings in development branches

Code Review Integration​

Validation Checklist for Reviews:

  • All validation errors resolved
  • Warnings reviewed and justified
  • Node names are descriptive and consistent
  • Graph topology is logical and complete
  • Agent types are available and appropriate
  • Configuration is production-ready

Review Guidelines:

# Reviewer workflow
git checkout feature-branch
agentmap validate all --fail-on-warnings

# Check for validation best practices
# - Clear node naming
# - Logical workflow structure  
# - Appropriate error handling
# - Complete documentation

Team Configuration Management​

Shared Configuration Standards:

# team-standards.yaml (example)
validation:
  required_checks:
    - csv_structure
    - node_references
    - agent_availability
    - configuration_completeness
  
  warning_policy: "review_required"
  error_policy: "blocking"
  
naming_conventions:
  nodes: "snake_case_descriptive"
  graphs: "purpose_based_naming"
  files: "workflow_purpose.csv"

CI/CD Integration​

Pipeline Integration Strategy​

Build Pipeline Integration:

# .github/workflows/validation.yml (example)
name: AgentMap Validation

on: [push, pull_request]

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      
      - name: Set up Python
        uses: actions/setup-python@v2
        with:
          python-version: '3.9'
          
      - name: Install AgentMap
        run: pip install agentmap
        
      - name: Validate Workflows
        run: |
          agentmap validate all --no-cache --fail-on-warnings
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
          ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

Environment-Specific Validation:

# Development: Allow warnings
agentmap validate all

# Staging: Review warnings
agentmap validate all --fail-on-warnings || echo "Review warnings before production"

# Production: Block on any issues
agentmap validate all --no-cache --fail-on-warnings

Deployment Gates​

Pre-Deployment Validation:

#!/bin/bash
# deploy.sh (example)

echo "🔍 Pre-deployment validation..."

# Clear cache for fresh validation
agentmap validate cache --clear

# Comprehensive validation with strict requirements
agentmap validate all --no-cache --fail-on-warnings

if [ $? -eq 0 ]; then
    echo "✅ Validation passed. Proceeding with deployment..."
    # Continue with deployment
else
    echo "❌ Validation failed. Deployment blocked."
    exit 1
fi

Performance Optimization in CI​

Cache Strategy for CI:

# Option 1: No cache (guaranteed fresh validation)
agentmap validate all --no-cache

# Option 2: Use cache for faster repeated builds
# (cache persists between runs on same runner)
agentmap validate all

# Option 3: Conditional cache clearing
if [ "$FORCE_FRESH_VALIDATION" = "true" ]; then
    agentmap validate cache --clear
fi
agentmap validate all

Performance Optimization​

Development Speed Optimization​

Fast Iteration Cycle:

# 1. Make small, focused changes
# 2. Validate immediately (leverage cache)
agentmap validate csv --csv modified_workflow.csv

# 3. Only run full validation when necessary
agentmap validate all

# 4. Use specific validation for targeted feedback
agentmap validate config --config updated_config.yaml

Selective Validation Strategy:

# When working on CSV files
agentmap validate csv --csv specific_workflow.csv

# When updating configuration
agentmap validate config --config specific_config.yaml

# When preparing for deployment
agentmap validate all --fail-on-warnings

Resource Management​

Cache Maintenance Schedule:

# Daily development cleanup (optional)
agentmap validate cache --cleanup

# Weekly deep cleanup
agentmap validate cache --clear

# Monitor cache effectiveness
agentmap validate cache --stats

Memory and Storage Optimization:

  • Keep workflow files reasonably sized (< 1000 rows for optimal performance)
  • Use descriptive but concise node names
  • Clean up unused workflow files regularly
  • Monitor cache directory size

Production Deployment Best Practices​

Pre-Production Validation​

Comprehensive Pre-Deployment Checklist:

# 1. Environment setup validation
echo "Checking environment variables..."
[ -z "$OPENAI_API_KEY" ] && echo "⚠️ OPENAI_API_KEY not set"
[ -z "$ANTHROPIC_API_KEY" ] && echo "⚠️ ANTHROPIC_API_KEY not set"

# 2. File accessibility validation
echo "Checking file paths..."
ls -la workflows/
ls -la configs/

# 3. Comprehensive validation
echo "Running comprehensive validation..."
agentmap validate all --no-cache --fail-on-warnings

# 4. Agent availability check
echo "Verifying agent availability..."
# Custom check for agent dependencies

# 5. Configuration verification
echo "Validating production configuration..."
agentmap validate config --config configs/production.yaml

Monitoring and Maintenance​

Production Monitoring Strategy:

# Regular validation in production environment
# (Should always pass, but good for monitoring)
agentmap validate all --config configs/production.yaml

# Log validation results for monitoring
agentmap validate all 2>&1 | tee validation.log

# Alert on validation failures
if ! agentmap validate all --fail-on-warnings; then
    # Send alert to monitoring system
    echo "Production validation failed!" | mail -s "AgentMap Alert" admin@company.com
fi

Health Check Integration:

# Include validation in application health checks
health_check() {
    # Other health checks...
    
    # Quick validation check
    agentmap validate all --fail-on-warnings &> /dev/null
    if [ $? -eq 0 ]; then
        echo "Validation: ✅ PASS"
    else
        echo "Validation: ❌ FAIL"
        return 1
    fi
}

Documentation and Knowledge Sharing​

Validation Documentation Standards​

Team Documentation Requirements:

  1. Validation Runbook: Document team-specific validation procedures
  2. Error Resolution Guide: Common errors and their solutions
  3. Configuration Standards: Team configuration conventions
  4. Deployment Procedures: Validation requirements for each environment

Example Team Runbook:

# Team Validation Runbook

## Daily Development
- Validate after significant changes: `agentmap validate csv --csv workflow.csv`
- Run full validation before committing: `agentmap validate all`

## Pre-Deployment
- Clear cache: `agentmap validate cache --clear`
- Full validation: `agentmap validate all --fail-on-warnings`
- Environment check: Verify all required environment variables

## Troubleshooting
- Cache issues: `agentmap validate cache --clear`
- Performance issues: `agentmap validate cache --stats`
- Error resolution: See error resolution guide

Knowledge Transfer​

Onboarding New Team Members:

  1. Validation Overview: Introduce validation system concepts
  2. Hands-on Practice: Walk through common validation scenarios
  3. Tool Familiarization: Practice with validation commands
  4. Best Practices Training: Team-specific guidelines and procedures

Regular Training Updates:

  • Share validation tips and tricks in team meetings
  • Document and share solutions to new validation challenges
  • Update team guidelines based on lessons learned
  • Conduct periodic validation system reviews

Next Steps​

  1. Implement Workflow: Choose validation practices that fit your development workflow
  2. Set Up Automation: Integrate validation into your CI/CD pipeline
  3. Train Team: Ensure all team members understand validation best practices
  4. Monitor Performance: Use cache statistics to optimize development speed
  5. Iterate and Improve: Regularly review and refine your validation practices