CLI Validation Commands

📍 AgentMap → Deployment → CLI Validation Commands

AgentMap provides comprehensive CLI validation commands built on the runtime facade pattern to ensure your workflows, configuration files, and system setup are correct before execution. These commands integrate seamlessly with development workflows and CI/CD pipelines.

Validation Architecture

Facade Pattern Implementation

All validation commands follow AgentMap's facade pattern, using runtime_api.py for business logic and cli_presenter.py for consistent output:

# Standard validation command pattern
from agentmap import validate_workflow
from agentmap.deployment.cli.utils.cli_presenter import print_json, print_err, map_exception_to_exit_code

def validate_command(args):
    try:
        # Use runtime facade for validation logic
        result = validate_workflow(graph_name, config_file=args.config)
        
        # Use CLI presenter for consistent output
        print_json(result)
        
    except Exception as e:
        print_err(str(e))
        exit_code = map_exception_to_exit_code(e)
        raise typer.Exit(code=exit_code)

Runtime API Integration

Validation commands use these runtime facade functions:

Function	Purpose	Commands Using
`validate_workflow()`	CSV and graph validation	`validate`
`validate_cache()`	Cache integrity checking	`validate-cache`
`ensure_initialized()`	Runtime system validation	Most commands

CLI Presenter Integration

All validation commands use the standardized CLI presenter utilities:

Consistent Error Handling: Maps validation exceptions to appropriate exit codes
JSON Output: Structured validation results with custom encoding for AgentMap objects
Stderr Management: Error messages output to stderr for script compatibility

Bundle-Based Validation

The validation system uses bundle analysis to provide comprehensive workflow validation:

# Example validation workflow
def validate_command(csv_file, graph, config_file):
    try:
        # Resolve CSV path using utility functions
        csv_path = resolve_csv_path(csv_file, csv)
        
        # Use runtime facade for validation
        result = validate_workflow(graph_name, config_file=config_file)
        
        # Bundle analysis results
        outputs = result["outputs"]
        metadata = result["metadata"]
        
        # Report comprehensive validation results
        print(f"Total nodes: {outputs['total_nodes']}")
        print(f"Total edges: {outputs['total_edges']}")
        
        if outputs["missing_declarations"]:
            print(f"Missing agents: {', '.join(outputs['missing_declarations'])}")
        
    except Exception as e:
        handle_command_error(e, verbose=False)

Validation Pattern Benefits

Facade Pattern Advantages:

Consistent Interface: All validation commands use the same runtime facade
Error Standardization: Unified exception handling and exit code mapping
Output Formatting: Consistent JSON output with custom AgentMap object encoding
Business Logic Separation: Validation logic encapsulated in the runtime layer

CLI Presenter Benefits:

Script Compatibility: Structured JSON output for automation
Human Readability: Optional pretty formatting for development
Error Handling: Standardized stderr output and exit codes

Command Overview

The validation system provides several specialized commands:

agentmap validate csv: Validate CSV workflow files with graph consistency checks
agentmap validate config: Validate YAML configuration files with schema validation
agentmap validate all: Comprehensive validation of both CSV and config files
agentmap validate cache: Manage validation cache for performance optimization

CSV Validation Command

Validate CSV workflow definition files with comprehensive structure and logic checks.

Basic Usage

agentmap validate csv [OPTIONS]

Options

Option	Short	Description	Default
`--csv`	`-c`	Path to CSV file to validate	Uses config default
`--no-cache`		Skip cache and force re-validation	`false`
`--config`		Path to custom config file	`agentmap_config.yaml`

Examples

Validate Specific CSV File:

agentmap validate csv --csv workflows/customer_onboarding.csv

Force Fresh Validation (Bypass Cache):

agentmap validate csv --csv workflows/customer_onboarding.csv --no-cache

Use Custom Configuration:

agentmap validate csv --csv workflows/flow.csv --config configs/development.yaml

Validate Default CSV (from config):

agentmap validate csv

Sample Output

Successful Validation:

🔍 Validating CSV file: workflows/customer_onboarding.csv
✅ CSV file format is valid
ℹ️ CSV contains 12 rows and 7 columns
ℹ️ Found 1 graph(s): 'customer_onboarding' (12 nodes)
ℹ️ Found 4 unique agent types: GPTAgent, HumanAgent, EmailAgent, ValidationAgent
ℹ️ Graph 'customer_onboarding' has multiple potential entry points: 'start', 'manual_entry'
ℹ️ Node 'final_approval' has no outgoing edges (terminal node)
✅ CSV validation passed!

Validation with Issues:

🔍 Validating CSV file: workflows/broken_workflow.csv

❌ CSV Validation Errors:
  1. Duplicate node 'validate_email' in graph 'customer_flow'
     Line 5
  2. Node 'process_payment' references non-existent target 'send_confirmtion' in Edge
     Line 6, Field: Edge, Value: send_confirmtion
     Suggestion: Valid targets: validate_email, process_payment, send_confirmation

⚠️ CSV Validation Warnings:
  1. Unknown agent type: 'GPTAgnet'
     Line 3, Field: AgentType, Value: GPTAgnet
     Suggestion: Check spelling or ensure agent is properly registered/available
  2. Node 'complex_analysis' has a large prompt (500+ characters)
     Line 8, Field: Prompt
     Suggestion: Consider breaking into smaller, focused prompts

ℹ️ Validation complete with 2 errors and 2 warnings

Validation Checks Performed

Structure Validation:

Required columns presence (GraphName, Node)
Optional columns recognition
Column alias support and normalization
Data format and consistency

Row-Level Validation:

Required field completeness
Input/Output field format validation
Routing logic consistency
Field value constraints

Graph Consistency:

Duplicate node detection within graphs
Node reference validation
Entry and terminal point identification
Routing path verification

Agent Validation:

Agent type availability checking
Agent registry verification
Custom agent accessibility

Config Validation Command

Validate YAML configuration files with comprehensive schema and environment checks.

Basic Usage

agentmap validate config [OPTIONS]

Options

Option	Short	Description	Default
`--config`	`-c`	Path to config file to validate	`agentmap_config.yaml`
`--no-cache`		Skip cache and force re-validation	`false`

Examples

Validate Default Configuration:

agentmap validate config

Validate Specific Configuration:

agentmap validate config --config configs/production.yaml

Force Fresh Validation:

agentmap validate config --config configs/staging.yaml --no-cache

Sample Output

Successful Validation:

🔍 Validating configuration file: agentmap_config.yaml
✅ Configuration file format is valid
ℹ️ Found 4 configuration sections: paths, llm, execution, memory
✅ Configuration schema validation passed
ℹ️ Custom agents directory will be created: ./custom_agents
ℹ️ OpenAI API key found in environment variable OPENAI_API_KEY
ℹ️ LLM providers configured: openai, anthropic
✅ Config validation passed!

Validation with Issues:

🔍 Validating configuration file: configs/production.yaml

❌ Config Validation Errors:
  1. Schema validation error: Expected boolean, got string
     Field: execution.parallel_execution, Value: "true"
  2. OpenAI API key appears to be a placeholder
     Field: llm.openai.api_key, Value: your_api_key_here
     Suggestion: Replace with your actual API key

⚠️ Config Validation Warnings:
  1. No API key configured for anthropic
     Field: llm.anthropic.api_key
     Suggestion: Set api_key in config or ANTHROPIC_API_KEY environment variable

ℹ️ Validation complete with 1 error and 1 warning

Validation Checks Performed

YAML Structure:

Valid YAML syntax
Root-level dictionary format
Section organization and naming
Common typo detection

Schema Validation:

Pydantic model compliance
Type checking and constraints
Required field validation
Field format verification

Path Validation:

File and directory existence
Path accessibility and permissions
Parent directory validation
Extension consistency checking

LLM Configuration:

API key format and presence
Environment variable checking
Model name validation
Provider-specific settings

Cross-Reference Validation:

Execution policy consistency
Inter-section dependency verification

Comprehensive Validation Command

Validate both CSV and configuration files in a single comprehensive check.

Basic Usage

agentmap validate all [OPTIONS]

Options

Option	Description	Default
`--csv`	Path to CSV file to validate	Uses config default
`--config`	Path to config file to validate	`agentmap_config.yaml`
`--no-cache`	Skip cache and force re-validation	`false`
`--fail-on-warnings`	Treat warnings as errors (exit code 1)	`false`
`--config-file`	Path to initialization config file	None

Examples

Validate All with Defaults:

agentmap validate all

Specify Both Files:

agentmap validate all --csv workflows/main.csv --config configs/prod.yaml

Strict Validation (Fail on Warnings):

agentmap validate all --fail-on-warnings

CI/CD Pipeline Usage:

agentmap validate all --no-cache --fail-on-warnings

Sample Output

Successful Validation:

🔍 Running complete validation
Files:
  CSV: workflows/customer_flow.csv
  Config: agentmap_config.yaml

🔍 CSV validation completed successfully
✅ CSV file format is valid
ℹ️ Found 1 graph(s): 'customer_onboarding' (8 nodes)

🔍 Config validation completed successfully  
✅ Configuration schema validation passed
ℹ️ LLM providers configured: openai

✅ All validation passed!

Validation with Mixed Results:

🔍 Running complete validation

🔍 CSV validation completed successfully
✅ All CSV checks passed

❌ Config Validation Errors:
  1. CSV path does not exist: ./missing_workflow.csv
     Field: csv_path

⚠️ Config Validation Warnings:
  1. No API key configured for anthropic
     Field: llm.anthropic.api_key

ℹ️ Complete validation finished with 1 error and 1 warning

Exit Codes

Exit Code	Condition
`0`	All validation passed, or warnings only (without `--fail-on-warnings`)
`1`	Validation errors found, or warnings with `--fail-on-warnings`

Cache Management Command

Manage the validation cache system for optimal performance and troubleshooting.

Basic Usage

agentmap validate cache [OPTIONS]

Options

Option	Description
`--stats`	Show cache statistics and performance metrics
`--clear`	Clear all validation cache files
`--cleanup`	Remove expired cache entries only
`--file PATH`	Clear cache for specific file only

Examples

View Cache Statistics:

agentmap validate cache --stats

Clear All Cache:

agentmap validate cache --clear

Remove Expired Entries:

agentmap validate cache --cleanup

Clear Cache for Specific File:

agentmap validate cache --file workflows/customer_flow.csv

Sample Output

Cache Statistics:

📊 Validation Cache Statistics
Total cache entries: 23
Valid cache entries: 18
Expired cache entries: 3
Corrupted cache entries: 2
Cache directory: ~/.agentmap/validation_cache
Total cache size: 256 KB

Cache performance:
Recent cache hits: 85%
Average cache age: 4.2 hours
Oldest valid entry: 18 hours ago

Recent activity:
✅ workflows/main.csv (cached 2 hours ago)
✅ agentmap_config.yaml (cached 4 hours ago)
⏰ workflows/old_flow.csv (expired 26 hours ago)
❌ workflows/broken.csv (corrupted cache)

Cache Operations:

# Clear all cache
agentmap validate cache --clear
# Cleared 23 cache entries

# Cleanup expired only  
agentmap validate cache --cleanup
# Removed 3 expired cache entries

# Clear specific file
agentmap validate cache --file workflows/test.csv
# Cleared cache for workflows/test.csv

Command Integration and Workflows

Development Workflow

Typical Development Cycle:

# 1. Start development - validate existing state
agentmap validate all

# 2. Make changes to CSV workflow
# Edit workflows/customer_flow.csv

# 3. Validate changes immediately  
agentmap validate csv --csv workflows/customer_flow.csv

# 4. Update configuration if needed
# Edit agentmap_config.yaml

# 5. Validate configuration changes
agentmap validate config

# 6. Final comprehensive validation
agentmap validate all

# 7. Commit changes (validation passing)
git commit -m "Updated customer workflow"

CI/CD Integration

Build Pipeline Integration:

# Pre-deployment validation script
#!/bin/bash
set -e

echo "🔍 Starting comprehensive validation..."

# Clear cache for fresh validation
agentmap validate cache --clear

# Run strict validation
agentmap validate all --no-cache --fail-on-warnings

if [ $? -eq 0 ]; then
    echo "✅ Validation passed - deployment authorized"
else
    echo "❌ Validation failed - deployment blocked"
    exit 1
fi

Environment-Specific Validation:

# Development: Allow warnings
agentmap validate all || echo "Review warnings before proceeding"

# Staging: Warn about issues but don't block
agentmap validate all --fail-on-warnings || echo "Address issues before production"

# Production: Strict validation required
agentmap validate all --no-cache --fail-on-warnings

Performance Optimization

Cache-Aware Development:

# During active development (use cache)
agentmap validate csv --csv current_workflow.csv

# For critical validations (bypass cache)
agentmap validate all --no-cache

# Regular maintenance
agentmap validate cache --cleanup

Batch Validation:

# Validate multiple workflows efficiently
for workflow in workflows/*.csv; do
    agentmap validate csv --csv "$workflow"
done

# Comprehensive project validation
agentmap validate all --config configs/development.yaml
agentmap validate all --config configs/staging.yaml  
agentmap validate all --config configs/production.yaml

Troubleshooting Common Issues

Cache Performance Problems:

# Check cache effectiveness
agentmap validate cache --stats

# If hit rate is low, clear and rebuild cache
agentmap validate cache --clear
agentmap validate all  # Rebuild cache

Stale Cache Issues:

# Force fresh validation
agentmap validate all --no-cache

# Clear cache for specific problematic file
agentmap validate cache --file problematic_workflow.csv

Validation Errors

File Not Found Errors:

# Check current directory and file paths
ls -la workflows/
ls -la configs/

# Use absolute paths if needed
agentmap validate csv --csv /full/path/to/workflow.csv

Permission Errors:

# Check file permissions
ls -la workflows/workflow.csv

# Check cache directory permissions
ls -la ~/.agentmap/validation_cache/

Environment Issues

Missing Dependencies:

# Verify AgentMap installation
agentmap --version

# Check Python environment
python --version
pip list | grep agentmap

Environment Variable Issues:

# Check required environment variables
echo $OPENAI_API_KEY
echo $ANTHROPIC_API_KEY

# Validate configuration with environment setup
agentmap validate config --config configs/production.yaml

Best Practices for CLI Usage

Command Selection

Use Specific Commands During Development:

# When working on CSV files
agentmap validate csv --csv specific_workflow.csv

# When updating configuration
agentmap validate config --config specific_config.yaml

# For comprehensive pre-commit checks
agentmap validate all

Flag Usage Guidelines

Cache Management:

Default behavior: Use cache for faster development iterations
--no-cache: Use in CI/CD pipelines and for troubleshooting
Cache commands: Use for maintenance and performance monitoring

Error Handling:

Default behavior: Allow warnings during development
--fail-on-warnings: Use for production deployments and strict validation
Exit codes: Integrate with scripts and automation

Performance Optimization

Development Speed:

# Fast iteration during development
agentmap validate csv --csv current_file.csv  # Uses cache

# Comprehensive validation before commits  
agentmap validate all  # Uses cache

# Clean validation for deployments
agentmap validate all --no-cache --fail-on-warnings

Resource Management:

# Regular maintenance
agentmap validate cache --cleanup  # Weekly
agentmap validate cache --clear    # Monthly or after major updates

Validation System Overview: Complete validation system architecture
CSV Validation Guide: Detailed CSV validation capabilities
Config Validation Guide: Configuration validation specifics
Validation Cache Management: Cache system details
Validation Best Practices: Development workflow integration
CSV Schema Reference: Complete CSV format specification
Configuration Reference: Configuration file format and options

Next Steps

Start Validating: Begin using validation commands in your development workflow
Integrate with CI/CD: Add validation to your deployment pipeline
Optimize Performance: Use cache management for faster development
Learn Best Practices: Review validation best practices for optimal workflows

Validation Architecture​

Facade Pattern Implementation​

Runtime API Integration​

CLI Presenter Integration​

Bundle-Based Validation​

Validation Pattern Benefits​

Command Overview​

CSV Validation Command​

Basic Usage​

Options​

Examples​

Sample Output​

Validation Checks Performed​

Config Validation Command​

Basic Usage​

Options​

Examples​

Sample Output​

Validation Checks Performed​

Comprehensive Validation Command​

Basic Usage​

Options​

Examples​

Sample Output​

Exit Codes​

Cache Management Command​

Basic Usage​

Options​

Examples​

Sample Output​

Command Integration and Workflows​

Development Workflow​

CI/CD Integration​

Performance Optimization​

Troubleshooting Common Issues​

Cache-Related Issues​

Validation Errors​

Environment Issues​

Best Practices for CLI Usage​

Command Selection​

Flag Usage Guidelines​

Performance Optimization​

Related Documentation​

Next Steps​

Validation Architecture

Facade Pattern Implementation

Runtime API Integration

CLI Presenter Integration

Bundle-Based Validation

Validation Pattern Benefits

Command Overview

CSV Validation Command

Basic Usage

Options

Examples

Sample Output

Validation Checks Performed

Config Validation Command

Basic Usage

Options

Examples

Sample Output

Validation Checks Performed

Comprehensive Validation Command

Basic Usage

Options

Examples

Sample Output

Exit Codes

Cache Management Command

Basic Usage

Options

Examples

Sample Output

Command Integration and Workflows

Development Workflow

CI/CD Integration

Performance Optimization

Troubleshooting Common Issues

Cache-Related Issues

Validation Errors

Environment Issues

Best Practices for CLI Usage

Command Selection

Flag Usage Guidelines

Performance Optimization

Related Documentation

Next Steps