Error Handling - SMDPfier
Description: SMDP-level behavior for Gymnasium environments

Skip to content

# Error Handling

SMDPfier provides comprehensive error handling with detailed context information to help debug option execution issues and build robust SMDP applications.

## Error Types

SMDPfier defines specialized exceptions that provide rich context about option failures, making debugging much easier than generic exceptions.

### SMDPOptionValidationError

Raised during precheck validation when an option is determined to be invalid before execution.

    from smdpfier.errors import SMDPOptionValidationError

    try:
        obs, reward, term, trunc, info = env.step(invalid_option_index)
    except SMDPOptionValidationError as e:
        print(f"Validation failed for option '{e.option_name}' (ID: {e.option_id})")
        print(f"Validation type: {e.validation_type}")
        print(f"Failed at step {e.failing_step_index} with action {e.action_repr}")
        print(f"Environment state: {e.short_obs_summary}")
        if e.base_error:
            print(f"Underlying error: {e.base_error}")

### SMDPOptionExecutionError

Raised during option execution when a primitive action fails at runtime.

    from smdpfier.errors import SMDPOptionExecutionError

    try:
        obs, reward, term, trunc, info = env.step(problematic_option_index)
    except SMDPOptionExecutionError as e:
        print(f"Execution failed for option '{e.option_name}' (ID: {e.option_id})")
        print(f"Failed at step {e.failing_step_index} with action {e.action_repr}")
        print(f"Environment state: {e.short_obs_summary}")
        if e.base_error:
            print(f"Underlying error: {e.base_error}")

## Error Context Fields

Both error types provide comprehensive context information:

Field | Type | Description
--- | --- | ---
`option_name` | `str` | Human-readable name of the failing option
`option_id` | `str` | Unique identifier for the option (hash-based)
`failing_step_index` | `int` | Which action in the sequence failed (0-indexed)
`action_repr` | `str` | String representation of the failing action
`short_obs_summary` | `str` | Abbreviated environment state description
`base_error` | `Exception \| None` | Original exception that caused the failure
`validation_type` | `str` | Type of validation that failed (ValidationError only)

### Error Context Example

    # When this option fails:
    problematic_option = Option([0, 1, 99], "problematic-sequence")

    # You get detailed context:
    # SMDPOptionExecutionError: Option execution failed
    #   option_name: "problematic-sequence"
    #   option_id: "abc123def456..."
    #   failing_step_index: 2
    #   action_repr: "99"
    #   short_obs_summary: "obs=[0.1, -0.3, 0.05, ...] (CartPole-v1)"
    #   base_error: ValueError("Invalid action 99 for Discrete(2) action space")

## Common Error Scenarios

### Invalid Action for Action Space

Most common error - action not in environment's action space.

    # CartPole has Discrete(2) action space (actions 0, 1)
    invalid_option = Option([0, 1, 2], "invalid-action")  # Action 2 is invalid

    try:
        obs, reward, term, trunc, info = env.step(invalid_option)
    except SMDPOptionExecutionError as e:
        print(f"Invalid action {e.action_repr} at step {e.failing_step_index}")
        # Output: "Invalid action 2 at step 2"

Solution strategies:

    # Strategy 1: Validate options at creation
    def create_safe_option(actions, name, action_space):
        """Create option with action space validation."""
        if isinstance(action_space, gym.spaces.Discrete):
            valid_actions = list(range(action_space.n))
            invalid_actions = [a for a in actions if a not in valid_actions]
            if invalid_actions:
                raise ValueError(f"Invalid actions {invalid_actions} for {action_space}")

        return Option(actions, name)

    # Strategy 2: Use masking to prevent invalid options
    def safe_availability(obs):
        """Only allow options with valid actions."""
        return [0, 1, 2]  # Indices of options with valid actions only

### Continuous Action Shape Mismatch

Continuous environments require actions with correct shape.

    # Pendulum expects actions with shape (1,)
    wrong_shape_option = Option([[1.0, 2.0]], "wrong-shape")  # Shape (2,) - wrong!
    correct_option = Option([[1.5]], "correct-shape")         # Shape (1,) - correct

    try:
        obs, reward, term, trunc, info = env.step(wrong_shape_option)
    except SMDPOptionExecutionError as e:
        print(f"Shape error: {e.base_error}")
        # Output: "Shape error: ValueError('Expected action shape (1,), got (2,)')"

### Environment State Issues

Some environments have state-dependent action validity.

    # Taxi environment - pickup only valid at passenger locations
    def taxi_example():
        env = SMDPfier(gym.make("Taxi-v3"), ...)

        # This might fail if taxi is not at passenger location
        pickup_option = Option([4], "pickup")  # Action 4 = pickup

        try:
            obs, reward, term, trunc, info = env.step(pickup_option)
        except SMDPOptionExecutionError as e:
            if "pickup" in str(e.base_error).lower():
                print("Pickup failed - taxi not at passenger location")
                # Try movement instead
                move_option = Option([0], "move-south")
                obs, reward, term, trunc, info = env.step(move_option)

### Early Termination During Option

Episode termination during option execution is handled gracefully (not an error), but understanding the behavior is important.

    # Long option that might cause episode termination
    long_option = Option([0] * 10, "ten-left-actions")

    obs, reward, term, trunc, info = env.step(long_option)

    # Check if option completed
    smdp_info = info["smdp"]
    if smdp_info["terminated_early"]:
        print(f"Episode ended after {smdp_info['k_exec']} of {smdp_info['option']['len']} actions")
        print(f"Partial duration: {smdp_info['duration_exec']} ticks")

## Error Handling Patterns

### Basic Try-Catch Pattern

    def robust_step(env, action):
        """Robust step with error handling."""
        try:
            return env.step(action)
        except SMDPOptionValidationError as e:
            print(f"Precheck failed: {e.option_name} at step {e.failing_step_index}")
            # Return safe fallback or re-raise
            raise
        except SMDPOptionExecutionError as e:
            print(f"Execution failed: {e.option_name} with action {e.action_repr}")
            # Return safe fallback or re-raise
            raise

### Fallback Strategy Pattern

    def execute_with_fallback(env, primary_option, fallback_options):
        """Try primary option, fall back to alternatives on failure."""

        # Try primary option
        try:
            return env.step(primary_option)
        except (SMDPOptionValidationError, SMDPOptionExecutionError) as e:
            print(f"Primary option failed: {e}")

            # Try fallback options
            for i, fallback in enumerate(fallback_options):
                try:
                    print(f"Trying fallback {i+1}: {fallback.name}")
                    return env.step(fallback)
                except (SMDPOptionValidationError, SMDPOptionExecutionError):
                    continue

            # All options failed
            raise RuntimeError("All options failed, no valid actions available")

    # Usage
    primary = Option([0, 1, 0, 1], "complex-sequence")
    fallbacks = [
        Option([0, 1], "simple-sequence"),
        Option([0], "minimal-action"),
    ]

    obs, reward, term, trunc, info = execute_with_fallback(env, primary, fallbacks)

### Logging and Monitoring Pattern

    import logging

    logger = logging.getLogger("smdp_errors")

    def monitored_step(env, action):
        """Step with comprehensive error logging."""
        try:
            result = env.step(action)

            # Log successful execution details
            smdp_info = result[4]["smdp"]  # info["smdp"]
            logger.info(f"Option '{smdp_info['option']['name']}' executed successfully")
            logger.debug(f"Steps: {smdp_info['k_exec']}, Duration: {smdp_info['duration_exec']}")

            return result

        except SMDPOptionValidationError as e:
            logger.error(f"Validation error: {e.option_name} failed at step {e.failing_step_index}")
            logger.error(f"Action: {e.action_repr}, State: {e.short_obs_summary}")
            logger.debug(f"Full error: {e}")
            raise

        except SMDPOptionExecutionError as e:
            logger.error(f"Execution error: {e.option_name} failed at step {e.failing_step_index}")
            logger.error(f"Action: {e.action_repr}, Base error: {e.base_error}")
            logger.debug(f"Full error: {e}")
            raise

### Error Recovery for RL Training

    class ErrorResilientAgent:
        def __init__(self, env):
            self.env = env
            self.error_counts = {}
            self.max_error_threshold = 5

        def step_with_recovery(self, action):
            """RL training step with error recovery."""
            try:
                return self.env.step(action)

            except (SMDPOptionValidationError, SMDPOptionExecutionError) as e:
                # Track error frequency
                error_key = (e.option_name, e.failing_step_index)
                self.error_counts[error_key] = self.error_counts.get(error_key, 0) + 1

                # If this error happens too often, it's a systematic issue
                if self.error_counts[error_key] > self.max_error_threshold:
                    print(f"Systematic error detected: {error_key}")
                    # Could trigger option blacklisting, environment reset, etc.

                # For training, reset environment and continue
                print(f"Error during training: {e.option_name}, resetting episode")
                obs, info = self.env.reset()
                return obs, 0.0, True, False, info  # Return terminal step

        def get_error_statistics(self):
            """Get error frequency statistics."""
            return dict(self.error_counts)

## Debugging Techniques

### Option Validation

    def validate_option_thoroughly(env, option):
        """Comprehensive option validation."""
        print(f"Validating option: {option.name}")
        print(f"Actions: {option.actions}")
        print(f"Action count: {len(option.actions)}")

        # Check action space compatibility
        action_space = env.unwrapped.action_space
        print(f"Environment action space: {action_space}")

        for i, action in enumerate(option.actions):
            if isinstance(action_space, gym.spaces.Discrete):
                if not (0 <= action < action_space.n):
                    print(f"  ❌ Action {i}: {action} invalid for {action_space}")
                else:
                    print(f"  ✅ Action {i}: {action} valid")
            elif isinstance(action_space, gym.spaces.Box):
                action = np.array(action)
                if action.shape != action_space.shape:
                    print(f"  ❌ Action {i}: shape {action.shape} != {action_space.shape}")
                elif not action_space.contains(action):
                    print(f"  ❌ Action {i}: {action} outside bounds {action_space}")
                else:
                    print(f"  ✅ Action {i}: {action} valid")

### Error Pattern Analysis

    def analyze_error_patterns(error_log):
        """Analyze common error patterns from logs."""
        error_types = {}
        failing_actions = {}
        failing_options = {}

        for error in error_log:
            # Group by error type
            error_type = type(error).__name__
            error_types[error_type] = error_types.get(error_type, 0) + 1

            # Group by failing action
            action = error.action_repr
            failing_actions[action] = failing_actions.get(action, 0) + 1

            # Group by option name
            option = error.option_name
            failing_options[option] = failing_options.get(option, 0) + 1

        print("Error Analysis:")
        print(f"Error types: {error_types}")
        print(f"Most failing actions: {sorted(failing_actions.items(), key=lambda x: x[1], reverse=True)[:5]}")
        print(f"Most failing options: {sorted(failing_options.items(), key=lambda x: x[1], reverse=True)[:5]}")

### Interactive Debugging

    def debug_option_interactively(env, option):
        """Step through option execution interactively."""
        print(f"Debug mode for option: {option.name}")
        print(f"Actions to execute: {option.actions}")

        obs, info = env.reset()
        print(f"Initial state: {obs}")

        for i, action in enumerate(option.actions):
            print(f"\nStep {i}: About to execute action {action}")
            input("Press Enter to continue...")

            try:
                obs, reward, term, trunc, info = env.step(action)
                print(f"Result: obs={obs[:3]}..., reward={reward}, term={term}, trunc={trunc}")

                if term or trunc:
                    print("Episode terminated!")
                    break

            except Exception as e:
                print(f"Error at step {i}: {e}")
                break

## Error Prevention

### Option Design Guidelines

    # ✅ Good: Simple, robust options
    good_options = [
        Option([0], "single-left"),           # Minimal failure risk
        Option([1], "single-right"),          # Minimal failure risk
        Option([0, 1], "balanced"),           # Self-correcting
    ]

    # ❌ Avoid: Complex, risky options
    risky_options = [
        Option([0] * 20, "extreme-long"),     # High termination risk
        Option([999], "invalid-action"),      # Invalid action
        Option([0, 1, 0, 1, 0, 1], "too-long"), # Long sequence risk
    ]

### Environment-Specific Safety

    def create_safe_cartpole_options():
        """Create CartPole options with built-in safety."""
        # CartPole actions: 0 (left), 1 (right)
        safe_options = [
            Option([0], "left"),
            Option([1], "right"),
            Option([0, 1], "left-right"),     # Balanced
            Option([1, 0], "right-left"),     # Balanced
            # Avoid long sequences that might cause termination
        ]
        return safe_options

    def create_safe_pendulum_options():
        """Create Pendulum options with proper action ranges."""
        # Pendulum action space: Box([-2.0], [2.0])
        safe_options = [
            Option([[-2.0]], "max-left"),
            Option([[2.0]], "max-right"),
            Option([[0.0]], "no-torque"),
            Option([[-1.0], [1.0]], "swing"),
            Option([[0.5], [0.5]], "gentle-right"),
        ]
        return safe_options

## Summary

Error Type | When It Occurs | How to Handle
--- | --- | ---
`SMDPOptionValidationError` | Precheck failure | Disable precheck or fix option
`SMDPOptionExecutionError` | Runtime action failure | Use fallbacks or fix option
Early termination | Episode ends during option | Normal behavior, check `terminated_early`

Best Practices: 1. Design robust options with minimal failure risk 2. Use action masking to prevent invalid options 3. Implement fallback strategies for critical applications 4. Log errors comprehensively for debugging 5. Test options thoroughly in development

Key Takeaway: SMDPfier's detailed error context makes debugging much easier than generic exceptions. Use this information to build robust, error-resilient SMDP applications.

Next: FAQ | See Also: Masking and Precheck