Bidding Benchmark Analysis

This tutorial demonstrates how to analyze Enertel bidding benchmark results to evaluate trading strategy performance and optimization outcomes. Bidding benchmarks simulate real-world trading scenarios using our forecasts to optimize bidding strategies for different asset types and market conditions.

Access Required

Bidding benchmark analysis requires special access beyond standard forecast subscriptions. Contact our team to request trial access to bidding benchmark data and examples.

Understanding Bidding Benchmarks

Each bidding benchmark is customized for specific client requirements, including:

• Asset type: Load buying, solar selling, wind selling, battery optimization
• Market participation: Day-ahead, real-time, ancillary services
• Trading strategy: DART trading, arbitrage, hedging
• Risk preferences: Conservative, moderate, aggressive bidding

Every benchmark has a unique game_id that identifies the specific optimization setup and parameters.

Game Session Types

A game session represents a single optimization run and contains four types of data:

Component	Description
Agents	Metadata about optimization agents, including forecast models used
Info	Run metadata including time range, parameters, and performance summary
Bids	Actual bids submitted by agents, including volume and market details
Paths	Detailed price predictions used to generate bidding decisions

Session Types

• live: Real-time optimization using current market conditions
• backtest: Historical simulation for strategy validation
• search: Parameter optimization to find best strategy settings

Prerequisites

1. Game ID: Obtain from the Enertel application frontend
2. API token: Create from your user profile
3. Benchmark access: Special permissions for bidding benchmark data

Setup

import pandas as pd
import requests
import arrow
import json
from datetime import datetime, timedelta
import matplotlib.pyplot as plt
import seaborn as sns

# Configuration
token = '<your-api-token>'
game_id = '<your-game-id>'  # Get this from the frontend
base_url = "https://app.enertel.ai/api"

# Common headers
headers = {'Authorization': f'Bearer {token}'}

Listing Game Sessions

Start by retrieving all available game sessions for your benchmark:

def get_game_sessions(token, game_id):
    """
    Retrieve all game sessions for a specific benchmark.
    """
    url = f"{base_url}/grid/game/{game_id}/sessions"
    
    print(f"Fetching sessions for game {game_id}...")
    start_time = arrow.get()
    
    try:
        response = requests.get(url, headers=headers, timeout=30)
        
        if response.status_code == 200:
            sessions = response.json()
            end_time = arrow.get()
            
            print(f"✓ Retrieved {len(sessions)} sessions in {end_time - start_time}")
            return sessions
        else:
            print(f"Error: {response.status_code} - {response.text}")
            return []
            
    except requests.exceptions.RequestException as e:
        print(f"Request failed: {e}")
        return []

# Get all sessions
sessions = get_game_sessions(token, game_id)

# Convert to DataFrame for analysis
if sessions:
    sessions_df = pd.DataFrame(sessions)
    print(f"\nSessions overview:")
    print(sessions_df[['id', 'type', 'created_at', 'status']].head())
else:
    print("No sessions found")

Analyzing Session Performance

Filter and Sort Sessions

def analyze_sessions(sessions):
    """
    Analyze and categorize game sessions by type and performance.
    """
    if not sessions:
        return None
    
    # Convert to DataFrame
    df = pd.DataFrame(sessions)
    
    # Convert timestamps
    df['created_at'] = pd.to_datetime(df['created_at'])
    
    # Analyze by session type
    print("=== Session Analysis ===")
    type_counts = df['type'].value_counts()
    print(f"Session types:")
    for session_type, count in type_counts.items():
        print(f"  {session_type}: {count}")
    
    # Recent sessions by type
    print(f"\nMost recent sessions:")
    recent_sessions = df.sort_values('created_at', ascending=False).head(10)
    print(recent_sessions[['id', 'type', 'created_at', 'status']])
    
    return df

# Analyze sessions
sessions_df = analyze_sessions(sessions)

Select Most Recent Live Session

def get_latest_live_session(sessions):
    """
    Find the most recent live trading session.
    """
    live_sessions = [s for s in sessions if s['type'] == 'live']
    
    if not live_sessions:
        print("No live sessions found")
        return None
    
    # Sort by creation time
    live_sorted = sorted(live_sessions, key=lambda x: arrow.get(x["created_at"]))
    latest_session = live_sorted[-1]
    
    print(f"Latest live session:")
    print(f"  ID: {latest_session['id']}")
    print(f"  Created: {latest_session['created_at']}")
    print(f"  Status: {latest_session.get('status', 'unknown')}")
    
    return latest_session

# Get latest live session
latest_session = get_latest_live_session(sessions)

Loading Session Data

Retrieve Detailed Session Data

def load_session_data(token, session_id, include_paths=False, include_distribution=False):
    """
    Load detailed data for a specific game session.
    
    Parameters:
    - include_paths: Include raw price paths (increases response time)
    - include_distribution: Include price distribution data (increases response time)
    """
    url = f"{base_url}/grid/game/session/{session_id}"
    
    params = {
        'distribution': 'true' if include_distribution else 'false',
        'paths': 'true' if include_paths else 'false'
    }
    
    print(f"Loading session {session_id} data...")
    start_time = arrow.get()
    
    try:
        response = requests.get(url, headers=headers, params=params, timeout=60)
        
        if response.status_code == 200:
            data = response.json()
            end_time = arrow.get()
            
            print(f"✓ Loaded session data in {end_time - start_time}")
            return data
        else:
            print(f"Error: {response.status_code} - {response.text}")
            return None
            
    except requests.exceptions.RequestException as e:
        print(f"Request failed: {e}")
        return None

# Load session data (start without paths for faster loading)
if latest_session:
    session_data = load_session_data(token, latest_session['id'])
    
    if session_data:
        print(f"\nSession data structure:")
        print(f"Keys: {list(session_data.keys())}")
        
        # Extract different data components
        agents_data = session_data.get('data', {}).get('agents', [])
        bids_data = session_data.get('data', {}).get('bids', [])
        info_data = session_data.get('data', {}).get('info', {})
        
        print(f"Agents: {len(agents_data)}")
        print(f"Bids: {len(bids_data)}")
        print(f"Info available: {'Yes' if info_data else 'No'}")

Analyzing Bidding Performance

Bid Analysis

def analyze_bids(bids_data):
    """
    Analyze bidding patterns and performance.
    """
    if not bids_data:
        print("No bid data available")
        return None
    
    # Convert to DataFrame
    bids_df = pd.DataFrame(bids_data)
    
    print("=== Bid Analysis ===")
    print(f"Total bids: {len(bids_df)}")
    print(f"Columns: {list(bids_df.columns)}")
    
    # Show sample bids
    print(f"\nSample bids:")
    print(bids_df.head())
    
    # Analyze bid characteristics
    if 'price' in bids_df.columns and 'quantity' in bids_df.columns:
        print(f"\nBid statistics:")
        print(f"Price range: ${bids_df['price'].min():.2f} - ${bids_df['price'].max():.2f}")
        print(f"Quantity range: {bids_df['quantity'].min():.1f} - {bids_df['quantity'].max():.1f} MW")
        print(f"Average bid price: ${bids_df['price'].mean():.2f}")
        print(f"Average bid quantity: {bids_df['quantity'].mean():.1f} MW")
    
    return bids_df

# Analyze bids
if 'bids_data' in locals() and bids_data:
    bids_df = analyze_bids(bids_data)

Agent Performance Analysis

def analyze_agents(agents_data):
    """
    Analyze agent performance and configuration.
    """
    if not agents_data:
        print("No agent data available")
        return None
    
    agents_df = pd.DataFrame(agents_data)
    
    print("=== Agent Analysis ===")
    print(f"Number of agents: {len(agents_df)}")
    print(f"Agent columns: {list(agents_df.columns)}")
    
    # Show agent details
    if not agents_df.empty:
        print(f"\nAgent details:")
        for idx, agent in agents_df.iterrows():
            print(f"Agent {idx + 1}:")
            print(f"  ID: {agent.get('id', 'N/A')}")
            print(f"  Type: {agent.get('type', 'N/A')}")
            print(f"  Model: {agent.get('model_id', 'N/A')}")
            if 'performance' in agent:
                print(f"  Performance: {agent['performance']}")
    
    return agents_df

# Analyze agents
if 'agents_data' in locals() and agents_data:
    agents_df = analyze_agents(agents_data)

Visualization and Reporting

Bid Distribution Visualization

def visualize_bid_performance(bids_df):
    """
    Create visualizations for bid analysis.
    """
    if bids_df is None or bids_df.empty:
        print("No bid data to visualize")
        return
    
    fig, axes = plt.subplots(2, 2, figsize=(15, 10))
    
    # Bid price distribution
    if 'price' in bids_df.columns:
        axes[0, 0].hist(bids_df['price'], bins=30, edgecolor='black', alpha=0.7)
        axes[0, 0].set_title('Bid Price Distribution')
        axes[0, 0].set_xlabel('Price ($/MWh)')
        axes[0, 0].set_ylabel('Frequency')
        axes[0, 0].grid(True, alpha=0.3)
    
    # Bid quantity distribution
    if 'quantity' in bids_df.columns:
        axes[0, 1].hist(bids_df['quantity'], bins=30, edgecolor='black', alpha=0.7)
        axes[0, 1].set_title('Bid Quantity Distribution')
        axes[0, 1].set_xlabel('Quantity (MW)')
        axes[0, 1].set_ylabel('Frequency')
        axes[0, 1].grid(True, alpha=0.3)
    
    # Time series of bids (if timestamp available)
    if 'timestamp' in bids_df.columns:
        bids_df['timestamp'] = pd.to_datetime(bids_df['timestamp'])
        daily_bids = bids_df.groupby(bids_df['timestamp'].dt.date).size()
        
        axes[1, 0].plot(daily_bids.index, daily_bids.values, marker='o')
        axes[1, 0].set_title('Daily Bid Count')
        axes[1, 0].set_xlabel('Date')
        axes[1, 0].set_ylabel('Number of Bids')
        axes[1, 0].tick_params(axis='x', rotation=45)
        axes[1, 0].grid(True, alpha=0.3)
    
    # Price vs Quantity scatter
    if 'price' in bids_df.columns and 'quantity' in bids_df.columns:
        axes[1, 1].scatter(bids_df['quantity'], bids_df['price'], alpha=0.6)
        axes[1, 1].set_title('Price vs Quantity')
        axes[1, 1].set_xlabel('Quantity (MW)')
        axes[1, 1].set_ylabel('Price ($/MWh)')
        axes[1, 1].grid(True, alpha=0.3)
    
    plt.tight_layout()
    plt.show()

# Create visualizations
if 'bids_df' in locals() and bids_df is not None:
    visualize_bid_performance(bids_df)

Performance Summary Report

def generate_performance_report(session_data, session_info):
    """
    Generate a comprehensive performance report.
    """
    print("=" * 60)
    print("BIDDING BENCHMARK PERFORMANCE REPORT")
    print("=" * 60)
    
    # Session overview
    print(f"\n📊 SESSION OVERVIEW")
    print(f"   Session ID: {session_info.get('id', 'N/A')}")
    print(f"   Type: {session_info.get('type', 'N/A')}")
    print(f"   Created: {session_info.get('created_at', 'N/A')}")
    print(f"   Status: {session_info.get('status', 'N/A')}")
    
    # Data summary
    data = session_data.get('data', {})
    agents = data.get('agents', [])
    bids = data.get('bids', [])
    
    print(f"\n📈 DATA SUMMARY")
    print(f"   Agents: {len(agents)}")
    print(f"   Total bids: {len(bids)}")
    
    # Bid analysis
    if bids:
        bids_df = pd.DataFrame(bids)
        if 'price' in bids_df.columns:
            print(f"   Avg bid price: ${bids_df['price'].mean():.2f}")
            print(f"   Price range: ${bids_df['price'].min():.2f} - ${bids_df['price'].max():.2f}")
        
        if 'quantity' in bids_df.columns:
            print(f"   Total volume: {bids_df['quantity'].sum():.1f} MW")
            print(f"   Avg bid size: {bids_df['quantity'].mean():.1f} MW")
    
    # Agent performance
    if agents:
        print(f"\n🤖 AGENT PERFORMANCE")
        for i, agent in enumerate(agents):
            print(f"   Agent {i+1}: Model {agent.get('model_id', 'N/A')}")
    
    print(f"\n{'=' * 60}")

# Generate report
if 'session_data' in locals() and 'latest_session' in locals():
    if session_data and latest_session:
        generate_performance_report(session_data, latest_session)

Advanced Analysis

Multi-Session Comparison

def compare_sessions(sessions, token, max_sessions=5):
    """
    Compare performance across multiple sessions.
    """
    live_sessions = [s for s in sessions if s['type'] == 'live'][-max_sessions:]
    
    comparison_data = []
    
    for session in live_sessions:
        session_data = load_session_data(token, session['id'])
        if session_data:
            bids = session_data.get('data', {}).get('bids', [])
            
            if bids:
                bids_df = pd.DataFrame(bids)
                comparison_data.append({
                    'session_id': session['id'],
                    'created_at': session['created_at'],
                    'total_bids': len(bids_df),
                    'avg_price': bids_df['price'].mean() if 'price' in bids_df.columns else None,
                    'total_volume': bids_df['quantity'].sum() if 'quantity' in bids_df.columns else None
                })
    
    if comparison_data:
        comparison_df = pd.DataFrame(comparison_data)
        print("Session Performance Comparison:")
        print(comparison_df)
        
        return comparison_df
    
    return None

# Compare recent sessions
# comparison_df = compare_sessions(sessions, token)

Best Practices

Data Loading Optimization

# Start with basic data (no paths/distribution) for faster loading
session_data = load_session_data(token, session_id, 
                                include_paths=False, 
                                include_distribution=False)

# Load detailed data only when needed for specific analysis
if detailed_analysis_needed:
    detailed_data = load_session_data(token, session_id, 
                                    include_paths=True, 
                                    include_distribution=True)

Error Handling

def robust_session_load(token, session_id, max_retries=3):
    """
    Load session data with retry logic.
    """
    for attempt in range(max_retries):
        try:
            data = load_session_data(token, session_id)
            if data:
                return data
        except Exception as e:
            print(f"Attempt {attempt + 1} failed: {e}")
            if attempt < max_retries - 1:
                time.sleep(2 ** attempt)  # Exponential backoff
    
    return None

Next Steps

1. Performance Analysis: Use the bid data to evaluate strategy effectiveness
2. Parameter Optimization: Compare different agent configurations
3. Risk Assessment: Analyze bid distributions for risk management
4. Strategy Refinement: Use insights to improve bidding algorithms

API Documentation

For complete API reference and additional endpoints, visit our comprehensive API documentation.

This tutorial provides the foundation for analyzing bidding benchmark results. The specific analysis will depend on your asset type, trading strategy, and performance objectives.