Agent Concepts

Overview

The agents module implements all bidding agent strategies for the DARTs game. Each agent is responsible for generating optimal Day-Ahead (DA) and Real-Time (RT) bids for each market period, using available market data and probabilistic forecasts. The agent framework is designed for extensibility, allowing new strategies to be added with minimal boilerplate.

All agents inherit from a common base interface from the Enertel forecasts product, which provides the standard workflow, shared logic for data preparation, price path generation, and bid formatting. Each agent implements its own optimization objective and decision logic.

---

Agent Interface & Workflow

1. Initialization

Each agent is initialized with:

• Market and forecast model parameters: IDs for DA/RT models, feature IDs, number of price paths, etc.
• Participation constraints: Minimum and maximum DA participation as a fraction of generation.
• Bid price parameters: Default bid prices, RT threshold factors, and price adjustments.
• Forecast data: Model inference results, indexed by timestamp, model, and scenario.

Use multiple models

Specify both a DA and an RT forecast model (or multiple of each) when configuring agents. Using multiple models for each market gives agents a richer set of price paths and often leads to more robust bidding strategies than relying on a single model.

2. Bid Generation Process

For each market period (game step), the agent follows this process:

1. Prepare Data: Extracts the relevant DA/RT price scenarios and generation for the current period. If forecast models are specified, generates quantile-based price paths from the forecast data; otherwise, uses actual prices (for baseline agents).
2. Optimize Bids: Solves an optimization problem to determine the optimal DA bid quantities, subject to participation constraints and the agent's objective.
3. Format Bids: Packages the DA/RT bids, prices, and metadata for submission to the game environment.

3. Optimization Procedure

• Forecast-based agents: Generate multiple price paths using the specified DA/RT models and scenario forecasts. The optimization is run multiple times with randomized/shuffled paths for robustness, and the median result is used.
• Baseline agents: Use actual prices or static rules (e.g., all-in DA/RT) and do not require forecasts.

Use 30–100 price paths

For forecast-based agents, use between 30 and 100 price paths per period. Too few paths can make bids noisy and overfit to a small sample; too many add compute cost with diminishing gains. The 30–100 range is a good balance for stability and performance.

Keep optimizations per step to 5–10

When running multiple optimization passes with randomized paths, keep the number of runs per step in the 5–10 range. This keeps the median bid stable without excessive runtime, and avoids overfitting to a single path ordering.

---

Agent Types, Behaviors, and Objective Functions

1. Base Agent (Abstract)

• Provides the standard interface and shared logic for all agents.
• Handles price path generation, bid formatting, and multi-run optimization.
• Requires subclasses to implement the core bid optimization logic.

2. Perfect Foresight Agent

• Behavior: Has perfect knowledge of future DA/RT prices for each period.
• Optimization: Directly maximizes realized revenue using actual prices (no forecasts).
• Objective Function:

For generation vector $g$, DA bid vector $q_{DA}$, DA prices $p_{DA}$, and RT prices $p_{RT}$:

$$\max_{q_{DA}} \sum_{t} \left[ p_{DA, t} \cdot q_{DA, t} + p_{RT, t} \cdot (g_t - q_{DA, t}) \right]$$

Subject to participation constraints (DA bids between min and max fraction of generation per period).

3. All-In Agent

• Behavior: Places all available capacity in either the DA or RT market, based on a static choice at initialization.
• Optimization: No optimization; simply returns all generation for the chosen market and zero for the other.
• Objective Function:

$$q_{DA, t} = \begin{cases} g_t & \text{if all-in DA} \\ 0 & \text{if all-in RT} \end{cases}$$

4. Revenue Expectation Agent

• Behavior: Maximizes expected revenue across all generated price paths.
• Optimization: Solves for DA bid quantities that maximize the mean revenue over all forecast scenarios, subject to participation constraints.
• Objective Function:

For $N$ price paths, let $p_{DA}^{(i)}$ and $p_{RT}^{(i)}$ be the DA/RT prices for path $i$:

$$\max_{q_{DA}} \frac{1}{N} \sum_{i=1}^N \sum_{t} \left[ p_{DA, t}^{(i)} \cdot q_{DA, t} + p_{RT, t}^{(i)} \cdot (g_t - q_{DA, t}) \right]$$

Subject to the same participation constraints as above.

5. Regret Minimization Agent

• Behavior: Minimizes average regret, defined as the difference between the achieved revenue and the maximum possible revenue for each price path.
• Optimization: For each scenario, computes the regret and solves for DA bids that minimize the mean regret across all paths.
• Objective Function:

For each path $i$, regret is:

$$\text{regret}^{(i)} = \sum_{t} \left[ \max(p_{DA, t}^{(i)}, p_{RT, t}^{(i)}) \cdot g_t - (p_{DA, t}^{(i)} \cdot q_{DA, t} + p_{RT, t}^{(i)} \cdot (g_t - q_{DA, t})) \right]$$

The agent solves:

$$\min_{q_{DA}} \frac{1}{N} \sum_{i=1}^N \text{regret}^{(i)}$$

Subject to participation constraints.

6. Mean–Variance Agent

• Behavior: Maximizes a mean–risk objective, balancing expected revenue and risk (variance or downside risk).
• Optimization: A risk aversion parameter controls the trade-off:
  • Positive values penalize downside deviations (risk-averse).
  • Negative values reward upside deviations (risk-seeking).
• Objective Function:

Let $R^{(i)}(q_{DA})$ be the revenue for path $i$:

$$\text{Expected revenue:}\quad \mu = \frac{1}{N} \sum_{i=1}^N R^{(i)}(q_{DA})$$ $$\text{Risk adjustment:}\quad \begin{cases} \frac{1}{N} \sum_{i=1}^N \max(\mu - R^{(i)}(q_{DA}), 0) & \text{if } \lambda \geq 0 \\ \frac{1}{N} \sum_{i=1}^N \max(R^{(i)}(q_{DA}) - \mu, 0) & \text{if } \lambda < 0 \end{cases}$$

The agent solves:

$$\max_{q_{DA}} \left[ \mu - |\lambda| \cdot \text{Risk adjustment} \right]$$

Subject to participation constraints.

---

Agent Configuration & Extensibility

• Configurable Parameters: All agents can be configured with different model sets, participation constraints, price path settings, and optimization parameters.
• Adding New Agents: New strategies are added by extending the base interface and implementing the core bid optimization with a custom objective. The rest of the workflow (data preparation, bid formatting, multi-run optimization) is handled by the base logic.
• Serialization: Each agent can be serialized for reproducibility and storage.

---

Optimization Details

• Constraints: DA bid quantities are always constrained to be within the configured min and max DA participation fraction of generation for each period.
• Forecast Handling: Agents can use any combination of DA/RT models for price path generation. The number of paths and models is configurable.
• Robustness: For forecast-based agents, multiple optimization runs with randomized price paths are used to avoid overfitting to a single scenario.

Compare baseline and forecast agents

Run Perfect Foresight and All-In agents alongside forecast-based agents. They give you an upper bound (perfect foresight) and simple benchmarks (all-in DA/RT) to interpret how well your forecast-driven strategies are performing.

---

Summary Table

Agent Type	Objective	Uses Forecasts?	Robustness (Multi-Run)	Baseline?
Perfect Foresight	Maximize realized profit	No	No	Yes
All-In	All-in DA or RT	No	No	Yes
Revenue Expectation	Maximize expected profit	Yes	Yes	No
Regret Minimization	Minimize mean regret	Yes	Yes	No
Mean–Variance	Mean–risk tradeoff	Yes	Yes	No

---

This guide describes the agent strategies, optimization procedures, and extensibility of the DARTs agent framework.