AI & ML Agent Simulation

Foundations

1. What is an AI Agent?

An AI Agent is a software system that can perceive its environment, reason about it, and take actions to achieve a goal, all with a degree of autonomy. Unlike a simple chatbot that responds to prompts, an agent can plan, use tools, and adapt its behavior based on feedback.

The Agent Loop

Every AI agent follows a core cycle:

Perceive: Receive input (sensor data, text, telemetry).
Reason: Use an LLM or ML model to interpret the situation and plan.
Act: Execute a tool call, send a command, or generate output.
Observe: Check the result and decide whether to continue or stop.

🛰️ Perceive

→

🧠 Reason

→

⚡ Act

→

👁️ Observe

→

🔄 Repeat

Key Insight: The difference between a "model" and an "agent" is autonomy. A model answers a question. An agent pursues a goal across multiple steps, making decisions along the way.

Architecture

2. Agent Architecture Patterns

Modern AI agents can be designed using different architectural patterns, each suited to different levels of complexity.

Pattern	How it Works	Space Application
ReAct Reason + Act	Agent alternates between reasoning ("I should check telemetry...") and acting (calling a tool). Most common pattern.	Mission Control assistant that interprets anomalies and queries databases.
Multi-Agent Collaboration	Multiple specialized agents communicate. One plans, another executes, a third validates.	Distributed satellite constellation management with specialized sub-agents.
Tool-Use Function Calling	Agent has access to external tools (APIs, calculators, databases) and decides which to invoke.	EO data retrieval agent that queries Copernicus, processes imagery, and reports.
Hierarchical Manager + Workers	A "manager" agent breaks a task into sub-tasks and delegates to worker agents.	Mission planning where one agent handles trajectory, another handles payload scheduling.

Space Context

3. AI Agents in Space Operations

Space is the ultimate environment for autonomous agents: communication delays make real-time human control impossible for deep-space missions, and the stakes of failures are astronomically high.

🏥 Medical Triage Agent

Assists astronauts during medical emergencies on Mars (20-min comm delay). Analyzes symptoms, recommends treatments, and escalates to Earth physicians when possible.

🛰️ Anomaly Detection Agent

Monitors spacecraft telemetry in real-time, identifies off-nominal behavior, and autonomously triggers corrective procedures before ground confirmation.

🌍 Earth Observation Agent

Onboard satellite agent that prioritizes imagery acquisition based on cloud cover predictions and urgent observation requests, maximizing data utility.

🔧 Maintenance Scheduling Agent

Predicts component failures on the ISS using sensor fusion and schedules crew EVA maintenance windows to minimize mission disruption.

Connection to APP-2: Remember the Mars Medical Emergency simulation from Week 4? That exercise introduced you to AI-assisted decision-making under communication delay. Today, we examine how those AI systems are built.

Implementation

4. Anatomy of an Agent System

Building a functional agent requires connecting several components. Here is the typical stack:

System Prompt (Instructions)

Defines the agent's role, constraints, and personality. "You are a spacecraft anomaly detection agent. Your priority is crew safety above all else."

LLM Backbone (The Brain)

A large language model (Gemini, GPT, Claude) serves as the reasoning engine that interprets situations and generates plans.

Tools (Capabilities)

Functions the agent can call: query a database, run a calculation, fetch satellite imagery, send an alert, or control a system.

Memory (Context)

Short-term (conversation history) and long-term (knowledge base, past mission logs) memory that informs the agent's decisions.

Orchestration Loop

The control flow that manages the perceive-reason-act-observe cycle, including error handling, retries, and termination conditions.

// Simplified Agent Loop (Pseudocode)
function runAgent(goal, tools, llm) {
    let context = [{ role: "system", content: SYSTEM_PROMPT }];
    let observation = "Mission start. Awaiting telemetry.";

    while (!isGoalComplete(observation)) {
        context.push({ role: "user", content: observation });

        // Step 1: LLM reasons about the situation
        const plan = await llm.generate(context);

        // Step 2: Extract and execute tool calls
        if (plan.toolCalls) {
            observation = await executeTool(plan.toolCalls, tools);
        } else {
            observation = plan.response; // Final answer
        }
    }
    return observation;
}
                    

Simulation

5. Mission Simulation Design

In today's hands-on lab, you will build a Mars Mission Agent that operates under realistic constraints. Here is the scenario:

🔴 Scenario: Olympus Station Emergency

It is Sol 247 at Olympus Station, a crewed Mars habitat. The following events unfold:

Alert: CO2 scrubber efficiency drops to 72% (nominal: 95%).
Constraint: Communication delay to Earth is 14 minutes one-way.
Crew Status: 4 crew members, 1 currently on EVA.
Resources: 48 hours of backup O2, spare scrubber filters in Storage Bay C.

Your agent must assess the situation, prioritize actions, and generate a response plan before Earth can be consulted.

🚀 Open Interactive Lab

Review

Summary of Big Ideas

✓ Agents vs. Models: An AI model answers a question. An AI agent pursues a goal across multiple autonomous steps, using tools and memory.
✓ Autonomy is Essential in Space: Communication delays to Mars (up to 24 min round-trip) demand agents that can operate independently and make critical decisions.
✓ The ReAct Pattern: The Reason-Act loop is the most common and practical pattern for building mission support agents.
✓ Trust and Guardrails: In safety-critical space applications, agents must have explicit constraints, human-in-the-loop checkpoints, and fail-safe behaviors.

🎓 Agent Intelligence Quiz

1. What is the primary difference between an AI "model" and an AI "agent"?

An agent is faster than a model.

An agent can autonomously pursue goals across multiple steps using tools, while a model responds to single inputs.

An agent always uses deep learning while models use rule-based systems.

2. Why are AI agents particularly critical for deep-space missions?

They are cheaper to develop than traditional software.

Astronauts prefer talking to AI over Mission Control.

Communication delays make real-time human control impossible, requiring autonomous decision-making.

3. In the ReAct agent pattern, what does the agent do after "acting" (executing a tool)?

It immediately stops and waits for human input.

It observes the result and decides whether to continue reasoning or report a final answer.

It resets its memory and starts over from scratch.

4. Which agent architecture would best suit managing an autonomous satellite constellation?

A single ReAct agent controlling all satellites directly.

A Multi-Agent system where specialized agents manage different subsystems and communicate.

A rule-based system with no AI involvement.

🌍 Local to Global

From ISU Strasbourg to Mars

The European Space Agency (ESA) operates ESOC (European Space Operations Centre) just 200km from ISU in Darmstadt, Germany. ESOC is actively researching AI-based autonomous operations for deep-space missions, including the Argonaut lunar lander program.

Texas Connection: NASA's Johnson Space Center in Houston runs the CHAPEA (Crew Health and Performance Exploration Analog) Mars habitat simulations, where AI agents are actively being tested for crew health monitoring.

🗺️ Case Study

CIMON on the ISS

CIMON (Crew Interactive MObile CompanioN) is an AI-powered flying robot aboard the International Space Station, developed by Airbus and DLR. It uses IBM Watson to assist astronauts with procedures, act as a hands-free database interface, and even detect stress levels during experiments.

🔗 Learn more about CIMON

📝 Glossary

LLM Large Language Model: AI trained on text data that can reason and generate language.
ReAct Reason + Act: Agent pattern of alternating thought and tool use.
MAS Multi-Agent System: Multiple specialized agents cooperating on a task.
RAG Retrieval-Augmented Generation: Using external knowledge to enhance LLM responses.
MCP Model Context Protocol: Standard for connecting AI agents to external tools and data.