Lesson 01: Context Windows

The Core Idea: As pioneered in Microsoft/OpenAI's seminal paper "Sparks of Artificial General Intelligence" (found in the References/ folder), the "intelligence" of an agent is often a direct reflection of its Context Window. The context window isn't just a memory buffer; it is the workspace of reason.

💾 Context RAM Simulator

Watch how new data "pushes out" old memories when the window is full.

Context Window Capacity: 5 Slots

🔬 The Theory

In the Sparks of AGI paper, researchers demonstrated that GPT-4 could solve complex, multi-domain problems not just because of its weights, but because its large context window allowed it to maintain a coherent "chain of thought."

⚠️ The "Lost in the Middle" Curve

LLMs are like humans: they remember the start of the meeting and the end, but zone out in the middle.

Prompt Start The "Dead Zone" Prompt End

Key Concepts for Students:

The In-Context Learning Phenomenon: Models can "learn" new tasks without weight updates simply by being shown examples in the context window.
Context Window as "RAM": If the weights are the Hard Drive (long-term knowledge), the Context Window is the RAM (active processing).
The "Lost in the Middle" Problem: Understanding that models often pay more attention to the beginning and end of a context window than the middle.

🛠️ The Lesson Plan (Day 1 Afternoon)

Part 1: Context Awareness (30 min)

Exercise: "The Amnesiac Agent."

Students start a fresh chat with a local model.
They give it 10 complex facts across 10 turns.
On the 11th turn, they ask a question that requires synthesizing Fact #1 and Fact #9.
Objective: Observe when the model starts "forgetting" or hallucinating.

Part 2: The "Sparks" Simulation (60 min)

Instruction: Use the Sparks_of_AGI.pdf to recreate the "Unicorn in TiKZ" experiment.
Challenge: Provide the model with a detailed architectural description of a building (referencing Dr. Sounny's dissertation themes). Ask the agent to generate Python code to render a 2D floor plan based only on the text provided in the context window.
Lesson: Show how feeding the model its own previous outputs (Self-Correction) within the window leads to "emergent" accuracy.

Part 3: Architecting the Memory (`agents.md`)

Action: Students will now manually update a memory.md file after every turn.
Discussion: Why does a shared file (agents.md) provide a "Permanent Context" that survives beyond a single session?
Lab Task: Create an "Agentic Persona" that reads a 50-page PDF (using the context window via RAG) and produces a 1-page executive summary without losing specific data points.