AI Meets Maps:
Building Geospatial Intelligence

Yesterday you built an AI chat interface: the user types a question, the LLM generates text. Today we take the next step: making the AI think spatially.

Imagine typing this into your app:

"Show me deforestation hotspots in the Amazon basin
from the last 6 months, and highlight any areas
where NDVI dropped below 0.2"

And the AI responds by:

1. Parsing the request into a geographic region (Amazon basin bounding box)
2. Querying relevant satellite data (Sentinel-2 NDVI time series)
3. Computing NDVI change detection over 6 months
4. Generating GeoJSON polygons for areas where NDVI < 0.2
5. Rendering those polygons on your Leaflet map with a red heatmap
6. Explaining the results in natural language alongside the visualization

Planet Labs has pioneered what they call Agentic Geospatial AI: an AI system that can autonomously search, retrieve, analyze, and visualize satellite imagery in response to natural language queries.

Our approach today is simpler but follows the same core architecture: an LLM that understands geospatial concepts and can call functions to interact with a map. We are building the minimum viable version of this pattern.

The entire pipeline flows through four stages. Your browser orchestrates each handoff.

What happens at each stage

1. User types a natural language query in the chat panel
2. Your JS code sends the query to the Gemini API, along with tool definitions
3. Gemini returns either text OR a functionCall object (with name + arguments)
4. Your code dispatches the function call to the matching JavaScript function
5. The map updates accordingly, and you send the result back to Gemini as a functionResponse
6. Gemini summarizes the result in natural language for the user

Function calling (also called "tool use") is the mechanism that transforms an LLM from a text generator into an agent that can take actions. Instead of outputting text, the model outputs a structured request to invoke a specific function.¹ This paradigm has been extended to geospatial contexts, where LLMs generate GeoJSON and invoke mapping functions via structured tool calls.⁸

When you call the Gemini API, you include a tools array that describes each function the AI is allowed to invoke. Think of it as handing the AI a toolbox and a manual for each tool.

// Define tools that Gemini can call
const tools = [{
  functionDeclarations: [{
    name: 'addMapMarker',
    description: 'Add a marker to the map at specified coordinates',
    parameters: {
      type: 'OBJECT',
      properties: {
        latitude:  { type: 'NUMBER',  description: 'Latitude in decimal degrees' },
        longitude: { type: 'NUMBER',  description: 'Longitude in decimal degrees' },
        label:     { type: 'STRING',  description: 'Marker popup label' }
      },
      required: ['latitude', 'longitude', 'label']
    }
  }]
}];

Anatomy of a Function Declaration

You need to define both the function declarations (for Gemini) and the JavaScript implementations (for Leaflet). Here are the essential functions for a geospatial AI assistant:

Here is the full cycle: declaring tools, sending a request, and dispatching the function call to Leaflet.

// 1. Define the tool declarations
const tools = [{
  functionDeclarations: [
    {
      name: 'addMapMarker',
      description: 'Add a labeled marker to the Leaflet map',
      parameters: {
        type: 'OBJECT',
        properties: {
          latitude:  { type: 'NUMBER', description: 'Latitude (WGS84)' },
          longitude: { type: 'NUMBER', description: 'Longitude (WGS84)' },
          label:     { type: 'STRING', description: 'Popup text' }
        },
        required: ['latitude', 'longitude', 'label']
      }
    }
  ]
}];

// 2. Send user query to Gemini with tools
const response = await model.generateContent({
  contents: [{ role: 'user', parts: [{ text: userQuery }] }],
  tools: tools
});

// 3. Check if the response contains a function call
const part = response.response.candidates[0].content.parts[0];

if (part.functionCall) {
  const { name, args } = part.functionCall;

  // 4. Dispatch to Leaflet
  if (name === 'addMapMarker') {
    L.marker([args.latitude, args.longitude])
      .addTo(map)
      .bindPopup(args.label)
      .openPopup();
  }
}

For an AI to be useful in Earth observation, it needs domain knowledge about how satellite data works. The most fundamental concept is the spectral index, especially NDVI.⁵ Rolf et al. (2024) argue that satellite data constitutes a distinct modality requiring specialized ML approaches.²

A system prompt is a special instruction block sent to the AI at the start of every conversation. It defines the AI's persona, knowledge, and behavioral rules. For geospatial AI, this is where you inject EO expertise.

What makes a great EO system prompt?

• Role definition: "You are an Earth Observation AI assistant"
• Domain knowledge: NDVI ranges, band names, coordinate systems
• Behavioral rules: "Always use WGS84 coordinates", "Always validate lat/lng ranges"
• Tool instructions: "Use function calling to display results on the map"
• Error handling: "If a location is ambiguous, ask the user to clarify"

Key design principles

Gemini is a multimodal model, meaning it can process text, images, audio, and video in a single request. For Earth observation, this means you can send a satellite image to Gemini and ask it to interpret what it sees. Recent work has explored foundation models for geospatial reasoning, assessing LLM capabilities in understanding geometries and spatial relations.⁶

// Send a satellite image + question to Gemini
const imagePart = {
  inlineData: {
    mimeType: 'image/png',
    data: base64EncodedImage  // base64 string of the tile
  }
};

const result = await model.generateContent([
  imagePart,
  { text: 'Analyze this Sentinel-2 image. Identify land cover types and estimate NDVI ranges for each visible zone.' }
]);

GeoJSON is an open standard format (IETF RFC 7946) for encoding geographic features using JSON.⁴ It is natively supported by Leaflet, Mapbox, Google Maps, and virtually every web mapping library. It is also the format we will teach our AI to generate.

{
  "type": "Feature",
  "properties": {
    "name": "ISU Strasbourg",
    "type": "university"
  },
  "geometry": {
    "type": "Point",
    "coordinates": [
      7.7676,
      48.5734
    ]
  }
}

One of the most powerful applications of function calling is having the AI generate valid GeoJSON from natural language descriptions. The user describes a geographic feature in plain English, and the AI produces the structured data.

Example conversation

For more complex shapes, you can define a addGeoJSON function that accepts raw GeoJSON:

{
  name: 'addGeoJSON',
  description: 'Add a GeoJSON FeatureCollection to the map. Use for complex shapes like country borders, river paths, or multi-polygon regions.',
  parameters: {
    type: 'OBJECT',
    properties: {
      geojson: { type: 'STRING', description: 'Valid GeoJSON string' },
      style:   { type: 'OBJECT', description: 'Leaflet path style options',
        properties: {
          color:       { type: 'STRING' },
          fillColor:   { type: 'STRING' },
          fillOpacity: { type: 'NUMBER' }
        }
      }
    },
    required: ['geojson']
  }
}

AI-generated spatial data can contain errors. You need a robust validation layer between Gemini's output and your map.

Common AI-generated GeoJSON errors

function validateCoordinates(lat, lng) {
  if (lat < -90 || lat > 90)   return { valid: false, error: 'Latitude must be between -90 and 90' };
  if (lng < -180 || lng > 180) return { valid: false, error: 'Longitude must be between -180 and 180' };
  return { valid: true };
}

An agentic AI goes beyond single-turn question answering. It can plan, execute, observe results, and adjust its approach over multiple steps. The foundational framework is the ReAct pattern (Reasoning + Acting).³ This paradigm is now being applied to Earth observation, with emerging work on LLM agents that autonomously query and analyze satellite data.⁷

The cycle repeats until the AI determines the task is complete. This is fundamentally different from a single prompt-response interaction.

Consider the user prompt: "Compare vegetation density in Strasbourg's urban center vs. the surrounding countryside."

An agentic AI would decompose this into multiple steps:

Academic References

1. [1] Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L., Cancedda, N., & Scialom, T. (2023). Toolformer: Language Models Can Teach Themselves to Use Tools. Advances in Neural Information Processing Systems (NeurIPS 2023), 36.
   DOI: 10.48550/arXiv.2302.04761
2. [2] Rolf, E., Klemmer, K., Robinson, C., & Kerner, H. (2024). Mission Critical: Satellite Data is a Distinct Modality in Machine Learning. Proceedings of the 41st International Conference on Machine Learning (ICML 2024).
   DOI: 10.48550/arXiv.2402.01444
3. [3] Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2023). ReAct: Synergizing Reasoning and Acting in Language Models. International Conference on Learning Representations (ICLR 2023).
   DOI: 10.48550/arXiv.2210.03629
4. [4] Butler, H., Daly, M., Doyle, A., Gillies, S., Hagen, S., & Schaub, T. (2016). The GeoJSON Format. RFC 7946, Internet Engineering Task Force (IETF).
   DOI: 10.17487/RFC7946
5. [5] Drusch, M., Del Bello, U., Carlier, S., Colin, O., Fernandez, V., Gascon, F., Hoersch, B., Isola, C., Laberinti, P., Martimort, P., Meygret, A., Spoto, F., Sy, O., Marchese, F., & Bargellini, P. (2012). Sentinel-2: ESA's Optical High-Resolution Mission for GMES Operational Services. Remote Sensing of Environment, 120, 25-36.
   DOI: 10.1016/j.rse.2011.11.026
6. [6] Ji, Y., Gao, S., Nie, Y., Majić, I., & Janowicz, K. (2025). Foundation Models for Geospatial Reasoning: Assessing the Capabilities of Large Language Models in Understanding Geometries and Topological Spatial Relations. International Journal of Geographical Information Science.
   DOI: 10.1080/13658816.2025.2511227
7. [7] Kao, C.-H., Zhao, W., Revankar, S., Speas, S., Bhagat, S., Datta, R., Phoo, C. P., Mall, U., Vondrick, C., Bala, K., & Hariharan, B. (2025). Towards LLM Agents for Earth Observation. arXiv preprint.
   DOI: 10.48550/arXiv.2504.12110
8. [8] Luo, Q., Xu, L., Lin, Q., Wu, S., Mao, R., Wang, C., Feng, H., Huang, B., & Du, Z. (2025). GeoJSON Agents: A Multi-Agent LLM Architecture for Geospatial Analysis, Function Calling vs Code Generation. arXiv preprint.
   DOI: 10.48550/arXiv.2509.08863
9. [9] Gramacki, P., Martins, B., & Szymański, P. (2024). Evaluation of Code LLMs on Geospatial Code Generation. Proceedings of the 7th ACM SIGSPATIAL Workshop on AI for Geographic Knowledge Discovery (GeoAI 2024).
   DOI: 10.1145/3687123.3698286

External Resources

• Gemini API: Function Calling Documentation
• Leaflet.js API Reference
• GeoJSON Specification
• geojson.io: Interactive GeoJSON Editor
• Turf.js: Advanced Geospatial Analysis in JavaScript
• Planet: Agentic Geospatial AI
• Sentinel-2 MSI User Guide (ESA)