ESPi Architecture & Query Flow

ESPi Architecture & Flow

For non-technical stakeholders, this functional flowchart shows the end-to-end journey of a user query in 6 simple steps, highlighting how the agent dynamically loads capabilities to build interactive responses:

1. User Submission: The user submits a question (e.g., "Compare CrowdStrike and Wiz licenses") along with what screen they are currently looking at on the platform.
2. Context Analysis: ESPi analyzes the request and uses previous chat memory and viewport context to determine user intent.
3. On-Demand Tool Search: If the query requires platform data, ESPi dynamically searches the Vector Tool Index for relevant skills (like contract summaries or capability metrics).
4. Execute Backend APIs: ESPi runs the matched tools to fetch live database records, calculate priorities, or render visual HTML reports.
5. Stream Markdown Response: The agent streams the response tokens in real-time, including thinking progress.
6. Rich Interactive UI: The browser renders the finalized text alongside live checklists, interactive citation cards, and actionable next steps.

High-Level Overview

ESPi is built on top of Spring AI and is backed by the Google Gemini model family (utilizing advanced reasoning capabilities). It acts as an agent orchestrator, meaning it doesn't just answer questions statically—it dynamically plans, reasons, searches the internal/external platform data, and delegates tasks to specialized sub-agents or rendering systems (like the stand-alone HTML Notebooks).

On-Demand Tool Discovery (The "Tool Search Tool" Pattern)

Rather than sending the entire library of available tools (contracts, capability colliders, notebooks, email, search, etc.) to the Gemini model on every request—which would bloat the context window, raise costs, and degrade model reasoning—ESPi implements Spring AI's Tool Search Tool pattern for on-demand discovery.

1. Minimal Initial Context: When the ChatClient request is built, the ToolSearchToolCallingAdvisor injects only one primary tool upfront: the toolSearchTool itself. This keeps initial token usage extremely low and avoids model confusion.
2. On-Demand Call: When the LLM starts reasoning and realizes it needs a specific platform capability (e.g., searching contracts or viewing a capability collider), it calls the toolSearchTool with a natural language query (e.g., toolSearchTool(query="commercial skills")).
3. Dynamic Expansion: The toolSearchTool executes under the hood, querying the Vector Store (Tool Index) to locate matching tool definitions.
4. Context Injection: The tool definitions that match (e.g., contractSummary, contractList, contractGet) are dynamically expanded and registered directly into the active prompt's tool options.
5. Execution of Discovered Tools: The LLM receives the newly discovered tools and is now able to invoke them directly (e.g., calling contractSummary) to get the actual platform business data.

This achieves massive token savings while maintaining access to a huge catalog of custom tools, allowing the agent to dynamically load capabilities on-the-fly.

Connector Legend

To help you read the sequence diagram, here is what each line style and connector represents:

ESPi Sequence Diagram

This diagram separates Infrastructure & Storage (like the Vector Store and ESPChatMemory) from high-level Platform Business Services.