Chat System

Overview

The CROW dashboard features a comprehensive Ask CROW AI chat application powered by a tool-calling agentic loop. Users can interact with their data through natural language queries, getting insights, exploring patterns, and analyzing interactions across all data sources.

AI Architecture

The chat system uses a single-agent tool-calling loop (not a multi-agent dispatch system). A single AI model iterates through tool calls to gather data before generating a final response.

How It Works

Query Received: User submits a natural language query via the dashboard
Context Assembly: The last 10 messages from the session are loaded for conversation context
System Prompt: A system prompt configures the AI as "CROW AI, an intelligent retail analytics assistant" with tool-selection guidance
Agentic Loop (up to 5 iterations):
- The AI model is called with the conversation history and available tools
- If the model returns tool calls, the tools are executed in parallel
- Tool results and source references are appended to the conversation
- The loop continues until the model generates a final text response (no more tool calls)
Response Formatting: Source references are appended as footnotes ([1] Product: "...", [2] Interaction #abc (web))
Message Persistence: Both user and assistant messages are stored in D1 database

AI Model

Model: @cf/meta/llama-3.3-70b-instruct-fp8-fast (Cloudflare Workers AI)
Gateway: Cloudflare AI Gateway (crow-ai-gateway) with caching enabled
Framework: Native Cloudflare Workers AI binding (env.AI.run()) with tool-calling support
Max Iterations: 5 per query (fallback to non-tool response if exceeded)
Error Handling: Falls back to a simple non-tool call if the agentic loop fails

Available Tools

The AI has access to 5 tools, each calling internal CROW APIs:

Tool	Description	Data Source
`search_products`	Semantic search across product catalog	Product Service via Vectorize
`search_interactions`	Semantic search across interaction history (web, CCTV, social)	Interaction Service via Vectorize
`search_patterns`	Semantic search across behavioral patterns	Pattern Service via Vectorize
`search_org_context`	Search organization knowledge base	QnA Service via Vectorize
`get_interaction_summary`	Get total counts and channel breakdown	Interaction Service summary endpoint

Tool Selection Guidance (from system prompt):

For COUNT questions ("how many", "total"): Always use get_interaction_summary first
For SEARCH questions ("find", "show me"): Use the appropriate search tool
For greetings or casual conversation: Respond naturally without tools

Source References

Each tool execution generates typed source references:

product - Product catalog entries with title
interaction - Interaction records with source type
pattern - Behavioral pattern insights with type
org_context - Organization knowledge base entries with context type

References are formatted as numbered footnotes appended to the response.

BFF Services

bff-chat-service

The primary chat backend, a BFF (Backend for Frontend) service on Cloudflare Workers.

Responsibilities:

Chat Session Management: Create, retrieve, and delete sessions scoped to organizations
Message Storage: Persist all user and assistant messages in D1 database with timestamps
Session Titles: Auto-generated from first user message (first 6 words, max 40 chars)
AI Orchestration: Runs the agentic tool-calling loop via runCrewAgenticLoop()
Organization Scoping: All operations validate organization ID via X-Organization-Id header
A2A Endpoint: Exposes /a2a/tasks/send for Agent-to-Agent protocol integration
Agent Card: Serves /.well-known/agent.json with skills (product-search, interaction-analysis, pattern-insights, org-context)

API Routes:

POST /api/v1/chat/sessions - Create new chat session
POST /api/v1/chat/sessions/:sessionId/messages - Send message and get AI response
GET /api/v1/chat/sessions/:sessionId/messages - Get paginated message history
GET /api/v1/chat/sessions/organization/:orgId - List sessions for organization
GET /api/v1/chat/sessions/:sessionId - Get session details
DELETE /api/v1/chat/sessions/:sessionId - Delete session and messages

Technology Stack:

TypeScript on Cloudflare Workers
Hono (OpenAPIHono) web framework
Drizzle ORM for D1 database access
Cloudflare Workers AI (@cf/meta/llama-3.3-70b-instruct-fp8-fast)
Cloudflare AI Gateway (crow-ai-gateway)

bff-qna-service

The Q&A and knowledge base service, providing semantic search over organization context via Vectorize.

Responsibilities:

Semantic Search: Vector-based search using @cf/baai/bge-m3 embedding model and Vectorize index
Document Indexing: Index content into Vectorize for retrieval
Reindexing: Full reindex from GitHub-hosted documents
Session-based Q&A: Create sessions and send messages for standalone Q&A (separate from chat)
Scheduled Indexing: Cron handler for automatic document reindexing

API Routes:

GET /api/v1/qna/search - Semantic search with query, topK, and organizationId parameters
POST /api/v1/qna/index - Index a single document
POST /api/v1/qna/reindex - Full reindex from source
POST /api/v1/qna/sessions - Create Q&A session
GET /api/v1/qna/sessions/:sessionId - Get session
POST /api/v1/qna/sessions/:sessionId/messages - Send Q&A message
POST /api/v1/qna/query - Direct query

Technology Stack:

TypeScript on Cloudflare Workers
Hono (OpenAPIHono) web framework
Cloudflare Workers AI for embeddings (@cf/baai/bge-m3)
Cloudflare AI Gateway (crow-ai-gateway)
Cloudflare Vectorize for vector storage and similarity search

Architecture

Data Flow

Dashboard sends message to API Gateway
API Gateway routes to bff-chat-service
Chat service loads session context (last 10 messages)
Agentic loop calls Workers AI with tools
Tools fetch data via API Gateway (interactions, patterns, products) and QnA service (org context)
AI generates response with source footnotes
Message stored in D1 database
Response returned to dashboard

Data Sources Accessed by Tools

Product Service: Product catalog via API Gateway (/api/v1/products/search)
Interaction Service: Customer interactions via API Gateway (/api/v1/interactions/organization/:orgId/search, /summary)
Pattern Service: Behavioral patterns via API Gateway (/api/v1/patterns/organization/:orgId/search)
QnA Service: Organization knowledge base via bff-qna-service (/api/v1/qna/search) using Vectorize semantic search

Dashboard Integration

Chat History Sidebar

The sidebar displays past chat sessions via ChatHistoryContext:

Sessions fetched from bff-chat-service per organization
Clickable sessions load their message history
Session titles auto-generated from first user message
Local sessions merged with remote sessions for immediate UI updates
Expandable/collapsible sidebar section

Ask CROW Page Features

Landing View: Clean initial state with search input and suggested prompts
Chat View: Scrollable message thread with animated transitions
User Messages: Displayed via MessageBubble component
Assistant Messages: Displayed with CROW avatar, markdown-rendered content, and copy button
Generating State: Shows "CROW is thinking" with avatar and subtitle during AI processing
Attachment Menu: Placeholder for future file attachment support
Session Sync: Active session synced between sidebar clicks and page state

Query Examples

Users can ask questions like:

Data Queries

"How many interactions do we have?"
"What are our top products?"
"Show me recent web interactions"

Pattern Analysis

"What behavioral patterns have been detected?"
"Show me patterns from the last week"
"What trends are emerging in CCTV data?"

Organization Context

"Tell me about our company"
"What is our target market?"
"Summarize our product catalog"

Performance

Response Times

Query Type	Expected Time
Simple greetings (no tools)	< 3 seconds
Single tool call	3-8 seconds
Multi-tool queries	5-15 seconds

Optimizations

AI Gateway caching enabled (skipCache: false)
Parallel tool execution within each iteration
Context limited to last 10 messages to manage token usage
Fallback to non-tool response on agentic loop failure

System Architecture - Overall architecture
Integration API - API details including MCP and A2A
Data Storage - Storage design
Dashboard Features - Full dashboard feature list

Overview​

AI Architecture​

How It Works​

AI Model​

Available Tools​

Source References​

BFF Services​

bff-chat-service​

bff-qna-service​

Architecture​

Data Flow​

Data Sources Accessed by Tools​

Dashboard Integration​

Chat History Sidebar​

Ask CROW Page Features​

Query Examples​

Data Queries​

Pattern Analysis​

Organization Context​

Performance​

Response Times​

Optimizations​

Related Documentation​