Skip to main content

Chat System

Overview​

The CROW dashboard features a comprehensive chat application powered by a multi-agentic AI system. Users can interact with their data through natural language queries, getting insights, generating reports, and exploring patterns across all data sources.

Multi-Agentic Architecture​

The chat system employs a multi-agentic setup where multiple AI agents work concurrently to provide comprehensive answers.

How It Works​

  1. Query Understanding: User's natural language query is analyzed
  2. Agent Dispatch: Multiple agents are dispatched concurrently:
    • Interaction Agent: Queries Interaction Service and raw interaction data
    • Pattern Agent: Queries Pattern Service for trend analysis
    • Product Agent: Retrieves product context and associations
    • Analytics Agent: Computes metrics and aggregations
  3. Concurrent Execution: Agents work in parallel across different data sources
  4. Result Aggregation: Responses from all agents are combined
  5. Response Generation: Unified answer generated using Gemini AI
  6. Streaming Response: Answer streamed back to user in real-time

Agent Framework​

  • Vercel AI SDK: Used for orchestrating multi-agent system
  • Gemini Models: All agents use Google Gemini for AI processing
  • Cloudflare AI Gateway: Routes all LLM requests for monitoring and optimization

Chat Service (Dashboard)​

The Chat Service is a BFF (Backend for Frontend) service specifically designed for the dashboard's chat application.

Purpose: Provide chat-specific functionality without being a generic reusable service.

Responsibilities​

  • Chat History Management: Store and retrieve past conversations
  • Message Storage: Persist all chat messages
  • Artifacts Management: Handle generated files, reports, and visualizations
  • Assets Storage: Manage images, charts, and other chat-related assets
  • Session Context: Maintain conversation context across messages
  • Multi-Agentic Orchestration: Coordinate agent dispatching and response aggregation

Technology Stack​

  • TypeScript service on Cloudflare Workers
  • Gemini AI models via Vercel AI SDK
  • Cloudflare AI Gateway for LLM request routing
  • D1 database for chat history
  • R2 storage for artifacts and assets
  • Vectorize for semantic search across past conversations

Capabilities​

  • Natural Language Queries: Ask questions in plain English
  • Context-Aware Responses: Understands organizational and product context
  • Multi-turn Conversations: Remembers conversation history within session
  • Export Generation: Create reports in various formats
  • Visualization Creation: Generate charts and diagrams
  • Depth Control: Configure analysis depth (quick vs. deep analysis)
  • Duration Filtering: Query specific time ranges (with user permission constraints)
  • Component Filtering: Focus on specific data sources (Web, Social, CCTV)
  • Product Filtering: Analyze specific products or product groups
  • Concurrent Processing: Multiple agents work simultaneously for faster responses

User Permissions Integration​

Chat access is controlled by user permissions:

  • Duration Constraints: Minimum duration (e.g., 1 year) set by admin
  • Component Constraints: Minimum 1 component, up to all 3 (Web, Social, CCTV)
  • Query Validation: System enforces constraints before processing queries

→ Learn more about User Permission Levels

Architecture​

Data Sources​

The multi-agentic system queries across multiple services:

  • Interaction Service: Raw interaction data via MCP-like interface
  • Pattern Service: Discovered patterns across time periods
  • Product Service: Product catalog and context
  • Organization Service: Organization context from Vectorize
  • Vectorize: Semantic search across all data
  • D1 Database: Metadata and structured queries
  • R2 Storage: Historical data and archives

Query Examples​

Users can ask questions like:

Data Queries​

  • "How many users viewed Product X last week?"
  • "What's the conversion rate for our landing page?"
  • "Show me the top 10 most viewed products"

Reports​

  • "Generate a weekly summary of user engagement"
  • "Create a report on cart abandonment rates"
  • "Analyze user behavior patterns for Product Y"

Analytics​

  • "Which products have the highest interaction rates?"
  • "What's the average time users spend on product pages?"
  • "Show me products with declining interest"

Patterns​

  • "What paths do users take before purchasing?"
  • "Identify common drop-off points"
  • "Show me user segments based on behavior"

Processing Flow​

Query Processing​

  1. Query Received: User submits natural language query
  2. Intent Classification: Determine query type
  3. Context Retrieval: Fetch relevant context from Vectorize
  4. Data Fetching: Query D1 for specific data
  5. Response Generation: Generate response using LLM
  6. Formatting: Format response for display

The ai-chat-service uses Vectorize for semantic search:

Export Capabilities​

The chat system supports multiple export formats:

Diagrams​

  • Mermaid: Flowcharts, sequence diagrams
  • Charts: Bar, line, pie charts
  • Tables: Formatted data tables

Documents​

  • PDF: Professional reports
  • LaTeX: Academic format
  • Markdown: Documentation

Data​

  • CSV: Spreadsheet-compatible
  • JSON: Programmatic access
  • Excel: Rich formatting

Conversation Features​

Multi-turn Context​

The chat maintains conversation context:

User: "Show me sales for Product X"
Bot: [Shows sales data]

User: "How does that compare to last month?"
Bot: [Shows comparison, understanding context]

Query Refinement​

Users can refine queries:

User: "Show me user interactions"
Bot: "I found 125,432 interactions. Filter by date, product, or source?"

User: "Filter by last month"
Bot: [Shows filtered results]

Integration​

Vectorize Integration​

  • Query embeddings for semantic search
  • Interaction context retrieval
  • Product similarity matching

D1 Integration​

  • User and organization context
  • Product catalog queries
  • Interaction metadata

R2 Integration​

  • Export file storage
  • Generated report access

Performance​

Response Times​

Query TypeTarget Time
Simple queries< 2 seconds
Complex analytics< 10 seconds
Report generation< 30 seconds

Optimizations​

  • Query result caching
  • Pre-computed aggregations
  • Streaming responses for long queries