System Architecture

Overview

CROW is a unified customer interaction intelligence platform that ingests data from three primary channels—website interactions, CCTV footage, and social media—then applies AI-driven processing to surface contextual insights. Built entirely on Cloudflare's edge infrastructure, the platform ensures low-latency global distribution and simplified operations.

The following diagram illustrates the high-level system architecture:

Platform Infrastructure

Cloudflare Edge Platform

The entire CROW platform is built on Cloudflare's edge infrastructure, providing:

• Global Distribution: 300+ data centers worldwide
• Low Latency: Sub-50ms response times globally
• Simplified Operations: No server management required
• Integrated Services: Seamless connectivity between all components

Core Infrastructure Components

Component	Cloudflare Service	Purpose
Compute	Workers	Serverless edge compute
Database	D1	SQLite-based relational database
Object Storage	R2	S3-compatible blob storage
Vector Database	Vectorize	AI embedding storage and search
Message Queue	Queues	Async message processing
Orchestration	Workflows	Complex multi-step processing
Real-time	Realtime SFU	WebRTC-based video streaming
Scheduling	Cron Triggers	Scheduled task execution

Data Channels

CROW ingests data from three primary channels, each with dedicated processing pipelines.

Website Interactions

Website interactions are captured via a lightweight JavaScript SDK deployed on client websites.

Key Features:

• Lightweight SDK (~5KB gzipped)
• Automatic event batching
• Offline support with retry logic
• Privacy-first design (no PII collection)
• Real-time event streaming

→ Learn more about Website Interaction Tracking

CCTV Footage

Real-time video analysis using Cloudflare Realtime SFU for low-latency video streaming.

Key Features:

• WebRTC-based low-latency streaming
• Frame extraction and analysis
• Motion detection optimization
• Privacy masking (face blur)
• Compressed archive storage

Social Media

Scheduled scraping of public social media mentions using Cron Triggers.

Key Features:

• Configurable scraping schedules
• Rate limit management
• Duplicate detection
• Sentiment pre-filtering
• Platform-specific adapters

Processing Layer

Cloudflare Queues

All incoming data flows through Cloudflare Queues for reliable, ordered processing.

Queue Architecture:

• interaction-queue: Website events and processed CCTV frames
• social-queue: Social media mentions
• processing-queue: AI processing tasks
• export-queue: Report generation tasks

Trigger Service

A dedicated Worker that monitors queues and initiates appropriate Workflows.

// Simplified trigger logic
export default {
  async queue(batch, env) {
    for (const message of batch.messages) {
      const { type, data } = message.body;
      
      switch (type) {
        case 'interaction':
          await env.WORKFLOW.create('process-interaction', data);
          break;
        case 'social':
          await env.WORKFLOW.create('analyze-social', data);
          break;
        case 'cctv':
          await env.WORKFLOW.create('process-video', data);
          break;
      }
    }
  }
};

Cloudflare Workflows

Complex, multi-step processing is orchestrated via Cloudflare Workflows.

Key Workflows:

• process-interaction: Handle website events
• analyze-social: Process social media mentions
• process-video: Analyze CCTV footage
• generate-insights: Create aggregated insights
• export-report: Generate exportable reports

AI Services

Three specialized AI services handle different processing needs.

ai-scraping-service

Handles product catalog discovery and web scraping tasks.

Capabilities:

• Multi-page crawling
• Dynamic content extraction
• Product data normalization
• Image analysis and OCR
• Competitor monitoring

→ Learn more about Product Scraping

ai-processing-service

Core AI processing for interaction analysis and insight generation.

Capabilities:

• Natural language understanding
• Sentiment analysis
• Entity extraction
• Behavior pattern recognition
• Anomaly detection

ai-chat-service

Powers the conversational interface for data querying.

Capabilities:

• Natural language queries
• Context-aware responses
• Multi-turn conversations
• Export generation
• Visualization creation

→ Learn more about Chat System

Data Storage

D1 Database

Primary relational database for structured data.

Schema Overview:

• Organizations, Users, API Keys
• Products and Categories
• Interaction metadata
• Processing jobs and status
• Configuration and settings

Key Features:

• SQLite compatibility
• Automatic replication
• Point-in-time recovery
• Global read distribution

R2 Object Storage

S3-compatible storage for binary data and archives.

Storage Buckets:

• raw-interactions: Original event data
• cctv-frames: Video frame extracts
• exports: Generated reports
• assets: Static assets

Key Features:

• Zero egress fees
• S3 API compatibility
• Lifecycle policies
• Automatic CDN integration

Vectorize

AI-native vector database for semantic search.

Vector Indexes:

• interaction-embeddings: Interaction text embeddings
• product-embeddings: Product description embeddings
• social-embeddings: Social media content embeddings

Key Features:

• High-dimensional vector storage
• Fast similarity search
• Metadata filtering
• Automatic index optimization

→ Learn more about Data Lake Architecture

API Layer

REST API

Standard RESTful endpoints for CRUD operations.

Endpoint Groups:

• /api/v1/organizations - Organization management
• /api/v1/products - Product catalog
• /api/v1/interactions - Interaction data
• /api/v1/insights - Generated insights
• /api/v1/exports - Export management

GraphQL API

Flexible querying for complex data relationships.

type Query {
  organization(id: ID!): Organization
  interactions(filter: InteractionFilter): [Interaction]
  insights(dateRange: DateRange): InsightsSummary
}

type Mutation {
  createProduct(input: ProductInput!): Product
  triggerExport(config: ExportConfig!): ExportJob
}

type Subscription {
  interactionCreated(orgId: ID!): Interaction
  insightGenerated(orgId: ID!): Insight
}

Agent2Agent (A2A) Protocol

Enterprise integration for AI agent communication.

Capabilities:

• Standardized agent messaging
• Capability discovery
• Task delegation
• Result aggregation

MCP Server

Model Context Protocol server for LLM integrations.

Supported Operations:

• Data retrieval for context
• Tool execution
• Resource access
• Prompt templates

→ Learn more about Integration API

Frontend Architecture

Split Application Design

Two separate applications optimized for their specific purposes.

Landing Application:

• Static site generation
• SEO optimized
• Fast initial load
• Authentication flows

Dashboard Application:

• React SPA
• Real-time updates
• Rich interactivity
• Complex visualizations

→ Learn more about Frontend Architecture

Security & Compliance

Authentication

• OAuth 2.0 / OIDC: User authentication
• API Keys: Machine-to-machine access
• JWT Tokens: Session management
• mTLS: Service-to-service security

Authorization

• RBAC: Role-based access control
• Organization Scoping: Data isolation
• API Rate Limiting: Abuse prevention
• Audit Logging: Compliance tracking

Data Protection

• Encryption at Rest: All storage encrypted
• Encryption in Transit: TLS 1.3 everywhere
• Data Residency: Regional data storage options
• GDPR Compliance: Privacy by design

Deployment & Operations

CI/CD Pipeline

Monitoring & Observability

• Cloudflare Analytics: Built-in metrics
• Worker Logs: Structured logging
• Error Tracking: Automatic error capture
• Performance Monitoring: Real-time latency tracking

Disaster Recovery

• Multi-region Replication: Automatic data replication
• Point-in-time Recovery: D1 database backups
• Failover: Automatic traffic rerouting
• Incident Response: Defined runbooks

Technology Stack Summary

Category	Technology	Purpose
Edge Compute	Cloudflare Workers	Serverless functions
Database	Cloudflare D1	Relational data
Object Storage	Cloudflare R2	Binary data & archives
Vector DB	Cloudflare Vectorize	AI embeddings
Queues	Cloudflare Queues	Message passing
Workflows	Cloudflare Workflows	Orchestration
Real-time	Cloudflare Realtime	Video streaming
Frontend	React + Next.js	User interfaces
API	REST + GraphQL	Data access
AI	OpenAI + Custom Models	Intelligence

Related Documentation

• Frontend Architecture - UI design and deployment
• Data Lake Architecture - Storage design
• Integration API - API design and protocols
• Services Architecture - Service architecture
• Website Interaction Tracking - SDK and ingestion
• Product Scraping - AI-powered scraping
• Chat System - Conversational interface
• User Signup Flow - Onboarding process