Skip to main content

Services Architecture

Overview​

CROW is built on Cloudflare Workers with a service-oriented architecture. Each service is a dedicated Worker with specific responsibilities, communicating through Cloudflare's service bindings and Queues.

Service Architecture​

API Workers​

REST API Worker​

Purpose: Handle REST API requests

Responsibilities:

  • HTTP request handling
  • Authentication and authorization
  • Rate limiting
  • Request validation
  • Response formatting

Bindings:

  • D1: Database queries
  • R2: File operations
  • KV: Caching

GraphQL Worker​

Purpose: Handle GraphQL queries and mutations

Responsibilities:

  • GraphQL query execution
  • Schema management
  • Real-time subscriptions

Bindings:

  • D1: Data fetching
  • Vectorize: Semantic search

A2A Worker​

Purpose: Agent2Agent protocol handling

Responsibilities:

  • A2A protocol handling
  • Capability discovery
  • Task coordination
  • Result aggregation

Bindings:

  • D1: Task state
  • Queues: Async tasks

MCP Worker​

Purpose: Model Context Protocol server

Responsibilities:

  • MCP protocol implementation
  • LLM tool execution
  • Context retrieval
  • Resource access

Bindings:

  • D1: Data access
  • Vectorize: Semantic search

Ingestion Workers​

SDK Ingestion Worker​

Purpose: Receive events from JavaScript SDK

Responsibilities:

  • Event validation
  • Data transformation
  • Session management
  • Queue dispatch

Bindings:

  • D1: Session storage
  • Queues: Processing queue

CCTV Ingestion Worker​

Purpose: Process CCTV video streams

Responsibilities:

  • WebRTC handling via Realtime SFU
  • Frame extraction
  • Storage management

Bindings:

  • R2: Frame storage
  • Queues: Processing queue

Social Scraper Worker​

Purpose: Scrape social media mentions

Responsibilities:

  • Cron-triggered execution
  • Platform-specific scraping
  • Duplicate detection

Bindings:

  • D1: Scrape state
  • Queues: Processing queue

AI Services​

ai-scraping-service​

Purpose: Intelligent web scraping

Capabilities:

  • Multi-page crawling
  • Dynamic content handling
  • Product extraction
  • Data normalization

ai-processing-service​

Purpose: Core AI processing

Capabilities:

  • Natural language understanding
  • Sentiment analysis
  • Entity extraction
  • Behavior pattern recognition

ai-chat-service​

Purpose: Conversational AI

Capabilities:

  • Natural language queries
  • Context-aware responses
  • Export generation
  • Visualization creation

Processing Layer​

Trigger Service​

Purpose: Route queue messages to workflows

Responsibilities:

  • Queue consumption
  • Message routing
  • Workflow initiation
  • Error handling

Cloudflare Workflows​

Purpose: Orchestrate multi-step processing

Key Workflows:

  • process-interaction: Handle website events
  • analyze-social: Process social mentions
  • process-video: Analyze CCTV footage
  • generate-insights: Create aggregated insights
  • export-report: Generate reports

Service Communication​

Service Bindings​

Workers communicate via Cloudflare service bindings:

// Example: REST Worker calling AI service
export default {
  async fetch(request, env) {
    const result = await env.AI_PROCESSING.fetch(
      new Request('http://internal/process', {
        method: 'POST',
        body: JSON.stringify({ data: '...' })
      })
    );
    return result;
  }
};

Queue-based Communication​

Asynchronous communication via Cloudflare Queues:

// Producer
await env.PROCESSING_QUEUE.send({
  type: 'interaction',
  sessionId: '...',
  data: { ... }
});

// Consumer
export default {
  async queue(batch, env) {
    for (const message of batch.messages) {
      await processMessage(message.body, env);
    }
  }
};

Deployment​

Wrangler Configuration​

Each service has its own wrangler.toml:

name = "crow-rest-api"
main = "src/index.ts"
compatibility_date = "2024-01-01"

[[d1_databases]]
binding = "DB"
database_id = "..."

[[r2_buckets]]
binding = "STORAGE"
bucket_name = "crow-storage"

[[queues.producers]]
binding = "QUEUE"
queue = "processing-queue"

CI/CD Pipeline​