Architecture Overview

This document provides a comprehensive overview of the Community Signal Moderation Bot architecture.

System Architecture Diagram

                                    ┌─────────────────────────────────────────┐
                                    │           External Services             │
                                    │  ┌─────────┐  ┌─────────┐  ┌─────────┐ │
                                    │  │ OpenAI  │  │ Authentik│ │ PeerTube│ │
                                    │  │   API   │  │   SSO   │  │  Video  │ │
                                    │  └────┬────┘  └────┬────┘  └────┬────┘ │
                                    │       │           │            │       │
                                    │  ┌────┴────┐  ┌───┴────┐  ┌───┴────┐  │
                                    │  │ Local AI│  │Discourse│  │ Outline │ │
                                    │  │(OpenWebUI)│ │ Forum  │  │  Wiki  │  │
                                    │  └────┬────┘  └────┬────┘  └────┬────┘ │
                                    │       │           │            │       │
                                    │  ┌────┴────┐  ┌───┴────────────┴────┐  │
                                    │  │Wikipedia│  │Google Knowledge Graph│  │
                                    │  │(MediaWiki)│ │    (optional)       │  │
                                    │  └────┬────┘  └────┬────────────────┘  │
                                    └───────┼────────────┼────────────────────┘
                                            │            │            │
┌─────────────┐                    ┌────────┴────────────┴────────────┴────────┐
│   Signal    │    JSON-RPC        │                                           │
│   Network   │◄──────────────────►│              Signal Bot (Node.js)         │
│             │                    │                                           │
└─────────────┘                    │  ┌──────────────┐  ┌──────────────────┐  │
                                   │  │ Command      │  │ Background       │  │
      ▲                            │  │ Handler      │  │ Schedulers       │  │
      │                            │  │ (40+ cmds)   │  │ - Announcements  │  │
      │                            │  └──────┬───────┘  │ - Reminders      │  │
      │                            │         │          │ - Rollups        │  │
      │                            │  ┌──────┴───────┐  │ - Cleanup        │  │
      │                            │  │ Utilities    │  └──────────────────┘  │
      │ signal-cli                 │  │ (70+ modules)│                        │
      │ daemon                     │  └──────┬───────┘                        │
      │                            │         │                                │
      │                            └─────────┼────────────────────────────────┘
      │                                      │
      │                                      ▼
┌─────┴─────────────────────────────────────────────────────────────────────────┐
│                              Docker Network                                    │
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐               │
│  │   PostgreSQL    │  │     Redis       │  │     MinIO       │               │
│  │   Database      │  │     Cache       │  │    S3 Storage   │               │
│  │   (~50 tables)  │  │                 │  │                 │               │
│  └─────────────────┘  └─────────────────┘  └─────────────────┘               │
│                                                                               │
│  ┌─────────────────┐  ┌─────────────────┐                                    │
│  │    Shlink       │  │    signal-cli   │                                    │
│  │  URL Shortener  │  │  (daemon mode)  │                                    │
│  └─────────────────┘  └─────────────────┘                                    │
└───────────────────────────────────────────────────────────────────────────────┘

Technology Stack

Layer	Technology	Version
Runtime	Node.js	20+
Language	TypeScript	5.x
Database	PostgreSQL	16
Cache	Redis	7
Signal Protocol	signal-cli	0.13.24 (master build primary)
Containerization	Docker Compose	v2
File Storage	MinIO (S3-compatible)	-

Core Components

1. Signal Bot (Node.js)

The main application orchestrates all bot functionality.

Key Files:

src/src/index-selfhosted.ts - Entry point
src/src/bot/signal-bot-v2.ts - Main bot logic (~7,600 LOC)
src/src/bot/command-handler.ts - Command routing (~27,300 LOC)

Responsibilities:

Connect to Signal network via signal-cli JSON-RPC
Process incoming messages
Route commands to handlers
Manage conversation context
Handle automated features (auto-scan, news, etc.)

2. signal-cli Daemon

signal-cli provides Signal protocol implementation via JSON-RPC.

Configuration:

signal-cli -a +1234567890 daemon --json-rpc --receive-mode=on-connection

Features:

Receives messages from Signal network
Sends messages and attachments
Manages groups and membership
Handles safety number changes

3. PostgreSQL Database

Stores all persistent data (~40 tables).

Key Tables:

signal_groups - Group settings and metadata
signal_messages - Message history
q_and_a_questions/answers - Q&A system
breakout_rooms - Temporary discussion groups
reminders - Scheduled reminders
memes - Meme library

See Database Schema for full documentation.

4. Redis Cache

Provides fast caching and session management.

Uses:

Rate limiting counters
Session data
Temporary state (dice games, RPS)
File search index cache

5. Background Schedulers

Automated processes running on intervals.

Scheduler	Purpose	Interval
Announcement	Deliver scheduled announcements	1 minute
Reminder	Deliver due reminders	1 minute
Rollup	Generate community digests	Configurable
Cleanup	Clean expired games/sessions	5 minutes
Watchdog	Monitor signal-cli health	60 seconds

Directory Structure

/Users/sac/Git/Community-Signal-Moderation-Bot/
├── src/                           # Source code directory
│   ├── src/                       # TypeScript source
│   │   ├── bot/                   # Bot core
│   │   │   ├── signal-bot-v2.ts   # Main bot class
│   │   │   ├── command-handler.ts # Command routing
│   │   │   ├── announcement-handler.ts
│   │   │   ├── signal-jsonrpc-client.ts
│   │   │   └── commands/          # Organized command modules
│   │   │       ├── core/
│   │   │       ├── fun/
│   │   │       ├── info/
│   │   │       └── utility/
│   │   ├── utils/                 # 70+ utility modules
│   │   │   ├── ai/               # AI clients
│   │   │   ├── unified-search.ts # Agentic search orchestrator (!ask)
│   │   │   ├── wikipedia-client.ts # Wikipedia MediaWiki API
│   │   │   ├── knowledge-graph-client.ts # Google KG API
│   │   │   ├── rss-search.ts     # RSS term-based search
│   │   │   ├── wiki-search.ts    # Wiki keyword + fetch
│   │   │   ├── wiki-embeddings.ts # Semantic embeddings
│   │   │   ├── minio-client.ts   # S3 storage
│   │   │   ├── shlink-client.ts  # URL shortening
│   │   │   ├── meme-manager.ts   # Meme handling
│   │   │   ├── breakout-manager.ts
│   │   │   └── ...
│   │   ├── db/                    # Database layer
│   │   │   └── postgres-client.ts # Data access (176 KB)
│   │   ├── scheduler/             # Background jobs
│   │   │   ├── announcement-scheduler.ts
│   │   │   ├── reminder-scheduler.ts
│   │   │   └── rollup-scheduler.ts
│   │   ├── config/                # Configuration
│   │   └── index-selfhosted.ts    # Entry point
│   ├── dist/                      # Compiled JavaScript
│   └── package.json
├── database/                      # SQL migrations
│   ├── 000_base_schema.sql        # Complete schema
│   └── 001-054_*.sql              # Incremental migrations
├── memes/                         # Local meme GIF files
├── docs/                          # Documentation
├── compose.yml                    # Docker Compose config
├── Dockerfile.selfhosted          # Docker build file
└── deploy.sh                      # Deployment script

Data Flow

Message Processing

1. User sends message in Signal group
        ↓
2. Signal network delivers to signal-cli daemon
        ↓
3. Daemon emits JSON-RPC notification
        ↓
4. Signal Bot receives notification
        ↓
5. Bot parses message and context
        ↓
6. If command → Route to CommandHandler
   If attachment → Auto-scan/archive
   If URL → Auto-process (news, social, git)
        ↓
7. Handler processes and responds
        ↓
8. Response sent via signal-cli
        ↓
9. User receives reply in Signal

Command Routing

// Message: "!ai What is the meaning of life?"

CommandHandler.handleMessage(message)
  ↓
parseCommand() → { command: 'ai', args: 'What is the meaning of life?' }
  ↓
routeCommand('ai', args, context)
  ↓
handleAiCommand(args, context)
  ↓
openai.chat.completions.create(...)
  ↓
sendReply(response)

External Services Integration

Service Communication

Service	Protocol	Auth Method
OpenAI	HTTPS REST	API Key
Local AI	HTTPS REST	API Key
PeerTube	HTTPS REST	Bearer Token
Discourse	HTTPS REST	API Key
Outline	HTTPS REST	API Key
Authentik	HTTPS REST	Bearer Token
MinIO	S3 Protocol	Access Key/Secret
Shlink	HTTPS REST	API Key
Wikipedia	HTTPS REST (MediaWiki Action API)	None (public)
Google Knowledge Graph	HTTPS REST	API Key (optional)

Network Architecture

The bot operates across multiple Docker networks:

┌─────────────────────────────────────────────────┐
│              signal-bot-network                  │
│  ┌────────────┐  ┌────────────┐  ┌───────────┐ │
│  │ signal-bot │  │  postgres  │  │   redis   │ │
│  └─────┬──────┘  └────────────┘  └───────────┘ │
└────────┼────────────────────────────────────────┘
         │
         │ (connected to external networks)
         │
┌────────┴─────────────┐  ┌──────────────────────┐
│   minio-network      │  │   postgres-network   │
│  ┌────────────────┐  │  │  ┌────────────────┐  │
│  │ minio-irregularchat│  │  │    shlink      │  │
│  └────────────────┘  │  │  └────────────────┘  │
└──────────────────────┘  └──────────────────────┘

Security Model

Admin Authorization

// Check performed for admin commands
isAdmin(senderUuid: string): boolean {
  return ADMIN_UUIDS.includes(senderUuid) ||
         ADMIN_PHONE_NUMBERS.includes(senderPhone);
}

Trust Verification

Safety number changes trigger verification flow
New members require vouch from existing member
Auto-removal for unverified accounts

Input Sanitization

!calc: Whitelist parser (no eval)
!meme: spawn() instead of exec() (no shell)
SQL: Parameterized queries only
URLs: Validated before processing

Performance Characteristics

Message Processing

Typical response time: 200-500ms
AI commands: 1-3 seconds
Video processing: 30-60 seconds

Database

~50 tables
Indexed for common queries
Connection pooling via pg Pool

Memory

Typical usage: 150-200MB
Peaks during video processing

Monitoring & Health

Health Endpoint

GET http://localhost:8080/health

{
  "status": "ok",
  "timestamp": "2026-01-23T12:00:00.000Z",
  "uptime": 86400,
  "memory": { "rss": 150000000, "heapUsed": 80000000 }
}

Watchdog Features

Daemon process monitoring
RPC connection checks
Message flow monitoring
Auto-restart on failure
Notification on restart

Database Schema - Table structure
Integrations - External service setup
Deployment Guide - Server operations
Configuration Reference - Environment variables

Architecture Overview

System Architecture Diagram

Technology Stack

Core Components

1. Signal Bot (Node.js)

2. signal-cli Daemon

3. PostgreSQL Database

4. Redis Cache

5. Background Schedulers

Directory Structure

Data Flow

Message Processing

Command Routing

External Services Integration

Service Communication

Network Architecture

Security Model

Admin Authorization

Trust Verification

Input Sanitization

Performance Characteristics

Message Processing

Database

Memory

Monitoring & Health

Health Endpoint

Watchdog Features

Related Documentation