Troubleshooting Guide

This guide covers common issues, their root causes, and solutions. Organized from most to least common.

Quick Reference Commands

Check Bot Status

./deploy.sh status           # Container status + startup logs
./deploy.sh logs -t 100      # Last 100 log lines
./deploy.sh follow           # Real-time log streaming

Verify Environment Variables

./deploy.sh env                                          # Show configured vars (masked)
ssh proxmox-main "docker exec signal-bot-selfhosted-signal-bot-1 printenv | grep ADMIN"

Database Operations

# Connect to database
docker exec -it signal-bot-postgres psql -U signal_bot signal_bot

# Run a query
docker exec signal-bot-postgres psql -U signal_bot -c "SELECT COUNT(*) FROM signal_messages"

Check Bot Health

ssh proxmox-main "docker exec signal-bot-selfhosted-signal-bot-1 curl -s http://localhost:8080/health | jq"

---

Common Issues

Bot Not Responding to Commands

Symptoms: Commands are ignored, no response from bot

Check:

1. Bot container running: ./deploy.sh status
2. Signal-CLI daemon healthy: Check logs for daemon or RPC errors
3. Message timestamps: Check if messages are too old (skip logic)

Solutions:

• Restart container: ./deploy.sh restart
• If daemon died silently, full restart: ./deploy.sh deploy
• Check watchdog logs for auto-restart notifications

---

Admin Commands Not Working

Symptoms: !gtg, !pending, !createuser etc. say “Permission denied”

Root Cause: Admin environment variables not passed to container

Verify:

docker exec signal-bot-selfhosted-signal-bot-1 printenv | grep ADMIN
# Should show:
# ADMIN_PHONE_NUMBERS=+12345678901
# ADMIN_UUIDS=770b19f5-...

Solution:

1. Check .env file has ADMIN_PHONE_NUMBERS and ADMIN_UUIDS
2. Verify compose.yml maps these variables:

   environment:
     ADMIN_PHONE_NUMBERS: ${ADMIN_PHONE_NUMBERS}
     ADMIN_UUIDS: ${ADMIN_UUIDS}

3. Sync compose.yml and restart: ./deploy.sh deploy

---

”relation does not exist” Database Errors

Symptoms: Commands fail with “relation X does not exist”

Root Cause: Database tables not created or missing columns

Solution:

# Apply base schema
docker exec -i signal-bot-postgres psql -U signal_bot signal_bot < database/000_base_schema.sql

# Or apply specific migration
docker exec -i signal-bot-postgres psql -U signal_bot signal_bot < database/010_breakout_summary_columns.sql

---

Database Connection Issues

Symptoms: “Connection refused” or “could not connect to server”

Possible Causes:

1. Wrong DB_HOST value
2. Postgres container not running
3. Network issues between containers

Check:

# Verify postgres is running
docker compose ps

# Test connection
docker exec signal-bot-selfhosted-signal-bot-1 psql -h signal-bot-postgres -U signal_bot -c "SELECT 1"

Solution: Use full container name signal-bot-postgres instead of postgres in DB_HOST to avoid DNS conflicts with other databases on shared networks.

---

Container Shows Old Code After Deployment

Symptoms: Code changes not reflected after ./deploy.sh

Root Cause: Docker cache using old layers

Solution:

./deploy.sh deploy --no-cache

---

MinIO Presigned URL Signature Errors

Symptoms: SignatureDoesNotMatch when accessing files

Root Cause: Mismatch between internal and public endpoints

Check:

./deploy.sh test-minio

Explanation: MinIO requires TWO clients:

• Internal client: http://minio-irregularchat:9000 (for API operations)
• Presign client: https://s3.irregular.chat (for URL generation)

The presigned URL must be generated using the PUBLIC endpoint so signatures match.

---

Security Issues

Command Injection Prevention

The bot has been hardened against command injection:

Command	Protection
!calc	Whitelist parser (only 0-9 + - * / ( ) .), no eval()
!meme	Uses spawn() instead of exec(), no shell interpretation

If you see suspicious input in logs:

[SECURITY] !calc blocked suspicious input: __import__('os')

This is normal - the input was rejected.

---

Meme Text Overlay Security (Fixed Dec 2025)

Previous Vulnerability: !meme 42 $(whoami) executed shell commands

Root Cause: Used exec() which spawns a shell

Solution Applied:

• Changed to spawn() which passes arguments directly (no shell)
• Container now runs as node user instead of root

---

Signal-CLI Daemon Issues

Daemon Silent Death (Fixed Jan 2026)

Symptoms: Bot stops responding, no error logs

Root Cause: Daemon process died, watchdog wasn’t detecting it

Current Protection:

• Multiple detection layers: process state + socket + ping + message flow
• Auto-restart on daemon death
• Notifications to Bot Dev room on restart

Manual Restart:

./deploy.sh restart

---

Skip Old Messages Bug (Fixed Jan 2026)

Symptoms: After outage, bot processes all backlogged messages

Root Cause: Only checked one envelope path for timestamps

Current Behavior: The bot checks both:

• notification.params.envelope.timestamp
• notification.params.result.envelope.timestamp

---

Integration Issues

Authentik SSO Not Working

Symptoms: !accountinvite and !createuser fail silently

Check logs for:

Authentik API not configured. Required: AUTHENTIK_BASE_URL and AUTHENTIK_API_TOKEN

Solution: Add to .env:

AUTHENTIK_BASE_URL=https://sso.irregularchat.com
AUTHENTIK_API_TOKEN=<your-token>
AUTHENTIK_INVITE_FLOW_ID=<flow-uuid>

Get flow ID:

curl -s -H "Authorization: Bearer <token>" \
  "https://sso.irregularchat.com/api/v3/flows/instances/" \
  | jq '.results[] | "\(.pk) - \(.slug) - \(.name)"'

---

PeerTube Upload Failures

Check:

./deploy.sh test-api

Common issues:

• Token expired: Get new long-lived token from PeerTube admin
• Network: Verify bot can reach http://peertube:9000
• Disk space: PeerTube needs space for transcoding

---

Local AI (Open WebUI) Errors

Symptoms: !lai returns errors

Check:

• LOCAL_AI_URL is set correctly
• LOCAL_AI_API_KEY is valid
• Model exists: LOCAL_AI_MODEL=irregularbot

---

Content Extraction Issues

News Article Extraction Failing

Symptoms: “Couldn’t extract article content for summary”

Current Behavior: Bot uses multi-strategy extraction:

1. Googlebot UA + Readability
2. Chrome UA + Readability
3. archive.today check
4. Wayback Machine fallback

Sites that will still fail:

• Heavy JavaScript rendering (SPAs)
• Hard paywalls
• Cloudflare bot protection
• Login-required content

The bot provides bypass/archive links as fallback.

---

Signal-Generated Long Message Files

Symptoms: Auto-scan triggers on text messages

Explanation: Signal converts messages over ~4KB to .txt attachments named like signal-2025-12-18-213038.txt.

Current Behavior: Bot skips these files automatically. They won’t be scanned or archived.

---

Development Issues

Missing Meme Files

Symptoms: !meme returns “No memes available”

Check:

docker exec signal-bot-selfhosted-signal-bot-1 ls -la /app/memes/

Solution: Ensure memes are synced:

./deploy.sh sync  # Syncs memes automatically

---

Full Quality Meme Images Not Working

Symptoms: !meme add <title> (reply to image) uses thumbnail

Explanation: Signal only provides thumbnails for quoted attachments. The bot now:

1. Saves attachment-only messages to database
2. Looks up original attachment by timestamp
3. Uses full-quality file

Limitation: Only works for images sent AFTER Dec 20, 2025.

---

Server Directory Structure Reference

/home/signal-bot-selfhosted/
├── .env                    # Environment variables
├── compose.yml             # Docker Compose configuration
├── Dockerfile.selfhosted   # Docker build file
├── src/                    # Synced bot source code
├── signal-data/            # Signal CLI account data (CRITICAL - backup!)
├── data/
│   ├── postgres/           # PostgreSQL data
│   ├── redis/              # Redis data
│   ├── backups/            # PostgreSQL backups
│   ├── memes/              # Persistent meme storage
│   └── pgadmin/            # pgAdmin data (debug profile only)
└── database/               # SQL migration scripts

---

Event Service Issues

Event Times Display Wrong (UTC Instead of Eastern)

Symptoms: Event shows “2:00 AM Feb 6” instead of “9:00 PM Feb 5”

Cause: Docker container runs in UTC. Date formatting without explicit timezone.

Fixed in v3.21.0: All date formatting now uses timeZone: 'America/New_York'

---

Event URL Returns 404

Symptoms: Clicking event link shows 404 page

Cause: URL path mismatch - /events/{id} vs /event/{id}

Correct URL format: https://event.irregulars.io/event/{id} (singular)

Fixed in v3.21.0: Bot now uses API-provided URLs

---

Event Service Not Responding

Check:

curl -s https://event.irregulars.io/health | jq

Common causes:

• Cloudflare Worker deployment needed
• D1 database binding issue
• Missing environment variables in wrangler.toml

Deploy event service:

cd event-service && npx wrangler deploy

---

Poll Voting Not Working

Symptoms: !vote <id> returns “Event not found”

Check:

1. Event ID is correct (use !events to see IDs)
2. Event has active polls (🗳️ indicator)
3. Partial ID is at least 8 characters

v3.21.0+: Partial event IDs now supported (e.g., !vote d6595bcd)

---

Related Documentation

• Database Schema - Table structure and migrations
• Deployment Guide - Full deployment instructions
• Configuration Reference - All environment variables