File Management
The bot provides comprehensive file management capabilities for the IrregularChat community, including file search, automatic organization, and community archive links through the signed-in member portal.
File Search (!files)
Section titled “File Search (!files)”Search for files in the IrregularChat shared archive.
Basic Usage
Section titled “Basic Usage”!files <query> - Search filenames for query!files -g <keyword> - Search file contents (grep)!files -s <query> - Smart AI-powered search!files --cat <name> - Browse a specific category!files --list - List all categoriesExamples
Section titled “Examples”!files drone - Find files with "drone" in name!files -g FPV manual - Find content mentioning "FPV manual"!files -s UAV tactics - Smart search with AI-generated synonyms!files --cat Tech - Browse files in the Tech categorySearch Operators
Section titled “Search Operators”The file search supports advanced query operators:
| Operator | Example | Description |
|---|---|---|
| OR | drone OR uav | Match files with either term |
| “quotes” | "drone strike" | Exact phrase match |
| NOT | drone NOT toy | Exclude files with “toy” |
| -term | drone -toy | Exclude files with “toy” |
| (default) | drone attack | Files must contain both terms |
Selecting Results
Section titled “Selecting Results”After a search returns results, you can:
- Reply with a number to get a direct download link for that file (RustFS presigned URL)
- Reply
tldr Nto get an AI summary of PDF file N
Note: Download links expire after 15 minutes for security. The bot maintains your search session for 10 minutes, allowing quick follow-up interactions.
Automatic File Organization
Section titled “Automatic File Organization”When files are shared in Signal groups, the bot automatically:
- Scans for viruses using ClamAV (optional)
- Normalizes filenames with consistent formatting
- Categorizes files based on group name and content keywords
- Copies to archive in the IrregularChat directory structure
- Syncs to pCloud via rclone for cloud backup
Directory Structure
Section titled “Directory Structure”Files are organized into categories based on the Signal group they’re shared in:
| Group Topic | Archive Directory |
|---|---|
| AI/ML, NLP, Machine Learning | AI-ML |
| Unmanned Systems, Drones, UAV, FPV | UnmannedSystems |
| Counter-UAS, C-UAS | UnmannedSystems/Counter_Unmanned_Systems |
| DragonOS, RF, SDR | DragonOS |
| Cyber, Security, Infosec | Cybersecurity |
| Red Team, Pentest | Red Teaming |
| Blue Team, SOC, Defensive | Blue Team |
| OSINT, Research, Intel | Research |
| Influence, IWAR, PSYOP | Influence |
| Space | Space |
| Tech, Technology | Tech |
| Business, Startup | Business |
| Fabrication, 3D Print | Fabrication |
| Leadership | Leadership |
| Off Topic, General | Off topic |
Subdirectory Routing
Section titled “Subdirectory Routing”Files are further categorized into subdirectories based on:
Content Keywords:
- Russia/Russian content ->
Adversary/Russia - China/Chinese content ->
Adversary/China - Reports ->
Reports - Analysis ->
Analysis - Guides ->
Documents/Guides - Policy/Doctrine ->
Documents/Policy_and_Doctrine
File Types:
- Videos (.mp4, .mov, .mkv) ->
Audio_Video - Audio (.mp3, .wav, .m4a) ->
Audio_Video - Documents (.pdf, .doc, .ppt) ->
Documents - Images (.png, .jpg, .gif) ->
Images
Filename Normalization
Section titled “Filename Normalization”Files are renamed for consistency:
- Date prefix added:
2024-01-15_original_name.pdf - Spaces converted to underscores
- Special characters removed
- Length limited to 200 characters
Technical Infrastructure
Section titled “Technical Infrastructure”This section describes the technical architecture for file storage, synchronization, and access.
When a user selects a file from !files search results, provide a direct link to that file. This is achieved using MinIO S3-compatible storage with presigned URLs that provide secure, time-limited (15 min) direct downloads.
Architecture Overview
Section titled “Architecture Overview”┌─────────────────────┐ ┌──────────────────────┐ ┌─────────────────┐│ Signal Groups │────▶│ Signal Bot (Docker) │────▶│ Local Archive ││ (file uploads) │ │ /app/irregularchat │ │ /datadrive/ │└─────────────────────┘ └──────────────────────┘ │ IrregularChat/ │ └────────┬────────┘ │ ┌──────────────────────────────┼──────────────────────────────┐ │ │ │ ▼ ▼ ▼ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ MinIO S3 │ │ pCloud │ │ Signal Bot │ │ (primary) │ │ (legacy sync) │ │ Direct Access │ │ Presigned URLs │ │ rclone backup │ │ (local files) │ │ s3.irregular │ └─────────────────┘ └─────────────────┘ │ .chat │ └─────────────────┘MinIO S3 Storage (Primary)
Section titled “MinIO S3 Storage (Primary)”MinIO provides S3-compatible object storage for direct file downloads:
- Public URL:
https://s3.irregular.chat - Bucket:
irregularchat - Link Type: Presigned URLs with 24-hour expiration
- Access: Anyone with the presigned URL can download (time-limited)
Note: RustFS was evaluated in Dec 2025 for its lower memory footprint (~74 MiB vs 16 GiB), but had issues with presigned URL signature validation via Cloudflare Tunnel. MinIO remains the recommended solution.
Advantages over pCloud:
- Direct file links (not folder navigation)
- No IP-binding restrictions
- Self-hosted, full control
- S3-compatible API
Docker Volume Mounting
Section titled “Docker Volume Mounting”The bot runs in Docker and accesses the file archive via a bind mount:
compose.yml (production):
services: signal-bot: volumes: # Mount the IrregularChat archive into the container - /datadrive/IrregularChat/Topics:/app/irregularchat| Host Path | Container Path | Purpose |
|---|---|---|
/datadrive/IrregularChat/Topics | /app/irregularchat | Read/write access to file archive |
The IRREGULARCHAT_PATH environment variable tells the bot where to find files:
IRREGULARCHAT_PATH=/app/irregularchatrclone Configuration
Section titled “rclone Configuration”rclone keeps the historical pCloud archive available as a private import source.
rclone remote configuration:
[pcloud]type = pcloudtoken = {"access_token":"...", "token_type":"bearer", "expiry":"0001-01-01T00:00:00Z"}Sync command:
# Sync local archive to pCloudrclone sync /datadrive/IrregularChat pcloud:IrregularChat --progress
# One-way sync (local → cloud), preserving cloud files not in localrclone copy /datadrive/IrregularChat pcloud:IrregularChat --progressSync schedule: Runs periodically via cron or systemd timer to keep cloud storage up-to-date.
Member file links
Section titled “Member file links”The bot now links archived files directly to the community file drive. Links to individual files open through the portal’s private Silo gateway; a member must sign in to view or download. Folder results open the archive with that folder path prefilled in search. Approved chat uploads use the portal’s chat-file gateway. Members can star files and copy a sign-in link from the drive.
The pCloud remote remains an archive import source. Public pCloud links are being retired. Earlier chat messages with those links may stop working; search again with !files for a current link.
Historical pCloud URL Investigation Log
Section titled “Historical pCloud URL Investigation Log”This section documents attempts to enable direct folder navigation in pCloud public folder links.
Problem Statement
Section titled “Problem Statement”When users search for files and request a link, they want to navigate directly to the folder containing the file, not the root of the public folder (IrregularChat/).
What Has Been Tried
Section titled “What Has Been Tried”1. Direct Download Links via getpublinkdownload API
Section titled “1. Direct Download Links via getpublinkdownload API”Approach: Use pCloud’s API to generate direct download links for files.
const response = await fetch( `https://api.pcloud.com/getpublinkdownload?code=${PUBLIC_CODE}&fileid=${fileId}`);const data = await response.json();// Returns: { hosts: ["..."], path: "/..." }// Construct URL: https://{host}{path}Result: FAILED
- API generates IP-bound download links
- Link only works from the server’s IP address
- Users get error: “This link was generated for another IP address”
Why: pCloud’s download API binds the generated URL to the requesting IP for security.
2. Path-Based Folder Navigation (#folder=path)
Section titled “2. Path-Based Folder Navigation (#folder=path)”Approach: Use hash fragment with folder path string.
const folderUrl = `${PCLOUD_PUBLIC_URL}#folder=${encodeURIComponent(folderPath)}`;// Example: https://u.pcloud.link/publink/show?code=...#folder=UnmannedSystems/DocumentsResult: FAILED
- URL loads but navigates to root folder (IrregularChat/)
- Path strings are not recognized by pCloud’s public folder viewer
3. Numeric Folder ID Navigation (#folder=folderid)
Section titled “3. Numeric Folder ID Navigation (#folder=folderid)”Approach: Use pCloud’s numeric folder IDs instead of path strings.
// Build folder index from pCloud APIconst response = await fetch(`https://api.pcloud.com/showpublink?code=${PUBLIC_CODE}`);// Returns folder structure with numeric folderids// Example: UnmannedSystems/Documents -> folderid=29123489281
const folderUrl = `${PCLOUD_PUBLIC_URL}#folder=${folderId}`;// Example: https://u.pcloud.link/publink/show?code=...#folder=29123489281Result: FAILED
- Index builds successfully (14,139 files, 1,445 folders)
- Numeric folderid is retrieved correctly
- URL still navigates to root folder (IrregularChat/)
What Hasn’t Been Tried
Section titled “What Hasn’t Been Tried”1. Query Parameter Instead of Hash (?folder=)
Section titled “1. Query Parameter Instead of Hash (?folder=)”https://u.pcloud.link/publink/show?code=...&folder=29123489281Untested - may or may not work with pCloud’s URL parser.
2. Different URL Structures
Section titled “2. Different URL Structures”https://u.pcloud.link/publink/show?code=...&path=/UnmannedSystems/Documentshttps://e.pcloud.link/publink/show?code=...#folder=29123489281https://my.pcloud.com/#page=publink&code=...&folder=291234892813. pCloud Web App URL Format
Section titled “3. pCloud Web App URL Format”Inspecting how pCloud’s own web interface constructs URLs when browsing folders.
4. File-Specific Links (#file=fileid)
Section titled “4. File-Specific Links (#file=fileid)”https://u.pcloud.link/publink/show?code=...#file=123456785. Combined Folder + File Parameters
Section titled “5. Combined Folder + File Parameters”https://u.pcloud.link/publink/show?code=...#folder=29123489281&file=filename.pdf6. pCloud’s publink API with Different Parameters
Section titled “6. pCloud’s publink API with Different Parameters”GET https://api.pcloud.com/showpublink?code=...&forcedownload=0&folderid=291234892817. Creating Per-Folder Public Links
Section titled “7. Creating Per-Folder Public Links”Instead of one public link for the entire IrregularChat folder, create individual public links for each category folder. This would give direct access but require significant setup.
Current implementation
Section titled “Current implementation”packages/shared-utils/src/bot-utils/file-search.ts generates signed-in portal links from archive paths. The portal checks each file against the Silo manifest on access. The investigation above is retained as history; its public-link code and folder URL examples are no longer used by the bot.
Virus Scanning
Section titled “Virus Scanning”When enabled, files are scanned with ClamAV before archiving:
- Clean files are archived normally
- Infected files are rejected and not stored
- Scan errors (ClamAV unavailable) allow the file through with a warning
Technical Details
Section titled “Technical Details”File Index
Section titled “File Index”The bot maintains an in-memory index of all files for fast searching:
- Index refreshes every hour
- Maximum 50,000 files indexed
- Stores filename, path, size, modified date, and category
pCloud Index
Section titled “pCloud Index”A separate index maps file paths to pCloud file IDs:
- Refreshes every 30 minutes
- Stores both file IDs and folder IDs
- Enables potential future folder navigation URLs
Search Sessions
Section titled “Search Sessions”Search results are cached in sessions:
- 10-minute expiration
- Links session to group/user context
- Supports follow-up queries without re-searching
Configuration
Section titled “Configuration”Environment variables for file management:
MinIO S3 Configuration (Primary)
Section titled “MinIO S3 Configuration (Primary)”| Variable | Default | Description |
|---|---|---|
MINIO_ENDPOINT | - | Internal MinIO API endpoint (e.g., http://minio-irregularchat:9000) |
MINIO_ACCESS_KEY | - | MinIO access key (S3 compatible) |
MINIO_SECRET_KEY | - | MinIO secret key (S3 compatible) |
MINIO_BUCKET | irregularchat | MinIO bucket name |
MINIO_REGION | us-east-1 | MinIO region (required for S3 API) |
MINIO_PUBLIC_URL | - | Public URL for presigned links (e.g., https://s3.irregular.chat) |
File Archive Configuration
Section titled “File Archive Configuration”| Variable | Default | Description |
|---|---|---|
IRREGULARCHAT_PATH | /app/irregularchat | Base path for file archive |
pCloud Configuration (Historical)
Section titled “pCloud Configuration (Historical)”| Variable | Default | Description |
|---|---|---|
| The bot no longer needs a public pCloud link to generate file search results. |
Archive Statistics
Section titled “Archive Statistics”The 2026-10-16 Silo import inventory contains 15,503 files and 166,872,255,447 bytes.
Largest Categories
Section titled “Largest Categories”| Directory | Size |
|---|---|
| UnmannedSystems | 3.7 GB |
| Research | 1.8 GB |
| Testing | 1.3 GB |
| Dragon RF Room | 229 MB |
| Influence | 132 MB |
| Warfare | 113 MB |
| Fabrication | 100 MB |
Related Commands
Section titled “Related Commands”- All Commands - Complete command reference