RSS Reader API Documentation

The RSS Reader API provides access to links, documents, and media shared across Signal groups. It aggregates content from multiple sources into a unified feed that can be filtered by category, group, or platform.

Base URL: https://rss.irregulars.io/api/rss (external) or http://localhost:8082/api/rss (internal)

Authentication

Most endpoints are publicly accessible for reading content metadata. However, document download URLs require authentication.

API Key Authentication

Include the API key in the request header:

X-Api-Key: <your-api-key>

Document Access Control

User Type	Metadata	Download URL
Anonymous	Visible	null (requires_auth: true)
Authenticated	Visible	Presigned URL (7-day expiry)

---

Endpoints

GET /api/rss/links

Fetch links with filtering and pagination. Aggregates content from news articles, social media, repositories, documents, and videos.

Query Parameters

Parameter	Type	Default	Description
category	string	all	Filter by category: news, social, videos, documents, repos
group	string	-	Filter by Signal group ID
platform	string	-	Filter by platform (e.g., youtube, twitter, instagram)
q	string	-	Search query (searches title and summary)
limit	number	50	Results per page (max: 100)
offset	number	0	Pagination offset

Response

{
  "success": true,
  "links": [
    {
      "type": "news",
      "id": "news-123456",
      "url": "https://example.com/article",
      "title": "Article Title",
      "summary": "Article summary...",
      "platform": "example.com",
      "group_id": "abc123...",
      "group_name": "IrregularChat: Tech",
      "posted_by_name": "username",
      "forum_url": null,
      "post_count": 3,
      "first_posted_at": "2026-01-25T10:00:00Z",
      "last_posted_at": "2026-01-26T15:00:00Z",
      "image_url": "https://example.com/image.jpg",
      "stars": null,
      "language": null
    }
  ],
  "total": 952,
  "offset": 0,
  "limit": 50,
  "has_more": true
}

Content Types

Type	Source	Description
news	news_links	News articles with og:image metadata
social	social_media_links	Instagram, Twitter/X, LinkedIn, Facebook, Reddit, TikTok
video	social_media_links + archived_videos	YouTube, Vimeo, Twitch, PeerTube uploads
file	archived_files	PDFs, documents uploaded to Signal
repo	repository_links	GitHub, GitLab repositories

Document Response (Authenticated)

{
  "type": "file",
  "id": "89",
  "url": "https://s3.irregular.chat/...?X-Amz-Signature=...",
  "title": "Document Title.pdf",
  "summary": "AI-generated summary...",
  "platform": "document",
  "group_name": "IrregularChat: Research",
  "requires_auth": null
}

Document Response (Unauthenticated)

{
  "type": "file",
  "id": "89",
  "url": null,
  "title": "Document Title.pdf",
  "summary": "AI-generated summary...",
  "platform": "document",
  "group_name": "IrregularChat: Research",
  "requires_auth": true
}

---

GET /api/rss/categories

Get all content categories with link counts.

Response

{
  "success": true,
  "categories": [
    {
      "name": "news",
      "display_name": "News",
      "description": "News articles and stories",
      "link_count": 952,
      "icon": "newspaper"
    },
    {
      "name": "social",
      "display_name": "Social Media",
      "description": "Social media posts",
      "link_count": 594,
      "icon": "phone"
    },
    {
      "name": "videos",
      "display_name": "Videos",
      "description": "YouTube, Vimeo and archived videos",
      "link_count": 218,
      "icon": "video"
    },
    {
      "name": "documents",
      "display_name": "Documents",
      "description": "PDFs and files",
      "link_count": 77,
      "icon": "folder"
    },
    {
      "name": "repos",
      "display_name": "Repositories",
      "description": "GitHub, GitLab and other code repositories",
      "link_count": 66,
      "icon": "code"
    }
  ]
}

---

GET /api/rss/groups

Get Signal groups that have shared links, with total link counts.

Response

{
  "success": true,
  "groups": [
    {
      "id": "abc123...",
      "name": "IrregularChat: Main",
      "description": "Main discussion group",
      "group_type": "community",
      "link_count": 450
    },
    {
      "id": "def456...",
      "name": "IrregularChat: Tech",
      "description": "Technology discussions",
      "group_type": "topic",
      "link_count": 230
    }
  ]
}

---

GET /api/rss/stats

Get overall statistics for the RSS feed.

Response

{
  "success": true,
  "stats": {
    "total_links": 1907,
    "news_count": 952,
    "social_count": 594,
    "repo_count": 66,
    "file_count": 77,
    "video_count": 218,
    "groups_with_links": 76,
    "last_updated": "2026-01-26T21:40:20.094Z"
  }
}

---

GET /api/rss/search

Search across all content types by keyword.

Query Parameters

Parameter	Type	Default	Description
q	string	required	Search query (min 2 characters)
semantic	boolean	false	Use semantic search (future feature)
limit	number	20	Results limit (max: 50)

Response

{
  "success": true,
  "query": "drone",
  "method": "keyword",
  "results": [
    {
      "type": "news",
      "id": "news-789",
      "url": "https://example.com/drone-article",
      "title": "New Drone Regulations",
      "summary": "Article about drones...",
      "platform": "defensenews.com",
      "group_name": "IrregularChat: sUAS"
    }
  ],
  "total": 15
}

---

GET /api/rss/entities/graph

Get the entity co-occurrence graph for visualization.

Query Parameters

Parameter	Type	Default	Description
min_articles	number	10	Minimum article count for a node
min_edges	number	5	Minimum co-occurrence count for an edge
limit	number	60	Maximum number of nodes
edge_limit	number	100	Maximum number of edges
include_outlets	boolean	false	Include news outlet nodes

Response

{
  "success": true,
  "nodes": [
    {
      "name": "United States",
      "type": "locations",
      "article_count": 142,
      "recent_count": 18,
      "prev_count": 25
    }
  ],
  "edges": [
    {
      "source": "United States",
      "target": "Donald Trump",
      "weight": 87,
      "pmi": 0.234
    }
  ],
  "meta": {
    "total_nodes": 60,
    "total_edges": 95,
    "total_available_nodes": 245,
    "min_articles": 10,
    "min_cooccurrence": 5,
    "total_articles": 1907
  }
}

---

GET /api/rss/entities/detail/:name

Get detailed information about a specific entity.

Path Parameters

Parameter	Type	Description
name	string	Entity name (URL-encoded)

Response

{
  "success": true,
  "entity": {
    "name": "United States",
    "type": "locations",
    "article_count": 142,
    "recent_count": 18,
    "articles": [
      {
        "id": "news-123",
        "title": "Article about US policy",
        "url": "https://example.com/article",
        "posted_at": "2026-02-28T10:00:00Z"
      }
    ],
    "cooccurring": [
      { "name": "Donald Trump", "type": "people", "count": 87 },
      { "name": "Congress", "type": "organizations", "count": 45 }
    ]
  }
}

---

GET /api/rss/entities/search

Search entities by name.

Query Parameters

Parameter	Type	Default	Description
q	string	required	Search query (min 2 characters)
limit	number	20	Results limit

---

GET /api/rss/entity-aliases

Get all entity aliases.

Response

{
  "success": true,
  "aliases": [
    { "canonical": "United States", "alias": "USA", "entity_type": "locations" },
    { "canonical": "United States", "alias": "US", "entity_type": "locations" }
  ]
}

---

POST /api/rss/admin/entity-aliases (Admin)

Add a new entity alias.

Request Body

{
  "canonical": "United States",
  "alias": "America",
  "entity_type": "locations"
}

---

DELETE /api/rss/admin/entity-aliases (Admin)

Remove an entity alias.

Request Body

{
  "alias": "America",
  "entity_type": "locations"
}

---

Platform Coverage

Social Media Platforms

Platform	Metadata	Thumbnails	Notes
YouTube	Title, description, channel	Yes (via yt-dlp)	Full metadata extraction
Instagram	-	Yes (via og:image)	API restrictions limit metadata
Twitter/X	-	Yes (via og:image)	Limited metadata
LinkedIn	-	Yes (via og:image)	Limited metadata
Facebook	-	Yes (via og:image)	Limited metadata
Reddit	-	No	Blocks scrapers
TikTok	-	No	Blocks scrapers
Twitch	-	Yes	Stream thumbnails
Vimeo	-	Yes	Video thumbnails

News Sources

News links automatically extract:

• Title from <title> or og:title
• Summary from og:description or meta description
• Image from og:image or twitter:image

Success rate: ~73% have images (depends on site support for Open Graph tags)

Documents

Documents stored in MinIO S3:

• PDF title extraction (when available)
• AI-generated summaries (optional)
• Category based on source Signal group
• Virus scanned before storage

---

Error Responses

All endpoints return consistent error format:

{
  "success": false,
  "error": "Error message description"
}

HTTP Status Codes

Code	Meaning
200	Success
400	Bad request (missing/invalid parameters)
401	Unauthorized (invalid API key)
500	Server error
503	Database not initialized

---

Rate Limiting

Currently no rate limiting is enforced. Please be respectful with request frequency.

---

Examples

Fetch Latest News

curl "https://rss.irregulars.io/api/rss/links?category=news&limit=10"

Fetch YouTube Videos

curl "https://rss.irregulars.io/api/rss/links?category=videos&platform=youtube"

Search for “drone”

curl "https://rss.irregulars.io/api/rss/search?q=drone&limit=20"

Fetch Documents (Authenticated)

curl -H "X-Api-Key: YOUR_API_KEY" \
  "https://rss.irregulars.io/api/rss/links?category=documents"

Filter by Group

curl "https://rss.irregulars.io/api/rss/links?group=sUz0ycsujy8FjbEAISdmOaOopRuOgK2HalesYqD%2Ba34%3D"

---

Changelog

2026-03-01

• Entity graph API: GET /entities/graph returns co-occurrence graph with nodes, edges, PMI, and community data
• Entity detail API: GET /entities/detail/:name returns article history and co-occurring entities for a specific entity
• Entity search API: GET /entities/search for searching entities by name
• Entity aliases API: CRUD endpoints for managing entity name normalization (e.g., “USA” → “United States”)
• Admin backfill: POST /admin/backfill-entities triggers entity extraction from existing articles

2026-01-26

• Documents require authentication: Document download URLs now require API key authentication. Metadata (title, summary, group) remains publicly visible.
• Presigned URLs: Documents use 7-day presigned URLs for secure access
• YouTube metadata: Full title, description, and thumbnail extraction via yt-dlp
• Video category: YouTube, Vimeo, Twitch links now categorized as “videos” instead of “social”