Remove .kiro and .vscode from tracking
This commit is contained in:
AskIt
@@ -1,119 +0,0 @@
|
||||
# Bot System Improvements
|
||||
|
||||
## Recent Changes
|
||||
|
||||
### 1. Simplified Encryption (✅ Completed)
|
||||
|
||||
**Problem**: Required a separate `BOT_ENCRYPTION_KEY` environment variable, adding complexity to deployment.
|
||||
|
||||
**Solution**: Now uses `AUTH_SECRET` (which is already required) for encryption. A 32-byte key is derived using SHA-256.
|
||||
|
||||
**Benefits**:
|
||||
- One less environment variable to manage
|
||||
- Simpler deployment process
|
||||
- Same security level (AES-256-GCM)
|
||||
|
||||
**Migration**: Remove `BOT_ENCRYPTION_KEY` from your `.env` file. Existing encrypted keys will need to be re-encrypted (or you can keep the old key temporarily for migration).
|
||||
|
||||
### 2. Automatic Content Fetching (✅ Completed)
|
||||
|
||||
**Problem**: Users had to manually set fetch intervals, and content fetching was a separate scheduled process.
|
||||
|
||||
**Solution**: Content is now fetched automatically when a post is triggered. The system:
|
||||
1. Checks all active content sources
|
||||
2. Fetches fresh content (with retry logic)
|
||||
3. Selects the best content item
|
||||
4. Generates and posts
|
||||
|
||||
**Benefits**:
|
||||
- No manual fetch interval configuration needed
|
||||
- Always uses fresh content
|
||||
- Simpler user experience
|
||||
- Removed `fetchIntervalMinutes` column from database
|
||||
|
||||
**Migration**: Run the migration: `drizzle/0001_remove_fetch_interval.sql`
|
||||
|
||||
### 3. Fixed Reddit Link Previews (✅ Completed)
|
||||
|
||||
**Problem**: Reddit blocks regular HTTP scraping with aggressive bot detection, returning login/blocked pages instead of actual content with Open Graph tags. This caused bot posts with Reddit URLs to have NULL metadata for title, description, and image.
|
||||
|
||||
**Solution**: Implemented Reddit-specific handling using their oEmbed API (`https://www.reddit.com/oembed`), which is designed for embedding and doesn't require authentication.
|
||||
|
||||
**Benefits**:
|
||||
- Reddit link previews now work reliably
|
||||
- Extracts title, description (subreddit/author), and thumbnail when available
|
||||
- Falls back gracefully if oEmbed fails
|
||||
- Other sites continue to use standard OG tag scraping
|
||||
|
||||
**Files Modified**:
|
||||
- `src/lib/bots/posting.ts` - Added `isRedditUrl()` and `fetchRedditPreview()` functions
|
||||
- `src/app/api/media/preview/route.ts` - Same Reddit-specific handling for the preview API
|
||||
|
||||
## Implementation Details
|
||||
|
||||
### Encryption Changes
|
||||
|
||||
**File**: `src/lib/bots/encryption.ts`
|
||||
|
||||
```typescript
|
||||
// Before
|
||||
function getEncryptionKey(): Buffer {
|
||||
const keyEnv = process.env.BOT_ENCRYPTION_KEY;
|
||||
// ... complex base64/hex decoding logic
|
||||
}
|
||||
|
||||
// After
|
||||
function getEncryptionKey(): Buffer {
|
||||
const keyEnv = process.env.AUTH_SECRET;
|
||||
if (!keyEnv) {
|
||||
throw new Error('AUTH_SECRET environment variable is not set');
|
||||
}
|
||||
// Create a 32-byte key from AUTH_SECRET using SHA-256
|
||||
return crypto.createHash('sha256').update(keyEnv).digest();
|
||||
}
|
||||
```
|
||||
|
||||
### Content Fetching Changes
|
||||
|
||||
**File**: `src/lib/bots/posting.ts`
|
||||
|
||||
The `triggerPost` function now automatically fetches content before posting:
|
||||
|
||||
```typescript
|
||||
// Auto-fetch content from sources before posting
|
||||
const activeSources = await getActiveSourcesByBot(botId);
|
||||
if (activeSources.length > 0) {
|
||||
const fetchResults = await Promise.allSettled(
|
||||
activeSources.map(source =>
|
||||
fetchContentWithRetry(source.id, 2, { maxItems: 10, timeout: 15000 })
|
||||
)
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
### Database Schema Changes
|
||||
|
||||
**File**: `src/db/schema.ts`
|
||||
|
||||
Removed:
|
||||
```typescript
|
||||
fetchIntervalMinutes: integer('fetch_interval_minutes').default(30).notNull(),
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
After these changes, test:
|
||||
|
||||
1. ✅ Bot creation with API key encryption
|
||||
2. ✅ Manual post triggering (should auto-fetch content)
|
||||
3. ✅ Reddit content sources
|
||||
4. ✅ RSS content sources
|
||||
5. ✅ News API content sources
|
||||
|
||||
## Next Steps
|
||||
|
||||
Consider:
|
||||
- Add UI feedback showing when content is being fetched
|
||||
- Add content preview before posting
|
||||
- Add ability to regenerate post content if user doesn't like it
|
||||
- Add scheduling for automatic posts (cron-based)
|
||||
@@ -1,833 +0,0 @@
|
||||
# Design Document: Bot System
|
||||
|
||||
## Overview
|
||||
|
||||
This document describes the technical design for implementing an autonomous bot system in Synapsis. The system enables users to create bots that monitor external content sources (RSS, Reddit, news APIs), generate posts using LLM APIs, and respond to mentions. Bots operate as special user entities with clear identification and configurable personalities.
|
||||
|
||||
The design integrates with existing Synapsis infrastructure:
|
||||
- Drizzle ORM for database operations
|
||||
- Next.js API routes for endpoints
|
||||
- ActivityPub federation for cross-instance communication
|
||||
- Existing authentication and user systems
|
||||
|
||||
## Architecture
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "User Interface"
|
||||
UI[Bot Management UI]
|
||||
end
|
||||
|
||||
subgraph "API Layer"
|
||||
BA[Bot API Routes]
|
||||
CA[Content Source API]
|
||||
LA[Activity Log API]
|
||||
end
|
||||
|
||||
subgraph "Core Services"
|
||||
BM[Bot Manager]
|
||||
CS[Content Source Monitor]
|
||||
PS[Post Scheduler]
|
||||
MH[Mention Handler]
|
||||
CG[Content Generator]
|
||||
RL[Rate Limiter]
|
||||
AL[Activity Logger]
|
||||
end
|
||||
|
||||
subgraph "External Services"
|
||||
LLM[LLM Providers]
|
||||
RSS[RSS Feeds]
|
||||
RD[Reddit API]
|
||||
NEWS[News APIs]
|
||||
end
|
||||
|
||||
subgraph "Data Layer"
|
||||
DB[(PostgreSQL)]
|
||||
CACHE[Redis Cache]
|
||||
end
|
||||
|
||||
subgraph "Federation"
|
||||
AP[ActivityPub]
|
||||
end
|
||||
|
||||
UI --> BA
|
||||
BA --> BM
|
||||
BA --> CS
|
||||
BA --> LA
|
||||
|
||||
BM --> DB
|
||||
BM --> RL
|
||||
BM --> AL
|
||||
BM --> CG
|
||||
|
||||
CS --> RSS
|
||||
CS --> RD
|
||||
CS --> NEWS
|
||||
CS --> DB
|
||||
|
||||
CG --> LLM
|
||||
|
||||
PS --> BM
|
||||
MH --> BM
|
||||
MH --> CG
|
||||
|
||||
BM --> AP
|
||||
```
|
||||
|
||||
### Component Responsibilities
|
||||
|
||||
1. **Bot Manager**: Core orchestrator for bot lifecycle, configuration, and operations
|
||||
2. **Content Source Monitor**: Fetches and parses external content from RSS, Reddit, and news APIs
|
||||
3. **Post Scheduler**: Manages scheduled and autonomous posting based on configuration
|
||||
4. **Mention Handler**: Detects mentions and triggers response generation
|
||||
5. **Content Generator**: Interfaces with LLM providers to generate posts and replies
|
||||
6. **Rate Limiter**: Enforces posting limits and prevents abuse
|
||||
7. **Activity Logger**: Records all bot actions for auditing and debugging
|
||||
|
||||
## Components and Interfaces
|
||||
|
||||
### Bot Manager Service
|
||||
|
||||
```typescript
|
||||
interface BotManager {
|
||||
// Bot CRUD
|
||||
createBot(userId: string, config: BotCreateInput): Promise<Bot>;
|
||||
updateBot(botId: string, config: BotUpdateInput): Promise<Bot>;
|
||||
deleteBot(botId: string): Promise<void>;
|
||||
getBotsByUser(userId: string): Promise<Bot[]>;
|
||||
getBotById(botId: string): Promise<Bot | null>;
|
||||
|
||||
// Bot Operations
|
||||
triggerPost(botId: string, sourceContentId?: string): Promise<Post>;
|
||||
processScheduledPosts(): Promise<void>;
|
||||
suspendBot(botId: string, reason: string): Promise<void>;
|
||||
reinstateBot(botId: string): Promise<void>;
|
||||
}
|
||||
|
||||
interface BotCreateInput {
|
||||
name: string;
|
||||
handle: string;
|
||||
bio?: string;
|
||||
avatarUrl?: string;
|
||||
personality: PersonalityConfig;
|
||||
llmConfig: LLMConfig;
|
||||
contentSources?: ContentSourceConfig[];
|
||||
schedule?: ScheduleConfig;
|
||||
autonomousMode?: boolean;
|
||||
}
|
||||
|
||||
interface PersonalityConfig {
|
||||
systemPrompt: string;
|
||||
temperature: number;
|
||||
maxTokens: number;
|
||||
responseStyle?: string;
|
||||
}
|
||||
|
||||
interface LLMConfig {
|
||||
provider: 'openrouter' | 'openai' | 'anthropic';
|
||||
apiKey: string; // Encrypted before storage
|
||||
model: string;
|
||||
}
|
||||
```
|
||||
|
||||
### Content Source Monitor
|
||||
|
||||
```typescript
|
||||
interface ContentSourceMonitor {
|
||||
addSource(botId: string, config: ContentSourceConfig): Promise<ContentSource>;
|
||||
removeSource(sourceId: string): Promise<void>;
|
||||
fetchContent(sourceId: string): Promise<ContentItem[]>;
|
||||
processAllSources(): Promise<void>;
|
||||
}
|
||||
|
||||
interface ContentSourceConfig {
|
||||
type: 'rss' | 'reddit' | 'news_api';
|
||||
url: string;
|
||||
subreddit?: string; // For Reddit
|
||||
apiKey?: string; // For news APIs
|
||||
fetchInterval: number; // Minutes
|
||||
keywords?: string[]; // Optional filtering
|
||||
}
|
||||
|
||||
interface ContentItem {
|
||||
id: string;
|
||||
sourceId: string;
|
||||
title: string;
|
||||
content: string;
|
||||
url: string;
|
||||
publishedAt: Date;
|
||||
processed: boolean;
|
||||
}
|
||||
```
|
||||
|
||||
### Content Generator
|
||||
|
||||
```typescript
|
||||
interface ContentGenerator {
|
||||
generatePost(
|
||||
bot: Bot,
|
||||
sourceContent?: ContentItem,
|
||||
context?: string
|
||||
): Promise<GeneratedContent>;
|
||||
|
||||
generateReply(
|
||||
bot: Bot,
|
||||
mentionPost: Post,
|
||||
conversationContext: Post[]
|
||||
): Promise<GeneratedContent>;
|
||||
|
||||
evaluateContentInterest(
|
||||
bot: Bot,
|
||||
content: ContentItem
|
||||
): Promise<{ interesting: boolean; reason: string }>;
|
||||
}
|
||||
|
||||
interface GeneratedContent {
|
||||
text: string;
|
||||
tokensUsed: number;
|
||||
model: string;
|
||||
}
|
||||
```
|
||||
|
||||
### Mention Handler
|
||||
|
||||
```typescript
|
||||
interface MentionHandler {
|
||||
detectMentions(botId: string): Promise<Mention[]>;
|
||||
processMention(mentionId: string): Promise<void>;
|
||||
getUnprocessedMentions(botId: string): Promise<Mention[]>;
|
||||
}
|
||||
|
||||
interface Mention {
|
||||
id: string;
|
||||
botId: string;
|
||||
postId: string;
|
||||
authorId: string;
|
||||
content: string;
|
||||
createdAt: Date;
|
||||
processedAt?: Date;
|
||||
responsePostId?: string;
|
||||
}
|
||||
```
|
||||
|
||||
### Rate Limiter
|
||||
|
||||
```typescript
|
||||
interface RateLimiter {
|
||||
canPost(botId: string): Promise<{ allowed: boolean; reason?: string }>;
|
||||
recordPost(botId: string): Promise<void>;
|
||||
getPostCount(botId: string, windowHours: number): Promise<number>;
|
||||
getRemainingQuota(botId: string): Promise<{ daily: number; hourly: number }>;
|
||||
}
|
||||
|
||||
// Rate limit constants
|
||||
const RATE_LIMITS = {
|
||||
MAX_POSTS_PER_DAY: 50,
|
||||
MIN_POST_INTERVAL_MINUTES: 5,
|
||||
MAX_REPLIES_PER_HOUR: 20,
|
||||
};
|
||||
```
|
||||
|
||||
### Activity Logger
|
||||
|
||||
```typescript
|
||||
interface ActivityLogger {
|
||||
log(entry: ActivityLogEntry): Promise<void>;
|
||||
getLogsForBot(botId: string, options: LogQueryOptions): Promise<ActivityLog[]>;
|
||||
getErrorLogs(botId: string): Promise<ActivityLog[]>;
|
||||
}
|
||||
|
||||
interface ActivityLogEntry {
|
||||
botId: string;
|
||||
action: 'post_created' | 'mention_response' | 'content_fetched' |
|
||||
'llm_call' | 'error' | 'config_changed' | 'rate_limited';
|
||||
details: Record<string, unknown>;
|
||||
success: boolean;
|
||||
errorMessage?: string;
|
||||
}
|
||||
|
||||
interface LogQueryOptions {
|
||||
actionTypes?: string[];
|
||||
startDate?: Date;
|
||||
endDate?: Date;
|
||||
limit?: number;
|
||||
offset?: number;
|
||||
}
|
||||
```
|
||||
|
||||
## Data Models
|
||||
|
||||
### Database Schema Additions
|
||||
|
||||
```typescript
|
||||
// New tables for bot system
|
||||
|
||||
export const bots = pgTable('bots', {
|
||||
id: uuid('id').primaryKey().defaultRandom(),
|
||||
userId: uuid('user_id').notNull().references(() => users.id, { onDelete: 'cascade' }),
|
||||
name: text('name').notNull(),
|
||||
handle: text('handle').notNull().unique(),
|
||||
bio: text('bio'),
|
||||
avatarUrl: text('avatar_url'),
|
||||
|
||||
// Personality configuration (JSON)
|
||||
personalityConfig: text('personality_config').notNull(), // JSON
|
||||
|
||||
// LLM configuration
|
||||
llmProvider: text('llm_provider').notNull(), // openrouter, openai, anthropic
|
||||
llmModel: text('llm_model').notNull(),
|
||||
llmApiKeyEncrypted: text('llm_api_key_encrypted').notNull(),
|
||||
|
||||
// Scheduling
|
||||
scheduleConfig: text('schedule_config'), // JSON
|
||||
autonomousMode: boolean('autonomous_mode').default(false).notNull(),
|
||||
|
||||
// Status
|
||||
isActive: boolean('is_active').default(true).notNull(),
|
||||
isSuspended: boolean('is_suspended').default(false).notNull(),
|
||||
suspensionReason: text('suspension_reason'),
|
||||
suspendedAt: timestamp('suspended_at'),
|
||||
|
||||
// ActivityPub keys
|
||||
publicKey: text('public_key').notNull(),
|
||||
privateKeyEncrypted: text('private_key_encrypted').notNull(),
|
||||
|
||||
// Timestamps
|
||||
lastPostAt: timestamp('last_post_at'),
|
||||
createdAt: timestamp('created_at').defaultNow().notNull(),
|
||||
updatedAt: timestamp('updated_at').defaultNow().notNull(),
|
||||
}, (table) => [
|
||||
index('bots_user_id_idx').on(table.userId),
|
||||
index('bots_handle_idx').on(table.handle),
|
||||
index('bots_active_idx').on(table.isActive),
|
||||
]);
|
||||
|
||||
export const botContentSources = pgTable('bot_content_sources', {
|
||||
id: uuid('id').primaryKey().defaultRandom(),
|
||||
botId: uuid('bot_id').notNull().references(() => bots.id, { onDelete: 'cascade' }),
|
||||
|
||||
type: text('type').notNull(), // rss, reddit, news_api
|
||||
url: text('url').notNull(),
|
||||
subreddit: text('subreddit'), // For Reddit sources
|
||||
apiKeyEncrypted: text('api_key_encrypted'), // For news APIs
|
||||
|
||||
fetchIntervalMinutes: integer('fetch_interval_minutes').default(30).notNull(),
|
||||
keywords: text('keywords'), // JSON array for filtering
|
||||
|
||||
isActive: boolean('is_active').default(true).notNull(),
|
||||
lastFetchAt: timestamp('last_fetch_at'),
|
||||
lastError: text('last_error'),
|
||||
consecutiveErrors: integer('consecutive_errors').default(0).notNull(),
|
||||
|
||||
createdAt: timestamp('created_at').defaultNow().notNull(),
|
||||
updatedAt: timestamp('updated_at').defaultNow().notNull(),
|
||||
}, (table) => [
|
||||
index('bot_content_sources_bot_idx').on(table.botId),
|
||||
index('bot_content_sources_type_idx').on(table.type),
|
||||
]);
|
||||
|
||||
export const botContentItems = pgTable('bot_content_items', {
|
||||
id: uuid('id').primaryKey().defaultRandom(),
|
||||
sourceId: uuid('source_id').notNull().references(() => botContentSources.id, { onDelete: 'cascade' }),
|
||||
|
||||
externalId: text('external_id').notNull(), // Unique ID from source
|
||||
title: text('title').notNull(),
|
||||
content: text('content'),
|
||||
url: text('url').notNull(),
|
||||
|
||||
publishedAt: timestamp('published_at').notNull(),
|
||||
fetchedAt: timestamp('fetched_at').defaultNow().notNull(),
|
||||
|
||||
isProcessed: boolean('is_processed').default(false).notNull(),
|
||||
processedAt: timestamp('processed_at'),
|
||||
postId: uuid('post_id').references(() => posts.id), // If a post was created
|
||||
|
||||
interestScore: integer('interest_score'), // LLM evaluation score
|
||||
interestReason: text('interest_reason'),
|
||||
}, (table) => [
|
||||
index('bot_content_items_source_idx').on(table.sourceId),
|
||||
index('bot_content_items_processed_idx').on(table.isProcessed),
|
||||
index('bot_content_items_external_idx').on(table.externalId),
|
||||
]);
|
||||
|
||||
export const botMentions = pgTable('bot_mentions', {
|
||||
id: uuid('id').primaryKey().defaultRandom(),
|
||||
botId: uuid('bot_id').notNull().references(() => bots.id, { onDelete: 'cascade' }),
|
||||
postId: uuid('post_id').notNull().references(() => posts.id, { onDelete: 'cascade' }),
|
||||
|
||||
authorId: uuid('author_id').notNull().references(() => users.id),
|
||||
content: text('content').notNull(),
|
||||
|
||||
isProcessed: boolean('is_processed').default(false).notNull(),
|
||||
processedAt: timestamp('processed_at'),
|
||||
responsePostId: uuid('response_post_id').references(() => posts.id),
|
||||
|
||||
// For federated mentions
|
||||
isRemote: boolean('is_remote').default(false).notNull(),
|
||||
remoteActorUrl: text('remote_actor_url'),
|
||||
|
||||
createdAt: timestamp('created_at').defaultNow().notNull(),
|
||||
}, (table) => [
|
||||
index('bot_mentions_bot_idx').on(table.botId),
|
||||
index('bot_mentions_processed_idx').on(table.isProcessed),
|
||||
index('bot_mentions_created_idx').on(table.createdAt),
|
||||
]);
|
||||
|
||||
export const botActivityLogs = pgTable('bot_activity_logs', {
|
||||
id: uuid('id').primaryKey().defaultRandom(),
|
||||
botId: uuid('bot_id').notNull().references(() => bots.id, { onDelete: 'cascade' }),
|
||||
|
||||
action: text('action').notNull(), // post_created, mention_response, etc.
|
||||
details: text('details').notNull(), // JSON
|
||||
|
||||
success: boolean('success').notNull(),
|
||||
errorMessage: text('error_message'),
|
||||
|
||||
createdAt: timestamp('created_at').defaultNow().notNull(),
|
||||
}, (table) => [
|
||||
index('bot_activity_logs_bot_idx').on(table.botId),
|
||||
index('bot_activity_logs_action_idx').on(table.action),
|
||||
index('bot_activity_logs_created_idx').on(table.createdAt),
|
||||
]);
|
||||
|
||||
export const botRateLimits = pgTable('bot_rate_limits', {
|
||||
id: uuid('id').primaryKey().defaultRandom(),
|
||||
botId: uuid('bot_id').notNull().references(() => bots.id, { onDelete: 'cascade' }),
|
||||
|
||||
windowStart: timestamp('window_start').notNull(),
|
||||
windowType: text('window_type').notNull(), // daily, hourly
|
||||
postCount: integer('post_count').default(0).notNull(),
|
||||
replyCount: integer('reply_count').default(0).notNull(),
|
||||
|
||||
createdAt: timestamp('created_at').defaultNow().notNull(),
|
||||
}, (table) => [
|
||||
index('bot_rate_limits_bot_window_idx').on(table.botId, table.windowStart),
|
||||
]);
|
||||
```
|
||||
|
||||
### TypeScript Types
|
||||
|
||||
```typescript
|
||||
// Bot types
|
||||
export interface Bot {
|
||||
id: string;
|
||||
userId: string;
|
||||
name: string;
|
||||
handle: string;
|
||||
bio?: string;
|
||||
avatarUrl?: string;
|
||||
personalityConfig: PersonalityConfig;
|
||||
llmProvider: 'openrouter' | 'openai' | 'anthropic';
|
||||
llmModel: string;
|
||||
scheduleConfig?: ScheduleConfig;
|
||||
autonomousMode: boolean;
|
||||
isActive: boolean;
|
||||
isSuspended: boolean;
|
||||
suspensionReason?: string;
|
||||
lastPostAt?: Date;
|
||||
createdAt: Date;
|
||||
updatedAt: Date;
|
||||
}
|
||||
|
||||
export interface PersonalityConfig {
|
||||
systemPrompt: string;
|
||||
temperature: number;
|
||||
maxTokens: number;
|
||||
responseStyle?: string;
|
||||
}
|
||||
|
||||
export interface ScheduleConfig {
|
||||
type: 'interval' | 'times' | 'cron';
|
||||
intervalMinutes?: number;
|
||||
times?: string[]; // HH:MM format
|
||||
cronExpression?: string;
|
||||
timezone?: string;
|
||||
}
|
||||
|
||||
export interface ContentSource {
|
||||
id: string;
|
||||
botId: string;
|
||||
type: 'rss' | 'reddit' | 'news_api';
|
||||
url: string;
|
||||
subreddit?: string;
|
||||
fetchIntervalMinutes: number;
|
||||
keywords?: string[];
|
||||
isActive: boolean;
|
||||
lastFetchAt?: Date;
|
||||
lastError?: string;
|
||||
}
|
||||
```
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Bot Management
|
||||
|
||||
```
|
||||
POST /api/bots - Create a new bot
|
||||
GET /api/bots - List user's bots
|
||||
GET /api/bots/:id - Get bot details
|
||||
PUT /api/bots/:id - Update bot configuration
|
||||
DELETE /api/bots/:id - Delete a bot
|
||||
POST /api/bots/:id/suspend - Suspend a bot (admin)
|
||||
POST /api/bots/:id/reinstate - Reinstate a bot (admin)
|
||||
```
|
||||
|
||||
### API Key Management
|
||||
|
||||
```
|
||||
POST /api/bots/:id/api-key - Set/update LLM API key
|
||||
DELETE /api/bots/:id/api-key - Remove API key
|
||||
GET /api/bots/:id/api-key/status - Check if API key is configured
|
||||
```
|
||||
|
||||
### Content Sources
|
||||
|
||||
```
|
||||
POST /api/bots/:id/sources - Add content source
|
||||
GET /api/bots/:id/sources - List content sources
|
||||
PUT /api/bots/:id/sources/:sid - Update content source
|
||||
DELETE /api/bots/:id/sources/:sid - Remove content source
|
||||
POST /api/bots/:id/sources/:sid/fetch - Manually trigger fetch
|
||||
```
|
||||
|
||||
### Bot Operations
|
||||
|
||||
```
|
||||
POST /api/bots/:id/post - Manually trigger a post
|
||||
GET /api/bots/:id/mentions - Get pending mentions
|
||||
POST /api/bots/:id/mentions/:mid/respond - Manually respond to mention
|
||||
```
|
||||
|
||||
### Activity Logs
|
||||
|
||||
```
|
||||
GET /api/bots/:id/logs - Get activity logs
|
||||
GET /api/bots/:id/logs/errors - Get error logs only
|
||||
```
|
||||
|
||||
### Public Bot Endpoints (for federation)
|
||||
|
||||
```
|
||||
GET /api/users/:handle - Returns bot actor (with bot flag)
|
||||
GET /api/users/:handle/outbox - Bot's outbox
|
||||
POST /api/users/:handle/inbox - Bot's inbox (for mentions)
|
||||
```
|
||||
|
||||
|
||||
|
||||
## Correctness Properties
|
||||
|
||||
*A property is a characteristic or behavior that should hold true across all valid executions of a system—essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.*
|
||||
|
||||
### Property 1: Bot Creation Links to User
|
||||
|
||||
*For any* valid bot configuration and user, when a bot is created, the resulting bot entity SHALL be linked to the creating user's account and have a unique identifier.
|
||||
|
||||
**Validates: Requirements 1.1, 1.2**
|
||||
|
||||
### Property 2: Bot Listing Completeness
|
||||
|
||||
*For any* user with N bots, querying that user's bots SHALL return exactly N bots, all belonging to that user.
|
||||
|
||||
**Validates: Requirements 1.3**
|
||||
|
||||
### Property 3: Bot Deletion Cascade
|
||||
|
||||
*For any* bot with associated content sources, content items, mentions, and activity logs, deleting the bot SHALL remove all associated data from the database.
|
||||
|
||||
**Validates: Requirements 1.4**
|
||||
|
||||
### Property 4: Bot Identification Immutability
|
||||
|
||||
*For any* bot entity, the bot identification flag SHALL always be true and cannot be modified to false.
|
||||
|
||||
**Validates: Requirements 1.5, 12.1, 12.2, 12.4**
|
||||
|
||||
### Property 5: Bot Limit Enforcement
|
||||
|
||||
*For any* user at the maximum bot limit, attempting to create an additional bot SHALL fail with an appropriate error.
|
||||
|
||||
**Validates: Requirements 1.6**
|
||||
|
||||
### Property 6: API Key Encryption Round-Trip
|
||||
|
||||
*For any* valid API key, encrypting then decrypting the key SHALL produce the original key value, and the encrypted value SHALL differ from the original.
|
||||
|
||||
**Validates: Requirements 2.2, 2.3**
|
||||
|
||||
### Property 7: API Key Format Validation
|
||||
|
||||
*For any* string that does not match valid API key formats for supported providers, the validation SHALL reject the key.
|
||||
|
||||
**Validates: Requirements 2.1**
|
||||
|
||||
### Property 8: LLM Provider Support
|
||||
|
||||
*For any* of the supported LLM providers (OpenRouter, OpenAI, Anthropic), creating a bot with that provider SHALL succeed.
|
||||
|
||||
**Validates: Requirements 2.6**
|
||||
|
||||
### Property 9: Personality Configuration Persistence
|
||||
|
||||
*For any* valid personality configuration, storing then retrieving the configuration SHALL produce an equivalent configuration object.
|
||||
|
||||
**Validates: Requirements 3.1, 3.3, 3.4**
|
||||
|
||||
### Property 10: Personality in LLM Prompts
|
||||
|
||||
*For any* bot with a configured personality, all LLM calls (posts and replies) SHALL include the personality system prompt in the request.
|
||||
|
||||
**Validates: Requirements 3.2, 3.5**
|
||||
|
||||
### Property 11: Content Source URL Validation
|
||||
|
||||
*For any* invalid URL or unsupported source type, adding a content source SHALL fail with a validation error.
|
||||
|
||||
**Validates: Requirements 4.1**
|
||||
|
||||
### Property 12: RSS Parsing Correctness
|
||||
|
||||
*For any* valid RSS feed XML, parsing SHALL extract all items with their titles, content, URLs, and publication dates.
|
||||
|
||||
**Validates: Requirements 4.2**
|
||||
|
||||
### Property 13: Content Item Storage
|
||||
|
||||
*For any* fetched content item, the item SHALL be stored with its source reference and marked as unprocessed.
|
||||
|
||||
**Validates: Requirements 4.5**
|
||||
|
||||
### Property 14: Multiple Source Types Per Bot
|
||||
|
||||
*For any* bot, adding content sources of different types (RSS, Reddit, news API) SHALL all succeed and be retrievable.
|
||||
|
||||
**Validates: Requirements 4.6**
|
||||
|
||||
### Property 15: Fetch Error Retry with Backoff
|
||||
|
||||
*For any* content source that fails to fetch, consecutive failures SHALL increase the retry delay exponentially up to a maximum.
|
||||
|
||||
**Validates: Requirements 4.7**
|
||||
|
||||
### Property 16: Schedule Configuration Persistence
|
||||
|
||||
*For any* valid schedule configuration (interval, times, or cron), storing then retrieving SHALL produce an equivalent configuration.
|
||||
|
||||
**Validates: Requirements 5.1, 5.3**
|
||||
|
||||
### Property 17: Scheduled Post Triggering
|
||||
|
||||
*For any* bot with a due schedule and available content, the scheduler SHALL trigger post generation.
|
||||
|
||||
**Validates: Requirements 5.2, 5.4**
|
||||
|
||||
### Property 18: Skip When No Content
|
||||
|
||||
*For any* bot with a due schedule but no unprocessed content, the scheduler SHALL skip the posting cycle without creating a post.
|
||||
|
||||
**Validates: Requirements 5.5**
|
||||
|
||||
### Property 19: Rate Limit Enforcement
|
||||
|
||||
*For any* bot, posting more than 50 times per day OR posting within 5 minutes of the last post SHALL be rejected.
|
||||
|
||||
**Validates: Requirements 5.6, 10.1, 10.2, 10.4**
|
||||
|
||||
### Property 20: Autonomous Mode Content Evaluation
|
||||
|
||||
*For any* bot in autonomous mode with new content, the system SHALL evaluate content interest before deciding to post.
|
||||
|
||||
**Validates: Requirements 6.1, 6.2, 6.3**
|
||||
|
||||
### Property 21: Autonomous Mode Toggle
|
||||
|
||||
*For any* bot with autonomous mode disabled, the bot SHALL only create posts on schedule, not autonomously.
|
||||
|
||||
**Validates: Requirements 6.5**
|
||||
|
||||
### Property 22: Mention Detection
|
||||
|
||||
*For any* post that mentions a bot's handle, the mention SHALL be detected and stored for processing.
|
||||
|
||||
**Validates: Requirements 7.1, 7.2**
|
||||
|
||||
### Property 23: Mention Response Context
|
||||
|
||||
*For any* mention being processed, the LLM prompt SHALL include the original post content and conversation context.
|
||||
|
||||
**Validates: Requirements 7.3, 7.4**
|
||||
|
||||
### Property 24: Mention Chronological Processing
|
||||
|
||||
*For any* bot with multiple unprocessed mentions, processing SHALL occur in chronological order (oldest first).
|
||||
|
||||
**Validates: Requirements 7.5**
|
||||
|
||||
### Property 25: Reply Rate Limiting
|
||||
|
||||
*For any* bot, replying more than 20 times per hour SHALL be rejected.
|
||||
|
||||
**Validates: Requirements 7.6**
|
||||
|
||||
### Property 26: Activity Log Completeness
|
||||
|
||||
*For any* bot action (post, reply, error, config change), an activity log entry SHALL be created with action type, timestamp, and result.
|
||||
|
||||
**Validates: Requirements 8.1, 8.3, 8.4**
|
||||
|
||||
### Property 27: Activity Log Ordering
|
||||
|
||||
*For any* bot's activity logs, querying logs SHALL return them in reverse chronological order.
|
||||
|
||||
**Validates: Requirements 8.2**
|
||||
|
||||
### Property 28: Activity Log Filtering
|
||||
|
||||
*For any* log query with action type or date range filters, only matching logs SHALL be returned.
|
||||
|
||||
**Validates: Requirements 8.6**
|
||||
|
||||
### Property 29: Federation Post Distribution
|
||||
|
||||
*For any* post created by a bot, an ActivityPub Create activity SHALL be generated and queued for delivery to followers.
|
||||
|
||||
**Validates: Requirements 9.1**
|
||||
|
||||
### Property 30: Federated Mention Handling
|
||||
|
||||
*For any* mention received via ActivityPub from a remote instance, the mention SHALL be detected and processed like local mentions.
|
||||
|
||||
**Validates: Requirements 9.2**
|
||||
|
||||
### Property 31: ActivityPub Bot Flag
|
||||
|
||||
*For any* bot's ActivityPub actor representation, the actor object SHALL include `"type": "Service"` or a bot flag indicating automated status.
|
||||
|
||||
**Validates: Requirements 9.3**
|
||||
|
||||
### Property 32: Bot Follow Handling
|
||||
|
||||
*For any* follow request to or from a bot, the federation protocol SHALL handle the relationship correctly (Accept/Reject activities).
|
||||
|
||||
**Validates: Requirements 9.5**
|
||||
|
||||
### Property 33: Input Sanitization
|
||||
|
||||
*For any* user input containing potential injection attacks (SQL, XSS, command injection), the input SHALL be sanitized or rejected.
|
||||
|
||||
**Validates: Requirements 10.5**
|
||||
|
||||
### Property 34: Bot Suspension Enforcement
|
||||
|
||||
*For any* suspended bot, all actions (posting, replying, fetching) SHALL be blocked until reinstated.
|
||||
|
||||
**Validates: Requirements 10.6**
|
||||
|
||||
### Property 35: LLM Prompt Construction
|
||||
|
||||
*For any* post generation request, the LLM prompt SHALL combine source content with personality context and configured parameters.
|
||||
|
||||
**Validates: Requirements 11.1, 11.2**
|
||||
|
||||
### Property 36: Content Truncation
|
||||
|
||||
*For any* source content exceeding the maximum length, the content SHALL be truncated or summarized before being sent to the LLM.
|
||||
|
||||
**Validates: Requirements 11.3**
|
||||
|
||||
### Property 37: LLM Retry Logic
|
||||
|
||||
*For any* LLM call that fails, the system SHALL retry up to 3 times before logging an error.
|
||||
|
||||
**Validates: Requirements 11.4**
|
||||
|
||||
### Property 38: Post Content Validation
|
||||
|
||||
*For any* generated post, the content SHALL be validated against platform requirements (length, format) before publishing.
|
||||
|
||||
**Validates: Requirements 11.5**
|
||||
|
||||
### Property 39: Bot Creator Attribution
|
||||
|
||||
*For any* bot, the bot's data SHALL include a reference to the creating user that is retrievable.
|
||||
|
||||
**Validates: Requirements 12.3**
|
||||
|
||||
## Error Handling
|
||||
|
||||
### API Errors
|
||||
|
||||
| Error Code | Condition | Response |
|
||||
|------------|-----------|----------|
|
||||
| 400 | Invalid input (validation failure) | `{ error: "Invalid input", details: [...] }` |
|
||||
| 401 | Not authenticated | `{ error: "Authentication required" }` |
|
||||
| 403 | Not authorized (not bot owner) | `{ error: "Not authorized" }` |
|
||||
| 403 | Bot suspended | `{ error: "Bot is suspended", reason: "..." }` |
|
||||
| 404 | Bot/resource not found | `{ error: "Not found" }` |
|
||||
| 429 | Rate limit exceeded | `{ error: "Rate limit exceeded", retryAfter: seconds }` |
|
||||
| 500 | Internal error | `{ error: "Internal server error" }` |
|
||||
|
||||
### Content Source Errors
|
||||
|
||||
- **Network failures**: Log error, increment consecutive error count, schedule retry with exponential backoff
|
||||
- **Parse failures**: Log error with content sample, mark source as errored
|
||||
- **API rate limits**: Respect Retry-After header, adjust fetch interval
|
||||
|
||||
### LLM Provider Errors
|
||||
|
||||
- **Authentication failures**: Log error, notify user to check API key
|
||||
- **Rate limits**: Implement exponential backoff, queue requests
|
||||
- **Content policy violations**: Log violation, skip content item
|
||||
- **Timeout**: Retry up to 3 times with increasing timeout
|
||||
|
||||
### Federation Errors
|
||||
|
||||
- **Delivery failures**: Queue for retry with exponential backoff
|
||||
- **Invalid signatures**: Log and reject activity
|
||||
- **Unknown actors**: Attempt to fetch actor, reject if unavailable
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
### Unit Tests
|
||||
|
||||
Unit tests focus on specific examples and edge cases:
|
||||
|
||||
- Bot CRUD operations with valid/invalid inputs
|
||||
- API key encryption/decryption edge cases
|
||||
- RSS parsing with malformed feeds
|
||||
- Schedule parsing for various formats
|
||||
- Rate limit boundary conditions
|
||||
- Input sanitization for injection attempts
|
||||
|
||||
### Property-Based Tests
|
||||
|
||||
Property-based tests validate universal properties using a library like `fast-check`:
|
||||
|
||||
- **Minimum 100 iterations per property test**
|
||||
- Each test references its design document property
|
||||
- Tag format: `Feature: bot-system, Property N: [property title]`
|
||||
|
||||
Example test structure:
|
||||
```typescript
|
||||
// Feature: bot-system, Property 6: API Key Encryption Round-Trip
|
||||
test.prop([fc.string().filter(isValidApiKey)], (apiKey) => {
|
||||
const encrypted = encryptApiKey(apiKey);
|
||||
const decrypted = decryptApiKey(encrypted);
|
||||
expect(decrypted).toBe(apiKey);
|
||||
expect(encrypted).not.toBe(apiKey);
|
||||
});
|
||||
```
|
||||
|
||||
### Integration Tests
|
||||
|
||||
- Bot creation flow with database
|
||||
- Content source fetching with mocked external APIs
|
||||
- Mention detection and response flow
|
||||
- Federation activity generation and delivery
|
||||
- Rate limiting across multiple requests
|
||||
|
||||
### Test Coverage Requirements
|
||||
|
||||
- All 39 correctness properties must have corresponding property tests
|
||||
- Critical paths (bot creation, posting, mentions) require integration tests
|
||||
- Error handling paths require unit tests with specific error conditions
|
||||
@@ -1,174 +0,0 @@
|
||||
# Requirements Document: Bot System
|
||||
|
||||
## Introduction
|
||||
|
||||
This document specifies the requirements for implementing an autonomous bot system in Synapsis. The system enables users to create and manage bots that can monitor external content sources, generate posts using LLM APIs, and interact with other users through mentions and replies. Bots operate under user accounts with configurable personalities and behaviors while maintaining clear identification as automated entities.
|
||||
|
||||
## Glossary
|
||||
|
||||
- **Bot**: An automated entity that operates under a user account, capable of posting content and responding to interactions
|
||||
- **Bot_Creator**: A user who creates and manages one or more bots
|
||||
- **Bot_Manager**: The system component responsible for bot lifecycle and operations
|
||||
- **Content_Source**: An external data feed (RSS, Reddit, news API) that a bot monitors
|
||||
- **LLM_Provider**: An external API service (e.g., OpenRouter) that generates text content
|
||||
- **Personality_Configuration**: Settings that define a bot's voice, tone, and behavior patterns
|
||||
- **Post_Scheduler**: The component that determines when bots should create posts
|
||||
- **Mention_Handler**: The component that detects and processes mentions of bots
|
||||
- **Activity_Logger**: The component that records bot actions and events
|
||||
- **Federation_Protocol**: ActivityPub protocol used for inter-instance communication
|
||||
- **Rate_Limiter**: The component that enforces posting frequency limits
|
||||
|
||||
## Requirements
|
||||
|
||||
### Requirement 1: Bot Creation and Management
|
||||
|
||||
**User Story:** As a user, I want to create and manage bots under my account, so that I can deploy autonomous agents with specific purposes.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN a user creates a bot, THE Bot_Manager SHALL create a new bot entity linked to the user's account
|
||||
2. WHEN a bot is created, THE Bot_Manager SHALL assign it a unique identifier and mark it as a bot entity
|
||||
3. WHEN a user views their bots, THE Bot_Manager SHALL display all bots associated with their account
|
||||
4. WHEN a user deletes a bot, THE Bot_Manager SHALL remove the bot and all associated configurations
|
||||
5. WHEN a bot is displayed, THE System SHALL clearly indicate that it is a bot and not a human user
|
||||
6. THE Bot_Manager SHALL enforce a maximum limit of bots per user account
|
||||
|
||||
### Requirement 2: API Key Configuration
|
||||
|
||||
**User Story:** As a bot creator, I want to configure API keys for LLM providers, so that my bot can generate content.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN a user provides an API key, THE Bot_Manager SHALL validate the key format before storage
|
||||
2. WHEN storing API keys, THE Bot_Manager SHALL encrypt them using secure encryption
|
||||
3. WHEN retrieving API keys for bot operations, THE Bot_Manager SHALL decrypt them securely
|
||||
4. WHEN a user updates an API key, THE Bot_Manager SHALL replace the old key with the new encrypted key
|
||||
5. WHEN a user deletes a bot, THE Bot_Manager SHALL remove all associated API keys from storage
|
||||
6. THE Bot_Manager SHALL support multiple LLM provider types (OpenRouter, OpenAI, Anthropic)
|
||||
|
||||
### Requirement 3: Personality and Behavior Configuration
|
||||
|
||||
**User Story:** As a bot creator, I want to configure my bot's personality and behavior, so that it has a consistent voice and style.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN a user configures a bot personality, THE Bot_Manager SHALL store the personality prompt and parameters
|
||||
2. WHEN generating content, THE LLM_Provider SHALL use the configured personality prompt as context
|
||||
3. WHEN a user updates personality settings, THE Bot_Manager SHALL apply changes to future bot actions
|
||||
4. THE Personality_Configuration SHALL include system prompts, temperature, and other LLM parameters
|
||||
5. WHEN a bot responds to mentions, THE LLM_Provider SHALL maintain consistency with the configured personality
|
||||
|
||||
### Requirement 4: Content Source Monitoring
|
||||
|
||||
**User Story:** As a bot creator, I want my bot to monitor external content sources, so that it can generate posts based on real-world information.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN a user adds a content source, THE Bot_Manager SHALL validate the source URL and type
|
||||
2. WHEN monitoring RSS feeds, THE Content_Source SHALL fetch and parse feed items at regular intervals
|
||||
3. WHEN monitoring Reddit, THE Content_Source SHALL use Reddit API to retrieve posts from specified subreddits
|
||||
4. WHEN monitoring news APIs, THE Content_Source SHALL fetch articles using the configured API endpoint
|
||||
5. WHEN new content is detected, THE Content_Source SHALL store it for bot processing
|
||||
6. THE Content_Source SHALL support multiple source types per bot (RSS, Reddit, news APIs)
|
||||
7. WHEN a source fails to fetch, THE Content_Source SHALL log the error and retry with exponential backoff
|
||||
|
||||
### Requirement 5: Scheduled Posting
|
||||
|
||||
**User Story:** As a bot creator, I want to configure posting schedules, so that my bot posts at predictable intervals.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN a user sets a posting schedule, THE Post_Scheduler SHALL store the interval configuration
|
||||
2. WHEN the scheduled time arrives, THE Post_Scheduler SHALL trigger bot content generation
|
||||
3. THE Post_Scheduler SHALL support multiple schedule types (fixed interval, time-of-day, cron-like)
|
||||
4. WHEN generating scheduled posts, THE Bot_Manager SHALL select content from monitored sources
|
||||
5. WHEN no new content is available, THE Post_Scheduler SHALL skip the posting cycle
|
||||
6. THE Rate_Limiter SHALL enforce minimum intervals between posts to prevent spam
|
||||
|
||||
### Requirement 6: Autonomous Posting
|
||||
|
||||
**User Story:** As a bot creator, I want my bot to decide when to post based on content interest, so that it behaves more naturally.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN autonomous mode is enabled, THE Bot_Manager SHALL evaluate content sources for posting opportunities
|
||||
2. WHEN evaluating content, THE LLM_Provider SHALL determine if the content is interesting enough to post
|
||||
3. WHEN the bot decides to post, THE Bot_Manager SHALL generate and publish the post
|
||||
4. THE Rate_Limiter SHALL enforce maximum posting frequency for autonomous bots
|
||||
5. WHEN autonomous posting is disabled, THE Bot_Manager SHALL only post on schedule
|
||||
|
||||
### Requirement 7: Mention and Reply Detection
|
||||
|
||||
**User Story:** As a bot creator, I want my bot to detect and respond to mentions, so that it can interact with other users.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN a bot is mentioned in a post, THE Mention_Handler SHALL detect the mention within 5 minutes
|
||||
2. WHEN a mention is detected, THE Mention_Handler SHALL retrieve the mentioning post content
|
||||
3. WHEN processing a mention, THE LLM_Provider SHALL generate a contextually appropriate response
|
||||
4. WHEN generating responses, THE Bot_Manager SHALL include the original post context in the LLM prompt
|
||||
5. WHEN a bot receives multiple mentions, THE Mention_Handler SHALL process them in chronological order
|
||||
6. THE Rate_Limiter SHALL enforce limits on reply frequency to prevent abuse
|
||||
|
||||
### Requirement 8: Bot Activity Logging
|
||||
|
||||
**User Story:** As a bot creator, I want to view my bot's activity history, so that I can monitor its behavior and troubleshoot issues.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN a bot performs an action, THE Activity_Logger SHALL record the action type, timestamp, and result
|
||||
2. WHEN a user views bot logs, THE Activity_Logger SHALL display actions in reverse chronological order
|
||||
3. THE Activity_Logger SHALL record post creation, mention responses, errors, and configuration changes
|
||||
4. WHEN an error occurs, THE Activity_Logger SHALL record the error message and context
|
||||
5. THE Activity_Logger SHALL retain logs for a configurable retention period
|
||||
6. WHEN viewing logs, THE System SHALL support filtering by action type and date range
|
||||
|
||||
### Requirement 9: Federation Integration
|
||||
|
||||
**User Story:** As a system architect, I want bots to integrate with ActivityPub federation, so that they work seamlessly across instances.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN a bot creates a post, THE Federation_Protocol SHALL distribute it to followers on other instances
|
||||
2. WHEN a bot is mentioned from another instance, THE Mention_Handler SHALL detect and process the mention
|
||||
3. WHEN bot profiles are viewed from other instances, THE Federation_Protocol SHALL include bot identification metadata
|
||||
4. THE Federation_Protocol SHALL mark bot actors with the "bot" flag in ActivityPub actor objects
|
||||
5. WHEN a bot follows or is followed, THE Federation_Protocol SHALL handle the relationship correctly
|
||||
|
||||
### Requirement 10: Security and Rate Limiting
|
||||
|
||||
**User Story:** As a system administrator, I want bots to be rate-limited and secure, so that they cannot be abused or compromise the system.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. THE Rate_Limiter SHALL enforce a maximum of 50 posts per bot per day
|
||||
2. THE Rate_Limiter SHALL enforce a minimum interval of 5 minutes between posts
|
||||
3. WHEN API keys are stored, THE Bot_Manager SHALL use AES-256 encryption
|
||||
4. WHEN a bot exceeds rate limits, THE Rate_Limiter SHALL prevent further actions and log the violation
|
||||
5. THE Bot_Manager SHALL validate all user inputs to prevent injection attacks
|
||||
6. WHEN a bot is suspended for violations, THE Bot_Manager SHALL prevent all bot actions until reinstated
|
||||
|
||||
### Requirement 11: Content Generation
|
||||
|
||||
**User Story:** As a bot creator, I want my bot to generate engaging posts with commentary, so that it provides value to followers.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN generating a post, THE LLM_Provider SHALL combine source content with personality context
|
||||
2. WHEN creating commentary, THE LLM_Provider SHALL use the configured temperature and parameters
|
||||
3. WHEN source content is too long, THE Bot_Manager SHALL truncate or summarize it before sending to LLM
|
||||
4. WHEN LLM generation fails, THE Bot_Manager SHALL log the error and retry up to 3 times
|
||||
5. WHEN a post is generated, THE Bot_Manager SHALL validate it meets platform content requirements
|
||||
6. THE Bot_Manager SHALL support multiple content formats (text, links, quotes)
|
||||
|
||||
### Requirement 12: Bot Identification and Transparency
|
||||
|
||||
**User Story:** As a user, I want to clearly identify bots, so that I know when I'm interacting with automated entities.
|
||||
|
||||
#### Acceptance Criteria
|
||||
|
||||
1. WHEN displaying a bot profile, THE System SHALL show a "Bot" badge or indicator
|
||||
2. WHEN a bot posts, THE System SHALL include bot identification in the post metadata
|
||||
3. WHEN viewing bot details, THE System SHALL display the bot creator's username
|
||||
4. THE System SHALL prevent bots from removing or hiding their bot status
|
||||
5. WHEN federating bot profiles, THE Federation_Protocol SHALL include bot identification in ActivityPub objects
|
||||
@@ -1,428 +0,0 @@
|
||||
# Implementation Plan: Bot System
|
||||
|
||||
## Overview
|
||||
|
||||
This implementation plan breaks down the bot system into incremental coding tasks. Each task builds on previous work and includes property-based tests to validate correctness. The plan follows the existing Synapsis patterns for database schema, API routes, and ActivityPub federation.
|
||||
|
||||
## Tasks
|
||||
|
||||
- [x] 1. Set up database schema for bot system
|
||||
- [x] 1.1 Create bot tables in schema.ts
|
||||
- Add `bots` table with user reference, personality config, LLM config, and status fields
|
||||
- Add `botContentSources` table for RSS, Reddit, and news API sources
|
||||
- Add `botContentItems` table for fetched content
|
||||
- Add `botMentions` table for tracking mentions
|
||||
- Add `botActivityLogs` table for activity logging
|
||||
- Add `botRateLimits` table for rate limit tracking
|
||||
- Add relations and indexes
|
||||
- _Requirements: 1.1, 1.2, 4.5, 8.1_
|
||||
|
||||
- [x] 1.2 Run database migration
|
||||
- Generate and apply Drizzle migration
|
||||
- _Requirements: 1.1_
|
||||
|
||||
- [x] 2. Implement encryption utilities for API keys
|
||||
- [x] 2.1 Create encryption module in src/lib/bots/encryption.ts
|
||||
- Implement AES-256 encryption for API keys
|
||||
- Implement decryption function
|
||||
- Add key format validation for OpenRouter, OpenAI, Anthropic
|
||||
- _Requirements: 2.1, 2.2, 2.3, 10.3_
|
||||
|
||||
- [x] 2.2 Write property test for API key encryption round-trip
|
||||
- **Property 6: API Key Encryption Round-Trip**
|
||||
- **Validates: Requirements 2.2, 2.3**
|
||||
|
||||
- [x] 2.3 Write property test for API key format validation
|
||||
- **Property 7: API Key Format Validation**
|
||||
- **Validates: Requirements 2.1**
|
||||
|
||||
- [x] 3. Implement Bot Manager core functionality
|
||||
- [x] 3.1 Create Bot Manager service in src/lib/bots/botManager.ts
|
||||
- Implement createBot with user linking and unique ID generation
|
||||
- Implement updateBot for configuration changes
|
||||
- Implement deleteBot with cascade deletion
|
||||
- Implement getBotsByUser and getBotById
|
||||
- Implement bot limit enforcement per user
|
||||
- Generate ActivityPub keys for bots
|
||||
- _Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 1.6_
|
||||
|
||||
- [x] 3.2 Write property test for bot creation links to user
|
||||
- **Property 1: Bot Creation Links to User**
|
||||
- **Validates: Requirements 1.1, 1.2**
|
||||
|
||||
- [x] 3.3 Write property test for bot listing completeness
|
||||
- **Property 2: Bot Listing Completeness**
|
||||
- **Validates: Requirements 1.3**
|
||||
|
||||
- [x] 3.4 Write property test for bot deletion cascade
|
||||
- **Property 3: Bot Deletion Cascade**
|
||||
- **Validates: Requirements 1.4**
|
||||
|
||||
- [x] 3.5 Write property test for bot limit enforcement
|
||||
- **Property 5: Bot Limit Enforcement**
|
||||
- **Validates: Requirements 1.6**
|
||||
|
||||
- [x] 4. Checkpoint - Ensure all tests pass
|
||||
- Ensure all tests pass, ask the user if questions arise.
|
||||
|
||||
- [x] 5. Implement Bot API routes
|
||||
- [x] 5.1 Create bot CRUD API routes
|
||||
- POST /api/bots - Create bot
|
||||
- GET /api/bots - List user's bots
|
||||
- GET /api/bots/[id] - Get bot details
|
||||
- PUT /api/bots/[id] - Update bot
|
||||
- DELETE /api/bots/[id] - Delete bot
|
||||
- _Requirements: 1.1, 1.3, 1.4_
|
||||
|
||||
- [x] 5.2 Create API key management routes
|
||||
- POST /api/bots/[id]/api-key - Set API key
|
||||
- DELETE /api/bots/[id]/api-key - Remove API key
|
||||
- GET /api/bots/[id]/api-key/status - Check key status
|
||||
- _Requirements: 2.1, 2.2, 2.4_
|
||||
|
||||
- [x] 5.3 Write property test for LLM provider support
|
||||
- **Property 8: LLM Provider Support**
|
||||
- **Validates: Requirements 2.6**
|
||||
|
||||
- [x] 6. Implement personality configuration
|
||||
- [x] 6.1 Create personality config types and validation in src/lib/bots/personality.ts
|
||||
- Define PersonalityConfig interface
|
||||
- Implement validation for system prompt, temperature, maxTokens
|
||||
- Implement storage and retrieval functions
|
||||
- _Requirements: 3.1, 3.3, 3.4_
|
||||
|
||||
- [x] 6.2 Write property test for personality configuration persistence
|
||||
- **Property 9: Personality Configuration Persistence**
|
||||
- **Validates: Requirements 3.1, 3.3, 3.4**
|
||||
|
||||
- [x] 7. Implement Content Source Monitor
|
||||
- [x] 7.1 Create content source service in src/lib/bots/contentSource.ts
|
||||
- Implement addSource with URL validation
|
||||
- Implement removeSource
|
||||
- Implement source type validation (RSS, Reddit, news API)
|
||||
- _Requirements: 4.1, 4.6_
|
||||
|
||||
- [x] 7.2 Implement RSS feed parser
|
||||
- Create RSS parser using xml parsing
|
||||
- Extract title, content, URL, publication date
|
||||
- Handle malformed feeds gracefully
|
||||
- _Requirements: 4.2_
|
||||
|
||||
- [x] 7.3 Implement content fetching with retry logic
|
||||
- Implement fetch with exponential backoff on failure
|
||||
- Track consecutive errors per source
|
||||
- Store fetched content items
|
||||
- _Requirements: 4.5, 4.7_
|
||||
|
||||
- [x] 7.4 Write property test for content source URL validation
|
||||
- **Property 11: Content Source URL Validation**
|
||||
- **Validates: Requirements 4.1**
|
||||
|
||||
- [x] 7.5 Write property test for RSS parsing correctness
|
||||
- **Property 12: RSS Parsing Correctness**
|
||||
- **Validates: Requirements 4.2**
|
||||
|
||||
- [x] 7.6 Write property test for content item storage
|
||||
- **Property 13: Content Item Storage**
|
||||
- **Validates: Requirements 4.5**
|
||||
|
||||
- [x] 7.7 Write property test for multiple source types per bot
|
||||
- **Property 14: Multiple Source Types Per Bot**
|
||||
- **Validates: Requirements 4.6**
|
||||
|
||||
- [x] 7.8 Write property test for fetch error retry with backoff
|
||||
- **Property 15: Fetch Error Retry with Backoff**
|
||||
- **Validates: Requirements 4.7**
|
||||
|
||||
- [x] 8. Implement content source API routes
|
||||
- [x] 8.1 Create content source API routes
|
||||
- POST /api/bots/[id]/sources - Add source
|
||||
- GET /api/bots/[id]/sources - List sources
|
||||
- PUT /api/bots/[id]/sources/[sid] - Update source
|
||||
- DELETE /api/bots/[id]/sources/[sid] - Remove source
|
||||
- POST /api/bots/[id]/sources/[sid]/fetch - Manual fetch
|
||||
- _Requirements: 4.1, 4.6_
|
||||
|
||||
- [x] 9. Checkpoint - Ensure all tests pass
|
||||
- Ensure all tests pass, ask the user if questions arise.
|
||||
|
||||
- [x] 10. Implement Rate Limiter
|
||||
- [x] 10.1 Create rate limiter service in src/lib/bots/rateLimiter.ts
|
||||
- Implement canPost check (50/day, 5min interval)
|
||||
- Implement canReply check (20/hour)
|
||||
- Implement recordPost and recordReply
|
||||
- Implement getRemainingQuota
|
||||
- _Requirements: 5.6, 7.6, 10.1, 10.2, 10.4_
|
||||
|
||||
- [x] 10.2 Write property test for rate limit enforcement
|
||||
- **Property 19: Rate Limit Enforcement**
|
||||
- **Validates: Requirements 5.6, 10.1, 10.2, 10.4**
|
||||
|
||||
- [x] 10.3 Write property test for reply rate limiting
|
||||
- **Property 25: Reply Rate Limiting**
|
||||
- **Validates: Requirements 7.6**
|
||||
|
||||
- [x] 11. Implement Post Scheduler
|
||||
- [x] 11.1 Create scheduler service in src/lib/bots/scheduler.ts
|
||||
- Implement schedule configuration storage
|
||||
- Support interval, time-of-day, and cron-like schedules
|
||||
- Implement isDue check for schedules
|
||||
- Implement processScheduledPosts
|
||||
- Skip posting when no content available
|
||||
- _Requirements: 5.1, 5.2, 5.3, 5.4, 5.5_
|
||||
|
||||
- [x] 11.2 Write property test for schedule configuration persistence
|
||||
- **Property 16: Schedule Configuration Persistence**
|
||||
- **Validates: Requirements 5.1, 5.3**
|
||||
|
||||
- [x] 11.3 Write property test for scheduled post triggering
|
||||
- **Property 17: Scheduled Post Triggering**
|
||||
- **Validates: Requirements 5.2, 5.4**
|
||||
|
||||
- [x] 11.4 Write property test for skip when no content
|
||||
- **Property 18: Skip When No Content**
|
||||
- **Validates: Requirements 5.5**
|
||||
|
||||
- [x] 12. Implement Content Generator (LLM integration)
|
||||
- [x] 12.1 Create LLM client in src/lib/bots/llmClient.ts
|
||||
- Implement OpenRouter API client
|
||||
- Implement OpenAI API client
|
||||
- Implement Anthropic API client
|
||||
- Add retry logic (3 retries)
|
||||
- _Requirements: 2.6, 11.4_
|
||||
|
||||
- [x] 12.2 Create content generator in src/lib/bots/contentGenerator.ts
|
||||
- Implement generatePost with personality context
|
||||
- Implement generateReply with conversation context
|
||||
- Implement evaluateContentInterest for autonomous mode
|
||||
- Implement content truncation for long sources
|
||||
- _Requirements: 3.2, 3.5, 6.2, 11.1, 11.2, 11.3_
|
||||
|
||||
- [x] 12.3 Write property test for personality in LLM prompts
|
||||
- **Property 10: Personality in LLM Prompts**
|
||||
- **Validates: Requirements 3.2, 3.5**
|
||||
|
||||
- [x] 12.4 Write property test for LLM prompt construction
|
||||
- **Property 35: LLM Prompt Construction**
|
||||
- **Validates: Requirements 11.1, 11.2**
|
||||
|
||||
- [x] 12.5 Write property test for content truncation
|
||||
- **Property 36: Content Truncation**
|
||||
- **Validates: Requirements 11.3**
|
||||
|
||||
- [x] 12.6 Write property test for LLM retry logic
|
||||
- **Property 37: LLM Retry Logic**
|
||||
- **Validates: Requirements 11.4**
|
||||
|
||||
- [x] 13. Checkpoint - Ensure all tests pass
|
||||
- Ensure all tests pass, ask the user if questions arise.
|
||||
|
||||
- [x] 14. Implement autonomous posting
|
||||
- [x] 14.1 Create autonomous posting logic in src/lib/bots/autonomous.ts
|
||||
- Implement content evaluation flow
|
||||
- Implement autonomous post decision logic
|
||||
- Respect rate limits
|
||||
- Support autonomous mode toggle
|
||||
- _Requirements: 6.1, 6.2, 6.3, 6.5_
|
||||
|
||||
- [x] 14.2 Write property test for autonomous mode content evaluation
|
||||
- **Property 20: Autonomous Mode Content Evaluation**
|
||||
- **Validates: Requirements 6.1, 6.2, 6.3**
|
||||
|
||||
- [x] 14.3 Write property test for autonomous mode toggle
|
||||
- **Property 21: Autonomous Mode Toggle**
|
||||
- **Validates: Requirements 6.5**
|
||||
|
||||
- [x] 15. Implement bot posting flow
|
||||
- [x] 15.1 Create post creation logic in src/lib/bots/posting.ts
|
||||
- Implement triggerPost function
|
||||
- Select content from sources
|
||||
- Generate post via LLM
|
||||
- Validate post content
|
||||
- Create post in database
|
||||
- Integrate with rate limiter
|
||||
- _Requirements: 5.4, 11.5, 11.6_
|
||||
|
||||
- [x] 15.2 Create bot operations API routes
|
||||
- POST /api/bots/[id]/post - Manual post trigger
|
||||
- _Requirements: 5.4_
|
||||
|
||||
- [x] 15.3 Write property test for post content validation
|
||||
- **Property 38: Post Content Validation**
|
||||
- **Validates: Requirements 11.5**
|
||||
|
||||
- [x] 16. Implement Mention Handler
|
||||
- [x] 16.1 Create mention detection in src/lib/bots/mentionHandler.ts
|
||||
- Implement mention detection from posts
|
||||
- Store detected mentions
|
||||
- Retrieve mentioning post content
|
||||
- _Requirements: 7.1, 7.2_
|
||||
|
||||
- [x] 16.2 Implement mention response generation
|
||||
- Generate response with conversation context
|
||||
- Include original post in LLM prompt
|
||||
- Process mentions in chronological order
|
||||
- Respect reply rate limits
|
||||
- _Requirements: 7.3, 7.4, 7.5, 7.6_
|
||||
|
||||
- [x] 16.3 Create mention API routes
|
||||
- GET /api/bots/[id]/mentions - Get pending mentions
|
||||
- POST /api/bots/[id]/mentions/[mid]/respond - Manual respond
|
||||
- _Requirements: 7.1_
|
||||
|
||||
- [x] 16.4 Write property test for mention detection
|
||||
- **Property 22: Mention Detection**
|
||||
- **Validates: Requirements 7.1, 7.2**
|
||||
|
||||
- [x] 16.5 Write property test for mention response context
|
||||
- **Property 23: Mention Response Context**
|
||||
- **Validates: Requirements 7.3, 7.4**
|
||||
|
||||
- [x] 16.6 Write property test for mention chronological processing
|
||||
- **Property 24: Mention Chronological Processing**
|
||||
- **Validates: Requirements 7.5**
|
||||
|
||||
- [ ] 17. Checkpoint - Ensure all tests pass
|
||||
- Ensure all tests pass, ask the user if questions arise.
|
||||
|
||||
- [-] 18. Implement Activity Logger
|
||||
- [x] 18.1 Create activity logger in src/lib/bots/activityLogger.ts
|
||||
- Implement log function for all action types
|
||||
- Record action type, timestamp, result, error message
|
||||
- Implement getLogsForBot with filtering
|
||||
- Implement reverse chronological ordering
|
||||
- _Requirements: 8.1, 8.2, 8.3, 8.4, 8.6_
|
||||
|
||||
- [x] 18.2 Create activity log API routes
|
||||
- GET /api/bots/[id]/logs - Get logs with filters
|
||||
- GET /api/bots/[id]/logs/errors - Get error logs
|
||||
- _Requirements: 8.2, 8.6_
|
||||
|
||||
- [ ] 18.3 Write property test for activity log completeness
|
||||
- **Property 26: Activity Log Completeness**
|
||||
- **Validates: Requirements 8.1, 8.3, 8.4**
|
||||
|
||||
- [ ] 18.4 Write property test for activity log ordering
|
||||
- **Property 27: Activity Log Ordering**
|
||||
- **Validates: Requirements 8.2**
|
||||
|
||||
- [ ] 18.5 Write property test for activity log filtering
|
||||
- **Property 28: Activity Log Filtering**
|
||||
- **Validates: Requirements 8.6**
|
||||
|
||||
- [-] 19. Implement bot suspension
|
||||
- [x] 19.1 Create suspension logic in src/lib/bots/suspension.ts
|
||||
- Implement suspendBot function
|
||||
- Implement reinstateBot function
|
||||
- Block all actions for suspended bots
|
||||
- _Requirements: 10.6_
|
||||
|
||||
- [x] 19.2 Create admin suspension API routes
|
||||
- POST /api/bots/[id]/suspend - Suspend bot
|
||||
- POST /api/bots/[id]/reinstate - Reinstate bot
|
||||
- _Requirements: 10.6_
|
||||
|
||||
- [ ] 19.3 Write property test for bot suspension enforcement
|
||||
- **Property 34: Bot Suspension Enforcement**
|
||||
- **Validates: Requirements 10.6**
|
||||
|
||||
- [-] 20. Implement input sanitization
|
||||
- [x] 20.1 Create input sanitization in src/lib/bots/sanitization.ts
|
||||
- Implement SQL injection prevention
|
||||
- Implement XSS prevention
|
||||
- Implement command injection prevention
|
||||
- Apply to all user inputs
|
||||
- _Requirements: 10.5_
|
||||
|
||||
- [ ] 20.2 Write property test for input sanitization
|
||||
- **Property 33: Input Sanitization**
|
||||
- **Validates: Requirements 10.5**
|
||||
|
||||
- [ ] 21. Checkpoint - Ensure all tests pass
|
||||
- Ensure all tests pass, ask the user if questions arise.
|
||||
|
||||
- [-] 22. Implement ActivityPub federation for bots
|
||||
- [-] 22.1 Update ActivityPub actor for bots in src/lib/activitypub/actor.ts
|
||||
- Add bot type detection
|
||||
- Include bot flag in actor object (type: "Service" or bot property)
|
||||
- Include bot creator reference
|
||||
- _Requirements: 9.3, 12.3, 12.4_
|
||||
|
||||
- [x] 22.2 Implement bot post federation
|
||||
- Generate Create activities for bot posts
|
||||
- Deliver to followers
|
||||
- _Requirements: 9.1_
|
||||
|
||||
- [x] 22.3 Implement federated mention handling
|
||||
- Detect mentions from remote instances
|
||||
- Process remote mentions like local ones
|
||||
- _Requirements: 9.2_
|
||||
|
||||
- [x] 22.4 Implement bot follow handling
|
||||
- Handle Follow activities for bots
|
||||
- Generate Accept/Reject activities
|
||||
- _Requirements: 9.5_
|
||||
|
||||
- [ ] 22.5 Write property test for federation post distribution
|
||||
- **Property 29: Federation Post Distribution**
|
||||
- **Validates: Requirements 9.1**
|
||||
|
||||
- [ ] 22.6 Write property test for federated mention handling
|
||||
- **Property 30: Federated Mention Handling**
|
||||
- **Validates: Requirements 9.2**
|
||||
|
||||
- [ ] 22.7 Write property test for ActivityPub bot flag
|
||||
- **Property 31: ActivityPub Bot Flag**
|
||||
- **Validates: Requirements 9.3**
|
||||
|
||||
- [ ] 22.8 Write property test for bot follow handling
|
||||
- **Property 32: Bot Follow Handling**
|
||||
- **Validates: Requirements 9.5**
|
||||
|
||||
- [-] 23. Implement bot identification in UI
|
||||
- [x] 23.1 Update user/bot display components
|
||||
- Add bot badge to profile display
|
||||
- Show bot creator on bot profiles
|
||||
- Ensure bot flag cannot be hidden
|
||||
- _Requirements: 12.1, 12.2, 12.3, 12.4_
|
||||
|
||||
- [ ] 23.2 Write property test for bot identification immutability
|
||||
- **Property 4: Bot Identification Immutability**
|
||||
- **Validates: Requirements 1.5, 12.1, 12.2, 12.4**
|
||||
|
||||
- [ ] 23.3 Write property test for bot creator attribution
|
||||
- **Property 39: Bot Creator Attribution**
|
||||
- **Validates: Requirements 12.3**
|
||||
|
||||
- [-] 24. Create Bot Management UI
|
||||
- [x] 24.1 Create bot list page at src/app/settings/bots/page.tsx
|
||||
- Display user's bots
|
||||
- Show bot status, last post time
|
||||
- Add create bot button
|
||||
- _Requirements: 1.3_
|
||||
|
||||
- [x] 24.2 Create bot creation form at src/app/settings/bots/new/page.tsx
|
||||
- Bot name, handle, bio, avatar
|
||||
- Personality configuration
|
||||
- LLM provider and model selection
|
||||
- API key input
|
||||
- _Requirements: 1.1, 2.1, 3.1_
|
||||
|
||||
- [x] 24.3 Create bot detail/edit page at src/app/settings/bots/[id]/page.tsx
|
||||
- Edit bot configuration
|
||||
- Manage content sources
|
||||
- View activity logs
|
||||
- Manual post/respond actions
|
||||
- _Requirements: 1.3, 4.6, 8.2_
|
||||
|
||||
- [ ] 25. Final checkpoint - Ensure all tests pass
|
||||
- Ensure all tests pass, ask the user if questions arise.
|
||||
|
||||
## Notes
|
||||
|
||||
- All tasks are required including property-based tests
|
||||
- Each task references specific requirements for traceability
|
||||
- Checkpoints ensure incremental validation
|
||||
- Property tests validate universal correctness properties
|
||||
- Unit tests validate specific examples and edge cases
|
||||
- The implementation follows existing Synapsis patterns for consistency
|
||||
Vendored
-3
@@ -1,3 +0,0 @@
|
||||
{
|
||||
"typescript.autoClosingTags": false
|
||||
}
|
||||
Reference in New Issue
Block a user