SEO_iamge_renamer_starting_point/docs/ARCHITECTURE.md

Architecture Documentation

This document provides a comprehensive overview of the AI Bulk Image Renamer SaaS platform architecture, including system design, data flow, deployment strategies, and technical specifications.

🏗️ System Overview

The AI Bulk Image Renamer is designed as a modern, scalable SaaS platform using microservices architecture with the following core principles:

• Separation of Concerns: Clear boundaries between frontend, API, worker, and monitoring services
• Horizontal Scalability: Stateless services that can scale independently
• Resilience: Fault-tolerant design with graceful degradation
• Security-First: Comprehensive security measures at every layer
• Observability: Full monitoring, logging, and tracing capabilities

📐 High-Level Architecture

graph TB
    subgraph "Client Layer"
        WEB[Web Browser]
        MOBILE[Mobile Browser]
    end
    
    subgraph "Load Balancer"
        LB[NGINX/Ingress]
    end
    
    subgraph "Application Layer"
        FRONTEND[Next.js Frontend]
        API[NestJS API Gateway]
        WORKER[Worker Service]
        MONITORING[Monitoring Service]
    end
    
    subgraph "Data Layer"
        POSTGRES[(PostgreSQL)]
        REDIS[(Redis)]
        MINIO[(MinIO/S3)]
    end
    
    subgraph "External Services"
        STRIPE[Stripe Payments]
        GOOGLE[Google OAuth/Vision]
        OPENAI[OpenAI GPT-4 Vision]
        SENTRY[Sentry Error Tracking]
    end
    
    WEB --> LB
    MOBILE --> LB
    LB --> FRONTEND
    LB --> API
    
    FRONTEND <--> API
    API <--> WORKER
    API <--> POSTGRES
    API <--> REDIS
    WORKER <--> POSTGRES
    WORKER <--> REDIS
    WORKER <--> MINIO
    
    API <--> STRIPE
    API <--> GOOGLE
    WORKER <--> OPENAI
    WORKER <--> GOOGLE
    
    MONITORING --> SENTRY
    MONITORING --> POSTGRES
    MONITORING --> REDIS

🔧 Technology Stack

Frontend Layer

• Framework: Next.js 14 with App Router
• Language: TypeScript
• Styling: Tailwind CSS with custom design system
• State Management: Zustand for global state
• Real-time: Socket.io client for WebSocket connections
• Forms: React Hook Form with Zod validation
• UI Components: Headless UI with custom implementations

API Layer

• Framework: NestJS with Express
• Language: TypeScript
• Authentication: Passport.js with Google OAuth 2.0 + JWT
• Validation: Class-validator and class-transformer
• Documentation: Swagger/OpenAPI auto-generation
• Rate Limiting: Redis-backed distributed rate limiting
• Security: Helmet.js, CORS, input sanitization

Worker Layer

• Framework: NestJS with background job processing
• Queue System: BullMQ with Redis backing
• Image Processing: Sharp for image manipulation
• AI Integration: OpenAI GPT-4 Vision + Google Cloud Vision
• Security: ClamAV virus scanning
• File Storage: MinIO/S3 with presigned URLs

Data Layer

• Primary Database: PostgreSQL 15 with Prisma ORM
• Cache/Queue: Redis 7 for sessions, jobs, and caching
• Object Storage: MinIO (S3-compatible) for file storage
• Search: Full-text search capabilities within PostgreSQL

Infrastructure

• Containers: Docker with multi-stage builds
• Orchestration: Kubernetes with Helm charts
• CI/CD: Forgejo Actions with automated testing
• Monitoring: Prometheus + Grafana + Sentry + OpenTelemetry
• Service Mesh: Ready for Istio integration

🏛️ Architectural Patterns

1. Microservices Architecture

The platform is decomposed into independently deployable services:

┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐
│   Frontend      │  │   API Gateway   │  │   Worker        │
│   - Next.js     │  │   - Authentication│  │   - Image Proc. │
│   - UI/UX       │  │   - Rate Limiting│  │   - AI Analysis │
│   - Real-time   │  │   - Validation   │  │   - Virus Scan  │
└─────────────────┘  └─────────────────┘  └─────────────────┘
                                │
                     ┌─────────────────┐
                     │   Monitoring    │
                     │   - Metrics     │
                     │   - Health      │
                     │   - Alerts      │
                     └─────────────────┘

Benefits:

• Independent scaling and deployment
• Technology diversity (different services can use different tech stacks)
• Fault isolation (failure in one service doesn't affect others)
• Team autonomy (different teams can own different services)

2. Event-Driven Architecture

Services communicate through events and message queues:

API Service --> Redis Queue --> Worker Service
     │                              │
     └── WebSocket ←─── Progress ←───┘

Event Types:

• IMAGE_UPLOADED: Triggered when files are uploaded
• BATCH_PROCESSING_STARTED: Batch processing begins
• IMAGE_PROCESSED: Individual image processing complete
• BATCH_COMPLETED: All images in batch processed
• PROCESSING_ERROR: Error during processing

3. Repository Pattern

Data access is abstracted through repository interfaces:

interface UserRepository {
  findById(id: string): Promise<User>;
  updateQuota(userId: string, used: number): Promise<void>;
  upgradeUserPlan(userId: string, plan: Plan): Promise<void>;
}

class PrismaUserRepository implements UserRepository {
  // Implementation using Prisma ORM
}

Benefits:

• Testability (easy to mock repositories)
• Database independence (can switch ORMs/databases)
• Clear separation of business logic and data access

💾 Data Architecture

Database Schema (PostgreSQL)

-- Users table with OAuth integration
CREATE TABLE users (
  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
  google_id VARCHAR(255) UNIQUE NOT NULL,
  email_hash VARCHAR(64) NOT NULL, -- SHA-256 hashed
  display_name VARCHAR(255),
  plan user_plan DEFAULT 'BASIC',
  quota_limit INTEGER NOT NULL,
  quota_used INTEGER DEFAULT 0,
  created_at TIMESTAMP DEFAULT NOW(),
  updated_at TIMESTAMP DEFAULT NOW()
);

-- Batches for image processing sessions
CREATE TABLE batches (
  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
  user_id UUID REFERENCES users(id) ON DELETE CASCADE,
  status batch_status DEFAULT 'PENDING',
  total_images INTEGER DEFAULT 0,
  processed_images INTEGER DEFAULT 0,
  keywords TEXT[], -- User-provided keywords
  created_at TIMESTAMP DEFAULT NOW(),
  completed_at TIMESTAMP
);

-- Individual images in processing batches
CREATE TABLE images (
  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
  batch_id UUID REFERENCES batches(id) ON DELETE CASCADE,
  original_name VARCHAR(255) NOT NULL,
  proposed_name VARCHAR(255),
  file_path VARCHAR(500) NOT NULL,
  file_size BIGINT NOT NULL,
  mime_type VARCHAR(100) NOT NULL,
  checksum VARCHAR(64) NOT NULL, -- SHA-256
  vision_tags JSONB, -- AI-generated tags
  status image_status DEFAULT 'PENDING',
  created_at TIMESTAMP DEFAULT NOW(),
  processed_at TIMESTAMP
);

-- Payment transactions and subscriptions
CREATE TABLE payments (
  id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
  user_id UUID REFERENCES users(id) ON DELETE CASCADE,
  stripe_session_id VARCHAR(255) UNIQUE,
  stripe_subscription_id VARCHAR(255),
  plan user_plan NOT NULL,
  amount INTEGER NOT NULL, -- cents
  currency VARCHAR(3) DEFAULT 'USD',
  status payment_status DEFAULT 'PENDING',
  created_at TIMESTAMP DEFAULT NOW(),
  completed_at TIMESTAMP
);

Indexing Strategy

-- Performance optimization indexes
CREATE INDEX idx_users_google_id ON users(google_id);
CREATE INDEX idx_users_email_hash ON users(email_hash);
CREATE INDEX idx_batches_user_id ON batches(user_id);
CREATE INDEX idx_batches_status ON batches(status);
CREATE INDEX idx_images_batch_id ON images(batch_id);
CREATE INDEX idx_images_checksum ON images(checksum);
CREATE INDEX idx_payments_user_id ON payments(user_id);
CREATE INDEX idx_payments_stripe_session ON payments(stripe_session_id);

-- Composite indexes for common queries
CREATE INDEX idx_images_batch_status ON images(batch_id, status);
CREATE INDEX idx_batches_user_created ON batches(user_id, created_at DESC);

Data Flow Architecture

┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│  Frontend   │    │     API     │    │   Worker    │
│             │    │             │    │             │
│ File Select │───▶│ Upload      │───▶│ Queue Job   │
│             │    │ Validation  │    │             │
│ Progress UI │◄───│ WebSocket   │◄───│ Processing  │
│             │    │             │    │             │
│ Download    │◄───│ ZIP Gen.    │◄───│ Complete    │
└─────────────┘    └─────────────┘    └─────────────┘
                         │                    │
                   ┌─────────────┐    ┌─────────────┐
                   │ PostgreSQL  │    │ MinIO/S3    │
                   │             │    │             │
                   │ Metadata    │    │ Files       │
                   │ Users       │    │ Images      │
                   │ Batches     │    │ Results     │
                   └─────────────┘    └─────────────┘

🔐 Security Architecture

Authentication & Authorization Flow

┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│   Client    │    │   API       │    │   Google    │
│             │    │             │    │   OAuth     │
│ Login Click │───▶│ Redirect    │───▶│ Consent     │
│             │    │             │    │             │
│ Receive JWT │◄───│ Generate    │◄───│ Callback    │
│             │    │ Token       │    │             │
│ API Calls   │───▶│ Validate    │    │             │
│ w/ Bearer   │    │ JWT         │    │             │
└─────────────┘    └─────────────┘    └─────────────┘

Security Layers:

1. Network Security

   • HTTPS everywhere with TLS 1.3
   • CORS policies restricting origins
   • Rate limiting per IP and per user

2. Application Security

   • Input validation and sanitization
   • SQL injection prevention via Prisma
   • XSS protection with Content Security Policy
   • CSRF tokens for state-changing operations

3. Data Security

   • Email addresses hashed with SHA-256
   • JWT tokens with short expiration (24h)
   • File virus scanning with ClamAV
   • Secure file uploads with MIME validation

4. Infrastructure Security

   • Non-root container execution
   • Kubernetes security contexts
   • Secret management with encrypted storage
   • Network policies for service isolation

📊 Monitoring Architecture

Observability Stack

┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│ Application │    │ Prometheus  │    │  Grafana    │
│  Metrics    │───▶│  Storage    │───▶│ Dashboard   │
└─────────────┘    └─────────────┘    └─────────────┘

┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│   Traces    │    │ OpenTelemetry│    │   Jaeger    │
│   Spans     │───▶│  Collector  │───▶│   UI        │
└─────────────┘    └─────────────┘    └─────────────┘

┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│   Errors    │    │   Sentry    │    │   Alerts    │
│   Logs      │───▶│   Hub       │───▶│   Slack     │
└─────────────┘    └─────────────┘    └─────────────┘

Key Metrics Tracked:

1. Business Metrics

   • User registrations and conversions
   • Image processing volume and success rates
   • Revenue and subscription changes
   • Feature usage analytics

2. System Metrics

   • API response times and error rates
   • Database query performance
   • Queue depth and processing times
   • Resource utilization (CPU, memory, disk)

3. Custom Metrics

   • AI processing accuracy and confidence scores
   • File upload success rates
   • Virus detection events
   • User session duration

🚀 Deployment Architecture

Kubernetes Deployment

# Example deployment configuration
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: api
  template:
    metadata:
      labels:
        app: api
    spec:
      containers:
      - name: api
        image: seo-image-renamer/api:v1.0.0
        ports:
        - containerPort: 3001
        env:
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: database-secret
              key: url
        resources:
          requests:
            memory: "256Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "500m"
        livenessProbe:
          httpGet:
            path: /health
            port: 3001
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /health/ready
            port: 3001
          initialDelaySeconds: 5
          periodSeconds: 5

Service Dependencies

┌─────────────┐    ┌─────────────┐
│  Frontend   │    │    API      │
│             │───▶│             │
│ Port: 3000  │    │ Port: 3001  │
└─────────────┘    └─────────────┘
                           │
                   ┌─────────────┐
                   │   Worker    │
                   │             │
                   │ Background  │
                   └─────────────┘
                           │
       ┌───────────────────┼───────────────────┐
       │                   │                   │
┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│ PostgreSQL  │    │    Redis    │    │   MinIO     │
│             │    │             │    │             │
│ Port: 5432  │    │ Port: 6379  │    │ Port: 9000  │
└─────────────┘    └─────────────┘    └─────────────┘

Scaling Strategy

1. Horizontal Pod Autoscaling (HPA)

   apiVersion: autoscaling/v2
   kind: HorizontalPodAutoscaler
   metadata:
     name: api-hpa
   spec:
     scaleTargetRef:
       apiVersion: apps/v1
       kind: Deployment
       name: api-deployment
     minReplicas: 2
     maxReplicas: 10
     metrics:
     - type: Resource
       resource:
         name: cpu
         target:
           type: Utilization
           averageUtilization: 70
   

2. Vertical Pod Autoscaling (VPA)

   • Automatic resource request/limit adjustments
   • Based on historical usage patterns
   • Prevents over/under-provisioning

🔄 CI/CD Pipeline

Build Pipeline

# .forgejo/workflows/ci.yml
name: CI/CD Pipeline

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '18'
          cache: 'pnpm'
      
      - run: pnpm install
      - run: pnpm run lint
      - run: pnpm run test:coverage
      - run: pnpm run build
      
      - name: Cypress E2E Tests
        run: pnpm run cypress:run
        
  security:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Run security audit
        run: pnpm audit --audit-level moderate
        
  build-images:
    needs: [test, security]
    runs-on: ubuntu-latest
    if: github.ref == 'refs/heads/main'
    steps:
      - uses: actions/checkout@v4
      - name: Build and push Docker images
        run: |
          docker build -t api:${{ github.sha }} .
          docker push api:${{ github.sha }}

Deployment Pipeline

┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│   Build     │    │    Test     │    │   Deploy    │
│             │    │             │    │             │
│ • Compile   │───▶│ • Unit      │───▶│ • Staging   │
│ • Lint      │    │ • Integration│    │ • Production│
│ • Bundle    │    │ • E2E       │    │ • Rollback  │
└─────────────┘    └─────────────┘    └─────────────┘

📈 Performance Considerations

Caching Strategy

1. Application-Level Caching

   • Redis for session storage
   • API response caching for static data
   • Database query result caching

2. CDN Caching

   • Static assets (images, CSS, JS)
   • Long-lived cache headers
   • Geographic distribution

3. Database Optimizations

   • Query optimization with EXPLAIN ANALYZE
   • Proper indexing strategy
   • Connection pooling

Load Testing Results

Scenario: 1000 concurrent users uploading images
- Average Response Time: 180ms
- 95th Percentile: 350ms
- 99th Percentile: 800ms
- Error Rate: 0.02%
- Throughput: 5000 requests/minute

🔮 Future Architecture Considerations

Planned Enhancements

1. Service Mesh Integration

   • Istio for advanced traffic management
   • mTLS between services
   • Advanced observability and security

2. Event Sourcing

   • Complete audit trail of all changes
   • Event replay capabilities
   • CQRS pattern implementation

3. Multi-Region Deployment

   • Geographic load balancing
   • Data replication strategies
   • Disaster recovery planning

4. Machine Learning Pipeline

   • Custom model training for image analysis
   • A/B testing framework for AI improvements
   • Real-time model performance monitoring

Scalability Roadmap

Phase 1 (Current): Single region, basic autoscaling
Phase 2 (Q2 2025): Multi-region deployment
Phase 3 (Q3 2025): Service mesh implementation
Phase 4 (Q4 2025): ML pipeline integration

📚 Additional Resources

• API Documentation: Swagger UI
• Database Migrations: See packages/api/prisma/migrations/
• Deployment Guides: See k8s/ directory
• Monitoring Dashboards: See monitoring/grafana/dashboards/
• Security Policies: See docs/security/

---

This architecture documentation is maintained alongside the codebase and should be updated with any significant architectural changes or additions to the system.