SEO_iamge_renamer_starting_.../packages/worker/.env.example

79 lines
1.8 KiB
Text
Raw Permalink Normal View History

feat(worker): complete production-ready worker service implementation This commit delivers the complete, production-ready worker service that was identified as missing from the audit. The implementation includes: ## Core Components Implemented: ### 1. Background Job Queue System ✅ - Progress tracking with Redis and WebSocket broadcasting - Intelligent retry handler with exponential backoff strategies - Automated cleanup service with scheduled maintenance - Queue-specific retry policies and failure handling ### 2. Security Integration ✅ - Complete ClamAV virus scanning service with real-time threats detection - File validation and quarantine system - Security incident logging and user flagging - Comprehensive threat signature management ### 3. Database Integration ✅ - Prisma-based database service with connection pooling - Image status tracking and batch management - Security incident recording and user flagging - Health checks and statistics collection ### 4. Monitoring & Observability ✅ - Prometheus metrics collection for all operations - Custom business metrics and performance tracking - Comprehensive health check endpoints (ready/live/detailed) - Resource usage monitoring and alerting ### 5. Production Docker Configuration ✅ - Multi-stage Docker build with Alpine Linux - ClamAV daemon integration and configuration - Security-hardened container with non-root user - Health checks and proper signal handling - Complete docker-compose setup with Redis, MinIO, Prometheus, Grafana ### 6. Configuration & Environment ✅ - Comprehensive environment validation with Joi - Redis integration for progress tracking and caching - Rate limiting and throttling configuration - Logging configuration with Winston and file rotation ## Technical Specifications Met: ✅ **Real AI Integration**: OpenAI GPT-4 Vision + Google Cloud Vision with fallbacks ✅ **Image Processing Pipeline**: Sharp integration with EXIF preservation ✅ **Storage Integration**: MinIO/S3 with temporary file management ✅ **Queue Processing**: BullMQ with Redis, retry logic, and progress tracking ✅ **Security Features**: ClamAV virus scanning with quarantine system ✅ **Monitoring**: Prometheus metrics, health checks, structured logging ✅ **Production Ready**: Docker, Kubernetes compatibility, environment validation ## Integration Points: - Connects with existing API queue system - Uses shared database models and authentication - Integrates with infrastructure components - Provides real-time progress updates via WebSocket This resolves the critical gap identified in the audit and provides a complete, production-ready worker service capable of processing images with real AI vision analysis at scale. 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
2025-08-05 18:37:04 +02:00
# SEO Image Renamer Worker Service - Environment Configuration
# Application Settings
NODE_ENV=development
WORKER_PORT=3002
HEALTH_CHECK_PORT=8080
# Redis Configuration
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_PASSWORD=your_redis_password
REDIS_DB=0
REDIS_URL=redis://localhost:6379
# Database Configuration
DATABASE_URL=postgresql://user:password@localhost:5432/seo_renamer
# AI Vision APIs (at least one is required)
OPENAI_API_KEY=your_openai_api_key
OPENAI_MODEL=gpt-4-vision-preview
OPENAI_MAX_TOKENS=500
OPENAI_TEMPERATURE=0.1
OPENAI_REQUESTS_PER_MINUTE=50
OPENAI_TOKENS_PER_MINUTE=10000
GOOGLE_CLOUD_VISION_KEY=path/to/google-service-account.json
GOOGLE_CLOUD_PROJECT_ID=your_project_id
GOOGLE_CLOUD_LOCATION=global
GOOGLE_REQUESTS_PER_MINUTE=100
VISION_CONFIDENCE_THRESHOLD=0.40
# Storage Configuration (MinIO or AWS S3)
# MinIO Configuration
MINIO_ENDPOINT=localhost
MINIO_PORT=9000
MINIO_USE_SSL=false
MINIO_ACCESS_KEY=minioadmin
MINIO_SECRET_KEY=minioadmin
MINIO_BUCKET_NAME=seo-images
# AWS S3 Configuration (alternative to MinIO)
# AWS_REGION=us-east-1
# AWS_ACCESS_KEY_ID=your_aws_access_key
# AWS_SECRET_ACCESS_KEY=your_aws_secret_key
# AWS_BUCKET_NAME=your_bucket_name
# Processing Configuration
MAX_CONCURRENT_JOBS=5
JOB_TIMEOUT=300000
RETRY_ATTEMPTS=3
RETRY_DELAY=2000
# File Processing
MAX_FILE_SIZE=52428800
ALLOWED_FILE_TYPES=jpg,jpeg,png,gif,webp
TEMP_DIR=/tmp/seo-worker
TEMP_FILE_CLEANUP_INTERVAL=3600000
# Virus Scanning (optional)
VIRUS_SCAN_ENABLED=false
CLAMAV_HOST=localhost
CLAMAV_PORT=3310
CLAMAV_TIMEOUT=30000
# Monitoring
METRICS_ENABLED=true
METRICS_PORT=9090
LOG_LEVEL=info
FILE_LOGGING_ENABLED=false
LOG_DIR=./logs
# Rate Limiting for AI APIs
OPENAI_REQUESTS_PER_MINUTE=50
OPENAI_TOKENS_PER_MINUTE=10000
GOOGLE_REQUESTS_PER_MINUTE=100
# Optional: Grafana
GRAFANA_PASSWORD=admin