video-edit/PROJECT_CONTEXT.md

📘 Project Context for AI Assistants

This document provides technical context for AI assistants working on this video editor project. It covers architecture decisions, implementation patterns, and key code locations.

---

🏗️ Architecture Overview

Tech Stack

• Backend: Flask 3.0 (Python 3.11+)
• Video Engine: FFmpeg with libx264 encoder
• Frontend: Vanilla JavaScript (no frameworks)
• Deployment: Python local / Docker / Docker Compose

Request Flow

1. User uploads video → /upload endpoint
2. Server assigns UUID, stores in uploads/ dir
3. User edits via UI → sends params to /process
4. FFmpeg processes with filters → outputs/ dir
5. User downloads → /download/<file_id> endpoint
6. Cleanup → /cleanup/<file_id> removes temp files

File Structure

c:\Projects\editor/
├── app.py                      # Flask backend, FFmpeg integration
├── templates/
│   └── index.html              # Single-page frontend app
├── requirements.txt            # Python dependencies
├── uploads/                    # Temporary uploaded files (created on first run)
├── outputs/                    # Processed video outputs (created on first run)
├── Dockerfile                  # Container image definition
├── docker-compose.yml          # Local development compose
├── prod-compose.yml            # Production registry-pull compose
├── .gitea/
│   └── workflows/
│       └── rebuild-prod.yaml   # CI/CD build and deployment
├── README.md                   # User documentation
└── PROJECT_CONTEXT.md          # This file (technical context)

---

🔑 Key Implementation Details

1. Video Processing (app.py)

FFmpeg Filter Chain

The app builds a complex FFmpeg filter chain dynamically based on user selections:

# Base filters array
filters = []

# Rotation (if not 0°)
if rotate and rotate != '0':
    # transpose values: 1=90°CW, 2=90°CCW, 3=180° (using 2,2)
    if rotate == '90':
        filters.append('transpose=1')
    elif rotate == '180':
        filters.append('transpose=2,transpose=2')
    elif rotate == '270':
        filters.append('transpose=2')

# Crop (if specified)
if crop_w and crop_h:
    filters.append(f'crop={crop_w}:{crop_h}:{crop_x}:{crop_y}')

# Scale with even dimension enforcement
if scale_percentage < 100:
    # trunc(value/2)*2 ensures dimensions are divisible by 2 (H.264 requirement)
    filters.append(f'scale=trunc(iw*{scale_percentage/100}/2)*2:trunc(ih*{scale_percentage/100}/2)*2')

# Final filter string
filter_str = ','.join(filters)

Critical: The trunc(iw*percentage/100/2)*2 pattern ensures output dimensions are always even numbers. H.264 encoder requires width and height divisible by 2.

File ID System

• UUID4 generates unique IDs for each upload
• original_filenames dict maps UUID → original filename
• Downloads use: {original_name}_processed_{timestamp}.mp4
• Timestamp format: YYYYMMDD_HHMMSS (e.g., 20240315_143022)

CRF Quality Control

• CRF (Constant Rate Factor) range: 18-32
• Lower CRF = higher quality, larger file
• Default: 23 (balanced)
• Passed directly to FFmpeg: -crf {quality}

2. Visual Crop Selection (index.html)

Canvas Overlay Technique

// Create canvas overlay matching video display size
cropCanvas.width = video.offsetWidth;
cropCanvas.height = video.offsetHeight;

// Calculate scale factors for actual video pixels
const scaleX = video.videoWidth / video.offsetWidth;
const scaleY = video.videoHeight / video.offsetHeight;

// Mouse drag draws rectangle on canvas
// Convert canvas coordinates → actual video pixels
const actualX = Math.round(rectX * scaleX);
const actualY = Math.round(rectY * scaleY);
const actualW = Math.round(rectW * scaleX);
const actualH = Math.round(rectH * scaleY);

Why this works: Video element display size ≠ actual video resolution. Scale factors convert screen pixels to video pixels for accurate crop commands.

Dynamic Button Creation

The crop button is dynamically injected into video-info after video loads:

function setupCropButton(video, canvas) {
    const videoInfoDiv = document.getElementById('video-info');
    
    // Create button HTML
    const cropButtonHtml = `
        <button id="draw-crop-btn" class="btn btn-primary">
            🎯 Click to Draw Crop Area
        </button>
    `;
    
    // Inject after codec paragraph
    videoInfoDiv.innerHTML += cropButtonHtml;
    
    // Add event listener to new button
    document.getElementById('draw-crop-btn').addEventListener('click', () => {
        enableCropDrawing(video, canvas);
    });
}

Caution: Using innerHTML += recreates all child elements. Event listeners must be re-attached after injection.

3. Trim Controls

Video Position Capture

Instead of complex timeline scrubbing, uses simple current position capture:

// "Set Start Here" button
video.currentTime // Gets current playback position in seconds

// Display position in real-time
function formatTime(seconds) {
    const mins = Math.floor(seconds / 60);
    const secs = Math.floor(seconds % 60);
    const ms = Math.floor((seconds % 1) * 100);
    return `${mins}:${secs.toString().padStart(2, '0')}.${ms.toString().padStart(2, '0')}`;
}

Design Decision: Timeline scrubbers added complexity without improving UX. Position-based buttons are more intuitive.

4. Scale and Dimension Display

Live Output Resolution Preview

// Calculate what output dimensions will be
let outputW = videoWidth;
let outputH = videoHeight;

// If cropped, use crop dimensions
if (cropW && cropH) {
    outputW = cropW;
    outputH = cropH;
}

// Apply scale percentage
outputW = Math.round(outputW * (scalePercentage / 100));
outputH = Math.round(outputH * (scalePercentage / 100));

// Force even dimensions (H.264 compatibility)
if (outputW % 2 !== 0) outputW++;
if (outputH % 2 !== 0) outputH++;

// Display to user
document.getElementById('output-resolution').textContent = `${outputW}x${outputH}`;

Important: Frontend preview must match backend FFMPEG logic exactly, or users see incorrect dimensions.

---

🎨 UI/UX Patterns

Feature Ordering Logic

Features are ordered by typical editing workflow:

1. Trim - Select video segment (temporal)
2. Crop - Select video area (spatial)
3. Output Scale - Apply to trimmed/cropped result
4. Compression Quality - Final output quality
5. Quick Actions - Rotate/mute (can apply anytime)

Reset Functionality

Complete reset clears:

• All input values (trim, crop, scale, quality)
• Canvas drawings (crop rectangle)
• Video preview (remove src)
• Button states (rotate, mute)
• Display texts (resolution, filename, time)
• Section visibility (hide edit/download sections)

// Comprehensive reset
video.src = '';
cropCanvas.width = 0;
cropCanvas.height = 0;
document.querySelectorAll('input[type="number"]').forEach(input => input.value = '');
document.getElementById('scale-slider').value = 100;
document.getElementById('mute-audio').classList.remove('active');
// ... etc

Button State Management

Visual feedback for toggles:

// Mute audio toggle
muteBtn.classList.toggle('active');
muteBtn.textContent = muted ? '🔇 Audio Muted' : '🔊 Mute Audio';

// Rotation state
document.getElementById('rotation-display').textContent = 
    rotationAngle === 0 ? '' : `Rotated ${rotationAngle}°`;

---

🐳 Deployment Configurations

Local Development

# Python
python app.py  # Runs on localhost:5000

# Docker
docker-compose up --build

Production (CI/CD)

.gitea/workflows/rebuild-prod.yaml:

1. Triggered on push to main
2. Builds Docker image with FFmpeg
3. Pushes to reg.dev.nervesocket.com/video-editor:latest
4. Calls Portainer API to redeploy container
5. Uses production compose pulling from registry

Registry: Private registry at reg.dev.nervesocket.com Deployment: Portainer API with webhook ID df5c509c-8002-463e-b009-3b05b8a1e2c3

---

🧩 Common Extension Patterns

Adding New FFmpeg Filter

1. Add UI control in index.html (slider, button, input)
2. Send parameter with form submission (line ~400)
3. Extract in app.py process_video: request.form.get('param_name')
4. Add filter to filters array: filters.append('filter_name=value')
5. Ensure filter order is logical (crop before scale, etc.)

Adding New Quick Action

1. Create button in Quick Actions section
2. Add click handler setting global variable
3. Send parameter in POST to /process
4. Handle in backend with FFmpeg flag or filter
5. Add reset logic to resetApp()

Handling New Video Codec Output

Currently hardcoded to H.264/AAC. To add codec selection:

1. Add codec dropdown in UI
2. Pass codec parameter to backend
3. Conditional FFmpeg args based on codec:

if codec == 'h264':
    cmd.extend(['-c:v', 'libx264', '-crf', quality])
elif codec == 'h265':
    cmd.extend(['-c:v', 'libx265', '-crf', quality])

---

🔍 Debugging Tips

FFmpeg Command Inspection

The app prints full FFmpeg command to console:

print("FFmpeg command:", ' '.join(cmd))
subprocess.run(cmd, check=True)

Copy this command and run manually to debug encoding issues.

Canvas Coordinate Debugging

Add console logs in mouse handlers:

console.log('Canvas coords:', x, y);
console.log('Video coords:', Math.round(x * scaleX), Math.round(y * scaleY));

Processing Failures

Common issues:

• Odd dimensions: Backend now handles with trunc(), shouldn't occur
• Invalid crop: Crop exceeds video bounds (validate in frontend)
• Missing FFmpeg: Docker image includes it, but local Python needs manual install
• Unsupported format: Check FFmpeg support with ffmpeg -formats

---

📦 Dependencies

Python (requirements.txt)

Flask==3.0.0
Werkzeug==3.0.1

System

• FFmpeg: Must be installed and in PATH
  • On Debian/Ubuntu: apt-get install ffmpeg
  • On Windows: Download from ffmpeg.org
  • Docker: Installed in Dockerfile

FFmpeg Codecs Used

• Video: libx264 (H.264/AVC)
• Audio: aac (AAC)
• Filters: crop, scale, transpose

---

🚨 Critical Implementation Notes

1. H.264 Even Dimension Requirement

Must enforce: Width and height divisible by 2

• Backend: trunc(iw*scale/2)*2
• Frontend: if (dim % 2 !== 0) dim++;
• Violation causes FFmpeg error: "width not divisible by 2"

2. File Cleanup

Currently, files are NOT auto-deleted. Manual cleanup endpoint exists but:

• /cleanup/<file_id> removes files
• Consider adding auto-cleanup timer (e.g., 1 hour after upload)
• Or cron job to clean old files

3. Memory Usage

original_filenames dict grows indefinitely in memory. For production:

• Persist to Redis/database
• Or tie to file existence (check disk, rebuild dict on startup)

4. Security Considerations

Current implementation is for trusted users:

• No file size limits (could DOS server)
• No rate limiting
• No authentication
• FFmpeg runs with user permissions (subprocess)
• For public deployment, add nginx upload limits, Flask rate limiting, user sessions

5. Browser Compatibility

• Canvas API: IE not supported (use Chrome/Firefox/Edge)
• Video Preview: Requires browser codec support (H.264 widely supported)
• File API: Drag-and-drop requires modern browser

---

🎯 Design Philosophy Explained

Why No Framework?

• Simplicity: Single HTML file, no build process
• Learning: Clear vanilla JS patterns
• Deployment: No npm, no bundling, just Python + FFmpeg
• Performance: No framework overhead for simple app

Why Percentage-Based Scaling?

• Intuitive: "50% smaller" clearer than "calculate dimensions"
• Flexible: Works regardless of input resolution
• Safer: Avoids invalid dimensions (too large/small)
• Chaining: Works naturally after crop (scale the crop, not original)

Why CRF Instead of Bitrate?

• Quality-based: Maintains consistent quality across different content
• Simpler: One slider, no need to understand bitrate implications
• Better results: Variable bitrate adapts to video complexity
• Industry standard: CRF 18-23 is professional range

---

🔮 Future Enhancement Considerations

If extending this project, consider:

1. Multiple Video Support: Concatenation, side-by-side
2. Audio Controls: Volume adjustment, audio replacement
3. Filters: Brightness, contrast, saturation, blur
4. Watermarks: Text or image overlays
5. Format Selection: WebM, GIF, HEVC output options
6. Batch Processing: Queue multiple jobs
7. Progress Bar: Real-time FFmpeg encoding progress (parse stderr)
8. Undo/Redo: State management for edit history
9. Presets: Save common settings as named presets
10. Cloud Storage: S3/Azure integration for large files

---

📞 Getting Help

When asking for assistance:

1. Describe the feature: User-facing behavior first
2. Specify location: Which file/function needs changes
3. FFmpeg context: If encoding-related, mention codec/format
4. Show errors: Full error messages help diagnose issues
5. Test command: If FFmpeg fails, what command would work manually?

Example good request:

"I want to add a brightness filter. Users should have a slider in the UI from -1.0 to 1.0. This should apply the FFmpeg eq=brightness=X filter before the scale filter. The parameter should be called 'brightness' and default to 0.0."

---

Last Updated: 2024-03-15 (after adding compression quality slider and filename preservation)

Current Version: Fully functional simple video editor with 7 main features (trim, crop, scale, quality, rotation, mute, visual controls)