Multi-Node Setup

Scale BitBonsai’s encoding capacity by adding worker nodes across multiple machines.

What is Multi-Node Encoding?

Multi-node encoding distributes video transcoding jobs across multiple physical machines, dramatically reducing total encoding time. How it works:

Main Node: Owns the PostgreSQL database, distributes jobs, hosts the web UI
Child Nodes: Worker-only nodes that pull jobs from the main node’s queue
Shared Storage: NFS mount ensures all nodes access the same video files
Automatic Load Balancing: Jobs are distributed based on each node’s current load and capacity

Example:

Main Node (192.168.1.100):
  - PostgreSQL database
  - Web UI (port 4210)
  - Encodes jobs (4 concurrent)

Child Node 1 (192.168.1.170):
  - Encodes jobs (8 concurrent)
  - Proxies API calls to main

Total capacity: 12 concurrent encoding jobs

Architecture

The main node owns the database and web UI. Child nodes are workers that connect to the main node’s database.

[Web UI] -> [Main Node] -> [Database]
                ↓
         [Child Nodes] -> [NFS Storage]

All nodes must mount the same NFS share to access video files.

Prerequisites

Before setting up multi-node:

Shared Storage Required

All nodes must access the same video files via NFS/SMB network mount.Without shared storage, child nodes cannot read source videos or write encoded files.

Main Node

PostgreSQL 15+ (8GB RAM recommended)
Docker + Docker Compose
NFS mount to video storage

Child Nodes

Docker installed
NFS mount to same path as main node
Network access to main node

Setup Instructions

Step 1: Configure NFS Server

On your NAS or file server:

# Example: Unraid NFS export
/mnt/user/videos 192.168.1.0/24(rw,sync,no_subtree_check,no_root_squash)

# Restart NFS
exportfs -ra

Step 2: Mount NFS on All Nodes

Mount the shared storage at the same path on every node:

# On main node (192.168.1.100)
sudo mkdir -p /mnt/videos
sudo mount -t nfs 192.168.1.1:/mnt/user/videos /mnt/videos

# On child node 1 (192.168.1.170)
sudo mkdir -p /mnt/videos
sudo mount -t nfs 192.168.1.1:/mnt/user/videos /mnt/videos

# On child node 2 (192.168.1.171)
sudo mkdir -p /mnt/videos
sudo mount -t nfs 192.168.1.1:/mnt/user/videos /mnt/videos

Make permanent in /etc/fstab:

192.168.1.1:/mnt/user/videos /mnt/videos nfs defaults,_netdev 0 0

Step 3: Deploy Main Node

On the main node (192.168.1.100):

# docker-compose.yml
version: '3.8'

services:
  postgres:
    image: postgres:15
    environment:
      POSTGRES_DB: bitbonsai
      POSTGRES_USER: bitbonsai
      POSTGRES_PASSWORD: secure_password
    volumes:
      - ./postgres-data:/var/lib/postgresql/data
    restart: unless-stopped

  backend:
    image: bitbonsai/backend:latest
    ports:
      - "3100:3100"
    environment:
      DATABASE_URL: postgresql://bitbonsai:secure_password@postgres:5432/bitbonsai
      NODE_MODE: MAIN
      CONCURRENT_JOBS: 4
    volumes:
      - /mnt/videos:/media
    depends_on:
      - postgres
    restart: unless-stopped

  frontend:
    image: bitbonsai/frontend:latest
    ports:
      - "4210:80"
    environment:
      API_URL: http://192.168.1.100:3100
    restart: unless-stopped

Start:

docker compose up -d

Step 4: Deploy Child Nodes

On child node 1 (192.168.1.170):

# docker-compose.yml
version: '3.8'

services:
  backend:
    image: bitbonsai/backend:latest
    ports:
      - "3100:3100"
    environment:
      DATABASE_URL: postgresql://bitbonsai:secure_password@192.168.1.100:5432/bitbonsai
      NODE_MODE: LINKED
      MAIN_NODE_URL: http://192.168.1.100:3100
      CONCURRENT_JOBS: 8
    volumes:
      - /mnt/videos:/media
    restart: unless-stopped

Start:

docker compose up -d

Repeat for additional child nodes (192.168.1.171, etc.)

Step 5: Verify Node Registration

Check the web UI at http://192.168.1.100:4210:

Go to Settings → Nodes
You should see:
- Main Node (192.168.1.100) - Status: ONLINE
- Child Node 1 (192.168.1.170) - Status: ONLINE
- Child Node 2 (192.168.1.171) - Status: ONLINE

How Job Distribution Works

BitBonsai uses smart load balancing:

User scans library → Jobs added to queue (status: QUEUED)
Scheduler polls every 5 seconds
For each node:
- Check current load (active jobs / max concurrent)
- If load < 100%, assign next QUEUED job to this node
Node picks up job → status changes to ENCODING
Job completes → status changes to COMPLETED

Load calculation:

Node Load = (Active Jobs / Max Concurrent Jobs) × 100

Example:
- Main Node: 3/4 jobs = 75% load
- Child Node 1: 6/8 jobs = 75% load
- Child Node 2: 2/8 jobs = 25% load ← Gets next job

Troubleshooting

Child Node Not Appearing in UI

Symptom: Child node started but not visible in Settings → Nodes Fix:

# On child node, check logs
docker logs bitbonsai-backend

# Look for:
# ✓ Connected to database at postgresql://192.168.1.100:5432/bitbonsai
# ✓ Registered as LINKED node with ID: node_xyz

# If missing, check:
# 1. DATABASE_URL points to main node's PostgreSQL
# 2. PostgreSQL port 5432 is accessible (firewall)
# 3. NODE_MODE=LINKED is set

Jobs Stuck in QUEUED

Symptom: Jobs remain QUEUED, no node picks them up Fix:

Check all nodes are ONLINE in Settings → Nodes

Verify NFS mounts are working:

docker exec bitbonsai-backend ls -la /media

Check logs for errors:

docker logs bitbonsai-backend | grep ERROR

File Not Found Errors

Symptom: Job fails with “ENOENT: no such file or directory” Fix:

Verify NFS mount path is identical on all nodes
Check NFS permissions (should be rw, not ro)

Test file access:

docker exec bitbonsai-backend cat /media/Movies/test.mp4

Performance Tips

Tip	Why
Use 10GbE network	NFS throughput critical for 4K video
Enable NFS async mode	Faster writes (use UPS for safety)
Match NFS mount paths	Avoids path rewriting overhead
Use SSD for database	Faster job updates and queries
Monitor node load	Add nodes when load > 80% consistently

GPU Acceleration - Add GPU support to nodes
Monitoring - Track multi-node performance
Troubleshooting - Fix common multi-node issues

Getting Started

Installation

User Guide

Advanced

Support

What is Multi-Node Encoding?

Architecture

Prerequisites

Shared Storage Required

Main Node

Child Nodes

Setup Instructions

Step 1: Configure NFS Server

Step 2: Mount NFS on All Nodes

Step 3: Deploy Main Node

Step 4: Deploy Child Nodes

Step 5: Verify Node Registration

How Job Distribution Works

Troubleshooting

Child Node Not Appearing in UI

Jobs Stuck in QUEUED

File Not Found Errors

Performance Tips

​What is Multi-Node Encoding?

​Architecture

​Prerequisites

Shared Storage Required

Main Node

Child Nodes

​Setup Instructions

​Step 1: Configure NFS Server

​Step 2: Mount NFS on All Nodes

​Step 3: Deploy Main Node

​Step 4: Deploy Child Nodes

​Step 5: Verify Node Registration

​How Job Distribution Works

​Troubleshooting

​Child Node Not Appearing in UI

​Jobs Stuck in QUEUED

​File Not Found Errors

​Performance Tips

​Related Guides

What is Multi-Node Encoding?

Architecture

Prerequisites

Setup Instructions

Step 1: Configure NFS Server

Step 2: Mount NFS on All Nodes

Step 3: Deploy Main Node

Step 4: Deploy Child Nodes

Step 5: Verify Node Registration

How Job Distribution Works

Troubleshooting

Child Node Not Appearing in UI

Jobs Stuck in QUEUED

File Not Found Errors

Performance Tips

Related Guides