Using Docker With FastApi, Postage, Opencv To Build A Media Asset Manager
Lasted edited: February 22, 2025
A work in progress. Maybe.
To make sure docker is working with the newest code changes - no cache
From the project root directory, first rebuild with no cache (you need to point to the location where Dockerfile isolated. Mine is at server/docker/Dockerfile):
docker build --no-cache -f server/docker/Dockerfile .
Then go to where the docker-compose is located:
docker-compose down
docker-compose build --no-cache
docker-compose up
Or to run in one command:
docker-compose down && docker-compose build --no-cache && docker-compose up
When you execute docker-compose it looks for docker-compose.yml
Useful Commands
To remove all unused images, containers, networks, and volumes
docker system prune
Shut down docker
docker-compose down
Bring docker up
docker-compose up -d
Build docker
docker-compose up --build
Show docker logs
docker-compose logs -f web
Activate virtual environment
source venv/bin/activate
Intro
I got all of this media on a synology nas that only has 512mb of memory. So, my options are limited as to what I can run on it. It’s basically a storage system. They’re photos app is pretty crapy. Just old and slow, like me.
Anyway I was figuring I’d just put a media asset manager (mam) on a linux box or a Mac mini and then use the mam to find whatever I need. I couldn’t find anything that does what I want so I decided to build my own. What with Cursor AI, Claude, Chatgpt, Gemini it shouldn’t be too hard. I guess it is when you know as little as I do.
The overwhelming opinions among the my ai friends is to use docker. And so, that’s what I’m doing. Of course, being as dumb as I am, it ain’t as easy it sounds.
What is Docker
I know nothing about docker so if anyone ever reads this, just go talk to AI instead. These notes that I write are basically for me to try to keep my thoughts straight and have a resource to find stuff when I need it. They also serve as a refresher when I move on to some other hobby for a year or two and then come back to this. Just relearning git this time around has taken me days.
Getting back to docker, my understanding is that the concept is that the actual app gets built inside of a docker container. Then that container can be moved from computer to computer with all the parts inside of it so you don’t gotta worry about different configurations.
In order to use Docker, there are two main files that I’ve become aware about.
Dockerfile
`Dockerfile` is basically the configuration file that sets up the container and then executes the command to start the app.
With Dockerfile, part of what you’re trying to do is duplicate is to duplicate the file structure from your app into the container.
So, if your app has the following structure:
project_root/
└── backend/
├── main.py
├── api/
├── services/
├── models/
└── requirements.txt
Here’s a basic Dockerfile.
# Use the official Python image from the Docker Hub
FROM python:3.9
# Set the working directory in the container
WORKDIR /code/backend
# Create necessary subdirectories
RUN mkdir -p /code/backend/api \
&& mkdir -p /code/backend/services
# Copy the requirements file into the backend directory
COPY backend/requirements.txt .
# Install dependencies
RUN pip install -r requirements.txt
# Copy the entire backend application code into the container
COPY backend/ .
# Command to run the application
CMD ["python", "main.py"]
To me, WORKDIR is fascinating. It acts like the project folder for the app within the container and will be on the same level as other directories like var, bin, etc.
So, it’s a two-step process to duplicate your app file structure.
First you set the working (project) directory in the container.
WORKDIR /code/backend
Second, copy the entire backend application code into the container
COPY backend/ .
In the COPY command you only have one parameter backend/ . because docker knows that the target directory within the container is the working directory /code/backend. And all the directories files within your app directory backend will be copied into container’s code/code/backend.
And you will wind up in the container with, in part, code/backend/services and so on.
docker-compose.yaml
`docker-compose.yaml` is the file that creates the app inside the container.
docker-compose.yaml
** Docker Memory Concerns**
One concern, I have, is the additional resources docker will use. This mam will use lots of resources with opencv and whisper. Then throw in resizing videos, and who knows what else, I’m gonna need every last bit of computing power.
Funny enough, I spent a few days trying to improve memory usage with Claude.AI. Claude got so frustrated with me that he finally told me, “Don’t be such a cheapskate. Throw in more ram.” I swear that’s what Claude told me. So, I did.
Project Structure on February 19, 2025
project_root/
├── backend/ # Backend directory for FastAPI application
│ ├── app/ # Application code for the backend
│ │ ├── __init__.py # Makes the app directory a package
│ │ ├── main.py # Entry point for the FastAPI application
│ │ ├── api/ # API endpoints
│ │ │ ├── __init__.py
│ │ │ ├── auth/ # User authentication endpoints
│ │ │ │ ├── __init__.py
│ │ │ │ └── auth.py # Authentication logic
│ │ │ ├── media/ # Media-related endpoints
│ │ │ │ ├── __init__.py
│ │ │ │ ├── upload.py # Endpoint for uploading media files
│ │ │ │ ├── process.py # Endpoint for processing media files
│ │ │ │ └── search.py # Endpoint for searching media assets
│ │ │ ├── metadata/ # Metadata-related endpoints
│ │ │ │ ├── __init__.py
│ │ │ │ └── metadata.py # Endpoint for managing metadata
│ │ │ └── health/ # Health check endpoints
│ │ │ ├── __init__.py
│ │ │ └── health.py # Health check logic
│ │ ├── services/ # Business logic and processing
│ │ │ ├── __init__.py
│ │ │ ├── opencv/ # OpenCV-related services
│ │ │ │ ├── __init__.py
│ │ │ │ ├── video_analysis.py # Video analysis logic
│ │ │ │ ├── image_processing.py # Image processing logic
│ │ │ │ └── models/ # Directory for OpenCV models (e.g., YOLOv4)
│ │ │ │ ├── __init__.py
│ │ │ │ └── yolov4.py # YOLOv4 model loading and inference logic
│ │ │ ├── whisper/ # Whisper-related services
│ │ │ │ ├── __init__.py
│ │ │ │ ├── transcription.py # Audio transcription logic
│ │ │ │ └── models/ # Directory for Whisper models
│ │ │ │ ├── __init__.py
│ │ │ │ └── whisper_model.py # Logic for loading and using Whisper models
│ │ │ ├── media_processes/ # General media processing services
│ │ │ │ ├── __init__.py
│ │ │ │ ├── video_resizing.py # Video resizing logic
│ │ │ │ └── color_conversion.py # Color conversion logic
│ │ │ └── database/ # Database interaction logic
│ │ │ ├── __init__.py
│ │ │ ├── models.py # Database models
│ │ │ └── queries.py # Database queries and operations
│ │ ├── alembic/ # Alembic migrations directory
│ │ │ ├── env.py # Alembic environment configuration
│ │ │ ├── script.py.mako # Alembic script template
│ │ │ └── versions/ # Directory for migration scripts
│ │ │ ├── __init__.py
│ │ │ └── <migration_file>.py # Individual migration scripts
│ │ ├── models/ # Pydantic models for request/response validation
│ │ │ ├── __init__.py
│ │ │ └── media.py # Models for media assets
│ │ ├── schemas/ # Database schemas (if using an ORM)
│ │ │ ├── __init__.py
│ │ │ └── media_schema.py # Schema definitions for media assets
│ │ └── utils/ # Utility functions and helpers
│ │ ├── __init__.py
│ │ └── file_management.py # File handling utilities
│ ├── requirements.txt # Your dependencies for the backend
├── frontend/ # Frontend directory for client-side code
│ ├── public/ # Public assets (e.g., images, favicon)
│ ├── src/ # Source code for the frontend application
│ │ ├── components/ # React/Vue components
│ │ ├── pages/ # Page components
│ │ ├── App.js # Main application file
│ │ └── index.js # Entry point for the frontend
│ ├── package.json # Frontend dependencies and scripts
│ └── README.md # Frontend documentation
├── media/ # Directory for storing uploaded media files
│ ├── audio/ # Subdirectory for audio files
│ ├── documents/ # Subdirectory for document files
│ ├── images/ # Subdirectory for image files
│ ├── other/ # Subdirectory for other file types
│ └── videos/ # Subdirectory for video files
└── server/ # Directory for server configuration files
├── nginx/ # Nginx configuration files
│ ├── sites-available/ # Nginx sites-available configurations
│ │ ├── your_site.conf # Example site configuration
│ ├── sites-enabled/ # Nginx sites-enabled configurations
│ │ └── your_site.conf # Symlink to sites-available
├── docker/ # Docker-related files
│ ├── Dockerfile # Dockerfile for the backend
│ └── docker-compose.yml # Docker Compose file
├── README.md # Documentation for server configurations
└── other_config_files # Any other server-related configuration files
Project Structure On February 18, 2025
project_root/
├── backend/ # Backend directory for FastAPI application
│ ├── main.py # Entry point for the FastAPI application
│ ├── api/ # API endpoints
│ │ ├── __init__.py
│ │ ├── auth/ # User authentication endpoints
│ │ │ ├── __init__.py
│ │ │ └── auth.py # Authentication logic
│ │ ├── media/ # Media-related endpoints
│ │ │ ├── __init__.py
│ │ │ ├── upload.py # Endpoint for uploading media files
│ │ │ ├── process.py # Endpoint for processing media files
│ │ │ └── search.py # Endpoint for searching media assets
│ │ ├── metadata/ # Metadata-related endpoints
│ │ │ ├── __init__.py
│ │ │ └── metadata.py # Endpoint for managing metadata
│ │ └── health/ # Health check endpoints
│ │ ├── __init__.py
│ │ └── health.py # Health check logic
│ ├── services/ # Business logic and processing
│ │ ├── __init__.py
│ │ ├── opencv/ # OpenCV-related services
│ │ │ ├── __init__.py
│ │ │ ├── video_analysis.py # Video analysis logic
│ │ │ ├── image_processing.py # Image processing logic
│ │ │ └── models/ # Directory for OpenCV models (e.g., YOLOv4)
│ │ │ ├── __init__.py
│ │ │ └── yolov4.py # YOLOv4 model loading and inference logic
│ │ ├── whisper/ # Whisper-related services
│ │ │ ├── __init__.py
│ │ │ ├── transcription.py # Audio transcription logic
│ │ │ └── models/ # Directory for Whisper models
│ │ │ ├── __init__.py
│ │ │ └── whisper_model.py # Logic for loading and using Whisper models
│ │ ├── media_processes/ # General media processing services
│ │ │ ├── __init__.py
│ │ │ ├── video_resizing.py # Video resizing logic
│ │ │ └── color_conversion.py # Color conversion logic
│ │ └── database/ # Database interaction logic
│ │ ├── __init__.py
│ │ ├── models.py # Database models
│ │ └── queries.py # Database queries and operations
│ ├── models/ # Pydantic models for request/response validation
│ │ ├── __init__.py
│ │ └── media.py # Models for media assets
│ ├── schemas/ # Database schemas (if using an ORM)
│ │ ├── __init__.py
│ │ └── media_schema.py # Schema definitions for media assets
│ └── utils/ # Utility functions and helpers
│ ├── __init__.py
│ └── file_management.py # File handling utilities
├── frontend/ # Frontend directory for client-side code
│ ├── public/ # Public assets (e.g., images, favicon)
│ ├── src/ # Source code for the frontend application
│ │ ├── components/ # React/Vue components
│ │ ├── pages/ # Page components
│ │ ├── App.js # Main application file
│ │ └── index.js # Entry point for the frontend
│ ├── package.json # Frontend dependencies and scripts
│ └── README.md # Frontend documentation
├── media/ # Directory for storing uploaded media files
│ ├── audio/ # Subdirectory for audio files
│ ├── documents/ # Subdirectory for document files
│ ├── images/ # Subdirectory for image files
│ ├── other/ # Subdirectory for other file types
│ └── videos/ # Subdirectory for video files
└── server/ # Directory for server configuration files
├── nginx/ # Nginx configuration files
│ ├── sites-available/ # Nginx sites-available configurations
│ │ ├── your_site.conf # Example site configuration
│ ├── sites-enabled/ # Nginx sites-enabled configurations
│ │ └── your_site.conf # Symlink to sites-available
├── docker/ # Docker-related files
│ ├── Dockerfile # Dockerfile for the backend
│ └── docker-compose.yml # Docker Compose file
├── README.md # Documentation for server configurations
└── other_config_files # Any other server-related configuration files