Lasted edited: February 14, 2025

A work in progress. Maybe.

Reminder - good branch add-object-identification-to-opencv

Commonly Used Commands

source venv/bin/activate

docker-compose down

docker-compose up -d

docker-compose up --build

docker-compose logs -f web

To Back Up Project Files

I realize that best practices includes using git and requirements.txt for the backup. But you better be backing up the project folder too for when your git inevitably gets all messed up and you need to start again. Look, I ain’t no large organization with some git expert to check every commit, push, and merge. So, you know I’m gonna screw up the git every so often. This is when a project folder may save you some time. So this is how I do it.

Deactivate Virtual Environment

deactivate

Shutdown Docker Container

docker-compose down

Change Into Directory/Folder Above Project To Be Backed Up

cd ..

Duplicate Project Folder With Date & Time To Back Up

cp -r project-folder-name project-folder-name-back-up-$(date +%Y%m%d_%H%M%S)

Example - media-manager folder to be backed up

cp -r media-manager media-manager-back-up-$(date +%Y%m%d_%H%M%S)

Git Help To Delete The Messy Branch

I’m a solo developer who believes that always working on a branch rather than main is the way to go. And this why.

Whenever I’m working on something, or I should say AI is working is working on something, it’s gonna get messed up. Deleting the branch and starting over works for me. So this is how I do it.

First Create A Branch

checkout -b branch-that-is gonna-get-messed up

Make sure that you’re in the new branch.

git status

Make sure again.

git status

Work. Modify files. Delete files. Add Files. Mess up the branch. Oh, no. Nothings working.

git status

See what’s change and make sure you’re still on the messed up branch.

Add all to git

git add .

Commit the changes with a message

git commit -m "This is a temporary commit before I delete branch-that-is gonna-get-messed up"

Go back to main

git status

I’m in main and all looks good.

Delete messed up branch with the capital D since it hasn’t been merged.

git branch -D branch-that-is gonna-get-messed up

Take a deep breath

git status

Moving Project To Cloud

When To Use Sudo

System updates and package installation

sudo apt update sudo apt upgrade sudo apt install

Initial Docker installation

sudo sh get-docker.sh sudo usermod -aG docker $USER

If you need to edit system files

sudo nano /etc/hosts

View system logs

sudo tail -f /var/log/syslog

When Not To Use Sudo

Docker commands (after adding user to docker group)

docker-compose up -d docker ps docker logs

Git operations

git clone git pull git push

Project files

cd your-project nano docker-compose.yml mkdir new-directory

Python/pip commands in your project

pip install -r requirements.txt python main.py

To Run Project

Change Into Project Folder directory

cd name-of-project-folder

Activate Virtual Environment

source venv/bin/activate

Start Docker Container

docker-compose up -d

Open Browser And Go To:

http://localhost:8000/docs

click on POST

click on Try it out

click on choose file to open file browser

click on file that you want to upload

click on open

click on execute

wait for results

To Find Info On Database

Go To backend/.env to find the database url.

DATABASE_URL=postgresql://myuser:mypassword@db:5432/mydatabase

This connection string tells you:

postgresql: The database type is PostgreSQL.

myuser: The username is myuser.

mypassword: The password is mypassword. (Important: In real applications, store passwords securely, not directly in code or configuration files. Use environment variables or secrets management.)

db:5432: The hostname is db (which is likely the name of your PostgreSQL service in your docker-compose.yml file), and the port is 5432.

/mydatabase: This is the database name!

Your application is connecting to a database named mydatabase.

Therefore, to connect manually using psql, you should use:

docker-compose exec db psql -U myuser -d mydatabase

The -d mydatabase part is crucial. It tells psql to connect to the mydatabase database.

Key takeaway: The DATABASE_URL (or similar connection string) is the definitive source for the database name your application is using. Always refer to it when you need to connect manually or perform database administration tasks.

Run Alembic Migration Within Docker With Web service name from docker-compose.yml

docker-compose run --rm web alembic revision --autogenerate -m "your message"

Git General Process

Make sure you’re on your working branch

git status

Add your changes

git add backend/app/services/opencv_service.py (example file name)

git add backend/app/main.py (example file name)

git add docker-compose.yml (example file name)

Create a meaningful commit

git commit -m “Add comprehensive video analysis with OpenCV”

Push to your branch

git push origin your-branch-name

**If everything is working well, merge to main

git checkout main
   	
git merge your-branch-name
   	
git push origin main

** To create new branch and move into it

git checkout -b name-of-new-branch

Initial Process To Create An App, Use Cursor Workspace, And Git

Create Project Directory Than Change Into It

mkdir my-project
cd my-project

Initialize Git in your project directory

git init

You should see a message like “Initialized empty Git repository in /your/path/.git/”

Create a .gitignore File

# Create and open .gitignore
touch .gitignore

Add common files/directories to ignore:

# Dependencies
node_modules/
venv/
.env

# IDE specific
.vscode/
.idea/
*.swp
*.swo

# Cursor specific
.cursor/
.cursor-log/

Open Project in Cursor

  • Launch Cursor

  • File > Open Folder > Select your project directory

Initial Commit

git add .
git commit -m "Initial commit"

Set Up Remote Repository on Github and get the repository url

git remote add origin <your-repository-url>
git branch -M main
git push -u origin main

Python Virtual Environment:

To set up

python -m venv venv

To activate

source venv/bin/activate

Docker with PostgreSQL

With Docker Compose you use a YAML configuration file, known as the Compose file, to configure your application’s services, and then you create and start all the services from your configuration with the Compose CLI.

The default path for a Compose file is compose.yaml (preferred) or compose.yml that is placed in the working directory. Compose also supports docker-compose.yaml and docker-compose.yml for backwards compatibility of earlier versions. If both files exist, Compose prefers the canonical compose.yaml.

DockerDocs

To Start PostgreSQL (using Docker)

docker run --name postgres-db -e POSTGRES_PASSWORD=password -p 5432:5432 -d postgre

Start the database

docker-compose up -d

Start the database and build for changes

docker-compose up --build

Stop the database

docker-compose down

Stop and remove volumes (will delete all data)

docker-compose down -v

View logs

docker-compose logs -f db

Restart the database

docker-compose restart db

To list all the services along with their current status

docker compose ps

Check PostgreSQL/Docker container status by executing the below command in the project’s root directory.

docker-compose ps

Check logs if needed

docker-compose logs db

To activate virtual environment

source venv/bin/activate

Make sure the environment is running before executing the below uvicorn command to start the web application.

uvicorn main:app --reload

Uvicorn command is used to start a web application using Uvicorn, a popular ASGI (Asynchronous Server Gateway Interface) server, often used with frameworks like FastAPI.

uvicorn invokes the Uvicorn server. It’s the command you use to run your ASGI-compatible application.

main:app specifies the location of your application.

main refers to the Python module or file where your application code resides (e.g., main.py). app refers to the actual application object within that module. This could be an instance of a FastAPI application, a Starlette application, or any other ASGI-compatible callable.

--reload is a crucial flag for development. It enables auto-reloading. When you make changes to your code, Uvicorn will automatically detect those changes and restart the server, so you don’t have to manually restart it every time you modify your application.  

Alembic commands are run where alembic.ini is located

Generates Migration

   alembic revision --autogenerate -m "Rename metadata to file_metadata in Asset model"

Apply Migration

   alembic upgrade head

If unable to install python face recognition models/opencv:

Error is generally “Please install face_recognition_models with this command before using face_recognition”

To fix:

pip install setuptools

Suggested FastApi Project Structure

Common FastAPI Project Structure

backend/
│
├── app/
│   ├── api/
│   │   ├── endpoints/
│   │   │   ├── assets.py
│   │   │   └── other_endpoints.py
│   │   └── __init__.py
│   ├── core/
│   │   ├── config.py
│   │   └── security.py
│   ├── db/
│   │   ├── base.py
│   │   ├── session.py
│   │   └── models/
│   │       ├── asset.py
│   │       └── other_models.py
│   ├── schemas/
│   │   ├── asset.py
│   │   └── other_schemas.py
│   ├── utils/
│   │   ├── metadata_extractor.py  # Utility for extracting metadata
│   │   └── other_utilities.py
│   ├── main.py
│   └── __init__.py
│
├── alembic/
│   ├── versions/
│   └── env.py
│
├── tests/
│   ├── test_assets.py
│   └── other_tests.py
│
├── .env
├── .gitignore
├── requirements.txt
└── README.md

Steps to Back Up and Recreate Your Virtual Environment

  1. Ensure requirements.txt is Up-to-Date If you’ve installed additional packages since creating your requirements.txt, you can update it with the current list of installed packages: txt This command will overwrite your existing requirements.txt with the current list of installed packages in your virtual environment.
  2. Recreate the Virtual Environment If you need to recreate your virtual environment, you can do so by following these steps: Delete the Existing venv: If it exists and you want to start fresh, delete it:
  3. Version Control: Keep your requirements.txt file under version control (e.g., Git) to ensure you always have a record of your project’s dependencies.
  4. Environment Variables: If you have any environment-specific settings, consider using a .env file to store them, and ensure it’s also backed up or documented.

To run Postgres migrations in docker

Stop the containers

docker-compose down

Rebuild and start

docker-compose up --build -d

Run the migration

docker-compose exec web alembic revision --autogenerate -m "add_metadata_columns"

docker-compose exec web alembic upgrade head

Adding fields to OpenCv for analysis

The error occurs because we’re trying to return a NumPy boolean (np.True_) in the response, which FastAPI can’t serialize to JSON. Let’s fix the OpenCVService class to convert NumPy types to Python native types:

def analyze_video(self, video_path: str) -> Dict[Any, Any]:
    """Analyze video file and extract various metrics."""
    try:
        # ... (previous code remains the same until return statement)

        return {
            "fps": float(fps),
            "total_frames": int(total_frames),
            "width": int(width),
            "height": int(height),
            "faces_detected": int(avg_faces),
            "motion_score": float(avg_motion),
            "brightness_score": float(avg_brightness),
            "is_stable": bool(avg_motion < 30),  # Convert to Python bool
            "scene_changes": [float(ts) for ts in scene_changes],  # Convert to list of floats
            "shot_types": {
                "close_up_percentage": float(close_up),
                "wide_shot_percentage": float(wide_shot),
                "action_shot_percentage": float(action_shot),
                "static_shot_percentage": float(static_shot)
            },
            "camera_motion": {
                "static_percentage": float(static),
                "pan_tilt_percentage": float(pan_tilt),
                "tracking_percentage": float(tracking),
                "rapid_movement_percentage": float(rapid)
            }
        }

    except Exception as e:
        print(f"Error analyzing video: {str(e)}")
        return {}

The key changes are converting NumPy types to Python native types using float(), int(), and bool(). Converting NumPy arrays to Python lists with list comprehension Ensuring all numeric values are explicitly converted to Python types.

After making these changes, rebuild your container:

docker-compose down

docker-compose up --build

Visit Emlekezik.com