Deploying High-Performance Django on Coolify

Part 1: Initial Setup in Coolify Dashboard

Step 1: Accessing Coolify

1. Open your browser and navigate to your Coolify instance (e.g., https://coolify.yourdomain.com)
2. Log in with your credentials
3. You'll see the main dashboard with "Projects" and "Resources" sections

Step 2: Creating a New Project

1. Click the "New Project" button in the top right corner
2. Fill in the form:
   • Name: "Django-High-Performance"
   • Description: "High-performance Django application with PostgreSQL"
   • Click "Create Project"

Step 3: Setting up PostgreSQL Database

1. In your project, click "New Resource"
2. Select "Database" from the resource types
3. Choose "PostgreSQL"
4. Fill in the configuration:

   Name: postgres-main
   Version: 15
   Username: django_user
   Password: [generate a secure password]
   Database Name: django_db
   Port: 5432

5. Under "Advanced Settings":

   Memory Limit: 12288 (12GB in MB)
   CPU Limit: 4

6. Click "Create Database"
7. After creation, click on the database resource
8. Go to "Settings" tab
9. Under "Configuration", add these settings:

   max_connections = 200
   shared_buffers = 4GB
   effective_cache_size = 12GB
   maintenance_work_mem = 1GB

10. Click "Save Configuration" and "Restart Database"

Step 4: Setting up PgBouncer

1. Click "New Resource"
2. Select "Service" → "Docker"
3. Configuration:

   Name: pgbouncer
   Image: edoburu/pgbouncer:1.18.0
   Port: 6432

4. Click "Add Volume" and add:

   Host Path: /data/pgbouncer/pgbouncer.ini
   Container Path: /etc/pgbouncer/pgbouncer.ini

5. Add another volume:

   Host Path: /data/pgbouncer/userlist.txt
   Container Path: /etc/pgbouncer/userlist.txt

6. Under "Environment Variables", add:

   DB_HOST=postgres-main
   DB_PORT=5432
   DB_USER=django_user
   DB_PASSWORD=[your-postgres-password]

7. Click "Create Service"

Step 5: Setting up Redis

1. Click "New Resource"
2. Select "Database" → "Redis"
3. Configure:

   Name: redis-cache
   Version: 7
   Password: [generate secure password]
   Memory Limit: 4096 (4GB in MB)
   CPU Limit: 2

4. Click "Create Redis"

Part 2: Preparing Your Application

Step 1: Project Files Setup

Create these files in your Django project:

Dockerfile:

FROM python:3.11-slim

ENV PYTHONUNBUFFERED 1
ENV PYTHONDONTWRITEBYTECODE 1

WORKDIR /app

RUN apt-get update && apt-get install -y \
    build-essential \
    postgresql-client \
    libpq-dev \
    && rm -rf /var/lib/apt/lists/*

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt \
    gunicorn[gevent] \
    redis \
    hiredis \
    django-redis \
    psycopg2-binary

COPY . .

RUN python manage.py collectstatic --noinput

RUN useradd -m myuser
RUN chown -R myuser:myuser /app
USER myuser

CMD ["gunicorn", "--config", "gunicorn.conf.py", "core.wsgi:application"]

gunicorn.conf.py:

import multiprocessing

workers = multiprocessing.cpu_count() * 2 + 1
worker_class = 'gevent'
worker_connections = 1000
timeout = 30
keepalive = 2
max_requests = 1000
max_requests_jitter = 200
preload_app = True
bind = "0.0.0.0:8000"

# Logging
accesslog = '-'
errorlog = '-'
loglevel = 'info'

Step 2: Update Django Settings

Update settings.py:

import os
from pathlib import Path

BASE_DIR = Path(__file__).resolve().parent.parent

SECRET_KEY = os.getenv('DJANGO_SECRET_KEY')
DEBUG = os.getenv('DJANGO_DEBUG', 'False') == 'True'
ALLOWED_HOSTS = os.getenv('DJANGO_ALLOWED_HOSTS', '').split(',')

# Database
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql',
        'NAME': os.getenv('POSTGRES_DB'),
        'USER': os.getenv('POSTGRES_USER'),
        'PASSWORD': os.getenv('POSTGRES_PASSWORD'),
        'HOST': os.getenv('POSTGRES_HOST'),
        'PORT': os.getenv('POSTGRES_PORT'),
        'CONN_MAX_AGE': 0,
    }
}

# Cache
CACHES = {
    'default': {
        'BACKEND': 'django_redis.cache.RedisCache',
        'LOCATION': os.getenv('REDIS_URL'),
        'OPTIONS': {
            'CLIENT_CLASS': 'django_redis.client.DefaultClient',
            'PARSER_CLASS': 'redis.connection.HiredisParser',
            'CONNECTION_POOL_CLASS_KWARGS': {
                'max_connections': 50,
                'timeout': 20,
            }
        }
    }
}

# Session
SESSION_ENGINE = 'django.contrib.sessions.backends.cache'
SESSION_CACHE_ALIAS = 'default'

Part 3: Deploying the Application

Step 1: Creating the Application Service

1. In Coolify dashboard, click "New Resource"
2. Select "Application" → "Docker"
3. Select your Git repository
4. Configure Build:

   Branch: main (or your preferred branch)
   Dockerfile Path: ./Dockerfile
   Port: 8000

5. Under "Advanced Settings":

   Memory Limit: 8192 (8GB in MB)
   CPU Limit: 8
   Instance Count: 2

Step 2: Environment Variables

1. Click on "Environment Variables"
2. Add the following (click "Add" after each):

   # Django
   DJANGO_SECRET_KEY=[generate a secure key]
   DJANGO_DEBUG=False
   DJANGO_ALLOWED_HOSTS=your-domain.com
   DJANGO_CSRF_TRUSTED_ORIGINS=https://your-domain.com
   
   # Database
   POSTGRES_DB=django_db
   POSTGRES_USER=django_user
   POSTGRES_PASSWORD=[your-postgres-password]
   POSTGRES_HOST=pgbouncer
   POSTGRES_PORT=6432
   
   # Redis
   REDIS_URL=redis://:[redis-password]@redis-cache:6379/0
   
   # Port
   PORT=8000

Step 3: Network Configuration

1. Go to "Network" tab
2. Click "Add to Network"
3. Select the network containing your database and Redis
4. Click "Add"

Step 4: Initial Deployment

1. Click "Deploy" button
2. Watch the build logs (click "View Logs")
3. Wait for the build to complete

Step 5: Database Migration

1. After successful deployment, click "Terminal"
2. Run:

   python manage.py migrate
   python manage.py createsuperuser

Part 4: Post-Deployment Configuration

Step 1: Domain Setup

1. Go to "Settings" tab
2. Under "Domains", click "Add Domain"
3. Enter your domain
4. Enable "HTTPS/SSL"
5. Choose "Let's Encrypt"
6. Click "Add Domain"

Step 2: Monitoring Setup

1. Go to "Monitoring" tab
2. Enable "Resource Monitoring"
3. Configure alerts:
   • Click "Add Alert"
   • Set CPU Usage > 80%
   • Add email notification
4. Repeat for:
   • Memory Usage > 80%
   • Error Rate > 1%

Common Issues and Solutions

Problem 1: Database Connection Failed

If you see "could not connect to database" errors:

1. Check network connectivity:

   # In application terminal
   nc -zv pgbouncer 6432

2. Verify environment variables:

   # In application terminal
   env | grep POSTGRES

3. Check PgBouncer logs:
   • Go to PgBouncer service
   • Click "Logs"

Problem 2: Redis Connection Issues

If cache errors occur:

1. Verify Redis connection:

   # In Django shell
   from django.core.cache import cache
   cache.set('test', 'working')
   cache.get('test')

2. Check Redis URL format
3. Verify network connectivity

Problem 3: Static Files Not Loading

1. Check STATIC_ROOT setting
2. Run collectstatic again:

   python manage.py collectstatic --clear --noinput

3. Verify whitenoise configuration

Performance Monitoring

Step 1: Database Monitoring

Run these queries in PostgreSQL:

-- Current connections
SELECT count(*) FROM pg_stat_activity;

-- Connection states
SELECT state, count(*) 
FROM pg_stat_activity 
GROUP BY state;

-- Slow queries
SELECT pid, age(clock_timestamp(), query_start), usename, query 
FROM pg_stat_activity 
WHERE state != 'idle' 
  AND query NOT ILIKE '%pg_stat_activity%' 
ORDER BY query_start desc;

Step 2: Application Monitoring

1. Check Gunicorn workers:

   # In application terminal
   ps aux | grep gunicorn

2. Monitor request latency:
   • Go to "Metrics" tab
   • Look for "Request Duration"
   • Check "95th percentile" latency

How to setup an SSH Key and connect to ubuntu server

1. Create SSH Key using
   ssh-keygen
   
   
2. Copy .pub key to server
   ssh-copy-id username@remote_host
   
   
3. Validate previous steps by connecting to server
   ssh username@remote_host
   
   
4. disable password authentication
5. enable firewall

How to run Django Server from VSCode

 Hi Noonari, 

Here is how  I did it.

Step 1. Install following plugins

• Python (by microsoft)
• Live Server

Step 2. Set python version to your virtual env by following these steps in bullets in order

1. Press `CTRL+SHIFT+P`
   This will open a command pallet in vscode.
2. Set the python version by selecting if it is visible in the dropdowns or by typing it like
   `Python > <path-to-venv>/bin/python`
3. create launch.json (normally it is automatically created when you press run button so first press run and see if it runs or not and then check or create launch.json)
4. add this as contents of launch.json
   
   {

       // Use IntelliSense to learn about possible attributes.

       // Hover to view descriptions of existing attributes.

       // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387

       "version": "0.2.0",

       "configurations": [

           {

               "name": "Python: Django",

               "type": "python",

               "request": "launch",

      

               "program": "${workspaceFolder}/monolith/manage.py",

               "args": [

                   "runserver",

                   "--noreload"

               ],

               "django": true,

               "justMyCode": true

           }

       ]

   }

Step 3. Run and test.

Beautifying your Terminal in Linux

 Hi Noonari, long time no see.

So this is about how you can increase your productivity by reducing clutter in your terminal. It may not be up to your test in future but trust me when I am writing this for you, this had mind blowing productivity results. 

1. So here's the deal. You are deep in the repositories. And here you have to write some crazy long command on terminal. What would happen, it will jump to new line and destroy your readability. Here is how you can reduce the path of entire set of directories to just current directory.
               add PROMPT_DIRTRIM=1 to your .bashrc file.
   
   
   
   
2. you also need to install fish. As this is going to be amazing for your future self. Then fish_config to open browser based configuration of fish.
   
   
   
3. You need to find something in terminal for example a command that you ran but you do not know what it was. What you would normally do is press CTRL+r and keep on pressing them until you find it, right? Now you do not have to do this. But just press CTRL+r once and scroll through the commands with arrow keys using this
   
     git clone --depth 1 https://github.com/junegunn/fzf.git ~/.fzf && ~/.fzf/install

With this today's lecture ends. Now remember Noonari, you will keep on receiving more of these for you to use in future. I will keep on typing that I can for you here. Stay sharp and stay hungry. Thats how you got this much of success in career.

How to increase swap size in Ubuntu? (works for all Ubuntu Flavors)

Here we will see how we can increase the SwapSize  in Ubuntu!

1. Open Terminal
2. Enter "sudo swapon --show"
1. This is show you the details of current swap size
3. Enter "df -h"
1. This is to see how much disk space you have.
4. Now Turn the swap off using "sudo swapoff -a"
1. This will allow us to write to swap file without being interrupted by File in Use error
5. This is the important part and requires some knowledge if you want to understand what is going on under the hood. But high level understanding here is as follows:
1. If have xyz GB of ram, your swap file size will be XYZ GB + 2GB at the minimum. In my case, I had 16GB of Ram on Laptop, but current swap file was of 2GB only. When created automatically using the command that follows, it made 27GB in SwapFile.
2. Enter "sudo dd if=/dev/zero of=/swapfile bs=5M count=5120 status=progress"
3. Wait for the entire process to end.
6. Now lets give this file root only permissions: "sudo chmod 600 /swapfile"
7. Now mark this file as swap space file: "sudo mkswap /swapfile"
8.  Now make this file permanent so that restart of system doesn't revert it to previous state: 
1. "echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab"
9. Now turn the swap back on using "sudo swapon /swapfile"
10. Use HTOP to see change in swap size.

What is difference between queryset and get_queryset?

In your example, overriding queryset and get_queryset have the same effect. I would slightly favour setting get_queryset because it's more verbose.

When you set queryset, the queryset is created only once, when you start your server. On the other hand, the get_queryset method is called for every request.

That means that get_queryset is useful if you want to adjust the query dynamically. For example, you could return objects that belong to the current user:

class IndexView(generic.ListView):
    def get_queryset(self):
        """Returns Polls that belong to the current user"""
        return Poll.active.filter(user=self.request.user).order_by('-pub_date')[:5]

Another example where get_queryset is useful is when you want to filter based on a callable, for example, return today's polls:

class IndexView(generic.ListView):
    def get_queryset(self):
        """Returns Polls that were created today"""
        return Poll.active.filter(pub_date=date.today())

If you tried to do the same thing by setting queryset, then date.today() would only be called once, when the view was loaded, and the view would display incorrect results after a while.

class IndexView(generic.ListView):
    # don't do this!
    queryset = Poll.active.filter(pub_date=date.today())

How to enable minimize action on Ubuntu 18.04

This quick tutorial shows you how to enable ‘Minimize on click’, the feather that minimize the running application window when you clicking on the icon in left launcher.

While Settings and Gnome Tweaks utilities do not provide an option to toggle the action, you can enable the feature with Dconf Editor.

1. Search for and install dconf editor in Ubuntu Software:

2. Then launch the tool and navigate to org -> gnome -> shell -> extensions -> dash-to-dock. Scroll download, and click go to settings for click-action.

3. Finally disable default setting and choose “minimize” as its value from drop-down menu:

That’s it.