initial commit

2024-10-25 17:28:07 +01:00 · 2024-10-25 17:28:07 +01:00 · 4684a5c726
commit 4684a5c726
2 changed files with 669 additions and 0 deletions
--- a/README.md
+++ b/README.md
@ -0,0 +1,669 @@
 # Building my Mastodon Setup with Docker
 So many admins of [mastodon](https://joinmastodon.org) instances prefer to use a server and install the OS and mastodon straight onto it. I on the other hand prefer to containerize my setup with [docker](https://www.docker.com). I find it neater and easier to upgrade, plus if I want to run other services on the same server I'm free to do so without messing up dependancies or versions. Now there are some ok(ish) guides on how to do this out there but I wanted pull together a few guides in order to produce a complete setup, a [traefik](https://traefik.io) reverse proxy in front of mastodon with full text search and translation enabled using [libretranslate](https://libretranslate.com), I also wanted to use object storage for my assets, in my case I'm running on [AWS](https://aws.amazon.com) so I'm using S3. I also set myself the challenge of removing software with dubious licenses such as Redis and ElasticSearch and opting for the more FOSS friendly [Valkey](https://valkey.io) and [OpenSearch](https://opensearch.org) componants. So here is my setup.
 ## The Setup
 A picture speaks a thousand words so let me show you the main companants of this setup. Each bit of software is running in it's own container:
 ```mermaid
 flowchart TD
    Users --> |HTTPS port: 80/443| A
    A[Traefik] -->|Web Interface port: 3000| B(mastodon)
    A -->|Streaming port: 4000| C(mastodon-streaming)
    B --> D(sidekiq)
    C --> D
    B --> E[postgres]
    C --> E
    B --> F[valkey]
    C --> F
    B --> G[opensearch]
    B --> H[S3]
    B --> I[libretranslate]
    Users --> | assets | H
 ```
 Now the easiest way to run all these companants and have them interconnected is to set up the entire system with [docker compose](https://docs.docker.com/compose/) but before we get going we are going to have to set up the basics. Now I built this on a [debian 12](https://debian.org) system but most of the compose setup will be identical, you'll just need to tweak the install commands for docker and compose.
 ### Install Docker and Compose
 1. First lets make sure theres no unoffical packages in place that will conflict by removing them.
 ``` bash
 for pkg in docker.io docker-doc docker-compose podman-docker containerd runc; do sudo apt-get remove $pkg; done
 ```
 2. Now lets set up dockers official apt repository:
 ``` bash
 # Add Docker's official GPG key:
 sudo apt-get update
 sudo apt-get install ca-certificates curl
 sudo install -m 0755 -d /etc/apt/keyrings
 sudo curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc
 sudo chmod a+r /etc/apt/keyrings/docker.asc
 # Add the repository to Apt sources:
 echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/debian \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
 sudo apt-get update
 ```
 3. Now install the tool chain:
 ``` bash
 sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin vim
 ```
 4. Verify that the installation is successful by running the hello-world image:
 ``` bash
 sudo docker run hello-world
 ```
 > 👉 Note
 > 
 > If you want to run docker as a none root user you'll need to add that user to the docker group by running the following command:
 > ```sudo usermod -aG docker $USER```
 ### Setup storage directories on the host
 I like to try and keep the persistant data neat and tidy on my system so I make sure group my configs and data directories together. For this I store them in */opt/containers/\<application_name>* in this case **mastodon**
 - Let's start by creating that folder structure, for ease we are going to use sudo to become root for these commands(you could just prefix all instructions with sudo if you really wanted):
 ``` bash
 sudo su -p
 mkdir -p /opt/docker/traefik
 mkdir -p /opt/docker/mastodon
 cd /opt/docker/mastodon
 ```
 - Now we are in the main directory lets create the directories where the conatiners are going to write their persistant data:
 ``` bash
 cd /opt/docker/traefik
 mkdir data
 mkdir logs
 cd /opt/docker/mastodon
 mkdir -p public/system
 mkdir postgres
 mkdir -p opensearch/data
 mkdir valkey
 mkdir -p lt/data/{key,local}
 mkdir lt/api_keys
 ```
 - Now lets set the permissions on those directories so the containers can write their data to them without having to tweak the containers:
 ``` bash
 chown -R 991:991 public
 chown -R 70:root postgres
 chown -R 1000:root opensearch
 chown -R 999:root valkey
 chown -R 1032:1032 lt
 ```
 ### Setting up traefik
 To access your site and ensure you have a valid SSL certificate we are going to run traefik as a reverse proxy, this means we can also force all traffic to be HTTPS which is great for security. Traefik will automtically take care of getting a cert from [LetsEncrypt](https://letsencrypt.org) and rotating it when its due to expire. All you have to do is provide an email address for registration.
 Ensure your in the */opt/docker/traefik* directory and create a new file call *compose.yml*
 ```bash
 vi compose.yml
 ```
 now add the follwoing to the file:
 > 💡 Tip
 > 
 > To enter **insert mode** in vim press i
 ```yaml
 services:
  traefik:
    image: "traefik:latest"
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=proxy"
      - "traefik.http.routers.traefik.entrypoints=web"
      - "traefik.http.routers.traefik.rule=Host(`${HOSTNAME}`)"
      - "traefik.http.middlewares.traefik-auth.basicauth.users=${TRAEFIK_PASSWORD}"
      - "traefik.http.middlewares.traefik-https-redirect.redirectscheme.scheme=https"
      - "traefik.http.middlewares.sslheader.headers.customrequestheaders.X-Forwarded-Proto=https"
      - "traefik.http.routers.traefik.middlewares=traefik-https-redirect"
      - "traefik.http.routers.traefik-secure.entrypoints=websecure"
      - "traefik.http.routers.traefik-secure.rule=Host(`${HOSTNAME}`)"
      - "traefik.http.routers.traefik-secure.middlewares=traefik-auth"
      - "traefik.http.routers.traefik-secure.tls=true"
      - "traefik.http.routers.traefik-secure.tls.certresolver=myresolver"
      - "traefik.http.routers.traefik-secure.service=api@internal"
      # Define the port inside of the Docker service to use
      - "traefik.http.services.traefik-secure.loadbalancer.server.port=8080"
    env_file: .env
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /opt/docker/traefik/logs:/logs:rw
      - /opt/docker/traefik/data/acme.json:/acme.json:rw
      - /opt/docker/traefik/data/traefik.yml:/traefik.yml:rw
      - /opt/docker/traefik/data/config.yml:/config.yml:rw
    networks:
      - proxy
 networks:
  proxy:
    external: false
 ```
 Save and exit the file.
 > 💡 Tip
 > 
 > To get out of vim just type ```:wq!```
 You're also going to need to set up some environment variables for this to work so once you've saved the file above you'll need to create a new file called *.env*
 ```bash
 vi .env
 ```
 Populate it with these details, and remember to update the domain:
 ```text
 HOSTNAME=traefik.example.com
 TRAEFIK_PASSWORD=admin:<GENERATED_PASSWORD>
 ```
 You'll need to generate a password for traefik basic_auth to be able to login to the dashboard API. you can do that with the following command:
 ```bash
 echo $(htpasswd -nb user password) | sed -e s/\\$/\\$\\$/g
 ```
 Paste the output user:password keypair into your *.env* file and save and exit.
 Theres a couple of other files (4 actually) we now need to prep. 3 are going to be empty files that the container will need and one will be the initial config startup for traefik.
 ```bash
 cd /opt/docker/traefik/data
 touch dynamic-config.yml
 touch acme.json
 cd /opt/docker/traefik/logs
 touch traefik.log
 ```
 Now for the important file to pull all this together and get traefik working.
 ```bash
 vi traefik.yml
 ```
 Enter the following information and update your email address:
 ```yaml
 api:
  dashboard: true
  debug: true
 entryPoints:
  web:
    address: ":80"
  websecure:
    address: ":443"
 log:
  level: ERROR
  filePath: "/logs/traefik.log"
  format: common
 serversTransport:
  insecureSkipVerify: true
 providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"
    exposedbydefault: false
  file:
    filename: dynamic-config.yml
 certificatesResolvers:
  myresolver:
    acme:
      email: <YOUR_EMAIL_ADDRESS>
      storage: acme.json
      httpChallenge:
        # used during the challenge
        entryPoint: web
 ```
 Now traefik is ready to run and accept HTTP and HTTPS connections. To get it up and running is pretty simple with docker compose, run the following command:
 ```bash
 cd /opt/docker/traefik
 docker compose up -d
 ```
 #### Test Traefik
 To test traefik you can browse direction to [https://traefik.example.com](https://traefik.example.com) where you will be prompted to enter your basic_auth details in your browser.
 ![ScreenSHot of Basic Auth Prompt](img/basic_auth.png)
 ### Setup object storage
 ##### Create a S3 Bucket
 ##### Generate an IAM key
 ##### Setup Cloudfront
 ### Setting up email with SES
 ##### Setup + Verify Domain
 ##### Generate SMTP credentials
 ### Create configuration files
 This first file is the all important docker compose file, so lets go ahead and create *compose.yml* and paste in the following content.
 ``` bash
 cd /opt/docker/mastodon
 touch .env.production
 touch .env.translate
 vi compose.yml
 ```
 Now lets paste in the following content:
 ``` docker
 services:
  db:
    restart: unless-stopped
    image: postgres:14-alpine
    shm_size: 256mb
    networks:
      - mastodon
    healthcheck:
      test:
        - CMD
        - pg_isready
        - -U
        - postgres
    volumes:
      - /opt/docker/mastodon/postgres:/var/lib/postgresql/data
    environment:
      - POSTGRES_HOST_AUTH_METHOD=trust
  valkey:
    restart: unless-stopped
    image: valkey/valkey:7-alpine
    networks:
      - mastodon
    healthcheck:
      test:
        - CMD
        - valkey-cli
        - ping
    volumes:
      - /opt/docker/mastodon/valkey:/data
  web:
    image: richarvey/mastodon-bird-ui:latest
    restart: unless-stopped
    env_file: .env
    #command: bash -c "rm -f /mastodon/tmp/pids/server.pid; bundle exec rails s -p 3000"
    command: bundle exec puma -C config/puma.rb
    networks:
      - traefik_proxy
      - mastodon
    healthcheck:
      test:
        - CMD-SHELL
        - wget -q --spider --proxy=off localhost:3000/health || exit 1
    depends_on:
      - db
      - valkey
      - os
    volumes:
      - /opt/docker/mastodon/public/system:/mastodon/public/system
    labels:
      - traefik.enable=true
      - traefik.http.routers.mastodon.entrypoints=web
      - traefik.http.routers.mastodon.rule=Host(`mastodon.squarecows.com`)
      - traefik.http.middlewares.mastodon-https-redirect.redirectscheme.scheme=https
      - traefik.http.routers.mastodon.middlewares=mastodon-https-redirect
      - traefik.http.routers.mastodon-secure.entrypoints=websecure
      - traefik.http.routers.mastodon-secure.rule=Host(`mastodon.squarecows.com`)
      - traefik.http.routers.mastodon-secure.tls=true
      - traefik.http.routers.mastodon-secure.tls.certresolver=myresolver
      - traefik.http.routers.mastodon-secure.service=mastodon
      - traefik.http.services.mastodon.loadbalancer.server.port=3000
      - traefik.docker.network=traefik_proxy
  streaming:
    image: ghcr.io/mastodon/mastodon-streaming:v4.3.1
    restart: unless-stopped
    env_file: .env
    command: node ./streaming/index.js
    networks:
      - traefik_proxy
      - mastodon
    healthcheck:
      test:
        - CMD-SHELL
        - wget -q --spider --proxy=off localhost:4000/api/v1/streaming/health ||
          exit 1
    depends_on:
      - db
      - valkey
    labels:
      - traefik.enable=true
      - traefik.http.routers.mastodon-api.entrypoints=web
      - traefik.http.routers.mastodon-api.rule=(Host(`mastodon.squarecows.com`)
        && PathPrefix(`/api/v1/streaming`))
      - traefik.http.middlewares.mastodon-api-https-redirect.redirectscheme.scheme=https
      - traefik.http.routers.mastodon-api.middlewares=mastodon-api-https-redirect
      - traefik.http.routers.mastodon-api-secure.entrypoints=websecure
      - traefik.http.routers.mastodon-api-secure.rule=(Host(`mastodon.squarecows.com`)
        && PathPrefix(`/api/v1/streaming`))
      - traefik.http.routers.mastodon-api-secure.tls=true
      - traefik.http.routers.mastodon-api-secure.tls.certresolver=myresolver
      - traefik.http.routers.mastodon-api-secure.service=mastodon-api
      - traefik.http.services.mastodon-api.loadbalancer.server.port=4000
      - traefik.docker.network=traefik_proxy
  sidekiq:
    #image: richarvey/mastodon-bird-ui:latest
    image: ghcr.io/mastodon/mastodon:v4.3.1
    restart: always
    env_file: .env
    command: bundle exec sidekiq
    depends_on:
      - db
      - valkey
      - os
    networks:
      - mastodon
    volumes:
      - /opt/docker/mastodon/public/system:/mastodon/public/system
    healthcheck:
      test:
        - CMD-SHELL
        - ps aux | grep '[s]idekiq 6' || false
  os:
    restart: always
    image: opensearchproject/opensearch:latest
    container_name: opensearch-node1
    environment:
      - node.name=opensearch-node1 # Name the node that will run in this container
      - discovery.type=single-node
      - bootstrap.memory_lock=true # Disable JVM heap memory swapping
      - DISABLE_SECURITY_PLUGIN=true
      - OPENSEARCH_JAVA_OPTS=-Xms512m -Xmx512m # Set min and max JVM heap sizes to at least 50% of system RAM
      - OPENSEARCH_INITIAL_ADMIN_PASSWORD=<OPENSEARCH_PASSWORD>
    networks:
      - traefik_proxy
      - mastodon
        #healthcheck:
        #test: ["CMD-SHELL", "curl --silent --fail localhost:9200/_cluster/health || exit 1"]
    volumes:
      - /opt/docker/mastodon/opensearch/data:/usr/share/opensearch/data
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    ports:
      - 9200:9200
      - 9600:9600
  libretranslate:
    image: libretranslate/libretranslate:v1.6.1
    restart: unless-stopped
    healthcheck:
      test:
        - CMD-SHELL
        - ./venv/bin/python scripts/healthcheck.py
    env_file:
      - .env
    volumes:
      - /opt/docker/mastodon/lt/data:/home/libretranslate/.local
      - /opt/docker/mastodon/lt/api_keys:/app/db
    networks:
      - traefik_proxy
      - mastodon
    ports:
      - 5000:5000
 networks:
  mastodon:
    external: false
  traefik_proxy:
    external: true
 ```
 > 👉 Note
 > 
 > There are a few variables you'll need to setup in the above script to match your environment and secure it, make sure you change these values.
 > 
 > - \<POSTGRES_PASSWD> \*1
 > - \<URL_MASTODON_URL> \*4 
 > - \<OPENSEARCH_PASSWORD> \*1
 Once you've changed these variables you'll need to save and exit vim.
 ### Setting up the environment variables
 Now to make this easy I've included a skeleton *.env.production* file for you to copy and populate with your own variables. But first we need to generate some keys and setup the DB.
 #### Generate keys for mastodon
 First we'll need to generate some keys you'll need to secure mastodon and you'll need to run the following command **twice** once for the SECRET_BASE_KEY and once for OTP_SECRET.
 > 👉 Note
 > 
 > make a note of them you'll need them for the config file
 ```bash
 docker compose run --rm -- web bundle exec rails secret
 ```
 Now we need to generate the VAPID_PRIVATE_KEY.
 > 👉 Note
 > 
 > yet again make a note of this output
 ```bash
 docker compose run --rm -- web bundle exec rails mastodon:webpush:generate_vapid_key
 ```
 #### Initialize the database
 Now we need to start setting up the database ready for mastodon to use. Run the following command which will create the DB and populate the initial tables:
 ```bash
 docker compose run --rm -- web bundle exec RAILS_ENV=production rails db:setup
 ```
 #### Generate DB AR Encryption Values
 Since mastodon 4.3.0 you'll also need to set up Active Record (AR) encryption keys. The following command will generate you the required keys.
 ```bash
 docker compose run --rm -- web bundle exec RAILS_ENV=production rails db:encryption:init
 ```
 > 👉 Note
 > 
 > Make a note of these values as we'll need them for the configuration file.
 #### Setup Libre Translate
 Mastodon allows you to translate posts into your native language by using libretranslate. You can run this service locally in your stack with very little setup. Let's look at the steps
 ##### Create the environments file
 ```bash
 vi libretranslate.env
 ```
 and enter the following details, you don't need to edit anything in this file:
 ``` text
 LT_DEBUG=true
 LT_UPDATE_MODELS=true
 LT_SSL=true
 LT_SUGGESTIONS=false
 LT_METRICS=true
 LT_API_KEYS=true
 LT_THREADS=12
 LT_FRONTEND_TIMEOUT=2000
 #LT_REQ_LIMIT=400
 #LT_CHAR_LIMIT=1200
 LT_API_KEYS_DB_PATH=/app/db/api_keys.db
 ```
 ##### Run the container
 Now lets start the container ready to set up the required files and generate an API key.
 ```bash
 docker compose exec -it lt bash
 ```
 ##### Install all the language packs
 We now need to download the language packs for libretranslate to use.
 ```bash
 for i in `/app/venv/bin/argospm list`;do /app/venv/bin/argospm install $i;done
 ```
 > 👉 Note
 > 
 > Note: this will take a fair bit of time so please do be patient
 ##### Generate an API key
 Mastodon will need an API key to access libretranslate. Still within the container lets create an API key that can make 120 requests per min:
 ```bash
 /app/venv/bin/ltmanage keys add 120
 ```
 > 👉 Note
 > 
 > Make a note of the output as this is the API key
 ### Create the mastodon environment variables
 We now need to gather those credentials we've just created and edit our *.env.production* file. Open that file and copy the below contents into that file and change the values in the **< >**.
 ``` bash
 vim .env.production
 ```
 Contents:
 ``` text
 LOCAL_DOMAIN=<URL_MASTODON_URL>
 SINGLE_USER_MODE=false
 SECRET_KEY_BASE=<GENERATED_SECRET_KEY_BASE>
 OTP_SECRET=<GENERATED_OTP_SECRET>
 VAPID_PRIVATE_KEY=<GENERATED_VAPID_PRIVATE_KEY>
 VAPID_PUBLIC_KEY=<GENERATED_VAPID_PUBLIC_KEY>
 DB_HOST=db
 DB_PORT=5432
 DB_NAME=postgres
 DB_USER=postgres
 DB_PASS=<DB_PASSWORD>
 REDIS_HOST=valkey
 REDIS_PORT=6379
 # This is blank on purpose
 REDIS_PASSWORD=
 S3_ENABLED=true
 S3_PROTOCOL=https
 S3_BUCKET=<AWS_S3_BUCKET>
 S3_REGION=<AWS_REGION>
 S3_HOSTNAME=<s3.AWS-REGION-1.amazonaws.com>
 AWS_ACCESS_KEY_ID=<AWS_ACCESS_KEY>
 AWS_SECRET_ACCESS_KEY=<AWS_SECRET_KEY>
 # Only needed if you use cloudfront
 S3_ALIAS_HOST=<CLOUDFRONT_URL>
 SMTP_SERVER=<EMAIL_SERVER>
 SMTP_PORT=587
 SMTP_LOGIN=<EMAIL_USERNAME>
 SMTP_PASSWORD=<EMAIL_PASSWORD>
 SMTP_AUTH_METHOD=plain
 SMTP_OPENSSL_VERIFY_MODE=none
 SMTP_ENABLE_STARTTLS=auto
 SMTP_FROM_ADDRESS=<EMAIL_ADDRESS>
 ES_ENABLED=true
 ES_HOST=http://os
 ES_PORT=9200
 ES_PRESET=single_node_cluster
 ES_USER=admin
 ES_PASS=<OPENSEARCH_PASSWORD>
 ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY=<GENERATED_AR_KEY>
 ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT=<GENERATED_AR_SALT>
 ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY=<GENERATED_AR_PRIMARY_KEY>
 LIBRE_TRANSLATE_ENDPOINT=http://libretranslate:5000
 LIBRE_TRANSLATE_API_KEY=<LT_API_KEY>
 ```
 ## enable admin
 ```bash
 docker compose exec web /bin/bash
 ```
 ```bash
 tootctl accounts deploy -approve <ADMIN_NAME>
 ```
 ## enable opensearch
 ```bash
 docker compose exec web /bin/bash
 ```
 ```bash
 tootctl search deploy
 ```
 ## get rid of yellow cluster warning
 ```bash
 docker compose exec os /bin/bash
 ```
 ```bash
 curl -X PUT -u admin:<OPENSEARCH_PASSWORD> "http://os:9200/_settings" -H 'Content-Type: application/json' -d'{
    "index" : {
        "number_of_replicas" : 0
    }
 }'
 ```
 ### libretranslate
 https://blog.gcn.sh/howtos/integrating-mastodon-and-libretranslate
--- a/img/basic_auth.png
+++ b/img/basic_auth.png