This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

Cache Management

This guide explains how to manage the Redis/Valkey cache used by Bovine Pages Server, including viewing cache contents, flushing specific sites, and troubleshooting cache-related issues.

Overview

Bovine Pages Server uses Redis or Valkey for caching to improve performance and enable advanced features. The cache stores:

File Content: Static files (HTML, CSS, JS, images, etc.) from repositories
Custom Domain Mappings: Domain-to-repository associations
Password Hashes: Hashed passwords for password-protected sites
Directory Listings: Generated directory listing HTML
Traefik Router Configurations: Dynamic router settings for custom domains

Cache Key Structure

Understanding the cache key format helps you target specific data for inspection or deletion.

File Content Cache

Format: username:repository:filepath

Examples:

squarecows:bovine-website:public/index.html
john:my-blog:public/assets/style.css
alice:docs:public/api/v1/reference.html

Custom Domain Mappings

Forward Mapping (domain → repository):

custom_domain:bovine.squarecows.com
custom_domain:www.example.com

Reverse Mapping (repository → domain):

squarecows:bovine-website
john:personal-site

Password Hash Cache

Format: password_hash:username:repository

Examples:

password_hash:squarecows:private-docs
password_hash:alice:team-wiki

Traefik Router Configurations

Format: traefik/http/routers/{sanitized-domain}/{property}

Examples:

traefik/http/routers/custom-bovine-squarecows-com/entrypoints
traefik/http/routers/custom-bovine-squarecows-com/rule
traefik/http/routers/custom-bovine-squarecows-com/service
traefik/http/routers/custom-bovine-squarecows-com/middlewares
traefik/http/routers/custom-bovine-squarecows-com/tls/certresolver
traefik/http/routers/custom-bovine-squarecows-com/priority

Accessing Redis/Valkey CLI

Docker/Podman Setup

Most deployments run Redis or Valkey in a Docker container. First, find your container:

# List all containers
docker ps

# Find Redis/Valkey container
docker ps | grep -E "redis|valkey"

# Example output:
# abc123def456   redis:7-alpine   "docker-entrypoint..."   redis

Connection Methods

Method 1: Direct Command Execution

# Redis
docker exec <container-name> redis-cli <command>

# Valkey
docker exec <container-name> valkey-cli <command>

Method 2: Interactive Shell

# Redis
docker exec -it <container-name> redis-cli

# Valkey
docker exec -it <container-name> valkey-cli

# You'll get an interactive prompt:
127.0.0.1:6379>

Method 3: Container Shell + CLI

# Enter container
docker exec -it <container-name> sh

# Then run CLI commands
redis-cli
# or
valkey-cli

With Password Authentication

If your Redis/Valkey requires authentication:

# Add -a flag
docker exec <container-name> redis-cli -a your-password <command>

# Or in interactive mode, authenticate after connecting
docker exec -it <container-name> redis-cli
127.0.0.1:6379> AUTH your-password
OK

Inspecting Cache Contents

View All Keys

# Redis
docker exec redis redis-cli KEYS "*"

# Valkey
docker exec valkey valkey-cli KEYS "*"

# Warning: KEYS is slow on large databases, use SCAN instead

Scan Keys (Recommended for Production)

# Scan all keys (safer than KEYS)
docker exec redis redis-cli --scan

# Scan with pattern
docker exec redis redis-cli --scan --pattern "*bovine*"

# Count total keys
docker exec redis redis-cli DBSIZE

View Specific Key Value

# Get a single key
docker exec redis redis-cli GET "custom_domain:bovine.squarecows.com"

# Example output: squarecows:bovine-website

Check Key Type and TTL

# Check key type
docker exec redis redis-cli TYPE "custom_domain:bovine.squarecows.com"

# Check time-to-live (-1 = no expiration, -2 = doesn't exist)
docker exec redis redis-cli TTL "custom_domain:bovine.squarecows.com"

# Check if key exists
docker exec redis redis-cli EXISTS "custom_domain:bovine.squarecows.com"

Search for Keys by Pattern

# Find all keys for a specific user
docker exec redis redis-cli --scan --pattern "squarecows:*"

# Find all custom domains
docker exec redis redis-cli --scan --pattern "custom_domain:*"

# Find all password hashes
docker exec redis redis-cli --scan --pattern "password_hash:*"

# Find all Traefik router configs
docker exec redis redis-cli --scan --pattern "traefik/http/routers/*"

Flushing Cache

Flush Cache for Specific Website

Custom Domain Site (e.g., bovine.squarecows.com)

Step 1: Find what repository the domain maps to

docker exec redis redis-cli GET custom_domain:bovine.squarecows.com
# Output: squarecows:bovine-website

Step 2: Delete all cached content

# Redis
docker exec redis sh -c "redis-cli --scan --pattern 'squarecows:bovine-website:*' | xargs -r redis-cli DEL"

# Valkey
docker exec valkey sh -c "valkey-cli --scan --pattern 'squarecows:bovine-website:*' | xargs -r valkey-cli DEL"

Step 3: Delete custom domain mappings

# Redis
docker exec redis redis-cli DEL custom_domain:bovine.squarecows.com
docker exec redis redis-cli DEL squarecows:bovine-website

# Valkey
docker exec valkey valkey-cli DEL custom_domain:bovine.squarecows.com
docker exec valkey valkey-cli DEL squarecows:bovine-website

Step 4: Delete Traefik router configurations

# Redis (sanitized domain: bovine.squarecows.com → bovine-squarecows-com)
docker exec redis redis-cli DEL traefik/http/routers/custom-bovine-squarecows-com/entrypoints
docker exec redis redis-cli DEL traefik/http/routers/custom-bovine-squarecows-com/rule
docker exec redis redis-cli DEL traefik/http/routers/custom-bovine-squarecows-com/service
docker exec redis redis-cli DEL traefik/http/routers/custom-bovine-squarecows-com/middlewares
docker exec redis redis-cli DEL traefik/http/routers/custom-bovine-squarecows-com/tls/certresolver
docker exec redis redis-cli DEL traefik/http/routers/custom-bovine-squarecows-com/priority

# Valkey (same commands, just replace redis-cli with valkey-cli)

Quick Method: Delete Everything Related to Domain

# Redis
docker exec redis sh -c "redis-cli --scan --pattern '*bovine.squarecows.com*' | xargs -r redis-cli DEL"
docker exec redis sh -c "redis-cli --scan --pattern '*bovine-squarecows-com*' | xargs -r redis-cli DEL"

# Valkey
docker exec valkey sh -c "valkey-cli --scan --pattern '*bovine.squarecows.com*' | xargs -r valkey-cli DEL"
docker exec valkey sh -c "valkey-cli --scan --pattern '*bovine-squarecows-com*' | xargs -r valkey-cli DEL"

Pages Domain Site (e.g., john.pages.example.com/blog)

Flush all cached content for a repository:

# Redis
docker exec redis sh -c "redis-cli --scan --pattern 'john:blog:*' | xargs -r redis-cli DEL"

# Valkey
docker exec valkey sh -c "valkey-cli --scan --pattern 'john:blog:*' | xargs -r valkey-cli DEL"

Delete password hash (if password protected):

# Redis
docker exec redis redis-cli DEL password_hash:john:blog

# Valkey
docker exec valkey valkey-cli DEL password_hash:john:blog

Flush Cache for Specific User

Delete all cached content for all repositories owned by a user:

# Redis
docker exec redis sh -c "redis-cli --scan --pattern 'john:*' | xargs -r redis-cli DEL"

# Valkey
docker exec valkey sh -c "valkey-cli --scan --pattern 'john:*' | xargs -r valkey-cli DEL"

Flush Entire Cache

⚠️ Warning: This deletes ALL cached data for ALL sites!

# Redis
docker exec redis redis-cli FLUSHDB

# Valkey
docker exec valkey valkey-cli FLUSHDB

# Flush all databases (if using multiple)
docker exec redis redis-cli FLUSHALL

Common Cache Management Tasks

Invalidate Cache After Content Update

When you push new content to a repository, the cache isn't automatically invalidated. To force a refresh:

# Delete all cached files for the repository
docker exec redis sh -c "redis-cli --scan --pattern 'username:repository:*' | xargs -r redis-cli DEL"

# The next request will fetch fresh content from Forgejo

Remove Stale Custom Domain Mapping

If a custom domain was changed or removed:

# 1. Find current mapping
docker exec redis redis-cli GET custom_domain:old-domain.com

# 2. Delete forward mapping
docker exec redis redis-cli DEL custom_domain:old-domain.com

# 3. Delete reverse mapping (using output from step 1)
docker exec redis redis-cli DEL username:repository

# 4. Delete Traefik router configs
docker exec redis sh -c "redis-cli --scan --pattern 'traefik/http/routers/custom-old-domain-com/*' | xargs -r redis-cli DEL"

Clear Password Cache

After changing a site's password in the .pages file:

# Redis
docker exec redis redis-cli DEL password_hash:username:repository

# Valkey
docker exec valkey valkey-cli DEL password_hash:username:repository

Monitor Cache Activity

# Watch commands in real-time
docker exec redis redis-cli MONITOR

# Get cache statistics
docker exec redis redis-cli INFO stats

# Check memory usage
docker exec redis redis-cli INFO memory

Automated Cache Management

Using the Cache Reaper Script

For automated cleanup of stale domain mappings, use the Cache Reaper script included in the repository:

cd reaper/
python3 reaper.py --forgejo-host https://git.example.com \
                  --redis-host localhost \
                  --redis-port 6379 \
                  --dry-run

See Cache Reaper documentation for details.

Scheduled Cache Cleanup

Create a cron job for regular cache cleanup:

# Edit crontab
crontab -e

# Run reaper daily at 2 AM
0 2 * * * cd /path/to/reaper && python3 reaper.py --forgejo-host https://git.example.com --redis-host localhost

Troubleshooting

Cache Not Working

Check if Redis/Valkey is running:

docker ps | grep -E "redis|valkey"
docker exec redis redis-cli PING
# Expected: PONG

Check Traefik can connect to Redis:

# Check Traefik logs for Redis connection errors
docker logs traefik | grep -i redis

Verify cache configuration in middleware:

Check enableRedisCache: true in Traefik middleware config
Verify redisAddress points to correct container/host
Test Redis connectivity: docker exec traefik ping redis

Custom Domain Not Working

Check if domain mapping exists:

docker exec redis redis-cli GET custom_domain:yourdomain.com
# Should return: username:repository

Check if Traefik router exists:

docker exec redis redis-cli --scan --pattern "traefik/http/routers/custom-yourdomain-com/*"
# Should return 6 keys

Re-register domain:

Delete existing mappings (see above)
Visit the pages URL: https://username.pages.example.com/repository/
Check cache again - mappings should be recreated

High Memory Usage

Check cache size:

docker exec redis redis-cli INFO memory | grep used_memory_human
docker exec redis redis-cli DBSIZE

Find largest keys:

docker exec redis redis-cli --bigkeys

# Or use redis-cli memory doctor
docker exec redis redis-cli --memkeys

Reduce cache size:

Lower cache TTL in middleware configuration
Run cache reaper more frequently
Flush unused cached content
Consider increasing maxmemory and enabling eviction policy

Authentication Issues

Test authentication:

# Without password
docker exec redis redis-cli PING
# Error: NOAUTH Authentication required

# With password
docker exec redis redis-cli -a your-password PING
# PONG

Update Traefik middleware configuration:

redisPassword: your-redis-password

Connection Exhaustion Issues (v0.1.5+)

Problem: Redis refusing connections or "too many clients" errors

Check current connections:

# Check connected clients
docker exec redis redis-cli CLIENT LIST | wc -l

# Check max clients setting
docker exec redis redis-cli CONFIG GET maxclients

Tune connection pooling settings:

# In Traefik middleware configuration
pages-server:
  plugin:
    pages-server:
      redisPoolSize: 10          # Idle connections per cache (default: 10)
      redisMaxConnections: 20    # Max connections per cache (default: 20)
      redisConnWaitTimeout: 5    # Wait timeout in seconds (default: 5)

Notes:

The plugin creates 3 cache instances, so total max = 3 × redisMaxConnections
Default: 60 max connections (3 × 20)
Increase redisMaxConnections if you have high concurrent load
Increase redisConnWaitTimeout to wait longer before falling back to memory cache
Ensure Redis maxclients > total max connections from all Traefik instances

Increase Redis maxclients:

# Temporarily (resets on restart)
docker exec redis redis-cli CONFIG SET maxclients 1000

# Permanently (in redis.conf)
maxclients 1000

Best Practices

Production Recommendations

Enable Redis/Valkey: Don't rely on in-memory cache in production
Set Appropriate TTLs: Balance freshness vs performance (default 300s is good)
Monitor Memory: Set up alerts for high memory usage
Regular Cleanup: Run cache reaper script daily or weekly
Backup Strategy: Redis persistence (RDB/AOF) for critical mappings
Security: Use password authentication for Redis/Valkey

Cache Invalidation Strategy

When to manually invalidate:

After repository content changes
After .pages file modifications
After password changes
When custom domain configuration changes

Automatic invalidation:

File cache expires after TTL (default 300s)
Directory listings expire after 60s
Password hashes expire after 60s

Performance Tuning

Optimize cache hit rate:

Increase TTL for rarely-changing content
Pre-warm cache for popular pages
Monitor cache hit/miss ratio

Optimize memory usage:

Set maxmemory limit in Redis
Enable eviction policy (e.g., allkeys-lru)
Compress large cached values

Reference Commands

Quick Reference Card

# === INSPECTION ===
# List all keys
docker exec redis redis-cli KEYS "*"

# Scan keys (production-safe)
docker exec redis redis-cli --scan

# Get key value
docker exec redis redis-cli GET key-name

# Check if key exists
docker exec redis redis-cli EXISTS key-name

# === DELETION ===
# Delete single key
docker exec redis redis-cli DEL key-name

# Delete by pattern
docker exec redis sh -c "redis-cli --scan --pattern 'pattern:*' | xargs -r redis-cli DEL"

# Flush entire cache
docker exec redis redis-cli FLUSHDB

# === MONITORING ===
# Real-time monitoring
docker exec redis redis-cli MONITOR

# Cache statistics
docker exec redis redis-cli INFO stats

# Memory usage
docker exec redis redis-cli INFO memory

# === MAINTENANCE ===
# Count keys
docker exec redis redis-cli DBSIZE

# Find large keys
docker exec redis redis-cli --bigkeys

# Test connection
docker exec redis redis-cli PING

Additional Resources

Configuration Guide - Cache configuration options
Cache Reaper - Automated cache cleanup
Troubleshooting - Common issues and solutions
Redis Documentation - Official Redis docs
Valkey Documentation - Official Valkey docs

Support

If you encounter cache-related issues:

Check this guide's troubleshooting section
Review Traefik logs: docker logs traefik
Review Redis/Valkey logs: docker logs redis
Report an issue with detailed logs