qsgen3/how-it-works.md

How qsgen3 Works

Table of Contents

1. Philosophy and Design Principles
2. Project Structure
3. Configuration System
4. Theme System
5. Creating New Themes
6. Content Processing Pipeline
7. Static File Handling
8. Template System
9. Output Generation
10. Command Line Interface
11. Dependencies and Requirements
12. Detailed Workflow
13. Troubleshooting and Debugging

Philosophy and Design Principles

Core Philosophy

qsgen3 is designed to be 100% design-agnostic. It does not impose any specific CSS frameworks, JavaScript libraries, or HTML structures on users. The generator's role is strictly to:

1. Process Markdown content into HTML
2. Combine content with user-chosen templates and styling
3. Generate a complete static site structure

Key Principles

• Minimal Dependencies: Only requires Pandoc for content processing
• In-Memory Operations: All content manipulation occurs in memory to improve performance and reduce storage wear
• Flexible Theme System: Supports easy switching between themes via configuration
• Template Agnostic: Works with any Pandoc-compatible HTML templates
• No Forced Assets: Only automatically links the main theme CSS; all other asset inclusion is explicit

Project Structure

A typical qsgen3 project follows this structure:

project-root/
├── bin/
│   └── qsgen3                 # Main generator script
├── site.conf                 # Main configuration file
├── content/                  # Markdown content
│   ├── posts/               # Blog posts
│   │   └── hello-world.md
│   └── pages/               # Static pages
├── layouts/                 # Pandoc HTML templates
│   ├── index.html          # Homepage template
│   ├── post.html           # Blog post template
│   └── rss.xml             # RSS feed template
├── static/                  # Static assets (CSS, images, etc.)
│   ├── css/
│   └── images/
├── themes/                  # Theme directory
│   └── theme-name/         # Individual theme
│       ├── layouts/        # Theme-specific templates (optional)
│       ├── static/         # Theme static assets (optional)
│       └── css/            # Alternative theme asset location
└── output/                 # Generated site (created by qsgen3)
    ├── index.html
    ├── rss.xml
    ├── posts/
    └── static/

Configuration System

Primary Configuration: site.conf

The site.conf file uses a simple key-value format:

# Site Metadata
site_name="My Awesome Site"
site_tagline="A brief description of my site"
site_url="http://localhost:8000"

# Theme Configuration
site_theme="minimal"
site_theme_css_file="css/style.css"

# Directory Paths
paths_content_dir="content"
paths_output_dir="output"
paths_layouts_dir="layouts"
paths_static_dir="static"

# Build Options
build_options_generate_rss=true
build_options_generate_sitemap=true
build_options_process_drafts=false

Configuration Loading Process

1. File Location: Defaults to $PROJECT_ROOT/site.conf
2. Override: Can be specified with -c <file> or --config <file>
3. Parsing: Simple line-by-line parsing of key="value" pairs
4. Storage: Values stored in QSG_CONFIG associative array
5. Validation: Basic validation for required keys and file existence

Key Configuration Variables

• site_theme: Name of the active theme (directory name in themes/)
• site_theme_css_file: Path to main CSS file relative to theme's static assets
• site_url: Base URL for the site (used in RSS and absolute links)
• paths_*: Directory paths (can be relative or absolute)
• build_options_*: Boolean flags for optional features

Theme System

Theme Architecture

Themes in qsgen3 provide two main components:

1. Templates: Pandoc HTML templates for different page types
2. Static Assets: CSS, JavaScript, images, and other resources

Theme Directory Structure

themes/theme-name/
├── layouts/              # Optional: Override default templates
│   ├── index.html       # Homepage template
│   ├── post.html        # Post template
│   └── rss.xml          # RSS template
├── static/              # Preferred: Standard static assets location
│   ├── css/
│   ├── js/
│   └── images/
└── css/                 # Alternative: Direct CSS location
    └── style.css

Theme Resolution Logic

1. Theme Selection: Based on site_theme in site.conf
2. Layout Override: If themes/theme-name/layouts/ exists, it overrides paths_layouts_dir
3. Static Asset Source:
   • First checks for themes/theme-name/static/
   • Falls back to themes/theme-name/ (for themes with assets at root level)
4. CSS File Location: Determined by site_theme_css_file relative to theme's static source

Theme Switching

Switching themes is accomplished by:

1. Changing site_theme in site.conf
2. Updating site_theme_css_file to match the new theme's CSS structure
3. Running qsgen3 to regenerate the site

Creating New Themes

Theme Development Overview

Creating a new theme for qsgen3 involves designing templates and static assets that work together to provide a cohesive visual and functional experience. Themes can range from minimal CSS-only styling to complex designs with custom layouts and interactive elements.

Step-by-Step Theme Creation

1. Create Theme Directory Structure

Start by creating a new directory in the themes/ folder:

mkdir -p themes/my-theme
cd themes/my-theme

Choose one of these organizational approaches:

Option A: Standard Structure (Recommended)

themes/my-theme/
├── layouts/              # Custom templates (optional)
│   ├── index.html       # Homepage template
│   ├── post.html        # Blog post template
│   └── rss.xml          # RSS feed template
└── static/              # Static assets
    ├── css/
    │   └── style.css    # Main stylesheet
    ├── js/              # JavaScript files (optional)
    │   └── theme.js
    └── images/          # Theme images (optional)
        └── logo.png

Option B: CSS-Only Structure

themes/my-theme/
└── css/
    └── style.css        # Main stylesheet only

2. Create the Main Stylesheet

Create your theme's primary CSS file. This is the only asset that qsgen3 will automatically link:

/* themes/my-theme/static/css/style.css */

/* Reset and base styles */
* {
    margin: 0;
    padding: 0;
    box-sizing: border-box;
}

body {
    font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
    line-height: 1.6;
    color: #333;
    background-color: #fff;
}

/* Header styles */
header {
    background: #2c3e50;
    color: white;
    padding: 1rem 0;
    margin-bottom: 2rem;
}

header h1 {
    max-width: 800px;
    margin: 0 auto;
    padding: 0 1rem;
}

/* Main content */
main {
    max-width: 800px;
    margin: 0 auto;
    padding: 0 1rem;
}

/* Post styles */
article {
    margin-bottom: 3rem;
    padding-bottom: 2rem;
    border-bottom: 1px solid #eee;
}

article h1, article h2 {
    color: #2c3e50;
    margin-bottom: 0.5rem;
}

article .meta {
    color: #666;
    font-size: 0.9rem;
    margin-bottom: 1rem;
}

/* Navigation and links */
nav ul {
    list-style: none;
    display: flex;
    gap: 1rem;
}

nav a {
    color: #3498db;
    text-decoration: none;
}

nav a:hover {
    text-decoration: underline;
}

/* Responsive design */
@media (max-width: 768px) {
    main {
        padding: 0 0.5rem;
    }
    
    nav ul {
        flex-direction: column;
        gap: 0.5rem;
    }
}

3. Create Custom Templates (Optional)

If you want to override the default templates, create custom Pandoc templates:

Homepage Template (layouts/index.html)

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>$site_name$</title>
    $if(css)$<link rel="stylesheet" href="$css$">$endif$
</head>
<body>
    <header>
        <h1>$site_name$</h1>
        $if(site_tagline)$<p>$site_tagline$</p>$endif$
    </header>
    
    <main>
        <section class="posts">
            <h2>Recent Posts</h2>
            $if(posts)$
            <ul class="post-list">
            $for(posts)$
                <li class="post-item">
                    <h3><a href="$it.post_url$">$it.post_title$</a></h3>
                    <div class="meta">
                        $if(it.post_date)$<time>$it.post_date$</time>$endif$
                        $if(it.post_author)$ by $it.post_author$$endif$
                    </div>
                    $if(it.post_description)$<p>$it.post_description$</p>$endif$
                </li>
            $endfor$
            </ul>
            $else$
            <p>No posts available.</p>
            $endif$
        </section>
    </main>
    
    <footer>
        <p>&copy; 2024 $site_name$. Generated with qsgen3.</p>
    </footer>
</body>
</html>

Post Template (layouts/post.html)

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>$title$ - $site_name$</title>
    $if(css)$<link rel="stylesheet" href="$css$">$endif$
    $if(description)$<meta name="description" content="$description$">$endif$
</head>
<body>
    <header>
        <h1><a href="/">$site_name$</a></h1>
        $if(site_tagline)$<p>$site_tagline$</p>$endif$
    </header>
    
    <main>
        <article>
            <header class="post-header">
                <h1>$title$</h1>
                <div class="meta">
                    $if(date)$<time datetime="$date$">$date$</time>$endif$
                    $if(author)$ by <span class="author">$author$</span>$endif$
                </div>
            </header>
            
            <div class="post-content">
                $body$
            </div>
        </article>
        
        <nav class="post-nav">
            <a href="/">&larr; Back to Home</a>
        </nav>
    </main>
    
    <footer>
        <p>&copy; 2024 $site_name$. Generated with qsgen3.</p>
    </footer>
</body>
</html>

4. Add JavaScript (Optional)

If your theme requires JavaScript functionality, add it to the static assets:

// themes/my-theme/static/js/theme.js

document.addEventListener('DOMContentLoaded', function() {
    // Add smooth scrolling
    document.querySelectorAll('a[href^="#"]').forEach(anchor => {
        anchor.addEventListener('click', function (e) {
            e.preventDefault();
            const target = document.querySelector(this.getAttribute('href'));
            if (target) {
                target.scrollIntoView({
                    behavior: 'smooth'
                });
            }
        });
    });
    
    // Add copy code button functionality
    document.querySelectorAll('pre code').forEach(block => {
        const button = document.createElement('button');
        button.textContent = 'Copy';
        button.className = 'copy-button';
        button.addEventListener('click', () => {
            navigator.clipboard.writeText(block.textContent);
            button.textContent = 'Copied!';
            setTimeout(() => button.textContent = 'Copy', 2000);
        });
        block.parentNode.appendChild(button);
    });
});

Important: Remember that qsgen3 will not automatically include JavaScript files. You must add <script> tags to your templates:

<!-- Add to your template's <head> or before </body> -->
<script src="/static/js/theme.js"></script>

5. Configure Theme Usage

Update your site.conf to use the new theme:

# Theme Configuration
site_theme="my-theme"
site_theme_css_file="css/style.css"  # Path relative to theme's static source

For CSS-only themes using the alternative structure:

site_theme="my-theme"
site_theme_css_file="style.css"  # Direct path if CSS is at theme root

6. Test Your Theme

Generate your site to test the theme:

./bin/qsgen3

Check the output:

• Verify CSS is applied correctly
• Test responsive design on different screen sizes
• Validate HTML structure
• Check that all assets are copied correctly

Theme Development Best Practices

CSS Guidelines

1. Use Relative Units: Prefer rem, em, and percentages over fixed pixels
2. Mobile-First Design: Start with mobile styles, then add desktop enhancements
3. Semantic Selectors: Use class names that describe content, not appearance
4. CSS Custom Properties: Use CSS variables for consistent theming

:root {
    --primary-color: #2c3e50;
    --secondary-color: #3498db;
    --text-color: #333;
    --background-color: #fff;
    --border-color: #eee;
    --font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
}

body {
    color: var(--text-color);
    background-color: var(--background-color);
    font-family: var(--font-family);
}

Template Guidelines

1. Conditional Content: Use Pandoc's conditional syntax for optional elements
2. Semantic HTML: Use appropriate HTML5 semantic elements
3. Accessibility: Include proper ARIA labels and alt text
4. Meta Tags: Include essential meta tags for SEO and social sharing

<!-- Good conditional usage -->
$if(description)$<meta name="description" content="$description$">$endif$
$if(author)$<meta name="author" content="$author$">$endif$

<!-- Semantic structure -->
<main role="main">
    <article>
        <header>
            <h1>$title$</h1>
        </header>
        <div class="content">
            $body$
        </div>
    </article>
</main>

Asset Organization

1. Logical Structure: Group related assets in appropriate directories
2. Naming Conventions: Use consistent, descriptive file names
3. Optimization: Optimize images and minimize CSS/JS when possible
4. Dependencies: Document any external dependencies clearly

Advanced Theme Features

Dark Mode Support

Add CSS custom properties and media queries for dark mode:

:root {
    --bg-color: #fff;
    --text-color: #333;
    --border-color: #eee;
}

@media (prefers-color-scheme: dark) {
    :root {
        --bg-color: #1a1a1a;
        --text-color: #e0e0e0;
        --border-color: #333;
    }
}

body {
    background-color: var(--bg-color);
    color: var(--text-color);
    transition: background-color 0.3s ease, color 0.3s ease;
}

Print Styles

Include print-specific styles:

@media print {
    header, footer, nav {
        display: none;
    }
    
    body {
        font-size: 12pt;
        line-height: 1.4;
    }
    
    a[href]:after {
        content: " (" attr(href) ")";
    }
}

Custom Fonts

If using custom fonts, include them properly:

/* Load fonts */
@import url('https://fonts.googleapis.com/css2?family=Inter:wght@400;600;700&display=swap');

/* Or use local fonts */
@font-face {
    font-family: 'CustomFont';
    src: url('/static/fonts/custom-font.woff2') format('woff2'),
         url('/static/fonts/custom-font.woff') format('woff');
    font-display: swap;
}

Theme Distribution

Documentation

Create a README.md for your theme:

# My Theme

A clean, minimal theme for qsgen3.

## Features
- Responsive design
- Dark mode support
- Clean typography
- Fast loading

## Installation
1. Copy theme to `themes/my-theme/`
2. Update `site.conf`:

site_theme="my-theme" site_theme_css_file="css/style.css"

3. Run `./bin/qsgen3`

## Customization
- Edit CSS custom properties in `style.css`
- Modify templates in `layouts/` directory
- Add custom JavaScript in `static/js/`

## Browser Support
- Modern browsers (Chrome 90+, Firefox 88+, Safari 14+)
- Graceful degradation for older browsers

Version Control

If sharing your theme:

1. Use semantic versioning
2. Tag releases appropriately
3. Include a changelog
4. Provide example configurations

Troubleshooting Theme Development

Common Issues

1. CSS Not Loading: Check site_theme_css_file path matches actual file location
2. Templates Not Found: Ensure template files are in layouts/ directory
3. Assets Missing: Verify static files are in correct directory structure
4. JavaScript Errors: Remember to include <script> tags in templates

Testing Checklist

• CSS loads correctly on all page types
• Templates render without Pandoc errors
• Responsive design works on mobile devices
• All static assets are accessible
• JavaScript functionality works (if applicable)
• Print styles are appropriate
• Accessibility standards are met
• Performance is acceptable

Content Processing Pipeline

Markdown Processing

1. Discovery: Recursively scans paths_content_dir for .md files
2. Metadata Extraction: Parses YAML front matter for post metadata
3. Content Conversion: Uses Pandoc to convert Markdown to HTML
4. Template Application: Applies appropriate template based on content type
5. Output Generation: Writes processed HTML to corresponding location in output/

Content Types

• Posts: Files in content/posts/ → output/posts/
• Pages: Files in content/pages/ → output/
• Index: Generated from post metadata → output/index.html
• RSS: Generated from post metadata → output/rss.xml

Metadata Handling

Each Markdown file can include YAML front matter:

---
title: "Post Title"
date: "2023-01-01"
author: "Author Name"
description: "Post description"
draft: false
---

# Post Content

Your markdown content here...

Static File Handling

Copy Strategy

Static files are copied in a specific order to handle theme overrides:

1. Root Static Files: Copy from paths_static_dir to output/static/
2. Theme Static Files: Copy from theme's static source to output/static/
3. Override Behavior: Theme files overwrite root files with same names

Copy Implementation

• Primary Tool: rsync with -av --delete flags
• Fallback: cp -R if rsync is unavailable
• Preservation: Maintains directory structure and file permissions

CSS File Linking

1. Availability: Theme CSS files are copied to output/static/
2. Verification: Script checks for CSS file existence after copying
3. Pandoc Integration: CSS path passed to Pandoc via --css flag
4. Path Format: Uses site-root-relative paths (e.g., /static/css/style.css)

Template System

Template Types

qsgen3 uses Pandoc templates with specific purposes:

• index.html: Homepage template (receives post list metadata)
• post.html: Individual post template (receives post content and metadata)
• rss.xml: RSS feed template (receives post list for syndication)

Template Variables

Templates receive data through Pandoc's variable system:

Post Templates

• $title$: Post title from front matter
• $date$: Post date
• $author$: Post author
• $body$: Converted HTML content
• Custom variables from YAML front matter

Index Template

• $site_name$: From site.conf
• $site_tagline$: From site.conf
• $posts$: Array of post metadata for listing

RSS Template

• $site_url$: Base URL for absolute links
• $posts$: Array of post data with URLs and content

Template Resolution

1. Theme Override: If theme provides templates, use theme's layouts/
2. Default: Use project's layouts/ directory
3. Fallback: Error if required template not found

Output Generation

Directory Structure

Generated output maintains a clean, predictable structure:

output/
├── index.html           # Homepage
├── rss.xml             # RSS feed
├── posts/              # Individual post pages
│   └── post-name.html
├── static/             # All static assets
│   ├── css/           # Stylesheets
│   ├── js/            # JavaScript (if provided by theme)
│   └── images/        # Images and media
└── css/               # Legacy: Index-specific CSS location
    └── theme.css      # Copy of main theme CSS for index page

File Naming

• Posts: content/posts/hello-world.md → output/posts/hello-world.html
• Pages: content/pages/about.md → output/about.html
• Index: Generated → output/index.html
• RSS: Generated → output/rss.xml

URL Structure

• Posts: /posts/post-name.html
• Pages: /page-name.html
• Static Assets: /static/path/to/asset
• CSS: /static/css/style.css (for posts), /css/theme.css (for index)

Command Line Interface

Basic Usage

./bin/qsgen3 [options]

Available Options

• -h, --help: Display usage information and exit
• -V, --version: Show script name and version, then exit
• -c <file>, --config <file>: Specify custom configuration file path

Path Resolution

• PROJECT_ROOT: Defaults to current working directory ($PWD)
• CONFIG_FILE: Defaults to $PROJECT_ROOT/site.conf
• Relative Paths: Configuration file path can be relative to project root

Exit Codes

• 0: Successful generation
• 1: Error (missing dependencies, configuration issues, processing failures)

Dependencies and Requirements

Required Dependencies

• Pandoc: Core dependency for Markdown processing and HTML generation
• Zsh: Shell interpreter (script written in Zsh)

Optional Dependencies

• rsync: Preferred tool for efficient file copying (falls back to cp)

System Requirements

• Operating System: Linux/Unix-like systems
• File System: Support for standard Unix file permissions
• Memory: Minimal requirements (all processing in memory)

Environment Setup

The script configures a consistent environment:

LC_ALL=C
LANG=C
umask 0022

Detailed Workflow

1. Initialization Phase

Start qsgen3
├── Parse command line arguments
├── Set PROJECT_ROOT (default: $PWD)
├── Determine CONFIG_FILE path
├── Set environment variables (LC_ALL, LANG, umask)
└── Initialize QSG_CONFIG array

2. Configuration Loading

Load Configuration
├── Check if CONFIG_FILE exists
├── Parse key="value" pairs line by line
├── Strip quotes from values
├── Store in QSG_CONFIG associative array
└── Validate required configuration keys

3. Dependency Checking

Check Dependencies
├── Verify Pandoc is available
├── Check Pandoc version compatibility
├── Verify other required tools
└── Exit with error if dependencies missing

4. Theme Processing

Process Theme Configuration
├── Read site_theme from configuration
├── Determine theme base path: themes/$site_theme
├── Check for theme layouts directory
│   ├── If exists: Override paths_layouts_dir
│   └── If not: Use default layouts
├── Determine theme static source
│   ├── Check themes/$site_theme/static/
│   ├── Fallback to themes/$site_theme/
│   └── Set QSG_CONFIG[theme_static_source_dir]
└── Log theme processing decisions

5. Output Preparation

Prepare Output Directory
├── Clean existing output directory
├── Create fresh output directory structure
└── Prepare for static file copying

6. Static File Processing

Copy Static Files
├── Copy from paths_static_dir to output/static/
│   ├── Use rsync -av --delete if available
│   └── Fallback to cp -R
├── Copy from theme static source to output/static/
│   ├── Theme files overwrite root files
│   └── Preserve directory structure
└── Log copy operations and results

7. CSS Path Determination

Determine CSS Linking
├── Read site_theme_css_file from configuration
├── Construct expected CSS file path in output/static/
├── Verify CSS file exists after copying
├── Set QSG_CONFIG[pandoc_css_path_arg] for Pandoc
└── Log CSS path decisions and warnings

8. Content Processing

Process Markdown Content
├── Scan paths_content_dir recursively for .md files
├── For each Markdown file:
│   ├── Extract YAML front matter
│   ├── Determine output path and template
│   ├── Run Pandoc with appropriate template and CSS
│   ├── Write generated HTML to output directory
│   └── Log processing results
└── Collect metadata for index and RSS generation

9. Index Generation

Generate Index Page
├── Collect all post metadata
├── Create YAML metadata file for Pandoc
├── Run Pandoc with index template
├── Apply CSS styling
├── Write output/index.html
└── Clean up temporary files

10. RSS Generation

Generate RSS Feed
├── Collect post metadata with URLs
├── Create YAML metadata for RSS template
├── Run Pandoc with RSS template
├── Generate absolute URLs using site_url
├── Write output/rss.xml
└── Clean up temporary files

11. Finalization

Complete Generation
├── Log final directory structure
├── Report generation success
├── Clean up any remaining temporary files
└── Exit with status code 0

Troubleshooting and Debugging

Common Issues

1. CSS Not Applied

Symptoms: Generated HTML doesn't show theme styling Causes:

• Incorrect site_theme_css_file path in site.conf
• CSS file doesn't exist in theme's static assets
• Theme static directory structure mismatch

Solutions:

• Verify CSS file path relative to theme's static source
• Check theme directory structure
• Enable debug logging to trace CSS path resolution

2. Theme Not Found

Symptoms: Warning about theme directory not found Causes:

• Typo in site_theme configuration
• Theme directory doesn't exist
• Incorrect theme directory structure

Solutions:

• Verify theme name spelling in site.conf
• Check themes/ directory exists and contains named theme
• Ensure theme directory has expected structure

3. Template Errors

Symptoms: Pandoc errors during HTML generation Causes:

• Missing required templates
• Template syntax errors
• Incompatible template variables

Solutions:

• Verify all required templates exist
• Check Pandoc template syntax
• Review template variable usage

4. Static File Copy Issues

Symptoms: Assets missing from output directory Causes:

• Permission issues
• Disk space problems
• Path resolution errors

Solutions:

• Check file permissions
• Verify available disk space
• Review path configurations for absolute vs. relative paths

Debug Logging

Enable detailed logging by modifying the _log function or adding debug statements:

# Enable debug logging
QSG_DEBUG=1 ./bin/qsgen3

Path Debugging

The script includes path resolution logic to handle both relative and absolute paths. If experiencing path issues:

1. Check that PROJECT_ROOT is correctly set
2. Verify configuration paths are relative to project root
3. Review log messages for path construction details

Configuration Validation

Ensure site.conf follows the correct format:

• Use double quotes for values: key="value"
• No spaces around the equals sign
• One configuration per line
• Comments start with #

---

This document reflects the current implementation of qsgen3 and its design philosophy of remaining completely design-agnostic while providing flexible theme and content management capabilities.