Stig-Ørjan Smelror
f72fe18873
- Add sophisticated file preservation functionality to _clean_output_dir() - Support .qsgen3_preserve file with shell glob patterns for selective preservation - Preserve shared articles and important files during site regeneration - Maintain backward compatibility (no preserve file = complete cleaning) - Add comprehensive documentation in how-it-works.md - Include example preserve file (.qsgen3_preserve.example) - Remove legacy images directory and update documentation - Fix workflow documentation formatting issues
1091 lines
30 KiB
Markdown
1091 lines
30 KiB
Markdown
# How qsgen3 Works
|
|
|
|
## Table of Contents
|
|
|
|
1. [Philosophy and Design Principles](#philosophy-and-design-principles)
|
|
2. [Project Structure](#project-structure)
|
|
3. [Configuration System](#configuration-system)
|
|
4. [Theme System](#theme-system)
|
|
5. [Creating New Themes](#creating-new-themes)
|
|
6. [Content Processing Pipeline](#content-processing-pipeline)
|
|
7. [Static File Handling](#static-file-handling)
|
|
8. [Template System](#template-system)
|
|
9. [Output Generation](#output-generation)
|
|
10. [Command Line Interface](#command-line-interface)
|
|
11. [Dependencies and Requirements](#dependencies-and-requirements)
|
|
12. [Detailed Workflow](#detailed-workflow)
|
|
13. [Troubleshooting and Debugging](#troubleshooting-and-debugging)
|
|
|
|
## Philosophy and Design Principles
|
|
|
|
### Core Philosophy
|
|
qsgen3 is designed to be **100% design-agnostic**. It does not impose any specific CSS frameworks, JavaScript libraries, or HTML structures on users. The generator's role is strictly to:
|
|
|
|
1. Process Markdown content into HTML
|
|
2. Combine content with user-chosen templates and styling
|
|
3. Generate a complete static site structure
|
|
|
|
### Key Principles
|
|
|
|
- **Minimal Dependencies**: Only requires Pandoc for content processing
|
|
- **In-Memory Operations**: All content manipulation occurs in memory to improve performance and reduce storage wear
|
|
- **Flexible Theme System**: Supports easy switching between themes via configuration
|
|
- **Template Agnostic**: Works with any Pandoc-compatible HTML templates
|
|
- **No Forced Assets**: Only automatically links the main theme CSS; all other asset inclusion is explicit
|
|
|
|
## Project Structure
|
|
|
|
A typical qsgen3 project follows this structure:
|
|
|
|
```
|
|
project-root/
|
|
├── bin/
|
|
│ └── qsgen3 # Main generator script
|
|
├── site.conf # Main configuration file
|
|
├── .qsgen3_preserve # Optional: File preservation patterns
|
|
├── content/ # Markdown content
|
|
│ ├── posts/ # Blog posts
|
|
│ │ └── hello-world.md
|
|
│ └── pages/ # Static pages
|
|
├── layouts/ # Pandoc HTML templates
|
|
│ ├── index.html # Homepage template
|
|
│ ├── post.html # Blog post template
|
|
│ └── rss.xml # RSS feed template
|
|
├── static/ # Static assets (CSS, images, etc.)
|
|
│ ├── css/
|
|
│ └── images/
|
|
├── themes/ # Theme directory
|
|
│ └── theme-name/ # Individual theme
|
|
│ ├── layouts/ # Theme-specific templates (optional)
|
|
│ └── css/ # Alternative theme asset location
|
|
└── output/ # Generated site (created by qsgen3)
|
|
├── index.html
|
|
├── rss.xml
|
|
├── posts/
|
|
└── static/
|
|
```
|
|
|
|
## Configuration System
|
|
|
|
### Primary Configuration: `site.conf`
|
|
|
|
The `site.conf` file uses a simple key-value format:
|
|
|
|
```bash
|
|
# Site Metadata
|
|
site_name="My Awesome Site"
|
|
site_tagline="A brief description of my site"
|
|
site_url="http://localhost:8000"
|
|
|
|
# Theme Configuration
|
|
site_theme="minimal"
|
|
site_theme_css_file="css/style.css"
|
|
|
|
# Directory Paths
|
|
paths_content_dir="content"
|
|
paths_output_dir="output"
|
|
paths_layouts_dir="layouts"
|
|
paths_static_dir="static"
|
|
|
|
# Build Options
|
|
build_options_generate_rss=true
|
|
build_options_generate_sitemap=true
|
|
build_options_process_drafts=false
|
|
```
|
|
|
|
### Configuration Loading Process
|
|
|
|
1. **File Location**: Defaults to `$PROJECT_ROOT/site.conf`
|
|
2. **Override**: Can be specified with `-c <file>` or `--config <file>`
|
|
3. **Parsing**: Simple line-by-line parsing of `key="value"` pairs
|
|
4. **Storage**: Values stored in `QSG_CONFIG` associative array
|
|
5. **Validation**: Basic validation for required keys and file existence
|
|
|
|
### Key Configuration Variables
|
|
|
|
- **`site_theme`**: Name of the active theme (directory name in `themes/`)
|
|
- **`site_theme_css_file`**: Path to main CSS file relative to theme's static assets
|
|
- **`site_url`**: Base URL for the site (used in RSS and absolute links)
|
|
- **`paths_*`**: Directory paths (can be relative or absolute)
|
|
- **`build_options_*`**: Boolean flags for optional features
|
|
|
|
## Theme System
|
|
|
|
### Theme Architecture
|
|
|
|
Themes in qsgen3 provide two main components:
|
|
|
|
1. **Templates**: Pandoc HTML templates for different page types
|
|
2. **Static Assets**: CSS, JavaScript, images, and other resources
|
|
|
|
### Theme Directory Structure
|
|
|
|
```
|
|
themes/theme-name/
|
|
├── layouts/ # Optional: Override default templates
|
|
│ ├── index.html # Homepage template
|
|
│ ├── post.html # Post template
|
|
│ └── rss.xml # RSS template
|
|
├── static/ # Preferred: Standard static assets location
|
|
│ ├── css/
|
|
│ └── js/
|
|
└── css/ # Alternative: Direct CSS location
|
|
└── style.css
|
|
```
|
|
|
|
### Theme Resolution Logic
|
|
|
|
1. **Theme Selection**: Based on `site_theme` in `site.conf`
|
|
2. **Layout Override**: If `themes/theme-name/layouts/` exists, it overrides `paths_layouts_dir`
|
|
3. **Static Asset Source**:
|
|
- First checks for `themes/theme-name/static/`
|
|
- Falls back to `themes/theme-name/` (for themes with assets at root level)
|
|
4. **CSS File Location**: Determined by `site_theme_css_file` relative to theme's static source
|
|
|
|
### Theme Switching
|
|
|
|
Switching themes is accomplished by:
|
|
|
|
1. Changing `site_theme` in `site.conf`
|
|
2. Updating `site_theme_css_file` to match the new theme's CSS structure
|
|
3. Running qsgen3 to regenerate the site
|
|
|
|
## Creating New Themes
|
|
|
|
### Theme Development Overview
|
|
|
|
Creating a new theme for qsgen3 involves designing templates and static assets that work together to provide a cohesive visual and functional experience. Themes can range from minimal CSS-only styling to complex designs with custom layouts and interactive elements.
|
|
|
|
### Step-by-Step Theme Creation
|
|
|
|
#### 1. Create Theme Directory Structure
|
|
|
|
Start by creating a new directory in the `themes/` folder:
|
|
|
|
```bash
|
|
mkdir -p themes/my-theme
|
|
cd themes/my-theme
|
|
```
|
|
|
|
Choose one of these organizational approaches:
|
|
|
|
**Option A: Standard Structure (Recommended)**
|
|
```
|
|
themes/my-theme/
|
|
├── layouts/ # Custom templates (optional)
|
|
│ ├── index.html # Homepage template
|
|
│ ├── post.html # Blog post template
|
|
│ └── rss.xml # RSS feed template
|
|
└── static/ # Static assets
|
|
├── css/
|
|
│ └── style.css # Main stylesheet
|
|
└── js/ # JavaScript files (optional)
|
|
```
|
|
|
|
**Option B: CSS-Only Structure**
|
|
```
|
|
themes/my-theme/
|
|
└── css/
|
|
└── style.css # Main stylesheet only
|
|
```
|
|
|
|
#### 2. Create the Main Stylesheet
|
|
|
|
Create your theme's primary CSS file. This is the only asset that qsgen3 will automatically link:
|
|
|
|
```css
|
|
/* themes/my-theme/static/css/style.css */
|
|
|
|
/* Reset and base styles */
|
|
* {
|
|
margin: 0;
|
|
padding: 0;
|
|
box-sizing: border-box;
|
|
}
|
|
|
|
body {
|
|
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
|
|
line-height: 1.6;
|
|
color: #333;
|
|
background-color: #fff;
|
|
}
|
|
|
|
/* Header styles */
|
|
header {
|
|
background: #2c3e50;
|
|
color: white;
|
|
padding: 1rem 0;
|
|
margin-bottom: 2rem;
|
|
}
|
|
|
|
header h1 {
|
|
max-width: 800px;
|
|
margin: 0 auto;
|
|
padding: 0 1rem;
|
|
}
|
|
|
|
/* Main content */
|
|
main {
|
|
max-width: 800px;
|
|
margin: 0 auto;
|
|
padding: 0 1rem;
|
|
}
|
|
|
|
/* Post styles */
|
|
article {
|
|
margin-bottom: 3rem;
|
|
padding-bottom: 2rem;
|
|
border-bottom: 1px solid #eee;
|
|
}
|
|
|
|
article h1, article h2 {
|
|
color: #2c3e50;
|
|
margin-bottom: 0.5rem;
|
|
}
|
|
|
|
article .meta {
|
|
color: #666;
|
|
font-size: 0.9rem;
|
|
margin-bottom: 1rem;
|
|
}
|
|
|
|
/* Navigation and links */
|
|
nav ul {
|
|
list-style: none;
|
|
display: flex;
|
|
gap: 1rem;
|
|
}
|
|
|
|
nav a {
|
|
color: #3498db;
|
|
text-decoration: none;
|
|
}
|
|
|
|
nav a:hover {
|
|
text-decoration: underline;
|
|
}
|
|
|
|
/* Responsive design */
|
|
@media (max-width: 768px) {
|
|
main {
|
|
padding: 0 0.5rem;
|
|
}
|
|
|
|
nav ul {
|
|
flex-direction: column;
|
|
gap: 0.5rem;
|
|
}
|
|
}
|
|
```
|
|
|
|
#### 3. Create Custom Templates (Optional)
|
|
|
|
If you want to override the default templates, create custom Pandoc templates:
|
|
|
|
**Homepage Template (`layouts/index.html`)**
|
|
```html
|
|
<!DOCTYPE html>
|
|
<html lang="en">
|
|
<head>
|
|
<meta charset="UTF-8">
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
<title>$site_name$</title>
|
|
$if(css)$<link rel="stylesheet" href="$css$">$endif$
|
|
</head>
|
|
<body>
|
|
<header>
|
|
<h1>$site_name$</h1>
|
|
$if(site_tagline)$<p>$site_tagline$</p>$endif$
|
|
</header>
|
|
|
|
<main>
|
|
<section class="posts">
|
|
<h2>Recent Posts</h2>
|
|
$if(posts)$
|
|
<ul class="post-list">
|
|
$for(posts)$
|
|
<li class="post-item">
|
|
<h3><a href="$it.post_url$">$it.post_title$</a></h3>
|
|
<div class="meta">
|
|
$if(it.post_date)$<time>$it.post_date$</time>$endif$
|
|
$if(it.post_author)$ by $it.post_author$$endif$
|
|
</div>
|
|
$if(it.post_description)$<p>$it.post_description$</p>$endif$
|
|
</li>
|
|
$endfor$
|
|
</ul>
|
|
$else$
|
|
<p>No posts available.</p>
|
|
$endif$
|
|
</section>
|
|
</main>
|
|
|
|
<footer>
|
|
<p>© 2024 $site_name$. Generated with qsgen3.</p>
|
|
</footer>
|
|
</body>
|
|
</html>
|
|
```
|
|
|
|
**Post Template (`layouts/post.html`)**
|
|
```html
|
|
<!DOCTYPE html>
|
|
<html lang="en">
|
|
<head>
|
|
<meta charset="UTF-8">
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
<title>$title$ - $site_name$</title>
|
|
$if(css)$<link rel="stylesheet" href="$css$">$endif$
|
|
$if(description)$<meta name="description" content="$description$">$endif$
|
|
</head>
|
|
<body>
|
|
<header>
|
|
<h1><a href="/">$site_name$</a></h1>
|
|
$if(site_tagline)$<p>$site_tagline$</p>$endif$
|
|
</header>
|
|
|
|
<main>
|
|
<article>
|
|
<header class="post-header">
|
|
<h1>$title$</h1>
|
|
<div class="meta">
|
|
$if(date)$<time datetime="$date$">$date$</time>$endif$
|
|
$if(author)$ by <span class="author">$author$</span>$endif$
|
|
</div>
|
|
</header>
|
|
|
|
<div class="post-content">
|
|
$body$
|
|
</div>
|
|
</article>
|
|
|
|
<nav class="post-nav">
|
|
<a href="/">← Back to Home</a>
|
|
</nav>
|
|
</main>
|
|
|
|
<footer>
|
|
<p>© 2024 $site_name$. Generated with qsgen3.</p>
|
|
</footer>
|
|
</body>
|
|
</html>
|
|
```
|
|
|
|
#### 4. Add JavaScript (Optional)
|
|
|
|
If your theme requires JavaScript functionality, add it to the static assets:
|
|
|
|
```javascript
|
|
// themes/my-theme/static/js/theme.js
|
|
|
|
document.addEventListener('DOMContentLoaded', function() {
|
|
// Add smooth scrolling
|
|
document.querySelectorAll('a[href^="#"]').forEach(anchor => {
|
|
anchor.addEventListener('click', function (e) {
|
|
e.preventDefault();
|
|
const target = document.querySelector(this.getAttribute('href'));
|
|
if (target) {
|
|
target.scrollIntoView({
|
|
behavior: 'smooth'
|
|
});
|
|
}
|
|
});
|
|
});
|
|
|
|
// Add copy code button functionality
|
|
document.querySelectorAll('pre code').forEach(block => {
|
|
const button = document.createElement('button');
|
|
button.textContent = 'Copy';
|
|
button.className = 'copy-button';
|
|
button.addEventListener('click', () => {
|
|
navigator.clipboard.writeText(block.textContent);
|
|
button.textContent = 'Copied!';
|
|
setTimeout(() => button.textContent = 'Copy', 2000);
|
|
});
|
|
block.parentNode.appendChild(button);
|
|
});
|
|
});
|
|
```
|
|
|
|
**Important**: Remember that qsgen3 will not automatically include JavaScript files. You must add `<script>` tags to your templates:
|
|
|
|
```html
|
|
<!-- Add to your template's <head> or before </body> -->
|
|
<script src="/static/js/theme.js"></script>
|
|
```
|
|
|
|
#### 5. Configure Theme Usage
|
|
|
|
Update your `site.conf` to use the new theme:
|
|
|
|
```bash
|
|
# Theme Configuration
|
|
site_theme="my-theme"
|
|
site_theme_css_file="css/style.css" # Path relative to theme's static source
|
|
```
|
|
|
|
For CSS-only themes using the alternative structure:
|
|
```bash
|
|
site_theme="my-theme"
|
|
site_theme_css_file="style.css" # Direct path if CSS is at theme root
|
|
```
|
|
|
|
#### 6. Test Your Theme
|
|
|
|
Generate your site to test the theme:
|
|
|
|
```bash
|
|
./bin/qsgen3
|
|
```
|
|
|
|
Check the output:
|
|
- Verify CSS is applied correctly
|
|
- Test responsive design on different screen sizes
|
|
- Validate HTML structure
|
|
- Check that all assets are copied correctly
|
|
|
|
### Theme Development Best Practices
|
|
|
|
#### CSS Guidelines
|
|
|
|
1. **Use Relative Units**: Prefer `rem`, `em`, and percentages over fixed pixels
|
|
2. **Mobile-First Design**: Start with mobile styles, then add desktop enhancements
|
|
3. **Semantic Selectors**: Use class names that describe content, not appearance
|
|
4. **CSS Custom Properties**: Use CSS variables for consistent theming
|
|
|
|
```css
|
|
:root {
|
|
--primary-color: #2c3e50;
|
|
--secondary-color: #3498db;
|
|
--text-color: #333;
|
|
--background-color: #fff;
|
|
--border-color: #eee;
|
|
--font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
|
|
}
|
|
|
|
body {
|
|
color: var(--text-color);
|
|
background-color: var(--background-color);
|
|
font-family: var(--font-family);
|
|
}
|
|
```
|
|
|
|
#### Template Guidelines
|
|
|
|
1. **Conditional Content**: Use Pandoc's conditional syntax for optional elements
|
|
2. **Semantic HTML**: Use appropriate HTML5 semantic elements
|
|
3. **Accessibility**: Include proper ARIA labels and alt text
|
|
4. **Meta Tags**: Include essential meta tags for SEO and social sharing
|
|
|
|
```html
|
|
<!-- Good conditional usage -->
|
|
$if(description)$<meta name="description" content="$description$">$endif$
|
|
$if(author)$<meta name="author" content="$author$">$endif$
|
|
|
|
<!-- Semantic structure -->
|
|
<main role="main">
|
|
<article>
|
|
<header>
|
|
<h1>$title$</h1>
|
|
</header>
|
|
<div class="content">
|
|
$body$
|
|
</div>
|
|
</article>
|
|
</main>
|
|
```
|
|
|
|
#### Asset Organization
|
|
|
|
1. **Logical Structure**: Group related assets in appropriate directories
|
|
2. **Naming Conventions**: Use consistent, descriptive file names
|
|
3. **Optimization**: Optimize images and minimize CSS/JS when possible
|
|
4. **Dependencies**: Document any external dependencies clearly
|
|
|
|
### Advanced Theme Features
|
|
|
|
#### Dark Mode Support
|
|
|
|
Add CSS custom properties and media queries for dark mode:
|
|
|
|
```css
|
|
:root {
|
|
--bg-color: #fff;
|
|
--text-color: #333;
|
|
--border-color: #eee;
|
|
}
|
|
|
|
@media (prefers-color-scheme: dark) {
|
|
:root {
|
|
--bg-color: #1a1a1a;
|
|
--text-color: #e0e0e0;
|
|
--border-color: #333;
|
|
}
|
|
}
|
|
|
|
body {
|
|
background-color: var(--bg-color);
|
|
color: var(--text-color);
|
|
transition: background-color 0.3s ease, color 0.3s ease;
|
|
}
|
|
```
|
|
|
|
#### Print Styles
|
|
|
|
Include print-specific styles:
|
|
|
|
```css
|
|
@media print {
|
|
header, footer, nav {
|
|
display: none;
|
|
}
|
|
|
|
body {
|
|
font-size: 12pt;
|
|
line-height: 1.4;
|
|
}
|
|
|
|
a[href]:after {
|
|
content: " (" attr(href) ")";
|
|
}
|
|
}
|
|
```
|
|
|
|
#### Custom Fonts
|
|
|
|
If using custom fonts, include them properly:
|
|
|
|
```css
|
|
/* Load fonts */
|
|
@import url('https://fonts.googleapis.com/css2?family=Inter:wght@400;600;700&display=swap');
|
|
|
|
/* Or use local fonts */
|
|
@font-face {
|
|
font-family: 'CustomFont';
|
|
src: url('/static/fonts/custom-font.woff2') format('woff2'),
|
|
url('/static/fonts/custom-font.woff') format('woff');
|
|
font-display: swap;
|
|
}
|
|
```
|
|
|
|
### Theme Distribution
|
|
|
|
#### Documentation
|
|
|
|
Create a `README.md` for your theme:
|
|
|
|
```markdown
|
|
# My Theme
|
|
|
|
A clean, minimal theme for qsgen3.
|
|
|
|
## Features
|
|
- Responsive design
|
|
- Dark mode support
|
|
- Clean typography
|
|
- Fast loading
|
|
|
|
## Installation
|
|
1. Copy theme to `themes/my-theme/`
|
|
2. Update `site.conf`:
|
|
```
|
|
site_theme="my-theme"
|
|
site_theme_css_file="css/style.css"
|
|
```
|
|
3. Run `./bin/qsgen3`
|
|
|
|
## Customization
|
|
- Edit CSS custom properties in `style.css`
|
|
- Modify templates in `layouts/` directory
|
|
- Add custom JavaScript in `static/js/`
|
|
|
|
## Browser Support
|
|
- Modern browsers (Chrome 90+, Firefox 88+, Safari 14+)
|
|
- Graceful degradation for older browsers
|
|
```
|
|
|
|
#### Version Control
|
|
|
|
If sharing your theme:
|
|
1. Use semantic versioning
|
|
2. Tag releases appropriately
|
|
3. Include a changelog
|
|
4. Provide example configurations
|
|
|
|
### Troubleshooting Theme Development
|
|
|
|
#### Common Issues
|
|
|
|
1. **CSS Not Loading**: Check `site_theme_css_file` path matches actual file location
|
|
2. **Templates Not Found**: Ensure template files are in `layouts/` directory
|
|
3. **Assets Missing**: Verify static files are in correct directory structure
|
|
4. **JavaScript Errors**: Remember to include `<script>` tags in templates
|
|
|
|
#### Testing Checklist
|
|
|
|
- [ ] CSS loads correctly on all page types
|
|
- [ ] Templates render without Pandoc errors
|
|
- [ ] Responsive design works on mobile devices
|
|
- [ ] All static assets are accessible
|
|
- [ ] JavaScript functionality works (if applicable)
|
|
- [ ] Print styles are appropriate
|
|
- [ ] Accessibility standards are met
|
|
- [ ] Performance is acceptable
|
|
|
|
## Content Processing Pipeline
|
|
|
|
### Markdown Processing
|
|
|
|
1. **Discovery**: Recursively scans `paths_content_dir` for `.md` files
|
|
2. **Metadata Extraction**: Parses YAML front matter for post metadata
|
|
3. **Content Conversion**: Uses Pandoc to convert Markdown to HTML
|
|
4. **Template Application**: Applies appropriate template based on content type
|
|
5. **Output Generation**: Writes processed HTML to corresponding location in `output/`
|
|
|
|
### Content Types
|
|
|
|
- **Posts**: Files in `content/posts/` → `output/posts/`
|
|
- **Pages**: Files in `content/pages/` → `output/`
|
|
- **Index**: Generated from post metadata → `output/index.html`
|
|
- **RSS**: Generated from post metadata → `output/rss.xml`
|
|
|
|
### Metadata Handling
|
|
|
|
Each Markdown file can include YAML front matter:
|
|
|
|
```yaml
|
|
---
|
|
title: "Post Title"
|
|
date: "2023-01-01"
|
|
author: "Author Name"
|
|
description: "Post description"
|
|
draft: false
|
|
---
|
|
|
|
# Post Content
|
|
|
|
Your markdown content here...
|
|
```
|
|
|
|
## Static File Handling
|
|
|
|
### Copy Strategy
|
|
|
|
Static files are copied in a specific order to handle theme overrides:
|
|
|
|
1. **Root Static Files**: Copy from `paths_static_dir` to `output/static/`
|
|
2. **Theme Static Files**: Copy from theme's static source to `output/static/`
|
|
3. **Override Behavior**: Theme files overwrite root files with same names
|
|
|
|
### Copy Implementation
|
|
|
|
- **Primary Tool**: `rsync` with `-av --delete` flags
|
|
- **Fallback**: `cp -R` if rsync is unavailable
|
|
- **Preservation**: Maintains directory structure and file permissions
|
|
|
|
### CSS File Linking
|
|
|
|
1. **Availability**: Theme CSS files are copied to `output/static/`
|
|
2. **Verification**: Script checks for CSS file existence after copying
|
|
3. **Pandoc Integration**: CSS path passed to Pandoc via `--css` flag
|
|
4. **Path Format**: Uses site-root-relative paths (e.g., `/static/css/style.css`)
|
|
|
|
## Template System
|
|
|
|
### Template Types
|
|
|
|
qsgen3 uses Pandoc templates with specific purposes:
|
|
|
|
- **`index.html`**: Homepage template (receives post list metadata)
|
|
- **`post.html`**: Individual post template (receives post content and metadata)
|
|
- **`rss.xml`**: RSS feed template (receives post list for syndication)
|
|
|
|
### Template Variables
|
|
|
|
Templates receive data through Pandoc's variable system:
|
|
|
|
#### Post Templates
|
|
- `$title$`: Post title from front matter
|
|
- `$date$`: Post date
|
|
- `$author$`: Post author
|
|
- `$body$`: Converted HTML content
|
|
- Custom variables from YAML front matter
|
|
|
|
#### Index Template
|
|
- `$site_name$`: From site.conf
|
|
- `$site_tagline$`: From site.conf
|
|
- `$posts$`: Array of post metadata for listing
|
|
|
|
#### RSS Template
|
|
- `$site_url$`: Base URL for absolute links
|
|
- `$posts$`: Array of post data with URLs and content
|
|
|
|
### Template Resolution
|
|
|
|
1. **Theme Override**: If theme provides templates, use theme's `layouts/`
|
|
2. **Default**: Use project's `layouts/` directory
|
|
3. **Fallback**: Error if required template not found
|
|
|
|
## Output Generation
|
|
|
|
### Directory Structure
|
|
|
|
Generated output maintains a clean, predictable structure:
|
|
|
|
```
|
|
output/
|
|
├── index.html # Homepage
|
|
├── rss.xml # RSS feed
|
|
├── posts/ # Individual post pages
|
|
│ └── post-name.html
|
|
├── static/ # All static assets
|
|
│ ├── css/ # Stylesheets
|
|
│ └── js/ # JavaScript (if provided by theme)
|
|
└── css/ # Legacy: Index-specific CSS location
|
|
└── theme.css # Copy of main theme CSS for index page
|
|
```
|
|
|
|
### File Naming
|
|
|
|
- **Posts**: `content/posts/hello-world.md` → `output/posts/hello-world.html`
|
|
- **Pages**: `content/pages/about.md` → `output/about.html`
|
|
- **Index**: Generated → `output/index.html`
|
|
- **RSS**: Generated → `output/rss.xml`
|
|
|
|
### URL Structure
|
|
|
|
- **Posts**: `/posts/post-name.html`
|
|
- **Pages**: `/page-name.html`
|
|
- **Static Assets**: `/static/path/to/asset`
|
|
- **CSS**: `/static/css/style.css` (for posts), `/css/theme.css` (for index)
|
|
|
|
## Command Line Interface
|
|
|
|
### Basic Usage
|
|
|
|
```bash
|
|
./bin/qsgen3 [options]
|
|
```
|
|
|
|
### Available Options
|
|
|
|
- **`-h, --help`**: Display usage information and exit
|
|
- **`-V, --version`**: Show script name and version, then exit
|
|
- **`-c <file>, --config <file>`**: Specify custom configuration file path
|
|
|
|
### Path Resolution
|
|
|
|
- **`PROJECT_ROOT`**: Defaults to current working directory (`$PWD`)
|
|
- **`CONFIG_FILE`**: Defaults to `$PROJECT_ROOT/site.conf`
|
|
- **Relative Paths**: Configuration file path can be relative to project root
|
|
|
|
### Exit Codes
|
|
|
|
- **0**: Successful generation
|
|
- **1**: Error (missing dependencies, configuration issues, processing failures)
|
|
|
|
## Dependencies and Requirements
|
|
|
|
### Required Dependencies
|
|
|
|
- **Pandoc**: Core dependency for Markdown processing and HTML generation
|
|
- **Zsh**: Shell interpreter (script written in Zsh)
|
|
|
|
### Optional Dependencies
|
|
|
|
- **rsync**: Preferred tool for efficient file copying (falls back to `cp`)
|
|
|
|
### System Requirements
|
|
|
|
- **Operating System**: Linux/Unix-like systems
|
|
- **File System**: Support for standard Unix file permissions
|
|
- **Memory**: Minimal requirements (all processing in memory)
|
|
|
|
### Environment Setup
|
|
|
|
The script configures a consistent environment:
|
|
|
|
```bash
|
|
LC_ALL=C
|
|
LANG=C
|
|
umask 0022
|
|
```
|
|
|
|
## Detailed Workflow
|
|
|
|
### 1. Initialization Phase
|
|
|
|
```
|
|
Start qsgen3
|
|
├── Parse command line arguments
|
|
├── Set PROJECT_ROOT (default: $PWD)
|
|
├── Determine CONFIG_FILE path
|
|
├── Set environment variables (LC_ALL, LANG, umask)
|
|
└── Initialize QSG_CONFIG array
|
|
```
|
|
|
|
### 2. Configuration Loading
|
|
|
|
```
|
|
Load Configuration
|
|
├── Check if CONFIG_FILE exists
|
|
├── Parse key="value" pairs line by line
|
|
├── Strip quotes from values
|
|
├── Store in QSG_CONFIG associative array
|
|
└── Validate required configuration keys
|
|
```
|
|
|
|
### 3. Dependency Checking
|
|
|
|
```
|
|
Check Dependencies
|
|
├── Verify Pandoc is available
|
|
├── Check Pandoc version compatibility
|
|
├── Verify other required tools
|
|
└── Exit with error if dependencies missing
|
|
```
|
|
|
|
### 4. Theme Processing
|
|
|
|
```
|
|
Process Theme Configuration
|
|
├── Read site_theme from configuration
|
|
├── Determine theme base path: themes/$site_theme
|
|
├── Check for theme layouts directory
|
|
│ ├── If exists: Override paths_layouts_dir
|
|
│ └── If not: Use default layouts
|
|
├── Determine theme static source
|
|
│ ├── Check themes/$site_theme/static/
|
|
│ ├── Fallback to themes/$site_theme/
|
|
│ └── Set QSG_CONFIG[theme_static_source_dir]
|
|
└── Log theme processing decisions
|
|
```
|
|
|
|
### 5. Output Preparation
|
|
|
|
```
|
|
Prepare Output Directory
|
|
├── Check for .qsgen3_preserve file in project root
|
|
├── If preserve file exists:
|
|
│ ├── Read file patterns (shell glob patterns)
|
|
│ ├── Create temporary backup directory
|
|
│ ├── Find and backup matching files from output directory
|
|
│ ├── Remove entire output directory
|
|
│ ├── Recreate clean output directory
|
|
│ ├── Restore preserved files maintaining directory structure
|
|
│ └── Clean up temporary backup directory
|
|
│ └── Remove entire output directory
|
|
│ └── Create fresh output directory
|
|
└── Log preservation and cleaning operations
|
|
```
|
|
|
|
#### File Preservation System
|
|
|
|
qsgen3 supports preserving specific files during the cleaning process to handle cases where content has been shared or bookmarked and should remain accessible even after title changes.
|
|
|
|
**Preserve File Format (`.qsgen3_preserve`):**
|
|
- Located in project root directory
|
|
- One pattern per line using shell glob patterns (`*`, `?`, `[]`)
|
|
- Lines starting with `#` are comments
|
|
- Empty lines are ignored
|
|
- Patterns are relative to the output directory
|
|
|
|
**Example preserve patterns:**
|
|
```bash
|
|
# Preserve specific shared articles
|
|
posts/my-important-shared-article.html
|
|
posts/viral-blog-post.html
|
|
|
|
# Preserve files by pattern
|
|
posts/legacy-*.html
|
|
archive/*
|
|
|
|
# Preserve all PDFs and downloads
|
|
*.pdf
|
|
downloads/*
|
|
```
|
|
|
|
**Benefits:**
|
|
- Maintains stable URLs for shared content
|
|
- Prevents broken links when content is renamed
|
|
- Flexible pattern matching for various preservation needs
|
|
- Backward compatible (no preserve file = complete cleaning)
|
|
|
|
### 6. Static File Processing
|
|
|
|
```
|
|
Copy Static Files
|
|
├── Copy from paths_static_dir to output/static/
|
|
│ ├── Use rsync -av --delete if available
|
|
│ └── Fallback to cp -R
|
|
├── Copy from theme static source to output/static/
|
|
│ ├── Theme files overwrite root files
|
|
│ └── Preserve directory structure
|
|
└── Log copy operations and results
|
|
```
|
|
|
|
### 7. CSS Path Determination
|
|
|
|
```
|
|
Determine CSS Linking
|
|
├── Read site_theme_css_file from configuration
|
|
├── Construct expected CSS file path in output/static/
|
|
├── Verify CSS file exists after copying
|
|
├── Set QSG_CONFIG[pandoc_css_path_arg] for Pandoc
|
|
└── Log CSS path decisions and warnings
|
|
```
|
|
|
|
### 8. Content Processing
|
|
|
|
```
|
|
Process Markdown Content
|
|
├── Scan paths_content_dir recursively for .md files
|
|
├── For each Markdown file:
|
|
│ ├── Extract YAML front matter
|
|
│ ├── Determine output path and template
|
|
│ ├── Run Pandoc with appropriate template and CSS
|
|
│ ├── Write generated HTML to output directory
|
|
│ └── Log processing results
|
|
└── Collect metadata for index and RSS generation
|
|
```
|
|
|
|
### 9. Index Generation
|
|
|
|
```
|
|
Generate Index Page
|
|
├── Collect all post metadata
|
|
├── Create YAML metadata file for Pandoc
|
|
├── Run Pandoc with index template
|
|
├── Apply CSS styling
|
|
├── Write output/index.html
|
|
└── Clean up temporary files
|
|
```
|
|
|
|
### 10. RSS Generation
|
|
|
|
```
|
|
Generate RSS Feed
|
|
├── Collect post metadata with URLs
|
|
├── Create YAML metadata for RSS template
|
|
├── Run Pandoc with RSS template
|
|
├── Generate absolute URLs using site_url
|
|
├── Write output/rss.xml
|
|
└── Clean up temporary files
|
|
```
|
|
|
|
### 11. Finalization
|
|
|
|
```
|
|
Complete Generation
|
|
├── Log final directory structure
|
|
├── Report generation success
|
|
├── Clean up any remaining temporary files
|
|
└── Exit with status code 0
|
|
```
|
|
|
|
## Troubleshooting and Debugging
|
|
|
|
### Common Issues
|
|
|
|
#### 1. CSS Not Applied
|
|
**Symptoms**: Generated HTML doesn't show theme styling
|
|
**Causes**:
|
|
- Incorrect `site_theme_css_file` path in site.conf
|
|
- CSS file doesn't exist in theme's static assets
|
|
- Theme static directory structure mismatch
|
|
|
|
**Solutions**:
|
|
- Verify CSS file path relative to theme's static source
|
|
- Check theme directory structure
|
|
- Enable debug logging to trace CSS path resolution
|
|
|
|
#### 2. Theme Not Found
|
|
**Symptoms**: Warning about theme directory not found
|
|
**Causes**:
|
|
- Typo in `site_theme` configuration
|
|
- Theme directory doesn't exist
|
|
- Incorrect theme directory structure
|
|
|
|
**Solutions**:
|
|
- Verify theme name spelling in site.conf
|
|
- Check themes/ directory exists and contains named theme
|
|
- Ensure theme directory has expected structure
|
|
|
|
#### 3. Template Errors
|
|
**Symptoms**: Pandoc errors during HTML generation
|
|
**Causes**:
|
|
- Missing required templates
|
|
- Template syntax errors
|
|
- Incompatible template variables
|
|
|
|
**Solutions**:
|
|
- Verify all required templates exist
|
|
- Check Pandoc template syntax
|
|
- Review template variable usage
|
|
|
|
#### 4. Static File Copy Issues
|
|
**Symptoms**: Assets missing from output directory
|
|
**Causes**:
|
|
- Permission issues
|
|
- Disk space problems
|
|
- Path resolution errors
|
|
|
|
**Solutions**:
|
|
- Check file permissions
|
|
- Verify available disk space
|
|
- Review path configurations for absolute vs. relative paths
|
|
|
|
#### 5. File Preservation Issues
|
|
**Symptoms**: Expected files not preserved during cleaning, or preservation not working
|
|
**Causes**:
|
|
- Incorrect patterns in `.qsgen3_preserve` file
|
|
- File paths don't match patterns
|
|
- Permission issues with temporary backup directory
|
|
- Malformed preserve file format
|
|
|
|
**Solutions**:
|
|
- Verify patterns use shell glob syntax (`*`, `?`, `[]`)
|
|
- Check that patterns are relative to output directory
|
|
- Ensure `.qsgen3_preserve` file is in project root
|
|
- Test patterns with `find output/ -name "pattern"` before adding to preserve file
|
|
- Enable debug logging to see preservation process details
|
|
- Verify file permissions allow temporary directory creation
|
|
|
|
**Example debugging:**
|
|
```bash
|
|
# Test if your pattern matches files
|
|
find output/ -name "posts/legacy-*.html"
|
|
|
|
# Enable debug logging to see preservation process
|
|
QSG_DEBUG=1 ./bin/qsgen3
|
|
```
|
|
|
|
### Debug Logging
|
|
|
|
Enable detailed logging by modifying the `_log` function or adding debug statements:
|
|
|
|
```bash
|
|
# Enable debug logging
|
|
QSG_DEBUG=1 ./bin/qsgen3
|
|
```
|
|
|
|
### Path Debugging
|
|
|
|
The script includes path resolution logic to handle both relative and absolute paths. If experiencing path issues:
|
|
|
|
1. Check that `PROJECT_ROOT` is correctly set
|
|
2. Verify configuration paths are relative to project root
|
|
3. Review log messages for path construction details
|
|
|
|
### Configuration Validation
|
|
|
|
Ensure site.conf follows the correct format:
|
|
- Use double quotes for values: `key="value"`
|
|
- No spaces around the equals sign
|
|
- One configuration per line
|
|
- Comments start with `#`
|
|
|
|
---
|
|
|
|
*This document reflects the current implementation of qsgen3 and its design philosophy of remaining completely design-agnostic while providing flexible theme and content management capabilities.*
|