Skip to main content

move.conf

move.conf is a massive nginx map configuration file containing 12,810 redirect rules that handle URL canonicalization, historical page moves, file reorganizations, and path.

Pathnginx/redirect/move.conf
LanguageNginx
Lines12896
Sourcenginx/redirect/move.conf
at 406d3e423

Read this when Use this page when tracing server routing, redirects, deployment configuration, or Nginx behavior around move.

Overview​

move.conf is a massive nginx map configuration file containing 12,810 redirect rules that handle URL canonicalization, historical page moves, file reorganizations, and path standardization across gwern.net. This file ensures that old URLs, externally-linked content, and reorganized resources continue to work as the site evolves.

The configuration uses nginx's map directive to define pattern-based redirects; every entry uses a case-sensitive regex key (prefixed with ~) rather than a literal string key. It serves as a historical record of site reorganization decisions while maintaining link permanence for external citations and bookmarks.

Key insight: This file implements the server-side portion of gwern.net's URL redirect strategy. Client-side redirects for in-page anchors (hash fragments) are handled separately via JavaScript (see style-guide#renaming).

File Structure​

Location: nginx/redirect/move.conf

Type: Nginx map configuration

Size: 12,836 lines, ~1.5MB

Format: Key-value redirect map

"~^/old/path$" "/new/path";

Nginx Map Syntax​

The file defines a map with this structure:

default "";

"~^/pattern$" "/destination";

Components:

  1. Default value: Empty string (no redirect)
  2. Source pattern: Regex or literal string (prefixed with ~^ for regex)
  3. Destination: New path (relative to gwern.net root)

Pattern format:

  • ~^ - Regex match (nginx syntax)
  • $ - End anchor (prevents partial matches)
  • .*$ - Match any trailing query strings or junk

(Named capture groups such as (?<u>...) are not used in move.conf; they appear only in the companion broken.conf's garbage-suffix-stripping rules.)

Major Redirect Categories​

1. Repository Restructuring (Lines 7-45)​

Purpose: Moved build scripts and source code from site root to /static/build/

Pattern: Build files, Haskell modules, shell scripts

"~^/Definition.hs$" "/static/build/Definition.hs";
"~^/LinkMetadata.hs$" "/static/build/LinkMetadata.hs";
"~^/sync-gwern.net.sh$" "/static/build/sync-gwern.net.sh";

Rationale: Separates source code from content, cleaner site organization

2. Metadata Reorganization (Lines 22-24)​

"~^/static/metadata/archive.hs$" "/metadata/archive.hs";
"~^/static/metadata/auto.yaml$" "/metadata/auto.gtx";
"~^/static/metadata/custom.yaml$" "/metadata/full.gtx";

Changes:

  • Moved out of /static/ directory
  • YAML format changed to GTX format
  • custom.yaml renamed to full.gtx

3. Logo Image Consolidation (Lines 25-38)​

All logo variants moved from /images/logo* to /static/img/logo/:

"~^/images/logo-smooth.svg$" "/static/img/logo/logo-smooth.svg";
"~^/images/logo-whitebg.png$" "/static/img/logo/logo-whitebg.png";

Affected files: 14+ logo variants (SVG, PNG, JPG, different themes)

4. The Great Image Renaming (Lines 47+)​

Scope: Thousands of images moved from /image/ to /doc/ subdirectories

Sections:

  • Fonts (Lines 48-194): Dropcap fonts consolidated under /static/font/dropcap/
  • Images (Lines 195+): Topic-specific images moved to match content structure

Pattern:

"~^/image/anime/eva/...jpg.*$" "/doc/anime/eva/...jpg";
"~^/image/biology/...png.*$" "/doc/biology/...png";

Rationale: Organize images alongside related content rather than in flat /image/ directory

5. SICP Page Extension Changes (Lines 40-45)​

"~^/sicp/Chapter-1\.1.page$" "/sicp/Chapter-1-1.md";
"~^/sicp/Chapter-1\.1$" "/sicp/Chapter-1-1";

Changes:

  • .page extension removed (changed to .md)
  • Dots in filenames changed to hyphens
  • Both with and without extension supported

6. File Format Corrections (Throughout)​

Hundreds of redirects fixing incorrect file extensions:

"~^/path/file.png.*$" "/path/file.jpg";  # PNG → JPG
"~^/path/file.jpeg.*$" "/path/file.jpg"; # JPEG → JPG

Common patterns:

  • .png → .jpg (image format standardization)
  • .jpeg → .jpg (extension normalization)
  • Trailing .*$ catches query strings and junk

7. Content Reorganization by Topic (Throughout)​

Files moved into topic-specific subdirectories:

"~^/doc/psychology/1996-berman.pdf.*$" "/doc/psychology/inner-voice/1996-berman.pdf";
"~^/doc/statistics/bayes/2007-pascual.pdf.*$" "/doc/statistics/bayes/abc/2007-pascual.pdf";

Pattern: Flat topic directory → topical subdirectory structure

8. Short URL Aliases (End of file)​

"~^/nadia$" "/doc/anime/nadia/index";
"~^/darcs$" "/doc/cs/haskell/darcs/index";
"~^/queueing$" "/doc/statistics/probability/queueing/index";
"~^/phenibut$" "/doc/nootropic/phenibut/index";

Purpose: Memorable short URLs redirect to full paths

Redirect Pattern Types​

1. Exact Match Redirects​

"~^/exact/path$" "/new/path";

Matches only the exact path, no variations.

2. Extension-Flexible Redirects​

"~^/path/file\.jpg.*$" "/new/path/file.jpg";

The .*$ suffix handles:

  • Query strings: file.jpg?version=2
  • Fragments: file.jpg#section
  • Garbage characters appended by broken crawlers

3. No Capture Group Redirects​

move.conf does not use named capture groups; every entry maps an explicit old path to an explicit new path. Parametric directory-level renames (which would need captures) are instead handled by rewrite rules in gwern.net.conf, and capture-based garbage-stripping lives in broken.conf.

Regex Patterns Used​

Anchors​

  • ~^ - Start anchor (nginx map syntax for regex)
  • $ - End anchor

Wildcards​

  • .* - Match any characters
  • .*$ - Match anything until end of URL

Escaping​

  • \. - Literal dot (escaped)
  • \- - Literal hyphen (in some contexts)

Character Classes​

  • [0-9] - Digits (used in some rules)

Comments and Documentation​

The file includes section headers:

## The great image renaming:
### Fonts:
### Images:
## Redirect the small ones too:

Header comment (Lines 3-4):

# NOTE: nginx redirects can only handle URL-level redirects for updating/moving things.
# They cannot handle links to IDs *within* pages, using hashes.
# To do that, we use custom client-side JS which read '.redirect-from-id' classes/attributes.
# See </style-guide#renaming> for more.

Key limitation: Hash-based redirects (e.g., #section-id) require JavaScript, as nginx never sees hash fragments.

Historical Context​

Repository Splitting​

Early redirects (lines 7-45) suggest gwern.net underwent a "great repo splitting" where:

  • Build scripts moved out of root
  • Source code consolidated under /static/build/
  • Clear separation between content and tooling

Image Organization Evolution​

The "great image renaming" (thousands of rules) shows transition from:

  • Old: Flat /image/ directory by topic
  • New: Images live alongside content in /doc/ subdirectories

This mirrors modern static site best practices (colocate assets with content).

Format Standardization​

Many redirects normalize file formats:

  • JPEG → JPG
  • PNG → JPG (when format was wrong)
  • Page extensions removed or standardized

Nginx Integration​

This file is included in the main nginx configuration via a map directive (in gwern.net.conf):

map $request_uri $new_uri {
    include /home/gwern/gwern.net/static/nginx/redirect/move.conf;   # intentional moves (first)
    include /home/gwern/gwern.net/static/nginx/redirect/broken.conf; # broken-URL repairs (second)
}

server {
    if ($new_uri != "") {
        rewrite ^(.*)$ $new_uri permanent;
    }
}

The map populates $new_uri which triggers a permanent rewrite when non-empty. Since regex map keys are evaluated first-match-wins in order of appearance, move.conf entries take precedence over broken.conf entries.

Maintenance Patterns​

Adding New Redirects​

move.conf is fully automated and must not be hand-edited (per the policy comment in gwern.net.conf). When moving/renaming a file:

  1. Use the move tooling (gwmv / renamer scripts), which generates the entry: "~^/old/path\.ext.*$" "/new/path.ext";
  2. The generated .*$ suffix catches query strings and trailing junk
  3. Duplicate keys are caught at sync time by remove-duplicates.py
  4. Test redirect (nginx -t) before deploying

Testing Redirects​

# Test nginx configuration
nginx -t -c /path/to/nginx.conf

# Test specific redirect
curl -I https://gwern.net/old/path
# Should return: HTTP/1.1 301 Moved Permanently
# Location: /new/path

Performance Considerations​

Map efficiency:

  • Nginx maps are hash-based for literal keys (O(1) lookup)
  • Regex patterns require sequential evaluation (O(n) worst case)
  • 12,000+ rules could impact performance on regex matches

Optimization opportunities:

  • Group common patterns
  • Use literal matches when possible (faster than regex)
  • Consider separating frequently-accessed redirects

Current approach:

  • Accepts linear regex evaluation cost
  • Values link permanence over microsecond performance
  • Redirects are infrequent (users access final URLs after first visit)

Companion File: broken.conf​

See broken.conf documentation for:

  • Broken crawler handling
  • Malformed URL cleanup
  • Security-related blocks
  • Regex substitutions for garbage removal

Division of responsibility:

  • move.conf: Legitimate redirects for moved content
  • broken.conf: Handle broken/malicious requests

This file embodies gwern.net's commitment to Cool URIs don't change:

"When you change a URI on your server, you can never completely tell who will have links to the old URI. They might have bookmarked your page. They might have scrawled the URI in the margin of a book. They might have written it on the back of an envelope." — Tim Berners-Lee

Evidence:

  • 12,810 redirect rules maintained indefinitely
  • Redirects span years of site evolution
  • Even minor file moves get permanent redirects
  • External citations continue working indefinitely

Common Redirect Patterns​

Pattern 1: Directory Reorganization​

"~^/doc/topic/file.pdf.*$" "/doc/topic/subtopic/file.pdf";

Moving files deeper into category hierarchy.

Pattern 2: Asset Consolidation​

"~^/images/asset.png$" "/static/img/asset.png";

Consolidating assets into standard directories.

Pattern 3: Extension Normalization​

"~^/path/file.jpeg.*$" "/path/file.jpg";

Standardizing file extensions.

Pattern 4: Short URL Creation​

"~^/short$" "/doc/category/long/path/index";

Creating memorable aliases for deep paths.

Build System Integration​

This file is:

  1. Fully autogenerated - Entries are produced by the move tooling (gwmv etc.); the policy in gwern.net.conf explicitly forbids hand-editing it (hand-written and one-off redirects go in broken.conf or gwern.net.conf rewrites instead)
  2. Linted - remove-duplicates.py detects shadowed duplicate keys at sync time
  3. Version controlled - Git tracks all redirect additions/changes
  4. Deployed atomically - Nginx reloads apply all redirects at once
See Also

Summary Statistics​

MetricValue
Total redirects12,810
File size1.5 MB
Lines12,836
Major categories8+ (repo split, images, fonts, logos, content reorg)
Oldest redirectsRepo splitting (early site history)
Newest redirectsOngoing content reorganization
Regex patterns100% (all keys use ~^ prefix)
Extension handlingExtensive .*$ suffix usage