Configuration

The command line Wyam application reads a configuration file typically named config.wyam (though you can change that with an argument) that sets up the environment and configures the pipelines. Preprocessor directives can appear anywhere in the configuration file, though they are always evaluated before processing the rest of the file (by convention they're usually at the top of the file).

The configuration file is evaluated as C# code, so you can make use of the full C# language and the entire .NET ecosystem. However, it's not necessary to know C# to write Wyam configuration files. The syntax has been carefully crafted to be usable by anyone no matter their level of programming experience. Some extra pre-processing is also done to the file to make certain code easier to write (which actually makes the syntax a superset of C#, though this extra magic is entirely optional).

A configuration file typically looks like this:

// Preprocessor directives

// Body code
// ...

When Wyam starts, the configuration file is compiled by Roslyn (a C# compiler) and starting with Wyam 1.0 a cached copy of the resulting compilation is stored on disk at config.wyam.dll by default. A hash of the config file is also stored at config.wyam.hash. These two files improve startup performance by allowing Wyam to skip the compilation phase if the config file has previously been processed and its contents haven't changed.

Preprocessor Directives

Preprocessor directives establish the Wyam environment and get evaluated before the rest of the configuration file. They're typically responsible for declaring things like NuGet packages and assemblies. Every preprocessor directive starts with # at the beginning of a line and extend for the rest of the line. The following directives are available (the current set of directives can always be seen by calling wyam help --directives).

Available preprocessor directives:

#nuget-source, #ns
Specifies an additional package source to use.

#recipe, #r
    -i, --ignore-known-packages    Ignores (does not add) packages for
                                   known recipes.
    <recipe>                       The recipe to use.
Specifies a recipe to use.

#assembly, #a
Adds an assembly reference by name, file name, or globbing pattern.

#theme, #t
    -i, --ignore-known-packages    Ignores (does not add) packages for
                                   known themes.
    <theme>                        The theme to use.
Specifies a theme to use.

#nuget, #n
    -p, --prerelease         Specifies that prerelease packages are
                             allowed.
    -u, --unlisted           Specifies that unlisted packages are
                             allowed.
    -v, --version <arg>      Specifies the version range of the package
                             to use.
    -l, --latest             Specifies that the latest available version
                             of the package should be used (this will
                             always trigger a request to the sources).
    -s, --source <arg>...    Specifies the package source(s) to get the
                             package from.
    -e, --exclusive          Indicates that only the specified package
                             source(s) should be used to find the
                             package.
    <package>                The package to install.
Adds a NuGet package (downloading and installing it if needed).

NuGet Packages

Any NuGet packages you specify in preprocessor directives are installed and then scanned for modules which are made available to the main configuration body. By default, Wyam will attempt to match requested packages with those on disk and will use the disk-based package when available. To force it to use a specific version or version range (and download and install it if necessary), use the --version <version> flag. To force it to always download and install the latest available version on the package feed, use the --latest flag, which will always result in a call to the configured source(s).

Wyam follows the same conventions as NuGet with regards to specifying version ranges. Specifically, the following summarizes how to specify version ranges:

1.0  = 1.0 ≤ x
(,1.0]  = x ≤ 1.0
(,1.0)  = x < 1.0
[1.0] = x == 1.0
(1.0) = invalid
(1.0,) = 1.0 < x
(1.0,2.0) = 1.0 < x < 2.0
[1.0,2.0] = 1.0 ≤ x ≤ 2.0

Note that many modules require their package to be installed before they can be used. For example, to make use of the Markdown module, you must install the Wyam2.Markdown package. To do this, you would add the following to your configuration file:

#n Wyam.Markdown

You can also specify the special Wyam2.All package which will download all of the official Wyam module packages at once:

#n Wyam.All

All NuGet packages you specify, along with their dependencies, are located and downloaded at startup. Walking the full dependency tree can be time consuming since queries have to be issued to the NuGet server(s) for each individual package. To improve performance, the full set of packages and their dependencies is cached in a packages.xml file. If Wyam has already calculated the dependency tree for a given package and version, that entire tree will be stored in this cache file and it won't need to be rewalked on the next run. This means that the first execution of Wyam may take longer as the dependency cache is populated but subsiquent executions should be much faster.

Assemblies

In addition to NuGet packages you can also load assemblies. All assemblies are loaded with the #assembly or #a directive. You can load all the assemblies in a directory by using a globbing pattern, or by specifying a relative or absolute path to the assembly. You can also load assemblies by name. If you specify a short name, Wyam will attempt to resolve the assembly with the same version as the currently loaded framework. Keep in mind that non-framework assemblies located in the GAC must be loaded by full name (including the version, public key token, etc.).

By default, the following assemblies are already loaded so you don't need to explicitly specify them:

  • System
  • System.Collections.Generic
  • System.Linq
  • System.Core
  • Microsoft.CSharp
  • System.IO
  • System.Diagnostics

Declarations

The code in your configuration is executed inside the context of an "invisible" class and method. Note that any declarations such as classes or methods will be automatically "lifted" into an outer scope. In this way, the configuration file is more like a C# script than a normal code file. Any classes that are declared in the configuration file will be placed in the global scope. Any methods will be placed in the wrapping class outside the default wrapping method that contains the rest of the configuration code.

// "using" statements are placed in the outer scope
// and are available throughout the script
using System.IO;

// classes are also placed in the outer scope
public static class Helpers
{
    public static string GetWriteExtension()
    {
        return ".html";
    }
}

// normal code is placed in a wrapper class and method
// that gets called when the script is evaluated
Pipelines.Add("Markdown",
    ReadFiles("*.md"),
    FrontMatter(Yaml()),
    Markdown(),
    WriteFiles(Helpers.GetWriteExtension())
);

Note that namespaces for all found modules as well as the following namespaces are automatically brought into scope for every configuration script so you won't need to explicitly add them:

  • System
  • System.Collections.Generic
  • System.Linq
  • System.IO
  • System.Diagnostics

Pipelines

Configuring a pipeline is easy, and Wyam configuration files are designed to be simple and straightforward:

Pipelines.Add(
    ReadFiles("*.md"),
    Markdown(),
    WriteFiles(".html")
);

or

 //Disable API doc generation while in development
if(inDev)
{
    Pipelines.Remove(Docs.RenderApi);
    Pipelines.Remove(Docs.ApiIndex);
    Pipelines.Remove(Docs.ApiSearchIndex);
}

or

//Insert a new pipeline called "Test" that dumps the pipeline docs using a custom json serialization to the log file
Pipelines.InsertAfter("Reference", "Test",
    Documents(Docs.Api),
    ForEach(
        Trace(DocToJson)
    )
 );

However, don't let the simplicity fool you. Wyam configuration files are C# scripts and as such can make use of the full C# language and the entire .NET ecosystem (including the built-in support for NuGet and other assemblies as explained above). One of the core modules even lets you write a delegate right in your configuration file for extreme flexibility.

Pipeline Names

Pipelines should be given names, which makes them easier to identify in trace messages and also makes them easier to refer to within templates (for example, to get all the documents generated by a previous pipeline). Just pass in the name of a pipeline as the first parameter to the Add() method. If no name is provided (as above), then pipelines will implicitly be given the names Pipeline 1, Pipeline 2, etc.

Pipelines.Add("Markdown",
    ReadFiles("*.md"),
    FrontMatter(Yaml()),
    Markdown(),
    WriteFiles(".html")
);

Child Modules

Some modules also accept child modules as part of their processing. For example, the FrontMatter module accepts a child module to handle parsing whatever front matter content is found. This way, the same module can be used to recognize front matter without it having to worry about what kind of content the front matter contains (such as YAML or JSON). For example:

Pipelines.Add("Markdown",
    ReadFiles("*.md"),
    FrontMatter(Yaml()),
    Markdown(),
    WriteFiles(".html")
);

Fluent Methods

Some modules allow method chaining to extend or modify their behavior. For example, the OrderBy module allows you to chain a call to ThenBy() if you would like to sort by multiple inputs. Or, you chain a call to Descending() if you would like to reverse the order.

Pipelines.Add("Markdown",
    ReadFiles("*.md"),
    FrontMatter(Yaml()),
    OrderBy(@doc.Get<DateTime>("Published"))
        .ThenBy(@doc["Title"])
        .Descending(),
    WriteFiles(".html")
);

Skipping Previously Processed Documents

If you know that a given pipeline doesn't use data from other pipelines and you'd like to prevent reprocessing of documents after the first pass, you can set the ProcessDocumentsOnce flag. Under the hood, this looks for the first occurrence of a given IDocument.Source and then caches all final result documents that have the same source. On subsequent executions, if a document with a previously seen IDocument.Source is found and it has the same content, that document is removed from the module output and therefore won't get passed to the next module. At the end of the pipeline, all the documents from the first pass that have the same source as the removed one are added back to the result set (so later pipelines can still access them in the documents collection if needed). The ProcessDocumentsOnce flag can be set when creating a pipeline:

Pipelines.Add("Markdown",
    ReadFiles("*.md"),
    FrontMatter(Yaml()),
    Markdown(),
    WriteFiles(".html")
).WithProcessDocumentsOnce();

Explicit Module Instantiation

Note that you supply the pipeline with new instances of each module. An astute reader will notice that in the example above, modules are being specified with what look like global methods. These methods are just shorthand for the actual module class constructors and this convention can be used for any module within the configuration script. The example configuration above could also have been written as:

Pipelines.Add("Markdown",
    new ReadFiles("*.md"),
    new Markdown(),
    new WriteFiles(".html")
);

Automatic Lambda Generation

Many modules accept functions so that you can use information about the current IExecutionContext and/or IDocument when executing the module. For example, you may want to write files to disk in different locations depending on some value in each document's metadata. To make this easier in simple cases, and to assist users who may not be familiar with the C# lambda expression syntax, the configuration file will automatically generate lambda expressions when using a special syntax. This generation will only happen for module constructors and fluent configuration methods. Any other method you use that requires a function will still have to specify it explicitly.

If the module or fluent configuration method has a ContextConfig delegate argument, you can instead use any variable name that starts with @ctx. For example:

Foo(@ctx2.InputFolder)

will be expanded to:

Foo(@ctx2 => @ctx2.InputFolder)

Likewise, any variable name that starts with @doc will be expanded to a DocumentConfig delegate. For example:

Foo(@doc["SomeMetadataValue"])

will be expanded to:

Foo((@doc, _) => @doc["SomeMetadataValue"])

If you use both @ctx and @doc, a DocumentConfig delegate will be generated that uses both values. For example:

Foo(@doc[@ctx.InputFolder])

will be expanded to:

Foo((@doc, @ctx) => @doc[@ctx.InputFolder])

If you need to use the ContextConfig delegate argument but do not need to access @doc or @ctx then you will need to specify the lambda. For example, one of the overrides for the WriteFiles module is a string to let you specify a file extension. However, we might want to override the output and write to a single file (with minified and combined CSS for example). To accomplish this, we will need to pass in the lambda, even thought we won't be using it.

WriteFiles((doc, ctx) => "css/style.css")

Execution Ordering

Be aware that the configuration file only configures the pipelines. Each pipeline is executed in the order in which they were first added after the entire configuration file is evaluated. This means that you can't declare one pipeline, then declare another, and then add a new module to the first pipeline expecting it to reflect what happened in the second one. The second pipeline won't execute until the entire first pipeline is complete, including any modules that were added to it after the second one was declared. If you need to run some modules, switch to a different pipeline, and the perform additional processing on the first set of documents, look into the Documents module.

GitHub