Generating A Jekyll Static Site with Gulp

Jekyll

Jekyll is a static site generator that takes a collection of content pages, and assets such as images, and generates a web site.

A Documentation Web Site

In an open-source project it is natural to extend the prototypical README to a more extensive documentation of anything ranging from architecture, to design, to usage instructions, to user manual. A README is fine, but once it gets too long, it is best to structure it into folders by subject. A large enough system would require also media assets, such as images.

Coming from the README origin, it is natural to use Markdown to write the documentation.

Such a documentation collection to be useful for readers, has to be presented as a web site that provides its readers with common HTML pages presentation, including search capability, and showing media assets on pages.

A Static Site Generator

We would like generate such a documentation web site automatically from a documentation folder, and its sub-folders, containing Markdown files and media assets.

Most often, the documentation is spread across multiple folders of software projects, and we need to automatically extract the Markdown files and images.

Jekyll is convenient as the target static site generator, because it can handle Markdown, and through a plugin privide full-text search capability.

But we needed an automated way to provide the source content to Jekyll.

Gulp to the Rescue

Gulp is workflow automation system built on the principle of pipelining files between tasks.

Our tasks pipeline:

  1. Extract Markdown files (with .md extension) from our project folder and its sub folders

  2. Give unique names to files (think of multiple README.md files)

  3. Modify image links in Markdown files to correspond to the Jekyll target folder structure

  4. Copy image files into Jekyll target folder

  5. Build the Jekyll web site

Let us consider each Gulp task in turn.

Assume the project folder, which includes both code and documentation files is in the path ./src

Extract Markdown Files and Give Unique Names

We give to files unique names in the Jekyll format for posts:

 YYYY-MM-DD-<name>

By concatenating the date to the folder and file name:

 d + "-" + path.dirname + "-" + path.basename

This is performed with gulp-rename

And then write them to a temporary folder:

    gulp.task('unique-names', function(cb) {

        // extract all the markdown files

        return gulp.src("./src/**/*.md")

    // rename files by sub folder name

        .pipe(rename(function (path) {

          var d = moment().format("YYYY-MM-DD");

          path.basename = d + "-" + path.dirname

                      + "-" + path.basename;

        }))

            // write files into a temporary folder

        .pipe(gulp.dest("../static_site/tmp"));

    });

Copy Image Files into Jekyll

    gulp.task('copy-images-jekyll', function() {

         // extract all image files

       return gulp.src('./src/**/*.jpg')

             // write image files into the images folder of Jekyll

         .pipe(gulp.dest('../static_site/images'));

    });

For simplicity, we consider here just JPEG files. The generalization to all types of image files is straightforward.

Images are written into Jekyll under the images folder in a the same folder structure as in the source src folder.

Generate Links To Images In Jekyll Format

We modify images links from the Markdown link structure of, say:

    ![screenshot](./screenshot.jpg)

into the Jekyll link structure of:

    {{ site.url }}/images/<folder>/screenshot.jpg

This is performed with gulp-replace

We then copy the files from the temporary folder into the _posts folder of Jekyll:

gulp.task('generate-images-links', ['unique-names'], function(){

    // get all markdown files in the templrary folder

  return gulp.src(['../static_site/tmp/**/*.md'])

        // replace image links

    .pipe(replace(/.\//g, '{{ site.url }}/images/{{ page.path | remove_first: "_posts/" | split: "/" | first }}/'))

        // write into jekyll _posts folder

    .pipe(gulp.dest('../static_site/_posts'));

});

Build and Server Jekyll

We use gulp-shell to execute Jekyll build and serve commands:

gulp.task('build-site-serve', shell.task([

  '(cd ../static_site; jekyll build)',

  '(cd ../static_site; jekyll serve)'

]));

The Whole Gamut

gulp.task('jekyll', ['clean-jekyll', 'unique-names',

    'copy-images-jekyll', 'generate-images-links', 'build-site-serve'

]);