Elsa Gonsiorowski

Generating Multiple Jekyll RSS Feeds

gonsie@me.com (Elsa Gonsiorowski) — Thu, 23 Jun 2022 00:00:00 +0000

Whelp, my “resolution” to post to this blog every 90 days or so has failed. It lasted about a year, but a few ~~weeks~~ months ago I got an email notification that my RSSby.email subscription was automatically unsubscribed due to inactivity. I guess one good thing is now I know that I’ll get an unsubscribe notice… but sorry to all my RSSby.email followers who now have to go through the hassle to resubscribe.

One of the things that has kept me from posting is that my blog is being indexed by Planet Emacs. I only found out from a comment that someone left, but apparently it’s been in there since March 2020. Now I feel bad for all those non-emacs posts! My solution is create an emacs-only RSS feed for my site.

Note: If you are using GitHub pages to deploy your site, skip to Step 4: Slightly Less Generation.

Step 1: Create the File Generator

Jekyll has the capability to generate whole pages. That is, no .md file exists in the source tree, but a .html file is generated and appears on the site. This requires writing a bit of ruby code (a Jekyll plugin), but a very easy to use template is included in the documentation (linked above).

module GenerateTagRSS
  class TagRSSGenerator < Jekyll::Generator
    safe true

    # loop to generate each feed file
    def generate(site)
      site.tags.each do |tag, posts|
        Jekyll.logger.info "\tGenerating feed for tag: #{tag}"
        # call to TagRSSPage instantiation function
        site.pages << TagRSSPage.new(site, tag, posts)
      end
    end
  end

  # Subclass of `Jekyll::Page` with custom method definitions.
  # TagRSSPage instantiation
  # Definition of what an RSS file looks like
  class TagRSSPage < Jekyll::Page
    def initialize(site, tag, posts)
      @site = site             # the current site instance.
      @base = site.source      # path to the source directory.
      @dir  = 'blorg/feeds'    # the directory the page will reside in.

      @basename = tag          # filename without the extension.
      @ext      = '.xml'       # the extension.
      @name     = tag + '.xml' # basically @basename + @ext.


      # Initialize data hash with a key pointing to all posts under current category.
      # This allows accessing the list in a template via `page.linked_docs`.
      @data = {
        'linked_docs' => posts
      }

      # Look up front matter defaults scoped to type `tags`, if given key
      # doesn't exist in the `data` hash.
      data.default_proc = proc do |_, key|
        site.frontmatter_defaults.find(relative_path, :tag, key)
      end
    end

    # Placeholders that are used in constructing page URL.
    def url_placeholders
      {
        :path       => @dir,
        :basename   => basename,
        :output_ext => output_ext,
      }
    end
  end
end

File GenerateRSSTags.rb

Now I don’t understand that code entirely, but I have verified that it does work.

Step 2: Template for the Generated Files

The R community has a similar “R topics” RSS aggregator, with an explicit requirement that included feeds are limited to on-topic posts. This post guides you through doing that by hand, for a single Jekyll tag. It provides a basic template for an RSS XML file, (only slightly adapted below).

  <?xml version="1.0" encoding="UTF-8"?>
  <rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
      <channel>
          <title>{{ site.title | xml_escape }}</title>
          <description>{{ site.description | xml_escape }}</description>
          <link>{{ site.url }}{{ site.baseurl }}/</link>
          <atom:link href="{{ page.url | prepend: site.baseurl | prepend: site.url }}" rel="self" type="application/rss+xml"/>
          <pubDate>{{ site.time | date_to_rfc822 }}</pubDate>
          <lastBuildDate>{{ site.time | date_to_rfc822 }}</lastBuildDate>
          <generator>Jekyll v{{ jekyll.version }}</generator>
          {% for post in page.linked_docs %}
              <item>
                  <title>{{ post.title | xml_escape }}</title>
                  <author>{{ site.email | xml_escape }} ({{ site.author | xml_escape }})</author>
                  <description>{{ post.content | xml_escape }}</description>
                  <pubDate>{{ post.date | date_to_rfc822 }}</pubDate>
                  <link>{{ post.url | prepend: site.baseurl | prepend: site.url }}</link>
                  <guid isPermaLink="true">{{ post.url | prepend: site.baseurl | prepend: site.url }}</guid>
                  {% for tag in post.tags %}
                  <category>{{ tag | xml_escape }}</category>
                  {% endfor %}
                  {% for cat in post.categories %}
                  <category>{{ cat | xml_escape }}</category>
                  {% endfor %}
              </item>
          {% endfor %}
      </channel>
  </rss>

File _layouts/rss.html

Some notes on this snippet:

This layout file has the .html extension. This is an assumption/requirement of Jekyll, our ruby generator code is what sets the output file extension to .xml.
The posts which will be included are passed via the page.linked_docs variable.

Of course it’s important to make sure Jekyll knows that the generated files should default to using this layout. That is done in the _config.yml file.

defaults:
  - scope:
      type: tag
    values:
      layout: rss

Snippet from _config.yml

Step 3: Getting it to Run

By default Jekyll will run any generators it finds in _plugins directory. I included a nice log message in the generation loop, so that I have some indication that each tag is being generated. Locally building my site is complete success!

The one slight issue is that GitHub pages won’t run unsupported plugins, aka the generator we just wrote. So much of this post feels moot. The whole reason I’m still using the Jekyll stack is that GitHub takes care of generating the site for me.

Step 4: Slightly Less Generation

Since I couldn’t get GitHub pages to actually generate the files from whole cloth, I had to do a bit by hand. This is relatively easy, given a list of tags which exist on my site, I simply create the following markdown file for each tag:

---
layout: rss
permalink: /blorg/feeds/emacs.xml
tagname: emacs
---

Jekyll frontmatter for tag feed file

Then, in the RSS layout (from Step 1), instead of looping over the page.linked_docs variable, I use:

{% for post in site.posts %}
{% if post.tags contains page.tagname %}

Updated liquid loop for _layouts/rss.html file.

(Don’t forget to close your if statement with an {% endif %} just before the end of the loop). This solution requires the manual step of me remembering to create the tag markdown file when I use a new tag, but it works!

Step 5: Profit

The final step is to share these RSS feeds with the world. I’ve created a whole feeds page which implements the following liquid to generate a nice list of topics with a link to each RSS feed (somewhat simplified here):

{% for tag in site.tags %}
  {% capture tagname %}{{ tag | first }}{% endcapture %}
  <a href="/blorg/feeds/{{ tagname }}.xml">{{ tagname }}</a>
{% endfor %}

Liquid snippet for generating a list of tags.

Of course, you can find all this code in my site’s GitHub repository.

Jekyll Portfolio

gonsie@me.com (Elsa Gonsiorowski) — Fri, 17 Dec 2021 00:00:00 +0000

So I recently learned that the future of Jekyll is, well, grim. I have quite the history with jekyll, I’ve been creating websites with it since 2012 (most especially this one). To mark the end of my jekyll era, I’ve collected a portfolio of all the sites I’ve developed and contributed to.

A few of them feature some clever liquid to make updating/expanding easy for others. Here I’ve collected my portfolio of sites, all of which are still live, thanks to github and the power of static sites. Looking over this collection, I mainly see a lot of acronyms…

My Website

gonsie.com
Converted to Jekyll site in 2012 (still in use)
Accent theme by Ankit Sultana (blog)

The only real blog in my portfolio, this site has a bunch of sections and cruft left around from past projects. Recently, I’ve created some post templates which allow me to easily post quotations and photos.

ROSS

ross.cs.rpi.edu
Launched 2013 (still in use)
Polar Bear Theme by Camille Diez

The ROSS (Rensselaer’s Optimistic Simulation System) website hosts much of the user documentation for the simulator I worked on during graduate school. This is/was one of my more successful sites. My idea to build up documentation for ROSS (which was severely lacking) was to require a ‘blog post’ to the website for nearly every pull request. Each blog post contains a jekyll tag, which places that entry in the proper section of the overall documentation table of contents.

XPDES

xpdes.org
2014
Hyde theme by Mark Otto

The Extreme Parallel Discrete-Event Simulation (PDES) was a consortium of PDES researchers from several institutions. The site had some liquid to group ‘people’ pages by institution; the posts were for meeting agendas. The consortium lasted until 2016.

NYχ

nychi.tech
2016
Long Haul theme by Brian Maier Jr.

The New York Capitol Hackathon for Inclusivity was a hackathon I help organize in April 2016. We needed a quick website. This site includes some nice liquid for automatically generating the navigation menu from all pages that exist.

IHPCSS

ihpcss.github.io
2018
Git-Wiki theme (and functionality) by Yehonal

This wiki theme is very cool with tons of functionality. I created this site because there was a lot of information that I wanted to organize for students at the International HPC Summer School. I hoped the wiki feature would allow other to add content, but the site was voted down by the organizing committee and the content migrated to a different system.

HUST

hust-workshop.github.io
I joined in 2019 (still in use)
SinglePaged Theme by Tim O’Brien

The HUST site was initially developed by Chris Bording, but I started co-organizing the workshop (and running the website) in 2019. This single-paged site uses post files to hold the content for each section. Any posts from the current year (as specified by the frontmatter in the index.md) are displayed. This allows us to easily keep the content file from previous years, without them getting in the way.

PSIP

bssw-psip.github.io
2019
Cayman Theme by Pietro Menna

This really simple site was for a bunch of markdown documentation whcih existed on GitHub for the Productivity and Sustainability Improvement Planning software improvement process. The coolest thing was an interactive Rate Your Project Survey which calculates a cumulative score for items checked off on the page (using javascript).

Jekyll Resume

gonsie.com/jekyll-resume/resume.html
2019
A jekyll implementation of the One Page Resume by Chris Coyier

I’ve used Chris Coyier’s one page resume since 2012 (slightly before I started using jekyll). In 2019 I created a template jekyll repository which I use to teach students about markdown and static websites. The template renders a resume written in markdown to a nice one page resume website. This site is really a demonstration of my markdown and css skills (rather than jekyll or liquid).

Developing ox-jekyll

gonsie@me.com (Elsa Gonsiorowski) — Sat, 12 May 2018 00:00:00 +0000

In a previous post, I surveyed existing org and jekyll integration workflows. This survey found two common approaches, one that is Jekyll-centric and one that uses Org-publish (Emacs-centric). Unfortunately, neither fit with my workflow and I decided to develop my own Org exporter.

I finally have some functioning code, so I figured I should share details about the project and the development process. You can find the project on GitHub.

Starting Point

In my survey I found some useful resources:

The Worg Documentation on Org Export Engine;
An existing ox-jekyll in org-octopress.

Great artists steal.

I want everything that org-octopress is doing with it’s jekyll exporter, I just want it built on markdown instead of HTML.

So, I’ve copied ox-jekyll.el and replaced “html” with “md”. After fixing a few function names, it works!

Most importantly, the front matter is in the right spot (with and without a table of contents). The next step is to make this work with the particulars of my publishing workflow.

My Publishing Workflow Details

There are a number of things I want my exporter to accomplish. Many of which are particular to my workflow.

I use a single org file per post.
Upon export, the date (YYYY-MM-DD) should get added to the file name.
Subtree level matches export heading level (this is just how I design my site’s CSS)
Code snippets should be easy to export with appropriate syntax. This is where many markdown converters differ, but jekyll currently wraps code blocks with { % highlight LANG %} / { % endhighlight %}. [Ignore the extra space before the %]
Jekyll front matter should be added above org-inserted table of contents. The manual HTML source block is just hideous.
It should be easy to move the generated file to a separate location (I think this addressed in the org-publish workflow).
Org-y way to add tags to the post.
Safe export of title (colon character mucks things up for Jekyll).
Fix up links and export connected images. This one is really a nice to have, not necessary.
TOC should be an ordered list (with the option to make it unordered) WTF is the HTML exporter doing making it an unordered list with numbers in the text.

Many of these are solved with code from org-octopress, particularly the yaml front matter.

Implementing Features

Here are some notes on my implementation.

Thinking about the Table of Contents

I had been including a table of contents at the top of each of my posts. I didn’t actually label it with “table of contents” and I prefer an un-numbered list. This seems to be very contrary to what the org-exporter likes to do. It pretty much does not allow for any fine-grained control of how the table of contents appears.

While I was able to get the code to do what I wanted, it ended up being way too much of a hack. In the end, I’ve decided that I don’t want a table of contents at all. Really, it will be up to the markdown / jekyll / liquid parser to create a TOC if necessary. This makes it easy to apply separate HTML styles to the TOC (to put it in a sidebar or something).

Preserving the Headline Level

I would like :headline-offset (as described here) to work on a file level. This allows me to make sure the contents of my post match up the CSS on the web site (the content’s final destination). This is definitely a more markdown-y way to do things rather than the org way.

Unfortunately, the :headline-offset option is only supported per subtree. I can hack it with a simple function that adds an “offset by 0” property to each subtree:

(defun org-jekyll-headline-offset (headline contents info)
  "proper headline offset"
  (let* ((info (plist-put info :headline-offset 0)))
    (org-md-headline headline contents info)))

I add it to the derived backend :translate-alist like so:

(org-export-define-derived-backend 'jekyll 'md
  ;; ...
  :translate-alist
  '((headline . org-jekyll-headline-offset)
  ;; ...

Now, this function is called each time a headline is translated by the export engine and my headline level is directly translated to markdown.

Setting the Date

I have a strange workflow where I want the #+DATE: tag to be the day I started working on a post and the date in the filename to be the day I hit publish. For my exporter, I can use the option date: to trigger this behavior. The date: option takes precedence over the variable org-export-with-date.

While I would like my code to support putting the #+DATE: property into the filename, this seems impossible with the export engine. The date property is stored in the info plist which I don’t have access to at file naming time.

Instead, I’ve implemented this functionality:

The date: option controls if the value of the #+DATE property is put into the Jekyll front matter.
I’ve added a new option to the exporter: org-jekyll-use-today-date. If t, the exporter will prepend the filename with today’s date.

Results

I’m quite happy with my exporter, I even used it to export this post! I still need to figure out how to use the org-publish workflow, but for now I can successfully export my blog posts.

As with most of my projects, you can find it on GitHub. Please feel free to file an issue and share any feed back you have.