README.md

weekly_snippets Gem

Standardizes different weekly snippet formats into a common format,
munges snippet text
according to user-supplied rules, performs redaction of internal information,
and publishes snippets in plaintext or Markdown format.

Contributed by the 18F team, part of the United States General Services
Administration: https://18f.gsa.gov/

Motivation

This gem was extracted from the 18F Hub Joiner
plugin. That
plugin manipulates Jekyll-imported data
by removing or promoting private data, building indices, and performing joins
between different data files so that the results appear as unified collections
in Jekyll's site.data object. It serves as the first stage in a pipeline
that also builds cross-references and canonicalizes data before generating
static HTML pages and other artifacts.

Installation

Add this line to your application's Gemfile:

gem 'weekly_snippets'

And then execute:

$ bundle

Or install it yourself as:

$ gem install weekly_snippets

Usage

The 18F Hub processes snippet data as CSV
files harvested from a
web-based spreadsheet, stored using timestamped
filenames in the
Jekyll _data folder. Since we have
experimented with different CSV column formats, we keep the data files
corresponding to each version in separate directories:

This map is then used to standardize batches of weekly snippets, converting
each different version to a common format, before joining the data with team
member information:

# Snippet data is expected to be stored in files matching the pattern:# _data/@source/snippets/[version]/[YYYYMMDD].csv## resulting in the initial structure:# site.data[@source][snippets][version][YYYYMMDD] = Array<Hash>## After this function returns, the `standardized` will be of the form:# site.data[snippets][YYYYMMDD] = Array<Hash>
standardized = ::WeeklySnippets::Version.standardize_versions(
@data[@source]['snippets'], snippet_versions)

Munge

To accommodate the preferred formats employed by some team members, the
18F Hub snippets.rb plugin
defines a Ruby block to
munge the snippet data
before converting it to a uniform
Markdown representation: