The Beagle Has Landed

Last winter I wrote Build Shit in CSS, a tutorial series. Unlike most my work, there was no need for this to be dynamic. It’s flat, it changes slowly, and it really belongs on Github pages.

And I hit a wall: include, extends, all the modern functionality I'd grown used to in template languages wasn't available when working with a flat site.

I poked at Jekyll for a while, felt stupid (getting Jekyll to do things that aren’t a blog isn’t obvious to me), and did something evil and hacky to get the project up and running. The page would build itself with javascript, including rendering the markdown instructions, and use querystrings to navigate between examples.

It worked. Lot’s of people liked Build Shit and nobody complained. But as I went to write Build Shit in Javascript, I ran into the same problem again.

It wasn’t going away.

So I made a Beagle.

2.

Now, a sane, community oriented developer might say “Why reinvent the wheel? Why not use Gulp/Jekyll/Pelican/Sphinx?”

It’s a fair question. The real answer is: because I wanted to. This was for fun, and to see if I could make the kind of API’s and programming experience I had in mind.

But also, Jekyll et al solve very specific use cases. I just want the power I’m used to having in Flask and Django, but with a dist folder.

Can you do this in Gulp? Yes. Definitely. More on that later.

3.

So what does Beagle do?

You need a package with:

  1. A dist folder, which is empty
  2. A src folder with an index.py in it, along with Jinja2 templates, SASS, javascript, and whatever else.

The index file should be full of functions that will execute to build the site. Here’s one that renders a page with markdown:

from beagle.decorators import action
from beagle.commands import Page

@action
def example():
    text = "# Hello world"
    return Page(template="index.html", outfile="index.html", context={
        "content": text,
    })

And I’m done. If src/index.html is {{ text|markdown }}, dist/index.html will become <h1>Hello world</h1>.

To run it, I create an app file at the top of the package.

# !/usr/bin/env python

import os
import sys
import beagle

from src import index

PROJECT_DIR = os.path.dirname(os.path.abspath(__file__))
SRC_DIR = os.path.join(PROJECT_DIR, "src")
DIST_DIR = os.path.join(PROJECT_DIR, "dist")

app = beagle.App(index, src=SRC_DIR, dist=DIST_DIR, watch="dev" in sys.argv)

This passes the two directories, and the index file full of actions to Beagle. It, in turn, looks at all the functions in the index for actions and runs them. Every action returns one or more Commands, which render templates, copy files, do post-processing, etc.

What's cool about this is the API is just lets Python be Python. It doesn't presume to know what the structure of the output should be. Want to keep blog posts in a Markdown folder and render each one with a post template? os.listdir the folder and iterate other them to make a list of Page objects. I can tap into a database with SQLAlchemy or resize images in PIL. The magic is there is no magic and no questions about compatibility.

4.

There are a couple of interesting problems Beagle had to solve.

Keep in mind, virtually everything I might want to do with a static site are command line tools. SASSC, Babel, concatenation, copying -- they all have bin or bash implementations. The framework can be dumb. It doesn't know or care what all the tools it taps into are written in, it just needs to use them.

First, I need to find all the actions in the index. That's easy enough: dir(module) will list all the objects. getattr each one, check it for a private property that the decorator sets, run the ones that match.

Second is autorebuilding. A state sitebuilder without a watch function isn't very useful. For that, there's a Python library called Watchdog that can call a function when a directory is modified.

To serve the assets in development, I can use Flask's static file function and point it at everything in the dist directory.

Working with Github pages is a bit tricky since they serve out of username.github.io/reponame/ instead of username.github.io. I don't want to be forced to write relative paths, so I added a url_prefix parameter to the app and use it to dynamically build the Flask routes. This way, the development server can mirror gh-pages' production if you'd like to host content there.

The rest is straightforward enough: mostly trying different things with the API's to figure out how I wanted commands to work, and what the default Jinja2 settings should be.

5.

Can't you do this in Gulp? Yes. And maybe you should.

But I find Gulp frustrating. Sure, common tasks like building SASS and Browserify with Babel are very well-documented, but the further off the beaten path you go, the rougher the edges.

I wanted to use Markdown with Nunjucks (a template language inspired by Jinja2), so I dug through gulp-nunjucks, gulp-nunjucks-render, gulp-nunjucks-html, gulp-nunjucks-render, gulp-nunjucks-render-env, and gulp-render-nunjucks, which is not the same thing gulp-nunjucks-render but it did come up once as a result when Googling an issue I was having with the latter.

I eventually managed to make gulp-nunjucks-render integrate with nunjucks-markdown. But finding that combination and how to configure it was a grinding process.

6.

In recent years, I've heard Python developers taking the position that we should just let Node handle all the front end tooling.

But why? Choices are good, after all.

After spending an afternoon trying to get the Gulp configurations right instead of building my project, I was starting to wonder about that.

I wondered enough to take a crack at it, to see what it would take to build a tool with an API that I'd personally enjoy using; something where you could spend time coding instead of trying to wrangle dependencies or configurations.

Python has many advantages for this. The standard library robust. Working with files and serving as glue to other utilities and libraries is its bread and butter. It doesn't matter if Babel is javascript and Sass is in C, because nine times out of 10 I'm going to use their command line interfaces anyway.

You can see the result on Github.

Will I maintain it for the long haul, or even use it again? Who knows. But I’d argue its worth trying.