I’m too stupid for AsyncIO

If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
The Zen of Python

I have a new toy project called Letters from a Feed. It takes blogs (and RSS feeds) of people I think are interesting and compiles them into a custom email newsletter for me.

This is meant to replaces one of the things I was using social networks for, without having to add a new app to check.

It’s a fun little project. But scanning all these feeds is slow, and the more feeds I add, the slower it gets.

This is a perfect use case for async functions.

1.

I’ve been thinking about this since I did Wes Bos’s Node course1. I like Python better, but the one thing Node excels at is calling multiple api’s.

import requests

def fetch_things():
    """
    This is slow because it has to do each call in order.
    If each call takes 1s to complete, the function will take 3s to run.
    """
    responses = [
        requests.get('https://example.com/api/object/1'),
        requests.get('https://example.com/api/object/2'),
        requests.get('https://example.com/api/object/3'),
    ]
    return responses

fetch = require('node-fetch');

exports.fetchThings = async () => {
  // This is less slow because each request can start
  // before the previous one ends. If each one takes 1s,
  // the whole function will take 1s.
  let requests = [
    fetch('https://example.com/api/object/1'),
    fetch('https://example.com/api/object/2'),
    fetch('https://example.com/api/object/3'),
  ]

  return await Promise.all(requests);
}

And yet, I still want the batteries, and, frankly stability, I find with the rest of the Python ecosystem.

With a lot of research, I managed to stick together a minimal working Flask example. It’s one view that fetches a list of urls in parallel before rendering the page.

It works, but it’s weird. Here it is reduced to it’s simplest form:

import asyncio
import aiohttp  # special library. requests.get doesn't do async.


async def fetch(url, session):
    # The async task

    async with session.get(url) as response:
    # I have no idea why this is a context manager

        html = await response.read()
        # The response.read() is sort of like 
        # a promise. Once it's back you can 
        # return a better data structure.

        return {
            "resp": response,
            "html": html,
        }


@app.route("/")
def fetch_things():

    urls = [
        "https://example.com/api/object/1",
        "https://example.com/api/object/2",
        "https://example.com/api/object/3",
    ]

    # Must share the session between calls
    session = aiohttp.ClientSession()

    tasks = [asyncio.ensure_future(fetch(url, session)) for url in urls]

    # Create a loop to run all the tasks in.
    loop = asyncio.get_event_loop()

    # Gather is like Promise.all
    responses = loop.run_until_complete(asyncio.gather(*tasks, return_exceptions=True))
    loop.close()
    return responses

Javascript always has an event loop. In Python you write your own, push events into it, and run it until it’s done.

Where I get lost is in the fetch function itself. There’s an async with statement within an async function. This is ugly, but also, why?

2.

Letters from a Feed syncs sources and sends email using a management command on a cron job.2 And of course, the more sources you add, the longer it takes to sync.

All that slowness is time spent waiting on servers to respond. If the app made all the requests at once and waited for them to come back, the delay would drop from a few minutes to a few seconds.

It uses Feedparser to read and normalize all the different formats, so I can’t just switch to aiohttp like I did in the last example.

Here’s my code:

# WARNING THIS CODE DOES NOT WORK
# DO NOT USE AS EXAMPLE.

import asyncio
from .models import Newsletter, Source

async def sync(source_id):
    source = Source.objects.get(id=source_id)
    source.update()

def sync_all(newsletter=None):
    qs = Source.objects.all()
    if newsletter:
        qs = qs.filter(newsletter=newsletter)
    ids = list(qs.values_list("id", flat=True))

    # Sync each one asynchronously
    loop = asyncio.get_event_loop()
    tasks = [sync(i) for i in ids]
    loop.run_until_complete(asyncio.gather(*tasks))
    loop.close()

Not bad. It opens a loop, puts some async functions in it, and gathers them when they’re done. There’s just one problem: it was still slow.

I added logging before and after the .update() and discovered that it wasn’t actually executing the tasks in parallel. Each source would finish syncing before the next one started.

Docs weren’t helpful. Most Stackoverflow comments suggest using async libraries instead of the existing ones. That seems wrong. I should be able to run normal Python code as async, right?

The answer is to add a 0s sleep function. (Edit: It turns out that’s wrong too. See postscript.)

# WARNING: Misleading example. Do not use.

import asyncio
from .models import Newsletter, Source

async def sync(source_id):
    source = Source.objects.get(id=source_id)
    await asyncio.sleep(0)  # This appeared to fix it!
    source.update()

def sync_all(newsletter=None):
    qs = Source.objects.all()
    if newsletter:
        qs = qs.filter(newsletter=newsletter)
    ids = list(qs.values_list("id", flat=True))

    # Sync each one asynchronously
    loop = asyncio.get_event_loop()
    tasks = [sync(i) for i in ids]
    loop.run_until_complete(asyncio.gather(*tasks))
    loop.close()

Why? Apparently gather was running the async functions (tasks3) synchronously unless something happened that caused the async function to “pause and continue.”

This makes no sense to me—shouldn’t all async functions be gathered asynchronously?—but it works.

3.

Armin Ronacher of Flask wrote last year that he didn’t understand asyncio. His interests are lower level than mine. He’s talking about the mechanics of the threads. I just want to solve half a dozen simple concrete problems.

As of today, my high-level use cases are still harder than they should be4, and the problem starts with the documentation.

From the docstring for coroutines:

The word “coroutine”, like the word “generator”, is used for two different (though related) concepts:

• The function that defines a coroutine (a function definition using async def or decorated with @asyncio.coroutine). If disambiguation is needed we will call this a coroutine function (iscoroutinefunction() returns True).

• The object obtained by calling a coroutine function. This object represents a computation or an I/O operation (usually a combination) that will complete eventually. If disambiguation is needed we will call it a coroutine object (iscoroutine() returns True).

Gentlemen5, neither of those bullet points are coherent sentences. You can’t document what a taco is by explaining it is a taco. This is not an explanation; it is a tautology.

On the same page, the docs present this example:

import asyncio

async def hello_world():
    print("Hello World!")

loop = asyncio.get_event_loop()
# Blocking call which returns when the hello_world() coroutine is done
loop.run_until_complete(hello_world())
loop.close()

You’ll notice this makes the same mistake I did: it’s not actually executing anything asynchronously because it never does a sleep or anything else that would cause a second task to start before the first is complete. If you’re trying to figure out what to do to make it work this will mislead you.

4.

A good place to start would be a useful hello world example. Here’s my attempt:

import random
import asyncio


async def print_async(message):
    # Randomizing the delay to prove it's actually
    # async. To always run your code as fast as
    # possible, sleep for 0 seconds.
    await asyncio.sleep(random.random())
    print(message)


def say_hello():
    """
    Say hello in many languages
    asynchronously.
    """

    messages = [
        "Hello world",
        "Hola Mundo",
        "Hallo Wereld",
        "Bonjour le monde",
        "Hallo Welt",
        "Helo Byd",
        "こんにちは世界",
        "你好,世界",
        "안녕 세상",
        "Привет мир",
        "שלום עולם",
    ]

    # Open loop
    loop = asyncio.get_event_loop()

    tasks = [print_async(m) for m in messages]
    loop.run_until_complete(asyncio.gather(*tasks))
    print("Done saying hello.")
    loop.close()

say_hello()

The good news is my example above should be enough for almost all my real world problems. The bad news is I have no idea if it’s the ‘one right way,’ because after browsing the docs and too many blog posts, the only thing I know for sure is I know nothing at all.

Postscript

Hacker News had great feedback on this post.

In a cooperative async model anything which would block will stop progress of every coroutine, so you would need an async version of the ORM library for this to work. When you await something, you're telling the event loop to stop running this coroutine, and wait for something (usually IO) to happen before resuming execution. Because of this, if you block on IO without using await, the coroutine doesn't know to yield and will simply pause; thus execution of every single coroutine is blocked.

This leads to a sort of infectious need to make everything async as even a single non cooperating coroutine can bring the whole show to a halt. It's essentially the red vs blue function problem of Python. However, there is actually a nice alternative, gevent. gevent will monkey patch all functions in the standard library which would block, e.g. reading from a socket, attaching an implicit await to them. If the author has used gevent, the example Django code would actually work as expected, since the code would execute until the database connection was written to/read from and then immediately await.

Because of the way I was testing the functions, it looked like they were executing in parallel. All the async functions would start first, and then they would end in a random order in quick succession.

But they weren’t actually running in parallel. The moment one began running .update(), all the running async functions paused.


There are a few options on how to proceed and have the requests actually execute asynchronously:

In researching this, I also learned requests now has native async support powered by gevent. This feels right, and addresses my main use case, but it doesn’t fit my existing code.

Another solution would be to have feedparser read from a string, and make the request with aiohttp. This seemed like it would work. Based on my new (but probably still wrong) understanding of asyncio, all the parsing and ORM work (which is fast) would happen synchronously, but now other operations would be able to execute while waiting on the response from the url.

This might work, but it’s a bad idea. Imagine trying to reason about this in order to fix a bug, or explaining it to the next person who has to maintain it.

Commenters on HN and Reddit suggested using a multiprocessing and thread pools.

from multiprocessing import Pool

from .models import Newsletter, Source


def sync(source_id):
    source = Source.objects.get(id=source_id)
    source.update()


def sync_all(newsletter=None):

    # Create a pool of 5 workers
    pool = Pool(5)

    qs = Source.objects.all()
    if newsletter:
        qs = qs.filter(newsletter=newsletter)
    ids = list(qs.values_list("id", flat=True))

    # The pool will run sync with each argument
    # spread across the 5 workers.
    pool.map(sync, ids)

There are a few things to like about this solution:

  1. It’s clear what’s happening, and doesn’t introduce the cognitive overhead of having threads that pause and play.
  2. The docs for multiprocessing are a lot more straightforward than the asyncio ones.
  3. Because the number of workers is defined, there’s no risk of spinning up too many as the dataset grows.
  4. It works! With a single worker, or without using the pool, the task takes about 6 seconds to complete. The more workers you add, the faster it runs. My example ran locally in 2 seconds—just a hair slower than the delay of the slowest url I’m scraping.

What’s cool about this is its portable. Suppose I had an app that needed to access multiple API’s inside the request response cycle, which is more sensitive to delays than a background task. If I know I’m running 5 API calls, I can spin up 5 workers to run them.6 Alternately, if a background tasks needs to make 100 API calls, it’s easy to give it 10 workers to speed it up but limit its impact on system resources.

Further Reading

  1. Asynchronous Python and Databases, focuses on SQLAlchemy, but does a much better job explaining what’s going on than any of the HTTP-centric posts I’ve been reading.
  2. gevent is very cool, but probably overkill for my limited used case.
  3. Philosopher-Programmer Carl Johnson responds, with an experiment comparing notes for Asyncio and Go.
  4. Philip Jones, the creator of Quart (like Flask but async), wrote Understanding Asyncio in response to this article, and does a much better job articulating the mental model behind asyncio.

  1. I’d highly recommend his work if you’re looking to learn Node or React or something. He’s good at balancing direct “do this next” instructions and explaining why it all works. 

  2. Celery was overkill for this project. 

  3. I still have no idea what the difference between a coroutine and a task is. 

  4. And to gain widespread adoption, they’re harder than they can be. 

  5. and pyladies. 

  6. I’m aware that spinning up workers has a performance cost. As usual, the answer is to measure it for the use case and decide if its acceptable.