SELECT * logo

SELECT *

Archives
Log in
Subscribe
August 20, 2026

On Course(s)

I’ve been teaching full-time at the University of Maryland since the fall of 2021. But I started tracking the university’s course offerings even earlier, thanks to the efforts of Ed Summers.

While he was at UMD, Ed wrote some code that scraped the university’s Schedule of Classes, otherwise known as Testudo (yes, that’s our mascot’s name as well, because we like turtles). The actual schedule website is useful to a point, but it only permits users to look at a department or an instructor or a class, all within a single term. If you want to do anything more - for example, tell whether the number of courses are increasing or decreasing - you’d need to have the data. Oh, and older terms just went away.

Ed’s code gave me the data, and then I started to think about how to add onto to it. First, the scraper work was slow - grabbing courses from a single term could take several hours - and you couldn’t narrow it down to a single department. The original output was a series of JSON files, one per course, and I wanted to keep that but offer alternatives, such as saving the output to a SQLite database. Here’s what one course listing looks like:

{"id":"JOUR405","title":"Breaking News With Numbers: Statistics for Journalists","credits":"3","description":"Prerequisite: Minimum grade of C- in JOUR201; and (MATH107 or MATH110; or must have completed a higher level math course).\nRestriction: Must be in a major in JOUR-Philip Merrill College of Journalism.","level":"Undergrad","grading_method":["Reg","P-F","Aud"],"gen_ed":["FSAR"],"sections":[{"id":"0101","instructors":["Derek Willis"],"seats":38,"open_seats":0,"waitlist":0,"days":"W","start":"12:30pm","end":"1:45pm","building":"KNI","room":"3202"}],"term":"202608","department":"Journalism","syllabus_count":3,"most_recent_syllabus":null,"updated":"2026-08-16T01:14:23.654170Z"}

So, taking a page from Simon Willison, who builds a lot of command-line utilities, I turned Testudo into a command-line utility. It still scrapes the Schedule of Classes and outputs JSON. But now it does more:

# Test with a single department
uv run python cli.py --test AAAS

# Scrape a single department
uv run python cli.py --department CMSC

# Run full scrape
uv run python cli.py --term 202608

Why stop there? The schedule offers the number of syllabi available for each course (usually 0, and the actual documents are only available to users with a UMD account), so I added that to the scraper options. It definitely slows things down. But I did manage to speed up the course scraping by parallelizing the work.

Another addition is semantic search, since part of the fun (for me at least) is finding courses when I don’t actually know the title or course number. Testudo can generate embeddings based on the title and description, and then offers a command to search for courses related to the idea of “data analysis”, for example.

Given my meager design talents, that’s probably where this story would end: a piece of software that provides a data pipeline. But at a certain point, I figured this data could make for a good news app. So that’s what Claude helped me build. Here’s the Testudo Course Explorer:

Ok, so still a meager design

The underlying data makes a lot of things possible, including pages for individual instructors, every course (including recent history), new course offerings and more. That semantic search? It’s here, too, and since the data covers multiple years, the app can show changes over time, or the patterns of when courses are offered or not. That also means we can tell which courses appeared in an early version of a semester’s listings and then subsequently was dropped. Might be some stories there.

Although I mentioned a SQLite database earlier, this app doesn’t use it, relying on the JSON files and HTML & JavaScript. A script builds all of the pages and they are served by GitHub Pages.

At some point in the next couple of years, this will all come to an end because the university will switch to a course listing system that requires authentication to access. I oppose that for several reasons - UMD is a public university, after all - but my opinion isn’t the one that matters in this case. And I doubt that new system will do everything I want it to, much less be a worthy successor of Ed Summers’ work.

Don't miss what's next. Subscribe to SELECT *:
Older → Call To Order
Bluesky
Powered by Buttondown, the easiest way to start and grow your newsletter.