June 29, 2020

26: Progress and APIs

After the high of last week's design breakthrough¹ I now am trapped in the lightning sand of API usage and design.

If you're just joining now this is A Warm Newsletter, a development log of an ill-advised attempt to make a new note-taking stack.

First, let's talk a bit about API usage

Up until now I've been writing relatively "pure" code, just taking in and outputting strings, no "side-effects" involved. But now that I'm implementing this append-only log architecture I have to figure out interacting with the real world.

In this case the "world" consists of a web browser, because that's where I've decided to implement Fancynote. I've gone back and forth on it for a while but ultimately, the browser is the most full featured and accessible application platform that has ever existed, and I'm already familiar with many parts of it.

It's incredible the breadth of API's built into webbrowsers today. You can write code that can run on billions of computers out there and can make use of API's to interact with MIDI devices, Webcams, USB storage, Peer-to-Peer networking, and so much more.

Today I've been trying to wrap my head around one of the more boring API's, IndexedDB, the browser's persistent data system.

I need a way to keep my notes-log around, even if I close my browser, or reboot my computer. I also need a way to lookup data from the log, and process it. IndexedDB is how you do that in a web-browser today.

For some undoubtedly good² but currently baffling to me reason, it's API is based on event handling. This means for everything you want to do (like initializing a database, or adding an entry) you have to do an action, and then define handlers for different events that could happen.

This is not the paradigm I normally use, either in API's I consume or code I write myself so it's got a bit of a learning curve to it. It would be alright on it's own, but, in a brilliant move, I've been learning it while trying to design and implement, an API of my own.

An interface for a log

As simple as I may have thought the append-only log architecture I described last week was, there's a lot of concrete details to figure out. I need to figure out how one defines new views, how those views are processed, and how the high level code can then interact with them.

On top of these requirements I'm not really sure how to evaluate the design of the API, other than striving for simplicity. And, I'm balancing many competing goals, like keeping everything strongly typed, or enabling efficient updating of the visual interface.

Keeping all this in my head at once has been frustrating and I've been left feeling like I didn't accomplish what I wanted to this weekend.

So what have I accomplished?

It's been about 6 weeks since I started working on Fancynote again in earnest so it seems like a good time to checkin. Since I started I've implemented a basic compiler complete with a couple functions and different markup tags. It works with a html and plaintext printer that outputs to a neat lil website. And, I've dug into various UI frameworks and modern 2D graphics.

Each week my routine is roughly:

1. Stop myself from working on fancynote during the week. Hype myself up instead.
2. Get to the weekend fired up.
3. Realize that the fancynote problems are hard, and I'm kinda tired, and I want to spend some time relaxing.
4. Alternate hacking on fancynote,relaxing, and miscellaneous weekend things.
5. Realize it's sunday midnight, rush to write this newsletter, and contextualize the little I've done as something interesting.

I always end up feeling a little like I've just scraped by. The only way I know how to handle a project like fancynote (large, amorphous, ever evolving), is a little by little. I know this. But it still can end up feeling like too little.

There was a really great twitter thread on how to shake this feeling a couple weeks ago:

https://twitter.com/mayli/status/1273384131997716481

Honestly, writing this newsletter is by far the most valuable thing I do to fight it. It forces me to think about what I've actually been working on, instead of what it feels like I've been doing, and to find a way to frame it as a step forward, towards whatever I'm going to do next. So thank you all for reading this :), I don't know if I'd be still keeping at it without you!

So what am I gonna do next?

Have made a first stab at the API design I know feel better equipped at actually design it! Ah, it always happens this way. I've also figured out IndexedDB enough to work with it. Next week I'm going to put this experience to good use and layout the basic API along with the first view: The "current" view. Basically, it's going to take the log and produce a view of the current state of ever note, combining all edits. Sounds boring but it should be a good test of things!

Thanks again for tuning in. See you next week!