Weekly API Evangelist Governance (Guidance)
I was immersed in what it takes to automate business all of this week. My definition of automation includes the current notion of agentic automation using AI, but also other ways we already are automating our businesses. The technical details of automation are just the beginning, and it is the business and politics of automation that are standing in the way of what we envision as modern business automation Something that doesn’t always get properly talked about in detail amidst today’s business climate.
Authentication
Ensuring that our automation has access to what it needs, while ensuring it does not have access to what it doesn’t need, is a top concern and discussion right now. We have all decided to make our digital resources accessible via the web, and have invited the web into our businesses and homes. The social contract of the web is being renegotiated right now as part of the recent AI shift in how we do business, and authentication is the front line of this negotiation.
Authorization
The details of not just who has access to our digital resources, but which portions of the digital resources is an ongoing negotiation in the social contract of the web. OAuth, SAML, and other standards exist to help us negotiate the scope of digital resources we have access to on the web. The authorization of the data, files, images, videos, and other digital resources we produce and manage daily is being renegotiated right now as part of the currently very charged automation discussion.
Accounts
To access information across the web you need an account. Actually, you need a lot of them. As part of the authentication and authorization discussion there are ways accounts are being reused, federated, and securely applied on the web. To be able to automate with the data you produce and own online, you need programmatic access to the accounts behind each digital relationship you enter into as a business or individual, providing another level of accounting you probably haven’t thought much about.
Applications
Some of the platforms we use require you to create an “application” before you can access your data as part of any automation. This is a legacy of the API management evolution of the integration discussion. If you want to pull your Facebook data, you will need to setup a developer application. Only then will you be able to get the keys you need to access even your own data using the platform APIs—whether you intend to use a 3rd-party service or write custom code to access.
Keys
To access any data via automation you will need keys to access it. Keys are a short or long series of letters and numbers that are generated as part of the gateways we employ to manage access to our digital resources. Keys can last for months or hours, and come with a variety of security protocols that help us manage the authentication and authorization of humans or bots who are looking to access and automate with any of the data we manage across platforms.
Plans
Most of the platforms we use to produce and manage our digital resources have formulated plans that describe how we can access and use the digital resources available there. Plans overlap between the SaaS and API realms, providing a handful of entry level, professional, enterprise, and other types of plans that describe what we get access to and often what it will cost for use to access our accounts, but also all of the data and media produced via our own accounts.
Pricing
Plans for the platforms we use often have pricing associated with them, but increasingly there can be a second set of pricing that overlaps with, or is adjacent to the plans for users. The pricing is looking to decouple the business of automation from the user and employ a usage and consumption based approach to governing how our digital resources are being used. Pricing is often based on the number of requests for any given digital resource, but we’ve seen a new dimension emerge with AI negotiation.
Models
When you visit the plans and pricing pages for platforms you will increasingly see a new dimension of pricing centered on models, or more specifically large-language models. Model-based pricing can replace earlier user and request based pricing, but is often just being added as a new dimension that will need negotiation at the runtime for business automation. The real question here comes down to whether or not you are running your models or someone else’s.
Tiers
Plans historically have contained the structure for tiers of access to digital resources available via a platform. As with pricing moving outside the plan, and the introduction of the models dimension, we are seeing added access tiers being considered as part of the access to digital resources. These new access tiers go beyond the tiers included in plans or models, and consider other financial, investment, business, and marketplace concerns that influence what you should have access to.
Limits
Next we begin a transition to the technical aspects of how we meter and enforce everything we’ve talked about so far. The APIs for platforms utilize rate limits to control what we have access to. Our rate limits are shaped by our accounts, applications, plans, pricing, models, and tiers. I am also beginning to use just the word “limits”, as they have expanded beyond just rates, and we are seeing performance limits being defined by our accounts, plans, and overall spend.
Headers
Limits are communicated as part of automation using HTTP headers. The story of what you have access to as part of automation is communicated via a mix for formal and custom HTTP headers that are send along with each API response. These HTTP Headers provide you with the accounting behind your limits as reflected through your account, plans, pricing, models, and tiers. HTTP Headers allow you to make decisions around automation based upon what you need.
Parameters
Some of what you will need to authenticate and shape your access will be a little more visible via the query parameters you send in with API calls, utilizing a mix of headers, but also more intuitive query parameters to governing what you have access to as part of automation. Ideally all of this was available via HTTP Headers, as it shapes the transport layer for our automation, but different providers will all you to communicate in different ways around platform access.
Pagination
One critical variable in the algorithm being outlined in this newsletter is the concept of pagination. Pagination can be tokenized, offset, or time-based, meaning how much information you provide or access can be shaped and chunked in different ways—these are ways that will have real world consequences depending on your account, plan, pricing, models, and tiers. Pagination works hand in hand with limits and other types of business calculations made via automation.
Status Codes
HTTP Status codes are used to communicated many of the dimensions of API automation expressed in this newsletter. 401 Unauthorized, 403 Forbidden, 405 Method Not Allowed, and 429 Too Many Requests are all formal HTTP status codes. HTTP Status Codes are essential to automation, and being able to communicate failure will shape automation and the overall experience or intended outcomes. The web is designed for automation, and we are already doing it—we just need to keep adapting it to the current shift happening with AI.
Reviews
An increasing number of platforms are requiring a review of your application before you access any data beyond your own. Formal reviews of any application requesting access to APIs has been common place for the last 15 years. Reviews provide important gates for keeping out bad actors and ensuring that applications will be adhering to terms of service, privacy policies, as well as the accounts, plans, pricing, and other constraints applied to how we use data.
Billing
Once you have access to digital resources and are successful in automating around them, you will need to manage the billing. Something that should also be automated and auditable. How much of a resource you are using, which models you are applying, and other criteria will shape your bill each month. Your ability to upgrade and downgrade plans, tiers, and models will increasingly play a role in automation across different platforms and domains.
Business
There is an additional business layer often being added to accounts requiring additional validation of the business behind the account, plans, and tiers being utilized via a platform. Platforms will require certain levels of business validation as part of Know Your Customer (KYC) efforts and trust initiatives, before you can gain additional access to platform resources via an API, ensuring that any person or bot involved with automation can be associated to a valid business operating within a specific region.
Marketplace
I am seeing additional marketplace constraints being applied to this automation stack. Salesforce has tiers of access for Slack’s API depending on if you’ve been through their application review and are published to the Slack Marketplace. Platforms are increasingly requiring more transparency for bot and agent automation, requiring them to be publicly published and seamlessly tied in with the overall monetization goals of the platform via a controlled marketplace.
Scaling
Let’s take everything we have covered so far for a single API, and scale to 25, 50, or 100 APIs. There is little standardization across this stack. The web brings a lot of the standardization, and then cloud and API management providers introduce the rest. Everything else is ad hoc, without much machine-readability to help reduce friction when it comes to automation. This newsletter entry represents each gear in the automation machine that will need greasing to do what y’all are looking to do.
The bird a nest, the spider a web, man friendship. - William Blake