SAIL: Self-promotion, Meta AI, Social Science, Vectors, Databases
Welcome to Sensemaking, AI, and Learning (SAIL) - a “when I feel like it” newsletter on AI and learning. I’m learning to be better.
Last week I spent time at the ASU-GSV conference in San Diego. It remains an interesting event - not to attend the sessions since by the time we gather we’ve all read and heard everything new and novel. It is a good opportunity to meet up with some really interesting humans doing really interesting things. It’s the conversation, not the conference, that matters. At some point, maybe we just drop panels and speakers and just all get together to talk.
During the event, we announced the launch of a Global Data Consortium. Write ups of the initiative can be found here and here. Sign up if you/your institution wants to review and comment on the concept/technical paper to be released in about a week. Universities need to get into the AI arena as producers, not only consumers. We’re planning on on-ramping that work with this project.
AI and Learning
I did a TEDx talk in January that is live now. I focused on what it means to be human in the age of AI.
What will social science research look like as AI grows in prominence? Here’s one angle: Automated Social Science: Language Models as Scientist and Subjects (h/t Shane Dawson).
AI Innovations, Senior Program Officer. I generally don’t post job openings, but this is relevant in that it gives insight into how the largest Foundation in the USA is planning on approaching AI. It’s sector wide - Biden requires all US agencies to have a Chief AI officer.
AI cheat sheets. Worth a skim for high level overviews of history of AI, AI jobs, chatgpt, prompting, etc.
The AI tools in education database. Kinda like the title. “this database is intended to be a community resource for educators, researchers, students, and other edtech specialists looking to stay up to date.”
AI and Technology
Meta doesn’t get enough credit for its AI capacity. They have likely the largest, cleanest, conversational data set in the world (across both Instagram and Facebook and possibly WhatsApp, but that is supposed to be encrypted). They’ve also been leaders in open sourcing their work. This is a great interview that outlines Meta’s AI planning. It covers significant barriers including energy (think small nuclear power plants to support ongoing model training), synthetic data, open source as a malicious power neutralizer, AI agents, GPU access (a less significant issue than it was a year ago), and rather other-worldly training budgets (think $10b-100b+. These ranges are in line with a recent Ezra Klein podcast with the CEO of Anthropic).
Meta AI released their Llama 3 LLM this week (8b & 70b parameter model with a 405b model still being trained. It’s now the highest ranking open source model on leader boards. When running on Grok chips the results are beyond impressive.
This is a sentiment worth thinking about: “all the major AI companies are spending billions producing almost exactly the same results using almost exactly the same data using almost exactly the same technology”… and “The half-life of excitement now is about 5 hours. Soon nobody will care.” I don’t disagree, but as we’re seeing, it’s the layers of support around the LLM that are making an impact, especially with function calling, RAG, DSPy…and emerging development and testing tool kits. It’s the ecosystem emerging that is giving viability channels to these LLMs - a gradual realization of Karpathy’s LLM as OS.
Short summary of vector databases and why they are important for LLMs/AI. Last week, I referenced a video on how GPTs work - there is a great overview at the 12:30min mark. Stay with it until at least the 18min mark. It’s an excellent description of how vectors work (and are created).