a quiet weekend is all we need.

AI News for 8/15/2024-8/16/2024. We checked 7 subreddits, 384 Twitters and 29 Discords (253 channels, and 3480 messages) for you. Estimated reading time saved (at 200wpm): 525 minutes. You can now tag @smol_ai for AINews discussions!

Jeremy Howard's return to Latent Space to talk about his team's extreme AI fueled productivity is worthwhile, we think, not least because of the dynamite song intro.

You can also enjoy conversations with Demis Hassabis or watch the new Sora demo, and mourn your SearchGPT waitlist rejection letter with the rest of us.

The Table of Contents and Channel Summaries have been moved to the web version of this email: !

AI Twitter Recap

all recaps done by Claude 3.5 Sonnet, best of 4 runs.

AI Model and API Updates

• Anthropic API Enhancements: @alexalbert__ announced the rollout of prompt caching in the Anthropic API, which cuts API input costs by up to 90% and reduces latency by up to 80%. @AnthropicAI confirmed this feature allows instant fine-tuning of model responses with longer prompts while reducing costs.

• New AI Models: @_philschmid reported the release of Grok-2 from xAI, which matches frontier models from Google DeepMind, OpenAI, Anthropic, Mistral AI, and Meta. It supports vision and text inputs and integrates external models for image generation. @Teknium1 noted that "Another model enters the frontier arena."

• Model Performance: @bindureddy claimed that "Sonnet 3.5 is way better than GPT-4 in key areas like coding and reasoning." @omarsar0 reported improvements in ChatGPT-4o-latest, particularly in reasoning capabilities.

AI Development and Research

• Intelligence Theory: @fchollet proposed that "Intelligence is the efficiency with which you operationalize past information in order to deal with the future," expressing it as a conversion ratio using algorithmic information theory.

• AI Research Challenges: @sarahookr discussed the challenges of building datasets for multilingual AI, involving 3000 collaborators worldwide for the Aya project.

• AI Safety and Regulation: @GoogleDeepMind shared a podcast featuring CEO Demis Hassabis discussing AI hype, future innovations, and safe AI development.

AI Tools and Applications

• Design Automation: @svpino demonstrated the Dora AI plugin for Figma, which can generate a complete landing page in under 60 seconds.

• Document Processing: @svpino highlighted Box\'s new AI API, enabling users to chat with documents, extract data, summarize content, and generate derived content from stored files.

• AI Agents: @_akhaliq reported on Salesforce\'s release of DEI, an open AI software engineering agents framework with a 55% resolve rate on SWE-Bench Lite.

Industry and Market Trends

• AI Integration: @scottastevenson observed that "Traditional ML experience can now be a yellow flag on your resume," emphasizing the rapid changes in AI application development over the past two years.

• AI Job Market: @savvyRL noted that "~80% roles are filled by personal network," highlighting the importance of networking in the AI job market.

• AI Acceleration: @bindureddy predicted increased AI acceleration, suggesting that OpenAI might launch a larger version of GPT-4 in response to uncensored posts from competitors.

Memes and Humor

• @kylebrussell joked about using Apple Vision Pro to catch up on cinema.\n- @teortaxesTex shared a meme about the consequences of "doing the bit" in reference to Cyberpunk: Edgerunners.\n- @giffmana humorously commented, "Guess the gang and i are doing something wrong then…" in response to a statement about AI progress.

AI Reddit Recap

/r/LocalLlama Recap

Theme 1. Advancements in Small and Efficient LLMs

• Will small models get exponentionally better? (Score: 100, Comments: 104): Phi3 3B, a small language model, can run on devices with limited resources like a Mac with 8GB RAM. The post author questions whether such small models will experience significant quality improvements in the coming years or if they are approaching their performance ceiling.

• Evolution of llama.cpp from March 2023 to Today | Gource Visualization (Score: 157, Comments: 23): The Gource visualization showcases the evolution of llama.cpp, an open-source project for running large language models, from March 2023 to the present. The video highlights the rapid growth and collaborative nature of the project, demonstrating the contributions of numerous developers and the expansion of the codebase over time.

• Flux.1 converted into GGUF - what interesting opportunity it offers in llm space? (Score: 76, Comments: 31): The author used a GGUF model of Flux in ComfyUI for image generation, noting its impressive speed and ability to operate within 8GB of VRAM. They shared links to the ComfyUI-GGUF GitHub repository and the Hugging Face model page, seeking opinions on potential new opportunities this development might bring to the LLM space.

Theme 2. New Model Releases and Benchmarks

• Hermes 3 - a NousResearch Collection (Score: 151, Comments: 37): NousResearch has released Hermes 3, a collection of open-source language models ranging from 2.7B to 70B parameters. The models, trained on a 2.3T token dataset, include Hermes 2 Base, Hermes 2 Pro, and Hermes 3 Pro, with the latter two incorporating constitutional AI and DPO techniques for improved performance and safety.

• Drummer\'s Rocinante 12B v1 (& v1.1!) - A workhorse with cranked up creativity! Your out-of-this-world adventure awaits! From the creators of Theia 21B and other stuff. (Score: 68, Comments: 36): Rocinante 12B, a new AI model from the creators of Theia 21B, has been released in versions v1 and v1.1. The model is described as a creative workhorse, designed to balance productivity with enhanced imaginative capabilities for various applications.

• "Grok-2 and Grok-2 mini now hold the top two spots on MathVista" hope they open source Grok mini soon (Score: 143, Comments: 42): Grok-2 and Grok-2 mini have achieved the top two positions on the MathVista leaderboard, demonstrating their strong performance in mathematical visual reasoning tasks. The post expresses hope that xAI will open-source the Grok mini model in the near future, potentially allowing wider access to this high-performing AI system.\n - Elon Musk\'s credibility is questioned, with users expressing skepticism about Grok\'s performance and xAI\'s intentions to open-source. Some argue Musk\'s past actions suggest he prioritizes control over openness.\n - The talent density at xAI is highlighted, with former employees from DeepMind, Anthropic, and OpenAI contributing to Grok\'s development. Grok 2 reportedly used more compute than GPT-4, potentially explaining its superior performance.\n - Debate ensues over the legitimacy of Grok\'s benchmark results, with some suggesting potential training on test datasets. However, it\'s noted that MathVista\'s test answers are not publicly released, countering these claims.

Theme 3. Local LLM Deployment and Infrastructure

• Online services are down, good thing you got local (Score: 82, Comments: 29): Perplexity, Anthropic, and OpenAI\'s ChatGPT are experiencing service outages according to a tweet by Kristi Leilani. This situation highlights the advantage of using local Large Language Models (LLMs), which can continue to function during cloud service disruptions.

• My Goofy Ass Inference Server (Score: 60, Comments: 24): The post describes a DIY inference server setup for running local Large Language Models (LLMs). The system consists of a Ryzen 7950X CPU, 128GB DDR5 RAM, and a 4090 GPU, capable of running models up to 70B parameters with acceptable performance, including the ability to run Llama 2 70B at about 7-8 tokens per second.

Theme 4. LLM Cognition and Reality Understanding

• LLMs develop their own understanding of reality as their language abilities improve (Score: 78, Comments: 35): Large Language Models (LLMs) demonstrate an increasing ability to develop their own understanding of reality as their language capabilities improve. This phenomenon suggests that LLMs are not merely processing language, but are forming coherent internal representations of the world, potentially leading to more advanced reasoning and problem-solving abilities. The development of this "understanding" in LLMs raises important questions about the nature of artificial intelligence and its potential to approach human-like cognition.

• Will small models get exponentionally better? (Score: 100, Comments: 104): Phi3 3B, a small language model, can run on devices with limited resources like a Mac with 8GB RAM. The post author questions whether such small models will experience significant quality improvements in the coming years or if they are approaching their performance ceiling.

All AI Reddit Recap

r/machinelearning, r/openai, r/stablediffusion, r/ArtificialInteligence, /r/LLMDevs, /r/Singularity

AI Image Generation and Models

• Flux image generation model: Used by Grok for image generation, developed by Black Forest Labs. Open-sourced and available on Hugging Face. Praised for its capabilities in r/StableDiffusion and r/FluxAI.

• Grok image generation controversy: Generated controversial images like Barack Obama doing cocaine and Donald Trump with guns, raising questions about AI guardrails.

• Creative AI applications: A user designed high-heel shoes using Flux and brought them to life using Kling image-to-video technology.

AI Model Comparisons and Speculation

• GPT-5 anticipation: A humorous video comparing various AI models to Dragon Ball Z characters, with GPT-5 as the most powerful. Sparked discussions about potential disappointment and competition from other models.

AI and Human Interaction

• AI imitation: A viral video shows humans imitating AI-generated videos, highlighting the circular nature of AI training and human behavior.

AI Discord Recap

A summary of Summaries of Summaries by Claude 3.5 Sonnet

1. LLM Advancements and Benchmarks

• Hermes 3 405B: Open-Source Powerhouse: Hermes 3 405B, a powerful new open-source AI model, excels at tasks like style transfer, summarization, and creative writing with parallel instructions, outperforming Meta's bf16 instruct model.
  • The model's response speeds are only slightly slower than GPT-3.5 sonnet, making it a strong contender for research and development. It also introduces new special tokens for 'thinking' such as <SCRATCHPAD>, <REASONING>, and <INNER_MONOLOGUE>.
• DeepSeek-Prover V1.5: Pushing Theorem Proving Boundaries: DeepSeek-Prover-V1.5 achieves new state-of-the-art performance on high school level miniF2F (63.5%) and undergraduate level ProofNet (25.3%) benchmarks for theorem proving.
  • The model leverages proof assistant feedback for Reinforcement Learning (RL) and Monte-Carlo Tree Search (MCTS), with open base, SFT, and RL weights available on Hugging Face.
• Llama3-8B-Instruct Matches Meta's Benchmarks: A user successfully reproduced Meta's GSM8k performance using Llama3-8B-Instruct with a specific prompt format and settings, as detailed in this HuggingFace dataset viewer.
  • This required adjusting the regex expression and creating a new .yaml file for the GSM8k-cot task. The user offered to share the .yaml file and plans to replicate the process for other datasets to reproduce Meta's results.

2. AI Model Optimization Techniques

• Batching LLM Jobs for Efficiency: A blog post titled Unlocking the Power of Job Batching: Transforming AI Workloads on Medium discusses the advantages of batching jobs for LLM workloads.
  • The post highlights efficiency gains and cost savings associated with batching, offering a practical approach to managing large-scale AI projects and addressing challenges like rate limiting and GPU utilization.
• Moonglow: Streamlining Remote GPU Access: Moonglow, a VSCode extension, allows users to connect Jupyter notebooks to remote cloud GPUs like those offered by Runpod, streamlining the process of starting, connecting to, and stopping GPU instances.
  • The tool eliminates the need for managing SSH keys, package installations, and other DevOps tasks, allowing users to seamlessly switch between cloud compute environments and manage resources directly within their IDE.
• OpenBLAS Optimization for Intel CPUs: A user shared their experience compiling OpenBLAS to optimize CPUs for running generative AI workloads, specifically for Intel Haswell architecture.
  • The release was compiled on Linux x86_64 Intel CPU but also includes targets for ARM, POWER, MIPS, and RISC-V architectures, showcasing efforts to optimize AI workloads across various hardware platforms.

3. Open-Source AI Developments

• Salesforce's DEI Framework for SWE Agents: Salesforce released DEI (Diversity Empowered Intelligence), an open-source AI software engineering agent organization that leverages SWE agents' unique expertise for enhanced problem-solving.
  • DEI achieved a 34.3% resolve rate on SWE-Bench Lite with a group of open-source SWE agents, surpassing the performance of individual agents and demonstrating the potential of collaborative AI systems in software engineering tasks.
• xLSTM: A Potential Transformer Replacement: A Hugging Face compatible xLSTM trainer was released, with the developer believing that xLSTM may eventually replace transformers.
  • The trainer is available on GitHub as helibrunna, potentially offering an alternative to traditional transformer architectures for certain NLP tasks.
• LlamaIndex's Multi-Agent System Framework: LlamaIndex is developing Llama-Agents, a multi-agent system framework focused on production use cases, featuring a microservices-based architecture and a control plane for task orchestration.
  • The framework aims to provide scalability and flexibility for complex AI tasks, showcasing the growing trend of modular and collaborative AI systems in production environments.

4. Multimodal AI Progress

• VITA: Open-Source Interactive Multimodal LLM: A new paper titled "VITA: Towards Open-Source Interactive Omni Multimodal LLM" introduces an open-source approach to interactive multimodal large language models.
  • The project aims to bridge the gap between the capabilities of closed-source models like GPT-4 and open-source alternatives, focusing on both multimodal processing and interactive experiences.
• ColPali: Novel Approach to Document Embedding: ColPali offers a new method for document embedding by directly embedding screenshots of PDF pages, including images, charts, and tables, into vector representations.
  • This approach eliminates the need for OCR, layout analysis, and text chunking, potentially offering a more efficient and user-friendly solution for document retrieval and ranking in multimodal AI systems.
• Boundary Attention for Image Segmentation: A new lightweight, bottom-up model called Boundary Attention has been proposed for inferring color-based boundaries with high precision in image segmentation tasks.
  • Unlike traditional methods, this model infers unrasterized boundaries, including contours, corners, and junctions, using a field of embeddings that encode three-way partitions and associated windowing functions.

5. AI Safety and Governance

• California's SB 1047 Amendment: California's bill SB 1047, aimed at preventing AI disasters, has passed the Appropriations Committee with significant amendments, removing the requirement for AI labs to submit safety test result certifications "under penalty of perjury".
  • Instead, the amended bill now requires AI labs to provide public statements outlining their safety practices, reflecting a shift in approach to AI governance and safety regulations.
• Goodfire AI's Interpretability Mission: Goodfire AI, a public benefit corporation, is working to advance understanding of AI by examining the inner workings of advanced AI models, bridging theoretical science and practical applications of interpretability.
  • The company is building infrastructure to empower developers to understand, edit, and debug AI models at scale, aiming to ensure the creation of safer and more reliable AI systems.
• OpenAI's Short Model Expiration Policy: OpenAI has implemented a notably shorter model expiration time of 3 months, contrasting with the more common 1-year expiration period offered by other providers like Modal.
  • This policy highlights OpenAI's distinct approach to model lifecycle management and user access, potentially impacting how researchers and developers plan their projects using OpenAI's models.

PART 1: High level Discord summaries

Nous Research AI Discord

• RedPajama-Data: Preparing Datasets for LLMs: A user shared a link to the RedPajama-Data repository which contains code for preparing large datasets for training large language models.
  • The repository aims to support the training of large language models with high-quality, diverse data.
• Sarvam AI: Voice-to-Voice Agent: Sarvam AI, an Indian company, has developed a voice-to-voice agent that can speak in both English and Indian languages.
  • The company offers an interactive experience that allows users to engage with the agent by speaking in any Indian language, which can then be used to explain products, share presentations, and schedule meetings.
• LLMs Develop Understanding of Reality: A new study from MIT explores how large language models (LLMs) are developing their own understanding of reality.
  • Researchers found that LLMs can generate descriptions of sensory experiences, like the scent of rain, despite lacking real-world experience, suggesting that these models may be drawing upon their training data to generate these responses.
• Hermes 3 405B: Powerful New Open-Source Model: Hermes 3 405B is a powerful new open-source AI model that excels at a mix of tasks, including style transfer, summarization, and creative writing, often with tons of parallel instructions.
  • It outperforms the Meta's bf16 instruct model in these use cases, with response speeds only slightly slower than GPT-3.5 sonnet, making it a strong contender for research and development.
• RAG: The New Trend in AI: Charlie Marsh initially thought this link was a joke, but now must learn about the 12 types of RAG.
  • RAG is gaining traction and is being widely adopted, Charlie Marsh must learn what it is and the 12 different types.

aider (Paul Gauthier) Discord

• Aider Embraces Prompt Caching: A member highlighted the potential benefits of prompt caching, particularly for large codebases, elaborate system prompts, and numerous examples.
  • They cited Claude Dev's implementation as a positive example and suggested exploring this feature within Aider.
• OpenRouter's Prompt Caching Roadmap: There was discussion about whether OpenRouter currently supports prompt caching.
  • A member from the OpenRouter team confirmed that they are actively working on implementing this feature.
• Aider's New Feature: Code in JSON: A member shared a link to a blog post discussing the release of Aider's new feature: Code in JSON, which allows for structured code output.
  • The post details the benefits of this new feature and addresses why Aider previously preferred plain text formats.
• Aider's Weak Model: Customizing Your Workflow: There was a question regarding the role and purpose of the weak model in Aider, which is used for tasks such as commit message generation and chat history summarization.
  • A member clarified that users can opt to use the main model for all tasks by setting the --weak-model flag to the main model in the Aider configuration.
• Structured Responses: An Ongoing Debate: A member presented an alternative approach to structuring LLM responses using the Instructor library, which involves providing a pre-defined structure and fitting LLM data into it.
  • Other members, however, argued that this method could negatively impact model performance, citing Paul's blog post showing that models generate lower-quality code when restricted to JSON output.

Stability.ai (Stable Diffusion) Discord

• Flux Dev: A Possible SDXL Contender?: Flux Dev is a new model making waves with its controlnet support and improved prompt adherence, some users even suggesting it could be more popular than SDXL.
  • The model's capabilities are generating excitement within the community, with users exploring its potential for a wide range of applications.
• Model Merging: A Tactic Under Scrutiny: A member proposed a model merging tactic using UltraChat, Mistral, and Mistral-Yarn.
  • The tactic has garnered mixed reactions, highlighting the ongoing exploration of techniques to improve model performance within the community.
• Dreamshaper-XL v2 Turbo: Same Face, Different Poses?: A new user reported that Dreamshaper-XL v2 Turbo consistently generates images with the same face but different poses.
  • The user shared their code and sought help understanding the issue, highlighting the challenges of achieving image diversity in AI image generation.
• ComfyUI: Upscaling and Image Diversity: The discussion focused on improving image quality and diversity in ComfyUI, particularly regarding upscaling.
  • Users shared techniques like noise injection and using descriptive prompts to achieve better results, demonstrating the community's commitment to enhancing ComfyUI's capabilities.
• Flux AI: Impressive, but Not Perfect: One user expressed their positive experience with Flux AI, highlighting its ability to produce good results even with poor prompts.
  • The user's interest in using custom Loras to further improve the model's capabilities indicates the ongoing pursuit of personalizing AI image generation.

HuggingFace Discord

• Hermes 3 Special Tokens For Thinking: Hermes 3 has new special tokens for "thinking" including <SCRATCHPAD>, <REASONING>, <INNER_MONOLOGUE>, <PLAN>, <EXECUTION>, <REFLECTION>, <THINKING>, <SOLUTION>, <EXPLANATION>, and <UNIT_TEST>.
  • The report also details new tokens for RAG, tool calling, and structured JSON output, with the full report available here.
• DeepSeek Prover V1.5: Proof Assistant Feedback: DeepSeek-Prover-V1.5 introduces significant improvements and achieves new state-of-the-art performance on high school level miniF2F and undergraduate level ProofNet benchmarks.
  • This model leverages proof assistant feedback for reinforcement learning and Monte-Carlo Tree Search, detailed in a paper available on arXiv (https://arxiv.org/abs/2408.08152).
• Hyperspace P2P AI Network: Peer-to-Peer AI Network: Hyperspace is now available for users to join as a peer-to-peer AI network, offering various ways to participate.
  • This network features over 17,745 unique nodes and 100+ models, enabling users to serve LLMs, embedding models, re-rankers, vectors, and more to consumers and developers.
• OpenBLAS: Optimized for Intel Haswell CPUs: A member is learning to compile OpenBLAS for optimizing CPUs to run genAI workloads.
  • This release was compiled on Linux x86_64 Intel CPU but there are targets for ARM, POWER, MIPS, and RISC-V.
• Deploying YOLO Models on Robots: Using Viam: A blog post was written on Hugging Face about deploying YOLO models hosted on Hugging Face onto robots/machines in the real world using Viam.
  • The post describes a custom integration for yolov5 and yolov8 models to use them for real-time classifications and detections, with source code and a full tutorial available.

LM Studio Discord

• ForgeUI Adds Full Precision Support for Flux-dev: ForgeUI now supports Flux-dev at full precision using GGUF checkpoints.
  • It's currently unclear if this support will extend to other platforms such as automatic1111 or ComfyUI.
• Evaluating Fine-Tuned Models with Quantization: A user is seeking advice on evaluating their fine-tuned model after observing that a quantized version using GPTQ performs better than the original model.
  • However, when using GGUF or AWQ for quantization, performance decreases, prompting a discussion about LM Studio's capabilities for private bug reporting.
• LM Studio Server Setup and Connectivity Issues: A user encountered an error attempting to connect LM Studio to Obsidian.
  • The discussion identified potential issues related to LM Studio's server running on the LM Studio side and the need for CORS configuration.
• P40 Power Consumption: Myths Debunked: A common misconception about multiple P40s consuming 1kW for inference is false.
  • When used for LLMs, they draw power sequentially, resulting in a total consumption close to a single GPU (around 250W).
• Tensor Split & GPU Bottlenecks: Disabling offload to the GTX with tensor split (set to 0,1 or the opposite in the configuration file) is crucial, as a 2GB GTX will bottleneck a T4 with 4GB combined memory.
  • Search for 'tensor split' to learn more about this configuration option.

Perplexity AI Discord

• Perplexity AI Integrates with Knowledge Base: A user inquired about integrating Perplexity with AI knowledge base tools to automatically tag or file useful information from searches.
  • The user aims to streamline their workflow by capturing and organizing valuable insights from Perplexity results within their knowledge base.
• Hermes 3 Powers Two Channels on Discord: Two separate Discord channels are currently using Hermes 3 models, with users engaging in prompts and conversations.
  • The experimental setup allows for diverse interactions with the models, potentially leading to valuable insights and developments within the community.
• Batching Jobs for LLM Workloads: A blog post titled Unlocking the Power of Job Batching: Transforming AI Workloads on Medium discusses the advantages of batching jobs for LLM workloads.
  • The post highlights the efficiency gains and cost savings associated with batching, offering a practical approach to managing large-scale AI projects.
• Starbucks Leadership Shuffle: Brian Niccol, CEO of Chipotle Mexican Grill, has been appointed as the new Chairman and CEO of Starbucks, effective September 9, 2024.
  • This comes after Laxman Narasimhan stepped down after 17 months, with Rachel Ruggeri, Starbucks' CFO, serving as interim CEO during the transition.
• Thailand's Political Landscape in Turmoil: Thailand's political landscape is in turmoil following the removal of Prime Minister Srettha Thavisin from office by the constitutional court.
  • This highlights the ongoing struggle between Thailand's military-backed conservative establishment and reformist parties, raising concerns about the stability of democratic institutions.

OpenAI Discord

• AI is Not a Magic Wand, Just a Tool: The discussion highlights the misconception that AI should be able to do everything, dismissing it as useless when it can't perform simple tasks like counting letters.
  • Users emphasized the importance of understanding AI as a tool with specific applications, similar to how a hammer is used for construction, not as a self-sufficient builder.
• TikTok Fuelled ChatGPT Hype: The conversation attributed the widespread popularity of ChatGPT to its free accessibility and TikTok's amplified enthusiasm, leading to a surge of users utilizing it for tasks like homework.
  • The discussion also touched upon the trend of emphasizing AI models' performance on benchmarks like LMSYS, generating excitement based on high scores without a nuanced understanding of their capabilities.
• Banning ChatGPT in Education is Counterproductive: The discussion debated the ethical implications of using AI for homework, with some arguing against banning ChatGPT, emphasizing its potential as a learning tool for students who understand how to utilize it.
  • Participants envisioned a future where AI integration into education systems will revolutionize learning, adapting to individual needs and providing a more efficient and personalized approach.
• Grok2's Token Limit and Context Window: The conversation explored the token limit of Grok2, with users sharing their experiences with encountering a message limit that prompted a request for summarization before continuing the conversation.
  • It was suggested that Grok2's context window could be limited to 8k tokens, impacting its ability to process longer conversations effectively.
• Gemini Voice vs ChatGPT Voice: A discussion arose regarding the emotional expressiveness of AI voice models, comparing Gemini Advanced Voice to ChatGPT's voice capabilities, which some perceived as more emotional and engaging.
  • The conversation also touched upon the lack of web search functionality in ChatGPT's Advanced Voice and its potential limitations compared to other models like Gemini Live.

Interconnects (Nathan Lambert) Discord

• OpenAI's ToS: A Legal Minefield: A former employee shared that their company was cleared to train on generations from OpenAI that third parties made and released under a permissive license, but couldn't directly make the generations themselves.
  • They suggested that using outputs for training may be a legal risk but with no one getting banned, it's not a major concern.
• SB 1047's Impact on AI: SB 1047, a California bill aimed at preventing AI disasters, has passed the Appropriations Committee with amendments.
  • The amendments remove the requirement for AI labs to submit certifications of safety test results "under penalty of perjury," and instead require public statements outlining their safety practices.
• Sentdex: From YouTube to Farm Life: Sentdex, a popular YouTuber known for teaching neural nets and Python programming, has gained significant recognition for his tutorials, including "Python plays Grand Theft Auto V" and "Neural Networks from Scratch in Python."
  • He is no longer actively creating content, but his work has impacted many, including the person asking about him. Sentdex is now focusing on his farm after achieving success through his projects, domain reselling, books, and YouTube channel.
• The Difficulty of Evaluating Models: A disagreement involving Nous Hermes on the Nous Discord, with accusations of rudeness directed towards an individual, highlighted the complexities of evaluating language models.
  • This individual was criticized for using default LM Harness settings, despite them not being explicitly mentioned in a paper, suggesting a potential misunderstanding or misinterpretation of the research.
• Deeply, the new very?: The author noticed a rise in the usage of the word 'deeply' in public discourse and believes it has become the universal adverb.
  • The author referenced Merriam-Webster's definition of the word 'cant' and suggested 'deeply' is replacing 'very' in similar fashion.

Latent Space Discord

• Salesforce's DEI Framework for SWE Agents: Salesforce released DEI (Diversity Empowered Intelligence), an open-source AI software engineering agent organization that leverages SWE agents' unique expertise.
  • DEI functions as a meta-module atop existing SWE agent frameworks, managing agent collectives for enhanced problem-solving, achieving a 34.3% resolve rate on SWE-Bench Lite with a group of open-source SWE agents, exceeding the best individual agent's performance by a large margin.
• DeepSeek-Prover-V1.5: Proof Assistant for RL & MCTS: DeepSeek-Prover-V1.5 harnesses proof assistant feedback for Reinforcement Learning (RL) and Monte-Carlo Tree Search (MCTS), achieving significant improvements.
  • It achieved new state-of-the-art (SotA) on both the high school level miniF2F bench (63.5%) and the undergraduate level ProofNet bench (25.3%).
• DSPy: Not Yet Commercialized, but Omar's Working on It: A member asked if there is a commercial company behind DSPy, and another responded that there isn't yet, but Omar is obviously working on it.
  • The member also noted that they went to Cursor's office meetup yesterday and were told there is no alpha to share yet, but Cursor says hi.
• New Latent Space Pod Episode Released: A new episode of the Latent Space Pod is available, featuring guest Jeremy Howard.
  • This episode delves into the founding journey of AnswerAI, the OpenAI governance crisis, and Howard's plans to scale AI research and development.
• Choosing the Right Embedding Model for RAG: This article guides users through the Hugging Face MTEB (Massive Text Embedding Benchmark) leaderboard to select suitable embedding models for their Retrieval Augmented Generation (RAG) applications.
  • It explains the difference between Bi-Encoder and Cross-Encoder models, how embedding models are benchmarked, and how to select a baseline embedding model for your use case.

Cohere Discord

• Cohere Startup Program: Helping Startups Integrate AI: The Cohere Startup Program offers discounts and support to Series B funded startups who want to integrate AI into their core operations.
  • This program provides access to Cohere's powerful AI tools and expertise, empowering startups to build innovative solutions.
• Cohere's Training on Oracle Fusion SaaS: A user is seeking information on how well Cohere is trained on Oracle Fusion SaaS applications.
  • This demonstrates the growing demand for AI solutions that can seamlessly integrate with existing enterprise software systems.
• Tokenizing with Cohere: AutoTokenizer vs llamatokenizer: The Cohere community is the best place to get an answer on the differences between AutoTokenizer and llamatokenizer.
  • The community at Cohere For AI is a valuable resource for open-science research and practical advice on using Cohere tools.
• LLM University API Key Usage: Production or Not?: A user is unsure if using Cohere API keys for small exercises in LLM University modules would be considered production deployment.
  • The question highlights the importance of understanding API usage policies, especially when using AI tools for educational purposes.
• R+ API: Missing Guidelines Layer: A user asked if there is a guidelines layer on top of the R+ API separate from the local model.
  • This concern suggests that the model may be generating hallucinations, which is a known issue in large language models, highlighting the need for robust safety and ethical considerations.

LlamaIndex Discord

• LlamaIndex's Multi-Agent System Framework: Llama-Agents: LlamaIndex is building a multi-agent system framework called Llama-Agents, which focuses on production use cases.
  • This framework prioritizes scalability and flexibility through a microservices-based architecture, featuring a control plane for task orchestration and key components for seamless operations.
• Generating Multimodal Reports with LlamaIndex's Agents: LlamaIndex is showcasing an automated multi-agent system capable of conducting research over a multimodal RAG (Retrieval Augmented Generation), compiling information into a knowledge bank.
  • This system dynamically generates multimodal reports that combine text and images, adapting to user queries and delivering comprehensive insights.
• Streamlining Control Flow with LlamaIndex Workflows: LlamaIndex is highlighting the power of workflows, demonstrating their ability to streamline complex processes with decorators and types for control flow definition.
  • Workflows enable event-driven process chaining and customization, empowering users to create sophisticated steps for intricate tasks and scenarios.
• Exploring LlamaIndex's Implementation of GraphRAG: LlamaIndex's implementation of GraphRAG shares similar ideas with the original Microsoft version, focusing on building communities and retrieving information based on them.
  • However, the extent of its differences with Microsoft's complex codebase is unclear, and LlamaIndex primarily referenced the paper for its implementation.
• Anthropic's Performance: Code Refactoring and Idea Iteration: A user reported initial negative experiences with Anthropic, but upon pasting their code into the platform and asking for assistance, it successfully identified and fixed the issues.
  • This highlights Anthropic's potential for code refactoring and idea iteration, particularly when using its sonnet-3.5 model.

LangChain AI Discord

• LangChain's Tool Arsenal Expands: A user inquired about tools built for LangChain agents beyond the LangChain documentation, leading to suggestions of exploring OpenAI Actions, MindSQL, and the Awesome LangChain repository.
  • These tools aim to empower developers with more flexibility in creating and customizing LangChain agents for specific use cases.
• Post-Tool Execution with LangGraph: A user, new to LangGraph, sought guidance on executing a function after tool usage within LangGraph's ToolNode.
  • The user hoped to find a parameter within LangGraph's ToolNode that allowed for function execution directly following tool usage.
• Llama Model Integration Trouble: A user experienced issues while using ChatHuggingface with a locally hosted Llama model.
  • The user requested assistance with identifying and resolving the error, prompting a suggestion to post the question in a relevant channel for more focused support.
• Optimizing Embeddings for Accurate Retrieval: A user reported a retrieval issue with irrelevant data being fetched, suspecting embedding problems.
  • The user, utilizing Ollama Embeddings and Chroma for embeddings and retrieval respectively, sought advice on choosing suitable embedding models and optimizing the entire process.
• Unveiling the Cache's Speed Boost Secrets: A user observed a speed increase with caching in .invoke() and .batch() operations, but found that .batch_as_completed() remained slow.
  • Despite the cache being populated after the first run, the user questioned whether .batch_as_completed() was actually utilizing the cache and sought an explanation for this behavior.

Eleuther Discord

• Boundary Attention: Lightweight Image Segmentation: A new lightweight, bottom-up model is proposed for inferring color-based boundaries with high-precision, using Boundary Attention.
  • This model, unlike traditional methods, infers unrasterized boundaries, including contours, corners, and junctions, from the bottom-up, using a field of embeddings that encode three-way partitions and associated windowing functions.
• Language Model Probability Computation Errors: A recent paper (View PDF) highlights that many recent linguistic studies have been incorrectly computing word probabilities in language models, particularly those using beginning-of-word (bow) tokenizers.
  • This paper proposes the correct methods for computing word probabilities, highlighting how inaccuracies in these computations can affect the measured outcomes in sentence comprehension and lexical optimization analyses.
• Fine-tuning Gemma-2-2b without LayerNorm: A member is looking for a collaborator or training script for fine-tuning Gemma-2-2b (or a similar model) without LayerNorm.
  • They are inspired by a previous attempt to fine-tune GPT2 without LayerNorm, resulting in only slightly worse performance, and they're curious if this method can be applied to larger models.
• Goodfire AI: Demystifying AI's Inner Workings: Goodfire AI is a public benefit corporation with a mission to advance humanity's understanding of AI by examining the inner workings of advanced AI models, bridging the gap between theoretical science and practical applications of interpretability.
  • They are building critical infrastructure that empowers developers to understand, edit, and debug AI models at scale, ensuring the creation of safer and more reliable systems.
• Llama3-8B-Instruct matches GSM8k results: A user reported success reproducing Meta's GSM8k performance using Llama3-8B-Instruct with a specific prompt format and settings: https://huggingface.co/datasets/meta-llama/Meta-Llama-3.1-8B-Instruct-evals/viewer/Meta-Llama-3.1-8B-Instruct-evals__gsm8k__details?row=0.
  • This required adjusting the regex expression and creating a new .yaml file for the GSM8k-cot task. The user offered to share the .yaml file and will need to do the same for other datasets to reproduce Meta's results.

DSPy Discord

• Neural Search Repositories Explored: One member shared a GitHub repository for Neural Search designed to enhance search functionality using neural networks.
  • Another member showcased a GitHub repository for a modular AI assistant that handles audio, image, and text processing.
• New Paper on Neural Networks for Text Retrieval: A member linked an arXiv paper titled "Neural Network for Text Retrieval" with contributions from various authors.
  • The paper explores the use of neural networks in text retrieval, discussing their advantages and applications.
• Self-Taught Evaluators for LLMs: A new approach called "Self-Taught Evaluator" aims to improve LLM evaluators without human annotations, using only synthetic training data.
  • This approach generates contrasting model outputs, trains an LLM-as-a-Judge to produce reasoning traces and final judgments, iteratively improving predictions.
• Hybrid RAG System for Enhanced Reasoning: A hybrid RAG system is introduced, incorporating optimizations that enhance retrieval quality, reasoning capabilities, and numerical computation ability.
  • This system utilizes refined text chunks and tables from web pages, attribute predictors to reduce hallucinations, LLM Knowledge Extractor and Knowledge Graph Extractor, and a reasoning strategy with all the references.
• WeKnow-RAG: Integrating Web Search and Knowledge Graphs: WeKnow-RAG integrates Web search and Knowledge Graphs into a "Retrieval-Augmented Generation (RAG)" system to enhance the accuracy and reliability of LLM responses.
  • It combines the structured representation of Knowledge Graphs with dense vector retrieval, improving LLM responses by utilizing both structured and unstructured information.

Modular (Mojo 🔥) Discord

• Mojo: General Purpose Programming Language: Mojo is intended to be a general-purpose programming language that aims to enable easy-to-read and efficient "Python-like" codebases across various domains, including AI, while also extending to fields beyond it.
  • However, for specific tasks like GPU shaders, Mojo requires Max for compilation due to the lack of alternative programming methods for Mojo on GPUs.
• Mojo's Runtime: Minimal but Mighty: Mojo will function as a language with a minimal runtime, with essential features like GPU scheduling and asynchronous operations being handled by Max.
  • This runtime is crucial for ensuring efficient execution of Mojo code, especially in performance-sensitive applications.
• String Indexing Debate: Code Points vs Grapheme Clusters: A member raised the concern that using code points for string indexing might not be the most efficient approach, suggesting that grapheme clusters could be a better choice, particularly in the context of string processing tasks.
  • Another member proposed an index_type parameter for Strings, allowing for cases like byte, codepoint, and grapheme, giving users maximum control over indexing and optimization based on their specific data and requirements.
• Mojo Installation Error on WSL Ubuntu 24.02 LTS: A user reported an error, "modular: error: invalid manifest: expiration has passed", while attempting to install Mojo on WSL running Ubuntu 24.02 LTS.
  • The error message suggests that the Mojo manifest file used for installation has expired, which can be addressed by checking for a newer version or potentially updating the environment setup and paths.
• Potential Memory Efficiency Improvements: A member expressed concern about the efficiency of using memcpy in combination with zeroing and index building, resulting in three passes over the memory.
  • They suggested that fusing the copy and indexing operations could potentially improve performance by reducing the number of passes over the memory, leading to more efficient use of memory resources.

OpenInterpreter Discord

• Raspberry Pi 5: Power-Efficient Choice for OpenInterpreter: A user pondered the advantages of using Raspberry Pi 5 over Umbrell for OpenInterpreter.
  • Another user suggested Raspberry Pi 5 due to its lower power consumption and ARM architecture, making it a more efficient option for running OpenInterpreter.
• Harnessing Gemini Models with OpenInterpreter OS: A user sought a beginner's guide on implementing Gemini models within the Open Interpreter OS environment.
  • A helpful user provided code snippets and installation instructions, recommending flags like --model, --api_key, --local, and --os for seamless execution.
• Alexa Echo Dot: Local Server Connection via Ollama: A user inquired about a possible workaround to connect an older Alexa Echo Dot to a local home server using Ollama.
  • No responses were provided regarding this topic.
• OpenInterpreter Discord: A Quiet Day: A user remarked on the low activity levels on the OpenInterpreter Discord server.
  • Another user confirmed that it was a relatively quiet day on the platform.

LAION Discord

• Musk/X: No Big Deal: A user stated that Musk/X seems to be doing fine as journalists and politicians are only focused on "Musk/X Bad!" and don't look into the details.
  • The user pointed out that things could escalate and "Stanford researchers" could dig further and find issues, but ultimately implying that things are fine and the media hype is overblown.
• Stanford Researchers: In Search of Problems: A user jokingly suggested that "Stanford researchers" might find issues with Musk/X in the future, even if there's nothing actually wrong.
  • Another user agreed and joked that "Stanford is working hard", implying that Stanford researchers are always looking for problems to solve.
• Moonglow: Streamlined GPU Access: Moonglow is a VSCode extension that allows you to connect your Jupyter notebooks to remote cloud GPUs, like those offered by Runpod.
  • Moonglow simplifies the process of starting, connecting to, and stopping a Runpod instance with A100s or H100s in under a minute, simplifying the workflow for ML research.
• Moonglow: Simplifying Cloud Compute: Moonglow eliminates the need for managing SSH keys, package installations, and other DevOps tasks, allowing seamless switching to cloud compute in seconds.
  • Users can pick any GPU they need (A40s, A100s, H100s, and more) and manage compute directly within their IDE, all while avoiding typical SSH hassles.
• Moonglow: Expanding Cloud Integration: Moonglow currently supports connecting notebooks in VS Code/Cursor to Runpod and AWS.
  • The team is open to expanding Moonglow's capabilities to support other setups, encouraging users to reach out if they have specific needs or requests.

DiscoResearch Discord

• xLSTM Trainer Released: A Hugging Face compatible xLSTM trainer was recently released by a member.
  • They shared a link to the repository on GitHub.
• xLSTM Poised to Replace Transformers?: The member believes that xLSTM may eventually replace transformers.
  • It remains to be seen how this will play out in the future.

Alignment Lab AI Discord

• Jala: Automating Data Labeling: Jala, an automated text data labeling interface, uses AI for high accuracy and efficiency, supporting various data types (e.g., CSV, JSON, TXT, XML) and scaling for large datasets.
  • It integrates with existing workflows for use cases like NLP, machine learning and AI model training, and data annotation, with automated content categorization capabilities.
• Jala: Join the Waitlist: Jala is coming soon! Sign up for the waitlist to be among the first to experience it and receive updates on its progress.
  • This innovative data labeling solution is available at Jala - Data Labeling Solution.

LLM Finetuning (Hamel + Dan) Discord

• OpenAI's Short Model Expiration: OpenAI has a much shorter model expiration time of 3 months compared to other providers, which typically offer 1-year expiration periods.
  • This shorter timeframe emphasizes OpenAI's approach to model lifecycle management and user access.
• Modal's Flexible Expiration Policy: Modal provides a standard 1-year expiration period for models, but allows users to extend this time after expiration.
  • This flexibility provides users with greater control and adaptability, accommodating varying project requirements.
• General Model Expirations: The prevalent model expiration time is 1 year, with most providers adhering to this standard, including Modal.
  • However, extensions are often possible with these providers, enabling continued model usage beyond the initial expiration.

The MLOps @Chipro Discord has no new messages. If this guild has been quiet for too long, let us know and we will remove it.

The Mozilla AI Discord has no new messages. If this guild has been quiet for too long, let us know and we will remove it.

The AI21 Labs (Jamba) Discord has no new messages. If this guild has been quiet for too long, let us know and we will remove it.

PART 2: Detailed by-Channel summaries and links

The full channel by channel breakdowns have been truncated for email.

If you want the full breakdown, please visit the web version of this email: !

If you enjoyed AInews, please share with a friend! Thanks in advance!