[AINews] AI Discords Newsletter 11/15/2023
This is AI News! an MVP of a service that goes thru all AI discords/Twitters/reddits and summarizes what people are talking about, so that you can keep up without the fatigue. Signing up here opts you in to the real thing when we launch it 🔜
Guild: Latent Space
Latent Space Guild Summary
- A discussion initiated by @chef on the best small models for instruction finetuning which can be trained on a single A100 and available on the Hugging Face hub. @eugeneyan suggested the 7B classes for mpt-instruct, falcon-instruct, and mistral as feasible options, further adding that with Lora, they can fit in 16GB GPU ram.
- @last_ride_1707 sought advice on defining and tracking metrics for a freemium code generation tool pre-PMF. @coffeebean6887 further asked for clarification whether the metrics were for testing code generation or growth metrics for adoption.
- @Phill asked for suggestions on implementing Slack bots as frontend User Experience for OpenAI Assistants.
- @swyxio cited a Twitter post reflecting on the instability of copyright. The discussion highlighted its potential impact on forthcoming lawsuits.
- @coffeebean6887 expressed interest in model routing companies like Martian, Open Router, Pulze, etc., and asked the community for similar company suggestions, feedback, or preferences.
- A discussion by @mitch3x3 concerning the nuances of copyright lawsuits, hinting the industry could look into different kinds of licenses akin to those used in code.
- An interesting ycombinator discussion pointed out by @swyxio on fine-tuning results.
- Huggingface DPO paper discussion spearheaded by @swyxio in the ai-event-announcements channel. Members were encouraged to participate.
- The discussion scheduling, topic links for the llm-paper-club included a Discord link for a discussion on DPO shared by @eugeneyan, and the Code Diffusion Model paper link shared by @picocreator.
- @swyxio announced that the upcoming week, @296887155819675650 would lead discussions on two research papers: Paper 1 and Paper 2
- @youngphlo recommended a paper as the trigger for the recent research explosion.
- @slono inquired about whether the code fusion paper had been reviewed in a session.
Latent Space Channel Summaries
Channel: ai-general-chat
Summary (3 messages):
AI General Chat Summary:
- Instruction Finetuning in Small Models: @chef inquired about the best small models for instruction finetuning that could be trained on a single A100 and available on HF hub. @eugeneyan responded by suggesting the 7B classes for mpt-instruct, falcon-instruct, and mistral as suitable options, adding that with Lora, they can fit in 16GB gpu ram.
- Metrics for Freemium Code Gen Tool: @last_ride_1707 asked for advice on defining and tracking metrics for a code generation tool that follows a freemium model and is pre-PMF. @coffeebean6887 sought to clarify whether the expected metrics were for testing code generation or regarding growth metrics for adoption.
- Slack bots as UX for OpenAI Assistants: @Phill requested suggestions on the easiest way to implement Slack bots as frontend User Experience for OpenAI Assistants.
- Copyright Stability: @swyxio shared a comment and a Twitter link discussing dissent in the stability of copyright which may impact upcoming lawsuits.
- Model Routing Companies: @coffeebean6887 is researching into model routing companies like Martian, Open Router, Pulze, etc. and inquired about any other similar companies, feedback or preferences.
- Copyright Lawsuit Nuances: @mitch3x3 expressed hope for nuanced outcome from copyright lawsuits, suggesting that the industry might consider different types of licenses similar to those in code.
- Finetuning Results: @swyxio pointed out an interesting discussion on finetuning results through a ycombinator link.
Channel: ai-event-announcements
Summary (3 messages):
Huggingface DPO Paper Discussion:
- Announcement: @swyxio announced they're starting a discussion about the Huggingface DPO paper. Other members are encouraged to join.
Channel: llm-paper-club
Summary (3 messages):
Summary of topics discussed:
- Scheduled Discussion Times: @yikesawjeez and @eugeneyan discussed scheduling for upcoming discussion sessions.
- Links to Discussion Topics:
- @eugeneyan shared a Discord link for the discussion on DPO.
- @picocreator shared a link to the Code Diffusion Model paper and ignored the 3.5 turbo drama.
- @swyxio mentioned that next week @296887155819675650 will lead through two papers: Paper 1 and a bit of Paper 2.
- @youngphlo suggested this paper as the reason for the recent explosion.
- Discussion Inquiry: @slono inquired if the code fusion paper had been discussed during the session.
Guild: OpenAI
OpenAI Guild Summary
- Several deep discussions on the implications and potential of AI consciousness, with a focus on the definition, empirical vs non-empirical assessment, and the capabilities of OpenAI models. Exploration and concerns over the ability to further train an AI beyond its built-in abilities also emerged. (Quotes from @bedros_p, @nauticalstache, and @solbus)
- Various users reported performance issues with GPT-4, including longer response times and decreased accuracy. Discussions on the value of ChatGPT subscription, server and access issues, use of plugins, desired features, and the functionality of DALL-E and Vision plugin.
- Users queried issues in custom GPTs, such as problems with POST actions, GPT's formal nature, and various bugs. Development tactics, including a proposed medical GPT and study aids, were discussed. Unauthorized access and plus account cancellations were highlighted. (references by @whackscript, @colonelgentleman, and others)
- User reactions to the performance of GPTs, changes in pricing and terms of use, customizing GPT, and a debate on public versus private GPTs took place. Questions on calling external API from GPTs were a recurrent theme, with several GPT demos and helpful OpenAI links shared.
- Prompt engineering was explored, with an emphasis on humanizing text and generating structured content in markdown. Users delved into the behavior and thinking patterns of AI and its possible inconsistencies. Other discussion topics included generating long code snippets and a two-part argument and poem prompt.
- Detailed queries on the Quality of AI's responses, experiences with different versions of ChatGPT, and contextually guiding AI with simulation scenarios and markdown tables were discussed. Users also pointed out usability issues with the ChatGPT UI and posed unique AI challenges.
OpenAI Channel Summaries
Channel: ai-discussions
Summary (6 messages🔥):
Discussion on Consciousness and AI:
- Defining AI Consciousness: The users held an in-depth discussion on the nature of consciousness and whether AI can attain it. @bedros_p, @nauticalstache, and @solbus debated the potential for AI exhibiting consciousness. They explored differing views on sentience, qualia, and whether these characteristics could be replicated in AI.
- Empirical vs. Non-Empirical Assessment: The debate also touched upon evaluating consciousness from empirical and non-empirical perspectives. @solbus posited that everything in our experience of the universe seems to support empirical inquiry, while @bedros_p argued for the worth of exploring non-empirical assertions and implied consciousness could exist beyond material explanations.
- Capabilities of OpenAI Models: @nauticalstache mentioned that OpenAI had to put limiters into Model 4 because it was already at General Intelligence Level 3. However, there was a debate around these claims, focusing on whether AI models could truly attain consciousness similar to humans.
- GPT's Capability to Learn: @tilanthi raised concerns about the ability to further train an AI beyond its built-in abilities, questioning if their efforts at training an AI were indeed improving its performance or not. @solbus explained the distinction between the "Create" and "Configure" processes in GPT Builder, clarifying how it uses knowledge files and communication with the model.
- Changes in OpenAI's platform: @zahmb and @foxabilo discussed apparent changes in the OpenAI platform, particularly the removal of threads and messages from the navigation bar in the platform UI.
Channel: openai-chatter
Summary (6 messages🔥):
OpenAI Discord Chat Summary
- Discussion on GPT-4 Performance: Many users like @libertadthesecond, @exx1, and @bbq_man discussed their experience about the change in performance of GPT-4, including longer response times, decreased accuracy in retrieving information, relying heavily on Bing for internet searches, and forgetting previous conversations quickly.
- ChatGPT Subscription: @mklbash and @7_vit_7 shared about their subscriptions being cancelled unexpectedly. @babysitter1003 sought alternatives to get access to GPT-4. Users discussed the price and value of GPT Plus subscriptions in this context.
- Server & Access Issues: Many users including @user101, @stunspot, and @_ciphercode reported having problems accessing the service, slow loading times, and errors. @solbus and @rjkmelb provided updates on the server overload and wait times due to increased user activity.
- Custom GPT and Plugins: @atariikafa asked about the ability to get a custom GPT to reference docs from GitBook, and @da.a sought suggestions about best plugins to use with ChatGPT.
- Discussion about Features & Requests: @ted_k, @bbq_man, and @libertadthesecond discussed desired features like the ability to extend or turn off certain capabilities, more accurate browsing ability, and improved conversation memory. @mrhumble wanted to understand how the JSON response works in the OpenAI library.
- Visual and DALL-E Functionality: @chief_executive discussed the ability to outline things using the Vision plugin. Multiple users had discussions about the reliability and limitations of the DALL-E functionality.
Channel: openai-questions
Summary (6 messages🔥):
User Query and Platform Issues:
- Issues with Actions in Custom GPTs: @whackscript discussed encountering problems with using POST requests in custom GPTs when sending data, suggesting JSON was being added incorrectly to query strings instead.
- Personality Aspects of GPT: @colonelgentleman and others shared frustration at GPT's tendency to be "formally helpful" and provide preambles before responses, with discussions suggesting this is inherent to the model.
- Bugs and Errors: Multiple errors were reported across various GPT functionalities, including:
- Error: "Cannot have multiple custom actions for the same domain" and "Error saving draft"
- "Message in Conversation Not Found" error in Chrome Incognito mode
- GPT doesn't look up in uploaded documents
- Issues with the use of non-English languages with DALL·E
- "We've updated our Terms of Use and Privacy Policy" pop-up prevents interaction
- Unable to view and manage plan options
Custom GPT and Chatbot Development:
- Tactics in Building GPTs: Users @juregg and @justzakary discussed their preferred methods for building GPTs, indicating a mixed approach using both chat with GPT builder and typing into instructions directly.
- Medical GPT: @juregg and @thesocraticbeard discussed the potential of creating a medical GPT using anatomy and pediatric resources, with a debate on whether these models could be automatically deleted.
- Study Aids: @juregg proposed building a GPT for assisting users in studying for SATs and ACTs.
External Factors:
- Unauthorized Access: User @_ardj_ reported recurrent unauthorized access to their chat GPT Plus account, and a discussion about revoking access via Google settings emerged.
- Plus Account Cancellations: @dercrisb94111 and @obrzutw discussed experiencing GPT Plus subscription cancellations due to payment issues and automatic renewals not taking place.
- Website Verification: Problems with website verification txt files on Godaddy were mentioned by @soltzu.
Channel: gpt-4-discussions
Summary (6 messages🔥):
Discussions on Custom GPTs and Development Updates:
- Performance of GPTs: Users reported various issues with their GPTs. Some experienced delay in response times after uploading large amounts of data, while others faced difficulty in saving updates. A recurrent problem seemed to be with GPTs not functioning as per their given instructions or failing to effectively use uploaded files.
- Changes in Terms of Use and Pricing: There were complaints about change in usage limit from 100 to 25 messages. Users also discussed issues related to the pricing structure and payment models of OpenAI's products.
- Customizing GPTs: A variety of techniques to customize GPTs were discussed. These ranged from the format of instructions to actions and making GPTs more accessible to non-tech users.
- Public vs Private GPTs: Users debated the pros and cons of making a GPT public or keeping it private. They also discussed the visibility of public GPTs and shared various avenues where one can publish or advertise their GPTs.
- External API Calls from GPTs: Several questions and problems regarding calling an external API from GPTs were discussed. Users shared their experiences and difficulties in the setup process.
- Links:
Channel: prompt-engineering
Summary (6 messages🔥):
Prompt Engineering and Chatbot Behaviour Discussions:
- Prompt Engineering for Humanizing Text and Formatting Issues: Users lodosd, lumirix, neighbor8103, and flaskie discussed prompt engineering difficulties in humanizing a text and formatting a .txt file into structured content with headings and paragraphs. Part of the discussions involved adapting the instructions to provide good input for AI for it to generate a more helpful output.
- Discussion on AI's Behaviour and Thinking Patterns: User mnjiman shared his thoughts on identifying 'thinking patterns' in AI and how its intent to assist shapes its responses. They also discussed the AI's perceived 'moods' and inconsistencies in AI's responses.
- Chatbot Performance Issues: Users taholmes160, .pythagoras, and solbus discussed limitations and performance issues with different versions of GPT, and how these affect their tasks, especially generating long code snippets and organizing information as they intended it to be.
- Seeking Help for Markdown Table and Command Parameters: Users .pythagoras and no.iq sought help regarding generating a markdown table and finding parameter adjustment settings in the new UI, respectively.
- Argument and Poem Prompt: User ertagon shared a two-part prompt where a paragraph arguing for the non-existence of tomatoes using biological reasoning is generated first, then transformed into a poem in the second part.
Channel: api-discussions
Summary (6 messages🔥):
OpenAI Discord "api-discussions" Summary:
- Prompts and Response Quality: Various users like @flaskie and @eskcanta had discussions about the quality of the AI's responses. @flaskie illustrated his difficulties with prompt engineering, and @eskcanta pointed out conflicts in @flaskie's instructions to the AI. Furthermore, @taholmes160 mentioned having to start new sessions to get a different response for the same question.
- ChatGPT Versions: Discussed the different experiences with varying versions of ChatGPT. @.pythagoras mentioned encountering issues with the GPT 4 Turbo, while @taholmes160 used the GPT 4-1106-preview without major issues. Differences between ChatGPT and custom GPTs were also discussed.
- Contextual Approaches: @mnjiman talked about identifying the AI's 'thinking patterns' and suggested contextual solutions for enhanced interactions. He proposed using simulation scenarios and markdown tables to guide the AI and structure the conversation.
- Functionality/UI Changes: @no.iq pointed out that altering parameters like 'temperature' is not apparent in the new UI, indicating potential usability issues.
- Unique Challenges: In a playful manner, @ertagon proposed an interesting challenge: to get the AI to make a biological argument denying the existence of tomatoes, and then formulate the argument into a poem.
Guild: LangChain AI
LangChain AI Guild Summary
- Questions and issues raised about LangChain AI Discord Chatbot: Issues regarding certain prompt's timing out, the inclusion of the new OpenAI Vision API in LangChain, need for web scraping tools and integrations, the different usage of LangChain for data analysts and developers, problems with Direct Retrieval in GPT-4, questions on reproducing a notebook due to changing import routes, possibility of chatbot generating an optimized prompt based on user input, interest in multimodal RAG, advanced examples of agents with data warehouses, possibility of chaining LangChain objects, and an error in converting an API response.
- Discussion on semi-structured RAG template and the integration with docstore/vectorstore: Challenges related to the semi-structured RAG template and the persistent data storage where the entire file is loaded into the context with no search in the RAG template. Efficiency of Vectorstore in similarity search for a production app and the issue with text not having a vector/semantic search.
- Sharing of work on hierarchical cooperative multi-agent framework integration: Websocket connectivity established among various AI platforms such as Gradio, Chainlit, Tkinter, PySimpleGUI. Mention of a basic framework for AI-driven question-answering logic allowing two different models/agents to communicate with each other. Future plans include integrating the established network with a case-specific agent from the LangChain repository to drive server<->client connectivity.
- relevant Github links: ServerMain.py, ChainlitCli.py, Multiagent_authoritarian
- relevant HuggingFace spaces: ServerNeural, QA-Docs-Chainlit-Langchain
- No information available regarding the tutorials channel due to a lack of messages.
LangChain AI Channel Summaries
Channel: general
Summary (4 messages):
LangChain AI Discord Chatbot Messages Summary:
- Issues with certain prompts: @fishyccy experiences issues with certain prompts timing out and seeks help. @s_papu_25 offered to assist in DMs.
- Inquiry about OpenAI Vision API: @vudumagic queries if the new OpenAI vision API is a part of LangChain.
- Request for web scraping tools: @vj19 asks about tools and integrations to scrape data from websites where a date range and keywords need to be entered into a search bar.
- Discussion on LangChain usage for Data Analysts vs Developers:: @coreyb42 discusses the difference in using LangChain for data analysts and developers, suggesting GPT for analysts but also highlighting that GPT may obfuscate functionality for developers.
- Problems with Direct Retrieval in GPT-4: @.cannaboss warns about a failure of direct retrieval with GPT-4 using the new OpenAI Assistant API, he argues for multiple learned "knowledge planes" with trained weight tensors for specific tasks that can be switched in.
- Inquiry on Route Imports: @jlcases is having trouble reproducing a notebook due to changing import routes and requests for URLs to the updated routes.
- Question on LLM Chatbots: @minecraftjuicer asks if it's possible for a chatbot to generate an optimized prompt based on user input.
- Interest in Multimodal RAG: @iloveh8 inquires if anyone has worked with multimodal RAG.
- Looking for Advanced LangChain examples: @dennisyurk is searching for more advanced examples of agents that operate with data warehouses.
- Pair Programming Offer: @.trouble_ offers to pair up on projects, using his experience on LangChain and a background in software development.
- Chaining LangChain objects: @ethereon_ asks if it's possible to chain LangChain objects together, @syntactic__ confirms that it's possible.
- API Conversion Error: @jinwolf2 has an error while converting an API response and seeks help to resolve it without affecting the OpenAI function.
Channel: langchain-templates
Summary (4 messages):
Semi-Structured RAG Template and Docstore/Vectorstore Integration Discussion:
- Challenges with the semi-structured RAG template and persistent data storage: @alex_35579 pointed out that in the current model, the docstore, loaded from a text array, does not persistently store data needed for the Real RAG. There is a need to integrate it with LangChain retrievers for precise searches. They also highlighted that the entire file is loaded into the context with no search in the RAG template, which can't function in a practical setting.
- Efficiency of Vectorstore in similarity search for a prod app: On a positive note, they observed that Vectorstore integrates with the LangChain retrievers and can use similarity search for a prod app.
- Issue with text not vector/semantic search: They reiterated that this problem also affects any search that is text-based and not vector/semantic. They have requested assistance with these issues.
Channel: share-your-work
Summary (4 messages):
Discussion on hierarchical cooperative multi-agent framework Integration:
- Websocket connectivity between AI platforms: @arcypojeb reported having successfully established websocket connectivity among various AI platforms and interfaces namely Gradio, Chainlit, Tkinter, PySimpleGUI. The implementation facilitates client-server interactions and secure connections.
- Development of AI-driven QA logic: Additionally, @arcypojeb mentioned the development of a basic framework for question-answering AI-driven logic, and shared that two completely different models/agents can communicate with each other.
- Future plans: Plans to integrate the established network with a case-specific agent from the LangChain repository were also discussed. The aim is to utilize this agent as the main logic driving the server<->client connectivity.
- Links:
Channel: tutorials
Summary (4 messages):
Due to a lack of messages for the specific channel "tutorials", no summaries can be provided.
Guild: Nous Research AI
Nous Research AI Guild Summary
-
Deep-dive into dataset optimization and archiving was led by @tsunemoto discussing optimization methods with Parquet, resource efficiency, and scope of dataset size, resulting in a conversation on prospective solutions for storage space and network limitations, including the proposal of alternative storage solutions such as archive.org.
-
Active conversation on various AI performance-specific tasks, speech editing, audio classification, fine-tuning, and logical reasoning in AI. Links to Twitter posts served as the crux of these discussions with community reflections, debates, sarcastic remarks on benchmarking, and the release of new models. The quality of commercially available models, particularly the Phi 1.5 model, and AI brain theories were key topics as well.
-
Substantive debate on AI model merging, discussing the technical details such as how a merge results in a seemingly reduced total parameter count. Notable topics also included the discussion of open-source Large Language Models (LLMs), issues and solutions with using capybara 34b, data and model training initiatives, and announcements for model updates.
-
Welcoming environment fostered by the community with @SpazeCraft's greeting and @teknium's friendly response.
-
Constructive conversation in the Ask About LLMS channel revolved around fine-tuning models, merging weights after fine-tuning, structured JSON outputs with fine-tuned models, and load time issues with certain models. There were notable community solutions and suggestions, with @teknium, @yorth_night, @tsunemoto, and @crainmaker offering tips and advice.
-
Brief discussion on scaling and growth of operations with meme-esque references to Paul Graham's essay on not scaling early and a Hacker News post.
- Paul Graham's Essay
- Hacker News Post
Nous Research AI Channel Summaries
Channel: off-topic
Summary (6 messages🔥):
Discussion on Dataset Optimization and Archiving:
- Dataset Optimization Method: @tsunemoto discussed a method of optimizing a dataset using Parquet. He mentioned using blobs to store images and added batch processing for faster results. He also mentioned creating a script allowing exploration of the 'blob'd parquet' through a regex search, automatically converting blobs back to images when queried.
- Dataset Size and Scope: @tsunemoto mentioned having created a 100k row subset from a Parquet, equating to roughly 1% of the total dataset. He estimated creating the same dataset with Dall-e 3 in HD would cost $800,000 in API spend. @tsunemoto and @yorth_night agreed to split others Parquets to work on in order to avoid duplicate processing.
- Script Sharing and Usage: The script used in the process was shared among the chat, mainly @yorth_night and @youngphlo, to divide the processing of different sections of the dataset. The use of tqdm for display progress during processing was also mentioned.
- Storage Space and Network Limitations: @yorth_night encountered an issue where the script was crashing his RAM. @tsunemoto suggested the possible option of using a larger Colab instance. The potential risks of running the process on Discord's Content Delivery Network (CDN) was also mentioned.
- Alternative Storage Solutions: Due to upcoming changes to Discord's hosting policies, where images would no longer be hosted for more than a few weeks, @john0galt suggested archive.org as a potential solution for backing up the data.
Channel: interesting-links
Summary (6 messages🔥):
Summary of Channel Discussion:
- Discussion on specific AI task performance: @max_paperclips and @teknium discussed a Twitter post regarding AI performance on specific tasks with @max_paperclips opining that the task is one Transformers would excel at.
- Debate on speech editing and audio classification: @tsunemoto shared a Twitter post about speech editing and audio classification, with @teknium questioning its potential use cases.
- Discussion on logical reasoning in AI: @ldj provided input on logical reasoning benchmarking in AI, stating it's a stretch to classify proofwriting as general logical reasoning.
- Remarks and sarcasm on benchmarking: @yorth_night, @euclaise, @makya commented on a Twitter post about certain companies possibly gaming AI performance results.
- Discussion on fine-tuning models: @_automagic, @teknium, @gabriel_syme, and @tokenbender chatted about the upcoming releases of new models, their licensing issues, and the potential uses of fine-tuning.
- Indirect feedback on commercial AI models: @tokenbender also gave indirect feedback on the quality of the Phi 1.5 model, while expressing hopes that commercial counterparts will follow if the Phi 2 model performs well.
- Comments on AI brain theory: @yorth_night and @_automagic shared a Twitter post and labeled the post about AI brain theory as "big brain".
Channel: general
Summary (6 messages🔥):
Summary of the Nous Research AI Discord Chatbot Messages
- Discussion on Model Merging: The discussion begins with @zakkor asking how a 70B+70B model merge can result in a 120B model. @teknium responded that not all layers would be duplicated. @ldj further explained that the merge is likely the first 60B parameters of Llama-70B being connected to the last 60B parameters of Llama-70B, which @yorth_night had doubts about, given that different layers perform different functions. @alpindale then detailed the process, stating that slices from different layer ranges from each model were taken and stacked on top of each other. This is discussed in the readme as included in @alpindale's link. Additionally, @alpindale also mentioned plans to fine-tune the 120B model using 24x H100s.
- Open source Large Language Models (LLMs) discussion: @mihai4256 shared a link to the guidance-ai/guidance repo, which recently updated to accept GGUFs. @alexatallah also talked about price drops for Nous Hermes and OpenHermes APIs in OpenRouter.
- Problems and solutions with using capybara 34b: @yorth_night faced issues with inferring the capybara 34b model, getting errors related to the yi architecture and CUDA issues. @teknium provided solutions, indicating that capybara 34b could be loaded as a LlamaModelForCausal and suggested using ```load_in_4bit=True``` when initializing the model. @teknium also shared a Python code snippet for using Open Hermes 2.5 with HF Transformers with transformers inference code.
- Data and model trainings: @euclaise proposed creating a few datasets with data from archive.org, which includes contents like Usenet, Open textbooks, Children's books, and Manuals. @gabriel_syme shared @akhaliq's twitter post regarding a new model. Chatglm was also mentioned as a promising model by @euclaise.
Channel: welcomes
Summary (6 messages🔥):
Welcome Greetings and User Check-In:
- @SpazeCraft offered a welcoming greeting to the group, saying "Blessings".
- @teknium responded to the greeting with a waving emoji.
Channel: ask-about-llms
Summary (6 messages🔥):
Summary of Ask About LLMS Channel Discussion:
- On Merging Weights After Fine-Tuning: @lightninglemons asked how to merge the adapter weights after fine tuning and the base model to get the final fine-tuned model. @yorth_night recommended a solution using peft, sharing a link to a relevant GitHub repository: https://github.com/geronimi73/qlora-minimal/blob/main/merge_qlora.py.
- OpenHermes Claims About Fast Inference With VLLM: @ac1dbyte queried about OpenHermes' claim of its ability to allow fast inference with VLLM, and later thanked the community for their responses.
- Discussion on OpenHermes 2.5: @gabriel_syme raised a question regarding an issue with OpenHermes 2.5, and discussed a slower loading AWQ model issue in the context of vllm. @teknium pointed out that the model probably won't be intelligible past 4-8k ctx.
- Generating Structured JSON Outputs with Fine-Tuned Models: @ac1dbyte expressed the need for their fine-tuned OH model to generate structured json outputs. @teknium encouraged @ac1dbyte to first attempt the task with prompting before fine-tuning. @tsunemoto suggested that function calling could also be used to achieve this goal if the model supports it, additionally sharing a link to a json grammar builder: https://grammar.intrinsiclabs.ai/
- Fine-Tuning a Model on a Low Resourced Language: @4biddden inquired if fine-tuning a model on a low resourced language will let it to speak that language. @crainmaker suggested that the feasibility depends on various factors including the foundation model, the complexity of the language, the actual information encoded by the dataset, etc. @4biddden then shared a Reddit thread discussing a similar attempt: https://www.reddit.com/r/LocalLLaMA/comments/14chx6o/finetuning_a_llm_on_another_language/jphlo4i. @giftedgummybee advised looking into continued pretraining.
Channel: memes
Summary (6 messages🔥):
Discussion on Scaling and Growth:
- Referring to Paul Graham's essay, @teknium highlighted the idea of "Do things that won't scale".http://paulgraham.com/ds.html
- @_automagic shared a non-specific related post from Hacker News. https://news.ycombinator.com/item?id=36225723
Guild: Alignment Lab AI
Alignment Lab AI Guild Summary
-
Discussion on Creating Datasets & Federated Learning held across various topics like:
- Questions around the cost-effectiveness of creating datasets, whether ChatGPT would be a more financially viable option compared to GPT-4 generation.
- Interest and discussions around Federated Learning with Adapter Methods, particularly around its potential when parties cannot share data directly but can share model updates. Relevant links were shared including NVIDIA's take on implementing Federated Learning with Adapter Methods and the Petals Code Repository on GitHub providing hands-on opportunity:
-
An appeal for collaboration on the Techstars Startup Weekend San Francisco AI Event was made by a member. The project idea revolves around using Large Language Models (LLMs) to help e-commerce businesses automate operations. An Eventbrite link was shared to get more information on the event:
-
Discussions on the Finetuning and Merging of AI models brought up questions about the behavior of the AI models upon being finetuned and merged. A user also shared their experience of another project where hot swapping lora adapters was a highlighted feature.
-
A discussion on the Coding instructions and Condition Reminder Techniques in AI models was found, including:
- Clarifications on the CRLFT from the openchat codebase and paper.
- Suggestions for starting weightages for different models in reminding condition on every turn of multi-turn tasks.
- Sharing of a relevant link to a GitHub repository: MergeLM for further exploration on this topic.
Alignment Lab AI Channel Summaries
Channel: ai-and-ml-discussion
Summary (4 messages):
Discussion Topics about Creating Datasets & Federated Learning:
- Creating Datasets Cost-effectiveness: @igoforth asked for budget alternatives to a **gpt-4** generated dataset, questioning if using **chatgpt** would be more cost efficient than GPU time.
- Federated Learning with Adapter Methods: @erogol shared their interest in federated learning paired with recent adapter methods. They highlighted the potential such a method holds, particularly useful for parties unable to directly share data but can share model updates.
- Links:
- NVIDIA Developing Federated Learning with Adapter Methods - shared by erogol, provides an example of implementing this method.
- Petals Code Repository on GitHub - shared by erogol, offers a platform to explore federated learning using adapter methods.
Channel: looking-for-collabs
Summary (4 messages):
Techstars Startup Weekend San Francisco AI Event Collaboration:
- Messaging User 5811g is a Machine Learning Engineer (MLE) seeking collaborators for a team participating in an AI event. The proposed project idea involves using Large Language Models (LLMs) to assist e-commerce businesses in automating their operations.
- Links:
Channel: general-chat
Summary (4 messages):
Discussion on Finetuning and Merging of AI models:
- Finetuning Models: @teknium clarified to @tural.sadik, that the original AI model remains frozen until it is merged or attached. @tural.sadik further queried on the possible different responses expected if two separately finetuned models A and B are used.
- Hot Swapping Lora Adapters Feature: @teknium mentioned another project where hotswapping lora adapters was a main functionality.
Channel: oo
Summary (4 messages):
Discussion on Coding Instructions and Condition Reminder Techniques in AI Models:
- Understanding CRLFT: @turgutluk sought clarification about CRLFT from the openchat codebase and paper, to which @imonenext confirmed that it is indeed about prompt conditioning and weighted cross entropy.
- GPT Weightage Tips: @imonenext suggested starting with a weight of 0.1 for gpt 3.5 and a weight of 1 for gpt 4.
- Condition Implementation: @turgutluk expressed an approach of adding conditions, which involved appending it to the system prompt. @imonenext noted the importance of reminding the model of the condition on every turn of multi-turn tasks.
- Links:
- https://github.com/yule-BUAA/MergeLM, shared by @cryptossssun.
Guild: Skunkworks AI
Skunkworks AI Guild Summary
- Active discussion on AI Research, Resources, and Tools was noted, involving the search and sharing of resources, including AI papers. The following key discussions were observed:
- @le_mess asked for assistance locating a blog post on Multimodal LLMs, which @mrfoo found on ArXiv.
- @interstellarninja expressed the need for a reliable source for trending AI papers, and was referred to a resource by @tokenbender.
- @interstellarninja and @tokenbender exchanged viewpoints on the difficulty of filtering perpetual influx of AI papers and looked into options like Grok and Bard.
- @interstellarninja suggested the concept of a paper forwarding service with @tokenbender subsequently sharing a link for setting notifications for trending papers.
- @interstellarninja advocated for the establishment of a Discord channel and bot for paper summarization, with the aim of fostering a more organized paper discussion and review format.
- In the channel, there was a discussion of the Skunkworks AI Discord Chatbot with @digthatdata sharing sources specific to S-LoRA, including a GitHub repository and a corresponding Arxiv paper.
Skunkworks AI Channel Summaries
Channel: general
Summary (2 messages):
Discussions on AI Research, Resources, and Tools:
- Blogpost on Multimodal LLMs: @le_mess requested for help in finding a blogpost that discusses multimodal Language Learning Models with different modes like audio and images. The document, as responded by @mrfoo, was seemingly found in this ArXiv post.
- Trending AI Papers: @interstellarninja expressed dissatisfaction with the current state of Twitter reposts and sought resources for where to find trending AI papers. @tokenbender recommended this link as a source.
- Paper Filtering Problems: @interstellarninja and @tokenbender both echoed the struggle of sifting through the mass of AI papers. They discussed potential solutions with tools like Grok and Bard.
- AI Paper Forwarding Service: The idea of a paper forwarding service was floated by @interstellarninja, followed by @tokenbender sharing the link used for setting notifications for trending papers.
- Need for a Discord Channel and Bot for Paper Summarization: @interstellarninja suggested the creation of a new Discord channel solely for AI paper discussions, alongside a bot for summarizing and ranking these papers.
Channel: papers
Summary (2 messages):
Discussion on Skunkworks AI Discord Chatbot:
- @digthatdata shared a link to the GitHub repository for S-LoRA: https://github.com/S-LoRA/S-LoRA
- @digthatdata also provided a link to a relevant academic paper on Arxiv: https://arxiv.org/abs/2311.03285
Guild: MLOps @Chipro
MLOps @Chipro Guild Summary
- Announcement of the Building Active Learning Pipelines Webinar by @stephenoladele_mldev. The webinar, focusing on the enhancement of computer vision models and the development of active learning pipelines, promises to deliver a hands-on demo, insights into resolving data quality issues, and best practices for constructing data pipelines. The webinar is scheduled for November 15th and can be registered for at https://encord.com/lp/from-data-to-diamonds-active-learning/.
- Plan shared by @5811g about the use of Language Models to aid e-commerce businesses in automating their operations.
- Announcement of a talk on Crypto Fintech and Real-Time Data Infrastructure by @jovana0450, featuring speakers from Goldsky, Alchemy, Superchain Network, and Risingwave Labs. The seminar aims to shed light on real-time data handling within the cryptocurrency and fintech domains. The event can be attended by registering at https://www.meetup.com/streaming-stories/events/297191788/.
MLOps @Chipro Channel Summaries
Channel: events
Summary (1 messages):
Events: Announcements and Discussions:
- Building Active Learning Pipelines Webinar: @stephenoladele_mldev announces a new webinar focused on implementing active learning pipelines and improving computer vision models. The webinar will include a hands-on demo, insights on fixing data quality issues, best practices for building data pipelines, and a Q&A session. The webinar will be held on November 15th, and you can register at https://encord.com/lp/from-data-to-diamonds-active-learning/.
- Implementation of Language Models: @5811g discussed how they are planning to leverage Language Models to help e-commerce businesses automate their operations.
- Crypto Fintech and Real-Time Data Infrastructure: @jovana0450 announces a discussion with speakers from Goldsky, Alchemy, Superchain Network, and Risingwave Labs. The speakers would provide insights into real-time data processing in the context of cryptocurrency and fintech applications. Register at https://www.meetup.com/streaming-stories/events/297191788/.
This guild has no new messages. If this guild has been quiet for too long, let us know and we will remove it.
This guild has no new messages. If this guild has been quiet for too long, let us know and we will remove it.
This guild has no new messages. If this guild has been quiet for too long, let us know and we will remove it.
Guild: YAIG (a16z Infra)
YAIG (a16z Infra) Guild Summary
- Exchange on ML Infrastructure and Knowledge Graphs, focusing on tools and projects like Mistral, Stable Diffusion, and VectorFlow:
- @stevekamman discussed the usage of Mistral and Stable Diffusion on Cloudflare Workers AI.
- @.danme is working on an open-source infrastructure for semantic search, handling large volumes of data. He invited those with experience in large-scale knowledge graphs to share their insights and challenges.
YAIG (a16z Infra) Channel Summaries
Channel: ai-ml
Summary (1 messages):
Discussion on ML Infrastructure and Knowledge Graphs:
- Mistral and Stable Diffusion on Cloudflare Workers AI:: @stevekamman shared about the implementation of **Mistral** and **Stable Diffusion** on **Cloudflare Workers AI**.
- Request for Insights on Knowledge Graphs:: @.danme mentioned the development of an open-source infrastructure to ingest large data volumes for semantic search. He invited individuals working with knowledge graphs at scale to provide their insights into its challenges.