Didi Azaria’s Post

We all know the new era of AI is upon us thanks to large language models like GPT-4. But as Andrej Karpathy cautions, demos alone don't make products. So how do we thoughtfully build LLMs into real systems? An insightful new guide has been written sharing some key patterns. Things like fine-tuning, caching, defensive UX - incredibly useful and applicable items.This is a masterclass in taking AI from hype to deployed impact. It distills troves of research into an actionable playbook. LLMs are a breakthrough technology, but realizing their potential requires diligence and know-how. This guide provides a great roadmap. Bookmark and save for when you can digest this 65min read. https://buff.ly/3Qnj1gv #ai #products #innovation

🤔 What is Chunking and why is it necessary for processing large documents with LLMs? What is Chunking? ➡ Chunking is the process of breaking a large document into smaller, manageable "chunks." This is crucial in the Retrieval-Augmented Generation (RAG) ecosystem, especially due to the context window size limitations of Large Language Models (LLMs). ➡ When documents are sent to an LLM for extraction or reasoning tasks, they need to fit within the model's context window. 🚨 Why Chunking is Necessary: ➡ LLMs have a limited context size or window, meaning they can only process a certain number of tokens (words and punctuation marks) at one time. ➡ For example, GPT-3.5 Turbo has a context size of 4,096 tokens. This limitation means very large documents cannot be sent in their entirety to the LLM. ➡ Instead, only the contextually relevant portions should be sent. To achieve this, the document needs to be chunked, allowing relevant chunks to be identified and processed efficiently. ➡ In summary, chunking ensures that LLMs can handle large documents by breaking them into smaller, relevant pieces that fit within their processing capabilities. If you have two documents: - Document A: 2,000 words (1,500 tokens) - Document B: 15,000 words (11,250 tokens) ➡ Document A fits easily within GPT-3.5 Turbo’s 4,096 token limit, so the entire document can be processed at once. However, Document B exceeds this limit, requiring chunking to break it into smaller, manageable parts. While Document A doesn't need chunking, Document B does, to ensure it fits within the model's context size. Avoid chunking for the best results if your documents are smaller than the LLM's context size (like Document A). However, extensive extractions can increase costs. ✅ The Solution: Unstract’s SaaS and Enterprise versions offer features like Summarized Extraction and Single-Pass Extraction. These options let you benefit from no chunking while keeping costs low and maintaining ease of use. To read in detail: https://lnkd.in/g6EB8VPE Try now: https://unstract.com/

I am starting to get asked about how we make the responses to #genai a little more focused & specific to the l&d world. Below is one great way to do this, and is a one of the best overview of the differences between ‘Fine Tuning’ and ‘RAG’ that I have seen for non-AI people. What does this mean for learning, we can’t just take the standard output of the #ai solutions that we are playing with at the moment, we need to consider how make them specific for our world - and it’s not an easy task! #genai #learninganddevelopment #aiedtech #edtech

Great post! kdb.ai is the one-of-a-kind vector database you need to store the embeddings for semantic search, and it allows for both unstructured and structured data for increased accuracy. #genai https://lnkd.in/gWDQbKem

Jay Leib and Isha Chopra brought the big guns with an interactive session to build a real time bespoke AI model - in 10 min or less! Did you know that you (or anyone) can be a citizen data scientist. As in build powerful AI?? What is an ai model? A rule that produces classification scores for documents. Language, metadata and more can power these algorithms. Also did you know Reveal-Brainspace has a lie detector #Ai model? “Roundabout ai model” But that is not it. There is a whole model library similar to a netflix qeueu for AI models… that you can pack and stack to make an uber model! Crazy cool!! So isha described how to get started to build your own ai model adventure… - past review decisions - search terms - taxonomy, wikis, coding manuals One hack is also to take legacy cases and their coding decisions—> output is a bespoke ai model just for you org—> and you OWN the IP- You can also do this concurrently with an active case! Why would you want to do this? - Industry specific linguistics - Linguistic context - Continual improvement - Jump starting a case How is this different from just keyword? - can integrate domain expertise (inside baseball) - can build from exemplar - Can incorporate prior workproduct and human decisions/ coding - Far easier than messing with Boolean logic alone! 🤖🤖So HOW DO YOU DO IT???🤖🤖 1- seed with coded docs, prior models, existing portable models 2- code- train batched algo picks 3- train- Machine alearning build new model (rinse repeat) 4- evaluate- do you like it or do you want to keep training? 5- save and go!! Pretty intuitive. One great example? Our amazing collaboration with DLA Piper #aiscension with Ilan Sherr Ie - 150m saved - 80k data set—> smoking gun in 10 mins - 1m docs took 1/19 the time to find 30 issues - 477,142—> done in 95 hours!! A great example of amplifying human intelligence-#HCAI ALL DAY LONG! So what does it look like in action?? 🤖—>start with the cutting edge awesomeness of … excel or even a text doc 🤖—> sit with domain experts and ask about the key issues, what you are thinking about with this type of matter 🤖—>if say an employment case start with 2 columns “term” and “weight” 🤖—> starting with “harass” add harass to term, for weight decide how important the term is with a number 0-1 Harass 1 Accomodation (can add -1 rank phrases to avoid “hotel accompdation” ) Performance review 1 Recruit 1 Etc. —>with Reveal can do both words and phrases (groups of words) and stemming is factored in as well. 🤖—> add in phrases that do not relate, and rank them 0- to -1. (Baseball, mortgage, hotel etc). 🤖—> export as CSV 🤖—> import model to Reveal 11 🤖—> turn on AI service and tag and choices 🤖—> link model to the tag in the case—go to classifiers, configure and add to tag 🤖—> in 5 minites we had completed the starting point for a powerful ai model. No excise to not try it out!! #customai #allinonai #portablemodels Reveal-Brainspace #aimodels

#openai have just announced the introduction of a new generation of #embeddingmodels and updates to #gpt4 Turbo and GPT-3.5 Turbo. ⬇️⬇️⬇️ 👉 Development of new #ai models Text embedding models are designed to convert text data into numerical representations, known as embeddings, which capture the semantic meaning of the text. A model being smaller and more cost-effective might mean it's optimized for specific tasks or scenarios.  ✅ text-embedding-3-small  The model requires 5 times less computational resources than its predecessors, resulting in lower costs for the same or more tasks. The model has been enhanced in terms of its ability to understand and process text and handle nuanced language. ✅ text-embedding-3-large This is OpenAi's best performing model which can create embeddings with up to 3072 dimensions. This high dimensionality allows the model to capture a wide array of linguistic and semantic nuances in the text. Higher-dimensional embeddings generally can encapsulate more complex patterns and subtleties in the data, leading to potentially more accurate and insightful representations. 👉 Updates to GPT-4 Turbo GPT4 Turbo can now interpret programming problems more accurately, provide more precise and relevant code solutions, and handle a wider variety of coding tasks effectively. Enhanced code generation can be particularly beneficial for developers and programmers, aiding in writing, debugging, and understanding code. ❗️ Also, an UTF-8 encoding bug was fixed, meaning that GPT-4 Turbo can now correctly handle and process text in various languages and scripts without errors or misinterpretations. 👉 GPT-3.5 Turbo The cost for processing the input text (the text that users input into the system for the model to respond to) has been halved. This could make it more cost-effective for users to input longer queries or documents, as the cost associated with the amount of text processed has been significantly reduced. 📣These updates represent a significant step forward in the capabilities of language models, particularly in areas requiring technical precision and multilingual support. If you want to find out more about #ai #marketing and #ecommerce, don't hesitate to contact me: 📧 flo@cleverplusplus.com

LLM's difference between Fine Tunings LLM and Retrieval-Augmented Generation (RAG). Since I am using both for my GenAI projects, clarifying the differences is good. Methodology: Fine Tuning: Involves adapting a pre-trained LLM to specific tasks or domains by training it further on a smaller, task-specific dataset. RAG: Integrates the retrieval of relevant documents from a large corpus into the generation process of an LLM, allowing it to pull in external knowledge for its responses. Knowledge Source: Fine Tuning: Relies on the knowledge embedded during pre-training and the nuances learned during fine-tuning. RAG: Uses both the embedded knowledge from pre-training and real-time retrieved information from external documents. Training Data: Fine Tuning: Requires task-specific data for the fine-tuning phase. RAG: Needs a large corpus of documents for retrieval and a set of question-answer pairs for training the generation process. Flexibility: Fine Tuning: Can be tailored to a wide range of tasks by fine-tuning on relevant data. RAG: Specifically designed for tasks that benefit from real-time external knowledge retrieval, like open-domain question answering. Complexity: Fine Tuning: Relatively straightforward, involving further training of a pre-trained model. RAG: More intricate due to the integration of retrieval and generation processes, often requiring joint training. Use Cases: Fine Tuning: Suitable for task-specific applications like sentiment analysis, text classification, or domain-specific chatbots, Summarization, Translation Code generation,Natural language inference . RAG: Ideal for tasks that require up-to-date or extensive external information, such as open-domain Q&A or tasks where the model's internal knowledge might be outdated. Creative writing, Essay generation, Code completion, Product reviews, Customer service chatbots. Got this nice image from a blog post. Flaticons was used for icon. #ai #llm #finetuning

3 (way too easy) steps to build a custom GPT. OpenAI just released GPT Builder, meaning you can now create custom, personalized GPTs with natural language (no code). 🤯 The key for these is leading with the use case - you want to be specific in the tasks it should perform, it's goals, and the output you’re looking for so that the results are consistent and repeatable. There are some pre-made GPTs that are useful off the bat - Data Analysis, The Negotiator, Creative Writing Coach - to name a few. But building your own for a specific use case to be shared within your org can be incredibly powerful. Here’s how: 1️⃣ Chat to start: Clearly outline your GPT's role, like “a B2B competitive intelligence expert specialized in delivering comprehensive reports tailored to senior decision-makers”. Be super detailed in your instructions and what you want the output to be - use examples for clarity. You can also ask it for feedback on what might be missing to enhance its effectiveness. 2️⃣ Configure the GPT - Fine-tune behavior and output in the 'Configure' tab. You can view how your chat converted into instructions, and make any modifications or clarifications. Adjust conversation starters to push users in the right direction, and upload relevant files to expand the GPT’s knowledge base (super impactful for internal use - think product info, pricing info, etc). 3️⃣ Test, test, test - Use the ‘Preview’ window to see how the GPT responds in real time. Continuously revise, save, and test after each modification - it may take a bit to work out the kinks. Share it with your team, and tweak it over time. There’s no ‘right’ way to set these up, so lean into feedback and iterations to improve the results. Have you dabbled in creating a custom GPT yet? Any cool use cases you’ve seen or are thinking about trying?

DSPy has dramatically changed how I develop applications with AI, particularly in super-charging older models like GPT-3.5. DSPy’s approach, which combines multi-step prompting with a detailed compilation process, not only fine-tunes AI’s performance but also offers a consistent and reliable output. To bring this concept to life, I embarked on creating a sophisticated test application—a story generator that leverages the full spectrum of DSPy’s offerings. This tool isn’t just any content generator; it’s a comprehensive platform for crafting narratives, equipped with an extensive array of customization options. From genre, theme, setting, and character details to writing style, tone, and pacing—the system allows for an unparalleled level of control over the storytelling process. It’s hosted on Google Colab for easy access. This tool shows DSPy’s potential in creating complex applications that follow a logical, structured approach to crafting narratives, providing explicit fine-tuning tailored to specific types of requests. Beyond just enhancing storytelling, DSPy opens up new areas of possibilities for rejuvenating older, less sophisticated models or smaller less capable models. This isn't just about supercharging language models; it's also a cost-effective strategy for leveraging older less intelligent Ai, boosting their performance to match or even exceed that of newer models. This efficiency makes DSPy a valuable tool for developers and creators looking to maximize their resources without sacrificing quality. Ite actually quite amazing at this. In the coming days, I plan to share additional examples that highlight how to further harness these older models using unique "mixture of experts" approach I've developed that I call “Recursive Unified Validators (rUv). While this work is still in progress, the potential it holds is immense, promising to unlock new levels of capability and efficiency in AI-driven projects. Keep an eye on my gist for sneak peeks. DSPy’s innovative framework demonstrates its capability to not just supercharge LLMs but also to significantly reduce operational costs by making older models vastly more capable at fraction of the cost. This paves the way for a broad spectrum of applications, from content creation to more complex AI-driven solutions, redefining what's possible with existing technologies. See comments for links to Google Colab and my GitHub.

🚀Excited to share my latest project: a custom chatbot that uses AI to parse and answer questions directly from PDFs! This project leverages the power of: 💠 Google Gemini's API: Offering high-performance NLP capabilities, including advanced text retrieval, efficient information extraction, and flexible text-to-text manipulation 💠Langchain library: To seamlessly connect different AI components 💠RAG technique: Enabling the chatbot to retrieve answers from relevant PDFs 💠SQL Database: To store frequently asked questions in the database to minimize API calls for cost reduction at production level. I've also written a detailed article on Medium explaining the project's journey and technical details: https://lnkd.in/gpSyJUd2 Do give it a read, I’d really appreciate it! Massive thanks to these amazing YouTube creators whose content was invaluable in my learning process: Krish Naik Alejandro AO Venelin V. Karndeep Singh I'm now gearing up to create the frontend for this chatbot and would love your suggestions! Feel free to drop your ideas and recommendations in the comments. Your insights will play a crucial role in shaping the next phase of this project. P.S. Don't forget to check out the project's GitHub repository: https://lnkd.in/gb2TGgbx #AI #chatbot #LLMs #PDFparsing #opensource #collaboration #gemini #googleai #langchain #google #artificialintelligence