Marcos F.

Using generative AI, NVIDIA operations built an AI Planner agent, developed on NVIDIA Inference Microservices (NIM). The agent leverages LLM, NeMo Retriever and CuOpt NIM to reduce re-planning time from hours to just seconds. https://nvda.ws/4bIfS2T

2

If you are benchmarking Computer Vision models and you use random inputs because you don’t have input data yet, be careful! If your model includes preprocessing, you may be getting higher times. Why? Usually the most expensive operation in preprocessing is the resize, and there’s a catch: It takes more time to resize a completely random image (noise) than a natural image! For example, here it is an example with the Pillow Python library: https://lnkd.in/d3RHYxEe In the notebook, you see that it takes almost twice the time for the random image when compared to a natural image or a monochromatic one. The solution is always use images that follow the expected distribution in production. For example in Nvidia Model Analyzer there is the “input-data” option (https://lnkd.in/dum_eRXF) where you can set the directory of input images or a JSON with the base64 string of the images. The exception is if you are analyzing just the model and you are happy with providing random images at the exact size the model is expecting (i.e. 224x224), so there is no resize involved.

8

In-Context Learning at Extreme Scales: My Thoughts As large language models (LLMs) continue to expand their context capacities, the domain of in-context learning (ICL) at extreme scales has piqued my interest. A recent paper by researchers at Carnegie Mellon University and Tel Aviv University provides a detailed exploration into this area, unveiling both impressive capabilities and notable challenges. 🔍 Key Insights: Performance Improvement: The study demonstrates that ICL can improve significantly by integrating hundreds to thousands of examples into the context window—far beyond traditional few-shot settings. This is particularly impactful in complex tasks like 77-way banking intent classification, where long-context ICL matches or surpasses traditional finetuning. Input Sensitivity: Long-context ICL shows reduced sensitivity to input shuffling compared to shorter contexts. However, organizing examples by label dramatically reduces performance, emphasizing the need for diverse context configurations. Source of Gains: Performance gains primarily arise from accessing more relevant examples, rather than aggregating knowledge across many examples during encoding. 📉 Challenges & Drawbacks: Sensitivity to Example Order: Despite reduced sensitivity, the performance still varies with input order, indicating potential instability in practical scenarios. Performance Saturation: The models often do not fully leverage their extended context capacities, showing performance saturation before reaching the maximum context length. Computational Cost: The increased computational cost associated with using large context windows could limit practical deployment, particularly in resource-constrained environments. Efficiency Issues: There is a noted inefficiency in how models utilize the encoded information within large datasets, suggesting that current architectures may need refinement to better use available data. 🚀 Future Directions: The paper suggests several exciting avenues for future research, including optimizing the trade-offs between computational costs and performance, exploring model adaptation strategies, and refining ICL approaches for specific applications or datasets. As we push the boundaries of what LLMs can achieve, effectively leveraging their full context could unlock new capabilities. This area is ripe for further exploration and could lead to major advancements in how we deploy AI. For those at the cutting edge of language model development, I highly recommend this paper for its thorough analysis and forward-looking perspective. Check out the full work at https://lnkd.in/d7AmGh4y. What are your thoughts on the future of in-context learning at such scales and the challenges discussed? #AI #MachineLearning #DeepLearning #LanguageModels #InContextLearning

17

Using generative AI, NVIDIA operations built an AI Planner agent, developed on NVIDIA Inference Microservices (NIM). The agent leverages LLM, NeMo Retriever and CuOpt NIM to reduce re-planning time from hours to just seconds. https://nvda.ws/4bIfS2T

20

1 Comment

🤔 Vanilla-RAG struggles with structured knowledge sources like knowledge graphs. GNN-RAG is a very neat idea to fix this! ⛳ Vanilla-RAG struggles with structured inputs like KGs because it relies heavily on LLMs for retrieval, which are not adept at handling the complex graph information inherent in KGs. This leads to suboptimal performance, especially on multi-hop and multi-entity questions that require traversing multiple relationships in the graph. ⛳ GNN-RAG integrates the strengths of both LLMs and and Graph Neural Networks (GNNs) to solve this issue: 💡 GNN: Excels at processing and reasoning over graph structures. It reasons over a dense KG subgraph to retrieve answer candidates for a given question. 💡LLM: Leverages its natural language processing abilities to further reason over the information provided by the GNN. 👉 Here's the workflow: 🔺 A GNN processes the KG to identify and retrieve candidate answers. 🔺The shortest paths connecting question entities to answer candidates in the KG are extracted to represent reasoning paths. 🔺These paths are verbalized and provided as input to the LLM for final reasoning and answer generation. GNN-RAG achieves state-of-the-art results on two widely used KGQA benchmarks, WebQSP and ComplexWebQuestions (CWQ) and outperforms existing methods, including GPT-4, particularly on multi-hop and multi-entity questions. Link to the paper: https://lnkd.in/euC7N85K

244

4 Comments

A very short backlog on AGI. 1. Implement hypernetwork in llama3 2. Implement queue Augmented llama 3. with push, pop, nop. how : use bidirectional encoder like Deberta for getting embedding of the queue, and concatenate to input. and output layer for action of push, pop, nop 3. neural projection operator in transformer block, for learning physics constraints on embedding. 4. Pretrained using PPO. Sebastian Raschka, PhD how much effort is this?

18

2 Comments

Cerebras and Neural Magic have reached a significant milestone in the realm of large language models (LLMs). Through innovative techniques and purpose-built hardware, we achieved up to 70% parameter reduction in LLMs without sacrificing accuracy. This breakthrough opens doors for more efficient training and deployment, democratizing access to LLMs across various sectors. #AI #MachineLearning #Innovation

10

1 Comment

Today, we’re launching Substrate. We’re also announcing our $8M Series Seed led by Lightspeed. We believe the most robust, reliable, and productive uses of AI are when many models are used in coordination with each other, with regular programming mixed in. Most people building with AI already know this. But the available tools are suboptimal, unless you work for Google. For the rest of us, we’re left creating an unwieldy mess of chained API calls to multiple providers, with many roundtrips in between. We took a hard look at this state of affairs, and recognized how much it is stifling progress. Building large multi-step AI workloads requires sophisticated high-performance tooling and infrastructure. Nobody wants to deal with more tooling and infrastructure… but everyone would benefit from simple, intuitive interfaces that abstract away a powerful system underneath. No tooling, no infrastructure – just elegant abstractions.

25

1 Comment

The best part of investing at the earliest stages of company building is every once in a while you meet founders who can see the future. Ben Guo & Rob Cheung are two of those founders and after a year of building in stealth they're launching ꩜Substrate, the first inference API optimized for multi-step AI workloads. 👉 https://substrate.run With ꩜Substrate, you connect nodes from a curated library that includes optimized ML models, built-in file and vector storage, a code interpreter, and logical control flow. By simply connecting nodes, you describe a graph workflow, which Substrate then analyzes and runs as fast as possible. Entire graphs of many nodes will often run on a single machine, with microsecond communication between tasks. We are thrilled to participate in their 💵 $8M Seed led by Lightspeed (Guru Chahal/Nnamdi Iregbulem) alongside our friends at South Park Commons (Arian Agrawal), Craft Ventures, Red Swan Ventures (Eric Rubin/Sandy Cass), joined by amazing 🧠 founders and 👨‍🏭 operators like Guillermo Rauch, Immad Akhund, William Gaybrick, Chris Best, Gokul Rajaram, Shreyas Doshi, Jana Messerschmidt, Michael Manapat, and Matthew Hartman.

23

1 Comment

There are concerns that filtering out LLM-generated text during data collection for model training is now impossible. Human-created text seems to be becoming the data equivalent of low-background steel, also known as pre-atomic steel, which is steel produced prior to the detonation of the first nuclear bombs in the 1940s and 1950s. To me, this seems like a false premise. It shouldn't matter if the content was created by a human, an animal, or AI. What matters is the quality of the content used for training the model. Current LLMs are already so advanced that a kind of bootstrapping occurs. For example, Meta used synthetic data to train their new Llama 3 models: “We found that previous generations of Llama are good at identifying high-quality data, so we used Llama 2 to help build the text-quality classifiers that are powering Llama 3. We also leveraged synthetic data to train in areas such as coding, reasoning, and long context. For example, we used synthetic data to create longer documents to train on.” What do you think?

3

3 Comments

🚀 Want to employ the power of #PyTorch for your scientific models without a steep learning curve? 👀 Check out our latest blog, where Travis Oliphant discusses how PyTorch's integration with the Array API makes it easy to use with familiar libraries like NumPy and scikit-learn.🔥 https://lnkd.in/dDRkEY3v #Innovation

10

🚀 Weekend Read: Delving into Transformer Reasoning Capabilities with Graph Algorithms 🚀 Looking for a thought-provoking read this weekend? I’m excited to dive into "Understanding Transformer Reasoning Capabilities via Graph Algorithms" by the talented team at Google Research and Columbia University. 🔍 What’s Inside: Transformer Scaling Regimes: Discover how transformers handle different classes of algorithmic problems, especially graph reasoning tasks. Theoretical Foundations: Gain insights into the theoretical underpinnings of transformers’ representational capabilities based on their depth, width, and additional tokens. Empirical Validation: See how transformers perform on graph reasoning tasks using the GraphQA benchmark compared to specialized graph neural networks (GNNs). 📊 Why It’s a Good Read: Innovative Framework: The paper introduces a novel representational hierarchy categorizing graph reasoning tasks by complexity and transformer capabilities. Performance Insights: Learn about the superior performance of transformers in tasks requiring long-range dependency analysis. Practical Impact: Explore the potential real-world applications of these insights in AI, from language modeling to computer vision. Check out the paper in the first comment. Let’s make this weekend insightful reading an excellent research! #AI #MachineLearning #Transformers #GraphAlgorithms #WeekendRead #ResearchDiscussion

11

1 Comment

Exploring the Pros and Cons of Multi-token Prediction in Language Models : A recent study, "Better & Faster Large Language Models via Multi-token Prediction," by researchers from FAIR offers a fascinating look at how predicting multiple tokens simultaneously can significantly enhance the efficiency of large language models like GPT and Llama. The researchers of the paper claim that the models developed via the technique discussed in the paper solves 12 % more problems on HumanEval and 17 % more on MBPP than comparable next-token models Key Highlights: Enhanced Efficiency: The multi-token prediction method boosts sample efficiency without additional training time or memory overhead, presenting a compelling case for its adoption in larger models. Improved Performance: This technique has shown to dramatically improve performance, particularly in complex coding tasks and natural language processing, consistently outperforming traditional models. Pros of Multi-Token Prediction: Scalability and Versatility: The method scales impressively with larger models and demonstrates benefits across various data types, including both code and natural language. Inference Speed: Models trained with this method can be up to 3× faster at inference, making them not only more intelligent but also quicker. Potential Challenges: Complexity in Training Dynamics: The method involves multiple output heads which could complicate model training dynamics and require careful tuning. Optimal for Large-Scale Models: While the benefits are clear for large models, smaller models may not see as significant improvements, possibly limiting its applicability in resource-constrained projects. Generalization across Tasks: The effectiveness across all types of tasks is not fully explored, which may limit its applicability in certain scenarios. Future Directions: The paper discusses promising future research paths, including automating the selection of the number of tokens to predict and optimizing vocabulary sizes for multi-token prediction. This approach represents an exciting advancement in AI training techniques. However, it also poses new challenges that necessitate further exploration to fully understand and mitigate potential drawbacks. Paper link : https://lnkd.in/d69Je8wU #AI #MachineLearning #LanguageModels #Innovation #Research

14

Explore fascinating free tutorials on machine learning! In "Neural Networks: Zero to Hero" Andrej Karpathy explains the fundamental themes of ML with the real usage of code. Details 👇 You start learning with a simple bigram character-level language model and gradually build it into a more complex Transformer model like GPT. Key topics include: - torch.Tensor usage for efficient neural network evaluation - language modeling framework - MLP implementation - Batch Normalization for robust training - prompting strategies and fine-tuning - tokenization, development of a Tokenizer for GPT models "Neural Networks: Zero to Hero" tutorials by Andrej Karpath on YouTube: https://lnkd.in/eAquTgX3

1