Andrew Kwok

I am proud to be working at Martian, where our AI research team is pushing the boundaries of scaling Mechanistic Interpretability, arguably one of the most important areas of AI research. In recent weeks, we have seen a surge of progress in AI interpretability, driven by both Anthropic and OpenAI. While their recent techniques (linked in our blog post) involve deep and painstaking human inspection of individual neurons, our approach aims to automate this understanding at scale. The Martian research team, led by Shriyash U. and Chaithanya B., outlines a groundbreaking insight in this blog post: to truly automate the understanding of neural networks, we must evolve not only our techniques but also our theoretical foundations. They propose category theory as the key to this evolution. As our founder Etan Ginsberg would say, for more on the technical details, dive into the blog post. For those who are “normies” like me, you might wonder why this is so important! The goal of Mechanistic Interpretability is to make AI models understandable, so we can "read" them like a book. Currently, AI models are largely black boxes—even their developers are often surprised by what they can do. Understanding how to prompt them, what they excel at, and their inherent biases is a process of trial and error. Imagine the Possibilities: -Having detailed insights into an AI model's knowledge and capabilities. -Understanding its biases, strengths, and weaknesses. Knowing how to effectively prompt a model to get the desired answers. This level of understanding would revolutionize our interaction with AI, making it more transparent and predictable. At Martian, this research powers our LLM Router, which predicts the best large language model (LLM) for a given prompt based on a customer's cost, latency, and output quality preferences, a highly practical real-world application of Mechanistic Interpretability.  Follow Martian to be tuned into more exciting developments from our team! P.S. - As I write this, our founders just came out of a meeting with the research team of a major global tech company. They were delving into our technology, and the customer said, “You guys are better than OpenAI.” While I’m not sure of the exact context, it’s still incredibly flattering! #AI #MachineLearning #ArtificialIntelligence #TechInnovation #MechanisticInterpretability #Research #MartianAI #AITransparency

7

This 6+ hour course and certification is free until the end of July. I strongly suggest you consider giving it a try! 😎 #softwareengineering #datascience #ai #technologynews #technews #itcareers #techcareers #programming

5

1 commentaire

As Yann Lecun says, intelligence is not correlated with a desire to dominate. It follows that the more intelligent AI becomes, the less it is likely to want to dominate us. Thus, we have nothing to fear. However...that doesn't mean that a desire to dominate could not drive some future AI. It just means that any such desire would not emerge from its intelligence. Indeed, a 'dominating AI' wouldn't have to be great at math and logic, just at manipulating others. Perhaps such an AI could also, ironically, easily manipulate the most intelligent AI's - and humans - precisely because the latter don't care about domination. They'll be too busy solving quantum consciousness or some such intellectual challenge while the dumber, Machiavellian AI's take over the world. I'll leave it to you to ponder what kind of humans might be all too happy to cooperate with those kinds of entities. To consider whether intelligence has been an effective antidote to the manipulation and domination of the masses that led (and continues to lead) to mass suffering and death. How strong is the assumption that intelligence will be the only objective for AI development, forever? Which stakeholders control the objectives and what are their objectives? Then again, perhaps you're too smart to bother with such matters 😉 https://lnkd.in/gUbvZXmQ

2

Another great post from Arnault 🦜 Gombert for MantisNLP in collaboration with Argilla. The latest in our impressive - 11 blog long - series on Reinforcement Learning with Human Feedback (RLHF) the key to incredible performance from #LargeLanguageModels such as ChatGPT, Gemini, and Claude. 👀 Check it out!

5

Preach. AI is great and I’m using genAI to expedite processes at work (and testing it as a makeshift therapist). As much as I’d love for it to actually replace my job it’s still not able to compute creatively, laterally, intersectionally, nor can it challenge perception, like actual humans. Difference is, insofar as generating fresh insight, we bottle our lived experience not only in our consciousness, but in the subconscious and in the body - that may manifest as intuition or viewpoint leaps. Also so many things determine a point of view - emotional, biological, cultural, social conditioning, etc. perhaps the next step would be corporeal embodiment to collect and layer over more nuanced data.

4

In this video lecture , I talk about constrained optimization. The first method to solve a constrained problem is by converting it into an unconstrained one i.e by the method of substitution. Ref book: Sydsaeter and Hammond Chapter 18 Useful for : Students entering Semester 3 , Economics Hons ( Course Advanced Mathematical Methods of Economics) ArthaPoint - One Stop Platform for Economics #advancedmme #ecohons https://lnkd.in/gYtSnGDQ

3

"A team of computer scientists and AI researchers from FAIR at Meta, INRIA, Université Paris Saclay and Google, has developed a possible means for automating data curation for self-supervised pre-training of AI datasets. The group has written a paper describing their development process, the technique they developed and how well it has worked thus far during testing. It is posted on the arXiv preprint server. As developers and users alike have been learning over the past year, the quality of the data that is used to train AI systems is tied very closely to the accuracy of results. Currently, the best results are obtained with systems that use manually curated data and the worst are obtained from systems that are uncurated." #datacuration #aidatasets

2

AGREE : a learning-based framework for improving LLMs by grounding their responses in retrieved passages and by providing citations to make answers more reliable LLMs has major issue towards their widespread deployment in the real world is that they can generated hallucinated answers that are not factual. In this paper, authors propose a new framework, AGREE, Adaptation for GRounding EnhancEment, that provides grounding of answers in retrieved passages by providing citations, making them more reliable and increasing user trust. 𝗞𝗲𝘆 𝗰𝗼𝗻𝘁𝗿𝗶𝗯𝘂𝘁𝗶𝗼𝗻𝘀: - Introduces a learning-based approach that adapts a base LLM to include accurate citations in its response, leveraging automatically created data - A test-time adaptation (TTA) method that iteratively improves responses of LLMs based on the citation information. - Extensive experiments on two LLMs over five datasets demonstrating the effectiveness of the proposed AGREE framework for improving grounding and citation generation. 𝗔𝗚𝗥𝗘𝗘 𝗙𝗿𝗮𝗺𝗲𝘄𝗼𝗿𝗸 i) Tuning LLMs - authors tune the LLM to self-ground the claims in their responses by providing citations to retrieved documents - authors sampled responses from the base model, use an NLI model to add citations to the sampled responses, and tune the base model with the best-grounded response. ii) Test-time adaptation(TTA) -Proposed framework is a form of retrieval augmented generation framework ,using an adapted LLM that is able to answer a query based on a set of given passages retrieved from the corpus, and, more importantly, self-ground its response to add citations to the passages as well as to find supported statements needing further investigation Detailed procedure is as follows: - Given a query Q and corpus D, we first retrieve based on the query to obtain an initial set of working passages - Next, iteratively at each iteration, the LLM generates a response to the query based on the working passages, adds citations to its response, and finds out any unsupported statements that do not have citations - Then, we add the cited passages to the list of relevant passages - Lastly, at each iteration, we update the working passages – if there are unsupported statements, we include additional information retrieved based on the unsupported statements. 𝗘𝘅𝗽𝗲𝗿𝗶𝗺𝗲𝗻𝘁𝗮𝗹 𝗥𝗲𝘀𝘂𝗹𝘁𝘀 𝗔𝗻𝗮𝗹𝘆𝘀𝗶𝘀 - across 5 datasets(NQ, StrategyQA,ASQA etc.), AGREE can generate responses that are better grounded in text corpus and provide accurate citations to its response, substantially outperforming all baselines - improvements obtained from training on in-domain datasets can effectively generalize to OOD datasets that contain different question types - TTA improves llama-2 answer correctness by 3.1 and 3.7 on NQ and ASQA, respectively - AGREE framework fine-tunes the base LLM to enable self-grounding without needing for additional in-context examples or NLI models 𝗣𝗮𝗽𝗲𝗿: https://lnkd.in/eCGa8AE8 #generativeai #largelanguagemodel

26

1 commentaire

Memory3: A New Approach to Language Modeling ... Memory3 is a novel architecture for LLMs that introduces an explicit memory mechanism to improve efficiency and performance. Here are the key points: 1. Concept of Explicit Memory:   - Memory3 introduces a new memory format called "explicit memory" that sits between model parameters and plain text retrieval.   - This explicit memory is designed to store knowledge more efficiently than traditional model parameters. 2. Memory Hierarchy:   - Memory3 establishes a memory hierarchy: plain text → explicit memory → model parameters.   - As you move up this hierarchy, the cost of storing information increases, while the cost of accessing decreases. 3. Knowledge Externalization:   - The model externalizes a significant portion of its knowledge into explicit memories, rather than storing everything in model parameters.   - This allows for a smaller backbone model that focuses on abstract knowledge and frequently used specific knowledge. 4. Explicit Memory Format:   - Each explicit memory is a compressed, sparse representation of knowledge derived from attention key-values.   - This format is much more compact than full text, allowing storage of larger knowledge bases. 5. Memory Sparsification:   - To make storage tractable, Memory3 uses aggressive sparsification techniques:    a. Only the first half of attention layers are used as "memory layers".    b. Only 8 out of 128 tokens are selected per attention head.    c. Grouped Query Attention (GQA) is used to reduce the number of key-value heads. 6. Inference Process:   - Before inference, reference texts are converted to explicit memories and stored.   - During inference:    a. The model processes input in chunks of 64 tokens.    b. For each chunk, it retrieves 5 relevant explicit memories.    c. These memories are integrated into the self-attention computation.    d. This process repeats for each 64-token chunk of input. 7. Two-Stage Pretraining:   - Warmup Stage: The model is trained without explicit memories, similar to traditional LLM pretraining.   - Continual Train Stage: Explicit memories are introduced, and the model learns to use them effectively. 8. Advantages:   - Allows for a smaller model size while maintaining or improving performance.   - Enables processing of very long contexts efficiently.   - Improves factuality and reduces hallucination by providing more explicit knowledge. 9. Differences from RAG:   - Memory3 integrates explicit memories directly into the model's attention layers, unlike RAG which typically encodes retrieved passages separately.   - Memory3 retrieves memories more frequently during generation (every 64 tokens) compared to RAG's typical one-time retrieval.   - The explicit memory format is more compact and efficient than RAG's full text passages. 10. Results:   - A 2.4B parameter Memory3 model outperformed larger models and RAG models on various benchmarks.

167

9 commentaires

🚀 ⚡ 𝐀𝐧 𝐢𝐧𝐭𝐫𝐨𝐝𝐮𝐜𝐭𝐢𝐨𝐧 𝐭𝐨 𝐨𝐩𝐭𝐢𝐦𝐢𝐳𝐚𝐭𝐢𝐨𝐧 𝐨𝐧 𝐬𝐦𝐨𝐨𝐭𝐡 𝐦𝐚𝐧𝐢𝐟𝐨𝐥𝐝𝐬 by Nicolas Boumal 🚀 ⚡ - an interesting math-intensive book about #optimization 𝕿𝖆𝖇𝖑𝖊 𝖔𝖋 𝕮𝖔𝖓𝖙𝖊𝖓𝖘: ⭐ Simple examples ⭐ Embedded geometry: first order ⭐ First-order optimization algorithms ⭐ Embedded geometry: second order ⭐ Second-order optimization algorithms ⭐ Embedded submanifolds: examples ⭐ General manifolds ⭐ Quotient manifolds ⭐ Additional tools ⭐ Geodesic convexity 𝕱𝖗𝖊𝖊 𝕯𝖔𝖜𝖓𝖑𝖔𝖆𝖉 𝕷𝖎𝖓𝖐: https://lnkd.in/dxSnXfbv #mathematics #manifolds

202

2 commentaires

How Much Research Is Being Written by Large Language Models? 17.5% (that is assumed to have been detected) https://buff.ly/3VdqVv0 This will mean all future papers on average will be referencing 3 to 5 papers containing LLM generated content significant error rates.

1

📢 Our latest blog post in partnership Argilla is out. We cover SimPO a simplified and faster 🏎️ variant of DPO for aligning LLMs to your use cases.

2

[Article] Our paper "Applying non-negative tensor factorization to centered data" is finally available on BMI, Vol.174, n°3. Congratulations to the authors Paul Fogel, Christophe Geissler, Hans-Jörg von Mettenheim and George Luta! >> More information on our website: https://lnkd.in/e6WQ8iu8 #AI #ESGData #Mazars

21

Researchers have developed a modified greedy algorithm with an improved cost function, enhancing computational efficiency for complex problems. By dynamically selecting the best cost function at runtime, this new approach outperforms traditional methods. Dive into the details of how this innovative algorithm is revolutionizing tensor contractions. Read more - https://lnkd.in/gtXmQXYy 😍 Follow AI Toolhouse for more such amazing content. 🌟 Explore 𝟑𝟔𝟎𝟎+ latest AI Tools here for FREE ➡️ https://lnkd.in/dpQB7xZU #AI #MachineLearning #TensorContraction #Algorithm #Research

1

The attacks on Lattice PQAs are coming thick and fast. Of course Symmetric Key Agreement is provably quantum safe and standards compliant. If you want to transition to a post quantum future, SKA from Arqit is your sleep safe at night option. #PQA #Lattice #NIST #Quantum #Encryption #PostQuantum #Cryptography

18

A new paper was published a few days ago proposing novel methods to consider using to attack Lattice algorithms.     Professor Phong Q. Nguyen, Director of Research in Computer Science at PSL Universite Paris is one of the elder statesman of lattice analysis with a strong track record defeating proposed systems. The work looks mathematically strong at first glance and applies to some popular PQA proposals where the lattices have extra structure, most notably FALCON which is one of the 4th round finalists in NIST standardizations process. The work shows how the ”effective dimension” of the problem could be roughly halved which would roughly halve the bits of security.   It is clear that the usefulness of FALCON must now be very carefully studied.   With Rainbow and SIKE already broken, and novel approaches to weaken Lattices now coming forward in some depth from the research community, it is no longer credible to confidently assert that Lattices can yet be relied upon solely. This is of course why the US Government issued National Security Memorandum 10, instructing National Security System users to move to Symmetric Key protections. Our announcement with Intel yesterdays shows that Arqit has the solution, in a Symmetric Key Agreement platform that is not only provably quantum safe, it is orchestrated from the cloud with no hardware required and now incredibly easy to deploy with Generally Available integrations with very important global vendors like Intel Corporation.    https://lnkd.in/d6MeEz2t #Lattice #PQA #PostQuantum #Cryptography #Encryption

3