AI Technology & Industry Review
In a recent technical report, OpenAI introduces Sora, a groundbreaking text-to-video model. Sora stands out for its ability to generate videos and images spanning a wide range of durations, aspect ratios, and resolutions, producing up to a minute of high-definition video content.
In a new paper PianoMime: Learning a Generalist, Dexterous Piano Player from Internet Demonstrations, a research team introduces PianoMime, a framework for training a robot to play the piano using internet-sourced demonstrations.
In a new paper OpenDevin: An Open Platform for AI Software Developers as Generalist Agents, a research team introduces OpenDevin, an Open Platform for AI Software Developers as Generalist Agents. This community-driven platform supports the development of AI agents that interact with software systems.
A DeepMind research team release PaliGemma, a robust and versatile vision language model with 3 billion parameters. PaliGemma excels in transfer learning across various vision and language tasks, achieving state-of-the-art performance in a multitude of open-world applications.
A research team from Saskatchewan University and Google introduces an innovative unsupervised method for automatic video highlight detection, eliminating the requirements for manual annotations while achieving superior performance compared to previous methods.
A Stanford University research team proposes Hypothetical Minds, builds on recent advancements in LLM-based agents designed for multi-agent environments, aiming to enhance adaptability in competitive, cooperative, and mixed-motive scenarios with concealed information.
A DeepMind research team introduces PEER, a innovative layer design leverages the product key technique for sparse retrieval from an extensive pool of tiny experts (over a million), which unlocks the potential for further scaling transformer models while maintaining computational efficiency.
A research team from Microsoft and University of Surrey introduces MInference (Milliontokens Inference), which employs a sparse calculation approach designed to expedite the pre-filling of long-sequence processing. It can reduce inference latency by up to 10 times on an A100 GPU while preserving accuracy.
In a new paper FACTS About Building Retrieval Augmented Generation-based Chatbots, an NVIDIA research team introduces the FACTS framework, designed to create robust, secure, and enterprise-grade RAG-based chatbots.
A Meta AI research team introduces Meta Large Language Model Compiler, a suite of robust, openly available, pre-trained models is specifically designed for code optimization tasks, aiming to provide a scalable, cost-effective foundation for further research and development in compiler optimization.
In a new paper How to Boost Any Loss Function, a Google research team provides a constructive, formal answer, demonstrating that any loss function can be optimized with boosting.
In a new paper RL on Incorrect Synthetic Data Scales the Efficiency of LLM Math Reasoning by Eight-Fold, a research team provides insights into how synthetic data affects performance, suggesting that a specific schema can achieve consistent gains over using only positive data, achieving performance by 8× in synthetic data volume.
A research team from University of Illinois Urbana-Champaign introduces HPTSA, a multi-agent system that significantly advances cybersecurity exploits, achieving up to 4.5 times better performance on a benchmark of 15 real-world vulnerabilities compared to previous efforts.
In a new paper Artificial Generational Intelligence: Cultural Accumulation in Reinforcement Learning, a research team from the University of Oxford and Google DeepMind introduces methods to achieve cultural accumulation in Reinforcement Learning (RL) agents. This research opens new pathways for modeling human culture through artificial systems.
In a new paper SleepFM: Multi-modal Representation Learning for Sleep Across Brain Activity, ECG and Respiratory Signals, a research team introduces SleepFM, the first attempt at developing a multi-modal contrastive learning (CL) approach for PSG analysis, outperforming baselines in tasks like demographic attribute prediction and sleep stage classification.
In a new paper Proofread: Fixes All Errors with One Tap, a Google research team introduces Proofread, an innovative Gboard feature powered by a server-side LLM. This feature allows for seamless sentence and paragraph corrections with a single tap. Launched on Pixel 8 devices, it benefits thousands of users daily.
“Global Vision, Ideas in Collision, Leading Cutting-Edge Innovations” – The 6th annual BAAI Conference successfully concluded on June 15. Over 200 AI scholars and industry leaders gathered to discuss the trajectories and applications of advanced AI technologies.
In a new paper TextGrad: Automatic ‘Differentiation’ via Text, a research team from Stanford University and CZ Biohub introduces TEXTGRAD, a robust framework that performs automatic differentiation through text. In this system, LLMs generate comprehensive, natural language suggestions to optimize variables in computation graphs.
In a recent new paper VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers, a Microsoft research team presents VALL-E 2, the latest advancement in neural codec language models. This innovation marks a milestone in zero-shot TTS synthesis by achieving human parity for the first time.
In a new paper Scalable MatMul-free Language Modeling, a research team introduces the first scalable MatMul-free language model, demonstrating that it is possible to completely eliminate MatMul operations from large language models (LLMs) while maintaining robust performance, even at billion-parameter scales.
A research team introduce Instruct-MusicGen, an innovative method that fine-tunes a pretrained MusicGen model to efficiently follow editing instructions, delivering superior performance across various tasks compared to existing benchmarks.
In a new paper Zipper: A Multi-Tower Decoder Architecture for Fusing Modalities, a Google DeepMind research team introduces Zipper, a multi-tower decoder architecture. This architecture can flexibly combine multimodal generative models from independently pre-trained unimodal decoders and can be reused and repurposed in new multimodal combinations.
In a new paper NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models, an NVIDIA research team introduces NV-Embed. This generalist embedding model significantly boosts the performance of decoder-only LLMs in embedding and retrieval tasks while maintaining simplicity and reproducibility.
In a new paper Your Transformer is Secretly Linear, a research team uncovers a near-perfect linear relationship in transformations between sequential layers and introduces a novel distillation technique that approximates certain layers linearly while preserving model performance.
In a new paper A Generalist Learner for Multifaceted Medical Image Interpretation, a research team proposes MedVersa, a generalist AI model designed to enable flexible learning and tasking for medical image interpretation.
In a new paper SpeechVerse: A Large-scale Generalizable Audio Language Model, a research team from Amazon AWS AI Labs introduces SpeechVerse, a robust multi-task framework that leverages supervised instruction fine-tuning to achieve strong performance across various speech tasks.
In a new paper Imagine Flash: Accelerating Emu Diffusion Models with Backward Distillation, a Meta GenAI research team introduces an innovative distillation framework aimed at enabling high-fidelity, diverse sample generation within just one to three steps. This framework surpasses existing competitors in both quantitative metrics and human evaluations.
An IBM research team introduces the Granite Code model family. Specifically optimized for enterprise software development workflows, these models excel across a spectrum of coding tasks, rendering them versatile and well-suited for diverse coding challenges.
a research team from Google and Verily introduce Med-Gemini, a family of highly proficient multimodal models is tailored for medical tasks, boasting the capacity to seamlessly integrate web search functionality and adapt efficiently to new modalities through customized encoders.
In a new paper KAN: Kolmogorov-Arnold Networks, a research team introduces Kolmogorov-Arnold Networks (KANs) as promising alternatives to MLPs. These networks showcase superior performance in both accuracy and interpretability domains.
In a new paper NeMo-Aligner: Scalable Toolkit for Efficient Model Alignment, a team of researchers from Nvidia introduces NeMo-Aligner, a toolkit designed for large-scale LLM model alignment that can efficiently harness the power of hundreds of GPUs for training.
A pioneering research group introduces MovieChat, a novel framework tailored to accommodate extensive video durations exceeding 10,000 frames. This innovative system achieves unprecedented performance in deciphering prolonged video content.
In a new paper TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding, a research team from CMU and Meta introduces TriForce—a hierarchical speculative decoding system tailored for scalable long sequence generation, reaching up to 2.31× on an A100 GPU.
In a new paper NExT: Teaching Large Language Models to Reason about Code Execution, a Google DeepMind research team proposes Naturalized Execution Tuning (NExT), a method aims to equip LLMs with the ability to scrutinize program execution traces and deduce runtime behaviors through chain-of-thought (CoT) rationales.
An NVIDIA research team presents ScaleFold, a novel and scalable training methodology tailored for the AlphaFold model, which accomplishes the OpenFold partial training task in a mere 7.51 minutes—over six times faster than the benchmark baseline—ultimately slashing the AlphaFold’s initial training time to a remarkable 10 hours.
A Google DeepMind research team introduce RecurrentGemma, an open language model built on Google’s innovative Griffin architecture, which reduces memory usage and facilitates efficient inference on lengthy sequences, thereby unlocking new possibilities for highly efficient small language models in environments where resources are limited.
A Google research team unveils the latest iteration of MobileNets: MobileNetV4 (MNv4). This cutting-edge model boasts an impressive 87% ImageNet-1K accuracy, coupled with an astonishingly low Pixel 8 EdgeTPU runtime of merely 3.8ms.
In a new paper LM Transparency Tool: Interactive Tool for Analyzing Transformer Language Models, a research team from Meta, University College London and Universitat Politècnica de Catalunya introduces the LM Transparency Tool (LM-TT), an open-source interactive toolkit designed for dissecting Transformer-based language models.
In a new paper Streaming Dense Video Captioning, a Google research team proposes a streaming dense video captioning model, which revolutionizes dense video captioning by enabling the processing of videos of any length and making predictions before the entire video is fully analyzed, thus marking a significant advancement in the field.
A collaborative effort involving researchers from 33 institutions presents AURORA-M, the inaugural open-source model not only excels in multilingual understanding and coding tasks but also underscores the collaborative ethos of the open-source community, promoting transparency and accessibility in AI development.
A research team from Huawei and Peking University introduces DiJiang, a groundbreaking Frequency Domain Kernelization approach, which facilitates the transition to a linear complexity model with minimal training overhead, achieving performance akin to LLaMA2-7B across various benchmarks, but at just 1/50th of the training cost.