Showing 1–11 of 11 results for author: Abascal, J

Phantom: General Trigger Attacks on Retrieval Augmented Language Generation

Authors: Harsh Chaudhari, Giorgio Severi, John Abascal, Matthew Jagielski, Christopher A. Choquette-Choo, Milad Nasr, Cristina Nita-Rotaru, Alina Oprea

Abstract: Retrieval Augmented Generation (RAG) expands the capabilities of modern large language models (LLMs) in chatbot applications, enabling developers to adapt and personalize the LLM output without expensive training or fine-tuning. RAG systems use an external knowledge database to retrieve the most relevant documents for a given query, providing this context to the LLM generator. While RAG achieves i… ▽ More Retrieval Augmented Generation (RAG) expands the capabilities of modern large language models (LLMs) in chatbot applications, enabling developers to adapt and personalize the LLM output without expensive training or fine-tuning. RAG systems use an external knowledge database to retrieve the most relevant documents for a given query, providing this context to the LLM generator. While RAG achieves impressive utility in many applications, its adoption to enable personalized generative models introduces new security risks. In this work, we propose new attack surfaces for an adversary to compromise a victim's RAG system, by injecting a single malicious document in its knowledge database. We design Phantom, general two-step attack framework against RAG augmented LLMs. The first step involves crafting a poisoned document designed to be retrieved by the RAG system within the top-k results only when an adversarial trigger, a specific sequence of words acting as backdoor, is present in the victim's queries. In the second step, a specially crafted adversarial string within the poisoned document triggers various adversarial attacks in the LLM generator, including denial of service, reputation damage, privacy violations, and harmful behaviors. We demonstrate our attacks on multiple LLM architectures, including Gemma, Vicuna, and Llama. △ Less

Submitted 30 May, 2024; originally announced May 2024.

TMI! Finetuned Models Leak Private Information from their Pretraining Data

Authors: John Abascal, Stanley Wu, Alina Oprea, Jonathan Ullman

Abstract: Transfer learning has become an increasingly popular technique in machine learning as a way to leverage a pretrained model trained for one task to assist with building a finetuned model for a related task. This paradigm has been especially popular for $\textit{privacy}$ in machine learning, where the pretrained model is considered public, and only the data for finetuning is considered sensitive. H… ▽ More Transfer learning has become an increasingly popular technique in machine learning as a way to leverage a pretrained model trained for one task to assist with building a finetuned model for a related task. This paradigm has been especially popular for $\textit{privacy}$ in machine learning, where the pretrained model is considered public, and only the data for finetuning is considered sensitive. However, there are reasons to believe that the data used for pretraining is still sensitive, making it essential to understand how much information the finetuned model leaks about the pretraining data. In this work we propose a new membership-inference threat model where the adversary only has access to the finetuned model and would like to infer the membership of the pretraining data. To realize this threat model, we implement a novel metaclassifier-based attack, $\textbf{TMI}$, that leverages the influence of memorized pretraining samples on predictions in the downstream task. We evaluate $\textbf{TMI}$ on both vision and natural language tasks across multiple transfer learning settings, including finetuning with differential privacy. Through our evaluation, we find that $\textbf{TMI}$ can successfully infer membership of pretraining examples using query access to the finetuned model. An open-source implementation of $\textbf{TMI}$ can be found $\href{https://github.com/johnmath/tmi-pets24}{\text{on GitHub}}$. △ Less

Submitted 21 March, 2024; v1 submitted 1 June, 2023; originally announced June 2023.

SNAP: Efficient Extraction of Private Properties with Poisoning

Authors: Harsh Chaudhari, John Abascal, Alina Oprea, Matthew Jagielski, Florian Tramèr, Jonathan Ullman

Abstract: Property inference attacks allow an adversary to extract global properties of the training dataset from a machine learning model. Such attacks have privacy implications for data owners sharing their datasets to train machine learning models. Several existing approaches for property inference attacks against deep neural networks have been proposed, but they all rely on the attacker training a large… ▽ More Property inference attacks allow an adversary to extract global properties of the training dataset from a machine learning model. Such attacks have privacy implications for data owners sharing their datasets to train machine learning models. Several existing approaches for property inference attacks against deep neural networks have been proposed, but they all rely on the attacker training a large number of shadow models, which induces a large computational overhead. In this paper, we consider the setting of property inference attacks in which the attacker can poison a subset of the training dataset and query the trained target model. Motivated by our theoretical analysis of model confidences under poisoning, we design an efficient property inference attack, SNAP, which obtains higher attack success and requires lower amounts of poisoning than the state-of-the-art poisoning-based property inference attack by Mahloujifar et al. For example, on the Census dataset, SNAP achieves 34% higher success rate than Mahloujifar et al. while being 56.5x faster. We also extend our attack to infer whether a certain property was present at all during training and estimate the exact proportion of a property of interest efficiently. We evaluate our attack on several properties of varying proportions from four datasets and demonstrate SNAP's generality and effectiveness. An open-source implementation of SNAP can be found at https://github.com/johnmath/snap-sp23. △ Less

Submitted 21 June, 2023; v1 submitted 25 August, 2022; originally announced August 2022.

Comments: 28 pages, 16 figures

Strongly refuting all semi-random Boolean CSPs

Authors: Jackson Abascal, Venkatesan Guruswami, Pravesh K. Kothari

Abstract: We give an efficient algorithm to strongly refute \emph{semi-random} instances of all Boolean constraint satisfaction problems. The number of constraints required by our algorithm matches (up to polylogarithmic factors) the best-known bounds for efficient refutation of fully random instances. Our main technical contribution is an algorithm to strongly refute semi-random instances of the Boolean… ▽ More We give an efficient algorithm to strongly refute \emph{semi-random} instances of all Boolean constraint satisfaction problems. The number of constraints required by our algorithm matches (up to polylogarithmic factors) the best-known bounds for efficient refutation of fully random instances. Our main technical contribution is an algorithm to strongly refute semi-random instances of the Boolean $k$-XOR problem on $n$ variables that have $\widetilde{O}(n^{k/2})$ constraints. (In a semi-random $k$-XOR instance, the equations can be arbitrary and only the right-hand sides are random.) One of our key insights is to identify a simple combinatorial property of random XOR instances that makes spectral refutation work. Our approach involves taking an instance that does not satisfy this property (i.e., is \emph{not} pseudorandom) and reducing it to a partitioned collection of $2$-XOR instances. We analyze these subinstances using a carefully chosen quadratic form as a proxy, which in turn is bounded via a combination of spectral methods and semidefinite programming. The analysis of our spectral bounds relies only on an off-the-shelf matrix Bernstein inequality. Even for the purely random case, this leads to a shorter proof compared to the ones in the literature that rely on problem-specific trace-moment computations. △ Less

Submitted 16 September, 2020; originally announced September 2020.

Comments: 31 Pages

ACM Class: F.2.2

A Non-iterative Parallelizable Eigenbasis Algorithm for Johnson Graphs

Authors: Jackson Abascal, Amadou Bah, Mario Banuelos, David Uminsky, Olivia Vasquez

Abstract: We present a new $O(k^2 \binom{n}{k}^2)$ method for generating an orthogonal basis of eigenvectors for the Johnson graph $J(n,k)$. Unlike standard methods for computing a full eigenbasis of sparse symmetric matrices, the algorithm presented here is non-iterative, and produces exact results under an infinite-precision computation model. In addition, our method is highly parallelizable; given access… ▽ More We present a new $O(k^2 \binom{n}{k}^2)$ method for generating an orthogonal basis of eigenvectors for the Johnson graph $J(n,k)$. Unlike standard methods for computing a full eigenbasis of sparse symmetric matrices, the algorithm presented here is non-iterative, and produces exact results under an infinite-precision computation model. In addition, our method is highly parallelizable; given access to unlimited parallel processors, the eigenbasis can be constructed in only $O(n)$ time given n and k. We also present an algorithm for computing projections onto the eigenspaces of $J(n,k)$ in parallel time $O(n)$. △ Less

Submitted 11 December, 2018; originally announced December 2018.

Critique of Barbosa's "P != NP Proof"

Authors: Jackson Abascal, Shir Maimon

Abstract: We review André Luiz Barbosa's paper "P != NP Proof," in which the classes P and NP are generalized and claimed to be proven separate. We highlight inherent ambiguities in Barbosa's definitions, and show that attempts to resolve this ambiguity lead to flaws in the proof of his main result. We review André Luiz Barbosa's paper "P != NP Proof," in which the classes P and NP are generalized and claimed to be proven separate. We highlight inherent ambiguities in Barbosa's definitions, and show that attempts to resolve this ambiguity lead to flaws in the proof of his main result. △ Less

Submitted 19 November, 2017; originally announced November 2017.

A Refutation of Guinea's "Understanding SAT is in P"

Authors: Jackson Abascal, Shir Maimon

Abstract: In this work, we summarize and critique the paper "Understanding SAT is in P" by Alejandro Sánchez Guinea [arXiv:1504.00337]. The paper claims to present a polynomial-time solution for the NP-complete language 3-SAT. We show that Guinea's algorithm is flawed and does not prove 3-SAT is in P. In this work, we summarize and critique the paper "Understanding SAT is in P" by Alejandro Sánchez Guinea [arXiv:1504.00337]. The paper claims to present a polynomial-time solution for the NP-complete language 3-SAT. We show that Guinea's algorithm is flawed and does not prove 3-SAT is in P. △ Less

Submitted 12 November, 2017; originally announced November 2017.

Incorporation of prior knowledge of the signal behavior into the reconstruction to accelerate the acquisition of MR diffusion data

Authors: Juan F P J Abascal, Manuel Desco, Juan Parra-Robles

Abstract: Diffusion MRI measurements using hyperpolarized gases are generally acquired during patient breath hold, which yields a compromise between achievable image resolution, lung coverage and number of b-values. In this work, we propose a novel method that accelerates the acquisition of MR diffusion data by undersampling in both spatial and b-value dimensions, thanks to incorporating knowledge about the… ▽ More Diffusion MRI measurements using hyperpolarized gases are generally acquired during patient breath hold, which yields a compromise between achievable image resolution, lung coverage and number of b-values. In this work, we propose a novel method that accelerates the acquisition of MR diffusion data by undersampling in both spatial and b-value dimensions, thanks to incorporating knowledge about the signal decay into the reconstruction (SIDER). SIDER is compared to total variation (TV) reconstruction by assessing their effect on both the recovery of ventilation images and estimated mean alveolar dimensions (MAD). Both methods are assessed by retrospectively undersampling diffusion datasets of normal volunteers and COPD patients (n=8) for acceleration factors between x2 and x10. TV led to large errors and artefacts for acceleration factors equal or larger than x5. SIDER improved TV, presenting lower errors and histograms of MAD closer to those obtained from fully sampled data for accelerations factors up to x10. SIDER preserved image quality at all acceleration factors but images were slightly smoothed and some details were lost at x10. In conclusion, we have developed and validated a novel compressed sensing method for lung MRI imaging and achieved high acceleration factors, which can be used to increase the amount of data acquired during a breath-hold. This methodology is expected to improve the accuracy of estimated lung microstructure dimensions and widen the possibilities of studying lung diseases with MRI. △ Less

Submitted 9 February, 2017; originally announced February 2017.

Closure and Nonclosure Properties of the Compressible and Rankable Sets

Authors: Jackson Abascal, Lane A. Hemaspaandra, Shir Maimon, Daniel Rubery

Abstract: The rankable and compressible sets have been studied for more than a quarter of a century, ever since Allender [1] and Goldberg and Sipser [6] introduced the formal study of polynomial-time ranking. Yet even after all that time, whether the rankable and compressible sets are closed under the most important boolean and other operations remains essentially unexplored. The present paper studies these… ▽ More The rankable and compressible sets have been studied for more than a quarter of a century, ever since Allender [1] and Goldberg and Sipser [6] introduced the formal study of polynomial-time ranking. Yet even after all that time, whether the rankable and compressible sets are closed under the most important boolean and other operations remains essentially unexplored. The present paper studies these questions for both polynomial-time and recursion-theoretic compression and ranking, and for almost every case arrives at a Closed, a Not-Closed, or a Closed-Iff-Well-Known-Complexity-Classes-Collapse result for the given operation. Even though compression and ranking classes are capturing something quite natural about the structure of sets, it turns out that they are quite fragile with respect to closure properties, and many fail to possess even the most basic of closure properties. For example, we show that with respect to the join (aka disjoint union) operation: the P-rankable sets are not closed, whether the semistrongly P-rankable sets are closed is closely linked to whether P = UP $\cap$ coUP, and the strongly P-rankable sets are closed. △ Less

Submitted 30 October, 2018; v1 submitted 5 November, 2016; originally announced November 2016.

ACM Class: F.1.1; F.4.1; F.2.2

ICT technologies for the refurbishment of wooden structure buildings

Authors: Ivan Arakistain, Jose Miguel Abascal, Oriol Munne

Abstract: Nowadays, one would think that after years of massive concrete and steel construction in Spain, there are not many wood structure buildings left to be refurbished except for some palaces or cathedrals. However, if we go for a walk and have a look at the old part of any city, we will realize that still most of the buildings have a wood structure. In spite of the fact that the majority of urban regu… ▽ More Nowadays, one would think that after years of massive concrete and steel construction in Spain, there are not many wood structure buildings left to be refurbished except for some palaces or cathedrals. However, if we go for a walk and have a look at the old part of any city, we will realize that still most of the buildings have a wood structure. In spite of the fact that the majority of urban regulations forbid their demolition, other bad practices such as casting and overloading the wood structure are very common. Considering that we want to reach a standard of sustainable construction, the economical and environmental costs, which are implied by the deficient refurbishment makes it well worth a previous study of the structure, which in most cases represents less than a 1% of the total budget. The main goal of this paper is to present most relevant parts of the whole process of diagnosis of a wood structure building by means of Non-Destructive Testing Techniques. Among the ones to be considered, we could mention the analysis of the building and its surroundings, on-site inspection of the building, structural diagnosis, definition of the corrective actions to be taken, definition of treatments, quality control and a maintenance plan. For the on-site inspection of the building, in the paper we will highlight the use of Non-Destructive methods such as resistograph drilling, X-ray imaging, ultrasound-based testing or moisture measurement. We will provide practical examples of all this. The aim of this paper is to give the audience an overall idea on how a pre-assessment work can enhance the refurbishment of a wood structure building while reducing costs and environmental impact. △ Less

Submitted 31 January, 2014; originally announced January 2014.

Wireless sensor network technology for moisture monitoring of wood

Authors: Ivan Arakistain, Jose Miguel Abascal, Oriol Munne

Abstract: Leaks represent a very important hazard for the buildings and they can affect all sorts of building materials and specially wood due to its hygroscopic properties. Excessive moisture content can affect in a negative way building processes such as the installation of wooden floors or the use of wood as a structural material. Moisture meters can provide prompt and non-destructive determination of wo… ▽ More Leaks represent a very important hazard for the buildings and they can affect all sorts of building materials and specially wood due to its hygroscopic properties. Excessive moisture content can affect in a negative way building processes such as the installation of wooden floors or the use of wood as a structural material. Moisture meters can provide prompt and non-destructive determination of wood moisture, and as such are among the most useful tools available to wood products manufacturers and scientists. However, a continuous monitoring system is needed in order to avoid excessive moisture content which can damage wooden floors as well as structural wood. Data and procedures are presented in order to develop a suitable monitoring tool based on wireless sensor networks to provide an electronic tool of active security both for the installation of wooden floors and for the proper maintenance of existent buildings which have a timber structure. △ Less

Submitted 3 July, 2013; originally announced July 2013.