“After working with Peter for more than a year on our company's core marketing optimization projects, I have tremendous respect for his intellect and, in particular, his ability to bridge business understanding with data science insights to produce models that deliver real value. Peter pushes himself to understand not just the mathematical minutiae of a brilliant model, but also the context in which they operate. That difference is what makes Peter so talented; he's a data scientist focused on practical results, ready to apply his intellect beyond data science. Not only that, Peter is fun to work with and quickly earns the respect of his colleagues. Anyone looking for a smart, results-focused and personable data scientist able to add value well beyond the realm of the quantitative would be lucky to count Peter on his or her team.”
About
Experience & Education
Publications
-
Equivariant Hamiltonian Flows
This paper introduces equivariant hamiltonian flows, a method for learning expressive densities that are invariant with respect to a known Lie-algebra of local symmetry transformations while providing an equivariant representation of the data. We provide proof of principle demonstrations of how such flows can be learnt, as well as how the addition of symmetry invariance constraints can improve data efficiency and generalisation. Finally, we make connections to disentangled representation…
This paper introduces equivariant hamiltonian flows, a method for learning expressive densities that are invariant with respect to a known Lie-algebra of local symmetry transformations while providing an equivariant representation of the data. We provide proof of principle demonstrations of how such flows can be learnt, as well as how the addition of symmetry invariance constraints can improve data efficiency and generalisation. Finally, we make connections to disentangled representation learning and show how this work relates to a recently proposed definition.
Other authorsSee publication -
Hamiltonian Generative Networks
The Hamiltonian formalism plays a central role in classical and quantum physics. Hamiltonians are the main tool for modelling the continuous time evolution of systems with conserved quantities, and they come equipped with many useful properties, like time reversibility and smooth interpolation in time. These properties are important for many machine learning problems - from sequence prediction to reinforcement learning and density modelling - but are not typically provided out of the box by…
The Hamiltonian formalism plays a central role in classical and quantum physics. Hamiltonians are the main tool for modelling the continuous time evolution of systems with conserved quantities, and they come equipped with many useful properties, like time reversibility and smooth interpolation in time. These properties are important for many machine learning problems - from sequence prediction to reinforcement learning and density modelling - but are not typically provided out of the box by standard tools such as recurrent neural networks. In this paper, we introduce the Hamiltonian Generative Network (HGN), the first approach capable of consistently learning Hamiltonian dynamics from high-dimensional observations (such as images) without restrictive domain assumptions. Once trained, we can use HGN to sample new trajectories, perform rollouts both forward and backward in time and even speed up or slow down the learned dynamics. We demonstrate how a simple modification of the network architecture turns HGN into a powerful normalising flow model, called Neural Hamiltonian Flow (NHF), that uses Hamiltonian dynamics to model expressive densities. We hope that our work serves as a first practical demonstration of the value that the Hamiltonian formalism can bring to deep learning.
Other authorsSee publication -
Online Learning with Gated Linear Networks
This paper describes a family of probabilistic architectures designed for online learning under the logarithmic loss. Rather than relying on non-linear transfer functions, our method gains representational power by the use of data conditioning. We state under general conditions a learnable capacity theorem that shows this approach can in principle learn any bounded Borel-measurable function on a compact subset of euclidean space; the result is stronger than many universality results for…
This paper describes a family of probabilistic architectures designed for online learning under the logarithmic loss. Rather than relying on non-linear transfer functions, our method gains representational power by the use of data conditioning. We state under general conditions a learnable capacity theorem that shows this approach can in principle learn any bounded Borel-measurable function on a compact subset of euclidean space; the result is stronger than many universality results for connectionist architectures because we provide both the model and the learning procedure for which convergence is guaranteed.
Other authorsSee publication -
Criticality & Deep Learning II: Momentum Renormalisation Group
arxiv.org
Guided by critical systems found in nature we develop a novel mechanism consisting of inhomogeneous polynomial regularisation via which we can induce scale invariance in deep learning systems. Technically, we map our deep learning (DL) setup to a genuine field theory, on which we act with the Renormalisation Group (RG) in momentum space and produce the flow equations of the couplings; those are translated to constraints and consequently interpreted as "critical regularisation" conditions in the…
Guided by critical systems found in nature we develop a novel mechanism consisting of inhomogeneous polynomial regularisation via which we can induce scale invariance in deep learning systems. Technically, we map our deep learning (DL) setup to a genuine field theory, on which we act with the Renormalisation Group (RG) in momentum space and produce the flow equations of the couplings; those are translated to constraints and consequently interpreted as "critical regularisation" conditions in the optimiser; the resulting equations hence prove to be sufficient conditions for - and serve as an elegant and simple mechanism to induce scale invariance in any deep learning setup.
Other authorsSee publication -
Criticality and Deep Learning, Part I: Theory vs. Empirics
arxiv.org
Motivated by the idea that criticality and universality of phase transitions might play a crucial role in achieving and sustaining learning and intelligent behaviour in biological and artificial networks, we analyse a theoretical and a pragmatic experimental set up for critical phenomena in deep learning. On the theoretical side, we use results from statistical physics to carry out critical point calculations in feed-forward/fully connected networks, while on the experimental side we set out to…
Motivated by the idea that criticality and universality of phase transitions might play a crucial role in achieving and sustaining learning and intelligent behaviour in biological and artificial networks, we analyse a theoretical and a pragmatic experimental set up for critical phenomena in deep learning. On the theoretical side, we use results from statistical physics to carry out critical point calculations in feed-forward/fully connected networks, while on the experimental side we set out to find traces of criticality in deep neural networks. This is our first step in a series of upcoming investigations to map out the relationship between criticality and learning in deep networks.
Other authorsSee publication
Projects
-
CriticalAI
- Present
Codebase for experiments in CriticalAI: Motivated by the idea that criticality and universality of phase transitions might play a crucial role in achieving and sustaining learning and intelligent behaviour in biological and artificial networks, we analyse a theoretical and a pragmatic experimental set up for critical phenomena in deep learning. On the theoretical side, we use results from statistical physics to carry out critical point calculations in feed-forward/fully connected networks…
Codebase for experiments in CriticalAI: Motivated by the idea that criticality and universality of phase transitions might play a crucial role in achieving and sustaining learning and intelligent behaviour in biological and artificial networks, we analyse a theoretical and a pragmatic experimental set up for critical phenomena in deep learning. On the theoretical side, we use results from statistical physics to carry out critical point calculations in feed-forward/fully connected networks, while on the experimental side we set out to find traces of criticality in deep neural networks. This is our first step in a series of upcoming investigations to map out the relationship between criticality and learning in deep networks.
https://www.github.com/eidonfiloi/criticalai -
Gholly
- Present
Generative Holographic Criticality - a web app for visualising the growth, statistics and dynamics of critical network architectures.
-
ReCog
https://github.com/eidonfiloi/SparseRecurrentNetwork
ReCog is an experimental framework for developing and testing deep, recurrent neural networks for sequence prediction. It is based on an experimental architecture that uses among others
- sparse autoencoders
- a custom cell unit containing feedforward, recurrent and feedback connections with a custom update logic
- gradient descent with momentum and adaptive weight updates using local gain matrices
- dropout and…https://github.com/eidonfiloi/SparseRecurrentNetwork
ReCog is an experimental framework for developing and testing deep, recurrent neural networks for sequence prediction. It is based on an experimental architecture that uses among others
- sparse autoencoders
- a custom cell unit containing feedforward, recurrent and feedback connections with a custom update logic
- gradient descent with momentum and adaptive weight updates using local gain matrices
- dropout and inhibition for regularization and algorithmic sparsification
- audio and text input preprocessing
It is written in Python to provide easy, simple developing-testing cycles (parallelization and cluster deployment is currently WIP as well as development in Scala/Spark).
Languages
-
English
Professional working proficiency
-
German
Professional working proficiency
-
Hungarian
Native or bilingual proficiency
Recommendations received
4 people have recommended Peter
Join now to viewOther similar profiles
Explore collaborative articles
We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.
Explore MoreOthers named Peter Toth in United States
67 others named Peter Toth in United States are on LinkedIn
See others named Peter Toth