Wednesday, April 29, 2020

Come and join us at LightOn, we have a 3-year PhD fellowship available for someone who can help us build our future photonic cores. Here is

As part of the newly EU-funded ITN project “Post-Digital”, LightOn has an opening for a fully-funded 3 year Ph.D. studentship to join its R&D team, at the crossroads between Computer Science and Physics.

The goal of this 3 year Ph.D. position is to theoretically, numerically, and experimentally investigate how optimization techniques can be used in the design of hybrid computing pipelines, including a number of photonic building blocks (“photonic cores”). In particular, the optimized networks will be used to solve large-scale physics-based inverse problems in science and engineering - for instance in medical imaging (e.g. ultrasound), or simulation problems. The candidate will first investigate how LigthOn’s current range of photonics co-processors can be integrated within task-specific networks. The candidate will then develop a computational framework for the optimization of electro-optical systems. Finally, optimized systems will be built and evaluated on experimental data. This project will be part of LightOn’s internal THEIA project, aiming at automating the design of hybrid computing architectures, including combinations of LightOn’s photonic cores and traditional silicon chips.

In the framework of the EU funded ITN Post-Digital network, this project involves collaborations and 3-month secondments with two research groups led by:

Daniel Brunner (Université Bourgogne Franche-Comté / FEMTO-ST Besançon), who will be the academic supervisor - The candidate will be registered as a Ph.D. student at UBFC.

The supervisor at LightOn will be Laurent Daudet, CTO - currently on leave from his position of professor of physics at Université de Paris.

Due to the EU funding source, please make sure you comply with the mobility and eligibility rule before applying. Application: Position to be filled no later than Sept 1st, 2020.

Send your application with a CV to jobs@lighton.io with [Post-Digital PhD] in the subject line. Shortlisted applicants will be asked to provide references. This project has received funding from the European Union’s Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement No 860830.

Tuesday, April 07, 2020

At LightOn, we just launched LightOn Cloud 2.0 that feature several Aurora Optical Processing Unit for use by the Machine Learning Community. the blog post about this can be found here. You can request access to the Cloud at https://cloud.lighton.ai/

Because building computational hardware makes no sense if we don't have a community that lifts us, the code used to generate the plots in that blog post is publicly available at the following link: https://github.com/lightonai/newma-md.

A key element of understanding the efficacy of overparameterized neural networks is characterizing how they represent functions as the number of weights in the network approaches infinity. In this paper, we characterize the norm required to realize a function f:Rd→R as a single hidden-layer ReLU network with an unbounded number of units (infinite width), but where the Euclidean norm of the weights is bounded, including precisely characterizing which functions can be realized with finite norm. This was settled for univariate univariate functions in Savarese et al. (2019), where it was shown that the required norm is determined by the L1-norm of the second derivative of the function. We extend the characterization to multivariate functions (i.e., networks with d input units), relating the required norm to the L1-norm of the Radon transform of a (d+1)/2-power Laplacian of the function. This characterization allows us to show that all functions in Sobolev spaces Ws,1(R), s≥d+1, can be represented with bounded norm, to calculate the required norm for several specific functions, and to obtain a depth separation result. These results have important implications for understanding generalization performance and the distinction between neural networks and more traditional kernel learning.

Ata Kaban, University of Birmingham.
Compressive Learning with Random Projections

By direct analogy to compressive sensing, compressive learning has been originally coined to mean learning efficiently from random projections of high dimensional massive data sets that have a sparse representation. In this talk we discuss compressive learning without the sparse representation requirement, where instead we exploit the

The advent of deep learning has considerably accelerated machine learning development, but its development at the edge is limited by its high energy cost and memory requirement. With new memory technology available, emerging Binarized Neural Networks (BNNs) are promising to reduce the energy impact of the forthcoming machine learning hardware generation, enabling machine learning on the edge devices and avoiding data transfer over the network. In this talk we will discuss strategies to apply BNNs to biomedical signals such as electrocardiography and electroencephalography, without sacrificing accuracy and improving energy use. The ultimate goal of this research is to enable smart autonomous healthcare devices.

In response to the development of recent efficient dense layers, this talk discusses replacing linear components in pointwise convolutions with structured linear decompositions for substantial gains in the efficiency/accuracy tradeoff. Pointwise convolutions are fully connected layers and are thus prepared for replacement by structured transforms. Networks using such layers are able to learn the same tasks as those using standard convolutions, and provide Pareto-optimal benefits in efficiency/accuracy, both in terms of computation (mult-adds) and parameter count (and hence memory).

LightOn’s OPU is opening a new machine learning paradigm. Two use cases have been selected to investigate the potentiality of OPU for particle physics:

End-to-End learning: high energy proton collision at the Large Hadron Collider have been simulated, each collision being recorded as an image representing the energy flux in the detector. Two classes of events have been simulated: signal are created by a hypothetical supersymmetric particle, and background by known processes. The task is to train a classifier to separate the signal from the background. Several techniques using the OPU will be presented, compared with more classical particle physics approaches.

Tracking: high energy proton collisions at the LHC yield billions of records with typically 100,000 3D points corresponding to the trajectory of 10,000 particles. Various investigations of the potential of the OPU to digest this high dimensional data will be reported.

Random projections belong to the major techniques used to process big data. They have been successfully applied to, e.g., (Nonnegative) Matrix Factorization ((N)MF). However, missing entries in the matrix to factorize (or more generally weights which model the confidence in the entries of the data matrix) prevent their use. In this talk, I will present the framework that we recently proposed to solve this issue, i.e., to apply random projections to weighted (N)MF. We experimentally show the proposed framework to significantly speed-up state-of-the-art weighted NMF methods under some mild conditions.

The workshop will take place at IPGG, 6 Rue Jean Calvin, 75005 Paris. The location is close to both the Place Monge and the Censier-Daubenton subway stations on line7. it is also close to the Luxembourg station on the RER B line. The location is close to bus stops on the 21, 24, 27, 47, and 89 routes. Note that strikes are still ongoing, and some of these options may not be available.

We will be in the main amphitheater, downstairs on your right when you enter the building. Please register in advance on our meetup group so as to help us in the organization of the workshop.

A big thank you to Scaleway for hosting us in their inspiring office and sponsoring the networking event afterwards.

So this is quite exciting. Our meetup group has 7 999 members and we are going to organize a meetup in a town that is paralyzed by strikes. During the course of existence of this meetup, we have seen worse.

For those of you who will not be able to make it, all information slides and link to streaming are below:

Tabular data are the most common within companies. Generating synthetic data that respects the statistical properties of the original data can have several applications: a machine learning that respects data privacy, improving the robustness of a model in relation to data drift, etc. Since 2018, there has been an increasing number of academic publications presenting the use of GANs on this type of data, particularly on patient medical data. We have performed a proof of concept on real data, and present the results of several models from the research, namely the Wasserstein GAN, the Wasserstein GAN with Gradient Penalty and the Cramér-GAN, with the objective of "model compatibility", i.e. the possibility of using synthetic data to replace real data to train a classifier.

Its goal is to integrate predictions of train delays into the SNCF mobile application. Every day, our model predicts delays for the next 7 days, at each stop, for every train in Paris area network. The challenge of this project is to improve the reliability of passenger information and to provide more relevant routes for the application users. We will present the project, from the definition of needs and exploratory data analysis, to its industrialization in the cloud and the reliability of its predictions.

This talk is focussed on AI and ML applications in retail. Discover how Carrefour is transforming through the introduction of the Google - Carrefour Lab by Elina Ashkinazi-Ildis, Director of the Lab. Then go further with the "shelf out detection" usecase presented by Kasra Mansouri, Data Scientist within Artefact.

RAPIDS makes it possible to have end-to-end data science pipelines run entirely on GPU architecture. It capitalizes on the parallelization capabilities of GPUs to accelerate data preprocessing pipelines, with a pandas-like dataframe syntax. GPU-optimized versions of scikit-learn algorithms are available, and RAPIDS also integrates with major deep learning frameworks.
This talk will present RAPIDS and its capabilities, and how to integrate it in your pipelines.

Neural networks have a reputation for being better at solving statistical or approximate problems than at performing calculations or working with symbolic data. In this paper, we show that they can be **surprisingly good** at more elaborated tasks in mathematics, such as symbolic integration and solving differential equations. We propose a syntax for representing mathematical problems, and methods for generating large datasets that can be used to train sequence-to-sequence models. We achieve results that outperform commercial Computer Algebra Systems such as Matlab or Mathematica.https://arxiv.org/abs/1912.01412

Publicis Sapient is a digital transformation partner helping companies and established organizations get to their future, digitally-enabled state, both in the way they work and the way they serve their customers. Within Publicis Sapient, the Data Science Team builds machine learning products in order to support clients in their transformation.

A significant share of visitors on a site do not return, making it crucial to identify levers that can decrease bouncing rate. For a client in the retail sector, we developed several models that are able to predict both the gender and the segment in which unlogged and unknown visitors fit in. This allows to personalize the experience from the first visit and prevent users from bouncing.

Internet users leave multiple traces of micro-conversions (searches, clicks, whishlist...) during their visit on an ecommerce site: these micro-conversions can be weak signals of an act of purchase in the near future. To analyze those signals, we built a solution to detect visitors that are likely to convert and target them in while optimizing media campaigns budgets.

The fields of neuroscience and artificial intelligence (AI) have a long and intertwined history. In more recent times, however, communication and collaboration between the two fields has become less commonplace.In https://bit.ly/2WLsMaQ, the authors argue that better understanding biological brains could play a vital role in building intelligent machines. They survey historical interactions between the AI and neuroscience fields and emphasize current advances in AI that have been inspired by the study of neural computation in humans and other animals. Finally they conclude by highlighting shared themes that may be key for advancing future research in both fields.

The recent M4 forecasting competition (https://www.mcompetitions.unic.ac.cy) has demonstrated that the use of one forecasting method alone is not the most efficient approach in terms of forecasting accuracy. In this talk, I will focus on an energy consumption forecasting use case integrating exogenous data such as weather conditions and open data. In particular, I will present a forecasting time series challenge and the best practices observed on the best submissions and showcase an interesting approach based on a combination of classical statistical forecasting methods and machine learning algorithms, such as gradient boosting, for increased performance. Generalizing the use of these methods can be a major help to address the challenge of electricity demand and production adjustment.