SummaryDeep learning is revolutionizing the field of Natural Language Processing (NLP), with breakthroughs in machine translation, speech recognition, and question answering. New language interfaces (digital assistants, messenger apps, customer service bots) are emerging as the next technologies for seamless, multilingual communication among humans and machines.
From a machine learning perspective, many problems in NLP can be characterized as structured prediction: they involve predicting structurally rich and interdependent output variables. In spite of this, current neural NLP systems ignore the structural complexity of human language, relying on simplistic and error-prone greedy search procedures. This leads to serious mistakes in machine translation, such as words being dropped or named entities mistranslated. More broadly, neural networks are missing the key structural mechanisms for solving complex real-world tasks requiring deep reasoning.
This project attacks these fundamental problems by bringing together deep learning and structured prediction, with a highly disruptive and cross-disciplinary approach. First, I will endow neural networks with a "planning mechanism" to guide structural search, letting decoders learn the optimal order by which they should operate. This makes a bridge with reinforcement learning and combinatorial optimization. Second, I will develop new ways of automatically inducing latent structure inside the network, making it more expressive, scalable and interpretable. Synergies with probabilistic inference and sparse modeling techniques will be exploited. To complement these two innovations, I will investigate new ways of incorporating weak supervision to reduce the need for labeled data.
Three highly challenging applications will serve as testbeds: machine translation, quality estimation, and dependency parsing. To maximize technological impact, a collaboration is planned with a start-up company in the crowd-sourcing translation industry.

Deep learning is revolutionizing the field of Natural Language Processing (NLP), with breakthroughs in machine translation, speech recognition, and question answering. New language interfaces (digital assistants, messenger apps, customer service bots) are emerging as the next technologies for seamless, multilingual communication among humans and machines.
From a machine learning perspective, many problems in NLP can be characterized as structured prediction: they involve predicting structurally rich and interdependent output variables. In spite of this, current neural NLP systems ignore the structural complexity of human language, relying on simplistic and error-prone greedy search procedures. This leads to serious mistakes in machine translation, such as words being dropped or named entities mistranslated. More broadly, neural networks are missing the key structural mechanisms for solving complex real-world tasks requiring deep reasoning.
This project attacks these fundamental problems by bringing together deep learning and structured prediction, with a highly disruptive and cross-disciplinary approach. First, I will endow neural networks with a "planning mechanism" to guide structural search, letting decoders learn the optimal order by which they should operate. This makes a bridge with reinforcement learning and combinatorial optimization. Second, I will develop new ways of automatically inducing latent structure inside the network, making it more expressive, scalable and interpretable. Synergies with probabilistic inference and sparse modeling techniques will be exploited. To complement these two innovations, I will investigate new ways of incorporating weak supervision to reduce the need for labeled data.
Three highly challenging applications will serve as testbeds: machine translation, quality estimation, and dependency parsing. To maximize technological impact, a collaboration is planned with a start-up company in the crowd-sourcing translation industry.

Max ERC Funding

1 436 000 €

Duration

Start date: 2018-02-01, End date: 2023-01-31

Project acronymDEPENDABLECLOUD

ProjectTowards the dependable cloud:
Building the foundations for tomorrow's dependable cloud computing

SummaryCloud computing is being increasingly adopted by individuals, organizations, and governments. However, as the computations that are offloaded to the cloud expand to societal-critical services, the dependability requirements of cloud services become much higher, and we need to ensure that the infrastructure that supports these services is ready to meet these requirements. In particular, this proposal tackles the challenges that arise from two distinctive characteristic of the cloud infrastructure.
The first is that non-crash faults, despite being considered highly unlikely by the designers of traditional systems, become commonplace at the scale and complexity of the cloud infrastructure. We argue that the current ad-hoc methods for handling these faults are insufficient, and that the only principled approach of assuming Byzantine faults is too pessimistic. Therefore, we call for a new systematic approach to tolerating non-crash, non-adversarial faults. This requires the definition of a new fault model, and the construction of a series of building blocks and key protocol elements that enable the construction of fault-tolerant cloud services.
The second issue is that to meet their scalability requirements, cloud services spread their state across multiple data centers, and direct users to the closest one. This raises the issue that not all operations can be executed optimistically, without being aware of concurrent operations over the same data, and thus multiple levels of consistency must coexist. However, this puts the onus of reasoning about which behaviors are allowed under such a hybrid consistency model on the programmer of the service. We propose a systematic solution to this problem, which includes a novel consistency model that allows for developing highly scalable services that are fast when possible and consistent when necessary, and a labeling methodology to guide the programmer in deciding which operations can run at each consistency level.

Cloud computing is being increasingly adopted by individuals, organizations, and governments. However, as the computations that are offloaded to the cloud expand to societal-critical services, the dependability requirements of cloud services become much higher, and we need to ensure that the infrastructure that supports these services is ready to meet these requirements. In particular, this proposal tackles the challenges that arise from two distinctive characteristic of the cloud infrastructure.
The first is that non-crash faults, despite being considered highly unlikely by the designers of traditional systems, become commonplace at the scale and complexity of the cloud infrastructure. We argue that the current ad-hoc methods for handling these faults are insufficient, and that the only principled approach of assuming Byzantine faults is too pessimistic. Therefore, we call for a new systematic approach to tolerating non-crash, non-adversarial faults. This requires the definition of a new fault model, and the construction of a series of building blocks and key protocol elements that enable the construction of fault-tolerant cloud services.
The second issue is that to meet their scalability requirements, cloud services spread their state across multiple data centers, and direct users to the closest one. This raises the issue that not all operations can be executed optimistically, without being aware of concurrent operations over the same data, and thus multiple levels of consistency must coexist. However, this puts the onus of reasoning about which behaviors are allowed under such a hybrid consistency model on the programmer of the service. We propose a systematic solution to this problem, which includes a novel consistency model that allows for developing highly scalable services that are fast when possible and consistent when necessary, and a labeling methodology to guide the programmer in deciding which operations can run at each consistency level.

SummarySeveral human pancreatic diseases have been characterized, being the diabetes the most common. Like others, this genetic disease is related to disrupted non-coding cis-regulatory elements (CREs) that culminate in altered gene expression. Although Genome Wide Association Studies support this hypothesis, it’s still unclear how mutations on CREs contribute to disease. The translation from the “non-coding code” to phenotype is an exciting and unexplored field that we will approach in this project with the help of the zebrafish as a suitable animal model. We aim to uncover the implications of the disruption of pancreas CREs and how they contribute to diabetes in vivo. For this we will study transcriptional regulation of genes in zebrafish. The similarities between zebrafish and mammal pancreas and the evolutionary conservation of pancreas transcription factors (TF) make it an excellent model to approach and study this disease. In this project we will characterize the zebrafish insulin producing beta-cell regulome, by determining the active CREs in this cell type and their bound TFs. Then we will compare this information with a similar dataset recently available for human beta-cells, to define functional orthologs in these species. Selected CREs will be tested by in vivo gene reporter assays in zebrafish, focusing on those functionally equivalent to human CREs where risk alleles have been associated with diabetes or those regulating genes involved in diabetes. Later these CREs will be mutated in the zebrafish genome to validate their contribution to diabetes. Finally we will translate this to predict new human disease-associated CREs by focusing on the regulatory landscape of diabetes-associated genes, without the need of having countless patients to uncover them. With this project we will create a model system that will allow the identification of new diabetes-associated CREs, which might have a great impact in clinical management of this epidemic disease.

Several human pancreatic diseases have been characterized, being the diabetes the most common. Like others, this genetic disease is related to disrupted non-coding cis-regulatory elements (CREs) that culminate in altered gene expression. Although Genome Wide Association Studies support this hypothesis, it’s still unclear how mutations on CREs contribute to disease. The translation from the “non-coding code” to phenotype is an exciting and unexplored field that we will approach in this project with the help of the zebrafish as a suitable animal model. We aim to uncover the implications of the disruption of pancreas CREs and how they contribute to diabetes in vivo. For this we will study transcriptional regulation of genes in zebrafish. The similarities between zebrafish and mammal pancreas and the evolutionary conservation of pancreas transcription factors (TF) make it an excellent model to approach and study this disease. In this project we will characterize the zebrafish insulin producing beta-cell regulome, by determining the active CREs in this cell type and their bound TFs. Then we will compare this information with a similar dataset recently available for human beta-cells, to define functional orthologs in these species. Selected CREs will be tested by in vivo gene reporter assays in zebrafish, focusing on those functionally equivalent to human CREs where risk alleles have been associated with diabetes or those regulating genes involved in diabetes. Later these CREs will be mutated in the zebrafish genome to validate their contribution to diabetes. Finally we will translate this to predict new human disease-associated CREs by focusing on the regulatory landscape of diabetes-associated genes, without the need of having countless patients to uncover them. With this project we will create a model system that will allow the identification of new diabetes-associated CREs, which might have a great impact in clinical management of this epidemic disease.