Audio-visual recognition (AVR) has been considered as a solution for speech recognition tasks when the audio is corrupted, as well as a visual recognition method used for speaker verification in multispeaker scenarios. The approach of AVR systems is to leverage the extracted information from one modality to improve the recognition ability of the other modality by complementing the missing information. The essential problem is to find the correspondence between the audio and visual streams, which is the goal of this paper. We propose the use of a coupled 3D convolutional neural network (3D CNN) architecture that can map both modalities into a representation space to evaluate the correspondence of audio-visual streams using the learned multimodal features. The proposed architecture will incorporate both spatial and temporal information jointly to effectively find the correlation between temporal information for different modalities. By using a relatively small network architecture and much smaller dataset for training, our proposed method surpasses the performance of the existing similar methods for audio-visual matching, which use 3D CNNs for feature representation. We also demonstrate that an effective pair selection method can significantly increase the performance. The proposed method achieves relative improvements over 20% on the equal error rate and over 7% on the average precision in comparison to the state-of-the-art method.

Sketch/Line Art colorization is a time consuming process, The automation of the process requires not just simple boundary detection but also semantical feature identification, user interaction and shading(which is not a problem for grey image colorization). We propose an deep end-to-end trainable colorization model that meanwhile small in size and has a almost-real-time performance. It identifies semantic features from the sketch and inpaint it with/without user interaction with realistic shading.

For decades, institutional data on radio telescope arrays have accumulated in American astronomical research institutions.

Radio array telescopes like SETI at Home and the NRAO Very Large Array have over time, amassed big, not just large, big data sets of astronomical data concerning the cosmos. Data points such as polarization, radio wave frequency, magnitude, longitude/latitude, and date/time help astronomers pinpoint and understand stars.

We seek to use TensorFlow integrated into Intel DevCloud to build a TensorFlow Machine Learning model which better interprets and analyzes radio telescope array big data and also to utilize machine vision to further enhance radio images created by astronomical big data.