Featured Research Projects

Kite: Building Conversational Bots from Mobile Apps

Task-oriented chatbots allow users to carry out tasks (e.g., ordering a pizza) using natural language conversation. The widely-used slot-filling approach for building bots of this type requires significant hand-coding, which hinders scalability. Kite is a practical system for bootstrapping task-oriented bots. Kite’s key insight is that while bots encapsulate the logic of user tasks into conversational forms, existing apps encapsulate the logic of user tasks into graphical user interfaces. A developer demonstrates a task using a relevant app, and from the collected interaction traces Kite automatically derives a task model, a graph of actions and associated inputs representing possible task execution paths. A task model represents the logical backbone of a bot, on which Kite layers a question-answer interface generated using a hybrid rule-based and neural network approach. Using Kite, developers can automatically generate bot templates for many different tasks. Read our MobiSys 2018 paper on Kite

SUGILITE is a new multi-modal, interactive, programming by demonstration (PBD) system that enables end users to add new capabilities to an intelligent assistant by programming automation scripts for tasks in any existing third-party Android mobile app using a combination of demonstrations and verbal instructions. SUGILITE leverages state-of-art machine learning and natural language processing techniques to comprehend the user’s verbal instructions that supply missing information in the demonstration, such as implicit conditions, user intents and personal preferences. The user’s demonstrations on the GUI are used for grounding the conversation and reinforcing the natural language understanding model. The system points the way to allowing the general public to more effectively use their smartphones, IoT devices and intelligent assistants, increasing the adoption, efficiency and correctness of uses of these technologies. The follow-up system EPIDOSITE extends SUGILITE to support programming for smart home devices. Read our CHI ’17 Paper about SUGILITE / Watch a SUGILITE demo video / Check out our GitHub Repository / Try out SUGILITE at Google Play

Atlasify is a novel information retrieval / interactive visualization system supporting exploratory search. As the Lead Student Researcher and Head Developer, I re-implemented the system with Leaflet on the front end and WikiBrain on the back end, enabling the system to dynamically compute the semantic relatedness for any given keywords and render the interactive map instantly. Since its beta release in June 2015, Atlasify has acquired thousands of active users and been featured on Wired, Phys.org and ACM Newsletter. Based on Atlasify, we are now conducting a variety of user behavior studies and investigating the HCI aspect of spatialization and spatial information retrieval system design. Read more about Atlasify or Try out the beta version of Atlasify

WikiBrain – A Java Library for Wikipedia-based Algorithms

WikiBrain is a Java-based library/framework wrote by Shilad Sen, Toby J Li, Brent Hecht and a group of undergraduate students at Macalester College. It democratizes access to a range of Wikipedia-based algorithms and technologies, enabling anyone with basic Java programming skills to utilize state-of-art semantic relatedness algorithms, page view data analysis and spatial queries in a few lines of code and easily analyze terabytes of Wikipedia data. After its debut at OpenSym/WikiSym ’14, WikiBrain has been used by many in both academia and industry. Learn more about WikiBrain / WikiBrain homepage