While robots are rapidly becoming more capable and ubiquitous, their utility is still severely limited by the inability of regular users to customize their behaviors. This EArly Grant for Exploratory Research (EAGER) will explore how examples of language, gaze, and other communications can be collected from a virtual interaction with a robot in order to learn how robots can interact better with end users. Current robots’ difficulty of use and inflexibility are major factors preventing them from being more broadly available to populations that might benefit, such as aging-in-place seniors. One promising solution is to let users control and teach robots with natural language, an intuitive and comfortable mechanism. This has led to active research in the area of grounded language acquisition: learning language that refers to and is informed by the physical world. Given the complexity of robotic systems, there is growing interest in approaches that take advantage of the latest in virtual reality technology, which can lower the barrier of entry to this research.

This EAGER project develops infrastructure that will lay the necessary groundwork for applying simulation-to-reality approaches to natural language interactions with robots. This project aims to bootstrap robots’ learning to understand language, using a combination of data collected in a high-fidelity virtual reality environment with simulated robots and real-world testing on physical robots. A person will interact with simulated robots in virtual reality, and his or her actions and language will be recorded. By integrating with existing robotics technology, this project will model the connection between the language people use and the robot’s perceptions and actions. Natural language descriptions of what is happening in simulation will be obtained and used to train a joint model of language and simulated percepts as a way to learn grounded language. The effectiveness of the framework and algorithms will be measured on automatic prediction/generation tasks and transferability of learned models to a real, physical robot. This work will serve as a proof of concept for the value of combining robotics simulation with human interaction, as well as providing interested researchers with resources to bootstrap their own work.

Dr. Matuszek’s Interactive Robotics and Language lab is developing robots that everyday people can talk to, telling them to do tasks or about the world around them. Their approach to learning to understand language in the physical space that people and robots occupy is called grounded language acquisition and is a key to building robots that can perform tasks in noisy, real-world environments, instead of being pre-emptively programmed to handle a fixed set of predetermined tasks.

Meet Your Professor: Dr. Frank Ferraro, 12-1 Mon 3/25, ITE 239

Prof. Ferraro’s teaching and research areas include AI, language understanding and machine learning

Meet Your Professor: Dr. Frank Ferraro

Find out about Machine Learning and NLP classes

2:00-1:00 Monday, 25 March 2019, ITE 239﻿

Next Monday, March 25th, join the Computer Science Education Club for its second installment of the “Meet Your Professor” Spring 2019 series featuring Professor Frank Ferraro. The series provides students with the opportunity to learn more about their professors, including how they achieved their position, what they believe makes an effective teacher, what research they conduct, and more.

Tim Finin named ACM fellow for contributions to knowledge sharing

UMBC’s Tim Finin, professor of computer science and electrical engineering (CSEE), has been named a fellow of the Association for Computing Machinery (ACM), a distinctive honor granted to less than one percent of all ACM members. ACM fellows are selected based on their work to advance computing over the course of a career, in areas such as mobile networks, computer architecture, robotics, and security. Finin was cited for his “contributions to the theory and practice of knowledge sharing in distributed systems and the World Wide Web.”

“It’s a great honor to be selected as an ACM fellow, since it is based on the recommendations of one’s peers and recognizes contributions to the field of computing,” says Finin. “I am especially honored since ACM fellows include so many pioneers of the field whose work and contributions I have studied and used over the past 40 years.”

“Dr. Finin has been a leader in our department ever since he came in as the chair in 1991,” says Anupam Joshi, professor and chair of CSEE. “He is one of our most accomplished researchers, and in addition to this fellowship, has been recognized both internally (as a Presidential Research Professor) and externally with numerous awards.”

Joshi continues, “Tim is a great teacher, and he has mentored a number of our mid-career and senior faculty, including me!”

Throughout his career, Finin has been involved with various aspects of ACM. As a graduate student at the University of Illinois Urbana-Champaign, Finin became a member of ACM’s special interest group on artificial intelligence (SIGART), which is one of his primary areas of focus. In the years since then, he has collaborated with numerous many UMBC faculty, students, and alumni, in addition to colleagues in industry and at other institutions, to move this work forward.

In the 1990s, and with support from the Defense Advanced Research Projects Agency (DARPA), Finin worked with UMBC faculty to develop new software standards to support the then-new concept of intelligent multiagent systems. The software, called the Knowledge Query and Manipulation Language, was used to develop intelligent applications and as the basis for faculty research and many Ph.D. dissertations.

The ACM’s Conference for Information and Knowledge Management awarded Finin and his collaborators with the 2018 Test of Time Award for a 1994 paper about this research that has continued to have an important impact on the research community.

Finin and a group of collaborators also worked on projects related building the semantic web. “The idea was to enhance the new web technologies with a way to enclose structured data that machines could use into ordinary web pages,” he explained. He adds that this allowed computers to understand the information on the web page without having to understand natural language.

Starting in the 2000s, Finin and his collaborators focused much of their work on blogs and then social media, including Facebook and Twitter. They explored how to analyze the data collected on these sites, and also how to protect and improve security and privacy features.

“I’ve only been able to do this because of the environment at UMBC,” Finin says, reflecting on the encouragement he has received to pursue new collaborations and areas of research.

“Based on my experience,” he shares, “I hope to mentor more faculty in the middle of their careers,” to help them access opportunities through organizations like ACM.

Finin currently oversees and mentors UMBC’s student chapter of ACM, which includes both undergraduate and graduate students. The student organization sponsors weekly talks and other events for people in the UMBC community who are interested in computing and related topics.

Finin joins Roy Rada, professor emeritus of information systems, who is also an ACM fellow.

“Her curriculum vitae stretches a mile, detailing her accomplishments in technology in the last 25 years. Her triumphs with NASA, National Institute of Science and Technology (NIST), National Science Foundation (NSF), the Center for Advanced Studies (CAS), IBM, her leadership in the recent launch of the Center for Accelerated Real-Time Analytics (CARTA) at UMBC, plus her personal pursuits in developing solutions in healthcare, and now the recent launch of her Blockchain company, are glimpses into the life of a leader intent on changing the world.”

Meet the Staff: Rebecca Dongarra

Name: Rebecca Dongarra

Educational Background: Bachelor of Arts in Biology from St. Mary’s College of Maryland, currently pursuing a master’s degree in Instructional Systems Development through UMBC

Hometown: West Friendship, Maryland

Current role: Academic Affairs Manager

Rebecca moved from the College of Natural and Mathematical Sciences as the Data and Events Coordinator to join CSEE. She was a team member on the STEM BUILD at UMBC Initiative grant supported by NIH from 2016 to 2018. Prior to working in higher education, Rebecca established herself as a small business owner and local community leader. When not working, Rebecca enjoys hiking, volunteering with YMCA and USA swim teams and keeping busy with gardening.

Prof. Cynthia Matuszek named one of AI’s 10 to Watch

Cynthia Matuszek named one of AI’s 10 to Watch

UMBC CSEE Professor Cynthia Matuszek was named as one AI’s 10 to Watch by IEEE Intelligent Systems. The designation is given every two years to a group of “10 young stars who have demonstrated outstanding AI achievements”. IEEE Intelligent Systems accepts nominations from around the world, which are then evaluated by the the publication’s editorial and advisory boards based on reputation, impact, expert endorsement, and diversity. Dr. Matuszek was recognized for her research that “combined robotics, natural language processing, and machine learning to build systems that nonspecialists can instruct, control, and interact with intuitively and naturally”.

Professor Matuszek joined UMBC in 2014 after receiving her Ph.D. in Computer Science from the University of Washington. At UMBC, she established and leads the Interactive Robotics and Language Lab that integrates research on robotics and natural language processing with the goal of “bringing the fields together: developing robots that everyday people can talk to, telling them to do tasks or about the world around them”.

Here is how she describes her research in the IEEE Intelligent Systems article.

Robot Learning from Language and Context

As robots become more powerful, capable, and autonomous, they are moving from controlled industrial settings to human-centric spaces such as medical environments, workplaces, and homes. As physical agents, they will soon be able help with entirely new categories of tasks that require intelligence. Before that can happen, though, robots must be able to interact gracefully with people and the noisy, unpredictable world they occupy.

This undertaking requires insight from multiple areas of AI. Useful robots will need to be flexible in dynamic environments with evolving tasks, meaning they must learn and must also be able to communicate effectively with people. Building advanced intelligent agents that interact robustly with nonspecialists in various domains requires insights from robotics, machine learning, and natural language processing.

My research focuses on developing statistical learning approaches that let robots gain knowledge about the world from multimodal interactions with users, while simultaneously learning to understand the language surrounding novel objects and tasks. Rather than considering these problems separately, we can efficiently handle them concurrently by employing joint learning models that treat language, perception, and task understanding as strongly associated training inputs. This lets each of these channels provide mutually reinforcing inductive bias, constraining an otherwise unmanageable search space and allowing robots to learn from a reasonable number of ongoing interactions.

Combining natural language processing and robotic understanding of environments improves the efficiency and efficacy of both approaches. Intuitively, learning language is easier in the physical context of the world it describes. And robots are more useful and helpful if people can talk naturally to them and teach them about the world. We’ve used this insight to demonstrate that robots can learn unanticipated language that describes completely novel objects. They can also learn to follow instructions for performing tasks and interpret unscripted human gestures, all from interactions with nonspecialist users.

Bringing together these disparate research areas enables the creation of learning methods that let robots use language to learn, adapt, and follow instructions. Understanding humans’ needs and communications is a long-standing AI problem, which fits within the larger context of understanding how to interact gracefully in primarily human environments. Incorporating these capabilities will let us develop flexible, inexpensive robots that can integrate into real-world settings such as the workplace and home.

​You can access a pdf version of the full IEEE AI’s 10 to Watch article here.

Prof. Milton Halem receives UMBC Research Faculty Excellence Award

Dr. Milton Halem, Research Professor in the Computer Science and Electrical Engineering Department, has been selected as the inaugural recipient of the UMBC Research Faculty Excellence Award. The award recognizes overall excellence in research, and where appropriate, significant contributions to teaching and service/leadership while at UMBC.

Dr. Halem joined UMBC in 2003, after retiring from a highly successful career at NASA Goddard Space Flight Center, where he still holds an Emeritus position as Chief Information Research Scientist to the Director of the Earth Sciences Directorate.

Upon joining UMBC, Dr. Halem served as the founding Director of UMBC’s Center for Hybrid Multicore Productivity Research (CHMPR) and today continues to serve as the UMBC Site Director for this major NSF-supported multi-institutional center that works closely with government and private sector sponsors in the areas of big-data computation, next generation computing and software tool development.

In 2013, Dr. Halem was instrumental in negotiating and securing a major equipment donation from NASA Goddard that significantly enhanced our high-performance computing capacity through the donation of a 512 -node supercomputer to the UMBC campus.

Dr. Halem’s scholarly achievements include more than 150 scientific publications in the areas of atmospheric and oceanographic sciences and computational and information sciences. He is most noted for his groundbreaking research in simulation studies of space-observing systems and for development of four-dimensional data assimilation for weather and climate prediction.

Over the years, Dr. Halem’s achievements have earned him numerous awards including the NASA Medal for Exceptional Scientific Achievement, the NASA Medal for Outstanding Leadership, and NASA’s highest award – the NASA Distinguished Service Medal – in 1996.

Keith J. Bowman, Dean of the College of Engineering and Information Technology, comments: “Dr. Halem’s exceptional vision and his unrelenting drive for excellence continue to serve our UMBC community well. His dedication to pushing the scientific and engineering boundaries serves as a model for our campus and beyond.”

Karl V. Steiner, Vice President for Research, adds: “ I could not think of a more deserving member of our UMBC research community to receive this inaugural Research Faculty Excellence Award than Milt Halem. UMBC is the academic home to over 180 Research Faculty who contribute their expertise and personal commitment to making UMBC a destination for cutting-edge research while providing our students with remarkable insights and opportunities. Milt Halem is clearly one the leaders in the field of high-performance computation and his energy and expertise have been a major factor in UMBC being recognized as a major contributor to high performance computation and data analytics.”

Marie desJardins, associate dean of the College of Engineering and Information Technology (COEIT) and professor of computer science and electrical engineering, will be leaving UMBC to take up a new position as founding dean of the College of Organizational, Computational, and Information Sciences at Simmons College in Boston.

“What I will remember most about my 17 years here is UMBC’s collaborative spirit. Because of the open environment and commitment to diversity, I’ve been able to work with colleagues across the university on a wide range of initiatives,” desJardins says.

During her tenure at UMBC, desJardins has applied her passion and expertise to implementing programs for students across all disciplines and majors, explains Keith J. Bowman, dean of the College of Engineering and Information Technology (COEIT). “She brought her passion and expertise to UMBC, and has changed the lives of faculty, students, and staff through her work,” Bowman says. “As COEIT’s founding associate dean, she has played a crucial role in establishing how the College operates, with a focus on supporting students at all levels. She has set an incredibly high bar in all areas of her work.”

One of desJardins’ many accomplishments was the development and launch of UMBC’s Grand Challenge Scholars Program, based on the National Academy of Engineering’s (NAE) Grand Challenges for Engineering. The program is open to students who are interested in working on interdisciplinary teams to address pressing challenges facing society. UMBC’s program is distinct because it is open to all majors, bringing together students studying everything from computing and mechanical engineering to the life sciences, social sciences, humanities, and the arts. The Grand Challenge Scholars Program is “a great match with so many things that UMBC and UMBC students are already doing: applied, project-based learning; service learning; entrepreneurial explorations; global involvement; and undergraduate research,” desJardins said of the program when it launched in 2016.

Marie desJardins during an experiential learning activity.

desJardins also reached students across the university through her work with the Honors Colleges, as an Honors Faculty Fellow. This role enabled her to teach a seminar called “Computation, Complexity, and Emergence,” where students from a range of majors shared their perspectives on interdisciplinary topics and learned how subjects they had not previously explored were relevant to their lives. desJardins also served as a chair of the Honors College Advisory Board during her tenure at UMBC.

Beyond her passion for expanding computer science education at UMBC, desJardins has also been steadfast in her work to increase access to computing education for K – 12 students. She has served as the lead principal investigator of CE21-Maryland, a series of projects implemented to increase opportunities for high school students to access computer science education. She was also instrumental in the creation of How Girls Code, an afterschool program and a summer camp at UMBC where girls in elementary and middle school develop computer science skills through engaging activities and learn about careers in the field.

In addition to her writing for academic and technical audiences, desJardins has written numerous articles for the public, including pieces for The Conversation and The Baltimore Sun about the need for computing education for students of all ages. She is particularly passionate about engaging girls and women in computer science.

In a recent op-ed in The Baltimore Sun, desJardins discussed the importance of computer science education in K – 12 schools, both to expand career opportunities for students of all backgrounds and identities and to make sure the world has a chance to benefit from a diverse talent pool in computing fields. “The need for computer science and computational thinking skills is becoming pervasive not just in the world of software engineers, but in fields as varied as science, design, marketing, and public policy,” she wrote.

Marie desJardins, standing at right, addresses high school teachers at the July 2015 CS Matters in Maryland pilot teacher workshop.

desJardins has also worked to support new faculty in her College, as they work to advance their careers, inviting assistant professors and lecturers in COEIT to participate in the Junior Faculty Initiative. The program introduces participants to university resources through units like the Faculty Development Center and Office of Student Disabilities Services. It also supports junior faculty through a series of workshops addressing topics such as time management, mentoring relationships, and conflict management, to acclimate faculty to UMBC.

Across the nation and the world, desJardins has been recognized as a leader in the field of artificial intelligence (AI). Earlier this year, she was named a fellow of the Association for the Advancement of Artificial Intelligence. In 2017, she was included on Forbes’ list of women advancing AI research. UC Berkeley, desJardins’ alma mater, also recently recognized her work to advance her field by presenting her with the Distinguished Alumni Award in Computer Science.

“UMBC has given me so many opportunities to learn, grow, and give back to the community around me, I will be forever grateful,” says desJardins. “No matter where I go from here, I will always consider myself to be part of the UMBC community.”

Adapted from a UMBC News story by Megan Hanks. All photos by Marlayna Demond ’11 for UMBC.

Global Impact, Promise & Perils of Blockchain

UMBC Professors Haibin Zhang (Computer Science and Electrical Engineering) and Karuna Joshi (Information Systems) will be panelists for an event focused on the impact, promise and perils of blockchain technologies at the bwtech@UMBC Research & Technology Park. The event will take place from 8:15 to 11:15am on Tuesday, March 20 2018 at and will include breakfast and time for networking. They will be joined on the panel by Marcus Edwards, from Northrop Grumman’s Cyber & Intelligence Mission Solutions, Steve Cook of Verizon Enterprise Solutions, Sean Manion of Science Distributed and moderator Razvan Miutescu of Whiteford, Taylor & Preston, LLP. For more information and tickets are available on eventbrite.

8:15 – 11:15 Tuesday, March 20, 2018

Cryptocurrency market capitalizations have soared over the past year and new innovative blockchain applications are continuing to emerge seemingly by the day. However, the average end-user is left to singularly make sense of a vast and global marketplace that is rapidly converging the core tenets of economics and technology development. The future prospects of traditional business and financial models is uncertain as blockchain technology leaves key decision makers in an untenable position to either adopt and adapt or simply be left behind. Industry and academic experts in the field will discuss the pros and cons and opportunities and challenges of this disruptive technology movement.

Robots acting in human-scale environments must plan under uncertainty in large state–action spaces and face constantly changing reward functions as requirements and goals change. We introduce a new hierarchical planning framework called Abstract Markov Decision Processes (AMDPs) that can plan in a fraction of the time needed for complex decision making in ordinary MDPs. AMDPs provide abstract states, actions, and transition dynamics in multiple layers above a base-level “flat” MDP. AMDPs decompose problems into a series of subtasks with both local reward and local transition functions used to create policies for subtasks. The resulting hierarchical planning method is independently optimal at each level of abstraction, and is recursively optimal when the local reward and transition functions are correct.

I will present empirical results in several domains showing significantly improved planning speed, while maintaining solution quality. I will also discuss related work within the same project on automated option discovery, abstraction construction, language learning, and initial steps towards automated methods for learning AMDPs from base MDPs, from teacher demonstrations, and from direct observations in the domain.

This work is collaborative research with Dr. Michael Littman and Dr. Stefanie Tellex of Brown University. Dr. James MacGlashan of SIFT and Dr. Smaranda Muresan of Columbia University collaborated on earlier stages of the project. The following UMBC students have also contributed to the project: Khalil Anderson, Tadewos Bellete, Michael Bishoff, Rose Carignan, Nick Haltemeyer, Nathaniel Lam, Matthew Landen, Keith McNamara, Stephanie Milani, Shane Parr (UMass), Shawn Squire, Tenji Tembo, Nicholay Topin, Puja Trivedi, and John Winder.

Dr. Marie desJardins is a Professor of Computer Science and the Associate Dean for Academic Affairs in the College of Engineering and Information Technology at the University of Maryland, Baltimore County. Prior to joining the faculty at UMBC in 2001, she was a Senior Computer Scientist in the AI Center at SRI International. Her research is in artificial intelligence, focusing on the areas of machine learning, multi-agent systems, planning, interactive AI techniques, information management, reasoning with uncertainty, and decision theory. She is active in the computer science education community, founded the Maryland Center for Computing Education, and leads the CS Matters in Maryland project to develop curriculum and train high school teachers to teach AP CS Principles.

Dr. desJardins has published over 125 scientific papers in journals, conferences, and workshops. She will be the IJCAI-20 Conference Chair, and has been an Associate Editor of the Journal of Artificial Intelligence Research and the Journal of Autonomous Agents and Multi-Agent Systems, a member of the editorial board of AI Magazine, and Program Co-chair for AAAI-13. She has previously served as AAAI Liaison to the Board of Directors of the Computing Research Association, Vice-Chair of ACM’s SIGART, and AAAI Councillor. She is a AAAI Fellow, an ACM Distinguished Member, a Member-at-Large for Section T (Information, Computing, and Communication) of the American Association for the Advancement of Science, the 2014-17 UMBC Presidential Teaching Professor, a member and former chair of UMBC’s Honors College Advisory Board, former chair of UMBC’s Faculty Affairs Committee, and a member of the advisory board of UMBC’s Center for Women in Technology.