Category: Staff profiles and activity

Dynamic new interactive technology which visualises the 3D structures inside DNA has been launched by a team of computational artists, game developers and scientists, working together to help the public better understand the cause of diseases.

CSynth is a software platform created by researchers at Goldsmiths, University of London and Oxford University. Described by its designers as ‘bio-visualisation made interactive’, it shows how cell machinery physically interacts with a structure as complex and compact as the genome.

Viewers can watch and explore the 3D models on a screen, or use a Virtual Reality headset to immerse themselves in genetic material and manipulate it themselves.

Traditionally, scientists have only been able to visualise and understand the genome – the complete set of genetic material present in a cell – in 2D presentations, on a screen or through graphs or histograms.

But as researchers gather more data about how cells work it is clear that a 3D structure is extremely important for gene regulation and how cells differentiate. For example, a white blood cell looks and behaves differently to a red blood cell even though its genome is exactly the same.

Subtle differences in the way the genome is folded can impact on whether genes can be switched on and off at particular times, which then dictates what a cell can do. Changes in the way chromatin is folded can cause rare blood diseases, for example, because it impacts on how genetic code is read by a cell.

Understanding this process is vital for seeking the cause of diseases such as diabetes or anaemia, and for the development of treatments for them.

Thanks to advances in genetic techniques, researchers are able to harness more information than ever before from biological data provided by patients and volunteers.

The CSynth software then integrates data from genome sequencing and computer modelling and presents it in an attractive and engaging way, using computer game technology.

The team have launched a complete software package that will also allow the import of public data, and help both the public and medical researchers gain a better understanding of how the genome is folded in a cell, and the complex mechanisms involved.

Professor Frederic Fol Leymarie and Professor William Latham from the Department of Computing at Goldsmiths are the computer artists and software designers behind CSynth, working with Steve Taylor, Head of Analysis, Visualisation and Informatics at the WIMM Centre of Computational Biology, and Professor Jim Hughes at the MRC Weatherall Institute of Molecular Medicine, University of Oxford. They are joined by Professor Stephen Todd, lead software architect at London Geometry Ltd and Visiting Professor in Computing at Goldsmiths, and Peter Todd, senior developer, London Geometry Ltd.

Steve Taylor said: “We have made a web-based interface where any researcher can load in the data from their experiments. Previously the software had to be installed and all the parameters were adjusted in text files by us behind the scenes. Now you can upload or drag and drop the data into a web page, and it will build a model allowing investigators to really get a handle on their data. You also get a fantastic user interface to interact with the model and overlay other data, such as genes and enhancers. We get asked a lot about making CSynth available for teaching and and now we can do this easily.”

Professor Fol Leymarie said: “Our body is made of trillions of cells, each one containing chromatin tightly folded. This very long molecular strand is not static, but rather keeps moving, vibrating, unfolding and refolding locally, more like a molecular dance.

“Furthermore, it keeps interacting with other molecular structures present in the cell and with itself. It is this dynamic nature that CSynth makes visible and interactive, so that a user – a researcher, student or even a curious member of the public – can load different data sequences, try out various parameters, compare various situations, to eventually get a much better, intuitive understanding, which we hope may help lead to new discoveries.”

VR is also popular with our undergraduate students. Last term our third years did our module 3D Virtual Environments and Animation, which includes VR development. They were the first students to use our newly refurbished VR lab which includes Oculus Rifts, HTC VIVEs, Oculus Go and Acer Windows Mixed Reality. At the end of term, they presented their projects for the module. There was a really wide range of fantastic VR experiences, from fairground games, to virtual bar-tending and therapy for fear of heights to virtual photography.

Dr Marco Gillies, Reader in Computing at Goldsmiths, gave a talk at the Virtual and Augmented Reality to Enhance Learning and Teaching in Higher Education Conference, that explained how virtual reality, and particularly interactive virtual characters, could enable us to learn the kinds of social skills that we need for work. These are the kinds of skills that are so hard to learn in a traditional way, anything from a doctor breaking bad news to a patient to a police officer interviewing a suspect. Professional social skills of this type are very different from our ordinary social lives, and handling them well can only be learned through experience.

Together with Dr Sylvia Pan, Lecturer in Computing at Goldsmiths, Marco has over 20 years experience of developing animated interactive characters that have realistic body language. Encountering a life-sized virtual human in immersive VR is a really powerful experience because body language cues like eye contact or personal space feel very realistic in VR. Making eye contact on a traditional computer just means looking out of the screen, but in VR it feels like eye contact in real life. That means that a VR conversation feels real. We can use this type of realistic social interaction to help train people to be better at their professional social skills.

The conference as a whole included many examples of how VR could be used to teach skills that you can only learn by experience. VR is much cheaper and safer than doing things for real, but much closer to real life than a book or video. In the next few years we will probably see a revolution in immersive media for education, and no where is this going to be more beneficial than in one of the most important skills we need in life: how to interact with other people.

Delivered by online learning platform Coursera, the MOOCs are designed to prepare students for the world-first BSc in Computer Science which has been designed by Goldsmiths and is being offered by the University of London.

The Introduction to Computer Programming MOOC gives participants foundation skills to write computer programs in programming language, as well as learning to create 2D and interactive graphics.

Lead instructor Dr Simon Katan, Lecturer in Computing, said: “In Introduction to Computer Programming, learners will be approaching the fundamentals of code through practical and creative exercises, and also explore how coders think and feel. We’ve drawn on our many years of teaching experience to deliver some cutting-edge pedagogy including our code adventure game Sleuth.”

The How Computers Work MOOC is designed for learners who are proficient with computers, smartphones and the internet but wish to improve their understanding of how they work, or go on to study computer science.

On this MOOC, learners can acquire key computer skills that can be applied to word processing applications, e-commerce, the internet and websites.

Instructor and Senior Lecturer in Computing Dr Marco Gillies said: “I’m really excited to be working with the University of London and Coursera to create a new way of learning computer science for the 21st Century.

“We’re bringing together the best learning technologies and the best teaching techniques to create a fantastic computer science learning experience for anyone, anywhere in the world.”

He added: “How Computers Work will introduce you to some fundamental computer science concepts and you’ll find out how they apply to the kind of computer applications you use every day. It’s a great foundation for starting to study computer science, but it will also give you a better understanding of the technologies that are so important to modern life.”

Sam Brenton, Director of Educational Innovation and Development for the University of London’s distance and flexible learning programmes, said: “This is an exciting time for the University as it prepares to launch one of its most innovative programmes, the BSc Computer Science.

“We anticipate a very broad appeal for this degree programme from students all over the world; not just those working in the technical field but also those from other industries.”

Head of Creative Computing, Simon Katan writes about the premier of his work ‘Clamour’ at the Roundhouse in Camden

Last Thursday my work Clamour premiered at the Roundhouse Camden’s Sackler Space. The work is an interactive mixed media theatre performance for live coder which is experienced simultaneously through audience smartphones, projection and sound. The aim is to wryly and reflexively interrogate how technology and social media mediate and influence our knowledge, relationships, and identities.

The piece tells the story of Sealand – a lone outpost surrounded by vast swathes of ocean – the sole surviving nation of global catastrophe. Now faced with an existential fight for its future, its people must forge the tools they require. New rules must be written, paving the way for a new state of being. With their resourcefulness and through working together what could go wrong? Throughout the performance, an inscrutable figure on stage (that’s me) controls all from their laptop – shaping divergent audience experiences with heuristic games that charm, frustrate and deceive. Yet it is the audience themselves, through gameplay with image, sound, and text, who determine the shape of the final performance.

I’ve been working on elements of the piece through various commissions over the last five years, and for this final stage I collaborated with digital artist Luke Fraser to bring everything together. Our development process involved numerous user testing sessions with Goldsmiths computing students to fine tune our interaction.

After a nerve wracking 15 minutes of dealing with the usual Wifi difficulties, the performance got underway. In such a performance it’s difficult to gauge reactions but a sprinkling of chuckles throughout gave me a good indication that things were progressing to plan. Read a review here http://www.savageonline.co.uk/our-journal/clamour/

A programme of regional touring is planned for Spring and Summer of 2019. You can find out more at clamour.org.uk.

Musicians will be able to use Artificial Intelligence to create new music and sound to share or sell, thanks to a project led by Goldsmiths.

At a time when many in the music industry worry their livelihoods are under threat from new technology, the MIMIC (Musically Intelligent Machines Interacting Creatively) project puts humans back in control of making music.

MIMIC will develop free, user-friendly web tools that harness the power of AI to listen to existing recordings and come up with new sounds and instruments interactively. Artists will own the sounds they create and can incorporate them into their music or sell them to others. The tools will meld the latest ‘deep learning’ AI methods with people’s creativity to empower a new generation of ‘cyborg’ musicians.

The £1m project is a collaboration between Goldsmiths, the University of Durham, the University of Sussex, and Google Magenta and has been funded by the UK’s Arts and Humanities Research Council (AHRC).

Mick Grierson, Professor of Computing at Goldsmiths and MIMIC project leader said: “In the past, to use these powerful Artificial Intelligence technologies you had to be an expert in programming: we want to make these technologies free and easy for anyone to use – from amateur music-makers and sonic experimenters to professional musicians.

“Rather than simply creating autonomous musical ‘robots’, we are harnessing Artificial Intelligence systems to augment human creativity. We’re inviting people to meld their musical talents and sonic curiosity to the very latest deep learning systems. Our interfaces will mean you don’t have to already know how to code to benefit from AI, you just have to want to make some noise. However, if you do want to code, you’ll be able to do so using a new language we will be creating specifically for making AI music systems.”

The MIMIC team aim to upload the first prototype web tools for people to experiment with within the next year. The tools will use a browser-based simplified live coding language written on top of JavaScript specially designed for musicians and artists. As well as working with the music industry, the team plan to produce learning materials for university, secondary school, and professional learners introducing them to how they can enhance their creativity with AI systems.

This post, written by Pete Wilton, was originally published on Goldsmiths News

The influence of ‘forgotten’ scientific papers has been demonstrated in a new study led by a researcher from Goldsmiths, University of London.

A team from Goldsmiths, the University of Chicago, Google, the University of Maryland, and Columbia University, developed a model that tracks ‘discursive influence’, or recurring words and phrases in historical texts that measure how scholars actually talk about a field, instead of just their attributions. To determine a particular scientific paper’s influence, the researchers can statistically remove it from history and see how scientific discourse would have unfolded without it.

Aaron Gerow, Lecturer in Computing at Goldsmiths, who led t
he study said: “Citations are one kind of impact, and discursive influence is a different kind. Neither one is the complete story, but they work together to give a better picture of what’s influencing science.”

The researchers report in the journal PNAS how they trained the model on massive text collections from computational linguistics, physics, and across science and scholarship (JSTOR) and then traced distinct patterns of influence. They found that scientists who persistently published in a single field were more likely to be ‘canonised’ in a way that compelled others to cite them disproportionate to their papers’ discursive contributions. On the other hand, discoveries that crossed disciplinary boundaries were more likely to have outsized discursive impact but fewer citations, likely because the ‘owner’ of the idea and her allies remain socially and institutionally distant from the citing author.

The model also sheds light on so-called ‘sleeping beauties’: papers that went relatively unacknowledged for years or even decades before experiencing a late burst of citations. For example, a 1947 paper on graphene remained obscure and forgotten until the 1990s with a resurgence of research interest in the material and an eventual Nobel Prize.

Study co-author James Evans, director of Knowledge Lab and professor of sociology at the University of Chicago, said: “Papers have a news cycle, when lots of people chat about them and cite them, and then they’re no longer new news. Our model shows that some papers have much more influence than citations will typically demonstrate, such as these ‘sleeping beauties,’ which didn’t have much influence early but come to be appreciated and important later.”

The study used a computational method known as ‘topic modeling’ that was invented by co-author David Blei of Columbia University. The authors said the same model can also be used to trace influence in other areas, such as literature and music. Text from poems or song lyrics, and even extra-textual characteristics such as stanza structure or chord progressions, could feed into the model to find under-credited influencers and map the spread of new concepts and innovations.