Category Archives: News

Below is a guest post from Dr Philip Fowler, who leads our award-winning bug-squishing project BashTheBug. This project aims to improve the diagnosis and treatment of tuberculosis, which remains one of the leading causes of death worldwide.

This project has a huge amount of data to get through, so Phil is working hard to make sure this is being done in the most efficient way possible. Read on to find out more.

– Helen

Focussing effort where it is needed: picking out the Bugs that are harder to Bash

BashTheBug has been running for a little over a year now and in that time 11,303 volunteers have classified 834,032 images of the bacterium that causes tuberculosis growing on 14 antibiotics at different strengths. These images correspond to a bit less than 4,000 different samples of M. tuberculosis since each image is shown, by default, to different 15 volunteers to generate a consensus.

The goal of the larger CRyPTIC project that BashTheBug belongs to is to match all this data with the genomes of each and every sample and thereby produce the most comprehensive and accurate catalogue of what genetic variants confer resistance to specific antibiotics. This is important because there is a shift towards using genomic methods to diagnose which antibiotics would be best to treat individual patient infections because genomics can be faster, cheaper and probably more accurate as well.

Too many new images?

The CRyPTIC project has produced a new dataset of 4,286 samples. These have been collected from people with tuberculosis from all over the world.

This dataset alone would need 900,060 classifications if we were to simply require each antibiotic lane to be seen by 15 different volunteers and, unless a lot more people joined the project, would take at least a year. Our problem is the project is producing around 1,000 samples a month, which would require 210,000 classifications a month, which our volunteers at present could not keep up with!

Ultimately the CRyPTIC project will collect at least 30,000 samples over the next few years, so we are only at the beginning!

Some images are easy…

What might help is we’ve found that some of the images of bacterial growth are easy to classify. For example, all 15 volunteers identify well number 2 as the first well in which there is growth.

If the volunteers find this easy, a computer might also, so we wrote some computer software (called AMyGDA) that tries to measure the growth in each of the wells on the 96-well plate. It does a good job on these simple cases, but is confused by cases where this is little growth, or there are artefacts on the image, like air bubbles, contamination or shadows.

We can identify the “easier” images based on how much growth there is, and whether the computer software agrees with the single reading we have of each plate done by a laboratory scientist. On our new dataset of 4,286 samples, this approach identifies 84% of the antibiotic lanes as easy to classify.

If we only send the remaining 16% of images to the volunteers, that reduces the number of classifications we need to complete this dataset down to 144,000 with a monthly growth rate of 34,000 which is much more achievable!

…and some are hard.

But this means you will all be seeing images that are harder to interpret and classify and therefore should be more of a challenge.

This is an example of an image that is harder to classify.

In our existing dataset, these images have typically elicited a range of answers. Some volunteers might say they cannot classify the image, whilst others would identify a range of wells as being the first with no growth. We can, of course, still form a consensus (I’d say well 5), but the variation is itself telling us something about how and why the image is hard to classify, which is potentially useful (for example, for training a machine learning classifier).

A few things to think about

Because the images should be, on average, more challenging now, you will have to make more frequent judgment calls about whether that blob in well 5 is an artefact or whether it “looks like” growth, and if you think it is, whether or not it is big enough to be relevant. Personally, I’d say for something to be growth it has to look like the growth in the positive control wells. If it is a lot smaller (like a dot) then I personally tend to ignore it. Don’t spend too long on individual images – rely on the collective power of the volunteers to allow us to extract a consensus from all your answers!

Focussing your efforts

In summary

– there are a lot of new images available to classify on our Zooniverse project page and
– they should be, on average, a lot more interesting and challenging

On the 3rd of July the Zooniverse team headed to London to take part in the Royal Society’s Summer Science Exhibition. Several Zooniverse projects were featured, including Galaxy Zoo, Penguin Watch and The Plastic Tide:

For those of you new to the Zoo, the Zooniverse is an online platform for Citizen Science research. It relies on volunteers to analyse data which then contributes to real research. This often results in new discoveries, publications and data sets useful to the wider research community. At the Royal Society, Zooniverse team members; Adam McMaster, Grant Miller, Cam Allen, Jim O’Donnell and Helen Spiers gave visitors a whistle stop tour of the Zooniverse and answered any questions people had about what we do. Visitors were surprised at the plethora of projects on the platform that they can contribute to, and got to ‘listen to the Zooniverse’ via a web page that plays a note for every classification made! You can listen to it here, it’s not the easiest thing to dance to but I think it makes nice background music.

Members of the Galaxy Zoo team, Coleman Krawczyk and Jen Gupta, shared the ‘Tactile Universe’ with visitors. The Tactile Universe is a project aimed at making astronomy more accessible to those with visual impairments. The team behind it has developed a way to 3D print galaxy images so that the brighter parts of the picture stick out more. This allows you to feel the shape of the galaxy rather than see it. People really seemed to enjoy trying to match up pictures of the galaxies with what they could feel on the ‘Tactile Universe’ tiles. Also on show was the Galaxy Zoo project, which asks volunteers to identify the type of galaxy in an image.

Do you want to count some penguins? If yes then Penguin Watch is the project for you! Fiona Jones, a member of the Penguin Watch team, and myself showed this project off to visitors at the exhibition. Visitors were interested to learn how something as simple as clicking on the penguins in an image can make a big difference to not only scientists but also the penguins themselves. This is because scientists can use the information they get from Penguin Watch to monitor and protect the penguin populations. We also had some cold weather gear and a ration pack which the team need to set up the Penguin Watch cameras in the cold arctic climate. (I still tried the gear on despite the 25 degree temperatures in sunny London!).

Showing visitors the big problem of plastic pollution in our oceans was The Plastic Tide team; Peter Kohler, Stefan Leutenegger, Karl-Mattias Tepp, and Arturo Castillo. The Plastic Tide project asks volunteers to look at photos of beaches taken by their drone and tag plastic and other rubbish. Visitors to the exhibition were very interested to learn that they could help with this project even from the comfort of their own homes! The team also gave a live demonstration of how they are using the data they get from the Zooniverse to train a computer to identify rubbish on beaches!

I wanted to let you know that a new company called 1715 Labs has been set up to make commercial use of the software created by the Zooniverse team. Specifically, the company will explore how other businesses might make use of our tools in order to classify and label images, text, audio and video.

We’ve been approached over the years by a number of companies with such projects in mind, but the Zooniverse policy has always been to accept only projects whose aim is academic research. (See our policy statement at https://www.zooniverse.org/help/lab-policies).

This is not changing. This policy will remain the same for Zooniverse projects, so you can be sure that any project you see at Zooniverse.org will continue to have as its goal the advancement of academic research. Projects developed by 1715 Labs will not appear at Zooniverse.org.

It’s also important that you, the volunteers, know that the Zooniverse will not be handing the new company any of your data or personal information. Indeed, according to the Zooniverse privacy policy we will not be able to. Instead, the company will use the same software as the Zooniverse to reach other crowds who can take part in any projects it creates.

The team who have been working on the Zooniverse will continue to do so, just as they always have. However, the possibility exists that some team members – including myself – may serve as paid consultants for 1715 Labs as the new company gets off the ground. This work will be managed separately from work for us in the Zooniverse.

1715 Labs is formally a spin-out company of the University of Oxford, where a large part of the Zooniverse team have been based from the beginning. It is currently led by Sophie Hackford (We’re currently recruiting a long-term CEO – if you’re interested, send me a CV). Normally, the researchers involved in leading such a spin-out would receive equity in the new company, and benefit financially from it.

However, in this case, we have given up any such rights, passing the shares instead to another new organisation, the 1715 Association. This means – unusually for a spin out company – no-one involved in Zooniverse owns shares or has a financial stake in the new company. (As noted above, some of the team may end up working for 1715 labs as consultants).

The 1715 Association is what’s known as a company limited by guarantee. If it receives money as a result of its ownership of shares in 1715 Labs, it must use it in accordance with its objects. The objects of the 1715 Association are to benefit citizen science research, especially through the Zooniverse. Should the company do well, therefore, the result will be additional funding for our work here at Zooniverse.org and the chance to build new, better, more interesting projects.

This is good news – we want the excellent software our developers have created to be used, and if it can benefit our research, then so much the better. Hopefully, businesses with data that needs labelling will be inspired by this link to the Zooniverse to work with 1715 Labs.

I’m looking forward to seeing what happens with this new venture. In the meantime, I’m happy to answer any questions in the comments below or over on Talk.

Occasionally we run studies in collaboration with external researchers in order to better understand our community and improve our platform. These can involve methods such as A/B splits, where we show a slightly different version of the site to one group of volunteers and measure how it affects their participation, e.g. does it influence how many classifications they make or their likelihood to return to the project for subsequent sessions?

One example of such a study was the messaging experiment we ran on Galaxy Zoo. We worked with researchers from Ben Gurion University and Microsoft research to test if the specific content and timing of messages presented in the classification interface could help alleviate the issue of volunteers disengaging from the project. You can read more about that experiment and its results in this Galaxy Zoo blog post https://blog.galaxyzoo.org/2018/07/12/galaxy-zoo-messaging-experiment-results/.

As the Zooniverse has different teams based at different institutions in the UK and the USA, the procedures for ethics approval differ depending on who is leading the study. After recent discussions with staff at the University of Oxford ethics board, to check our procedure was up to date, our Oxford-based team will be changing the way in which we gain approval for, and report the completion of these types of studies. All future study designs which feature Oxford staff taking part in the analysis will be submitted to CUREC, something we’ve been doing for the last few years. From now on, once the data gathering stage of the study has been run we will provide all volunteers involved with a debrief message.

The debrief will explain to our volunteers that they have been involved in a study, along with providing information about the exact set-up of the study and what the research goals were. The most significant change is that, before the data analysis is conducted, we will contact all volunteers involved in the study allow a period of time for them to state that they would like to withdraw their consent to the use of their data. We will then remove all data associated with any volunteer who would not like to be involved before the data is analysed and the findings are presented. The debrief will also contain contact details for the researchers in the event of any concerns and complaints. You can see an example of such a debrief in our original post about the Galaxy Zoo messaging experiment here https://blog.galaxyzoo.org/2015/08/10/messaging-test/.

As always, our primary focus is the research being enabled by our volunteer community on our individual projects. We run experiments like these in order to better understand how to create a more efficient and productive platform that benefits both our volunteers and the researchers we support. All clicks that are made by our volunteers are used in the science outcomes from our projects no matter whether they are part of an A/B split experiment or not. We still strive never to waste any volunteer time or effort.

We thank you for all that you do, and for helping us learn how to build a better Zooniverse.

Hello, my name is Sophia Vaughan, I’m a Physics student from Oxford University but I don’t spend all of my time studying physics, I also like to knit and go to science fiction conventions (especially Star Trek ones).

I’m posting on this blog because I’ve just started a summer project here at Zooniverse and while I’m here hoping to create a new project for the Zooniverse. But don’t worry you don’t need to be a scientist, student or a professor to create a project, all you need is Zooniverse account! If all goes well you’ll see future blog post(s) from me on how to build your own project for using the project builder.

Right now, Science Scribbler is working to process a dataset related to Huntington’s Disease. Many of the circular objects marked by volunteers as part of the project are organelles. These are smaller compartments inside of each cell, and each type of compartment has a different job. In Huntington’s Disease, some of the compartments are dysfunctional, but details about how the disease leads to this dysfunction aren’t really known. By comparing a diseased cell to a non-diseased cell we’re hoping to learn about organelle changes due to the disease.

How do Zooniverse volunteers contribute to your research?

Zooniverse volunteers are amazing! Without them, this project would be moving at a glacial speed!

For this project, volunteers are asked to place marks in the centers of objects and outline the objects. This information will first tell us where all of the organelles are so we can begin to answer our biological question. And second, using this dataset as a standard, we hope to create a computer algorithm that will use just the center points to find the outlines of the objects, making the task easier and faster in the future.

The problem of segmentation – marking out parts of a dataset in order to analyze is – is a common problem across many imaging techniques and disciplines. One of the benefits of working on these problems at Diamond Light Source is that there are always people and projects that need improved segmentation. Our next project on Science Scribbler will focus on a new imaging technique and a new segmentation problem. This switch in focus will give the original project some time to process the data, come up with potential computer algorithms to improve the process and think about the next steps. And it will give a new project and new researchers a chance to interact with the Zooniverse to jumpstart their research!

What are your favourite other citizen research projects and why?

I love Wildwatch Kenya! I find it super chill and it’s always fun when you find an animal in the picture!

And finally, when not at work, where are we most likely to find you?

Long walks with my husband and dogs (Jinx and Charm) and reading good books (right now, re-reading the Harry Potter series).

Part three in a multi-part series exploring the visual and UX changes to the Zooniverse classify interface

Coming soon!

Today we’ll be going over a couple of visual changes to familiar elements of the classify interface and new additions we’re excited to premier. These updates haven’t been implemented yet, so nothing is set in stone. Please use this survey to send me feedback about these or any of the other updates to the Zooniverse.

Keyboard shortcut modal

New modals

Many respondents to my 2017 design survey requested that they be able to use the keyboard to make classifications rather than having to click so many buttons. One volunteer actually called the classifier “a carpal-tunnel torturing device”. As a designer, that’s hard to hear – it’s never the goal to actively injure our volunteers.

We actually do support keyboard shortcuts! This survey helped us realize that we need to be better at sharing some of the tools our developers have built. The image above shows a newly designed Keyboard Shortcut information modal. This modal (or “popup”) is a great example of a few of the modals we’re building – you can leave it open and drag it around the interface while you work, so you’ll be able to quickly refer to it whenever you need.

This behavior will be mirrored in a few of the modals that are currently available to you:

Add to Favorites

Add to Collection / Create a New Collection

Subject Metadata

“Need Help?”

It will also be applied to a few new ones, including…

Field Guide

New field guide layout

Another major finding from the design survey was that users did not have a clear idea where to go when they needed help with a task (see chart below).

Survey results show a mix of responses

We know research teams often put a lot of effort into their help texts, and we wanted to be sure that work was reaching the largest possible audience. Hence, we moved the Field Guide from a small button on the right-hand side of the screen – a place that can become obscured by the browser’s scrollbar – and created a larger, more prominent button in the updated toolbar:

By placing the Field Guide button in a more prominent position and allowing the modal to stay open during classifications, we hope this tool will be taken advantage of more than it currently is.

The layout was the result of the audit of every live project I conducted in spring 2017:

Field Guide

Mode item count

5

Mode label word count

2

Min item count

2

Min label word count

2

Max items count

45

Max label word count

765

Using the mode gave me the basis on which to design; however, there’s quite a disparity between min and max amounts. Because of this disparity, we’ll be giving project owners with currently active projects a lot of warning before switching to the new layout, and they’ll have the option to continue to use the current Field Guide design if they’d prefer.

Tutorial

Another major resource Zooniverse offers its research teams and volunteers is the Tutorial. Often used to explain project goals, welcome new volunteers to the project, and point out what to look for in an image, the current tutorial is often a challenge because its absolute positioning on top of the subject image.

No more!

In this iteration of the classify interface, the tutorial opens once as a modal, just as it does now, and then lives in a tab in the task area where it’s much more easily accessible. You’ll be able to switch to the Tutorial tab in order to compare the example images and information with the subject image you’re looking at, rather than opening and closing the tutorial box many times.

A brand-new statistics section

Another major comment from the survey was that volunteers wanted more ways to interact with the Zooniverse. Thus, you’ll be able to scroll down to find a brand-new section! Features we’re adding will include:

Your previous classifications with Add to Favorites or Add to Collection buttons

Interesting stats, like the amount of classifications you’ve done and the amount of classifications your community have done

Links to similar projects you might be interested in

Links to the project’s blog and social media to help you feel more connected to the research team

Links to the project’s Talk boards, for a similar purpose

Possibly: A way to indicate that you’re finished for the day, giving you the option to share your experience on social media or find another project you’re interested in.

The statistics we chose were directly related to the responses from the survey:

Survey results

Respondents were able to choose more than one response; when asked to rank them in order of importance, project-wide statistics were chosen hands-down:

Project-wide statistics are the most important

We also heard that volunteers sometimes felt disconnected from research teams and the project’s accomplishments:

“In general there is too less information about the achievement of completed projects. Even simple facts could cause a bit of a success-feeling… how many pictures in this project over all have been classified? How much time did it take? How many hours were invested by all participating citizens? Were there any surprising things for the scientists? Things like that could be reported long before the task of a project is completely fullfilled.”

Research teams often spend hours engaged in dialog with volunteers on Talk, but not everyone who volunteers on Zooniverse is aware or active on Talk. Adding a module on the classify page showing recent Talk posts will bring more awareness to this amazing resource and hopefully encourage more engagement from volunteers.

Templates for different image sizes and dimensions

When the project builder was created, we couldn’t have predicted the variety of disparate topics that would become Zooniverse projects. Originally, the subject viewer was designed for one common image size, roughly 2×3, and other sizes have since been shoehorned in to fit as well as they can.

Now, we’d like to make it easier for subjects with extreme dimensions, multimedia subjects, and multi-image subjects to fit better within the project builder. By specifically designing templates and allowing project owners to choose the one that best fits their subjects, volunteers and project owners alike will have a better experience.

Very wide subjects will see their toolbar moved to the bottom of the image rather than on the right, to give the image as much horizontal space as possible. Tall subjects will be about the same width as they have been, but the task/tutorial box will stay fixed on the screen as the image scrolls, eliminating the need to scroll up and down as often when looking at the bottom of the subject.

Wide and tall subjects

Let’s get started!

I’m so excited for the opportunity to share a preview of these changes with you. Zooniverse is a collaborative project, so if there’s anything you’d like us to address as we implement this update, please use this survey to share your thoughts and suggestions. Since we’re rolling these out in pieces, it will be much easier for us to be able to iterate, test, and make changes.

We estimate that the updates will be mostly in place by early 2019, so there’s plenty of time to make sure we’re creating the best possible experience for everyone.

Thank you so much for your patience and understanding as we move forward. In the future, we’ll be as open and transparent as possible about this process.

In this week’s edition of our Who’s who in the Zoo series meet Dr Sam Blickhan who leads the development of new Humanities projects here in the Zooniverse.

– Helen

Name: Sam Blickhan

Location: Adler Planetarium, Chicago IL, USA

Tell us about your role within the team:

I’m the IMLS Postdoctoral Fellow and Humanities Lead. I started in March of 2017 as a Postdoc, and started to take on more Humanities Lead duties in 2018.

What did you do in your life before the Zooniverse?

Before coming to the Zooniverse I was a student. I did my undergrad at the University of Iowa, studying Medieval English Literature and classical Voice Performance. Then I did a Masters in Musicology at Oxford, and went on to do a PhD in Musicology at Royal Holloway, University of London, writing about the palaeography of medieval music notation. I’ve always been interested in technologies of writing and the development of language, as well as digital approaches to research and teaching, so being able to work on transcription projects with Zooniverse is a really great way to continue that academic work in a not-so-traditional format.

What does your typical working day involve?

Always coffee. Depending on what I’m working on, my day could involve communication with research teams, project planning/design/development with my Zooniverse colleagues, grant writing, data analysis, preparing conference papers & presentations, professional development, and/or researching & writing articles. It varies quite a bit from day to day, which I love — I never, ever get bored, and I get to meet lots of interesting people!

How would you describe the Zooniverse in one sentence?

Zooniverse is as much or as little as you want it to be; a way to relax, to learn, and engage with others.

Tell us about the first Zooniverse project you were involved with

When I was in grad school I used to volunteer on Seafloor Explorer as a way to relax — I felt like I could turn off the part of my brain that was doing lots of critical thinking, while still doing something productive and interesting. The first project I was actually a part of developing was Anti-Slavery Manuscripts.

What’s been your most memorable Zooniverse experience?

I’m lucky to be able to travel regularly for my job, and I love meeting people from other platforms and/or institutions and learning about how their crowdsourcing projects work. I was part of a panel discussion at the IMLS (Institute of Museum and Library Services) in 2017, and got to meet some people who have been doing really amazing work for a long time. It was exhilarating to be able to have conversations about public research methods and access to archival materials and data, and learn from these larger communities of researchers and advocates, as well as show them what tools and opportunities for research we at Zooniverse have to offer.

What are your top three citizen science projects?

SCOTUS Notes (www.scotusnotes.org) is one of my favorite new projects. I’m a bit of a politics junkie, so reading the Justices’ comments can be a very interesting look at the thought processes behind these Supreme Court decisions. I think that the American Soldier project (https://www.zooniverse.org/projects/tkotwim/the-american-soldier) is also incredible. The soldiers’ responses to the questions are often very moving and powerful to read. Outside the Zooniverse, I’m a big fan of the Colored Conventions Project (http://coloredconventions.org/) — I was lucky enough to meet some of their team at a conference last year, and they do an incredible job of engaging their community in a positive way, and reminding the public how important it is to keep re-examining history and working to create space for those who have been overlooked. I’ve learned a lot from listening to them and watching their work.

What advice would you give to a researcher considering creating a Zooniverse project?

Look at as many other projects as you can! Really spend some time classifying, but also try to get a sense of what the volunteer and research communities are like across projects. Reach out to other project owners, as well as volunteer and Talk moderators, and ask about their experiences. It’s hard to anticipate what it will be like to run a project if you haven’t yet participated in one. Learn by doing!

When not at work, where are we most likely to find you?

Since it’s summer, I’m probably outside running, biking, or on a patio with friends. I love to make things and enjoy knitting & embroidery in particular. Music is a big part of my life, too — I go to lots of concerts and play music with friends whenever I can.

Part two in a multi-part series exploring the visual and UX changes to the Zooniverse classify interface

The breakdown

Today and in the next post, we’ll take a look at the reasoning behind specific changes to the classifier that we’ve already started to roll out over the past few months. We’ve had good discussions on Talk about many of the updates, but I wanted to reiterate those conversations here so there’s just one source of information to refer back to in the future.

In case you missed it, the first blog post in this series previews the complete new classify layout.

As a reminder, if you have feedback about these changes or anything else on the site you’d like to see addressed, please use this survey link.

Navigation bar

Updated navigation bar

We started with a rethinking of each project’s navigation bar. The new design features cleaner typography, a more prominent project title, and visual distinction from the sitewide navigation. It also includes the project’s home page background image, giving the project visual distinction while keeping the classify interface itself clean and legible. It’s also responsive: on smaller screen heights, the height of the navigation bar adjusts accordingly.

The most important goal we solved in making this change was to separate the project navigation from the site navigation. During my initial site research and in talking to colleagues and volunteers, many found it difficult to distinguish between the two navigations. Adding a background, a distinct font style, and moving the options to the right side of the page accomplishes this goal.

Neutral backgrounds

Classify interface with neutral background

In conjunction with adding the background image to the navigation bar, the background image was removed from the main classify interface. It was replaced with a cool light grey, followed quickly by the dark grey of the Dark Theme.

Legibility is one of the main goals of any web designer, and it was the focus of this update. By moving to clean greys, all of the focus is now on the subject and task. There are some really striking subject images on Zooniverse, from images of the surface of Mars to zebras in their natural habitat. We want to make sure these images are front and center rather than getting lost within the background image.

The Dark Theme was a suggestion from a Zooniverse researcher – they pointed out that some subject images are similar in tone to the light grey, so a darker theme was added to make sure contrast would be enough to make the image “pop”. We love suggestions like this! While the team strives to be familiar with every Zooniverse project, the task is sometimes beyond us, so we rely on our researchers and volunteers to point out anomalies like this. If you find something like this, you can use this survey to bring it to my attention.

Another great suggestion from a Zooniverse volunteer was the addition of the project name on the left side of the screen. This hasn’t been implemented yet, but it’s a great way to help with wayfinding if the interface is scrolled to below the navigation bar.

Updated task section

New task section

By enclosing the task and its responses in a box rather than leaving it floating in space, the interface gives a volunteer an obvious place to look for the task across every project. Adjusting the typography elevates the interface and helps it feel more professional.

One of the most frequent comments we heard in the 2017 survey was that the interface had far too much scrolling – either the subject image or the task area was too tall. The subject image height will be addressed at a later date, but this new task area was designed specifically with scrolling in mind.

I used the averages I found in my initial project audit and the average screen height (643 px) based on Google Analytics data from the same time period to design a task area that would comfortably fit on screen without scrolling. It’s important to note that there are always outliers in large-scale sites like Zooniverse. While using averages is the best way to design for most projects, we know we can’t provide the most optimal experience for every use case.

You’ll also notice the secondary “Tutorial” tab to the right of the “Task” label. This is a feature that’s yet to be implemented, and I’ll talk more about it in the next post.

And more to come

The next installments in this series will address the additional updates we have planned, like updated modals and a whole new stats section.