New Hit: Segment and label the objects in this image (Tool fixed)

ExpandCollapse

New Member

I'm new to MTurk as a requester and looking to get some MTurk workers to pick up my hits. I'm still trying to improve my image annotation tool and work out our best to balance reward and worker qualifications.

This is my hit, please search "Segment and label the objects in this image (Tool fixed)"

ExpandCollapse

TV Reviewing Meme Panda

I'm new to MTurk as a requester and looking to get some MTurk workers to pick up my hits. I'm still trying to improve my image annotation tool and work out our best to balance reward and worker qualifications.

This is my hit, please search "Segment and label the objects in this image (Tool fixed)"

ExpandCollapse

TV Reviewing Meme Panda

TLDR: Mechanical Turk forums are typically for American workers who expect American wages. It's clear that your task is either not in need of American workers or you're not willing to pay american wages. These forums aren't the best place to reach the type of workers you're looking for.

Its image annotation at a penny, that's always going to be difficult to make worth it but the poor layout, design, and server resources behind the HIT make it pretty much impossible.

The obvious issue with a penny HIT & images is that if the image loads slowly (which these did) you're already off to a rough start. Requester has them loaded on his crappy private server instead of allowing AWS to handle it for him & deliver content to workers quickly.

You'd think only having to annotate one object would make these somewhat okayish, even passable except that aside from the load times the images are also shared between all workers. So if you get an image that has all or most objects annotated, you're left twiddling your thumbs with nothing to really do or wasting time looking around for something to do. Some images are completely covered by annotation boxes and if you click one it throws an annoying dialog about how you're not allowed to edit someone else's work.

And to top it off the dropbox you have to use once closing in the object annotation to select what kind of object you're labeling is of course clunky and slows down the submission time considerably. It would be nice if the Requester split these up into "annotate doorways", "annotate mirrors" etc so that the dropdown wasn't necessary but that isn't the case for the HITs when I gave them a test run.

Click to expand...

Pros

You only have to annotate one object.
Perhaps this penny will be distributed heads up so its lucky?

Click to expand...

"No, A6_Foul_Out, you're just a meme stuck on loop." -
"THOSE WITHOUT BOWS WILL SUFFER NO CONSEQUENCES HERE."
"We are all One, under TurkerView."​

ExpandCollapse

New Member

Thansk for the feedback. We're still experimenting with the different options and trying to work out the best way to get our data annotated. I'll post again on our next iteration to see if we do any better.