9 Crowd Labeling Techniques Right Under Your Nose: 7. Straightforward Asking
Ingenious ways in which tech companies get users to label their data
This post is part of a series on techniques for high quality data annotation through the user’s voluntary efforts. Check out the links to the introduction and the other techniques in the bottom of this article.
With that said, enjoy your read.
7. Straightforward Asking
Yeah, asking. You see, depending on how badly a user needs to solve a problem or access some specific information, she might fall into a temporal state of exploitable numbness. It is possible to take advantage of this moment by asking the user to label data.
Type of Data: Explicit data
Type of Labeling: Pairings of ‘data + label’ created between incomplete/incorrect pieces of information displayed to the user and their correct version, provided by her when she completes a puzzle.
Examples: reCaptcha by Google, Face Tagging on Facebook, Google Photos and Apple iOS, Bounding boxes on the Auto-Skew feature of apps such as U Scanner or Adobe Scan, “Suggest a better translation” button on Facebook, Google Translate.
Example 1: reCaptcha
Let’s suppose your rent is due tomorrow. You’re waiting for your salary, which should land in your bank account at anytime today. In order to make sure that the automatic payment scheduled for tomorrow happens smoothly, you login to check your balance. You input your user name and password, and the system shows you a series of puzzle-looking questions, but honestly speaking, you just need to know the number, and the questions are not so difficult anyways. You haste through the tests trying to solve them correctly so you can proceed to the next screen. See what happened there? Win-win.
Now, let’s be frank about the internet. There is a lot of malicious people trying to steal information. Because of that, any additional security measure is never extra. Validating whether it is in fact you, and not some malicious robot, the one requesting access to your bank’s server is critical. “I’m not a robot” validation is crucial for the usability of several aspects of the internet. Nevertheless, the true cleverness behind it comes in the way it’s enforced. For years Alphabet has used us to produce label street imagery to build self driving cars. Yeah, self driving cars.
Is that even closely related to our internet banking task? Of course not, and that is irrelevant. What’s important is that users are vulnerable and desperate to access their information fast, and because for that they made their best effort to “pass the tests”. Which ultimately elevates the accuracy of the responses provided to the reCaptchas and hence the data that is labeled through it.
Example 2: Face tagging on Facebook, Google Photos and Apple iOS
Remember the time before Facebook suggested you tagging a bunch of people on every picture you upload? Tagging people on pictures is a good way of doing the right thing and letting them know that you have shared their face in public. Before Facebook started suggesting the persons in the picture they didn’t even knew whether there was people in the image or not. But when they realized they could tap into our social string and we would gladly mark the location of ours and our friend’s faces in the pictures we uploaded, the smartest face recognition system was conceived. See, by willingly providing Facebook with millions of images with bounding boxes around the faces on them, and even with the link back to the accounts being tagged, we users single handedly created the highest quality training data set for facial recognition of all times (some users might create incorrect tags for various reasons, but regardless of this, for a data annotation team is much cheaper to evaluate the correctness of facial labelings on pictures that to label them from scratch).
Now think about all the narcissistic users that post their faces daily. They represent another extremely interesting dataset for computer vision experts. See, you can go beyond detection if you have several pictures of a single user to train siamese networks that enable facial recognition. Just take a look at this simple (yet great!) tutorial on facial recognition using open software. You know the drill: the main limitation in ML is high quality data availability.
Example 3: Adobe Scan Corner Detector
The promise of an app that allows you to forgo the Scanner whatsoever and do everything from the phone? I’m in! but wait, there is a catch. It sucks at corner detection. The solution? Provide the user with a tool to manually select the corners in case the automatic detector fails (which it does… a lot) and collect the data. Thus, the following data is properly stored: reference image, coordinates of the incorrectly generated bounding box, coordinates of the correct bounding box as manually input by the user. Isn’t that great? I mean, incentives are in place. The user is not going to input incorrect data because she wants her document properly scanned from the picture. So Adobe can almost certainly be sure that the bounding boxes are correct, which eliminates the need for data massaging.
Example 4: Contributing a Better Translation on Facebook
Another great example are the automatic translations on Facebook. Depending on the language that you use the service in and the on in which your “friends” post content in, the app will automatically show translated text. And translations are sometimes inaccurate (to say the least). Now, you are given the chance to improve it in case you think it has a mistake, and then submit the correction. There you go.
What can be achieved with this labeled data?
Given the nature of the examples provided it’s pretty straight forward to visualize the applications of this sort of labeled data. Thanks to the face labelings collected by Facebook now every picture with you on it is available without no human being lifting a finger, thus enabling further social interactions among users. Thanks to the faces we labeled in the photos app on iOS, Apple obtained a juicy facial database on top of which it developed the FaceID system that it now sold to us in the iPhone 8 (and beyond).
Adobe Scan, and other scanner apps providers can build CNN that automatically detect the corners of documents in pictures. Huge businesses are build around this very simple application (Sansan’s business card manager tool, for example).
Introduction | 1. P2P Connection Schemes | 2. Voting Systems | 3. Content Categorization | 4. Viewership | 5. Following on Search Results | 6. Autocomplete| 7. Straightforward Asking | 8. Correction of Human Mistakes | 9. Data Labeling Tools
