Microsoft Researchers Claim Their Deep Learning System Can Beat Humans

Microsoft Research in its new academic paper claims that its latest artificial intelligence called ‘deep learning system’ can outperform humans by one metric.

Deep Learning is a trend in Machine Learning, which is fast and powerful and implements Artificial Intelligence.

As reported by VentureBeat, the 2012 system got a 4.94 percent error rate for the correct image classification of the widely recognised ImageNet data set, as compared to the 5.1 percent error rate among humans. The challenge was to correctly identify objects in the images and then selecting the most accurate category for those images out of 1000 options.

The paper reports that human performance yields a 5.1 percent top-5 error on the ImageNet dataset. The reported human performance is estimated on a random subset of 1500 test images. The system’s result (4.94%) exceeds the reported human-level performance. According to the report, the two major types of human errors come from fine-grained recognition and class unawareness. This is where algorithms perform better.

The system, besides correctly predicting the dominant label in an image also gave four other predictions. For example, in the horse cart image (Figure 1, row 1, col 1), the other object which the algorithm identified was a mini bus, as can be seen in the background of the image.

Figure 1: Example validation images successfully classified by the system. For each image, the ground-truth label and the top-5 labels predicted are listed.

The image below (figure 2), shows some examples of images misclassified by the system. Some of the predicted other labels still make some sense. The error is due to the existence of multiple objects, small objects, or large intra-class variance.

For example, figure 1 (shown above) shows some example of fine-grained objects successfully recognized by the system, such as: ‘coucal’, ‘komondor’, and ‘yellow lady’s slipper’. While humans would have recognized these objects as a bird, a dog, and a flower, it is unlikely for most humans to tell their species. The algorithm makes mistakes in cases that are not difficult for humans, especially for those requiring context understanding or high-level knowledge. For example, the ‘spotlight’ image in figure 2, shown above, the algorithm identifies all three images as ‘spotlight’, however, they have different context.

As per the report, the algorithm gives a superior result on a particular dataset, but it does not imply that the machine vision can beat human vision on object recognition in general as they have their own limitations.

However, the researchers feel that their research and experiment show tremendous potential of machine algorithms to match human level on visual recognition.

No matter how great a machine is developed by scientists worldwide, it might never be able to match the human level of intelligence or perception. Afterall, the machine is a human invention itself.