0.8
Sorting media using crowdsourcing.   
Home
LIRIS

Crowd Powered Sort

What is CPS?

Crowd Powered Sort is a java application allowing an easy use of crowdsourcing to place media continuously along one or several axis.

What CPS should not be used for?

If you can use a "real" algorithm (i.e, one that does not require any human intervention) to place your media along the axis you want, then you definitely should not use it. Indeed, while this software is free, the workers performing the comparisons must be paid, which could turn out to be rather expensive.

You should also avoid using it if you want to place your media in a discrete small number of categories : the results of CPS are continuous and, actually, the hole interest of CPS is in this continuity. It is much more complex!

Principle

The coordinate along each axis is found by sorting the media using a splitsort and then associates to their rank in the list a coordinate. The splitsort separates comparisons from the rest of the algorithm in order them to be performed by humans: this is where crowdsourcing enters. Indeed, comparisons are performed by human workers rather than machines. Therefore, CPS can be used to sort media along very subjective axes, such as happiness or pleasure.

Why CPS?

CPS was ordered by the LIRIS (Laboratoire d'InfoRmatique en Images et Systemes d'information), a French laboratory studying, among other things, the way the emotion conveyed by a media could be recognized by an algorithm. In order to generate a database containing a huge amount of media (several thousands or even more than ten thousand) tagged with the correct level of emotion, CPS was designed. However, though its primary use is rather specific, you can use it to have media put on any axis you want by human workers.

Crowdsourcing provider

CPS uses self-service, a crowdsourcing solution provided by CrowdFlower. Unlike Amazon's Mechanical Turk, it is not necessary to be american to use it.

How does it work ?

If you want more information on how this works, you are at the right place! This website contains CPS's documentation and useful PHP scripts. It is divided in several parts: