Components

There are several components interacting with each other when you use CPS. They can be all on the same machine or on different ones, depending on your needs and what you have at your disposal.

1. First of all is the server CPS Host, hosting CPS i.e the java bytecode (the content of the bin directory. It is in charge of downloading jobs results, parsing them, using them for a splitsort iteration and then uploading a new job containing new comparisons.

2. When the workers have answered all of the questions, CrowdFlower sends a webhook. But instead of having it sent directly to you, it first arrives in the Antechamber. This component consists in several PHP scripts triggered by the webhook. Their aim is to send you an email when a job is finished, therefore filtering these webhooks (there can be up to tens of thousands of units per job, the end of each being spotted. Your webmail may not appreciate such a flood).

   Furthermore, since CrowdFlower decided unilateraly (and without leaving the slightest warning for the users) to stop supporting the downloading of results as a CSV files, the Antechamber earned a new role : it parses the JSON formatted results and returns a csv file in order for CPS to work without a huge re-writing. The mail is sent after this operation.

3. Since you must display the media you want to sort for the workers to compare them, a server, Media Host must host all of them.
4. At last, CrowdFlower and its workers also interacts with these components, though you cannot host them where you please.

Here is a scheme that summarizes how these elements interact with each other. An array symbolises information flux.

This server host the CPS program, i.e the one whose documentation you are browsing right now. Since what this program does is discussed in pretty much every where else in this website, it will not be detailed here. However, in order for this to work as well as possible, here is what the server must have:

• A MySQL database.
• An Internet access allowing it to download and upload data from the antechamber. The bandwidth does not have to be huge because only text files of several Mo will be exchanged.
• A Java Virtual Machine (JVM).
• At least a few Go of free space.

It also should have as much computing power as possible. Indeed, the get another label program, which is called in order to increase the quality of the data, requires a lot of calculations.

As its name indicates it, this server is in charge of receiving webhook and preparing it to be sent to you. Depending on their signal attibute, the webhooks will trigger the sending of an e-mail to the user after parsing their payload attribute (the one alleged to contain the job results). This server must have:

• An Internet connection. Well, you know, the http protocol, used by the webooks, demands it.
• A mail server. This might be a little tickier to obtain if you want to install it yourself.
• An Apache server configured to be able to run PHP scripts.

Since it will most likely send less than one e-mail per day and parse a several Mo big JSON file as rarely, there is no need for the father of all servers here. The simplest web server you can get should be just fine. PHP scripts you can use as well as more details on how this component works are given a dedicated section.

This server will host the media you want to sort, i.e the "real" media, not only their names like in the media.xml file. The HITs the workers will download will contain media: these have to be stored somewhere! Here's what this server must have:

• An internet connection (again) allowing distant users to download its content. Indeed, the media it contains will most likely not travel by homing pigeon.
• A web server, like Apache, to allow distant users to connect with it.

Depending on the type of media you want to use, their may be a need for a rather high bandwidth and/or computing power, for example if you want to sort videos.