CPS: User Manual

Antechamber ?

In older days, one had to go through a gentleman's antechamber and wait here until he could see him. The same applies here: rough results sent by crowdflower will be waiting here before they are retrieved by CPS. To further the analogy, this component will warn you when results are are waiting for you (just like a servant) and prepare them for you.

As stated above, these are the two functions of this component:

Receive the results from CrowdFlower, store them and parse them: while CPS can read csv files, it can not read JSON. Why ?
Send you a notification e-mail when results are ready, i.e when you can launch a new iteration.

Mail Address

You have to edit the main.php file of the antechamber in order for the $address variable to contain yours.

From JSON to CSV

As stated in the introduction, it is necessary to change the format of the result file from JSON to CSV. This is done using PHP's JSON parser, json_decode, which converts a JSON formatted string into an array. Here is the function:

00001	function json2csv($content,$path)
00002	/* Transforms the JSON formatted string $content in a csv fomatted
00003	* one and prints it in the file whose path is $path. */
00004	{
00005		echo '$content:'.$content."\n";
00006		$jsonInput = json_decode($content,true);
00007		 // echo "\n".'job_id:'.$jsonInput['id']."\n";
00008		
00009		$res = '_unit_id,_created_at,_id,_started_at,_trust, _worker_id,_country,_region,_city,_ip,which_media_conveys_the_most,idMedia1,idMedia2,axis,urlmedia1,urlmedia2'."\n";
00010		$j = $jsonInput["results"]["judgments"];
00011		$count = count($j);
00012		for($i=0;$i<$count;$i++)
00013		{
00014			 //  initialisation
00015			$line = '';
00016			// building line
00017			$line .= $j[$i]["unit_id"].'.';
00018			$line .= $j[$i]["created_at"].'.';
00019			$line .= $j[$i]["started_at"].'.';
00020			$line .= $j[$i]["id"].'.';
00021			$line .= $j[$i]["which_media_conveys_the_most"]["confidence"].'.';
00022			$line .= $j[$i]["worker_id"].'.';
00023			$line .= $j[$i]["country"].'.';
00024			$line .= $j[$i]["region"].'.';
00025			$line .= $j[$i]["city"].'.';
00026			$line .= $j[$i]["ip"].'.';
00027			$line .= $j[$i]["judgment"].'.';
00028			 // adding the content of the 'data' field:
00029			$line .= $j[$i]["data"][idmedia1"].'.';
00030			$line .= $j[$i]["data"][idmedia2"].'.';
00031			$line .= $j[$i]["data"]["axis"].'.';
00032			$line .= $j[$i]["data"]["urlmedia1"].'.';
00033			$line .= $j[$i]["data"]["urlmedia2"].'.';
00034			 // adding the line to the results
00035			$res .= $line."\n";
00036			echo $line."\n";
00037		}
00038		save($res,$path);
00039	}

Configuration

As you could see above, the json2csv method easily transforms a JSON formatted file into a CSV formatted one.However, its configuration is not definitive! Indeed, I could not find a way to know for sure the field in which the answer to the question the workers are asked is stored. Therefore, you may have to edit this function!

If such a modification is necessary, it must be made at the line 27 of the function by replacing

$j[$i]["judgment"]

$j[$i]["where the judgment actually is"]

In order to know where it is, look at a json formatted results file you retrieved and locate a field containing either "Media 1", "Media 2" or anything else the turkers may answer to your question. The possible answers are set in the CML template.

Once your modification is complete, go to the antechamber, at the page where you can see your JSON files (i.e Raw Results), select the file you just received in the menu below and click on "Tranform it!". It will regenerate the corresponding csv file correctly.

Notification mail

When CrowdFlower gathered all the judgements necessary for a job (i.e, the group of comparisons uploaded all at once), it will post a webhook at an adress you set. These are used by the antechamber to know when to send you an e-mail inviting you to perform a new iteration. However, since webhooks are also supposed to be sent when each unit (one of the comparison CPS asked for) have all the judgements it needs, it also filters them: you probably don't want your mailbox to be filled with thousands of mail within days...

How it works

Note: You have to set the address of the main.php script of the antechamber as the one where webhooks are fired by CrowdFlower if you want his to work (and you most likely do).

The way the PHP scripts deal with webhooks is pretty straight forward: when a Webhook is received, its payload is decoded using json_decode(). Then, depending on the value of the signal attribute, different things can happen:

If it corresponds to the end of a job ($signal === 'job_complete'):
1. The identifier of the job is retrieved.
2. The content of the webhook, i.e its payload attribute, is saved in webhook.json.
3. It is then parsed in order for it to be transformed into a csv file.
4. An email is sent to the user.
Otherwise, if it spots the end of a splitsort ($signal === 'sort_finished'), here is what happens (quch a webhook is sent by CPS itself, not by CrowdFlower):
1. An email is sent at the address you have set with the content of the payload as an attachment (it should the results.xml file.
2. The identifier of the results file is calculated: it is the date when it was retrieved.
3. This file is saved.

In every case, the date and hour of the post are stored in a logfile, as well as the value of the signal attribute.

include('webhookUtils.php');

// YOUR ADDRESS:span>
$address = 'jean-noel.bettinelli@ecl2012.ec-lyon.fr';

if (isset($_POST['signal']))
{
	$jsonPayload = json_decode($_POST['payload'],true);
	$triggerIteration = 'job_complete';
	$triggerFinished = 'sort_finished';
	if ($_POST['signal'] === $triggerIteration)
	{
		$job_id = $jsonPayload["id"];
		save($_POST['payload'],'raw/'.$job_id.'.json');
		json2csv($jsonPayload,'csv/'.$job_id.'.txt');
		echo 'Parsing of the JSON file finished: sending notification email.'."\n";
		jobCompleteNotification($address,$job_id);
	}
	else if ($_POST['signal'] === $triggerFinished)
	{
		finishedNotification($address,$_POST['payload']);
		echo 'Sort Finished: sending notification mail.'."\n";
		$job_id = date('Ymd-Hi').'-CPS_results.xml';
		save($_POST['payload'],'finalResults/'.$job_id);
	}
	else
		$job_id = $jsonPayload["job_id"];
	append2log($_POST['signal'],$job_id);
}