[Corpora-List] Robot Commands Treebank (www.TrainRobots.com) =?windows-1252?Q?=96_?=Feedback Welcome

[Kais Dukes]

Dear Srdjan,

Thanks for taking a look at the website. I probably should have clarified that there are two different aspects to this - the data collection part and the semantic treebank.

Over 10,000 sentences have been collected through the game, and 3,394 of these have been annotated in the treebank so far.

The way data collection works, is that once users have signed in their previous score is displayed and the game indefinitely repeats a three-round cycle. For incentive, as well as to provide examples of accurate commands, the first two rounds of each cycle are voting rounds, in which players rate previous commands. Points are awarded for accurate voting compared to the majority. Similarly, players who enter high-quality commands gain bonus points when their command is voted as accurate. In the third round of each cycle, players enter new commands for a random image pair selected from 1,000 preconstructed board configurations. The cycle repeats until the player logs off, with voting scores and commands saved to a database.

The more interesting part is the semantic treebank (http://www.trainrobots.com/treebank.jsp) – this contains annotations for part of the data that has been collected so far.

Kind Regards,

-- Kais Dukes
University of Leeds

______________________________________
From: srdjan [tarzanka at gmail.com]
Sent: 03 October 2013 08:38
To: Kais Dukes
Cc: corpora at uib.no
Subject: Re: [Corpora-List] Robot Commands Treebank (www.TrainRobots.com) – Feedback Welcome

Hello,

I have registered, and gave the game a go. For the UX perspective there are a couple of peculiarities. For instance, why is user constantly presented with an option to choose the 1-6 checkboxes, with labels that never change? And then assigned points for choosing one of them?

Best regards,

Srdjan Prodanovic


On Wed, Oct 2, 2013 at 10:48 PM, Kais Dukes <sckd at leeds.ac.uk<mailto:sckd at leeds.ac.uk>> wrote:
Dear Corpora List,

TrainRobots (www.TrainRobots.com<http://www.TrainRobots.com>) is a game with a purpose for annotating robot commands. Building on initial data collected through the game, the Robot Commands Treebank provides semantic annotation for robotic spatial commands. Collected through crowdsourcing, this first release of treebank provides annotation for 3,394 sentences (41,158 words). Commands are organized into scenes. These are pairs of images showing a game board before and after each command.

In the treebank, natural language sentences have been mapped to a compositional Robot Control Language (RCL). Each RCL statement forms a semantic tree that provides instructions to a spatial planner for command execution. The treebank also provides word-aligned annotations between leaf nodes in RCL statements and words in the corresponding annotated sentence.

This first version of the treebank focuses on commands that use qualitative spatial relations involving proximity ('near') and adjacent ('above', 'in front of', …) relations. The semantic treebank is freely available for research. The website includes a color-coded visualization of the semantic trees that have been annotated so far.

http://www.trainrobots.com/treebank.jsp

This data is currently being used to develop a statistical semantic parser as well as spatial natural language understanding component.

Feedback on the treebank is most welcome, as are any ideas for improving the website.

Kind Regards,

-- Kais Dukes
University of Leeds
_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora at uib.no<mailto:Corpora at uib.no>
http://mailman.uib.no/listinfo/corpora

_______________________________________________
UNSUBSCRIBE from this page: http://mailman.uib.no/options/corpora
Corpora mailing list
Corpora at uib.no
http://mailman.uib.no/listinfo/corpora