Demonstrations#

We have recorded demonstrations from Mechanical Turk and put them in a separate repository.

Demonstration Format#

Each demonstration is saved as a JSON file. The root object generated by core/record.js contains the following fields:

  • taskName (string)

  • utterance (string)

  • reward (number): Reward as defined by the task

  • rawReward (number): 1 if succeeded and -1 if failed

  • states: a list of state objects

    • One state is recorded for the initial state

    • Two states are recorded for each event, one before the event resolves and one after the event resolves

Each state object has the following fields:

  • time (number): Time elapsed since the episode started

  • action: An action-specific object (not present for the initial state) with the following common keys:

    • type (string)

    • timing (number): the eventPhase property of the JS event object. This is 1 before the event resolves (capturing state) and 3 after the event resolves (bubbling state).

  • dom: The DOM info as generated by getDOMInfo()

    • The event target will have a special key recordingTarget set to true.

Recording Your Own Demonstrations#

  1. Start the recording server:

    # Create an output directory
mkdir out/
python -m miniwob.record out/

  2. Append ?record=true to the URL of the task you want to record. For example, for the click-test task, go to

    file:///path/to/miniwob-plusplus/html/miniwob/click-test.html?record=true

    (Note: For recent versions of Chrome, you might have to retype the whole URL instead of just appending ?record=true.)

  3. To view the results, open viewer/viewer.html while the recording server is running. The URL should be like

    file:///path/to/miniwob-plusplus/html/viewer/viewer.html