Environments List

Original Tasks

These are the original MiniWoB tasks.

Name	Description
bisect-angle	Find the line that bisects an angle evenly in two.
book-flight	Search for flight results.
choose-date	Learn to operate a date picker tool.
choose-list	Choose an item from a drop down list.
circle-center	Find the center of a circle.
click-button	Click on a specific button in a generated form.
click-button-sequence	Click on buttons in a certain order.
click-checkboxes	Click desired checkboxes.
click-collapsible	Click a collapsible element to expand it.
click-collapsible-2	Find and click on a specified link, from collapsible elements.
click-color	Click the specified color.
click-dialog	Click the button to close the dialog box.
click-dialog-2	Click a specific button in a dialog box.
click-link	Click on a specified link in text.
click-menu	Click menu items.
click-menu-2	Find a specific item from a menu.
click-option	Click option boxes.
click-pie	Click items on a pie menu.
click-scroll-list	Click multiple items from a scroll list.
click-shades	Click the shades that match a specified color.
click-shape	Click on a specific shape.
click-tab	Click on a tab element.
click-tab-2	Click a link inside a specific tab element.
click-test	Click on a single button.
click-test-2	Click on one of two buttons.
click-widget	Click on a specific widget in a generated form.
copy-paste	Copy text and paste it into an input.
copy-paste-2	Copy text from a specific textarea and paste it into an input.
count-shape	Count number of shapes.
count-sides	Count the number of sides on a shape.
drag-box	Drag the smaller box into the larger box.
drag-circle	Drag an item in a specified direction.
drag-cube	Drag a 3D cube to show a specific face.
drag-items	Drag items in a list, in a specified direction
drag-items-grid	Drag items in a 2D grid around.
drag-shapes	Drag shapes into a box.
drag-sort-numbers	Drag numbers into sorted ascending order.
email-inbox	Navigate through an email inbox and perform some actions.
enter-date	Use the date input to pick the correct date.
enter-password	Enter the password into the form.
enter-text	Enter given text to a textfield.
enter-text-2	Convert given text to upper or lower case.
enter-text-dynamic	Enter dynamically generated text to a textfield.
enter-time	Enter the specified time into the input.
find-midpoint	Find the shortest mid-point of two points.
find-word	Find nth word in a block of text.
focus-text	Focus into a text input.
focus-text-2	Focus on a specific text input.
grid-coordinate	Find the Cartesian coordinates on a grid.
guess-number	Guess the number.
highlight-text	Highlight all the text.
highlight-text-2	Highlight the specified paragraph.
identify-shape	Identify a randomly generated shape.
login-user	Enter user login details into the form.
navigate-tree	Navigate a file tree to find a specified file or folder.
number-checkboxes	Draw a given number using checkboxes.
read-table	Read information out from a table.
read-table-2	Read multiple pieces of information out from a table.
resize-textarea	Resize a textarea in a given direction.
right-angle	Given two points, add a third point to create a right angle.
scroll-text	Scroll through a text area element and enter last word into text area.
scroll-text-2	Scroll through a text area in a given direction.
search-engine	Search through a bunch of results to find a specified link.
simple-algebra	Solve for X.
simple-arithmetic	Perform some arithmetic math operations.
social-media	Interact with a social media feed.
terminal	Use the terminal to delete a file.
text-editor	Modify a text’s style in a text-editor.
text-transform	Enter slightly transformed text into a text box.
tic-tac-toe	Win a game of tic-tac-toe.
use-autocomplete	Use autocomplete element efficiently.
use-colorwheel	Use a color wheel.
use-colorwheel-2	Use a color wheel given specific random color.
use-slider	Use a slider to select a particular value.
use-slider-2	Use sliders to create a given combination.
use-spinner	Use a spinner to select given number.
visual-addition	Count the total number of blocks.

No-delay Tasks

The UI elements in some tasks have animation delays or change the state when the browser is defocused. We provide the “no-delay” version without these issues.

Name	Description
book-flight-nodelay	[book-flight] Removed animation.
choose-date-nodelay	[choose-date] Removed animation.
click-collapsible-nodelay	[click-collapsible] Removed animation.
click-collapsible-2-nodelay	[click-collapsible-2] Removed animation.
click-pie-nodelay	[click-pie] Removed animation.
use-autocomplete-nodelay	[use-autocomplete] Removed delay.

Additional Tasks

These tasks were introduced in MiniWoB++. Some are harder versions of the existing tasks, while some are completely new.

Name	Description
click-checkboxes-large	[click-checkboxes] Click at least 5 out of up to 12 checkboxes.
click-checkboxes-soft	[click-checkboxes] Paraphrased entries.
click-checkboxes-transfer	[click-checkboxes] Train and test on different number of targets.
click-tab-2-hard	[click-tab-2] Varying number of tabs from 2 to 6.
login-user-popup	[login-user] Random popup.
multi-layouts	Fill in forms of varying layouts.
multi-orderings	Fill in forms with shuffled field orderings.
social-media-all	[social-media] Do some action on all matching entries.
social-media-some	[social-media] Do some action on some matching entries.
email-inbox-forward-nl	[email-inbox-forward] varied instruction texts (30 templates).
email-inbox-forward-nl-turk	[email-inbox-forward] varied instruction texts (100 templates).
email-inbox-nl-turk	[email-inbox] varied instruction texts (100 templates for each subtask).

Debug Tasks

These are easier versions of existing tasks, suitable for debugging.

Name	Description
choose-date-easy	[choose-date] December only.
choose-date-medium	[choose-date] December or November only.
click-tab-2-easy	[click-tab-2] One 1 tab.
click-tab-2-medium	[click-tab-2] Choose between a link or ‘no match’.
click-test-transfer	[click-test] Different buttons during train and test.
email-inbox-delete	[email-inbox] No scrolling + 1 subtask.
email-inbox-forward	[email-inbox] No scrolling + 1 subtask.
email-inbox-important	[email-inbox] No scrolling + 1 subtask.
email-inbox-noscroll	[email-inbox] No scrolling.
email-inbox-reply	[email-inbox] No scrolling + 1 subtask.
email-inbox-star-reply	[email-inbox] No scrolling + 2 subtasks.
unicode-test	Click on the button with the correct Unicode text.

Flight Search Tasks

These are ports of the FormWoB tasks in the original World of Bits paper.,

• The prompt is a list of key-value pairs (e.g., Departure City: New York).

• If the required fields are not filled, or if the agent navigates away from the page, the reward is -1.,

• Otherwise, the reward is the fraction of key-value pairs that are satisfied.

Name	Description
flight.Alaska	Search for flights on the Alaska Airlines website (original).
flight.Alaska-auto	Search for flights on the Alaska Airlines website (harder).
flight.AA	Search for flights on the American Airlines website.

Hidden Test Tasks

These are tasks intended to be used as the hidden test set. They were originally not available from the OpenAI website.

Name	Description
ascending-numbers	Click on the numbers in ascending order.
buy-ticket	Buy a ticket that matches the requested criteria.
daily-calendar	Create an event on a daily calendar.
drag-single-shape	Drag a randomly generated shape in a specified direction.
drag-shapes-2	Drag shapes into boxes, categorized by type.
draw-circle	Draw a circle around a marked point.
draw-line	Draw a line through a marked point.
find-greatest	Find the card with the greatest number.
form-sequence	Perform a series of instructions on a form.
form-sequence-2	Perform a series of instructions on a form.
form-sequence-3	Perform a series of instructions on a form.
generate-number	Generate a random number that meets certain criteria.
hot-cold	Find and click on the hot area.
odd-or-even	Mark each number as odd or even.
order-food	Order food items from a menu.
phone-book	Find a contact in a phone book.
sign-agreement	Sign a user agreement.
stock-market	Buy from the stock market below a specified price.

Excluded Tasks

The following tasks require the agent to wait for events to happen before acting, and a ‘no-delay’ version is impossible to make.

Name	Description
chase-circle	Keep your mouse inside a moving circle.
moving-items	Click moving items before they disappear.
simon-says	Push the buttons in the order shown.
button-delay	Wait a certain period of time before clicking the second button.
hover-shape	Hover over the colored shape.