Home

User Errors

Dictation errors are represented as the percentage of words written differently from the dictation text. Participants were not penalised for using lower case letters to begin capitalising proper nouns, although erratic capitalisations of common nouns were counted as an error, unless they appeared at the beginning of a sentence. By the end of the twelve sessions, even those novices most resistant to grammatical convention had discovered that Dasher's language model rewards capitalising properly. In a few cases, a word or phrase on the recording was misunderstood. These cases were not counted as errors since the users successfully generated the text that they intended to write. Here we see that errors greater than 10% are rare after about thirty to forty minutes of experience with a particular mode, and it is not uncommon for even novice users to complete some trials with no errors at all.

Information inefficient button pressing

• Loops

An ideal user would generate text by always selecting the correct zooming-in action without mistake. Real users, however, make information inefficient button presses. It would seem reasonable to expect that users make fewer information inefficient selections as they gain experience using Dasher. Interestingly, however, this does not always seem to be the case. Menu Dasher users sometimes overshoot their target box and have to cycle through the other boxes and backup option to get back to their intended box, costing a number of extra button pushes equal to the total number of boxes, B. The percentage of button presses corresponding to needless looping back through menu options is shown here.

Although novice uniform menu users show an initial improvement after an hour of experience, users still spent between 15 - 25 % of their button presses on unnecessary cycling. Of these users, those who spent another hour using the non-uniform menu mode did not show significant improvement. One might expect that eventually users would get better at hitting their target without overshooting, but even an expert menu user spends nearly a fifth of total clicks on looping unnecessarily through menu options!

For expert user C, it is particularly easy to see how a larger number of boxes penalises a user who overshoots his target; this user alternated between having four and six selection boxes in non-uniform menu mode and because each loop required two extra button pushed to start a cycle over again, the percentage of the time user C spent on looping was consistently higher for the six-box trials.

• Backing up

One might again expect users to learn to move toward their target letter or string with less and less backing up required, but neither direct nor menu users stop zooming out.   For direct mode users, a zooming out event is accomplished by one press of a button, but for menu users, backing up requires a number of button pushes equal to the number of boxes, B.  Again, the disadvantage of a larger number of selection boxes is illustrated by user C's non-uniform menu results.

• Total information inefficient button presses for Menu Dasher users

Given that users don't stop making unnecessary loops, direct users have the clear advantage of not ever needing to loop. The only information inefficient button press a direct mode user can make is a one-button press that zooms out when a mistake is made. Menu users must also zoom out to correct errors, and the cost for their zoom-outs is again proportional to the number of selection boxes available. Thus, menu users have two possible ways of making inefficient navigating decisions, and each of these costs them a factor of B more button presses than the direct mode users!

Plotting the total percentage of technically unnecessary button presses that menu mode user are making per trial, the results are shocking, especially when compared to the percentage of zoom-outs that direct mode users make. In the beginning, the majority of button presses novices make are information inefficient, and even after an hour or two of practice, many users, including experts, are still using 20 - 40% of their button presses inefficiently!

By comparing the percentage of the total information inefficient button presses zoom-outs in direct mode (i.e. the percentage of zoom-outs shown above) to the following plots showing the total number of information inefficient button pushes made by menu modes users, it's easy to see why Direct Dasher users are performing so much better.

This may not even be the full story; with experience, Direct Dasher users seem to begin to use the occasional information inefficient zoom out to help them write faster. Instead of navigating directly to a target string of characters by a completely information efficient series of buttons like, say, {up down down up down down up up}, more experienced direct users tend to learn to use the back-up button to align the target text such that they can navigate towards it by only changing zoom-in buttons once, like, say, {back up up up up up up up up down}. Thus, direct mode users seem to be able to write more quickly by aligning their target text between the two halves of the screen in a favourable way, even when this means using theoretially information inefficient gestures. 

Next trials under consideration

• Direct mode, expert users, s=0, Austen training text, updated software. In previous user trials, the same passages from Emma were used as dictation tasks and Dasher was trained using the remaining portion of the novel. In these trials, Dasher was trained using a corpus consisting of text of different genres taken from a wide variety of sources. The results therefore do not represent the upper bounds on the writing speeds that learning or experienced users can achieve. It is also expected that results will improve given software modifications which remove disorienting screen flashes and allow users to click more quickly. It would be useful to determine the maximum writing speeds and information rates attainable.
• Chris mode, novice & expert users. When an expert user spent 10 minutes using Direct Dasher with no safety margin and a very large non-uniformity, information rates and writing speeds were comparable to Direct Dasher results. This suggests a two-button adaptation of Direct Dasher in which one of the zoom-in buttons is controlled by Dasher, leaving the user to control the zoom-out button and the timing of one zoom-out button.
• Direct or Chris mode, variable safety margin. For the purpose of these trials, the safety margin was set somewhat arbitrarily. Making this overlap smaller would theoretically increase information efficiency, but may cause problems when users can't distinguish on which side of a boundary their desired text sits. More work should be done to quantify exactly how small this overlap can be before it has a negative effect on user performance. Theoretically, the performance of the above users would improve by 7% if the safety margin could be set to zero.
• Menu mode, B=2.  It was initially thought that one of the advantages of the direct mode was that it would operate essentially as a two-button mode, relying primarily on only the two zooming-in buttons with the third button used only rarely. As this third button is turning out to be an important feature in direct mode, it could be interesting to reconsider ways in which users who can't perform time-critical tasks could most effectively communicate with only two buttons.
• Compass mode. A four-button mode allows users to navigate in both dimensions of the Dasher canvas.