DragonDictate is a combined navigator/dictation software package. Version 2.0 is the latest version offered (available since January 1996). The particular version implemented is the Classic Version which uses a 30,000 word vocabulary. DragonDictate version 1.3 was installed on an IBM PC compatible computer, with a Pentium processor running at 90 Mhz, 16 MB of RAM, a Sound Blaster 16 sound card, and a color monitor.
The evaluation of DragonDictate was done in two stages. The first stage evaluated the dictation accuracy and learning capability of DragonDictate. The second stage evaluated the ease of navigation performed while working in Windows. The navigational ability of DragonDictate was evaluated by noting how well the software was able to accommodate opening and closing various Windows applications. The Windows applications used were Lotus 1-2-3, WordPerfect, MatLab, Netscape, Eudora, and Windows Program Manager.
The dictation and learning capability of DragonDictate were measured
by dictating a standard passage (see Appendix D) consisting of 313 dictation words and commands
into WordPerfect using DragonDictate. The passage was dictated six times,
recording the number of mistakes, correcting the mistakes as they occurred
(using the technique described in the DragonDictate User's Guide [Ref.
1, pp. 20-28], and the length of time required to complete the dictation.
The errors were calculated as a fraction of the total number of commands
to give a percentage of each error type as well as the total amount of
errors. For this study there were four types of mistakes that could be
measured, which are listed and described below:
These measures of performance were taken against the passage in Appendix D, which was dictated into WordPerfect. The results are depicted in Figures 1 and 2.
Figure 1. Number of recognition errors performed Vs Trials
Figure 1 shows that with each trial, the number of errors made by DragonDictate decreased. The number of type 2 errors decreased with each trial due to those words not previously listed in the choice menu becoming candidates within the selections listed in the choice list. Eventually these words became recognized as the primary, or first selection, choices in the list. This means that they became the words that were recognized by DragonDictate as the input words uttered by the user. The other Error types became less frequent also, thus contributing to the improvement on the overall errors performed by the software.
Figure 2. DragonDictate Accuracy Vs Trials
Figure 2 demonstrates that with each use DragonDictate generally improved in its accuracy. This supports Dragon Systems, Inc.'s claim that DragonDictate performance improves with usage. The greatest degree of accuracy reached during this evaluation was 98.03%. This was achieved within a controlled environment where the user was able to control the level of background noise. During this evaluation there was very little to absolutely no background noise present. With some background noise (maintenance man drilling in the adjacent room with the door closed) DragonDictate achieved an accuracy of 93.5%.
Along with the improvement of accuracy, the amount of time required to dictate the control passage decreased (Figure 3). As shown, with each successive use of DragonDictate, the length of time required to input the control passage was reduced. This was due to the improved level of accuracy. As accuracy improved, the user was able to increase the speed at which he dictated the text. Less time was expended correcting errors performed by the software. The longest input time was 20:35 (mm:ss) with an accuracy of 72.73%, the fastest input time was 9:45 with an accuracy of 98.03%.
Figure 3. Accuracy Vs. Input time
Navigation with DragonDictate was flawless. All that was required to ensure reliable navigation was that the program being controlled by voice was properly added to the DragonDictate program group in windows. The technique for doing this is described in the User's Guide [Ref. 1: p. 18]. It is also necessary to ensure that the program being controlled is placed within the group and properly named. For example, Wordperfect 6.1 need only be named Wordperfect, while Lotus 1-2-3 may still be named Lotus 1-2-3. Other non-supported programs may still be controlled by training the name of the program. For example, Matlab is not supported and therefore is not part of DragonDictate's vocabulary. It is therefore required that the user train this particular word in order to start the program by voice. However, it is not necessary to train any of the commands within the menus of non-supported programs. DragonDictate is capable of tracking all of the commands within the menu and many of the button controlled commands as well.
DragonDictate performed very well as an input device for the Windows operating environment. As a dictation input into word processing software and in conjunction with Matlab it proved to be outstanding. After some continuous use the software was able to adapt to the user's speech patterns and was able to improve accuracy to 98.03% within a quiet test environment. DragonDictate maintained an accuracy of over 90% in a noisy environment. The noisy environment was caused by a maintenance man drilling into a wall adjacent to the lab in which the evaluation was being performed. As a navigational input for Windows it performed equally well, though more work was required by the user in order to ensure that non-supported program applications were able to be initiated by voice. This is described in detail in Appendix C.