How to: Validate Tuning Changes
This content is no longer actively maintained. It is provided as is, for anyone who may still be using these technologies, with no warranties or claims of accuracy with regard to the most recent product version or service release.
Use Re-recognizer to determine the benefit results from tuning changes in Speech Server before deploying those changes to a deployed system. Use this tool to run log data against modified recognition settings or against updated grammar files and then view reports to see details about how the tuning changes improve recognition results.
To enable the logging of grammars for use in Analytics and Tuning Studio, enable the logging of call analysis events. For more information, see How to: Enable Event Tracing of Call Activity.
Consider using Re-recognizer to analyze changes made after running Grammar Tuning Advisor.
Note
A user must be a member of the Administrators group to use Re-recognizer. If a user is not a member of the Administrators group and tries to run Re-recognizer, the user might receive a message like this: "Remoting call invoked by invalid user {user name} (expected Network Service, Local System, or an administrator)."
Note
Grammars might not be available for some turn instances in the database, because a grammar is logged only on first use, when it is cached, and not generally thereafter until a service recycle. If the log file containing the grammar is not imported into the same database, grammars for turns associated with the grammar might appear in the Validation Re-recognition Settings dialog box as incomplete, or Re-recognizer or Grammar Tuning Advisor might skip the turn, displaying the text "Re-recognition cannot occur without any grammars" in the Output pane. To ensure that re-recognition cycles take place only on grammars that are complete in the database, use the Custom SQL query "Finding Turns for Which a Complete Grammar Is Available in the Database." For more information, see Custom SQL Queries.
Validating Tuning Changes
To run Re-recognizer
In Analytics and Tuning Studio, open a view, and then select a particular set of calls.
On the Analytics and Tuning Studio toolbar, click Re-recognizer.
In the Validation Re-recognition Settings dialog box, select the appropriate settings.
For more information, see Validation Re-recognition Settings Dialog Box.
Click OK.
When the cycle is finished, click Display Recognition Results Comparison in the Output pane.
For more information, see Re-recognition Results Comparison View.
Note
If a re-recognition cycle fails to complete because of a timeout or other error, the obtained results are saved to the tuning database and can be appended to later. To append to the original incomplete results, run the cycle again using the original cycle name.
Running Multiple Cycles
It is possible to run two re-recognition cycles in parallel from Analytics and Tuning Studio. The number of simultaneous recognition cycles is limited to two. To execute two cycles in parallel, simply begin a second cycle when the first is under way.
Do not attempt to start more than two cycles. The limit on two simultaneous recognition sessions results in skipped recognitions and incomplete results.
Selecting Only Turns Containing Audio
To select only turns for which audio is available in the database, the following should be added to the Custom turn SQL filter.
TurnInfo.SpeechRequestId IN (SELECT EngineRecoAudio.SpeechRequestId from EngineRecoAudio WHERE EngineRecoAudio.SpeechRequestId IS NOT NULL)
See Also
Other Resources
Speech Application Analysis and Tuning
Speech Recognition Tuning