Use Custom Pronunciations (.NET)

There may be some scenarios in which you can improve the performance of speech recognition in your applications by specifying custom pronunciations for words in your application's vocabulary that are uncommon or whose pronunciation is unusual. Unusual words for which you may want to create custom pronunciations include proper names, place names, fictional words, slang, or words that are specific to an educational or medical discipline. Using the information and examples in this topic, you can create custom pronunciations for the specialized vocabulary in your application.

When to Create Custom Pronunciations

You create custom pronunciations to improve the accuracy of speech recognition for vocabulary in your application that the speech recognition engine does not interpret well. Typically, you will only need to create custom pronunciations for words that are not common to a language and that do not follow the typical pronunciation rules for the orthography of a language. The Microsoft speech recognition engine is well equipped with tools that enable it to determine the correct pronunciation of words it is familiar with, as well as for words it has never encountered.

Speech recognition engines include a default lexicon that specifies which words can be recognized, and how each word must be pronounced to be recognized. The scope of a speech recognition engine's lexicon is typically a single language, and the lexicon contains a large number of words that the speech recognition engine can recognize in that language. This provides the speech recognition engine with a substantial native vocabulary.

In addition to its lexicon, a speech recognition engine also has an acoustic model that describes the sounds of a language, and a language model that describes how the sounds of a language can be combined into meaningful phrases. This gives the speech recognition engine an understanding of its language that transcends the words in its lexicon. The speech recognition engine uses this understanding to create pronunciations for words it encounters that are not in its lexicon, and it does this almost instantaneously.

However, if your application includes words that feature unusual spelling or atypical pronunciation of familiar spellings, then the speech recognition engine may not create the pronunciation that works best for your application. In these cases, you can specify a custom pronunciation that may improve the recognition accuracy for the specialized vocabulary in your application. It is important to test the performance of your custom pronunciations to confirm that they provide an improved speech recognition experience for your intended audience.

Methods of Incorporation

You can create custom pronunciations either inline in a grammar, or in a lexicon file that a grammar references. A lexicon is a list of words together with their pronunciations. When deciding whether to implement custom pronunciations inline in a grammar or in a linked lexicon, consider the following:

• Custom pronunciations specified inline in grammars apply only to the single occurrence of a word in the grammar.

• Custom pronunciations specified in lexicons apply to all occurrences of a word in a grammar.

• A lexicon linked from a grammar is only active while the grammar is active for recognition.

When deciding which pronunciation to use for a word or phrase during speech recognition, a speech recognition engine looks for pronunciations at the following locations in order:

1. Inline in grammar documents

2. In lexicon files linked from a grammar document

3. In the speech recognition engine's internal lexicon

If there are custom pronunciations specified for the same word both inline in a grammar and in a linked lexicon, the speech recognition engine uses only the inline pronunciations. Similarly, if there are custom pronunciations specified in a lexicon linked from a grammar, the speech recognition engine uses those pronunciations instead of, not in addition to, the pronunciations given in the engine's internal lexicon.

If the speech recognition engine does not find a pronunciation for a word, either in grammars or lexicons to which it currently has access, it will create a pronunciation using the rules of its language model and acoustic model.

You can determine the pronunciation that the speech recognition engine associates with a phrase using the EmulateRecognize or EmulateRecognizeAsync methods of the SpeechRecognitionEngine and SpeechRecognizer classes. You load a grammar that contains the phrase and pass in the phrase using one of the EmulateRecognize or EmulateRecognizeAsync methods, The speech recognition engine outputs the pronunciation associated with the phrase in the Pronunciation property of the recognition result. This can help you to decide whether or not to provide a custom pronunciation for a phrase.

You specify custom pronunciations using characters from a phonetic alphabet. A phonetic alphabet contains combinations of letters, numbers, and characters which are known as "phones". Phones describe the sounds of speech for a particular language. Similar to those used in dictionaries, phonetic spellings describe how words should be pronounced for successful speech recognition in a specified language.

Creating Inline Custom Pronunciations

You can create custom pronunciations inline in XML-format grammars that are based on the Speech Recognition Grammar Specification (SRGS) Version 1.0. To add custom pronunciations inline in a grammar document, you use special attributes that were created by Microsoft for the grammar Element and the token Element. Remember that inline custom pronunciations apply only to a single occurrence of a word. Use the following steps to create custom pronunciations inline in a grammar document:

Create Inline Custom Pronunciations

1. Add the following declarations to the grammar element:

   • sapi:alphabet="x-microsoft-ups". This informs the grammar that you will use Microsoft's Universal Phone Set (UPS) to specify pronunciations. You will typically use UPS to specify pronunciations in US English. This attribute is case-sensitive.

   • xmlns:sapi=https://schemas.microsoft.com/Speech/2002/06/SRGSExtensions. This provides a link to the namespace that defines Microsoft's custom attributes for the grammar and token elements.

2. Create a token element that encloses the word for which you want to specify a custom pronunciation. Add an empty sapi:pron attribute.

   • <token sapi:pron=""> habanera </token>

3. Using the phone tables below , look up the phones that correspond to the sound of each syllable in your word. Use the Sample Pronunciations below to understand how to combine phones to describe the sound of a syllable.

4. In the sapi:pron attribute, enter the phones that describe the word's pronunciation. Phones are case-sensitive and must be space-delimited. Optionally use markers for syllable emphasis (S1, S2) and for separating syllables (.) to further refine the pronunciation. See Suprasegmentals later in this topic.

Here is an example of a grammar that specifies custom pronunciations inline.

<?xml version="1.0" encoding="UTF-8"?>

<grammar 
  version="1.0" mode="voice" root="sauce"
  xml:lang="en-US" tag-format="semantics/1.0" 
  sapi:alphabet="x-microsoft-ups" 
  xml:base="https://www.contoso.com/"
  xmlns="http://www.w3.org/2001/06/grammar"
  xmlns:sapi="https://schemas.microsoft.com/Speech/2002/06/SRGSExtensions">
  
  <rule id="sauce" scope="public">
    <item> Please bring more </item>
      <one-of>
        <item><token sapi:display="habanero" sapi:pron="H AE . B AX . S1 N EH lng . R O"> habanera </token></item>
        <item><token sapi:display="habanera" sapi:pron="H AE . B AX . S1 N EH lng . R AX"> habanera </token></item>
      </one-of> 
    <item> sauce. </item>
  </rule>

</grammar>

  <item>
    <token sapi:display="ストップ" sapi:pron="ストップ"> すとっぷ </token>
    <tag>cancel</tag>
  </item>

Creating Custom Pronunciations Using a PLS Lexicon

If your application uses specialized words repeatedly across multiple grammars, or if you are generating pronunciations separately from the grammar generation, you can create a lexicon that contains the words and their pronunciations. A lexicon is a separate document that you can link to one or more grammars. If the speech recognition engine loads a grammar that is linked to a lexicon, it uses the pronunciations that the lexicon contained when the grammar was loaded. Any pronunciations inline in a grammar still take precedence over pronunciations in a linked lexicon.

You author lexicons as XML documents that follow the format of the Pronunciation Lexicon Specification (PLS) Version 1.0. Use the following steps to author a PLS lexicon:

Create a Lexicon

1. Start with a new, blank XML document.

2. Enter the XML declaration: <?xml version="1.0" encoding="UTF-8"?>.

3. Enter the opening tag of the lexicon element. This must declare the single language-culture of the words in the lexicon, and the phonetic alphabet used to construct the pronunciations. Here is an example of the opening tag of a lexicon element:

   <lexicon version="1.0" 
     xmlns="http://www.w3.org/2005/01/pronunciation-lexicon"
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
     xsi:schemaLocation="http://www.w3.org/2005/01/pronunciation-lexicon 
     http://www.w3.org/TR/2007/CR-pronunciation-lexicon-20071212/pls.xsd"
     alphabet="x-microsoft-ups" xml:lang="en-US">
   

4. Add a lexeme element for each word for which you want to define one or more pronunciations:

   • <lexeme></lexeme>. The parent element for defining words and their pronunciations.

5. Within the lexeme element, add a grapheme element and a phoneme element.

   • <grapheme></grapheme>. Contains the written form of a word.

   • <phoneme></phoneme>. Contains phones that describe the pronunciation of a word.

6. In the grapheme element, enter the word for which you want to specify a pronunciation.

7. Using the phone tables, look up the phones that correspond to the sound of each syllable in your word. Use the Sample Pronunciations to understand how to combine phones to describe the sound of a syllable.

8. In the phoneme element, enter the phones that describe the word's pronunciation. Phones are case-sensitive and must be space-delimited. Optionally include markers for syllable emphasis (S1, S2) and for separating syllables (.) to further refine the pronunciation. See Suprasegmentals.

9. If you want to specify multiple pronunciations for the same word, add more phoneme elements within the lexeme element. When specifying multiple pronunciations for a word, you can designate one pronunciation as preferred by adding the attribute/value pair prefer="true" to the phoneme element, for example: <phoneme prefer="true">1 l eh d </phoneme>.

10. Continue adding lexeme elements for each word whose pronunciation you want to specify.

11. Add the closing lexicon tag: </lexicon>, and save the document with a .pls extension.

You can optionally include an <example></example> element within the lexeme element that contains a sample usage of the grapheme.

Here is an example of a completed lexicon document that specifies pronunciations for fictional names:

<?xml version="1.0" encoding="UTF-8"?>

<lexicon version="1.0" 
  xmlns="http://www.w3.org/2005/01/pronunciation-lexicon"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
  xsi:schemaLocation="http://www.w3.org/2005/01/pronunciation-lexicon 
  http://www.w3.org/TR/2007/CR-pronunciation-lexicon-20071212/pls.xsd"
  alphabet="x-microsoft-ups" xml:lang="en-US">

  <lexeme>
    <grapheme> Klhtr </grapheme>
    <phoneme> K L EH . S1 T AA R </phoneme>
  </lexeme>

  <lexeme>
    <grapheme> Eanor </grapheme>
    <phoneme> S1 I . AX . N O R </phoneme>
  </lexeme>

  <lexeme>
    <grapheme> Puntahrik </grapheme>
    <phoneme> P UH N . S1 T AA . R IH K</phoneme>
  </lexeme>

</lexicon>

See Pronunciation Lexicon Reference for a summary listing of the elements and attributes used in creating lexicons, including information and examples related to creating lexicons for System.Speech.

Link the Lexicon to the Grammar

Now that you have created your lexicon, you must link to it from your grammar, using the following steps.

Link a Lexicon to a Grammar

1. After the opening tag of the grammar element, and before the first rule element, enter a lexicon element: <lexicon />

2. Add a uri attribute to the lexicon element and enter the path and name of the lexicon file. For example: <lexicon uri="c:\MyLexicon.pls" /> OR <lexicon uri= “https://contoso.com/lexiconstore/MyLexicon.pls”/>

The following is an example of a grammar that references the lexicon of fictional names shown above:

<?xml version="1.0" encoding="UTF-8"?>

<grammar 
  version="1.0" mode="voice" root="warriors"
  xml:lang="en-US" tag-format="semantics/1.0" 
  sapi:alphabet="x-microsoft-ups" 
  xml:base="https://www.contoso.com/"
  xmlns="http://www.w3.org/2001/06/grammar"
  xmlns:sapi="https://schemas.microsoft.com/Speech/2002/06/SRGSExtensions">

  <lexicon uri="c:\test\Warriors.pls" />
  
  <rule id="warriors" scope="public">
    <item> The warrior's name is </item>
    <one-of>
      <item> Klhtr </item> 
      <item> Eanor </item>  
      <item> Puntahrik </item>
    </one-of>
  </rule>

</grammar>

UPS Phone Tables

The following tables contain the most commonly used phones for US English from Microsoft's Universal Phone Set (UPS).

Consonants

The following table lists the most commonly used consonant phones for US English from Microsoft's Universal Phone Set (UPS).

Vowels

The following table lists the most commonly used vowel phones for US English from Microsoft's Universal Phone Set (UPS).

Suprasegmentals

Suprasegmentals are optional markers that indicate the division of a word into syllables, identify which syllables receive emphasis (stress), and specify the length of syllables. The following suprasegmental markers of the UPS are commonly used to create pronunciations for words in US English:

Sample Pronunciations

The following table contains a sampling of words in US English and their phonetic spellings (pronunciations) using UPS phones. Use this table to understand which phones represent sounds in US English with which you are already familiar, and as a guide to combining phones to create syllables and words.

See Also

Reference

EmulateRecognize()

EmulateRecognizeAsync()

EmulateRecognize()

EmulateRecognizeAsync()

Concepts

grammar Element

token Element