Adding pause to speech?

[kencmds]

I am building a notes app, and we’d like to add the ability for the app to read the notes back to the user. My approach currently is to loop through the notes and add the note content to a string, and have the app read back the contents of the string. The only issue is that speech does not take much of a pause between sentences, so it becomes a little difficult to know where one note ends and the next begins. Is there a way to introduce “silence” between words? I’ve tried inserting a period and a comma between notes but I didn’t really notice any effect.

SDK 44, iOS/Android.

[regex]

What package are you using to implement text to speech?

By the way, right off the bat, I can think that instead of turning all notes in a string you could turn each note in a string and put them in an array, then make the reader loop the array and give a manual timeout between a speech and the next.

in pseudo js it would be

var notes = ["note 1", "note 2", "note 3"]

notes.forEach(function(note, index) {
    setTimeout(function(){
        // Text2SpeechPackage.read(note)
    }, ((1 * index ) / index) * 100); // not sure, but this should block the foreach for 1 second
});

or

for (var i = 0; i < notes.length; i++){
    // Text2SpeechPackage.read(notes[i])
    await timer(3000); // this is apparently from ES7 onwards
}

[kencmds]

@regex I am using expo-speech.

Thank you for the looping suggestion. I did try that, however I did run into difficulties allowing the user to stop the audio - I found it complicated to break the for loop. I still don’t have an implementation of it that works smoothly.

[ccheever]

I did some Googling and it seems like there is no pause API (at the library level) for TTS on Android, so it’s hard to do much better than feeding the TTS engine one word at a time, and pausing if someone hits pause.

You can break out of for loop in JS with break

> for (let i = 0; i < 1000; i++) { if (i > 3) { break; } console.log(i); }

0

1

2

3

undefined

>

[kencmds]

@ccheever appreciate you looking into it. In my experience, when i do a for loop through an array and have each item spoken aloud, the entire for loop executes instantly. So when I stop the speech with .stop(), that will stop the item that is currently speaking, but calling break() at that time doesn’t stop the loop - the rest of the items will then be spoken unless i hit stop for each individual one.

Hence, why I was wondering if I could add characters that would create some silence between the spoken text - but I haven’t had any luck in that regard so far.

[krschacht]

@kencmds If you’re seeing the entire loop execute immediately, you’re probably missing an await somewhere.

Or, instead of doing a simple loop, you could do text-to-speech on the first note and connect a callback which gets fired at the end of that speaking. expo-speech has an onDone event. When onDone gets fired for your first speech, then queue up speaking for your next note.

Alternatively, you could use another text-to-speech system. Most of the big ones (Google TTS and Amazon TTS) support SSML, which is a markup language for speech. SSML includes “pause” tags called <break. e.g. 音声合成マークアップ言語（SSML）  |  Cloud Text-to-Speech のドキュメント  |  Google Cloud

[kencmds]

Thanks everyone for your help. I think I solved this. When a user clicks the play button, I do a for loop like this:

for (let i = 0; i < notes.length; i++) {
        await playSound(notes[i].content, i);
        if (!notesPlayingRef.current) {
          break;
        }
      }

And the playSound function returns a promise that resolves after the audio is done (the content of the note is read aloud, then a tone plays using expo-av). Function goes like this:

  const playSound = (noteContent, index) => {
    let delayTime = 200;
    if (index === 0) {
      delayTime = 0;
    }
    return new Promise((resolve) => {
      setTimeout(() => {
        if (notesPlayingRef.current) {
          Speech.speak(noteContent, {
            onDone: async () => {
              if (index === notes.length - 1) {
                setNotesPlaying(false);
                resolve(true);
              } else {
                if (notesPlayingRef.current) {
                  sound.setOnPlaybackStatusUpdate((status) => {
                    if (status.didJustFinish) resolve(true);
                  });
                  await sound.replayAsync();
                }
              }
            },
            quality: "Enhanced",
          });
        }
      }, delayTime);
    });
  };

This approach seems to be working. If anyone has suggestions for doing this better, please let me know - thanks again.