Tools Blog Learn Quizzes Smile API Log In / Sign Up
Tools Blog Learn Quizzes Smile API Log In / Sign Up
« Return to the tutorials list
We have updated our privacy policy to let you know that we use cookies to personalise content and ads. We also use cookies to analyse our traffic and we share information about your use of our site and application with our advertising and analytics partners. By using this website or our application you agree to our use of cookies. Learn more about the way this website uses cookies or remove this message.

Sync Audio with text using JavaScript

Difficulty: 30 / 50 Tweet
sound-waves

A client of mine wanted a way to highlight text on a page as audio was playing, thus helping children that had difficulties reading along. In this tutorial I will show you the solution I implemented.

My client has some complex XML files that contain data about the media files and text, but for the sake of clarity, in this tutorial I will use a much simpler JSON file which basically contains the text from the audio file and markers that define the start and end time of a word or group of words in the audio track.

The code makes use of the timeupdate event and it uses the data from the JSON to sync audio with the text.

I am pasting the code below in case you want to use it in your own projects.

    
        <audio id="audiofile" src="//pathtoyouraudiofile.mp3" controls></audio><br>
        <div id="subtitles"></div>
        <script>
        ( function(win, doc) {
            var audioPlayer = doc.getElementById("audiofile");
            var subtitles = doc.getElementById("subtitles");
            var syncData = [
                  { "end": "0.225","start": "0.125","text": "There" },
                  {"end": "0.485","start": "0.225","text": "were" },
                  /* ... and so on ... full json is in the demo */
                ];
            createSubtitle();

            function createSubtitle()
            {
                var element;
                for (var i = 0; i < syncData.length; i++) {
                    element = doc.createElement('span');
                    element.setAttribute("id", "c_" + i);
                    element.innerText = syncData[i].text + " ";
                    subtitles.appendChild(element);
                }
            }

            audioPlayer.addEventListener("timeupdate", function(e){
                syncData.forEach(function(element, index, array){
                    if( audioPlayer.currentTime >= element.start && audioPlayer.currentTime <= element.end )
                        subtitles.children[index].style.background = 'yellow';
                });
            });
        }(window, document));
        </script>
    

Update: March 12 2018 - Sync vtt subtitles with playing audio/video file

Since this small tutorial was published in 2015, I have written a package that uses vtt subtitle files to display the text of the audio being played.

The NPM package can be installed using (npm install audio-sync-with-text --save) and then you can use it with:


    var audioSync = require('audio-sync-with-text');
    //init:
    new audioSync({
        audioPlayer: 'audiofile', // the id of the audio tag
        subtitlesContainer: 'subtitles', // the id where subtitles should show
        subtitlesFile: './MIB2-subtitles-pt-BR.vtt' // the path to the vtt file
    });

Here's a demo that shows how each of the code approaches works! Demo

comments powered by Disqus