Saturday, December 19, 2009

My Little Contribution to Language Learning Reader screenshot
I’ve been working on a “secret” project that may help people learn languages. It provides a way to listen to audio while reading along with the corresponding text. As the audio plays, the corresponding section of text is highlighted.  You can also click on any section of the text to immediately begin playing the audio from that point.
My dream is that all language learning podcasts will race to make their content available in this way.
Listening and reading are critical components of language learning. But like oil and water, they’re not easy to combine, so each is generally practiced in isolation. Try listening to an audiobook while reading along with the real book.  What if you want to hear the last sentence again?  It’s a pain to rewind the audio.  What if you want jump to another part of the book?  What if you just listen for a while, but then need to refer to the book again later?  It’s frustrating because the audio and the book are not connected in any way.
But what if you could combine the two?
Peanut-butter jelly time! Embedded here is an example of a very short Chinese lesson (thanks ChinesePod). Try clicking the play button, or anywhere on the transcript:

I have scaled this up to entire books! Click here to see the full book, Alice in Wonderland, in all it’s English glory: Reader screenshot
(text from Project Gutenberg, audio from LibriVox)

  • Instantly accessible from any web browser -- no special software to download/install.
  • You can still listen without reading, or read without listening. When you want to listen to a given part, or read a given part, you don’t have to struggle to find the right place in both the audio and text.
  • Since you’re reading along in a web-page, your favorite pop-up dictionary will continue to serve you!

The intent is that anyone can create their own synchronized text/audio to share with the world.  The player is just a tool that can play back an audio file with a given transcript. Anyone can host audio and transcripts on their own website, and use the player to present them together. The player can also be embedded in a webpage, as demonstrated with the Chinese lesson above.
Why would any language learning podcast NOT make their content available in this fashion?

What’s Next
  • Connect with other people who are interested in seeing this type of idea come alive.
  • Make existing content available in this format (like synchronizing content from LibriVox,  Project Gutenberg).
  • Gather feedback from users to see what kind of crazy things become possible when you have text and audio aligned (e.g. easily generating flashcards with text and audio).
I welcome your feedback!  And feel free to notify your favorite podcast if you’d like to see their content available in this type of format.


  1. Hi Jim,
    As a matter of fact, you've been making a little yet BLESSING contribution to people who are learning language, as long as they are able to catch the idea behind this tool. And I am the one who not only understands this idea but also has been practicing "secretly" (I mean alone) this method. I also made use the two big public sources you mention above for my study material, but synced these two materials (audio from LibriVox and text from Project Gutenberg) myself :'(. It is easier but not less portable when using laptop, and harder and portable when using hand phone. What type of synchronization I made an effort is, I bet, a manual and effortful syncing ^_^, yet the advantage is I know exactly what and why I am doing for and know how it works for my language learning. If I am not wrong there is a website freely offering the service like yours, but I forget it :).

    Your little idea and its amazing realization is a blessing thing in language learning. Although I just want to add a point that I think it is invaluable thing in learning something. How good the software was, but if users do not know the main reason the software was made and why each feature given on it, it's useless. Just like you said on "What's next" on this page, read: "Connect with other people who are interested in seeing this type of idea come alive." I hope everyone can some day find the same, perfect amazing idea behind

    Finally I want to say that actually I've been looking for such idea be realized (for free is better ^_^), and I've already come up with

    Thank you so much

  2. Hi Rivo,

    Great to hear from you, thanks so much for your comment.

    It's great to hear you have started synchronizing content. Would you like me to post it in the DingLabs Library to share with others? I don't have a way for others to upload yet, but I can post it for you, if you would be willing.


  3. This comment has been removed by a blog administrator.

  4. This comment has been removed by a blog administrator.

  5. This comment has been removed by a blog administrator.

  6. This comment has been removed by a blog administrator.