Will AI Help You Be Fluent In French?
Watch my mind-blowing experiment
When I saw the Matrix way back in 1999 and saw Neo being uploaded with martial arts skills all I could think was “I would upload every language on the planet.” It would be its own superpower. Every African dialect? Cantonese? All 800 languages they speak in Papua New Guinea? You could travel anywhere and have in depth conversations with the residents.
If you are someone who is dreading the new AI age, you need to consider what AI is about to do for us and languages, including instant translation and dubbing which uses your own voice.
We’ll begin with the new abilities of text-to-voice generation. We all remember the weird robot voice that these apps used to create - they were grating and unpleasant to listen to. I just uploaded the paragraph above to Eleven Labs and gave it a sample of my voice. Here is the result (again, this is not me but an AI version of my voice.)
Whoa. Pretty impressive.
I then tried the video translation software HeyGen. I was interested in the technology, but also I was really curious to hear myself speaking in French. Since we live part time in Montpellier I hear myself speaking halting, mediocre French on a regular basis, but what would I sound like if I were fluent? Would listening to my own voice speak the language help with my accent and confidence? I hoped so.
I took a 30 second video as I walked through the streets of Montpellier.
This YouTube video walked me step by step through the translation process using either Eleven Labs or HeyGen. Eleven Labs translates your voice, but HeyGen will take a video of you and not only dub your words, it will alter your lips to match! Bonkers. Here is my first attempt:
You will see that there are glitches. At the 00:24 mark my voice cuts out completely. Perhaps because I turned away and my mouth was not visible to the camera? Also, the synching of my lips is okay, not great. I also asked a native French speaker to watch it and she said that the beginning sounds like “French French” but after the glitch it sounds like “Canadian French,” and then it switches back to “French French.”
All of these things aside, it is still F***ING UNBELIEVABLE. The voice sounds very much like mine. It will certainly only be a matter of months before the software is nearly perfect.
Because of the glitches, I decided to try filming at home, staring straight at the camera without any wind or background noise.
This time I chose to speak in Italian:
Much better than the one I did outside! Plus, my friend Christopher says that the Italian is “100% perfect in terms of grammar and pronunciation.”
Roberto and I work on our French daily, and I am determined to be fluent regardless of what AI can do. It’s good for my brain and I enjoy speaking it. However, if I could wear a pair of glasses that would instantaneously translate for me, I would start going to French plays and films immediately instead of say, in five years (thanks to @PRANATHFERNANDO for sharing the glasses news).
I suspect instant translation will be a game changer for immigrants and refugees in new countries. Think how much more quickly they will be able to find employment. When people were fleeing Ukraine relief workers in Poland and Germany were desperate for translators to help people get resettled. Translators lives were endangered by helping American soldiers in Afghanistan. I also believe that if soldiers could understand the residents of a foreign country it might not be so easy to see those people as “other.”
For you, our readers, I think the take-away is that if you are considering moving to France but the language barrier is holding you back, you should stop panicking. In no time at all you will be able to easily order your steak frites in perfect French or any number of other languages.
Jusqu’à la prochaine fois (until next time)
Carolyn & Roberto
This is totally brain-popping! Way to go, Carolyn, making friends with our new life partners - AI, that is. And getting totally convincing Italian on the way. Very fun.
very cool!