I have written before about my lazy and largely unsuccessful journey to become a polyglot. (TL;DR Duolingo doesn’t work, LanguageTransfer is cool, and the only thing that makes a difference is crossing the cringe minefield and talking to people before you know how to talk to people.)

I stand by what I said, but talking to people is, like, a lot of work and super embarrassing. Here’s a way to practice your foreign language of choice that doesn’t require talking to people, in a way that feels a lot like talking to people, because you are talking, and getting natural language replies, and you have to understand what’s going on.

The system has two parts (in the future, I’d like it to have three):

1. Speak at your local AI in a foreign language
2. It replies in text in the foreign language, correcting your grammar
3. (Coming soon: Moonshine STT reads it aloud)

Part I: Dictation in your target language, with Handy

I wrote about how handy Handy is. But what’s extra handy about it, is that it’s multilingual, and it can even translate between languages!

Although it’s forgiving, you do have to get the pronunciation close enough for it to be unambiguous.

1. Install Handy (or any other equally good multilingual dictation tool)
2. In the settings, change the model to Canary 180M Flash. I have found that this is the sweet spot for a speech-to-text model that runs in near realtime, and is multi lingual and accurate enough for the purpose. If your target language isn’t supported by Canary 180M Flash, or you have a more or less powerful laptop, the optimal model might be different.
3. Set the language (in the General tab) to your target language; in my case, Spanish.
4. Set a hotkey, if you haven’t already.

Test this set up by opening a text editor, pressing your dictation hotkey, and transcribing some foreign language speech.

Part II: The Language Teacher, Powered by Llama.cpp + Qwen {#part-ii-the-language-teacher-powered-by-llamacpp-qwen}

Follow my tutorial to install Llama.cpp and Qwen. For language learning, any modern model will probably work okay. So you could use Ollama, or Jan.ai, or (if you wanna be uncool) some free vaguely-privacy-respecting cloud model like Brave AI or Lumo.

Putting it all together!

Fire up Handy so that’s running in your tray, and start your llama.cpp server (or AI chat of choice).

The third ingredient is the prompt. We don’t want to just say, talk to me. Because it’s much more fun to try to learn something we want to learn anyway. The foreign language is just something that’s getting between us and what we want, so we won’t get obsessed with grammar, we will focus on understanding, and being understood!

Despite running several math websites, I kind of suck at doing arithmetic in my head. I also … wasn’t good at counting in Spanish. So my starting prompt was this:

I’m learning Spanish. Please have a conversation with me in the language. I want to learn basic mathematics, especially how to do arithmetic (multiplication etc) in my head and on paper without a calculator. Teach them to me step by step, in Spanish, with quizzes and exercises. I will reply in Spanish. Below your response in Spanish, include corrections to my Spanish usage, and a brief English translation of what you said (esp any difficult words that haven’t been used in the conversation before)

Working Around Context Length Limitations

Depending on how powerful your system is, there likely won’t be enough context length available for more than a few near-realtime conversations. My solution is a prompt specifically for saving the current state of the lessons, so I can start where I left off in a fresh session:

Write a careful, thoughtful lesson plan, including what we have covered so far, all the way through to fluency in both Spanish and Math. Include a list of words that have been introduced and I have learned to use, as well as a list of key terms and mathematical concepts that I have not yet learned. Make it concrete, so it is easy to start where we left off, even without access to the chat history, without repeating or skipping over any material. The idea is that each session, I paste the lesson plan as the starting prompt, do a bunch, and then have the AI update the lesson plan, and start fresh the next day, starting from just the lesson plan.

But we’re missing the best part!

You’re right. The hardest part of learning Spanish isn’t speaking it, it’s understanding it. People speak so fast, and all the words run together!

For a long time, I’ve been trying to get a full 3-part system working, without luck. I have at last succeeded! (It’s still a bit janky.)

Stay tuned for the sequel to this post, Immersive, Fully Local Speech-to-Speech Foreign Language Learning in ~30 lines of Bash.