Tuesday, December 30, 2014

How to Make a Gold Duolingo Tree

Finishing Duolingo isn't enough; you want all the skills to be "gold," and you want them to stay gold. In this post, I'll talk about how to "regild" a completed tree, what to do about weak skills if you're half-way through a tree, and how to keep a tree gold from the very start. I'll also discuss how to do this in a way that actually teaches you something--it shouldn't be an exercise in aesthetics!

Those who are impatient can just skip straight to the end and read what I recommend as the best practice. For everyone else, I'll explain what it is we're talking about and how I reached the conclusions I did.

The Decaying Tree

When you first finish a skill, Duolingo turns its icon gold, and five "strength bars" (like on your phone) appear beside it. If you don't do regular "strengthening" exercises, though, the strength bars will gradually weaken from five to one, and at anything less than five, the skill appears in its original color--not gold. This is what we mean by "strong skills" and "weak skills." "Regilding the tree" means doing enough exercises to turn the whole tree gold again.


People who do new lessons but never do review lessons get used to the way a wave of color chases them down their tree. Those who took the placement test and skipped over a large part of the tree are often stunned when a huge slab abruptly turns color all at once. In both cases, Duolingo is sending a message that those are skills that ought to be reviewed. This post is for those who want to do something about weak skills and who want to enhance their language learning at the same time.

Why Bother?

As Duolingo warns, if you don't review what you learned, you'll start to forget it. Lots of people eventually abandon Duolingo because they get about a third of the way down the tree and then they have terrible trouble finishing new lessons, due to the fact that they've forgotten too much of the earlier material. Doing regular review lessons spares you from this.

Duolingo has a decent algorithm that tries to target the things you most need to review. Done right, the review lessons really shouldn't end up being a waste of time--they should help you solidify your understanding of the language. And they often contain material you didn't see when you did the skill for the first time.

Two Kind of Strengthening Exercises

To reverse this decay process, you can either do a skill-specific strengthening exercise or a general strengthening exercise. To do a skill-specific one, you pick a weakened skill and click on its icon. On the right, you'll see a colored box showing the strength of the skill, and inside that box is a button to strengthen the skill. Press that button and Duolingo gives you a review lesson targeting the skill as a whole. That is, it doesn't just repeat one of the lessons--it asks you questions drawn from all of them.

Because it only takes 17 correct answers to complete a lesson, and because Duolingo usually has far more than 17 possible questions per lesson, when you do a review lesson you will frequently see sentences you never saw when you did the skill in the first place. Usually not entirely new words, but you'll definitely see new ways to use the old words.


Depending on how weak the skill is and how big it is (i.e. how many words it covers), a single strengthening exercise may not be enough to make it gold again. Repetition may be necessary. On the other hand, occasionally you'll strengthen some entirely different skill that just happens to share a few words with the one you were trying to strengthen.

To do a general strengthening exercise, you go back to the main menu and click the blue "Strengthen Skills" button on the far right of the screen." This brings up a review lesson that has a random mix of things you have already studied but that Duolingo thinks are weakest.


Because the benefits of the general exercise are spread across the entire tree, it usually strengthens anywhere from zero to three skills--some people have reported as many as 13.

In short, your tree weakens over time, and you have to do strengthening exercises if you want it to stay (or become) gold. The question is how often to do them and what kind to do. To answer that, we'll start with a look at Duolingo's algorithm--what's known of it, anyway.

How it Works

Duolingo does not publish much about the algorithms they use for weakening and strengthening, but they did make at least one posting with hints about it. Further, a Duolingo user with the handle pinkodoug has made an extensive study of the Duolingo JavaScript and he has some plausible ideas about their algorithms for strengthening and weakening.

Drawing from those sources plus my own experiments, here's my best guess as to what they do. (N.B. these are my conclusions, so if they're wrong blame me--not the sources above.)

To Duolingo, strength is about words, not skills. Strong words have a score of 100%, and they decay over time until they reach zero. All words are decaying all the time, but some words decay faster than others.

You can actually see this if you click on the "Words" tab at the top of the home screen.


This shows all the words that Duolingo thinks you've studied so far. It tells you how long ago you practiced each one, and it gives each word's current strength using a bar system that goes from 1-4 (not 1-5 the way skills do). If you sort by "Last Practiced," you'll see that the older words tend to be weak while the latest words are almost all 4-bars strong. This is because all words weaken over time, but, as I said before, some words decay faster than others. We'll get back to this last point in a minute.

Every skill is associated with a set of words--probably the same words that you see in the individual lesson descriptions. Whenever those words have an average strength of 2.5 bars or more, the skill is gold. Otherwise, it shows the original color, and the skill-strength declines based on some function of the average word strength.

Whenever you do a review lesson, it targets some number of your weakest words for review. A skill-specific review only picks words from that one skill. A general review picks words from every lesson you've completed so far. Otherwise they're the same--with one important exception: in the tips that Duolingo offered, they said that skill-specific strengthening turns your tree gold faster than general strengthening does.

As long as you complete the review lesson, the targeted words all get set back to four bars whether you get them right or not. The catch is that getting them right makes them decay slower in the future. Getting them wrong or using the hints makes them decay faster. This is why some people report making tremendous efforts to strengthen every single skill only to watch them start decaying again a few days later.

The fact that some people seem to keep their trees gold without much effort suggests that there actually is an optimal way to do this. To determine what that might be, I did a few experiments--with help from a friend.

Experiments

Greg's French Tree

For the past twelve months, I have been slowly going through Duolingo's French tree, following a rule of doing two review lessons per day followed by one new lesson. In that time, I have completed most of the tree, and it has never weakened. That is, although an individual skill has weakened two or three times, I simply strengthened it specifically and thus ended the day with a solid-gold tree. During that time, I had three different week-long vacations during which I couldn't do lessons. Even so, when I got back, the tree was still gold. Others have reported similar experiences.

This suggested to me that it is possible to have a durable gold tree without a whole lot of work. That is, it seemed that as few at two general strengthening exercises per day might be enough, if done daily for a long enough time.

Naomi's Italian Trees

My friend Naomi in New Zealand was interested in exploring this, and she agreed to help do a few experiments. She had finished her English-to-Italian tree (aka the "forward tree") in just a few months and wanted to gild it. She had also just completed the Italian-to-English tree (aka the "reverse tree") in a single day and wanted to keep it gold.

The experiment was very simple: On her forward tree, she did two general strengthens per day. On the reverse tree, she did three general strengthens per day. Back when Duolingo allowed you to fail, she would persist until she completed that number, although, as it happens, she rarely failed an exercise.

Here's what happened.

With three general strengthens per day, the reverse tree got worse for about 40 days, leveled off for about a month, and then started improving in an irregular sort of way.

The forward tree decayed steadily, reaching 57 weak skills on day 91. At that point, we changed our strategy.

Starting on day 92, Naomi began doing three skill-specific exercises instead of two or three general ones. She selected the skills to strengthen by picking the weak skills highest in her tree. She applied the same algorithm to both trees.

We only have 13 days of data with the new method, but the results speak for themselves.



This graph shows how many weak skills were left each day after Naomi did her strengthening exercises. This shows that the reverse tree was 100% regilded on day 99 and that it has remained so for six days now. On the present trend, the forward tree will be regilded in two more weeks.

Obviously this strongly suggests doing the per-skill exercises rather than the general ones--just as Duolingo had advised people. In fact, the per-skill exercises appear to be more than three-times as effective at regilding. It also suggests that two exercises per day was not enough.

We adopted one additional rule: substitute general exercises for specific ones if there are no weak skills left, but never do more than two general exercises on the same day. That is, if the tree has any weak skills, then Naomi still does three strengthening exercises--padding them out with general ones as needed, but if the tree was solid gold to begin with, then she only does two generals, not three. The idea is that, like with my French tree, a durable gold tree should eventually need only two per day to maintain it.

Note: Throughout the process, Naomi reported that the algorithm seemed to have a knack for finding exactly the things she was weakest on. She felt the exercise improved her Italian enormously, even if she did spend 91 days making no visible progress on regilding her trees.

Greg's Spanish and Italian Trees

During this time, I let my own Spanish and Italian trees decay. (Both had been gold on day zero of Naomi's experiment.) Over the next month or two I'll try to reproduce the results from Naomi's trees, but I wanted to share one key result that I got on the first day:

For the best learning experience, you should strengthen from bottom-up instead of top-down.

The reason to prefer bottom up is that it drills you in the skills you most-recently learned--the skills you are most likely to need help in. You'll discover that Duolingo has a lot more material for those skills that you never saw when you did them the first time. Most people have a weak grip on the last skills in a tree, so it makes a great deal of sense to prioritize strengthening them first. In contrast, the stuff at the top of the tree is far easier to do, but it's also far less valuable to review.

The exception would be for someone trying to resurrect a long-unused tree or for someone who has become overwhelmed and is considering quitting. Those folks should strengthen from the top down because it's equivalent to starting over, in a way, without losing everything you learned the first time. (In my view, no one should ever delete their Duolingo tree and start over.)

Other Issues

I always advise people to avoid timed practices and not to use the mouse-over hints unless absolutely necessary. I'll explain my reasoning for both suggestions.

Timed Practices

You buy the ability to do timed practices at the lingot store. It's a one-time purchase, but it's still a bad deal. The way it works is that when do a review Duolingo offers you the option to do timed practice instead of the normal kind. In fact, it becomes the default.


In timed practice, you only have a few seconds before the clock runs out and you lose the round. Every correct answer puts more time on the clock, so you need to go like lightening to keep ahead. In theory, this helps you develop the skills to handle conversation in real time.

Many of the people I've talked to who reported doing lots and lots of review lessons with little progress on regilding their trees told me they were doing timed practices. I think this strengthens the tree slowly because time pressure encourages people to skip long sentences. The earlier post from Duolingo said that every word counts, so it would make sense that skipping the long sentences would really hurt you.

Given that Duolingo's new UI is much more forgiving of mistakes, it probably makes more sense to try to get the same effect by simply going through the exercises as fast as you can, accepting the fact that you'll make more errors. As a bonus, when you do make an error, you can stop and study what you did wrong. Anyway, if you want to regild your tree, you should probably stay away from timed practice.

Hints

The other thing that seems to be strongly associated with lack of success is using hints. Using a hint means doing a lesson and mousing over a word to get Duolingo to show you the definition.


Unfortunately Duolingo takes this as a hint that you don't know that word at all, and it makes that word decay faster. Do use the hints if you really don't know the word. (We want to help Duolingo figure out what you really need to study, after all.) But don't use them just because you want to be 100% sure of a word before you press return.

Summary

Plan to do three exercises per day. If you can't invest that much effort, you probably can't do this in a reasonable time. With that, here is the system I recommend for making durable gold trees with Duolingo:
  • Always do a skill-specific strengthening exercise if there are any weak skills in your tree at all. Only do a general strengthening exercise if there are no more weak skills to do.
  • If there are multiple weak skills to choose from, always pick the one that is furthest down the tree. (That is, prefer skills you learned later over ones you learned earlier.) But, if you are effectively starting over (e.g. after a long absence or because you're overwhelmed) then pick the highest (that is easiest) weak skills first.
  • For an incomplete tree, do two strengthening exercises and then do one new exercise every day. If you're not ready for a new exercise, do a third strengthening exercise instead.
  • For a complete tree that isn't all gold, do three strengthening exercises every day, using general strengthening exercises if you run out of skill-specific ones.
  • For a complete, solid-gold tree, only do two strengthening exercises per day--both of which have to be general ones, of course.
Never do a timed practice. Never use hints.

Finally, take your time. Don't try to regild your tree in a day or two of inhumanly focused effort--followed by a month of neglect. A steady effort over a long period of time is the best way to do this. You'll learn the most that way, and it won't burn you out.

If you follow this plan, assuming that you are doing your best on the exercises, your tree will eventually be "durable gold," meaning it will stay gold with minimal effort on your part, even if you go on vacation for a week or so. How long it will take to get there depends on a lot of things, but a month or two seems typical. (It depends on how well you do the exercises and on how deeply decayed your tree was in the first place.) Have at it, and best of luck!

I would love to hear from anyone who gives this a try. Leave a comment and let me know how it went.

58 comments:

Anonymous said...

This was great, thanks!

Unknown said...

Great advice and very in-depth! My tree is currently all gold but incomplete, but I've gone through the process of having to regild it several times and always wondered why the colours returned so quickly. After reading your article, it was most definitely the fact that I kept using the hints to be 100% sure (phew, am I glad that the hearts system is gone!).

Anyways, I will use your idea of 2 general strengthens + a new lesson per day as I continue with my tree. Thanks! :)

Ariel said...

Very interesting! I've been trying to guild my Spanish tree for the past few months but it's proving difficult. I will definitely try out some of your tips but I have a few questions.

First, let me give you some background. I started Duolingo nearly a year ago in February, primarily on mobile. I did the shortcut tests through all the checkpoints I could, then slowly started making my way through the last section. After a few months the previous sections ungilded and I had trouble trying to regild AND make progress through the rest of the tree. In June I finally got fed up (especially because my Spanish background was pretty good before starting Duolingo so I resented having to spend so much time re-gilding skills I already knew well) and quit the app. I came back to it in September (of course my tree was a mess by then) and finished the rest of the tree while trying to just keep the most recent lessons gilded. Currently about half of the section after the last checkpoint is gilded and the other half is at 4/5 strength. The rest of the tree still needs some work.

SO, my question is: When you say the best method for dealing with weak skills is to focus on the ones lower down on the tree, how weak do you mean? Should I be trying to gild the skills in the last section which are all at 4/5, first and then move up (which is what I've been doing)? Or should I pay more attention to the ones higher up on the tree which have 1 or 2/5 strength.

Also, I have a question about your criticism of the timed practice feature. You conclude that it's useless in terms of regilding because people tend to skip long sentences in order to beat the clock. But I've never done this.

My usual practice goes like this: I switch between mobile and laptop depending what is convenient for me on a given day. If I'm on mobile I do 2 skill specific reviews in Spanish each day and if I'm on the laptop I do a general timed practice (or more if I don't score a 20 on the first try). I was doing the timed practice because a) it gives more XP, b) it strengthens more skills on the general setting, and c) it just seemed to me that the skills I practice on mobile decayed faster--not sure if this is true or if I was just assuming that algorithm more heavily-weighted getting a correct answer in a time crunch. But anyways, when I used timed practice, most of the time I score a 20, sometimes I get 1 or 2 wrong, sometimes I run out of time. But I never skip sentences and I rarely use hints. So, do you have any other theories on why the timed practices may slow gilding?

I will try out your method of skill-specific practices but in truth, I do prefer the timed practices for the challenge. Whenever I complete a practice on mobile really fast, having known every answer instantly, I always feel a bit gypped by not be able to get extra credit for that. (Also, it makes up for the much-missed duels. really wish they hadn't gotten rid of that feature)

Also, I just started German so I definitely will taking your advice so that it stays gold the whole way. So, far I've just past the first checkpoint and it's looking good!

Greg Hullender said...

When I say "the lowest weak ones" I mean "the lowest ones that aren't gold." I don't care how many bars they have. The reason is that the later skills all make use of material from the earlier ones. The final dozen or so skills in any tree should be a fair sample of all the grammar in the whole tree. Strictly in terms of learning the language, they should be the most valuable to review. Since most people tend to rush the end of the tree, they're doubly important.

If you are only partly done with your tree, the most recently done skills are the ones for which you have had the least practice. Again, they're the most valuable to review. In this case, I suggest doing only two strengthening exercises per day (regardless of whether they actually strengthen anything and regarless of how many weak skills there are) plus one new exercise OR three strengthens, if you don't feel ready for a new lesson.

The only reason to review skills way up at the top of the tree is if you abandoned the tree for a while and you're afraid you've forgotten the material. In that case, you can either start at the very top or else start at the lowest place where you're confident you do understand the material and then work your way down.

I'm not sure why timed practice is so ineffective for keeping a tree gold. I just know that I've heard lots of anecdotal reports that it makes it easy to turn skills gold for a little while, but that they always weaken again fast.

Making a tree gold should be a means to better understanding a language. If you think the timed practice is helping you, then stick with it and don't worry about whether the tree is gold. I'm not sure, though, why you can't just do the regular, untimed practice really fast.

jmeltonga said...

Thanks for these suggestions. I find them most encouraging and have started putting them into practice tonight. About a year ago, I did as you said... I began at the top of the tree and it did help me get back into the swing again. I became discouraged after losing a streak of almost 300 days while on vacation. Recently, I have been at a stalemate in progressing on the tree. I have felt in limbo with no progress, just keeping up mostly with immersion, but with no sense of accomplishment. I think your suggestions will help me.

Simon said...

Your advice is similar to what I have been doing. I found the general strengthening a week or so after starting Italian and started using it a lot. I think I initially was doing multiple new lessons a day but I stopped that after a while. What I do now is I repeat the lesson I did the previous day, then do a new lesson and then do general practice for the rest of the hour. When I get home from work I repeat the new lesson I did earlier in the day. I usually only use the hints when I really can't remember, .

The result is that I've never had a problem with my tree degrading after returning from a month long trip, I only had the first two skills degraded. The rest remained gold. When I left I had just passed a 100 day streak.

At some point I also realised that getting things wrong is good because it helps Duolingo learn what you need to practice, so I stopped working too much about getting things exactly right everytime the time.

Caco said...

Wow. This post is amazing. I bet you had a lot of fun while doing these researchs!

In my case, I have never had lots of problems with weakened lessons, since from my first week in Duolingo, I regild every ungilded lesson. Some days it meant I didn't have to do any strenghten lessons, some days it meant I had to do 8. It's like ungilded circles rezlly piss me off so I need to get rid of them quickly.

Because of the way my mind works, I go through periods in which I'm eager conquer the tree. In theese periods, I can learn 3 new skills perd day or so, I score a lot of XP and I still regild the weakened skills. But after that I need to stop myself myself, otherwise strengthening (and learning itself) are not bearable. In these cases I set a "milestone", a meaningful skill or checkpoint in which I will stop and "take a rest" (from that point I only learn one new lesson (10 points at least) per day).

This technique turned out to work pretty well to me, but maybe it doesn't work well for others. Actually maybe my only secret is how much I hate coloured skills.

As Ariel said, I've never skipped an exercise during timed practice, but I don't use it a lot, because I play Duolingo from a tablet 95% of the times. So I'm not sure if it didn't work well with strenghtening.

What I regret A LOT now is constantly clicking on hints. Of course I click on them when I don't remember a word, but 70% of the time is to check how smart Duolingo is. I want to see if it gives the exact word I think the sentence needs, for example a perfectly well conjugated verb form or things like this.

Caco said...

I'm sorry for the typos and omitted/repeated words... I'm too clumsy when typing long texts on a tablet. Oh! And congratulations for the research and the post!

PS: I deleted and rewrote this comment becuase it also had typos.. OMG...

Libertas said...

I pretty much followed your advice not knowing there were other ways to strengthen skills. I had always strengthened specifics skills everyday before I moved on to learn a new skill.

After I finished my tree, I went back from top down until my tree was gold again. I couldn't do that every day, but most days I could. Now it seems like I only have two or three skills to review.

I didn't even know about the general skills

Ariel said...

So, you were right! Doing skill specific non-timed strengthening does regild the fastest. I think the issue with the timed practice is that while you sometimes can strengthen multiple levels or topics at once, the majority of the time you don't strengthen anything (at least that you can see, maybe because the effect is too spread out); however, with the non-timed skill-specific, you always strength at least one thing and sometimes multiple (assuming you did enough of the lesson correctly). Anyways, my Spanish tree is all gold now so I'll only have to do two strengthens a day to keep it up and I can focus on my German more. Thanks!

Elias said...

I'm surprised at your warning against timed practice. I tend to find that timed practice works the best. Is your warning only valid if I fail a number of tasks in the challenge? Or do words fade more quickly after a timed practice?

I didn't think about the mouseover hints - I'd often use it if I was unsure about the exact translation of a word, not least since English is my second language, so the exact English phrase sometimes eludes me. But I'll stop doing that.

Unknown said...

That was the exact answer I was looking for. Ty so much :)

Unknown said...

Thanks so much! I had no idea! Also, I keep clicking on the words to figure out the pronunciation. I'm guessing duolingo is taking that as a "hint" and docking ... whatever they dock. The Dutch voice sounds synthesized and is frequently difficult to understand so I like to click on the words to hear them a couple of times before moving on. Is there another way to work on pronunciation? Thanks, again!

Greg Hullender said...

When you're translating entire sentences, I don't think there's any penalty for having Duolingo play the sentence over again. Nor do I think there is any penalty for using the "slow" mode during the listening exercises. In the multiple-word-choice questions, I also don't think it matters how many times you ask Duo to pronounce the words.

What hurts you is when you hover over a word and Duolingo gives you some definitions for it. (Whether it pronounces it or not.) But in that context, you should always be able to just play the whole sentence again.

Unknown said...

This artricle was a tremendous help for me - thankyou

Unknown said...

Thanks so much, Greg!

Delle said...

Thanks for taking the time to write this well thought-out "walk through." I think duo lingo is a good language learning too and a great game for people who love word puzzles. I love testing out and yes, the more I did this the more my tree started looking like a demented Christmas tree. Its been 5 weeks now. Im almost done with the tree, and I do tend to ignore the degrading process and just move on. I can't really gild it in the time I have in a day anymore so I will try your method and will try to be patient, which is not my highest skill. There are endless new words to learn in redoing skills, but there is also a lot of boring repetition. So I wish there were a third option besides regilding individually or generally--I wish there was the opportunity to test out again! I thought about buying the timed option but I guess since it is so inefficient I will avoid that. Thanks again for the great game plan! (And yes I read the whole article top to bottom, traditionally.)

Chris said...

The dutch voice of google translate sounds more real. So you could try to type the words in there.

Unknown said...

This analysis is consistent with my experience, with one important exception. I learned French and Spanish decades ago, and have been using Duolingo for review, not for first-time learning. Therefore, it makes no difference if I strengthen skills from bottom up or top down. In fact, while in the middle of re-gilding skills from the bottom up, I've seen yet another skill decay on me more than once.

I completed each tree in 8 [sic] days, blitzing through the French tree to prepare for a suddenly-arranged trip to France. With French, I tested out of nothing, and re-did the lessons instead of doing "strengthen skills," thereby racking up a lot of XPs. I got to level 12, 58% fluency, in less than two weeks. It was a struggle to keep the tree gold because decay was massive -- Duolingo doesn't fully understand how to treat people whose initial knowledge is near the limit of what Duolingo can teach. I then took two weeks off for my trip to France. When I returned, the tree was about 60% decayed. I re-gilded slowly, maybe 4 skills a day, bringing me up to level 16 and 61% fluency. Once it was fully re-gilded, I started doing two general reviews a day, and that has, miraculously, been sufficient to keep the French tree all gold for several weeks now.

Spanish was a different story, since my knowledge of it was considerably fresher and more advanced compared to French, and still is, for that matter. I tested out of everything except past subjunctive -- I "lost heart" too often. I quickly got to level 12 and 53% fluency, from which it has been inching up. At this point about 8 skills decay per day. I'm looking forward to the day I can start doing general reviews.

In my opinion, Duolingo over-weights the amount of drill you do, measured in XPs. It under-weights prior mastery or learning achieved outside of Duolingo. My French quiz score after completing the tree was 4.64/5.0, and for Spanish it was 5.0/5.0, but Duolingo says that my French fluency is 61% and my Spanish fluency only 58%. Because I did so much more drill in French than in Spanish, my French level is now 16 and my Spanish level is 12 -- but my grasp of Spanish is much better, which Duolingo doesn't recognize.

Greg Hullender said...

Using Duolingo for review is definitely a different story. I had a similar experience to what you describe, except substitute Italian for French.

I lean toward advising people to never try to place out of anything--no matter how well you know the language. If the lessons are too easy, you'll blitz through them in just a couple of minutes each, and you'll find the material you really need to review along the way. If you place out of most of the tree, you'll have a hard time doing reviews because of vocabulary you never learned--among other things.

Another article I should write at some point is "When is it time to stop doing Duolingo and what do you do next?"

Unknown said...

After having tested out all the 64 Spanish skills but one, those skills decay, I imagine, because words that were never presented in the test-out exams are left with a strength of zero, so Duolingo has pressure to raise their strength. In my case, reviewing words that I never encountered in a lesson isn't a problem because I really do know most of them already.

Unknown said...

Thank you for this, I'm trying to keep my tree nice and golden. I didn't know about hovering over the words and that decaying them, I usually do it if i can't spell the word in English because my spelling is really bad and English is my first language.

lhp said...

I reached two gold trees, in German and Italian, more than a month ago, and have since tried to keep them gold. At first it was really hard, as all sorts of units turned colors each day, from one to four in each language, and I knew I did not have the time to keep this up. So I tried various ways, doing general reviews and specific reviews, and using vocabulary flashcards, although I hardly used them before (my aim is to keep the trees gold while spending the least time necessary, and being able to review the units that I need reviewing, rather than units that seem to turn colors arbitrarily).
And now I think I have found a formula that works for me: starting with a gold tree, I choose one skill to strengthen, then I do two or three flashcard sets. This has kept my trees gold every day for the last week (no units turning colors overnight).

Unknown said...

I have a python script which checks my Duolingo scores for me and updates a Google spreadsheet, which I run before and after each session. Since I am using the Duolingo API, I have access to more information than you can see from the interface, in particular the actual numeric score for each vocabulary word. These are decimals which seems to vary continuously between 0.0 and 1.0. (The skill scores, on the other hand, are only stored as 0.0, 0.25, 0.5, 0.75, 1.0). I add all of these up to calculate a vocabulary score, which would be equal to the number of words in my vocabulary, if everything was perfect. Of course it never is.

I also count the number of colored skills, like you did in your experiments, although I actually count the number of missing bars, to reflect that some skills are less gold than others. So, if the skill is 4/5, that is one missing, if it is 1/5, that is 4 missing.

I fell behind during my summer vacation, and my Swedish tree currently has 13 missing skill bars, out of 55 lessons, and a vocabulary strength of 1579.8/2070. The number that I really use for my goal is the deficit in the vocabulary strength; for me right now, this is 490.2. This seems to me like a real measure of how much I need to practice. I find that, although the number of skills which lose gold each day varies (at this point in my tree, it's between 1 and 5), the loss of vocabulary strength is more constant over time, generally between 15 and 20 points per day. These numbers are before my practice, which is 4 general practice, which gets me between 2 and 7 gold skills, and between 16 and 35 vocab points. The 16 is from the beginning of record keeping, about a month ago; it's almost always over 30 now; the benefits of my practice have been generally getting higher over time, and if anything the rate of vocabulary decay is going down, despite the fact that I am still doing new lessons. I expect this is partly a result of me doing better at the practice, but it also may have something to do with the changed distribution of vocabulary scores. I originally started with 4 practiced per day because that was what it took to make progress; now I could drop down to 3, and maybe even 2.

When I first started this method, I came to the conclusion that, although I can get more consistent daily progress on turning lessons gold if I do specific skill practice, I actually improve the total vocabulary score more by doing general practice, so that's what I've been doing. My justification is that, as long as the trend is for my vocab score to get closer to max, then my tree must eventually get gold too. Looking at your graph makes me question that assumption though. I may need to try going back to specific practices.

I am also learning French, although I am have't gotten nearly as far on the tree; I currently have 30 skills complete, all gold. I do one learning lesson and up to two general practices per day, as necessary to keep the tree gold. My vocabulary score is 737.8/788, which makes a deficit of 50.2. I'll repeat, this deficit is with the tree gold. My daily vocab score losses in French are between 4.3 and 6.8, substantially smaller than Swedish. However, my recovery rate is also quite a bit lower. In Swedish, I have been getting over 8 (vocab) points per review lesson, but French generally only gets about 3-4. I make a lot more mistakes in French, so that may be why. Alternatively, it may be a case of diminishing returns, because there is not so much low-scoring vocabulary to catch up on, or it may just be because I'm not as far in the French tree.

If you'd like to do some more detailed research, I could share my python script with you.

Greg Hullender said...

Wow, you really got into it! I think that's a little more work than I'm willing to do at the moment, but thanks for the offer.

Ydan Stark said...

Awesome post. I just finished my Portuguese tree and I was wondering if I should go back up or start strengthening from the beginning.

Árpád said...

Greg, your article is much-much appreciated, thank you!
It helped me a lot, because I realized, that I simply strenghtened too much, causing the tree to fade quickly. It sounds weird and absurd, but may be true: I did 10 refresh lessons a day, completing them quicky, and using the hints whenever possible. This method not only overwhelmed me, but also has a high possibility of causing the tree fade quicky, because Duo assumes that I do not know the words included.
But now I will slow myself and follow your method, and... let's see! :)

Greg E said...

Brendan, that's awesome that you have got into the API. Anything one can do to give some more motivation and satisfaction to an exercise is always good. Like charting your running times and watching them go down. Or your weight.

I have some Python skills and would love to try your script. Not sure how to proceed. I wil try to find you on Google+

Ciao, Greg E

Unknown said...

Greg E- I've shared the script on Google Drive. I hope it is documented well enough in the comments that you can get it to work.

Greg Hullender said...

Remember to report back so we know how well it worked! :-)

Greg E said...

Brendan, I have it, thanks. 500+ lines of Python, you're a dedicated hacker. Will give it a go and report back. (I'm learning Italian by the way, also attending proper Italian classes, and ALSO have an Italian Exchange student with us, from Ancona. Muliti-pronged attack). Ciao.

Carl Kruse said...

Greg, what an incredible amount of research and work on gilding the Duolingo tree. Thanks for your efforts. I'm on German level 15 but can only work on Duolingo intermittently and find little in my tree remaining gold for much time. And yes, I surely must be forgetting a good chunk of what was learned before. I've taken a few days and gone back and regilded my tree following some of the advice you've given, e.g., not looking at hints. Tree seems well lit now. ;-) And more than aesthetics, I re-enforced alot of the language.

Thanks again for your research/post.

Carl Kruse

David said...

This strategy helped me turn my tree gold pretty quickly, but it seems slow in being able to have it persist as gold. I have continued to do three lessons each day, using the general exercise to fill out the total after strengthening any areas that aren't gold. After quite a few weeks (maybe a couple months, not sure), I don't think I've had the tree gold at the start of the day twice in a row. Maybe once. It's been gold at the start of the day only three times in the last ten days.

I'm guessing that eventually the three-a-day exercises (mostly general, as usually there's just one non-gold) would lead to longer streaks of starting the day gold, but I'm wondering if it might not be more efficient to continue to do the focused exercises, rotating among the topics. That's what I'm going to try starting today.

Greg E said...

Ciao tutti. Well I have just experienced some Duo decay as per this blog. I was on the Italian tree, about 2/3 through, all gold, and neglected it for about two or three months, and most topics decayed to half strength or less. But I plunged in and did about 10 sessions per day for a couple of weeks, and have recovered. (I wish I could see more than the last 7 days of my activity, it's motivating to see a full timeline of your efforts). It seems to be staying gold pretty well, only about one decay per day now, or even less. I worked from the top down, contrary to Greg H's advice, it just seemed neater. Was weird to go back to "Ciao ragazzi" and all that early stuff.

I'm happy really to do whatever Duo asks of me. All language practice is good. If Duo makes you slog a bit longer than you expected, to get a Skill back to gold, that's ok. As Greg H says, you will get a few new words and sentences each time. I have been very pleased to find that early stuff is much easier for me than the first time. And some phrases are becoming automatic - "gli animali" - which is the payoff of repetition. [ I should underline, as I mentioned in an earlier post, that I'm also doing a normal classroom Italian course, two hours per week, so Duo is just an adjunct. ]

Message to Brendan: thanks for the Python script, I think I'm not going to have time to pursue that now, apologies. Very kind of you to provide it.

Ciao ragazzi !
Greg E

Unknown said...

I worked out a simple approach.

Click the Button/Tab labeled "Words". Now click the rightmost column heading, labeled "Strength". The words will now be displayed in decreasing order of strength. That is, weakest/most-decayed words will be on top. The words with strength less than maximum are most likely to be presented when you click the "Strengthen Skills" button on the home page. You can review those words before the lesson by hovering the mouse cursor over the word.

Every day I do as many lessons as it takes to return all words to the maximum strength level. At this point, that is about 2 to 5 lessons (20 to 50 XP) per language per day. Some words seem to be dropped from the curriculum; their strength level neither decays nor gets strengthened ("yeux" in French and "creia" and "identificacion" in Spanish). I just ignore them.

Only once did my tree depart from solid gold in the almost three months since I returned from a vacation break and re-gilded my two trees. What appears to have happened is a set of words were initially presented together in a lesson, and they all decayed together at about the same rate. I expect this to happen less and less as word strengths from the same lesson diverge as a result of typos, carelessness, haste and differential learning and forgetting.

Unknown said...

I meant to write "increasing order of strength." I don't see a way to edit the error.

Greg E said...

Well today I have finished the Italian tree ! I never thought I'd get there. Over the last few weeks I slogged along and did about 80 to 160 XP a day. You can get in the groove - do a few lessons, take a break, have a coffee, do a few more, do some work, do a few more. One thing that was very useful - the lessons on mobile DL are much easier and quicker, especially for re-gilding some thing you're not very interested in, like Subjunctive Imperfect. But I prefer to stay with the Mac web page for stuff I'm more interested in, like new vocabulary. What was also very motivating, was printing off the tree and calculating my % finished after every new line of lessons, ie. 1 to 3 topics. There are 414 lessons in the current Italian tree. That was painful to calculate.

My decay rate was quite variable. At one stage, for a week or two, nothing much decayed. Then in the space of a week, 20 lessons decayed. That was depressing. Now it's fairly stable again, decaying only slowly. I'll try this blog's suggestions on maintaining the tree.

My final thoughts on DL ? Worth doing, but you would never learn a language from DL alone. You need proper classes, and a lot of other material to refer to. And DL is very incomplete - the biggest failing IMO is that as you get to the end of the course, there are no grammar summaries - for example, nothing at all on Subjunctive as far as I recall.

Thanks again to all contributors here, it has kept me motivated.
Ciao.



Robin Lancaster said...

I would like to respectfully submit a question to be pondered. I'm not in anyway looking to stir up a nest, but am just asking an innocent question.

Is your goal to have a golden tree or to learn the language?

When I see something in my tree that is not golden, it is almost always, without fail, an area I know I am weak in. I look at it as an opportunity to work on that area. Occasionally I will even work on an area that is still golden, but *I* know is weak even if Duo does not recognize it yet.

Just something to consider from an old language learner.

Greg Hullender said...

You left a comment without even reading the first paragraph?

Greg E said...

Ciao Greg, I have a nice graph of 3 months of keeping the tree gold. Initial decay was about 4-6 lessons a day, falling to 1-3. I have just re-gilded the lessons that decayed, no other revision. Hope this image link works ...

[img]http://imgur.com/SnJGcfV[/img]

Greg E said...

Didn't work, but you can easily paste the link into a browser. (Or maybe Greg you would kindly fix it). Further notes: some of the data is a bit smoothed, ie. I was away for a few days and didn't do any DuoLingo and maybe ten lessons decayed, so I have spread them back over the preceding few days. It still seems to be decreasing slowly, heading towards 1 to 2 lessons decayed per day.

I am ahead of my regular Italian classes now, eg. we are only just starting the Congiuntivo, and we have not yet done a few tenses like Trapassato Prossimo, Futuro Anteriore, Passato Remoto, so I guess I'm fore-armed. Duo's been fun, but it inevitably has many gaps. It can only be about 1/4 at most of a serious language project, but hey it's free and fun, and you can fill time on the bus/train/airport.

Greg Hullender said...

Unfortunately Blogger makes it hard to edit other people's posts. (I'll bet that's a feature, not a bug.) But anyone can copy and paste the link. Interesting graph!

Yes, I definitely agree that Duolingo is insufficient by itself for learning a language. However, I've found that a self-taught student can organize his/her study program around it, and then it's great for generating grammar exercises.

Charlie said...

The problem I see with Duolingo's decay algorithm is that there's an underlying assumption that you aren't doing anything with the language __except__ Duolingo. But of course, once you reach a certain level, you can then have regular conversations, read novels, write a blog, etc. If you are doing that then the decay is meaningless and it may be time to quit Duolingo.

Greg Hullender said...

There definitely comes a time when Duolingo has nothing more to teach you in a given language.

Unknown said...

This is a great article! Just the help I need to keep pushing ahead with learning German ;)

Charlie said...

I'm currently trying to decide whether to give up DuoLingo for two languages: French and Italian.

In the case of French, I've spoken it for many years, have given talks in it and even taught a weeklong course in test-driven development in French. I started to use DuoLingo as one way to keep it up, since it had been a few years since I used it. I did other things at the same time, such as joining a conversational group and a reading group and I regularly read in French.

I quickly moved up the tree in French by challenging the levels but then I decided to focus on other things. Currently my tree is rather decayed and Duo says I am 4% fluent. It's a bit silly, but I am somewhat tempted to bring the tree back up in order to impress the cartoon owl character! But really, I think that would be a waste of my time and I should either ignore the apparent decay or just delete French as a language.

Italian is a bit of a different story. I'm probably only at a B2 level in Italian, but have spent long periods travelling in Italy, where I only used Italian. Besides DuoLingo, I belong to some conversation groups and a reading group and have a blog in Italian. In spite of Duo being somewhat basic, I still find I get something out of the drills. For example, I just had a good drill on adverbs and there were a few whose meaning I was shaky on. They don't come up in my conversations precisely because I'm shaky on them. In this case, the decay of the tree actually meant something important for me.

I think the general rule is to keep using Duo - or any tool - so long as it gives you benefit, and preferably more benefit for your time than the alternative methods available to you.

This is as good a place as any to comment that I really like your blog and I'll probably be showing up in more comments as I look at some of the old posts that interest me. As it happens, I live in Poulsbo, so we may run into one another in the real world some day. For now, you can visit me at braniitaliani.wordpress.com.

DownhamC said...

Many thanks, Greg, for taking the time to write such an informative blog. Very helpful and has kept me encouraged.

Rafael Marchioli Bernardes said...

Very good post! Thank you!

Kags said...

An interesting article.
Has anyone tested whether there is a difference in the persistence of strengthening via the app or via the website? My perception is that using the web, skill-specific strengthening exercises maintains a skill at gold longer than doing the equivalent on the app? I certainly _feel_ as if I've put more effort in (more typing full answers, rather than just drag and dropping words, or choosing pairs), and it seems as though things are staying gilded longer.

Richard said...

Wonderful post Greg H. Thank you.

Unknown said...

I'm doing the challenge in habitica :) thanks for this post. Very helpful.

Greg E said...

Hi all, well I've regilded my tree yet again (Italian from English), a few weeks ago. This is the third time, ie. the second re-gild. Wasn't hard this time, I seem to have learned much of the grammar & vocab fairly well and can race through it. This time the tree is decaying very slowly, only a lesson every few days. But maybe I re-gilded so quickly that it's all at the same level, and it's all going to decay in a huge bang ! I think I've reached the level now where DL can't help me much, although it's a small motivation to keep it gold. I need conversation now, I'm running a conversation group in Meetup, and Skyping with my old exchange student in Ancona, and still going to my regular classes, and I might try a lesson from the tutors on iTalki. CIao.

Rhubarbia DeVil said...

Thanks; this article is really useful. My Spanish tree is full and I re-guild every day and it does feel under control. However I can't get above 19% fluency no matter how I try. I wondered if it is because I can't type the accents over letters when I'm using my mobile. Duolingo gives me a tick but grumbles about this and I don't know what effect it has.

healigans said...

Extremely helpful. Thank you for taking the time to share the information. I was becoming quite dispirited with the 'unwinding' of my skills, which I believed to be solid. Now I understand what is happening and what you suggest, I am feeling purposeful and directed. Thank you again. Happy learning.

Geoff Hom said...

Super helpful. Thank you. I fell into the "Strengthen skills" trap; silly me for trusting DuoLingo's UI. The ongoing debacle that is Strengthen Skills makes me wary of DuoLingo. But they're free and I respect their purpose.

AKJackson said...

I seriously almost gave up on duolingo. I took pride in my gold lessons but every time i tried to move forward to the next lesson, a few more past gold were now not. I felt compelled to complete those first and in the end i was stalled.

Thank you so much for this post as it's the only reason why i started up again.

I muscled through a day of getting full gold again (only about half of the full japanese course) and then did as suggested with 2 strengthening lessons and at least one new lesson. Now i only ever have maybe one non gold while most of the time it's just the new lessons needing attention.

Seriously dude, Thank you!

Greg Hullender said...

I'm glad it still works! And I'm very glad it worked for you.

Gerard said...

Thanks much. I am going to work my bottom up now until its all GOLD. then strengthen bar. what if I have some weak ones up top manana. do I redo those then go back to the last???