Hey PokerDad, we’re still plugging away, to my surprise. H. continues to like the sessions, most days. We are now reviewing three sets of recordings, which last between 3 and 10 minutes apiece and average, I’d say, around 6 minutes. It seems obvious to me that my memory of the material is much superior to what it would have been if we had not done the review, but the question is whether it helps H. We haven’t done any sort of testing, however, on the material we’ve reviewed, apart from some Story of the World tests, which we do orally at dinnertime sometimes. Does H. answer the questions over material we’ve reviewed more accurately than the questions of material we haven’t reviewed? I think so, but the effect frankly does not seem to be very large. I think it’s plausible that testing memory (more than just reviewing information) might improve recall, as your study discovered. Reproducing the information is a very different activity from passively listening. Perhaps we will add tests on things from time to time. I’ve thought of doing so occasionally.
Well, for quite a while we’ve thought we’d have to change something in our memory approach, partly because I can’t see doing this over the long haul, and partly because (as I indicated just above) I’m not entirely convinced that H. is experiencing all the potential benefits of using this amount of time every day on reviving memories of things learned.
So now we’re thinking of switching from reviewing information by simply listening to my summaries, to regular quizzes. Besides, H. desperately needs a regular oral examination, I’ve thought, and this would be the perfect melding of memory work and narration/recitation. (Actually, if the answers are “short answers,†then there really wouldn’t be so much of the latter after all. But it would be good initial training for narration.)
The question is how we do this. Well, I just read a bit to H. from The Story of the World about Columbus and asked eight questions. H. seemed to like answering the questions; he got, I think, 6 of 8 right. My guess is that doing a whole bunch of similar questions all at once would prove to be very taxing.
I will transcribe the questions just so you get an idea of what I have in mind.
- What was Christopher Columbus trying to find when he sailed due west across the Atlantic Ocean from Spain?
- What was the name of the Portuguese explorer who rounded the Cape of Good Hope in 1487, but turned back because his crew was afraid of what might lie ahead?
- What was the name of the Portuguese explorer who finally did manage to sail around Africa and reach India?
- Where did Columbus go first, to look for money for his trip? And what did that king’s scientist’s say about his proposal?
- Who funded Columbus’ trip, and why did they do so?
- When did Columbus set sail, and what were the names of his three ships?
- When Columbus’ desperate, scurvy men threatened to mutiny, what did Columbus offer them?
- Where did Columbus land, and what did he call the people that he found there?
What I am fairly sure of is that doing this at the same intervals as pure information review would ensure that H. learns the material much better. He might not be exposed to as much information, because less information can be put into simple question-and-short-factual-answer format than relatively many declarative sentences, but there is no doubt in my mind that he would learn the answers to those questions much better. It’s not just a matter of quality over quantity; the actual quantity of information retained would be greater despite less information initially imparted (I think–obviously, this could be tested).
We’ve been doing three or four recordings per day (i.e., we read 3-4 things that I summarize), sometimes two, sometimes five or (very rarely) six. But if we average 3.5 recording sessions, and there are 8 questions per, that would work out to 28 questions per day on average. The actual average would probably be somewhere from 20 to 40. That’s a lot. Then if we’re reviewing according to the same schedule, that works out to approximately 84 questions after a month, and 112 after three months.
Whereas listening to 30 minutes of recordings seems doable, if a little much to expect, answering a hundred questions a day sounds pretty ridiculous, on its face.
But! If the answers are coming quickly from H., as they did this evening, then well, it might not be so bad. If I had recorded as usual, I would probably have used 2.5 or 3 minutes, while the questions themselves took a little less than 1.5 and if H’s answers were very short and came fast enough, the whole review might take just a couple minutes. So in terms of time spent, it might not be any more and could be less.
Clearly, the best way to find that out is to try devising questions for a day, instead of making sentences. No need to speculate further; we’ll see how it goes tomorrow.
I’m inclined to think I should have H. answer the questions right after I’ve made them. Dr. Jones’ method actually specifies “one minute, one hour,” but we never do that and it sounds impractical. But we might do a review ten minutes later, followed by a review 24-36 hours later. Anyway, the immediate review shouldn’t take long, and it would ensure much better recall the next and subsequent times.
The big question, however, is whether we should do the reviews according to a one day, one week, one month, etc., sequence. Now, there is one very good reason to review according to this sequence, namely, it ensures that the information is reviewed when it is apt to be forgotten. But there is surely a difference between the requirements for passive review versus requirements for effortful examination. For one thing, quizzing is more efficient; for another, we can actually see whether the information is “taking.â€
I’m inclined to think that questions might be typed in. (It didn’t take me long at all to type in those eight questions above; but it would undoubtedly be a pain to type in 20-40 of such questions every day. Maybe I’ll have H. type them in, as typing practice, mwa-ha-ha.)
In that case, we could use software to automate the review according to some algorithm. I see two problems with that. First, the software might require me to write an answer, which is even more of a pain, when an answer really isn’t necessary (we’ll consult the book if we really can’t remember). Well, that’s easily finessed. Second, the questions would have to be formulated so that the context is not required, so a difficult question can be understood when separated from its fellows. But I guess I could get into the habit of formulating unconfusing questions.
Any suggestions as to software?
(Enjoying this very relevant discussion.)
Looks great, and glad to hear you’re still doing these reviews. Even though they’ve been tedious, I’m sure they have paid some dividends already and if continued will prove very valuable. How many things did I once know that is now lost? The majority of my education I’m sure.
As for software, I’ve been using Anki daily since Chris1 posted that youtube about it. Works well. But, you should be sure that’s the route you want to go. There’s a timer limit that you can set so that no single review goes past a certain time. I prefer the short answer or “fill in the blank” methods for what I study, but that research paper I posted said open free recall response is the absolute best.
As for typing in questions - I personally think this is THE BEST IDEA.
I type in every single question I plan on asking myself. If I can’t come up with a good question about something, then it’s probably not worth me taking the time to remember. Incidentally, I would have flunked your quiz, LOL…
Here’s why I like H typing in the questions (if this is something that is do-able for him and practical).
- Typing the questions is a review in of itself. It can serve as the first quiz actually (sort of).
- He practices his typing (but somehow I doubt this is going to be a big issue is your household; your kids will learn to type)
- There are no surprises. He will know if he aware of the answer or not.
And, I do like having some sort of answer - it’s for feedback purposes, and doesn’t have to be an all-inclusive answer. On the software, they really recommend simple questions and simple answers. However, I often do very complex questions and complex answers - and I still feel that I get a benefit.
Now here’s another thing that you can do with software; you can use a template to create reverse cards. Suppose an answer is Sir Francis Drake - though not a question, it’s a prompt… what’s significant about him? What can you say about him? You click the button and there’s the question (jeopardy style). It might be beneficial.
Another idea that perhaps you ought not discount, is combing quizzes with your summaries. In that article, the “study” sessions served as feedback (the student would spot something they got wrong, or missed within their latest test, and pay more attention to the forgotten material) - instant feedback (ie, an answer) is quick. A study review of a summary could be a bit more in-depth perhaps. So, suppose he remembered Ferdinand Magellan did X and Y, but forgot Z. Next summary he goes over, he might notice that he forgot Z.
Or maybe that whole summary as feedback is sub-optimal and if he wants to recall X, Y, and Z there’s all three as a quick answer so that (if using software especially) he’d see all three once he clicks the button.
And because he remembers the basic facts so well after say three times through the material as a quiz, when he hears the summary, he catches some more obscure content that maybe didn’t process the first time around.
I think that last sentence is a big purpose behind The Well Trained Mind’s concept of cycling through - they pick up more detail once their existing schema has grown.
In that research paper that I posted, there were a few studies where the subjects actually started to recall MORE information as time went on; and it was due to summaries providing the feedback on what was missed.
What I like about short answer or free recall is that it will sort of force him (through practice) to articulate. Not only is that a memory enhancer as discussed already, but there ought to be something said for being able to put the thoughts in order and present them… I think you mentioned that, his narratives improving.
Anki doesn’t use equal spaced repetition. It tries to prolong the repetitions out as far as possible. The reason is that the quiz is actually most effective the more DIFFICULT it is to remember (but not forgotten). If you really have to think to draw it out (like a few seconds), that’s good. With that in mind, there are usually 4 choices once you see the answer: do it again (forgot it), good, easy, very easy. It’s a bit subjective… if you do it again, it will mix it back in with whatever questions you’re reviewing that day… I’ve had ones where I keep forgetting within the same session; but some of that is a product of an obtuse subject matter and a price I’m willing to pay.
One thing that software can give you that a paper method wouldn’t, is that Anki can show you statistics. You can divide by subject which is nice, you can edit the question (or answer) if needed (such as a misspelling or mistake or especially if it’s perpetually forgotten which is called a leach, you can adjust it to make it more useful). There’s a way to put sound in and pictures and all that, but I don’t know how to do it - I’m sort of doing basic question answer stuff.
Hope this helps in some way or fashion. I’m VERY interested in how this goes and what you find useful or not, and what works or not… so be sure to update from time to time. 
Hey PokerDad, thanks for the helpful replies.
I tried out Mnemosyne, because it was recommended in that massive article I linked to at the end of my last, but have decided to switch to Supermemo because it seems to have the most features, largest community, and general support. I assume that it has these advantages over the similar Anki, although Anki was written after Supermemo. I’m impressed by all the time geeks have put into this sort of thing. Not terribly surprised, but impressed.
No chance of H. typing in the questions; he’s still slow and we’re talking a few dozen questions per day. That’s more writing than he does, on average, per day, so it’s much too much to expect.
As my wife pointed out, with the right software (Supermemo, not Mnemosyne), we can reuse the flashcards with baby E. when he starts reading the stuff.
I agree about having an answer. With this sort of system it’s just too much to expect one of us to know the answer or to look it up in a book. It will save time in the end to have the answer there.
Good point about the WTM 4-year cycles. But doesn’t it throw a monkey wrench in (in a good way) to that method if, after four years, the child remembers what he learned four years ago? Doesn’t that make it possible to accelerate learning even more on the next pass through the material? I’d expect so.
Doing summaries as well as questions is a non-starter, I’m afraid, for me. Maybe with a parent who had more time than me. The up front cost of a quiz-type approach is significant, and includes the time it takes to devise the questions and answers (more time-consuming and difficult than simply summarizing), and type this in–that’s above and beyond what we do when we listen to summaries.
I like the idea that H. can continue to do this himself when he’s reading more to himself, can spot the relevant questions, and types fast enough to type them in himself.
DadDude & PokerDad,
Glad to see you are stilling dedicated to working the memorization methods. I wanted to ask you for input. I downloaded Mnemosyne and found it unsatisfactory. I am studying Hebrew and it could not handle the diacritical markings copied into it. Also I was unable to make the print larger. Small print is a killer for me and would also be for a young child whose visual acuity is not well developed yet. Do SuperMemo or Anki have these same problems? Mnemosyne is a free download however and would work for most things I imagine.
As for testing, absolutely the next step is to move away from passive reception to active recall and explanation. You might try a Cloze test. It is very easy. When the material is repeated just leave a blank where the details you want remembered are. Columbus sailed the ocean blue in _______. That sort of thing. It works greats for remembering discrete detail. The final stage is being able to express a more or less “comprehensive” understanding of the event. A three year olds understanding will not be the same as a seven year olds, etcetera so adjust accordingly.
SuperMemo15 supports diacritics and works for me with Arabic:
http://www.supermemo.com/help/new15.htm
http://www.super-memo.com/supermemo15.html
And I believe that Anki does as well, but I had other issues with it.
~ Ayesha
Dr. Jones: SuperMemo has ways of automating the construction of “Cloze” items (I just learned this word a half-hour ago as I was reading on the Supermemo site!).
I’m frankly pretty excited about this. I’m thinking of adding a feature to Reading Bear, and to the upcoming nonfiction e-readers we’re planning after the reading tutorial is finished, that tests items according to spaced repetition. I’ve got to do research and see if this is feasible, if it’s already essentially being done, and generally inform myself of the ins and outs.
In the meantime, I’m already getting good practical experience with H., who likes SuperMemo, as long as we limit sessions to a couple of times a day.
Ayesha, are you using it with any kids?
Is anyone else here using one of these programs? How about a report?
I use the cloze method A TON (called it fill in the blank earlier);
I will be watching for updates from DadDude especially since he’s going to use Supermemo; I’ve only used the older version 7, and with more advanced features and support, I would be willing to switch over. Currently am using Anki, and Dr. Jones, you can adjust the font size in the card layout display, you can also adjust the font style, etc.
Dr Jones, quick question… I’ve read one of Lev Vygotsky’s books, and in it he describes how children need help in constructing memories at first (I believe this was from his book anyway). Young children, without training, have a hard time describing what they did that day if you ask them, or what they learned or studied. Obviously if they do something big like visit Disneyland or something, they can tell you they did that, but they typically won’t go into detail of the things they did without some sort of co-construction. Co-construction, in this example, would be if the parent asked the child “what rides did you go on?” and then as the child thought about it for a moment, the parent helps… “remember when you went on the Dumbo ride? and then what did we do after that?”… “remember we went over and rode the Pinocchio ride” etc etc and the child is then able to reconstruct the memory with ASSISTANCE.
My question is, when do you think this co-construction can begin? Do you have any other thoughts about how to train an early memory, not so much to remember facts or figures, but just to have the ability to actually remember things in general? Specifically if there are things that can help develop the brain structures required so that a child isn’t having to co-construct when 5 or 6 years old. Thanks!
Next memory seminar you do, I will put it in a calendar with an alarm so that I don’t get hyper-focused and miss out.
Oh, and diacritical marks will copy easily into Anki. Not trying to sell you on one ware over the other - they’re both good; it was just Anki was free, LOL
Edit: DadDude, the most challenging part is in learning how to phrase, or how to construct the question so as to streamline the learning process; obviously you’re reading the 20 recommendations list… I found that I got better at it, but part of that is figuring out what’s working and what’s not working WITH ME. If I were doing this for someone else, I’d have to just go off of feel, experience, or intuition. You can look over any particular card’s statistics… sort of how test makers would look for [ugh, can’t remember what they called them!] poor questions that had low correlations.
Hey DadDude,
I used SM15 to create lists of classical/Qur’anic Arabic vocabulary words by Chapter, and paused the project for sometime. I also am having some computer issues, and once I complete a project in the next 2 - 3 months, I intend to begin using SM15 again for my children and for myself.
I also like this great little (inexpensive) book by Dr. Bill Klemm, “The Memory Medic”, who is an American neuroscientist and memory expert, that has some good advice on learning that will be helpful for my children as they get older:
and his blog is informative as well: http://thankyoubrain.blogspot.com/
I am going to read this thread from the beginning, because I am interested in memorization techniques related to Qur’anic memorization:
Famous Mauritanian Method:
ttp://www.sunniforum.com/forum/showthread.php?10435-Methods-of-Memorization-in-Mauritania
Article by Michael Tipper, a memory expert who visited Kuwait and discusses the Qur’anic memorization methods:
http://www.happychild.org.uk/acc/tpr/features/0108text.htm
and to better understand how you are using SMS with your children.
The ‘one minute’ schedule reminds me of a different schedule used in foreign language learning programs developed by Dr. Paul Pimsleur, that used graduated recall (spaced repetition). (http://www.pimsleur.com/)
“Pimsleur’s 1967 memory schedule was as follows: 5 seconds, 25 seconds, 2 minutes, 10 minutes, 1 hour, 5 hours, 1 day, 5 days, 25 days, 4 months, 2 years.[3]”
http://en.wikipedia.org/wiki/Pimsleur_language_learning_system#cite_ref-2
And his research: http://www.eric.ed.gov/ERICWebPortal/search/detailmini.jsp?_nfpb=true&_&ERICExtSearch_SearchValue_0=ED012150&ERICExtSearch_SearchType_0=no&accno=ED012150
When memorizing a new Qur’anic lesson, I use this schedule up to the 1 day mark, and then use a common daily review schedule either on the Mauritanian Method ( http://www.sunniforum.com/forum/showthread.php?10435-Methods-of-Memorization-in-Mauritania ) or a less rigourous review method that is common in the Indian subcontinent:
http://fearthedunya.wordpress.com/2008/06/24/how-to-memorize-the-quran-part-i/
http://fearthedunya.wordpress.com/2008/06/24/how-to-memorize-the-quran-part-i/
And since it is a foreign language, one also has to study the WFW meanings and verse translations to understand basic meanings), and for deeper meanings, studies include the details of exegesis, Arabic grammar, etc. which also helps with retaining it.
I polled the Well-Trained Mind Forums and finally sparked a discussion. I am instructed by various people that five-year-olds should not memorize things. Well!
I punished them with a philosophical essay.
http://www.welltrainedmind.com/forums/showthread.php?t=375753
Thanks for your replies, above, Ayesha and PokerDad. I have to find time for all I have to read!
What a coincidence! I just finished reading your posts on that thread this morning! I have been lurking on the WTM forum more and more as my daughter gets closer and closer to the age when we must decide whether to homeschool her or enroll her in private school. I promise, though, that I am NOT stalking you! :biggrin:
LOLOL. Thanks for the laugh, I needed it. :laugh:
While sifting through non-fiction bestsellers on AMZN last night, I stumbled onto a cognitive psychology book that piqued my interest and might be apropos to the WTM debate. The book is called “Why Don’t Students Like School”
Apparently this a pedagogical piece from a cognitive psych view; FYI, cognitive psych was perhaps my favorite class in college; I learned a lot, had a perfect grade, and my prof is in the top 50 of influential psychologist of the 20th century… so I have quite a bit of respect for the field…
I will get the book to read it for myself, but in browsing, it appears that the gist of the book is that thinking cannot take place absent information, and information is accumulated in a particular way (memorization? I think so to a large part, this is discussed in the book)… will keep you posted.
EDIT: I read through the thread, and concur with your thoughts. Very well put, IMO.
It’s a great book (so far), I’ve read a lot of it. Your gist is absolutely right, but he covers several other topics, not just the necessity of information for thinking.
It would be great to pull facts from that book in particular to try out spaced repetition on myself! It’s very meaty.
Why is spaced repetition not better known? Why isn’t it used in schools?
http://larrysanger.org/2012/05/why-is-spaced-repetition-not-better-known/
been waiting for your next installment, thanks!!
the biggest “ah-hah!” moment for me regarding spaced repetition was really in reviewing the supermemo site a few years back and discovering that there’s a huge difference in how you phrase things or set up the cards. I did cards for years, but no where near the sophistication that I do now… and the whole concept of active review vs. passive review (my cards used to be just for passive review)… LOL @ the person that believed “testing” had no merit to long term retention in the WTM thread - very clearly established that it DOES.
Thanks again DadDude!!
Well, now it has been exactly two weeks since we started using spaced repetition software systematically. I’m now a true believer. I’ve learned a few things:
(1) The questions should be as brief and easy as possible. (The Supermemo tips emphasize this, but I didn’t pay attention closely enough.) The nice thing is that you can revisit difficult questions and make them easier at any time.
(2) It’s counterproductive to do too much in one day. Keep the number of questions added under 20, and better yet under 15. Then your review sessions won’t be too arduous, you’ll enjoy yourself, and you’ll be able to remember things more easily. Simply avoid the more recherche details, ones that your child does not really need to know. Also, if your child insists on learning all the names of the seas in the Mediterranean and all the capitals of Central American countries, in one day, added on top of your usual quantity of questions, put on the breaks. It will not take one day, it will take several days…
(3) Review twice a day, once in the morning for the day’s newly-scheduled items, and evening for new items added during the day as well as items missed in the morning (which are automatically scheduled, unless you tell the Supermemo system not to do so).
The main reason I’m so enthusiastic about spaced repetition for homeschooling (and as a homework supplement, I imagine) is the 95% guarantee that the information put into the system is actually memorized. Pick any 15 simple items from your child’s school day. You can guarantee that your child will know them almost perfectly in five years, almost all of them, as long as you stick with daily use of the software. If you’re adding 15 easy questions a day (rough guess), my guess is that the total review time can be around 30 minutes (rough guess). I think it is time extremely well spent–30 minutes daily for a guarantee of near-perfect memory of a huge number of fundamental facts about everything. Huge number = over 25,000 “Q & A” formatted facts in five years.
It is frankly a travesty that educational psychologists have not insisted that all new educators learn about this method, and that school systems and individual teachers do not use it. If you want a single tool that could solve the problem of ignorance among students in the U.S., have them use Supermemo (or better a user-friendlier but less powerful version of the same) for a half-hour a day.
We’re continuing to listen to recordings from a month ago, and as soon as we have no more one-month-old recordings to listen to (except poetry and myth, which I still summarize without making recordings about–they’re easy to do), we’ll switch to our “one season later” review–in fact, we started recording summaries exactly one month and one day ago!
DadDude,
Do you feel that this program follows the above method? http://www.freeworldu.org/static/index.aspx We use the FREE version. I have done a few lessons. If they get the answers wrong we review. Not sure if anyone else has tried this program. Any reviews both positive and negative would be greatly appreciated If you click on flash card index you can try any of the lessons… Thank you kindly…
Susan Khan