I am not sure how long, but after XXX time the previous destinations drop off. I know it will work for about 1 week after that I don't know it is just they are no longer there.
You can save your home or work as a preferred destination which makes finding out how long to get back home easier.
Actually, it just turns out that I am retarded and that I originally had a set a few addresses in memory. When I cleared them all out to import my phonebook, I stopped looking at the memory screen which had all my previous destinations. Just found it this morning.