There is some emerging science behind the perception that time passes more quickly as we age — or appears to pass more quickly, I should say. It's documented to some extent in this article: Physics explains why time passes faster as you age.
My personal theory has been that the longer we live, the less of a percentage each unit of time is in our lives. So, if you are 5 years old, your most recent year, the fifth year, is 20% of your life. But if you are 80 years old, the most recent year (the 80th year) is only a little more than 1% of your life.
Because we forget so much, I believe that, in a certain sense, an 80 year-old doesn't feel like they've lived much longer than a 5 year-old, so the percentage is of some importance. That's just my theory, which is totally unprovable by the way and certainly not part of the physicist's explanation.
However, as referenced in this article, Bejan, is theorizing on the matter. You can read the whole piece or quickly read the 4 main points that I have cribbed, below.
Time is happening in the mind’s eye. It is related to the number of mental images the brain encounters and organizes and the state of our brains as we age. When we get older, the rate at which changes in mental images are perceived decreases because of several transforming physical features, including vision, brain complexity, and later in life, degradation of the pathways that transmit information. And this shift in image processing leads to the sense of time speeding up.
There’s an inversely proportional relationship between stimuli processing and the sense of time speeding by, Bejan says. So, when you are young and experiencing lots of new stimuli—everything is new—time actually seems to be passing more slowly. As you get older, the production of mental images slows, giving the sense that time passes more rapidly.
Another factor in time’s perceived passage is how the brain develops. As the brain and body grow more complex and there are more neural connections, the pathways that information travels are increasingly complicated. They branch like a tree and this change in processing influences our experience of time, according to Bejan.
Finally, brain degradation as we age influences perception. Studies of saccadic eye movements in elderly people show longer latency periods, for example. The time in which the brain processes the visual information gets longer, which makes it more difficult for the elderly to solve complex problems. They “see” more slowly but feel time passing faster, Bejan argues.