I don't think it is necessarily about harder or more time, but just that your body get used to the same routine. You muscle build "memory" of the movement and don't have to work as hard. Changing be simply doing different exercises for the same muscle if you are weight training or changing your route if you are walking so that you hit hills at different times. From what I have read, changing up your workout every 8 weeks or so is a good thing.
(Misty I have no idea what you do for your work out.. I'm just throwing this out there as part of the conversation. It's not necessarily directed at you

)