I only count them on days I consciously workout (at the gym or home) - I don't count them for things like walking around the zoo, cleaning house, etc.
I work out on average 3 days per week right now (wish it was more), 30 minutes of cardio and 30 minutes of lifting weights. So on average I get about 9 activity pts per week.