First, what do you mean by "struggling to lose?" How much exactly have you lost? Are you losing, say, a pound a week? Second, what's your height and current weight?
Muscle DOES weight more than fat -- a certain volume of muscle (i.e., a cubic inch) will weigh more than the same volume of fat. The flip side of that is that a given weight of muscle (i.e., a pound) will take up less room than the same weight of muscle. This is why it's important to track your measurements and/or body fat % as well as weight, because, yes -- even a beginning exerciser will gain muscle, even through just cardio. You are right that it may not be a huge factor and the effect tapers off after the first few weeks, but it's not total bunk, either.
More importantly, depending on your current weight, and given your activity level, you may not be eating enough calories. If you don't eat enough to sustain your resting metabolism and activity, your body can start to work hard to PRESERVE fat stores rather than burning them. You might check out
http://www.jimkaras.com/loss_math01.cfm to see if your calorie intake is in line with your actual needs.