it's often a combination of parts, I would not lose sleep over it. You can stop it by standing slightly to disconnect your butt from the saddle or stabilizing the top tube with your knees. Getting all tense and sighting the bike just makes it worse. It seems like some bike-wheel combinations just wobble for some riders. Sometimes.

That said I have had carbon, steel and aluminum bikes that wobbled, or didn't