Yeah, most of what you see is the tail. A decent size comet nucleus is on the order ot tens of miles across. A comet tail can extend a few million miles. So yes, a comet tail can be longer than the diameter of the Sun.

Rob