Okay. I'm taking an astrophysics class and upon hearing that the centers of large stars can reach temperatures in excess of 100 million K, I did some rough calculations. I found that the average velocity of a hydrogen nucleus at a temperature of hundreds of millions of degrees, not accounting for relativity, would be over the speed of light. They would of course actually be moving near the speed of light with an increased mass from the mass of their kinetic energy (E=MC^2).

So, at the centers of the biggest stars you have temperatures so high that the particles involved gain relativistic mass.

I got to thinking - in the largest stars, could the thermal energy of the particles making up the star be high enough that its mass is a significant fraction of the star's total mass?

I calculated the total thermal energy present inside a 60 solar mass, 1,000,000 solar luminosity star. Assuming an average path length for a photon of 100,000 lightyears (like for our sun) of scattering before it is emitted from the surface of the star, you get a figure of 1.23*10^45 Joules of heat energy and radiation present inside the star. Thats 1/146 of a SOLAR MASS when you convert energy to mass - meaning that more than 1/9,000 of the mass of this large star comes only from its thermal energy!!!

If you assume that because the star is more massive the photons will scatter even more on their way out, you get an even more incredible result. I took the cube root of 60 for my multiplier for the path length, thinking that the total amount of matter from the core to the surface along any one line would be the deciding factor, I could be wrong. Anyways, if you multiply the path length by the cube root of 60 (3.91) you get that a full 1/37.5 solar mass is present in the form of thermal energy and radiation. This would mean that a full 1/2241 of the star's mass comes from its thermal energy alone.

Am I totally off my rocker here?

In addition this got me thinking - say you have a large gas cloud that collapses. It will heat up, as the gravitational potential energy is converted into heat. Since conservation of energy holds and this heat energy can be considered mass (due to the relativistic effects on the atoms if nothing else) then does this mean that this gravitational potential energy has mass as well? And therefore, could large diffuse objects have larger masses than small compact objects with the same matter content and temperature?

At such temperatures and pressures, we're not dealing with gas. The movement of the particles must be restricted comparatively. Since this energy cannot be converted to momentum, isn't that part of the reason stars radiate their energy from a coherent mass instead of breaking apart?

Just some starter thoughts until someone smarter can point us both in the right direction.

You're not off your rocker, you just need to check your math about the speed getting faster than light. The core temperature of the Sun is a little over 10 million Kelvin, which means the "average velocity of a hydrogen nucleus" is about 500 km/s. The speed of light is 300,000 km/s. Now you said "hundreds of millions of Kelvin", which can happen in the cores of massive stars nearing supernova, but even that gives nuclei at only about 1% of the speed of light. The electrons can get relativistic, especially in the cores of evolving stars (and when they make white dwarfs), but they don't have much mass-- the only times the nucleons get relativistic is in neutron stars and black holes. Your numbers about 1/2000 or so of the mass of the Sun being thermal is a bit high too-- it's more like 1 part in a million (and the negative gravitational self-energy reduces the mass more than this increases it.) But still your point does have an important application-- mass does get altered in the way you imagine, and it can sometimes matter: the mass of a neutron star does require relativistic corrections (but it is not more mass than you'd think, it's less, owing to gravitational self-energy reductions in the mass).

Alright. Must've made a math error in my rough calculations. Oops! In fact I'm almost certain I know what it was, I probably forgot to transfer between meters and kilometers per second.

Still - if kinetic energy can be considered mass, I have a hard time seeing any large problem in my mass fraction calculations.

I didn't bother calculating the values in the sun, I figured that'd be very small. I took an extreme example, a star with 60 times the mass of the sun and a million times as much energy being produced per second and emitted per second. Depending on how long it takes energy to diffuse out away from the core it would presumably contain at least a million times as much thermal energy stored in the thermal motion of its atoms/molecules/plasma.

Applying the relativistic mass correction to an object traveling at one percent the speed of light I get that its mass increases to 1.00005 times it original value - thats one part in 20,000. In principle it seems to me that the very largest stars with tremendous luminosities and very high temperatures might have mass fractions of thermal energy higher than 1 in 100,000. I know theres pretty much no practical application to this but its something awesome to think about.

Luminosity (J/s) * path time (~100,000 years) = internal heat energy of the star when it is in equilibrium. I still get the same value for the thermal energy of the enormous star in Joules... and thus the same mass increase. I'm not even considering relativistic speed, I'm just considering good old E=MC^2. That IS applicable here, yes?

I don't think there was a problem with those, the results were very small. Also, they did not include the gravitational self-energy, which changes the sign of the correction but not its overall magnitude. We have a very small effect here, but it's not zero-- you're right.
Agreed, we often forget about relativistic corrections. If you look at it in absolute numbers, the relativistic corrections even for the relatively small mass of the Sun are roughly of order the mass of the whole Earth.