I don't know the mechanics of how this is actually set up, so I may be wrong in the details.
IIRC, thermoelectric generators use a pair of conductors at different temperatures to generate a current flow from the Seebeck effect. In the case you're talking about, the "hot" conductor sits next to the radioisotope source, and the "cold" conductor sits (presumably) somewhere that is thermally insulated from the radioisotope source. If you keep this cold conductor facing away from the sun, it will cool continuously by radiation (I presume you could help this along with some radiator fins), because it "sees" nothing but starlight and the 3K background radiation.
So it's the balance between incoming and outgoing thermal radiation that does the cooling, rather than access to a source of "cold" molecules.
Or so I think.

Grant Hutchison