Generally speaking, in general relativity, there are two kinds of time: coordinate time and proper time. The former is the one that is gravity dependent, but the latter is not. When scientists report a calculated age for the universe, it is always a proper time, and therefore independent from gravity. However, a cosmology is just a model for the universe that includes general relativity as the theory of spacetime. The age of the universe is very much dependent on the details of the model, and cannot be derived purely from observation.

In most cases the age is arrived at in much the way you suggest, by using measured values of the Hubble constant to get the "expansion age" of the universe (see, i.e., "How Old is the Universe?). But we can also use more "ordinary" astrophysics to derive ages for stars. We then assume that the age of the oldest stars must be a minimum age for the universe, since it cannot be younger than the stars that are in it. 13 to 15 billion years is a good range, consistent with most cosmologies, and not mtoo outlandish.

__________________
The point of philosophy is to start with something so simple as not to seem worth stating, and to end with something so paradoxical that no one will believe it. -- Bertrand Russell