Expansion of the space is to observe if there is a difference – the space inside the atoms remains small and intergalaxy grows – it could be if there are antigravity particle in Cosmic Voids (Dark Energy), or there are more and more gravitationally neutral particles (neutrinos ?) with short range repulsive force.
If the space grows only, or there is just the space as space, we would grow with the space and no observe the expansion.