There's a difference between an inertial frame of reference and just any frame of reference. In special relativity, the inertial frame of reference just keeps on going in a straight line because there are no gravitational effects allowed in SR, so you, an observer, can't change direction or speed without changing which inertial frame you're in.

In general relativity, I don't know how this whole thing works, but I would guess that the "moving in straight lines" is more precisely "moving along a geodesic". Maybe.