Poor thermal conductivity of rock. Heat couldn't be transported from below fast enough to keep up with the heat radiated away into space. A thick blanket of rock makes an effective insulator, without the sun's input, the surface would be a lot cooler...most of our surface heat comes from the sun.

It's also not jut thermal inertia keeping the core warm. First, the same insulating effect means a relatively low amount of radioactive decay would keep the core warm for a considerable amount of time. Second, the inner core is very compact solid iron, the outer core is liquid. Removing heat doesn't cool the outer core, it just allows more of it to solidify, similar to how ice water at the surface stays at the freezing point until it has completely frozen or melted. So while the core is losing heat, it's not necessarily dropping in temperature.