Exactly. If you didn't start out in orbit of a body, simply falling towards it under the influence of gravity won't put you in orbit- the gravity field doesn't change your energy in relation to it, it just converts potential energy to kinetic energy and then back. You'll pass by it and eventually get to the same distance and speed that you started with. An orbit works the same way, but if your velocity is high enough the gravity well won't actually stop you and complete the ellipse- that is a hyperbolic path. Whether you hit the star or not in this case is not a factor of velocity but of direction- if your path goes through the star, you go splash, and if not, you continue on. To enter orbit, you would need to change your velocity, slowing in relation to the star. Aerobraking at such speeds (using a star or planet's "atmosphere" to slow down via classic friction) would be be, shall we say, problematic.

Mike