I wasn't saying they're necessarily aggressive. Indeed, as you said aggressiveness is probably a bottleneck for civilizations. Too aggressive, and you self-destruct.
My point was that the aliens don't necessary even
realize we're here or don't see us important enough. We think we know what intelligent life or life is and take it granted others think in the same way. A truly alien alien species don't even have to be aggressive, they would simply not notice us. We think a human being is immensely more important than an ant, they may not. That's the ant analogy.
Imagine some sort of silicon-based rock-eaters on Mars that are driven to extinction when earthlings terraform the planet before we even notice they're there, if you will. Another example is Arkady and Boris Strugasky's
Roadside Picnic.
We can modify our instincts only by changing ourselves. But first, the most rational way to proceed is to modify our societies. Despite all our technological advances, our culture is not much different from bronze age one despite what we may think.
The greatest threat to our survival is ourselves... A starfaring civilization would be practically immortal. Which leads to the Fermi paradox, and that worries me.