Aliens may have been here for a long time, but are of suhc a radically different evolution, biology and what have you that we can't see them and don't notice them, and they are equally ignorant of us!

Also, it's reasonable to suppose that interstellar travel - by whatever means it might be achieved - is an expensive and risky process. Dropping in on inhabited planets whose natives might have an attitude is probably illegal. For pragmatic, not ethical, reasons. The logistics of interstellar warfare would be mind-boggling, and the odds would always be with the defence, even if they were less advanced. There may very well be a bunch of aliens galumphing about out there, but they're probably sticking to empty worlds and asteroid belts with lots of valuable minerals to mine.