How to settle free planets in enemy territory?

I can't figure out how to settle planets that are not in my territory. 2 turns after I land a colony ship, the planet gets culture flipped. I have time only to fast build a single building! I tried to build a Consulate, but it doesn't seem to help.

I also tried building an influence starbase next to the planet I'm trying to settle on, but the AI alsways takes offence at that and declares war after about 5 turns of having a starbase there (and the relations were warm before I sent the constructor!). No such problems ever happened in GalCiv2, I think.