The point you have to remember is that all of these people, every character, are caught up in something much bigger than they are.

World-weary Rick came to Casablanca to avoid it. Committed, idealistic Victor needs to be in the thick of it, doing his part. Ilse is drawn to both men: the brave one who puts his cause first, and the apparently cynical one who would put them first. When she tell Rick that he must think for all of them, it is because she knows that he won't be blinded by either idealism or love, because he's distanced from both. Rick no longer trusts either Ilse or love - he likes to think he's learned from his mistakes. But in the end he realises that, in the bigger context, Victor is more important than he, and Victor will not function well without Ilse. So he lets her go, and he can do that because he was always self-sufficient - he loves her, he wants her, but he doesn't need her.

As Meat Loaf pointed out "Two out of three ain't bad!" And they'll always have Paris.