As a Bear of Very Little Brain, and at the risk of sounding like the Master of the Obvious, your White interlocutor has taken the comment as inferring that they, personally, got a free pass.
They sit there and think "I've worked my dangly bits off to get where I am and here's this person telling me that I got it all handed to me on a silver plate." So, lo and behold, you become another whingeing PoC.
Facts are not the issue when the information presented appears as anecdotal.
In such cases, you need to show a smoking gun. An admission by a teacher that they actively suppressed the achievements of Black pupils becuase of their race. Written evidence that edication boards and authorities condoned and even encouraged this. Even then, all you'll get is: "Well, if I'd known at the time, of course, I'd have spoken up." Yeah, right!
White Supremacy, White Privilege are hurtful concepts to many White people because they perceive them as lessening and undermining their own achievements. If they have worked hard, and made sacrifices - as many have and do - to reach their current place, then they resent the implication that their race made it easier for them.
This is why society is structured in such a way as to keep the races apart as much as possible. Because if more White people could or would actually see, with their own eyes, the extra barriers that are placed in front of Black people, they would be less complacent. Because even if most people are as dim as a Toc-H lamp, a lot of them are still decent and kindly folk. But they are taught, trained, not to notice, to stay in their protective bubble. America is the land of opportunity, they are told, where all men are equal. They are indoctrinated to believe that a failure to succeed is a failure of the person, the individual, themselves - nothing to do with the structural and systemic prejudice built into society. OK, I'm starting to sound like A Mere Hack, but you see my point, yes?
They respond that way because they are programmed to, by history and the needs of predatory capitalism. They are creations of a system and thus forced to see criticism of the system as criticism of themselves. They have been assimilated. Resistance, if attempted, would have been futile.
