To those socialized in a particular way, being with a background is considered being an asshole. It’s especially a noticed issue with women and more conservative areas.

In my experience I usually think this in situations where I went out of my way to help someone and am treated poorly for it. So it depends on if you consider “not helping someone when they need it” being an asshole, which I guess would probably be a case by case thing for most people. But yes I agree, setting boundaries gets you out of a lot of these situations too.