Most of the people who make nasty comments on others are bad/ toxic/ negative and unhappy people. People who live in mud sling mud on others ;-) The best people that at least I have met in my life are positive, encouraging, friendly, lovely and too busy with activities to be spending time on gossiping.
I might remember the unfavorable feedback/ comments made on me by the toxic people but it doesn't mean that I let it get to me. Most of the negative people are hypocrites, honestly it is hilarious to see the negative minds come up with their judgment, yes they are nasty but at the same time a mirror of themselves.
So I say - have a good memory but pretty much do your own thing. Do not even try to make people see their hypocrite or negative behavior, it is a waste of time. If you feel good and are not doing anything to hurt someone intentionally, then you are doing everything you can!