Surely people who think street harassment is no big deal have dealt with panhandlers. Does anyone walking down the street in a big city hear "How are you doing" or "Excuse me" or "Can I talk to you" from a stranger and not automatically suspect that person wants something from the exchange?
They aren't being neighborly. And if you nod, or say hello, or acknowledge they have spoken to you, they will not stop until they get what they want. And we know this, and we shut down every shred of decency and friendliness and generosity in our soul and ignore them. Or we give them what they want.
How is "Baby you are looking fine today" viscerally different than "Buddy you are looking wealthy and generous"?