When it’s just for your own peace of mind, or just some muppet downvoting you all the time, that’s fine. However, harassment strictu sensu often causes social repercussions that harm the person being harassed, even if they’re oblivious to that. And often harassers don’t just stop at one person, they pick multiple targets; it is not the sort of people who you want in a community.

So often it’s simply better for you and everyone else to report them instead.