It's the "appeal to authority" fallacy (ie. the "authority" being the quoted 'people':
https://yourlogicalfallacyis.com/appeal-to-authority).
Essentially your guy is saying that the crowd is right and you're wrong because the moral authority (in this instance) is with the crowd (ie. many people believe that you're wrong, therefore you're wrong - no reasoning required). I have a natural aversion to the 'crowd' mentality. The 'crowd' is often irrational, stupid, blinded by emotion and powerful. In the words of the late, great ,Terry Pratchett, "The intelligence of that creature known as a crowd is the square root of the number of people in it.".
https://yourlogicalfallacyis.com/