It depends on whether what they say is coming from them or if it's something they are citing; "I am extremely attractive" can be countered with "you are wrong", but "People say I am extremely attractive" cannot be, because I did not come up with the opinion, others did.

"They are wrong" is then valid, or "That is not correct" if I have misinterpreted them.