That says more about other people than him, tho, let's be honest.
If it were another "more acceptable" relationship (no age gap, no power imbalance, two popular idols), except one was freer (because that's what it is. He was free(r) to talk about it) to talk about the relationship and did so...because it's their relationship, nothing to be ashamed of and something he's probably happy with, then...he can? People dragging HER because he's mentioning/references his relationship with her? Yeah, that's not on him unless he started getting real personal in sharing what they do when they're together, which, to my knowledge, he didn't.
Now, as for conjectures on his personality or character, I have nothing to say on that because I don't know dude (re: "Japanese fetish" as I see on this thread or whatever else fans have come up with). But toxic people being toxic (fans and non-fans) ain't on him, it's on them.