James Smith: 
Zoomup‍ Sorry but I personally feel your advice is part of the problem, not part of the solution.

Don't be sorry. Why be sorry? Speak out and be confident about your opinions. I am sick and tired of people saying “sorry but…”. That's a modern problem, people are just so insecure in these “modern” times. Speak your mind.

Z.

It sounds as if he is a bit insecure about how great you are, my husband agrees. ?

“Other colleagues have commented on how rude he can be towards me but mostly keep clear of it (I don't blame them).”

Silence is complicity. How can your colleagues be making you "feel welcome" if they are not telling this guy to wind his neck in. The next time a colleague comments to you about how rude he is, say “Yes he is rude. Thanks for standing up for me.” This guy is obviously comfortable enough to be openly hostile and downright rude, safe in the knowledge that nobody is going to say anything to him. Given his hostility towards you, direct confrontation may not be the best solution, but if you can get the others to understand your point of view, perhaps one or more could have a word with him and tell him to knock it off. He may be more likely to take it under advisement coming from someone else.

Saying “all women are prostitutes” is disgusting misogyny - not even cloaked in “humour” or “banter”, but I do understand that it may depend on the power dynamic of your relative roles/ages and the team dynamic as to how comfortable you might be to clapback. If you are at the same level, you could just roll your eyes and say something like “Tell me you're an incel, without telling me you're an incel” or “Are you including your mother in that statement?” or “Hey colleague X,  Bob's calling your wife/daughter a prostitute”. You can do this in a lighthearted way, or you can say it deadpan to your colleagues “you realise he is saying that about your wife/mother/sister/daughter, don't you?” or “Are you going to defend your wife/mother/sister/daughter or do you agree with Bob's comments?”. Or you could try standing up and loudly saying “Does anyone else want to take a turn at being attacked by X? It seems like it has been my turn for X months and it is becoming quite exhausting. Anyone?”

Failing that, you could try a direct approach and ask things like:

“What are you basing that on?”
“Can you explain what you mean by that?”
“What are you hoping to achieve by saying that?” 
“Is that directed at me?”
“Is that your attempt at humour?”
“Are you hoping for some sort of reaction?”  
“Do you treat everyone like this or just me?”
“Is there something you want to say?”
“Do you have a problem?”
“Can we just focus on the work?”
“What an unusual point of view for 2021.”
“This again? Aren't you bored of this now?"
“What a sad view of the world you have”

As others have mentioned, I would discuss things with your line manager. Even if you don't want to make an official complaint (yet) for fear of repercussions of being seen as “making a fuss”, you should make your line manager aware of the things that are being said so that s/he can keep an eye on things. Your line manager may think it is “banter” and may not be aware that it is crossing a line. If your line manager knows it is going too far, then the next time they witness something offensive, they can and should call him on it there and then without the reprimand being associated with you.

You could also start logging the offensive comments so that if you do decide to involve HR at a later date, you will have a record of what was said, when, where and who else was present at the time.