I think fair can apply to men in the sense of pleasing countenance. In The Last Debate Legolas was described as fair of face beyond the measure of Men, and when Legolas saw Imrahil he recalled the legend of Imrazor, who was said to have had an Elven wife.
Imrahil refers to Legolas as one of the fair folk, and Legolas later comments about Imrahil: 'That is a fair lord and a great captain of Men.'
Of course 'fair' can refer to various things. For example Tolkien noted:
'Vanyar thus comes from an adjectival derivative *wanja from the stem WAN. Its primary sense seems to have been very similar to English (modern) use of 'fair' with reference to hair and complexion; though its actual development was the reverse of the English: it meant 'pale, light coloured, not brown or dark', and its implication of beauty was secondary.'
I think Celegorm the Fair was so called, at least in part, because he had golden hair, in early texts anyway -- Christopher Tolkien kept the 'title' the Fair but for The Silmarillion removed the reference to his golden hair, on account of a statement in Appendix F (quoted in part below).
I don't now why Hirluin the Fair was called this, but I think that Imrahil was exceptionally Elvish looking whether or not the legend was true, and thus fair in the sense of pleasing looking or well featured; also noting that in Appendix F the Eldar were said to be tall '... fair of skin and grey-eyed, though their locks were dark, save...' so pale of skin could be included as well.
I think we could have more than one reference here, as Elfwine might have been fair of face like Imrahil, as in having pleasing features, and here arguably including 'pale of skin'. And maybe he was fair of hair like his father, although granted the likeness to Imrahil would arguably be more striking if both were dark haired.