Is it ok to not talk to non-mahram (permanently unmarriageable) men even if Islam said that we can in some cases?

All perfect praise be to Allaah, The Lord of the Worlds. I testify that there is none worthy of worship except Allaah and that Muhammad is His slave and Messenger.

It is permissible for you to refrain from talking to non-mahram men even if it is a permissible speech. However, if there is a need and a benefit in talking to them while there is no suspicion or fear of temptation, then it is more appropriate not to refrain from talking to them in order to achieve the benefit.

Allaah knows best.