All perfect praise be to Allaah, The Lord of the Worlds. I testify that there is none worthy of worship except Allaah and that Muhammad, sallallaahu ‘alayhi wa sallam, is His slave and Messenger.
The question is somewhat ambiguous. It appears that what your husband said is an oath that he will not accept you as his wife throughout his life. So this is not a divorce or 'eelaa' (swearing not to have sexual relations with you). It is probable that he said so before he got married to you. If so, then as soon as the marriage contract was conducted, he broke his oath, and he is thus obliged to expiate for breaking his oath. Please refer to Fatwa 87181.
It is also probable that he said so after he got married to you, and with his oath, he intended that he will not continue his marital life with you and that he will divorce you one day. In which case, if he divorces you, he fulfills his oath, and if he does not divorce you, then he is obliged to expiate for breaking his oath. However, breaking the oath does not happen until it is too late for fulfilling the oath, which is the time of death of either of you, or by being determined not to do what he had sworn upon, such as refusing to divorce you or deciding to continue the marriage with you; in which case he is considered as having broken the oath according to the view of some scholars.
In any case, we advise your husband not to hasten to divorce and to expiate for breaking his oath and continue his marital life with his wife, especially if she is religious and is of good moral character.
Allaah knows best.