All perfect praise be to Allah, The Lord of the Worlds. I testify that there is none worthy of worship except Allah, and that Muhammad is His slave and Messenger.
It is impermissible for you to give up fasting in Ramadhaan because of these whisperings. Rather, it is incumbent on you to resist them and try to get rid of them. You have to fast the whole month and not give in to these delusions. If you gave up fasting on any of the days of Ramadhaan (because of that reason), then it is a serious sin that necessitates repentance and making up for the missed fasting days. However, it does not count as an act of disbelief that takes the doer out of the fold of Islam.
Allah Knows best.