Praise be to Allah, the Lord of the World; and blessings and peace be upon our Prophet Muhammad and upon all his Family and Companions.
There is no harm in just looking at a church as the Prophet did not deny that when Um Habiba () and Um Salamah () looked at a church. But the Prophet denied the pictures and drawings contained in the church. Al Bukhari and Muslim reported from Aisha () that she said: Um Habiba and Um Salama mentioned a church they had seen in Ethiopia and in the church there were pictures. When they told the Prophet of this, he said, "Those people are such that if a pious man amongst them died, they build a place of worship over his grave and paint these pictures in it. Those people will be Allah's worst creatures on the Day of Resurrection."
As for entering the church and looking at the idols and statues that are in it, this is more likely to be forbidden. Al Bukhari reported from Omar Ibn Al Khattab () that he said: "We do not enter your Churches because of idols and pictures that are in them".
Anyway, if the purpose behind going to the church is to see the statues and crosses that are in it or to participate with the Christians in their feasts then it is strictly prohibited to enter it. We believe that it is safer not to go to the Church as to avoid seeing bad and evil things.
Allah knows best.