All perfect praise be to Allah, the Lord of the worlds. I testify that there is none worthy of worship except Allah and that Muhammad, sallallahu ‘alayhi wa sallam, is His slave and Messenger.
You should know, dear brother, that it is impermissible to accuse a Muslim of being deviant or an innovator based on assumption or mere doubts, let alone accusing a Muslim of disbelief, so fear Allah, the Almighty, and repel such thoughts. If you are a resident of this city or staying in it, then you have to seek to pray the Friday prayer with the Muslim community, and it is impermissible to miss it without an excuse. It is not valid to pray the Friday prayer alone, because one of the conditions for the validity of the Friday prayer is to offer it in congregation.
The Encyclopedia of Fiqh reads, "As for the Friday prayer, one of the conditions for its validity is offering it in congregation; it is not valid to offer it without congregation according to the consensus of the scholars."
If you miss the Friday prayer without an excuse, you bear a sin for that, and you are obliged to pray the Thuhr anyway. Ibn Al-Munthir wrote, "They (scholars) unanimously agreed that if a resident (i.e. not a traveler) missed the Friday prayer, he performs a four-Rak'ah (unit of prayer) prayer instead."
As for the fact that the Muslim community is small in that city, the view adopted at Islamweb is that there is no minimum number of persons required for a valid Friday prayer. The only condition is that there is a sufficient number of worshipers that is customarily considered a congregation, since no number was specified in Shariah.
Shaykhul-Islam Ibn Taymiyyah chose the view that the minimum number of worshipers required for the Friday prayer to be valid is three: one imaam and two listeners. Some scholars held the same opinion as well. Some scholars even held that the minimum number of worshipers required for the Friday prayer to be valid is two: the imaam and one listener. Ash-Shawkaani chose this view in his book Nayl Al-Awtaar. In any case, there is no explicit text that stipulates a certain number of worshipers to establish the Friday prayer. As for the congregation, it is one of the conditions for the validity of the Friday prayer by consensus. Obviously, the Friday prayer requires a congregation; as long as there is a number of worshipers that is customarily considered a congregation, it is obligatory to hold the Friday prayer and it is valid.
Regarding your belief that the masjid is inside their house, it is not a condition to hold the Friday prayer in a masjid according to the view adopted at Islamweb. Rather, it may be held in any place in the city suitable for holding the prayer in congregation. Ibn Hajar Al-Haytami wrote, "It is not a condition to hold the Friday prayer in a masjid; rather, if there is any place in town that accommodates the people, even if it is not a masjid, it is an obligation to hold the Friday prayer there." [Al-Fataawa Al-Fiqhiyyah Al-Kubra]
Therefore, you should strive to attend the Friday prayer and fulfill the obligation that Allah assigned you with; and do not miss it except without a valid excuse.
Allah knows best.