All perfect praise be to Allaah, The Lord of the Worlds. I testify that there is none worthy of worship except Allaah, and that Muhammad is His slave and Messenger.
It is not permissible to be pessimistic about the visit of this person merely because of certain harms that occur during his visit, because pessimism involves Shirk (polytheism), thinking ill of Allaah and expecting disasters.
If you feel pessimistic, put your trust in Allaah and say, as the Prophet , instructed us in a Hadeeth on the authority of Ibn ‘Umar and his father: "Allaahumma laa khayra illa khayruk wala tayra illa tayruk wala illaaha ghayruk (O Allaah, there is no good but Yours; There is no omen but rather there is reliance on You. There is no deity worthy of worship but You)” [Ahmad]
Also, you are not permitted to prevent this man from visiting you merely for this reason. The Prophet mentioned that a person who refrains from doing something out of pessimism is committing Shirk. [Ahmad] In another Hadeeth, the Prophet stated that pessimism is to do or not do something based on this. [Ahmad]
These two Hadeeths emphasize that a person will not be held accountable for pessimism unless they act according to it. Thus, preventing this man from visiting you is a practical application of pessimism.
However, if you are sure that the things that are broken or burnt in your house are a result of him looking at them, then he has an evil eye and you should hide things that may attract his attention or sit with him in a place that does not contain valuable items. You must also protect yourself with Quranic verses and Prophetic supplications so that the evil eye will not harm you or your property.
Ibn Al-Qayyim mentioned that the effects of envy and the evil eye afflict only the person who does not protect himself with the Islamically legislated Athkaar (mentionings and remembrances of Allaah). As for the person who does protect himself with them, nothing will harm him except with the permission of Allaah.
You should advise this man with gentleness and wisdom and explain to him that if a person sees something he admires, he should say, "Maa shaa' Allaah” or, “Tabaarak Allaah" (That is what Allaah wills, or, blessed be Allaah). Additionally, it is permissible to prevent him from visiting you if he continues to be envious and corruptive after you have explained the issue to him. Scholars even suggest that severer actions may be taken against him. For example Imaam Ibn Al-Qayyim, the Hanbali scholar, held that an Imaam must prevent an envious person from intermixing with people, confine him and provide for him until he dies. Ibn Al-Qayyim quoted this ruling from the Hanbali scholars and others and gave it predominance. The reason behind this ruling was not pessimism but the prevention of harm.
Allaah Knows best.