All perfect praise be to Allah, the Lord of the worlds. I testify that there is none worthy of worship except Allah and that Muhammad, sallallaahu ‘alayhi wa sallam, is His slave and Messenger.
The basic principle is that the impurity was not transferred to your hands. Whenever you doubt whethersomething has become impure, you should act upon the original state of that thing, which is that of purity. In other words, if you are certain that something was pure and then doubt whether it has become impure, then you should know that certainty is not overruled by doubt.
The fatwa of the Permanent Committee for Iftaa reads, "The basic principle is that all things are ritually pure; therefore, no object or place is declared impure unless there is evidence indicating its impurity."
Accordingly, your hand does not become impure by touching the mentioned clothes while you are not certain that you touched the spot stained with wet impurity. It is possible that you touched a pure spot of the clothes, so the transfer of the impurity is doubtful in this case.
If the questioner suffers from Waswaas (obsessive whisperings), we advise him to ignore them, as that is an effective remedy for them.
Allah knows best.