“…The more likely is that it is ascribed to wool (soof) because linguistic analogy supports it. And because the wearing of wool was from the characteristics/signs of the Prophets, ascetics and worshippers. This view was given preference and chosen by al-Siraj in ‘al-Lum’ (p.40), al-Qushayri in ‘al-Risalah’ (4/3,4 with its commentary) al-Kalabadhi in ‘al-Tarruf’ (p.29,30) Ibn Taymiyyah in ‘al-Sufiyyah wal-Fuqara’ (2/17) and al-Yafi in ‘Nashr al-Mahasin al-Ghaliyah’ (2/343).
Abu Nasr al-Siraj and others mentioned that the name ‘Sufi’ was present during the time of the followers (tabieen) and at the end of the companions era. The evidence for this being that Hasan al-Basri (Allah the Most High be pleased with him) said: I saw a Sufi in the tawaf, I gave him something but he did not take it.
And Sufyan al-Thawri (Allah the Most High be pleased with him) said: If it were not for Abu Hashim al-Sufi I would not have known the finer points of ostentation (riya).
He then mentioned that Makka al-Mukarramah became deserted during a time before the coming of Islam such that no one would perform tawaf of the House (Kabah). A sufi man would come from a far away land and perform tawaf of the House and then leave. He said: If this is correct then it proves that this name was known before Islam and was attributed to the people of virtue and uprightness.
Imam Abul Abbas Ahmad bin Taymiyyah (Allah the Most High have mercy on him) said in the treatise ‘Al-Sufiyyah wal-Fuqara’: The wording (meaning al-Sufiyyah) is reported from more than one of the Imam’s and Shuyukh such as Imam Ahmad bin Hanbal, Suleiman al-Darani and Sufyan al-Thawri…”