I think the X Factor which himesh refers to is not about looks or clothes or style, it is more about the singing quality.
I think waht they mean is that in many such shows there have been many ME TOO singers who ape the existing singers and hence are not able to make a place for themselves after the show. If you see singers like Vinit, Rahul Vaidya, Aneek are very good technically and with their voice but they are all lesser versions of Sonu. Singers like Raja, Toshi are modelled on Sukhvinder.
In order to cast an impression and create an identity in the minds of the viewers the singer needs to be different and original. Only then he will make a huge impact once the show is over.
Singers like Sukhwinder, HR, Kailash Kher, Kunal Ganjawala have made a huge mark because they have their own singing style which gave them an instant identity. So you have the classic voices like a Sonu, Shaan, Abhijeet, Udit and these other guys have found their niches between them and have won audiences.
For me X-Factor in a singer is his ability to be different from this ME TOO type of singing. Its okay to copy a Kavita or a Sonu or a Udit but important is what you bring yourself to the table. After a certain stage (where you have a good voice and sing in sur) it will be only those with a difference who will survive.