tag:blogger.com,1999:blog-7631116270195175228.post5184111433943937776..comments2024-03-14T04:06:37.165-07:00Comments on Text & Data Mining by practical means: Uncertainty coefficients for Features Reduction - comparison with LDA techniqueCristian Mesianohttp://www.blogger.com/profile/04880057603671195464noreply@blogger.comBlogger5125tag:blogger.com,1999:blog-7631116270195175228.post-57856112734981006832013-12-18T10:27:23.690-08:002013-12-18T10:27:23.690-08:00Please can you answer me : PCA or LDA reduce numbe...Please can you answer me : PCA or LDA reduce number of coefficents or number of frame for feature vector in speech recgnition?Anonymoushttps://www.blogger.com/profile/10973903462883625780noreply@blogger.comtag:blogger.com,1999:blog-7631116270195175228.post-73111362538197820762012-05-10T09:19:30.052-07:002012-05-10T09:19:30.052-07:00Hi Lev,
exactly, for this experiment I ranked the ...Hi Lev,<br />exactly, for this experiment I ranked the features using this formulation.<br />The vectors have been built using the first 3000 features scored through TF-DF and sorted in decrescent order (by score).<br />The feature selection has been done over these vectors.<br />I didn't play a lot with C or kernel params just because I wanted highlight that feature selection plays a primary role respect param optimization. (of course, if you train with wrong param the classifier doesn't work well :) ).<br />The graph you mentioned has been built as following:<br />I sorted the features by TFDF score, then for each feature I plotted the respective entropy.<br />In this case there is a clear correlation between features having high TFDF score and High entropy: BUT THIS IS A TOY PROBLEM!! in the real scenario this correlation is much more smoothed.<br />About the accuracy:<br />LDA as clustering algorithm is by its nature unsupervised, but you can use it to extract features from the corpus. <br />In the paper I used for the comparison test, the authors built the vectors using the features extracted through LDA, then they trained an SVM and they measured the accuracy as True positive (being a boolean case it's not so wrong).<br />Of course I did the same to compare using the same metric.<br />Actually it should be better measure the sensitivity in this case because the two classes are not balanced in term of size.<br />Please let me know If I answered all your questions!<br />I hope to see you again, maybe as a follower clicking "join this site" on the right panel.<br />cheers<br />cristianCristian Mesianohttps://www.blogger.com/profile/04880057603671195464noreply@blogger.comtag:blogger.com,1999:blog-7631116270195175228.post-40790878394581603692012-05-09T23:40:22.821-07:002012-05-09T23:40:22.821-07:00Hi Christian,
I went through your note and I hav...Hi Christian, <br /><br />I went through your note and I have several questions.<br /><br />The weights you used are I(L,X) / H(X) of feature X ?<br /><br />You write "from the above training set I extracted (after stemming and filtering process) all the words and I used them to build the boolean vectors." <br />Does that mean that you haven't counted the frequency of the words in the docs but used only its inclusion or non-inclusion?<br /><br />Why haven't you search for the best C of SVM?<br /><br />What do you mean exactly by "The above graph shows the entropy of the first 3000 features sorted by TF-DF score." ? TF-IDF makes sense only if we think documents+feature domain but not features domain only so I cannot imagine what do you exactly use for sorting. I must be misunderstanding something here. <br /><br />This is not clear for me what you measured on accuracy in case of LDA since it is an unsupervised method. Can you explain it a bit.Levhttps://www.blogger.com/profile/11632008869150411325noreply@blogger.comtag:blogger.com,1999:blog-7631116270195175228.post-3520479496632783762012-05-09T13:32:09.588-07:002012-05-09T13:32:09.588-07:00Thanks a lot for the info!
Actually I'm not su...Thanks a lot for the info!<br />Actually I'm not surprise that there are app for the features selection entropy based because it is a method very general purpose and easy to implement.<br />As you know I encourage always people in implementing by your own the algorithm, because it's the best way to understand advantages and limits of it and to adapt it to your specific problem (in this case there are many variants).<br />cheers<br />c.Cristian Mesianohttps://www.blogger.com/profile/04880057603671195464noreply@blogger.comtag:blogger.com,1999:blog-7631116270195175228.post-11536485888219196392012-05-09T13:09:11.551-07:002012-05-09T13:09:11.551-07:00Here is a web-based app that uses mutual informati...Here is a web-based app that uses mutual information for feature selection http://www.simafore.com/blog/?Tag=keyconnectBR Deshpandehttps://www.blogger.com/profile/03132933574358283998noreply@blogger.com