mahout下載和安裝

1、1：下載地址我下載的是： mahout-0.3.tar.gz          17-Mar-2010 02:12   47M  2：解壓tar -xvf mahout-0.3.tar.gz3:配置環(huán)境export HADOOP_HOME=/home/hadoopuser/hadoop-0.19.2 export HADOOP_CONF_DIR=/home/hadoopuser/hadoop-0.19.2/conf4：使用看看先bin/mahout -help會列出很多可以用的算法5：使用kme

2、ans聚類看看先bin/mahout kmeans -input /user/hive/warehouse/tmp_data/complex.seq   -clusters  5 -output  /home/hadoopuser/1.txt關(guān)于 kmeans需要的參數(shù)等等通過如下命令可以查看：bin/mahout kmeans -helpmahout下處理的文件必須是SequenceFile格式的，所以需要把txtfile轉(zhuǎn)換成sequenceFile。SequenceFile是hadoop中的一個(gè)類，允許我們向文件中寫入二進(jìn)制的鍵

3、值對，具體介紹請看eyjian寫的mahout中提供了一種將指定文件下的文件轉(zhuǎn)換成sequenceFile的方式。（You may find Tika (/tika) helpful in converting binary documents to text.）使用方法如下：$MAHOUT_HOME/bin/mahout seqdirectory -input <PARENT DIR WHERE DOCS ARE LOCATED> -output <OUTPUT DIRECTORY> <-c <CHARSET

4、NAME OF THE INPUT DOCUMENTS> UTF-8|cp1252|ascii.> <-chunk <MAX SIZE OF EACH CHUNK in Megabytes> 64> <-prefix <PREFIX TO ADD TO THE DOCUMENT ID>>舉個(gè)例子：bin/mahout seqdirectory -input /hive/hadoopuser/ -output /mahout/seq/ -charset UTF-8運(yùn)行kmeans的簡單的例子：1：將樣本數(shù)據(jù)集放到hdfs中指定文件下,應(yīng)

5、該在testdata文件夾下$HADOOP_HOME/bin/hadoop fs -put <PATH TO DATA> testdata例如：bin/hadoop fs   -put /home/hadoopuser/mahout-0.3/test/synthetic_control.data  /user/hadoopuser/testdata/2：使用kmeans算法$HADOOP_HOME/bin/hadoop jar $MAHOUT_HOME/examples/target/mahout-examples-$MAHOUT_VERSI

6、ON.job org.apache.mahout.clustering.syntheticcontrol.kmeans.Job例如：bin/hadoop jar /home/hadoopuser/mahout-0.3/mahout-examples-0.1.job org.apache.mahout.clustering.syntheticcontrol.kmeans.Job3：使用canopy算法$HADOOP_HOME/bin/hadoop jar $MAHOUT_HOME/examples/target/mahout-examples-$MAHOUT_VERSION.job org.ap

7、ache.mahout.clustering.syntheticcontrol.canopy.Job例如：bin/hadoop jar /home/hadoopuser/mahout-0.3/mahout-examples-0.1.job org.apache.mahout.clustering.syntheticcontrol.canopy.Job4：使用dirichlet 算法$HADOOP_HOME/bin/hadoop jar $MAHOUT_HOME/examples/target/mahout-examples-$MAHOUT_VERSION.job org.apache.maho

8、ut.clustering.syntheticcontrol.dirichlet.Job5：使用meanshift算法meanshift : $HADOOP_HOME/bin/hadoop jar $MAHOUT_HOME/examples/target/mahout-examples-$MAHOUT_VERSION.job org.apache.mahout.clustering.syntheticcontrol.meanshift.Job6：查看一下結(jié)果吧bin/mahout vectordump -seqFile /user/hadoopuser/output/data/part-000

9、00這個(gè)直接把結(jié)果顯示在控制臺上。Get the data out of HDFS  and have a look All example jobs use testdata as input and output to directory outputUse bin/hadoop fs -lsr output to view all outputsOutput:KMeans is placed into output/pointsCanopy and MeanShift results are placed into output/clustered-points英文參

10、考鏈接：/MAHOUT/syntheticcontroldata.htmlTriJUG: Intro to Mahout Slides and Demo examplesFirst off, big thank you to TriJUG and all the attendees for allowing me to present Apache Mahout last night.  Also a big thank you to Red Hat for providing a most excellent meeting s

11、pace.  Finally, to Manning Publications for providing vouchers for Taming Text and Mahout In Action for the end of the night raffle.  Overall, I think it went well, but thats not for me to judge.  There were a lot of good questions and a good sized audience.The slides f

12、or the Monday, Feb. 15 TriJUG talk are at: Intro to Mahout Slides(Intro Mahout (PDF).For the “ugly demos”, below is a history of the commands I ran for setup, etc.  Keep in mind that you can almost always run bin/mahout <COMMAND> help to get syntax help for any given command.Heres th

13、e preliminary setup stuff I did:1. Get and preprocess the Reuters content per2. Create the sequence files: bin/mahout seqdirectory input <PATH>/content/reuters/reuters-out output <PATH>/content/reuters/seqfiles charset UTF-83. Convert the Sequence Files to Sparse Vectors, using the Eucli

14、dean norm and the TF weight (for LDA): bin/mahout seq2sparse input <PATH>/content/reuters/seqfiles output <PATH>/content/reuters/seqfiles-TF norm 2 weight TF4. Convert the Sequence Files to Sparse Vectors, using the Euclidean norm and the TF-IDF weight (for Clustering): bin/mahout seq2sp

15、arse input<PATH>/content/reuters/seqfiles output <PATH>/content/reuters/seqfiles-TF-IDF norm 2 weight TFIDFFor Latent Dirichlet Allocation I then ran:1. ./mahout lda input  <PATH>/content/reuters/seqfiles-TF/vectors/ output  <PATH>/content/reuters/seqfiles

16、-TF/lda-output numWords 34000 numTopics 202. ./mahout org.apache.mahout.clustering.lda.LDAPrintTopics input <PATH>/content/reuters/seqfiles-TF/lda-output/state-19 dict <PATH>/content/reuters/seqfiles-TF/dictionary.file-0 words 10 output <PATH>/content/reuters/seqfiles-TF/lda-output

17、/topics dictionaryType sequencefileFor K-Means Clustering I ran:1. ./mahout kmeans input <PATH>/content/reuters/seqfiles-TFIDF/vectors/part-00000 k 15 output <PATH>/content/reuters/seqfiles-TFIDF/output-kmeans clusters <PATH>/content/reuters/seqfiles-TFIDF/output-kmeans/clusters2.

18、Print out the clusters: ./mahout clusterdump seqFileDir /Volumes/Content/grantingersoll/content/reuters/seqfiles-TFIDF/output-kmeans/clusters-15/ pointsDir /Volumes/Content/grantingersoll/content/reuters/seqfiles-TFIDF/output-kmeans/points/ dictionary /Volumes/Content/grantingersoll/content/reuters/seqfiles-TFIDF/dictionary.file-0 dic