實(shí)驗(yàn)2-熟悉常用的HDFS操作-題目

1、實(shí)驗(yàn)2熟悉常用的HDFS操作云計(jì)算課程機(jī)房上機(jī)實(shí)驗(yàn)報(bào)告題目：熟悉常用的 HDFS操作姓名日期實(shí)驗(yàn)環(huán)境：實(shí)驗(yàn)內(nèi)容與完成情況：岀現(xiàn)的問(wèn)題：解決方案（列岀遇到的問(wèn)題和解決辦法，列岀沒(méi)有解決的問(wèn)題）：實(shí)驗(yàn)?zāi)康?. 理解HDFS在Hadoop體系結(jié)構(gòu)中的角色:2. 熟練使用HDFS操作常用的Shell命令：3. 熟悉HDFS操作常用的Java APIo實(shí)驗(yàn)平臺(tái)操作系統(tǒng)：LinuxHadoop版本：或以上版本JDK版本：1.6或以上版本Java IDE： Eclipse實(shí)驗(yàn)內(nèi)容和要求編程實(shí)現(xiàn)以下指左效用,并利用Hadoop提供的Shell命令完成相同任務(wù)：（1）向HDFS中上傳任意文本文件，如果指左的文件

2、在HDFS中已經(jīng)存在，由用戶指泄是追加到原有文件末尾還是覆蓋原有的文件; 開(kāi)啟服務(wù)：wengubuntu:/usr/local/hadoop/sbtn$ ./start allshThis script is Deprecated Instead use startdfsSh and startyarn.shStarttng namenodes on localhostlocalhost: starting nanenode, logging to /usr/locat/hadoop/logs/hadoop-wen-nan node-ubuntu.outShell命令:wcnubuntu:/u

3、sr7loubl/hBdoop/btn$ wenQubuntu:/usr/local/hadoop/bin$ 0 wenubuntu:/usr/local/hadoop/binS hadoop :未找到命令 wenaubuntu:/usr/tocal/hadoop/btnS wenubuntu:/usr/local/hadoop/binS wenubuntu:/usr/local/hadoop/bin$ Higtrls Hl,girls wenubuntu:/usr/local/hadoop/bin$ txt weneubuntu:/usr/locaX/hadoop/btn$ Hi,girls

4、./hadoop fs est e text.txt echo $?hadoop fs -appendToFile local.txt texttxt./hadoop /hadoop /hadoop /hadoop./hadoopwenubuntu:/usr/local/hadoop/bin$ |appendToFile localtxt texttxt appendToFile local.txt text.txt cat text.txtcopyFromLocal local.txt text.cat text.txt Console S3翼憑HdFsApi Java Applicatio

5、n/usr/lib/jvrn力ava-7-open4log4j;WARN No appenders could be found for logger (qoop. log4j:WARN Please inreiabze the Iog4jsystem properly/user/tlny/text.txt 不存花/usr/local/hadoop/text.txt 己上傳至 /userAlny/text.txt（2）從HDFS中下載指左文件，如果本地文件與要下載的文件名稱相同，則自動(dòng)對(duì)下載的文件重命名; Shell命令：wenubuntu:/usr/local/hadoop/btn$ exp

6、ort PATH=$PATH:/usr/local/hadoop/btn wen(dubuntu:/usr/local/hadoop/bin$ if $(hadoop fs est -e /usr/local/hadoop/tex t.txt);> then $(hadoop fs copyToLocal text.txt /text.txt);> else $(hadoop fs -copyToLocal text.txt ./text2txt);> ftv(Enubuntu:/us/loc己l/hadoop/btn$ .2/1Consoledeterminated >

7、; HdfsApLOZ Java Application/usi7Ut>/|vni/javaa7-openjdk. for more /usr/tocal/hadoopAext.txt 已殍在.甬新命名為：/usr/local/hadoopAext.Ext_0(3)將HDFS中指定文件的內(nèi)容輸出到終端中：wenubuntu;/usr/local/hadoop/bin$ hadoop fs cat text*txt Hl,girlswenubuntu: /jisr/local/hadoop/btn$ （4）顯示HDFS中指左的文件的讀寫(xiě)權(quán)限、大小、創(chuàng)建時(shí)間.路徑等信息;Shell命令wen

8、ubuntu:/usr/local/hadoop/bin$ hadoop fs cat text.txtHi,girlswenQubuntu:/usr/local/hadoop/btn$ hadoop fs ls h text.txt -rw-r-r-1 wen supergroup9 2016-12-15 18:38 text.txtwenubuntu:/usr/local/hadoop/btn$（5）給左HDFS中某一個(gè)目錄，輸出該目錄下的所有文件的讀寫(xiě)權(quán)限、大小、創(chuàng)建時(shí)間、路徑等信息，如果該文件是目錄，則遞歸輸岀該目錄下所有文件相關(guān)信息; Shell命令：wenubuntu:/usr/l

9、ocal/hadoop/bin$ hadoop fs R -h /user/tiny-rw-r-r-3 wwn supergroup8 2016-12-15 18:51 /user/tiny/text.txtubuntu:/usr/local/hadoop/bin$JAVA代碼:（6）提供一個(gè)HDFS內(nèi)的文件的路徑，對(duì)該文件進(jìn)行創(chuàng)建和刪除操作。如果文件所在目錄不存在，則自動(dòng)創(chuàng)建目錄:Shell命令：2/1wengubuntu:/usr/local/hadoop/bin$ if $(hadoop fs -test -d dirl/dir2);> then $(hadoop fs touch

10、z dirl/dtr2/filename);> else $(hadoop fs -nkdir -p dirl/dir2 && hdfs dfs -touchz dirl/dir2/filename);> ftwcngubuntu:/us/lo5l/hadoop/btn$ hadoop fs m dirl/dir2/filenane16/12/15 19:11:24 INFO fs.TrashPolicyDefault: Namenode trash configuration: Del etlon Interval = 0 ntnutes, Enptter Int

11、erval = 0 ntnutes.Deleted dirl/dir2/filenamewenBubuntu: /usr/locral/hadooD/binSJAVA結(jié)果:J Console S3 X 瀆 砧昌吃cterminated> ndfsApi_O6 Java Application /usr/lib/jvm/java-7openjdk-annd64/bi 胸 system properly.og4-WARN S*ee hi(p7/logging .apach e.o rg/log4g 112/faq.htmlff rtocon fig for more info. 創(chuàng)建文件夾;

12、/user/tlny/tnput胡建丈件;$erAiny/lnputAex<.<xt（7）提供一個(gè)HDFS的目錄的路徑，對(duì)該目錄進(jìn)行創(chuàng)建和刪除操作。創(chuàng)建目錄時(shí)，如果目錄文件所在目錄不存在則自動(dòng)創(chuàng)建相應(yīng)目錄；刪除目錄時(shí),由用戶指左當(dāng) 該目錄不為空時(shí)是否還刪除該目錄：Shell命令：wenQubuntu:/usr/local/hadoop/bin$ hadoop fs -nkdir -p dirl/dir2 wenubuntu:/usr/locat/hadoop/bin$ hadoop fs -rmdtr dirl/dir2 wenQubuntu:/usr/local/hadoop/b

13、tn$ hadoop fs -rm -r dtrl/dtr2 rm: 'dirl/dir2': No such file or directoryWAnffliihiint-u r /usr/locat/hadoop/btn$JAVA結(jié)果: console &其筑喩噩區(qū)昌蘭 ：腎-=Q<terminated> HdfsApi_07 Java Application /usr/lib/jvm/java-7-openjdk-amd64/bin/java (2016 1215 "F午7: log4j:WARN Please initialize the

14、Icg4j system properly目錄不為空.不AM沬/user/tiny/inpui（8）向HDFS中指上的文件追加內(nèi)容,由用戸指泄內(nèi)容追加到原有文件的開(kāi)頭或結(jié)尾; Shell命令：hadoop fs appendToFile local.txt texttxt hadoop fs get text.txtcat text» local.txthadoop fs copyFonLocal f text.txt texttxt more local.txtwenubuntu:/usr/local/hadoop/bin$ wcnubuntu:/usr/local/hadoop/

15、bin$ get:ext.txt': File existswenQubuntu:/usr/local/hadoop/bin$ wenQubuntu:/usr/local/hadoop/btn$ wenubuntu:/usr/local/hadoop/btn$ Hi,girlsHi,boyswenubuntu:/usr/local/hadoop/bin$JAVA代碼:Console S3 X % lit 砧 空twminatedA Hdf$ApiJ>8 ova ApplicotionJ/usi/lib/lvm/warmpemdharnch og4j;WARN Seehupy/to

16、/togl.i/faq.htmlFnoconifig formore info, 三追 ttO 內(nèi)容別文件末 fiK/w$er/(inynpu t/texc. tx t（9）刪除HDFS中指左的文件;Shell命令：A/enfiubuntu:/usr/local/hadoop/bin$ hadoop fs text.txt16/12/15 19:34:55 INFO fsTrashPolicyDefault: Namenode trash configuration: Del etton interval = 0 minutes, Emptier interva

17、l = 0 minutesDeleted text.txtgnubuntu:/us/loc:al/hadoop/b5$JAVA結(jié)果:2/1vterm 泊 ated> Hdf5Api_O9 Java Application/u5r/lib/jvm/java-7-openjdk-amd64/birlog4j:WARN Please initLaliiethe Iog4j system properly.log4j:WARN Se http:/k>gslr)/tog4/L2/f3q.htmWnaconfig for more info. 文件:M嫁：/user/(

18、lny/cexL(xt（10）刪除HDFS中指左的目錄，由用戶指左目錄中如果存在文件時(shí)是否刪除目錄;Shell命令：wenubuntu:/usr/local/hadoop/bin$ hadoop fs mdir dirl/dtr2 rmdir: 'dirl/dir2' : No such file or directorywenQubuntu:/usr/local/hadoop/bin$ hadoop fs (n -R dirl/dtr2 rm: 'dirl/dir2': No such file or directorywenubuntu:/us/local/hadoop/bin$JAVA代碼:（IDHDFS中,將文件從源路徑移動(dòng)到目的路徑。Shell命令：wenubuntu:/usr/local/hadoop/bin$ hadoop fs text*txt input mv: "text.txt': No such file or directoryJAVA結(jié)果:termi nate> H<jf$ApLi 1 Java Application /usr