15-Hadoop-socket測(cè)試

Hadoop socket 測(cè)試 分類(lèi) hadoop 2011 11 01 14 35 89 人閱讀 評(píng)論 0 收藏 舉報(bào) 概要 hadoop 上以 streaming 方式運(yùn)行 map 任務(wù) map 任務(wù)為 socke client 端 和在外部系統(tǒng) 運(yùn)行的 server 交互 代碼 server cpp view plaincopy to clipboardprint 1 include 2 include 3 include 4 include 5 include 6 include 7 include 8 include 9 include 10 include 11 include 12 define SERVPORT 9899 服務(wù)器監(jiān)聽(tīng)端口號(hào) 13 define BACKLOG 1000 最大同時(shí)連接請(qǐng)求數(shù) 14 define MAXDATASIZE 100 每次最大數(shù)據(jù)傳輸量 15 16 using namespace std 17 18 int main 19 20 int sock fd client fd sock fd 監(jiān)聽(tīng) socket client fd 數(shù)據(jù)傳輸 socket 21 int sin size 22 char buf MAXDATASIZE 23 struct sockaddr in my addr 本機(jī)地址信息 24 struct sockaddr in remote addr 客戶(hù)端地址信息 25 26 if sock fd socket AF INET SOCK STREAM 0 1 27 28 perror socket 創(chuàng)建出錯(cuò) 29 exit 1 30 31 my addr sin family AF INET 32 my addr sin port htons SERVPORT 33 my addr sin addr s addr INADDR ANY 34 bzero 35 36 if bind sock fd struct sockaddr 39 exit 1 40 41 42 if listen sock fd BACKLOG 1 43 44 perror listen 出錯(cuò) 45 exit 1 46 47 48 while 1 49 50 sin size sizeof struct sockaddr in 51 if client fd accept sock fd struct sockaddr 54 continue 55 56 printf received a connection from s n inet ntoa remote addr sin addr 57 if fork 58 59 子進(jìn)程代碼段 60 if send client fd Hello you are connected n 26 0 1 61 62 perror send 出錯(cuò) 63 64 65 int recvbytes 0 66 if recvbytes recv client fd buf MAXDATASIZE 0 1 67 68 perror recv 出錯(cuò) 69 exit 1 70 71 72 cout buf endl 73 close client fd 74 exit 0 75 76 close client fd 77 78 include include include include include include client cpp view plaincopy to clipboardprint 1 include 2 include 3 include 4 include 5 include 6 include 7 include 8 include 9 include 10 include 11 define SERVPORT 9899 12 define MAXDATASIZE 100 每次最大數(shù)據(jù)傳輸量 13 using namespace std 14 15 int main int argc char argv 16 17 int sock fd recvbytes 18 char buf MAXDATASIZE 19 struct hostent host 20 struct sockaddr in serv addr 21 22 讀取輸入 23 string input 24 while cin input 25 26 cout input endl 27 28 29 if argch addr 49 bzero 50 51 連接 52 if connect sock fd struct sockaddr 55 exit 1 56 57 58 接受數(shù)據(jù) 59 if recvbytes recv sock fd buf MAXDATASIZE 0 1 60 61 perror recv 出錯(cuò) 62 exit 1 63 64 65 發(fā)送數(shù)據(jù) 66 if send sock fd hello server 13 0 1 67 68 perror send 出錯(cuò) 69 70 buf recvbytes 0 71 printf Received s buf 72 close sock fd 73 return 0 74 include include include include include include 編譯程序 g o client client cpp g o server server cpp 測(cè)試 測(cè)試機(jī)上運(yùn)行 server server hadoop 上運(yùn)行任務(wù) hadoop streaming input user test txt output user result mapper client domain reducer cat file client jobconf mapred reduce tasks 1 jobconf mapred map tasks 5 jobconf mapre job name socket test input 內(nèi)容 user test txt 為 1324 jobconf mapred reduce tasks 1 jobconf mapred map tasks 5 mapper 任務(wù) 5 個(gè) reduce 任務(wù) 1 個(gè) server 輸出 received a connection from 10 hello server received a connection from 10 hello server received a connection from 10 hello server received a connection from 10 hello server received a connection from 10 hello server 運(yùn)行結(jié)果 hadoop fs cat user result part 00000 1324 Received