Hive編程入門-weibo-大數據文檔資料

集團技術平臺部-核心系統(tǒng)-海量數據天照1–OpenSourceMapReduceframework–支持SQL語義的大規(guī)模數據分析工具–離線/數據倉庫應用?為超大數據集設計的計算/擴展能力?統(tǒng)一的元數據管理整體架構?Client端應用程序?元數據?編程接口?環(huán)境變量–HIVE_HOME–HIVE_CONF_DIR–HADOOP_HOME/HADOOP_CONF_DIR–hive-default.xml（默認不做修改）/hive-site.xml（用戶的配置）–perties?選擇一個元數據服務器–int/bigint/smallint/tinyint–boolean–double/float–string?沒有精度/長度設定(col_namedata_type,...)等同于：…–INSERTOVERWRITET–INSERTOVERWRITETABLE[WHEREwhere_condition]–join_table:table_referenceJOINtable_factor[{LEFT|RIGHT|FULL}OUTER|LEFTSEMI]JOINtable_referencejoin_condition–table_factor:tbl_name[alias]|table–join_condition:ONequality_expression(ANDequality_expression)*equality_expression:-e'quotedquerystring'Sqlfromcommandline$HIVE_HOME/bin/hive-e'selecta.colfromtab1a‘$HIVE_HOME/bin/hive-e'selecta.colfromtab1a'-hiveconfmapred.reduce.tasks=32$HIVE_HOME/bin/hive-f/home/my/hive-script.sql$HIVE_HOME/bin/hive-i/home/my/hive-init.sqlquitorexittocomeoutofinteractiveshell.Usethistosetvalueofparticularconfigurationvariable.Onethingtonotehereisthatifyoumisspellthevariablename,cliwillnotshowanerror.Thiswillgiveallpossiblehadoop/hiveconfigurationvariables.JARresourcesarealsoaddedtotheJavaclasspath.ThisisrequiredinordertoreferenceobjectstheycontainsuchasUDF's.executedfscommandcommandfromhiveshell?<querystring>executeshivequeryandprintsresultstostdoutexecutessqlfromfileandprintsresultstostdouthive>setmapred.reduce.tasks=32;//設置值hive>setmapred.reduce.tasks;//顯示值hive>selecta.*fromtab1;hive>addjar/home/tianzhao/hive/hive-1.1.2/lib/hive_contrib.jar;hive>CREATETEMPORARYFUNCTIONdate_formatAS'com.taobao.hive.udf.UDFDateFormat';hive>source/home/tianzhao.blj/test_others_hive/union.sql;顯示partition的信息?Java語言編寫的用戶自定義函數–UDF-1:1–UDAF-N:1–UDTF-1:N–CREATETEMPORARYFUNCTIONfunction_nameASclass_name–implementevaluate()function–extendsUDAF/GenericUDAF–implementUSING'my_map_script'(AScolName(','colName)*)?(clusterBy?|distribute)REDUCEUSING'my_reduce_script'(AS