元數(shù)據(jù)管理-實(shí)用

1、元數(shù)據(jù)管理冷鵬 （2006年編寫）2n數(shù)據(jù)集成技術(shù)數(shù)據(jù)集成技術(shù)n 數(shù)據(jù)展現(xiàn)分析數(shù)據(jù)展現(xiàn)分析n 數(shù)據(jù)倉庫建模數(shù)據(jù)倉庫建模n 廠商解決方案廠商解決方案n 談元數(shù)據(jù)管理談元數(shù)據(jù)管理n 數(shù)據(jù)倉庫基礎(chǔ)數(shù)據(jù)倉庫基礎(chǔ)n 項(xiàng)目開發(fā)管理項(xiàng)目開發(fā)管理3元數(shù)據(jù)管理工具元數(shù)據(jù)管理工具n Informatica MetaManagern DAG MetaCentern IBM MetaStagen Oracle OWBn CA Decisionbase Transformern 4元數(shù)據(jù)元數(shù)據(jù)MDC的開放信息模型的開放信息模型分析與設(shè)計(jì)（Analysis and Design） 對象與組件（Object and Com

2、ponent）數(shù)據(jù)庫與數(shù)據(jù)倉庫（Database and Warehousing） 業(yè)務(wù)工程（Business Engineering） 知識管理（Knowledge Management） OMG組織的組織的CWM模型模型UML MOF（元對象設(shè)施）XMI（XML元數(shù)據(jù)交換）5元數(shù)據(jù)的重要性元數(shù)據(jù)的重要性及時(shí)發(fā)現(xiàn)系統(tǒng)變化及時(shí)發(fā)現(xiàn)系統(tǒng)變化找出需要調(diào)整的找出需要調(diào)整的ETL程序程序給出倉庫中數(shù)據(jù)和指標(biāo)的來源及規(guī)則給出倉庫中數(shù)據(jù)和指標(biāo)的來源及規(guī)則調(diào)整倉庫的數(shù)據(jù)模型調(diào)整倉庫的數(shù)據(jù)模型促進(jìn)項(xiàng)目組成員的溝通促進(jìn)項(xiàng)目組成員的溝通監(jiān)控?cái)?shù)據(jù)質(zhì)量監(jiān)控?cái)?shù)據(jù)質(zhì)量采用元數(shù)據(jù)跟用戶確認(rèn)新的需求采用元數(shù)據(jù)跟用戶確認(rèn)新的需

3、求6技術(shù)元數(shù)據(jù)技術(shù)元數(shù)據(jù)7業(yè)務(wù)元數(shù)據(jù)業(yè)務(wù)元數(shù)據(jù)8業(yè)務(wù)專家業(yè)務(wù)專家元數(shù)據(jù)元數(shù)據(jù)開發(fā)人員開發(fā)人員項(xiàng)目經(jīng)理項(xiàng)目經(jīng)理架構(gòu)師架構(gòu)師誰需要元數(shù)據(jù)誰需要元數(shù)據(jù)業(yè)務(wù)經(jīng)理業(yè)務(wù)經(jīng)理分析人員分析人員業(yè)務(wù)定義業(yè)務(wù)定義如：某銷售指標(biāo)不正確，找出銷售指標(biāo)的計(jì)算方法及計(jì)算項(xiàng)來源業(yè)務(wù)專家業(yè)務(wù)專家9分析人員分析人員開發(fā)人員開發(fā)人員業(yè)務(wù)專家業(yè)務(wù)專家項(xiàng)目經(jīng)理項(xiàng)目經(jīng)理架構(gòu)師架構(gòu)師誰需要元數(shù)據(jù)誰需要元數(shù)據(jù)業(yè)務(wù)經(jīng)理業(yè)務(wù)經(jīng)理元數(shù)據(jù)元數(shù)據(jù)需求分析定義需求分析定義如：分析指標(biāo)定義、報(bào)表如：分析指標(biāo)定義、報(bào)表定義、數(shù)據(jù)來源定義定義、數(shù)據(jù)來源定義10開發(fā)人員開發(fā)人員業(yè)務(wù)專家業(yè)務(wù)專家項(xiàng)目經(jīng)理項(xiàng)目經(jīng)理架構(gòu)師架構(gòu)師誰需要元數(shù)據(jù)誰需要元數(shù)據(jù)業(yè)務(wù)經(jīng)理業(yè)務(wù)經(jīng)

4、理元數(shù)據(jù)元數(shù)據(jù)不同開發(fā)小組之間的交流不同開發(fā)小組之間的交流如：DBA工程師與ETL工程師 DBA工程師與BI工程師 . 分析人員分析人員11元數(shù)據(jù)元數(shù)據(jù)開發(fā)人員開發(fā)人員業(yè)務(wù)專家業(yè)務(wù)專家項(xiàng)目經(jīng)理項(xiàng)目經(jīng)理架構(gòu)師架構(gòu)師誰需要元數(shù)據(jù)誰需要元數(shù)據(jù)業(yè)務(wù)經(jīng)理業(yè)務(wù)經(jīng)理業(yè)務(wù)管理與信息消費(fèi)業(yè)務(wù)管理與信息消費(fèi)如：數(shù)據(jù)分析、業(yè)務(wù)決策、給前系統(tǒng)提出改進(jìn)建議，對數(shù)據(jù)倉庫系統(tǒng)二期、三期提出更多的業(yè)務(wù)需求分析人員分析人員12開發(fā)人員開發(fā)人員業(yè)務(wù)專家業(yè)務(wù)專家項(xiàng)目經(jīng)理項(xiàng)目經(jīng)理架構(gòu)師架構(gòu)師誰需要元數(shù)據(jù)誰需要元數(shù)據(jù)業(yè)務(wù)經(jīng)理業(yè)務(wù)經(jīng)理元數(shù)據(jù)元數(shù)據(jù)數(shù)據(jù)倉庫項(xiàng)目管理控制數(shù)據(jù)倉庫項(xiàng)目管理控制如：定義項(xiàng)目范圍、控制項(xiàng)目風(fēng)險(xiǎn)、評估不同系統(tǒng)集成時(shí)產(chǎn)

5、生的影響（影響分析）分析人員分析人員13元數(shù)據(jù)元數(shù)據(jù)開發(fā)人員開發(fā)人員業(yè)務(wù)人員業(yè)務(wù)人員項(xiàng)目經(jīng)理項(xiàng)目經(jīng)理架構(gòu)師架構(gòu)師誰需要元數(shù)據(jù)誰需要元數(shù)據(jù)業(yè)務(wù)經(jīng)理業(yè)務(wù)經(jīng)理項(xiàng)目架構(gòu)設(shè)計(jì)項(xiàng)目架構(gòu)設(shè)計(jì)如：了解多個(gè)業(yè)務(wù)系統(tǒng)的結(jié)構(gòu)、關(guān)聯(lián)關(guān)系，數(shù)據(jù)轉(zhuǎn)換規(guī)則設(shè)計(jì).分析人員分析人員14元數(shù)據(jù)的存儲區(qū)元數(shù)據(jù)的存儲區(qū)15數(shù)據(jù)倉庫元數(shù)據(jù)結(jié)構(gòu)圖數(shù)據(jù)倉庫元數(shù)據(jù)結(jié)構(gòu)圖16元數(shù)據(jù)管理示意圖元數(shù)據(jù)管理示意圖源數(shù)據(jù)源數(shù)據(jù)EDW（ODS+DW+DM）前端工具前端工具connections暫存區(qū)暫存區(qū)元數(shù)據(jù)元數(shù)據(jù)數(shù)據(jù)庫數(shù)據(jù)庫17Informatica Metadata MangerMetadata Object Browser and Graphi

6、cal ReportsPowerCenter Integration ServerSuperGlue WarehouseMetadata SourcesSG Integration RepositoryCWM Class MetaModelPresentation ServerMetadata XConnectsSG ConsoleRole based dashboards and Metadata ReportsWin NT, 2000, XPWarehouse- Oracle 8.1.7, 9.2.0.1 and above, 10g- SQL Server 2000 SP3- DB2 U

7、DB 8.x EE/EEEIntegration Repo- Oracle 8.1.7, 9.2.0.1 and above, 10g- SQL Server 2000 SP3- DB2 UDB 7.x EE/EEE and aboveREPOSITORIES18PowerCenter Integration ServerInformatica Metadata ManagerMetadata Object Browser and Graphical ReportsSuperGlue WarehouseSG Integration RepositoryCWM Class MetaModelPr

8、esentation ServerMetadata XConnectsSG ConsoleRole based dashboards and Metadata ReportsWin NT, 2000, XPSERVERSPC 7.1.1 Integration ServerPC 7.1.1 on : NT, SOLARIS HP Unix, IBM AIX(PC supported versions)PA 4.1.1 Apps ServerBrowser IE 6.0 using : iPlanet 6.0, IIS 6.0 OR Apache 1.3BEA WL 6.1 SP4 on :-

9、NT 4.0 SP5, NT 2000 SP2, Win 2003- SOLARIS 7.0, 8.0- HP Unix 11.0, 11i- IBM AIX 4.3, 5.1Websphere 5.1 on :- NT 4.0 SP5, NT 2000 SP2, Win 2003- SOLARIS 7.0, 8.0- IBM AIX 4.3, 5.1Metadata Sources19Informatica Metadata ManagerMetadata Object Browser and Graphical ReportsPowerCenter Integration ServerSu

10、perGlue WarehouseSG Integration RepositoryCWM Class MetaModelPresentation ServerMetadata XConnectsSG ConsoleRole based dashboards and Metadata ReportsWin NT, 2000, XPETL- PowerCenter 5.x, 6.x, 7.x*- PowerMart 5.x, 6.x, 7.x*BI- PowerAnalyzer 4.x*- BOBJ 5.0, 5.1.x, 6.x- COGN Impromptu 7.x- COGN Report

11、Net 1.x- MSTR Designer 7.x- DB2 CubeViewsXCONNECTS* (on Oracle 8.1.7 and above, SQL Server 2000 SP3 and DB2 7.x EE/EEE and above)Databases- Oracle 8.1.7, 9i, 10g- SQL Server 2000- Sybase 12.x- TERADATA V2 R5 - DB2 UDB 7.1,7.2,8.1 EE/EEE- IBM Informix 9.2Design Tools- ERwin 3.0 to 3.5.2, 4.0 SP1 to 4

12、.1- Oracle Designer 1.3.2, 2.1.2, 6.0, 6i, 9i - Sybase PowerDesigner PDM 6.1.x, 7.5 to 9.5- IBM Rational ER 98(I) to 2000,2000e to 2002- Microsoft VISIO Database (ERX)- Embarcadero ERStudio 5.1 to 6.0 Metadata Sources20Source definition 源系統(tǒng)地址源系統(tǒng)地址, 網(wǎng)絡(luò)連接網(wǎng)絡(luò)連接, 訪問方式訪問方式Location, network connection, acc

13、ess methods計(jì)算機(jī)系統(tǒng)計(jì)算機(jī)系統(tǒng), 操作系統(tǒng)操作系統(tǒng)Computer platform definition, operating system definition數(shù)據(jù)庫服務(wù)器名數(shù)據(jù)庫服務(wù)器名,訪問方式訪問方式,數(shù)據(jù)結(jié)構(gòu)數(shù)據(jù)結(jié)構(gòu),有關(guān)的表有關(guān)的表,列列Database server name, access methods, schema, table, elements of interests源數(shù)據(jù)庫鏈接說明源數(shù)據(jù)庫鏈接說明 Source database links逆工程與源系統(tǒng)設(shè)計(jì)獲取定義說明逆工程與源系統(tǒng)設(shè)計(jì)獲取定義說明Reverse engineering and so

14、urce design capture specification元數(shù)據(jù)交換說明元數(shù)據(jù)交換說明Metadata exchange definition 21Staging area definition 源至?xí)捍鎱^(qū)映像（Source-staging mapping）暫存區(qū)至目標(biāo)區(qū)映像Staging-target mapping中間文件的傳輸說明File transport specification中間文件的說明及使用File specification and usage中間文件的前后處理(清理與壓縮)說明File pre/post event processing specificatio

15、n暫存區(qū)數(shù)據(jù)庫的前后處理及中間處理說明及使用Staging database usage and pre/post process specification 22Target modeling definition 星結(jié)構(gòu)Star schema事實(shí)表定義說明Fact table definition度量Measurement數(shù)字鍵Numeric key交替鍵Foreign key索引Index definition定義數(shù)據(jù)（如數(shù)據(jù)類型，數(shù)值大小和省缺值）有效性制約Validation constraint數(shù)據(jù)域值定義Data range validation分區(qū)定義說明Partition s

16、pecification存儲說明Storage specification維表定義與說明Dimension table definition主鍵Primary key緩慢變化的維Slowly changed dimension維的等級/層次Dimension hierarchy維的共享Dimension sharing維的版號化Dimension versioning數(shù)據(jù)庫完整性限定Database integrity constraints23Target modeling definition 維表定義與說明Dimension table definition主鍵Primary key緩慢

17、變化的維Slowly changed dimension維的等級/層次Dimension hierarchy維的共享Dimension sharing維的版號化Dimension versioning數(shù)據(jù)庫完整性限定Database integrity constraints24Mapping definition 源至目標(biāo)映像Source-target mapping基本數(shù)據(jù)到綜合數(shù)據(jù)的映像Basic data-summary mapping標(biāo)準(zhǔn)字典Standard dictionary 轉(zhuǎn)換字典Conversion dictionary參照表Lookup tables映像表Mapping

18、tables 25ETL definition 轉(zhuǎn)換規(guī)則和方法Conversion rule and method數(shù)據(jù)清洗說明Data cleaning specifications批作業(yè)說明Batch job specification加載頻率Loading frequency累加時(shí)間片控制說明Incremental time slice control specification回退說明與準(zhǔn)則Rollback specification and policy剝離與修改數(shù)據(jù)準(zhǔn)則Strip and modification in bound data policy 轉(zhuǎn)換函數(shù)庫Conversio

19、n function library傳輸定義與說明Transportation definition加載定義與說明Loading definition26ETL definition 會期/對話管理定義與說明Session management definition預(yù)定作業(yè)排時(shí)Scheduling批作業(yè)前后事件說明及處理方法Pre-batch and post-batch job events批作業(yè)日志Batch logging會期/對話日志Session log拒收日志Rejected log壞數(shù)據(jù)日志Bad data log 27Summary management definition

20、事實(shí)表/維表至總計(jì)方陣映像Fact/dimension-summary cube mapping總計(jì)至總計(jì)映像Summary-summary mapping總計(jì)方陣設(shè)計(jì)Summary cube layout聚合定義說明Aggregate definition實(shí)體化視圖/方陣定義說明Materialized view/cube definition方陣間關(guān)系Relationships between cubes嵌入式實(shí)體化視圖/方陣之間的依賴性Dependency of nested materialized views/cubes方陣組 Grouping of cubes28Summary m

21、anagement definition 刷新過程的定義與說明Refreshing definition刷新前后事件處理說明Pre-refreshing and post refreshing events刷新方法Refreshing methods刷新組Refreshing groups刷新執(zhí)行順序Refreshing sequence刷新頻率Refreshing frequency刷新過程處理軟件Refreshing procedures刷新作業(yè)的監(jiān)控檢查Refreshing monitoring29OLAP definition 查詢機(jī)制Query engine虛擬方陣/視圖定義Virtual cube/view definition訪問權(quán)限管理說明Privilege management specification用戶接口定義與說明User interface definition訪問方式Access method結(jié)果數(shù)據(jù)提交方法Delivery method and definition結(jié)果數(shù)據(jù)傳輸定義與說明Result set transportation definition用戶登入/登出User log in/log off用戶活動日志以及出錯處理User logging and error han