矩陣式項目管理課件

MetricsandBenchmarkingPresentedtoWayneKellyPrincipalConsultantQualityAssurancePracticeApril27,2004MetricsandBenchmarkingPresenContentsIntroductiontoMetricsWhatisMetrics?WhyMetrics?ProjectMetricsProgramObjectivesMetrics-CustomerNeedMeasuresandMetricsImprovementCycleMeasures–DataElementsToolsforMeasurementMetricsCalculationMetricsRoadmapMetricsAnalysis-Examples2ContentsIntroductiontoMetricItisnotsufficienttoproducesoftwaredefectmetrics-likehowmanyraised,howmanyoutstanding,fixedbyseverityandurgency.Wemustprovethattheactivitiesofthetestteamareacosteffectiveandefficientuseofstaff.TestMetricsWithoutTears-DavidVaughnQuote3of37ItisnotsufficienttoproducIntroductiontoMetrics4of37IntroductiontoMetrics4of37WhatIsMetrics?Metrics-Derivedfromtheword“metre”-Quantitativeindicatorofperformanceofas/wprocessorproduct.Metricsisacannot-do-withoutprojectmanagementtoolVisibilityintoprojectexecutionAnalysisofprocessstrengthsandweaknessesyoucannotmanageorimprovewhatyoucannotmeasure“Youcan’tcontrolwhatyoucan’tmeasure”-TomDeMarco5of37WhatIsMetrics?Metrics-DeriEstablishameasurementandcontinousimprovementcultureacrossthecompany.ImprovedqualityofsoftwareacceptedintoproductionHelpmanagementandemployeesmakewell-informedanddecisionsAlertmanagementaboutriskstoprojectgoals,schedulesandsoftwarequalityProvidesanobjectiveassessmentofthestateofthesoftwareWhyMetrics?6of37EstablishameasurementandcoWhyMetrics?ContinuedMetricsprovideinputsforfutureestimationsandplanningTheactualperformanceofthepastinquantitativetermsbecomesastartingpointforfutureestimatesHelpsinsettinggoalsandtargetsBetterresourceplanning

MetricsidentifytheareasforimprovementsForexample,metricscanidentifythephasewhichintroducesthemaximumnumberoferrorsMetricscanbeusedtoeliminateproblemareasandrootcauses7of37WhyMetrics?Continued7of37SeniorManagementProjectManagementProjectTeamsBenchmarkingSettingimprovementgoalsPerformanceacrossgroupsHowlargeisproject,ReducedefectsBetterestimatesattasklevelConsistency,predictabilityBetterestimation,projectcontrolEffort,defect,schedule,sizeCostofQuality,Effort&DefectDistribution,Quality,ProductivityEstimationAccuracy,Quality,Productivity,DevelopmentCost,ROIWhyisitMeasuredWhatisMeasured

Measurementsdrivecontinuousimprovementacrosstheorganization8of37ProjectManagementProjectBenchMetricsProgramObjectivesMustbeclearlydefinedandPOSITIVE

DrivenbycriticalbusinessobjectivesMeasuresattheenterprise,program,projectandapplicationlevelsProvidequantitativeinformationofkeydrivers(i.e.schedule,estimatingaccuracy,defects)thatimprovesdecisionmakingintimetoaffectthebusinessoutcomeandsuccessfulmanagementofaproject9of37MetricsProgramObjectivesMustCUSTOMERPERSPECTIVE"WHATCUSTOMERNEEDSMUSTWESERVE?”LEARNINGPERSPECTIVE"TOACHIEVEOURGOALS,HOWMUSTOURORGANIZATIONLEARN,INNOVATEANDIMPROVE?”“WHATFINANCIALOBJECTIVESMUSTWEACCOMPLISH?WHATISTHECOSTPerFUNCTIONPOINT?”FINANCIALPERSPECTIVE“TOSATISFYOURCUSTOMERS,INWHICHINTERNALBUSINESSPROCESSESMUSTWEEXCEL?”INTERNALPERSPECTIVEMultipleperspectivesofaMetricsprogram10of37CUSTOMERPERSPECTIVE"WHATCUSTMeasuresandMetrics11of37MeasuresandMetrics11of37MeasuresMeasuresarethefundamentalbuildingblocksofameasurementsystemMeasuresDefectsSizeEffortScheduleCost12of37MeasuresMeasuresarethefundaMetricsMetricsarederivedoutofmeasuresMeasuresMetricsDefectsSizeEffortScheduleCostProductivityDeliveredQualityDefectRemovalEfficiencyCostofQualityReviewEffectivenessInternalPerspective13of37MetricsMetricsarederivedoutMetricsMetricsarederivedoutofmeasuresMeasuresMetricsDefectsSizeEffortScheduleCostProductivityDeliveredQualityDefectRemovalEfficiencyCostofQualityRevieweffectivenessReturnonInvestmentTotalCostofOwnershipInternalPerspectiveFinancialPerspective14of37MetricsMetricsarederivedoutImprovementCycleGatherAnalyseActImproveMeasuresMetricsDefectsSizeEffortScheduleCostProductivityDeliveredQualityDefectRemovalEfficiencyCostofQualityRevieweffectivenessReturnonInvestmentTotalCostofOwnershipLearningPerspective15of37ImprovementCycleGatherAnalyseMeasures-DataElementsDefects#ofdefectsDefectdescriptionStageinjected&detectedSeverityStatusEffortPlannedvs.actualActivities(Task,review,rework)ResourcegroupsWeekendingdatePhasewisepersoneffortMeasuresMetricsDefectsSizeEffortScheduleCost

ScheduleStartdate(Planned,Actual)Enddate(Planned,Actual)Milestonedate(Planned,Actual)SizeProgramreferenceLanguageusedSourceLinesofCode(InitialandFinal)FPoutofSLOCCostPerPhasePlanVsActGroup16of37Measures-DataElementsDefectsMeasures–ToolsUsed

toAutomaticallyCapturetheDataElementsDefects

ExcelTemplateClearQuestDefectLogsEffortExcelTemplateTimeTrackingMSProjectPlanViewMeasuresMetricsDefectsSizeEffortScheduleCost

ScheduleWBSExcelTemplateMSProjectPlanViewSizeSLOCcountersdependingonthelanguageLaborUnitCostProjectBudgets17of37Measures–ToolsUsed

toAutoMeasures–Size

Exampleoftools18of37Measures–Size

ExampleoftooMeasures–Effort

ExampleofTools19of37Measures–Effort

ExampleofTMeasures–Defect

ExampleofTools20of37Measures–Defect

ExampleofTMetricsAreCalculatedUsingMeasuresMetricsUnitofMeasureMeasuresFormulaProductivityQualityDefectRemovalEfficiencyCostofQualityReviewEffectivenessFP/pmDelivereddefects/pm%%%SizeEffortDefectsSizeDefectsEffortDefectsSizeoftheproduct/TotalEffortinpersonmonthsDefectsfoundduringAcceptanceTesting/SizeOftheproductTotaldefectsdetectedbeforeQA/TotaldefectsdetectedincludingQAReview+Test+Training+Reworkeffort*100/TotaleffortfortheprojectDefectsdetectedbymeansofreviews/Defectsdetectedbymeansofreviews+testing21of37MetricsAreCalculatedUsingMMetricsRoadmap22of37MetricsRoadmap22of37MetricProgram

CriticalSuccessFactorsDefineclearobjectivesandkeepthemPOSITIVEDrivenbykeybusinessissuesEnsurethattheprocessisrolledouteffectively–targetasubset,test,validateandimproveBeginwithmanualmethodsthen,considerautomation(DataDrill)Train,Train,TrainProvidetimefortheprocessestomature;variabilitytendstobeveryhighinitially–expectnoiseandchangeAssisttheprojectteamsinunderstanding,analyzingandimprovingtheprocessesthroughmetricreportingCommunicate,Communicate,Communicate23of37MetricProgram

CriticalSucceObtainExecutiveSponsorDefinethebusinessandITobjectivesforthemeasurementandimprovementprogram(mustbedatadriven)ConductJADsessiontoidentifypossiblemeasuresSelectmeasuresthatalignwithprogramobjectivesandimproveITperformanceIdentifytheinformationrequirement,author,datasourceandtoolsIdentifythecoresetorphasedimplementationplanformeasuresandvalidatewithauthor,contributoranduserthattheyalignwithproject,departmentandorganizationalgoalsImplementingAMetricsProgram24of37ImplementingAMetricsProgramDevelopthemetriccollectionandreportingsolutionincludingautomation,collection,graphingandreportingprocessesTest,finetune,verifyandpilotallreportsandmeasuresPackagemultiplemeasuresintoametricsreleaseIdentifyandtrainacoreteamofauthors,contributorsandusersintheinterpretationanduseofallmeasurestoimprovebusinessresultsProductionrolloutbeginsMonitor,improve,retireandimplementnewmeasuresastheorganizationmaturesinprocessandmeasurementcapabilitiesCapture,communicateandinformallofProgramresultsIntegrateintoaBusinessperformanceProgramImplementingAMetricsProgram25of37DevelopthemetriccollectionAlignmentwiththeITGovernanceand/orPMOProgramAlignmentwithcertificationobjectives(CMM/CMMI/Iso,etc)DefinitionofmeasurementrolesandresponsibilitiesIntegrationintotheteam,departmentandorganizationalprocessesConsistencyinthecodingandcollectionofmetricsProduction-izingthecollectionandreportingprocessOverallreportdesignandinterpretationTQMleadershipandfacilitationAlignmentwiththeorganizationalscorecardRelevantindusrtyfiguresforbenchmarkingobjectivesMetricsProgram

Dependencies26of37AlignmentwiththeITGovernaTheBusinessPerformanceProgram

27of37TheBusinessPerformanceProgr1Initial2Repeatable3Defined4Managed5OptimizingDatadefinitionandcollectionisstandardizedacrosstheOrganizationDataisusedtounderstandtheprojectQuantitativelyEstablishPCBFocusonreducingvariabilityDataiscollectedandusedinalldefinedprocessesDataissystematicallysharedacrossprojects

StandardizemetricscollectionImprovemetricsreliabilityOrganizationlevelProcessDatabaseIndividualprojectsplanandmanagedataStartcollectionofmeasuresProjectlevelmetrics

Dataisusedtoevaluateandselectprocessimprovements

ImprovemeancapabilityMetricsRoadmap

inTermsofCMMLevelsPredictabilityGoesUp28of371Initial2Repeatable3Defined4MaLevel2-Whatyoucanexpect…DeploymentMetricsatProjectlevelformonitoringandcontrollingSQAisidentifiedtohelpprojectsinmetricscollectionToomuchoffollow-upfordataRegularstatustrackingofprojectsusingmetricsBenefits

CultureisbeingestablishedProjectleveldecisionsaretakenbasedonmetricsTrackingbecomeseasierinprojectsAwareness/CultureMetricsidentificationanddefinitionOrientationonvariousmetricsAlittlecynicismfromprojectsideFearofmetricsusageagainstindividualsInfrastructureUseofsimpletoolsformetricsManualcollationofdatafromprojects29of37Level2-Whatyoucanexpect…Level3-Whatyoucanexpect…DeploymentSupportfromSQAformetricsgetsreducedCapturingmetricsbecomesawayoflifeImprovementingoodnessofdataBenefits

DecisionsaretakenatorganizationlevelbasedonmetricsProcessimprovementsaredrivenbasedonmetricsImprovementsinEffortestimationaccuracy,CostofQuality,Scheduleadherence,Productivity,Quality,etc..Awareness/CulturePurposeofmetricscollectionisknowntoeveryoneCynicismgoesawayManyaretrainedinMetrics,FPcounting,EstimationusingpastprojectsdataInfrastructureITwiderepositoryformetricsStandardizedtoolsacrossprojectsCollationofdatabecomeseasier30of37Level3-Whatyoucanexpect…MetricsAnalysis-Examples31of37MetricsAnalysis-Examples31Example–Effortanalysis32of37Example–Effortanalysis32ofExamples–Defectanalysis33of37Examples–Defectanalysis33oExamples–Codechanges34of37Examples–Codechanges34of3Examples–Defectanalysis35of37Examples–Defectanalysis35oExamples–Defectcategorization36of37Examples–DefectcategorizatiQuestionsand

Answers

YourFavoriteandMostEffectiveGraphsWayneKelly–PrincipalConsultantLizaRoberts–BusinessDevelopmentManagerlroberts@804-334-289537of37Questionsand

Answers

YourFMetricsandBenchmarkingPresentedtoWayneKellyPrincipalConsultantQualityAssurancePracticeApril27,2004MetricsandBenchmarkingPresenContentsIntroductiontoMetricsWhatisMetrics?WhyMetrics?ProjectMetricsProgramObjectivesMetrics-CustomerNeedMeasuresandMetricsImprovementCycleMeasures–DataElementsToolsforMeasurementMetricsCalculationMetricsRoadmapMetricsAnalysis-Examples39ContentsIntroductiontoMetricItisnotsufficienttoproducesoftwaredefectmetrics-likehowmanyraised,howmanyoutstanding,fixedbyseverityandurgency.Wemustprovethattheactivitiesofthetestteamareacosteffectiveandefficientuseofstaff.TestMetricsWithoutTears-DavidVaughnQuote40of37ItisnotsufficienttoproducIntroductiontoMetrics41of37IntroductiontoMetrics4of37WhatIsMetrics?Metrics-Derivedfromtheword“metre”-Quantitativeindicatorofperformanceofas/wprocessorproduct.Metricsisacannot-do-withoutprojectmanagementtoolVisibilityintoprojectexecutionAnalysisofprocessstrengthsandweaknessesyoucannotmanageorimprovewhatyoucannotmeasure“Youcan’tcontrolwhatyoucan’tmeasure”-TomDeMarco42of37WhatIsMetrics?Metrics-DeriEstablishameasurementandcontinousimprovementcultureacrossthecompany.ImprovedqualityofsoftwareacceptedintoproductionHelpmanagementandemployeesmakewell-informedanddecisionsAlertmanagementaboutriskstoprojectgoals,schedulesandsoftwarequalityProvidesanobjectiveassessmentofthestateofthesoftwareWhyMetrics?43of37EstablishameasurementandcoWhyMetrics?ContinuedMetricsprovideinputsforfutureestimationsandplanningTheactualperformanceofthepastinquantitativetermsbecomesastartingpointforfutureestimatesHelpsinsettinggoalsandtargetsBetterresourceplanning

MetricsidentifytheareasforimprovementsForexample,metricscanidentifythephasewhichintroducesthemaximumnumberoferrorsMetricscanbeusedtoeliminateproblemareasandrootcauses44of37WhyMetrics?Continued7of37SeniorManagementProjectManagementProjectTeamsBenchmarkingSettingimprovementgoalsPerformanceacrossgroupsHowlargeisproject,ReducedefectsBetterestimatesattasklevelConsistency,predictabilityBetterestimation,projectcontrolEffort,defect,schedule,sizeCostofQuality,Effort&DefectDistribution,Quality,ProductivityEstimationAccuracy,Quality,Productivity,DevelopmentCost,ROIWhyisitMeasuredWhatisMeasured

Measurementsdrivecontinuousimprovementacrosstheorganization45of37ProjectManagementProjectBenchMetricsProgramObjectivesMustbeclearlydefinedandPOSITIVE

DrivenbycriticalbusinessobjectivesMeasuresattheenterprise,program,projectandapplicationlevelsProvidequantitativeinformationofkeydrivers(i.e.schedule,estimatingaccuracy,defects)thatimprovesdecisionmakingintimetoaffectthebusinessoutcomeandsuccessfulmanagementofaproject46of37MetricsProgramObjectivesMustCUSTOMERPERSPECTIVE"WHATCUSTOMERNEEDSMUSTWESERVE?”LEARNINGPERSPECTIVE"TOACHIEVEOURGOALS,HOWMUSTOURORGANIZATIONLEARN,INNOVATEANDIMPROVE?”“WHATFINANCIALOBJECTIVESMUSTWEACCOMPLISH?WHATISTHECOSTPerFUNCTIONPOINT?”FINANCIALPERSPECTIVE“TOSATISFYOURCUSTOMERS,INWHICHINTERNALBUSINESSPROCESSESMUSTWEEXCEL?”INTERNALPERSPECTIVEMultipleperspectivesofaMetricsprogram47of37CUSTOMERPERSPECTIVE"WHATCUSTMeasuresandMetrics48of37MeasuresandMetrics11of37MeasuresMeasuresarethefundamentalbuildingblocksofameasurementsystemMeasuresDefectsSizeEffortScheduleCost49of37MeasuresMeasuresarethefundaMetricsMetricsarederivedoutofmeasuresMeasuresMetricsDefectsSizeEffortScheduleCostProductivityDeliveredQualityDefectRemovalEfficiencyCostofQualityReviewEffectivenessInternalPerspective50of37MetricsMetricsarederivedoutMetricsMetricsarederivedoutofmeasuresMeasuresMetricsDefectsSizeEffortScheduleCostProductivityDeliveredQualityDefectRemovalEfficiencyCostofQualityRevieweffectivenessReturnonInvestmentTotalCostofOwnershipInternalPerspectiveFinancialPerspective51of37MetricsMetricsarederivedoutImprovementCycleGatherAnalyseActImproveMeasuresMetricsDefectsSizeEffortScheduleCostProductivityDeliveredQualityDefectRemovalEfficiencyCostofQualityRevieweffectivenessReturnonInvestmentTotalCostofOwnershipLearningPerspective52of37ImprovementCycleGatherAnalyseMeasures-DataElementsDefects#ofdefectsDefectdescriptionStageinjected&detectedSeverityStatusEffortPlannedvs.actualActivities(Task,review,rework)ResourcegroupsWeekendingdatePhasewisepersoneffortMeasuresMetricsDefectsSizeEffortScheduleCost

ScheduleStartdate(Planned,Actual)Enddate(Planned,Actual)Milestonedate(Planned,Actual)SizeProgramreferenceLanguageusedSourceLinesofCode(InitialandFinal)FPoutofSLOCCostPerPhasePlanVsActGroup53of37Measures-DataElementsDefectsMeasures–ToolsUsed

toAutomaticallyCapturetheDataElementsDefects

ExcelTemplateClearQuestDefectLogsEffortExcelTemplateTimeTrackingMSProjectPlanViewMeasuresMetricsDefectsSizeEffortScheduleCost

ScheduleWBSExcelTemplateMSProjectPlanViewSizeSLOCcountersdependingonthelanguageLaborUnitCostProjectBudgets54of37Measures–ToolsUsed

toAutoMeasures–Size

Exampleoftools55of37Measures–Size

ExampleoftooMeasures–Effort

ExampleofTools56of37Measures–Effort

ExampleofTMeasures–Defect

ExampleofTools57of37Measures–Defect

ExampleofTMetricsAreCalculatedUsingMeasuresMetricsUnitofMeasureMeasuresFormulaProductivityQualityDefectRemovalEfficiencyCostofQualityReviewEffectivenessFP/pmDelivereddefects/pm%%%SizeEffortDefectsSizeDefectsEffortDefectsSizeoftheproduct/TotalEffortinpersonmonthsDefectsfoundduringAcceptanceTesting/SizeOftheproductTotaldefectsdetectedbeforeQA/TotaldefectsdetectedincludingQAReview+Test+Training+Reworkeffort*100/TotaleffortfortheprojectDefectsdetectedbymeansofreviews/Defectsdetectedbymeansofreviews+testing58of37MetricsAreCalculatedUsingMMetricsRoadmap59of37MetricsRoadmap22of37MetricProgram

CriticalSuccessFactorsDefineclearobjectivesandkeepthemPOSITIVEDrivenbykeybusinessissuesEnsurethattheprocessisrolledouteffectively–targetasubset,test,validateandimproveBeginwithmanualmethodsthen,considerautomation(DataDrill)Train,Train,TrainProvidetimefortheprocessestomature;variabilitytendstobeveryhighinitially–expectnoiseandchangeAssisttheprojectteamsinunderstanding,analyzingandimprovingtheprocessesthroughmetricreportingCommunicate,Communicate,Communicate60of37MetricProgram

CriticalSucceObtainExecutiveSponsorDefinethebusinessandITobjectivesforthemeasurementandimprovementprogram(mustbedatadriven)ConductJADsessiontoidentifypossiblemeasuresSelectmeasuresthatalignwithprogramobjectivesandimproveITperformanceIdentifytheinformationrequirement,author,datasourceandtoolsIdentifythecoresetorphasedimplementationplanformeasuresandvalidatewithauthor,contributoranduserthattheyalignwithproject,departmentandorganizationalgoalsImplementingAMetricsProgram61of37ImplementingAMetricsProgramDevelopthemetriccollectionandreportingsolutionincludingautomation,collection,graphingandreportingprocessesTest,finetune,verifyandpilotallreportsandmeasuresPackagemultiplemeasuresintoametricsreleaseIdentifyandtrainacoreteamofauthors,contributorsandusersintheinterpretationanduseofallmeasurestoimprovebusinessresultsProductionrolloutbeginsMonitor,improve,retireandimplementnewmeasuresastheorganizationmaturesinprocessandmeasurementcapabilitiesCapture,communicateandinformallofProgramresultsIntegrateintoaBusinessperformanceProgramImplementingAMetricsProgram62of37DevelopthemetriccollectionAlignmentwiththeITGovernanceand/orPMOProgramAlignmentwithcertificationobjectives(CMM/CMMI/Iso,etc)DefinitionofmeasurementrolesandresponsibilitiesIntegrationintotheteam,departmentandorganizationalprocessesConsistencyinthecodingandcollectionofmetricsProduction-izingthecollectionandreportingprocessOverallreportdesignandinterpretationTQMleadershipandfacilitationAlignmentwiththeorganizationalscorecardRelevantindusrtyfiguresforbenchmarkingobjectivesMetricsProgram

Dependencies63of37AlignmentwiththeITGovernaTheBusinessPerformanceProgram

64of37TheBusinessPerformanceProgr1Initial2Repeatable3Defined4Managed5OptimizingDatadefinitionandcollectionisstandardizedacrosstheOrganizationDataisusedtounderstandtheprojectQuantitativelyEstablishPCBFocusonreducingvariabilityDataiscollectedandusedinalldefinedprocessesDataissystematicallysharedacrossprojects

Standardizemetrics