Using Earned Value in Scientific Research David Roberts & Sheila Roberts CUPE International.
Discussion of EVM in Scientific Research Exploring a behavioural approach to adopting EVM. Challenges Mindset Scientific work and mindset Earned Value Management work and mindset Culture Can Science & Management cultures work with EVM? What Outcomes and Benefits would be mutually attractive to Science & Management? Implementation Approach Evidence Is there compelling evidence for the use of EVM? What are the options for change to using EVM? Collaboration What we did: working together to design EVM Light What others said in response: Stakeholder Quotes/Perspectives Learning Agility & Engagement Summary of our working approach. Learning EVM Evolution: Tailoring around the Teams: so far-so good. 2
The Challenge: Can the value of Scientific Research be improved using EVM? An international scientific programme; is seeking to drive greater value from current vaccine stockpiles future Global Pandemic It is a challenging test environment for EVM providing PM insight in the years ahead. 3
Scientific Concerns & Methods Can vaccine stockpiles prevent the next Pandemic? Pandemicpandemic? In global health, emergency vaccine stockpiles are like the insurance policy you never really wanted to take out; you resent the cost and have mixed feelings about never making a claim. Ideally, you never have to use it. 4
Global trends in Urbanisation (UN 2015) The worst-case scenario is an outbreak in huge cities, such as Mumbai, Beijing or Lagos, where even diseases that are relatively difficult to transmit, like Ebola, could prove difficult to contain. Such so-called mega-cities are a growing trend. 5
Scientific Research & Work Flow Review Existing Stockpiles Scientific Work Research Next Mutations Research Vaccine Scalability Improved Insurance Policy 6
Scientific Mindset Scientific Information takes the form of research findings and statistical data Work packages comprise Project planning, laboratory testing and mathematical modelling. Visualisation of this work can be expressed using Antigenic Maps. 7
Scientific Paradigm 8
Value Management Principles 9
EVM Earning Value Better Performance What and why? Traditional Planned Value: PMB Baseline: Task driven plans; Fixed scope, schedule, costs Focus Forwards Cost & Schedule Variance Agile Planned Value: PRL Baseline: Fixed time & costs; Multiple sprint plans, changes, delivery Focus Forwards Functionality & Features Variance Estimate to Completion or Estimate at Completion 10
Earned Value Management Mindset Management Information takes the form of Plans, Budgets, Accounting Systems, and Control Reports. Findings and statistical data provide Forecasts and Lessons. 11
EVM Paradigm Normal Earned Value Management Planned value is at the heart of project work; EVM focusses on cost/schedule estimates, variations and forecasts to agreed budgets, using measurements, ratios and indices. Improved EVM performance requires changes to planned project processes and/or practices, innovation & smarter ways of working, leading to better project outcomes & higher project-team/stakeholder satisfaction. Inspires PMs to deliver stakeholder goals 12
Common & Contrasting Cultures Science Activities in common: Measurement Data Analysis Statistics Lessons Attitudes can be different: Team or self focus Attract resources Freedom in decisions Puzzles or Discovery Management Activities in Common: Measurement Data Analysis Statistics Lessons Attitudes can be different: Organisational focus Return on resources Decisions to control Accountability Activities quite similar but attitudes quite dissimilar 13
Blending Science & Management Methods Can Earned Value Management work in Science? Science and management are seen as polar opposites; management simply gets in the way of good science! Little understood evidence that PM or VM works (Gartner et al), despite 100 years of scientific management. 14
Does EVM Work? GAO Cost Estimating & Assessment Guide:- -17 Best Practices checklist for cost estimates 48 case studies; 9 graded levels of technology Project personnel should use the GOA guide as applicable to Office of Science Projects. 15
Does EVM Work? Journal of Cost Analysis and Parametrics:- - According to David Christensen and Kirk Payne, in research conducted by the U.S. Air Force in the early 1990s: A project's cumulative CPI at 20% project completion, doesn't change by more than 10% for remaining project. If a project is underperforming financially at 20% project completion, it won't improve unless significant corrective measures are taken. 16
Paradigm Shift or Evolution Can Earned Value Management really bring about a transformation in science? 17
Paradigm Shift or Evolution? Can Earned Value Management really bring about a transformation in science? 18
Nudge Theory would suggest EVM Light EVM is now mandated for publically-funded projects in USA; An implementation standard ANSI 748;and APM Compass: Software Tools too; Tailoring EVM to fit Scientific Research Culture and Projects. 19
Part Two: What should EVM Light look like - Implementation and Learning. 20
Stakeholder Interests (1-4) 1.Scientists 2.Universities Discovering Enabling & Solving Capability EVM De-Risking & Delivering 3. Research Institute 4. Government - Demonstrating Value 21
Stakeholder Value is Key to EVM 22
Stakeholder Value Drivers Scientists: Puzzle Solving & Discovering - As a researcher I want to explore what is unknown and publish discoveries Universities: Enabling Capability - As a University I want to build an International reputation for high quality research Research Institute: De-Risking & Delivering As an Institute I want to commission and implement the outcomes of scientific research Government: Demonstrating Value As a Government Department I want VFM from taxes 23
Magic Recipe EVM Light Ingredients Outcomes EVMS Expertise Behavioural EVM Plan Timely Reports Healthy Debate Better decisions 24
PROJECT BOARD EVMS DECISION MAKING Approvals Plans Controls Updates Changes Risk Oversight Current Performance Forecast to Completion Reliability EVMS & Integrity Compliance Outcomes Reports and Forecasts CONTRACT Scope CHANGES ANALYSIS & CONTROL LESSONS PROJECT Agreed Contract with SoW Appoint PM & Scientific Teams Control A/C Mgmt Risk Register Create Create OBS OBS EVMS ENGINE Project Plan WBS WP PP RAM (CAs & CAMs) EVTs Create Project Schedule Confirm Milestones Allocate budgets to CA/WPs Cost/Schedule Risk Analysis Establish PMB PROJECT CONTROL Update cost schedule Baseline Maintenance Risk Management Issues & Changes PROJECT ANALYSIS PMB Variances C/ SPI Cost Performance s CLIENT :PROJECT PERFORMANCE Reporting m/q Reports annual Forecasts /EAC CHANGE REQUEST Authorised change Re-Plan/Prg. Re-Baseline Update EVMS FINANCIAL SYSTEM Planned Value (BCWS) Budget Allocation Distributed Budget Funding Payments FORECASTING In year outcomes EAC + TCPI CV + SV +VAC Planning assumptions RESEARCH FUNDS 25
EVM Implementation Agile or Traditional Project Environments. Assess EVM Maturity - 5 Level Model (APM Compass) Establish Expectations for Project Reporting & Control Verify/Confirm Current EVM Practices (e.g. Plans, Budgets, Control Accounts/ Control Account Managers) Clarify how the EVM gap is to be filled.(training or Outsourced Services) Validate Customer/Supplier satisfaction with new practice Identify Lessons and implement them in Planning. 26
Solution Outcome Solutions and Outcomes Allow Scientific Research to flourish Allocate techniques based on science Use sprints / prioritized plans Use scientific progress data Update plan when outcomes are known Identified CPR 1 and 5 only Research progresses Plans are current EVM raises important discussion points Collaboration between scientists & managers 27
Solution Solutions and Benefits Allow Scientific Research Enable high quality reputation De-risk delivery of outcomes Confirm value from public spending Benefits Objective Progress measurement techniques Project is continually aligned to objectives Securing funding into the future Confidence in collaboration 28
Learning 29
EVM in Complex Environments 30
Collaborate to Overcome Blockers Government Department Research Institute University Historical Beliefs: Scientists Minimise Management restricts scientific / academic freedom Maximise Research Quality power + prestige + income Comlpy with Governance pre-requisite for future spending Overcome Too cumbersome - too costly without proven benefits? 31
Findings to date: What the Project Managers Say EVM enabled us to show our funders that we, as an academic led-consortium, can manage large multimillion dollar budgets carefully and provide excellent value in our work EVM helps us to more easily identify any areas where spending may have varied against what we forecast and to re-plan accordingly Getting research staff to understand the uses of EVM is a challenge - scientists always prefer to be doing science! 32
Thank you for sharing your time with us. David and Sheila Roberts CUPE International 2018 Office: 01202-555711.