Today s Agenda Why does this matter? A Dangerous Mind Data Collection Data Analysis Data Interpretation Case Studies 2
Why is Data Collection & Analysis Important? 3
Anecdotes vs. Data Rescue groups cherry pick all of the highly adoptable animals. The shelters are full of nothing but pit bulls. In 2015, rescue partners pulled 1,500 animals. Of those: 30% Neonatal Kittens 30% Medical Needs 40% Highly Adoptable Pit Bull Type dogs make up 20% of total dog intake. On average, pit bull type dogs are held for 35 days prior to an outcome versus 14 days for other dogs. Which is more useful for designing and managing programs? 4
Ways to Use Data Analysis to Drive Impact Identify new program opportunities Geo target S/N or TNR programs Monitor compliance with policy Determine Length of Stay Communicate with community Design adoption incentives Identify best/worst kennels Monitor program effectiveness Incentivize and reward staff Motivate volunteers and donors Identify opportunities for better process Save Time Save Money Save Lives
Salt Lake County Animal Services 10K sq ft 88 dogs kennels all indoor 65 cat cages stainless steel 9700 animals LRR 58.5% Statistics available through public records request
Required Statistics for $
The Evolution Historical Data Analyzing both numbers and percentages Evaluating hold times based on outcome
Continued Evolution
Today s Agenda Why does this matter? A Dangerous Mind Data Collection Data Analysis Data Interpretation Case Studies 10
Your Brain Incredibly Powerful, Slightly Dangerous 11
Cognitive Bias Cognitive bias describes the inherent thinking errors that humans make in processing information. These thinking errors can prevent us from accurately understanding reality, even when we have all the needed data and evidence to form an accurate view. Said another way, cognitive bias is the common tendency to acquire and process information by filtering it through one's own likes, dislikes, and experiences. 12
Negativity Bias Negativity Bias is a tendency to notice, pay more attention, or give more weight to negative experiences or information over positive. What is an animal welfare example of negativity bias? 13
Frequency Illusion Frequency Illusion is the phenomenon in which people who just learn or notice something start seeing it everywhere. What is an animal welfare example of frequency illusion?
Today s Agenda Why does this matter? A Dangerous Mind Data Collection Data Analysis Data Interpretation Case Studies 15
Steps in Data Analysis Process Collect & Validate Analyze Interpret & Act 7
Accurate Data Collection is Key: Beware GIGO! Garbage IN Garbage OUT If input data are not complete, accurate, and timely, then the resulting output is unreliable and of no useful value 16
GIGO Example: Cat Intake Zip codes with shelter locations 17
A Few Words on Shelter Software Different features and functions mean no one size fits all approach Data you input, you should be able to extract and analyze Standard and custom reports usually available Use.csv or.xls exports Well worth having staff and/or volunteer member(s) with expertise on your system 18
Google Sheets or Microsoft Excel? Free Cloud based Multiple users can be working on or looking at simultaneously Best for small data sets Will timeout if working on a sheet with many tabs or more than a few thousand rows Lots of add-ons for analysis Pivot tables are easier for beginners Need a license Saved on your computer Only one user can be working on, have to email versions around Best for any data sets Can process lots of data quickly No add-ons available, pretty much everything you need is built in Pivot tables are better 18
Demographic Data can be a Helpful Overlay Demographics The characteristics of human populations and population segments, especially when used to identify consumer markets. Age, Race/Ethnicity, Income, Education, Employment, Language Spoken, Housing Type, Access to Vehicle, etc. Data Sources Zip code summary www.esri.com/data/esri_data/ziptapestry Detailed zip data www.factfinder.census.gov City/County data www.quickfacts.census.gov
Shelter Animals Count Basic Data All Data by Species (Cat/Dog) and age (</> 5 mo) Annual beginning and ending shelter count Intake Stray/At Large, Relinquished by Owner, Owner Intended Euthanasia, Transferred in from Agency, Other Outcomes Adoption, Returned to Owner, Transferred to another Agency, Returned to Field, Other Live Outcome Died in Care, Lost in Care, Shelter Euthanasia, Owner Intended Euthanasia Resources Data Matrix & Definitions What s Your Rate? Calculation Guidance 22
Additional Data is Very Helpful Shelter Animals Count minimum data will allow you to calculate release rates and do basic intake and outcome analysis. Additional data will allow you to do deeper analysis that will benefit your organization so we strongly suggest adding: Breed/Type Intake Address Length of Stay 23
Geographic Information Systems Also Powerful Resources www.mapline.com www.openheatmap.com
Tools Available via ASPCA X Maps Spot Project http://aspcapro.org/resource/saving-lives-research-data/x-maps-spot-tools-gis
Data Collection & Validation Tips Things to do BEFORE you start an analysis 1. Identify what data you want to analyze and why 2. Verify how that data is currently being captured in your system (i.e. who, when, how) 3. Consider whether the data is processed or modified after it is entered (i.e. intact status) 4. Validate that data values are within realm of reality (i.e. negative LOS) 5. Look for anomalies or outliers that need to be explored 21
Don t Despair Help is Available! Where to look for help: Your local government Area universities Professors Class projects Interns Your volunteers! LinkedIn 42
Today s Agenda Why does this matter? A Dangerous Mind Data Collection Data Analysis Data Interpretation Case Studies 28
Data Analysis - OR Lies, Damn Lies, and Statistics Which organization has the best life saving impact? Shelter A Shelter B Shelter C Population Served 600,000 500,000 100,000 Intake 10,000 25,000 5,000 Euthanasia 5,000 6,000 1,500 Euthanasia as % of Intake 50% 24% 30% Euthanasia per 1,000 Pop. 8.3 12 15 Reliance on any single metric will not give a full picture of performance 25
Rescue Groups Should Look At Metrics Too! Which rescue group has the most life saving impact? Kitten Rescue Fluffy Dog Recsue Medical Rescue Foster Homes 10 15 20 Avg. LOS (Days to Adopt) 45 20 60 Annual Animals Placed 240 270 120 Annual Budget $36,000 $81,000 $120,000 Cost Per Animal $150 $300 $1,000 How does YOUR organization s impact match with identified community needs? 25
Let Your Data Speak Good Graphs Part 1 8% 38% 38% 48% 12% 70% 12% Population Served: 1,065,897 Intake per 1,000: 9.7 Euthanasia per 1,000: 4.6 Population Served: 669,561 Intake per 1,000: 6.7 Euthanasia per 1,000: 0.8 26
Good Graphs Part 2 Hygeine To compare, keep same axis scale 38% Use Data Labels to show #s Start data axis at zero Waterfall style graphs give great data visualization 28
Pivot Tables are Your Friend!! A pivot table is a tool in Google Sheets & Excel that allows you to explore large sets of data interactively. Once you create a pivot table, you can quickly transform huge amounts of data into a meaningful summary. 29
Pivot Table Example Cat Intake & Outcomes Nearly 50,000 rows of outcome data becomes.... Pivot Table Tutorials Microsoft https://youtu.be/qmgilhilqr0 Google https://youtube.com/watch 30
Data Analysis Tips & Tricks 1. Look at numbers (#) and percentages (%) 2. Use graphs to visualize data; Waterfall style graphs are especially useful 3. Use good graph hygiene; always start graph axis at zero to avoid distortion 4. For comparison between graphs, keep same axis maximum and scale 5. For comparison between geography, normalize per 1,000 human population 6. Two words: PIVOT TABLES!! 33
Beware Data Analysis Pitfalls Garbage IN = Garbage OUT Boiling the Ocean Analysis Paralysis 34
Keep the Big Picture in Mind Goal: Maximize Lifesaving Impact Decrease Intake Know your INTAKE What pets are coming into shelter? Why? Increase Live Release Know your OUTCOMES What happens to pets in shelter? Why? Spay Neuter Trap Neuter Return Surrender Prevention Adoption Kitten Foster Return to Field 37
Today s Agenda Why does this matter? A Dangerous Mind Data Collection Data Analysis Data Interpretation Case Studies 38
Correlation is NOT Causation Studies have shown that people who eat yogurt regularly have a healthier body weight than those who do not eat yogurt regularly Can we therefore say that if you eat yogurt you will have a healthier body weight? YOGURT CAUSES HEALTHY WEIGHT Or, might it be that people who make healthier diet choices overall tend to eat more yogurt? HEALTHY PEOPLE EAT YOGURT We can say that eating yogurt is CORRELATED with healthy weight, we cannot say that eating yogurt CAUSES healthy weight Most results have multiple contributing causes Resist the urge to oversimplify cause and effect 38
Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Does Seasonal Kitten Intake Cause Higher Euthanasia? 2014 Feline Intake 2014 Feline Euthanasia 3000 3000 2500 2500 2000 2000 1500 1500 1000 1000 500 500 0 0 Cat Intake Kitten Intake Kitten Euthanasia Cat Euthanasia 40
Data Interpretation Tips & Tricks 1. Don t get too attached to your initial hypothesis, look for other explanations 2. Remember that correlation does not imply causation 3. Recall that most outcomes are the result of multiple different variables resist the urge to over simplify! 4. Always ask yourself questions What else could be going on here? What else has changed? 41
Today s Agenda Why does this matter? A Dangerous Mind Data Collection Data Analysis Data Interpretation Case Studies 42
CAGE TYPE CASE STUDY
Does Cage Type Impact LOS for Cats at Adopt & Shop?
Cage Type Data Collection & Analysis Process What data to analyze, and why? Adult cats, In a single kennel for entire stay, after Nov Pull data from system and process Date of outcome DOB = Age Check data for errors and anomalies Eliminate records with negative LOS Calculate metrics and graph Average LOS by kennel type, range Interpret and act....all grate front kennels?
Cage Type Data Collection & Analysis Process What can we conclude? Cats in Sample 19 11 9
CITY OF LOS ANGELES CASE STUDY
Intake Dogs Puppies Cats Kittens Euthanasia Dogs Puppies Cats Kittens Thousands Thousands 2010 Los Angeles Intake & Outcomes - Numbers 70 2010 Intake 60 2010 Euthanasia 60 50 40 30 20 10 0 58 33 1 15 9 50 40 30 20 10 0 62% Save Rate 22 9 0 7 7
Intake Dogs Puppies Cats Kittens Euthanasia Dogs Puppies Cats Kittens Thousands Thousands Biggest Opportunities for Lifesaving in LA in 2010? 70 2010 Intake 60 2010 Euthanasia 60 50 40 30 20 10 0 58 33 1 15 9 50 40 30 20 10 0 22 9 0 7 7 % of Intake 38 27 23 44 72
2010 LA Outcomes by Animal Type - Percentages
NKLA Program Started 2011 Used 2010 data analysis and community input to design several programs with aim to make Los Angeles No Kill by 2017 Initial programs included: Grants for zip code targeted spay neuter (low income) Adoption subsidies for rescue groups Kitten nursery/foster programs New adoption facilities
Intake Dogs Puppies Cats Kittens Intake Dogs Puppies Cats Kittens Thousands Thousands 2010 vs 2014 Los Angeles Intake - Numbers 70 2010 Intake 70 2014 Intake 60 50 40 30 20 10 0 58 33 1 15 9 60 50 40 30 20 10 0 52 29 2 13 9
Euthanasia Dogs Puppies Cats Kittens Euthanasia Dogs Puppies Cats Kittens Thousands Thousands 2010 vs 2014 Los Angeles Euthanasia - Numbers 25 2010 Euthanasia 25 2014 Euthanasia 20 15 22 9 0 20 15 77% Save Rate 10 7 10 12 4 0 5 0 7 5 0 4 4
2014 Outcomes by Animal Type - Percentages
Next Steps for NKLA CATS, CATS, CATS!!! Shifting S/N grant focus to cats More kitten nursery/ foster capacity Cat adoption promotions Targeting TNR to areas with high kitten intake
LENGTH OF STAY CASE STUDY
Do different transfer models impact LOS at Adopt & Shop? 3 different transfer models 1. A&S Staff select animals 2. Third Party select animals 3. Source shelter staff selects animals Our assumptions LOS will be lowest when the A&S staff select the animals LOS will be highest when shelter staff select animals since they may be sending those that they cannot adopt
LOS Collection & Analysis Process What data to analyze, and why? All animals by source shelter Pull data from system and process Date of outcome Date of intake = LOS Total LOS / total number of entries = AVG LOS Check data for errors and anomalies Eliminate records with negative LOS, eliminate anomalies Calculate metrics and graph Average LOS by source shelter, grouped by transfer model Interpret and act....source more animals from shelters with a lower LOS?
LOS Collection & Analysis Process Were our assumptions correct?
LOS Collection & Analysis Process Is the correlation between how the animal is selected? Or LRR? What about causation? Shelter Published Live Release Rate
FOSTER CAPACITY CASE STUDY 49
What is it really going to take to save 1800 kittens? 2015 Statistics ~200 foster homes <1000 kittens/mothers went through program Minimal foster bottleneck Assumptions We are going to need A LOT of new foster homes
How many fosters homes are needed to save 1800 kittens in 12 months? Historical data to analyze AVG # litters per foster annually, AVG # kittens in a litter, AVG LOS in a foster home Pull data from system and process Total # kittens fostered/total # litters = AVG # kittens per litter Total # litters/total # foster homes = AVG # of litters per foster home Check data for errors and anomalies Calculate metrics and graph Interpret and act Do we have enough foster homes?
LOS Collection & Analysis Process Raw Data Processed Data
Charting the Results 160 140 120 7% 3% 2% 100 80 60 40 20 22% 65% 0 1 Litter 2 Litters 3 Litters 4 Litters 5 Litters 6 Litters 1 Litter 2 Litters 3 Litters 4 Litters 6 Litters Chart both numbers and percentages to see the full picture.
Results Findings 87% of our foster homes (FH) take 1-2 litters annually AVG litter size is 2 kittens 226 active foster homes Calculations 1800 kittens/2 kittens per litter = 900 litters 900 litters/2 litters per FH = 450 FHs needed Optional Paths Recruit an additional 224 FHs Increase # of kittens per litter (1800 kittens/3 kittens per litter = 600 litters/2 litters per FH = 300 FHs needed) Increase # of litters per FH (1800 kittens/2 kittens per litter = 900 litters/3 litters per FH = 300 FHs needed) Increase # of kittens per litter and litters per FH (1800/3 kittens per litter = 600 litters/3 litters per FH = 200 FHs needed)
ADOPTER DISTANCE CASE STUDY 49
How far are adopters willing to drive in L.A. traffic?
Where should we spend our marketing dollars? Historical data to analyze AVG All adopters # litters with per foster addresses annually, AVG # kittens in a litter, AVG LOS in a foster home Pull data from system and process Pull data from system and process Standard report available with adopter information Total # kittens fostered/total # litters = AVG # kittens per Remove foster and employee adoptions litter Check data for errors and anomalies Total # litters/total # foster homes = AVG # of litters per foster Out of home state Calculate metrics and heat map Interpret and act How far from the store locations should we be marketing to potential adopters?
Heat Mapping
Questions? April Harris april@adoptapet.com Follow Found Animals! 58