Basic Statistics for Paid Search Advertising
Download as PPTX, PDF1 like1,812 views
The document presents an overview of basic statistical concepts and tools for effectively managing paid search advertising campaigns. It covers topics such as statistical sampling, data analysis, common misconceptions, and methods for prediction and forecasting, all while emphasizing the importance of robust data representation and validity. The presentation aims to correct misuses of statistics and to ensure participants understand data collection and interpretation without relying on complex formulas.
Basic Statistics for Paid Search Advertising
- 1. BASIC STATISTICS for PAID SEARCH ADVERTISING Katharine Mission-Estenzo SGS.com Search Engine Marketing Lead Research & Testing Specialist and Quality Management Coordinator PPC Pinas Meetup 2013 May 31, 2013 Cypress Towers, Taguig
- 2. 2 OBJECTIVE & SCOPE Introduce Statistical Concepts and Tools to efficiently manage campaigns and results Correct common misuses and misconceptions on Basic Statistical concepts Not a Statistics crash course - Guaranteed formula-free presentation!
- 3. 3 TOPICS Statistical Sampling and Analysis Charts and Graphs Common Numerical Misuses Prediction and Forecasting Statistical Process Control
- 4. 4 STATISTICS Collection Analysis Interpretation Presentation
- 5. 5 STATISTICAL SAMPLING? Population Target Population Samples Selected SAMPLING TECHNIQUES Simple Random Sampling Systematic Sampling Stratified Sampling Probability-proportional-to-size sampling Accidental / Purposive Sampling Quota Sampling Clustered Sampling SAMPLING PROCESS Define the population of concern Specify a sampling frame Develop a sample plan Implementing the sample plan Sampling and data collecting
- 6. 6 WHY USE A SAMPLE? • Lower Costs • Faster Data CollectionResearch • Validity of Results • Robustness of Statistical Model • Statistical Significance Testing SAMPLINGERRORS • History • Instrumentation • Selection • Sampling Distortion
- 7. 7 SIGNIFICANT VS STATISTICALLY SIGNIFICANT SIGNIFICANT Important Essential Meaningful STATISTICALLY SIGNIFICANT Pattern Behavior Not by Chance Before making conclusions, always make sure that you have sufficient sample size. All test results are invalid if: insufficient sample size sampling errors
- 8. 8 SAMPLING ERROR = MARGIN OF ERROR Sampling Error • Failure to capture the profile of the true population- under representation. Margin of Error • The difference of the estimated value to the true population value
- 9. 9 GRAPH IT! 0 200 400 600 800 1000 1200 1400 1600 HACCP ISO 22000 GMP FSSC 22000 BRC Page Visits January 2013 0 200 400 600 800 1000 1200 1400 1600 1800 Jan February March April Monthly Page Visits Jan - Apr 2013 HACCP GMP FSSC 22000 HACCP 33% ISO 22000 22% GMP 18% FSSC 22000 15% BRC 12% Page Visits January 2013 Discrete/ count data – Impressions, Clicks, Conversions Comparing data based on a single category/ criteria Change in magnitude/ quantity Continuous data – CTR, Conv Rate, CPC Tracking changes over time Trends Correlations Portions/ percentages of a whole – Geo performances One variable at a time Limit your data – use bar charts for more than six variables Avoid using 3D rotation - deceiving
- 10. 10 COMBINATION GRAPHS $0.00 $1.00 $2.00 $3.00 $4.00 $5.00 $6.00 0 10 20 30 40 50 60 Madrid Valencia Mallorca Zaragosa Tenerife Conversions vs CPA 33.39% 31.90% 27.38% 27.16% 22.26% 24.07% 28.33% 28.86% 17.47% 19.06% 18.79% 16.98% 14.47% 13.14% 11.33% 11.88% 12.42% 11.84% 14.16% 15.11% 0.00% 10.00% 20.00% 30.00% 40.00% 50.00% 60.00% 70.00% 80.00% 90.00% 100.00% Jan February March April Monthly Search Traffic Share HACCP ISO 22000 GMP FSSC 22000 BRC Clicks Impressions 0 5000 10000 15000 20000 25000 Algeria Nigeria Saudi Kurdistan Kenya When using combination graphs (or even simple graphs), keep in mind that your objective is to simplify data presentation. Present trends and changes in the simplest form. Do not complicate your graphs just to give the impression of “advanced” analysis and/or analytical skills.
- 11. 11 LIES, DAMNED LIES AND STATISTICS The Danger of Averages Bill Gates walk into a bar; on average, everybody in the bar is a millionaire. The average human has one breast and one testicle. ~Des McHale The interesting thing about averages is that they hide the truth very effectively. ~Avinash Kaushik
- 12. 12 MEASURES OF CENTRAL TENDENCY Day Earning (USD) Day 1 350.00 Day 2 400.00 Day 3 400.00 Day 4 5,500.00 Day 5 150.00 Day 6 300.00 Day 7 400.00 Day 8 400.00 Day 9 400.00 Day 10 400.00 Total 8,700.00 ON IMPULSE: My average daily earning is USD 870.00. MEAN Average Minimal differences Widely dispersed data Extremes and outliers MEDIAN Middle value Most resistant to outliers and extreme values If data points are even, this is the mean of the 2 middle values MODE Most often appears Most likely to be sampled Not unique – data set may be mutli-modal
- 13. 13 Percentage Fallacies and Misuses Using pure percentage values to measure effectiveness CTR Conversion Rates Averaging Percentages – valid or not? Trials Successes % 10 6 60.00% 25 10 40.00% 30 10 33.33% 40 5 12.50% Totals 105 31 145.83% AVERAGE = 36.46 % AVERAGE = 29.52 %
- 14. 14 The Excuse of Trends and Seasonality TREND - General tendency of a series of data points to move in a certain direction over time Consecutive data points moving in a single direction Majority of data points moving in a single direction Extreme values, singular peak values and outliers (Noise) are flattened in trend analysis SEASONALITY – Characteristic of a time series in which the data has regular and predictable changes on a specific period recurring every calendar year Always check previous data for the same time period Not all holidays are causal to seasonality
- 15. 15 PREDICTION AND FORECASTING TIME SERIES A sequence of data points measured successively in uniform time intervals Use of a statistical model to predict future values based on previous observations ! Assuming that conditions stay the same. REGRESSION ANALYSIS A technique for estimating the relationships between variables The value of a dependent variable is affected by the behavior of the values of the independent variables ! Check data for conformance to statistical assumptions.
- 16. 16 STATISTICAL PROCESS CONTROL FMEA – Failure Mode and Effects Analysis Identifying potential mistakes before they happen to determine whether the effects are tolerable or not FME(C)A – includes criticality analysis Efficient assessment of best option Evaluate effects of proposed changes on processes & performances Manage risks associated with system failures and changes Standardize procedures and practices
- 17. 17 Design Measure Analyze Improve Control DMAIC – Six Sigma Core Concept Campaign Objectives Nature of Business Advertising Channels Type of Testing Gap analysis/ Benchmark Historical Data Data Collection/ Testing Identify sources of variation Identify critical factors Validation of results Discover process relationships Implement optimization/ improvements FMEA Documentation Develop Control Plan Monitoring
- 18. 18 REMINDERS TEST! Don’t rely on assumptions. Efficiency – cost, time, energy Always define objectives and targets clearly Plan carefully – ensure objectives are met Understand your data – how, where, what and when Statistics – Bane or Boon?
Editor's Notes
- #6: For PPC, more often we use ymmetrical samplingWe divide samples equally among control and test conditions. Ensure ads are rotated evenly for split testingSampling plan includes parameters to be measured, range of values, sampling technique, sample size
- #7: Sampling Error – failure of the subset (sample) to represent/ capture the characteristics of the population of interestHistory – related to time – trendsInstrumentation – tools – tracking toolsSelection – subjects not evenly distributedSampling Distortion – failure to collect sufficient sample size
- #8: A significant change in conversions may not be statistically significant. It means that change may be due chances and/or such behavior may not last longStat sig – p value refers to probability that a given value will occur by chanceStat sig – when you repeat the test in the future given the same circumstances, results will be more or less the sameMec toolThe more samples, the more accurate results you getKey importance of sampling – Representativeness-in PPC sampling is important to help manage numerous campaigns, avoid making decisions based on small changes that may be due to chances. To make optimization efforts maximize their effectiveness
- #9: When you commit sampling errors, margin of error becomes greaterThe more samples, the lower the margin of error becomes
- #10: Correlations – use to indicate relationship variablesHowever, correlation does not imply causationThere may be additional factors to fully explain and measure the relationships of variables
- #11: 1st graph– easily see what are needs attention: low conversions, higher CPA2nd graph – good for comparing share percentages month by month. However, changes in magnitude for monthly search traffic is not captured. Ex. Jan has 300 visits (100%) and February may have 200 (100%)3rd graph – area charts good if proportion is high- maybe 30% at least
- #12: The event of having an extreme value could distort the average valueSample: Average monthly conversion for 1 year, campaign may have consistent values ranging from 3ly 0-40 but for a month you peaked conversion at 150- average monthly conversions may be greater than the true “usual” monthly values
- #13: How to get the median- arrange data from highest to lowest and get the middle valueMedian 50% falls below and 50% falls aboveOutliers – peaks, extremely large value, very far from the usual valuesIn PPC – using average position is not that advisableAve CPC – check daily for actual cpc
- #14: Percentages depend on their base population – it is a measure of proportion-CTR for keywords – consider search volumes-conversion rates – compare with the ad serving share (more fair % because this is portion of the whole)Do not sum up % then divide by the data points. Always get total success divided by total # of trials
- #15: Advanced Trend and Seasonality analysis – Deseasonalizing – get seasonality index and divide each data point with itAt least 3 yearsNo data on trends or seasonality- deduction analysis of all campaign elements
- #16: Time series is not a simple sequence nor is it based on a single variable only- it is a statistical model considering all variables that may affect the variable of interestApplicationsTime Series – annual returns and gains, ROI’s, speech recognition, earthquake predictionsRegression – ex. finding out what affects CPCBe careful to use in PPC because, certain parameters may not have linear relationshipsStatistical tests have assumptions to fulfill before the results are considered statistically correct
- #17: FMEA is a design tool sample is the PPC Best Practices guide
- #18: You’ve been doing this
- #19: Review concepts before planning your testsKnow your data- how is it gathered, where it may come from, what is it for and when – concerns related to time