Description  | Specific Indicators | Ontario Public Health Standards (OPHS) | Corresponding Health Indicator(s) from Statistics Canada and CIHI | Corresponding Health Indicator(s) from Other Sources | Data Sources |  Alternative Data Sources  | Analysis Check List |  Method of Calculation |  Basic Categories |  Indicator Comments  | Definitions |  Cross-References to Other Indicators |  Cited References  |  Other References  | Changes Made


Description

• Proportion of vendors willing to sell tobacco to minors during vendor compliance and enforcement checks.

Specific Indicators

• Rate of vendor non-compliance with the prohibition on tobacco sales to youth under the age of 19 years.

Ontario Public Health Standards (OPHS)  

The Ontario Public Health Standards (OPHS) establish requirements for the fundamental public health programs and services carried out by boards of health, which include assessment and surveillance, health promotion and policy development, disease and injury prevention, and health protection. The OPHS consist of one Foundational Standard and 13 Program Standards that articulate broad societal goals that result from the activities undertaken by boards of health and many others, including community partners, non-governmental organizations, and governmental bodies. These results have been expressed in terms of two levels of outcomes: societal outcomes and board of health outcomes. Societal outcomes entail changes in health status, organizations, systems, norms, policies, environments, and practices and result from the work of many sectors of society, including boards of health, for the improvement of the overall health of the population. Board of health outcomes are the results of endeavours by boards of health and often focus on changes in awareness, knowledge, attitudes, skills, practices, environments, and policies. Boards of health are accountable for these outcomes. The standards also outline the requirements that boards of health must implement to achieve the stated results.  

Link to OPHS

http://www.ontario.ca/publichealthstandards

Outcome Related to this Indicator

• Societal Outcome (Chronic Disease Prevention): There is increased adoption of behaviours and skills associated with reducing the risk of chronic disease of public health importance.
• Board of Health Outcomes (Chronic Disease Prevention): Tobacco vendors are in compliance with the Smoke-Free Ontario Act.
• Board of Health Outcomes (Chronic Disease Prevention): Youth have reduced access to tobacco products.

Assessment and/or Surveillance Requirement Related to this Indicator (Chronic Disease Prevention):

• The board of health shall conduct epidemiological analysis of surveillance data in the areas of comprehensive tobacco control.

Health Protection Requirement Related to this Indicator (Chronic Disease Prevention):

• The board of health shall implement and enforce the Smoke-Free Ontario Act in accordance with provincial protocols, including but not limited to the Tobacco Vendor Compliance Check Protocol, 2006 (or as current).¹

Corresponding Health Indicator(s) from Statistics Canada and CIHI

• None

Corresponding Health Indicator(s) from Other Sources

National Advisory Group on Monitoring and Evaluation (NAGME): (Select "Indicators for Monitoring Tobacco Control: A Resource for Decision Makers, Evaluators and Researchers")

• Proportion of young people reporting that they have been sold tobacco products by a retailer
• Proportion of young people reporting that they have received tobacco from a social source

http://www.ctcri.ca/~ctcri/en/index.php?option=content&task=view&id=30&Itemid=49

Centers for Disease Control (CDC) Key Outcome Indicators for Evaluating Comprehensive Tobacco Control Programs: (Select "Chapter 2. Goal Area 1: Preventing Initiation of Tobacco Use Among Young People")

• Proportion of successful attempts to purchase tobacco products by young people (Indicator 1.11.1)
• Proportion of young people reporting that they have received tobacco products from a social source (Indicator 1.11.4)

http://www.cdc.gov/tobacco/tobacco_control_programs/surveillance_evaluation/key_outcome/

Data Sources

Numerator & Denominator: Tobacco Vendor Compliance Data
Original source: Public Health Unit
Distributed by: Public Health Unit
Suggested citation (see Data Citation Notes):
Prior to May 31, 2006: Internal PHU Tobacco Vendor Compliance Data [month, year - month, year], Extracted: [month,year]

Post May 31, 2006: MOHLTC Tobacco Inspection System [month, year - month, year], Extracted: [month,year]

Alternative Data Source(s)

The following questions are found in the Canadian Community Health Survey's (CCHS) Smoking module for 2000-01 (core content) and the Youth Smoking module for years 2003 and 2005 (core content). The Youth Smoking module was optional content in 2007 but not selected by Ontario.

Analysis Check List

Tobacco Vendor Compliance data:

• The Ministry of Health Promotion determines the required number of vendors that have compliance checks conducted within each public health unit.
• Following the implementation of the Smoke-Free Ontario Act in 2006, all public health units were required to conduct annual compliance and enforcement checks in all vendors in the province. Comparisons between public health units will be possible.²
• Prior to 2006, the required number of vendors that have compliance checks conducted within each public health unit was based on the size of the health unit vendor population (see Table 1). The public health unit determined its sampling methodology (i.e., random sample, high risk vendors). As such, the methodology may have varied by public health unit and over time. Trend analyses should be done cautiously.³

Table 1. Annual vendor compliance check sample size determination prior to 2006

CCHS 

• It is recommended that public health units use the Share File provided by the Ministry of Health and Long-Term Care rather than public use file (PUMF) provided by Statistics Canada. The Share File has a slightly smaller sample size because respondents must agree to share their information with the province to be included; however, the share file has more variables and fewer grouped categories within variables. The Share File is a cleaner dataset for Ontario analysis because all variables that were not common content, theme content or optional content for Ontario have been removed.
• There may be slight differences between results from the share file and data published on the Statistics Canada website for the Health Indicators because rates calculated for Health Indicators use the master CCHS data file.
• Not applicable respondents should be excluded; however, it is important to understand who these respondents are from the questionnaire skip patterns to be able to describe the relevant population.
• Users need to consider whether or not to exclude the ‘Refusal, 'Don't Know' and ‘Not Stated' response categories in the denominator. Rates published in most reports, including Statistics Canada's publication Health Reports generally exclude these response categories.  In removing not stated responses from the denominator, the assumption is that the missing values are random, and this is not always the case.  This is particularly important when the proportion in these response categories is high.
• Estimates must be appropriately weighted (generally the share weight for the CCHS) and rounded. 
• Users of the CCHS Ontario Share File must adhere to Statistics Canada's release guidelines for the CCHS data when publishing or releasing data derived from the file in any form. Refer to the appropriate user guide for guidelines for tabulation, analysis and release of data from the CCHS. In general, when calculating the CV from the share file using the bootstrap weights, users should not use or release weighted estimates when the unweighted cell count is below 10. For ratios or proportions, this rule should be applied to the numerator of the ratio.  Statistics Canada uses this approach for the tabular data on their website. When using only the Approximate Sampling Variability (CV) lookup tables for the share file, data may not be released when the unweighted cell count is below 30. This rule should be applied to the numerator for ratios or proportions.  This provides a margin of safety in terms of data quality, given the CV being utilized is only approximate.
• Before releasing and/or publishing data, users should determine the CV of the rounded weighted estimate and follow the guidelines below:
• Acceptable (CV of 0.0 - 16.5) Estimates can be considered for general unrestricted release. Requires no special notation. 
• Marginal (CV of 16.6 - 33.3) Estimates can be considered for general unrestricted release but should be accompanied by a warning cautioning subsequent users of the high sampling variability associated with the estimates. Such estimates should be identified by the letter E (or in some other similar fashion). 
• Unacceptable (CV greater than 33.3) Statistics Canada recommends not to release estimates of unacceptable quality. However, if the user chooses to do so then estimates should be flagged with the letter F (or in some other fashion) and the following warning should accompany the estimates: "The user is advised that...(specify the data)...do not meet Statistics Canada's quality standards for this statistical program. Conclusions based on these data will be unreliable and most likely invalid". These data and any consequent findings should not be published. If the user chooses to publish these data or findings, then this disclaimer must be published with the data.
   
• Caution should be taken when comparing the results from Cycle 1.1 (2000/01) to subsequent years of the survey, due to a change in the mode of data collection.  The sample in Cycle 1.1 had a higher proportion of respondents interviewed in person, which affected the comparability of some key health indicators.  Please refer to http://www.statcan.gc.ca/imdb-bmdi/document/3226_D16_T9_V1-eng.pdffor a full text copy of the Statistics Canada article entitled "Mode effects in the Canadian Community Health Survey: a Comparison of CAPI and CATI"

RRFSS:

• Users should refer to the most recent RRFSS Manual of Operations for a complete list of RRFSS analysis guidelines.
• Denominator Data - cell size less than 30 not to be released (based on unweighted data).
• Numerator Data - cell size less than 5 not to be released (based on unweighted data).
• Coefficients of variation (CV) should be calculated for every estimate. The following categories determine the release of the data:
• CV between 0 and 16.5:  estimate can be released without qualification.
• CV between 16.6 and 33.3:  estimate can be released with qualification:  interpret with caution.  High variability.
• CV greater than 33.3:  estimate should not be released, regardless of the cell size.
• 95% confidence intervals should accompany all released estimates.  In general the simple computation of the C.I. for a proportion assuming SEp = sqrt(pq/n) and CI95% = p +/- 1.96*SEp is sufficient. However, if estimates are close to 0 or 100% and the simple computation confidence intervals include values less than zero or greater than 100 then the Fleiss 2^nd edition computation for skewed estimates should be employed.
• Weighting:
• General household weight will be applied for questions related to the individual. 
• Household weight is to be applied for questions about households when we wish to determine the population/number of people affected. 
• Household weight is not to be applied for child proxy questions (bicycle helmet use, car seat safety), dog and cat immunization modules, or other questions that relate to the household rather than the respondent. 
• Household weights are not required to be recalculated for sub-population based questions; for example mammography in women ages 35+ years and 50-74 years.
• If the weights supplied with the data set (health unit wave specific, health unit cumulative total, all health units combined wave specific, all health units combined cumulative total) are not appropriate for the required analysis, then a time-specific weight must be calculated.  For example, a new weight is required for all seasonal modules.
• If the cell size of ‘Don't Know' responses is 5% or greater, ‘Don't Know' responses should be included in the denominator of the analyses and reported separately.
• If the cell size of ‘Refusal' responses is 5% or greater, ‘Refusal' responses should be included in the denominator of the analyses and reported separately.
• When an indicator is being compared between groups (e.g. health units, time periods, gender), if any one group has ‘Don't Know' and/or ‘Refusal' responses that are 5% or greater, ‘Don't Know' and/or ‘Refusal' responses should be included in the denominator of the analyses and reported separately for all groups.
• A provincial sample is not available with the RRFSS.
• Refer to the RRFSS Data Dictionaries at http://www.rrfss.ca/ for more information about module questions and indicators.

Method of Calculation

Number of vendor compliance and enforcement checks that resulted in a sale of tobacco to a minor Total number of vendor compliance and enforcement checks

X 100

Basic Categories

• Geographic areas for: Tobacco vendor compliance data and CCHS - 36 public health units;   RRFSS - participating health units that chose the Tobacco- Minors' Access to Tobacco module

Indicator Comments

• The Tobacco Control Act (TCA) was enacted in 1994, making it an offence to sell or supply tobacco to a person who is less than 19 years of age.
• On May 31^st, 2006, the Smoke-Free Ontario Act (SFOA) amended the TCA. While maintaining the prohibition of tobacco sales to minors, the SFOA added new restrictions on smoking in enclosed workplaces and public places, and the display, promotion and handling of tobacco products.
• The Ministry of Health Promotion Scopes of Service Agreement specifies that all public health units conduct two annual inspections (one for sales to minors and one for retail displays) encompassing at all tobacco vendors.² The compliance/enforcement checks are to be conducted in accordance with the Protocol for Determination of Tobacco Vendor Compliance, which was revised in May, 2006.¹
• Tobacco vendors are categorized into eight vendor types: 1) chain convenience store, 2) independent convenience store, 3) discount/dollar store, 4) supermarket/grocery store, 5) gas station, 6) tobacconist, 7) restaurant/bar, and 8) other (e.g. bowling alley, billiard hall, arena, etc.)³
• The validity of the test shopping procedure used during compliance and enforcement checks has been questioned recently. Under the standard protocol, test shoppers (hired youth under the age of 19 years) must dress to appear their age and must tell the truth if a store clerk asks for their age. Two studies have tested alternative test shopping protocols against the standard protocol. In the DiFranza study,⁴ the "smoker protocol" allowed youth to dress as they chose, purchase other items with the tobacco product, lie about their age, and present their own valid ID if requested. In the Landrine study,⁵ the "familiarity protocol" had the test shoppers enter the store 4 times to purchase other non-tobacco items before purchasing tobacco on the fifth visit. Both studies found that the alternative test shopping protocols were almost six times more likely to result in a successful tobacco purchase attempt in comparison to the standard protocol. These findings reflect a more realistic purchasing pattern of young smokers and suggest that the standard protocol may underestimate the ability of young smokers to purchase tobacco in their own communities.
• High levels of vendor compliance with minors' access to tobacco legislation do not appear to have had a significant effect on decreasing youth smoking rates. Underage smokers are turning instead to noncommercial sources of cigarettes (e.g., friends, family members, strangers).⁶
• Despite the concerns regarding validity and efficacy of the standard test shopping procedure, compliance and enforcement checks still remains a mandatory program for assessing minors' access to tobacco products in both the US and in Ontario.
• Two amendments to the Smoke-Free Ontario Act may have an indirect impact on youth smoking rates, which in turn may impact the rates of youth access to tobacco products. These amendments include: a) a ban on smoking inside a vehicle while children under the age of 16 years are present, and b) a ban on the sale of flavoured cigarillos and the sale of single cigarillos.⁷

Definitions

• Compliance checks - staged attempts to purchase tobacco by youth under 19 years of age who have been hired to attempt tobacco purchase from retailers with no enforcement history and are under the supervision of TEOs. Should a sale occur during a compliance check, a warning letter is issued for non-compliance and an enforcement check occurs within 3 months of the first sale to a minor.
• Enforcement checks - staged attempts to purchase tobacco by youth under 19 years of age who have been hired to attempt tobacco purchase from retailers with a history of selling tobacco to underage youth. Should a sale occur during an enforcement check a charge is laid.

Cross-Reference(s) to Other Indicator(s)

• Smoking Status (Section 5A: Smoking)

Cited References

1. Ontario Ministry of Health Promotion. Protocol for Determination of Tobacco Vendor Compliance, May 2006
2. Ontario Ministry of Health Promotion. Scope of Service. Local Capacity Building: Prevention and Protection, January 2007.
3. Ontario Ministry of Health and Long-Term Care. Protocol for Determination of Tobacco Vendor Compliance, March 2000.
4. DiFranza JR, Savageau JA, Bouchard J. Is the standard compliance check protocol a valid measure of the accessibility of tobacco to underage smokers? Tob Control. 2001;10:227-232.
5. Landrine H, Klonoff EA. Validity of assessments of youth access to tobacco: The familiarity effect. Am J Public Health. 2003;93:1883-1886.
6. Cummings K, Hyland A, Perla J, Giovino G. Is the prevalence of youth smoking affected by efforts to increase retailer compliance with a minors' access law? Nicotine Tob Res. 2003;5;465-471.
7. Government of Ontario. Smoke-Free Ontario Act. Available at http://www.e-laws.gov.on.ca/html/statutes/english/elaws_statutes_94t10_e.htm (Accessed on March 3, 2009).

Other Reference(s)

• DiFranza JR. Best practices for enforcing state laws prohibiting the sale of tobacco to minors. J Public Health Manag Pract. 2005;11:559-565.
• Corporate Research Group. Evaluation of retailers' behaviour towards certain youth access-to-tobacco restrictions: Final report findings: 2007. Ottawa, ON:2008.
• Levy DT, Friend KB. Strategies for reducing youth access to tobacco: A framework for understanding empirical findings on youth access policies. Drugs Educ Prev Policy 2002;9:285-303.
• Staff G, Rogers T, Schooley M, Porter S, Wiesen E, Jamison N. Key outcome indicators for evaluating comprehensive tobacco control programs. Atlanta, GA: Centers for Disease Control and Prevention; 2005. Available at: http://www.cdc.gov/tobacco/tobacco_control_programs/surveillance_evaluation/key_outcome/
• Copley TT, Lovato C, O'Connor S. Indicators for monitoring tobacco control: A resource guide for decision-makers, evaluators and researchers. On behalf of the National Advisory Group on Monitoring and Evaluation. Toronto, ON: Canadian Tobacco Control Research Initiative, 2006. Available at: http://www.ctcri.ca/~ctcri/en/index.php?option=content&task=view&id=30&Itemid=49