Article Text
PDF
Abstract
Objectives There is great interest in evaluating gene–environment interactions with chemical exposures, but exposure assessment poses a unique challenge in case–control studies. Expert assessment of detailed work history data is usually considered the best approach, but it is a laborious and time-consuming process. We set out to determine if a less intensive method of exposure assessment (a job exposure matrix (JEM)) would produce similar results to a previous analysis that found evidence of effect modification of the association between expert-assessed lead exposure and risk of brain tumours by a single nucleotide polymorphism in the ALAD gene (rs1800435).
Methods We used data from a study of 355 patients with glioma, 151 patients with meningioma and 505 controls. Logistic regression models were used to examine associations between brain tumour risk and lead exposure and effect modification by genotype. We evaluated Cohen's κ, sensitivity and specificity for the JEM compared to the expert-assessed exposure metrics.
Results Although effect estimates were imprecise and driven by a small number of cases, we found evidence of effect modification between lead exposure and ALAD genotype when using expert- but not JEM-derived lead exposure estimates. κ Values indicated only modest agreement (<0.5) for the exposure metrics, with the JEM indicating high specificity (∼0.9) but poor sensitivity (∼0.5). Disagreement between the two methods was generally due to having additional information in the detailed work history.
Conclusion These results provide preliminary evidence suggesting that high quality exposure data are likely to improve the ability to detect genetic effect modification.
- Cancer
- polymorphisms
- lead
Statistics from Altmetric.com
What this paper adds
-
Expert assessment of occupational exposures in case–control studies is considered the best approach but is a laborious process.
-
We examined if a less intensive method of exposure assessment, the job exposure matrix (which has not been previously evaluated), would produce similar results in the context of gene–environment interactions.
-
A gene–environment interaction that was previously detected using expert assessment was not found when using a job exposure matrix.
-
The results suggest that high quality exposure data are needed to detect gene–environment interactions.
Introduction
Case–control studies are often the most feasible study design for the investigation of occupational exposures and rare diseases. However, occupational exposure assessment can be quite challenging in this setting, given the low prevalence of exposure to many specific agents in the study population and the inability to obtain workplace records to reconstruct exposures accurately.
Job exposure matrices (JEM) and expert assessment of self-reported job histories are two occupational exposure assessment methods that have been commonly utilised in case–control studies.1 There is some overlap between these procedures since both are typically developed by industrial hygienists who base their decisions on literature reviews and/or knowledge of industrial processes and occupational tasks. A JEM is a database that uses coded job and industry titles to assign exposures to a specific agent,1 ,2 while expert assessment involves the evaluation of detailed self-reported work history information from structured questionnaires by industrial hygienists to assign exposures.1 ,3 ,4 Thus, there may be a gain in accuracy over a JEM because experts can account for within-job variability of exposures. Expert assessment of self-reported job histories has some limitations, including its dependence on subject recall of detailed work information and the experience and subjectivity of the expert. Nonetheless, expert assessment of subject specific job history information currently is considered by many to be the best exposure assessment method for case–control studies.5 However, expert assessment is also a costly and time-consuming process1; as such, there is great interest in determining if less labour-intensive methods can produce similar results.
The issue of accurate exposure assessment in case–control studies is assuming even greater attention than in the past with the growing interest in studies of gene–environment interactions. It has been shown that even small errors in the assessment of environmental factors can result in biased estimates of interaction parameters and substantial decreases in study power.6 ,7 Also, unless we use the most accurate exposure assessment methods in studies evaluating genotypes and environmental exposures, there is a greater probability of detecting associations with genotypes rather than environmental exposures, since genotyping typically has a lower degree of classification error.8
While there have been comparisons of occupational exposure assessment methods in previous case–control studies,9–15 none, to our knowledge, have compared exposure assessment methods in the context of gene–environment interactions. We evaluated occupational exposure assessment for lead by a JEM in a case–control study of adult brain tumours that previously reported evidence of effect modification of expert-assessed lead exposure by the G177C polymorphism (rs1800435) of the δ-aminolevulinic acid dehydratase (ALAD) gene (G=ALAD1 and C=ALAD2).16 We determined whether risk estimates produced by the JEM-based exposure assessment were similar to those produced by the expert assessment. We also present the agreement, sensitivity and specificity of the JEM exposure estimates in comparison to the expert estimates.
Materials and methods
Study population
The study population has been described in detail previously.16 ,17 Adult patients diagnosed with a primary glioma (489), meningioma (197) or acoustic neuroma (96) during 1994–1998 at one of three hospitals were enrolled into the study (92% participation). Controls were 799 patients admitted to the same hospitals for various non-neoplastic conditions that were frequency matched (1:1 ratio) to cases based on categories of sex, age, race, hospital and residential proximity to the hospital (86%).
Blood samples were obtained from 382 patients with glioma (78%), 158 patients with meningioma (68%) and 540 controls (62%). Given the small number of patients with acoustic neuroma, they were excluded from this analysis. The study protocol was approved by the institutional review board of each participating institution, and written informed consent was obtained from each patient or proxy.
Genotyping
Genotyping of the rs1800435 polymorphism was conducted by the Core Genotyping Facility of the National Cancer Institute using a medium-throughput Taqman assay.18 There was a 90% concordance rate for quality control duplicate samples, and 94% of samples were successfully genotyped (355 glioma cases, 151 meningioma cases, 505 controls). There was no statistically significant departure from Hardy–Weinberg equilibrium among controls.
Data collection
A trained research nurse administered an in-person interview to each patient or proxy (8%).16 For all patients, a life-time occupational history (job title, type of industry, dates, activities, and materials and chemicals used) was obtained along with information on other potential risk factors for brain tumours. In addition to providing a standard work history, participants answered one of 63 questionnaires on specific jobs with detailed questions developed by an industrial hygienist regarding specific tasks and workplace conditions within each job.19
Lead exposure assessment
A single expert, blinded to the case/control status of the participants, carried out the assessment over an 18-month period. For the expert assessment, exposure guidelines were constructed based on a comprehensive literature review of peer-reviewed articles and technical reports linking job titles and industries with air level and/or blood level measurements of lead by decade.16 These guidelines were used in conjunction with detailed questions on tasks (welding, soldering, painting, sanding, etc) and workplace conditions (ventilation, use of personal protective equipment, etc) provided by the subject to assign an intensity (estimated airborne concentration (0, 5–9, 10–29, 30–49, 50–249 or ≥250 μg/m3)), a frequency (time exposed to lead (<1, 1–9, 10–29 or 30–40 h/week)) and a probability (likelihood of lead exposure actually occurring (0, 1–9, 10–49, 50–89 or ≥90%)) for each job reported by a study participant.
The JEM, which had been developed by different investigators for a different study of brain tumours, assigned intensity and probability of lead exposure (none=0, low=1, medium=2, high=3) independently for each three-digit occupation and industry code from the 1980 Census list of occupations and industries.20 A separate metric for frequency of lead exposure was not developed but was incorporated into the metric for intensity. The intensities and probabilities were based on published literature, computerised exposure databases, technical reports and expert experience. Each final intensity and each final probability score was obtained by multiplying the occupation and industry scores for each job to create a 7-point scale (0, 1, 2, 3, 4, 6, 9). If either the occupation or industry code was missing, the final intensity and probability score was obtained by squaring the individual occupation or industry score. For example, if the industry intensity score was missing, but the occupational intensity score was 2, the occupational score would be squared to give a final intensity score of 4.
Participants were considered ‘ever exposed’ if they had at least one lead-exposed job at any exposure intensity but probability of ≥10% (≥2 for the JEM). These probability levels were chosen to provide reasonable sensitivity and high specificity of the exposure assessment methods, minimising the attenuation of effect estimates by exposure misclassification in population-based case–control studies.21 Ever exposure was set to missing for individuals with incomplete data for any job in their work history, unless exposure to lead was indicated for at least one job with complete data.
For the expert assessment method, lifetime cumulative lead exposure was calculated by multiplying the number of years in each lead-exposed job by the midpoint of the estimated airborne concentration range and the estimated frequency of exposure and then summing the cumulative exposures across jobs. For the JEM, the product of the number of years in each job and the estimated intensity score for each job was summed across jobs to derive a cumulative exposure metric. Cumulative lead exposure was set to missing for individuals with incomplete information for any job in their lifetime job history. In order to facilitate comparison between the two methods, the cumulative expert and JEM exposure metrics were categorised into four groups based on percentile distributions among the controls (unexposed, ≤80th percentile, >80th–95th percentile, >95th percentile, unknown). Unexposed individuals comprised 60% and 70% of the distribution of controls for the cumulative expert and JEM exposure metrics, respectively.
For a subset of subjects whose characterisation as ever versus never exposed differed according to the exposure assessment method, an industrial hygienist (PAS), who supervised the expert assessment in the original study, reviewed the subjects' work history information and JEM assessment to identify reasons contributing to the differences.
Statistical analyses
Unconditional logistic regression was used to evaluate the associations of glioma and meningioma with exposure to lead (ever exposure and categories of cumulative lead exposure). Effect modification was evaluated with likelihood ratio tests comparing nested unconditional logistic regression models that did and did not include cross-products terms for lead exposure and ALAD genotype. ORs and 95% CIs for the association between lead exposure and glioma and meningioma were calculated for ALAD1 homozygotes and ALAD2 carriers; given the small number of ALAD2 homozygotes (n=1 glioma case; n=3 meningioma cases; n=6 controls) they were combined with the heterozygous participants in the analyses. The matching variables age, sex, race, hospital and residential proximity to the hospital were included in all analyses.
Akaike's information criterion22 (AIC) and Bayesian information criterion23 (BIC) were calculated to compare the goodness of fit between models using the expert-based exposure data to models using the JEM-based exposure data. Smaller values of these statistics indicate models that better fit the data.
Compared to the expert exposure assessment metrics, sensitivity, specificity per cent agreement and Cohen's κ statistic (which assesses agreement beyond chance) were calculated for the similarly derived JEM estimates for meningioma cases and controls. For cumulative lead exposure, JEM sensitivity and specificity were assessed for successful classification of subjects as ‘highly’ exposed (ie, in either of the two highest categories of exposure). Weighted Cohen's κ statistics were also calculated for cumulative exposure where a single category difference in agreement was given a weight of 0.66 and a two category difference in agreement was given a weight of 0.33. All statistical analyses were completed in STATA (Version 10, College Station, Texas, USA).
Results
The distribution of matching factors among cases and controls has been previously reported16 and is not discussed here. As seen in table 1, the JEM, for both the ever and cumulative exposure metrics, classified a greater proportion of participants as unexposed compared to the expert-derived estimates. For both metrics, the JEM and expert assessments classified different numbers of individuals as ‘unknown’. In some instances, exposure values could not be assigned based on the JEM because of missing occupation and industry codes, but the expert was able to assign exposure values using the detailed work history information. In other instances, where the JEM assigned exposure values if one of the two codes (occupation or industry) was missing, the expert determined that exposure could not be assigned based on the occupation or industry title alone, and in the absence of detailed work history information, set the exposure values to missing.
Risk of glioma and meningioma with exposure to lead determined by an expert and a job exposure matrix by ALAD G177C (rs1800435) genotype in the NCI Brain Tumour Study
There was no evidence of an overall association between lead exposure and glioma or of effect modification of the relationship between lead and glioma by ALAD genotype using either method of exposure assessment (table 1). There was evidence of an association between lead exposure and meningioma among individuals with the highest category of expert-assessed cumulative lead exposure (OR 2.7 (95% CI 1.0 to 7.8)), but no similar evidence was found when examining the JEM cumulative exposure metric (OR 0.9 (95% CI 0.3 to 2.8)). Also for meningioma, the metric based on expert assessment indicated borderline evidence of effect modification by ALAD genotype with ever exposure to occupational lead (p=0.09) and statistically significant evidence of effect modification with cumulative lead exposure (p=0.04). Neither metric derived from the JEM showed any evidence of effect modification of meningioma risk with ALAD genotype.
For glioma, model fit statistics were very similar when comparing logistic regression models using the expert-assessed exposure data to models using the JEM-assessed exposure data (results not shown). While for meningioma, fit was similar for the overall expert and JEM models (results not shown), AIC and BIC were lower for the expert models that included genotype effect modification terms compared to the JEM models with these terms (expert ever exposure: AIC=654, BIC=743; JEM ever exposure: AIC=670, BIC=769; expert cumulative exposure: AIC=640, BIC=747; JEM cumulative exposure: AIC=650, BIC=757), indicating a better fit of the models to the expert-assessed exposure data than to the JEM-assessed exposure data.
Tables 2 and 3 show the cross-classifications of meningioma cases and controls with respect to the expert- and JEM-derived ever versus never and cumulative exposure metrics, respectively. In general, the expert assessment tended to classify more subjects as exposed or into higher categories of exposure when compared to the JEM. For example, among all meningioma cases and controls, the expert assessment classified 40% of subjects as ever-exposed, while the JEM classified 25% as ever-exposed.
Comparison of ever lead exposure determined by expert assessment with exposure determined by a job exposure matrix for meningioma cases and controls in the NCI Brain Tumour Study
Comparison of cumulative lead exposure determined by expert assessment with exposure determined by a job exposure matrix for meningioma cases and controls in the NCI Brain Tumour Study
When examining all subjects, ALAD1 homozygotes and ALAD2 carriers, the sensitivity and specificity for the JEM ever/never exposure metric was approximately 0.5 and 0.9, respectively, for each of the three groups. The percentage of subjects showing exact agreement of exposure classification among these groups was approximately 75% and the corresponding κ value was approximately 0.4 for all three groups.
When considering cumulative exposure, the sensitivity (specificity) of the JEM for successful classification of ‘high’ exposure (>80th percentile, ie, categories 2 and 3) was approximately 0.5 (0.9) among the three groups (all subjects, ALAD1 homozygotes and ALAD2 carriers). The percentage of subjects showing exact agreement of exposure classification among these groups was approximately 60%. κ (weighted-κ) Values for these three groups were approximately 0.3 (0.4).
An expert industrial hygienist (PAS) examined the JEM and work history information for the 31 ALAD2 carriers in table 2 for which the JEM and expert exposure assessments did not agree with respect to having ever been exposed to lead. For these 31 individuals, the JEM and expert-assessed exposures differed for 43 of the 169 (25%) reported jobs because of differences in the assignment of probability or intensity of exposure between the two exposure assessment methods. For 37 of these 43 jobs, which varied widely with respect to occupation and industry titles, the expert assessment indicated exposure where the JEM did not. For most of these jobs (27), this was because work history data used in the expert assessment indicated the occurrence of lead exposure that would not be expected based on examining occupation and industry titles alone. For example, the JEM indicated zero exposure to lead for a job as an editor in the newspaper industry. However, the job history indicated that the individual spent part of his time in the production area during a time period when molten lead was still being used for typesetting. In only one instance did the work history data indicate a lack of lead exposure for a job which otherwise would have been expected based on the occupation and industry titles alone. For nine jobs, differences were attributed to the expert's knowledge of technical details for those particular jobs, and for six jobs the potential for lead exposure was uncertain and could have been argued either way.
A similar examination of sources of discrepancies between the cumulative exposure metrics was difficult given differences in the intensity scales and incorporation of frequency of exposure data between the two exposure assessment methods. However, patterns were similar to those observed for the ever versus never exposure analysis.
Discussion
An association between lead and meningioma and its modification by the ALAD rs1800435 polymorphism was observed based on expert assessment of exposure but not when using a JEM. Based on κ statistics, there was fair to moderate agreement between the exposure metrics derived from the JEM and expert exposure assessment methods.24 While the JEM displayed reasonable specificity compared to the expert assessment, its sensitivity was only modest. As expected, the κ statistics, sensitivity and specificity values did not vary appreciably when subjects were stratified by genotype. Although neither the expert nor the JEM approach is perfect, we believe that in this case, the expert approach is likely to be more accurate because, unlike the JEM, expert assessment has the ability to account for within-job variability by using detailed questionnaire-based work history information (eg, specific tasks, control measures, etc) specific to the study at hand.
The exposure prevalence for lead, based on expert assessment, was approximately 40% among controls (table 1) which may seem high. We believe the prevalence is realistic because of the calendar time of the study, and because we considered all exposures (including low exposures). The mean blood lead level in the US population circa 1970 was 12.8 μg/dl.25 Although we estimated occupational exposure prevalence in our study, this figure does indicate fairly ubiquitous exposure to lead.
The risk estimates and corresponding 95% CI and p values observed for the expert-assessed lead data differ slightly from those previously reported.16 This is because the previous analyses considered only those jobs with an exposure intensity of greater than or equal to 10 μg/m3 to be exposed to lead. To facilitate comparison between the expert and JEM assessment methods, we did not impose this restriction in the current analysis.
Expert- and JEM-based exposure assessments have been compared in previous case–control studies.9–14 In our study, we observed slightly higher levels of agreement between expert- and JEM-based exposure assessments than observed in other studies examining various exposures.9–11 For example, in a case–control study of glioma, Benke et al calculated a κ of 0.33 for ever exposure to lead,9 while we calculated a κ of 0.5 for ever exposure to lead among meningioma cases and controls. Even though we observed a higher κ value, 0.5 only represents a moderate level of agreement.24 As with our study, previous evaluations of various exposures in case–control studies observed poor sensitivity, yet high specificity, for JEMs compared to expert assessments.12–14 Rybicki et al, for example, observed a sensitivity of 0 and a specificity of 0.93 when comparing lead exposure estimates derived from a JEM versus expert assessment.12
Although in this paper we consider expert assessment as the more accurate method, it is also imperfect. The quality of the assessment depends on the experience of the expert,5 and there may be differences in exposure assignment as the study progresses, although the latter issue can be somewhat mitigated with detailed and standardised rules.1 ,26 While the ability to account for within-job variability is a strength of expert assessment because of the potential gain in accuracy, this gain may be offset by limitations in the ability of participants to recall detailed work information. The use of self-reported job histories also raises issues of response bias (ie, cases indicate greater exposures to lead because of their disease status), but this is not likely to be a problem in our study given that our questionnaire was designed and administered in such a way as to assess the potential for exposure to a wide variety of agents without prior knowledge of what exposures would be of most interest. Thus, any resulting misclassification of exposure would likely be non-differential, and the risk estimates would most typically be biased towards the null.
Use of a biomarker for cumulative lead exposure such as bone lead measurements rather than questionnaires would have been ideal. However, evaluation of the association between lead exposure and brain tumours was not the primary objective of this study when it was initiated, and, as such, biomarker data for lead exposure were not collected. In a previous comparison of exposure assessment methods including biomarker data, Tielemans et al found that assessment of individuals as exposed versus unexposed to chromium by job-specific questionnaires compared better to urinary measurements than when using a JEM to assess exposure.15 Although urinary chromium concentrations were clearly increased in subjects classified as exposed by the job-specific questionnaire, the exposed group from the job-specific questionnaire was restricted to those individuals that were determined to be highly exposed, and κ statistics indicated only poor to moderate agreement. While in the absence of actual measurement data, expert assessment is considered the best approach to date for assessing past exposures in population-based case–control studies,5 resources should be directed towards developing better methods that address the limitations of expert assessments.
Expert assessment has been reported to provide greater statistical power than other methods (including JEM-based exposure assessment) for detecting associations between exposure and disease.27 In the analysis of gene–environment effect modification, statistical power becomes an even greater issue as studies typically require large sample sizes to detect effect modification.28 It has been demonstrated that even small errors in the assessment of environmental factors can result in biased interaction parameters and substantially increased sample size requirements for the detection of effect modification.6 ,7 In our analyses, misclassification of exposure by the JEM as compared to expert assessment resulted in smaller ORs and less likelihood of detecting an effect. These results indicate that investigators would benefit from using the most accurate method of exposure assessment available, since the attenuating effects of exposure misclassification would result in increased sample size requirements to detect effect modification29 that would offset any savings from using a less costly method of exposure assessment.1
As genome-wide association studies identify genetic polymorphisms associated with disease, there is increasing interest and need for evaluating interaction with environmental factors. Although we recognise the need for replication of the effect modification results given the small sample size of variant carriers exposed to lead, preliminary findings suggest that high quality exposure data are likely to improve the ability to detect genetic effect modification.
References
Footnotes
-
Linked articles 054585.
-
Funding This study was funded under contract N01-CO-12400 from the Intramural Research Program of the National Cancer Institute, National Institutes of Health, Department of Health and Human Services.
-
Competing interests None.
-
Ethics approval The study was approved by the Human Subjects Review Board of the National Cancer Institute.
-
Provenance and peer review Not commissioned; externally peer reviewed.