Important information about creating custom weights: Users can create longitudinal weights for multiple survey years by going to the BLS Web site at www.bls.gov/nls and choosing the NLSY97 link. Picking "Create a set of custom weights" on this page brings up the NLSY97 Custom Weighting program. To create a set of custom weights, users should first type in their e-mail address, then select the survey years corresponding to their research and pick the "Create Custom Weights" button. The NLS server will generate a set of longitudinal weights and e-mail the users a compressed file in Winzip format. The NLSY97 sampling weights, which are constructed in each survey year, provide the researcher with an estimate of how many individuals in the United States are represented by each NLSY97 respondent. Individual case weights are assigned to produce group population estimates when used in tabulations.
This sampling weights section includes the following information:
Methodology for Calculating Weights
Summary of NLSY97 Weights for Rounds 1 through 11
Weighting is a challenging subject. Researchers must first decide if they should or should not weight the sample. If researchers decide to weight, they must then determine which weight variable to use. This can be a difficult decision because as of Round 11 there are more than 30 different pre-created weight variables available in the NLSY97 dataset.
What does weighting do? All NLS data sets use over-sampling. Over-sampling is the selection of a large number of additional respondents that match certain criteria. This over-sampling was done to allow researchers to measure more precisely the changes in key populations like blacks and Hispanics. Over-sampling impacts population descriptors, such as means and medians, because the NLSY97 has more respondents who are black or Hispanic than what really exists in the U.S. If the data are not adjusted, the greater number of black and Hispanic respondents skews population averages toward black and Hispanic averages. Adjusting the data by weighting reduces the impact of each black and Hispanic respondent and removes that bias. If a user attempts to summarize characteristics of the population but ignores weights, results are biased.
Weights for the NLSY97 range from a high of around 1.7 million to a low of 86,000. What do these numbers mean? All NLSY97 weights contain two implied decimal places. To interpret the weight, divide by 100. For example, a respondent with a weight of 1.7 million represents 17,000 people, while a respondent with weight of 86,000 represents 860 people. If a NLSY97 respondent has a weight of 0, he or she did not participate in that survey round.
User Note: If you are creating descriptive statistics such as means, medians, and standard deviations, it is suggested that you weight your results. If you are running a more complex analysis such as doing a regression, we suggest that you do not weight. For more details on when to use weights, see Practical Usage of Weights. Users looking at the NLSY97 data set will find a large number of weight variables in each round. Weights are found under the “Sample Design & Screening” Area of Interest in Investigator. Weights also can be found by searching for the word “Weight”in the Word in Title search option. Figure 1 shows the sampling weight variables available for both round 1 and round 11. Weights in other rounds have very similar question names and variable titles.
Figure 1. Sampling Weight Variables for Round 1 and Round 11
Variable Number
Question Name
Variable Title
Year
R12361.00
SAMPLING_WEIGHT
Round 1 Sampling Weight
1997
R12361.01
SAMPLING_WEIGHT_CC
Round 1 Sampling Weight Cumulative Cases Method
1997
R12362.00
CS_SAMPLING_WEIGHT
Round 1 Cross-Sectional Sampling Weight
1997
R12362.01
SAMPLING_PANEL_WEIGHT
Round 1 Sampling Weight Panel Method
1997
T00421.00
SAMPLING_WEIGHT_CC
Round 11 Sampling Weight Cumulative Cases Method
2007
T00422.00
SAMPLING_PANEL_WEIGHT
Round 11 Sampling Weight Panel Method
2007
There are four different types of weight variables (see the "Question Name" column in Figure 1 above):
Sampling_Weight. Provides a weight for everyone who participated in that particular round of surveying.
Sampling_Weight_CC. Provides a weight for everyone who participated in that particular round of surveying, using a special method of combining the cross-sectional and over-sample cases. This method makes the weight of an oversampled person invariant to which sample the person was drawn from. This reduces the variation in weights and hence improves the statistical efficiency of weighted estimators.
CS_Sampling_Weight. Gives all over-sample members a weight of zero. This weights only cross-sectional participants and provides a simple method of dropping over-sample cases. This weight is useful if a user has a slow computer or needs a simple method of reducing the number of respondents being analyzed without distorting the findings.
Sampling_Panel_Weight: This weights only people who are in every round from 1 to N. Those not in every round get a 0 weight. It is used when data are needed on individuals who participated in all rounds.
For more details about how the weights were calculated, see Methodology for Calculating Weights.
User Notes: Need a quick check to see the impact of weights for a complex statistical situation? Use R12361.01, which is the Round 1 Sampling Weight Cumulative Cases Method. This variable provides a weight for every NLSY97 respondent and adjusts for the over-sampling of blacks and Hispanics. It does not adjust for round-by-round non-response.
Researchers should weight the observations using the weights provided if tabulating sample characteristics in order to describe the population represented (i.e., computing sample means, totals, or proportions). The use of weights may not be appropriate without other adjustments for the following applications:
Samples Generated by Dropping Observations with Item Nonresponses: Often users confine their analysis to subsamples of respondents who provided valid answers to certain questions. In this case, a weighted mean will not represent the entire population, but rather those persons in the population who would have given a valid response to the specified questions. Item nonresponse due to refusals, don't knows, or invalid skips is usually quite small, so the degree to which the weights are incorrect is probably quite small. In the event that item nonresponse constitutes a small proportion of the variables under analysis, population estimates (i.e., weighted sample means, medians, and proportions) are reasonably accurate. However, population estimates based on data items that have relatively high nonresponse rates--such as family income--may not necessarily be representative of the underlying population of the cohort under analysis.
Data from Multiple Waves: Because the weights are specific to a single wave of the study, and because respondents occasionally miss an interview but are contacted in a subsequent wave, a problem similar to item nonresponse arises when the data are used longitudinally. In addition, the weights for a respondent in different years occasionally may be quite dissimilar, leaving the user uncertain as to which weight is appropriate. In principle, if a user wished to apply weights to multiple wave data, weights would have to be recomputed based upon the persons for whom complete data are available. In practice, if the sample is limited to respondents interviewed in a terminal or end point year, the weight for that year can be used.
Users may also create longitudinal weights for multiple survey years by using the NLSY97 Custom Weighting program at www.bls.gov/nls.
Regression Analysis: A common question is whether one should use the provided weights to perform weighted least squares when doing regression analysis. Such a course of action may lead to incorrect estimates. If particular groups follow significantly different regression specifications, the preferred method of analysis is to estimate a separate regression for each group or to use indicator variables to specify group membership; regression on a random sample of the population would be misspecified. Users uncertain about the appropriate method should consult an econometrician, statistician, or other person knowledgeable about the data before specifying the regression model.
User Notes: The NLSY97 data set contains two sampling weight variables for survey rounds 1 through 3: SAMPLING_WEIGHT and CS_SAMPLING_WEIGHT. The first set includes all NLSY97 respondents. These weights (when divided by 100) will add up to an estimate of the number of U.S. residents in the sample age range in 1997. The second set contains weights only for respondents in the cross-sectional sample; all oversample cases have a zero weight. These weights are also designed to produce an estimate of the number of U.S. residents in the sample age range. Since there are fewer respondents if the oversample is omitted, however, each black or Hispanic or Latino respondent in the cross-sectional sample has a larger value.
For research that includes analysis by race, using the regular sampling weights rather than the cross-sectional weights will produce results with higher precision for black and Hispanic or Latino youths. For research that focuses only on non-black, non-Hispanic youths or that does not include any analysis by race/ethnicity, using the cross-sectional weights will save processing time.
Methodology for Calculating Weights
The assignment of individual respondent weights involved a number of different adjustments. Complete details are found in the NLSY97 Technical Sampling Report, which has step-by-step descriptions of the entire adjustment process. Some of the major adjustments are
Adjustment One. Computation of a base weight, reflecting the case's selection probability for the screening sample. This step also corrects for missed housing units and caps the base weights in the supplemental sample to prevent extremely high weights;
Adjustment Two. Adjustment for nonresponse to the screener;
Adjustment Three. Development of a combination weight to allow the black and Hispanic cases from the cross-sectional sample to be merged with those from the supplemental sample (non-Hispanic, non-blacks in the supplemental sample were not eligible for the NLSY97 sample);
Adjustment Four. Adjustment of the weights for nonresponse to NLSY97 interviews;
Adjustment Five. Poststratification of the nonresponse-adjusted weights to match national totals.
Calculation of Weights under New “Cumulating Cases” Strategy
Starting in round 4, a new “Cumulating Cases” strategy has been used to calculate weights. Instead of calculating separate CX and SU base weights and then later combining the separate sample weights, a Horvitz-Thompson approach to weighting is used.
In the Horvitz-Thompson approach, the weights are determined across samples depending only on the overall selection probability (into either sample) of the individual element, giving a single unified set of weights for the cumulated cases. This approach is straightforward. Only Adjustments 1 and 3 above are modified. The probability for a case to be in either sample is simply the sum of the probabilities to be in each sample because the samples are independently drawn. Thus, the base weight for a case is the inverse of the sum of sample selection probabilities for a case:
Under the old strategy, the separate CX and SU step 1 weights were the reciprocal of the selection probabilities for just that sample:
The only other change is that Adjustment 3 is now unnecessary.
Summary of NLSY97 Weights for Rounds 1 through 11
Table 1 summarizes the weights for the first eleven NLSY97 rounds. The sum of the weights for all of these weights is equal, but when there are fewer positive weights to share this sum, the weights tend to increase. The positive weights for Round 1 respondents who are nonrespondents for any of the other rounds are spread around the round’s (or panel’s) respondents.Table 1: Summary Table for NLSY97 Rounds 1-11 Weights Under "Cumulating Cases" Strategy
Round 1
Round 2
Round 3
Round 4
Round 5
Round 6
N (> 0)
8,984
8,386
8,209
8,081
7,883
7,898
Sum
19,378,453
19,378,453
19,378,453
19,378,453
19,378,453
19,378,453
Mean
2,157.00
2,310.81
2,360.64
2,398.03
2,458.26
2,453.59
Standard Deviation
931.01
1,011.70
1,021.15
1,052.08
1,066.22
1,092.47
Minimum (> 0)
760.71
846.23
858.76
889.08
866.55
864.75
5th percentile
887.2
938.12
969
983.16
997.49
992.86
25th percentile
1,072.03
1,135.96
1,168.73
1,185.60
1,222.83
1,204.79
Median
2,596.59
2,777.05
2,806.46
2,869.30
2,955.45
2,955.01
75th percentile
2,909.73
3,111.65
3,154.13
3,203.60
3,271.03
3,293.50
95th percentile
3,268.16
3,556.58
3,649.77
3,786.59
3,831.66
3,904.73
Maximum
15,761.82
16,718.19
16,646.80
16,950.37
17,277.02
17,167.26
Round 7
Round 8
Round 9
Round 10
Round 11
N (> 0)
Sum
Mean
Standard Deviation
Minimum (> 0)
5th percentile
25th percentile
Median
75th percentile
95th percentile
Maximum
Because the samples are multi-stage stratified random samples instead of simple random samples, respondents tend to be clustered in geographic areas (for more information, see Sample Design & Screening Process). In general, these clusters tend to be alike in a variety of ways for a variety of reasons. For example, there may be cultural differences by locality or ecological differences in labor market conditions. Depending upon the degree of this homogeneity, the conventionally computed standard deviations for the variables, which assume a simple random sample, may be too small. However, by controlling the rate at which particular strata are sampled, multi-stage stratified random samples can improve upon simple random samples. The ratio of the correct standard error to the standard error computed under the assumption of a simple random sample is known as the design effect. The NLSY97 Technical Sampling Report provides design effects for the various strata.
As respondents in the cohort get older, mobility may mix the respondents more uniformly through the country, reducing the clustering of the sample as well as the design effects. Many of the persons who started out in the same PSU will have moved to different areas and may no longer be affected by similar unobservable labor market conditions. As this occurs, the error terms in a regression will more closely approximate the standard error computed for a completely random sample. However, some correlation due to respondents coming from the same household or neighborhood will most likely remain.
By examining the geocode data for the NLSY97, it may be possible to control for some of the environmental factors generating design effects or, if desired, to compute design effects based upon county or metropolitan area clusters.
Reference
Moore, Whitney; Steven Pedlow; Parvati Krishnamurty; and Kirk Wolter. National Longitudinal Survey of Youth 1997 (NLSY97) Technical Sampling Report. Chicago: NORC, 2000.
