Chapter 2: Sample Design & Fielding Procedures

• 2.1 Sample Design
• 2.2 Screening
• 2.3 Sampling Process
• 2.4 Interview Schedule & Fielding Periods
• 2.5 Interview Methods & Target Universe
• 2.6 Reasons for Noninterview
• 2.7 Sample Representativeness &Attrition
• 2.8 Sample Weights
• 2.9 Practical Usage of Weights
• 2.10 Design Effects
• 2.11 Confidentiality and Consent

Return to Table of Contents

2.1 Sample Design

Because this section contains a large amount of detail, a rough overview of the sample design is first provided. In 1978, the National Opinion Research Center (NORC) at the University of Chicago created a list of housing units in selected areas of the United States for the first NLSY79 interview. NORC interviewers went to a random sample of these homes and performed a short interview called the screener, which provided basic information on every resident of the household. NORC also extracted a random sample of Department of Defense records to ensure the survey included members of the military.

Together these two processes provided information, such as name, age, sex, race, and address, on more than 155,000 people. This information was then used to identify all individuals aged 14 to 21 as of December 31, 1978. Based on this screener information, each appropriately aged individual was assigned to one of the sample groups. Then, in 1979, NORC interviewers asked individuals on this list to participate in the first NLSY79 interview. Any person who completed the first round interview is considered a member of the NLSY79 cohort.

Three independent probability samples comprise the NLSY79. These samples are designed to represent the entire population of youth aged 14 to 21 as of December 31, 1978, residing in the United States on January 1, 1979. The three samples are:

(1) a cross-sectional sample (6,111) designed to be representative of the noninstitutionalized civilian segment of young people living in the United States in 1979 and born January 1, 1957, through December 31, 1964

(2) a set of supplemental samples (5,295) designed to oversample civilian Hispanic or Latino, black, and economically disadvantaged, non-black/non-Hispanic youth born in the same time period

(3) a military sample (1,280) designed to represent the population born January 1, 1957, through December 31, 1961, serving in the military as of September 30, 1978. The inclusion of the military sample allows comparative civilian/military analyses by ensuring more than the pro rata share of cohort members in the military

Beginning in 1986, additional information was collected about children born to female NLSY79 respondents. The child sample, when weighted, is representative of American children born to the population of women born in 1957 through 1964 and living in the United States in 1979. The sampling procedures used by NORC to select the civilian and military subsamples differed and are discussed separately below. For additional information on NLSY79 sampling procedures, see Frankel et al. (1983) and the Interviewer’s Manual for the 1978 household screening (NORC 1978). Sampling issues related to the Children of the NLSY79 are discussed in Baker et al. (1993) and in the NLSY79 Child & Young Adult Data Users Guide.

Return to top

2.2 Screening

To find people of the correct age and ethnic composition, NORC screened a large number of individuals in two separate procedures. First, household screening interviews were conducted to select the NLSY79 civilian cross-sectional and supplemental subsamples from the civilian population. Then, a second screening was done to identify the military sample. While the civilian screening made use of field interviewers going to preselected households, the military sample was drawn from Department of Defense internal records.

NORC administered the civilian sample screening interview in approximately 75,000 dwellings and group quarters. These interviews occurred in 1,818 sample segments of 202 Primary Sampling Units (PSUs), which included most of the fifty states and the District of Columbia. The screening interview was designed to elicit information that would identify persons eligible for inclusion in the NLSY79 sample. The civilian screening interviews were completed within 91.2 percent of the cross-sectional and 91.9 percent of the supplemental occupied dwelling units selected for screening.

Cross-Sectional Sample: Approximately 18,000 of the screening interviews were carried out among 918 sample segments in 102 Primary Sampling Units (PSUs), which were selected from the NORC Master Probability Sample of the United States.

Supplemental Sample: A total of 57,000 screening interviews for the supplemental sample were carried out among 900 sample segments in a 100-PSU sample specifically designed to produce statistically efficient samples of Hispanics or Latinos, blacks, and economically disadvantaged, non-black/non-Hispanics. The NLS sample design, which selected every eligible person connected to the household, generated a representative sample of siblings and spouses living in the same household and satisfying the age restrictions stated above. However, NLSY79 samples do not contain nationally representative samples of siblings and spouses of all ages and living arrangements. When the NLSY79 is used to study sibling pairs and married couples, care must be used in generalizing from the findings of such studies.

Procedures were also developed to establish “linkages” between dwellings and certain types of individuals who might be temporarily absent. As part of the initial screening for the civilian sample, household respondents were asked if there were any persons with primary family connections to the household who were away from the household at the time. Included in this group were college students, military personnel, and those in prisons or other institutions. Household screener respondents were also asked to name persons who might occasionally stay at the dwelling who did not have any other “usual place of residence.” For each individual identified in this process, an attempt was made to determine whether the individual would be “linked” to some other household, e.g., college students living off campus in their own dwelling units. All individuals without other linkages were included in the household composition for purposes of subsampling.

Military Sample: Persons on active military duty as of September 30, 1978, were sampled from rosters provided by the Department of Defense. No formal screening interview was conducted.

Return to top

2.3 Sampling Process

Civilian Samples: All civilian sample selection was accomplished through a multi-stage stratified area probability sample of dwelling units and group quarter units. A moderate degree of oversampling of dwelling units within sample listing segments was employed in order to increase the sample composition with respect to the targeted groups of the supplemental sample.

Base year samples of Hispanics or Latinos, blacks, and economically disadvantaged, non-black/non-Hispanics were selected from individuals identified in both the 102 PSU cross-sectional sample and the 100 PSU special purpose sample. To the extent that individuals identified in the screening phase were obtained with different probabilities of selection (because of selective oversampling), the weighting of base year samples attempts to minimize these probability differences. Since the use of oversampling tends to decrease sample efficiency (i.e. variance), attempts were made to hold required oversampling to a minimum.

Figure 2.3.1 Number of Respondents Born Each Month by Birth Year ¹

¹ The month and year of birth is taken from the 1978 screener (R00003. and R00005.). Respondents were asked about their dates of birth again in 1981, but the use of these values does not change the results indicated above. The 1957-63 value can be found by averaging the total number of birth dates reported for each month over all of the years.

Assignment of a youth to either the cross-sectional or supplemental sample was made using information collected during the household screening interviews and a set of coding instructions prepared by NORC. During the 1978 household screening interviews—from which the sample of NLSY79 respondents was drawn—information was collected on the sex, race, and ethnic origin of each household member and on the total income of the family unit during the past 12 months. A detailed set of coding procedures transformed these raw data into a combined racial/ethnic identifier and an economically disadvantaged qualifier. These criteria were used not only to assign a youth to one of the various subsamples, e.g., the economically disadvantaged, non-black/non-Hispanic supplemental sample, but also to specify the primary race/ethnicity variable, which provides the basis for weighting.

Other technical information on the sample assignment process can be found in (1) the Household Screener and Interviewer’s Manual (NORC 1978), which includes a copy of the screening instrument and detailed instructions to interviewers for administering the race, ethnic origin, and family income questions; (2) the Technical Sampling Report (Frankel et al. 1983), which describes the NLSY79 sample selection procedures for the civilian and military subsamples; (3) the 10/4/78 NORC memorandum, which provides the rules used to assign race and poverty status from responses to the screening questions; (4) a copy of the 1978 poverty income levels by family size and farm-nonfarm residence; and (5) the “Race, Ethnicity, & Nationality” section of this User’s Guide, which summarizes information in these documents as it relates to the assignment of “Hispanic,” “Black,” and “Non-Hispanic, Non-Black” origins used in the sample identification code variable (R01736.) and the race/ethnicity variable (R02147.). Base year interviews with the three subsamples were conducted between January and mid-August 1979. Table 2.3.1 summarizes base year completion rates for each subsample.

Table 2.3.1 Base Year Interview Completion: NLSY79

Cross-Sectional Sample: Following the initial screening process, 6,812 individuals from the cross-sectional sample were designated to be interviewed in the base year; of those, 90 percent or 6,111 respondents were actually interviewed in 1979. The cross-sectional sample is designed to maximize the statistical efficiency of samples which are “cross-sectional” with respect to the rural population. Specifically, through the several stages of sample selection (counties, enumeration districts-block groups, sample listing units), probabilities of selection are based upon either total population or total housing units. Except for the economically disadvantaged supplemental sample, sampling of non-black/non-Hispanic respondents was restricted to the 102 PSU National Sample.

Supplemental Sample: After screening, 5,969 individuals from the supplemental sample were designated for base year interviews, and of these, 89 percent or 5,295 respondents were actually interviewed. Stratification specifically relevant for Hispanics or Latinos, non-Hispanic blacks, and economically disadvantaged, non-black/non-Hispanics was used. Probability proportional to size procedures were based on size measures for these groups rather than for the general population, making it possible to more nearly equalize the distribution of the targeted groups among the various sampling units than would otherwise be the case.

Military Sample: Of the 1,793 military youth selected for interviews, 1,280 or 72 percent were interviewed in 1979. Selection of the military sample was accomplished in two stages. In the first stage, a sample of approximately 200 “military units” was selected. These units were selected with probabilities proportional to the number of persons born in 1957 through 1961 and serving in the military unit as of September 30, 1978.

Within selected units, persons born in 1957 through 1961 were subsampled with probabilities inversely proportional to the first-stage selection probability. Females were oversampled at a rate approximately six times that of males in order to produce approximately 850 males and 450 females. Within each sex, the sample was stratified on the basis of branch of military service (Army, Navy, Air Force, and Marine Corps) and geographic location (Eastern U.S., Western U.S., Europe, Far East, other). Of those interviewed in 1979, 824 military respondents were male and 456 were female (see Table 2.3.2). The entire military sample was eligible for interview from 1979–84.

Table 2.3.2 NLSY79 Military Respondents Interviewed in 1979 by Gender, Race & Military Branch

Child Sample: The number of children assessed during a given child survey year is a function of the number of children born to interviewed NLSY79 mothers, the number of children living in the homes of those mothers, and, finally, the number of those children actually interviewed. Of the 5,842 NLSY79 females eligible for the first child interview in 1986, more than 2,900 mothers and 4,971 children were interviewed. From this sample of eligible children, assessment data were collected for 4,786. Table 1.5.1 in Chapter 1 presents sample sizes across subsequent survey years.

Return to top

2.4 Interview Schedule & Fielding Periods

The original interview schedule, which called for yearly personal interviews with NLSY79 respondents, was retained from 1979 through 1986. In 1987, budget constraints dictated a limited phone interview rather than a personal interview. Personal interviews resumed with the 1988 round and continued yearly until 1994. Since 1994, NLSY79 respondents have been interviewed every other year (1996, 1998, etc.).

The initial NLSY79 interviews were conducted between late January and mid-August 1979. The next several interviews were fielded in the first six months of the year; subsequent surveys have typically begun in late spring and ended in the fall or early winter. Table 2.4.1 provides information on the fielding periods for NLSY79 respondents.

Table 2.4.1 NLSY79 Main Youth Fielding Periods

From 1979 until 1986, timing of the fielding period was designed to allow all respondents still in school to be interviewed before they left to take temporary summer jobs. Detailed information was collected for jobs held by respondents while they were in school. Since the youngest respondents in the survey were 23 years old in 1988, the shift in fielding periods after 1987 had a relatively small impact on information on jobs held while in school. An attempt was made during the initial survey years to keep the fielding period for an individual respondent approximately the same from year to year in order to assure that the time between interviews was approximately twelve months.

Researchers conducting analyses on topics where time periods are critical should carefully examine the reference period of the questions, the actual interview date, and the duration since the preceding interview.

Return to top

2.5 Interview Methods & Target Universe

During each survey round, NORC attempts to reach all youth within the active samples. No respondents have been routinely excluded from locating efforts with the exception of respondents who have died or, in a small number of cases, were judged to be extremely difficult. The permanent NLSY79 sample designated for interviewing during the 1979–84 interview years consisted of all civilian and military youth who were interviewed in the base year and who were alive at the survey date.

In 1985, when interviewing of the full military sample ceased, the total NLSY79 sample size dropped from 12,686 to 11,607. Retained for interviewing in post-1984 surveys were 201 military respondents randomly selected from the entire military sample of 1,280; the remaining 1,079 military respondents were eliminated from the sample. The 201 military members who were retained included (1) 51 cases that would have been selected as part of a random sample of youth including the military and (2) 150 additional cases selected to provide a sufficient number of original military sample members to avoid overly large sampling variability for the military sample. Beginning in 1991, the 1,643 members of the economically disadvantaged, non-black/non-Hispanic supplemental sample were no longer interviewed. Unless otherwise noted, eligible sample sizes reported in NLS publications include deceased and difficult-to-field respondents but exclude those respondents dropped from the sample. Additional information on numbers and characteristics of noninterviewed respondents can be found in the "Reasons for Noninterview" section later in this chapter.

NLSY79 respondents reside in each of the 50 states as well as the District of Columbia, U.S. territories, and countries abroad. Locating respondents is a coordinated effort of NORC’s central office, its locating shop, and local-level field staff. Prior to fielding, NORC’s central office sends a short, informative “advance letter” to each respondent reminding him or her of the upcoming interview and confirming the respondent’s current address and phone number. Field staff locating efforts begin with this information and locator sheets.

At the local level, interviewers are responsible for contacting all respondents in their caseloads and for tapping additional local resources (post offices, departments of motor vehicles and vital statistics, etc.) to locate those respondents who have moved. If an interviewer is unsuccessful in locating a respondent, the case is transferred to the field manager who undertakes additional locating strategies.

In the event that such local-level efforts fail, the case is forwarded to NORC’s locating shop in Chicago where the complete hard copy files on each respondent can be accessed and used for additional locating efforts. Respondents who cannot be located are only a small percentage of the total not interviewed in a given survey year. (For more information about noninterviews, refer to section 2.6 in this chapter.)

In addition to its comprehensive locating efforts, NORC makes every effort to convert initial respondent refusals to completed interviews. For uncooperative respondents, NORC sends “refusal conversion letters” designed to encourage continued participation in the survey. These letters are often written by field managers who have personal knowledge of specific respondents and can customize the letter to an individual respondent’s concerns. Over the years, NORC has successfully conducted interviews with 33–50 percent of respondents who initially refused. NORC’s extensive locating methods and its conversion strategy, combined with its close monitoring of response rates for each of the subsamples of the NLSY79, have resulted in relatively high retention rates for a longitudinal panel of this duration.

While personal interviewing has been the primary contact method used for all but one of the NLSY79 surveys, it is not the exclusive method. In previous rounds telephone contact to complete the survey occurred under certain circumstances, e.g., when the respondent resided in a remote area or field staff determined that phone contact was the preferred method of interviewing a specific respondent. The number of telephone interviews has increased in recent years as respondents have become more dispersed or expressed a preference for phone interviews. The percent of surveys conducted by telephone for each survey year are shown in Table 2.5.1.

Table 2.5.1 Percent of NLSY79 Interviews Conducted by Telephone, 1979-2004

In rare cases, interviews are conducted in whole or in part with a proxy, a person other than the respondent (for example, four in 1991, two in 1992). A variable, entitled ‘Interview Conducted with Proxy Respondent,’ is present in the data to identify these interviews. In order to conduct such an interview, individual approval must be obtained by the NORC central office and the circumstances documented.

A Spanish version of all survey instruments, except the Young Adult questionnaire, is prepared and NORC employs bilingual, Spanish-speaking interviewers. During the 1992 interview, for example, 104 respondents requested the use of a Spanish version of the questionnaire.

The average length of a personal interview is approximately one hour. The 1987 telephone interviews were completed within about 40 minutes, while the administration of the child assessments added approximately 45 minutes to the total survey administration time for each child.

Until 1989, the NLSY79 was conducted using only paper-and-pencil interviews (PAPI). PAPI interviews were performed by interviewers filling in the relevant fields of large printed questionnaire booklets. While these booklets were inexpensive to produce, interviewers could make mistakes in following complicated skip patterns and filling in answers. Moreover, after all interviews were completed, additional office staff were needed to transcribe the information collected. Computer-assisted personal interviews (CAPI) were designed to eliminate many of these problems. For CAPI interviews, interviewers take laptop computers into the field instead of questionnaire booklets. A computer program automatically selects the next question, prevents interviewers from entering illegal values, and warns interviewers about questionable answers. The computer also eliminates the need for data transcription except for specific items collected verbally and coded later.

While the majority of interviews in 1990 were collected using PAPI materials, a subset of one fourth of respondents was administered their survey using CAPI methods in order to test the viability and reliability of CAPI administration. Due to the success of these experiments, the NLSY79 interviews became fully CAPI administered beginning in 1993. Users interested in the results of these experiments should consult Olsen (1991).

From 1979 until 1994, each respondent was paid $10 upon completion of the interview. Respondents were paid $20 beginning with the 1996 interview and $40 beginning with the 2002 interview. NLSY79 mothers participating in the child assessments receive an additional $10 per child. Through 1994, young adults were paid $10. Beginning with the 1996 interview they received $20 and in 2004 received $40. NLSY79 respondents who participated in the 1980 ASVAB testing were paid $50 each.

Retention vs. Response Rates

Retention rates for NLSY79 respondents throughout the first 16 rounds of the survey exceeded or were very near 90 percent. Rounds 17 and 18 saw retention rates of approximately 85 percent. Retention rate is calculated by dividing the number of respondents interviewed by the number of respondents remaining eligible for interview. All 1979 (round 1) respondents including those reported as deceased are eligible for interviews, with the exception of those who have been permanently dropped from the sample. In the round 21 (2004) survey, 7,661 civilian and military respondents out of the 9,964 eligible were interviewed, for an overall retention rate of 76.9 percent. Retention rates for each survey are shown in Table 2.5.2. This table also shows which interviews were conducted with paper-and-pencil interviewing (PAPI) and which were executed with computer-assisted personal interviewing (CAPI).

The number of respondents can also be expressed as a percentage of the number of base year respondents not known to be deceased. This is referred to as the response rate and is reported in Table 2.5.3 for each survey round. As of 2004, 396 main respondents (nearly 4 percent of the respondents eligible for interview) had been reported as deceased. The response rate for those believed to be alive is 81.1 percent.

Table 2.5.2 Sample Sizes & Retention Rates by Sample Type: NLSY79

Table 2.5.3 Response Rates (Excluding Deceased Respondents) by Sample Type: NLSY79

Return to top

2.6 Reasons for Noninterview

A ‘Reason for Noninterview’ variable is constructed for each survey year (excluding 1979) in the NLSY79 and provides an explanation of why an interview could not be conducted or completed with a respondent. The cause of noninterview is assigned by the NORC interviewer to each respondent designated as a member of the eligible sample for a given survey year. Typical coding categories have included reasons such as an interview being refused by the respondent or by the respondent’s parent, the youth and/or the family unit not being successfully located, or the respondent being reported as deceased.

Beginning in the 1980s, two administrative categories were added. One reflected a decision by NORC not to refield certain cases that were determined to be extremely difficult to interview. The second category indicates that, due to funding cutbacks, interviews would not be attempted with certain members of one or more of the NLSY79 subsamples. Thus, beginning in 1985, interviews ceased for 1,079 respondent members of the military subsample; each was permanently assigned a reason for noninterview of “military sample dropped.” A second group of respondents, those belonging to the supplemental economically disadvantaged, non-black/non-Hispanic sample, was similarly dropped from interviewing beginning with the 1991 survey. The target universe for each survey year—that is, the respondents whom NORC attempts to interview—thus includes all respondents interviewed in the initial survey year exclusive of those who were (1) reported deceased at an earlier interview; (2) dropped from the sample; or (3) judged to be extremely difficult to interview.

The tables below present the number of respondents not interviewed across survey years by reason, gender, race/ethnicity, and sample type.

Table 2.6.1 Reasons for Noninterview: NLSY79 1979-2004

Table 2.6.2 Reasons for Noninterview by Gender: NLSY79 1980-2004

Table 2.6.3 Reasons for Noninterview by Race/Ethnicity: NLSY79 1980-2004

Key H: Hispanics or Latinos, B: Blacks, NB/NH: Non-Black/Non-Hispanics

Table 2.6.4 Reasons for Noninterview by Sample Type: NLSY79 1980-2004

Key C: Cross-sectional, S: Supplemental, M: Military