The NLSY97 collects information about a variety of standardized achievement tests commonly taken by young adults in junior high school and high school; these data are described in the "Achievement Tests" subsection. In round 1, most NLSY97 respondents participated in the administration of a national test, the computerized version of the Armed Services Vocational Aptitude Battery (ASVAB), described in the "Administration of the CAT-ASVAB" subsection. Finally, certain NLSY97 respondents take the Peabody Individual Achievement Test (PIAT) Math Assessment as a part of the interview. The administration of this assessment is discussed in the final subsection.
Table 1 summarizes the NLSY97 User's Guide subtopics available in this section and any global universe restrictions affecting those subtopics. Users should note that certain questions or groups of questions within a subtopic may have additional restrictions. Please consult the NLSY97 questionnaire and codebook for more information on particular questions.
4.1 Table 1. Aptitude, Achievement & Intelligence Scores Subtopics and Universe Restrictions
|
NLSY97 User's Guide Subtopic |
Round 1 Universe |
Rounds 2-6 Universe |
Rounds 7-9 Universe |
|
4.1.1 Achievement Tests |
all ages |
11th grade or higher |
11th grade or higher |
|
all ages |
-- |
-- |
|
|
4.1.3 PIAT Math Test |
all ages; 9th grade or lower |
age 12 as of 12/31/96; 9th grade or lower in rd. 1
|
-- |
Both the youth and the parent questionnaires have collected information about the youth's performance on various standardized achievement tests. In addition, scores are available for some respondents from the transcript survey, which collects transcripts as respondents leave high school.
Youth Questionnaire. Achievement test information was collected in round 1 from respondents who were enrolled in or who last attended the 9th grade or higher and from respondents with a GED or a high school diploma. The NLSY97 questioned these respondents about three specific achievement tests--the SAT I, American College Test (ACT), and Advanced Placement (AP) tests. For each test reported, the respondent stated the grade level(s) when it was taken and the highest score earned on each test (SAT I Verbal, SAT I Math, and ACT). Those who took AP exams were also asked for the subjects of the tests (e.g., biology, history, economics).
In rounds 2-9, similar information was gathered from respondents who last attended the 11th grade or higher. For the SAT I and ACT, the survey asked for the grade level(s) when each test was taken and the highest score received. In these surveys, respondents who took AP exams were only asked for the subject areas of these tests and did not report their scores.
User Notes: In the data set, the variables for to each
test are identified only by loop numbers; users must know which loop
numbers correspond to each test to use these data. For example, the
round 1 data include questions YSCH-8200.01 and YSCH-8200.02, for which
the question text reads, "What grades were you in when you took the [SAT
I or ACT]?" For all questions such as this, variables with "TEST 01" in
the title provide information about the SAT I and variables with "TEST
02" in the title pertain to the ACT. Similarly, for questions about the
ACT (for example: YSCH-9200.01-YSCH-9200.05, "Highest Score R Received
on ACT Test"), the test number in the variable title indicates the ACT
subtest to which the variable applies. For these questions, the sections
are coded as follows:
|
Parent Questionnaire (round 1). If the youth had been exclusively home-schooled since the seventh grade, the responding parent was asked about achievement tests taken. These questions included whether the youth had ever taken a standardized achievement test (e.g., SAT I, ACT, or California Achievement Test [CAT]). If the NLSY97 respondent had taken one or more of these tests, the parent reported the year when that youth last took a test with a math component and the highest score he or she ever received on that component. The parent then reported the same information for the last test the youth took with a verbal component.
Transcript Survey. In 1999-2000 and in 2004, school transcripts were collected for a total of 6,232 respondents. If these transcripts included scores on achievement tests such as the ACT, PSAT, SAT I, SAT II, or AP tests, those scores are reported in the transcript data file. More information is provided in section 4.2.5, "School & Transcript Surveys."
Comparison to Other NLS Surveys: Achievement test scores were collected during special transcript or school surveys for the NLSY79, the Young Women, and the Young Men. These scores were reported by the schools or coded from the respondent's transcript, rather than being reported by the respondent as in the NLSY97. NLSY79 data include PSAT, SAT I, or ACT scores for about a quarter of civilian respondents, as well as scores for smaller numbers of respondents on a variety of other achievement and intelligence tests. Available scores for respondents in the Young Men's and Young Women's cohorts are primarily from the California Test of Maturity and the Otis/Beta/Gamma; a few respondents have SAT I or ACT scores recorded, and a wide variety of other tests are also included. For more precise information, see the appropriate cohort's User's Guide.
Survey Instruments: Questions on achievement tests are found in the schooling section (question names begin with YSCH) of the Youth Questionnaire and in section PC8 of the round 1 Parent Questionnaire. The data from the Transcript Survey may also contain achievement tests scores.
|
Related User's Guide Sections |
4.1.2 Administration of the
CAT-ASVAB
4.1.3 PIAT Math Test 4.2.1 College Experience 4.2.4 School Experience 4.2.5 School & Transcript Surveys |
|
Main Area of Interest |
Achievement Tests |
| Supplemental Areas of Interest |
College Experience Ed. Status & Attainment Expectations School Experience Transcript Survey |
From the summer of 1997 through the spring of 1998, most NLSY97 round 1 respondents participated in the administration of the computer-adaptive form of the Armed Services Vocational Aptitude Battery (CAT-ASVAB). In this study, also called the Profile of American Youth 1997, the Department of Defense (DOD) used the NLSY97 sample as part of a larger effort to establish new norms for the CAT-ASVAB, a military enlistment test.
The Armed Services Vocational Aptitude Battery measures the respondent's knowledge and skills in the topical areas listed in Figure 1. The flexibility of the computer-adaptive version allows the test program to route youths through each topical series of questions according to their responses. If a respondent answers a question correctly, the next question will be more difficult; if the respondent answers the question incorrectly, his or her next question will be comparatively easier. The CAT-ASVAB also permitted the administration of an "easy" form of the test to the youngest NLSY97 youths, those born in 1983 and 1984. In the easy form, the first question in each section is easier than the standard form.
4.1.2 Figure 1. Contents of the CAT-ASVAB
|
Arithmetic reasoning |
Electronics information |
Numerical operations |
|
Assembling objects |
General science |
Paragraph comprehension |
|
Auto information |
Mathematics knowledge |
Shop information |
|
Coding speed |
Mechanical comprehension |
Word knowledge |
Several different types of ASVAB scores are created. First, the data includes final ability estimates that summarize the respondent's performance on each subtest. Because computer-adaptive testing means that respondents do not answer the same questions, their raw scores cannot be directly compared. To provide a score that can be compared across respondents, DOD created the final ability estimates using Item Response Theory (IRT). These scores are computed on a comparable scale and thus, can be compared across respondents--that is, a lower score indicates poorer performance, and a higher score indicates better performance.
Second, NLS staff have created a summary percentile score variable for four key subtests; this variable is similar to the AFQT score that is familiar to experienced NLSY79 users. This variable, ASVAB_MATH_VERBAL_SCORE_PCT, was created as follows: NLS staff first grouped respondents into three-month age groups. That is, the oldest cohort included those born from January through March of 1980, while the youngest were born from October through December 1984. Custom sampling weights were computed for the sample of 7,093 respondents who have scores on all four exams and were assigned for each respondent's scores. Within each three-month age group and using the sampling weights, NLS staff assigned percentiles for the theta scores for the tests on Mathematical Knowledge (MK), Arithmetic Reasoning (AR), Word Knowledge (WK), and Paragraph Comprehension (PC) based on the weighted number of respondents scoring below each score. Percentile scores for WK and PC were added to get an aggregate Verbal score (V) for which an aggregated intra-group, internally normed percentile was then computed. The percentile scores for MK, AR and two times the aggregated percentile for V were then summed. Finally, within each group NLS staff computed a percentile score, using the weights, on this aggregate score, yielding a final value between zero and 99.
| User Notes: Users should note that, although the formula
is similar to the AFQT score generated by the Department of Defense for
the NLSY79 cohort, this variable reflects work done by NLS program staff
and is neither generated nor endorsed by the Department of Defense.
More information about the statistical methods used to create both the IRT subtest scores and ASVAB_MATH_VERBAL_SCORE_PCT is provided in appendix 10 in the NLSY97 Codebook Supplement. |
In addition to scores on the tests, the ASVAB data available in the current data set include the following sets of variables:
The number of items answered in each section.
An estimate of the precision of the ability scores (not included for coding speed and numerical operations).
Answers to the "online questionnaire," which respondents answered during the ASVAB administration. The questions collected information on the respondents' school experience, family background, reasons for taking the ASVAB, and perceptions of the test-taking environment.
Question timings for each of the items in the online questionnaire.
During the NLSY97 screening process, two additional nationally representative samples were identified to complete the CAT-ASVAB. The first group, the Student Testing Program (STP), consisted of students, screened during the spring and summer of 1997, who expected to be in the 10th through 12th grades in the fall of 1997. Included were many respondents who also participated in the main NLSY97 survey, as well as youths who refused to participate in or were not eligible for the NLSY97.
The second sample, the Enlistment Testing Program (ETP), was a nationally representative sample of youths 18 to 23 years old as of June 1, 1997. This group provided the normative information that will be used by the Department of Defense to determine the score distribution of military-eligible youths and will help to assess the impact of these tests on minority and female military eligibility. The sample for this group, none of whom were eligible for the NLSY97, was drawn during the same screening period.
The administration of the CAT-ASVAB was conducted by NORC representatives according to standard ASVAB procedure guidelines; respondents were paid $75 for their participation. Groups of five to ten persons were tested under standardized conditions at more than 280 test sites throughout the United States. Many respondents took the CAT-ASVAB at Sylvan Learning Centers; others were administered the test at temporary sites including hotels, community centers, and libraries. A total of 7,127 NLSY97 respondents (or 79.3 percent of the NLSY97 sample) completed this test: 5,452, or 80.8 percent, of the cross-sectional sample and 1,675, or 74.9 percent, of the supplemental sample.
| User Notes: Respondents who did not take the ASVAB are assigned a -4, valid skip, in the data. For most NLSY97 variables, a valid skip indicates that the respondent was not supposed to be asked a particular question. However, all respondents were eligible for the ASVAB administration, and a valid skip for these variables means that the respondent chose not to participate. |
Comparison to Other NLS Surveys: The ASVAB, in paper-and-pencil form, was administered to NLSY79 respondents in 1980. Scores are available for 93.9 percent of that cohort's members. For more information, consult the NLSY79 User's Guide.
|
Related User's Guide Sections |
4.2.1 College Experience |
| Main Area of Interest | Achievement Tests |
| Supplemental Areas of Interest |
College Experience Common Variables School Experience |
The Peabody Individual Achievement Test (PIAT) measures academic achievement of children ages five and over. It is among the most widely used brief assessments of academic achievement, with demonstrably high test-retest reliability and concurrent validity. One of the PIAT subtests, the Mathematics Assessment, was given to round 1 respondents not yet enrolled in the 10th grade. In rounds 2 through 6, this test was given to respondents who were age 12 as of December 31, 1996, and who were in the 9th grade or lower in round 1. By comparing the test score on the PIAT Math Assessment with information on the respondent's math courses from the Youth Questionnaire and the transcript survey, researchers may be able to assess correlations between curriculum and math performance. The round 6 test was restricted to respondents still enrolled in high school.
The version used is the PIAT-Revised (PIAT-R), published in 1989. Administration of this test is relatively straightforward. Respondents enter the assessment at an age-appropriate item (although this is not essential to the scoring) and establish a "basal" by attaining five consecutive correct responses. A "ceiling" is reached when five of seven items are incorrectly answered. Created variables report a standard score (CV_PIAT_STANDARD_SCORE) and a percentile (CV_PIAT_PERCENTILE_SCORE); creation of these variables is described in the NLSY97 Codebook Supplement. Further information about the PIAT Math Assessment is found in Markwardt (1989).
Notes on the PIAT-R Math Test: As with any educational or psychological test of this nature, the PIAT-R does have certain limitations. To reiterate some of the cautions stated in Markwardt (1989), users of the data should be careful to avoid the following pitfalls:
This test is not designed for use as a diagnostic test; due to the nature of the test and the wide range of content, it does not include enough items to measure any specific skill area reliably.
This test is not designed to provide a highly precise assessment of achievement. Although it is a reliable general measure of achievement in the area of mathematics, it cannot make fine distinctions regarding the absolute attainment level of a respondent.
The items included in the PIAT-R Math Assessment were selected to represent a cross-section of various curricula in use across the United States. Although this allows for meaningful comparisons of an individual's scores and the normative national sample, the user must consider curricular and environmental differences of respondents in interpreting the results.
New norms were released in 1998 (Markwardt). The NLSY97 uses the original and the updated norms for rounds 1-3 and the updated norms for rounds 4, 5, and 6. Researchers should use the variables CV_PIAT_STANDARD_SCORE and CV_PIAT_PERCENTILE_SCORE for original norms and CV_PIAT_STANDARD_UPD and CV_PIAT_PERCENTILE_UPD for the updated norms. This math subtest shows a great degree of improvement in student performance. Average performance has risen at grades 5 through 12, and the performance level of above-average students has increased across almost the entire grade range (grades 2 through 12). Nevertheless, as on most other subtests, the performance of below-average students has declined at grades 1 through 12. Throughout the NLSY97 rounds, the test content and administration are unchanged; only the scoring has been updated.
Comparison to Other NLS Surveys: The Children of the NLSY79 age five and older have been administered an earlier version of the PIAT Math Assessment. See the Child and Young Adult Data Users Guide for more information.
Survey Instruments: This assessment is given during the PIAT section (question names begin with YPIA) of the youth interview.
|
Related User's Guide Sections |
|
| Main Areas of Interest |
Created Variables PIAT |
| Supplemental Area of Interest | School Experience |
Markwardt, Jr., Frederick C. Peabody Individual Achievement Test-Revised. Circle Pines, MN: American Guidance Service, Inc., 1998.