PSAT, SAT, etc .

[James A. Landau]

In a message dated 5/21/2003 11:42:19 AM Eastern Standard Time,
hpst at EARTHLINK.NET writes:

> I wonder if it might be possible for us to obtain an old copy of the verbal
> parts of one of these tests so that we could examine it and discuss its
> questions in terms of dialectical differences on this list.

In a message dated 5/21/2003 4:04:59 PM Eastern Standard Time,
hpst at EARTHLINK.NET writes:

> I am not taking a position on this either, but I would be interested in
> looking at an old test in order to see whether or not other questions on the
> PSAT or SAT are equally as controversial in terms of "correct" answers as
> this one is, and I cannot think of a better group of people to investigate
> this subject than members of the ADS.
>
> Perhaps questions are culture/dialect neutral perhaps they are not but since
> I have no evidence for or against my proposition I think that this is a
> subject worthy of investigation.

1)  Either ETS or the College Entrance Examination Boards (I don't know
which) publishes pamphlets with sample questions for students preparing to take the
SAT's and PSAT's.  I imagine new versions of these pamphlets are issued
regularly.  I imagine that these pamphlets are updated and re-issued regularly.
One could contact ETS or CEEB and ask if a file of these pamphlets from a number
of years was available.

2)  Some years ago there was a big fuss about whether the SAT's were
"culturally biased".  The literature spawned by this debate probably contains samples
of grammar questions about which there was disagreement as to whether the
SAT's grammatical rules were correct.

(One example:  the analogy "cup:saucer" was attacked on the ground that,
allegedly, to most African-Americans a cup was something one placed on the table,
not in a saucer.)

3)  The theory behind the SAT is that there is a high correlation between the
skills needed to get a high score on the SAT and the skills needed to do well
in college.  "Correlation" is used here in the technical sense defined by
statisticians.

I don't know if the theory is correct, or more exactly how accurate this
theory is.  However I do know that the CEEB tracks SAT scores versus graduation
rates by college, and therefore the data to examine the theory does exist.

Hence the SAT is not supposed to test for real-world English usage but rather
tests that grammatical knowledge which, as best as the CEEB and ETS can
determine, correlates highly with success in college.  If this means testing for a
"book" knowledge of English rather than real-world usage of English, so be it.
 As an extreme example, consider someone whose native language is not English
who takes the SAT.  S/he has no home usage of English, right or wrong.

4)  The CEEB refuses to admit it, but the SAT is a rather good facsimile of
an IQ test.  Both test for similar skills.  The difference is that the CEEB
only talks about a correlation with success in college, whereas the makers of IQ
tests claim that the score on the test "reflects" (rather than merely has a
statistical relationship with) the person's "intelligence".

I mention IQ tests because, to the best of my knowledge, all tests that are
scored in IQ points include questions on "general knowledge".  For example, I
once on an IQ test encountered a question about the infield fly rule in
baseball.  The "general knowledge" portion of the IQ test means that the IQ test is
necessarily culturally biased, no matter how conscientious the author of the
test may be.

The SAT's do not have a general knowledge section per se; however many of
their questions test the students' knowledge base as well as their
problem-solving ability.  Example: the verbal analogy questions, which assume an extensive
and exact vocabulary on the part of the student.  Hence it is probably
impossible to devise an SAT that was not culturally biased.  (One could, however,
devise an SAT that was biased culturally towards African-Americans or Hispanics or
Lithuanians).

Therefore the honest thing that the CEEB and ETS should do is to devise not a
culture-neutral test but rather a test that accurately reflects the cultural
bias of the colleges that the test-takers want to get into.  Therefore as the
cultural bias of colleges in the USA changes, e.g. as "diversity" gets
accepted, then the SAT SHOULD change to reflect the changes in the colleges who use
it.

In a message dated 5/21/2003 2:13:41 PM Eastern Standard Time,
JMB at STRADLEY.COM writes:

> I think you're making a leap here, based on the evidence you cite.  <snip>
> One problematic question, out of more than a thousand, doesn't tell us much.
>
> <snip>
>     Note that I'm not taking a position at this time on whether such tests
> are invidious.  I just don't think that the discussion over this one question
> gives any evidence for that position.

Thank you.  This is the first sensible thing anybody has said on this thread!

            James A. Landau
            systems engineer
            FAA Technical Center (ACB-510/BCI)
            Atlantic City Int'l Airport   NJ   08405   USA