I once read an article in the New York Times that reported a fact so implausible my immediate impulse was to debunk it. Here it is:
More than 18 percent of marriages by Chinese- and Japanese-Americans [are] to American Jews.
18 percent – chai percent! – is a huge number. I call bullshit. But how to prove it?
My initial attempt to fact-check this quote last September failed miserably. I tried to demonstrate statistically that, given the size of the populations in question, the purported numbers were patently absurd and ridiculous.
Once I got my hands on some relevant statistics, I remained thoroughly skeptical, but was at least satisfied that what the New York Times had reported was within the realm of statistical plausibility.
Defeated in my quest to fact-check the New York Times, but unwilling to let all my effort go to waste, I ended up writing some ridiculous post linking the latest bizarre beauty trends in Japan to the desire to find a nice Jewish boy (See ‘Bagel head’: My theory if you’re interested in a bit more detail about my statistical endeavors).
Having failed to debunk the original fact, and having read it in the New York Times, I began to feel comfortable citing that 18% as an actual fact over the past year. So it was, perhaps in connection with JLSA’s upcoming mixer with APALSA (but not), I related that fact to a friend… and was again struck by the overwhelming impulse to debunk it. There’s just no way.
So I decided to go back to the source: New York Times. And so I discovered, simply “fact”-checking was the wrong approach all along. I needed to be source-checking.
The best statistical evidence for the trend appears in a research paper published in 2000 by Colleen Fong and Judy Yung. They found that more than 18 percent of marriages by Chinese- and Japanese-Americans were to American Jews — who constitute about 2 percent of the nation’s population.
The context — i.e. the source — is important, and ignoring it the first time around was obviously the completely wrong approach to fact-checking.
But do you know who else is wrong? The New York Times, for characterizing the cited paper as anything approaching statistical evidence that “more than 18 percent of marriages by Chinese- and Japanese-Americans were to American Jews.” Here’s what the paper actually studied:
A total of nineteen women and twenty-four men were interviewed for this study.
It gets better:
Most of the interviewees were born in the 1940s or 1950s, representing the generation most affected by the repeal of anti-miscegenation laws and the civil rights movement.
I don’t need to tell you why this doesn’t quite add up to statistical evidence for 18% across the United States:
Most interviewees live in the greater San Francisco Bay Area, where Chinese Americans and Japanese Americans have consistently resided since the beginning of their immigration in the mid nineteenth century.
But don’t worry, I can keep going:
Among the women are thirteen of Chinese descent (six foreign-born) and six of Japanese descent (all U.S. born). Among the men are thirteen of Chinese descent (six foreign-born) and eleven of Japanese descent (three foreign-born). All except one has attended college and most hold professional occupations, thus corresponding to the middle-class background of Asian Americans who have outmarried, according to statistical studies.
Seriously though, you get it, right?
Interviewees were obtained from four sources: 1) a snowball sample beginning with our Asian American acquaintances; 2) responses to one of several classified ads which appeared in Asian American and Chinese-language newspapers; 3) responses to one of several classified ads which appeared in a San Francisco Bay Area free weekly newspaper; and 4) responses to letters requesting an interview sent to interracial couples who had filed for marriage licenses at the Alameda County Courthouse in 1986.
Of course, I don’t blame Fong and Yung. They never intended to compile a statistical survey. They were merely interested in examining “factors involved in contemporary Asian-white heterosexual marriages.”
No, I blame the New York Times, which somehow managed to turn this
Eighteen percent of the Chinese and Japanese American women and men we interviewed were married to Jewish partners.
The best statistical evidence for the trend appears in a [study which found that] more than 18 percent of marriages by Chinese- and Japanese-Americans were to American Jews — who constitute about 2 percent of the nation’s population.
Though to be fair to the Times editors, they were probably ecstatic to have simply found anything that even smelled like evidence in connection with one of their trend pieces.