August 31, 2007

A Couple of Thoughts on Rick Sander's Study

by PG

I'm just looking at the background memo on the study that UCLA law professor Rick Sander wants to do on the effect of affirmative action on African Americans' bar passage rates. I think that the California bar ought either to do the analyses themselves or turn over the data under a confidentiality agreement, but I would expect Sander to be quite careful in how he makes comparisons.

Sander's "mismatch hypothesis" roughly states that of two black students with the same LSAT scores and undergraduate grades, the one who uses admissions preferences to go to an elite school is more likely to make lower grades and fail the bar than the one who attends a school that would have admitted him even without affirmative action. It makes sense that students who had the same admissions qualifications would do worse on an elite law school's grading curve than on a less elite school's, but I'm doubtful that this is as disastrous as Sander implies. A B average at a top five school is worth more in employment opportunities than a B+ average at a mid-level school; for one thing, the top schools impose grade-blind first round interviewing, so a candidate has the opportunity to charm and impress an interviewer before she sees his grades, whereas lower-ranked schools only allow their best students to see these interviewers. Therefore more worrisome than the stuff about grades would be the claims about black students at elite schools failing the bar exam at a higher proportion to their white classmates than black students at non-elite schools who had the same entering qualifications do in proportion to their white classmates.

1) "In one analysis, Sander found that blacks who passed up an elite school offer to attend a less elite school were half as likely to later fail the bar." I'd be curious as to how that analysis came out for the whites who made a similar choice, because elite law schools aren't necessarily better than less elite schools at preparing students for the bar exam. For example, the University of Southern California's law school, though ranked 16 by US News, has a California bar passage rate of 81.3%, less than 2% higher than the U of San Diego law school, which has a rate of 79.5% despite a ranking of 85. UC Davis, ranked 34, has a lower pass rate than USF, ranked 100. St. John's, ranked 70, has a NY rate of 88.8%; Columbia, ranked 5, is less than 2% higher at 90.6%.

And which bars are students choosing to take based on the schools they attend? Heck, if I have the choice between Stanford and Marquette, I can go to Stanford and take the notoriously difficult CA bar, or not have to take the bar at all and still be able to practice in Wisconsin (you didn't really think those 100% passage rates from UW-Madison and Marquette could be calculated exactly like other schools', did you?). Comparing different state bar exams to one another would make for some very sloppy conclusions, yet law students frequently look nationwide in making their choices about schools, bars and employment. Maybe UT's having a lower rate than Baylor can be explained by the greater likelihood of UT students to work outside Texas, thus drawing some of the better students out of the TX bar exam pool.

2) The concern that information on bar-takers' race comes from the bar-takers themselves, and may thus be
unreliable, is described as "probably the silliest of the four arguments. Essentially all racial data in the United States is self-reported. The bar has been using these self-reports for over twenty years in reporting the results of the bar exam by race." However, some data is required to be reported; for example, the Census is not optional. In contrast, while I haven't signed up for a bar exam, I have signed up for the MPRE, and its form asked if I was willing to have my information made available for research analysis. What percentage of test-takers refuse that request? And to what extent can we estimate the refusals' tracking with race -- especially with Caucasians?

August 31, 2007 07:15 PM | TrackBack

You keep nibbling away at a tiny corner of the big picture like a rat nibbling away at a block of cheese.

Are you short-sighted? Put your glasses on, step back (way, way back) and look at the big, big picture in front of you.

Then, post a blog supporting ANYONE (Sander et al., the Californian Bar, anyone) doing any serious investigation of the issue. Then keep quiet. Like I'm going to do. Right now.

Posted by: Timothy Don-Hugh Mak at September 2, 2007 07:07 AM

The above post: I think that the California bar ought either to do the analyses themselves or turn over the data under a confidentiality agreement

Mr. Mak's comment: post a blog supporting ANYONE (Sander et al., the Californian Bar, anyone) doing any serious investigation of the issue

Mr. Mak, if you aren't bothering to read the post, your comments are unlikely to be useful.

Posted by: PG at September 3, 2007 03:39 PM

I've come to the considered view you are beyond redemption. There are sparks of insight in your elegant (arrogant?) writing but you're so fixed, so fused in your fashionable pro-Obama left-liberal cultism you can't see the wood for the trees.

First you miss the subtle reference to the lovely film Ratatouille that just opened in Australia (you appear to lack the facile flexibility of mind to take the rat comparison as anything but an insult, rather than a jocular (if very veiled) film reference). Perhaps you're too "serious" to see a "cartoon"?

Second you have (again) completely missed the big picture. Let me try to paint it for you one more time (I'm a masochist, but not so masochistic I'm going to do this again. My head is starting to bleed from repeatedly smashing it against a brick wall of "intellectual dishonesty").

(1) Race-based decision-making is presumptively unconstitutional under the Equal Protection Clause. That much (I hope) we can agree on.
(2) AA should ONLY be implemented if there are (a) demonstrable benefits to those it is supposed to benefit and (b) no significant focused negative effects on any other group competing for the benefit. I can't believe you'll fight me on this one, but you've shocked me before.
(3) AA should be stopped IMMEDIATELY if there is ANY evidence that it is ACTIVELY detrimental to its intended beneficiaries. There should be a sense of URGENCY about this. Many AA minorities will be admitted this Fall, going into debt up to (and beyond) $160k, only to fail in their chosen profession BECAUSE of AA (if Sander's research is to be believed).
What should be happening now is an urgent study, leading to a complete cessation of all AA in public education until further conclusive studies can be completed. Only after it is PROVEN that AA genuinely benefits its ostensible beneficiaries should it be re-introduced. Instead of comprehending the sense of betrayal, the feelings of inadequacy, the financial worries of marginal students hoping for the best but being set up to fail, you blithely keep chewing at the corner of the big picture like Remy's brother Emile chewing at a block of cheese. No doubt it's fun for cashed-up well-connected students and academics to find tiny holes in Sander's proposed study (like a rat finding holes in Swiss cheese), but wait until the cheese turns rancid, and minorities find out they've been duped - for decades. Remy could smell rancid cheese from a mile off. Can you?

Posted by: Timothy Don-Hugh Mak at September 6, 2007 07:12 AM

Mr. Mak,

Why do you think I took the rat comparison as an insult? My sole response to your comment was to point out that in the same post on which you were commenting, I had done what your comment was urging me to do: support the CA bar and/or researchers seeking the bar's data, in doing analysis of the effect of AA on bar passage rates. That I wouldn't take your order that I "keep quiet," I would hope was predictable.

I understand what you and many others perceive to be the "big picture" -- that AA is unconstitutional. (Technically, a pure textualist reading of the Equal Protection Clause does not privilege race, because race is not mentioned at all, but as I am not a pure textualist, I am willing to roll with the current "strict scrutiny" standard.) The problem is that good research and statistics is not about the "big picture," it's about being very careful, very precise, and having as few holes as possible.

For example, your brief's analysis, perhaps because it was done while holding down a fulltime job and traveling in the outback, appears to have made statistical errors, especially in construing percentages. You may see this as a "tiny hole" in the "big picture," but if the analysis is to hold up under criticism -- particularly from competing experts in a sophisticated courtroom -- you cannot be so dismissive of such errors.

A constructive response to Sander's proposal is not merely supportive, but also highlights issues of which Sander should be aware (and quite likely is, even though it's not explicitly reflected in what he's posted). If you cannot accept that some of those who think the research should be done will be skeptical of assuming that the results will be bad for AA, then you will have difficulty building a coalition to force the bars and law schools to hand over their data. In particular, those underrepresented minority students who do feel that AA has assisted them in getting access without having been any detriment to their ability to pass the bar, will want to see stringent standards in the empirical evidence that counters their anecdotal experience, especially if such evidence will be used to end the program that helped them.

That is the big picture for those for whom the goal is honesty about the effect of AA, rather than a pre-determined goal of ending AA and only instrumental to that, finding some evidence against it in order to get back into court. However, even the instrumentalists should be wary of what shoddy analysis will do to their bid for Supreme Court reconsideration.

Posted by: PG at September 6, 2007 03:33 PM

Ok, agreed. Then let's have the data.

How about you and I both lobby the Californian Bar to disclose the information.

Are you willing to draft up a joint email to them? I'm happy to sign on to anything your beautifully, concisely written prose conceives.

I'm not holding my breath by the way, so don't be concerned - I'm not about to suffocate in my own delusional hope of anyone from the left lifting a finger to help get access to the data.

After all, altruism only goes so far.

Posted by: Timothy Don-Hugh Mak at September 9, 2007 10:59 PM

Mr. Mak,

My altruism will extend to a 39 cent stamp. I am doubtful that an email from me would do more than a blog post already has -- the former probably would get filtered as spam, anyway -- but I'm happy to write a letter if you can provide the relevant contact information (i.e. a mailing address that would allow the letter to reach the relevant subcommittee). Because I don't associate myself with your views on affirmative action nor on race generally, I will not have your signature with mine, but let me know if you want a copy.

Posted by: PG at September 9, 2007 11:37 PM

Gayle E. Murphy
Senior Executive, Admissions
Committee of Bar Examiners of the State of California
Office of Admissions
180 Howard Street
San Francisco, CA 94105-1639

By the way, just to correct you, I don't have any views on "race".

Because for me, it doesn't exist.

Posted by: Timothy Don-Hugh Mak at September 10, 2007 01:10 AM

Mr. Mak,

If race doesn't exist for you, I'm surprised that your brief cited Wikipedia's article on the subject. Oughtn't you have refused to acknowledge the existence of such a concept?

FYI, the body of the letter I am sending:

Re: Analysis of bar-score data

Gayle E. Murphy
Senior Executive, Admissions
Committee of Bar Examiners of the State of California
Office of Admissions
180 Howard Street
San Francisco, CA 94105-1639

Dear Ms. Murphy,

I am in my last semester at Columbia Law School, an institution that I hope has prepared me in the long run for what I anticipate to be a varied legal career, and in the short run for the New York bar exam, which I must pass in order to commence that career. Given Columbia graduates’ failure rate of nearly 10%, I will study for the bar with a healthy awareness that my passage is by no means guaranteed.

As a female student of color, product of Southern public schools, I consider myself to contribute to the diversity of my law school. I am a supporter of measures to ensure that a wide range of law school applicants have access to the schools that will best afford them opportunities to become leaders in their fields. Still, for any law school graduate, the bar exam stands as a potential barrier to such leadership. A graduate who never passes the bar exam never can become a law firm partner, a corporation’s general counsel, a state’s attorney general. Her hard work and talent are stymied.

If an unintended effect of affirmative action is to reduce the number of students from diverse backgrounds who can take advantage of the opportunities that affirmative action is meant to afford them, by reducing the number who pass the bar, then we all need to know that this is happening. I realize that the Committee of Bar Examiners has several concerns about what might occur if it analyzes bar-score data with an eye to test-takers’ race, law school grades and application package: breaches of educational privacy, misrepresentations of the results, accidental statistical errors that become the basis for overturning programs, and challenges to the legitimacy of the bar exam itself.

Nonetheless, a refusal to perform these analyses, or to let outside researchers perform them with the Bar Examiners’ data (subject, of course, to enforceable guarantees of confidentiality on individuals’ information), does not help anyone. Blinding ourselves to what may be unpleasant truths about the effect of affirmative action on some students improves no one's position, and in particular, does not improve the dignity of the profession as represented by the Committee of Bar Examiners. I urge you to reconsider the decision on whether to allow researchers access to the data, and hope that you will find a way to increase the sum of our knowledge on this important question.

Posted by: PG at September 10, 2007 02:03 AM

Beautifully, elegantly written.

I'm sure you'll pass the bar exam with flying colors.

Posted by: Timothy Don-Hugh Mak at September 11, 2007 11:58 PM
Post a comment

Remember personal info?

Sitting in Review
Armen (e-mail) #
PG (e-mail) #
Dave (e-mail) #
Craig (e-mail) #
About Us
Senior Status
Chris Geidner #
Jeremy Blachman #
Nick Morgan #
Wings & Vodka #
Recent Opinions
Persuasive Authority
De Novo Reporter

Powered by
Movable Type 3.2