March 16, 2004
The Grading of Young Lawyers
by Nick Morgan
Every 1-L quickly learns that when you first enter law school, there’s absolutely no escaping a great deluge of advice. In my first year I quickly noticed two recurring themes in the advice: (1) welcome to the place where grades mean everything—not “everything” to your overachieving undergrad ego, but “everything” as in your career, stature, intrinsic worth; and (2) don’t worry about grades; everyone gets a B, except that one guy who’s adversely possessed his own table in the library and that other guy who drinks beer in class. This is all to suggest that shared wisdom about grades appears to be ambivalent. And it turns out that a similar ambivalence can be found in the administration of grades.
Should it be obvious which of the two themes is the more sensible advice? Isn’t it true that, besides the LSAT, the GPA accounts for more filing of law students into the various strata of the legal profession than any other factor? For starters, it’s fairly obvious that the legal industry is largely a prestige-mongering culture. Consider clerkships and jobs at big, reputable firms—employers don’t even feign indifference to grades. Even graduates of the most prestigious law schools have plenty left to compete for because their targets are correspondingly elite. The U.S. Supreme Court, for instance, typically recruits clerks who graduate at or near the very top of their classes from a handful of the best law schools.
So if grades generally determine who sits where in the great legal meritocracy, they must actually mean something, right? At the extremes, they probably mean a lot. I, for one, don’t doubt that the person with the single highest GPA at my law school exceeds the rest of us in some combination of (a) astuteness of legal thinking, (b) willingness to put up with long hours of work, and (c) sleeping with his / her professors. The reverse is probably true for the student with the single lowest GPA. I am not, however, confident that such factors (well, the first two at least) reliably account for grade differences at the center of the curve. Somewhere between the curve’s center and its extremes, legitimate reasons for earning good grades become factors weak enough to raise concern. I suspect that law school grades—while certainly informative—do not adequately reflect the student qualities that drive employment decisions—qualities like work ethic, intelligence, and a generalized ability to succeed.
Although I’m amused to hear students claim that professors simply throw a pile of exams down a stairwell and give As to the ones that fall farthest, the myth of arbitrariness is surely exaggerated. A measure of arbitrariness will inevitably inform most judgments of merit to the extent that achievement requires no mastery of mathematics. Legal education, however, presents a special problem beyond the inevitably arbitrary, for reasons Professor Solum mentioned in this symposium:
[L]egal academics do not share a core set of disciplinary assumptions and research methods. Historians are trained to immerse themselves in the original materials—the archives, records, pamphlets, and diaries. Sociologists are trained to do both survey and experimental research. Economists are trained to use a set of mathematical techniques. But what do academic lawyers know how to do? What do law schools teach? … The answer to this question is up for grabs, in this post-formalist, post-realist, interdisciplinary era.
The obvious counterpart is that what students are expected to learn—and how they are expected to demonstrate learning—is up for grabs too. And that’s what makes being a law student wonderful, at times. I wouldn’t dare suggest that law schools normalize their pedagogy. Instead, law professors should more carefully examine the considerable challenges facing students who must not only learn the law, but learn each professor’s peculiar vision of the law, each professor’s unique and often unshared theoretical assumptions. Misunderstanding these assumptions could easily be the difference between an “A” and a “C”—a misunderstanding that may reflect a student’s mind-reading prowess, but little of the student’s legal potential.
Some, but not enough, law professors explicitly define their vision of the law to the classroom, and use substantial class time to address the nature of that vision itself, not only its application to case materials. The best law professor I have had took the extra step and explained what, according to his vision of the law, constituted meritorious legal thinking. I suspect that such an explanation is often lacking from classrooms where “there’s no right answer” is so commonly intoned.
There is no way to fix the inherent malleability of the law, or to eliminate the element of unavoidable arbitrariness in grading. But there is at least a partial solution, which requires putting students on explicit notice. While class time is a great opportunity for that, the exam should be the principle focus of efforts to explicitly communicate standards of merit. A question on my exam for torts—a class almost exclusively devoted to “policy arguments,”—asked us to “explain the justice” of spreading liability among the twelve negligent cartoons who blew each other up. The professor wants to hear “policy arguments,” I guessed, because “justice” is a concept of legal and moral theory. The grading key guessed otherwise: the professor was looking for ordinary, formal application of the Restatement.
Although I have no way of knowing how often exams are ambiguously drafted, anecdotes seem to abound. I doubt much good would come of advancing a new set of standards for writing exams, and I’m not sure that such would be pedagogically wise. What I’d like to see is a system—or at least a culture—of peer review over exam drafting. According to this proposal, professors would draft their exams, then hand them over to a couple of colleagues with similar specializations. The colleagues would take the exams as if they were students, and then have the drafting professor grade them. Discussion and revision would ensue, hopefully producing clearer, more correct, and fairer tests for students to take. It’s likely that some professors already put their exams through a similar process, but only widespread practice could hope to restore student confidence in the “justice” of grading.
A cursory review of any law school’s penal sanctions for cheating should convince you that the school’s administration takes grades very seriously. So do students, so do employers, and so, by proxy, do clients. Professors and administrators need to more carefully investigate failures of communication that cause avoidable unfairness in grading. They owe it to students, to employers, and to the integrity of the profession.
March 16, 2004 12:00 AM
Nick, I think the premise of your piece is right on but I do question your conclusion/solution. No doubt professors could, and should draft better exams (though I think professors would be better served by having them reviewed by professors not in that professor's specialization...I could elaborate but that would be an entire comment unto itself).
Upon reflection, I truly believe that the root of the problem is the lack of accountability that seemingly pervades the legal academy. That is, what is accomplished by testing a boatload of material in one sitting is absolutely beyond me. Does this approach really bear any rational relationship to real law practice? Do partners at firms throw their associates to the wolves and review their work 4 months later? Do law clerks only know if they competently drafting an opinion 1 day before the opinion is to be released?
Somewhere along the way, I believe that the notion of "thinking like a lawyer" gets lost in the attempt to just understand what the bejesus is going on in one's classes. Not that law students need to be spoon fed - but throughout the semester, there needs to be a few places where a student can put it all together (Thanksgiving or Easter break does not suffice). It's no wonder why many students' grades improve from first semester to second.
In sum, I think that we law students will be able to grasp legal concepts in a more meaningful way (and be a bit more motivated) if we are periodically held accountable for things. What some may criticize as paternalistic at first blush actually might be a better reflection of what the practice of law is really like, while at the same time, allowing students to actually think like lawyers.
Your point about grading is excellent. For the most part, law professors are not trained on how to give exams and grade them. The AALS Conference for New Law Teachers has a lecture on this topic, but that's about it. As a result, new law professors struggle a great deal trying to figure out a good match between a) the course they want to teach, b) the exam they want to give, and c) how to grade that exam. Good professors work incredibly hard at it, but we do so only because we care. For the most part, this sort of thing is under the radar; students are mostly obvlivious to these efforts, as are law school administrations.
Law school exams measure absolutely nothing about a law student except his/her ability to take a law school exam. If a lawyer ever has to take a law school exam in court to win a case, I will be sure to take whichever top graduate from any given year I can get my hands on to serve as co-counsel. Other than that, grades mean almost nothing to me if I am to try and judge the "abilities" of a lawyer for any given real-life purpose.
I'll be blunt: because our profession is stodgy and old fashioned by nature, it's behind the times. In no other academic field are students of our intelligence levels and proven abilities judged by such a foolish yardstick as the "end of semester exam" accounting for 100% of grades. Do I sound bitter? Well I shouldn't: I walked into law school exams in courses I had all but ignored throughout the entire semester, and walked out with the highest grade in the class. This means I beat others who spent much more time studying and learning the topic consistently throughout the semester. Is that a fair measure of our respective abilities? I think not.
Nick, I was that guy that drank beer and class and I still got a B. What the devil does that mean?
The thing is, what else if not exams? There must be some way of comparing students. Even if some consider exams not to be the 'right' way, others may find the 'new' way to be unfair for them as well.
visit my cool website :)
As to the question of "what else if not exams?", the answer is as simple as looking to other fields of study. The problem is not the use of exams per se, but the use of a single exam to judge an entire semester.
MBA professors would probably laugh at the notion of evaluating an MBA student on the basis of a single exam throughout the whole semester, for example. Most MBA students are graded on the basis of numerous projects, reports and case studies they complete, along with an occasional exam. The same is true of engineering students and even, to some degree, medical students.
The only reason law schools haven't caught up to these other fields is--and I cringe with anger in writing this--"tradition."
Well, tradition, along with the fact that I'm sure Law Professors aren't just filled with glee at the prospect of continual assignments & grading chores. I'm sure they're much happier grading a batch of exams at the end than forcing themselves to read a number of short briefs or memos or case studies throughout the length of the course.
Maybe I'm wrong, but I would suspect Professors in the other fields you mentioned probably resisted the change to more frequent grades.
British undergrad operates this same way--basically all grades determined by massive exams at the end. And British universities aren't really thought of as all that consumer-frienly. first off, they can't really charge anything. quite unlike America, where colleges charge a ton, and as a result, are fairly responsive to their students needs. Plus, there are a lot of very good schools (unlike Oxford/Cambridge). So, you see much more grading done throughout the semester.
Law Schools, however, are more like the Oxford/Cambridge scenario--not too responsive, and they know they're the only game in town. If you want to whine about Harvard/Stanford/Michigan/Chicago/Columbia etc bases your whole grade on just one exam, well, tough, thousands of folks would happily fill your place. That, unfortunately, is a big part of why I wouldn't expect to see much change.
Unless, of course, firms start deciding this method of grading doesn't appropriately approximate the quality of students and begin hiring from schools that adopt a different (more gradual) grading approach.
Our complaints really won't matter much I think...
My experience is generally consistent with anonymous lawprof's view that many professors do work rather hard to craft and grade exams fairly. So I don't think improving fairness in grading is a lost cause, despite some of the skepticism in this comment thread. Many of my professors took extraordinary measures to level the field--many of them agonize over the choice between a 3 hour exam or a take home, some of them offered the option of a graded midterm or graded paper, and some put their exams through countless revisions. So professor apathy wasn't really the focus of my article; rather, I think peer review could potentially be a very powerful tool. Even the most brilliant scholars in every field want a fresh pair of eyes to go over their scholarship. It's my contention that the clarity and precision of exams is just as important, and possibly more so, than scholarship, because grades have such a profound impact on the profession and the academy.
Peer review can be helpful, but only moderately so. The problem is that law professors are free to teach their classes in very individualistic (idiosyncratic?) ways. As a result, it can be hard to assess another professor's exam, even in a class that you also teach. Some classes are focused on doctrine, others on policy, others on theory; you need to have a pretty good idea about how the other professor teaches her class to offer helpful advice. And if the exam is for a class the reviewer doesn't teach, it's quite difficult for peer review to be helpful; for example, a professor of field A probably doesn't remember much about field B and can't tell if a question/hypo on a proposed exam is very good.
There are some things that peer review can catch; questions that are too long, poorly written sentences, unclear directions, etc. You don't need to be well-versed in the field (or the professor's individual view of the field) to notice these sorts of things. Peer review can be quite helpful for that.
A comment and a query (made by a law professor willing to use his name):
1. The "skill" of decifering the formal and informal clues that law professors give concerning their exam/grading "styles" is in some respects similar to the "skill" of decifering the formal and informal clues that judges give concerning their decision-making "styles." Though I do not mean to suggest success in the classroom necessarily means success in a courtroom, I do think both law school and legal practice consistently reward what I like to call "legal shrewdness."
2. I would love to give a lot more weight --- perhaps as much as 50% of the final grade or more --- to in-class performance throughout the semester. But I fear students would revolt if I put even more pressure on the daily classroom experience. Would such a change in grading approaches alleviate (or exacerbate) some of your concerns?
You raise a good point about legal shrewdness that law student Tony Rickey mentioned here. I've at times thought that picking up on such clues could reflect skills very like those that make a successful attorney, but I'm mostly worried about the tendency of some professors to equivocate about their vision of the law--although, again, I can't know how broad a problem this is.
As for graded participation: it would probably improve the quality of classtime by encouraging participation, but it may be quite a challenge to grade participation fairly. A grading key for an exam, at least, offers each student the same opportunity to score the same number of points, but grading participation by quality seems like a wide open field where intuitive judgments of "good participation" would prevail. On the other hand, professors could simply grade by the frequency of a student's participation, in which case the grade would reflect willingness to speak up but perhaps little of the student's legal potential. Perhaps there's a good way to do it that I'm overlooking, who knows?
Nick, I completely agree with your comments. They came at a needed time: when I was just devastated by my own grades. I have never experienced academic frustration before; I went to Stanford and then to a Top Ten law school on the East Coast. Aside from being appalled at my fellow law students' cutthroat competitiveness, willingness to blindly accept rather than question the law, and overcarrerist discussion of "networking ideas" during the FIRST WEEK of law school, I am DEVASTATED by my own grades. I may drink beer in class once in a while, but being that my tolerance is obscenely large, i still took lucid, obsessive notes on those few days and on every other... I still studied a great amount, believed I understood the material, and even had fellow students thinking I was going to be "one of those" who did well. Well, big surprise to see the C's that awaited me. After first semester, I sat down with my professors the way you're supposed to, talked to the academic advisors, etc. and got NOTHING out of it. The same cliched advice one of a million law books I've read have said: "Make up lots of hypotheticals." "IRAC, IRAC, IRAC". "take practice exams." "outline yourself." "find a good study group". I did all of this, and except for running out of time at the end of two exams, I assumed I at least got in the B range on my second semester exams (by now, that was the height of my expectations). Imagine my horror yesterday to discover that I had done WORSE and now it is likely I will not return next year, pending "special evaluation" by the dean. I believe that part of the reason exams are so puzzling and arbitrary is because the professor does not teach towards the exam. We are expected to use the material in a way that most of us have never done and many of us never learn to do correctly--aside from the few who have mastered professor mind reading as to the right way to "IRAC". I am one of the unlucky who fell behind the B majority; now I am left with a sense of failure and an uncertainty about what to do with the rest of my life. I still feel that I would have made a good lawyer; what I did not make was a good law student.
"Tradition" is really the enemy. As suggested by another commenter, why not use the methods that work in the other colleges? The reason you will hear is that "it has worked so far!"
What the present way "works" to produce is trial and error of how to survive to the end of law school. Well, I think we can all agree that we hoped for more out of law school.
Who knew that what is called the mastery of law is really getting a handle on the history of the law and how the elements of the law got that way. Supposedly, we should follow the reasoning of the court in our casebooks and do something similar on our exams using the law applicable to the given facts. The reasoning supposedly is accomplished by plugging into one-IRAC-fits-all formula while racing along in an exam.
"Survivors" survive by matching every fact to any law that applies. The top grade winners throw law at the facts and get Bs plus. Wow, what a mind.
This, you don't need casebooks to accomplish, just commercial outlines. You don't have to think, just rote listing of applicable law and reciting the facts. This is not learning the law, this is learning exam survival.
How do I know what students do to survive? I am a 3L and have run study groups from day one. We copy our exams, share with each other the prof's comments - these are either numbers indicating what you did wrong, e.g.
1. Incorrect law
2. Incomplete analysis
3. Irrelevant issue
4. and so on
or they are comments such as (actual comments):
-a check mark means it meets prof's standards
-a bracket with an undecipherable comment means something only that professor knows for sure.
-"you have no idea what this question was about."
-"I apologize to you for not motivating you to study harder."
-"No, we talked about this."
-"We had a case similar to this in class."
Any learning associated with one's performance is purely coincidental.
The latter comments reflect the prof's annoyance with your answers. I don't blame the prof. I would be annoyed too. By my count through school, ten percent of us "get" how to pass exams. Some of us really try to learn law and wander around in conflict wondering: is it best to memorize the cases? Better to memorize law? Better to practice entirely answering hypothetical fact patterns? We don’t know.
We had a professor who would base critical exam performance on knowledge of the contents of footnotes in cases so that, in his words, "you can't get that out of commercial outlines." High-school kind of vendetta tactics. This is graduate school!
The other disciplines have paid attention to what actually educates their students and trains them to practice what they have learned.
Med schools design their curricula and lesson plans based on conceptual understanding. Everything is demonstrated - the practicum is everything. Competence is based on competency, not an abstract, synthetic, cheap demonstration of throwing law at the facts. How long would we survive as lawyers by throwing law at the facts?
I reread my law exams and I am embarassed at my high grades for such simplistic answers.
Finally, to ensure that med students transition well into their profession, medical practitioners have taken responsibility for their med-students and have required paid internships paid residences and certify them as competent. The ABA has relegated that entire process to state bar exams - the same useless measure of the level of preparation for passing exams - certainly not a measure of the readiness to practice law. Every practicing lawyer will confirm there is no lawyer in the profession more useless and unskilled than a newly graduated and newly admitted attorney.
The bar exams try to weed out the ones "not quite prepared." In California, half of us fail every exam - more like defoliation than weeding. Does that mean that half of us were not educated and trained well enough? Of course not - it is a disgrace and abomination that half of us are allowed to go into debt over $100,000 and not be allowed to practice law; that the same government responsible for accrediting and certifying the institutions that teach us, fail and doom over 6000 people a year. We receive our JDs but are denied licenses.
Traditionalists will look through this comment for an alternative, but will accept no alternative. Any deviation from the norm has been considered suspect and “spoon feeding.” To those who support tradition as the defense for producing incompetent lawyers, I can only say that insisting on ineffective education and training because it is our heritage contributes neither to the profession nor to society. Your dues to the ABA would be better spent on ensuring lawyer competency and letting the existing state and national methods for sanctions and the market system weed out the ones without the right stuff. As was acknowledged by UCLA, testing is not the answer.