Debriefing the July 2011 California Bar Exam

The California State Bar has just released the results to the July 2011 California Bar Exam and I am again writing about how well one must due to pass.  Different bar review companies offer percentage estimates ranging from 65% to 70%.  I tabulated some statistics from the July 2011 California Bar Exam and reached different results.

For the uninitiated, the exam consists of three components over three days.  On the morning of the first and third day, exam takers attempt three essay questions and have three hours to complete the task.  These essays very widely in depth and breadth of subject matter, some involve a large number of issues that require cursory treatment, others involve a very specific issue that requires extensive treatment.  On the afternoon of the first and third days exam takers attempt the California Performance Test (CPT) which is a three-hour writing assignment.  Each essay is worth 100 points and each CPT is worth 200 points for a total raw score of 1000 points.  That score is scaled.  On the July 2011 exam, the range of possible scores was 40-100 and the average score (by my calculation) was a 62.1 which is two points higher than it was in on the February 2011 Exam.  The total number of points that could be earned was then scaled with the following formula:

Scaled written score = (Raw written score x 2.9096) – 341.9199

The scaled written score is then multiplied by .65 to get the total number of essay points toward the final score.  The second day of the Bar exam consists of 200 multiple choice questions on the Multistate Bar Examination (MBE).

My amazing wife passed the July 2011 exam (by how much one will never know) and she told me that she thought the MBE was really hard.  When I took my exams (Maryland 7/2010; California 2/2011), I thought the MBE was easy.  It turns out statistically she was correct.  Getting a 50th percentile score on this exam would have earned you substantially more points than earning a 50th percentile score on the February exam, even though there was only a difference of two scaling points between the tests.

The scaling is:

Scaled MBE Score = 3.5 * (40.4+ (Raw score x (.8 or .9)))

Notably, this MBE issue had a higher mean number of correct questions up to 122 from 127.  However, the scaling indicates that the exam was more difficult, with higher scores being earned for fewer correct responses.  As shown below, an exam taker could score an average of 70 on the essays (just 8 points above average) and pass the exam with only 64 MBE questions correctly answered. On the February 2011 exam, an exam taker with an average essay score of 70 (ten points above average) would have to answer 75 MBE questions correctly to pass.

Here is a chart that shows what I am talking about.  The green line represents the 5th percentile of exam takers, the red line represents the 95th percentile of exam takers.  The lines are substantially further apart than they were in February 2011, primarily because of a large number of exam takers with very high scores (some 570 people decimated the exam earning scores over 1600, whereas only a handful did that well in February 2011).  Sure some people did better or worse, but 90% of exam takers fall between those two lines.  The blue line is set of passing scores showing the average raw essay on the x-axis and the average raw MBE on the y-axis.  The objects are various bar exam companies' ideas of what passing scores are.  The horizontal line is the national average MBE raw score (127), and the vertical line is my guess as to the average essay score (62.1).  Unlike February 2011, the average exam taker passed, but not by much.

By my math if you scored a 62.5 as an average essay and 120 raw on the MBE, you would have passed the exam. Which is much less than scores bar review companies advertise.