A Public Defender Efficacy Study

The results of a study of Denver's felony criminal convictions have been published. Denver District Judge Morris Hoffman explains in a New York Times op-ed today, Free-Market Justice.

When two economists from Emory University, Paul Rubin and Joanna Shepherd, agreed last year to collaborate with me on an econometric study of how effective public defenders really are, I had to guard against confirmation bias. I was positive that public defenders would prove more effective than their private counterparts. Mr. Rubin and Ms. Shepherd, with their occupational faith in markets, were equally positive of just the opposite. In the end, the economists were right, though with an interesting twist. (The full study has been published in the Ohio State Journal of Criminal Law.)

....The results were surprising. The average sentence for clients of public defenders was almost three years longer than the average for clients of private lawyers.

In trying to understand the study, I found myself confused by the "variables" selected and omitted for the study -- and I'm still trying to figure out why the study concludes that those they chose measure "effectiveness."

The study excluded people who received probation.

The study also excluded acquittals at trial and considered only those who received prison sentences.

I would think that when a lawyer achieves a result of probation or acquittal for a client charged with a serious felony, that's an indicator of effectiveness.

The study apparently didn't consider death penalty cases and count as effective those public defenders who succeeded in getting a life verdict or less.

It also contains a suspect supposition: that defendants of low means are more apt to break the family piggy bank to hire private lawyers when they are both facing serious time and innocent.

I'm not a statistician, so I can't parse the variables for you, but I'm not putting much stock in this study.

Public defenders (and I have never been one) -- particularly in Denver -- are exceedingly well trained and provide excellent representation -- for lower pay and little glory. This study makes no sense to me.

On a related issue, the winners of the 2006 Public Defender Blog Awards have been announced.

Update: It appears I may have misinterpreted the New York Times op-ed. I thought when it stated acquittals and probation counted for zero, it meant for nothing, as in they didn't count. A commenter below points out zero meant zero days and they were counted in the equation.

< Not-So-Free Speech | New Saddam Hanging Video Shows His Neck Ripped Open >
  • The Online Magazine with Liberal coverage of crime-related political and injustice news

  • Contribute To TalkLeft

  • Display: Sort:
    There is a point to the study (5.00 / 2) (#3)
    by scribe on Mon Jan 08, 2007 at 11:44:22 AM EST
    and it's a shot at the existence of public defenders and at their independence.  And, for that matter, at the right to counsel.


    If, on the one hand, it's ecomonically inefficient for the defendants to have PDs, because they get worse deals for their clients when the charges are "minor", then the Rethugs' policy argument is:  "do away with the requirement for PDs in 'minor' cases, because the defendants get a better deal without them.  That "minor case" exception is the one which will swallow the whole.

    Similarly, prosecutors (and cops) can tell defendants "you don't really need a lawyer.  Here's this scientific study and it tells you you'll get a better deal with us than you will through that skanky PD.  So, deal with us."

    Moreover, if you look at the criminal justice system through the "administrative" rather than "trial" model - a distinction in which "administrative" minimizes the "jury trial" and "beyond a reasonable doubt" requirements and which Prof. Doug Berman over at Sentencing Law and Policy has explored over the years in the contexts of Blakely/Booker, sentencing-for-uncharged/unconvicted-crimes, and elsewhere - why have PD.  Economically, it reduces the system costs (dollars, cents and time) to not have PDs.  

    And, let's not forget the distinction in ineffective assistance cases, where denial of the defendant's private counsel of choice results in a new trial, while denial of the defendant's public defender of choice is harmless error at best.  Scalia made a big show talking about wanting the best lawyer money can buy, during recent oral argument in a case on that distinction, but the distinction remains.

    And, eliminating PDs would, of course, aid the administration of the Courts by acting as a "laxative on the docket" (h/t Grits for Breakfast, I think, for the phrase).  This surcease would not last long, though, as idle government employees would have their time sucked up by more cases, not fewer.

    Of course, this all comes down to the ultimate (what some would call) amorality of economics - it recognizes only costs and profits, and decrees the former must be eliminated and the latter maximized, and now.  If a dollar sign cannot be placed before it, it doesn't count.  And, sadly, one cannot place a dollar sign before liberty and all those other, quaint concepts written in the Constitution and given lip service by the Rethugs.

    And, similar to Andy Napolitano, I see a bright future at a right-wing think tank for the Judge sponsoring this survey.  He's just the character they'll want, and he'll be able to get away from all those nasty brown people in his courtroom....

    The findings were (none / 0) (#9)
    by Edger on Mon Jan 08, 2007 at 02:21:37 PM EST
    The average sentence for clients of public defenders was almost three years longer than the average for clients of private lawyers.

    Private laywers with self interest (fees) are working harder for their clients that the PD's?


    Public defenders with overwhelming caseloads (5.00 / 1) (#10)
    by scribe on Mon Jan 08, 2007 at 03:38:25 PM EST
    are doing the best they can.

    And, I don't trust things which might skew the nice, round numbers the survey came up with, like:
    (a) counting probation as a "zero" (the same as an acquittal), and
    (b) counting a life sentence as 110 years.

    Under that calculating basis, and even in a sample of 5,000 or so cases, a five life sentences for indigent defendants will outweigh half a thousand drug convictions with five-years' probation.

    The math?

    5 life sentences rep'd by PDs @ 110 years incarceration each = 550 years
    500 five-year probation rep'd by private attorneys @ zero years incarceration each = zero years

    No, this is a hit job on PDs - using junk science - masquerading as something judicially approved.  The more I think about it, the more I opine this judge should be turned in to the ethics authorities for lending the dignity of his office to a scam.


    Thanks scribe (none / 0) (#11)
    by Edger on Mon Jan 08, 2007 at 03:46:25 PM EST
    didn't smell right about Hoffman's statement, which is why I phrased that possible interpretation as a question - and I hope other people who read it won't interpret it that way, but I'm afraid that many might...

    correction (none / 0) (#12)
    by Edger on Mon Jan 08, 2007 at 03:47:08 PM EST
    something didn't smell right

    Interesting (none / 0) (#1)
    by syinco on Mon Jan 08, 2007 at 11:03:10 AM EST
    I'm looking forward to reading this later, in particular to look at how they attempted to set up a fair comparison.

    As you suggest, it seems particularly strange that they would exclude probation, and only somewhat less so to exclude acquittals.

    Thanks for posting.

    Link (none / 0) (#2)
    by syinco on Mon Jan 08, 2007 at 11:10:56 AM EST
    Here's a link to the actual paper.


    Probation & acquittals are taken into account (none / 0) (#4)
    by lewke on Mon Jan 08, 2007 at 12:03:16 PM EST
    The study excluded people who received probation.

    The study also excluded acquittals at trial and considered only those who received prison sentences.

    This is incorrect.  The study counted acquittals and probation as zero days in prison, since the length of the jail time is zero.  This is stated both in the article and the paper.  The paper also says that if someone gets jail time and probation, only the days from jail time is counted.

    also (none / 0) (#5)
    by lewke on Mon Jan 08, 2007 at 12:05:34 PM EST
    There were no death penalty cases in Denver in 2002.

    Making sure crime pays (none / 0) (#6)
    by atlanta lawyer on Mon Jan 08, 2007 at 01:13:12 PM EST
    I've always suspected that in drug cases, private clients are more likely to get probation offers from DA's.  I suspect they think, look, if this guys out there dealing drugs, gets locked up, gets a low/ROR bond from a magistrate, a free lawyer, and all he has to do is report to probation, and all the while he's making good $ selling drugs, what's the incentive to stop.  But if he had to throw down 5, 10, or $15K to a private lawyer, he'll do the cost benefit analysis and figure out that crime doesn't pay, or if not, at least the taxpayers didn't subsidize his business by paying for his lawyer.  

    I also suspect that there is a certain personal dynamic that goes on.  If your a private lawyer, you walk into court representing one client, you can argue, look, this guy is different from everyone else on the calendar and hears why.  And if you happen to know, or especially if you're friends with the DA (it's a small world afterall) then, consciously or not, there's a likelihood of a bit of a favor being done.   But even if the DA likes the PD, he/she can't do a favor to 80% of the cases on the calendar.  And as a PD, it's hard to argue "this guy deserves a break" without implying, "the other 10 clients I have with the same charges and history don't deserve a break."
    PD clients tend to get rubberstamp disposition offers.  It's easier for a private attorney to distinguish their client and to help craft a more individualized and rehabilitory (I may have made that word up) sentence.

    nice theory ... (none / 0) (#8)
    by Sailor on Mon Jan 08, 2007 at 02:11:58 PM EST
    ... but I think DAs are wrong if they feel that way. In my experience the folks who got lawyers had to keep selling to afford the fees.

    I think it's a good point about the "It's easier for a private attorney to distinguish their client "


    i keep looking at this, and (none / 0) (#7)
    by cpinva on Mon Jan 08, 2007 at 01:37:28 PM EST
    something just doesn't pass the "smell test". why do i get the impression, admittedly without doing a thorough analysis, that somehow this is skewed in the private counsel's favor?

    are PD's that much more inept than private counsel? maybe they get more guilty indigent clients?

    as i said, i haven't done a backwards analysis, but something just seems out of whack here. 36 months seems like an awful lot, for a true side-to-side comparison.

    not very convincing (none / 0) (#14)
    by syinco on Tue Jan 09, 2007 at 12:18:11 AM EST
    After reading this, it's not at all clear to me that the discrepancy they suggest is statistically significant (though like JM, I'm not a statistician).  By one measure, they cite a three year discrepancy (which does sound significant); by another, they find that, in terms of essentially percentage-wise sentence reduction, private counsel is 94% effective vs. 92% for PDs (which isn't clearly a significant difference to me).  

    Combine that with the questionable decision to account for probation/acquittal simply as zero as well as some other questionable assumptions they made and the fact that the study as written makes it difficult to decipher how/why they addressed variables as they did, and I don't know if we're looking at statistical noise or something meaningful.

    Regardless, while the "marginal indigence" theory has at least some intuitive appeal, it seems quite a stretch to suggest that this is a major factor in the discrepancy.  It seems worthy of mention as a possible contributor, but they give it surprising (and I think unsubstantiated) weight in the study as well as the op-ed.  

    Even if their analysis is correct and statistically meaningful, I did not read anything that convinced me that the discrepancy is not due to more conventional assumptions.  To go so far as to conclude the op-ed with the suggestion of tightening indigency parameters seems irresponsible.

    Flawed study (none / 0) (#15)
    by joeblow on Tue Jan 09, 2007 at 08:58:04 AM EST
    I'd say the study is highly inaccurate, and probably invalid, based on the absence of two very important sentencing factors.  Due to lack of information, the authors were forced to ignore the defendant's criminal history and custody status (did the defendant post bail/bond or were they incarcerated while awaiting trial). To their credit, the authors are willing to recognize the study's weaknesses, although they understate the importance of these two factors.

    The authors note, "It is important to recognize that all measures of effectiveness--even sentence outcomes--may have built-in biases that cut against public defenders, including the possibility that public defender clients may tend to be in custody rather than on bond (putting substantially more pressure on them to plea bargain) and that public defender clients may tend to have more prior felonies (subjecting them to greater penalties if convicted). Although our statistical techniques could in theory control for these variables, the data available to us did not allow us to do so."(227)

    These two factors, prior criminal history and whether the defendant is in or out of custody, are probably the two most important factors in explaining the different outcomes that result from the same charges (aside from the facts underlying the charges).  

    The fact that the researchers were unable to control for custody creates a structural bias within the study against PD's.  PD clients are, by definition, poor and unlikely to post bail/bond.  A defendant's incarceration acts as an institutional pressure upon the defendant themselves to seek a speedy disposition, often in-spite of a more favorable outcome at a later point in time.  From the in-custody defedant's point of view, it is a better choice to end the incarceration ASAP than to remain incarcerated and fight their case.  Since PD's represent a higher percentage of defendants who are incarcerated, the PD disposition statistics will reflect this institutional effect without controlling for it.

    The importance and effect of a defendant's prior criminal record to the disposition of their case should be obvious to everyone.   It's probably the most important sentencing factor. By failing to control for the defendant's criminal record, this study is highly inaccurate and therefore invalid. In addition, the lack of control for the custody status of defendants creates an anti-PD bias in the study.  IMHO, it's a worthless study.

    How did they deal with the practice of (none / 0) (#16)
    by JSN on Tue Jan 09, 2007 at 10:21:17 AM EST
    converting a parole/probation violation into a contempt charge and a returning the PD client to supervision instead of being their being sent to prison?

    The proof of such studies is independent replication. The problem is that such studies are expensive and the independent replication may differ from the original study in important ways.

    The discussion in one of the law blogs devoted considerable attention to the issue of true economic status of the client (were they really indigent). Caseload is also an important factor my wife and I have observed several counties where the case load was so high the PD client conference took place an hour before the hearing where the case was decided. Under such circumstances the PD is doing plea bargaining exclusively.

    re (none / 0) (#17)
    by Deconstructionist on Wed Jan 10, 2007 at 04:46:16 PM EST
      I've never been a public defender but have worked with and otherwise obseved countless numbers of them and privately retained counsel in many different jurisdictions over the years.

      Anectdotally, I would say that by far the worst representation comes from  certain privately retained lawyers. Incompetent or insufferably lazy folks do not long survive as PDs because their work is subject to monitoring and supervision. Any hack with a license can claim to be a criminal defense lawyer and criminal defendants are not always the most sophisticated and discerning consumers.

       That said, it is very difficult for PD offices to retain their best because of the nature of the work and compensation. One has to be truly committed to the cause to spend years and years in a PD's office. The disparity in starting salaries while large is something many will bear for a few years to gain invaluable experience,  but down the road people often have far more need for money due to raising a family and the disparity in compensation only grows with experience.

       I think there are more cases of outright ineffective assistance (in real terms not in Strickland terms) with private counsel but, I do also believe that, with a significant number of exceptions, the most talented and experienced folks are found in private practice.

      In sum, I think PDs mostly do a very good job and relatively rarely a inexcusably bad job, but I think the private bar has more people at the extremes --good and bad.


    hoffman's op-ed piece (none / 0) (#18)
    by mollybloom on Wed Jan 10, 2007 at 11:22:21 PM EST
    One conclusion to be drawn from Hoffman's study is that judges on the Denver District Court give harsher sentences to poor (i.e. represented by the public defender) defendants than to defendants who show up with retained counsel.  Another conclusion is that clients who are able to purchase mitigation (expensive drug/alcohol treatment; sympathetic evaluations from high-dollar mental health workers) fare better than clients who can't buy such evidence.  

    Hoffman himself believes the study proves that "marginally indigent defendants represented by the public defender tend to be guilty."  Based on this statement, a third conclusion to be drawn is that the sentencing disparity stems from judges who only extend the presumption of innocence to clients with enough resources to buy it, in the form of retained counsel.