People being imprisoned because they smoke marijuana is not among them. If the possession and use of marijuana was legalized tomorrow the impact on prison populations would be below neglible. (Many excellent reasons for legalizing the recreational use of marijuana exist but decreasing prison populations is not among them.) Furthermore, if we immediately released every convict whose only offense was selling small amounts of marijuana we'd reduce the prison population by far less than 1%.

  Getting that out of the way, we could begin to address real issues.  This begins with acknowledging that even imposing alternative sentences on  all non-violent first offenders standing alone won't help very much in the long run. Why? Because we have many violent offenders and we have many repeat offenders.

  The real questions are  how do we reduce the incidence of violence in our society and, probably most importantly, what can we do that will be effective to reduce recidivism after a first offense.

  One fallacy,  I believe, is the notion that eliminating drug prohibition will reduce crime dramatically because it will mean less laws for people to violate. I don't believe that is true.

  Many people sell illegal drugs quite simply because they want the money. If we eliminate that method of making money, they will still want money and many of them will not then choose to go get a legal job, but rather will choose other illegal methods of making money.

  Ultimately, the answers lie in somehow creating a society where far fewer people are willing to break laws to get money and also where the use of violence is less prevalent both to get money and to "solve" disputes between persons and groups.

  The whole idea that the "criminal justice system" is the locus where these problems will be solved is largely  erroneous. The criminal justice system basically operates where society has failed.