How Social Engineering (almost always) Fails

George Rebane

Big government’s (centrally controlled) social programs have a daunting history of failures that more often than not spill over to spread their hurt into other unintended areas.  An excellent example of this – 'The State Against Blacks' - is given by economist Walter Williams in the 22jan11 WSJ.  Today we have entered the epoch of large government programs that range from long-entrenched farm subsidies through various targeted welfare and ‘stimulus’ programs to nationalized healthcare.  In this piece I would like to give a scientist’s view of how to look at and think about such programs with the goal of understanding their large-scale operational characteristics.  And further, how eventual failure is built into the very structural make-up of every such program.

From the gitgo I have to declare that this will not be an easy read, and damn near impossible for the innumerate (see numeracy).  My apologies are tempered by the reality that EVERY significant social issue comes down to understanding ‘the numbers’ involved in its framing and implementation.  Here ‘the numbers’ actually refer to a broader toolset from the system sciences.  So let’s dive in.

We begin by positing that every tax funded, government designed and operated social program is designed to benefit a ‘deserving’ cohort of people, and exclude the ‘undeserving’ cohort.  Because it costs money to benefit everyone served by the social program, the government has to discriminate between the Deserving and the Undeserving, else the program will quickly spend more than budgeted, or be unsustainable and go broke.  Examples of such government programs abound.

Given this basis, the social program’s design must include operationally usable definitions of the Deserving with respect to the program’s goals, and the attendant presumption that if you are not of the Deserving, then you belong to the Undeserving.  The social program’s design also includes elements that define its operational structure including who will vet the Deserving from all the applicants, and how the program’s benefits will be delivered to the successful applicants – we’ll call them the program’s Recipients and the unsuccessful applicants will be labeled the Rejects.  All of these considerations will make up the program’s policy, which includes what we may view as an operational threshold comprised of multiple explicitly defined factors used to cull the applicants into the Recipients and the Rejects.

And here comes the realworld rub.  Almost all such human-devised programs (they can be formally viewed as systems) are prone to imperfections and errors in their design, construction, and application.  We will here focus on two types of errors arising from the operation of the program that will ultimately be reflected in the program’s cost and, therefore, feasibility/sustainability.  The first type of error – let’s call them Type1 errors - is that in vetting the applicants we will refuse some Deserving people and assign them to the Rejects.  The second type of error – calling them Type2 errors – will cause the acceptance of some Undeserving applicants and assign them to join the Recipients.

Now when we consider these errors in terms of the program’s admission policy, we intuitively grasp that if we make that policy too tight, we will reject too many of the Deserving which translates into having an excessive Type1 error rate.  In response to this we can loosen the acceptance threshold and immediately see a drop in Type1 errors as we accept more Deserving people into the program as Recipients of its benefits.  The unfortunate counterpart to loosening acceptance policy is that we begin to accept some of the Undeserving into the ranks of the Recipients, in other words our Type2 error rate begins to rise.

The scheme of things should now start becoming clear.  The operation of realworld programs (systems) is almost always so that as you loosen your admission threshold to minimize Type1 errors (reduce rejection of the Deserving), the inevitable result is that you increase Type2 errors by letting in more and more of the Undeserving.  Only fools and politicians will argue at the onset of the next social program that such a situation will be eliminated by all manner of hokey prudence.  The interplay of Type1 (green) and Type2 (blue) errors is illustrated in Figure 1.

 

[At this point the technically astute reader will already recognize the direction of this explication as it parallels the technical developments in signal processing and generalized experiment design.  Yes, we are heading toward the beloved ROC Curve.]

Looking at Figure 2, we can draw the total Recipients population curve consisting of the Deserving and the Undeserving, again as a function of the program’s admission policy.  At any given time the total Deserving population is usually a (small?) fraction of the country’s total population or a sub-population (e.g. people of African descent).  So multiplying such a population by the complement of the Type1 error rate (i.e. one minus Type1 fractional error) lets us plot the number of Deserving Recipients in the program as a function of how tight or loose we make our admission threshold.  This is shown as the green curve in Figure 2.

But we know from Figure 1 that at any given acceptable admission threshold we will also have a positive Type2 error rate.  Multiplying this error rate by the total population of the Undeserving will give us the number of these people included in the program Recipients.  This is shown as the blue curve in Figure 2.  Adding the Deserving and Undeserving Recipients will give us the total number of Recipients in our program who receive benefits.  Depending on how loose the admission policy threshold is made, the total number of Recipients can approach the total population of the country.

The problem now begins to reveal itself as we consider the size of the Undeserving Recipient cohort.  Because the total population of the Undeserving is usually very large, even a very low Type2 error rate will admit a surprisingly large number of such recipients into the program.  And we always have to keep in mind, that attempting to tighten up on admissions (e.g. the popular ‘eliminating fraud and corruption’ argument) will inevitably increase Type1 errors and exclude some of the people we want to receive benefits.

All of this is comes down to dollars and cents, and its effect on program cost is shown in Figure 3, again plotted against the now familiar admissions policy axis.  The vertical axis may be scaled in either total program cost or its cost rate (say, dollars per year) of the social program.  The latter being important when the program competes annually with other programs for a share of some overarching budget.  But the real takeaway should be understanding how the attempt to benefit all the Deserving becomes impossibly (or is it outrageously?) expensive as we try to capture that last percent or two.  Somewhere the line must be draw no matter the tears.

In sum, when we line up the different curves of Figures 1, 2, and 3, we see how the real world of social programs operates.  And all this should be understood before we begin to consider how humans game all systems, where the tragedies of the commons arise, how such programs are constructed and must needs be operated by government workers who definitely are not among the brightest bulbs on the tree.

Before concluding, I have to point out that the actual error curves of Figure 1 can come in various shapes and forms.  Most certainly they are never nicely symmetric as I have shown here for illustrative purposes.  More than likely they are as shown in Figure 4 which emphasize the different rates at which Type1 and Type2 errors vary as the admissions threshold is changed.  The one thing you should never plan on is that the error curves will look like the ones in the lower right of Figure 4.  Here it is easy to devise an admissions threshold in the ‘valley’ where both error types are zero – a perfect piece of social engineering.  People who attempt to convince you that their social program will have such a valley, and admit only the Deserving while reliably rejecting the Undeserving, are either charlatans, liars, thieves, or simply the well-meaning ignorant.  None of them belong in public service.

Finally, we can combine the Type1 and Type2 errors for a social program into a single curve that represents how efficiently it will trade off serving the Deserving at the expense of also serving the Undeserving.  The combined curve shown in Figure 5 is the analog of the Receiver Operating Characteristic (ROC) curve devised in the early days of radio.  To fit within our context, I will rename it the Social Program Effectiveness Tradeoff (SPET) curve.  Over a spectrum of defined admissions policies each program will yield the appropriate error curves introduced in Figure 1.  Than as seen in Figure 5, the SPET curve plots the fraction of Deserving (1 – Type1) against the fraction Undeserving (Type2) that the program will admit.  The SPET curve is generated from the Figure 1 error curves by plotting their indicated values as the admissions threshold is swept from the tightest to the loosest policy.

Each social program will intrinsically operate within the Type1 and Type2 errors shown in Figure 1 and combined into its SPET curve as shown Figure 5.  All that program management can do then is pick an ‘operating point’ for the program in terms of its admissions policy.  That selects a point on its SPET curve and everyone is off to the races.  It takes some work, but these error curves can indeed be estimated as the social program is designed, and the information on its admissions and cost tradeoffs made as I have described above.  The embarrassing question at this point is, how many programs receive such consideration before Congress and then some bureaucracy launches into the next spending spree costing hundreds of billions of dollars.  Does anyone think that this was done before Obamacare was committed into thousands of pages of unread and still not understood legislation?

As voters we will continue to be subjected to hyper-ventilated arguments to move the admissions threshold this way or that by the citing of anecdotal and heartrending evidence of the current policy rejecting some pitiful Deserving person(s).  And the obverse citations of how some callous and greedy Undeserving have been receiving benefits, therefore advising that admissions should be made stricter.  As Walter Williams says in the above linked article, “Politicians exploit economic illiteracy.”

Considering the above, we should now be able to understand the factors that underlie all such social programs and demand that we receive usable information (fixed and variable costs, size of populations, admissions criteria, error rates, success metric of similar programs, …) from our politicians and bureaucrats before deciding whether to open or close our wallets to their always compelling entreaties.  The alternative is to continue as we have been, leaving reason by the wayside.

Comments

[Mikey McD]

Kudos to the welfare sales team; job security requires State agencies to push their products... I recall a representative going 'door to door' in the maternity ward (labor and delivery) signing up new mothers for "WIC" (Women Infant Child) benefits.

The WIC 'saleswomen' cited that it would be 6 months, if ever, before the mother would be asked to qualify for the then already received benefits via proving low income status (never mind the fact that our income was probably higher than that of the delivering Doctor).

The State had sent the diaper, formula, Cheerio fairy to help; 'for free.' I recall the debate with my wife (insert "Tragedy of the Commons" discussion here) it went something like this... 'everyone' else is signing up, we should get something for all the State taxes we pay...

[Todd Juvinall]

Also, wasn't the welfare department advertising for people not long after the reform was signed in the 90's. They were lonely I guess.

[George Rebane]

One could not have asked for a more timely illustration of how social programs are managed into failure than the explication given in today’s (24jan11) leading WSJ editorial.

http://online.wsj.com/article/SB10001424052748704881304576094132896862582.html?mod=WSJ_Opinion_AboveLEFTTop

Since setting the operational threshold, as described in the post, is a subjective policy decision based on many often fuzzy criteria, the editorial points out the specific nature of the follies that will now attend Obama’s latest lies about effective regulatory reform. Somehow, we expected nothing less.

[Dixie Redfearn]

Any way to dumb that down to a sentence or 2?

[D. King]

Oh George!
Leave it to you to explain the unexplainable (with graphs). A system that is doomed to failure from its inception begs the question; why? Is it a game played by a cruel elite or the realized musings of the insane?

Maybe it’s a need for this:

http://tinyurl.com/4q49pgs

[George Rebane]

OK Dixie. You can never afford to help ALL the Deserving, and attempting that will cause you to go broke so that in the end you CANNOT help ANY of the Deserving. Given any level of funding, you have to make a deliberate, subjective, and cold-hearted choice of what fraction of the Deserving you are able to help. Without understanding the kinds of errors your admissions policy has built in, and the effects of such errors (as explained in this piece), you cannot select the proper admissions threshold or cutoff point that will give you a sustainable program based on reason.

Instead, the politicians/bureaucrats will continue stabbing in the dark using politically correct/acceptable feel-good criteria that give rise to the kinds of social programs foisted on us over the years. (Obamacare is turning out to be the poster child of such a program.) Because the voters are ignorant of all this, they keep electing the same sleazebags and/or idiots who promise anything to get (re)elected. The result is the unfolding economic catastrophe we have today. Jefferson predicted the endpoint of all this – ‘A nation ignorant and free, that never was and never shall be.’

[D. King]

The true shame is that organizations like the Salvation Army and Catholic hospitals are vilified.

[Barry Pruett]

Thanks George! The theoretical aspect makes a lot of sense...it would be nice if we had the data available for computation. Do you think that it is possible to accurately predict the error rates though? That seems like a tough task given the information provided by the recepient is the only information upon which to base the computations.

[George Rebane]

It is indeed possible to measure the Type1 and Type2 error rates to arbitrarily small uncertainty intervals by running a test of the admissions policy, and then expending resources (beyond program norms) to investigate and confirm the assessments of Deserving and Undeserving made within the tested admissions policy. Such procedures are well-known to professionals in the field. I suspect that this is not done because the answers may embarrass the program designers and its political supporters/sponsors. Bamboozling the taxpayers is much easier and cheaper.

[Aaron Klein]

In regards to the innnumerate, I've heard it said that 4 out of 3 people have difficulty with fractions, so that could be a larger part of your readership than you might think... ;)

[George Rebane]

Friends, like yourself, have advised me of that. According to the late Steve Allen, one of the reasons for our country's rampant dumbth is that we have been taught by public education's purveyors of self-esteem that the student is never responsible for not understanding the lesson - it is always the teacher's fault if learning does not take place. Coming from a culture that rails against that notion, I therefore see RR readers in a much brighter light.

[Greg Goodknight]

Part of the problem is that far too many of the teachers didn't understand their lessons, either. One Federal study released in the last decade found after closely following one cohort that the lower a student's SAT when entering college, the higher the probability they were teaching K-12 ten years after receiving their baccalaureate.

As a result of the whole language, whole math choices of the Grass Valley School District, we got chased out of our local public schools and into Mt. Saint Mary's, our local St. Sensible. There were a couple turkey teachers there, but most were competent, and they had a sensible and competent curriculum. About that time I became embroiled in the Math Wars, as the struggle over whole or "fuzzy" math became called. One of the professors on my side of the math wars was an avowed communist in one of the CSU's, who once revealed that the prospective teachers at his CSU entered his classes with a 4th grade understanding of mathematics, and the goal was to get them to the 7th grade level. IIRC the Education Department on that campus got so fed up with so many F's being awarded by the Math faculty that they created their own math class where the boneheads could succeed, at least as far as the letter grade was concerned.

The far left prof eventually did a few things with Lynn Cheney towards the promotion of rational mathematics curriculums, showing that bad math does make for strange bedfellows.

The GVSD still does a poor job teaching math, but after they crashed and burned with half the kids greeted with "Mathland" in the first grade being in the bottom quartile when tested at the end of the 3rd grade circa '98, they floundered a bit spent a few years trying to patch it up and then bought the text that I tried to get them to accept a classroom's worth of free books when my kid was still there. All Saxon Publishing wanted was a standardized test when they started and again after they finished, but Linda Brown, GVSD Ass't Superintendent and co-Principal of Hennessey, was repulsed by the idea of a mere "drill and kill" text, which is how educationists describe arithmetic practice. Their detritus, the kids crippled by the experiment, are almost finished making their way through the local schools but the kids that were in the 1st grade when it started are now graduating from college, if they made it there.

[George Rebane]

Thanks for that great explication Greg. I have it from a reliable source that math at NUHS is woefully lacking. I see echoes of that in results of the TechTest merit scholarship exam that I'm involved with.

[Greg Goodknight]

While the math department has probably strengthened since one particular mean spirited incompetent chair left under a cloud about 5 years ago, don't blame the high school too much; if kids don't have a solid foundation in arithmetic before they begin actual study of Algebra (8th grade by the excellent California standards that may be on their way out), there isn't much chance of succeeding. If you don't understand fundamental arithmetic, you don't have much of a chance of understanding a generalization of fundamental arithmetic.

The GVSD under Byerrum and Brown did very poorly; I was told by an insider that the NUHS Geometry X entry exam was put in place specifically to nudge most Lyman Gilmore grads away from the honors class they were not prepared for.

Doing a little googling, I see that NC Supt of Schools Holly Hermansen is the wife of retired GVSD wreaker John Byerrum; funny I never noticed that. Kind of puts into perspective Hermansen's rejection of my FOIA to her office for documents relating to the secret donor of $50K (in McAteer's days) to the county to promote the International Baccalaureate in the GVSD and NUHSD.

I understand the IB is still on the backburner.

[George Rebane]

Agreed, and thanks for the insight into that world Greg.