Thursday, April 7, 2016

The Incremental Cost of a High School Education in Kansas

A few years ago I was at a conference where a vendor presented a product for adults to get a high school diploma (not a GED).  The cost of the product was about $10,000, but this was *justified* because high school graduates make about $5,000 a year more, so the investment *obviously* pays off.   There are logical challenges to this argument (that will be dealt with in the second half of this piece), but the presentation of data caught my attention and raised some questions in my mind around the cost and benefits of a high school diploma.

Yesterday, the Kansas Center for Economic Growth (KCEG) released a study that again caught my attention relating to the value of a high school graduation.  The first part of the study is especially interesting because it relates the incurred additional costs of lack of education (what is the cost to social service system/prisons of failing to educate) and future salary costs of the individual.  KCEG is largely (as I see it) a center-left (for Kansas) think tank, so the piece provides great information in quantifying education value, but I think there are some more sophisticated and better ways to attack the problem.

BACKGROUND 

Two key findings of the KCEG piece caught my eye:
  • Kansas high school graduates earn far more than their peers who do not graduate ($5,400 annually)
  • High school graduates save the state money (by saving on social service, prisons, etc.) ($80,000 lifetime)
I don't question that these are factually true in direction, and in fact the direct causal links can be imagined (many jobs require a high school diploma, not having a job can lead to more social service cost).  But I do have a few areas of improvement and extension for the study:
  1. Bright-lining graduation may not be accurate.  Even dropouts get the benefit of often 10 or 11 years of public education, so comparing the benefits of having a high school diploma versus not is not comparing value of education versus no education.  Instead, it's simply  comparing education level 12 to education level 10. This creates bias in an ROI calculation, because drop outs would likely be much worse off if receiving no education.
  2. Public education isn't going away. The KCEG study is interesting in that alludes to education being beneficial then compares back to total education costs (state level).  The problem here is that public education is an institution that will continue to provide education (certainly education is generally beneficial), and the real policy question seems to be about how much we should fund public education.  If we're looking at the benefit of graduating high school as a metric, then our cost should be related to the incremental costs of making borderline students graduate, assuming future existence of public education as an institution at similar funding levels.
  3. The initial logic of the data is problematic.  The logic in the benefit studies takes the differentials in outcomes between graduates and non-graduates and states this as a result of graduating, which isn't quite true.  I expound upon this in the second part of this blog entry.

ADDING TO THE DIALOGUE

The initial work by KCEG points out that the value of a high school education is high, and I will go with that assumption (specifically the $80,000 future State cost) for the purposes of this section.  My two initial critiques of the work come down to incrementallity, or the concept that the differential in cost between graduates and non-graduates isn't the cost of a full 12 year education.  The KCEG study seems to look at the entire cost of education but only looks at benefits on the binary output of: did you finish the last couple of years (graduate)?

I will attempt to correct this basis difference by estimating the actual cost of an incremental graduate.  Because most dropouts go to school only a couple years fewer than graduates, it may seem like annualized costs for two years of school would be a good proxy.  This is false however because although the last two years of high school are how that degree is attained, it may be more costly to provide the supports/environment that will keep borderline dropout kids in school.  In essence the cost to create the behavior of staying in school may not be equivalent to the actual cost of two years of school.  With that background I had two additional research questions:
  1. Can we improve graduation rates by spending more money on education?
  2. How much is the cost of an incremental graduate (with other values staying the same)?
Whereas KCEG took an axe to the problem of cost, I looked to use a statistical scalpel to create very accurate estimates of the impact of spending more or less money. I realized I had some data on my desktop that relates spending, various school district attributes, and also contains graduation rates.  The data was substantial enough and contained enough covariates to estimate a spending:graduation elasticity, so I went to model building.

(NON NERDS CAN SKIP)
The model relates per-pupil spending, district sizes, fixed-effects for years, free lunch percent (a poverty-level stand in) to graduation rates per district.  Fairly simple model, and it ends up looking like this:

The model makes a priori sense, the two most interesting variables are free lunch (poverty) and spending per pupil. Poverty appears to play the biggest role in graduation rates, which matches empirical research.

I do have a methodological concern though: there is a high level of multi-colinearity between spending, district size (FTE), and drop out rate; making it difficult to ensure stable variance explanation.  A thick example is that we know that rural schools have higher graduation rates, partially due to demographic issues but that they also spend more money due to lack of economies of scale.  As an additional coefficient test I repeated the analysis using only districts above the 500 FTE mark, and 1200 FTE mark, to demonstrate that the coefficients are robust.


For all these tests my coefficient of interest stayed statistically significant and in the 2.0-3.0 range, as a central estimator I will use 2.5 which seems reasonable going forward.  But how much does that mean an incremental student costs?  Well a 1 point change in the natural logarithm of stude.... screw it, this is why I learned calculus.

A quick calculus solution is obvious, we'll assume we're holding all other variables constant so we will just throw them away as they would go to zero in the first derivative anyway.  Here's our solution:
Y = Graduation Rate
B0 = Constant (including all other resolved attributes in equation)
2.5 = Estimate of our coefficient of interest.
X = Per Pupil Spending
Our equation from the regression is thus:
Y = B0 + 2.5* ln(X)
First derivative:
dY = 2.5dX/X
Simple interpretation: the change in Y is 2.5 the change in X divided by X, OR the change in Y is 2.5 times the percent change in X.  Now for some some math.

Using a reasonable set of estimates we can calculate the cost of an incremental graduate to be $70,000.  This uses 30,000 as an estimate of future graduates per year, $6.1 billion in annual State + Local + Fed education spending (pulled from KPI website, if someone has a better source please let me know), and a current graduation rate of about 86%.

Before we go to extrapolation, there are some caveats, specifically assumptions one would have to make:

  1. Current relationships between spending and graduation would hold up when more money is applied.
  2. The current relationship isn't measuring other outside factors that impact graduation rates (communities that see more potential spend more and also expect more out of their kids in graduating).
  3. The kids who are currently not graduating in the difficult districts (KCK for instance) are *inside the model* like the ones we see graduating from better districts.

In other words: Does Levi think we could just add $70,000 for each kid that isn't graduating currently and defer KCEG's future estimate of $80,000 in costs?  Maybe, but it might fail in many regions as well, due to outside effects.  On the other hand, I think this is an interesting starting point to look at how money impacts incremental education outcomes.


ESTIMATING BENEFITS

The other issue with the KCEG study is the notion that benefits can be measured by the difference in outcomes. The problem here can be viewed as an issue of statistical sampling: the difference between high school graduates and non-high school graduates isn't simply a diploma, but there are non-random processes that push certain kids to get a degree or not a degree including talent, socioeconomic and structural factors.

The additional problem is that those outcomes are also correlated with outcomes beyond the degree, so that the numbers represented in the KCEG study don't only represent the difference between graduating and not-graduating, but also those pre-existing factors (in this way, graduation can be seen as intervening).  What evidence do we have that graduation tends to covary with other factors that impact outcome?  A few, but our regression from above demonstrates that substantially that graduation rates are correlated to poverty and district size (here likely a proxy for urbanity).  Here's how poverty relates to graduation:


What does this all mean?  That we know that graduation from high school is not a random process, and that the factors that impact graduation would also likely impact post-graduation outcomes, making estimates of the value of education somewhat biased.  Also, because we know other things impact both graduation rates and long-term outcomes, it may be more effective to attack those other causes than education itself (e.g.: attack childhood poverty rather than spend more on education).

Partial Conclusion: Further research needed, not from me.

CONCLUSION

All in all, I like the KCEG study, especially in its attempt to quantify the benefit to education. However, I have two major improvements that could be made to these numbers:
  • Quantifying the cost of incremental high school graduates. This post is an initial attempt at quantifying that cost, at around $70,000, which is under KCEG's impact to social service costs $80,000.  Potentially education is an effective way to decrease future government costs costs (though there is a time-value issue and a full distributional impact issue here for further analysis), but more work needs to be done to prove the cost-benefit.
  • Needs a better estimate of value of high school graduation.  This is because current graduation is not a result of random processes, but has many exogenous causes that will also impact downstream outcomes, and poverty play such a large role such that it may be more effective to solve poverty first.



1 comment:

  1. To be at the highest point of watchfulness will require (an) availability (b) obligation (C) watchfulness and (d) sharpness. We've to be prepared in light of the fact that none of us know the hour or the day that the Lord will come or when passing could take us away. http://www.mordocrosswords.com/2016/07/reckoning-via-sundial-for-example.html

    ReplyDelete