Look I’m from New Zealand – so when everyone around me started talking about ATARs I just smiled and nodded.

In fact I probably couldn’t talk to most people in New Zealand about education. I’m from the “pre-NCEA” era – where a single end of year exam for five courses, scaled to fit a within-course normal distribution, determined most of anything. As a result, these more complex design criteria are well outside of my lived experience.

But it turns out ATARs are a very important part of an individuals assessment in Australia – providing a measure of how well a student performed relative to their peers, and determining their university admission.

This raises a question – how is a good performance on your ATARs associated with future earnings? Luckily for us Elyse Dwyer and Silvia Griselda at e61 decided to find out.

tl;dr a higher ATAR is associated with higher average earnings – but there is significant variability in income by ATAR. As a result, even though we’d expect the type of person who receives a higher ATAR to end up with higher earnings at 30, there is a lot more going on under the hood.

ATARs and earnings

The earning profiles of individuals between 19 and 33 are shown below, with each ATAR bucket also conditional on the individual going to university (well except the No Uni one). Lets chat about them

As we can see, the earnings of those who do not go to university is higher between the ages of 19 and 23. Anyone else who went to university when most of their high school mates went to work can attest to this truth – at this stage people love to go on about how they are not earning cash while your still at school.

Nonetheless, continuing education is a form of investment – so you should expect a payoff in future earnings. And that is what you see, all the other lines move above the no university line, showing that the median individual who went to university did end up earning more than the median individual that didn’t – irrespective of their ATAR.

Then it comes time to compare all the other lines. As we can see, the median individual with a higher ATAR earns more as well. And all individuals that used their ATAR for university entry earn more than those who entered through alternative pathways. So what does this tell us?

1. Individuals with higher ATARs may have characteristics that lead to higher income – harder work ethos, greater capability, better family and friend networks.
2. Higher ATARs may allow individuals to select into higher value educational opportunities.
3. There may be a direct return from having a high ATAR on your CV above and beyond everything else.

The note digs into a few more mechanisms, the main thing I want to note is that no-one is saying the higher ATAR itself causes higher earnings – just that there is an association worth further investigation.

And figure 2 shows that these things get more complex – there is significant dispersion in the earnings of individuals at 30 on the basis of their ATAR.

For a specific example, someone in the bottom 25% of earnings for the top ATAR bucket (the 25th percentile) earns less than someone in the 25% of earnings who didn’t go to university. And they earn about the same as the median “below 70” ATAR individual who did go to university!

I remember that I had friends from high school who went straight into work – and now own and run the business they were working at. I had friends who were clued on whose grades ended up a bit below mine, who went to university and earn significantly more than I ever will.

Unlike how high schools make us feel, that final grade doesn’t determine our full life path and opportunities – it influences it, but even if high school went poorly there are opportunities out there.

Not just that – never confuse a grade or your income with your self worth. You matter just as much as someone everyone calls smart, or someone who seems to have limitless resources – and the person who you view as unsuccessful matters just as much as well. Your power comes from being able to make your own choices, and own the outcomes – so make these choices with kindness.

ATARs and the SAT debate

Can this information about ATARs tell us anything about the standardised testing debate in the US at the moment? Probably not, but its worth thinking about.

The ATAR takes an average of course performance – like a GPA does. However, it scales the scores based on measures of how difficult that subject is – which is more similar to the weighting used for an SAT.

Concerns about implicit discrimination and the interuptions from COVID led to a large number of US univeristies dropping SAT requirements. However, the NY Times has come out stating that – relative to other metrics universities can use – standardised testing does appear discriminatory. Specifically, the advantage of advantaged students is starker when comparing GPAs to SATs.

To understand this issue we’d want to think through two issues:

1. Is it important: What is the return associated with gaining university entrance.
2. What drives differential selection by group: What is the reason why individuals from certain backgrounds are not being selected into study – and does that choice influence the benefit or opportunity cost of such study.

To understand the importance of univeristy entrance we’d want some idea of what would have happened to people who gained entrance if they didn’t – or people who didn’t gain entrance if they did.

To understand selection we would want to look into not just university selection criteria, but the nature of applications and the decision to study. If individuals from disadvantaged backgrounds are not studying because of other family responsibilities, then the policy choice of “removing SATs” isn’t really dealing with the lack of opportunity faced by that person – it is instead a lack of material resources to be able to make worthwhile investments in their human capital.

I think the e61 note informs this by showing just how heterogeneous these returns can be – hinting that such investments in the self may be very uncertain. SAT bans don’t do anything about that, and if they do restrict access to scholarships (say due to a shift to sport and GPA based support) they may make access worse for truly disadvantaged individuals.

So I guess the work did provide some insights to this debate after all!

Why do I say “may”, well this depends on the cause doesn’t it – why does this inequality exist.

Now I don’t disagree that educational attainment is associated with income inequality, and the narrowing of gaps in educational attainment has been associated with lower income inequality.  I also don’t disagree that, looking at a person in isolation, lower mobility can be associated with lower opportunity.

But I am from a low income area and I am not sure we can blame universities for the fact that they primarily have students from higher deciles.

When I was growing up I was repeatedly told university was a waste of time, and to look for a apprenticeship or undertake some type of vocational training.  I was told that you only went to university if you already knew people, and that I would end up with no money and lots of debt.  Attitudes may have changed in my hometown at this point, but I have heard similar stories from students who have come from small towns in recent years.

My unpopular opinion is that university entrance is not a particularly high bar for anyone with the capability to do the “highly remunerated” work mentioned in the NZ Herald article.  If people from poor communities are not undertaking university education it is likely because those schools, those communities, and those families, do not believe that university is a worthwhile endeavour for them.  The “long read” gives an example of this right up front, but to me that shows that the issue is almost entirely not about universities – and instead about varying communities attitude to undertaking higher education.

If opportunity is there and there is a failure it could be due to these incorrect expectations.  But there may be other systemic issues that lead to less education by these groups.  These are complex but we can have the following failures that we may view as relevant:

1. Peer-group effects:  Wanting to do the same thing as your friend.
2. Family effects:  Undertaking the same thing as your parents.
3. Low income insecurity:  Being unwilling to take on debt because you view your life through the lens of low income, and have a different perception of risk associated with the student loan.
4. Low income family effects:  Being unable to undertake investment in human capital as you are required to help within your family due to limited income – familial bonds are important.
5. Cultural effects:  A community, or group, that identifies themselves either without reference to university education – or against it.

If that is a failure then we need to address it.  If it is about expectations then we need to make the case to these communities and these groups that their children can succeed in such an environment.  If it is one of the others then we need to consider how they are relevant and how to make sure there is a functional opportunity to get education.

Finally, we should also make sure we aren’t contradicting ourselves when we talk about how important vocational work is while also saying people should go to university. If vocational jobs are in such high demand people from low decile areas may be making good choices rather than simply chasing what they see as “status seeking” education …

"We then narrow our focus to black families in the South, where state-wide minimum teacher salary laws created sharp differences in teacher wages between adjacent counties. These differences had large impacts on schooling attainment"https://t.co/WOnenclZxu

— Noah Smith (@Noahpinion) September 12, 2018

Interesting.  Education, and schooling attainment, seems to be a big issue over here in NZ as well – with a political consensus that we care about this issue, but conflict about “how” school quality relates to this (or is even measured).  So lets have a chat.

Take schooling attainment as a measure of human capital, improved attainment suggests greater human capital attainment and income, and inequality in this measure as a measure of inequality in human capital  – when thinking about peoples opportunities, these types of inequalities are of pretty significant interest.  What role does school quality, or characteristics that are related to school quality, have on these schooling attainment outcomes.

In this paper the “school quality” variable is a function of two things: the teacher to pupil ratio and a measure of the average level of teacher salaries.  Fewer pupils per teacher may improve student outcomes as they get more teacher time, higher teacher wages may improve outcomes by attracting more qualified teachers or increasing the effort of the marginal teacher – hey it might not do these things, but the goal here is to consider what the data is saying first

The three effects in the paper from considering this school quality variable to analyse educational outcomes?  Well copy and pasting from the paper we have:

1. School quality is strongly related to schooling attainment for most parental education groups, with effects that are largest for children with the least educated parents and smallest for those with the most educated parents. The general pattern suggests that higher school quality contributes to a closing of between-family gaps in human capital.
2. Estimated effects of the individual school quality measures are only slightly attenuated when we fit a model that includes both (teacher to pupil ratios (PT) and average salaries (W)) reflecting the limited correlation between PT and W across states.
3. Estimates of teacher effects are little affected by the addition of several state-level controls—the average level of education of whites aged 25–55, the state-level white male unemployment rate (among those aged 16 and older), and the mean value of homes in the state. However, the addition of these covariates leads to greater attenuation in the estimated coefficients of the pupil-teacher ratio. We conclude that the effects of teacher wages are reasonably robust to other controls, whereas the pupil-teacher effects are more sensitive, and are likely overstated in the simplest models.

They also concentrate on the role of teacher wages specifically by looking at “cross-border” differences in schooling attainment, when states on each side of this border will have had different minimum teacher salary laws.  If this type of analysis to tease out causal effects sounds familiar it should – but this paper doesn’t seem to take a change in the minimum in two otherwise comparable county areas of different states, rather the fact that states have a varying minimum, so I don’t think we can really rule out that the measure could still be a proxy for other differences due to state policies [in other words it isn’t a difference-in-difference study, it is an IV model of differences between the country pairs].  They summarise:

In summary, our analysis shows that policies that increased teacher salaries substantially improved educational attainment among black children in the South in 1940. We
believe additional research will be useful to help fully develop our understanding of the mechanisms whereby these policies affected educational outcomes. One important possibility is that the higher salaries for black teachers in states with high minimum salaries led to a better-educated teacher workforce.

At face value this looks like a cool study that indicates higher teacher pay was associated with better educational outcomes, especially among disadvantaged groups.   Thoughts?

She’s partly right. She’s right that independently schooled pupils perform better at A-levels, on average. It’s also true that Oxford and Cambridge have very high entry standards, which favours independently-schooled pupils. However, that does not fully explain the low rate of admission for state-schooled pupils.

It’s fairly easy to check because the percentage of state-schooled pupils admitted is one of the Performance Indicators published by HESA each year for all universities. The indicators helpfully include a benchmark that accounts for, among other things, the entry qualifications of students. That means we can compare the actual state-schooled intake for each university against a benchmark that takes Ms Hartley-Brewer’s concerns into account, along with other considerations, such as age, ethnicity and sex.

The chart below shows that we would not expect Oxford and Cambridge to take a high proportion of state-schooled pupils, largely because of their high entry requirements. However, even against that adjusted benchmark, they underperform.

This leads the SMF to express concern that school choice may not lift educational achievement because some parents do not consider it important. They then recommend Government intervention to promote the primacy of academic success. The line they’re treading between free choice and paternalism is a fine one. One the one hand, they want free school choice to improve the quality of schooling. On the other hand, they have a prescriptive view of what school quality means.

It is a difficult area because the parents cannot be assumed to be acting perfectly in the interests of their child. But, if we really think that the Government knows a child’s interests better than the parents then should we be promoting school choice as a mechanism for improvement? And if we want variation in schools to reflect the local community’s values then is it right to intervene when parents’ values are not the same as policy-makers’?

One thing I missed during the election was the debate about whether government should pay for tertiary education.  There were a number of people saying “we pay for kids to go to secondary school, why not pay for them to go to university?”.

We have to be clear about why we treat these things differently in order to answer this question – it isn’t just about cost, it is about equality of opportunity.

We fund secondary education as we believe it constitutes a minimum level of education and investment in ALL individuals in New Zealand – a level that is required to give people a fair crack at life and civic engagement. Some parents also decide to have additional tuition to regular secondary education, like Chemistry tuition Singapore, to give their children extra opportunities when entering a college.

We do not believe that tertiary education is required for all roles, all types of engagement, and for all people.  As a result, we are instead subsidising a group of people who will undertake this type of education (and receive the return associated with this investment) by taxing those who are not interested in this type of higher level study.  In other words, it is a transfer of resources from people on lower incomes who are less willing/able to take on higher education to those who (over their lifetime) will be on high incomes.

Secondary school education is paid for (and correspondingly compulsory for a long period) to offer opportunities.  Tertiary education is not as heavily subsidised, as we are generally against regressive income transfers.  We already subsidise it to a LARGE degree on the basis of assumed spillovers, and we offer interest free loans on the basis of “equalising opportunity” – but larger subsides would largely be a transfer to the rich hidden in the language of “transformational change” … just like industrial subsides to capitalists.  I find it perplexing that people view such regressive transfers as “left wing” ….

I am not a fan of school zoning.  Its main outcome is to reduce school choices for poorer families.  Although the intentions underpinning the policy are probably noble, it has unintended consequences which on the whole harm the prospects of children from poorer households.

Zoning adds another incentive to move into a community filled with people that are “like” you.  As we know from Schelling, a small incentive for such things can quickly lead to complete segregation.  Rather than enforcing greater equality in the school system, zoning is a feel good policy that ends up reinforcing broader inequalities!

If we honesty want to ensure education provides for everyone in society, David suggests:

It is too simplistic to presume that alternatives to the public system will be sufficient to generate education improvements.  Indeed, the US experience demonstrates that there is a mix of outcomes from charter schools.  But what the US experience with charter schools has provided is the opportunity to learn from their experimentation.  My reading of this evidence is that there are at least three areas that schools can focus on to improve education outcomes:

• Ensuring that the school maintains standards about expected student behaviour
• Openly assessing, reviewing, and improving teaching methods
• Directing better teachers towards students who have the greatest need for improved education outcomes.

Comparisons like the one above rely on averages only but who’s average? Comparing the distribution of skills across countries shows that things aren’t as dire as all that.
You might well respond that Japan is still far more skilled than any other nation surveyed, which is true. Sadly for the Japanese, having skills and using them are very different things and there is no prize for possessing the most unused skills. If there was, they would definitely win it.
As skilled as the Japanese are, they make very little use of their talents in the workplace. By contrast, the English and Americans make excellent use of the skills they have, which goes some way to explaining why their wages are so high despite their relatively poor literacy and numeracy. That’s no reason for the UK to be complacent but we shouldn’t forget that there is also good news in the OECD report.

Computers are an important part of modern education, yet many schoolchildren lack access to a computer at home. We test whether this impedes educational achievement by conducting the largest-ever field experiment that randomly provides free home computers to students like those you can now find on sale at stores like 25pc. Although computer ownership and use increased substantially, we find no effects on any educational outcomes, including grades, test scores, credits earned, attendance and disciplinary actions. Our estimates are precise enough to rule out even modestly-sized positive or negative impacts. The estimated null effect is consistent with survey evidence showing no change in homework time or other “intermediate” inputs in education.

Note that they only gave computers to children, they didn’t then change lessons and teaching to take advantage of them. Consequently, the message is more that computers alone are not enough, rather than suggesting the computers won’t help.