Given the excitement around these methods, Chicago University has recently run the IGM Economic Experts Panel asking economic experts on whether the “ Randomized control trials are a valuable tool for making significant progress in poverty reduction”. The results of the poll are summarized in the graph below. 

The chart above highlights respondents’ agreement distribution. What struck me most from the results was Angus Deaton’s strong disagreement with the statement – especially given that he is an expert in the field.

Why does Deaton strongly disagree? 

To answer that we would like to think about what the RCT is and how does it fit to answer the policy question. Let’s shed some light on it.

What is an RCT? 

RCT is a technique used predominantly in medical sciences, but also applied in economics quite intensively , especially in the last few decades. The technique works in the following way. Researchers randomly select a group of people to allocate them a clinical intervention (such as an anti-cancer pill). The comparison group (which is called the control group) is also randomly selected where they received a placebo intervention (such as a sugar pill). 

Then the researchers compare the difference between the groups to quantify the significance of the treatment (clinical intervention or “treatment effect”). 

In economic research, RCT is often applied in poverty alleviation schemes to help quantify the effect of the policy intervention. However, it has been applied much more widely giving insights about the labour market, behavioural economics, health economics, taxation, and industrial economics.

So an RCT tells me what a policy does? 

RCT gives us an empirical treatment effect given specific conditions. This is the type of thing economists will often call a stylised fact.

However, stylised facts cannot give us general policy effects – they tell us what the policy response was in a specific set of circumstances, but we need to be able to generalize that effect to apply it in other circumstances. 

This is where Deaton gets concerned, and where some of the push-back against RCT stems from. 

To get a policy effect we still need a model – simply scaling up an RCT involves imposing an implicit model about how the policy and behavioural responses work, one that assumes the scale of the policy change does not matter and that there are no general equilibrium effects.

This matters.  If we provided a minimum income payment in Treviso, Italy we may find certain changes in prices and labour supply responses in that community.  However, we could not then take that result and “scale it up” across Italy as a whole – as Treviso was not a closed system in the same way an entire country may be, and the larger scale of the policy would influence prices and labour market responses differently as a result (eg if a minimum income increased demand for particular goods, doing so in a small region may not change the price for that good – while doing it for the whole country would).

How economic models fit in here? 

Economic models provide the mechanism for generating generalisability.  At the same time, models and RCT results should work in a recipricotive way. 

Given the same conditions as the RCT, a good economic model should be able to replicate the result – or at least key attributes of it.  Given the ability to replicate an RCT for those conditions, the model then embeds key assumptions about why that result held and a description of the systems that make up the question at hand – this allows an economist to ask counterfactual questions about what would happen if the policy introduced was much larger.

However, it isn’t all one way. Models should in turn be reevaluated if a robust body of RCT evidence suggests that – for a given set of conditions – the models results are false.  RCTs provide the pieces of evidence that models should be able to replicate, while models provide a framework for understanding what can’t be measured and how other, counterfactual, policy changes will work.

Examples of policy implementations (treatments):

To clarify let’s talk about specific examples of how the RCT can be used. 

Minimum wage and labour market 

Let’s consider an example with minimum wage increase and the labour market outcomes. Card and Kruger (1993) found that the minimum wage increase in New Jersey led to employment increases in the state compared to the other state (Pennsylvania), where the same policy was not applied. 

Now if we want to take this result and generalise it to the population level, saying that if we increase minimum wage, it will lead to an increase in employment rate, we are making a mistake. Why? Because the same increase in the minimum wage in all states would have different impacts due to the composition of those states, the overall change in prices in the economy, and the capital structure and industries that are viable across the US economy.

However, it showed there were real shortcomings with models that could ONLY indicate that an increase in the minimum wage could reduce employment. This helped to generate a literature that has more carefully considered the role of minimum wages given the potential for market power and strategic interaction in the market for low wage workers. 

What is the solution then? 

In Deaton’s view too much is being asked of RCTs, and indeed people need to recognise how to “transport” the results to another context:

“More generally, demonstrating that a treatment works in one situation is exceedingly weak evidence that it will work in the same way elsewhere; this is the ‘transportation’ problem: what does it take to allow us to use the results in new contexts, whether policy contexts or in the development of theory?

It can only be addressed by using previous knowledge and understanding, i.e. by interpreting the RCT within some structure, the structure that, somewhat paradoxically, the RCT gets its credibility from refusing to use. If we want to go from an RCT to policy, we need to build a bridge from the RCT to the policy.”

Deaton’s concern, which is reasonable, is that RCTs are treated as a sole source of truth. But such a focus isn’t just misleading, it would be bad science.

Card and Kruger’s paper did not tell us that a higher minimum wage would increase employment – it taught us that reality is complicated, and the evaluation of policy must be based on trying to understand how this works, using both evidence and theory.  Duflo, Kremer, and Banerjee similarly see the importance of both – in her Economist as Plumber article Duflo notes:

“However, because the economist-plumber intervenes in the real world, she has a responsibility to assess the effects of whatever manipulation she was involved with, as rigorously as possible, and help correct the course: the economist-plumber needs to persistently experiment, and repeat the cycle of trying something out, observing, tinkering, trying again”

Deaton’s concern is that people will experiment and measure without ever trying to model and understand what they are doing – thereby generating a stream of published studies but no understanding.  Those that are more positive about the RCT revolution instead see such experimentation as part of this very iterative process that helps to describe the “transport” problem that Deaton is concerned about.

To sum it up 

Predicting a policy result from a given policy involves an implicit model – irrespective of the number of RCTs that have been run. However, these RCT provide a discipline that any worthwhile predictive model needs to be able to replicate – they provide the true stylised facts (if done properly) that a predictive model must match to be credible.

Note:  PP on twitter asked for a post on MMT, and asked me to try to avoid making it technical.  I attempted to do that at much as I could – however, I was forced to use words like endogenous, as it was the cleanest way of trying to get across the point that by using supply and demand savings and investment are jointly determined … and the interest rate is set as the price.  The very idea that economists only think the causal chain goes only one way or the other is patently ridiculous – and does not represent economists, no matter how much people keep saying it does.  So try to keep that point in the back of your head until at least the end of the post if you read it

I have nothing inherently against modern monetary theory, its proponents, or the value judgments involved.  But my impression is that MMT theorist view central bank independence and the framing of government policy as an ideological device to “shrink government” and so have decided to create a “strawman” mainstream economics to attack, rather than directly admitting they want a larger state (and the trade-offs involved in that).  For me this simply lacks transparency!

So how does MMT differ from mainstream economics.  Well in the words of Bill Mitchell (who I choose because he is clear, both here and on his blog – which is a good thing!) it comes from economists accepting three false premises:

1. A government has to borrow to spend
2. There is a fixed supply of savings at a point in time
3. Governments crowd out investment for that fixed supply of savings, pushing up interest rates

Supposedly all three of these are in the core of economics, and they are all wrong.  Huzzah.

Ok, so if that is MMT then I’m not sure who in the world they are actually arguing with.  The government can print money, and this is in any graduate macro book, so that doesn’t hold as a premise.  Savings and investment are determined by supply and demand, they aren’t a “fixed thing”, so that isn’t a premise in mainstream economics (Sidenote:  Why do people keep saying “savings determines investment” or “investment determines savings” – I’ve never heard economists talk like this … remember the money multiplier is a ceteris paribus example, not a description of the causal device).   Crowding out is actually a premise – but it comes from government demand pushing up demand for underlying goods and services … because government demand for things is just like the demand of any institution.

Let me restate these premises in terms of what the mainstream actually has:

1. A government can finance spending through taxation, selling bonds, or issuing money.  In the end, prices and expectations adjust such that someone pays for government consumption and investment.  More specifically the government has to match spending to taxes over time for a certain inflation target!
2. Savings and investment are determined endogenously by demand and supply factors in the economy, where the “price” is the REAL interest rate (perhaps I should use the world natural/fundamental here) – as savings and investment are factors that are involved with transferring consumption (the thing we really want) over time due to technology, the rate of return on investment, our time preference, etc etc
3. Additional government demand for goods and services will push up the price of those goods and services and push up the REAL interest rate in the economy … remember the real interest rate is a price, when the government is trying to push up investment of consumption this increases the demand for these given an underlying PRODUCTION FUNCTION, crowding out private investment and consumption … the real interest rate rises as investment/consumption demand has been pushed up and the lift in relative prices has to occur in a way that makes private agents defer consumption/investment in terms of the quantity of goods and services.  No amount of hammering the S=I identity in the face of fiat currency changes this

These premises actually sound pretty good to me!

I remember my dad used to say “it isn’t money that matters when we think about people, it is the actual good and services that are made and consumed”.  He didn’t take the same point out of it I did, but I think on this statement he was right – we need to actually think about goods and services, capital, and labour here.

Now there are MMT people who claim they do (back to Bill again) – that they have a production function (which seemed to be a bit missing earlier) and they have a Phillips curve (tells us how this production function and prices pressures relate through time).  This is good, these two things are necessary!

But if that is what they are doing, then their inherent model IS the mainstream model.  The three “fallacies” that they mention don’t actually exist – and that third point they list down is WRONG … there is crowding out.  Instead their argument is that the “optimal size of government” is larger than they hear other people saying … which is both an empirical and subjective question that people have already written (and should continue writing) countless books on.

Yes, people should discuss this, and discuss trade-offs.  But misinterpreting mainstream economics and pretending to offer an alternative in order to sell your view as not being “subjective” (which all policy conclusions are) is both misleading and irritating for people who view themselves as part of the mainstream.  Personally I like the idea of “changing the frame” to think about issues – but to me that is just a good way of researching, rather than a sign of a militant revolution inside economics

A much better critique of MMT (albeit more technical) can be found here.

The two I’m 100% going back to when I get a chance are the links on inequality (pet interest issue of mine, and all economists) and the nice summary of the actions taken by the Fed over the past four years.

There is also a piece where an academic economist attacks some consultants who attacked an OP-Ed they did, which attacked some work done by the consultants (or at least came to a different conclusion).

This is all well and good, and when it is the specialist field of the academic economist I would place more weight on their words than on a consultant (this is coming from a consultant/forecaster).  But this doesn’t just cut one way – if the academic economist is correct that their “clients” are the community they work in, then I would expect more academic economists to come out to help educate the public, and analysts such as myself, about the issuess.  Given how little this actually happens, it appears that at least the consultant he is criticising serves his clients in a better fashion that much of academia serves theres

This comment is not meant to be harsh at all – hopefully it illustrates the huge respect I have for academic economists, and my burning pashion for them to get more involved in the public discourse!  Because that is exactly how I feel.

My point here is that high CEO pay is not the disease, but the symptom – of the fact that CEOs have too much power. Treating the symptom is not sufficient, and might even be counter-productive.

I would note that, in terms of thinking about “excessive power” we need to ask ourselves about the framework that business works within.  Large businesses will be subject to waste, empire building, asymmetric information, and organisational issues – but just because there are issues does not mean that anything can be done to improve them, it may just be the way things are.

The key point is that an “excessive wage” just like a “price” that is “out of whack” is a signal, a symptom, of some underlying issue.  It should be a call to try and understand how the allocation of resources is working in the market, and whether any issues exist, not a call to arbitrarily mess around with prices.

Last week, I heard a number of commentators at the sustainable economics conference claim that neo-classical economics was:

1. Based on falsified views of the individual,
2. Static,
3. Had no supply side.

Then I saw an attack on “neo-classical economics” from Roger Kerr at the Business Roundtable (and more) which seemed to imply:

1. It ignores institutions,
2. It ignores transaction costs,
3. It is static.

I was surprised by these attacks.  More than surprised, I felt like the attacks were based on a straw man version of neo-classical economics – one that in many ways never existed, and if it was floating around it was during the 1950’s-1970’s when a lot of the focus was on a narrow neo-classical synthesis in macro theory.

Neo-classical economics is a term for the “core” of economic theory – primarily modern mainstream microeconomics.  I have discussed here how we get from scarcity to neo-classical economics, and I have discussed neo-classical economics in more detail here.

This “core” is different to the core in the 1970’s – as many of the fringe elements of theory have now shifted their way inside the core of economics (think game theory, endogenous growth theory, transaction cost economics).  However, this is the point, neo-classical economics has evolved and it is this modern version that is taught in universities (at least it is at Victoria) nowadays – contrary to the claims at the sustainability conference that economics hadn’t changed.

The reason I am so defensive about the definition of neo-classical economics is because people see it as the current core – which according to my definition it is.  Setting up an alternative definition of neo-classical economics and knocking it down is either equivalent to setting up a straw man to attack, or directly misleading people to make it sound like modern economists are incompetent.

In essence, my prior belief is that the future is not predetermined per see, but that there are current factors that influence future outcomes that are observable – as a result, we can use knowledge about the causal or empirical relationship between these factors to get some idea regarding what could happen and some of the risks around it.

However, in the face of genuine uncertainty I would believe we have no knowledge.  This specific view also indicates that the distinction between free will and determinism is unobservable – as there is no way to disentangle the relationship between cause and effect in a way that tells us whether there is choice, or whether the causal mechanism in itself determines the future.

Yet, a recent study that appears to show a mildly statistically significant relationship between people’s predictions of what will happen and what does happen BEFORE what occurs has been in any way determined.  In essence, there is complete uncertainty but people’s ability to judge what will happen in the face of this is greater than we would expect from chance! [ht Chris Blattman, Marginal Revolution *].

To me, this also provides a test of determinism vs free will – at least along some level of interaction.  Why?  If it is possible for people to “see the future” before it is ex-post determined then the future must in some sense exist before it appears to exist.

In the face of free will, we can still judge what will happen on the future given information, but we would not expect people to outperform chance in the face of no information.  In the face of determinism we would expect the ability to judge the future with no information would be related to the strength of the precognitive ability of the person – if, among people, this is on average greater than zero we would expect a statistically significant deviation from chance.

This is all very interesting, but I would like to see the results replicated and further testing done before I even begin to shift my posterior probability regarding such things.

Dualists about the mind and brain – those who hold that there are thinking substances like souls in the world as well as all the ordinary physical stuff – say that the mind sees and thinks and wants and calculates. Contemporary neuroscience dismisses this as crude, but Hacker argues that it just ends up swapping the mind with the brain, saying that the brain sees and thinks and wants and calculates. He says, “Merely replacing Cartesian ethereal stuff with glutinous grey matter and leaving everything else the same will not solve any problems. On the current neuroscientist’s view, it’s the brain that thinks and reasons and calculates and believes and fears and hopes. In fact, it’s human beings who do all these things, not their brains and not their minds. I don’t think it makes any sense to talk about the brain engaging in psychological or mental operations.”

Now, I agree with Peter Hacker that language is important – and if we are not careful some of the claims that come out of neuroscience can sound incredibly deterministic.

However, there are two issues that come up here:

1. Hacker immediately assumes dualism, that the mind and brain are separate.  But is this true?  Could it be that what we believe is consciousness is really just an evolutionary device to make us function as we do, it has certainly given us an advantage as a species.  This is very very deterministic to be sure – but this doesn’t mean it is false.
2. The mind and body, even if they are separate, are heavily inter-related.  Decisions are made on the basis of information, costs, and benefits, that are related to how our brain functions.  In the same way that economists see choice as “determined” by costs and benefits, it is unsurprising that a neuroscientist would talk about the functioning of the brain as “determining” outcomes.

Now, I would prefer to move solely to the second point, I will accept dualism.  In that case, the methodological issue that Hacker is uncomfortable with is how we translate what a neuroscientist is saying – and exactly the same issue exists in economics.

Ultimately, even if we say that costs and benefits “determine” choice we are not really arguing about determinism and free will directly.  However, it may SOUND like this type of analysis assumes determinism – as it is saying “given a set of costs and benefits this outcome/choice will result”.

The key issue is, ex-ante could the individual (or the mind) decide to make a different choice – determinism would say that, given these costs and benefits this choice must be made, free will implies that the individual has the ability to make any choice.  This is not an issue economists and neuroscientists delve into because they are not trying to figure out whether we have determinism or free will – they are just trying to understand the environment that influence choice.  In fact, it is something we CAN’T OBSERVE and so it is an issue you can’t really solve.

Now I think Harker is the one mixing the ideas of studying the costs and benefits that are processed in the brain, and the actual direct choice.  The fact is that the brain DOES determine these costs and benefits, it does determine the feelings of fear and emotion – and that is what neuroscients are studying.

How individuals subjectively interpreted these costs and benefits is important – but as economists know this is unobservable, and requires both an ability to “live” someone elses life AND an answer to whether people have free will or whether their actions are deterministic.  I hardly see how criticising neuroscients for not studying the unanswerable sections of choice makes any sense – other than as a warning regarding how they frame there own results.

Politicians are in charge of the modern economy in much the same way as a sailor is in charge of a small boat in a storm. The consequences of their losing control completely may be catastrophic (as civil war and hyperinflation in parts of the former Soviet empire have recently reminded us), but even while they keep afloat, their influence over the course of events is tiny in comparison with that of the storm around them. We who are their passengers may focus our hopes and fears upon them, and express profound gratitude toward them if we reach harbor safely, but that is chiefly because it seems pointless to thank the storm.

If I’m honest, I think that the belief that government can create growth magically stems from the fact that people want to feel like they have control of things – economic growth is something that impacts upon our daily lives that we have no control over, but if we can tell ourselves we have control it is easier to live our lives.

In the same way our forefathers would worship the sun, or a “god of the harvest” our modern society worships government policies that “will provide economic growth”.

Discuss

However, this does not mean that economists do not have some sort of moral obligation associated with their analysis.  In order to cover this off Robin Hanson discusses the “efficiency economist pledge” (ht Robbie A). :

I pledge to be an efficient economist, who helps clients find win-win deals to resolve social conflicts.

Reading the full pledge, it seems that the goal is to use economic advice to provide the client with the knowledge and tools they need to get what they are after.

This is as opposed to a moral principle I have heard discussed by Eric Crampton which pertains specifically to doing work for government.  Namely that when there is a difference between the social optimality of a policy and the policy that suit policy wonks, it is not right to provide advice that specifically benefits your client at a cost to broader society. [Do I have this right Eric?] Update:  Eric lays out his view more fully here.

If I have described Eric’s additional moral requirement appropriately, then my main counter to this has always been “it isn’t the economists fault that they are taking the incentives involved, it is just an illustration of government failure”.  The illustration should be made more apparent IMO, but I do not blame analysts for acting in their self interest.

Now although I believe much of this attack is excessive, and I believe the role of maths in economics is very very important, like all protestations there is a grain of truth to be found.

The way I see it, the role of maths is as a language – Mario Rizzo covers that off in more detail here.  Now, maths is not the only language that can provide clarity and solidity to the discussion of a given issue – words are still effective if treated with the same structural integrity.  The prime example of this is the work of Spinoza.

Maths was useful as it helped to drive structure and clear up arguments.  It helps to clarify thoughts, and acts as a clear linguistic device.

However, as with any language the main issue with maths in economics is when it loses its ability as a clear linguistic device,  namely when:

1. people using it become overconfident of their assumptions, thereby missing some implicit assumptions in their modeling
2. people use mathematical notation to hide their results.

However, both of these errors aren’t a problem of too much mathematical use in economics – they are an issue of too little concentration on the formal, logical, elements of using maths as a language!

The lesson here is that economists have to be aware of exactly what the elements of their models represent, and should be able to evaluate these elements when moving on to make recommendations – at least in some sense.  The maths is not the problem, and an arbitrary crusade against mathematics would only serve to reduce the usefulness of economic analysis.

An economist with a strong eye for mathematics and a strong nose for elements of reality can overcome these issues and use mathematics to provide a clear, incisive, and valuable contribution to the study of the economy and society as a whole.  I am glad to say that I believe many theoretical economists do possess these traits – hence why I am confident about the future of the discipline.

Note:  I am not a theoretical economist, and spend most of my time using words and numbers – not maths.  Even from my position of weakness I can see the important role that mathematical rigor plays in economics.