More defence

Posted on by

Ok, so there were a couple of comments in the previous post that I think I need to discuss in order to explain why macroforecasters have value.

First, from Eric Crampton:

I think both Falkenberg and Cowen are going after the ones selling themselves as having crystal balls, not the folks who are providing specialized reports for specific clients on how the likely macro environment, which everybody knows about, is likely to affect those clients in particular (bolding by me).

The thing is, no-one knows about the macro environment – this is the gigantic kicker.  It isn’t necessarily about theory, or about have the information around, it is about framing the information and making it useful.  Firms have a great idea about their own firm, they have a relatively good understanding of the specifics of their industry, but they are most uncertain about the macro environment.

And that is one of the key things macroforecasters provide, they provide a service that brings together information, describes the economic situation given that situation, and then discusses risks heading forward – for some reason there is this impression that either:

  1. This is a useless service or
  2. This is an unnecessary service

But I think all you have to do is look around economists, and see how much they debate each other about “what has happened” let alone what is going to happen.  In such a case, having someone you can trust to give you a consistent and open explanation of such events probably has a bit of value right.

Just because all the bits are available doesn’t mean there isn’t value in a service that puts it into something workable and useful – I would have assumed that people would have at least recognised this point.

Also, Paul Walker stated:

That’s only because macro is witchcraft while micro is actual economics!!!!!

If there is a micro question I would trust a microeconomist more than I would if there was a macro questions and I had an answer from a macroeconomist.

However, part of the reason for this is that the questions are so very different.  If a microeconomist thinks they can build a better macro model then they should really just do it or STFU.

Why do I say this?  I was initially trained with micro, I have a slight understanding of how micro models work.  And trying to apply that level of detail to an aggregate economy would be nigh on impossible.

Macroeconomists are forced to use more imperfect tools because of the way their questions work – not because they are necessarily inferior.

Update:  Anti-Dismal comments here.  He suggests you read his comments and Eric’s – and you should.  But in that case don’t forget to read my comments.

I’m sticking to my guns here that a lot of the criticism falls in these two camps:

  1. Macroforecasting isn’t being compared to an appropriate counterfactual,
  2. The service value of the product appears to be virtually completely ignored – which is a pain given that this is the primary purpose of it.
  • http://offsettingbehaviour.blogspot.com/ Eric Crampton

    Let me rephrase then: instead of “that everybody knows about”, then “about which nobody, forecasters included, has specific superior knowledge”. There’s the stuff out there that everybody knows, which may or may not be right, then there’s individual guesses, which can’t systematically improve on what “everybody knows”.

    The best measure of “what everybody knows” is encapsulated in futures prices. Beyond that, it’s hard to see much useful.

    I would never, ever pretend that I can build a better macro model. I tend to think rather that it might be impossible to build a macro model that does better than just looking at futures prices.

  • http://www.tvhe.co.nz Matt Nolan

    @Eric Crampton

    “Let me rephrase then: instead of “that everybody knows about”, then “about which nobody, forecasters included, has specific superior knowledge”. There’s the stuff out there that everybody knows, which may or may not be right, then there’s individual guesses, which can’t systematically improve on what “everybody knows”.”

    I have no problem with just looking at future pricing – but there is value from having someone describe what it means, and go through the information to paint risks that can’t be captured in a single price. There is a large information role – and I think that is getting put too much to one side in the discussion.

    Also, there is value in knowing what has happened “around you” not just what will happen – for example when a firm experiences an increase in demand they want to know whether it was the result of a shift towards that industry, general growth, or because they are getting market share. The data is sitting around, but the collation and interpretation is something that firms place a value on.

    “I would never, ever pretend that I can build a better macro model. I tend to think rather that it might be impossible to build a macro model that does better than just looking at futures prices.”

    I’ve never met a microeconomist who feels it would be right to build a “better macro model” – as macro questions aren’t really amiable to that approach and they realise that.

    Future prices are useful at telling you something – but I find the idea that no value could be added around that for a firm as completely inappropriate.

  • http://offsettingbehaviour.blogspot.com/ Eric Crampton

    Again, Matt, I’ll distinguish between
    1) making new macro forecasts of the overall economy – which seem unlikely to add much to what’s already out there
    and
    2) taking the existing forecasts as well as futures prices and packaging them up with analysis for industry or firm-specific purposes, or tweaking models to get some industry-specific effects.

    I can buy that there’s value added from the latter.

  • http://www.tvhe.co.nz Matt Nolan

    @Eric Crampton

    I guess it depends how we define macro forecasts. If all people did was publish a bunch of numbers, sure. But the discussion and explanation of the numbers and the painting of risks are all factors that have some value.

    Market prices move with information, people that solely do macro forecasts would describe what this information is for people and then discuss what different outcomes and the associated change in market pricing mean. I’d also note that their are a lot of thing macro forecasters forecast where no clear futures pricing is available.

    Trust me, I think these sorts of forecasters can massively oversell their value (which is why you see a lower number of firm hired economic forecasters – and a complete collapse in the market for just “economic” forecasts), but it doesn’t mean they have no value.

    I mean, isn’t saying that someone who describes available information for clients has no value the same as saying that a psychiatrist has no value (as they often tell people to do things they know they should do anyway) or that a teacher has no value (as they are teaching things that are freely available on the internet, or the text book they’ve already purchased).

  • http://www.tvhe.co.nz rauparaha

    @Matt Nolan
    You’re painting a picture of a macro forecaster’s value resting on their qualitative analysis. Nonetheless, the centrepiece of any forecast publication is the numbers, so what value do you think the ‘bunches of numbers’ add to the information that’s already available?

  • http://www.tvhe.co.nz Matt Nolan

    @rauparaha

    Really, I thought the centerpiece was the executive summary for most people, and that is half description of what has been and half discussion around the set of risks for the future.

    If it turns out that clients decide they value the numbers highly for some reason, when all the discussion and description has been offered, it isn’t up to me to judge what they put a value on.

    And if it turns out they do put value on the numbers, it might be because they express things that are, in some sense, available – but are easier to interpret and use in that form.

    Furthermore, even if the numbers were sitting at the very front of any forecast – I doubt a forecaster could stay in business long by just printing piles of numbers. Without a discussion about what is going on, the risks, and how to interpret different figures I severely doubt anyone would remain a forecasters client.

    It is a service, I find all these attacks on the provision of a service because “what do they do” to be strange – it reminds me of times when people say “agriculture” is the only real input in the economy, everything else is just playing with it and “stealing ” value.

  • http://www.tvhe.co.nz rauparaha

    @Matt Nolan

    Does anyone really suggest that macro forecasters don’t add value that clients are willing to pay for? If that were the case then I doubt many would be in business any more. I think some people are just suggesting that the numbers generated by the various forecasters don’t add to experts’ current understanding of the economy.

    When you’re sitting down fiddling with numbers in R you could instead be doing qualitative analysis, writing more reports or talking with clients. Plainly it must be valuable to do the quantitative forecasting or you’d spend far less time doing it. Hence my question: what value do you perceive the quantitative forecasting you do to have, given the wide availability of futures prices and others’ forecasts?

  • http://www.tvhe.co.nz Matt Nolan

    @rauparaha

    “I think some people are just suggesting that the numbers generated by the various forecasters don’t add to experts’ current understanding of the economy.”

    Remove the term experts, and you have what was stated – it is being implicitly suggested that a “marginal forecaster” doesn’t add value for the client they work for, apart from say signaling. I felt this criticism was extreme, and felt that the idea of providing a service was being ignored – there seemed to be an implication that “unless you are providing a magical method of cutting down RMSE you have nothing to add to a firms understanding of the general economic situation”.

    “When you’re sitting down fiddling with numbers in R you could instead be doing qualitative analysis, writing more reports or talking with clients. Plainly it must be valuable to do the quantitative forecasting or you’d spend far less time doing it.”

    It is useful to make sure we have a model, and general relationships, set up such that we can discuss the data sensibly when we do the qualitative things. It would be hard to provide a qualitative service unless we also made sure we collated all the relevant information we can – which is a time consuming and costly process.

    Surely, one of the services we provide is a filter for information – picking out important and interesting things and providing them in an easy to consume package for consumption by a client. I have strong faith that this is one of the primary purposes of macroforecasters – given the sheer number of clients that have said that is the sort of service they pay for.

  • http://www.tvhe.co.nz rauparaha

    @Matt Nolan
    I said experts because I don’t think that most people who pay for forecasting services have a strong understanding of the current macroeconomic situation. If they did then they probably wouldn’t need the service, which is why many people who view themselves as experts consider the service unnecessary. Obviously, it’s easy to pick on forecasts that hold a lot of factors constant but difficult to improve upon them.

    However, I don’t think that the forecasts of the marginal forecaster add much to experts’ understanding of the situation. I think the process of forecasting is primarily important as a tool for the forecaster to better understand the situation so that they can explain it clearly to their client. What I find slightly odd is that forecasting publications include many predictions, often without quantification of the uncertainty around them, while the forecasters tend to shy away from being tied to particular numbers when questioned.

  • http://www.tvhe.co.nz Matt Nolan

    @rauparaha

    “I said experts because I don’t think that most people who pay for forecasting services have a strong understanding of the current macroeconomic situation.”

    I 100% agree with you rauparaha. I was just pointing out that the position I had been arguing against in this post excluded the expert part – it seemed to assume that the knowledge was just there already for everyone.

    “However, I don’t think that the forecasts of the marginal forecaster add much to experts’ understanding of the situation.”

    I agree. Especially as we use the same methods.

    Just to note though, this was not the position I was arguing against at all.

    “What I find slightly odd is that forecasting publications include many predictions, often without quantification of the uncertainty around them, while the forecasters tend to shy away from being tied to particular numbers when questioned.”

    I don’t like it when forecasters work to attach to much certainty to their forecasts – and I think there is an institutional issue with such things. To me, it seems obvious that value comes from the discussion of risks and making data easier to interpret/eat for sure.

    I’ve always been told – as soon as you make a forecast talk like you know it will be wrong, but just make sure you can discuss why it will be wrong. It is those reason that people want – and so that is what you should provide.

  • http://antidismal.blogspot.com/ Paul Walker

    “If a microeconomist thinks they can build a better macro model then they should really just do it or STFU.”

    I was joking to a large degree. But the serious part of my comment would be that macro models may not be worth building in the sense that macro is after all aggregated micro. so while macro aggregates can tell us something has happened in many cases explaining what has happened and why requires a micro model. My view on this is explained in the following example, which is from stuff I’m working on currently on the so-called “knowledge economy”. It refers to the post 1995 productivity increases showing up in the US data:

    “However while this macro level data can tell us that something has changed, it can not tell us what changed and why. The productivity data is the aggregated result of changes at the micro level, in this case at the level of the firm. Such changes require a microeconomic explanation. Without an understanding of the firm level effects of changes in the importance of knowledge/information in the production process we will be unable to fully characterise the knowledge economy and thus will be unable to measure it correctly.”

    So macro can tell us some things but explaining those things maybe a micro issue.

  • http://www.tvhe.co.nz Matt Nolan

    @Paul Walker

    “I was joking to a degree”

    Definitely – I just felt it was an important area to cover off in any case. It made me think of the fact that the disciplines ask different questions, and I thought that would be a point I should raise.

    And I agree with you that “macro should essentially be micro at a very aggregate level” – I said as much in the comments to the previous post. The issue is that this is nigh on impossible – given data availability and the such.

    However, when the first best option isn’t available, it doesn’t mean we can’t use the lessons of economics (namely as always that there is scarcity and people respond to incentives) to describe what is going on in some sense – and what risks are associated with the economic environment. The inability to provide the first best in a world with perfect data availability and no time constraint does not mean that the next best option is to do absolutely nothing.

  • http://www.tvhe.co.nz rauparaha

    “The inability to provide the first best in a world with perfect data availability and no time constraint does not mean that the next best option is to do absolutely nothing.”

    Is the next best to sit on the sidelines and snigger at the people who can’t achieve the first best?

  • Andrew Coleman

    Hmmmm

    (a) I always thought most macro-forecasters were paid for providing numbers so that their clients could defend ex post “poor” decisions by saying and by being able to verify that they used a “reputable” brand of forecasters when they made their decision. ” But Westpac said that inflation was most likely to be 3 percent; we used this number to justify our investment.” In short they are paid for being the fall guy, a most useful service. This also is consistent with the long disclaimers that macroeforecasters provide their clients, to ensure they can be pillorised but not made legally liable.

    (b) At least one of the exceptions to this rule was Infometrics in the early days; they were paid because Gareth made his presentations funny and insightful. The other exception is that some macroforecasters use off-the wall forecasts to generate publicity

    (c) It has been well known for at least two decades that a small VAR estimated with Bayesian techniques using a Minnesota prior provides forecasts that are as good as large structural forecasting models. This is because, I suspect, (1) there are some genuine time-series patterns behind many aggregates that a forecasting VAR picks up and (2) most large structural models are junk, in the sense they contain too many false restrictions that offsets any advantages the of the true restrictions they impose.

    Consequently, it is possible to provide cheap numerical forecasts for the main macro variables that are nearly as good if not better as expensive numerical forecasts. However, these forecasts are all a sophisticated version of ” the economy will return to its average growth path,” and for entertainment purposes a more elaborate story is desired. Hence structural models are used to make a story. This seems to be their main use in Government agencies.

    (d) For many series, the real issue is timing: one is reasonably sure that eventually the adjustment will take place, but one is much less sure when it will take place, possibly because the timing is largely random. What is curious is that more macro-forecasters don’t concentrate on the medium term picture, which they probably have more chance of getting right. It maybe the case that if they did this they wouldn’t change their story so often, and thus wouldn’t be so entertaining (see (2)). Or it may the case that their clients are much more interested in timing than ultimate accuracy, as in much of the world a lack of timeliness is punished much more heavily than a lack of accuracy, for reasons (as a notorious sluggard) I don’t quite understand.

    (f) Not clear that microeconomists should waste their time building better macro-models, for the same reason you shouldn’t cast pearls before swine. For two decades microeconomists have been teaching macroeconomists the importance of matching (and macroeconomists have been learning), but there aren’t many macro-forecasting models that are centred around fluctuations in matching opportunities (with the exception of models of real-estate markets).

  • http://www.tvhe.co.nz Matt Nolan

    @Andrew Coleman

    “I always thought most macro-forecasters were paid for providing numbers so that their clients could defend ex post “poor” decisions”

    I have no doubt there is an element of that for sure. However, that merely pushes back the responsibility one step. Surely if the discussion and explanation wasn’t adding value, the person would lose the ability to use it as an excuse in the future.

    “Consequently, it is possible to provide cheap numerical forecasts for the main macro variables that are nearly as good if not better as expensive numerical forecasts. However, these forecasts are all a sophisticated version of ” the economy will return to its average growth path,” and for entertainment purposes a more elaborate story is desired.”

    “For many series, the real issue is timing: one is reasonably sure that eventually the adjustment will take place, but one is much less sure when it will take place, possibly because the timing is largely random. What is curious is that more macro-forecasters don’t concentrate on the medium term picture, which they probably have more chance of getting right”

    While this is very true, I would note that the primary interest of clients in such cases is usually on the relatively short term. I am pretty sure most macroforecasters would go out about 5 years at least – but they need to focus on where their market is when they write things up.

    “Not clear that microeconomists should waste their time building better macro-models, for the same reason you shouldn’t cast pearls before swine”

    I’d say the main issue would be to do with data more than anything else – what you get out is only as good as what you put in, and a lot of data around the world is not that pretty.

    “For two decades microeconomists have been teaching macroeconomists the importance of matching”

    I’ve long stuck to a rule that cutting edge macroeconomics is an applied version of micro from 20 years earlier … however I digress.

    ….

    Overall my key point remains that the key value comes from collating information in a way that can be used by clients – it is a service, and in that sense should be seen as such. The presentation issue you raised is very important – providing information in a way that improves a clients understanding, or illustrates risks on the horizon (in this case through a presentation) is a major part of what a macroforecaster should do.

    I felt that the discussion of macroforecasters in the initial set of articles completely ignored this point – and then aimed to sure that no value could be gained from such a venture.

    I also have no doubt that some of these attacks on macroforecasters stem from the fact that, some, act like they have a crystal ball. I don’t like that either, but I just don’t think it is right to tar everyone with the same brush.