Economic Impact Studies: The Number is not your friend

I’m excited today to introduce a new guest blogger, Peter Mallow.  Pete’s a one of a kind.

Really?  Really.

How many people do you know who have developed subdivisions, fought with small town planning commissions, run a civil engineering firm and completed a Ph.D. dissertation on the use of statistical regional analysis and modelling methods to link  health services to health outcomes within a geographic region?

I didn’t think so.

One thing that always strikes me about Pete is that he has developed this amazing ability to understand and work with data that is way over my head, but connect that back to immediate real-world planning and economic development issues in a way that shines new light on the whole scene.  Sometimes it’s like having a guide to the underworld of how we think… or don’t think… about the data behind our development assumptions.  And he isn’t afraid to call it as he sees it, or to tread in where others might fear to go.

Pete and I have done a very well-received presentation a few times called Don’t get Fooled Again: Understanding and Interpreting Economic Development Projections.    You can see an annotated version of that presentation here and listen to a podcast of that presentation here.  Now that Pete is an official Ph.D. as of a couple of weeks ago, he has kindly agreed to write here on a somewhat regular basis to help us think through the assumptions and mistakes that we make when dealing with economic impact studies… and how even us non-math types can handle them better.

So….I am delighted to introduce you to the esteemed Dr. Peter Mallow!


An article published in the Journal of the American Planning Association in 2002 by Brent Flyvbjerg suggested that economic evaluations for large projects – typically the kinds of evaluations done when public dollars are sought to support a project – are intentionally underestimated.  Flyvbjerg followed up this claim with a recent article in the journal Cities, in which he accused planners of deliberately lying about the projected economic impact of projects as shown in those evaluations and attempting to cover up these lies through its professional organization, the American Planning Association.

I think Flyvberg has identified a crucial problem regarding the economic evaluations used to support large projects.  But I think he focused on the wrong piece of the equation.  There will always be an occasional “bad apple” who has intentionally mislead, or a major miscommunication that leaves one side of a disagreement feeling cheated.  As I read these articles, I am reminded of a far more common problem that I see when it comes to understanding and using economic evaluations:


With all the demands on a planner’s or economic developer’s time, some ignorance is to be expected.  Almost by definition, these professionals have to be generalists first and specialists last.  Because of that, communities typically feel that they need an expert to tell them what economic impact the new project will generate, or how the cost and benefits will stack up, or how many TIF dollars will be generated.  In some cases, planners and economic developers simply rely on the project applicant’s own economic evaluations because the community doesn’t have the resources to hire their own number-generating consultant or buy the software package that promises to spit out that number for you.  In other cases, they get their own consultant or software, but the fundamental problem remains the same.

It’s easy, very easy, to rely on an expert or software package that promised to give you a “certain” answer.  That certain answer? It’s The Number, the one specific number that everyone clings to promote the project.

You have most likely encountered The Number. The Number tells you how many new jobs, how much investment, how much tax revenue, etc. is going to come as a result of the public investment.

People love The Number.  Decision makers love having a clear answer to point to, and the media loves having a number it can easily report.  The Number becomes its own living creature, and the project skeptics and detractors can only wait for the inevitable.

The inevitable?

The inevitable, unexplained, and unexplored uncertainty that is buried in the assumptions and methods behind The Number.  At some point in the future, a seemingly innocuous assumption will change and the actual impact of the project will change dramatically.  When that happens, the project skeptics and detractors will pounce.

And guess who gets caught in the crossfire.

So what do we do?  Too often, planners and economic developers think that their only choice is to run with The Number and deal with the inevitable down the road.  After all, they will argue, life is not certain, and time doesn’t allow us to pick apart the assumptions, methods, and the results embedded in The Number.

And many of you would probably add: Mallow, the money won’t allow it, either.

That’s all true.  But, let me propose an alternative.

If somebody, anybody, tries to sell you on The Number, the only thing you can be certain of is that The Number will be wrong.  Think about a stock investment for a minute.  You have no idea what the value will be in five years…or one year.  You hope it will be higher than today, but would a trustworthy financial advisor assure you that the $10,000 you invest in a stock today will be worth $15,552.35 in one year, or $32,535.97 in five years?

Of course not.  They will give you a range of the most likely future values under different growth or loss scenarios.

Instead of accepting The Number, demand to know the plausible range of jobs, tax dollars, economic impact, etc., given what we know.  Or ask for a range of scenarios — what happens to The Number if the building doesn’t fill up as fast as they project, or the average payroll turns out to be less, or the cost of steel spikes in the middle of construction.  What then?

You can do that.  And you and your community will be better off for it.  After all, you’re not evil.  And you’re not even really ignorant.  You just need to know what to demand, and demand it.


5 thoughts on “Economic Impact Studies: The Number is not your friend”

  1. You don’t always need a consultant. I have utilized some very inexpensive software put out by Georgia Tech University that provides a careful, thorough, and unbiased analysis of the economic impact of projects.

  2. Good idea. But if the number is the enemy, can I say that the “quality”, the “process” and the “values added” have to be targeted and counted ?

  3. One of the problems with these economic impact studies is that economic impact is not real money. You are trading real money today for Monopoly money tomorrow. At some conceptual level that only an economist could love, your communtity may be $ X,000.00 better off, but you can’t pay for a project with that.

    The problems come when you try to monetize that while applying the same logic. That’s when you find out its not real money and its the local government that floated the bonds that gets left holding the bag. This came back to bite a local government in Washington very bad a few years back. The state legislator now requires an independent financial assessment of public facilites proposals.

    Unfortunately, it’s the state DOT that’s getting the biggest haricut. They floated huge bonds based on toll revenute forecasts that turned-out to be based on very faulty traffic forecasts.

Leave a Reply

Your email address will not be published. Required fields are marked *