In these challenging economic times, praising the virtues of the market economy is not exactly in fashion, and for good reason: the “efficient markets”-hypothesis has been blown out of the water spectacularly.
But even with all it’s failings and inefficiencies, the market economy is the greatest creator of wealth for all of mankind throughout history: feudalism failed because it was hereditary and not meritocratic, socialism failed because of it’s inefficient command-and-control nature that ignores feedback. Markets are fallible and flawed, but given all available systems of economic organization, it is still the best we have – the price mechanism is the best feedback mechanism ever to exist.
Similar things can be said about Agile software projects. They are by no means assured success, nor are they ever perfect in execution or outcomes.

But I would argue that when Agile works, it is not because of individual practices, but because as a whole it mimics a market economy with a high level of accuracy:

  • Short iterations, fast customer feedback mimic the markets pricing mechanisms: either customers like what has been built, or they don’t, just as in a market customers will either buy your product at a given price point, or not.
  • Self-organizing teams mimic the autonomous actors in a marketplace: teams self-organizing means people will naturally supply their “services” in a team where they are needed, they find a “market” for their skills within the team, rather than being told by a “socialist” central planning committee what to do (even if it wastes their efforts).
  • YAGNI/simplicity means you do not add in features unless they are “saleable” immediately: doing the simplest thing possible that works means you only do what there is demand for, and what there is the most demand for at any given time. Your “supply” responds to what there is the highest demand for, nothing else.

Short iterations, fast feedback, self-organizing teams and a YAGNI-approach to features are all intrinsically market-based principles, Agile works (most of the time) due to the same reasons that markets work (most of the time).
If you have experienced failure in supposedly Agile projects, ask yourself: was the team REALLY self-organizing? Did management really trust the team to work it out for themselves?
Did you REALLY do frequent releases, getting early and frequent customer feedback?

I would argue that for an Agile project to be successful, following market-based principles is far more important than any individual practice. Agile, just like a market is not infallible, it is still subject to the whims, skill and intelligence of its actors, but all things being equal, market principles trump old school socialist command-and-control hierarchical management of resources any day, be they financial or human skills.