Corrections Are Healthy

Amazing how so many conservatives can easily see the power of darwinian selection in so many contexts, except in the (biological) context for which this selection model was originally formulated...

Cynicism aside, yes, this post does have a good insight. The people and techniques by which the economy is managed do benefit from being changed when their mistakes have serious bad consequences. What doesn't kill you makes you stronger, right?

Fixing the ambiguity in what I said: When the people and techniques by which the economy is managed are changed when their mistakes have serious bad consequences, the economy benefits... not the people and techniques that get thrown out!

Bernardo, it amazes me that you can look at this example of natural selection in economics, yet not realize that it works only because intelligent actors are producing the strategies to which the selection applies.

Well, there's no evidence that it only works because the actors are intelligent.

How intelligent are these investors anyway, often looks to me like they just follow each other like flock animals.

But it would be a fun experiment to mutate/select investment strategies in a Darwinian manner and measure how effective the results are, in dollars.
I expect it would do badly in the long term, since it would be competing against intelligent actors. In the biological case, early simple organisms would only have competed against other early simple organisms. It could be illustrative nonetheless.

Quick search didn't find any experiments like this, but apparently KBC Financial Products uses darwinian software as part of its routine financial modelling.

jez: My impression is that AI has been heavily applied towards financial modeling, with variable results. People don't tend to publish such papers, however, because (a) if it works, you'd rather keep it to yourself and make money, (b) if it doesn't work, that's not a very fun paper to write.

It's not too hard to "demonstrate" evolution in a simulation like that, but the trick is slipping little bits of knowledge into the system before it starts.

Somebody could probably write a natural selection system that will use proven criteria among blue-chip companies. But you'll wait a long, long time for success from a system that considers every type of investment, including Cheez-Its and Zimbabwean dollars. And you'll wait even longer if the system considers all possible data, including the phases of the moon and the per capita pounds of peanuts consumed in America each year.

That sounds silly, right? Who would invest in Cheez-Its? (Or even worse, Zimbabwean dollars!) But that's exactly my point: You already know which investments and criteria are silly, because you're an intelligent agent. You use heuristics to quickly rule out the overwhelming majority of options as inherently unworkable. But a true evolutionary system can't do that. It doesn't know in advance about the shelf-life of Cheez-Its or the government of Zimbabwe. (Unless, of course, you cheat so that you can get fawning press coverage and more research grants for having "proved" evolution.)

This is not to say that these types of thumb-on-the-scale exercises in emergent knowledge aren't a good idea. If they work, that's great! But those results shouldn't be confused with anything that might happened with the emergence of new species.

BB: No one is claiming parity in the timelines involved, simply in the ability of the concept to work and to potentially provide a mechanism by which biological development in our natural world could have occurred. Many conservatives (usually of the social ilk) consider evolution and natural selection in biology to be synonymous with heresy while not doubting or questioning at all the ability of those mechanisms to work in the financial world, and that's what so rightly is amazing to both Bernardo and myself.

Emergence of new species via evolution and natural selection obviously would take *a lot* longer than the relatively insignificant time after which investments are judged to be successful or not.

mark: what matters is number of generations, not clock time. You could conceivably prepare algorithms in a simulated market or using historical market data in a vastly accelerated manner.

BB: Genetic algorithm experiments purposefully don't rule out the "silly" solutions. The selection criteria cull the "silly" ones automatically, that's the whole point.
But there is usually a problem when you have too many inputs. You thought about the algorithms scanning the news (which brings up all kinds of natural language problems and so on); I really only had stock performance history in mind: pure numbers. Even then, the number of independent variables is a worry.
But in fields where there aren't that many variables, keeping the silly options open has yielded unexpected and novel results.

I've been thinking of simpler experiments and my best idea so far is to switch from financial markets to gambling, poker would be interesting. Could test them against each other and against online players, and turn over generations very fast.

It could be that the default strategy of "betting at random" is hard to beat, as it's close to neurotic.

Evolutionary algorithms tend to be pretty useless unless they're severely constrained. It comes down to basic information theory: information is conserved, minus entropy. Using an evolutionary algorithm requires a different sort of (complementary) information than writing procedural code, but no less information. And (as implied by "complementary"), if you know one then you probably know the other. TANSTAAFL.

My MS thesis was based on my research into evolutionary algorithms, and my study led me to strongly believe that natural selection and random mutation aren't sufficient to create new information. Getting interesting results requires a sophisticated selection criteria, and developing that selection criteria is at least as hard as simply writing a procedure to accomplish your goal. Bootstrapping tends to be unworkable due to the inevitable entropy I mentioned earlier.

I have some experience of evolutionary algorithms, but in very constrained systems. Even then it doesn't work amazingly.

In the case of life, physics contributes all sorts of nuance to the selection criteria, although it is easy to suppose that it (physics) is undirected. (The metaphysical question "why is physics the way it is" is much less well understood.)

If we want to use selection to breed useful algorithms, then yes we need to apply a lot of information and intelligence to choosing selection criteria. That is not an issue with life; the selection criteria arrive out of the box. We do not use genetic algorithms to make self-replicating agents, so we take on the burden of tuning criteria, which is a difficult problem.

MW: In your opinion, what is it that impedes those random mutations which would add information from happening? I can think of only one: brittleness, ie a journey from one near-optimal solution to another through gradual mutation entails many generations being far from either optimum. This is the kind of problem you would very clearly see if you attempted to mutate assembly language machine code. (Of course, DNA is clearly a lot less brittle than machine code.)
Do you have other objections?

jez: To believe that biological life is due to evolution is to believe that natural selection and random mutations are transforming information from the environment into biological matter, with some loss due to entropy. There are two problems with that:

1. There doesn't appear to be a lot of information in the universe. There are a very limited number of physical laws that pretty much explain everything. We probably don't know all the laws yet, but I suspect if we did they could be contained in a single book. Cosmic background radiation exhibits almost no information whatsoever, for example.

Most naive experiments with evolutionary algorithms begin with a few simple rules and hope to generate complex results from them, but cannot do so.

2. If there is a lot of information in the universe, where did it come from? This is a different question from "where did all the energy/matter come from?" because the presence of energy and matter does not necessitate the existence of information.

As for your question: random mutations don't net add information. They may add information locally, but every transaction is degraded by entropy so in the net there's a loss of information. As with Brownian motion, the more interactions you have the more the system as a whole tends towards the average. Contrary to the arguments of believers in biological evolution, longer time spans and larger populations actually make biological evolution harder to believe, not easier.

In my experience, information is never present without intelligence. I've never seen an example of a non-intelligent system that gains information over time, and I think the burden of proof lies with anyone who argues that it's possible.