Maxwell’s demon and Shannon’s demon

Named after the legendary physicist James Clerk Maxwell, Maxwell’s demon is a thought experiment in physics in which a hypothetical demon controls a microscopic, frictionless door separating two chambers of gas. The demon selectively sorts fast and slow molecules in the gas to pass through the door so that the fast molecules go to one chamber and the slow molecules go to the other, thereby creating a temperature difference without expending any of the system’s energy. This procedure would lead to an increase in the free energy of the system, which could then be used to do useful work, seemingly violating the second law of thermodynamics. Put differently, the demon extracts useful work from heat, which is the random energy of the molecules, all at no apparent thermodynamic cost to the system. This is magic for a physicist, and like magic, it is impossible. The modern resolution of this paradox is that there is in fact a cost to the system, as the demon with its finite memory must eventually erase its records, and erasing information dissipates at least of heat per erased bit, creating entropy in the process (Landauer 1961; Bennett 1982).

In a nod to Maxwell’s demon, Shannon’s demon, which is named after the founder of information theory, Claude Shannon, is a somewhat analogous thought experiment in investing in which the random up-and-down noise of asset prices (loosely analogous to heat in physics) can seemingly “magically” create wealth by rebalancing one’s portfolio in just the right way so as to maximize the long-run growth rate (Cover 1998). It is as if Shannon’s demon is able to sort profitable outcomes from unprofitable ones. Unlike Maxwell’s demon, however, Shannon’s demon exists in the sense that it is a real, mathematically rigorous strategy that does not violate any laws of physics or finance. The connection to thermodynamics is purely one of loose analogy; the actual concepts are information-theoretic.

The setup

In order to most clearly illustrate the concepts at play, I have necessarily chosen a very simple (and therefore unrealistic) example, in which there are only two assets to invest in: a risk-free asset with a negative interest rate and a risky asset modeled by a coin flip. Nonetheless, the results and machinery I develop here generalize to more realistic settings, though of course there are additional nuances and complications that arise in those cases, such as estimating return probabilities, accounting for transaction costs, and so on. In this post, however, I will ignore such complications in order to focus on the core ideas.

We will think of the risk-free asset as reminiscent of a Swiss bank account: the account is so safe and so desirable that it pays a negative interest rate; hence, its associated gross return satisfies On the other hand, the risky asset, which I’ll model as a coin flip, will be highly volatile in its outcome, where heads comes with a very handsome (gross) return of with probability and tails with a very steep loss (with probability ). In addition, I’ll set , for some , i.e.  is the gross return of one heads-and-tails round trip. Finally, I let denote the fraction of our money that we allocate to the risky asset (hence, is the fraction that we allocate to the risk-free asset). After every period, we rebalance our portfolio to maintain these fractions. If we disallow shorting the risky asset, then we limit ourselves to , and if we disallow leverage (shorting the risk-free asset), we require . Our goal is to choose intelligently so that we make as much money as possible in the long run (or lose the least possible).¹ I’ll let denote the initial wealth.

In what follows, let denote the allocation vector and the random gross returns vector, where takes the value with probability and with probability . Note that corresponds to a portfolio consisting solely of the risk-free asset, and corresponds to a portfolio consisting solely of the risky asset. The portfolio gross return for a particular period is then given by so the wealth after periods is where is the portfolio gross return in period . This multiplicative branching of wealth, starting from the initial wealth , is sketched in Figure 2.

For the specific example below, I’ll set:

1. : a negative interest rate of 3% per period.
2. : a 100% net gain on heads.
3. : it’s a fair coin.
4. : this implies , i.e. a 60% net loss on tails.
5. : initial wealth is normalized to 1.

from coin_flip_with_riskless_asset_model import CoinFlipWithRisklessAssetModel
coin_flip_model = CoinFlipWithRisklessAssetModel(r=0.97, gamma_heads=2.0, p=0.5, alpha=0.80)

The risky asset is a lemon

Obviously the risk-free asset is a lemon, as it bleeds 3% each period. (By “lemon” I simply mean an asset that will almost surely grind its way to a 100% loss over time.) While it is less obvious, the risky asset is also a lemon, which can be seen using Monte Carlo simulations.

import numpy as np
from finlib.ensemble_of_returns_paths import EnsembleOfReturnsPaths # finlib: https://github.com/jordanmckittrick/finlib
from finlib.simulating_performance import crp_performance # finlib: https://github.com/jordanmckittrick/finlib

seed = 12345
rng = np.random.default_rng(seed)
num_paths = 10_000
num_periods = 500

weights_for_risky_asset = np.array([0.0, 1.0]) # 0% in risk-free, 100% in risky
gross_returns_tensor = coin_flip_model.generate_random_gross_returns(rng, num_periods=num_periods, num_paths=num_paths)
perf_risky_matrix = crp_performance(weights_for_risky_asset, gross_returns_tensor)
ensemble_risky = EnsembleOfReturnsPaths.from_gross(perf_risky_matrix)

growth_rate_summary_df_risky = ensemble_risky.summarize_across_paths(ensemble_risky.running_growth_rate, threshold=0.0) 
wealth_summary_df_risky = ensemble_risky.summarize_across_paths(ensemble_risky.running_wealth_ratio, threshold=1.0) 

Figure 3 summarizes 10,000 simulated wealth paths over 500 periods (coin flips) for the risky asset. The shaded region indicates the middle 95% of these wealth paths.

Clearly the typical path through log-wealth space is downward sloping in the long run (top panel of Figure 3). In addition, after only 100 coin flips, we see that fewer than 10% of the paths have made money, and after 250 flips, that fraction has fallen to just over 1%. After all 500 flips, it is essentially zero (bottom panel).

Indeed, the wealth path tends to zero with probability one. To see this, we begin with the observation that since the sequence of random variables is i.i.d., the strong law of large numbers implies that the empirical growth rate, , converges almost surely to the expected growth rate (which I’ll also simply call the “growth rate”): which defines in nats per period. For the risky asset, our weights vector is ; hence, since . Let be any number satisfying . Then, by the definition of almost sure convergence, with probability one there exists some positive integer such that for all , we have . This implies that for we have with probability one, which proves the claim that the risky asset is indeed a lemon.

Note that this result implies that, regardless of the starting allocation, any buy-and-hold strategy for these two assets will, with probability one, tend to a 100% loss in the long run, as the wealth of a portfolio adopting the buy-and-hold strategy is just a convex combination of the wealth of each asset in the portfolio: every buy-and-hold strategy is also a lemon. In order to avoid this fate, we will need to adopt a different strategy. In this post, I will consider the strategy of rebalancing our portfolio to some fixed value of after every period, not allowing to drift with the asset prices. The resulting portfolio is called a constant rebalanced portfolio (CRP). The question then becomes: what is the optimal value of ?

The expected wealth is a mirage

The expected value path for the risky asset, however, tells a very different story from the typical path. After 500 periods, the expected value path (not shown in Figure 3) for the risky asset sits around , which is many orders of magnitude better than both the sampled median and the sampled mean, both of which are shown in the figure. This is because of the small number of rare paths with enough heads to make a killing. In fact, using the independence of the coin flips, we see that the expected wealth after periods diverges to infinity exponentially fast:

Given that a wealth path tends to zero with probability one, the expected value path is a mirage, as the expected value is dominated by a small number of rare events, which are not representative of the typical outcome.

Let’s now consider the probability that the path after periods at least hasn’t lost any money, i.e. that . For this simple model, we can actually compute this probability exactly. Letting denote the number of heads in flips, we have

where the critical fraction 0.569 solves . For , this yields 0.001. On the other hand, we have . So much for the expected value path being representative of the typical outcome.

To further underscore this point, we can compute an upper bound on this same tail probability by applying a Chernoff bound. The actual probability of heads is , while the smallest empirical probability of heads that would yield a non-negative growth rate is . Of all the empirical distributions of heads/tails that would achieve a non-negative growth rate, is the “closest” to the actual distribution in the sense of minimizing Kullback-Leibler divergence; however, it depends on . We can do away with the dependence by observing that replacing with would slightly decrease the KL divergence and thus very slightly weaken the Chernoff bound. Doing so, we find the resulting KL divergence, , to be given by The Chernoff bound then implies that , which decays exponentially in (Cover and Thomas 2006, ch. 11). Figure 5 plots the exact tail probability against this bound.² In what follows, I will drop the “KL” subscript from and simply write for the Kullback-Leibler divergence.

How to salvage a portfolio of lemons

Earlier I invoked the strong law of large numbers to show that the risky asset is a lemon: because its expected growth rate, is negative whenever (for our fair coin), it will almost surely tend towards a 100% loss in the long run, and as I just showed, the probability of not losing money decays exponentially fast. Instead of letting our portfolio consist solely of the risky asset, however, we can instead choose to maximize This maximizer is unique because is a strictly concave function of . If we think of as a function of (since ), we can find the optimal fraction to allocate to the risky asset by taking the derivative with respect to and setting it equal to zero. Doing so, we find that the optimal fraction is given by where I’ve defined , and .

Though Claude Shannon never published any of his ideas on investing, it is widely claimed that he explained an investing idea, which has since become known as “Shannon’s demon”, during a lecture at MIT (Cover 1998). During that lecture, Shannon reportedly considered a two-asset portfolio consisting of cash and a coin flip with , , , and . In that case, we have , and , which implies that the optimal fraction to allocate to the risky asset is . In other words, Shannon’s demon would invest half of its money in cash and half in the coin flip, rebalancing after every flip to maintain this allocation.

We see immediately that is positive (i.e. we invest at least some of our money in the risky asset) if and only if . If , then and we invest all of our money in the risk-free asset. If , then and we would need to short the risky asset in order to maximize our expected growth rate. In our case, we are restricting ourselves to long-only portfolios (i.e. ), so we would simply invest all of our money in the risk-free asset. Finally, if , then we would need to buy the risky asset on margin (i.e. short the risk-free asset), but again, in this post we’re restricting ourselves to long-only portfolios. Since the growth rate diverges to negative infinity at and , we see that the optimal is always in the open interval .

Our formula for can also be written in a form that is perhaps more recognizable: which, when , reduces to the most general form of the Kelly criterion for a single, two-state risky asset (Kelly 1956). In that context, is often referred to as the “edge”. If we not only let but also (implying ), then we recover the Kelly criterion for -to- odds: , or “edge over odds”.

For our specific example, we have . In the more general, multi-asset case, however, we will need to solve problems of this type numerically. At this growth-optimal allocation, the growth rate is bits/period (or about nats/period). This means that the typical path doubles our money every periods – not bad for a portfolio of lemons.

####### FIND THE OPTIMAL CRP STRATEGY #####
opt_result_for_CRP = coin_flip_model.solve_growth_rate_maximization_problem()
f_star = opt_result_for_CRP.x
weights_for_optimal_CRP = np.array([1 - f_star, f_star])
f_star_from_formula = coin_flip_model.growth_optimal_allocation
assert np.isclose(f_star, f_star_from_formula)
assert np.isclose(f_star, 0.38, atol=0.01) # the prose quotes f* ≈ 0.38

Let’s now simulate the performance of a portfolio using this optimal weighting to see whether my theoretical claims hold water. (For the reader more easily convinced by theory than by simulation, one of the canonical papers providing a theoretical justification for maximizing is (Breiman 1961).) Using the same sequences of coin flips, I have simulated the performance of a portfolio using the optimal CRP strategy, as seen in Figure 6. A striking feature of this simulation is that after only roughly 100 periods, about two-thirds of wealth paths are above break-even.

This may seem like magic or sleight of hand, but it most certainly is not. Whereas Maxwell’s demon mechanically sorts incoming molecules by operating a door, Shannon’s demon mechanically rebalances the portfolio by buying and selling assets so that the portfolio weight vector is maintained using the optimal value . This technique, which is sometimes referred to as “volatility harvesting”, is somewhat reminiscent of a mean-reversion strategy, since the “demon” sells the risky asset if it has just gone up and buys it if it has just gone down, though to be clear, the random variables here are i.i.d. – there is no serial correlation to exploit. By rebalancing to the optimal allocation , the expected growth rate is positive, and the strong law of large numbers implies that the empirical growth rate will converge to this expected growth rate almost surely; hence, the wealth path will tend to infinity with probability one.

Figure 7 shows this convergence happening. Each panel shows the same law at work, though with two very different portfolios: a cloud of empirical growth rates approaching its asymptotic values, the spread decaying as . For the risky asset it is bits per period, which almost surely guarantees long-run ruin. For the optimal CRP it is bits per period, and the guarantee reverses: wealth will almost surely tend to infinity. Note, however, that even after 500 periods, a not insignificant proportion (about 17%) of wealth paths for the optimal portfolio are below water.

While it is true that the expected return is a linear function of , the expected growth rate – which is the quantity that actually drives long-run performance (Breiman 1961) – is a strictly concave function of . This feature is readily seen in the bottom panel of Figure 8. The values of that fall between the two black “X” marks in Figure 8 are those that yield a positive expected growth rate, and hence almost surely . Such gambles have been defined elsewhere as “favorable” (Breiman 1961).

The concavity of shows the value of diversification in a new light. In traditional mean-variance portfolio theory, diversification is typically justified as a means of reducing risk (Markowitz 1952).³ However, in the context of maximizing long-run wealth (our goal), diversification emerges from an attempt to maximize the growth rate.

The connection to information theory

Since is the best achievable growth rate, let’s take a closer look at it. We have where and are alternative probabilities of heads and tails, respectively, where I’ve let for notational convenience, and where is the Kullback-Leibler divergence from the model to the actual distribution . Importantly, is the risk-neutral probability distribution for this setup, i.e. the expected gross return of any portfolio under is exactly the risk-free return : which is independent of , a property we will come back to in a moment.

Using our formula for , we see that the optimal portfolio’s excess growth rate over the risk-free asset is then equal to : it is the excess number of nats needed to encode the outcomes drawn from using a code optimized for the risk-neutral distribution . Since with equality if and only if , we see that, at least for the unconstrained investor, there is an edge whenever the actual distribution of returns differs from the risk-neutral distribution. By “edge” in this context, I mean that the excess growth rate is positive: . This definition of edge reduces to the usual definition, namely (implying ), in the context of long-only portfolios. Nonetheless, the general condition is clear: for the unconstrained investor, there is an edge if and only if . Importantly, this tells us that, with probability one, in the long run, such an unconstrained investor will always be able to beat the risk-free asset by an exponential amount whenever the actual distribution of returns differs from the risk-neutral distribution , though for the long-only investor this also requires .⁴

We have discovered a formula for . Let’s now try to find a formula for more generally, i.e. where we no longer require . To do this, notice that there is another probability distribution hiding in plain sight: since for all , if we simply divide by , we see that we have derived a new probability distribution that depends on : Observe that is just a tilted (reweighted state-by-state and renormalized) version of the risk-neutral distribution by the discounted portfolio gross returns , and that it is indeed a probability distribution since , and both terms are non-negative.

We can now derive the key result of this post (in nats): or

We’ve worked in nats throughout; dividing the result by re-expresses this in bits. From this result, we see that the excess growth rate, , is a purely information-theoretic quantity: it is the difference between the KL divergence from the risk-neutral distribution to the actual distribution and the KL divergence from the tilted risk-neutral distribution to the actual distribution . The first term is independent of and determines an upper bound on the excess growth rate, and this upper bound can always be achieved for the unconstrained investor by choosing , which makes . For the unleveraged, long-only investor, the maximum is achieved by choosing so as to minimize subject to the constraint . In either case, we see that a necessary condition for a positive excess growth rate is that the actual distribution of returns differs from the risk-neutral distribution . This is the two-asset form of the horse-race decomposition of gambling growth rates in (Cover and Thomas 2006, ch. 6): the race is the special case where the payoff matrix is diagonal, with the track’s implied probabilities, the bettor’s allocation, and the track take. Finally, we see that it is possible to guarantee exponentially growing wealth from a portfolio of lemons, provided that

where is the set of allowed values of , e.g.  for an unleveraged, long-only investor.

Conclusion

Unlike Maxwell’s demon, Shannon’s demon is, in the sense made clear in this post, real, as it is a mathematically sound strategy for increasing the growth rate of a portfolio, which is what drives long-run performance. More scientifically, the key takeaways from this post are:

1. Long-run performance is determined by the expected growth rate, , not the expected return, .
2. Unlike the expected return, which is a linear function of , the expected growth rate is a concave function of .
3. Since typically we have , diversification is required to maximize the long-run growth of a portfolio. Diversification isn’t simply a means of reducing risk.
4. The excess growth rate for a given investment fraction is the difference between two measures of information: the KL divergence from the risk-neutral distribution to the actual distribution , and the KL divergence from the tilted risk-neutral distribution to the actual distribution .
5. It is necessary for the actual distribution of returns to differ from the risk-neutral distribution in order for the excess growth rate to be positive.
6. The optimal growth rate is achieved when the tilted risk-neutral distribution matches the actual distribution .
7. One can achieve exponentially growing wealth almost surely from a portfolio of lemons provided that can be made sufficiently close to for some feasible .

To make these points, I have had to assume a lot, e.g. I’ve assumed a simple, two-state model in which is known; I’ve assumed is unique; I’ve assumed there are no transaction costs, etc. Importantly, even under these idealized assumptions, we saw that around 17% of wealth paths for the optimal CRP were under water after 500 periods. I have also limited us to CRP strategies where is a fixed constant. Rolling back one or more of these assumptions is a natural next step and could be the subject of one or more future posts.

References