Monday, 31 December 2018

Book review: Why Nations Fail

Acemoglu and Robinson argue against the “ignorance hypothesis” implicit in most development economics - that poor nations remain poor due to bad economic decisions. Rather, they propose, success and failure come down to the quality of political institutions in that country. These institutions need to be:
  • Centralised enough that they can enforce the rule of law, property rights, and a monopoly on violence. 
  • Inclusive, rather than absolutist or extractive. 

Inclusive institutions take input from a wide range of the people they govern, and uphold their rights and liberties, without seizing too much of their wealth. A lack of inclusive institutions leads to many problems:
  • When a high percentage of people’s incomes are taken away (e.g. medieval serfs), that reduces their incentive and ability to invest or innovate.
  • Rulers who prosper by extracting wealth from their populace are primarily concerned with remaining in power rather than improving their countries, and so block “creative destruction” and social mobility. For instance, many absolutist regimes (such as the Ottoman and Russian empires) banned the construction of railways for decades after they became common elsewhere, out of fear that they would lead to instability. 
  • Growth can occur under extractive institutions if they redirect resources towards important sectors - as in the Soviet Union’s space program. However, this growth is fundamentally unsustainable. 
  • Extractive institutions, once set up, are difficult to remove - they often survive regime changes because they are useful to the new rulers. See marketing boards in countries such as Sierra Leone, which were set up by the British to extract money from farmers, but which became even worse after independence. Or the fact that slavery flourished in Africa even after the UK and US banned the slave trade, since slaves were the backbone of agriculture. 
  • By contrast, Acemoglu and Robinson argue, inclusive institutions create a virtuous circle whereby most people want to uphold the rule of law since they benefit so much from it. 

How well does this theory explain historical developments? The authors survey a wide range of countries. In Europe, they discuss two historical events in particular depth - the Black Death and the Glorious Revolution. The first, they argue, was a turning point which distinguished Western Europe from Eastern Europe. In both regions labour became scarce, giving serfs more power, which in Western Europe soon led to the abolition of serfdom overall. In Eastern Europe, however, landowners responded by cracking down on the rights of serfs and entrenching their own power. In this way small initial differences were amplified into disparities which persist to this day. Similarly, while similar parliaments battled similarly authoritarian rulers in France, Spain and England during the 16th century, small differences led to large divergences. England’s monarchs started out less able to monopolise income from its colonies, and unable to raise taxes without parliamentary consent. This led to more income going to merchants and opponents of the monarchy, and gradual erosion of royal power. These trends culminated in the Glorious Revolution, which placed England under the effective rule of a (relatively) pluralistic parliament, and paved the way for its dominance during the Industrial Revolution. Parliament was much less inclined to grant royal monopolies than recent kings (who had relied upon them for income), and much more open to hearing petitions from its subjects.

In Africa, by contrast, political institutions were not centralised enough to take advantage of the Industrial Revolution. Under colonialism, this did start to change, but for the wrong reasons. Firstly, institutions arose to systematically exploit the European demand for slaves (e.g. justice systems which penalised all crimes with slavery). Secondly, when colonial powers wanted to extract revenue from a population without centralised institutions, they would often empower an authoritarian leader who was tasked with collecting taxes. The institutions that formed under these leaders were fundamentally perverted by their extractiveness and lack of accountability.

It’s not just that corrupt institutions take a cut of a pie whose size is determined by economic policies. Rather, they can also nullify otherwise-useful economic reforms. For example, having an independent central bank is usually considered a valuable check on governmental power. But when the governor of the central bank of Sierra Leone criticised the corruption of its dictator in 1980, he was murdered soon afterwards. In cases like this legal changes are useless since real power lies elsewhere. And even national leaders may be hamstrung by the need to appease various power blocs - as in Ghana, where Prime Minister Kofi Busia took the IMF’s advice to implement unpopular currency reforms, and was promptly deposed by the military, who reversed the changes. These seem like paradigmatic demonstrations of why the ignorance hypothesis is a bad explanation for the poverty of states with unstable or corrupt governments - a category which includes almost all very poor states.

Overall, I think the key idea of this book is important and underrated. It’s the sort of claim which is very obvious in hindsight, but which I’d somehow managed to avoid integrating into my worldview until now (although I explored some related ideas in a previous blog post - and in fact Why Nations Fail fits nicely into the conflict/mistake dichotomy I discuss in that post). The historical coverage is also very broad, and I haven’t done justice to it in this summary by a long shot. The main flaw, however, is its presentation of its claims as narratives rather than hypotheses. In doing so, the book made its arguments more dramatic but also borderline unfalsifiable. A few examples which showcase this effect: the claim that Rome becoming politically absolutist was what led to its fall (400-odd years later); the way in which the empowerment of serfs in Western Europe and the disempowerment of serfs in Eastern Europe after the Black Death are both used to argue for the same overall hypothesis; and the description of China’s economic takeoff as primarily driven by political changes. The last is true at one level of description, since it was sparked by Deng Xiaoping’s rise to power. However, since policy changes will very often be accompanied by political shifts, I think more work needs to be done in specifying what it would mean for the authors' political institutions hypothesis to be false. After all, the political institutions which brought Deng Xiaoping to power, and which he used to turn China around, were exactly the same ones which Mao had controlled: the stark differences between the two leaders were very much due to their different ideas. Other Asian tigers are even clearer examples of when new policies, not new institutions or new political systems, were the driving force behind great reductions in poverty.

A similar ambiguity is present in the authors' discussion of culture - while they reject cultural differences as the main explanation for why some nations prosper and others fail, they ignore that culture is critical to the functioning of political institutions. For example, in some countries corruption is common at all levels of government, and bribing officials to get things done is a normal part of life. In others, bribery is totally unacceptable - and those countries are much better off as a result. Yet I doubt that the authors have any principled way of determining whether examples like this one showcase differences in political institutions which support their hypothesis, or differences in culture which rebut it. If they do, it's never explained in the book.

One last thing I’ve been speculating on: in the book, ‘absolutist’ and ‘extractive’ are used nearly interchangeably, because historically, any groups not represented in political processes got screwed over. But is the last century or so the first in which extractive non-absolutist regimes played an important role? Modern Western states are non-absolutist because all adults get to vote. But they’re extractive because they’re so large. If you add up income taxes, VAT, corporate taxes, payroll taxes, property taxes, and all the others, it wouldn’t surprise me if the average person loses more than 50% of their potential income to the government - a burden comparable to that of medieval serfs. It’s true that today we get services for that money - but we also have to contend with mountains of regulations, which I think have a considerable pernicious effect on national wealth. However, this is just a side note - far more important are the ways in which Acemoglu and Robinson's arguments contribute to our understanding of why some nations remain poor, and what can be done to fix that.

Saturday, 24 November 2018

How democracy ends: a review and reevaluation

Last month I attended a talk by David Runciman, the author of a recent book called How Democracy Ends. I was prepared for outrage-stirring and pearl-clutching, but was pleasantly surprised by the quality of his arguments, which I’ve summarised below, along with my own thoughts on these issues. Note, however, that I haven’t read the book itself, and so can’t be confident that I’ve portrayed his ideas faithfully.

Which lessons from history?

Many people have compared recent populist movements with the stirrings of fascism a century ago. And indeed it’s true that a lot of similar rhetoric being thrown around. But Runciman argues that this is one of the least interesting comparisons to be made between these two times. Some things that would be much more surprising to a denizen of the early 20th century:
  • Significant advances in technology
  • Massive transformations in societal demographics
  • Very few changes in our institutions
The last of those is particularly surprising in light of the first two. Parliamentary democracies in the Anglophone world have been governed by the same institutions - and in some cases, even the same parties - for centuries. Continental European democracies were more disrupted by World War 2, but have been very stable since then, despite the world changing in many ways overall. That’s true even for institutions that are probably harmful - consider the persistence of the electoral college in the US, the House of Lords in the UK, the monarchy in Australia (despite their ardent republicanism movement), first-past-the-post voting systems in many countries, and so on. (In fact, Runciman speculates that Americans voted for Trump partly because of how much confidence they had in the durability of their institutions - a confidence which so far seems to have been well-founded.)

So history gives us pretty good evidence for the robustness of democratic institutions to the normal flow of time - but not to exceptional circumstances. In fact, an inability to make necessary changes may well render them more fragile in the face of sharp opposition. If and when pressure mounts, are they going to snap like the democratic institutions of 1930s Europe did? Runciman argues that they won’t, because of the nature of the demographic changes the West has seen. There are three particularly important axes of variation:
  • Wealth: the average person is many times wealthier than they were a century ago, and the middle class is much larger.
  • Education: we’ve gone from only a few percent of people getting tertiary education (and many of the remainder not finishing high school) to nearly 50% of young people being in university in many western countries.
  • Age: in the last century, the median age has risen by over ten years in most western countries.
These three factors are some of the most powerful predictors of behaviour that we have, and so we should take them into account when judging the likelihood of democratic failure. For instance, wealthier and more educated people are much less likely to support populist or extremist groups. But Runciman focuses the most on age, which I think is the correct approach. Wealth is relative - even if people are actually much richer, they can feel poor and angry after a recession (as they did in the 1930s, despite still being many times wealthier than almost all their ancestors). Education may just be correlated with other factors, rather than the actual cause of lasting differences in mindset. But there are pretty clear biological and social reasons to think that the behaviour and priorities of older people are robustly and significantly different from those of younger people. You need only look at the age distribution of violent crime, for example, to see how strong this effect is (although it may have lessened somewhat over recent years, since marriage rates are declining and single men cause more trouble).

In short: the failures of democracy in the 30’s were based on large populations of young men who could be mobilised in anger by militaristic leaders - see for instance the brownshirts in Germany and blackshirts in Italy. But that’s not what the failure of democracy in our time would look like, because that group of people is much smaller now. For better or worse, older populations are less disruptive and more complacent. To see where that might lead, consider Japan: an ageing population which can’t drag itself out of economic torpor, resistant to immigration, dominated for decades by one political party, betting the country's future on using robots to replace the missing workforce.

Changes ahead

During a Q&A after the talk, I pointed out that Japan is very different to western countries in its particularly strong culture of social conformity and stability. Age trends notwithstanding, I have much more difficulty imaging the same quiet tolerance of slow decline occurring in the US or UK. So, given that government institutions are very difficult to change, where will people direct their frustration if lacklustre growth continues in the coming decades?

In response, Runciman raised two possibilities. Firstly, that people will “go around their governments”, finding new domains in which politics is less relevant. We could call this the “Wild West” possibility. Of course, there’s no longer an uncolonised West to explore - but there is the internet, which isn’t democratically run and probably never will be. We already see fewer young men working full-time, because the alternative of spending most of their time gaming has become more appealing. As virtual worlds become even more immersive, it seems plausible that people will begin to care much less about political issues.

One problem with the idea of “going around governments”, though, is that governments are just much bigger now than they used to be. And as technology companies profit from the growing role of the internet, there’ll be pressure for governments to intervene even more to fight inequality. So a second option is a more Chinese approach, with increasingly autocratic Western governments exerting heavy pressure on (and perhaps eventually control over) tech companies.

A more optimistic possibility is for the internet to make democracy more accountable. Runciman invites us to consider Plato’s original argument against direct democracy (in which people vote on individual issues) - that it would lead to rule by the poor, the ignorant, and the young, all of whom necessarily outnumber the wealthy, wise and old. This argument turned out not to apply for representative democracy, since elected representatives tend to be wealthy, educated and old despite their constituents being the opposite. But now it’s inapplicable for a different reason - that although our representatives haven’t changed much, the rest of us are starting to look much more like them. So maybe it’ll become feasible to implement a more direct democracy, facilitated by the internet and modern communication technology. (This still seems like a bad idea to me, though.)

Base rates and complacency

The last section was a little speculative, so let’s take a step back and think about how to make predictions about these sorts of events in general. Runciman’s analysis above provides good reasons not to draw a specific parallel between the rise of fascism last century and recent political events. But it would take extraordinary evidence to exempt us from extrapolating broader historical trends, in particular the fact that states always collapse eventually, and that the base rate for coups and other forms of internal strife is fairly high. Are the extraordinary changes we’ve seen since the industrial revolution sufficient to justify belief in our exceptionalism?

It’s true that since World War 2, almost no wealthy democracies have descended into autocracy or chaos (Turkey and Ireland being two edge cases). It’s also true that, despite widespread political disillusionment, norms against violence have held to a remarkably large degree. But drawing judgements from the historical period “since World War 2” is a classic case of the Texan Sharpshooter’s Fallacy (and possibly also anthropic bias?). In fact, this recent lull should make us skeptical about our ability to evaluate the question objectively, because people are in general very bad at anticipating extreme events that haven't occurred in living memory. I think this is true despite these possibilities being discussed in the media. For example, while there’s a lot of talk about Trump being a potential autocrat, few Americans are responding by stockpiling food or investing in foreign currencies or emigrating. This suggests that hostility towards Trump is driven primarily by partisan politics, rather than genuine concern about democratic collapse. An additional data point in favour of this hypothesis is how easily the Republican political establishment has fallen in line.

Another key question which isn’t often discussed is the nature of modern military culture. Historically, this has been a major factor affecting governmental stability. But, apart from vague intuitions about modern militaries being fairly placid, I find myself remarkably ignorant on this subject, and suspect others are as well. What facts do you know about your country's military, about the character of its commanders or the distribution of power within it, that make you confident that it won't launch a coup if, for example, one of its generals is narrowly defeated in a disputed presidential election (as in Gore vs Bush)? Note that military demographics haven’t changed nearly as much as those of our societies overall. They’re still primarily composed of young working-class men without degrees - a group that’s unusually angry about today’s politics. So while I am pretty convinced by Runciman’s arguments, this is one way in which they may not apply. Also consider that warfare is much less hands-on than it used to be, and firepower much more centrally concentrated, both of which make coups easier.

And what about extreme events?

So far I've looked at societal collapse from a political point of view. But many historical transitions were precipitated by natural disasters or diseases. See, for instance, the Mayan collapse, or the Little Ice Age, or the Black Death. Today, we're much safer from natural disasters, both because of our technology and because of the scale of our societies - few people live in countries in which the majority of a population can be struck by a single natural disaster. Similarly, we're also much safer from natural diseases. But we're much more vulnerable to severe man-made disasters, which I think are very likely to occur over the next century. Since this post is focused on political collapse as a distinct phenomenon to technological disaster, I won’t discuss extreme risks from technology here. However, it's worthwhile to look at the ways in which smaller technological harms might exacerbate other trends. AI-caused unemployment and the more general trend towards bimodal outcomes in western countries are likely to cause social unrest. Meanwhile terrorism is going to become much easier - consider being able to 3D-print assassin drones running facial recognition software, for instance. And due to antibiotic overuse, it's likely that our safety from disease will decline over the coming years (even without the additional danger of bioterrorism using engineered diseases). Finally, I think we're much softer than we used to be - it won't take nearly as much danger to disrupt a country. Runciman is probably correct that we’re less susceptible to a collapse into authoritarianism than we were in the past - but the same trends driving that change are also pushing us towards new reasons to worry.


In addition to the talk by Runciman, this post was inspired by discussions with my friends Todor and Julio, and benefited from their feedback.

Wednesday, 10 October 2018

Some cruxes on impactful alternatives to AI policy work

Cross-posted to Less Wrong.

Ben Pace and I (Richard Ngo) recently did a public double crux at the Berkeley REACH on how valuable it is for people to go into AI policy and strategy work: I was optimistic and Ben was pessimistic. During the actual event, we didn't come anywhere near to finding a double crux on that issue. But after a lot of subsequent discussion, we've come up with some more general cruxes about where impact comes from.

I found Ben's model of how to have impact very interesting, and so in this post I've tried to explain it, along with my disagreements. Ben liked the goal of writing up a rough summary of our positions and having further discussion in the comments, so while he edited it somewhat he doesn’t at all think that it’s a perfect argument, and it’s not what he’d write if he spent 10 hours on it. He endorsed the wording of the cruxes as broadly accurate.

(During the double crux, we also discussed how the heavy-tailed worldview applies to community building, but decided on this post to focus on the object level of what impact looks like.)

Note from Ben: “I am not an expert in policy, and have not put more than about 20-30 hours of thought into it total as a career path. But, as I recently heard Robin Hanson say, there’s a common situation that looks like this: Some people have a shiny idea that they think about a great deal and work through the details of, that folks in other areas are skeptical of given their particular models of how the world works. Even though the skeptics have less detail, it can be useful to publicly say precisely why they’re skeptical.

In this case I’m often skeptical when folks tell me they’re working to reduce x-risk by focusing on policy. Folks doing policy work in AI might be right, and I might be wrong, but it seemed like a good use of time to start a discussion with Richard about how I was thinking about it and what would change my mind. If the following discussion causes me to change my mind on this question, I’ll be really super happy with it.”

Ben's model: Life in a heavy-tailed world

A heavy-tailed distribution is one where the probability of extreme outcomes doesn’t drop very rapidly, meaning that outliers therefore dominate the expectation of the distribution. Owen Cotton-Barratt has written a brief explanation of the idea here. Examples of heavy-tailed distributions include the Pareto distribution and the log-normal distribution; other phrases people use to point at this concept include ‘power laws’ (see Zero to One) and ‘black swans’ (see the recent SSC book review). Wealth is a heavy-tailed distribution, because many people are clustered relatively near the median, but the wealthiest people are millions of times further away. Human height and weight and running speed are not heavy-tailed; there is no man as tall as 100 people.

There are three key claims that make up Ben's view.

The first claim is that, since the industrial revolution, we live in a world where the impact that small groups can have is much more heavy-tailed than in the past.
  • People can affect incredibly large numbers of other people worldwide. The Internet is an example of a revolutionary development which allows this to happen very quickly.
  • Startups are becoming unicorns unprecedentedly quickly, and their valuations are very heavily skewed.
  • The impact of global health interventions is heavy-tail distributed. So is funding raised by Effective Altruism - two donors have contributed more money than everyone else combined.
  • Google and Wikipedia qualitatively changed how people access knowledge; people don't need to argue about verifiable facts any more.
  • Facebook qualitatively changed how people interact with each other (e.g. FB events is a crucial tool for most local EA groups), and can swing elections.
  • It's not just that we got more extreme versions of the same things, but rather that we can get unforeseen types of outcomes.
  • The books HPMOR and Superintelligence both led to mass changes in plans towards more effective ends via the efforts of individuals and small groups.

The second claim is that you should put significant effort into re-orienting yourself to use high-variance strategies.
  • Ben thinks that recommending strategies which are safe and low-risk is insane when pulling out of a heavy-tailed distribution. You want everyone to be taking high-variance strategies.
  • This is only true if the tails are long to the right and not to the left, which seems true to Ben. Most projects tend to end up not pulling any useful levers whatever and just do nothing, but a few pull crucial levers and solve open problems or increase capacity for coordination.
  • Your intuitions were built for the ancestral environment where you didn’t need to be able to think about coordinating humans on the scale of millions or billions, and yet you still rely heavily on the intuitions you’re built with in navigating the modern environment.
  • Scope insensitivity, framing effects, taboo tradeoffs, and risk aversion, are the key things here. You need to learn to train your S1 to understand math.
  • By default, you’re not going to spend enough effort finding or executing high-variance strategies.
  • We're still only 20 years into the internet era. Things keep changing qualitatively, but Ben feels like everyone keeps adjusting to the new technology as if it were always this way.
  • Ben: “My straw model of the vast majority of people’s attitudes is: I guess Facebook and Twitter are just things now. I won’t spend time thinking about whether I could build a platform as successful as those two but optimised better for e.g. intellectual progress or social coordination - basically not just money.”
  • Ben: “I do note that never in history has change been happening so quickly, so it makes sense that people’s intuitions are off."
  • While many institutions have been redesigned to fit the internet, Ben feels like almost nobody is trying to improve institutions like science on a large scale, and that this is clear low-hanging altruistic fruit.
  • The Open Philanthropy Project has gone through this process of updating away from safe, low-risk bets with GiveWell, toward hits-based giving, which is an example of this kind of move.

The third claim is that AI policy is not a good place to get big wins nor to learn the relevant mindset.
  • Ben: “On a first glance, governments, politics and policy looks like the sort of place where I would not expect to find highly exploitable strategies, nor a place that will teach me the sorts of thinking that will help me find them in future.”
  • People in policy spend a lot of time thinking about how to influence governments. But governments are generally too conventional and slow to reap the benefits of weird actions with extreme outcomes.
  • Working in policy doesn't cultivate the right type of thinking. You're usually in a conventional governmental (or academic) environment, stuck inside the system, getting seduced by local incentive gradients and prestige hierarchies. You often need to spend a long time working your way to positions of actual importance in the government, which leaves you prone to value drift or over-specialisation in the wrong thing.
  • At the very least, you have to operate on the local incentives as well as someone who actually cares about them, which can be damaging to one’s ability to think clearly.
  • Political landscapes are not the sort of environment where people can easily ignore the local social incentives to focus on long-term, global goals. Short term thinking (election cycles, media coverage, etc) is not the sort of thinking that lets you build a new institution over 10 years or more.
  • Ben: “When I’ve talked to senior political people, I’ve often heard things of the sort ‘We were working on a big strategy to improve infrastructure / international aid / tech policy etc, but then suddenly public approval changed and then we couldn’t make headway / our party wasn’t in power / etc.’ which makes me think long term planning is strongly disincentivised.”
  • One lesson of a heavy-tailed world is that signals that you’re taking safe bets are anti-signals of value. Many people following a standard academic track saying “Yeah, I’m gonna get a masters in public policy” sounds fine, sensible, and safe, and therefore cannot be an active sign that you will do something a million times more impactful than the median.
  • The above is not a full, gears-level analysis of how to find and exploit a heavy tail, because almost all of the work here lies in identifying the particular strategy. Nevertheless, because of the considerations above, Ben thinks that talented, agenty and rational people should be able in many cases to identify places to win, and then execute those plans, and that this is much less the case in policy.

Richard's model: Business (mostly) as usual.

I disagree with Ben on all three points above, to varying degrees.

On the first point, I agree that the distribution of success has become much more heavy-tailed since the industrial revolution. However, I think the distribution of success is often very different from the distribution of impact, because of replacement effects. If Facebook hadn't become the leading social network, then MySpace would have. If not Google, then Yahoo. If not Newton, then Leibniz (and if Newton, then Leibniz anyway). Probably the alternatives would have been somewhat worse, but not significantly so (and if they were, different competitors would have come along). The distinguishing trait of modernity is that even a small difference in quality can lead to a huge difference in earnings, via network effects and global markets. But that isn't particularly interesting from an x-risk perspective, because money isn't anywhere near being our main bottleneck.

You might think that since Facebook has billions of users, their executives are a small group with a huge amount of power, but I claim that they're much more constrained by competitive pressures than they seem. Their success depends on the loyalty of their users, but the bigger they are, the easier it is for them to seem untrustworthy. They also need to be particularly careful since antitrust cases have busted the dominance of several massive tech companies before. (While they could swing a few elections before being heavily punished, I don’t think this is unique to the internet age - a small cabal of newspaper owners could probably have done the same centuries ago). Similarly, I think the founders of Wikipedia actually had fairly little counterfactual impact, and currently have fairly little power, because they're reliant on editors who are committed to impartiality.

What we should be more interested in is cases where small groups didn't just ride a trend, but actually created or significantly boosted it. Even in those cases, though, there's a big difference between success and impact. Lots of people have become very rich from shuffling around financial products or ad space in novel ways. But if we look at the last fifty years overall, they're far from dominated by extreme transformative events - in fact, Western societies have changed very little in most ways. Apart from IT, our technology remains roughly the same, our physical surroundings are pretty similar, and our standards of living have stayed flat or even dropped slightly. (This is a version of Tyler Cowen and Peter Thiel's views; for a better articulation, I recommend The Great Stagnation or The Complacent Class). Well, isn't IT enough to make up for that? I think it will be eventually, as AI develops, but right now most of the time spent on the internet is wasted. I don't think current IT has had much of an effect by standard metrics of labour productivity, for example.

Should you pivot?

Ben might claim that this is because few people have been optimising hard for positive impact using high-variance strategies. While I agree to some extent, I also think that there are pretty strong incentives to have impact regardless. We're in the sort of startup economy where scale comes first and monetisation comes second, and so entrepreneurs already strive to create products which influence millions of people even when there’s no clear way to profit from them. And entrepreneurs are definitely no strangers to high-variance strategies, so I expect most approaches to large-scale influence to already have been tried.

On the other hand, I do think that reducing existential risk is an area where a small group of people are managing to have a large influence, a claim which seems to contrast with the assertion above. I’m not entirely sure how to resolve this tension, but I’ve been thinking lately about an analogy from finance. Here’s Tyler Cowen:
"I see a lot of money managers, so there’s Ray Dalio at Bridgewater. He saw one basic point about real interest rates, made billions off of that over a great run. Now it’s not obvious he and his team knew any better than anyone else.
Peter Lynch, he had fantastic insights into consumer products. Use stuff, see how you like it, buy that stock. He believed that in an age when consumer product stocks were taking off.
Warren Buffett, a certain kind of value investing. Worked great for a while, no big success, a lot of big failures in recent times."
The analogy isn’t perfect, but the idea I want to extract is something like: once you’ve identified a winning strategy or idea, you can achieve great things by exploiting it - but this shouldn’t be taken as strong evidence that you can do exceptional things in general. For example, having a certain type of personality and being a fan of science fiction is very useful in identifying x-risk as a priority, but not very useful in founding a successful startup. Similarly, being a philosopher is very useful in identifying that helping the global poor is morally important, but not very useful in figuring out how to solve systemic poverty.

From this mindset, instead of looking for big wins like “improving intellectual coordination”, we should be looking for things which are easy conditional on existential risk actually being important, and conditional on the particular skillsets of x-risk reduction advocates. Another way of thinking about this is as a distinction between high-impact goals and high-variance strategies: once you’ve identified a high-impact goal, you can pursue it without using high-variance strategies. Startup X may have a crazy new business idea, but they probably shouldn't execute it in crazy new ways. Actually, their best bet is likely to be joining Y Combinator, getting a bunch of VC funding, and following Paul Graham's standard advice. Similarly, reducing x-risk is a crazy new idea for how to improve the world, but it's pretty plausible that we should pursue it in ways similar to those which other successful movements used. Here are some standard things that have historically been very helpful for changing the world:
  • dedicated activists
  • good research
  • money
  • public support
  • political influence
My prior says that all of these things matter, and that most big wins will be due to direct effects on these things. The last two are the ones which we’re disproportionately lacking; I’m more optimistic about the latter for a variety of reasons.

AI policy is a particularly good place to have a large impact.

Here's a general argument: governments are very big levers, because of their scale and ability to apply coercion. A new law can be a black swan all by itself. When I think of really massive wins over the past half-century, I think about the eradication of smallpox and polio, the development of space technology, and the development of the internet. All of these relied on and were driven by governments. Then, of course, there are the massive declines in poverty across Asia in particular. It's difficult to assign credit for this, since it's so tied up with globalisation, but to the extent that any small group was responsible, it was Asian governments and the policies of Deng Xiaoping, Lee Kuan Yew, Rajiv Gandhi, etc.

You might agree that governments do important things, but think that influencing them is very difficult. Firstly, that's true for most black swans, so I don't think that should make policy work much less promising even from Ben's perspective. But secondly, from the outside view, our chances are pretty good. We're a movement comprising many very competent, clever and committed people. We've got the sort of backing that makes policymakers take people seriously: we're affiliated with leading universities, tech companies, and public figures. It's likely that a number of EAs at the best universities already have friends who will end up in top government positions. We have enough money to do extensive lobbying, if that's judged a good idea. Also, we're correct, which usually helps. The main advantage we're missing is widespread popular support, but I don't model this as being crucial for issues where what's needed is targeted interventions which "pull the rope sideways". (We're also missing knowledge about what those interventions should be, but that makes policy research even more valuable).

Here's a more specific route to impact: in a few decades (assuming long timelines and slow takeoff) AIs that are less generally intelligent that humans will be causing political and economic shockwaves, whether that's via mass unemployment, enabling large-scale security breaches, designing more destructive weapons, psychological manipulation, or something even less predictable. At this point, governments will panic and AI policy advisors will have real influence. If competent and aligned people were the obvious choice for those positions, that'd be fantastic. If those people had spent several decades researching what interventions would be most valuable, that'd be even better.

This perspective is inspired by Milton Friedman, who argued that the way to create large-scale change is by nurturing ideas which will be seized upon in a crisis.
"Only a crisis - actual or perceived - produces real change. When that crisis occurs, the actions that are taken depend on the ideas that are lying around. That, I believe, is our basic function: to develop alternatives to existing policies, to keep them alive and available until the politically impossible becomes the possible."
The major influence of the Institute of Economic Affairs on Thatcher’s policies is an example of this strategy’s success. An advantage of this approach is that it can be implemented by clusterings of like-minded people collaborating with each other; for that reason, I'm not so worried about policy work cultivating the wrong mindset (I'd be more worried on this front if policy researchers were very widely spread out).

Another fairly specific route to impact: several major AI research labs would likely act on suggestions for coordinating to make AI safer, if we had any. Right now I don’t think we do, and so research into that could have a big multiplier. If a government ends up running a major AI lab (which seems pretty likely conditional on long timelines) then they may also end up following this advice, via the effect described in the paragraph above.

Underlying generators of this disagreement

More generally, Ben and I disagree on where the bottleneck to AI safety is. I think that finding a technical solution is probable, but that most solutions would still require careful oversight, which may or may not happen (maybe 50-50). Ben thinks that finding a technical solution is improbable, but that if it's found it'll probably be implemented well. I also have more credence on long timelines and slow takeoffs than he does. I think that these disagreements affect our views on the importance of influencing governments in particular.

We also have differing views on what the x-risk reduction community should look like. I favour a broader, more diverse community; Ben favours a narrower, more committed community. I don't want to discuss this extensively here, but I will point out that there are many people who are much better at working within a system than outside it - people who would do well in AI safety PhDs, but couldn't just teach themselves to do good research from scratch like Nate Soares did; brilliant yet absent-minded mathematicians; people who could run an excellent policy research group but not an excellent startup. I think it's valuable for such people (amongst which I include myself), to have a "default" path to impact, even at the cost of reducing the pressure to be entrepreneurial or agenty. I think this is pretty undeniable when it comes to technical research, and cross-applies straightforwardly to policy research and advocacy.

Ben and I agree that going into policy is much more valuable if you're thinking very strategically and out of the "out of the box" box than if you're not. Given this mindset, there will probably turn out to be valuable non-standard things which you can do.

Do note that this essay is intrinsically skewed since I haven't portrayed Ben's arguments in full fidelity and have spent many more words arguing my side. Also note that, despite being skeptical about some of Ben's points, I think his overall view is important and interesting and more people should be thinking along similar lines.

Thanks to Anjali Gopal for comments on drafts.

Saturday, 15 September 2018

Realism about rationality

Epistemic status: trying to vaguely gesture at vague intuitions.

Cross-posted to Less Wrong, where there's been some good discussion. A similar idea was explored here under the heading "the intelligibility of intelligence", although I hadn't seen it before writing this post.

There’s a mindset which is common in the rationalist community, which I call “realism about rationality” (the name being intended as a parallel to moral realism). I feel like my skepticism about agent foundations research is closely tied to my skepticism about this mindset, and so in this essay I try to articulate what it is.

Humans ascribe properties to entities in the world in order to describe and predict them. Here are three such properties: "momentum", "evolutionary fitness", and "intelligence". These are all pretty useful properties for high-level reasoning in the fields of physics, biology and AI, respectively. There's a key difference between the first two, though. Momentum is very amenable to formalisation: we can describe it using precise equations, and even prove things about it. Evolutionary fitness is the opposite: although nothing in biology makes sense without it, no biologist can take an organism and write down a simple equation to define its fitness in terms of more basic traits. This isn't just because biologists haven't figured out that equation yet. Rather, we have excellent reasons to think that fitness is an incredibly complicated "function" which basically requires you to describe that organism's entire phenotype, genotype and environment.

In a nutshell, then, realism about rationality is a mindset in which reasoning and intelligence are more like momentum than like fitness. It's a mindset which makes the following ideas seem natural:
  • The idea that there is a simple yet powerful theoretical framework which describes human intelligence and/or intelligence in general. (I don't count brute force approaches like AIXI for the same reason I don't consider physics a simple yet powerful description of biology).
  • The idea that there is an “ideal” decision theory.
  • The idea that AGI will very likely be an “agent”.
  • The idea that Turing machines and Kolmogorov complexity are foundational for epistemology.
  • The idea that, given certain evidence for a proposition, there's an "objective" level of subjective credence which you should assign to it, even under computational constraints.
  • The idea that morality is quite like mathematics, in that there are certain types of moral reasoning that are just correct.
  • The idea that defining coherent extrapolated volition in terms of an idealised process of reflection roughly makes sense, and that it converges in a way which doesn’t depend very much on morally arbitrary factors.
  • The idea that having having contradictory preferences or beliefs is really bad, even when there’s no clear way that they’ll lead to bad consequences (and you’re very good at avoiding dutch books and money pumps and so on).

To be clear, I am neither claiming that realism about rationality makes people dogmatic about such ideas, nor claiming that they're all false. In fact, from a historical point of view I’m quite optimistic about using maths to describe things in general. But starting from that historical baseline, I’m inclined to adjust downwards on questions related to formalising rational thought, whereas rationality realism would endorse adjusting upwards. This essay is primarily intended to explain my position, not justify it, but one important consideration for me is that intelligence as implemented in humans and animals is very messy, and so are our concepts and inferences, and so is the closest replica we have so far (intelligence in neural networks). It's true that "messy" human intelligence is able to generalise to a wide variety of domains it hadn't evolved to deal with, which supports rationality realism, but analogously an animal can be evolutionarily fit in novel environments without implying that fitness is easily formalisable.

Another way of pointing at rationality realism: suppose we model humans as internally-consistent agents with beliefs and goals. This model is obviously flawed, but also predictively powerful on the level of our everyday lives. When we use this model to extrapolate much further (e.g. imagining a much smarter agent with the same beliefs and goals), or base morality on this model (e.g. preference utilitarianism), is that more like using Newtonian physics to approximate relativity (works well, breaks down in edge cases) or more like cavemen using their physics intuitions to reason about space (a fundamentally flawed approach)?

Another gesture towards the thing: a popular metaphor for Kahneman and Tversky's dual process theory is a rider trying to control an elephant. Implicit in this metaphor is the localisation of personal identity primarily in the system 2 rider. Imagine reversing that, so that the experience and behaviour you identify with are primarily driven by your system 1, with a system 2 that is mostly a Hansonian rationalisation engine on top (one which occasionally also does useful maths). Does this shift your intuitions about the ideas above, e.g. by making your coherent extrapolated volition feel less well-defined? I claim that the latter perspective is just as sensible, and perhaps even more so - see, for example, Paul Christiano's model of the mind, which leads him to conclude that "imagining conscious deliberation as fundamental, rather than a product and input to reflexes that actually drive behavior, seems likely to cause confusion."

These ideas have been stewing in my mind for a while, but the immediate trigger for this post was a conversation about morality which went along these lines:

R (me): Evolution gave us a jumble of intuitions, which might contradict when we extrapolate them. So it’s fine to accept that our moral preferences may contain some contradictions.
O (a friend): You can’t just accept a contradiction! It’s like saying “I have an intuition that 51 is prime, so I’ll just accept that as an axiom.”
R: Morality isn’t like maths. It’s more like having tastes in food, and then having preferences that the tastes have certain consistency properties - but if your tastes are strong enough, you might just ignore some of those preferences.
O: For me, my meta-level preferences about the ways to reason about ethics (e.g. that you shouldn’t allow contradictions) are so much stronger than my object-level preferences that this wouldn’t happen. Maybe you can ignore the fact that your preferences contain a contradiction, but if we scaled you up to be much more intelligent, running on a brain orders of magnitude larger, having such a contradiction would break your thought processes.
R: Actually, I think a much smarter agent could still be weirdly modular like humans are, and work in such a way that describing it as having idealised “beliefs” is still a very lossy approximation. And it’s plausible that there’s no canonical way to “scale me up”.

I had a lot of difficulty in figuring out what I actually meant during that conversation, but I think a quick way to summarise the disagreement is that O is a rationality realist, and I’m not. This is not a problem, per se: I'm happy that some people are already working on AI safety from this mindset, and I can imagine becoming convinced that rationality realism is a more correct mindset than my own. But I think it's a distinction worth keeping in mind, because assumptions baked into underlying worldviews are often difficult to notice, and also because the rationality community has selection effects favouring this particular worldview even though it doesn't necessarily follow from the community's founding thesis (that humans can and should be more rational).

Tuesday, 4 September 2018

I'm so meta, even this acronym

This is my 52nd blog post within the span of one calendar year, since I (re)started blogging on September 5th, 2017. My writing productivity has far exceeded my expectations, and I'm very happy about managing to explore so many ideas. Here are some metrics.

Breakdown by topic:
  • 10 posts on computer science, machine learning, and maths
  • 10 posts on philosophy
  • 7 posts on politics and economics
  • 6 posts on modern life and the future of society
  • 5 posts on history and geography
  • 3 posts on intelligence
  • 12 miscellaneous posts (including this one)

Most popular (in order):
  1. In search of All Souls
  2. Is death bad?
  3. What have been the greatest intellectual achievements?
  4. Utilitarianism and its discontents
  5. The unreasonable effectiveness of deep learning
  6. Yes, you should be angry about the housing crisis
  7. Oxford vs Cambridge

Longest posts (in order of word count):
  1. A brief history of India. 5915
  2. Proof, computation and truth. 5627
  3. Utilitarianism and its discontents. 5550
  4. Yes, you should be angry about the housing crisis. 5291
  5. The unreasonable effectiveness of deep learning. 3887
  6. In search of All Souls. 3873
  7. Which neural network architectures perform best at sentiment analysis? 3291

Word clouds:

Daily users (since I signed up for Google analytics):
 I really need a better way of getting readers than just the spikes from sharing on facebook.

Overall word count: 92,808 words (as a comparison, the first Harry Potter book was 76,944 words; The Hobbit was 95,022 words). This is over double the 44,816 words of material (excluding academic essays; most of it found here) which I'd written over the previous few years. Also note that this is a significant underestimate of how much I've actually written this last year, because I have at least a dozen drafts on the go at any one time, and right now also an extra half-dozen which I've written during my internship and am still mulling over.

However, I've been thinking lately that the focus for the next year will be on quality rather than quantity. This year has been fantastic in terms of intellectual exploration, but I'm not sure that any of my posts would actually contribute robustly valuable knowledge to people who already know about the topic. By contrast, I think my friends Jacob and Tom have blog posts which do so, because they're more careful and thorough. So I'd like to move in that direction a little more.

On the other hand, this blog is useful to me in many ways even if it doesn't make novel intellectual contributions. Probably the biggest is the fact that I started seriously reading current machine learning research while I was writing summaries of key ideas in deep learning. Without that, I wouldn't have learned nearly as much and may well not have gotten either my current internship or my upcoming job. Meanwhile, writing book reviews pushes me to read and understand books more thoroughly. In general, I'm glad to have something which feels like a tangible and permanent record of my personal intellectual progress, because I do worry about losing touch with my past self. Now I can be much more confident that my future self won't lose touch with me.

A compendium of conundrums

Logic puzzles

None of the puzzles below have trick answers - they can all be solved using logic and a bit of maths. Whenever a group of people need to achieve a task, assume they're allowed to confer and come up with a strategy beforehand. They're listed roughly in order of difficulty. Let me know of any other good ones you find!

Two ropes
I have two ropes which each, if lighted at one end, takes 1 hour to burn all the way to the other end. However, they burn at variable rates (e.g. the first might take 55 minutes to burn 1/4 of the way, then 5 minutes to burn all the rest; the second might be the opposite). How do I use them to time 45 minutes?

25 horses
I have 25 horses, and am trying to find the 3 fastest. I have no timer, but can race 5 at a time against each other; I know that a faster horse will always beat a slower horse. How many races do I need to find the 3 fastest, in order?

Monty hall problem (explanation taken from here)
The set of Monty Hall's game show Let's Make a Deal has three closed doors. Behind one of these doors is a car; behind the other two are goats. The contestant does not know where the car is, but Monty Hall does. The contestant picks a door and Monty opens one of the remaining doors, one he knows doesn't hide the car. If the contestant has already chosen the correct door, Monty is equally likely to open either of the two remaining doors. After Monty has shown a goat behind the door that he opens, the contestant is always given the option to switch doors. Is it advantageous to do so, or disadvantageous, or does it make no difference?

Four-way duel
A, B, C and D are in a duel. In turn (starting with A) they each choose one person to shoot at, until all but one have been eliminated. They hit their chosen target 0%, 33%, 66% and 100% of the time, respectively. A goes first, and of course misses. It's now B's turn. Who should B aim at, to maximise their probability of winning?

Duck in pond
A duck is in a circular pond with a menacing cat outside. The cat runs four times as fast as the duck can swim, and always runs around the edge of the pond in whichever direction will bring it closest to the duck, but cannot enter the water. As soon as the duck reaches the shore it can fly away, unless the cat is already right there. Can the duck escape?

Non-transitive dice
Say that a die A beats another die B if, when both rolled, the number on A is greater than the number on B more than 50% of the time. Is it possible to design three dice A, B and C such that A beats B, B beats C and C beats A?

Wine tasting
A king has 100 bottles of wine, exactly one of which is poisoned. He decides to figure out which it is by feeding the wines to some of his servants, and seeing which ones drop dead. He wants to find out before the poisoner has a chance to get away, and so he doesn't have enough time to do this sequentially - instead he plans to give each servant some combination of the wines tonight, and see which are still alive tomorrow morning.
a) How many servants does he need?
b) Suppose he had 100 servants - then how many wines could he test?

Crawling on the planet's face
Two people are dropped at random places on a featureless spherical planet (by featureless I also mean that there are no privileged locations like poles). Assume that each person can leave messages which the other might stumble across if they come close enough (within a certain fixed distance).
a) How can they find each other for certain?
b) How can they find each other in an amount of time which scales linearly with the planet's radius?

Dropping coconuts
I have two identical coconuts, and am in a 100-floor building; I want to figure out the highest floor I can drop them from without them breaking. Assume that the coconuts aren't damaged at all by repeated drops from below that floor - but once one is broken, I can't use it again.
a) What's the smallest number of drops I need, in the worst case, to figure that out?
b) Can you figure out an equation for the general case, in terms of number of coconuts and number of floors?

Pirate treasure
There are 5 pirates dividing up 100 gold coins in the following manner. The most senior pirate proposes a division (e.g. "99 for me, 1 for the next pirate, none for the rest of you"). All pirates then vote on this division. If a majority vote no, then the most senior pirate is thrown overboard, and the next most senior pirate proposes a division. Otherwise (including in the case of ties) the coins are split up as proposed. All pirates are entirely selfish, and have common knowledge of each other's perfect rationality.
a) What will the most senior pirate propose?
b) What about if there are 205 pirates?
c) Can you figure out a solution for the general case, in terms of number of coins and number of pirates?

Self-sorting
There are n people, all wearing black or white hats. Each can see everyone else's hat colour, but not their own. They have to sort themselves into a line with all the white hats on one end and all the black hats on the other, but are not allowed to communicate about hat colours in any way. How can they do it?

Knights and knaves
You are travelling along a road and come to a fork, where a guardian stands in front of each path. A sign tells you that one guardian only speaks the truth, and one only speaks lies; also, one road goes to Heaven, and ones goes to Hell. You are able to ask yes/no questions (each directed to only one of the guardians) to figure out which is which.
a) Can you figure it out using two questions?
b) How about one?

What is the name of this god? (explanation taken from here)
Three gods A, B, and C are called, in no particular order, True, False, and Random. True always speaks truly, False always speaks falsely, but whether Random speaks truly or falsely is a completely random matter. Your task is to determine the identities of A, B, and C by asking three yes/no questions; each question must be put to exactly one god. The gods understand English, but will answer all questions in their own language, in which the words for yes and no are da and ja, in some order. You do not know which word means which.

A game of greed
You have a pile of n chips, and play the following two-player game. The first player takes some chips, but not all of them. After that players alternate taking chips; the only rule is that you cannot take more than the previous player did. The person who takes the last chip wins. Is it the first player or the second player who has a winning strategy, and what is it?

Heat-seeking missiles
Four heat-seeking missiles are placed at the corners of a square with side length 1. Each of them flies directly towards the missile on its left at a constant speed. How far does each travel before collision? (Assume they're ideal points which only "collide" when right on top of each other).

Blind maze
You're located within a finite square maze. You do not know how large it is, where you are, or where the walls or exit are. At each step you can move left, right, up or down; if there's a wall in the given direction, then you don't go anywhere (but you don't get any feedback telling you that you bumped into it). Is there a sequence of steps you can take to ensure that you will eventually find the exit?

Hats in lines
There are 100 prisoners in a line, facing forwards. Each is wearing a black or white hat, and can see the hat colour of everyone in front of them, but not their own or that of anyone behind them; also, they don't know the total number of hats of each colour. Starting from the back of the line, each person is allowed to say either "black" or "white", and is set free if they correctly say the colour of their hat, but shot otherwise. Everyone in the line can hear every answer, and whether or not they were shot afterwards.
a) How many people can be saved for certain, and using what strategy?
b) Suppose that the number of prisoners is countably infinite (i.e. in correspondence with the natural numbers, with number 1 being at the back). How can they save all but one?
c) Suppose that the number of prisoners is countably infinite, and none of them can hear the answers of the other prisoners. How can they save all but finitely many?

Prisoners and hats
Seven prisoners are given the chance to be set free tomorrow. An executioner will put a hat on each prisoner's head. Each hat can be one of the seven colors of the rainbow and the hat colors are assigned completely at the executioner's discretion. Every prisoner can see the hat colors of the other six prisoners, but not his own. They cannot communicate with others in any form, or else they are immediately executed. Then each prisoner writes down his guess of his own hat color. If at least one prisoner correctly guesses the color of his hat, they all will be set free immediately; otherwise they will be executed. Is there a strategy that they can use which guarantees that they will be set free?

Prisoners and switch
There are 100 immortal prisoners in solitary confinement, whose warden decides to play a game with them. Each day, one will be chosen at random and taken into an empty room with a switch on the wall. The switch can be in the up position or the down position, but isn't connected to anything. The prisoner is allowed to change the switch position if they want, and is then taken back to their cell; the switch will then remained unchanged until the next prisoner comes in. The other prisoners don't know who is chosen each day, and cannot communicate in any other way.
At any point, any prisoner can declare to the warden "I know that every single prisoner has been in this room already". If they are correct, all the prisoners will be set free; if not, they will all be executed.
a) What's a strategy that's guaranteed to work?
b) Does it still work if the warden is allowed to take prisoners into the room as often as he wants, without the other prisoners knowing? If not, find one that does.

Prisoners and boxes
Another 100 prisoners are in another game. They are each given a piece of paper on which they can write whatever they like. The papers are then taken by the warden, shuffled, and placed into boxes labelled 1 to 100 (one per box). One by one, each prisoner will be taken into the room with the boxes, and must find their own piece of paper by opening at most 50 boxes. If they do so, they're set free. To make things easier for them, before anyone else goes inside, the warden allows one prisoner to look inside all the boxes and, if they choose, to swap the contents of any two boxes (the other prisoners aren't allowed to move anything). Find the strategy which saves the greatest number of prisoners for certain.

Blue eyes (explanation taken from here)
A group of people with assorted eye colors live on an island. They are all perfect logicians -- if a conclusion can be logically deduced, they will do it instantly. No one knows the color of their own eyes. Every night at midnight, a ferry stops at the island. Any islanders who have figured out the color of their own eyes then leave the island, and the rest stay. Everyone can see everyone else at all times and keeps a count of the number of people they see with each eye color (excluding themselves), but they cannot otherwise communicate. Everyone on the island knows all the rules in this paragraph.
On this island there are 100 blue-eyed people, 100 brown-eyed people, and the Guru (she happens to have green eyes). So any given blue-eyed person can see 100 people with brown eyes and 99 people with blue eyes (and one with green), but that does not tell him his own eye color; as far as he knows the totals could be 101 brown and 99 blue. Or 100 brown, 99 blue, and he could have red eyes.
The Guru is allowed to speak once (let's say at noon), on one day in all their endless years on the island. Standing before the islanders, she says the following:
"I can see someone who has blue eyes."
Who leaves the island, and on what night?

Quine
Can you write a quine: a program that, when executed, prints its own source code?

Cheating on a string theory exam (puzzle taken from here)
You have to take a 90-minute string theory exam consisting of 23 true-false questions, but unfortunately you know absolutely nothing about the subject. You have a friend who will be writing the exam at the same time as you, is able to answer all of the questions in a fraction of the allotted time, and is willing to help you cheat — but the proctors are alert and will throw you out if they suspect him of communicating any information to you. You and your friend have watches which are synchronized to the second, and the proctors are used to him often finishing exams quickly and won't be suspicious if he leaves early.
a) What is the largest value N such that you can guarantee that you answer at least N out of the 23 questions correctly?
b) (Easier). The obvious answer is 12, but in fact you can do better than that, even though it seems like 12 is the information-theoretic limit. How come?

The hydra game (explanation taken from here)
A hydra is a finite tree, with a root at the bottom. The object of the game is to cut down the hydra to its root. At each step, you can cut off one of the heads, after which the hydra grows new heads according to the following rules:
  • If you cut off a head growing out of the root, the hydra does not grow any new heads.
  • Otherwise, remove that head and then make n copies of its grandfather subtree (as in the diagram below), where n is the number of the step you're on
What strategy can you use to eventually kill the hydra?

Physical puzzles

Balancing nails
Picture a nail hammered vertically into the floor (with most of it still sticking out). You're trying to balance as many other nails on it as you can, such that none of them touch the ground. How do you do so?

Hanging pictures
Consider a picture hanging by a string draped over some nails in the wall, in a way such that if any single nail is removed, the picture will fall to the ground.
a) Is it possible for 2 nails?
b) How about n nails?

Two-piece pyramid
Consider the two identical shapes shown below. Each has two planes of symmetry, and a square base. Is it possible to put them together to create a regular pyramid? (For a fun discussion of this problem in the contexts of machine learning, see a few minutes into this video).



Plane on a treadmill
Suppose that a plane were on a gigantic treadmill, which was programmed to roll backwards just as fast as the plane was moving forwards. Could the plane ever take off?

Pennies game
Two players take turns to place pennies flat on a circular table. The first one who can't place a penny loses. Is it the first or the second player who has a winning strategy?

Joining chains
You have four chains, each consisting of three rings. You're able to cut individual rings open and later weld them closed again. How many cuts do you need to make to form one twelve-ring bracelet?


Going postal
Alice and Bob live far apart, but are getting married and want to send each other engagement rings. However, they live in Russia, where all valuable items sent by post are stolen unless they're in a locked box. They each have boxes and locks, but no key for the other person's lock. How do they get the rings to each other?

Nine dots puzzle
Without lifting your pen from the paper, draw four straight lines that go through the centres of all 9 dots.

Mutilated chessboard
Consider a chessboard missing two diagonally opposite corner squares. Is it possible to cover all the remaining squares with dominos (where each domino covers two adjacent squares)?Related image

Safe sex
Suppose a man wants to have safe sex with three women, but only has two condoms. How can he do so, while ensuring that no STD is passed from anyone to anyone else?

Wristcuffs
Two people are tied together as in the following diagram. Without being able to undo or cut the ropes, how can they get free?

Thursday, 30 August 2018

Book review: Why we sleep

I read this book because I knew that it would tell me to sleep more, and I hoped it would cite enough scary statistics that I'd be likely to actually follow through. Well, it worked - I'm keeping a copy on my bedside table for the foreseeable future, just as a reminder. In addition to the exhortations to get more sleep, it contains a variety of other interesting and important facts about sleep.

What is sleep?
  • Human sleep consists of cycles lasting about 1.5 hours, each of which contains first a period of NREM (Non-Rapid Eye Movement) sleep, then a period of REM sleep. In brain scans, the former consists of slow, deep brain waves, while the latter shows the same frenetic activity as an awake brain. As the night goes on, cycles feature a higher proportion of REM sleep. This means that if you cut your sleep short by 25%, you're actually missing out on somewhere between 60% and 90% of REM sleep. 
  • REM sleep is when the majority of dreams happen. While it's uncommon for dreams to replay events from our everyday lives, they do often reflect our emotional preoccupations. To prevent ourselves from flailing around during dreams, we enter a state of sleep paralysis, where our brains are unable to control our voluntary muscles. Eyes are an exception - hence the name REM. It's definitely not true that REM is the only valuable type of sleep - in fact, immediately after sleep deprivation the brain prioritises catching up on NREM. 
  • The slow waves of NREM sleep are useful for transferring memories from one part of the brain to the other - in particular, from short- to long-term storage. 
  • Walker's theory is that NREM sleep is used to prune away unnecessary connections, while REM reinforces useful connections. He uses the analogy of a sculptor who alternates between carving away whole chunks of marble (NREM) and then adding fine detail on whatever's left (REM). From this perspective, it makes sense that REM sleep is concentrated in later cycles. However, it's unclear whether this is the scientific consensus. 
  • There are two systems controlling sleep and wakefulness. The circadian system follows the day/night cycle, making you tired in the evening and alert in the morning (the exact timings vary by person, making some people "night owls" and some "morning larks"). In addition, "sleep pressure" is controlled by adenosine, which builds up while you're awake and is cleared away during sleep. Caffeine works by temporarily blocking adenosine receptors, but doesn't prevent it from continuing to build up.

What's it good for?
  • There's a very strong link between NREM sleep and memory. The formation of long-term memories suffers if we don't get enough sleep (even several days after the events we want to remember). This is true both for memories about facts and experiences and for "muscle memory" of actions like playing an instrument. When sleep-deprived, we also have worse short-term memory.
  • REM sleep is important in emotional regulation and creativity. After sleep deprivation, the responses of the amygdala (responsible for strong emotions) can be amplified by over 60%, due to weakened links between it and the prefrontal cortex (responsible for "rational" decision-making). Dreams during REM sleep allow us to make unusual and creative connections between different topics - many great intellectuals report that their best ideas just "came to them" upon waking.
  • Sleep deprivation massively reduces our ability to concentrate. In addition to slower reaction times, when tired we lapse into "micro-sleeps" during which we're totally unresponsive. Walker emphasises that tiredness is a far bigger cause of traffic accidents than drunk-driving, that drivers systematically underestimate how tired they are, and that drivers who micro-sleep often don't brake at all before collisions.
  • In the long term, sleep deprivation increases the risk of Alzheimer's (since toxins are flushed from the brain during sleep), heart attacks (by provoking a stress response from the sympathetic nervous system and raising blood pressure) and cancer (by devastating the immune system). All of these seem to be very big effects - e.g. sleep-deprived patients are twice to three times as likely to suffer calcification of their coronary arteries.
  • Note that most of the effects above are noticeable even after small amounts of sleep deprivation, like getting one or two hours less sleep for one or two nights. In fact, even the one-hour sleep reduction from Daylight Savings Time causes a spike in heart attacks.
  • Sleep is also linked to many mental illnesses - e.g sleep deprivation triggers mania or depression in bipolar patients. Most mental illnesses disrupt sleep, which exacerbates their other negative effects.
  • REM sleep promotes the formation of neural links in infants, who have far more neural connections than adults. It is also important for their language learning.
  • Walker's broad answer to the question of what sleep is useful for: EVERYTHING. In addition to the above, sleep helps us overcome traumatic memories, reduces athletes' injury rates, makes us look more attractive, reduces food cravings, and so on and so on...

The evolution of sleep
  • I guess it shouldn't be a surprise that sleep is so broadly useful: once it started, it makes sense that many metabolic processes would take advantage of it. And they've had a long time to do so: sleep is ancient, with all animal species demonstrating some form of sleep-like behaviour. 
  • Even unicellular bacteria have active and passive phases corresponding to the planet's light/dark cycle.
  • However, the length of sleep required varies wildly for different animals, from 4 hours for elephants to 19 for brown bats. 
  • Only birds and mammals have proper REM sleep - it is a relatively recent adaptation. It also seems to be absent in aquatic mammals, whose two brain hemispheres sleep separately. 
  • Humans seem to be naturally biphasic: modern hunter-gatherer tribes sleep for 7-8 hours at night, and then nap for 30-60 minutes in the afternoon. It's biologically natural to be sleepy after lunch. Biphasic sleep significantly decreases mortality from heart disease. 
  • Walker hypothesises that descending from the trees to sleep on the ground allowed us to gain more REM sleep (particularly difficult in trees due to sleep paralysis), and therefore was important in boosting human cognitive development; also, that fire was vital in making ground-sleeping safer.

How to sleep better
  • Alcohol is an extremely powerful suppressor of REM sleep. Since it stays in your system for hours, it's best not to drink in the evenings. 
  • Light, especially blue light, signals your circadian system to wake up. Unfortunately LED screens provide a lot of blue light. Avoid using screens in the hours before bed, or at least phase out the blue light (e.g. using flux). 
  • In addition to light, our bodies use decreasing temperatures as a signal to sleep. Lowering room temperature often helps with insomnia. Apparently your core temperature will also fall after a hot bath. 
  • Caffeine has a half-life of 5 to 7 hours, so if you drink it in the afternoon, a significant amount will still be in your system at bedtime. 
  • The circadian rhythm of a teenager is naturally a few hours later than that of an adult, so teens shouldn't be forced to get up too early. Unfortunately, schools aren't taking much notice of this. 
  • Apparently sleeping pills cause lower-quality sleep and have severe long-term side-effects, so they should be avoided (with the exception of melatonin). 
  • For serious sleep problems, Cognitive Behavioural Therapy for Insomnia (CBT-I) works fairly well and should be the first step.

As you can probably tell from the above, Walker is very much a cheerleader for sleep. This does bias him in some noticeable ways - e.g. his overt scorn towards coffee. He also blurs causation and correlation at some points throughout the book, so I'd be surprised if all of the deleterious effects mentioned above are as significant as he claims. But the overall picture is stark enough that I'm now very worried about the ongoing sleep loss epidemic.

Friday, 24 August 2018

Feeling, complicated

It was very striking to me to contrast the two recent successes of OpenAI: one, OpenAI Five, beating some of the best humans at a complex game in a sophisticated virtual environment; and the other, Dactyl, fumblingly manipulating blocks in ways that children master at young ages. This is not to diminish how much of an achievement Dactyl is - no other reinforcement learning system has come close to this sort of performance in a physical task. But it does show that the real world is very complicated, compared with even our most advanced virtual worlds. To be fair, the graphics and physics engines used to render videos are becoming very good (and as movies show, practically indistinguishable from real life when enough work is put in). Audio generation is worse, except on human voices, which are now very convincing - but background sounds aren't a crucial component of our environment anyway. The biggest experiential difference between current simulations and the real world seems to be tactile sensations, or the lack thereof. OpenAI couldn't get realistic simulation of tactile sensations even in the very simple Dactyl environment (and eventually decided to do without them).

This may be due to the intrinsic difficulty of generating tactile feedback, but may also be because of the type of situation in which it's required. You can get impressive visual output from a static landscape. By contrast, we get tactile feedback mostly from physical interactions with objects in our environment - but modelling objects which can interact with each other is very hard! Consider how many things I can do with a piece of paper: write on it, tear it, crumple it, blow it away, make a paper plane out of it, set it on fire, soak it in water, eat it, cut holes in it, braid it into ropes, and so on. Many of these effects depend on molecular-level physics, which we're very far from being able to simulate on a large scale. And even the macroscopic effects rely on friction, which is also very difficult to model efficiently. If we add in fluid dynamics, then it seems plausible that it will take half a century or more before any simulated world is advanced enough to model all the interactions I listed above in real time. And that's just paper, not machines or electronics!

An alternative approach is to constrain the types of interactions which are allowed - e.g. a simulation with only rigid bodies. In such an environment, we could develop efficient approximations to trade quality for speed (a tradeoff which the graphics community has been wrestling with for some time). Friction would still be a major difficulty, as well as the ability to feel surface textures, but it's likely that immersive, interactive simulations with these properties will be developed within the next decade or two. The reason visual rendering is so advanced is because it's so crucial to the multi-billion-dollar video game and film industries. Now that VR is becoming a thing, those market pressures will be pushing hard for realistic environments with tactile feedback - and in doing so, increasing the effective number of people working on AI even faster than the nominal number.

Monday, 20 August 2018

Book review: The Complacent Class

"The best lack all conviction, while the worst
Are full of passionate intensity."

                                                   W. B. Yeats

The idea that things aren't going great these days is pretty widespread; there's a glut of books pointing out various problems. Cowen's achievement in this one is in weaving together disparate strands of evidence to identify the zeitgeist which summarises the overall trend - in a word, complacency. There are at least two ways in which people can be complacent. Either they're living pretty good lives, and want to solidify their positions as much as possible. Or they're unsatisfied with their lives, but unwilling to mobilise or take the risks which could improve their situations. (People in the middle of the economic spectrum showcase aspects of both). What's the opposite of complacency? Dynamism and risk-taking - traits which have always been associated with immigrants, and with America, the land of immigrants. Such traits aren't always expressed in positive ways, of course. Says Cowen: "Our current decade can be understood by comparing it to the 1960s and early 1970s. The Watts riots of 1965 put 4,000 people in jail and led to thirty-four killed and hundreds injured; during an eighteen-month period in 1971–1972, there were more than 2,500 domestic bombings reported, averaging out to more than five a day. I’m not advocating these tactics, of course. My point is that, today, there is an entirely different mentality, a far more complacent one, and one that finds it hard to grasp that change might proceed on such a basis. Yet in the 1960s and 1970s, not only did riots and bombings happen, but large numbers of influential intellectuals endorsed them, defended them, and maybe led them to some degree." In comparison to this, even people who are passionate about social change lack anywhere near the same sense of urgency today.

Cowen identifies many metrics which support the narrative of increasing complacency. Over the last few decades, interstate migration has gone down, job mobility has gone down, startup formation has gone down, and business churn and turnover have gone down. What's gone up? Market concentration and "matching": our ability to tailor our lives so that we're only exposed to things we're already comfortable with, whether that be music and movies similar to those we've seen before, or partners from the same class background as us. One particularly perverse result of better matching is increasing racial, economic and political segregation, reversing the progress made in the first half of the 20th century. A particularly notable cause of this is NIMBY movements, many of which have succeeded in ossifying their neighbourhoods by stifling new development. Segregation is also very pronounced in the incarceration industry.

In our personal lives, complacency involves prioritising comfort and security above all: physical security, with games like dodgeball and even tag being banned in schools; emotional security, via safe spaces and trigger warnings; and even corporate security, with companies hyper-focused on protecting their brands and other intangible assets (which have gone from less than 20% to over 80% of the value of the S&P 500 over the last 40 years). LGBT activists have moved from pushing the boundaries of societal norms to pursuing the most traditional of institutions, marriage. Nobody really has a bold vision for the future. (Relatedly, the portion of the federal budget allocated for discretionary spending has been falling sharply.)

But, Cowen says, this mindset can't keep limping along indefinitely, and will eventually face a crisis. In fact, we can think of it as a cyclic process: our current appetite for calm was whetted in the riotous and violent 70s and 80s - but as people become more disillusioned, it will eventually give way to similar turmoil, the start of which we're already seeing.

Causes and complacencies

It's instructive to evaluate The Complacent Class with reference to Cowen's last two books. In The Great Stagnation, he argues that America's growth has slowed down because it ran out of "low-hanging fruit" like the technological advances of the early 20th century. Further, the problem is even worse than it seems, because growth in sectors like healthcare, finance and government spending contributes less to people's welfare than it used to. In Average is Over, he predicts the effects of the next low-hanging fruit: AI. He argues that those who can work well with technology - in broad strokes, the smart and the conscientious - will be able to replace dozens of less-skilled workers, and will be richly rewarded for it (which contributes to increasing credentialism). We should expect to see even more middle-class jobs crowded out, and even more wealth captured by a smaller proportion of people. Returns to being based in a good location are also increasing, driving the clustering of university graduates into relatively few cities. Meanwhile, Cowen predicts that the poor will be squeezed into places with much lower housing costs, even if they end up resembling "shantytowns". He notes that America's population is ageing, and that the elderly voting demographic usually gets what it wants, even at the cost of society overall.

Okay, so how does this relate to complacency? I think that's very unclear. If many of the trends cited in The Complacent Class can be explained by the ideas that we're in a "great stagnation" and that "average is over", without reference to individual attitudes, then they're not really evidence for complacency in the psychological sense. In fact, how do we know that there aren't other explanations for all of them? The ageing population comes to mind as a major possibility (although I'm not sure how many of Cowen's statistics already control for age). Perhaps it's useful to identify an overall pattern of "complacent" behaviour even if different changes have different causes, some psychological and some from technological and international shifts - but this sort of pattern has very little predictive power, since we don't know which other domains we can apply it to.

This lack of clarity around what complacency actually means makes it somewhat unfalsifiable. For Cowen, complacency is demonstrated both by the rich erecting barriers to the advancement of the poor, and by the poor not breaking through those barriers. But what if the rich did nothing while their social and financial dominance eroded away - wouldn't that also count as complacency? What if the poor are actually willing to take more risks now (like the financial risk of going to university) but it's just paying off less - would that really make them "complacent"? Cowen claims that more relaxed codes of dress and manners display "a culture of the static and the settled", but don't they also lower implicit class barriers and therefore promote dynamism? There's a case that doing graduate degrees is ambitious and valuable, but there's a similarly strong case that it's a complacent replacement for making things happen in the real world. The very same companies which match us with our preferred options also allow us to sample more variety - whether that's in songs, shopping, or sexual partners. And so on. More generally, I think we should be biased against claims of the form "it's a big problem that the next generation have the wrong attitude towards X", because they have occurred so commonly throughout history, usually sounded convincing, and were usually wrong.

Nevertheless, there's undeniably some truth to Cowen's core argument. Almost none of the physical technologies around us (buildings, cars, trains, rockets, household appliances) have seen significant progress over the last half-century; nor have systems like healthcare, law, politics or education. But more importantly, people aren't even surprised by this stasis: the radical expectations of the mid-20th century have given way to doubt that our lives will be any better than our parents', plus a generous helping of political disillusionment. It's true that IT has made massive leaps, but as Cowen notes, "a lot of the internet’s biggest benefits are distributed in proportion to our cognitive abilities to exploit them". People who don't highly value near-unlimited access to information or niche communities may even find that the downsides of the internet (addiction to games or porn, mental health problems exacerbated by social media, news media's race to the bottom) outweigh its upsides. So I do believe that westerners today are, psychologically, more complacent than they were a few decades ago, and that this shows through in attitudes towards risk and expectations for the future. It's also likely that increasingly complacent behaviour which isn't caused by a complacent mentality still leads to an overall culture of complacency, although disentangling cause and effect here is tricky. Either way, Cowen's ideas are as thought-provoking as usual and should be taken into account by anyone interested in understanding America.