Wednesday, 25 May 2022

Science-informed normativity

The debate over moral realism is often framed in terms of a binary question: are there ever objective facts about what’s moral to do in a given situation? The broader question of normative realism is also framed in a similar way: are there ever objective facts about what’s rational to do in a given situation? But I think we can understand these topics better by reframing them in terms of the question: how much do normative beliefs converge or diverge as ontologies improve? In other words: let’s stop thinking about whether we can derive normativity from nothing, and start thinking about how much normativity we can derive from how little, given that we continue to improve our understanding of the world. The core intuition behind this approach is that, even if a better understand of science and mathematics can’t directly tell us what we should value, it can heavily influence how our values develop over time.

Values under ontology improvements


By “ontology” I mean the set of concepts which we use to understand the world. Human ontologies are primarily formulated in terms of objects which persist over time, and which have certain properties and relationships. The details have changed greatly throughout history, though. To explain fire and disease, we used to appeal to spirits and curses; over time we removed them and added entities like phlogiston and miasmas; now we’ve removed those in turn and replaced them with oxidation and bacteria. In other cases, we still use old concepts, but with an understanding that they’re only approximations to more sophisticated ones - like absolute versus relative space and time. In other cases, we’ve added novel entities - like dark matter, or complex numbers - in order to explain novel phenomena.


I’d classify all of these changes as “improvements” to our ontologies. What specifically counts as an improvement (if anything) is an ongoing debate in the philosophy of science. For now, though, I’ll assume that readers share roughly common-sense intuitions about ontology improvement - e.g. the intuition that science has dramatically improved our ontologies over the last few centuries. Now imagine that our ontologies continue to dramatically improve as we come to better understand the world; and that we try to reformulate moral values from our old ontologies in terms of our new ontologies in a reasonable way. What might happen?


Here are two extreme options. Firstly, very similar moral values might end up in very different places, based on the details of how that reformulation happens, or just because the reformulation is quite sensitive to initial conditions. Or alternatively, perhaps even values which start off in very different places end up being very similar in the new ontology - e.g. because they turn out to refer to different aspects of the same underlying phenomenon. These, plus intermediate options between them, define a spectrum of possibilities. I’ll call the divergent end of this spectrum (which I’ve defended elsewhere) the “moral anti-realism” end, and the convergent end the “moral realism” end.


This will be much clearer with a few concrete examples (although note that these are only illustrative, because the specific beliefs involved are controversial). Consider two people with very different values: an egoist who only cares about their own pleasure, and a hedonic utilitarian. Now suppose that each of them comes to believe Parfit’s argument that personal identity is a matter of degree, so that now the concept of their one “future self” is no longer in their ontology. How might they map their old values to their new ontology? Not much changes for the hedonic utilitarian, but a reasonable egoist will start to place some value on the experiences of people who are “partially them”, who they previously didn’t care about at all. Even if the egoist’s priorities are still quite different from the utilitarian’s, their values might end up significantly closer together than they used to be.


An example going the other way: consider two deontologists who value non-coercion, and make significant sacrifices to avoid coercing others. Now consider an ontological shift where they start to think about themselves as being composed of many different subagents which care about different things - career, relationships, morality, etc. The question arises: does it count as “coercion” when one subagent puts a lot of pressure on the others, e.g. by inducing a strong feeling of guilt? It’s not clear that there’s a unique reasonable answer here. One deontologist might reformulate their values to only focus on avoiding coercion of others, even when they need to “force themselves” to do so. The other might decide that internal coercion is also something they care about avoiding, and reduce the extent to which they let their “morality” subagent impose its will on the others. So, from a very similar starting point, they’ve diverged significantly under (what we’re assuming is an) ontological improvement.


Other examples of big ontological shifts: converting from theism to atheism; becoming an illusionist about consciousness; changing one’s position on free will; changing one’s mind about the act-omission distinction (e.g. because the intuitions for why it’s important fall apart in the face of counterexamples); starting to believe in a multiverse (which has implications for infinite ethics); and many others which we can’t imagine yet. Some of these shifts might be directly prompted by moral debate - but I think that most “moral progress” is downstream of ontological improvements driven by scientific progress. Here I’m just defining moral progress as reformulating values into a better ontology, in any reasonable way - where a person on the anti-realist side of the spectrum expects that there are many possible outcomes of moral progress; but a person on the realist side expects there are only a few, or perhaps just one.


Normative realism


So far I’ve leaned heavily on the idea of a “reasonable” reformulation. This is necessary because there are always some possible reformulations which end up very divergent from others. (For example, consider the reformulation “given a new ontology, just pretend to have the old ontology, and act according to the old values”.) So in order for the framework I’ve given so far to not just collapse into anti-realism, we need some constraints on what’s a “reasonable” or “rational” way to shift values from one ontology to another.


Does this require that we commit to the existence of facts about what’s rational or irrational? Here I’ll just apply the same move as I did in the moral realism case. Suppose that we have a set of judgments or criteria about what counts as rational, in our current ontology. For example, our current ontology includes “beliefs”, “values”, “decisions”, etc; and most of us would classify the claim “I no longer believe that ‘souls’ are a meaningful concept, but I still value people’s souls” as irrational. But our ontologies improve over time. For example, Kahneman and Tversky’s work on dual process theory (as well as the more general distinction between conscious and unconscious processing) clarifies that “beliefs” aren’t a unified category - we have different types of beliefs, and different types of preferences too. Meanwhile, the ontological shifts I mentioned before (about personal identity, and internal subagents) also have ramifications for what we mean when talking about beliefs, values, etc. If we try to map our judgements of what’s reasonable into our new ontology in a reflectively consistent way (i.e. a way that balances between being “reasonable” according to our old criteria, and “reasonable” according to our new criteria), what happens? Do different conceptions of rationality converge, or diverge? If they strongly converge (the “normative realist” position) then we can just define reasonableness in terms of similarity to whatever conception of rationality we’d converge to under ontological improvement. If they strongly diverge, then…well, we can respond however we’d like; anything goes!


I’m significantly more sympathetic to normative realism as a whole than moral realism, in particular because of various results in probability theory, utility theory, game theory, decision theory, machine learning, etc, which are providing increasingly strong constraints on rational behavior (e.g. by constructing different types of dutch books). In the next section, I’ll discuss one theory which led me to a particularly surprising ontological shift, and made me much more optimistic about normative realism. Having said that, I’m not as bullish on normative realism as some others; my best guess is that we’ll make some discoveries which significantly improve our understanding of what it means to be rational, but others which show us that there’s no “complete” understanding to be had (analogous to mathematical incompleteness theorems).


Functional decision theory as an ontological shift


There’s one particular ontological shift which inspired this essay, and which I think has dragged me significantly closer to the moral/normative realist end of the spectrum. I haven’t mentioned it so far, since it’s not very widely-accepted, but I’m confident enough that there’s something important there that I’d like to discuss it now. The ontological shift is the one from Causal Decision Theory (CDT) to Functional Decision Theory (FDT). I won’t explain this in detail, but in short: CDT tells us to make decisions using an ontology based on the choices of individual agents. FDT tells us to make decisions using an ontology based on the choices of functions which may be implemented in multiple agents (and by expanding the concepts of causation and possible worlds to include logical causation and counterpossible worlds).


Because of these shifts, a “selfish” agent using FDT can end up making choices more similar to the choices of an altruistic CDT agent than a selfish CDT agent, for reasons closely related to the traditional moral intuition of universalizability. FDT is still a very incomplete theory, but I find this a very surprising and persuasive example of how ontological improvements might drive convergence towards some aspects of morality, which made me understand for the first time how moral realism might be a coherent concept! (Another very interesting but more speculative point: one axis on which different versions of FDT vary is how “updateless” they are. Although we don’t know how to precisely specify updatelessness, increasingly updateless agents behave as if they’re increasingly altruistic, even towards other agents who could never reciprocate.)


Being unreasonable


Suppose an agent looks at a reformulation to a new ontology, and just refuses to accept it - e.g. “I no longer believe that ‘souls’ are a meaningful concept, but I still value people’s souls”. Well, we could tell them that they were being irrational; and most such agents care enough about rationality that this is a forceful objection. I think the framing I’ve used in this document makes this argument particularly compelling - when you move to a new ontology in which your old concepts are clearly inadequate or incoherent, then it’s pretty hard to defend the use of those old concepts. (This is a reframing of the philosophical debate on motivational internalism.)


But what if they said “I believe that I am being irrational, but I just refuse to stop being irrational”; how could we respond then? The standard answer is that we say “you lose” - we explain how we’ll be able to exploit them (e.g. via dutch books). Even when abstract “irrationality” is not compelling, “losing” often is. Again, that’s particularly true under ontology improvement. Suppose an agent says “well, I just won’t take bets from Dutch bookies”. But then, once they’ve improved their ontology enough to see that all decisions under uncertainty are a type of bet, they can’t do that - or at least they need to be much unreasonable to do so.


None of this is particularly novel. But one observation that I haven’t seen before: the “you lose” argument becomes increasingly compelling the bigger the world is. Suppose you and I only care about our wealth, but I use a discount rate 1% higher than yours. You tell me “look, in a century’s time I’ll end up twice as rich as you”. It might not be that hard for me to say “eh, whatever”. But suppose you tell me “we’re going to live for a millennium, after which I’ll predictably end up 20,000 times richer than you” - now it feels like a wealth-motivated agent would need to be much more unreasonable to continue applying high discounts. Or suppose that I’m in a Pascal’s mugging scenario where I’m promised very high rewards with very low probability. If I just shrug and say “I’m going to ignore all probabilities lower than one in a million”, then it might be pretty tricky to exploit me - a few simple heuristics might be able to prevent myself being dutch-booked. But suppose now that we live in a multiverse where every possible outcome plays out, in proportion to how likely it is. Now ignoring small probabilities could cause you to lose a large amount of value in a large number of multiverse branches - something which hooks into our intuitive sense of “unreasonableness” much more strongly than the idea of “ignoring small probabilities” does in the abstract. (Relatedly, I don’t think it’s a coincidence that utilitarianism has become so much more prominent in the same era where we’ve become so much more aware of the vastness of the universe around us.)


Why am I talking so much about “reasonableness” and moral persuasion? After all, agents which are more rational will tend to survive more often, acquire more resources, and become more influential: in the long term, evolution will do the persuasion for us. But it’s not clear that the future will be shaped by evolutionary pressures - it might be shaped by the decisions of goal-directed agents. Our civilization might be able to “lock in” certain constraints - like enough centralization of decision-making that the future is steered by arguments rather than evolution. And thinking about convergence towards rationality also gives us a handle for reasoning about artificial intelligence. In particular, it would be very valuable to know how much applying a minimal standard of reasonableness to their decisions would affect how goal-directed they’ll be, and how aligned their goals will be with our own.


How plausible is this reasoning?


I’ve been throwing around a lot of high-level concepts here, and I wouldn’t blame readers for feeling suspicious or confused. Unfortunately, I don’t have the time to make them clearer. In lieu of that, I’ll briefly mention three intuitions which contribute towards my belief that the position I’ve sketched in this document is a useful one.


Firstly, I see my reframing as a step away from essentialism, which seems to me to be the most common mistake in analytic philosophy. Sometimes it’s pragmatically useful to think in terms of clear-cut binary distinctions, but in general we should almost always aim to be able to ground out those binary distinctions in axes of continuous variation, to avoid our standard bias towards essentialism. In particular, the moral realism debate tends to focus on a single binary question (do agents converge to the same morality given no pre-existing moral commitments?) whereas I think it’d be much more insightful to focus on a less binary question (how small or large is the space of pre-existing moral commitments which will converge)?


Secondly, there’s a nice parallel between the view of morality which I’ve sketched out here, and the approach some mathematicians take, of looking at different sets of axioms to see whether they lead to similar or different conclusions. In our case, we’d like to understand whether similar starting intuitions and values will converge or diverge under a given approach to ontological reformulation. (I discuss the ethics-mathematics analogy in more detail here.) If we can make progress on meta-ethics by actually answering object-level questions like “how would my values change if I believed X?”, that helps address another common mistake in philosophy - failing to link abstract debates to concrete examples which can be deeply explored to improve philosophers’ intuitions about the problem.


And thirdly, I think this framing fits well with our existing experiences. Our values are strongly determined by evolved instincts and emotions, which operate using a more primitive ontology than the rest of our brains. So we’ve actually got plenty of experience in struggling to shift various values from one ontology to another, and the ways in which some people manage to do so, and some remain unreasonable throughout.  We just need to imagine this process continuing as we come to understand the world far better than we do today.

Friday, 15 April 2022

Three intuitions about effective altruism: responsibility, scale, self-improvement

This is a post about three intuitions for how to think about the effective altruism community.

Part 1: responsibility

The first intuition is that, in a global sense, there are no “adults in the room”. Before covid I harboured a hope that, despite the incessant political squabbling we see worldwide, in the face of a major crisis with global implications, there were serious people who would come out of the woodwork to ensure that it went well. There weren’t. And that’s not just a national phenomenon, that’s a global phenomenon. Even countries like New Zealand, which handled covid incredibly well, weren’t taking responsibility in the global way I’m thinking about - they looked after their own citizens, but didn’t try to speed up vaccine distribution overall (e.g. by allowing human challenge trials), or fix everyone else’s misunderstandings.

Others developed the same “no adults in the room” intuition by observing failures on different issues. For some, AI risk; for others, climate change; for others, policies like immigration or housing reform. I don’t think covid is a bigger failure than any of these, but I think it comes much closer to creating common knowledge that the systems we have in place aren’t capable of steering through global crises. This naturally points us towards a long-term goal for the EA community: to become the adults in the room, the people who are responsible enough and capable enough to steer humanity towards good outcomes.

By this I mean something different from just “being in charge” or “having a lot of power”. There are many large power structures, containing many competent people, which try to keep the world on track in a range of ways. What those power structures don’t have is the ability to absorb novel ideas and take novel actions in response. In other words, the wider world solves large problems via OODA loops that take decades. In the case of climate change, decades of advocacy led to public awareness which led to large-scale policies, plus significant reallocation of talent. I think this will be enough to avoid catastrophic outcomes, but that’s more from luck than skill. In the case of covid, the OODA loop on substantially changing vaccine regulations was far too long to make a difference (although maybe it’ll make a difference to the next pandemic).

The rest of the world has long OODA loops because people on the inside of power structures don’t have strong incentives to fix problems; and because people on the outside can’t mobilise people, ideas and money quickly. But EA can. I don’t think there’s any other group in the world which can allocate as much talent as quickly as EA has; I don’t think there’s any other group which can identify and propagate important new ideas as quickly as EA can; and there are few groups which can mobilise as much money as flexibly.

Having said all that, I don’t think we’re currently the adults in the room, or else we would have made much more of a difference during covid. While it’s not itself a central EA concern, it’s closely related to one of our central concerns, and would have been worth addressing for reputational reasons alone. But I do think we were closer to being the adults in the room than almost any other group - particularly in terms of long-term warnings about pandemics, short-term warnings about covid in particular, and converging quickly towards accurate beliefs. We should reflect on what would have been needed for us to convert those advantages into much more concrete impact.

I want to emphasise, though, that being the adults in the room doesn’t require each individual to take on a feeling of responsibility towards the world. Perhaps a better way to think about it: every individual EA should take responsibility for the EA community functioning well, and the EA community should take responsibility for the world functioning well. (I’ve written a little about the first part of that claim in point four of this post.)

Part 2: scale, not marginalism

Historically, EA has thought primarily about the marginalist question of how to do the most good per unit of resources. An alternative, which is particularly natural in light of part 1, is to simply ask: how can we do the most good overall? In some sense these are tautologically equivalent, given finite resources. But a marginalist mindset makes it harder to be very ambitious - it cuts against thinking at scale. For the most exciting projects, the question is not “how effectively are we using our resources”, but rather “can we make it work at all?” - where if it does work it’ll be a huge return on any realistic amount of investment we might muster. This is basically the startup investor mindset; and the mindset that focuses on megaprojects.

Marginalism has historically focused on evaluating possible projects to find the best one. Being scale-focused should nudge us towards focusing more on generating possible projects. On a scale-focused view, the hardest part is finding any lever which will have a big impact on the world. Think of a scientist noticing an anomaly which doesn’t fit into their existing theories. If they tried to evaluate whether the effects of understanding the anomaly will be good or bad, they’d find it very difficult to make progress, and maybe stop looking. But if they approach it in a curious way, they’re much more likely to discover levers on the world which nobody else knows about; and then this allows them to figure out what to do.

There are downsides of scaling, though. Right now, EA has short OODA loops because we have a very high concentration of talent, a very high-trust environment, and a small enough community that coordination costs are low. As we try to do more large-scale things, these advantages will slowly diminish; how can we maintain short OODA loops regardless? I’m very uncertain; this is something we should think more about. (One wild guess: we might be the one group best-placed to leverage AI to solve internal coordination problems.)

Part 3: self-improvement and growth mindset

In order to do these ambitious things, we need great people. Broadly speaking, there are two ways to get great people: recruit them, or create them. The tradeoff between these two can be difficult - focusing too much on the former can create a culture of competition and insecurity; focusing too much on the latter can be inefficient and soak up a lot of effort.

In the short term, it seems like there are still low-hanging fruit when it comes to recruitment. But in the longer term, my guess is that EA will need to focus on teaching the skillsets we’re looking for - especially when recruiting high school students or early undergrads. Fortunately, I think there’s a lot of room to do better than existing education pipelines. Part of that involves designing specific programs (like MLAB or AGI safety fundamentals), but probably the more important part involves the culture of EA prioritising learning and growth.

One model for how to do this is the entrepreneurship community. That’s another place where returns are very heavy-tailed, and people are trying to pick extreme winners - and yet it’s surprisingly non-judgemental. The implicit message I get from them is that anyone can be a great entrepreneur, if they try hard enough. That creates a virtuous cycle, because it’s not just a good way to push people to upskill - it also creates the sort of community that attracts ambitious and growth-minded people. I do think learning to be a highly impactful EA is harder in some ways than learning to be a great entrepreneur - we don’t get feedback on how we’re doing at anywhere near the same rate entrepreneurs do, so the strategy of trying fast and failing fast is much less helpful. But there are plenty of other ways to gain skills, especially if you’re in a community which gives you support and motivation to continually improve.

Saturday, 2 April 2022

Book review: Very Important People

New York’s nightclubs are the particle accelerators of sociology: reliably creating the precise conditions under which exotic extremes of status-seeking behaviour can be observed. Ashley Mears documents it all in her excellent book Very Important People: Status and Beauty in the Global Party Circuit. A model turned sociology professor, while researching the book she spent hundreds of nights in New York’s most exclusive nightclubs, as well as similar parties across the world. The book abounds with fascinating details; in this post I summarise it and highlight a few aspects which I found most interesting.

Here’s the core dynamic. There are some activities which are often fun: dancing, drinking, socialising. But they become much more fun when they’re associated with feelings of high status. So wealthy men want to use their money to buy the feeling of having high-status fun, by doing those activities while associated with (and ideally while popular amongst) other high-status people, particularly beautiful women.

Unfortunately, explicit transactions between different forms of cultural capital are low-status - it demonstrates that you can’t get the other forms directly. So the wealthy men can’t just pay the beautiful women to come party with them. Instead an ecosystem develops which sells sufficient strategic ambiguity to allow (self- and other-) deception about the transaction which is taking place, via incorporating a series of middlemen.

Specifically, wealthy men pay thousands at these nightclubs for table charges and “bottle service” - already-expensive alcohol marked up by 5x or much more. The nightclubs pay “promoters” to scout out and bring along dozens of beautiful women each night. Those women get access to an exclusive venue with many wealthy men - but by itself that’s not enough to motivate regular attendance, at least not from the prettiest. And most are careful not to ruin their reputations by actually accepting payments from the promoters. Instead, in order to bring enough girls, promoters each need to do a bunch of emotional labour, flirting, relationship-building, and many non-cash payments (food, transport, even accommodation). I’m strongly reminded of Michael Sandel’s book What Money Can’t Buy - the intuitions about the corrosive effects of money are the same, they're just applied to a much less high-minded setting.

Some interesting features of this system:

  • At a top club, a promoter might get paid $1000 a night to bring out a dozen models or women who look like models. Notably, model-like beauty is much more highly-prized than conventional beauty - e.g. the clubs don’t allow access to women who aren’t unusually tall. Everyone selects for models even when they don’t personally find the model look as attractive, because the fashion industry has established this as the Schelling look for high-status women. (For more on how this happens, see Mears’ other book, Pricing Beauty; and the responses to my tweet about it).

  • The markup on increasingly large champagne bottles is determined less by the amount of champagne, and more by how ostentatious the purchase is. The biggest purchases, costing over 100k per bottle, therefore come with incredibly elaborate fanfare: all music stops, spotlights shine on the buyer, a whole train of staff bring out the drinks, etc.

  • The nightclub profits by creating an atmosphere of “suspended reality” where a large group of people who all individually believe that buying status in this way is tacky can still convince themselves that all the other people don’t think it’s tacky. Most of the profits don’t actually come from the biggest spenders, but rather the next tier down, who are inspired by the atmosphere, and anchored by stories of the biggest purchases.

  • In contrast to the predominantly-white clients and models, promoters are disproportionately black. Mears talks about them having “colour capital”, and using some stereotypes to their advantage in order to catch attention. They need to be very charismatic and attractive in order to consistently convince girls to come along with them while not making their relationship seem too transactional.

  • In some sense the whole system is grounded in the models’ sex appeal, but I think that the models’ prestige is just as important - as mentioned above, models are preferred to women who most men find more attractive, as well as preferred to women who have more transactional attitudes towards sex.

  • Basically the same dynamics play out internationally as well - promoters offer girls free flights, food and accommodation in exchange for attendance at nightclubs in St Tropez, etc. On those trips the transactionality is usually a bit more obvious.

  • How can promoters afford to regularly wine and dine so many girls? Often they have deals with restaurants who give them leftover food in exchange for making the restaurant look more glamorous. Other times, wealthy men will host the dinners before the parties start. At the nightclub itself, they all drink for free.


If I were a bit more cynical I might also say that the “fun” part of high-status fun is also mainly a strategic ambiguity which helps facilitate the status transaction - if people couldn’t convince themselves and others that they were having fun, their attempts to seem prestigious would be much more obvious. Perhaps it’s worth considering what differences you’d expect in a world where this is true vs false. (For example, might you expect that the highest-status men actually don’t spend much time dancing, drinking, or even socialising?)

The same might be true, to a lesser extent, of other types of high-status fun - which, in my circles, often involves quick-witted exchanges on arbitrary topics. Overall, though, after reading this book I do feel much luckier that silicon valley is largely disdainful of conspicuous consumption and other negative-sum status games; long may it stay that way.

Thursday, 10 March 2022

Beyond micromarriages

tl;dr micromarriages aren't fully analogous to micromorts, which makes it tricky to define them satisfactorily. I introduce an alternative unit: QAWYs (Quality-Adjusted Wife Years), where 1 QAWY is an additional year of happy marriage.

I once compiled a list of concepts which I’d discovered were much less well-defined than I originally thought. I’m sad to say that I now have to add Chris Olah’s micromarriages to the list. In his words: “Micromarriages are essentially micromorts, but for marriage instead of death. A micromarriage is a one in a million chance that an action will lead to you getting married, relative to your default policy.”

It’s a fun idea, and helpful in making small probabilities feel more compelling. But upon thinking about it more, I’ve realised that the analogy doesn’t quite work. The key difference is that micromorts are a measure of acute risk - i.e. immediate death. For activities like skydiving, this is the main thing to worry about, so it’s a pretty good metric. But most actions we’d like to measure using micromarriages (going to a party, say, or working out more) won’t lead you to get married immediately - instead they flow through to affect marriages that might happen at some later point.

So how can we measure the extent to which an action affects your future marriages, even in theory? One option is to track how it changes the likelihood you’ll get married eventually. But this is pretty unhelpful. By analogy, if micromorts measured an action’s effect on the probability that you’d die eventually, then all actions would have almost zero micromorts (with the possible exception of some life-extension work during the last few decades). Similarly, under this definition the micromarriages you gain from starting a new relationship could be mostly cancelled out by the fact that this relationship cuts off other potential relationships.


An alternative is to measure actions not by how much they change the probability that you’ll get married eventually, but by how much you expect them to causally contribute to an eventual marriage. The problem there is that many actions can causally contribute to a marriage (meeting someone, asking them out, proposing, etc) and there’s no principled way of splitting the credit between them. I won’t go into the details here, but the basic problem is the same as one which arises when trying to allocate credit to multiple contributors to a charitable intervention. E.g. if three different funders are all necessary for getting a project off the ground, in some sense they can all say that they “caused” the project to happen, but that would end up triple-counting their total impact. (In this case, we can use Shapley values to allocate credit - but the boundaries between different “actions” are much more arbitrary than the boundaries between different “agents”, making it harder to apply Shapley values to the micromarriage case. Should we count the action “skipping meeting someone else” as a contributor to the marriage? Or the action “turning your head to catch sight of them”? This is basically a rabbit-hole without end - and that’s not even getting into issues of marriage identity across possible worlds.)[1]


Fortunately, however, there’s another approach which does work. When thinking about mortality, the medical establishment doesn’t just measure acute risks, but also another category of risk: chronic risks, like smoking. When smoking, you don’t get a binary outcome after each cigarette, but rather a continual degradation of health. So chronic risks are instead measured in terms of the expected decrease in your lifespan - for example, with units of microlives, where one microlife is one millionth of an adult lifespan (about half an hour); or with quality-adjusted life years (QALYs), to adjust for ill health and disability.


Analogously, then, the most straightforward metric for guiding our romantic choices is the expected increase in the time you’ll spend married - which we could measure in microwives, where one microwife is an additional half-hour of marriage. But I don’t think this is the best unit, because most people could accumulate many more microwives by dropping their standards, even if that’ll lead to unhappy marriages. So it’s important to adjust for how good we expect the marriage to be! My proposed unit: quality-adjusted wife years (QAWYs). Note that these are gender-neutral: QAWYs can involve either being a wife or having a wife (or both) [2]. An intervention gains 1 QAWY if it increases the expected amount of time you’ll spend happily married by 1 year (or the amount of time you’ll spend in a half-as-happy marriage by 2 years, etc). We do need some benchmark for a “happy marriage”; I’ll arbitrarily pick the 90th percentile of marriages across the population. Some factors which affect QAWY evaluation include spouse compatibility, age of marriage, diminishing marginal utility, having children [3], and divorce probability. Conveniently, QAWYs don’t require the assumption of lifelong marriage - they can naturally account for the possibility of multiple consecutive (or even concurrent) marriages. With QAWY’s combination of theoretical elegance and pragmatic relevance, I look forward to their widespread adoption.


(To borrow a disclaimer from Chris' original post: seriously? Nope. I'm about 90% joking. I do think the general idea can sometimes be helpful, though.)


1. Alongside QAWYs, some version of micromarriages may still be useful - we just need to adjust them to measure an acute one-off event rather than a continuing chronic contributor to marriage. The most natural one is probably to think of a micromarriage as a one-in-a-million chance of first meeting your future spouse at a given event.

2. Unfortunately, this is still not fully inclusive. In formal contexts please use Quality-Adjusted Wedded Years instead.

3. The likelihood of which should of course be measured in units of microchildren (but not microkids, which I'm reserving for a very small chance of a very small joke, like this one).

Friday, 25 February 2022

My attitude towards death

The philosophy and psychology of death seem weirdly under-discussed - particularly by the wider silicon valley community, given how strongly anti-death many people in it are. This post is an attempt to think through some relevant considerations, primarily focused on my own intuitions and emotions. See also this old blog post - I mostly still agree with the points I made in it, but when thinking about it now I frame things pretty differently.

Fearing death, loving life

Let’s first distinguish two broad types of reasons for wanting to avoid death: fearing death, and loving life.* Perhaps these seem like two sides of the same coin - but, psychologically speaking, they feel very distinct to me. The former was particularly dominant when I was in primary school, when a part of me emerged that was very afraid of death (in a way that wasn’t closely linked to fear of missing out on any particular aspects of life). That part is still with me - but when it comes to the surface, its fear feels viscerally unpleasant, so I learned to suppress it pretty strongly.


Arguments for why death is bad usually focus on positive reasons - living longer allows people to experience more happiness, and more of the other good things in life. These have resonated with me more over time, as I started to think about death on a more intellectual level. However, one difficulty with these arguments is that many parts of me pursue goals in a fairly myopic way which doesn’t easily extrapolate to centuries, millennia, or longer. For example, it’s hard to imagine what career success or social success look like on the scale of millennia - and even when I try, those visions are pretty different from the versions of those concepts that I currently find motivating on a gut level. Extrapolating hedonistic goals is easier in some ways (it’s easy to imagine being happy for a very long time) but harder in other ways (the parts of me which care most about happiness are also the most myopic).


Dissolving fear


In practice, then, most of my motivation for avoiding death in the long term stems from fear of death. Although that fear comes out only rarely, I have a strong heuristic that fear-based motivation should be transmuted to other forms of motivation wherever possible. So what would happen if I talked more to the part that’s scared of death, to try and figure out where it’s coming from? By default, I expect it’d be uncooperative - it wants to continue being scared of death, to make sure that I act appropriately (e.g. that I stay ambitious). Can I assure it that I’ll still try hard to avoid death if it becomes less scared? One source of assurance is if I’m very excited about a very long life - which I am, because the future could be amazing. Another comes from the altruistic part of me, whose primary focus is increasing the probability that the future will in fact be amazing. Since I believe that we face significant existential risk this century**, working to make humanity’s future go well overlaps heavily with working to make my own future go well. I think this broad argument (along with being in communities which reward longtermist altruism) has helped make the part of me that’s scared of death more quiescent.


Indeed, probably my main concern with my current attitude towards death is actually that I’m not scared enough about existential risk - I think that, if my emotions better matched my credences, that’d help motivate me (especially to pursue particularly unusual or ambitious interventions). This doesn’t seem like a crucial priority, though, since my excitement- and interest-based motivations have been working fairly well so far (modulo some other productivity gaps which seem pretty orthogonal).


Generalising to others


So far I’ve talked primarily about my own experience. I’m curious about how well this generalises to other people. It seems like fear of death is a near-universal emotion (it’s striking that the first recorded story we have is about striving to escape death), but my guess is that most people have it much less strongly than I did.


Since most people aren’t very openly concerned with avoiding death in the long term, I feel uncertain about the extent to which they’ve suppressed versus dissolved that fear. My guess is that in western societies most people have mainly suppressed it, and that the hostility they often show to longevity research or cryonics is a psychological defense mechanism. If so, then overcoming those defense mechanisms to convince people that death is not inevitable might unlock a lot of suppressed excitement about the future. However, I’m wary of assuming that other people are too similar to me - perhaps other people’s fear of death is just more myopic than mine.


There also seem to be some people who started off with a long-term fear of death, then dissolved it, usually by significantly changing their conception of personal identity - via meditation, or drugs, or philosophical argument. The big question is whether this change is more like an empirical update, or more like a value shift (to be clear, I don’t think that there’s a bright line between the two - but something can be much more like one or the other). If the former, then perhaps fear of death is just a “mistake” that many people make. Whereas the latter suggests that death is really bad according to some people’s values, and mostly fine for others, even though they may in other ways be psychologically similar. Both of these conclusions seem a bit weird; let’s try to get a bit more clarity by digging into arguments about personal identity now.


Continuity of self


The core question is how much we should buy into the folk view of personal identity - the view that there’s a single “thread” of experience which constitutes my self, where I survive if that thread continues and “die” if it breaks. I consider thought experiments about duplicates to provide strong evidence against this position - it seems very compelling to me that, when two identical copies of myself are created, there is no fact of the matter about which one is “really me”. Insofar as many people have intuitions weighing the other way, that’s probably because we evolved in an environment where identical duplication didn’t happen. In a future where duplication exists, and we continue being subject to evolution, I can easily imagine the mental concept of survival-of-self being straightforwardly replaced by the concept of survival-of-a-copy-of-myself.


The main alternative to caring about continuity is caring about level of similarity - identifying with a successor if they are sufficiently psychologically similar to you. This might leave you identifying with many successors, or ones that are very disconnected from you in time or space. However, it’s also consistent with identifying only with successors with a level of similarity that, in practice, will only be achievable by copying or uploading you (although I expect that really buying into the similarity theory of personal identity will make most people more altruistic, like it did for Parfit).***


The strongest argument in favour of the folk view arises when considering large universes, like quantum multiverses or spatially infinite universes. In a quantum multiverse there are many copies of myself, and I tend to experience being the ones with more measure. But what does that even mean? If I expect that N slightly different copies of myself will branch off soon, and all of them will have the experience of being me, how can I anticipate being more likely to “find myself” as a given one of them? There's something here which I don't understand, and which makes me hesitant to fully dismiss the idea of a thread of experiences (a confusion which Yudkowsky explores in these two posts). I think the appropriate response is to be cautious until we understand this better - for instance, I would currently strongly prefer being non-destructively rather than destructively uploaded.


Generalising to society


When we stop thinking on an individual level and start thinking on a societal level, many more pragmatic considerations arise - especially related to how widespread longevity might shift the overall balance of power in the world. I do think these are important; here, though, I want to focus on a couple of broader philosophical considerations.


I previously talked about the part of myself which wants to make the future amazing. Partly that stems from imagining all the different ways in which the world might dramatically improve, including defeating death. Partly it’s an aesthetic preference about the trajectory of humanity - I want us to flourish in an analogous way to how I want to live a flourishing life myself. But there’s also a significant utilitarian motivation - which is relevant here because utilitarianism doesn’t care about death for its own sake, as long as the dead are replaced by new people with equal welfare. Indeed, if our lives have diminishing marginal value over time (which seems hard to dispute if you’re taking our own preferences into account at all), and humanity can only support a fixed population size, utilitarianism actively prefers that older people die and are replaced.


Now, I don’t think we’ll hit a “fixed population size” constraint until well after we’re posthuman, so this is a pretty abstract consideration. By that point, hopefully we won’t need to bite any bullets - we could build a flourishing civilisation which extrapolates our more human-specific values as well as possible, and also separately build the best utilitarian civilisation (assuming we can ensure non-conflict between them). But I’m also open to the idea that the future will look sufficiently weird that many of the concepts I’ve been using break down. For example, the boundaries between different minds could blur to such an extent that talking about the deaths of individuals doesn’t make much sense any more. I find it hard to viscerally desire that for myself, and I expect that most people alive today are much less open to the possibility than I am, but I can imagine changing my mind as we come to understand much more about how minds and values work.



* Upon reflection, I might also add a third distinct motivation - the celebration of immortality. I get this feeling particularly when I read fiction with very long-lived characters. But since it's much weaker than the other two, I won't discuss it further.

** At least double digit percentage points, although my specific estimate is pretty unstable.

*** On a side note: I feel very uncertain about how much information about my brain (in the form of my blog posts, tweets, background information about my life, etc) would be sufficient for future superintelligences to recreate me in a way that I’d consider a copy of myself. I haven’t even seen any rough bounds on this - maybe worth looking into.

Monday, 14 February 2022

Some limitations of reductionism about epistemology

This post is largely based on a lightning talk I gave at a Genesis event on metacognition, with some editing to clarify and expand on the arguments. 

Reductionism is the strategy of breaking things down into smaller pieces, then trying to understand those smaller pieces and how they fit together into larger pieces. It’s been an excellent strategy for physics, for most of science, for most of human knowledge. But my claim is that, when the thing we're trying to understand is how to think, being overly reductionist has often led people astray, particularly in academic epistemology.


I’ll give three examples of what goes wrong when you try to be reductionist about epistemology.  Firstly, we often think of knowledge in terms of sentences or propositions with definite truth-values - for example, “my car is parked on the street outside”. Philosophers have debated extensively what it means to know that such a claim is true; I think the best answer is the bayesian one, where we assign credences to propositions based on our evidence. Let’s say I have 90% credence that my car is parked on the street outside, based on leaving it there earlier - and let’s assume it is in fact still there. Then whether we count this as “knowledge” or not is mainly a question about what threshold we should use for the definition of “knows” (one which will probably change significantly depending on the context).


But although bayesianism makes the notion of knowledge less binary, it still relies too much on a binary notion of truth and falsehood. To elaborate, let’s focus on philosophy of science for a bit. Could someone give me a probability estimate that Darwin’s theory of evolution is true? [Audience answer: 97%] Okay, but what if I told you that Darwin didn’t know anything about genetics, or the actual mechanisms by which traits are passed down? So I think that 97% points in the right direction, but I think it’s less that the theory has a 97% chance of being totally true, and more like a 97% chance of being something like 97% true. If you break down all the things Darwin said into a list of propositions: animals inherit from their parents, and 100 different things - almost certainly at least one of these is false. That doesn’t change the fact that overall, the theory is very close to true (even though we really have no idea how to measure or quantify that closeness).


I don’t think this is a particularly controversial or novel claim. But it’s surprising that standard accounts of bayesianism don’t even try to account for approximate truth. And I think that’s because people have often been very reductionist in trying to understand knowledge by looking at the simplest individual cases, of single propositions with few ambiguities or edge cases. By contrast, when you start looking into philosophy of science, and how theories like Newtonian gravity can be very powerful and accurate approximations to an underlying truth that looks very different, the notion of binary truthhood and falsehood becomes much less relevant.


Second example: Hume’s problem of induction. Say you’re playing billiards, and you hit a ball towards another ball. You expect them to bounce off each other. But how do you know that they won’t pass straight through each other, or both shoot through the roof? The standard answer: we’ve seen this happen many times before, and we expect that things will stay roughly the same. But Hume says that this falls short of a deductive argument, it’s just an extrapolation. Since then, philosophers have debated the problem extensively. But they’ve done so in a reductionist way which focuses on the wrong things. The question of whether an individual ball will bounce off another ball is actually a question about our whole systems of knowledge: I believe the balls will bounce off each other because I believe they’re made out of atoms, and I have some beliefs about how atoms repel each other. I believe the balls won’t shoot through the roof due to my beliefs about gravity. If we try to imagine the balls not bouncing off each other, you have to imagine a whole revolution in our scientific understanding.


Now, Hume could raise the same objection in response: why can’t we imagine that physics has a special exception in this one case, or maybe that the fundamental constants fluctuate over time? If you push the skepticism that far, I don’t think we have any bulletproof response to it - but that’s true for basically all types of skepticism. Yet, nevertheless, thinking about doing induction in relation to models of the wider world, rather than individual regularities, is a significant step forward. For example, it clears up Nelson Goodman’s confusion about his New Riddle of Induction. Broadly speaking, the New Riddle asks: why shouldn’t we do induction on weird “gerrymandered” concepts instead of our standard ones? For any individual concept, that’s hard to answer - but when you start to think in a more systematic way, it becomes clearer that trying to create a model of the world in terms of gerrymandered concepts is hugely complex.


Third example: in the history of AI, one of the big problems that people have faced is the problem of symbol grounding: what does it mean for one representation in my AI to correspond to the real world. What does it mean for an AI to have a concept of a car - what makes the internal variable in my AI map to cars in the real world? Another example comes from neuroscience - you may have heard of Jennifer Aniston neurons, which fire when they recognise a single person, across a range of modalities. How does this symbolic representation in your brain relate to the real world?


The history of AI is the history of people trying to solve this from the ground up. Start with a few concepts, add some more to them, branch out, do a search through them, etc. This research program, known as symbolic AI, failed pretty badly. And we can see why when we think more holistically. The reason that a neuron in my brain represents my grandmother has nothing to do with that neuron itself, it’s because it’s connected to my arms which make me reach out and hug her when I see her, and the speech centers in my brain which remind me of her name when I talk about her, and the rest of my brain which brings up memories when I think of her. These aren’t things you can figure out by looking at the individual case, nor is it something you can design into the system on a step by step basis, as AI researchers used to try to do.


So these are three cases where, I claim, people have been reductionist about epistemology when they should instead have taken a much more systems-focused approach.

Strevens on scientific explanation

This post discusses two books by Michael Strevens: The Knowledge Machine: How Irrationality Created Modern Science and Thinking Off Your Feet: How Empirical Psychology Vindicates Armchair Philosophy. I loved the former, which tries to answer the core question in philosophy of science: why does science work so well? It’s a masterful synthesis of previous ideas and important new arguments, and it’ll be my go-to recommendation from now on for people interested in the field. The latter was… slightly less persuasive. But let’s start with the good stuff.

The Knowledge Machine begins with a review of two of the key figures in philosophy of science: Popper and Quine. Historically, philosophers of science focused on identifying a “scientific method”: a specific way of generating theories, designing experiments, and evaluating evidence which, when followed, led scientists to the truth. Popper’s influential account of the scientific method focused on scientists trying to refute their hypotheses. He claimed that only severe tests which attempt to falsify a hypothesis can give us reason to provisionally accept it. Along with other early philosophers of science, Popper’s work promoted (according to Strevens) “an ideal of the scientist as a paragon of intellectual honesty, standing up for truth in the face of stifling opposition from the prevailing politics, culture, and ideology”.


However, over the last half-century or so a range of criticisms of this flattering view of science have emerged. Most prominent is Kuhn, who in his book The Structure of Scientific Revolutions characterises scientists as constrained within a specific paradigm of thinking, unable to rationally decide between different paradigms. Soon afterwards, in his book The Sleepwalkers, Koestler described the emergence of early science not as the triumph of a superior method, but rather as “a history of collective obsessions and controlled schizophrenias”. More recently, Feyerabend’s book Against Method espoused a position he called “epistemological anarchism”, arguing that “anything goes” in the pursuit of truth.


Strevens focuses on Kuhn, but his arguments are in line with the positions of the others. He summarises a range of case studies of scientists ignoring inconvenient data, deciding questions via political maneuvering, and generally behaving in ways no “scientific method” would endorse. These case studies also reiterate a point made by Quine: that a theory can never be fully falsified, since it’s always possible to argue that it’s consistent with new evidence - e.g. by tacking on new parameters, or appealing to experimental mistakes. This line of thinking provides a useful counterbalance to earlier idolisation of the scientific method, but in doing so it reopens the core question of philosophy of science: if the scientific method isn’t what makes science work so well, then what does?


Strevens’ core idea is to strip down scientific methodology to the bare basics. Instead of trying to understand the success of science as being due to a shared methodology for generating theories, or for designing experiments, or for interpreting evidence, or for rejecting theories, we should understand it as being due to a shared norm about what types of evidence to accept. He calls it the Iron Rule of Explanation: resolve disagreements via empirical tests. The law results in a strict separation between “private” and “public” science: scientists in private proceed on the basis of hunches, aesthetic intuitions, rivalries, visions from god, or whatever other motivations they like. But in public, these are all sterilised: only empirical tests count. This forces scientists to search more and more deeply for key empirical data, rather than trying to build castles of arguments which aren’t ever tested - and which can therefore be washed away even after millennia of work, as the example of theology shows. In a particularly evocative passage, Strevens portrays science as a communal process for producing solid facts:


Science, then, is built up like a coral reef. Individual scientists are the polyps, secreting a shelly carapace that they bequeath to the reef upon their departure. That carapace is the sterilized public record of their research, a compilation of observation or experimentation and the explanatory derivation, where possible, of the data from known theories and auxiliary assumptions. The scientist, like a polyp, is a complete living thing, all too human in just the ways that the historians and sociologists of science have described. When the organism goes, however, its humanity goes with it. What is left is the evidential exoskeleton of a scientific career.


Strevens dates the birth of the iron rule to Newton, and in particular the famous passage where he says that he will “feign no hypotheses” about why the laws of gravity work the way they do. Newton thereby accepts a shallower and more instrumental conception of “explanation” than previous scientists, who searched for theories built on mechanisms that made intuitive sense. Strevens claims that the counterintuitive nature of the iron rule is why it took so long for science to get started. Shouldn’t the lack of a mechanism which implements Newtonian gravity be a significant strike against it? Shouldn’t the intuitions which led us to a theory be a key part of our arguments for it? And why can’t our other beliefs - say, about the existence of god - help inform our scientific theories?


Strevens agrees that excluding information which we believe is relevant is, in a sense, irrational (hence the book’s subtitle: “how irrationality gave birth to modern science”). But he argues that it’s necessary for the success of science, because it pushes scientists towards doing the difficult work required to find the truth:


We live in a Tychonic world - a world in which great competing stories about the underlying nature of things can be distinguished by, and only by, scrutinizing subtle intricacies and minute differences. Humans in their natural state are not much disposed to attend to such trifles. But they love to win. The procedural consensus imposed by the iron rule creates a dramatic contest within which the trifles acquire an unnatural luster, becoming, for their tactical worth, objects of fierce desire. The rule in this way redirects great quantities of energy that might have gone toward philosophical or other forms of argument into empirical testing. Modern science’s human raw material is molded into a strike force of unnervingly single-minded observers, measurers, and experimenters, generating a vast, detailed, varied, discriminating stock of evidence.


I think this explanation points in the right direction - but it’s incomplete. Why do we need the iron rule to create a dramatic contest, rather than just competing to find any type of compelling evidence? It’s true that early thinkers didn’t understand that we lived in a Tychonic world, and so underrated empirical evidence. But after seeing many examples of the power of empirical evidence (and more specifically, the power of advance predictions), why wouldn’t they update towards empirical evidence being a powerful way to identify the truth, without enshrining it as the only way to identify the truth? In other words, Strevens’ proposal of science-as-competition works almost as well without the iron rule, as long as scientists reward progress towards truth in an unbiased way.


So a complete version of Strevens’ explanation needs to identify why scientists will predictably overrate non-empirical evidence for theories, and reward that evidence more than it deserves. There may be a range of sociological considerations in play - for example, if observers tend to underestimate how much work has gone into finding evidence, then the reputational payoff for doing the hardest empirical work might be disproportionately low, meaning that scientists will focus on other ways to win the game. But for now I want to focus on the hypothesis that we find non-empirical arguments more persuasive than we should because of a range of cognitive biases. To illustrate this point, let’s dig into Strevens’ previous book - a more philosophical work named Thinking off your feet: how empirical psychology vindicates armchair philosophy.

The perils of philosophy


Having been so impressed by his latest book, I was surprised by how much I disagreed with this one. I ended up only skimming through most chapters, so take this summary with a pinch of salt, but my impression of Strevens’ case was as follows. The standard mode of inquiry in philosophy, known as conceptual analysis, aims to discover the “essential natures” of things by consulting our intuitions, especially intuitions about complex edge cases. Conceptual analysis has come under fire over the last few decades from skeptics who point out that almost no concepts can be characterised in terms of necessary and sufficient conditions - most of them are inherently vague or imprecise. Strevens agrees with this point. Nevertheless, he claims, conceptual analysis is still useful because the process of trying to identify essential natures helps us understand even entities which lack them.


What’s bizarre is that Strevens sees so clearly the difficulty of science - which forces us to adopt the strict restriction of the iron rule - yet still thinks that philosophy can make progress basically by accident, by aiming at the wrong target entirely. Perhaps this would be compelling if there were historical examples of this working well, but the ones Strevens identifies are underwhelming, to say the least. Consider, for example, the thought experiment of a swan which spontaneously appears due to random particle fluctuations. Strevens claims that arguing about whether this is “really” a swan helps us understand the “causal-explanatory structure” of normal swans - e.g. the ways in which their properties are explained by their ancestry. To be honest, my main response here is an incredulous stare. I have no idea what valuable knowledge about swans biologists lack, which this type of philosophising has provided, or could ever provide. And I don’t think that’s a coincidence - these types of thought experiments are usually designed to steer as far clear from any empirical uncertainties as possible (and sometimes further), to make the conceptual debate clearer.


Or consider Strevens’ analysis of the concepts of belief and desire in everyday psychology. He argues that conceptual analysis is valuable in this case because this approach to psychology is “all or nothing”: in the face of empirical investigation, concepts like belief and desire “either stand firm or suffer a catastrophic collapse”. To me this seems totally wrongheaded - our understanding of belief and desire has undergone extensive shifts as we’ve gradually learned more about things like the subconscious, behavioural reinforcement, addiction, prospect theory, dual process theory, signalling theory, evolutionary psychology, and so on. By contrast, conceptual analysis of belief has been stuck in an unproductive merry-go-round of definitions and counterexamples for decades.


This is not to say that there has been no progress in philosophy - see, for instance, Tom Adamczewski’s list of philosophy success stories. But it seems like Strevens, and many other philosophers, dramatically overestimate how useful philosophy has been. I claim that this is because common cognitive biases (like the bias towards essentialism, and confirmation bias, and hindsight bias) make philosophical arguments seem more insightful than they actually are. And if these biases are common even amongst the brightest thinkers, it answers the question I posed above about why the iron rule is still necessary. By ruling out these types of arguments, the iron rule doesn’t just steer us towards useful research, it also protects us from cognitive biases which make conceptual arguments seem disproportionately valuable.


I don’t want to point the finger only at philosophy; I think many other humanities and social sciences have similar problems. But as one of the fields which makes least use of empirical data, philosophy is a particularly easy clear illustration of my core claim: science succeeds because the iron rule of explanation (“resolve disagreements via empirical tests”) mitigates cognitive and sociological biases in our judgments of the strengths of different types of evidence.


There’s much more room to elaborate on the specific types of biases involved; I do that to some extent in this blog post, and it’s also a core theme of Yudkowsky’s writings (along with how to do better than the standard scientific process). But one point to note is that this formulation assumes that human reasoning is actually pretty good in general - that, if we get rid of these biases, we’re capable of thinking in a broadly reliable way about domains that are very far removed from our everyday experience. So in some sense, an explanation for why science succeeds needs to also be a story about human intelligence, and the mental models which we can build using it. But I’ll save that for another post.