>>>Imagine a device that, over time, learns you so well that it can offload more and more executive function, then more and more fine-grained motor planning, then eventually enough of your cognition that the old tissue is scarcely needed.
But I think in this quote, it seems quite possible the wearers of such devices may no longer have consciousness at all. I don't think their consciousness _necessarily_ has to continue existing in the earing or something, maybe these people just become gradually less and less conscious over time, to the point where they have no conscious experiences and no self-aware thoughts at all.
Thank you. I agree that is possible that consciousness *isn't* transferred, but I mainly set out to argue that the converse isn't proven true either. My main intent is to show that the standard reading of assuming the earring is truly malicious (instead of reasonably benevolent and misunderstood) is shakier than it sounds on closer inspection.
Consider it an argument made on the balance of probability, rather than one where I feel confident mustering the full force of Empiricism, Science and Reason. We won't know for sure until we make better BCIs, or at least very good AGI/ASI.
Good analysis. I think you are right that for the wearer, the morality is up to them: how much do you value perceived autonomy. But in the context of a soon to be possible LLM advisor, the morality question is more: will others demand that people wear them, so as to protect themselves from negative externalities from bad decisions? Or only in cases of demonstrated bad choices, like a slap drone from Banks culture?
Will people demand them? Maybe. Probably? There's a lot of people out there, but I presume you mean a super-majority of society. My AI timelines are short enough that I think there simply isn't enough time for LLMs to take over social interactions or life choices specifically before they're simply automating *everything*. At that point? If we're talking genuine AGI or ASI, you probably are better off heeding its advice, and it presumably (if aligned) will tailor its advice for you. If you're the kind of person who would be better off without complete reliance on AI, it'll tell you not to rely on it. And at least I'm considering a post-scarcity world where genuine competition at the human level quickly becomes obsolete (or at least it's a form of capitalism where humans need not apply).
The sad part is, there simply are too many moving parts to forecast how things will turn out, beyond my pretty strong confidence that we're getting AGI in the near term anyway.
"My AI timelines are short enough that I think there simply isn't enough time for LLMs to take over social interactions or life choices"
Well you may be right, but even right now, if you typed a daily diary into claude and asked it what to do in major decisions, what % of people would that result in better outcomes for? I feel like there will be at least SOME time between then and now for these mechanics to play out.
I dare say 90% of people, if they're using Opus 4.7. In fact, I've done this myself. It's helped a lot, just confounded by the fact that I have ADHD and depression. I believe I still retain my frontal lobe, no promises on how atrophied it might be though.
I enjoyed this a lot! reading this i had a few responses, and boiled them into topics:
1) Good dismissal of lazy earring critiques
So many people have a instinctual suspicion of the earring and grasp for justification later. Leads to misinterpretations like the ring is lying about promises and subverting values. I agree with you that this is a lazy cop-out. To properly engage with the earring parable we must respond even in the cases where it truly does enact our desires.
Reminds me of lazy critiques of consequentialism trying to argue that the utilitarians will lead to bad outcomes. They are conceding before they’ve even begun! To truly argue against this requires accepting that the ring is still a bad idea for some reason. Which is a much more difficult task. Stories like the Egan you cite aren’t relevant to the parable when they’re talking about practical failure-modes.
In some sense it’s a revisit of the question of evil. Why do bad things happen aka Why not simply put on the earring and push the fix everything button. Even the slatestarcodex Answer to Job admits that worlds without suffering are preferable.
This is the real strength of Dostoevsky’s Notes From the Underground; He actually argues that man has interest in suffering itself. Perhaps he does not succeed and the underground man does not seem so enviable, but the work is unique (to my limited knowledge) of truly biting the bullet.
2) Comparison to real life
ADHD is good example for your argument. People still argue against taking medication, but it’s comically evil to prevent adhd patients from accessing medicine because they ‘are ungrateful for what little freedom they have’. And the mechanical fear of externalizing feels hollow from rationalists who got convinced about ‘systematized winning’ from a blog post, or enjoy ‘shut up and calculate’ strategies. Like, buddy you’re already outsourcing your decisions into an otherworldly decision maker, it’s called game theory.
But these real world comparisons miss the parable. This is not a real world problem. The Cold Rain doesn’t say “never make a shorter path”, it says “never take the shortest path”. You can try taking the shortest path for a million years and never come close to succeeding. What kind of insane mind looks at all the problems in the world and complains about what happens when we run out of them? Of course it’s philosophers.
The closest mortals get to experiencing these questions is during creative acts, where the gaps between actual and potential is thinnest. When an amateur writer sits down at a typewriter they usually end up with some kind of marty-sue character who trivially wins everything and everyone loves them. Satisfying for the writer’s fantasies, but rarely interesting to read for anyone else. Not even for the character themselves sometimes! (see Watchmen or Megamind).
For more practical case-law, consider the decision of whether to ban LowInputRuns in trackmania. Speedrunning itself is a narrow domain where we’ve pushed up against human limits, which makes for interesting discussion. This video has an unintentional easter egg when a detractor complains the map they love is reduced to a piano-tiles (@12m30s), and Dostoevsky wrote “the whole work of man really seems to consist in nothing but proving to himself every minute that he is a man and not a piano-key!” https://www.youtube.com/watch?v=Jkft49A8pOk
3) Folklore of why a non-benevolent earring would warn against wearing
You argue that a predatory ring could just lie and would have no reason to perform this minimal informed consent and would not have allowed itself to be sealed in the vault without a fight. This behavior rules out animal like spirits and undirected deceivers, but it does not rule out other folk dangers.
For example: vampires require an invitation to enter a building. They are intelligent predators who can lie about other things, but can’t enter a sacred space without explicit acknowledgement. The narrative meaning of such a limitation is a view where possession must be invited. Even prayer is asked for, and a person wearing a WWJD bracelet must invite the changes.
This earring gives a classic Faustian bargain: material good-life in exchange for servitude. It obtains consent in an archaic fashion and then follows through with everything your heart desires. To a traveling wiseman, this sets off more alarm bells than an animalistic intelligence scared of being locked away. Should you find yourself placed in a parable such as this I hope you’re genre aware enough for some doubt before bending the knee.
God. I feel terrible for seeing your reply at a time when I am absolutely exhausted and don't have the energy to give it the response it deserves. TLDR, very good points, and not just because you mostly agree with me.
1. Agreed on utilitarianism/consequentialism. A *sane*, intelligent utilitarian, if provided with robust evidence that a different approach to ethics provides better results, would simply appropriate it on utilitarian grounds. We're computationally bounded entities, after all, and if something simpler like rule utiliarianism is more tractable/practical, we should use it. On the record, I've called deontology "utiliarianism for dummies", mostly because I don't think I'm a dummy, and to encourage people who can work through second and third order consequences semi-reliably to actually attempt that exercise.
2. I haven't read Notes From The Underground (yet). Even worse, I can't see the rest of your comment while typing this, since Substack helpfully makes my draft take up all the space on screen (while using the mobile app). Sorry, I hope I get around to a second comment at some point, but if not, this is the kind of engagement I crave, the kind which makes this exercise worth it. Thanks!
I appreciate it 👍. I wanted to highlight agreement because even though I explore the positions against the earring, I still don't have an actual argument for why not just put it on. That question is the jewel of the parable and requires cutting away these practical arguments against it.
Notes from the Underground is a neat read. I liked part 1 and admit I don't care for part 2 at all. Character study. The earring in 1860's socialism is known as The Crystal Palace.
Great article.
>>>Imagine a device that, over time, learns you so well that it can offload more and more executive function, then more and more fine-grained motor planning, then eventually enough of your cognition that the old tissue is scarcely needed.
But I think in this quote, it seems quite possible the wearers of such devices may no longer have consciousness at all. I don't think their consciousness _necessarily_ has to continue existing in the earing or something, maybe these people just become gradually less and less conscious over time, to the point where they have no conscious experiences and no self-aware thoughts at all.
Thank you. I agree that is possible that consciousness *isn't* transferred, but I mainly set out to argue that the converse isn't proven true either. My main intent is to show that the standard reading of assuming the earring is truly malicious (instead of reasonably benevolent and misunderstood) is shakier than it sounds on closer inspection.
Consider it an argument made on the balance of probability, rather than one where I feel confident mustering the full force of Empiricism, Science and Reason. We won't know for sure until we make better BCIs, or at least very good AGI/ASI.
Good analysis. I think you are right that for the wearer, the morality is up to them: how much do you value perceived autonomy. But in the context of a soon to be possible LLM advisor, the morality question is more: will others demand that people wear them, so as to protect themselves from negative externalities from bad decisions? Or only in cases of demonstrated bad choices, like a slap drone from Banks culture?
Will people demand them? Maybe. Probably? There's a lot of people out there, but I presume you mean a super-majority of society. My AI timelines are short enough that I think there simply isn't enough time for LLMs to take over social interactions or life choices specifically before they're simply automating *everything*. At that point? If we're talking genuine AGI or ASI, you probably are better off heeding its advice, and it presumably (if aligned) will tailor its advice for you. If you're the kind of person who would be better off without complete reliance on AI, it'll tell you not to rely on it. And at least I'm considering a post-scarcity world where genuine competition at the human level quickly becomes obsolete (or at least it's a form of capitalism where humans need not apply).
The sad part is, there simply are too many moving parts to forecast how things will turn out, beyond my pretty strong confidence that we're getting AGI in the near term anyway.
"My AI timelines are short enough that I think there simply isn't enough time for LLMs to take over social interactions or life choices"
Well you may be right, but even right now, if you typed a daily diary into claude and asked it what to do in major decisions, what % of people would that result in better outcomes for? I feel like there will be at least SOME time between then and now for these mechanics to play out.
I dare say 90% of people, if they're using Opus 4.7. In fact, I've done this myself. It's helped a lot, just confounded by the fact that I have ADHD and depression. I believe I still retain my frontal lobe, no promises on how atrophied it might be though.
Haha yes, I think you are right that part was literary flourish rather than prediction!
I enjoyed this a lot! reading this i had a few responses, and boiled them into topics:
1) Good dismissal of lazy earring critiques
So many people have a instinctual suspicion of the earring and grasp for justification later. Leads to misinterpretations like the ring is lying about promises and subverting values. I agree with you that this is a lazy cop-out. To properly engage with the earring parable we must respond even in the cases where it truly does enact our desires.
Reminds me of lazy critiques of consequentialism trying to argue that the utilitarians will lead to bad outcomes. They are conceding before they’ve even begun! To truly argue against this requires accepting that the ring is still a bad idea for some reason. Which is a much more difficult task. Stories like the Egan you cite aren’t relevant to the parable when they’re talking about practical failure-modes.
In some sense it’s a revisit of the question of evil. Why do bad things happen aka Why not simply put on the earring and push the fix everything button. Even the slatestarcodex Answer to Job admits that worlds without suffering are preferable.
This is the real strength of Dostoevsky’s Notes From the Underground; He actually argues that man has interest in suffering itself. Perhaps he does not succeed and the underground man does not seem so enviable, but the work is unique (to my limited knowledge) of truly biting the bullet.
2) Comparison to real life
ADHD is good example for your argument. People still argue against taking medication, but it’s comically evil to prevent adhd patients from accessing medicine because they ‘are ungrateful for what little freedom they have’. And the mechanical fear of externalizing feels hollow from rationalists who got convinced about ‘systematized winning’ from a blog post, or enjoy ‘shut up and calculate’ strategies. Like, buddy you’re already outsourcing your decisions into an otherworldly decision maker, it’s called game theory.
But these real world comparisons miss the parable. This is not a real world problem. The Cold Rain doesn’t say “never make a shorter path”, it says “never take the shortest path”. You can try taking the shortest path for a million years and never come close to succeeding. What kind of insane mind looks at all the problems in the world and complains about what happens when we run out of them? Of course it’s philosophers.
The closest mortals get to experiencing these questions is during creative acts, where the gaps between actual and potential is thinnest. When an amateur writer sits down at a typewriter they usually end up with some kind of marty-sue character who trivially wins everything and everyone loves them. Satisfying for the writer’s fantasies, but rarely interesting to read for anyone else. Not even for the character themselves sometimes! (see Watchmen or Megamind).
For more practical case-law, consider the decision of whether to ban LowInputRuns in trackmania. Speedrunning itself is a narrow domain where we’ve pushed up against human limits, which makes for interesting discussion. This video has an unintentional easter egg when a detractor complains the map they love is reduced to a piano-tiles (@12m30s), and Dostoevsky wrote “the whole work of man really seems to consist in nothing but proving to himself every minute that he is a man and not a piano-key!” https://www.youtube.com/watch?v=Jkft49A8pOk
3) Folklore of why a non-benevolent earring would warn against wearing
You argue that a predatory ring could just lie and would have no reason to perform this minimal informed consent and would not have allowed itself to be sealed in the vault without a fight. This behavior rules out animal like spirits and undirected deceivers, but it does not rule out other folk dangers.
For example: vampires require an invitation to enter a building. They are intelligent predators who can lie about other things, but can’t enter a sacred space without explicit acknowledgement. The narrative meaning of such a limitation is a view where possession must be invited. Even prayer is asked for, and a person wearing a WWJD bracelet must invite the changes.
This earring gives a classic Faustian bargain: material good-life in exchange for servitude. It obtains consent in an archaic fashion and then follows through with everything your heart desires. To a traveling wiseman, this sets off more alarm bells than an animalistic intelligence scared of being locked away. Should you find yourself placed in a parable such as this I hope you’re genre aware enough for some doubt before bending the knee.
God. I feel terrible for seeing your reply at a time when I am absolutely exhausted and don't have the energy to give it the response it deserves. TLDR, very good points, and not just because you mostly agree with me.
1. Agreed on utilitarianism/consequentialism. A *sane*, intelligent utilitarian, if provided with robust evidence that a different approach to ethics provides better results, would simply appropriate it on utilitarian grounds. We're computationally bounded entities, after all, and if something simpler like rule utiliarianism is more tractable/practical, we should use it. On the record, I've called deontology "utiliarianism for dummies", mostly because I don't think I'm a dummy, and to encourage people who can work through second and third order consequences semi-reliably to actually attempt that exercise.
2. I haven't read Notes From The Underground (yet). Even worse, I can't see the rest of your comment while typing this, since Substack helpfully makes my draft take up all the space on screen (while using the mobile app). Sorry, I hope I get around to a second comment at some point, but if not, this is the kind of engagement I crave, the kind which makes this exercise worth it. Thanks!
I appreciate it 👍. I wanted to highlight agreement because even though I explore the positions against the earring, I still don't have an actual argument for why not just put it on. That question is the jewel of the parable and requires cutting away these practical arguments against it.
Notes from the Underground is a neat read. I liked part 1 and admit I don't care for part 2 at all. Character study. The earring in 1860's socialism is known as The Crystal Palace.
This reads like a transhumanist manifesto. great work