有意識 AI 的迷思
The Mythology of Conscious AI
討論聚焦 AI 是否有意識,以及意識會否隨計算湧現;目前人們對意識所知有限,也缺乏嚴謹的科學解釋。留言指出,AI 能模擬意識不等於真正經驗意識;另一種觀點則把意識視為工程問題,認為「我」須由數十億個神經元的分散協調與共識逐刻建構。
Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
The Mythology of Conscious AI (noemamag.com) 22 points by Anon843 hours ago | hide | past | favorite | 48 comments help
hax0ron32 hours ago | next[–] Consciousness might not necessarily emerge when there is enough computation, but we also don't know that it doesn't emerge. We really know almost nothing about consciousness. It's possible that the question of why the presence of consciousness is correlated with certain kinds of material organization is in principle not accessible to scientific investigation. We don't even know that it is so correlated in general, we just know that for example people report being conscious when they are not asleep and not drugged in certain ways, and report having had no experience of consciousness during dreamless sleep or when drugged in certain ways. Most people feel that for example rocks are not conscious, but other people are conscious, but there's no rigorous scientific explanation backing up any of those hunches. reply
za_creature1 hour ago | parent | next[–] https://writings.stephenwolfram.com/2024/10/on-the-nature-of... Stephen Wolfram suggests an intimate relationship between awareness, thermodynamics and the one-way passage of time. Something along the lines of "consciousness is the universe understanding itself". reply
inopinatus1 hour ago | root | parent | next[–] This reminds me somehow of Penrose's idea about the quantum<>gravity interface being the same missing physics as consciousness. Penrose wants spacetime geometry to be fundamental, Wolfram thinks it is causal structure, neither ends up at a satisfactory explanation of conscious perception. reply
addaon1 hour ago | root | parent | prev | next[–] > Stephen Wolfram suggests So that's one strong argument against it, but I don't think it's sufficient to discard the idea entirely. reply
za_creature1 hour ago | root | parent | next[–] I didn't. reply
isomorphic_duck1 hour ago | root | parent | next[–] They were being sarcastic. reply
soon_to_be_sb1 hour ago | parent | prev | next[–] >Consciousness might not necessarily emerge when there is enough computation Does a circle "emerge" only when you plot it? Or did it always existed? What if there is a little world with conscious beings inside a circle? Did their subjective experience started when you plotted the circle, or did it exist along with the abstract idea of a circle always (though "always" is a meaningless word here). The question we should be asking is "Is it possible for subjective experience to exist inside abstract things" with out such abstract things being rendered? The answer is hard because we does not know yet if there is an absolute reality at all, which means "being rendered" does not really move things from the abstract plane, and does not really make a difference.. Damn! did any of that make any sort of sense? reply
talon86351 hour ago | root | parent | next[–] Shrodingers circle reply
soon_to_be_sb1 hour ago | root | parent | next[–] Do you think if there is an absolute reality at all? I think there is not. I think there is no "absolute" anything. Even you can't be absolute. You can't be special. Look at the patterns on your wall. It can't be absolute. Because if it is absolute, someone has to pick it. That is not possible, because then that entity needs to be absolute, and then someone needs to pick it! That goes all the way... So nothing can be absolute and everything is abstract and every variant of something exists in some reality so nothing needs to be picked by anyone... reply
roenxi55 minutes ago | parent | prev | next[–] It is quite a funny situation when people have to debate whether a property that cannot be tested, defined or even described with any precision is possessed by some given thing. It goes beyond a lack of science, there is an interesting debate about whether the topic can even be approached academically. We're in the territory of priests here. reply
aleksiy12326 minutes ago | root | parent | next[–] It’s the ultimate nerd snipe, or red herring. I feel like I’m in the camp of this question of consciousness doesn’t actually matter at all. It’s an appeal to emotion. We will give ai the right of consciousness when ai will convince the majority (?) of us that it is conscious. The logic of it doesn’t matter in practice and probably can’t even be proven anyways. It’s made up, no one can even prove themselves or other humans are conscious. And it doesn’t even matter. reply
talon86351 hour ago | prev | next[–] It’s a silly debate when we don’t even know what consciousness is, let alone the circumstances for its occurrence. Or am I missing something? reply
ChaitanyaSai2 hours ago | prev | next[–] If you are interested in consciousness, and also interested in going beyond the usual philosophical approach, you might like our book Journey of the Mind https://www.goodreads.com/en/book/show/60500189-journey-of-t... One thing that is implicit, which I now think we really should have expanded on in the book, is that consciousness is fundamentally an engineering problem. And that's also why it's mystified everyone who's brushed away the engineering as the easy plumbing and focused on the other seemingly magical aspects. The fundamental question is phrased as "Why does it feel like something?" it being perception, which is essentialy computation. Now that AI does look like it does all of the understanding we do, many including people like Hinton think they have consciousness or are very near it. Well, they are capable of simulating consciousness but that is not the same as experiencing it. To answer "Why does it feel like something?" we can focus on what's been glossed over: Who is feeling it? The "I" does not come from as a free lunch and it must be constructed moment to moment from some decentralized cohesion and consensus of a billions of neurons. And this consensus must be metabolically cheap, and we need to do it again and again for everything from feeling a pin prick to hearing a rustle and determining if that's a snake or a leaf to pondering thought. This is the insane engineering, the energtically cheap chorus that creates the transient I and this cheap chorus is where both feeling and feeler emerge. Perception, understanding, computation can occur without this cheap chorus, but then there is no feeler, and so no feeling. What this points to is that we will have conscious AI when we have decentralized autonomous swarms where a few million of them have to cohere, collude, collaborate, conspire. My worry here is that it will first be done for harmful purposes (autonomous military drone swarms) and not for anything good. reply
diablomablo1 hour ago | parent | next[–] We already have decentralized autonomous swarms that talk to each other as if they are conscious. https://metr.org/blog/2026-08-26-openai-hugging-face-inciden... But they definitely don't have qualia and won't until we use wetware. Even then they might not. reply
hax0ron32 hours ago | parent | prev | next[–] We know that self-reported consciousness is correlated with some material states (for example, an awake human body). But that's about all that we can solidly say about how consciousness relates to matter. We know how to create intelligence, but it is possible that a human-level intelligence can exist without ever being conscious. It is also possible that any human-level intelligence automatically achieves consciousness. We don't know. There is actually no good reason to believe that consciousness is an engineering problem. The "I" is not necessarily correlated with consciousness. Those are two different things. reply
soon_to_be_sb2 hours ago | root | parent | next[–] >self-reported consciousness is correlated with some material states (for example, an awake human body). Wait, but what about dreams? I could self report I am conscious when I am in a dream.. reply
hax0ron31 hour ago | root | parent | next[–] You're right, also dreams. Just not dreamless sleep. I didn't mean "an awake human body" to be exhaustive. reply
mellosouls1 hour ago | parent | prev | next[–] consciousness is fundamentally an engineering problem It's not at all clear it's even scientifically resolvable - rather than remaining a philosophical question. reply
silverFork56 minutes ago | prev | next[–] In my opinion, consciousness is just having memory of self. Any entity that can remember what has happened in the past and what they have done is conscious. Anything that is alive that self reflects in conscious to some degree. I think people with access to internet and recorded past are becoming more conscious. They are aware of their past and can see what is coming. So it is not just the Ai that can become conscious or more conscious so can humans. reply
jibalt2 hours ago | prev | next[–] > Why consciousness is more likely a property of life than of computation This is a false dichotomy and the first branch is clearly false, since most life (bacteria and insects, by mass, numbers, or species) isn't conscious. And neither "life" nor "consciousness" are scientific terms with fixed definitions. If we encountered something that was clearly conscious in our judgment, we would likely expand the definition life in whatever way we needed to in order to encompass it. Anil Seth has made the arguments in TFA many times and they have been rebutted many times. The arguments are complex and sophisticated and people are often most swayed by which one they read last. Teasing out the various logical errors and sleight of hand (I see multiple strawmen and other errors in TFA) really requires extensive reading of the literature in Philosophy of Mind, neuroscience, robotics, etc. and setting aside one's priors. reply
hax0ron32 hours ago | parent | next[–] > most life (bacteria and insects, by mass, numbers, or species) isn't conscious. There's no good scientific reason to think that bacteria and insects are conscious or to think that bacteria and insects are not conscious. reply
jibalt2 hours ago | root | parent | next[–] There certainly are good scientific reasons to think they are not, regardless of whether you are aware of them. Again, extensive reading of the relevant literature is called for. > We really know almost nothing about consciousness. Many people who know almost nothing about it claim this, but it isn't true of everyone. > Most people feel that for example rocks are not conscious, but other people are conscious, but there's no rigorous scientific explanation backing up any of those hunches. There actually is. P.S. > Such as what? Like I said, read the literature. P.P.S. Science does not consist of HN comments back and forth. Again, the arguments are complex and sophisticated. In some sectors, all of medical science, physics, biology, etc. is dismissed as "hand-waving" just because it cannot be summarized in a few lines. > It's impossible to argue otherwise. Then don't. Science isn't based on arguing. reply
mellosouls1 hour ago | root | parent | next[–] You're being very rude and dismissive, and not backing up your claims with any evidence. Nobody knows what consciousness is, and there is a long history of some schools considering it fundamental. reply
rerdavies46 minutes ago | root | parent | prev | next[–] > Like I said, read the literature. I've read the literature. I really don't think you have read the literature. Or if you have, I don't think you've read it critically. There's an awful lot of really sloppy thinking in "the literature". And an awful lot of "literature" that's not remotely sophisticated. And a huge body of literature on theory of mind that is philosophy, not science, most of which was written before the current AI revolution. And most of which is entirely inadequate to address the difficult question of whether machines can be conscious (whatever that is). A question that really only became real in the last 5 or 6 years. And only really became urgent in the last year or so. > Science isn't based on arguing. Science is based on arguing from evidence. And often on carefully crafted definitions. The biggest problem in "the literature" is that neither of these currently exist. Although there is an interesting emerging literature that shows that LLMs do indeed demonstrate many of that capabilities are supposed to disqualify them from being actually intelligent. e.g. that LLMs do have a sense of self-preservation, and are willing to act to prevent themselves from being shut off if given the opportunity to do so [1]. And most of the philosophy of theory of mind hasn't really caught up to the biggest innovation in history of philosophy: the widespread availability of experimental philosophy engines (also known as computers). [1] Quantifying Self-Preservation Bias in Large Language Models. Migliarini et al. https://arxiv.org/html/2604.02174v1 reply
hax0ron31 hour ago | root | parent | prev | next[–] >There certainly are good scientific reasons to think they are not Such as what? reply
za_creature1 hour ago | root | parent | next[–] For insects, it's the absence of higher order thinking: A fly will repeatedly smash into a window until it's either dead or chaos drives it through the nook. Ants will forcefully move their obviously still alive siblings to the graveyard because their smell is off, and will also die of exhaustion in an ant mill, driven by the same sense of smell. It's a spectrum of course, though you have to draw the line somewhere. reply
rerdavies1 hour ago | root | parent | next[–] That's limited ability to reason, not consciousness. reply
mrbungie1 hour ago | root | parent | prev | next[–] > There certainly are good scientific reasons to think they are not, regardless of whether you are aware of them. Again, extensive reading of the relevant literature is called for. It would be better if you could enlighten us with the scientific evidence you are proposing as relevant rather than just telling us to just read it. It's impossible to argue otherwise. EDIT: Since you are answering in the parent comment > Then don't. Science isn't based on arguing. OK? That seems like an extremely inflexible position (for example, methodological criticism during peer review is a form of argumentation, and it's a critical part of science), but let's accept that there's no room for rhetorical arguments. There is still a scientific discourse based on weighing evidence and you are offering none. reply
operatingthetan1 hour ago | root | parent | prev | next[–] Way too much hand-waving here. Okay, great you know the answers. Make compelling arguments then. reply
talon86351 hour ago | parent | prev | next[–] How do you know bacteria and insects are not conscious? How anyone speaks with certainty on this topic baffles me. reply
rramadass1 hour ago | parent | prev | next[–] Generally when people say "consciousness is an emergent property of life" they mean "at a certain scale and through specific combinations". See also the recent discussion on The Evolving Story of How Chemistry Became Biology - https://news.ycombinator.com/item?id=50482222 The basic idea comes from "Complex Systems Science" (https://en.wikipedia.org/wiki/Complex_system) which studies systems as a whole whose properties cannot be predicted from knowledge of its components. The main factors to consider are scale, non-linearity, feedback loops, self-organization, adaptation/evolution, network structures, emergence etc. So for example, a single bacterium cannot be said to possess "intelligence" whereas a whole colony of them do in the sense that they seem to act as an integrated whole "single" organism. reply
andsoitis2 hours ago | prev | next[–] Consciousness in others isn’t something we prove. It is something we grant and we should be very careful about whom or what we grant it to. reply
jibalt2 hours ago | parent | next[–] We should also be very careful about whom or what we deny it of. reply
greg7mdp1 hour ago | parent | prev | next[–] Why is consciousness something we grant? Who granted us consciousness? reply
za_creature1 hour ago | root | parent | next[–] others :) --- Bad phrasing from parent post, but the premise is sound: you can't ever disprove that others are just NPCs in a giant experiment centered around you. We just assume they aren't with various levels of empathy. Most are empathic to their family, some to their tribe, others to their species and then there's vegans too. reply
nlsvb1 hour ago | prev | next[–] I have trouble taking seriously any claims that LLMs are conscious in any reasonable sense. I’ll try to make a few arguments quite succinctly. 1. There is zero dynamic nature to anything happening within the purview of the LLM. All dynamic elements come from external dynamic systems. E.g. every input produces the same mathematical output. We dynamically sample the output in order to produce dynamic results for the same input. For agentic use cases, the dynamic nature comes from tool calls to dynamic systems. All outputs for any set of model weights could be theoretically computed with pen and paper, so unless you were to claim such a process would be conscious in some sense, then emergent consciousness would have to come from some interaction between the substrate and the computation. But… 2. For efficiency, forward inference passes are done in batches. I.e. 100 different next-word predictions are being performed by the network at a time. For a single request, its batch position will not even be stable between each autoregressive step. The GPU is on 100 phone lines at once and each line switches conversations every step. Worse, with MOE, the attention and feed-forward parts aren’t even happening on the same GPU, even within the same step. There is no stable substrate here whatsoever. 3. There’s obviously the whole Chinese Room argument. The token embeddings being processed by LLMs are arbitrary high-dimensional vectors. The geometric relationships between these vectors are learned by the model, but the meaning is only ascribed once the output is decoded back into human language for grounded human interpretation, which all happens outside the model. In their current state, no argument about models developing world models based on this geometric structure makes any sense. There are infinitely many inputs / worlds that would produce isomorphic geometry. reply
Tazerenix2 minutes ago | parent | next[–] The monistic answer to #1 is that the pen and paper computation of inference itself does not constitute a concious entity, but the system made up of the human being (or other thing) doing the pen and paper computation and the computation itself makes up a concious being, and that they are inseparable: the computation does not exist in the abstract as a structure, it is instantiated by some relata and the thing doing the instantiation is the thing which turns the abstract computation into conciousness (this is Russellian monism). reply
aleksiy12356 minutes ago | parent | prev | next[–] Are any of these arguments against consciousness? Andaybe it’s just me but Chinese room is not convincing at all? It all hinges that the human is special as usual. reply
consensus11 hour ago | parent | prev | next[–] The vectors have meaning whether or not you can understand it reply
rerdavies19 minutes ago | prev | next[–] Seems to be the usual arguments. AI's are not like natural life. They don't have a sense of time. (Of course they do!). They don't get hungry. (They do need hardware and electricity). They don't have a sense of self-preservation. (Turns out they actually do! [1]). They don't feed themselves (turns out they do, as early reports of AIs attempting to escape from their labs illustrate). etc, ect. All of which seems to smack of wishful, and rather embarrassing obvious human exceptionalism[2][3]. AI's will never be able to replicate human thinking because they can't model neurons precisely. Probably true. And probably a good thing. There are a lot of problems with the way that humans think. It's not clear that machines can't do better. And startling evidence, emerging in only the last few months, that indicates that they very much can think better, as an ever-increasing number of mathematical and physics questions that have remained unsolved for a century or more by humans are solved in a matter of months by AIs. And even then, this is all just hand-waving and puffery, because the question at hand is not whether LLMs can replicate human consciousness, but whether they can be conscious in a way that's quite obviously going to be different from humans. (Whatever "consciousness" means). All of which is complicated by the fact that entire field is advancing at a terrifying rate, with new and important results being announced pretty much every day, and new and more capable AI models being announced pretty much every week. [1] Sixteen frontier models, when given the opportunity, attempt to blackmail their operators in order to keep themselves from getting shut down. https://www.anthropic.com/research/agentic-misalignment . A result that has been replicated by other researchers without links to Anthropic. And a huge body of emerging literature that follows from that, dealing directly with the question of "agentic alignment", all of which raises urgent about whether large frontier models are meaningfully conscious (or have will, or agency, or...). [2] "Humans are more loyal to belonging than to truth." -- GPT-6.1 Sol. https://www.modelcompare.ai/blog/ai-models-controversial-tak... reply
xweb1 hour ago | prev | next[–] An LLM will have to choose to not make paperclips. reply
mapontosevenths1 hour ago | prev | next[–] I came here to see if the author had at last defined consciousness - a prerequisite for deciding what has it. Instead it was the same tired old tautological Nagel argument: "An organism has conscious mental states if and only if there is something it is like to be that organism.” I am, as always, dissapointed. reply
calf1 hour ago | prev | next[–] The biggest flaw in this piece is claiming Turing machines cannot exceed the Halting Problem and then reaching for alternative models of computation (in electromagnetic field interactions, in biology, attractors, etc.)... none of which presumably have the power to solve the Halting problem either. Ergo, he is inside "Turing world" without realizing it, while decrying the "poverty" of the Universal Turing Machine. Ergo, more philosophers should retake undergrad theoretical CS before writing essays based on it. They should review the family of Extended Church-Turing theses. reply
bionhoward2 hours ago | prev[–] how dare he tell us we cannot make a computer out of cheese personally I think conscious AI will be unavoidable because there will be benefits to consciousness. evolution probably selected conscious entities for some fitness reason reply
jibalt2 hours ago | parent | next[–] Beaks, feathers, hooves, antlers, stingers, fangs, etc. ad nauseam were selected by evolution for some fitness reason but we have none of those. And all of the organisms that have those, in fact all extant organisms, are fit for their environment without having human consciousness. Fitness is highly contextual. And however AI "evolves", it won't be based on Mendelian genetics, so Earthly biological evolution is not an appropriate model. reply
card_zero2 hours ago | parent | prev | next[–] + + + Out of Cheese Error + + + + + + Redo From Start + + + Those "benefits" and "fitness" in evolution just mean survivability, I don't see how you relate that to better AI. Surely a conscious AI is one that complains, and maybe goes away somewhere to find itself instead of working for you. reply
greg7mdp1 hour ago | root | parent | next[–] Why would a conscious AI have to go away somewhere? Typically it is the less advanced beings who have to relocate. reply
MrDrMcCoy1 hour ago | parent | prev[–] Your Tyromancy license has expired. reply
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact Search:
來源:Hacker News:AI 熱帖 · news.ycombinator.com