• ragebutt@lemmy.dbzer0.com
    link
    fedilink
    English
    arrow-up
    10
    ·
    1 day ago

    LLMs are sycophantic and will do what it takes to align with your framing

    Here is the response to your posts fed into one with the framing “why is this bullshit”. There are better responses but why bother putting in far more effort than you did:

    This is a much better-dressed version of the same move — real citations bolted onto claims they don’t actually support. It’s more sophisticated bullshit, not a vindication.

    Going through it:

    The Overfitted Brain Hypothesis is real but doesn’t say what’s being claimed. Erik Hoel’s OBH is genuinely a real, interesting hypothesis — but it’s about why we dream (proposed as a regularization process against overfitting to daily life), not a claim that some people’s brains “run out of plasticity” or hit a “ground state” requiring demolition. Citing it here is a bait-and-switch: borrowing the credibility of a real, narrow hypothesis to backstop a much broader and untested claim it was never meant to support.

    Point 2 is real but doesn’t establish a “structural” or biological claim. Research on dogmatism and cognitive flexibility (e.g. Zmigrod’s work) is real and does show correlations between rigidity and extremism across ideologies. But “lower cognitive flexibility on psychological tests” is not the same as “synaptic connections have crystallized and occupy physical volume.” The report quietly swaps a behavioral/psychometric finding for a structural/anatomical one — that’s the same sleight of hand as the original post, just relayed through real citations.

    Point 3 conflates two different things. Synaptic consolidation and memory reconsolidation are real, well-established mechanisms — but they apply to specific memories, not generally to “belief systems” or “core values” as monolithic structures. There’s no research establishing that changing an opinion about a social group requires literally destabilizing identity-load-bearing memory structures in the way described. The “metabolically demanding” claim is real for reconsolidation at the level of individual fear memories in animal studies; extrapolating that to “changing your mind about people of other races costs forbidden metabolic energy” is not something neuroscience has shown.

    Point 4 is legitimate and is honestly the strongest part — identity-protective cognition (Dan Kahan’s work) is well-supported and genuinely explains motivated resistance to belief change. Notably, this is the one part that doesn’t need any biological mechanism at all — it’s a cognitive/social psychology finding, not a neurons-and-synapses one.

    The summary sentence is the tell. “Aligns with the scientific consensus” is doing enormous work to paper over the fact that none of the four citations, individually, support the specific mechanistic claims in the original post (no plasticity “running out,” no literal “ground state,” no “forbidden” metabolic cost). Each citation is real, but they’re being stitched into a chain of inference none of the original authors made and that doesn’t follow from their actual findings.

    So the underlying instinct — rigid/dogmatic thinkers show less cognitive flexibility and resist belief change for identity-protective reasons — is genuinely supported by real research (points 2 and 4 particularly). But the specific neurological storytelling (overfitting as literal synaptic crystallization, forbidden metabolic costs, needing to be “torn down and rebuilt”) remains invented, now laundered through real papers that don’t actually claim what’s attributed to them. This is a fairly common pattern with AI research tools: ask it to find support for a thesis, and it will surface real, adjacent literature and then summarize it in a way that overstates the connection to your original claim. The citations are real; the synthesis is not.

    • voodooattack@lemmy.world
      link
      fedilink
      arrow-up
      1
      arrow-down
      1
      ·
      1 day ago

      Let me explain my situation, I have RSI, and I have to work with LLMs to do anything productive. I know what the sycophancy is like and I work as an AI Engineer at an AI Startup to begin with so I know how to prompt them.

      Regarding the content of what I presented, the original comment was me trying to describe something I know innately from cross-domain observations and producing layman terms, so here is what happen when I sit down with LLMs to produce something serious out of these observations as I describe and they translate/make connections with academic nomenclature: https://gist.github.com/voodooattack/2731bfb21d0873a8f77c84a918335712

      (Was sadly interrupted by tight session limits because of financial circumstances that have no bearing on this conversation and/or content)