• 1 Post
  • 121 Comments
Joined 3 years ago
cake
Cake day: July 19th, 2023

help-circle

  • https://www.patreon.com/MaxRead/posts/ai-x-risk-and-ai-173181621

    worth the full read, but here’s the parts I liked

    But “x-risk” is not the only kind of “existential” risk about which AI enthusiasts and their converts tend to worry. This recent Ezra Klein column, written in the wake of OpenAI’s release of hundreds of mathematical proofs, is a nice articulation of something I’ve come to think of as “n-risk,” for “ennui risk” (with the mutual understanding that it’s not pronunciation-accurate so don’t bother making a nitpicky little comment about it): fear of the loss of comprehension, control, and meaning in the face of impossibly sophisticated godlike machines poses.

    We see the threat of “n-risk” in some of the emotional reaction of mathematicians faced with the results of OpenAI models bulldozing through open problems: a sense of having been surpassed and left behind, having the meaning from which one derives one’s worth totally emptied. In some sense, on a conceptual level and as a discursive object, x-risk is preferable to n-risk: it’s striking, dramatic, and offers us humans an ongoing chance at great feats of heroism and genius. N-risk has no such narrative reassurances. Maybe it’s better to burn out (as your body is turned into a paperclip) than to fade away.

    Here you might ask: Is the cause of melancholy and despair on the part of some mathematicians simply the knowledge that a computer system has surpassed their ability to understand or produce proofs? Or is it the realization that said computer system is privately owned and confidentially developed in service of accumulation rather than the pursuit of shared human knowledge? Or is it the contempt with which the people who work for the computer system companies and the hanger-on fans of these computer systems seem to regard the mathematicians? Or is it the sudden recognition that the pursuit of mathematical truths has already been so totally devalued by an anti-intellectual social system that you can no longer feel secure that the tradition to which you have devoted your life will continue as a space of intellectual inquiry, even given the development of these new tools? And so on.

    I mean, realistically, it’s all of the above.





  • Hmm, I guess they could have written a lot of code lines, but its not like the code they write would be equivalent to million LOC of a coherent whole.

    I’d have guessed it was better for a training set to have lots of different mostly-known-good solutions to common problems and good visibility of the solution space, rather than one linux-sized program, even with the same bug rate between the two. I’d imagine leetcode is monetizing their user generated code this way too. IDK.

    I do actually have a decent GPU i bought 4 years ago in my PC, so maybe I’ll do a few simple experiments again… my preferred approach is to pick a simple but nonlinear problem e.g. binary multiplication of two large numbers, where you e.g. skew the training sample distribution or omit some number pairs and see if the resulting model can learn correct multiplication anyway.

    For that particular experiment, I’d also point you towards some of the 90s literature on “genetic programming”, because you could reuse most of your scaffolding for it. No GPU req’d, ought to learn it fairly quickly.


  • Mercor has ~$1.5B revenue so napkin math would put at 10-20 million contractor hours per year. I’m not very fast but I can type out about a hundred lines an hour fairly regularly, especially boring code like unit tests, data cleaning or OO design patterns. LoC is a bad metric but still. For comparison, Linux and Chrome are ~40M LoC each, AAA games are 3-10M LoC.

    And to your point, they can do data augmentation by rotating nodes in the ASTs, changing variable names, and other tricks to make it even fatter.

    For comparison, the Software Heritage Archive of all open source is ~2 Petabytes / ~30B files. But not sure how much of it is AI slop at this point, or for dead languages / platforms.


  • So, I was thinking about how modern LLMs got better at coding in the last 1 year, despite not having much good new training data.

    Half the job posts I see on linkedin are from snorkel, handshake and mercor etc to train llms for $50-90/hr. So I think there’s been quite a bit of new training data created as those companies scaled up.

    So everyone is paying openai to pay mercor to pay contractors to write code and then regurgitate it, rather than just paying for what they want directly.

    idk. I’m just skeptical that there was actually no human in the loop and that the gains were from unit tests / synthetic TDD.












  • https://www.publishersweekly.com/pw/by-topic/industry-news/publisher-news/article/101215-publishing-s-ai-reckoning.html h/t naked capitalism

    Smaller publishers, too, must cope with the challenge of identifying AI-generated text in books authors have submitted, and it isn’t always easy. At Microcosm, an independent publisher and distributor in Portland, Ore., co-owner and vice president Elly Blue says her team discovered an author’s undisclosed AI use almost by accident, when a freelance editor flagged what looked like plagiarism, only to find out it was AI-generated text. That discovery prompted Microcosm to run AI detection across its entire pipeline, which turned up several more cases, including two manuscripts fully generated from prompts, and one where an author had trained an LLM on his own past writing. “The book sounded like him, but it made no sense,” Blue recalls.

    Confronting the authors had mixed results. “Some denied it outright and insisted, ‘That’s just what my voice sounds like,’ ” Blue says. “One, surprisingly, owned it and said he’d take the book elsewhere.”

    These episodes led directly to Microcosm instituting a new contract clause that states that if an author’s undisclosed AI-generated text can’t be revised into something genuinely their own, Microcosm retains the rights and can publish the work itself, since the author didn’t actually write it.

    Like if people are using LLMs to submit to the local zine distro, what are we even doing; but the detector is not a good thing either. It isn’t homework and they shouldn’t need turnitin. :(