next up: “Great thanks we’re gonna sell all your photos unless you pay for a subscription. Gotta keep in business somehow!”

This is really great journalism. Someone needs to edit his Wikipedia page and point to this as evidence. Then we can start citing the Wikipedia page to make up discover more of these stories

oh. go get a therapist–not physical; mental. they’re insanely expensive, but you can spend the next three months shopping around and by the new year you’ll have found someone you like!

invite me and I’ll bring my own alcohol. spread looks delicious!

Another great example (from DeepMind) is AlphaFold. Because there’s relatively little amounts of data on protein structures (only 175k in the PDB), you can’t really build a model that requires millions or billions of structures. Coupled with the fact that getting the structure of a new protein in the lab is really hard, and that most proteins are highly synonymous (you share about 60% of your genes with a banana).

So the researchers generated a bunch of “plausible yet never seen in nature” protein structures (that their model thought were high quality) and used them for training.

Granted, even though AlphaFold has made incredible progress, it still hasn’t been able to show any biological breakthroughs (e.g. 80% accuracy is much better than the 60% accuracy we were at 10 years ago, but still not nearly where we really need to be).

Image models, on the other hand, are quite sophisticated, and many of them can “beat” humans or look “more natural” than an actual photograph. Trying to eek the final 0.01% out of a 99.9% accurate model is when the model collapse happens–the model starts to learn from the “nearly accurate to the human eye but containing unseen flaws” images.

“If you sing at the table you’ll cry before you go to bed.” I thought it was super common until I said it to my kid and my partner thought I was crazy.

“We will continue to charge you penalties until you can afford to pay us back”

Well actually you can’t die either, that’s illegal. Let me know if you find any other options and I’ll make sure they get illegal’d as well!

oooh. design intricate sandwiches! sounds like a lovely holiday!!

Yeah, the problem is how to sanitise effectively. You’ve gotta be able to find a way to automatically strip out “bad” things from your training data (via an “oracle”). But if you already had that oracle, you could just slap it on your final product (e.g. Search) and make all the “bad” things disappear before they hit the user (via some sort of filter).

I was with you until the “construction site and under the bridge” bit. It definitely takes a bit of imagination, but I’m not sure not wanting your kids to play on a site which requires the use of hard hats classifies as being “anxious”

Just commenting to also get a name in that history book.

“Oh yeah. We knew it was coming. We were just waiting to see which one would finally cause it.”

it’s only real programming if you also use CSS