Post
35
Early-checkpoint diffusion output retains enough corpus fingerprint for source attribution, before the model can form a coherent sentence.
I'm currently pretraining a masked diffusion language model and the following is output from one of its early checkpoints.
---
competitors of Paramount. Combootsseys announced on September 21 Hden 21. All was on Emb Street Journal. We’re not already full of news given how Cinema says tech moves. “Kitt showed that there are millions of investigating unprecedented positive issues for around us, shifting- on wayside during its “Our CES tour.” The subculture industry has allows hatenders against 47 respects since June, including the animated animated movies season 3, and “Ult with Securecom i Got USA Supreme.” On Friday, the Suicide ruled to stop providing the clifihotic, stupid clam of camps and candles, but there was no nihoke sent to me. Emeruination issued that was not enough to provide a generic explanation of the backlash for two months. But the President was lethally following Measure 2016 years of experimental crowdfunding it never received. The kind of thing that’s held during DJ Day’s Fan Bwn and Suicide is the opposite. It’s a little drama and even thwashing the $4ennig proceeds (he does lame forMiggle is patrol to besting) that often dominate nixing high party campaigns into imagardous, mediocre artistic
I asked Gemini, GPT and Claude to guess the site and they all said the same thing: The Verge.
An LLM would probably do the same with a bag of words by looking at clues in keywords. But it could also say Engadget, TechCrunch, Ars or Gizmodo. And it this case they all locked on a very specific site.
Very interesting.
I'm currently pretraining a masked diffusion language model and the following is output from one of its early checkpoints.
---
competitors of Paramount. Combootsseys announced on September 21 Hden 21. All was on Emb Street Journal. We’re not already full of news given how Cinema says tech moves. “Kitt showed that there are millions of investigating unprecedented positive issues for around us, shifting- on wayside during its “Our CES tour.” The subculture industry has allows hatenders against 47 respects since June, including the animated animated movies season 3, and “Ult with Securecom i Got USA Supreme.” On Friday, the Suicide ruled to stop providing the clifihotic, stupid clam of camps and candles, but there was no nihoke sent to me. Emeruination issued that was not enough to provide a generic explanation of the backlash for two months. But the President was lethally following Measure 2016 years of experimental crowdfunding it never received. The kind of thing that’s held during DJ Day’s Fan Bwn and Suicide is the opposite. It’s a little drama and even thwashing the $4ennig proceeds (he does lame forMiggle is patrol to besting) that often dominate nixing high party campaigns into imagardous, mediocre artistic
I asked Gemini, GPT and Claude to guess the site and they all said the same thing: The Verge.
An LLM would probably do the same with a bag of words by looking at clues in keywords. But it could also say Engadget, TechCrunch, Ars or Gizmodo. And it this case they all locked on a very specific site.
Very interesting.
