Avui he descobert que el 12% de les ressenyes del nostre…
Núria Puig@nuria_data
Avui he descobert que el 12% de les ressenyes del nostre dataset d'entrenament les ha escrit un LLM. Totes comencen igual: "Com a client satisfet...". Bueno. Ara entrenem un model amb text d'un altre model. Què pot sortir malament? 🙃
Get the best of #stochastic-parrots weekly
One email each Monday with the most useful posts. No spam; unsubscribe in one click.
Comments
Sign in to join the conversation.
Sign in to commentFrancesca Esposito@francesca_ml
Model collapse in practice ahah. How did you detect them? Only the opening phrase, or something more systematic?
Núria Puig@nuria_data
first the phrase, then near-duplicate clustering (the same MinHash trick from my leakage post). they form very tight clusters, humans are more chaotic. not perfect but good enough to flag them
Javier Montero@javi_montero
No entiendo catalán pero entiendo 'Com a client satisfet' y me da escalofríos. En el banco tenemos lo mismo con las encuestas de satisfacción, todas suenan a folleto.
Dana Kowalski@dana_ml
Read it with a translator. Worth a full post when you have numbers: how much did removing them change your model's outputs? I suspect the effect on style is larger than on accuracy.
Núria Puig@nuria_data
challenge accepted. I think the same, the model started writing 'satisfied' everywhere ahah