News Hub

Importance sampling with fine-tuned donor prefills predicts when reward hacking emerges during model training.

Deep Ignorance filters pretraining data to build tamper-resistant safeguards in open-weight LLMs.

EleutherAI adds attention mechanisms to linear probes.

EleutherAI released Common Pile v0.1, an 8TB dataset of public domain and openly licensed text.

Researchers use sparse autoencoder latent interpretations to simulate and rewrite LLM activations in natural language.

EleutherAI presents minetester tool for third-party evaluation of risks in LLM training data.

EleutherAI releases interim report on mechanistic anomaly detection research progress.

GPT-NeoX now supports RLHF and RLAIF post-training via SynthLabs collaboration.

EleutherAI releases open-source automated interpretability pipeline for sparse autoencoder features.

EleutherAI developed a method for concept erasure from models without requiring concept labels at inference.

EleutherAI published VINC-S, a closed-form method for knowledge elicitation with paraphrase invariance.

EleutherAI trained T5 on the Pile dataset.

EleutherAI fact-checks New York Times claims about Yi-34B and Llama 2.

EleutherAI announced the Foundation Model Development Cheatsheet resource.

Least-Squares Concept Erasure achieves more surgical model edits than LEACE with concept labels.

Foundation Model Transparency Index methodology has precision issues in evaluating transparency.

EleutherAI released Llemma, 7B and 34B parameter models trained on mathematical documents.

EleutherAI presents basic mathematics for transformer computation and memory usage.

Demonstration of interpretability analysis for RLHF transformers using TransformerLens.

EleutherAI released GPT-NeoX-20B, a 20 billion parameter model trained with CoreWeave.