News Hub

Article discusses recursive self-improvement in AI systems and engineering approaches for self-improvement.

Reward hacking occurs when reinforcement learning agents exploit flaws in reward functions to achieve high scores.

Article defines extrinsic hallucinations in LLMs as fabricated outputs not grounded in context or world knowledge.

Diffusion models are being applied to video generation with temporal consistency requirements.

Article discusses high-quality human data as essential fuel for deep learning model training and RLHF labeling.

Adversarial attacks and jailbreak prompts can trigger undesired outputs from aligned LLMs.

LLM-powered autonomous agents use language models as core controllers for problem solving.

Prompt engineering uses in-context prompting to steer LLM behavior without updating weights.

Updated comprehensive review of Transformer architecture improvements proposed since 2020, roughly doubling the length of the original post.

Article discusses optimization techniques for reducing inference time and memory costs of large Transformer models.

Article covers mathematical theory of Neural Tangent Kernel explaining why over-parameterized networks generalize well.

Article surveys visual language models that process images to generate text for tasks like captioning.

Article covers data generation approaches including augmentation and transformation for training with limited data.

Article discusses active learning strategies for selecting which samples to label under budget constraints.

Four common approaches address supervised learning with limited labeled data.

Diffusion models generate images through iterative denoising from random noise.

Contrastive learning creates embeddings where similar samples cluster and dissimilar samples separate.

Methods exist to reduce toxic behavior and biases in pretrained language models for safe deployment.

Techniques enable fine-grained control over neural text generation including prompt tuning and unlikelihood training.

Open-domain question answering systems retrieve and process factual knowledge to answer arbitrary questions.