Meta AI

Meta released Llama 3.2, introducing vision-capable open models for the first time. The release included lightweight 1B and 3B text-only models optimized for edge devices, plus 11B and 90B multimodal models with vision understanding. These were the first open-source models to natively integrate vision and language capabilities, enabling on-device AI applications while maintaining competitive performance.

• First open-source vision-language models
• 1B and 3B models for edge devices
• 11B and 90B multimodal variants
• On-device AI capabilities
• Fully open-source

Meta released Llama 3.1 with 8B, 70B, and 405B parameter models, representing the largest and most capable open-source LLM ever released. The 405B model matched or exceeded GPT-4o and Claude 3.5 Sonnet on many benchmarks while remaining fully open-source. The release included 128K context windows and multilingual support across eight languages, marking a major milestone for open AI.

• 405B largest open-source LLM ever
• Matched GPT-4o on many benchmarks
• 128K token context window
• Multilingual support (8 languages)
• Fully open-source weights

Meta released Llama 3 with 8B and 70B variants, trained on 15 trillion tokens (7x more than Llama 2). The model achieved state-of-the-art results among open models, surpassing GPT-3.5 on many benchmarks. Llama 3 70B even challenged GPT-4 on certain tasks. Meta's focus on data quality, scaling laws, and training stability resulted in exceptional performance.

• 15 trillion tokens (7x Llama 2)
• State-of-the-art open models
• 8B and 70B variants
• Surpasses GPT-3.5 on many tasks
• Focus on data quality and scaling

Meta released Llama 2, an open-source large language model available for free for research and commercial use. Models ranged from 7B to 70B parameters and outperformed most open-source models on benchmarks. This marked a major shift toward open foundation models.

• 7B, 13B, and 70B parameter models
• Free for research and commercial use
• Trained on 2 trillion tokens
• Outperformed other open-source models
• Enabled open-source AI ecosystem

Facebook AI (now Meta AI) introduced RoBERTa, showing that BERT was significantly undertrained. By training longer with bigger batches over more data and removing the next sentence prediction objective, RoBERTa outperformed BERT on all GLUE tasks.

• Trained longer with bigger batches
• 10× more data than BERT
• Removed next sentence prediction
• Dynamic masking patterns
• Outperformed BERT on all GLUE tasks