LLaMA

In February 2023, Meta AI launched LLaMA (Large Language Model Meta AI), a large language model ranging from 7B to 65B parameters.^[30] Subsequently, Meta AI released LLaMA 2 in July 2023^[31] and LLaMA 3 in April 2024.^[32]

MTIA v1

The MTIA v1 is Meta's first-generation AI training and inference accelerator, developed specifically for Meta's recommendation workloads. It was fabricated using TSMC's 7 nm process technology and operates at a frequency of 800 MHz. In terms of processing power, the accelerator provides 102.4 TOPS at INT8 precision and 51.2 TFLOPS at FP16 precision, while maintaining a thermal design power (TDP) of 25 W.^[37][38][39]

The accelerator is structured around a grid of 64 processing elements (PEs), arranged in an 8x8 configuration, and it is furnished with on-chip and off-chip memory resources along with the necessary interconnects. Each PE houses two processor cores (one with a vector extension) and several fixed-function units optimized for tasks such as matrix multiplication, accumulation, data movement, and nonlinear function calculation. The processor cores utilize the RISC-V open instruction set architecture (ISA), with extensive customization to perform the required compute and control tasks.

The accelerator's memory subsystem uses LPDDR5 for off-chip DRAM resources and can be scaled up to 128 GB. Additionally, it possesses 128 MB of on-chip SRAM that is shared amongst all the PEs for faster access to frequently used data and instructions. The design encourages parallelism and data reuse, offering thread and data-level parallelism (TLP and DLP), instruction-level parallelism (ILP), and memory-level parallelism (MLP).

MTIA accelerators are mounted on compact dual M.2 boards, enabling easier integration into a server. The boards connect to the host CPU via PCIe Gen4 x8 links and have a power consumption as low as 35 W. The servers hosting these accelerators utilize the Yosemite V3 server specification from the Open Compute Project. Each server houses 12 accelerators, interconnected through a hierarchy of PCIe switches, allowing workloads to be distributed across multiple accelerators and executed concurrently.

MTIA v2

MTIA v2 is Meta's second-generation AI training and inference accelerator, significantly enhancing performance and efficiency for AI workloads, particularly in recommendation and ranking models. Fabricated with TSMC's 5 nm technology, it operates at 1.35 GHz and provides 708 TOPS at INT8 precision (with sparsity) and 354 TFLOPS at FP16 precision, representing substantial improvements over MTIA v1.^[40]

Key architectural enhancements include an 8x8 grid of processing elements (PEs), increased local PE storage (384 KB per PE), on-chip SRAM (256 MB), and off-chip LPDDR5 memory (128 GB). Memory bandwidth improvements are also significant, with local memory at 1 TB/s per PE, on-chip memory at 2.7 TB/s, and off-chip LPDDR5 at 204.8 GB/s.

MTIA v2 features an improved network on chip (NoC) architecture for low-latency coordination between PEs. The system supports up to 72 accelerators in a rack-based setup, using PCIe Gen5 links for enhanced bandwidth and scalability.

The software stack, fully integrated with PyTorch 2.0, includes the Triton-MTIA compiler backend for high-performance kernel optimization, improving developer productivity. Early results show a 3x performance improvement over MTIA v1, with a 6x increase in model serving throughput and a 1.5x improvement in performance per watt.

Comparison of MTIA chips

User controls

Meta AI offers options for users to customize their interaction with its features. Users are able to mute the AI chatbot on platforms like Facebook, Instagram, and WhatsApp^[41], temporarily halting notifications from the chatbot. Some platforms also offer the ability to hide certain AI elements from their interface. To locate the relevant settings, users can consult the platform's help documentation or settings menu.

Concerns

Since May 2024, the Meta AI chatbot has summarized news from various outlets without linking directly to original articles, including in Canada, where news links are banned on its platforms. This use of news content without compensation has raised ethical and legal concerns, especially as Meta continues to reduce news visibility on its platforms.^[42]