Meta, which has reportedly developed a number of custom computer chips to help it perform artificial intelligence and video game tasks, communicated with the outside world about the products for the first time this week.
The social media giant first revealed the company’s internal chip project earlier this week, and has since discussed their investment in artificial intelligence technology infrastructure at a virtual event Thursday evening.
Meta previously announced a “year of efficiencies” that saw it lay off at least 21,000 employees and cut costs significantly. Since then, investors have been closely watching Meta’s investments in AI and related data center hardware.
Although it is expensive for a company to design and develop its own computer chips, Alexis Bjorlin, Meta’s vice president of infrastructure, said Meta believes the investment can be justified if it improves the company’s performance. The company has also been tweaking its data center design to focus more on energy-efficient technologies, such as liquid cooling and reducing overheating.
One of the computer chips called the Meta Scalable Video Processor (MSVP), processes video and delivers it to users while reducing energy consumption. Bojorin said that “no commercial product” can process and deliver 4 billion videos per day as efficiently as Meta aspires to.
The other processor is the first chip in the company’s Meta Training and Inference Accelerator (MTIA) family, which is designed to help Meta handle a variety of specialized artificial intelligence tasks. The new MTIA chip specializes in “inference,” which is the use of trained AI models to make predictions or take action.
The new AI inference chip helps strengthen Meta’s recommendation algorithm, which determines what content and ads are shown in a user’s feed, Bojorin said. She declined to answer the manufacturer of the chip, but a blog post revealed that the processor is “made on TSMC’s 7-nanometer process,” suggesting that it is being OEM’d by TSMC.
Bojorin also said Meta has “several generations of roadmaps” for its AI chip family, including tasks for training AI models, but she declined to provide more details beyond the new reasoning chip. It was previously reported that Meta canceled one AI inference chip project and started another, but won’t launch a specific product until 2025. Bojorin declined to comment.
Because Meta doesn’t sell cloud computing services like Alphabet and Microsoft do, the company doesn’t have much incentive to publicly discuss its internal data center chip project.
“Looking at what we’ve shared, these are the first two chips we’ve developed, and it’s definitely a great way to get a glimpse of the projects we’re working on internally.” She said, “We don’t advertise these things, and we don’t need to, but you know, the world is interested.”
Aparna Ramani, vice president of engineering at Meta, said the company’s new hardware is designed to work efficiently with its self-developed PyTorch software, which has become one of the most popular tools used by third-party developers to develop artificial intelligence applications.
The new hardware could eventually be used to perform tasks related to the metaverse, such as virtual reality and augmented reality, as well as emerging generative artificial intelligence techniques. Generative AI is a collective term for AI software that can create engaging text, images and video.
Ramani also said Meta has developed a generative AI programming assistant for the company’s programmers to help streamline its software development and operations process. The new assistant is similar to the GitHub Copilot tool that Microsoft launched in 2021 with the help of artificial intelligence startup OpenAI.
In addition, Meta announced that the company’s supercomputer, Research SuperCluster (“RSC”), has entered its second and final phase. The supercomputer contains 16,000 Nvidia A100 GPUs, which Meta uses to train its LLaMA big language model and other technologies.
Ramani said Meta remains committed to its belief in advancing the technology sector by contributing to open source technology and artificial intelligence research. The company revealed that its largest LLaMA big language model, LLaMA 65B, contains 65 billion parameters and is trained using 1.4 trillion tokens. Tokens refer to the data used to train artificial intelligence.
Neither OpenAI nor Google, among others, have publicly disclosed similar metrics for their respective big language models, but this week it was revealed that Google’s PaLM 2 model uses 3.6 trillion tokens and contains 340 billion parameters.
Unlike other technology companies, Meta released its LLaMA big language model to researchers so they could study the technology. But the LLaMA big language model was then leaked to a wider audience, allowing many developers to integrate the technology into their own applications.
Ramani said Meta “is still considering all of our open source collaborations, and of course, I want to reiterate that our philosophy remains one of open science and cross-collaboration.”
