Development details of Tesla’s FSD V12 system have been revealed.
Although Elon Musk has predicted that FSD V12 will change the technical route, what is surprising is that Tesla only started training this neural network-based intelligent driving algorithm at the beginning of this year. Just four months later, the new system was ready to replace the old system; eight months later, the new FSD V12 was unveiled during Elon Musk’s live broadcast.
Behind this is a changing technical route, from rule-driven to data-driven; from modular design to end-to-end. It also brings new challenges.
FSD V12: only neural networks
In general, Tesla FSD V12 has only one core feature: no rule code, only neural networks.
What’s the meaning?
Most of the common autonomous driving systems on the market adopt a modular design, including three modules: perception, decision-making, and control. Each task uses its own algorithm model. Among them, the AI algorithm is mainly used in the perception module, and the decision-making and control modules are still conventional codes based on if-else logic.
That is, the code written by the algorithm engineer will establish a set of rules for the autonomous driving system, such as stopping when the light is red, passing when the light is green, driving in the middle of the lane, etc.
Therefore, the shortcomings of this system are obvious. The rules-setting standards are determined by engineers of various companies. The driving style can easily not match the driver’s habits, resulting in a poor experience. It is better to drive it yourself.
The Tesla FSD V12 only has neural networks, which means that in the past, the major modules of perception, decision-making, and control were not needed in the design. Just determine the neural network architecture and then input the data for training.
A neural network processes all input signals and outputs driving decisions. Based on real human driving data, the system can learn how to drive and drive better.
This is the so-called transition from rule-driven to data-driven.
From the input of various environmental information, the system determines how to drive in this situation based on rules; to the training of human driving data, after the system fully learns human driving habits, it will drive itself in the actual driving environment based on the input environmental information. Decide how to open it.
If there is a situation that is not handled well, enter more data specifically for this scenario. It is a training method similar to ChatGPT, but a version more suitable for cars.
Before deciding to change the technical route, Tesla’s autonomous driving team showed Elon Musk that a system based on neural networks could handle better situations.
On a road littered with trash cans, downed traffic cones, and random obstacles, the car was able to accurately maneuver around them, cross lane lines, and break some traffic rules when necessary.
Before the live broadcast, Elon Musk also conducted a test on FSD developed based on neural networks. During a total of 25 minutes of driving, Elon Musk only stepped on the accelerator when the system was too cautious, but never touched the steering wheel. There was also a time when the system performed better than he expected.
My human neural network fails here.How to treat
In fact, long before Elon Musk announced that FSD V12 would become an end-to-end technical route, this concept had already emerged among autonomous driving players.
Because the end-to-end autonomous driving system is easy to develop, there is no need to write massive amounts of code in the early stage (there are more than 300,000 lines of C++ code in the FSD V11 version control stack), and there is no need for engineers to design rules in advance. Just keep inputting human driving data, and the system will learn by itself.
But this also places high demands on autonomous driving players. For example, the input must be a large amount of high-quality data to better help the system learn.
Elon Musk found that neural network-based self-driving systems only started to perform well when fed more than 1 million videos.
At the beginning of this year, Tesla had already input 10 million human driving videos into this system, and they were all screened by experienced drivers.
Tesla’s fleet of nearly 2 million vehicles around the world also provides approximately 160 billion frames of video for training every day. Tesla expects that the number of videos used for training will reach billions of frames in the future.
This is a challenge for data volume, data annotation, computing power, etc.
Moreover, the reason why end-to-end technology has not become popular among autonomous driving players on a large scale is that there is a key problem: this will increase the unexplainability of the autonomous driving system.
At this stage, end-to-end autonomous driving is still a “black box”, and there is no way to accurately explain why the system does not handle well in certain situations.
Therefore, the solution given by Tesla is to feed more data in a targeted manner if the system is not processed well during testing.
For example, during Elon Musk’s live broadcast, the system almost ran a red light. The solution he gave was to input more traffic lights, especially videos of left-turn signals.
In addition, Elon Musk also set an indicator for the team to display the number of miles traveled by the FSD system without human intervention in real time. If intervention occurs, the corresponding problem will be dealt with.
More importantly, if learning continues in this way, a new problem will arise: the system will not only learn the smooth operation of experienced drivers but also learn the behaviors of human drivers that do not comply with traffic rules.
For example, when encountering a stop sign, more than 95% of people will pass slowly rather than stop completely. This means regulators need to clearly regulate standards.
The U.S. National Highway Safety Commission is studying whether to allow autonomous driving systems to operate without fully complying with traffic regulations.
In short, the launch of the Tesla FSD V12 is indeed of great significance for autonomous driving. Now that the entire process can be AI-enabled, it is more possible to move towards AGI, that is, artificial general intelligence.
When will autonomous driving have its ChatGPT moment? The gears of fate may start turning from this moment on.
