It’s The Math – How China Is Avoiding The AI Bubble
moon of alabama
The Artificial Intelligence bubble in the United States is still growing. It is consuming a large share of the available financial resources. But its output, in form of real products, is so far rather meager.
There are some code generation tools which are, at times, helpful, but, unless subsidized, very expensive. There is ChatGTP and other squawk boxes which in the end are just new forms of inherently unreliable web search tools.
What is missing are useful mass applications billions of people are willing to pay for.
Still – a large number of ‘very important people’ believe that the Large Language Models, which are at the core of OpenAI’s and Anthropic products, will one day reach the capabilities of sentient beings. That is, in my view, utter bullshit, but who am I to tell you.
The NY Times has a long write up (archived) about Oracle founder Larry Ellison and his bet of nearly everything he owns on the A.I. bubble. It states:
The story of A.I. has been as much a financial story as a technological one, a question of how to structure the mind-boggling investments required to train and run the models. Few people doubt that this technology is going to change everything. What’s less clear is when the profits are going to start rolling in and how big they are going to be. “To me, it’s a math problem,” says Asad Ramzanali, the director of A.I. at a policy center at Vanderbilt University. “We are making trillions of dollars in investments on the back of tens of billions of dollars in revenues.”
That is why Ed Zitron and others will rightly tell you that the huge investments spent on AI are making no sense at all.
Where are the products and the customers that will allow to recuperate the hundreds of billions of shady dollars spent on data centers for AI?
The financial structure of the data center build-out makes it especially vulnerable to a crash. The deals themselves are built on enormously complicated debt and equity schemes that involve circular financing. The hyperscalers are investing heavily in the same companies they are counting on to buy their computing power. It’s what economists call an interlocking liability structure. If their customers struggle to monetize their products, they will be hit extra hard — and so will their investors, which include a lot of everyday Americans. And these are just the U.S. companies. The A.I. boom has been a global phenomenon; an A.I. collapse would be as well.
The NY Times presume that everything the U.S. does is copied throughout the world. If the bubble burst in the U.S. it will, in consequence, also burst in other places – especially in China.
This is a misunderstanding of what the Chinese AI models are, and what Chinese AI companies are doing.
Throughout the last year several Chinese companies have surprised the public by offering AI models that are nearly as capable as the best ones U.S. companies produce, but are offered at a price that just a tenth of the cost of the (heavily subsidized) U.S. models.
Moreover Chinese companies have published the weights and source code of their models and allow anyone with the adequate hardware to run them on their own premises.
The Economist has made an attempt to understand this:
How China gets better bang for its buck than America in AI (archived)
Its investment lags far behind America’s. Its models do not
The American AI splurge looks especially profligate compared with Chinese parsimony. In 2026 Chinese tech titans are forecast to invest less than a tenth as much in data centres as their American counterparts (see chart).
Their models appear only fractionally less powerful owing to this frugality. K3, an advanced model launched last month by Moonshot AI, a Beijing-based startup, is 95% as clever (on widely used benchmarks) as Fable 5, a frontier model from Anthropic, another top American lab. It is also 70% cheaper to use. On August 3rd Alibaba, a Chinese tech giant, released a model which reportedly scored among the world’s best by some measures.
The authors at the Economist go on to muse about cheaper data center costs in China. The country, they say, has also less chances to invest in overpriced U.S. hardware. It’s buyers are more stingy and the search for real applications is more important for China than the quest for a ‘singularity’ or some ‘general artificial intelligence’, which U.S. companies pursue.
This is all, like in the NY Times piece quote, mostly bullshit.
The real discriminator between U.S. and Chinese AI companies is the math they use.
U.S. models run on algorithms that are computationally wasteful, while Chinese models use smarter methods, which need less compute, to achieve a similar quality.
Language models, which, given some text input, produce a related text output, are nothing new.
Most are based on so called neural network algorithms which are a simulation of a rudimentary function of the human brain. Neural networks can be trained to recognize an input to then produce a related output.
The problem with neural networks and their early variants is that the training is very time consuming which limits the size of the models.
In 2018 researchers at Google found an elegant way to circumvent these restriction:
The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train.
U.S. companies jumped onto the Transformer model. It was seen as a chance to create ever bigger and better models with the hope that a huge sized model would eventually reach or exceed the capabilities of a human brain.
The advantage of the Transformer model is that the math necessary to train it can be parallelized. Computer chips built to produce graphics have special hardware to process pixels. Instead of one pixel at a time they compute complete pictures by computing the (color) value of thousands of pixels at the same time. The use of Graphic Processing Units (GPUs) allows for parallelized (matrix) operations on thousands of parameters. (A video by Grant Sanderson, Large Language Models explained briefly (vid), gives a good introduction to this.)
The disadvantage of the Transformer model is the exponential behavior of its attention mechanism. To digest one sentence of input the model has to process each element therein against all other elements held in the same context. That is an exponential operation. For a context of 500,000 input words (i.e to summarize a book of that length) the model will require 250 billion operations. OpenAI, Anthropic and others use a lot of tricks to optimize their computing but they can not escape the basic exponential constrain of the algorithm they use.
This is the reason why the input of current Transformer models is limited and why these models need huge data centers with millions of GPUs to respond to users questions.
For lack of compute capacity the Chinese model developers had to take different routes. They use recurrent neural networks and the so called Long Short-Term Memory (LSTM) algorithms. These have been known since the late 1990s and are the basis for most machine language translations. Apple’s SIRI is using of a LSTM model.
The LSTM algorithm has disadvantages. In its original form its training can not be parallelized. But it is able to ‘forget’ unnecessary context and its compute of an answer is (mostly) linear, not exponential.
The more recent extended LSTM algorithms allow for parallelizing the training of such models while keeping the advantage of a (nearly) linear response algorithm.
The recently revealed Kimi model of the Chinese company Moonshot.ai is based on a (modified) xLSTM algorithm, called Kimi Delta Attention. It also splits the model into a network of 800 ‘experts’ only a few of which are activated to answer a users specific question (see Kimi K3 Explained in 13 minutes (vid)).
While Kimi has yet to reach the parameter size of the newest U.S. model it has nearly the same capabilities while using much less compute than is needed to run an equivalent OpenAI model.
Other Chinese AI providers are also using similar algorithms which need much less compute than the U.S. preferred Transformer models.
The U.S. AI model providers, OpenAI, Anthropic, Google, Meta and SpaceX, are all stuck with the Transformer model algorithms they use to build their models. It is what they have been building on for years and it is what their engineers are experts in. They have promised to build or buy a huge amount of computing capacity because that is what their models need.
It is this choice of an algorithm that has created the U.S. AI bubble.
The Chinese modelers, for lack of unlimited venture capital, have had to find better ways to solve the problems. Their use of different algorithms allows them to deliver nearly similar results for much less money.
There is also a difference in attitude. Where U.S. financiers like Larry Ellison are striving for some god-like general artificial intelligence, the Chinese (and some European) developers and financiers are much more interested in solving real world (industry) problems.
A robot to fill the dish washer can operate sufficiently without being able to solve mathematical conjectures. Moreover its internal AI model will have to run on a local GPU and not inside some far away data center.
An abundance of resources has allowed U.S. model developers to build huge, very capable, but enormously expensive general AI models. Those companies have yet to find use cases that will pay for the cost of running these models.
Instead of reaching for the stars, like U.S. companies do, the Chinese are out to solve down-to-earth problems.
Restricted resources have forced the Chinese developers to find better algorithms. Their primary orientation is on useful applications that will generate appropriate returns for their costs.
At some point the AI bubble in the U.S. will blow up. There will be a lot of collateral damage. Many people will lose their money. There will be a number of useless empty data centers up for rent.
The NY Times and the Economist presume that this will cause China to have similar problems.
That is unlikely to be the case as Chinese AI is based on a different math, and different economic models.
posted by Satish Sharma at
06:35
0 Comments:
Post a Comment
Subscribe to Post Comments [Atom]
<< Home