Not happy about losing out to new entrants, AWS is considering other options. “We’re thinking about different solutions in the short and long term to give our customers the experience they’re looking for,” says Kapoor, who declined to give details.
Queue at processing units for AI
The inadequacies of cloud service providers are affecting their customers, who include some big names in the technology industry. According to Jeremy King, chief technology officer, social network Pinterest is increasingly using AI to better serve users and advertisers. The company is considering using Amazon’s new chips. “We need more GPUs, like everyone else,” King admits. “The chip shortage is a real thing.”
OpenAI, which develops ChatGPT and licenses the underlying technology to other companies, relies heavily on Azure chips to deliver its services. The GPU shortage has forced the company to place usage limits on the tools it sells. That was unfortunate for customers like the company behind the Jamie AI assistant, which aggregates meeting audio using OpenAI technology. Plans for a public release have been delayed by at least five months, partly because they wanted to refine their system, but also because of usage restrictions, says Louis Morgner, co-founder of startup. The problem hasn’t gone away. “We’re only a few weeks away from (Jamie’s) release to the public and then we need to keep a close eye on the scalability of our system given the limitations of our service providers,” says Morgner.
“The industry is seeing strong demand for GPUs,” said Niko Felix, spokesman for OpenAI. “We continue to work to ensure our API customers are able to serve their needs.”
At this point, any compound that makes a is startup Access to processing capacity is crucial. Investors, friends, neighbors: Startup managers use a wide variety of relationships to gain more power for their AIs. For example, Astria gained additional capacity on AWS, a close partner of the Amazon platform on whose technology Astria relies, with the help of Emad Mostaque, CEO of Stability AI.
The startup In accounting, Pilot, which uses OpenAI technology to classify some routine data, gained early access to GPT-4 after enlisting the help of friends from college, employees and VCs with ties to the company. It’s not clear if these links resulted in Pilot being removed from the waitlist, but he’s now spending around $1,000 a month on OpenAI, and these connections will come in handy when he needs to increase his quota , says Waseem Therefore, its CEO. “If you don’t use this generative AI technology, someone else will, and it’s powerful enough that you don’t want to risk it,” says Folger. “You want to offer your customers the best results and know what’s happening in the industry.”
Organizations are not only struggling to access more capacity, they are also trying to do less with more. Companies experimenting with generative AI are now obsessed with “optimization”: enabling them to render with satisfactory results on the cheapest GPUs. It’s like saving money by trading in an old energy-guzzling fridge that only holds a few drinks for a modern mini-fridge that runs on solar energy most of the time.
Companies are attempting to write better instructions for how chips should process programming instructions by attempting to reformat and limit the amount of data used to train AI systems, and then reducing the inference code to the minimum required to perform the task at hand. That means creating several smaller systems: maybe one imager that produces animals and another that produces humans, and switching from one to the other based on user preference.
