This is The Age of AI Series, where we talk to the foremost entrepreneurs and innovators around the planet using ML to transform industries. (Join our special mailing list!)
Edge AI involves everything from autonomous cars to drones, and smart doorbells to water filters. But to build AI models into millions of small devices, you need specialized semiconductor chips.
Today’s episode is a real dynamite. We discuss the most bleeding-edge chip technology on the market for deploying deep learning models.
My guest is Albert Liu, the Founder and CEO of KNERON, an American-Taiwanese chip design and hardware platform company.
It costs $100M just to be a player in this space. Albert and I discuss everything from the technology of chips to the global semiconductor supply chain crisis!
Although this episode is quite technical, you’ll still find some valuable things if you’re a non-techie. Make sure you scroll down to read the summary and key takeaways.
Here’s what we discuss:
- 01:46 — Why we need specialized chips and hardware for edge AI
- 04:46 — Kneron’s current market share, and secret sauce
- 09:22 — “Reconfigurable” AI chips: what are they and why are they cool?
- 13:14 — Limitations of chips and how to overcome them
- 16:38 — How to design world-class AI chips: decisions around performance, power, and more
- 22:34 — AI chips proven 1000x better than GPUs??!
- 26:00 — Technical differences between specialized AI chips and GPUs
- 30:59 — Manufacturing bleeding-edge chips: getting to 7 nanometers and below
- 34:06 — Money and supply chain: what it costs to start an AI chip company
- 38:15 — The current global crisis in semiconductor supply chains: what it’s really like
- 41:17 — Why the first 2 years of a chip company incredibly tough
- 45:49 — The NEXT generation of AI chips, coming soon: going beyond convolutions!
Aman’s 2-Minute Summary and Key Takeaways
Why we need special chips for Edge AI
For most devices that use deep learning, such as autonomous vehicles, you don’t have the luxury of processing data on the cloud — you have to process it on the device’s computer chip itself. It’s also better for data privacy. But since AI models can be quite heavy, current chips are too bulky and inefficient to handle them on the edge.
Analogy: Say you want to deliver a letter to your neighbour down the street. Current chips are like using a semi truck for that half-mile journey, while specialized chips are more like a bicycle. The bicycle will beat a semi-truck every single time!
Running a GPU server for a month could get you an electricity bill of $500. By using specialized chips built solely for deep learning, you can get a HUGE performance boost while also greatly reducing size, power consumption AND cost — the bill would only be $1! This is the order of magnitude of change we’re talking about.
Reconfigurable Chips
Kneron claims their patented chips are “reconfigurable” — i.e they can be adapted to a much larger variety of models, from ResNet to YOLO etc. Their current generation supports convolutions, but their newer generations will also allow recurrent networks.
What it costs
Getting a decent chip out takes 2 years of R&D and a team of around 100 people, before you can do anything in the market. It adds up to $70M easily: $10M – $15M each year for salaries alone, and then a couple rounds of manufacturing and testing. To put out 2 generations of chips, you need a minimum of $100M or you can’t even play the game.
The global semiconductor crisis is real!
Albert told me that when he was working at Qualcomm, it would take him 2 months to get a chip back from the foundry, and a max of 3.5 months. In Q1 of 2021, the lead time has become 18-24 months. They can’t test new designs quickly, because fabricating prototypes alone will take a couple years.
This has serious consequences for all companies involved. Not every company can stay alive for 2 years without any product!
Final Thoughts
Kneron doesn’t just design chips, they’re also building an ecosystem of easy-to-use software and hardware tools that developers can use to build their own edge AI apps. So in that sense, Kneron is really a platform company.
Competition is heating up in the AI chip space, but Kneron appears to be one of the forerunners in design and have already captured some pivotal early market share. They’re also well-funded — they raised $68M in Series A alone.
Regardless of that, I’m more excited about the technology itself. Reconfigurable, specialized chips and hardware platforms for deep learning will lead to the creation of intelligent yet extremely compact devices that we can’t even imagine at the moment.
They will truly help leapfrog us into the Age of AI.
(Ethics Policy: These opinions are 100% my own as an independent observer and educator. I don’t own stock in guests’ companies or their competitors, nor do I get paid by them in any form for any reason at the time of publishing, unless specifically stated. Episodes are also not intended to be an automatic endorsement of any company or its products and services.)
