I think the combination, especially in AI systems where accuracy doesn't need to be perfect and can be modeled in analog, can perfectly comeback.
i.e. If some circuitry is interfacing with an analog counterpart where 16-bit precision is enough at the digital side and the analog circuit can produce outputs matching that accuracy much more efficiently than digital while still being able to be adjusted externally by the digital, why not?
I think it can open up some new doors in AI, though what excites me is that what else other than AI could it be applied to? Circuit simulations? Signal generation? Perhaps some analog "programming languages"?
Analog has a limited dynamic range, in addition to its well-known problems with noise, sensitivity to random variation within devices or the in surrounding environment, etc. Most AI systems are using floating point compute because of its much greater dynamic range compared to fixed point, even though the latter is way more efficient.
For ML inference, the dynamic range and rounding noise of the 16-bit or 8-bit floating-point numbers preferred now for this application is well within the analog capabilities.
Analog processing cannot compete with 32-bit floating-point numbers, when their precision and/or dynamic range is required.
The power requirements for analog processing depend a lot on the required signal-to-noise ratio (SNR) and dynamic range, like also for digital processing.
Increasing the SNR for digital processing requires the processing of more bits, which means either the multiplication of digital blocks such as adders, or the increase of the clock frequencies, when the existing digital blocks must process more bits during the same time, in serial manner. Both methods increase the power consumption.
Increasing the SNR for analog processing means the use of more complex circuits, with more devices, and the use of larger devices, e.g. larger transistors, larger capacitors, larger resistors, so that the effect of various noise sources will be relatively smaller (due to averaging). This also increases the power consumption.
The relative power consumptions for analog and digital processing depend a lot on the technologies used, but in general analog processing needs much less power when the requirements for SNR/dynamic range are low, but the power increases rapidly with increased requirements and at some threshold the power needed becomes larger than for implementing the same task by digital processing.
In modern CMOS technologies designed for making digital devices, the analog components have become worse and worse in comparison with old technologies. That has moved the threshold where digital processing becomes preferable lower and lower.
For example, because of this, at every smartphone generation where a new CMOS process is used, an important part of the migration of the previous designs from the old process to the new process is to identify some old analog blocks whose functions could be implemented digitally in the new process, so that those analog blocks could be replaced by digital blocks, because otherwise the new chip could be worse than the old due to worse analog parts, even if the digital parts have been improved.
The parent article makes the point that this trend which was continuous for the last few decades can be reversed now due to the advances in packaging and interconnection technologies, which can make a modern multi-chip device to be similar in price to a monolithic device.
If the restriction of using a single chip is removed, then it is no longer necessary to use the bad analog components provided by the modern CMOS technologies but it is possible to use a separate analog chip, made with an appropriate manufacturing process.
In this case the threshold where digital processing becomes preferable to analog processing can be moved again to higher SNR/dynamic range values, where it was many years ago.
I've seen some very limited discussion about new modes of analog processing. For example, you can generatively produce an FPGA bitstream with a genetic algorithm that does strange things. E.g. totally disconnected logic components evolving such that they influence the operation of each other via RF leakage within the die. There are no doubt other methods. Constructions which model neurons are also a possibility, as are other things.
A lot of the analog in those devices is very high speed and must be put on, say, sapphire substrate or invokes some other complicated method, requiring wafer level integration or another package. The energy budget may be worse for digital at our current tech level but other effects like package size and package/wafer/process cost dominate.
Spintronics, phononics, photonics, ... could also lead to new computing and reduced sizes.
I often feels that when digital components are numerous enough, you fall back into statistical/analog thinking anyway (even if it's just a case of a few thousands of VMs respawning on need). No more discrete perfect information.. but flocks of potentials.
A 16 bit signal already represents an almost 100dB signal to noise ratio, which is extremely difficult to do operations on. I think 8 bits would be more realistic, but it would still suffer from the problems mentioned by the other reply.
i.e. If some circuitry is interfacing with an analog counterpart where 16-bit precision is enough at the digital side and the analog circuit can produce outputs matching that accuracy much more efficiently than digital while still being able to be adjusted externally by the digital, why not?
I think it can open up some new doors in AI, though what excites me is that what else other than AI could it be applied to? Circuit simulations? Signal generation? Perhaps some analog "programming languages"?