This is a departure from my usual tech blog. In the current series titled Artificial-Intelligence-based-chip-design, I/we present the use of Artificial Intelligence to design FPGA chips for Ultra-Low-Latency based speech transformers. This showcases our products as well as our skills in the respective domains. This started as an experiment to lower latency for a seq-to-seq LSTM based speech synthesis system. Post a thorough analysis, we broke it down into 1. mapping the matrix multiplication of deep learning transformers into an FPGA chip(Transformers On Chip(part-1)) and 2. co-designing the FPGA chip for the appropriate workload(ChipDesign(part-2)). In part-2 we treat the resources on FPGA chip, Workload, other design parameters as design variables. Finally, we feed the design variables into the Deep Reinforcement Learning agent to learn. Post learning the expectation from Deep Reinforcement Learning agent being the optimal placement of blocks to maximize certain goals like latency, etc. The Deep Reinforcement Learning Algorithm is supposed to figure out a balance that speeds up computation for tolerable accuracy losses(if at all).To use a boxing term, pound for pound, the same hardware micro-designed by a Deep Reinforcement Learning Agent for carrying a specific load.

This is departure from my usual tech blog. In the current series titled Artificial-Intelligence-based-chip-design, I/we present the use of Artificial Intelligence to design FPGA chip for Ultra-Low-Latency based speech transformers. This showcases our products as well as our skills in the respective domains. This started as an experiment to lower latency for a seq-to-seq LSTM based speech synthesis system. Post a thorough analysis, we broke it down into 1. mapping the matrix multiplication of deep learning transformers into an FPGA chip(Transformers On Chip(part-1)) and 2. co-designing the FPGA chip for the appropriate workload(ChipDesign(part-2)). In part-1 we look specifically at resources available on an FPGA chip and a few different strategies of hardware mapping of the workload(matrix multiplication) on the above chip.

The current article is part of a bigger series titled Meditating-with-microprocessors,in which I demonstrate the use of Artificial Intelligence to tune microprocessors for Ultra Low Latency and Realtime loads. There are 5 parts to the series Artificial Intelligence based Hardware(Microprocessor) tuning: Implementing a very simple idea(part-1) , A crashcourse in Microarchitecture and Linux CPUIDLE interface(part-2), Trading off power for UltraLowLatency(part-3), Artificial Intelligence guided Predictive MicroProcessor tuning(part-4), Appendix:Tools of the trade(part-5) . In the current article, we drill down into the inner workings of Ftrace and EBPF. With EBPF based techniques and frameworks Linux tooling has gained tracing superpowers. It has capabilities that now exceed that of Dtrace on Solaris. This is not an exhaustive section and it does not do a breadth-first scan of Linux tooling. I am highlighting a couple of tools, why they work for us, and what makes them special. Moreover, tools play a vital role in verifying or rebutting the theories we make about the systems we design. Building/designing a system is like building a beehive, little blocks you put together making it a whole. But these little blocks fit more snuggly if the theory is rock-solid. Tools provide us the evidence for that.

The current article is part of a bigger series titled Meditating-with-microprocessors,in which I demonstrate the use of Artificial Intelligence to tune microprocessors for Ultra Low Latency and Realtime loads. There are 5 parts to the series Artificial Intelligence based Hardware(Microprocessor) tuning: Implementing a very simple idea(part-1) , A crashcourse in Microarchitecture and Linux CPUIDLE interface(part-2), Trading off power for UltraLowLatency(part-3), Artificial Intelligence guided Predictive MicroProcessor tuning(part-4), Appendix:Tools of the trade (part-5) . In the current article, we explore the use of Artificial Intelligence to reduce latency and save power at the same time. As we have understood, left to itself, the microprocessor will sense the load and configure itself for it(suitable c-states, P-states, etc. ). However, this process is reactive at best, and if latency matters then it is already late. By reactive, I mean the arrival of a remote message triggers an interrupt and from there, one thing leads to another before the microprocessor configures itself for the load. Causal is another word for it. With an Artificial Intelligence based model can we reduce latency by predicting load and preconfigure the microprocessor for load and then postconfigure to save power once the load has tided over. In other words making the process more proactive and predictive.

The current article is part of a bigger series titled Meditating-with-microprocessors,in which I demonstrate the use of Artificial Intelligence to tune microprocessors for Ultra Low Latency and Realtime loads. There are 5 parts to the series Artificial Intelligence based Hardware(Microprocessor) tuning: Implementing a very simple idea(part-1) , A crashcourse in Microarchitecture and Linux CPUIDLE interface(part-2), Trading off power for UltraLowLatency(part-3), Artificial Intelligence guided Predictive MicroProcessor tuning(part-4), Appendix:Tools of the trade(part-5) . In the current article, I present irrefutable evidence that if the processor is configured correctly with a goal in mind(for e.g. UltraLowLatency), the OS jitter caused by the processor can nearly be eliminated. The system can be much more predictable. Furthermore, It has a huge impact on the latency of the system for good. A substantial improvement in latency due to configuration on the Core, but beyond substantial due to Uncore. The improvement due to Uncore is to be expected because there is a whole lot more circuitry on the Uncore. However, this is a trade off that comes at the cost of expending more power.

The current article is part of a bigger series titled Meditating-with-microprocessors,in which I demonstrate the use of Artificial Intelligence to tune microprocessors for Ultra Low Latency and Realtime loads. There are 5 parts to the series Artificial Intelligence based Hardware(Microprocessor) tuning: Implementing a very simple idea(part-1) , A crashcourse in Microarchitecture and Linux CPUIDLE interface(part-2), Trading off power for UltraLowLatency(part-3), Artificial Intelligence guided Predictive MicroProcessor tuning(part-4), Appendix:Tools of the trade(part-5) . In the current article I lay the technical groundwork for later articles in the series to build on. The term's and technologies I'll be using later must be understood really well. So we look at 3 concepts primarily. Firstly, Architecture of a modern microprocessor in short ,really really short. Just enough to make a mental model of the microprocessor to work with. Secondly, How does does software(Linux) interface with the microprocessor. Again just enough to make sense of the data we gather form the microprocessor using various UltraLowLatency profilers and tracers. Thirdly, help a programmers like you to get started.

In the current series titled Meditating-with-microprocessors, I demonstrate the use of Artificial Intelligence to tune microprocessors for Ultra-Low-Latency and Realtime loads. The techniques, in general, can be extended to other components of a computer system like storage devices, memory, etc. However, the article series and my work is currently restricted to Intel microprocessors only. In future, we may extend this to other hardware components of a computer system. This is a very specialized and intense field and hence I intend to break it down using the first-principles approach into simpler pieces of technology that are easy to understand. There are 5 parts to the series Artificial Intelligence based Hardware(Microprocessor) tuning: Implementing a very simple idea(part-1), A crashcourse in Microarchitecture and Linux CPUIDLE interface(part-2), Trading off power for UltraLowLatency (part-3), Artificial Intelligence guided Predictive MicroProcessor tuning (part-4), Appendix:Tools of the trade(part-5) . In the balance then, this is a documentation of my journey navigating these utterly specialized fields ( microarchitecture and Artificial Intelligence ), how to marry them together, the issues I faced, the respective solutions, what (how much) are the benefits if any, and what to have in the toolbox.