Generative Control Inference

By

In the previous post, I gave a peek into generative control, a new idea that can control some system by learning its intrinsic characteristics as a generative world model. It can avoid the over-shoot and over-expoloration problem from PID control and reinforcement learning. Rewrite the formuation a bit, we have
\[[v,w,x] = g(z), \quad z \sim \mathbb{D},\]
in which \(v\) is the collection of control objectives to be minimized, \(w\) contains the output control signals, and \(x\) represents the input signals that are thought to offer information to the control problem.

During training, we are presented with a dataset of \([v,w,x]\) triplets, each representing the control objective \(v\) achieved under the output and input condition \([w,x]\). We have an algorithm that can produce \(g\), which is a generative model that can disambiguate the uncertainty of multiple acceptable \(v\), \(w\) or \(x\)'s, each under the condition of the other outputs. This uncertainty is usually the reason why traditional system identification methods fail for complex controlling.

In our formulation, the control problem can be described as: given inputs \(x=x_0\), output the control signals \(w\) such that the objectives \(v\) are minimized. We use the following iterative algorithm to solve this problem.
\[\begin{align}& \text{Initialize } v_0, w_0 \\& \text{While } t = 0 \rightarrow T \\& \quad \quad z_t = \underset{z}{\text{argmin}} ~ \| g(z) - [v_{t-1} - \epsilon, w_{t-1}, x_0] \| \\& \quad \quad [v_t, w_t, \_] = g(z_t) \\& \text{Output } w_T \text{.}\end{align}\]

This algorithm works by iteratively reconstructing \([v,w,x]\) triplets that forces \(x=x_0\) and \(v\) going downwards by \(\epsilon\). At equilibrium, the control problem is solved because according to the system world model \(g\), objectives \(v\) cannot go down any more and inputs \(x\) cannot approach \(x_0\) any closer.

Since this algorithm is deterministic by fixed initialization \(v_0\) and \(w_0\), it disambiguated the intrinsic system uncertainty, which would present in the data. In turn, this makes it possible to fit a neural network \(h\) such that \(w_T = h(x_0)\), where direct regression from data fails.

We used this modeling on many problems in the home appliances sector, delivering control networks \(h\) on very cheap hardware whose size span from a few KBs to a few MBs. Soon we will share an online SDK, where you can upload data and download a C program that can be compiled anywhere to deploy \(h\).

Comments

Post a Comment

Popular posts from this blog

Serving Llama-2 7B using llama.cpp with NVIDIA CUDA on Ubuntu 22.04

By
This blog post is a step-by-step guide for running Llama-2 7B model using llama.cpp, with NVIDIA CUDA and Ubuntu 22.04.  llama.cpp is an C/C++ library for the inference of Llama/Llama-2 models. It has grown insanely popular along with the booming of large language model applications. Throughout this guide, we assume the user home directory (usually /home/username) is the working directory. Install NVIDIA CUDA To start, let's install NVIDIA CUDA on Ubuntu 22.04. The guide presented here is the same as the CUDA Toolkit download page provided by NVIDIA, but I deviate a little bit by installing CUDA 11.8 instead of the latest version. At the time of writing, PyTorch 2.0 stable is released for CUDA 11.8 and I find it convenient to keep my deployed CUDA version in sync with that. $ wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb $ sudo dpkg -i cuda-keyring_1.1-1_all.deb $ sudo apt update $ sudo apt install cuda-11-8 After
Read more

A Perplexity Benchmark of llama.cpp

By
Without further ado, here are the results (explanations and discussions later): Table 1: Perplexity on wikitext-2 test set.   Model \ Quantization q4_0 q4_1 q5_0 q5_1 q8_0 fp16 llama-7b 6.157 6.0915 5.9846 5.948 5.9063 5.68 llama-13b 5.385 5.3608 5.285 5.2702 5.2547 5.09 llama-30b 4.2707 - - - - 4.1 alpaca-30b 4.4521 - - - - - llama-2-7b 5.9675 6.0398 5.8328 5.8435 5.7897 - llama-2-7b-chat 7.7641 7.7853 7.5055 7.5392 7.5014 - llama-2-13b 5.2172 5.2115 5.1343 5.1289 5.1005 - llama-2-13b-chat 6.62
Read more

A Peek into Generative Control

By
Chinese version:  一窥生成式控制论 - 知乎 (zhihu.com) As a peek into some of my most recent work in applying generative modeling into various industries, I'm proudly presenting an idea that illustrates how powerful generative models can be for industrial controling. My team and I are working to rapidly expand this idea into many different areas, and we still haven't seen its limit yet. A General Formulation of Generative Models \[y = g(z), \quad z \sim \mathbb{D}\] where \(y\) is a sample from data, \(g\) is a generator neural network written as a function, and \(z\) follows a pre-defined distribution \(\mathbb{D}\). It is easy to identify that generative adversarial networks (GANs) naturally result in models in the above fashion. For autoregressive language models (transformers or otherwise), \(z\) is the concatenation of all the random sampling variables used during the decoding process. For stable diffusion, \(z\) is the concatenation of all the noise added in the diffusion process. A
Read more