# Differential Equations on Probability Distributions

## Example Problems

### Solve the equation $$-z_i \cdot q_{\phi}(\mathbf{z}) = \frac{\partial}{\partial z_i}\,q_{\phi}(\mathbf{z})$$

Let $$q_{\phi}(\mathbf{z}) = q_{\phi}(z_1|z_2,\dots,z_K) \cdot q_{\phi}(z_2,\dots,z_K)$$, where $$K$$ is the number of dimensions of $$\mathbf{z}$$. We shall first solve the differential equation on $$z_1$$:

\begin{align} & -z_1 \cdot q_{\phi}(\mathbf{z}) = \frac{\partial}{\partial z_1}\,q_{\phi}(\mathbf{z}) \\ \implies & -z_1 \cdot q_{\phi}(z_1|z_2,\dots,z_K)\cdot q_{\phi}(z_2,\dots,z_K) = \frac{\partial}{\partial z_1}\,q_{\phi}(z_1|z_2,\dots,z_K)\cdot q_{\phi}(z_2,\dots,z_K) \\ \implies & -z_1 \cdot q_{\phi}(z_1|z_2,\dots,z_K)\cdot q_{\phi}(z_2,\dots,z_K) = q_{\phi}(z_2,\dots,z_K)\cdot \frac{\partial}{\partial z_1}\,q_{\phi}(z_1|z_2,\dots,z_K) \\ \implies & -z_1 \cdot q_{\phi}(z_1|z_2,\dots,z_K) = \frac{\partial}{\partial z_1}\,q_{\phi}(z_1|z_2,\dots,z_K) \\ \implies & -z_1\,\partial z_1 = \frac{1}{q_{\phi}(z_1|z_2,\dots,z_K)}\,\partial q_{\phi}(z_1|z_2,\dots,z_K) \\ \implies & \int -z_1\,\partial z_1 = \int \frac{1}{q_{\phi}(z_1|z_2,\dots,z_K)}\,\partial q_{\phi}(z_1|z_2,\dots,z_K) \\ \implies & -\frac{1}{2} z_1^2 + C(z_2,\dots,z_K) = \log q_{\phi}(z_1|z_2,\dots,z_K) \\ \implies & \exp\left(-\frac{1}{2} z_1^2\right)\cdot\exp\left(C(z_2,\dots,z_K))\right) = q_{\phi}(z_1|z_2,\dots,z_K) \end{align}

Since $$q_{\phi}(z_1|z_2,\dots,z_K)$$ is a probability distribution, we have:

$\begin{equation*} \exp\left(C(z_2,\dots,z_K)\right) = 1 / \int \exp\left(-\frac{1}{2} z_1^2\right) \,\mathrm{d}z_1 = \frac{1}{\sqrt{2\pi}} \end{equation*}$

thus we have:

$\begin{equation*} q_{\phi}(\mathbf{z}) = \frac{1}{\sqrt{2\pi}} \exp\left(-\frac{1}{2} z_1^2\right)\cdot q_{\phi}(z_2,\dots,z_K) \end{equation*}$

We then solve the differential equation on $$z_2$$. We have:

\begin{align} & -z_2 \cdot q_{\phi}(\mathbf{z}) = \frac{\partial}{\partial z_2}\,q_{\phi}(\mathbf{z}) \\ \implies & -z_2 \cdot \frac{1}{\sqrt{2\pi}} \exp\left(-\frac{1}{2} z_1^2\right)\cdot q_{\phi}(z_2,\dots,z_K) = \frac{\partial}{\partial z_2} \frac{1}{\sqrt{2\pi}} \exp\left(-\frac{1}{2} z_1^2\right)\cdot q_{\phi}(z_2,\dots,z_K) \\ \implies & -z_2 \cdot \frac{1}{\sqrt{2\pi}} \exp\left(-\frac{1}{2} z_1^2\right)\cdot q_{\phi}(z_2,\dots,z_K) = \frac{1}{\sqrt{2\pi}} \exp\left(-\frac{1}{2} z_1^2\right)\cdot\frac{\partial}{\partial z_2} q_{\phi}(z_2,\dots,z_K) \\ \implies & -z_2 \cdot q_{\phi}(z_2,\dots,z_K) = \frac{\partial}{\partial z_2} q_{\phi}(z_2,\dots,z_K) \end{align}

If we let $$\mathbf{z}’ = z_2,\dots,z_K$$, the form of the differential equation on $$z_2$$ is now exactly the same as the differential equation on $$z_1$$. Use the same method, we can solve the equation on $$z_2$$, and further on $$z_3, \dots, z_K$$. Finally ,we can get the solution:

$\begin{equation*} q_{\phi}(\mathbf{z}) = \frac{1}{\left(\sqrt{2\pi}\right)^K}\prod_{i=1}^K \exp\left(-\frac{1}{2} z_i^2\right) \end{equation*}$

which is a $$K$$-dimensional unit Gaussian.