D-DL4CV-Lec14f-Feature_Inversion

Feature Inversion

Strategy

We’ll give a feature vector for an image in any layer, we want to generate an image that

• Gives similar feature vector at the same layer
• “looks natural” (This can be done by regularization)

Mathematical Expression

$${\mathbf{x}}^{\ast }={\text{argmin}}_{\mathbf{x}\in {\mathbb{R}}^{H\times W\times C}}\ell (\Phi (\mathbf{x}),{\Phi }_0)+\lambda R(\mathbf{x})$$

Explanation:

1. $\Phi (\mathbf{x})$ : Feature vector of new image
2. ${\Phi }_0$ : Given feature vector
3. $\ell (\Phi (\mathbf{x}),{\Phi }_0)$ : Compute distance between two feature vector

$$\ell (\Phi (\mathbf{x}),{\Phi }_0)={\left|\Phi (\mathbf{x})-{\Phi }_0\right|}^2$$

4. $R(\mathbf{x})$ : Total variation regularizer (encourages spatial smoothness)

$$R_{V^{\beta }}=\sum _{i,j}{\left((x_{i,j+1}-x_{i,j}{)}^2+(x_{i+1,j}-x_{i,j}{)}^2\right)}^{\beta /2}$$

Example

This example gives us the images generated when we use feature vectors from different depth

We can see feature vectors from deeper layers get rid of more features of the original image