The following code will show that the recomposed matrix is not the same as the original:
auto p = glm::perspective(glm::radians(45.0f), 1.0f, 0.1f, 100.0f);
auto t = glm::translate(glm::identity<glm::mat4>(), 0.0f, 0.0f, -5.0f);
glm::vec3 scale, translation, skew;
glm::quat rotation;
glm::vec4 perspective;
auto A = p * t;
glm::decompose(A, scale, rotation, translation, skew, perspective);
auto B = glm::recompose(scale, rotation, translation, skew, perspective);
If you check it, B is exactly 5 times smaller than A (scaled -1/z times, as z in translations)
The algorithm for decomposing the 4x4 matrix seems to be like thsi:
Let M4 be the target 4x4 matrix, we conjure up the composition:
$$\begin{align}
M_4 = P * T * M_3 & \\\
P &=
\begin{pmatrix}
1 & 0 \\\
p & p_3
\end{pmatrix}\\\
T &=
\begin{pmatrix}
1 & t \\\
0 & 1
\end{pmatrix}\\\
M_3 &=
\begin{pmatrix}
M & 0 \\\
0 & 1
\end{pmatrix}\\\
T\cdot M_3 &=
\begin{pmatrix}
M & t \\\
0 & 1
\end{pmatrix}\\\
M_4 = P\cdot T\cdot M_3 &=
\begin{pmatrix}
M & t \\\
p\cdot M & p_3 + p\cdot t
\end{pmatrix}
\end{align}$$
The algorithm seems to be constructing this $T\cdot M_3$ from $M_4 $, and then calculate $P$ from $(p, p_3) = M_{4\ 3}\cdot \left(T\cdot M_3\right)^{-1}$. Its purpose seems to be to zero out the last row.
Then $M$ is then QR decomposed into the other factors. As you can see, M4[3][3] is not 1 in the general case.
In fact, if using an actual perspective, that element will be -z from translation, and is not 1 in general. The code at the beginning put in this scaling factor and does not give it back anywhere:
// Normalize the matrix.
if(epsilonEqual(LocalMatrix[3][3], static_cast<T>(0), epsilon<T>()))
return false;
for(length_t i = 0; i < 4; ++i)
for(length_t j = 0; j < 4; ++j)
LocalMatrix[i][j] /= LocalMatrix[3][3];
Removing this fixes the problem with the example.
I understand the need to check if you can actually invert the matrix. But what about using the $M_4$ and $M$ determinants? They have the identity $\left|M_4\right| = p_3 \times \left|M\right|$, if both determinants are not 0 or infinity, this construction of factoring out the last row seems to hold anyways.
The following code will show that the recomposed matrix is not the same as the original:
If you check it, B is exactly 5 times smaller than A (scaled -1/z times, as z in translations)
The algorithm for decomposing the 4x4 matrix seems to be like thsi:
Let M4 be the target 4x4 matrix, we conjure up the composition:
The algorithm seems to be constructing this$T\cdot M_3$ from $M_4 $ , and then calculate $P$ from $(p, p_3) = M_{4\ 3}\cdot \left(T\cdot M_3\right)^{-1}$ . Its purpose seems to be to zero out the last row.
Then$M$ is then QR decomposed into the other factors. As you can see,
M4[3][3]is not 1 in the general case.In fact, if using an actual perspective, that element will be -z from translation, and is not 1 in general. The code at the beginning put in this scaling factor and does not give it back anywhere:
Removing this fixes the problem with the example.
I understand the need to check if you can actually invert the matrix. But what about using the$M_4$ and $M$ determinants? They have the identity $\left|M_4\right| = p_3 \times \left|M\right|$ , if both determinants are not 0 or infinity, this construction of factoring out the last row seems to hold anyways.