Example 17 Given that $a, b, c, d$ are non-zero real numbers, $f(x)=\frac{a x+b}{c x+d}, x \in \mathbf{R}$, and $f(19)=$ 19, $f(97)=97$. If for any real number $x$, when $x \neq-\frac{d}{c}$, $f[f(x)]=x$, try to find the only number outside the range of $f(x)$.