Let $S$ be the solution set of the original system. It suffices to check each operation separately, since a sequence of them is just repeated application.

Swap two rows. Swapping equations $i$ and $j$ doesn't change which equations must hold simultaneously — it's the same list of conditions in a different order. So the solution set is unchanged.

Scale a row by $c\neq0$. Replacing equation $E_i:\;a_{i1}x_1+\cdots=b_i$ by $cE_i$ keeps the same solutions: any $\mathbf{x}$ satisfying $E_i$ satisfies $cE_i$ (multiply both sides by $c$), and any $\mathbf{x}$ satisfying $cE_i$ satisfies $E_i$ (multiply both sides by $1/c$, valid since $c\neq0$). Every other equation is untouched, so the full solution set is unchanged.

Add a multiple of row $j$ to row $i$. Replace $E_i$ by $E_i+cE_j$ (row $j$ itself is kept as-is). If $\mathbf{x}\in S$, it satisfies both $E_i$ and $E_j$, hence satisfies $E_i+cE_j$ — so $\mathbf{x}$ still satisfies the new system. Conversely, if $\mathbf{x}$ satisfies the new system, it satisfies $E_j$ (unchanged) and $E_i+cE_j$; subtracting $cE_j$ from the latter recovers $E_i$, so $\mathbf{x}$ satisfies the original system too. Every other equation is untouched, so the solution set is exactly $S$ again.

Since each operation individually preserves the solution set, so does any sequence of them — including the sequence that produces row-echelon form.

Let $\Delta=ad-bc\neq0$ and $B=\dfrac1\Delta\begin{pmatrix}d&-b\\-c&a\end{pmatrix}$. Multiply directly:

$$AB=\frac1\Delta\begin{pmatrix}a&b\\c&d\end{pmatrix}\begin{pmatrix}d&-b\\-c&a\end{pmatrix} =\frac1\Delta\begin{pmatrix}ad-bc&-ab+ba\\cd-dc&-cb+da\end{pmatrix} =\frac1\Delta\begin{pmatrix}\Delta&0\\0&\Delta\end{pmatrix}=I.$$

The other order needs its own check — matrix multiplication doesn't commute, so $AB=I$ alone doesn't yet give $BA=I$ for free. Multiplying with the factors reversed: $$BA=\frac1\Delta\begin{pmatrix}d&-b\\-c&a\end{pmatrix}\begin{pmatrix}a&b\\c&d\end{pmatrix} =\frac1\Delta\begin{pmatrix}da-bc&db-bd\\-ca+ac&-cb+ad\end{pmatrix} =\frac1\Delta\begin{pmatrix}\Delta&0\\0&\Delta\end{pmatrix}=I.$$

Since $AB=BA=I$, $B$ is exactly the inverse required by Definition 3.3, so $A^{-1}=B$.

It suffices to show $B^{-1}A^{-1}$ satisfies the defining property of $(AB)^{-1}$ — that it multiplies with $AB$, on both sides, to give $I$ — since Definition 3.3 already guarantees an inverse is unique when one exists.

Using associativity of matrix multiplication: $$(AB)(B^{-1}A^{-1})=A\big(BB^{-1}\big)A^{-1}=A\,I\,A^{-1}=AA^{-1}=I,$$ $$(B^{-1}A^{-1})(AB)=B^{-1}\big(A^{-1}A\big)B=B^{-1}\,I\,B=B^{-1}B=I.$$

So $B^{-1}A^{-1}$ is a two-sided inverse of $AB$; by uniqueness of inverses, $(AB)^{-1}=B^{-1}A^{-1}$.

Let $A=\begin{pmatrix}a&b\\c&d\end{pmatrix}$, $B=\begin{pmatrix}e&f\\g&h\end{pmatrix}$. By Definition 3.2, $AB=\begin{pmatrix}ae+bg&af+bh\\ce+dg&cf+dh\end{pmatrix}$, so

$$\det(AB)=(ae+bg)(cf+dh)-(af+bh)(ce+dg).$$

Expanding both products: $$=acef+adeh+bcfg+bdgh-\big(acef+adfg+bceh+bdgh\big) =adeh+bcfg-adfg-bceh.$$

Grouping the four surviving terms by $(eh-fg)$: $$=ad(eh-fg)-bc(eh-fg)=(ad-bc)(eh-fg)=\det(A)\det(B).$$

($\Leftarrow$) If $\det(A)\neq0$, Theorem 3.1 already exhibits an explicit inverse, so $A$ is invertible.

($\Rightarrow$) Suppose instead $\det(A)=ad-bc=0$; we show $A$ cannot be invertible, by producing a nonzero $\mathbf{x}$ with $A\mathbf{x}=\mathbf{0}$. There are three cases, covering every possibility for which rows of $A$ are zero.

Both rows zero. If every entry of $A$ is $0$, then $A\mathbf{x}=\mathbf{0}$ for every $\mathbf{x}$, including $\mathbf{x}=(1,0)\neq\mathbf{0}$.

Row $1=(a,b)\neq(0,0)$. Then $ad=bc$ forces row $2=(c,d)$ to be a scalar multiple of row $1$ (take the multiple $k=c/a$ if $a\neq0$, or $k=d/b$ if $b\neq0$; one of these is always well-defined since row $1\neq(0,0)$, and $ad=bc$ is exactly what makes it work for the other coordinate too). Then $\mathbf{x}=(-b,a)$ — nonzero, since row $1\neq(0,0)$ — satisfies $A\mathbf{x}=\mathbf{0}$: row $1\cdot\mathbf{x}=-ab+ba=0$, and row $2\cdot\mathbf{x}=-cb+da=da-cb=ad-bc=0$ by hypothesis.

Row $1=(0,0)$ but row $2=(c,d)\neq(0,0)$. This is the case the previous paragraph's labeling skips over, and it needs its own construction: take $\mathbf{x}=(-d,c)$ — nonzero, since row $2\neq(0,0)$. Row $1\cdot\mathbf{x}=0$ automatically, since row $1$ is entirely zero. Row $2\cdot\mathbf{x}=c(-d)+d(c)=-cd+dc=0$ always, regardless of $\det(A)$ — this case needs no help from $ad-bc=0$ at all, since a single zero row already forces $A$ to be singular.

Every case produces a nonzero $\mathbf{x}$ with $A\mathbf{x}=\mathbf{0}$. If $A$ were invertible, multiplying both sides by $A^{-1}$ would give $\mathbf{x}=A^{-1}(A\mathbf{x})=A^{-1}\mathbf{0}=\mathbf{0}$, contradicting $\mathbf{x}\neq\mathbf{0}$. So $A$ is not invertible.