Hello fellow student of Stroh, how goes it?
Anyways, to solve this problem heres what you do.
5A^2-6A=3I
5/2A^2-2A=I
A(5/2A-2I)=I
A^-1 * A (5/2A-2I) = A^-1 * I
5/2A-2I=A^-1
So as weed need to find AB=I, we know that B=A^-1 (A^-1*A*B=A^-1*I) and the above found A^-1, just multiply that by A to get your proper form.
A(5/2A-2I)=I
A B =I