We can guess the answer by looking at the coefficients. We’re looking at factors of 2 for the constant. There are only 2 factors: 1 and 2. So we know the constant part of one factor is 2 and of the other it’s 1.
The coefficient of the x² term is 1 so the x part of the factor is x in each factor.
That leaves y, so we give it the coefficients a and b.
We have:
(x+ay+1)(x+by+2)=x²+bxy+2x+axy+aby²+2ay+x+by+2≡x²-4y²+3x-2y+2.
Comparing coefficients: a=-b because there are no xy terms.
ab=-4 equating y² terms, so a²=4, and a=±2, b=∓2.
2a+b=-2 equating y terms, so 2a-a=-2, and a=-2, consistent with a=±2, so b=2.
The x terms balance 2x+x=3x.
The factorisation is (x-2y+1)(x+2y+2)=x²-4y²+3x-2y+2.