5x2-8xy+y2-(2x2-3xy+7y2) is the way to write the question.
The next step is to apply distribution of the negative outside the bracketed expression. The minus sign switches all the signs within this expression:
5x2-8xy+y2-2x2+3xy-7y2.
Next, look for similar terms and group them:
5x2-2x2 - 8xy+3xy +y2-7y2.
Now perform arithmetic in these groups:
3x2 - 5xy - 6y2 which is the answer for this subtraction.
7x+xy-5-(2x2-3xy+7y2).
Note that xy is the only common term in the expressions.
7x+xy-5-2x2+3xy-7y2,
7x + xy+3xy -5-2x2-7y2,
7x+4xy -5-2x2-7y2. This look untidy so rearrange the terms by degree (greatest first):
-2x2+4xy-7y2+7x-5 is tidier. The degree is obtained by adding the exponents of variables in each term.
x2 has degree 2, as do xy and y2 so these come first. x has degree 1 and there's only one term, that is, 7x. The constant has degree 0 so comes last.