This is one of those problems that would be much easier to explain by any way other than writing. It's easier to follow if you can see the problem being written out and have things pointed to.
Warning: I think I'm right, but I could be wrong.
Look up a thing called "pascal's triangle." It will make this much easier.
Let's do the expansion a bit... Don't worry too much about this part, it's just a chart to use further down. I listed more than we need below, but it's there to show a pattern.
(1+3x)^1 = 1 + 3x
(1+3x)^2 = 1 + 6x + 9x^2
(1+3x)^3 = 1 + 9x + 27x^2 + 27x^3
(1+3x)^4 = 1 + 12x + 54x^2 + 108x^3 + 81x^4
(1+3x)^5 = 1 + 15x + 90x^2 + 270x^3 + 405x^4 + 324x^5
(1+3x)^6 = 1 + 18x + 135x^2 + 540x^3 + 1215x^4 + 1944x^5 + 972x^6
If you were expanding (1+y)^n you would get things like 1+y, 1+2y+y^2, 1+3y+3y^2+y^3 and so on. That's what Pascal's triangle gives us.
Look at the triangle, row 12 (1, 12, 66...). If you wanted to expand (1+y)^12 you would get 1+12y+66y^2+220y^3...
But we're not expanding (1+y). We're expanding (1+3x). So instead of y, we have 3x.
As we expand (1+3x) we get 1+3x, 1+2(3x)+(3x)^2, 1+3(3x)+3((3x)^2)+((3x)^3), and so on.
The question asks about the coefficient of x^2 being 324, so it's asking about 324x^2. We want to know when that 324 happens.
Remember we're not doing (1+x)^n, but (1+3x)^n. That means we get BLAH(3x)^2 = BLAH(9x^2) = 324x^2. (we don't know what BLAH is yet)
BLAH(9x^2) = 324x^2
Divide both sides by 9x^2
BLAH = 36
Look at the triangle again.
See how the left side is 1, 1, 1, forever? That's the stuff that makes the 1 (all by itself).
See how the next side in is 1, 2, 3, 4, and so on? That's the stuff that makes the number sitting next to the x (not the x^2, x^3, etc.).
See how the next side in is 1, 3, 6, 10, 15, 21, 28, 36, and so on? That's the stuff that makes the number sitting next to the x^2 (not the x, x^3, x^4, etc.).
We care about the x^2, so we're interested in the 3rd step in on the left side (1, 3, 6, 10, ...).
The top row of the triangle (the 1 all by itself) is row 0. The 1, 1 is row 1. The 1, 2, 1 is row 2, and so on.
Remember BLAH = 36? On the triangle, left side, 3rd step in, 9th row is 36. The row number is n. That means n = 9.
Answer: n = 9
Another problem: If the coefficient of x^3 in the expansion of (1+4x)^n is 640, find n.
It's not x^3, it's (4x)^3 = 64x^3
640(x^3) = BLAH(64x^3)
Divide both sides by 64x^3
10 = BLAH
We're doing x^3, so it's the 4th step in from the left side.
Look for the 10. It's in the 5th row. (remember the very top is row 0)
Answer: n = 5.
Another problem: If the coefficient of x^2 in the expansion of (1+2x)^n is 264, find n.
It's not x^2, it's (2x)^2 = 4x^2
264(x^2) = BLAH(4x^2)
Divide both sides by 4x^2
66 = BLAH
We're doing x^2, so it's the 3rd step in from the left side.
Look for the 66. It's in the 12th row.
Answer: n = 12.