The probability is just the ratio of the area of the circle to the area of the square.
Suppose the square has side a then its area is a2 and the radius of the inscribed circle is a/2 and its area is πa2/4.
The ratio is therefore (πa2/4)/a2=π/4=0.7854, making the probability about 78.54% that a point in the square also lies in the circle.
(The circle touches the square at just 4 tangential points, so if you were thinking of points on the circumference of the circle and the perimeter of the square, the probability is vanishingly small because a point has zero dimensions and the circumference contains an infinite number of points.)