On the drawing, split the square in half by drawing a vertical line through the middle. From the top of the shaded portion, where the vertical crosses, draw a line down to the corner at B. This is a right triangle, having base 5 (being half of the side-length) and hypotenuse 10 (because its also a radius line of the circle centered at B). What is the length of the other leg, being the height of the triangle?
Note that the segment from the top of this triangle down to A has the same length. If we label the top of the original triangle as "X", what sort of triangle is AXB? So what is the measure of angle ABX? What portion of the circle does this represent? So what is the area of the contained wedge?
Take the wedge ABX and subtract out the triangle AXB. This leaves you with a curvy-sided slice.
You know the measure of angle ABX. What then is the measure of XBC? What is the area of this sector?
If you subtract the curvy-sided slice from this sector area, you will get one-fourth of the area outside of the shaded part. So what is the total area of the unshaded part? Given the total area of the square, what is the shaded area?