I'm just wondering if there is any simple/efficient way to check if square falls inside a triangle. or at least one corner falls inside or some overlaps. e.g. considering the figure below, I should be able to tell that the 3 squares fall inside. i.e. square 1 is obviously inside, one corner of square 2 is inside, and 3 overlaps.

Please specify how the objects are specified - are you given the coordinates of the corners? Also, what happens if the square overlaps the triangle but no corners are inside the triangle - is that considered "inside"?
–
BitwiseSep 15 '12 at 20:45

given the (x,y) coordinates of each vertex in a triangle and squares (or in worst scenario, can be calculated). overlaps. hm.:'-( . . .i never thought of that one, but that would be considered inside.
–
r mSep 16 '12 at 22:59

6 Answers
6

Consider your triangle as three vectors all in a fixed rotation order: A->B, B->C and C->A

Now, consider for each triangle vertex you have four vector from that vertex to each square vertex. You compute the cross-product between each triangle edge-vector and each triangle-square vector (12 in total). If the cross-products are all the same sign (or zero), your square is inside.

Conceptually you are trying to determine whether the square vertices are on the left or right side of your line. You don't actually care whether it's left or right, or whether you're in clockwise or anticlockwise order... Only that the square vertices are on the same side of all the triangle vectors.

When I say 'sign' of the cross product, naturally I mean that your 2D co-ords are actually (x,y,0), and you are looking at the sign of the z-value after. To handle every other case, it might be easiest to just check whether any of the square edges cross the triangle edges.
–
paddySep 16 '12 at 22:22

The method in my answer will tell you if the square is entirely inside the triangle. The method in my comment (about bisecting edges) will tell you if the square intersects the triangle (hence overlapping). The case not handled, of course is where the triangle is completely inside the square. You might need to clarify your problem further. What other types of triangle-square intersections are classed as 'inside'?
–
paddySep 16 '12 at 23:25

hi paddy, currently i'm working on a simplifying assumption that a triangle would always be bigger than squares, hence cases are square being completely inside and square that overlaps or has a corner inside.
–
r mSep 16 '12 at 23:53

1

This fails even for simple cases. Did you actually test it?
–
hirschhornsalzSep 18 '12 at 22:20

Create the triangle image with 0-1 values (this is the hard part);
then for each square, create its 0-1 image, which is very simple;
add both images, and calculate the values at different triangle or square coordinates. You can even calculate the area of the overlapping region.

What you are actually trying to do is determine whether the square is linearly separable from the triangle, i.e. if there is a line you that separates the two objects. If such a line exists, then they do not intersect. There are several algorithms to test linear separability. The upside is that they are general so it will work with other polygons. The downside is that because of their generality they might not use specific characteristics of your problem to simplify the solution.

Since you asked about matlab, an because other answers explained the direct approach, I will mention some available solutions. You might want to take a look at polyxpoly (if you have the Mapping Toolbox). It can handle more general cases. Otherwise, there is a bunch of contributions on the File Exchange, see e.g. Curve Intersect 2. Polygon_Intersection on the other hand returns the overlapping regions, not just intersection points.