By using a MEX file, I am trying to loop through array "group_ptr" with size 1000×1 to make a cell with size 4×1. group_ptr(i) contains the group ID which "i" belongs to. The 4 groups have different number of elements. My MEX file does other things beside to creating this cell. It takes one input and generates 4 outputs. When it gets to making the cell, MATLAB crashes.

However, when I comment the part for making the cell and reduce the number of outputs to 3, MEX file works fine. I would be grateful if someone could help me what the problem could be.

Below is the part of my code which creates this cell. The MEX file has 4 output which the last one is this cell.

mxSetCell( CellArray_ptr, i, A );
mxDestroyArray( A ); // get rid of this line

Get rid of the mxDestroyArray call. When you call the mxSetCell function, three things happen:

1) The address of A gets put into the i'th cell location in the CellArray_ptr variable

2) The type of A gets changed from "temporary" to "sub-element"

3) The address of A gets removed from the garbage collection list.

After the mxSetCell function call, A is literally now part of the CellArray_ptr array. Its ultimate clearing will depend entirely on what happens to CellArray_ptr. When you call mxDestroyArray on A, you are invalidating part of a legitimate variable. Hence when MATLAB tries to access or clear CellArray_ptr later on it crashes since it accesses invalid memory.

You can find this detailed information in the JUG MEX document (James's Ultimate Guide to MEX programming) that I have written. You can find the document on the FEX ... umm ... wait a minute ... I haven't actually published it to the FEX yet ... well, when I finally get around to finishing this document and posting it to the FEX then you can find all these gory details (along with undocumented API function details, etc.). I will try to finish that up soon and post it.

I can probably send you a preliminary incomplete doc if you are interested, with all the usual caveats about potential errors in the doc and using undocumented functions. Just e-mail me through the link and let me know.