Knowing that I’m working with an unsafe API, but with knowledge of the guarantees of that API as far as thread safety etc etc, my question is this:

Is it okay to pass MyStruct::struct_method(&mut self) to a C API in place of unsafe extern "C" fn(*mut MyStruct) for callback registration where MyStruct is #[repr(C)]?

I am able to do this in practice and it works well so far, though I have to transmute the function pointer when passing to my bindgen generated bindings.

I just want to make sure I’m not shooting myself in the foot before I go much further down this path. I would really like to not have to write a bunch of unsafe extern "C' fn functions that simply call back into my struct methods if possible.

It does look like I should extern "C" my methods so that I can be sure of the calling convention…

BTW, you may notice that I actually have to transmute to a function that takes no arguments… though, the callback actually does take arguments, that is just an artifact of how the API works.

I am able to do this in practice and it works well so far, though I have to transmute the function pointer when passing to my bindgen generated bindings.

This is usually a bad idea. Unless you declare otherwise there’s no guarantee that struct_method() will have the "C" ABI. I’m also not sure whether it’s guarantee’d that method invocation in Rust is equivalent to invoking a function who’s first argument is a pointer to self. For example, with some C++ compilers the this pointer is normally passed in via a register. The funny thing about UB is that it can appear to work for your use case, but the slightest change in environment (e.g. a rustc upgrade or changing OS) can result in nasal demons.

A better way is to have a “trampoline” function which takes a *mut c_void then casts that pointer back to *mut MyStruct so it can invoke the method normally. It can get annoying to write these trampolines, but that’s normally the best way to do it without UB. This is what we did when passing callbacks to libsignal-protocol-c.

This “tarmpoline” trick is particularly important when you want to pass a Rust closure to C. I usually make a helper function which will generically “write” the trampoline function and return a (extern "C" fn (*mut c_void), *mut c_void). It shouldn’t be too hard to alter that helper function to take a closure that’ll invoke your struct method.

xnor:

BTW , you may notice that I actually have to transmute to a function that takes no arguments… though, the callback actually does take arguments, that is just an artifact of how the API works.

Uhh… that doesn’t sound like a great idea. I feel like changing a function pointer’s arity with a cast/transmute could mess up with the bookkeeping automatically injected by the compiler for managing the stack…

Is it okay to pass MyStruct::struct_method(&mut self) to a C API in place of unsafe extern "C" fn(*mut MyStruct) for callback registration where MyStruct is #[repr(C)] ?

Given the level of indirection, it is not that much about MyStruct being #[repr(C)] that matters here (it’s mostly about alignment issues or if C directly manipulates the pointee), but really about MyStruct::struct_method having been declared extern "C".

It so happens that in the code you’ve linked to, you do declare these methods as such. But it is important to mention for someone just skimming at your post.

Now, transmuting the arity of a function is something extremely dangerous to do, but after looking at what the C library does, their t_method (i.e., a void (*) (void)) is actually an opaque pointer type that is dynamically casted (back) to the right arity. So this is not really a Rust FFI issue, but an “issue” with the API of the C library. Given that the arity of the t_method type is ignored, I’d personally have expressed that at the type level by requiring an opaque type with the size of a (function) pointer.

Thanks!
I’m likely going to be writing helper macros for this stuff anyways so using trampolines is probably not that big of a deal because I should be able to generate them with my macros … And, this helper function for closures, I’m gonna have to look a little closer to get my head around how this works but that sounds very useful!

So, with that in mind, is there any real difference, besides the mangling that happens, between:

impl MyStruct {
pub extern "C" fn my_method(&mut self)

and

pub extern "C" fn plainfunc(*mut MyStruct)

As far as the “safety” their being sent to a C API?

impl MyStruct {
pub
extern "C"
fn my_method(&mut self)

gives the function

pub
extern "C"
fn MyStruct::my_method (_: &mut MyStruct)

So, the question is whether such function is equivalent to

extern "C"
fn foo (_: *mut MyStruct)

Or if, at least, the former function can be safely transmuted into the latter.

For two types A and B, what does using a f: fn(A) where a fn(B) is expected mean?

It means that f, a function that expects its first and only argument to be a A, may actually be fed a B. This is known to be sound when transmuting any x: B into a A is sound.

(With subtyping notation, it implies that if B : A, then fn(A) : fn(B). It is called contravariance).

So the question becomes: can a *mut MyStruct be transmuted into a &'_ mut MyStruct?

The answer is: no. Not always at least. It is unsafe.

Funnily enough, the fn transmutation can still be made soundly by requiring that the caller of the transmuted function use unsafe. This is achieved by transmuting the function into an unsafeextern "C" fn (*mut MyStruct). This means that the call is sound only as long as the caller respects some invariant (in this case, that the argument is a valid pointer to a (mutable and) unaliased MyStruct).

So, if you know that the C function will only feed such a pointer to the callback, then feeding this transmuted unsafe callback to the C function is sound.

Usually, we cannot really know that about C (the non-aliasing guarantee is non-existant in that language), so in practice you just hope it gives a valid (thus non-NULL) pointer, which in single threaded programs can be an acceptable assumption (Ideally, for guaranteed soundness, using shared & _ references with interior mutability makes it sound in single-threaded contexts, and with an added Sync bound on the pointee makes it sound in any context).