trying to install the yt package, most recent version 3.2.1 on python 3.5.0 using pip
pip3 install -U yt
error output attached. The same package installs fine with python 3.4.3, same compiler and also build from scratch, so the suspicion is that it is related to the new 3.5.0 version. The system installed is the current Fedora 22, gcc 5.1.1-4.

It looks like the problem is that yt uses OpenMP whereas OpenMP doesn't support atomic operations:
In file included from /home/alex/Python/include/python3.5m/pyatomic.h:12:0,
from /home/alex/Python/include/python3.5m/Python.h:53,
from build/src.linux-x86_64-3.5/yt/utilities/lib/geometry_utils.c:4:
/usr/lib/gcc/x86_64-redhat-linux/5.1.1/include/stdatomic.h:40:1: sorry, unimplemented: â€˜_Atomicâ€™ with OpenMP
typedef _Atomic _Bool atomic_bool;
^
I'm not sure that Include/pyatomic.h should be included by the main Include/Python.h header. We already skipped Include/pyatomic.h on C++ because this header is incompatible with C++.

"When I just comment out the << #include "pyatomic.h" >> line, python 3.5.0 will no longer compile"
Sure. My idea is to "disable" the header with we are not building Python itself. There is a nice define for that: Py_BUILD_CORE. See attached patch.
Since all symbols in pyatomic.h are prefixed by _Py, this header is fully part of the Python private API and so it's fine to modify it in a bugfix release (3.5.0 => 3.5.1).

/home/alex/Python-3.5.0/Modules/_ctypes/_ctypes.c:2062:15: error: ‘_PyThreadState_Current’ undeclared (first use in this function)
if (Py_EnterRecursiveCall("while processing _as_parameter_")) {
^
Ah yes, if you check the fix for C++ (changeset cb05b6d7aacd), I also had to modify pystate.h.
Please try pyatomic-2.patch which hides more CPython internals in public headers.

I created a venv with a Python patched with pyatomic-2.patch: I successfully installed yt. yt depends on numpy & Cython: good news, numpy & Cython were compiled correctly. These two libraries are well known users of the Python C API. It's not enough to check if this change breaks modules on PyPI, but it's still a good news :-)

> Would there be a way to expose these internals rather than hiding them?
Here is the issue is that pyatomic.h cannot be compiled on OpenMP. We had the same issue with C++. In fact, it doesn't make sense to compile pyatomic.h differently to access an atomic variable from an extension module. We must always use exactly the same implementation, otherwise bad things will happen.
A solution for that is to hide the implementation details and only expose high level APIs.
For example, pyatomic.h must be completly hidden.
A consequence is that the _PyThreadState_Current variable must be hidden to. _PyThreadState_Current is an implementation detail, you must not access it directly.
The PyThreadState_GET() macro uses directly the _PyThreadState_Current variable. So the solution to expose the "PyThreadState_GET" symbol (not necessary as a macro) is to define it as an alias to the PyThreadState_Get() *function*.
The advantage of using a function is that we don't expose implementation details to third-party extensions, it avoids the risk of ABI incompatibilies.

Understood and agreed. Second patch looks good to me.
Cython calls PyThreadState_GET() in pretty much every helper function that deals with exceptions, but I doubt that the potential speed difference is going to be relevant in the real world. And we target CPython's API level anyway, not the ABI, so the C code will just adapt at compile time.

Should it work with /configure '--with-cxx-main=g++' && make?
Because currently it doesn't:
g++ -c -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -Werror=declaration-after-statement -I. -IInclude -I./Include -DPy_BUILD_CORE -o Programs/python.o ./Programs/python.c
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
In file included from Include/pyatomic.h:10:0,
from Include/Python.h:53,
from ./Programs/python.c:3:
/usr/lib64/gcc/x86_64-pld-linux/5.2.0/include/stdatomic.h:40:9: error: ‘_Atomic’ does not name a type
[...]
(testing 3.5.0 + patch for this issue from hg)

I think this should be reconsidered. I understand the desire to compile Python with OpenMP. But the resolution here is hiding _Py_atomic symbols all the time, even when OpenMP isn't involved, and even when building a standard extension module.

Case in point: if I want to include "internal/pystate.h" from _pickle.c, I get the following error:
"""
In file included from ./Include/internal/pystate.h:12:0,
from /home/antoine/cpython/default/Modules/_pickle.c:10:
./Include/internal/ceval.h:14:5: error: unknown type name ‘_Py_atomic_int’
_Py_atomic_int calls_to_do;
^
"""

Yes, why not fix this by putting the offending code under "#ifndef _OPENMP"?
What would be even better if is pyatomic.h wasn't included in Python.h, which should be fine since pyatomic.h doesn't have any public APIs. Maybe we can do that for 3.8.

(Ah, Benjamin restarted the discussion, so I reopen this issue.)
> I understand the desire to compile Python with OpenMP.
I'm not sure that I understood the use case. Do you want to only compile Python core ("python3" binary") or just stdlib C extensions, or both?
> But the resolution here is hiding _Py_atomic symbols all the time, even when OpenMP isn't involved, and even when building a standard extension module.
Sorry, but I don't understand the problem. Why is it an issue to hide _Py_atomic symbols?

This issue is very specific to *third party* extensions, not C extensions which are part of the standard library. That's why I asked if it wouldn't be better to open a new issue, if your use case concerns the stdlib and/or Python core.