So this just a quick post, detailing a fix that I spent a good half day trying to get find. I'm currently working an idea for cool RL algorithms involving a bunch of agents living in an interactable world (think dwarf fortress, rimworld etc.), and I wanted to write the actual logic in C++ to get a speedup over the python interpreter.
The simplest solution that came to mind was to write the backend logic in C++, such as world generation and AI logic, and then have python bindings to make the environment viewable by humans and/or interfaceable from a python reinforcement learning agent.
As it turns out, a solution called Boost exists, that allows us to compile C++ binaries with a python interface. I consulted a bunch of tutorials, but kept running into weird errors like
"Py Unicode Type", referenced from: boost::python::to_python value<char const* const&>::get_pytype() const in hello.o
make[3]: *** No rule to make target `/Users/kevin/anaconda3/envs/py3/lib/libpython3.so', needed by `pylib.so'. Stop.
Fatal Python error: PyThreadState_Get: no current thread
Turns out, the issue was all because I was using a python installation provided by Anaconda, which is statically linked. Boost does not like this and will crash if it is passed a reference to the Anaconda libraries.
In the end, I finally stumbled upon a setup that can compile correctly. I don't actually have any significant knowledge on how Cmake works, but for anyone who encounters the same issues I was having, the key was to unlink the python libraries in CMakeLists.txt:
target_link_libraries(pylib ${Boost_LIBRARIES} ${PYTHON_LIBRARIES})
changes to:
target_link_libraries(pylib ${Boost_LIBRARIES})
Then, so make doesn't complain while building, add the line to CMakeLists:
set_target_properties(pylib PROPERTIES LINK_FLAGS "-undefined dynamic_lookup")
With this change, hopefully the Boost-ed C++ library should compile correctly and create your python library! For reference, here is the base setup I ended up with: https://github.com/kvfrans/BaseBoost