Re: wxLauncher Test Builds [Updated: 2016-09-04]

I can try Iss Mneur's patch to fix case sensitivity in FSO build names later. Guess I should test it on Linux too if no one else can.

Collection of replies to the various comments since my last post:

Having a distributable OS X build is certainly good but we still need to get CI working again before we can move forward with 0.12.0 RC3.

Sure, if the scripts now work with Python 2, I can remove Python 3 from the ReadMe. Note that Python is only required to build wxL, not to use it.

For the CI, can't we set up the scripts to build wxWidgets on the CI ser ver exactly once and then use those built libraries for all future CI tests?

Still not enthusiastic about prebuilt wxWidgets libraries (are they release or debug?) for official use if it requires regex hacking and additional CMake tinkering to get wx-config to work. I really don't think it's that hard to spend the ~5-10 minutes it takes to download, configure, and build the libraries if you copy/paste the commands from the wxL ReadMe. I'm even less enthusiastic about storing the prebuilt libraries in the wxL repo since they take up a lot of space. And I still don't understand why Homebrew requires special patches for wxWidgets 3.0.2 to work when I can use the vanilla 3.0.2 source without issues.

Okay, I guess people can get SDL2.framework from the official SDL2 site rather than the FSO source tree. I'll update the ReadMe.

I don't know if ~/Library/Frameworks is a standard location for storing libraries, but the CMake find_library() command is able to find SDL2.framework if it's stored there. I figure it's a better choice than asking people to put it in /Library/Frameworks since in the first case it's in their home folder rather than a system one. Note that placing the framework there is only required to build wxL, not to run it. It can be removed as soon as the user is finished building wxL.

IIRC wxWidgets 2.8 can be either Unicode or non-Unicode. wxL releases have always been built with Unicode builds of wxWidgets. I think 3.0 is Unicode-only? In which case --enable-unicode is not required to build wxWidgets. Will need to look into that and update the ReadMe if needed.

Yes I'd also prefer Cmd-Q for quitting wxL but I have no idea how to get it working again and am doubtful it's worth trying to track down assuming it's even fixable in the first place.

As for the case where the user has both Homebrew SDL2 and SDl2.framework, the user will need to explicitly specify the path to their SDL2.framework (i.e., set the SDL2_FRAMEWORK CMake variable) when running CMake. The CmakeLists.txt will need to be updated to check if SDL2_FRAMEWORK has already been specified by the user and look for it with the CMake find_library() command if it has not. I don't think there's any way to tell CMake to look for a framework and not a library built by something like Homebrew.

Honestly though, I'm really not sure how much time it's worth investing (and complexity worth adding) in making wxL as easy to build on OS X as possible, given how rarely people need to do it and how few people (only a handful of devs) need to do it, and especially if we don't do these sorts of things for Windows, where building wxL is more common. My $0.02. By complexity I mean the required changes to the CMakeLists.txt. In the 5+ years since I joined the SCP I have never heard of an OS X player having to build wxL.

I don't think there's any value in maintaining 2.8 compatibility except where needed to keep Linux people happy.

Re: wxLauncher Test Builds [Updated: 2016-09-04]

I can try Iss Mneur's patch to fix case sensitivity in FSO build names later. Guess I should test it on Linux too if no one else can.

Mac was already tested and confirmed, guessing he tested on Windows at least, so I doubt there's much besides Linux left to test it on. I plan on trying to compile wxLauncher for FreeBSD at some point for S&Gs.

Having a distributable OS X build is certainly good but we still need to get CI working again before we can move forward with 0.12.0 RC3.

No argument there, and having a system that can automatically retrieve and utilize the necessary dependencies the same way for both release builds, dev builds, and CI builds would greatly increase the effectiveness of the CI system overall.

Sure, if the scripts now work with Python 2, I can remove Python 3 from the ReadMe. Note that Python is only required to build wxL, not to use it.

Removing Python 3 I agree with, but in the OS X specific section we may want to clarify again that there are some required Python modules that don't ship with OS X. Guessing that's already in the README but at least mentioning to re-read that part might be helpful then.

For the CI, can't we set up the scripts to build wxWidgets on the CI ser ver exactly once and then use those built libraries for all future CI tests?

Not sure about that. Could build it every time at least I guess, only an extra few minutes right? But if we just assembled a prebuilt library and used that for official builds, CI builds, and development, it would solve this problem as well.

Still not enthusiastic about prebuilt wxWidgets libraries (are they release or debug?) for official use if it requires regex hacking and additional CMake tinkering to get wx-config to work. I really don't think it's that hard to spend the ~5-10 minutes it takes to download, configure, and build the libraries if you copy/paste the commands from the wxL ReadMe. I'm even less enthusiastic about storing the prebuilt libraries in the wxL repo since they take up a lot of space. And I still don't understand why Homebrew requires special patches for wxWidgets 3.0.2 to work when I can use the vanilla 3.0.2 source without issues.

I built release builds as I believe that's what we've been using all along for prebuilt libs for FSO. We aren't debugging the libs themselves I think, so I don't believe we need the added debug bulk. There's also the chance that for whatever reason, other users might run into difficulties compiling the libraries themselves and get frustrated before they even get to trying to compile our code. Now, there's no one saying you should _have_ to use our libs, as the mechanism to override the lib location would always remain.

Regex replacement was one such possible solution the problem, it's definitely not the only one. Using a temp directory that is always the same would work just as well, and /tmp/ I think would be a good candidate for that. I have the libs currently set up to install to /tmp/wxmac/ so that no further changes are required to anything. I just configure wxLauncher to find the library there and it works. The lib could be stored on bintray if not in the repository, downloaded, extracted to /tmp/, and then nothing else would need to be done. And this would only be done if wxWidgets were not detected during the configuration in an existing location. This a mechanism already established and tested in the current FSO CMake files. Nothing groundbreaking here, we just would need to apply established concepts to this project from FSO.

I'm honestly not sure how you were able to compile stock wxWidgets 3.0.2 without error on Mac, when both myself and the people apparently maintaining the wxmac build in Homebrew all ran into the same compilation error. I am on 10.11 until my new work Macbook comes in and I can upgrade this one, and I was on Xcode 7.3.x at the time, now I'm on Xcode 8. I could try again with the new SDK to see if the patches are still necessary. That said, if a dedicated maintainer also feels those patches add value to the wxWidgets source base when compiled on OS X, it seems like a useful thing to consider using ourselves. It would actually make the library closer to the Homebrew wxmac release as well.

I don't know if ~/Library/Frameworks is a standard location for storing libraries, but the CMake find_library() command is able to find SDL2.framework if it's stored there. I figure it's a better choice than asking people to put it in /Library/Frameworks since in the first case it's in their home folder rather than a system one. Note that placing the framework there is only required to build wxL, not to run it. It can be removed as soon as the user is finished building wxL.

It is a better choice then, I agree. Downloading it automatically if it isn't found, like the prebuilt lib, could save people from even having to read the readme though, which is always nice, but at this point I wouldn't push for this too hard as it isn't hard to download a lib and put it in a user folder yourself

As for the case where the user has both Homebrew SDL2 and SDl2.framework, the user will need to explicitly specify the path to their SDL2.framework (i.e., set the SDL2_FRAMEWORK CMake variable) when running CMake. The CmakeLists.txt will need to be updated to check if SDL2_FRAMEWORK has already been specified by the user and look for it with the CMake find_library() command if it has not. I don't think there's any way to tell CMake to look for a framework and not a library built by something like Homebrew.

Is there a way to detect which is did find and issue a warning during the configuration process? It might save people the trouble from making a build, giving it out on the forums, and then getting a bunch of reports that it doesn't work. I imagine that the framework copy step would not even work correctly though, so I'm guessing something would blow up at some point if the framework isn't available. This is more just to help avoid confusion for any devs down the line as it doesn't improve the current process as long as you understand the caveats already. Not as important as fixing CI and getting a new release out the door anyway.

Honestly though, I'm really not sure how much time it's worth investing (and complexity worth adding) in making wxL as easy to build on OS X as possible, given how rarely people need to do it and how few people (only a handful of devs) need to do it, and especially if we don't do these sorts of things for Windows, where building wxL is more common. My $0.02. By complexity I mean the required changes to the CMakeLists.txt. In the 5+ years since I joined the SCP I have never heard of an OS X player having to build wxL.

There haven't _been_ many builds of wxLauncher, and one reason I'm guessing is that it's been such a bear to get it compiled on OS X. It sounds like you're describing a catch-22. No one has built wxL on Mac because it's difficult to build because no one has built it on Mac because it's difficult to build...

I'm just suggesting that maybe it doesn't have to be that way. I don't see how I'm suggesting a large investment as my suggestions already mirror completed tasks in FSO. Yes we either need someone to understand them and implement them here, or ask m|m to help us out since he did most of those, etc, but again, no one is talking about reinventing the wheel. If it was easier to compile, we could spend more time actually developing it, and less time trying to get it to compile for the next release. It helps that the process is better documented in the readme now though.

Re: wxLauncher Test Builds [Updated: 2016-09-04]

Disclaimer: given that I am still sick (although feeling slightly better), the people I met through uh social networking apps earlier this week and tried to meet IRL all flaked on me I'm typing this in Notepad++ which has ****ty accessibility, and that now I have to fix the aforementioned wxL bug on Linux (the platform where my assistive technology skills are the least developed), I'm not really in the cheeriest of moods. That said....

No argument there, and having a system that can automatically retrieve and utilize the necessary dependencies the same way for both release builds, dev builds, and CI builds would greatly increase the effectiveness of the CI system overall.

As would having a system that builds stock wxWidgets without modifications using a single standardized set of configuration commands, meaning among other things no support for libraries built with additional patches or with Homebrew.

Removing Python 3 I agree with, but in the OS X specific section we may want to clarify again that there are some required Python modules that don't ship with OS X. Guessing that's already in the README but at least mentioning to re-read that part might be helpful then.

The only required python module that's not in the standard distribution is markdown and yes the OS X section of the ReadMe will mention that, as it currently does and always has.

I built release builds as I believe that's what we've been using all along for prebuilt libs for FSO. We aren't debugging the libs themselves I think, so I don't believe we need the added debug bulk.

I like to use debug builds of wxWidgets with debug build releases of wxL. wxWidgets can act strangely (e.g., wxWidgets assertions mysteriously triggered, yes this has happened) and having those debug symbols can be helpful. I do use release builds of wxWidgets with official release builds of wxL though.

There's also the chance that for whatever reason, other users might run into difficulties compiling the libraries themselves and get frustrated before they even get to trying to compile our code. Now, there's no one saying you should _have_ to use our libs, as the mechanism to override the lib location would always remain.

If someone is following the wxL ReadMe but unable to build the libraries, then something is wrong with the ReadMe and it needs to be updated. Building wxL should always work if the ReadMe is followed to the letter. Yes that was impossible to do with the old set of ReadMe instructions; this should be fixed with the updated instructions. I will update the ReadMe to clarify that even though I specify the exact versions of software I used, the user may have success with earlier or later versions (e.g., Xcode 8.0 rather than 7.3.1). And as I have said more than once with Iss Mneur's support, I do not want to have to support multiple ways to build wxL.

Regex replacement was one such possible solution the problem, it's definitely not the only one. Using a temp directory that is always the same would work just as well, and /tmp/ I think would be a good candidate for that. I have the libs currently set up to install to /tmp/wxmac/ so that no further changes are required to anything. I just configure wxLauncher to find the library there and it works. The lib could be stored on bintray if not in the repository, downloaded, extracted to /tmp/, and then nothing else would need to be done. And this would only be done if wxWidgets were not detected during the configuration in an existing location. This a mechanism already established and tested in the current FSO CMake files. Nothing groundbreaking here, we just would need to apply established concepts to this project from FSO.

This is admittedly a more palatable approach than regex hacking, but what if the path the prebuilt libraries use is taken? Unlikely but a possibility we have to account for.

I'm honestly not sure how you were able to compile stock wxWidgets 3.0.2 without error on Mac, when both myself and the people apparently maintaining the wxmac build in Homebrew all ran into the same compilation error. I am on 10.11 until my new work Macbook comes in and I can upgrade this one, and I was on Xcode 7.3.x at the time, now I'm on Xcode 8. I could try again with the new SDK to see if the patches are still necessary. That said, if a dedicated maintainer also feels those patches add value to the wxWidgets source base when compiled on OS X, it seems like a useful thing to consider using ourselves. It would actually make the library closer to the Homebrew wxmac release as well.

As I have said multiple times, I have no interest in supporting Homebrew, and furthermore I see no value in getting a set of libraries that's closer to the Homebrew version. Is the Homebrew version officially sanctioned by the wxWidgets team? If it is, then perhaps I stand corrected.

Please try building the wxWidgets 3.0.2 official release using the instructions in the wxL ReadMe and report back with your results. I used the Clang compiler that ships with Xcode 7.3.1 on 10.11 but can't imagine there being an issue with using the Clang compiler that ships with Xcode 8. The configuration command in the ReadMe specifies Clang as the compiler because we need both C++11 and 10.9 support. I'm sure you know this and I'll add it to the ReadMe, but you can use the -j option with the make command to speed up the compilation process.

Also I am double especially not enthusiastic about libraries that were not only not built from an official wxWidgets release but that also include patches that are not strictly required for it to compile. It's unnecessarily veering even further away from an official release.

It is a better choice then, I agree. Downloading it automatically if it isn't found, like the prebuilt lib, could save people from even having to read the readme though, which is always nice, but at this point I wouldn't push for this too hard as it isn't hard to download a lib and put it in a user folder yourself

I have little sympathy for people who want to build wxL but can't be bothered to even look at the ReadMe. It's called a "ReadMe" for a reason.

If I'm reading what you wrote above correctly, are we in agreement then that anyone who is unable or unwilling to download and copy a framework to ~/Library/Frameworks should not be considered able to build (let alone work on) wxL?

Is there a way to detect which is did find and issue a warning during the configuration process? It might save people the trouble from making a build, giving it out on the forums, and then getting a bunch of reports that it doesn't work. I imagine that the framework copy step would not even work correctly though, so I'm guessing something would blow up at some point if the framework isn't available. This is more just to help avoid confusion for any devs down the line as it doesn't improve the current process as long as you understand the caveats already. Not as important as fixing CI and getting a new release out the door anyway.

Yes CMake can check if the detected SDL2 library's name ends in "framework". The CMakeLists.txt revisions I'm working on do just this, with a fatal error issued if the library is not a framework. I do not want to support builds of wxL that do not use SDL2.framework. As I've said a few times in this thread and Iss Mneur has agreed with, I don't want to support multiple build setups, most notably mixing Homebrew into the build process. Using an SDL2.framework is the only supported way to build wxL. anyone who wants to use Homebrew to build any part of wxL or any dependency for it is on their own as far as I'm concerned. I see no value in supporting Homebrew.

There haven't _been_ many builds of wxLauncher, and one reason I'm guessing is that it's been such a bear to get it compiled on OS X. It sounds like you're describing a catch-22. No one has built wxL on Mac because it's difficult to build because no one has built it on Mac because it's difficult to build...

You'll have to ask Iss Mneur for why there have been few releases of wxL. Another possibility is that there hasn't been a need for frequent releases. I still don't understand how OS X taking some work to build figures in. Yes it really was a pain before the wxL ReadMe instructions were revised. Hence the new instructions that I worked very hard on getting right. I don't want to support versions of wxWidgets that were built in some other way.

Also, it's arguably even harder to build wxL on Windows, where Python isn't even included and prebuilt libraries aren't even possible because of CRT/SxS issues. And yet there actually have been contributors on Windows such as m!m and AdmiralRalwood.

I'm just suggesting that maybe it doesn't have to be that way. I don't see how I'm suggesting a large investment as my suggestions already mirror completed tasks in FSO. Yes we either need someone to understand them and implement them here, or ask m|m to help us out since he did most of those, etc, but again, no one is talking about reinventing the wheel. If it was easier to compile, we could spend more time actually developing it, and less time trying to get it to compile for the next release. It helps that the process is better documented in the readme now though.

OS X support for wxL has fallen to me because no one else wants to do it. If someone else would like to provide OS X wxL support they can support any number of build setups they like using any sets of instructions they like. But I want one and only one way to build wxL if I'm the one doing this, and having the option to use prebuilt libraries, especially ones that were not built using official wxWidgets releases, unnecessarily makes this task even harder. As I said earlier, it's hard enough getting wxL to work with exactly one setup of wxWidgets/SDL2/etc. I do not want to have to provide support for multiple build setups.

With the new build instructions that took a lot of research and experimentation to get right (particularly with respect to configuring wxWidgets) it should no longer be particularly difficult to build wxL on OS X.

Also I have never heard an OS X user express interest in working on wxL, let alone be deterred because of the build instructions. The only other SCP C++ coders I know with a Mac are Swifty and Echelon9. To the best of my knowledge, neither one has ever contributed to wxL nor has either one expressed interest in doing so, and I am not sold on their being suddenly interested if they weren't required to build wxWidgets to work on wxL.

Re: wxLauncher Test Builds [Updated: 2016-09-04]

The FSO community is the only development community I've ever really been involved with outside of work. I got involved in FSO because of my involvement with FotG. However, for quite a while after I was working with FotG, I had no real interest in messing with FSO. I had tried several times back when the VCS was still CVS and the build system was not terribly well documented or supported. This seemed to be because the devs had gotten used to knowing what was needed and kept up with changes, but it increased the barrier to entry for new developers. Shortly after the move to SVN, the build system had been stabilized a bit more and I could make a build without a bunch of configuring and tweaking things, all of which were new concepts to me as a very new C++ developer. But once I had it figured out, I was able to make improvements to the cross platform support, submit some patches, and even wrote the original nightly build system, all because the barrier to entry had been lowered enough for me to figure some things out. But initially I remember running into issues with wading through dependency requirements that I found offputting at first. We can act all day like devs shouldn't be offput by those kinds of things but the fact is, if a new developer comes by this project, it's like a new TV show. If it takes too much effort to become interesting, ADD will kick in and they'll move on to something else. Only if they had a real need or desire will they maybe give it another shot later, as I did. But many of those devs aren't going to pop into IRC and let us know they had issues. They'll just be gone. We won't know of the issues they ran into. Just because the build system works well enough for us doesn't mean it can't be improved upon, and the barrier to entry lowered even further. It kind of feels like arguing against client side form validation when you already have server side validation in place. Sure, you don't need it, and the server side works perfectly well. But if the experience can be improved further using concepts that are already in place elsewhere, where is the harm? It can only benefit us in the long wrong if done correctly.

As I was writing this, I removed the patches from my wx directory. It now compiles correctly somehow. I have upgraded to Xcode 8, not sure if that had anything to do with it or not. This is for the release build. I don't know why there would be issues with asserts from using release libs with a debug wxL build. I can provide prebuilt un-patched debug build probably as well though, if those build too, so that would no longer have to be an issue.

I merely thought that mirroring the official Homebrew wxMac release meant that anyone using wxmac from Homebrew already should expect wxL built on the same code to work just as well as any wxMac app they might build themselves, not that we should support Homebrew in any way. Don't get me wrong on this please, I honestly hate needing homebrew for anything, except the occasional command line utility like htop or something. It seems to cause more issues with projects like this when the wrong libs get found and used.

Guess I forgot the markdown requirement was mentioned in the mac section, my bad. Thought it had only been covered up in general requirements or something.

I don't really expect /tmp/wxmac/{release,debug} to be taken. /tmp is a standard path, and the likelihood something else already exists there should be incredibly low. That said, proper error handling in the implementation could take care of the low chance that it happens. We would simply have to mention that the folder needs to be removed from /tmp. Flow could be like:

if no wxWidgets specified on command line or found via auto detection: if /tmp/wxmac exists: if exists /tmp/wxmac/<release|debug>/IS_FSO_WX_BUILD: use that location else: throw warning/error that this location is in use already (highly unlikely but should be accounted for) else: download and extract FSO WX build to /tmp/wxmac/<release|debug> touch /tmp/wxmac/<release|debug>/IS_FSO_WX_BUILD empty file so that CMake knows this is our lib and not a pre-existing file.

This should allow us to download and extract once, and trust that the lib is safe to use as long as the IS_FSO_WX_BUILD file remains in place.

If I can manage to compile wxLauncher on FreeBSD I can probably reproduce the error you're getting on Linux, if so I'll look into it. IssMneur might have some more thoughts on that too, as I thought he had a Linux development environment too. I wouldn't spend more time on it than you are comfortable doing right now.

Re: wxLauncher Test Builds [Updated: 2016-09-04]

Debug built fine too, so I can build both release and debug wxWidgets 3.0.2 without patches on my Mac now somehow.

Edit: If we were to implement support for the prebuilt libs, I have a proposed update to the Readme already typed up so no one else needs to mess with that. It also covers updating the SDL2 and python recommendation changes. I have left manually compiling wxWidgets as the 'default' method, with only mentioning that the prebuilt lib can be used as an alternative. We could reverse the direction the readme encourages the user to go but I figured you would prefer it stay closer to something like this. I can submit a PR now that only covers the SDL2.framework/python2 changes for now. The prebuilt lib part could just be added later if we get that support added.

Building - OS X (tested on 10.11, El Capitan)=============================================- Get Xcode from the Mac App Store (used 7.3.1)This will include git which will be needed later.- Get Markdown for Python (link is provided above). (used 2.6.6)This will require root privileges to install globally.- Clone the wxLauncher repository.- Get CMake 3 (used 3.6.1).- Get the latest stable version of wxWidgets 3 (used 3.0.2). Once you'vedownloaded and extracted the source tarball, do these things, assuming 3.0.2: * cd wxWidgets-3.0.2/ * Either "mkdir build-debug" or "mkdir build-release" (your choice) * cd <TheBuildFolderYouJustMade> * Type the following to configure, adjusting according to the notes that follow: * ../configure --enable-stl --enable-unicode --enable-debug --disable-shared --with-macosx-version-min=10.9 CC=clang CXX=clang++ CXXFLAGS="-stdlib=libc++ -std=c++11" OBJCXXFLAGS="-stdlib=libc++ -std=c++11" LDFLAGS=-stdlib=libc++ - If you want a release build rather than a debug build, leave out the '--enable-debug' * makeIf you skip this step, a prebuilt lib will be downloaded for you to /tmp/wxmac.- Get SDL2 for Mac: https://www.libsdl.org/download-2.0.php

Extract the SDL2.framework and copy it to your/Users/<YourUsername>/Library/Frameworks folder. Create the Frameworksfolder if it does not exist.- Run CMake either by using cmake at the command line or by using theCMake.app GUI, selecting Xcode as your generator. - A few notes on configuring the CMake variables: * Set wxWidgets_CONFIG_EXECUTABLE to point to the version of wxWidgets you built. wxWidgets_CONFIG_EXECUTABLE is located at /yourWxWidgetsBuildFolder/wx-config * Omit the wxWidgets_CONFIG_EXECUTABLE option to use the prebuilt lib. * cd <wxLauncherSourceFolder> * mkdir build * cd build * /path/to/CMake.app/Contents/bin/cmake -G Xcode -DwxWidgets_CONFIG_EXECUTABLE=/path/to/wxWidgets/Build/Folder/wx-config -DUSE_OPENAL=1 ../- Once you have your Xcode project set up, build the "ALL_BUILD" target tobuild wxlauncher.app in /wxLauncherBuildFolder/SelectedBuildConfig/ , ortype "xcodebuild -configuration <SelectedBuildConfig>" at the terminal inyour wxLauncher build folder. Make sure that the build configuration youchoose (Debug or Release) matches the build configuration you used whenyou built wxWidgets (if you built it yourself).

Important known issues on OS X:- After startup or after a FS2 Open binary is (re-)selected, checkboxes onthe advanced settings page may not appear until after a moment or afterthe user interacts with the advanced settings page, such as by clicking onthe flag list.