I've started to get this prompt come up every time I opened IntelliJ: Do you want the application 'IntelliJ IDEA 14 CE.app' to accept incoming network connections?. This happened after doing an automatic update within IntelliJ itself. After a bit of messing around, I was able to come up with a solution to make this prompt disappear.

I checked the Security & Privacy settings, and IntelliJ was listed as allowed to accept incoming network connections there.

This is what I ended up doing. Open the Terminal.app and type in the text below. Make sure that IntelliJ is not running while you do this.