Hi! What is the best way to integrate sjavac with Gradle?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Hi! What is the best way to integrate sjavac with Gradle?

Fredrik Öhrström
As part of the OpenJDK build project I wrote sjavac, which is a wrapper around javac. It creates a database file that stores all the dependency information necessary to do proper incremental compiles of Java from the command line.

I.e. it knows which packages depend on which packages both for your sources and your references to classes and jars. It will only recompile your source, when the public api of the dependencies change (true both for sources and classes).
You can read more about it here, and find precompiled binaries and sources (http://fredrikohrstrom.blogspot.se/).

sjavac is definitely not a build system, therefore it makes sense to integrate it with make, ant and of course Gradle. Please give sjavac a spin and if you like it, then I have a couple of brainstorming questions on how to integrate it with Gradle. :-)

Can you give me a hint of which layer in Gradle you think is suitable for adjusting to sjavac? Is there something similar to the CompilerAdapter in ant?

Ant performs its own incremental analysis before sending a subset of the files to the Compiler, I would like to turn this off, but this requires changes to ant, or to build completely new Compiler tool, but then you have to change your xml build files significantly. My workaround in ant is to remember the list of sources from the initial compile, and always add those to the subsequent incremental compiles. Not perfect, but handles most use cases except where you want to reduce the set of the sources through filtering rules, then you have to clean and recompile. I am really striving for 100% incremental correctness, so this is not really acceptable.

Does Gradle track dependencies to jars/classes? You probably want Gradle to trigger a compilation, not only when a source file change, but when the timestamp of the dependent jar/class change. sjavac can then figure out if the public api of the jars/classes changed and do a recompilation if necessary. You can use the output from sjavac to gather such dependencies, if you like.

So I suppose the choice to try sjavac in gradle should be a simple command line switch, or simple addition to the build file. This switch would make sure that when a dependency for a module (source/class/jar) indicates that the module might need recompilation Gradle then sends all sources for that module to sjavac (no incremental filtering) . Gradle can then use the output from sjavac to detect if anything actually was recompiled, thus triggering further modules recompiles.

Also ant is interesting since the selecting rules for sources can be arbitrary complex, and often are. Even though the common use case is all sources below a root. sjavac handles this complexity with an @file that lists explicitly all sources "-if Source1.java -if Source2.java". I do not know how Gradle selects sources for compilation, if Gradle allows similarily complex selection rules, we will have to feed sjavac explicitly, and not just the roots.

The sjavac preferred solution is to have multiple source roots and copy files from other roots into the bin directory, sjavac (-copy) will do this incrementally. But you can of course take the bin output from sjavac and copy it to the real bin target, if you like, but then you are responsible for incrementally deleting classes that disappear.

I suppose it would be nice to have a programming API to sjavac as well, instead of forking an external process, or calling the go method with command line style strings. Do you have any preferences? sjavac is in development at this very moment. :-)

//Fredrik
Reply | Threaded
Open this post in threaded view
|

Re: Hi! What is the best way to integrate sjavac with Gradle?

Luke Daley-2
Hi Fredrik,

The soon to be released Gradle 2.1 features our very own incremental compilation implementation for Java. You can read about it in the release notes available for the release candidate: http://www.gradle.org/docs/release-candidate/release-notes#incremental-java-compilation

This isn’t based on Ant, and uses sophisticated byte code and source analysis instead of timestamps for detecting changes and invalidation. If you’d be interested in merging your work (if that makes sense), or helping to advance this implementation that would be very welcome.

  

On 29 August 2014 at 7:31:46 am, Fredrik Öhrström ([hidden email]) wrote:

As part of the OpenJDK build project I wrote sjavac, which is a wrapper around javac. It creates a database file that stores all the dependency information necessary to do proper incremental compiles of Java from the command line.

I.e. it knows which packages depend on which packages both for your sources and your references to classes and jars. It will only recompile your source, when the public api of the dependencies change (true both for sources and classes).
You can read more about it here, and find precompiled binaries and sources (http://fredrikohrstrom.blogspot.se/).

sjavac is definitely not a build system, therefore it makes sense to integrate it with make, ant and of course Gradle. Please give sjavac a spin and if you like it, then I have a couple of brainstorming questions on how to integrate it with Gradle. :-)

Can you give me a hint of which layer in Gradle you think is suitable for adjusting to sjavac? Is there something similar to the CompilerAdapter in ant?

Ant performs its own incremental analysis before sending a subset of the files to the Compiler, I would like to turn this off, but this requires changes to ant, or to build completely new Compiler tool, but then you have to change your xml build files significantly. My workaround in ant is to remember the list of sources from the initial compile, and always add those to the subsequent incremental compiles. Not perfect, but handles most use cases except where you want to reduce the set of the sources through filtering rules, then you have to clean and recompile. I am really striving for 100% incremental correctness, so this is not really acceptable.

Does Gradle track dependencies to jars/classes? You probably want Gradle to trigger a compilation, not only when a source file change, but when the timestamp of the dependent jar/class change. sjavac can then figure out if the public api of the jars/classes changed and do a recompilation if necessary. You can use the output from sjavac to gather such dependencies, if you like.

So I suppose the choice to try sjavac in gradle should be a simple command line switch, or simple addition to the build file. This switch would make sure that when a dependency for a module (source/class/jar) indicates that the module might need recompilation Gradle then sends all sources for that module to sjavac (no incremental filtering) . Gradle can then use the output from sjavac to detect if anything actually was recompiled, thus triggering further modules recompiles.

Also ant is interesting since the selecting rules for sources can be arbitrary complex, and often are. Even though the common use case is all sources below a root. sjavac handles this complexity with an @file that lists explicitly all sources "-if Source1.java -if Source2.java". I do not know how Gradle selects sources for compilation, if Gradle allows similarily complex selection rules, we will have to feed sjavac explicitly, and not just the roots.

The sjavac preferred solution is to have multiple source roots and copy files from other roots into the bin directory, sjavac (-copy) will do this incrementally. But you can of course take the bin output from sjavac and copy it to the real bin target, if you like, but then you are responsible for incrementally deleting classes that disappear.

I suppose it would be nice to have a programming API to sjavac as well, instead of forking an external process, or calling the go method with command line style strings. Do you have any preferences? sjavac is in development at this very moment. :-)

//Fredrik