Known issues with --parallel?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Known issues with --parallel?

Xavier Ducrohet
Hello,

I have a project with 70ish projects and I'm trying to run with --parallel and I keep getting errors (this is with 1.10). A lot of those projects are android projects, there's also a lot of custom logic in some of the projects (either regular java projects or on top of the android plugin).

Every time I get an error it's also during a compileJava task, either in an Android project or in a Java project.

The weird part is that the error never makes sense. For example we have the source of the jsr305 lib in our tree (we're using it from source) and I'll get errors like:

/path/to/src/main/java/javax/annotation/CheckForSigned.java:1: class, interface, or enum expected
package javax.annotation;
^

Obviously this file is correct, and compiles fine without parallelization. All breakages I've seen are always the same and always in the Java compile task on code that is not broken.

Is this is a known issue? Are there specifics to projects configs and inter-projects dependencies that could trigger this?

I have another projects that is about 20ish java-only projects and I don't have any issue building it in parallel but it has basically no custom logic to build it.

thanks
Xavier
Reply | Threaded
Open this post in threaded view
|

Re: Known issues with --parallel?

Luke Daley-2
This hasn't been reported before.

There are a few different ways that Gradle can manage the Java compiler.
Are there any custom settings on the JavaCompile task? like using a
different javac executable? setting useAnt = true?

> Xavier Ducrohet <mailto:[hidden email]>
> 12 February 2014 11:01 am
> Hello,
>
> I have a project with 70ish projects and I'm trying to run with
> --parallel and I keep getting errors (this is with 1.10). A lot of
> those projects are android projects, there's also a lot of custom
> logic in some of the projects (either regular java projects or on top
> of the android plugin).
>
> Every time I get an error it's also during a compileJava task, either
> in an Android project or in a Java project.
>
> The weird part is that the error never makes sense. For example we
> have the source of the jsr305 lib in our tree (we're using it from
> source) and I'll get errors like:
>
> /path/to/src/main/java/javax/annotation/CheckForSigned.java:1: class,
> interface, or enum expected
> package javax.annotation;
> ^
>
> Obviously this file is correct, and compiles fine without
> parallelization. All breakages I've seen are always the same and
> always in the Java compile task on code that is not broken.
>
> Is this is a known issue? Are there specifics to projects configs and
> inter-projects dependencies that could trigger this?
>
> I have another projects that is about 20ish java-only projects and I
> don't have any issue building it in parallel but it has basically no
> custom logic to build it.
>
> thanks
> Xavier

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Known issues with --parallel?

Xavier Ducrohet
No. In case of our jsr305 project, the whole build.gradle is:

apply plugin: 'java'
sourceSets {
    main {
        java {
            srcDirs = ['ri/src/main/java']
        }
    }
}

Other projects where I've seen issues are similarly simple. We never configure anything special like this. Most of the custom logic I mentioned about is mostly in the android projects (adding custom tasks, before or after the Java compile, customizing packaging etc...)



On Tue, Feb 11, 2014 at 5:29 PM, Luke Daley <[hidden email]> wrote:
This hasn't been reported before.

There are a few different ways that Gradle can manage the Java compiler. Are there any custom settings on the JavaCompile task? like using a different javac executable? setting useAnt = true?

Xavier Ducrohet <mailto:[hidden email]>
12 February 2014 11:01 am

Hello,

I have a project with 70ish projects and I'm trying to run with --parallel and I keep getting errors (this is with 1.10). A lot of those projects are android projects, there's also a lot of custom logic in some of the projects (either regular java projects or on top of the android plugin).

Every time I get an error it's also during a compileJava task, either in an Android project or in a Java project.

The weird part is that the error never makes sense. For example we have the source of the jsr305 lib in our tree (we're using it from source) and I'll get errors like:

/path/to/src/main/java/javax/annotation/CheckForSigned.java:1: class, interface, or enum expected
package javax.annotation;
^

Obviously this file is correct, and compiles fine without parallelization. All breakages I've seen are always the same and always in the Java compile task on code that is not broken.

Is this is a known issue? Are there specifics to projects configs and inter-projects dependencies that could trigger this?

I have another projects that is about 20ish java-only projects and I don't have any issue building it in parallel but it has basically no custom logic to build it.

thanks
Xavier

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|

Re: Known issues with --parallel?

Szczepan Faber-2
Hey Xav,

I haven't heard about this specific problem before. We have clients that use --parallel with very big multi-project builds with tons of custom code. Here're some things to look for:

- are you running a "clean build"? In past I've seen problems with --parallel compilation (although different than yours) related to the custom layout of the build directories. Basically, the individual build dirs were configured to live in root project's build dir. This caused trouble in parallel build when root projects 'clean' task was clashing on the 'compile' task of one of the subproject, who already completed his individual 'clean' task. This custom layout of build dirs is supported but requires laying out another task relationship rule (mustRunAfter).

- do you configure the compile task in any way? If not, one thing worth trying is using the compiler daemon: http://www.gradle.org/docs/current/dsl/org.gradle.api.tasks.compile.JavaCompile.html If toggling the compiler daemon makes a difference in the stability of the build we can dig further. You might also try Luke's suggestion

- more generic suggestion: can you imagine anything in the build that could possibly clash with compile tasks?, e.g. sharing some directories between projects, configuration tasks that alter other tasks' during execution time, some execution-time project couplings (e.g. one project reaching out to other project directly, instead of using a project dependency)?

Hope that helps!


On Wed, Feb 12, 2014 at 3:25 AM, Xavier Ducrohet <[hidden email]> wrote:
No. In case of our jsr305 project, the whole build.gradle is:

apply plugin: 'java'
sourceSets {
    main {
        java {
            srcDirs = ['ri/src/main/java']
        }
    }
}

Other projects where I've seen issues are similarly simple. We never configure anything special like this. Most of the custom logic I mentioned about is mostly in the android projects (adding custom tasks, before or after the Java compile, customizing packaging etc...)



On Tue, Feb 11, 2014 at 5:29 PM, Luke Daley <[hidden email]> wrote:
This hasn't been reported before.

There are a few different ways that Gradle can manage the Java compiler. Are there any custom settings on the JavaCompile task? like using a different javac executable? setting useAnt = true?

Xavier Ducrohet <mailto:[hidden email]>
12 February 2014 11:01 am

Hello,

I have a project with 70ish projects and I'm trying to run with --parallel and I keep getting errors (this is with 1.10). A lot of those projects are android projects, there's also a lot of custom logic in some of the projects (either regular java projects or on top of the android plugin).

Every time I get an error it's also during a compileJava task, either in an Android project or in a Java project.

The weird part is that the error never makes sense. For example we have the source of the jsr305 lib in our tree (we're using it from source) and I'll get errors like:

/path/to/src/main/java/javax/annotation/CheckForSigned.java:1: class, interface, or enum expected
package javax.annotation;
^

Obviously this file is correct, and compiles fine without parallelization. All breakages I've seen are always the same and always in the Java compile task on code that is not broken.

Is this is a known issue? Are there specifics to projects configs and inter-projects dependencies that could trigger this?

I have another projects that is about 20ish java-only projects and I don't have any issue building it in parallel but it has basically no custom logic to build it.

thanks
Xavier

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email






--
Szczepan Faber
Principal engineer@gradle; Founder@mockito
Reply | Threaded
Open this post in threaded view
|

Re: Known issues with --parallel?

Xavier Ducrohet
It seems like our custom build directories config could be the culprit. I'm running into another build breakage that's unrelated but I'll confirm as soon as I can verify the build. 

It's a bit surprising because I was building from a clean slate (without calling the clean task).

thanks!


On Tue, Feb 11, 2014 at 11:25 PM, Szczepan Faber <[hidden email]> wrote:
Hey Xav,

I haven't heard about this specific problem before. We have clients that use --parallel with very big multi-project builds with tons of custom code. Here're some things to look for:

- are you running a "clean build"? In past I've seen problems with --parallel compilation (although different than yours) related to the custom layout of the build directories. Basically, the individual build dirs were configured to live in root project's build dir. This caused trouble in parallel build when root projects 'clean' task was clashing on the 'compile' task of one of the subproject, who already completed his individual 'clean' task. This custom layout of build dirs is supported but requires laying out another task relationship rule (mustRunAfter).

- do you configure the compile task in any way? If not, one thing worth trying is using the compiler daemon: http://www.gradle.org/docs/current/dsl/org.gradle.api.tasks.compile.JavaCompile.html If toggling the compiler daemon makes a difference in the stability of the build we can dig further. You might also try Luke's suggestion

- more generic suggestion: can you imagine anything in the build that could possibly clash with compile tasks?, e.g. sharing some directories between projects, configuration tasks that alter other tasks' during execution time, some execution-time project couplings (e.g. one project reaching out to other project directly, instead of using a project dependency)?

Hope that helps!


On Wed, Feb 12, 2014 at 3:25 AM, Xavier Ducrohet <[hidden email]> wrote:
No. In case of our jsr305 project, the whole build.gradle is:

apply plugin: 'java'
sourceSets {
    main {
        java {
            srcDirs = ['ri/src/main/java']
        }
    }
}

Other projects where I've seen issues are similarly simple. We never configure anything special like this. Most of the custom logic I mentioned about is mostly in the android projects (adding custom tasks, before or after the Java compile, customizing packaging etc...)



On Tue, Feb 11, 2014 at 5:29 PM, Luke Daley <[hidden email]> wrote:
This hasn't been reported before.

There are a few different ways that Gradle can manage the Java compiler. Are there any custom settings on the JavaCompile task? like using a different javac executable? setting useAnt = true?

Xavier Ducrohet <mailto:[hidden email]>
12 February 2014 11:01 am

Hello,

I have a project with 70ish projects and I'm trying to run with --parallel and I keep getting errors (this is with 1.10). A lot of those projects are android projects, there's also a lot of custom logic in some of the projects (either regular java projects or on top of the android plugin).

Every time I get an error it's also during a compileJava task, either in an Android project or in a Java project.

The weird part is that the error never makes sense. For example we have the source of the jsr305 lib in our tree (we're using it from source) and I'll get errors like:

/path/to/src/main/java/javax/annotation/CheckForSigned.java:1: class, interface, or enum expected
package javax.annotation;
^

Obviously this file is correct, and compiles fine without parallelization. All breakages I've seen are always the same and always in the Java compile task on code that is not broken.

Is this is a known issue? Are there specifics to projects configs and inter-projects dependencies that could trigger this?

I have another projects that is about 20ish java-only projects and I don't have any issue building it in parallel but it has basically no custom logic to build it.

thanks
Xavier

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email






--
Szczepan Faber
Principal engineer@gradle; Founder@mockito

Reply | Threaded
Open this post in threaded view
|

Re: Known issues with --parallel?

Luke Daley-2
Is this just a custom $buildDir location?

> Xavier Ducrohet <mailto:[hidden email]>
> 13 February 2014 5:03 am
> It seems like our custom build directories config could be the
> culprit. I'm running into another build breakage that's unrelated but
> I'll confirm as soon as I can verify the build.
>
> It's a bit surprising because I was building from a clean slate
> (without calling the clean task).
>
> thanks!
>
>
>
> Szczepan Faber <mailto:[hidden email]>
> 12 February 2014 5:25 pm
> Hey Xav,
>
> I haven't heard about this specific problem before. We have clients
> that use --parallel with very big multi-project builds with tons of
> custom code. Here're some things to look for:
>
> - are you running a "clean build"? In past I've seen problems with
> --parallel compilation (although different than yours) related to the
> custom layout of the build directories. Basically, the individual
> build dirs were configured to live in root project's build dir. This
> caused trouble in parallel build when root projects 'clean' task was
> clashing on the 'compile' task of one of the subproject, who already
> completed his individual 'clean' task. This custom layout of build
> dirs is supported but requires laying out another task relationship
> rule (mustRunAfter).
>
> - do you configure the compile task in any way? If not, one thing
> worth trying is using the compiler daemon:
> http://www.gradle.org/docs/current/dsl/org.gradle.api.tasks.compile.JavaCompile.html 
> If toggling the compiler daemon makes a difference in the stability of
> the build we can dig further. You might also try Luke's suggestion
>
> - more generic suggestion: can you imagine anything in the build that
> could possibly clash with compile tasks?, e.g. sharing some
> directories between projects, configuration tasks that alter other
> tasks' during execution time, some execution-time project couplings
> (e.g. one project reaching out to other project directly, instead of
> using a project dependency)?
>
> Hope that helps!
>
>
>
>
>
> --
> Szczepan Faber
> Principal engineer@gradle; Founder@mockito
> Xavier Ducrohet <mailto:[hidden email]>
> 12 February 2014 12:25 pm
> No. In case of our jsr305 project, the whole build.gradle is:
>
> apply plugin: 'java'
> sourceSets {
>     main {
>         java {
>             srcDirs = ['ri/src/main/java']
>         }
>     }
> }
>
> Other projects where I've seen issues are similarly simple. We never
> configure anything special like this. Most of the custom logic I
> mentioned about is mostly in the android projects (adding custom
> tasks, before or after the Java compile, customizing packaging etc...)
>
>
>
>
> Luke Daley <mailto:[hidden email]>
> 12 February 2014 11:29 am
> This hasn't been reported before.
>
> There are a few different ways that Gradle can manage the Java
> compiler. Are there any custom settings on the JavaCompile task? like
> using a different javac executable? setting useAnt = true?
>
> Xavier Ducrohet <mailto:[hidden email]>
> 12 February 2014 11:01 am
> Hello,
>
> I have a project with 70ish projects and I'm trying to run with
> --parallel and I keep getting errors (this is with 1.10). A lot of
> those projects are android projects, there's also a lot of custom
> logic in some of the projects (either regular java projects or on top
> of the android plugin).
>
> Every time I get an error it's also during a compileJava task, either
> in an Android project or in a Java project.
>
> The weird part is that the error never makes sense. For example we
> have the source of the jsr305 lib in our tree (we're using it from
> source) and I'll get errors like:
>
> /path/to/src/main/java/javax/annotation/CheckForSigned.java:1: class,
> interface, or enum expected
> package javax.annotation;
> ^
>
> Obviously this file is correct, and compiles fine without
> parallelization. All breakages I've seen are always the same and
> always in the Java compile task on code that is not broken.
>
> Is this is a known issue? Are there specifics to projects configs and
> inter-projects dependencies that could trigger this?
>
> I have another projects that is about 20ish java-only projects and I
> don't have any issue building it in parallel but it has basically no
> custom logic to build it.
>
> thanks
> Xavier

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: Known issues with --parallel?

Xavier Ducrohet
yes it is. I've been sidetracked and haven't been able to confirm that this really fixes the issue (build it broken with something else outside of my direct control).


On Wed, Feb 12, 2014 at 5:15 PM, Luke Daley <[hidden email]> wrote:
Is this just a custom $buildDir location?

Xavier Ducrohet <mailto:[hidden email]>
13 February 2014 5:03 am

It seems like our custom build directories config could be the culprit. I'm running into another build breakage that's unrelated but I'll confirm as soon as I can verify the build.

It's a bit surprising because I was building from a clean slate (without calling the clean task).

thanks!



Szczepan Faber <mailto:[hidden email]>
12 February 2014 5:25 pm

Hey Xav,

I haven't heard about this specific problem before. We have clients that use --parallel with very big multi-project builds with tons of custom code. Here're some things to look for:

- are you running a "clean build"? In past I've seen problems with --parallel compilation (although different than yours) related to the custom layout of the build directories. Basically, the individual build dirs were configured to live in root project's build dir. This caused trouble in parallel build when root projects 'clean' task was clashing on the 'compile' task of one of the subproject, who already completed his individual 'clean' task. This custom layout of build dirs is supported but requires laying out another task relationship rule (mustRunAfter).

- do you configure the compile task in any way? If not, one thing worth trying is using the compiler daemon: http://www.gradle.org/docs/current/dsl/org.gradle.api.tasks.compile.JavaCompile.html If toggling the compiler daemon makes a difference in the stability of the build we can dig further. You might also try Luke's suggestion

- more generic suggestion: can you imagine anything in the build that could possibly clash with compile tasks?, e.g. sharing some directories between projects, configuration tasks that alter other tasks' during execution time, some execution-time project couplings (e.g. one project reaching out to other project directly, instead of using a project dependency)?

Hope that helps!





--
Szczepan Faber
Principal engineer@gradle; Founder@mockito
Xavier Ducrohet <mailto:[hidden email]>
12 February 2014 12:25 pm

No. In case of our jsr305 project, the whole build.gradle is:

apply plugin: 'java'
sourceSets {
    main {
        java {
            srcDirs = ['ri/src/main/java']
        }
    }
}

Other projects where I've seen issues are similarly simple. We never configure anything special like this. Most of the custom logic I mentioned about is mostly in the android projects (adding custom tasks, before or after the Java compile, customizing packaging etc...)




Luke Daley <mailto:[hidden email]>
12 February 2014 11:29 am

This hasn't been reported before.

There are a few different ways that Gradle can manage the Java compiler. Are there any custom settings on the JavaCompile task? like using a different javac executable? setting useAnt = true?

Xavier Ducrohet <mailto:[hidden email]>
12 February 2014 11:01 am
Hello,

I have a project with 70ish projects and I'm trying to run with --parallel and I keep getting errors (this is with 1.10). A lot of those projects are android projects, there's also a lot of custom logic in some of the projects (either regular java projects or on top of the android plugin).

Every time I get an error it's also during a compileJava task, either in an Android project or in a Java project.

The weird part is that the error never makes sense. For example we have the source of the jsr305 lib in our tree (we're using it from source) and I'll get errors like:

/path/to/src/main/java/javax/annotation/CheckForSigned.java:1: class, interface, or enum expected
package javax.annotation;
^

Obviously this file is correct, and compiles fine without parallelization. All breakages I've seen are always the same and always in the Java compile task on code that is not broken.

Is this is a known issue? Are there specifics to projects configs and inter-projects dependencies that could trigger this?

I have another projects that is about 20ish java-only projects and I don't have any issue building it in parallel but it has basically no custom logic to build it.

thanks
Xavier

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|

Re: Known issues with --parallel?

Xavier Ducrohet
In reply to this post by Szczepan Faber-2
OK I finally managed to look into this. Comments below.


On Tue, Feb 11, 2014 at 11:25 PM, Szczepan Faber <[hidden email]> wrote:
Hey Xav,

I haven't heard about this specific problem before. We have clients that use --parallel with very big multi-project builds with tons of custom code. Here're some things to look for:

- are you running a "clean build"? In past I've seen problems with --parallel compilation (although different than yours) related to the custom layout of the build directories. Basically, the individual build dirs were configured to live in root project's build dir. This caused trouble in parallel build when root projects 'clean' task was clashing on the 'compile' task of one of the subproject, who already completed his individual 'clean' task. This custom layout of build dirs is supported but requires laying out another task relationship rule (mustRunAfter).


We did have issue with build folders. I fixed those and now I'm still seeing a problem but later down the line in another projects where the java compile fails again. This time though it's an Android project (continuing below)...
 
- do you configure the compile task in any way? If not, one thing worth trying is using the compiler daemon: http://www.gradle.org/docs/current/dsl/org.gradle.api.tasks.compile.JavaCompile.html If toggling the compiler daemon makes a difference in the stability of the build we can dig further. You might also try Luke's suggestion


Yes. The previous projects that failed where just java projects with no special config. This time though it's an Android project and we custom create/configure the Java compile tasks.

It's line 792.
 
- more generic suggestion: can you imagine anything in the build that could possibly clash with compile tasks?, e.g. sharing some directories between projects, configuration tasks that alter other tasks' during execution 
time, some execution-time project couplings (e.g. one project reaching out to other project directly, instead of using a project dependency)?


There really shouldn't but we have some custom logic in there's that's grown organically and was done by someone not super familiar with Gradle. I need to clean it up. sigh.

I mostly wanted to know if there were some known issue. It seems like there's probably something wrong in our setup. I'll have to dig in further.
 
Hope that helps!


On Wed, Feb 12, 2014 at 3:25 AM, Xavier Ducrohet <[hidden email]> wrote:
No. In case of our jsr305 project, the whole build.gradle is:

apply plugin: 'java'
sourceSets {
    main {
        java {
            srcDirs = ['ri/src/main/java']
        }
    }
}

Other projects where I've seen issues are similarly simple. We never configure anything special like this. Most of the custom logic I mentioned about is mostly in the android projects (adding custom tasks, before or after the Java compile, customizing packaging etc...)



On Tue, Feb 11, 2014 at 5:29 PM, Luke Daley <[hidden email]> wrote:
This hasn't been reported before.

There are a few different ways that Gradle can manage the Java compiler. Are there any custom settings on the JavaCompile task? like using a different javac executable? setting useAnt = true?

Xavier Ducrohet <mailto:[hidden email]>
12 February 2014 11:01 am

Hello,

I have a project with 70ish projects and I'm trying to run with --parallel and I keep getting errors (this is with 1.10). A lot of those projects are android projects, there's also a lot of custom logic in some of the projects (either regular java projects or on top of the android plugin).

Every time I get an error it's also during a compileJava task, either in an Android project or in a Java project.

The weird part is that the error never makes sense. For example we have the source of the jsr305 lib in our tree (we're using it from source) and I'll get errors like:

/path/to/src/main/java/javax/annotation/CheckForSigned.java:1: class, interface, or enum expected
package javax.annotation;
^

Obviously this file is correct, and compiles fine without parallelization. All breakages I've seen are always the same and always in the Java compile task on code that is not broken.

Is this is a known issue? Are there specifics to projects configs and inter-projects dependencies that could trigger this?

I have another projects that is about 20ish java-only projects and I don't have any issue building it in parallel but it has basically no custom logic to build it.

thanks
Xavier

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email






--
Szczepan Faber
Principal engineer@gradle; Founder@mockito

Reply | Threaded
Open this post in threaded view
|

Re: Known issues with --parallel?

Szczepan Faber-2
On Fri, Feb 14, 2014 at 1:10 AM, Xavier Ducrohet <[hidden email]> wrote:
OK I finally managed to look into this. Comments below.


On Tue, Feb 11, 2014 at 11:25 PM, Szczepan Faber <[hidden email]> wrote:
Hey Xav,

I haven't heard about this specific problem before. We have clients that use --parallel with very big multi-project builds with tons of custom code. Here're some things to look for:

- are you running a "clean build"? In past I've seen problems with --parallel compilation (although different than yours) related to the custom layout of the build directories. Basically, the individual build dirs were configured to live in root project's build dir. This caused trouble in parallel build when root projects 'clean' task was clashing on the 'compile' task of one of the subproject, who already completed his individual 'clean' task. This custom layout of build dirs is supported but requires laying out another task relationship rule (mustRunAfter).


We did have issue with build folders. I fixed those and now I'm still seeing a problem but later down the line in another projects where the java compile fails again. This time though it's an Android project (continuing below)...
 
- do you configure the compile task in any way? If not, one thing worth trying is using the compiler daemon: http://www.gradle.org/docs/current/dsl/org.gradle.api.tasks.compile.JavaCompile.html If toggling the compiler daemon makes a difference in the stability of the build we can dig further. You might also try Luke's suggestion


Yes. The previous projects that failed where just java projects with no special config. This time though it's an Android project and we custom create/configure the Java compile tasks.

It's line 792.

After a quick look it's hard to say what might be wrong. First thing that I would look for is missing task dependencies: if the task depends on some other task's output, are the task dependencies configured correctly? (for example, line 804+). Parallel build often flushes problems with missing task relationships.

Hope that helps!

- more generic suggestion: can you imagine anything in the build that could possibly clash with compile tasks?, e.g. sharing some directories between projects, configuration tasks that alter other tasks' during execution 
time, some execution-time project couplings (e.g. one project reaching out to other project directly, instead of using a project dependency)?


There really shouldn't but we have some custom logic in there's that's grown organically and was done by someone not super familiar with Gradle. I need to clean it up. sigh.

I mostly wanted to know if there were some known issue. It seems like there's probably something wrong in our setup. I'll have to dig in further.
 
Hope that helps!


On Wed, Feb 12, 2014 at 3:25 AM, Xavier Ducrohet <[hidden email]> wrote:
No. In case of our jsr305 project, the whole build.gradle is:

apply plugin: 'java'
sourceSets {
    main {
        java {
            srcDirs = ['ri/src/main/java']
        }
    }
}

Other projects where I've seen issues are similarly simple. We never configure anything special like this. Most of the custom logic I mentioned about is mostly in the android projects (adding custom tasks, before or after the Java compile, customizing packaging etc...)



On Tue, Feb 11, 2014 at 5:29 PM, Luke Daley <[hidden email]> wrote:
This hasn't been reported before.

There are a few different ways that Gradle can manage the Java compiler. Are there any custom settings on the JavaCompile task? like using a different javac executable? setting useAnt = true?

Xavier Ducrohet <mailto:[hidden email]>
12 February 2014 11:01 am

Hello,

I have a project with 70ish projects and I'm trying to run with --parallel and I keep getting errors (this is with 1.10). A lot of those projects are android projects, there's also a lot of custom logic in some of the projects (either regular java projects or on top of the android plugin).

Every time I get an error it's also during a compileJava task, either in an Android project or in a Java project.

The weird part is that the error never makes sense. For example we have the source of the jsr305 lib in our tree (we're using it from source) and I'll get errors like:

/path/to/src/main/java/javax/annotation/CheckForSigned.java:1: class, interface, or enum expected
package javax.annotation;
^

Obviously this file is correct, and compiles fine without parallelization. All breakages I've seen are always the same and always in the Java compile task on code that is not broken.

Is this is a known issue? Are there specifics to projects configs and inter-projects dependencies that could trigger this?

I have another projects that is about 20ish java-only projects and I don't have any issue building it in parallel but it has basically no custom logic to build it.

thanks
Xavier

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email






--
Szczepan Faber
Principal engineer@gradle; Founder@mockito




--
Szczepan Faber
Principal engineer@gradle; Founder@mockito
Reply | Threaded
Open this post in threaded view
|

Re: Known issues with --parallel?

Xavier Ducrohet
I think dependencies are probably the reason indeed. Not for the basic android plugin but from all the custom logic we have on top of it for this particular project. I'm going to take a deeper look. thanks.


On Fri, Feb 14, 2014 at 1:14 AM, Szczepan Faber <[hidden email]> wrote:
On Fri, Feb 14, 2014 at 1:10 AM, Xavier Ducrohet <[hidden email]> wrote:
OK I finally managed to look into this. Comments below.


On Tue, Feb 11, 2014 at 11:25 PM, Szczepan Faber <[hidden email]> wrote:
Hey Xav,

I haven't heard about this specific problem before. We have clients that use --parallel with very big multi-project builds with tons of custom code. Here're some things to look for:

- are you running a "clean build"? In past I've seen problems with --parallel compilation (although different than yours) related to the custom layout of the build directories. Basically, the individual build dirs were configured to live in root project's build dir. This caused trouble in parallel build when root projects 'clean' task was clashing on the 'compile' task of one of the subproject, who already completed his individual 'clean' task. This custom layout of build dirs is supported but requires laying out another task relationship rule (mustRunAfter).


We did have issue with build folders. I fixed those and now I'm still seeing a problem but later down the line in another projects where the java compile fails again. This time though it's an Android project (continuing below)...
 
- do you configure the compile task in any way? If not, one thing worth trying is using the compiler daemon: http://www.gradle.org/docs/current/dsl/org.gradle.api.tasks.compile.JavaCompile.html If toggling the compiler daemon makes a difference in the stability of the build we can dig further. You might also try Luke's suggestion


Yes. The previous projects that failed where just java projects with no special config. This time though it's an Android project and we custom create/configure the Java compile tasks.

It's line 792.

After a quick look it's hard to say what might be wrong. First thing that I would look for is missing task dependencies: if the task depends on some other task's output, are the task dependencies configured correctly? (for example, line 804+). Parallel build often flushes problems with missing task relationships.

Hope that helps!

- more generic suggestion: can you imagine anything in the build that could possibly clash with compile tasks?, e.g. sharing some directories between projects, configuration tasks that alter other tasks' during execution 
time, some execution-time project couplings (e.g. one project reaching out to other project directly, instead of using a project dependency)?


There really shouldn't but we have some custom logic in there's that's grown organically and was done by someone not super familiar with Gradle. I need to clean it up. sigh.

I mostly wanted to know if there were some known issue. It seems like there's probably something wrong in our setup. I'll have to dig in further.
 
Hope that helps!


On Wed, Feb 12, 2014 at 3:25 AM, Xavier Ducrohet <[hidden email]> wrote:
No. In case of our jsr305 project, the whole build.gradle is:

apply plugin: 'java'
sourceSets {
    main {
        java {
            srcDirs = ['ri/src/main/java']
        }
    }
}

Other projects where I've seen issues are similarly simple. We never configure anything special like this. Most of the custom logic I mentioned about is mostly in the android projects (adding custom tasks, before or after the Java compile, customizing packaging etc...)



On Tue, Feb 11, 2014 at 5:29 PM, Luke Daley <[hidden email]> wrote:
This hasn't been reported before.

There are a few different ways that Gradle can manage the Java compiler. Are there any custom settings on the JavaCompile task? like using a different javac executable? setting useAnt = true?

Xavier Ducrohet <mailto:[hidden email]>
12 February 2014 11:01 am

Hello,

I have a project with 70ish projects and I'm trying to run with --parallel and I keep getting errors (this is with 1.10). A lot of those projects are android projects, there's also a lot of custom logic in some of the projects (either regular java projects or on top of the android plugin).

Every time I get an error it's also during a compileJava task, either in an Android project or in a Java project.

The weird part is that the error never makes sense. For example we have the source of the jsr305 lib in our tree (we're using it from source) and I'll get errors like:

/path/to/src/main/java/javax/annotation/CheckForSigned.java:1: class, interface, or enum expected
package javax.annotation;
^

Obviously this file is correct, and compiles fine without parallelization. All breakages I've seen are always the same and always in the Java compile task on code that is not broken.

Is this is a known issue? Are there specifics to projects configs and inter-projects dependencies that could trigger this?

I have another projects that is about 20ish java-only projects and I don't have any issue building it in parallel but it has basically no custom logic to build it.

thanks
Xavier

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email






--
Szczepan Faber
Principal engineer@gradle; Founder@mockito




--
Szczepan Faber
Principal engineer@gradle; Founder@mockito