Gradle has an option to choose the number of thread (--parallel-thread). I'm guessing that this is used when building different projects in parallel.
In the long term, we're going to have 3 types of parallelizations:
- project level
- task level inside a single project
- the task itself (I'm already doing this in a few task in the Android plugin).
Right now each of my task creates a new thread pool based on the number of cores available which is not great on its own and definitively doesn't work well when building projects in parallel.
Have you thought about having a global thread pool used by all three types? This would allow us to both avoid creating new threads in every tasks but also control better the number of threads running concurrently.
Yes, absolutely. We also have the problem with parallel test execution and —parallel, and will run into it when we introduce parallel compilation for the native languages.
The idea is to have an API that tasks can use to spawn pieces of work that can be run concurrently, and Gradle can take care of assigning work to threads. Possibly this might just be something simple like a java.util.concurrent.Executor subtype, or we might make it more structured so that each piece of work that a task spawns can also have its own dependencies, inputs and outputs, incremental state and so on, as a kind of light-weight tasks.
--
Adam Murdoch
Gradle Co-founder
http://www.gradle.orgVP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com
Locked
