Is it possible to elect a local gradle project to satisfy other projects deps?

[davide.cavestro]

Hi guys,
I'd like to know if there is a way to somewhat elect a local gradle project to be considered for satisfying other projects dependencies.
This way we could - especially on early developments - avoid packaging and uploading snapshot artifacts over and over on our private maven repo only to see how they behave on a project we have in the same workspace. This scenario applies even for testing a high invasive refactoring.
It would be something similar to inline plugins for grails ( http://johnrellis.blogspot.com/2010/03/grails-inplace-plugin-tip.html ).
PS: I asked the same thing to STS/gradle integration developers (they really made a great work!!!) at https://issuetracker.springsource.com/browse/STS-1314?focusedCommentId=41768&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-41768 but they forwarded me to this forum.

Cheers
Davide

[Szczepan Faber]

Hey,

Interesting idea. I guess the main use case is when you work with the
IDE and you want take advantage of the build on save (eclipse) or
build on ctrl+F9 (idea) of the dependency project.

I think it should be already possible with vanilla gradle + IDE. It's
not great but it should work if you configure a project dependency and
make sure it is higher on the classpath in the consumer project.

As far as STS goes I could imagine a feature that allows to replace
selected binary dependency with a selected project from the workspace
:)

For building gradle from CLI we plan to make it easy to toggle binary
dependency with a project dependency. Our main use case is to allow
working on the subset of projects in a big multi-module project.

Cheers!

On Fri, Jun 17, 2011 at 8:54 AM, davide.cavestro
<[hidden email]> wrote:

> Hi guys,
> I'd like to know if there is a way to somewhat elect a local gradle project
> to be considered for satisfying other projects dependencies.
> This way we could - especially on early developments - avoid packaging and
> uploading snapshot artifacts over and over on our private maven repo only to
> see how they behave on a project we have in the same workspace. This
> scenario applies even for testing a high invasive refactoring.
> It would be something similar to inline plugins for grails (
> http://johnrellis.blogspot.com/2010/03/grails-inplace-plugin-tip.html ).
> PS: I asked the same thing to STS/gradle integration developers (they really
> made a great work!!!) at
> https://issuetracker.springsource.com/browse/STS-1314?focusedCommentId=41768&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-41768
> but they forwarded me to this forum.
>
> Cheers
> Davide
>
> --
> View this message in context: http://gradle.1045684.n5.nabble.com/Is-it-possible-to-elect-a-local-gradle-project-to-satisfy-other-projects-deps-tp4497854p4497854.html
> Sent from the gradle-user mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

--
Szczepan Faber
Principal engineer@gradleware
Lead@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[davide.cavestro]

Hi Szczepan,
adding a project dependency seems not enough (at least on STS, where it always gets automatically discarded when I refresh gradle deps).
But to further elaborate on your suggestion, maybe at the state of the art we could add a somewhat shared flatDir repo before the usual one and use it for upload of local artifacts and for deps resolution (your higher priority) on consumer project.

i.e.

repositories {
    flatDir(name: 'fileRepo', dirs: "$projectDir/../repo") // flat dir repo "shared" at the upper level
    ... // usual private maven repo
}
...
uploadArchives {
    repositories {
        add project.repositories.fileRepo //
        ... //usual private maven repo
    }
...

Is it right?
This way we can't take advantage of the build on save (using it on demand would be a great enhancement) but at least we keep our changes local.
We also have the same use case to allow working on the subset of projects in a big multi-module project. Could you tell me a bit more about your plans to make it easy to toggle binary dependency with a project dependency?

Cheers
Davide

Szczepan Faber wrote

Hey,

Interesting idea. I guess the main use case is when you work with the
IDE and you want take advantage of the build on save (eclipse) or
build on ctrl+F9 (idea) of the dependency project.

I think it should be already possible with vanilla gradle + IDE. It's
not great but it should work if you configure a project dependency and
make sure it is higher on the classpath in the consumer project.

As far as STS goes I could imagine a feature that allows to replace
selected binary dependency with a selected project from the workspace
:)

For building gradle from CLI we plan to make it easy to toggle binary
dependency with a project dependency. Our main use case is to allow
working on the subset of projects in a big multi-module project.

Cheers!

[Szczepan Faber]

> adding a project dependency seems not enough (at least on STS, where it
> always gets automatically discarded when I refresh gradle deps).

Oups :)

> But to further elaborate on your suggestion, maybe at the state of the art
> we could add a somewhat shared flatDir repo before the usual one and use it
> for upload of local artifacts and for deps resolution (your higher priority)
> on consumer project.
>
> i.e.
> repositories {
>    flatDir(name: 'fileRepo', dirs: "$projectDir/../repo") // flat dir repo
> "shared" at the upper level
>    ... // usual private maven repo
> }
> ...
> uploadArchives {
>    repositories {
>        add project.repositories.fileRepo //
>        ... //usual private maven repo
>    }
> ...
>
> Is it right?
> This way we can't take advantage of the build on save (using it on demand
> would be a great enhancement) but at least we keep our changes local.

I guess it is a possible approach given the current state of Gradle.
However, I'm not sure if I like that. The reason I want to have a
project dependency inside the IDE is that the changes are picked up
immediately. If I need to run 'gradle assemble' for the dependency
project to work with the latest changes I can as easily run 'gradle
install' (provided I use maven plugin and use mavenLocal() to share
intra-project dependencies). But then we're back at the starting point
of the discussion.

> We also have the same use case to allow working on the subset of projects in
> a big multi-module project. Could you tell me a bit more about your plans to
> make it easy to toggle binary dependency with a project dependency?

Not sure what specifically would you like to know :) Consider a large,
hierarchical project consisting of 50+ modules. It should be easy  to
check out / work with / build only the selected subproject(s). This
may mean the 'project dependencies' are satisfied via some repo
instead of being a part of a multi-module build run.

Cheers!

>
> Cheers
> Davide
>
>
> Szczepan Faber wrote:
>>
>> Hey,
>>
>> Interesting idea. I guess the main use case is when you work with the
>> IDE and you want take advantage of the build on save (eclipse) or
>> build on ctrl+F9 (idea) of the dependency project.
>>
>> I think it should be already possible with vanilla gradle + IDE. It's
>> not great but it should work if you configure a project dependency and
>> make sure it is higher on the classpath in the consumer project.
>>
>> As far as STS goes I could imagine a feature that allows to replace
>> selected binary dependency with a selected project from the workspace
>> :)
>>
>> For building gradle from CLI we plan to make it easy to toggle binary
>> dependency with a project dependency. Our main use case is to allow
>> working on the subset of projects in a big multi-module project.
>>
>> Cheers!
>>
>>
>
>
> --
> View this message in context: http://gradle.1045684.n5.nabble.com/Is-it-possible-to-elect-a-local-gradle-project-to-satisfy-other-projects-deps-tp4497854p4498136.html
> Sent from the gradle-user mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

--
Szczepan Faber
Principal engineer@gradleware
Lead@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[davide.cavestro]

Since I'm refactoring our modules and transforming each of them in a single gradle project, I started considering them as single projects and linking them only through compile dependencies.
At a certain point I thought I'd have to switch to a multi-module build and search for a way to obtain optional project dependencies (in a scenario where a developer checked out some modules and wants to modify them) while the scenario where the developer didn't checkout only a certain module and wants to resolve its dependencies through the dependency manager was always covered by the maven repo.
Now: it's clear that I've not read enough about project deps/multi-module builds, but from your words I guess adding mavenLocal() to repositories and invoking the install command should be enough to share a packaged artifact in a local repo. The missing bit would be a way to automate the call to install command on save from IDE. Maybe STS has enough hooks fo that. I'll take a closer look.

Cheers
Davide

Szczepan Faber wrote

Not sure what specifically would you like to know :) Consider a large,
hierarchical project consisting of 50+ modules. It should be easy  to
check out / work with / build only the selected subproject(s). This
may mean the 'project dependencies' are satisfied via some repo
instead of being a part of a multi-module build run.

[kdvolder]

It seems that this could be very useful. (Davide is not the first person
I see asking this kind of question :-).

It seems to me that neither Gradle nor the IDE is in the perfect position
to do this right... but if Gradle and the IDE work together we may be
able to get a nice solution.

What I'm thinking is, an "ExternalDependency" as returned by the
Gradle project model in the tooling API should have some type of
identifying 'key' associated with it (similar to how maven etc.
identify/resolve deps).

Also a project should have this type of identifier associated with it
(i.e. the key it uses to 'publish' to the repo).

Then when we update/refresh dependencies, we can check whether projects
that exist in the IDE workspace match those returned by "ExternalDepencies"
and then represent those as project dependencies instead of jar entries
in the classpath container.

I'm almost certain that Gradle already has all this info... but it
isn't available from the tooling API. If it were, then it would
be pretty easy to for the IDE to 'connect the dots' so to speak,
because the IDE knows what the user has in their workspace.

Kris

PS: Yes, it seems also like a good idea suggested by Szcepan to make something
in the IDE that lets a user declare that some project corresponds to some jar and do these
changes... but it more or less means the user has to use some GUI tools
to basically redeclare for the IDE what Gradle builds scripts already declare
for Gradle builds. So while its better than nothing (and something I'm really
considering now :-) I think we can do better than that.

----- Original Message -----

> Since I'm refactoring our modules and transforming each of them in a
> single
> gradle project, I started considering them as single projects and
> linking
> them only through compile dependencies.
> At a certain point I thought I'd have to switch to a multi-module
> build and
> search for a way to obtain optional project dependencies (in a
> scenario
> where a developer checked out some modules and wants to modify them)
> while
> the scenario where the developer didn't checkout only a certain
> module and
> wants to resolve its dependencies through the dependency manager was
> always
> covered by the maven repo.
> Now: it's clear that I've not read enough about project
> deps/multi-module
> builds, but from your words I guess adding mavenLocal() to
> repositories and
> invoking the install command should be enough to share a packaged
> artifact
> in a local repo. The missing bit would be a way to automate the call
> to
> install command on save from IDE. Maybe STS has enough hooks fo that.
> I'll
> take a closer look.
>
> Cheers
> Davide
>
> Szczepan Faber wrote:
> >
> >
> > Not sure what specifically would you like to know :) Consider a
> > large,
> > hierarchical project consisting of 50+ modules. It should be easy
> >  to
> > check out / work with / build only the selected subproject(s). This
> > may mean the 'project dependencies' are satisfied via some repo
> > instead of being a part of a multi-module build run.
> >
> >
>
>
> --
> View this message in context:
> http://gradle.1045684.n5.nabble.com/Is-it-possible-to-elect-a-local-gradle-project-to-satisfy-other-projects-deps-tp4497854p4499287.html
> Sent from the gradle-user mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Mike Mills]

On Sat, Jun 18, 2011 at 11:40 AM, Kris De Volder <[hidden email]> wrote:

> It seems to me that neither Gradle nor the IDE is in the perfect position
> to do this right... but if Gradle and the IDE work together we may be
> able to get a nice solution.
>
> What I'm thinking is, an "ExternalDependency" as returned by the
> Gradle project model in the tooling API should have some type of
> identifying 'key' associated with it (similar to how maven etc.
> identify/resolve deps).
>
> Also a project should have this type of identifier associated with it
> (i.e. the key it uses to 'publish' to the repo).
>

From my perspective, if you have two projects being built by Gradle,
then Gradle could keep a reference to those projects soemwhere in
.gradle.

If the second project declares a depedency such as:

compile group: "projectA", name: "core", version: "1.0.0"

Then there should be a cmd line option on gradle so that before it
tries to resolve the binary dependency from a declared repository, it
should first look for a project named "projectA:core" that has a
declared version of 1.0.0.

If Gradle has already built such a project then it should
automatically perform a build or jar or uploadArtifacts on that
project/module (checking first that the project is still on the local
HDD), then include the created binary as part of the classpath. Better
still somehow link the appropriate configurations as if it was a
larger multimodule build.

This would save having to do some conditional declaration as part of
the dependencies.

This flag could also be used in the eclipse or idea plugins to
automatically create a project with projectA:core as a module of
projectB.

-Mike

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[davide.cavestro]

@Kris
your idea seems to me very clean and reasonable. Please take also a look at documentation for IvyDE "Dependencies resolution in workspace" feature as additional "inspiration" ( http://ant.apache.org/ivy/ivyde/history/latest-milestone/cpc/workspace.html ). What we are discussing about somewhat resemble that feature.

Cheers
Davide

PS: they also has so e support for WTP projects ( http://ant.apache.org/ivy/ivyde/history/latest-milestone/cpc/wtp.html ) but I just thaven't tested it (not yet).

kdvolder wrote

It seems that this could be very useful. (Davide is not the first person
I see asking this kind of question :-).

It seems to me that neither Gradle nor the IDE is in the perfect position
to do this right... but if Gradle and the IDE work together we may be
able to get a nice solution.

What I'm thinking is, an "ExternalDependency" as returned by the
Gradle project model in the tooling API should have some type of
identifying 'key' associated with it (similar to how maven etc.
identify/resolve deps).

Also a project should have this type of identifier associated with it
(i.e. the key it uses to 'publish' to the repo).

Then when we update/refresh dependencies, we can check whether projects
that exist in the IDE workspace match those returned by "ExternalDepencies"
and then represent those as project dependencies instead of jar entries
in the classpath container.

I'm almost certain that Gradle already has all this info... but it
isn't available from the tooling API. If it were, then it would
be pretty easy to for the IDE to 'connect the dots' so to speak,
because the IDE knows what the user has in their workspace.

Kris

PS: Yes, it seems also like a good idea suggested by Szcepan to make something
in the IDE that lets a user declare that some project corresponds to some jar and do these
changes... but it more or less means the user has to use some GUI tools
to basically redeclare for the IDE what Gradle builds scripts already declare
for Gradle builds. So while its better than nothing (and something I'm really
considering now :-) I think we can do better than that.

[kdvolder]

Hi Davide,

Took a quick look at the page you are pointing to. It seems like a nice
feature to have. I agree that the user experience editing across
projects will be much better if Eclipse understands project
dependencies rather than have '.jar' dependency.

As I was trying to explain however, it is currently not possible to
implement this. The knowledge to know that some jar was produced by
another project in the workspace (and could be replaced with
a project dependency is not available to STS/Eclipse, because
the tooling API doesn't expose enough information about projects
/ dependencies.

IvyDE can do this because they have full access to all this info
from Ivy.

So to implement this for Gradle, we'd need an extension of the
Tooling API. So, I can't do this alone.

I'll put some more thought into it and raise a Gradle Jira request
for the API extension.

Without the info coming from Gradle, I could implement something
that lets the user manually remap some specific jars to project
dependencies. (Essentially, provide a UI so that the user can
provide the requisite knowledge, lacking a way to extract it
from Gradle itself).

It may be something to consider, since it would
be flexible in letting users customize this remapping, but
if you have many project's, I can't imagine it being fun
to have to manually declare all these remappings (especially,
if you know that the knowledge is already essentially
encoded in the gradle build files).

If you, or other people on this list, think that the 'manual
remapping' would be worth the effort. Go ahead and raise an
STS Jira for it.

We can then take the discussion further on the respective
Jiras and off of this mailing list.

Kris
----- Original Message -----

> @Kris
> your idea seems to me very clean and reasonable. Please take also a
> look at
> documentation for IvyDE "Dependencies resolution in workspace"
> feature as
> additional "inspiration" (
> http://ant.apache.org/ivy/ivyde/history/latest-milestone/cpc/workspace.html
> ). What we are discussing about somewhat resemble that feature.
>
> Cheers
> Davide
>
> PS: they also has so e support for WTP projects (
> http://ant.apache.org/ivy/ivyde/history/latest-milestone/cpc/wtp.html
> ) but
> I just thaven't tested it (not yet).
>
>
>
> kdvolder wrote:
> >
> > It seems that this could be very useful. (Davide is not the first
> > person
> > I see asking this kind of question :-).
> >
> > It seems to me that neither Gradle nor the IDE is in the perfect
> > position
> > to do this right... but if Gradle and the IDE work together we may
> > be
> > able to get a nice solution.
> >
> > What I'm thinking is, an "ExternalDependency" as returned by the
> > Gradle project model in the tooling API should have some type of
> > identifying 'key' associated with it (similar to how maven etc.
> > identify/resolve deps).
> >
> > Also a project should have this type of identifier associated with
> > it
> > (i.e. the key it uses to 'publish' to the repo).
> >
> > Then when we update/refresh dependencies, we can check whether
> > projects
> > that exist in the IDE workspace match those returned by
> > "ExternalDepencies"
> > and then represent those as project dependencies instead of jar
> > entries
> > in the classpath container.
> >
> > I'm almost certain that Gradle already has all this info... but it
> > isn't available from the tooling API. If it were, then it would
> > be pretty easy to for the IDE to 'connect the dots' so to speak,
> > because the IDE knows what the user has in their workspace.
> >
> > Kris
> >
> > PS: Yes, it seems also like a good idea suggested by Szcepan to
> > make
> > something
> > in the IDE that lets a user declare that some project corresponds
> > to some
> > jar and do these
> > changes... but it more or less means the user has to use some GUI
> > tools
> > to basically redeclare for the IDE what Gradle builds scripts
> > already
> > declare
> > for Gradle builds. So while its better than nothing (and something
> > I'm
> > really
> > considering now :-) I think we can do better than that.
> >
> >
>
>
> --
> View this message in context:
> http://gradle.1045684.n5.nabble.com/Is-it-possible-to-elect-a-local-gradle-project-to-satisfy-other-projects-deps-tp4497854p4505525.html
> Sent from the gradle-user mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[walter.dicarlo]

Davide,

I am not sure I fully understand your needs but I have developed a
Gradle plugin, called Gradle Workspace Plugin, which allows to create
conditional dependencies between projects and artifacts.

Have a look to the announcement I have sent

http://gradle.1045684.n5.nabble.com/ANN-Gradle-Workspace-Plugin-td3238704.html

In few words, the plugin allows to:

- load project specific properties from a CSV file. For example you
can specify the properties for the other my plugin (gradle-svnkit) to
connect a project to a Subversion repository

- use the method ws_project(<project_name>) in the dependency clousure
to add a dependency either from the project or, if not present or
disabled, from its jar

- manage the build.gradle of all sub-projects with one file workspace.gradle

So, in few words, you specify the dependencies of the projects with
ws_project() method and then you can control if the dependency is with
the project or its artifact.

Note that it is a plugin I have developed in my spare time. This means
that it needs some code cleaning and generalization.

Hope it can help you

Don't hesitate to provide any comment/feddback

Regards,

Walter

On 17 June 2011 08:54, davide.cavestro <[hidden email]> wrote:

> Hi guys,
> I'd like to know if there is a way to somewhat elect a local gradle project
> to be considered for satisfying other projects dependencies.
> This way we could - especially on early developments - avoid packaging and
> uploading snapshot artifacts over and over on our private maven repo only to
> see how they behave on a project we have in the same workspace. This
> scenario applies even for testing a high invasive refactoring.
> It would be something similar to inline plugins for grails (
> http://johnrellis.blogspot.com/2010/03/grails-inplace-plugin-tip.html ).
> PS: I asked the same thing to STS/gradle integration developers (they really
> made a great work!!!) at
> https://issuetracker.springsource.com/browse/STS-1314?focusedCommentId=41768&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-41768
> but they forwarded me to this forum.
>
> Cheers
> Davide
>
> --
> View this message in context: http://gradle.1045684.n5.nabble.com/Is-it-possible-to-elect-a-local-gradle-project-to-satisfy-other-projects-deps-tp4497854p4497854.html
> Sent from the gradle-user mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[davide.cavestro]

I've just filed https://issuetracker.springsource.com/browse/STS-1858 to take the discussion further there as you suggested.
Opening a jira for gradle tooling api is up to you (if suitable).

Cheers
Davide

kdvolder wrote

Hi Davide,

Took a quick look at the page you are pointing to. It seems like a nice
feature to have. I agree that the user experience editing across
projects will be much better if Eclipse understands project
dependencies rather than have '.jar' dependency.

As I was trying to explain however, it is currently not possible to
implement this. The knowledge to know that some jar was produced by
another project in the workspace (and could be replaced with
a project dependency is not available to STS/Eclipse, because
the tooling API doesn't expose enough information about projects
/ dependencies.

IvyDE can do this because they have full access to all this info
from Ivy.

So to implement this for Gradle, we'd need an extension of the
Tooling API. So, I can't do this alone.

I'll put some more thought into it and raise a Gradle Jira request
for the API extension.

Without the info coming from Gradle, I could implement something
that lets the user manually remap some specific jars to project
dependencies. (Essentially, provide a UI so that the user can
provide the requisite knowledge, lacking a way to extract it
from Gradle itself).

It may be something to consider, since it would
be flexible in letting users customize this remapping, but
if you have many project's, I can't imagine it being fun
to have to manually declare all these remappings (especially,
if you know that the knowledge is already essentially
encoded in the gradle build files).

If you, or other people on this list, think that the 'manual
remapping' would be worth the effort. Go ahead and raise an
STS Jira for it.

We can then take the discussion further on the respective
Jiras and off of this mailing list.

Kris

[davide.cavestro]

@Walter
I just passed over your plugin cause I erroneously thought the emphasis was on loading project properties from a CSV file. Now I'll surely take a look.

Cheers
Davide

walter.dicarlo wrote

Davide,

I am not sure I fully understand your needs but I have developed a
Gradle plugin, called Gradle Workspace Plugin, which allows to create
conditional dependencies between projects and artifacts.

Have a look to the announcement I have sent

http://gradle.1045684.n5.nabble.com/ANN-Gradle-Workspace-Plugin-td3238704.html

In few words, the plugin allows to:

- load project specific properties from a CSV file. For example you
can specify the properties for the other my plugin (gradle-svnkit) to
connect a project to a Subversion repository

- use the method ws_project(<project_name>) in the dependency clousure
to add a dependency either from the project or, if not present or
disabled, from its jar

- manage the build.gradle of all sub-projects with one file workspace.gradle

So, in few words, you specify the dependencies of the projects with
ws_project() method and then you can control if the dependency is with
the project or its artifact.

Note that it is a plugin I have developed in my spare time. This means
that it needs some code cleaning and generalization.

Hope it can help you

Don't hesitate to provide any comment/feddback

Regards,

Walter

[davide.cavestro]

@Walter
I tried your WSProjectPlugin with no success.

After building it and putting the generated jar into my root project "libs" folder I made it available through

buildscript {
    dependencies {
        classpath fileTree(dir: "libs", include: "*.jar")
    }
}

So far so good. The problem is how to use it. Provided that I have a multi-project structure like the following

.
|-- build.gradle
|-- libs
|   `-- gradle-ws-plugin-0.3.jar
|-- settings.gradle
|-- src
|-- sublib1
|   |-- build.gradle
|   `-- src
`-- sublib2
    |-- build.gradle
    `-- src

I tried configuring deps on root project as

dependencies {
    compile ws_project(':sublib1')
    compile ws_project(':sublib2')
//  compile project (':sublib1')
//  compile project (':sublib2')
}

but it seems they never contributes as compilation dependencies. While if I use the gradle way (such as compile project (':sublib1') ) it works, (provided that I include sublib1 from settings.gradle.
Any idea?
Having developed such plugins, you could also be interested in http://issues.gradle.org/browse/GRADLE-1014

PS: the svn organization of your project vitiates the default checkout instructions provided at http://code.google.com/p/gradle-wdc-plugins/source/checkout

Cheers
Davide

[davide.cavestro]

Szczepan Faber wrote

The reason I want to have a
project dependency inside the IDE is that the changes are picked up
immediately. If I need to run 'gradle assemble' for the dependency
project to work with the latest changes I can as easily run 'gradle
install' (provided I use maven plugin and use mavenLocal() to share
intra-project dependencies). But then we're back at the starting point
of the discussion.

Just wondering if in the meantime we could - just as a raw workaround even for GRADLE-1014 - intercept the dependency resolution (with a custom resolver?) and search within the rootProject subprojects if there is one having a compatible groupId/artifactId/version. If found, invoking the maven install task, then leaving the resolution process continue.
This way we get some sort of raw automatic dependency resolution against projects (although deep IDE integration remains far, far away).

Cheers
Davide

[davide.cavestro]

Just got the ugliest workaround running. Having a multiproject that includes some projects, and every project having only external dependencies, it simply introduces before compileJava a dependency on the maven install task for projects (within the multiproject) related to external dependencies.

It's only a raw attempt to mimic a sort of smarter dependency resolution.

subprojects {
	apply plugin: 'java'
	apply plugin: 'maven'

	afterEvaluate { project ->

		def subprojectsMap = [:]
		rootProject.subprojects.each {subprj->
			subprojectsMap["${subprj.group}:${subprj.name}"] = subprj
		}
		
		project.configurations.each {config->
			config.dependencies.each {dep->
				def matchingprj = subprojectsMap["${dep.group}:${dep.name}"]
				if (matchingprj) {
					println "On ${project} will use local ${matchingprj} instead of ${dep}"
					if (project.tasks.find {tsk->tsk.name=='compileJava'}) {
						compileJava.dependsOn(":${matchingprj.name}:install")
						compileTestJava.dependsOn(":${matchingprj.name}:install")
					}
					if (project.tasks.find {tsk->tsk.name=='compileGroovy'}) {
						compileGroovy.dependsOn(":${matchingprj.name}:install")
						compileTestGroovy.dependsOn(":${matchingprj.name}:install")
					}
				}
			}
		}
	}
}

This workaround is only for multi-project builds... and that's a pity, but if I check out only a single project, it continues to build downloading dependencies from the maven repo.

Any remark/suggestion?

Cheers
Davide

[Adam Murdoch]

On 08/09/2011, at 1:54 AM, davide.cavestro wrote:

Just got the ugliest workaround running. Having a multiproject that includes
some projects, and every project having only external dependencies, it
simply introduces before /compileJava/ a dependency on the maven install
task for projects (within the multiproject) related to external
dependencies.

It's only a raw attempt to mimic a sort of /smarter dependency resolution/.


This workaround is only for multi-project builds... and that's a pity, but
if I check out only a single project, it continues to build downloading
dependencies from the maven repo.

Any remark/suggestion?

If you want to substitute in projects from within the same (multi-project) build, as it sounds like you're doing above, another approach might be: Once the build scripts for all projects have been executed, run through the set of dependencies, and replace external dependencies that refer to projects in the same build with actual project dependencies.

For example, in the root build.gradle:

gradle.projectsEvaluated {

    allprojects {

        configurations.each { config ->

            config.dependencies.withType(ExternalDependency).each { dep ->

                def targetProject = findTargetProject(dep)

                if ( targetProject ) {

                    config.dependencies.remove(dep)

                    dependencies.add(config.name, targetProject)

                }

            }

        }

    }

}

def findTargetProject(ExternalDependency dep) {

    // iterate over all projects, look for match on group+name, return if found

}

(this code probably needs a tweak based on which version of Gradle you are using, but you get the idea)

What's nice about this approach is that you get all the goodness of Gradle knowing about the dependencies between the projects: IDE support understands this, you can use 'buildNeeded' and 'buildDependent', etc.

For projects outside the build, you'd need to use a different approach. One option: Once all the projects have been evaluated, run through the set of dependencies. For each one which refers to a project that can be built locally (however it is you decide that), add an 'install<Project>' task of type GradleBuild, which runs the install task for the project. Then, add a dummy file dependency to the configuration which depends on this install task:

gradle.projectsEvaluated {

    allprojects {

        configurations.each { config ->

            config.dependencies.withType(ExternalDependency).each { dep ->

                def targetProjectDir = findTargetProject(dep)

                if ( targetProjectDir ) {

                    def installTask = rootProject.task("install$dep.name", type: GradleBuild) {

                        dir = targetProjectDir

                        tasks = ['install']

                    }

                    dependencies.add(config.name, files() { builtBy installTask })

                }

            }

        }

    }

}

def findTargetProjectDir(ExternalDependency dep) {

    // Figures out where to find the project on the local filesystem, if available.

}

This is definitely a use case I want to tackle soon after Gradle 1.0 is available. We'd start probably with a plugin that packages up the above logic. You'd apply the plugin and provide us with some mapping from external dependency to target project directory. We'd then improve the plugin over time:

* Add some support to the IDE plugins, so that locally available projects are dragged into the IDE project with the appropriate 

dependencies between them.

* Use the artifacts directly out of the external projects, as we do for project dependencies, rather than routing through maven cache.

* Add a conventional location to look for local projects, and use them automatically if found. For example, I might just check out a bunch of source trees into, say, a '~/gradle' directory and Gradle will just wire them together automatically.

* Make 'buildNeeded' work across build boundaries. Maybe make 'buildDependents' work across build boundaries in the other direction, so that it builds all local projects that depend on the project.

* Make external builds first-class citizens, so that you can refer to their tasks and publications directly from the build and the command-line, see them in the UI, and so on. By doing this, buildSrc can become a regular external, stand-alone project which happens to live in a conventional location.

There's lot of interesting stuff we can do in this space.

--

Adam Murdoch
Gradle Co-founder
http://www.gradle.org
VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com

[davide.cavestro]

Adam Murdoch wrote

replace external dependencies that refer to projects in the same build with actual project dependencies.

I previously tried the replace way, but (most probably for my inexperience with gradle) something went wrong.
While adapting your snippet for gradle-1.0-milestone-3, I noticed the external dependency is never actually removed, hence leading to unresolved deps. So given a multiproject including prj1 and prj2, with prj2 having an external dependency on the main artifact of prj1, if I launch following script from prj2

gradle.projectsEvaluated {
    allprojects {
        configurations.each { config ->
            config.getDependencies (ExternalDependency).each { dep ->
                def targetProject = findTargetProject(dep)
                if ( targetProject ) {
                    println "replacing external dep with ${targetProject}"
                    println "initial deps count: ${config.dependencies.size()}"
                    config.dependencies.remove (dep)
                    println "deps count after removal: ${config.dependencies.size()}"
                    dependencies.add(config.name, targetProject)
                    println "final deps count: ${config.dependencies.size()}"
                }
            }
        }
    }
}

def findTargetProject(ExternalDependency dep) {
    allprojects.find {prj-> prj.group==dep.group && prj.name==dep.name }// iterate over all projects, look for match on group+name, return if found
}

I get this output

replacing external dep with project ':prj1'
initial deps count: 1
deps count after removal: 1
final deps count: 2
:prj1:compileJava UP-TO-DATE
:prj1:processResources UP-TO-DATE
:prj1:classes UP-TO-DATE
:prj1:jar UP-TO-DATE
:prj2:compileJava
:: problems summary ::
:::: WARNINGS
		module not found: org.acme#prj1;latest.integration

I guess config.dependencies returns some sort of detached view, hence removal instruction has no effects cause it does not operates on actual dependencies structure (if you assign it to a variable you can see that it is a LinkedHashSet instance, and it actually removes the element).

Adam Murdoch wrote

This is definitely a use case I want to tackle soon after Gradle 1.0 is available. We'd start probably with a plugin that packages up the above logic. You'd apply the plugin and provide us with some mapping from external dependency to target project directory. We'd then improve the plugin over time
...
There's lot of interesting stuff we can do in this space.

That's great news!!! I wonder if IDE plugin developers could eventually benefit from API extensions for such mapping...
Thank you very much

Cheers
Davide

[Luke Daley-2]

On 08/09/2011, at 9:59 AM, davide.cavestro wrote:

> Adam Murdoch wrote:
>>
>> replace external dependencies that refer to projects in the same build
>> with actual project dependencies.
>>
> I previously tried the/ replace way/, but (most probably for my inexperience
> with gradle) something went wrong.
> While adapting your snippet for gradle-1.0-milestone-3, I noticed the
> external dependency is never actually removed, hence leading to unresolved
> deps. So given a multiproject including prj1 and prj2, with prj2 having an
> external dependency on the main artifact of prj1, if I launch following
> script from prj2
>
> I get this output
>
>
> I guess /config.dependencies/ returns some sort of detached view, hence
> removal instruction has no effects cause it does not operates on actual
> dependencies structure (if you assign it to a variable you can see that it
> is a LinkedHashSet instance, and it actually removes the element).

That's correct, as of milestone 4 though it's the real collection.

There unfortunately isn't a way to do this pre milestone 4, without removing private restrictions. Let me know if you need to go this route and want some help.

--
Luke Daley
Principal Engineer, Gradleware
http://gradleware.com


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[davide.cavestro]

Luke Daley-2 wrote

That's correct, as of milestone 4 though it's the real collection.
There unfortunately isn't a way to do this pre milestone 4, without removing private restrictions. Let me know if you need to go this route and want some help.

I see, on m3 we get the result of calling DefaultDomainObjectContainer.getAll(), hence a new instance of LinkedHashSet.
I am going to try milestone 4: if hit some blocking regression (even with STS integration) I will surely appreciate some help with m3.
Thank you for your willingness.

Cheers
Davide

[davide.cavestro]

With gradle-1.0-milestone-4 it works!!!
To avoid concurrent modification exceptions I just had to decouple iteration from the set of dependencies that's going to be modified. I did it with config.dependencies.withType(ExternalDependency).asList().each { dep ->.
So this is the ultimate version

gradle.projectsEvaluated {
    allprojects {
        configurations.each { config ->
            config.dependencies.withType(ExternalDependency).asList().each { dep ->
                def targetProject = findTargetProject(dep)
                if ( targetProject ) {
		    config.dependencies.remove(dep)
                    dependencies.add(config.name, targetProject)
                }
            }
        }
    }
}

def findTargetProject(ExternalDependency dep) {
    allprojects.find {prj-> prj.group==dep.group && prj.name==dep.name }// iterate over all projects, look for match on group+name, return if found
}

Now I had some problems with IDE integration. When I refresh gradle dependencies from STS it complains for UNRESOLVED DEPENDENCIES.
I suppose STS inferences the dependencies from the DSL, so it is not aware of our dependencies replacement machinery (moreover I put it into the rootProject build script to share it with all subprojects).
The funny thing is that it always fails resolving testRuntime config deps: I guess STS uses that configuration (that in turn extends runtime and compile) to collect all project dependencies and pass them to the ide project descriptor through its api. Maybe I should warn Kris De Volder from STS...
BTW I will give a try to the projects outside the build approach as soon as possible.

Cheers
Davide