Quantcast

improvements to conflict resolution

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

improvements to conflict resolution

Szczepan Faber
Hey,

I've started speccing improvements to conflict resolution here:
https://github.com/gradle/gradle/blob/master/design-docs/conflict-resolution-spec.md
(there's a big DSL mock-up down the bottom).

It would be good to start discussing the DSL additions here on the
mailing list. Please give feedback about the DSL or the use cases,
etc. I haven't decided myself on the DSL :) I'll keep you in loop on
how the design goes.

Cheers!
--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: improvements to conflict resolution

Heinrich Filter
Hi Szczepan,

I’m new around the Gradle community but would love to start contributing more. Please feel free to point out any “conventions” that I miss.

For me the “replacedBy” keyword is a bit confusing. In your example: ‘guava' was replace by 'google-collections'. In my mind that implies that 'google-collection' is the *newer* dependency and not, as you state in the doc, the older version.

I would think that ‘guava' rather *replaces* or *supersedes* 'google-collections’.

Or am thinking about this the wrong way?

Regards,
Heinrich

On 31 Jul 2014, at 3:27 PM, Szczepan Faber <[hidden email]> wrote:

> Hey,
>
> I've started speccing improvements to conflict resolution here:
> https://github.com/gradle/gradle/blob/master/design-docs/conflict-resolution-spec.md
> (there's a big DSL mock-up down the bottom).
>
> It would be good to start discussing the DSL additions here on the
> mailing list. Please give feedback about the DSL or the use cases,
> etc. I haven't decided myself on the DSL :) I'll keep you in loop on
> how the design goes.
>
> Cheers!
> --
> Szczepan Faber
> Core dev@gradle; Founder@mockito
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>


**********************************************************************
This email and all content are subject to the following disclaimer

http://content.momentum.co.za/content/legal/disclaimer_email.htm

**********************************************************************

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: improvements to conflict resolution

Szczepan Faber
This is a typo, the spec is work in progress. Initially we thought
about 'replaces' but we've changed that into 'replacedBy' but I forgot
to reverse the source and target :)

So:

google-collections is replaced by guava
guava replaces google-collections

Thanks for pointing this out. I'll do more speccing and get back to
the mailing list.

Cheers!

On Fri, Aug 1, 2014 at 9:38 AM, Heinrich Filter
<[hidden email]> wrote:

> Hi Szczepan,
>
> I'm new around the Gradle community but would love to start contributing more. Please feel free to point out any "conventions" that I miss.
>
> For me the "replacedBy" keyword is a bit confusing. In your example: 'guava' was replace by 'google-collections'. In my mind that implies that 'google-collection' is the *newer* dependency and not, as you state in the doc, the older version.
>
> I would think that 'guava' rather *replaces* or *supersedes* 'google-collections'.
>
> Or am thinking about this the wrong way?
>
> Regards,
> Heinrich
>
> On 31 Jul 2014, at 3:27 PM, Szczepan Faber <[hidden email]> wrote:
>
>> Hey,
>>
>> I've started speccing improvements to conflict resolution here:
>> https://github.com/gradle/gradle/blob/master/design-docs/conflict-resolution-spec.md
>> (there's a big DSL mock-up down the bottom).
>>
>> It would be good to start discussing the DSL additions here on the
>> mailing list. Please give feedback about the DSL or the use cases,
>> etc. I haven't decided myself on the DSL :) I'll keep you in loop on
>> how the design goes.
>>
>> Cheers!
>> --
>> Szczepan Faber
>> Core dev@gradle; Founder@mockito
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>    http://xircles.codehaus.org/manage_email
>>
>>
>
>
> **********************************************************************
> This email and all content are subject to the following disclaimer
>
> http://content.momentum.co.za/content/legal/disclaimer_email.htm
>
> **********************************************************************
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>



--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: improvements to conflict resolution

Szczepan Faber
More thoughts on conflict resolution, Adam-style :)

1. The starting use case is avoiding conflicting guava and
google-collections on the same classpath. We'll enable that by
offering an API to declare that a module (group+name)
replaces/supersedes/deprecates a different module (group+name). This
use is a starting point to more interesting use cases which I will
describe in later. While designing the DSL and the behavior we need to
be cautious about those other use cases, so that we can develop this
feature incrementally. We will use guava an collections as an example.

2. In a standalone project, a guava-collections issue is not that
problematic, it can be sorted out by editing the build file (change
the dependencies, add an 'exclude', add force rule, etc.). The use
case is getting interesting in on an enterprise level, where the
organisation wants to solve the problem in the shared plugin suite,
instead of getting each team solve the problem on their own.
Well-known deprecations like collections-guava could be even encoded
in Gradle out of the box, but that's a different topic.

3. Current APIs for influencing the dependency resolution (forcing
versions, dependency resolve rules) offer only "unconditional"
manipulation to the dependencies that are being resolved. Let's map
the collections-guava problem to the current API: we could force
collections to become guava via dependency resolve rule. The effect
is:
 - only collections in graph - guava is used
 a) only guava in graph - guava is used
 b) both collections in graph - guava is used (collections are forced
into guava)
 c) only collections in graph - guava is used
This is what I mean by 'unconditional' manipulation - if collections
appear in graph it is _always_ replaced with guava. And this is not
quite what we want...

4. We want the replacement rule to get activated only when both
collections and guava are in the graph. If only collections appear in
the graph - there is no need to replace it with guava. Perhaps guava
and collection is not the best example because early guava is
completely backwards compatible with latest collections. So in theory,
we could unconditionally replace collections with an early version of
guava via existing dependency resolve rule API. However, let's imagine
that replacement source and target are not that quite compatible or
that someone is using an _early_ version of collections which is not
quite compatible with _early_ guava.

5. The implementation is tricker than our current forcing/dependency
resolve rules. Consider dependency graph traversal scenarios:
 - first we encounter collections, we happily use it and continue traversing.
 - then we encounter guava - oups - we need to evict already resolved
module collections and continue traversing. (It's more complicated
than that, as we resolve conflicts only when we cannot traverse any
further).

In fact, it does not really differ from the standard, version-based
conflict resolution. I wanted to point out the difference between
existing dependency resolve rules implementation. The latter does not
need to consider already-resolved modules.

6. Other features to consider:
 - module is replaced with a set of modules ('spring' ->
'spring-core', 'spring-aop', etc.)
 - set of modules is replaced with a single module (I don't know any
real example for that)
 - api module and impl module use consistent version

7. Given that there are more ways to influence the dependency
resolution, and specifically, to replace/force certain dependencies
with others we should think about improving diagnostics on why certain
dependency ended up in the graph and why this particular version is
used. Our reports and API already provides 'selection reason'
information but I'm afraid it's not enough. A dependency can be
manipulated during resolution by different rules, applied by different
plugins and a flat information like 'dependency resolve rule was used'
is not enough IMHO. There can be a stream of actions that affects the
selected version of a dependency.

8. I made up my mind about the DSL. I'm easy and I can totally be
convinced to something else. Yet, here's my current preference:

 a) I would reuse some existing low-level, imperative API, like the
component rules or dependency resolve rules, for example:

  eachDependency { details ->
    if (details.target.name == 'google-collections') {
     details.replacedBy 'com.google.guava:guava'
    }
  }

 b) I would add some higher level api that would use a). For example:

 dependencies {
   components {
     modules("com.google.collections:google-collections").deprecatedBy("com.google.guava:guava")
     //or replacedBy(), supersededBy()
   }
 }

Thoughts?

On Fri, Aug 1, 2014 at 11:30 AM, Szczepan Faber <[hidden email]> wrote:

> This is a typo, the spec is work in progress. Initially we thought
> about 'replaces' but we've changed that into 'replacedBy' but I forgot
> to reverse the source and target :)
>
> So:
>
> google-collections is replaced by guava
> guava replaces google-collections
>
> Thanks for pointing this out. I'll do more speccing and get back to
> the mailing list.
>
> Cheers!
>
> On Fri, Aug 1, 2014 at 9:38 AM, Heinrich Filter
> <[hidden email]> wrote:
>> Hi Szczepan,
>>
>> I'm new around the Gradle community but would love to start contributing more. Please feel free to point out any "conventions" that I miss.
>>
>> For me the "replacedBy" keyword is a bit confusing. In your example: 'guava' was replace by 'google-collections'. In my mind that implies that 'google-collection' is the *newer* dependency and not, as you state in the doc, the older version.
>>
>> I would think that 'guava' rather *replaces* or *supersedes* 'google-collections'.
>>
>> Or am thinking about this the wrong way?
>>
>> Regards,
>> Heinrich
>>
>> On 31 Jul 2014, at 3:27 PM, Szczepan Faber <[hidden email]> wrote:
>>
>>> Hey,
>>>
>>> I've started speccing improvements to conflict resolution here:
>>> https://github.com/gradle/gradle/blob/master/design-docs/conflict-resolution-spec.md
>>> (there's a big DSL mock-up down the bottom).
>>>
>>> It would be good to start discussing the DSL additions here on the
>>> mailing list. Please give feedback about the DSL or the use cases,
>>> etc. I haven't decided myself on the DSL :) I'll keep you in loop on
>>> how the design goes.
>>>
>>> Cheers!
>>> --
>>> Szczepan Faber
>>> Core dev@gradle; Founder@mockito
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe from this list, please visit:
>>>
>>>    http://xircles.codehaus.org/manage_email
>>>
>>>
>>
>>
>> **********************************************************************
>> This email and all content are subject to the following disclaimer
>>
>> http://content.momentum.co.za/content/legal/disclaimer_email.htm
>>
>> **********************************************************************
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>     http://xircles.codehaus.org/manage_email
>>
>>
>
>
>
> --
> Szczepan Faber
> Core dev@gradle; Founder@mockito



--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: improvements to conflict resolution

Adam Murdoch

On 1 Aug 2014, at 9:18 pm, Szczepan Faber <[hidden email]> wrote:

More thoughts on conflict resolution, Adam-style :)

1. The starting use case is avoiding conflicting guava and
google-collections on the same classpath. We'll enable that by
offering an API to declare that a module (group+name)
replaces/supersedes/deprecates a different module (group+name). This
use is a starting point to more interesting use cases which I will
describe in later. While designing the DSL and the behavior we need to
be cautious about those other use cases, so that we can develop this
feature incrementally. We will use guava an collections as an example.

2. In a standalone project, a guava-collections issue is not that
problematic, it can be sorted out by editing the build file (change
the dependencies, add an 'exclude', add force rule, etc.). The use
case is getting interesting in on an enterprise level, where the
organisation wants to solve the problem in the shared plugin suite,
instead of getting each team solve the problem on their own.
Well-known deprecations like collections-guava could be even encoded
in Gradle out of the box, but that's a different topic.

Absolutely we should. There are some backwards compatibility implications if we were to add more meta-data about certain modules over time.


3. Current APIs for influencing the dependency resolution (forcing
versions, dependency resolve rules) offer only "unconditional"
manipulation to the dependencies that are being resolved. Let's map
the collections-guava problem to the current API: we could force
collections to become guava via dependency resolve rule. The effect
is:
- only collections in graph - guava is used
a) only guava in graph - guava is used
b) both collections in graph - guava is used (collections are forced
into guava)
c) only collections in graph - guava is used
This is what I mean by 'unconditional' manipulation - if collections
appear in graph it is _always_ replaced with guava. And this is not
quite what we want...

4. We want the replacement rule to get activated only when both
collections and guava are in the graph.

I would think about it somewhat differently. We don’t want to conditionally activate some rule here. Instead, we want to use some facts about collections and guava to both detect conflicts between the two and also to resolve such conflicts.

So, collections was replaced by guava. This is a fact that is true regardless of whether one or both or neither of them are in the graph.

This fact implies a few further facts:

- Collections and guava conflict with each other. They provide the same API and shouldn’t both be present in a classpath. When resolving a dependency graph, we have to select one of these to use.
- Guava is newer than collections, so if we happen to resolve conflicts by selecting the newer of two conflicting modules, we should select the guava module over the collections module.

We’d use these facts in other places. For example, when reporting on dependencies that are out-of-date, we should consider versions of guava as candidate upgrades for any dependencies on collections. Same when we’re offering suggestions in the IDE or when generating a dependency lock file.

For this use case, what we’re after is a DSL to let you state these facts about various modules. We don’t want an DSL where you provide imperative code that implements the mechanics of detecting or resolving the conflicts. This allows us to implement everything efficiently and also to reuse these facts in other contexts beyond resolving a dependency graph.


If only collections appear in
the graph - there is no need to replace it with guava. Perhaps guava
and collection is not the best example because early guava is
completely backwards compatible with latest collections. So in theory,
we could unconditionally replace collections with an early version of
guava via existing dependency resolve rule API. However, let's imagine
that replacement source and target are not that quite compatible or
that someone is using an _early_ version of collections which is not
quite compatible with _early_ guava.

Compatibility is a different concern.

You can think of the different versions of a given module in the same way as replacement of one module by another. If I have A version 1.2 and A version 1.3, then we can say 'A version 1.2 was replaced by A version 1.3’, with the same implications.

All we’re doing with replacement is saying something about the history of some thing. This doesn’t say anything about its compatibility over time. Or, more accurately, this doesn’t say anything more than what the version of a thing says about compatibility.

Similar to the history of a thing, there are also facts about the compatibility of a thing over time that we want to use as input to resolving a dependency graph, as part of resolving conflicts. If I have A and B in the dependency graph and they conflict, and A is newer than B, then I can select A only if A is compatible with B. Otherwise I have to either fail the resolve or find some C that is newer than A and B and that is compatible with both of them.

We’d also use facts about compatibility when resolving a graph where things are provided - eg the android SDK or the Windows API or whatever: If I have A and B is in the dependency graph and they conflict, and A is provided (and so can’t be substituted) and A is not compatible with B, then I have to fail the resolve.

Facts about compatibility are useful in other places beyond dependency resolution. For example, when updating a dependency lock file, we might skip candidates that are not compatible with what we’re currently using. Or when offering suggestions in the IDE we might distinguish between candidates that are compatible and not compatible.



5. The implementation is tricker than our current forcing/dependency
resolve rules. Consider dependency graph traversal scenarios:
- first we encounter collections, we happily use it and continue traversing.
- then we encounter guava - oups - we need to evict already resolved
module collections and continue traversing. (It's more complicated
than that, as we resolve conflicts only when we cannot traverse any
further).

In fact, it does not really differ from the standard, version-based
conflict resolution. I wanted to point out the difference between
existing dependency resolve rules implementation. The latter does not
need to consider already-resolved modules.

That’s right. The things we currently call ‘dependency resolve rules’ are dependency (edge) replacement rules. They don’t have anything with conflict handling. They’re very much not ‘everything that affects dependency resolution’ rules.


6. Other features to consider:
- module is replaced with a set of modules ('spring' ->
'spring-core', 'spring-aop', etc.)
- set of modules is replaced with a single module (I don't know any
real example for that)
- api module and impl module use consistent version

7. Given that there are more ways to influence the dependency
resolution, and specifically, to replace/force certain dependencies
with others we should think about improving diagnostics on why certain
dependency ended up in the graph and why this particular version is
used. Our reports and API already provides 'selection reason'
information but I'm afraid it's not enough. A dependency can be
manipulated during resolution by different rules, applied by different
plugins and a flat information like 'dependency resolve rule was used'
is not enough IMHO. There can be a stream of actions that affects the
selected version of a dependency.

This is something the general-purpose model rules has to address, and so the plan is that the ad hoc rule handling that we currently have in a few places in dependency management will be replaced by the general-purpose stuff, and will pick up the insight and reporting goodness that it provides.


8. I made up my mind about the DSL. I'm easy and I can totally be
convinced to something else. Yet, here's my current preference:

a) I would reuse some existing low-level, imperative API, like the
component rules or dependency resolve rules, for example:

 eachDependency { details ->
   if (details.target.name == 'google-collections') {
    details.replacedBy ‘com.google.guava:guava'

The eachDependency() method is used to replace one dependency edge with another. It doesn’t make any sense to say something about the history of a module here.

If we were to add a low level mechanism, it should be given the meta-data for a given module and allow the rules to say things like:

- This module conflicts with modules with these ids.
- This module is newer than modules with these ids.
- This module is or is not compatible with modules with these ids.


   }
 }

b) I would add some higher level api that would use a). For example:

dependencies {
  components {
    modules("com.google.collections:google-collections").deprecatedBy("com.google.guava:guava")
    //or replacedBy(), supersededBy()
  }
}

Thoughts?

We should only add the high level API.


On Fri, Aug 1, 2014 at 11:30 AM, Szczepan Faber <[hidden email]> wrote:
This is a typo, the spec is work in progress. Initially we thought
about 'replaces' but we've changed that into 'replacedBy' but I forgot
to reverse the source and target :)

So:

google-collections is replaced by guava
guava replaces google-collections

Thanks for pointing this out. I'll do more speccing and get back to
the mailing list.

Cheers!

On Fri, Aug 1, 2014 at 9:38 AM, Heinrich Filter
<[hidden email]> wrote:
Hi Szczepan,

I'm new around the Gradle community but would love to start contributing more. Please feel free to point out any "conventions" that I miss.

For me the "replacedBy" keyword is a bit confusing. In your example: 'guava' was replace by 'google-collections'. In my mind that implies that 'google-collection' is the *newer* dependency and not, as you state in the doc, the older version.

I would think that 'guava' rather *replaces* or *supersedes* 'google-collections'.

Or am thinking about this the wrong way?

Regards,
Heinrich

On 31 Jul 2014, at 3:27 PM, Szczepan Faber <[hidden email]> wrote:

Hey,

I've started speccing improvements to conflict resolution here:
https://github.com/gradle/gradle/blob/master/design-docs/conflict-resolution-spec.md
(there's a big DSL mock-up down the bottom).

It would be good to start discussing the DSL additions here on the
mailing list. Please give feedback about the DSL or the use cases,
etc. I haven't decided myself on the DSL :) I'll keep you in loop on
how the design goes.

Cheers!
--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

  http://xircles.codehaus.org/manage_email




**********************************************************************
This email and all content are subject to the following disclaimer

http://content.momentum.co.za/content/legal/disclaimer_email.htm

**********************************************************************

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email





--
Szczepan Faber
Core dev@gradle; Founder@mockito



--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email




--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
CTO Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: improvements to conflict resolution

Szczepan Faber

Thanks for great feedback!

2 sie 2014 02:05 "Adam Murdoch" <[hidden email]> napisał(a):

On 1 Aug 2014, at 9:18 pm, Szczepan Faber <[hidden email]> wrote:

More thoughts on conflict resolution, Adam-style :)

1. The starting use case is avoiding conflicting guava and
google-collections on the same classpath. We'll enable that by
offering an API to declare that a module (group+name)
replaces/supersedes/deprecates a different module (group+name). This
use is a starting point to more interesting use cases which I will
describe in later. While designing the DSL and the behavior we need to
be cautious about those other use cases, so that we can develop this
feature incrementally. We will use guava an collections as an example.

2. In a standalone project, a guava-collections issue is not that
problematic, it can be sorted out by editing the build file (change
the dependencies, add an 'exclude', add force rule, etc.). The use
case is getting interesting in on an enterprise level, where the
organisation wants to solve the problem in the shared plugin suite,
instead of getting each team solve the problem on their own.
Well-known deprecations like collections-guava could be even encoded
in Gradle out of the box, but that's a different topic.

Absolutely we should. There are some backwards compatibility implications if we were to add more meta-data about certain modules over time.


3. Current APIs for influencing the dependency resolution (forcing
versions, dependency resolve rules) offer only "unconditional"
manipulation to the dependencies that are being resolved. Let's map
the collections-guava problem to the current API: we could force
collections to become guava via dependency resolve rule. The effect
is:
- only collections in graph - guava is used
a) only guava in graph - guava is used
b) both collections in graph - guava is used (collections are forced
into guava)
c) only collections in graph - guava is used
This is what I mean by 'unconditional' manipulation - if collections
appear in graph it is _always_ replaced with guava. And this is not
quite what we want...

4. We want the replacement rule to get activated only when both
collections and guava are in the graph.

I would think about it somewhat differently. We don’t want to conditionally activate some rule here. Instead, we want to use some facts about collections and guava to both detect conflicts between the two and also to resolve such conflicts.

So, collections was replaced by guava. This is a fact that is true regardless of whether one or both or neither of them are in the graph.

This fact implies a few further facts:

- Collections and guava conflict with each other. They provide the same API and shouldn’t both be present in a classpath. When resolving a dependency graph, we have to select one of these to use.
- Guava is newer than collections, so if we happen to resolve conflicts by selecting the newer of two conflicting modules, we should select the guava module over the collections module.

We’d use these facts in other places. For example, when reporting on dependencies that are out-of-date, we should consider versions of guava as candidate upgrades for any dependencies on collections. Same when we’re offering suggestions in the IDE or when generating a dependency lock file.

For this use case, what we’re after is a DSL to let you state these facts about various modules. We don’t want an DSL where you provide imperative code that implements the mechanics of detecting or resolving the conflicts. This allows us to implement everything efficiently and also to reuse these facts in other contexts beyond resolving a dependency graph.


If only collections appear in
the graph - there is no need to replace it with guava. Perhaps guava
and collection is not the best example because early guava is
completely backwards compatible with latest collections. So in theory,
we could unconditionally replace collections with an early version of
guava via existing dependency resolve rule API. However, let's imagine
that replacement source and target are not that quite compatible or
that someone is using an _early_ version of collections which is not
quite compatible with _early_ guava.

Compatibility is a different concern.

You can think of the different versions of a given module in the same way as replacement of one module by another. If I have A version 1.2 and A version 1.3, then we can say 'A version 1.2 was replaced by A version 1.3’, with the same implications.

All we’re doing with replacement is saying something about the history of some thing. This doesn’t say anything about its compatibility over time. Or, more accurately, this doesn’t say anything more than what the version of a thing says about compatibility.

Similar to the history of a thing, there are also facts about the compatibility of a thing over time that we want to use as input to resolving a dependency graph, as part of resolving conflicts. If I have A and B in the dependency graph and they conflict, and A is newer than B, then I can select A only if A is compatible with B. Otherwise I have to either fail the resolve or find some C that is newer than A and B and that is compatible with both of them.

We’d also use facts about compatibility when resolving a graph where things are provided - eg the android SDK or the Windows API or whatever: If I have A and B is in the dependency graph and they conflict, and A is provided (and so can’t be substituted) and A is not compatible with B, then I have to fail the resolve.

Facts about compatibility are useful in other places beyond dependency resolution. For example, when updating a dependency lock file, we might skip candidates that are not compatible with what we’re currently using. Or when offering suggestions in the IDE we might distinguish between candidates that are compatible and not compatible.



5. The implementation is tricker than our current forcing/dependency
resolve rules. Consider dependency graph traversal scenarios:
- first we encounter collections, we happily use it and continue traversing.
- then we encounter guava - oups - we need to evict already resolved
module collections and continue traversing. (It's more complicated
than that, as we resolve conflicts only when we cannot traverse any
further).

In fact, it does not really differ from the standard, version-based
conflict resolution. I wanted to point out the difference between
existing dependency resolve rules implementation. The latter does not
need to consider already-resolved modules.

That’s right. The things we currently call ‘dependency resolve rules’ are dependency (edge) replacement rules. They don’t have anything with conflict handling. They’re very much not ‘everything that affects dependency resolution’ rules.


6. Other features to consider:
- module is replaced with a set of modules ('spring' ->
'spring-core', 'spring-aop', etc.)
- set of modules is replaced with a single module (I don't know any
real example for that)
- api module and impl module use consistent version

7. Given that there are more ways to influence the dependency
resolution, and specifically, to replace/force certain dependencies
with others we should think about improving diagnostics on why certain
dependency ended up in the graph and why this particular version is
used. Our reports and API already provides 'selection reason'
information but I'm afraid it's not enough. A dependency can be
manipulated during resolution by different rules, applied by different
plugins and a flat information like 'dependency resolve rule was used'
is not enough IMHO. There can be a stream of actions that affects the
selected version of a dependency.

This is something the general-purpose model rules has to address, and so the plan is that the ad hoc rule handling that we currently have in a few places in dependency management will be replaced by the general-purpose stuff, and will pick up the insight and reporting goodness that it provides.


8. I made up my mind about the DSL. I'm easy and I can totally be
convinced to something else. Yet, here's my current preference:

a) I would reuse some existing low-level, imperative API, like the
component rules or dependency resolve rules, for example:

 eachDependency { details ->
   if (details.target.name == 'google-collections') {
    details.replacedBy ‘com.google.guava:guava'

The eachDependency() method is used to replace one dependency edge with another. It doesn’t make any sense to say something about the history of a module here.

If we were to add a low level mechanism, it should be given the meta-data for a given module and allow the rules to say things like:

- This module conflicts with modules with these ids.
- This module is newer than modules with these ids.
- This module is or is not compatible with modules with these ids.


   }
 }

b) I would add some higher level api that would use a). For example:

dependencies {
  components {
    modules("com.google.collections:google-collections").deprecatedBy("com.google.guava:guava")
    //or replacedBy(), supersededBy()
  }
}

Thoughts?

We should only add the high level API.


On Fri, Aug 1, 2014 at 11:30 AM, Szczepan Faber <[hidden email]> wrote:
This is a typo, the spec is work in progress. Initially we thought
about 'replaces' but we've changed that into 'replacedBy' but I forgot
to reverse the source and target :)

So:

google-collections is replaced by guava
guava replaces google-collections

Thanks for pointing this out. I'll do more speccing and get back to
the mailing list.

Cheers!

On Fri, Aug 1, 2014 at 9:38 AM, Heinrich Filter
<[hidden email]> wrote:
Hi Szczepan,

I'm new around the Gradle community but would love to start contributing more. Please feel free to point out any "conventions" that I miss.

For me the "replacedBy" keyword is a bit confusing. In your example: 'guava' was replace by 'google-collections'. In my mind that implies that 'google-collection' is the *newer* dependency and not, as you state in the doc, the older version.

I would think that 'guava' rather *replaces* or *supersedes* 'google-collections'.

Or am thinking about this the wrong way?

Regards,
Heinrich

On 31 Jul 2014, at 3:27 PM, Szczepan Faber <[hidden email]> wrote:

Hey,

I've started speccing improvements to conflict resolution here:
https://github.com/gradle/gradle/blob/master/design-docs/conflict-resolution-spec.md
(there's a big DSL mock-up down the bottom).

It would be good to start discussing the DSL additions here on the
mailing list. Please give feedback about the DSL or the use cases,
etc. I haven't decided myself on the DSL :) I'll keep you in loop on
how the design goes.

Cheers!
--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

  http://xircles.codehaus.org/manage_email




**********************************************************************
This email and all content are subject to the following disclaimer

http://content.momentum.co.za/content/legal/disclaimer_email.htm

**********************************************************************

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email





--
Szczepan Faber
Core dev@gradle; Founder@mockito



--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email




--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
CTO Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: improvements to conflict resolution

Daz DeBoer-2
In reply to this post by Szczepan Faber



On Fri, Aug 1, 2014 at 5:18 AM, Szczepan Faber <[hidden email]> wrote:

 b) I would add some higher level api that would use a). For example:

 dependencies {
   components {
     modules("com.google.collections:google-collections").deprecatedBy("com.google.guava:guava")
     //or replacedBy(), supersededBy()
   }
 }

Thoughts?


We're designing a similar DSL for targeting version-selection rules to a particular module or set of modules (https://github.com/gradle/gradle/blob/master/design-docs/component-metadata.md#story-build-script-targets-versionselection-rule-to-particular-module).

I'm not sold on that DSL, but we do need a way to say "this block applies to _all_ modules".

I guess we can either choose to use a string-based selector for choosing the modules that apply:

dependencies {
    components/versions/some-other-rule-type {
        modules('*') {
        }
        modules('com.google.guava:*') {
        }
        modules('com.google.guava:guava') {
        }
    }

Or use explicit parameters:

dependencies {
    components/versions/some-other-rule-type {
        allModules {
        }
        group('com.google.guava') {
        }
        group('com.google.guave').name('guava') {
        }
    }

I think I prefer the first approach. But we should definitely aim for consistency here.
-- 

Darrell (Daz) DeBoer
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: improvements to conflict resolution

Szczepan Faber
Thanks Daz for input! Is the implementation of version-selection rules
happening soon?

It looks like that we're getting to consensus regarding the API:

dependencies {
    components {
        modules(moduleSelectorNotation) { detailsObject ->
          detailsObject.replacedBy(moduleSelectorNotation)
        }
    }
}

1. I'm curious if there is a plan to align dependency resolve rules
and version selection rules in some way?
2. Do we want to use an existing imperative API
"dependencies.components.eachComponent" as a way to pass down the
'replacedBy' declarations to the resolution algorithm? E.g.
declarative data 'dependencies.components.modules' would form one of
the 'eachComponent' rules behind the hood. Based on Adam's feedback, I
assume that we _don't_ want to do this.

Cheers!

On Sun, Aug 3, 2014 at 11:09 PM, Daz DeBoer
<[hidden email]> wrote:

>
>
>
> On Fri, Aug 1, 2014 at 5:18 AM, Szczepan Faber <[hidden email]> wrote:
>>
>>
>>  b) I would add some higher level api that would use a). For example:
>>
>>  dependencies {
>>    components {
>>
>> modules("com.google.collections:google-collections").deprecatedBy("com.google.guava:guava")
>>      //or replacedBy(), supersededBy()
>>    }
>>  }
>>
>> Thoughts?
>>
>
> We're designing a similar DSL for targeting version-selection rules to a
> particular module or set of modules
> (https://github.com/gradle/gradle/blob/master/design-docs/component-metadata.md#story-build-script-targets-versionselection-rule-to-particular-module).
>
> I'm not sold on that DSL, but we do need a way to say "this block applies to
> _all_ modules".
>
> I guess we can either choose to use a string-based selector for choosing the
> modules that apply:
>
> dependencies {
>     components/versions/some-other-rule-type {
>         modules('*') {
>         }
>         modules('com.google.guava:*') {
>         }
>         modules('com.google.guava:guava') {
>         }
>     }
>
> Or use explicit parameters:
>
> dependencies {
>     components/versions/some-other-rule-type {
>         allModules {
>         }
>         group('com.google.guava') {
>         }
>         group('com.google.guave').name('guava') {
>         }
>     }
>
> I think I prefer the first approach. But we should definitely aim for
> consistency here.
> --
>
> Darrell (Daz) DeBoer
> http://www.gradleware.com



--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: improvements to conflict resolution

Lóránt Pintér
Hi,

With eachDependency { … } it was not possible to replace an external dependency with a project dependency. Will this be possible with this notation? It would be awesome if this was possible.

-- 
Lóránt

On Monday 4 August 2014 at 11:30, Szczepan Faber wrote:

Thanks Daz for input! Is the implementation of version-selection rules
happening soon?

It looks like that we're getting to consensus regarding the API:

dependencies {
components {
modules(moduleSelectorNotation) { detailsObject ->
detailsObject.replacedBy(moduleSelectorNotation)
}
}
}

1. I'm curious if there is a plan to align dependency resolve rules
and version selection rules in some way?
2. Do we want to use an existing imperative API
"dependencies.components.eachComponent" as a way to pass down the
'replacedBy' declarations to the resolution algorithm? E.g.
declarative data 'dependencies.components.modules' would form one of
the 'eachComponent' rules behind the hood. Based on Adam's feedback, I
assume that we _don't_ want to do this.

Cheers!

On Sun, Aug 3, 2014 at 11:09 PM, Daz DeBoer



On Fri, Aug 1, 2014 at 5:18 AM, Szczepan Faber <[hidden email]> wrote:


b) I would add some higher level api that would use a). For example:

dependencies {
components {

modules("com.google.collections:google-collections").deprecatedBy("com.google.guava:guava")
//or replacedBy(), supersededBy()
}
}

Thoughts?

We're designing a similar DSL for targeting version-selection rules to a
particular module or set of modules

I'm not sold on that DSL, but we do need a way to say "this block applies to
_all_ modules".

I guess we can either choose to use a string-based selector for choosing the
modules that apply:

dependencies {
components/versions/some-other-rule-type {
modules('*') {
}
modules('com.google.guava:*') {
}
modules('com.google.guava:guava') {
}
}

Or use explicit parameters:

dependencies {
components/versions/some-other-rule-type {
allModules {
}
group('com.google.guava') {
}
group('com.google.guave').name('guava') {
}
}

I think I prefer the first approach. But we should definitely aim for
consistency here.
--

Darrell (Daz) DeBoer



--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: improvements to conflict resolution

Daz DeBoer-2
Hi Lorant
There's quite a bit of work involved to get to the point where we can substitute a project dependency for an external dependency. We've spent some time discussing this, but it's not something we're currently working on.
Daz


On Mon, Aug 4, 2014 at 6:03 PM, Lóránt Pintér <[hidden email]> wrote:
Hi,

With eachDependency { … } it was not possible to replace an external dependency with a project dependency. Will this be possible with this notation? It would be awesome if this was possible.

-- 
Lóránt

On Monday 4 August 2014 at 11:30, Szczepan Faber wrote:

Thanks Daz for input! Is the implementation of version-selection rules
happening soon?

It looks like that we're getting to consensus regarding the API:

dependencies {
components {
modules(moduleSelectorNotation) { detailsObject ->
detailsObject.replacedBy(moduleSelectorNotation)
}
}
}

1. I'm curious if there is a plan to align dependency resolve rules
and version selection rules in some way?
2. Do we want to use an existing imperative API
"dependencies.components.eachComponent" as a way to pass down the
'replacedBy' declarations to the resolution algorithm? E.g.
declarative data 'dependencies.components.modules' would form one of
the 'eachComponent' rules behind the hood. Based on Adam's feedback, I
assume that we _don't_ want to do this.

Cheers!

On Sun, Aug 3, 2014 at 11:09 PM, Daz DeBoer



On Fri, Aug 1, 2014 at 5:18 AM, Szczepan Faber <[hidden email]> wrote:


b) I would add some higher level api that would use a). For example:

dependencies {
components {

modules("com.google.collections:google-collections").deprecatedBy("com.google.guava:guava")
//or replacedBy(), supersededBy()
}
}

Thoughts?

We're designing a similar DSL for targeting version-selection rules to a
particular module or set of modules

I'm not sold on that DSL, but we do need a way to say "this block applies to
_all_ modules".

I guess we can either choose to use a string-based selector for choosing the
modules that apply:

dependencies {
components/versions/some-other-rule-type {
modules('*') {
}
modules('com.google.guava:*') {
}
modules('com.google.guava:guava') {
}
}

Or use explicit parameters:

dependencies {
components/versions/some-other-rule-type {
allModules {
}
group('com.google.guava') {
}
group('com.google.guave').name('guava') {
}
}

I think I prefer the first approach. But we should definitely aim for
consistency here.
--

Darrell (Daz) DeBoer



--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:





--
Darrell (Daz) DeBoer
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: improvements to conflict resolution

Adam Murdoch
In reply to this post by Lóránt Pintér

On 5 Aug 2014, at 10:03 am, Lóránt Pintér <[hidden email]> wrote:

Hi,

With eachDependency { … } it was not possible to replace an external dependency with a project dependency. Will this be possible with this notation?

Not initially, but that’s the eventual plan. It’s not quite so simple because external -> project dependency replacement also need the task graph to be modified, whereas external -> external replacement does not.


It would be awesome if this was possible.

-- 
Lóránt

On Monday 4 August 2014 at 11:30, Szczepan Faber wrote:

Thanks Daz for input! Is the implementation of version-selection rules
happening soon?

It looks like that we're getting to consensus regarding the API:

dependencies {
components {
modules(moduleSelectorNotation) { detailsObject ->
detailsObject.replacedBy(moduleSelectorNotation)
}
}
}

1. I'm curious if there is a plan to align dependency resolve rules
and version selection rules in some way?
2. Do we want to use an existing imperative API
"dependencies.components.eachComponent" as a way to pass down the
'replacedBy' declarations to the resolution algorithm? E.g.
declarative data 'dependencies.components.modules' would form one of
the 'eachComponent' rules behind the hood. Based on Adam's feedback, I
assume that we _don't_ want to do this.

Cheers!

On Sun, Aug 3, 2014 at 11:09 PM, Daz DeBoer



On Fri, Aug 1, 2014 at 5:18 AM, Szczepan Faber <[hidden email]> wrote:


b) I would add some higher level api that would use a). For example:

dependencies {
components {

modules("com.google.collections:google-collections").deprecatedBy("com.google.guava:guava")
//or replacedBy(), supersededBy()
}
}

Thoughts?

We're designing a similar DSL for targeting version-selection rules to a
particular module or set of modules

I'm not sold on that DSL, but we do need a way to say "this block applies to
_all_ modules".

I guess we can either choose to use a string-based selector for choosing the
modules that apply:

dependencies {
components/versions/some-other-rule-type {
modules('*') {
}
modules('com.google.guava:*') {
}
modules('com.google.guava:guava') {
}
}

Or use explicit parameters:

dependencies {
components/versions/some-other-rule-type {
allModules {
}
group('com.google.guava') {
}
group('com.google.guave').name('guava') {
}
}

I think I prefer the first approach. But we should definitely aim for
consistency here.
--

Darrell (Daz) DeBoer



--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:




--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
CTO Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: improvements to conflict resolution

Adam Murdoch
In reply to this post by Szczepan Faber

On 4 Aug 2014, at 7:30 pm, Szczepan Faber <[hidden email]> wrote:

Thanks Daz for input! Is the implementation of version-selection rules
happening soon?

It looks like that we're getting to consensus regarding the API:

dependencies {
   components {
       modules(moduleSelectorNotation) { detailsObject ->
         detailsObject.replacedBy(moduleSelectorNotation)

The argument to replacedBy() needs to be a module id, not a module selector - you can’t replace something with 'myorg:*’ because we have no idea what modules there are in ‘myorg’. At least, we don’t yet.

I’m definitely not sold on `modules(‘*’) { }` being better than `all { }`

       }
   }
}

1. I'm curious if there is a plan to align dependency resolve rules
and version selection rules in some way?

Yes. Whatever pattern we use to target things for the component rules, we should use the same pattern for the other things.

2. Do we want to use an existing imperative API
"dependencies.components.eachComponent" as a way to pass down the
'replacedBy' declarations to the resolution algorithm? E.g.
declarative data 'dependencies.components.modules' would form one of
the 'eachComponent' rules behind the hood. Based on Adam's feedback, I
assume that we _don't_ want to do this.

I think it’s still an open question as to whether we use the component (instance) meta-data rules or introduce a new type of meta data for streams of work and how they map to modules over time.

If we decide to add this to component meta-data then we would use whatever DSL happens to be available for defining the component meta-data, which happens to be `dependencies.components.eachComponent`. We’d later add the modules(_) { … } convenience DSL.


Cheers!

On Sun, Aug 3, 2014 at 11:09 PM, Daz DeBoer
<[hidden email]> wrote:



On Fri, Aug 1, 2014 at 5:18 AM, Szczepan Faber <[hidden email]> wrote:


b) I would add some higher level api that would use a). For example:

dependencies {
  components {

modules("com.google.collections:google-collections").deprecatedBy("com.google.guava:guava")
    //or replacedBy(), supersededBy()
  }
}

Thoughts?


We're designing a similar DSL for targeting version-selection rules to a
particular module or set of modules
(https://github.com/gradle/gradle/blob/master/design-docs/component-metadata.md#story-build-script-targets-versionselection-rule-to-particular-module).

I'm not sold on that DSL, but we do need a way to say "this block applies to
_all_ modules".

I guess we can either choose to use a string-based selector for choosing the
modules that apply:

dependencies {
   components/versions/some-other-rule-type {
       modules('*') {
       }
       modules('com.google.guava:*') {
       }
       modules('com.google.guava:guava') {
       }
   }

Or use explicit parameters:

dependencies {
   components/versions/some-other-rule-type {
       allModules {
       }
       group('com.google.guava') {
       }
       group('com.google.guave').name('guava') {
       }
   }

I think I prefer the first approach. But we should definitely aim for
consistency here.
--

Darrell (Daz) DeBoer
http://www.gradleware.com



--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email




--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
CTO Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: improvements to conflict resolution

Szczepan Faber
> The argument to replacedBy() needs to be a module id, not a module selector
> - you can't replace something with 'myorg:*' because we have no idea what
> modules there are in 'myorg'. At least, we don't yet.

Ok. Can you elaborate why we can't replace something with 'myorg:*'?
I'm curious, why cannot I say that: please replace module A with any
module found in the graph that matches some spec?

> I'm definitely not sold on `modules('*') { }` being better than `all { }`

Sure, tell us what's your preference regarding the dsl.

> 1. I'm curious if there is a plan to align dependency resolve rules
> and version selection rules in some way?
>
>
> Yes. Whatever pattern we use to target things for the component rules, we
> should use the same pattern for the other things.

That would be nice. It's getting hard to grasp all kinds of rules
applicable in various parts of the model.

We also have a strategy that there are a bunch of 'listener'
interfaces that can be directly added to Gradle instance. It was
somewhat nice that a single method (Gradle.addListener) can be used
and documents the listeners for extending Gradle's behavior. Not sure
how this kind of strategy would have worked for all kinds of rules
that alter dependency resolution but nevertheless I wanted to share
this thought.

> 2. Do we want to use an existing imperative API
> "dependencies.components.eachComponent" as a way to pass down the
> 'replacedBy' declarations to the resolution algorithm? E.g.
> declarative data 'dependencies.components.modules' would form one of
> the 'eachComponent' rules behind the hood. Based on Adam's feedback, I
> assume that we _don't_ want to do this.
>
>
> I think it's still an open question as to whether we use the component
> (instance) meta-data rules or introduce a new type of meta data for streams
> of work and how they map to modules over time.
>
> If we decide to add this to component meta-data then we would use whatever
> DSL happens to be available for defining the component meta-data, which
> happens to be `dependencies.components.eachComponent`. We'd later add the
> modules(_) { ... } convenience DSL.

To me, it makes sense the reuse an existing mechanism
("dependencies.components.eachComponent") + convenience DSL. However,
I need to think about it some more.

I did spike with eachComponent initially. I had some problem with the
fact that 'metadata.resolve()' needed to be executed earlier than
before. However, that was probably something manageable.

> Cheers!
>
> On Sun, Aug 3, 2014 at 11:09 PM, Daz DeBoer
> <[hidden email]> wrote:
>
>
>
>
> On Fri, Aug 1, 2014 at 5:18 AM, Szczepan Faber <[hidden email]> wrote:
>
>
>
> b) I would add some higher level api that would use a). For example:
>
> dependencies {
>   components {
>
> modules("com.google.collections:google-collections").deprecatedBy("com.google.guava:guava")
>     //or replacedBy(), supersededBy()
>   }
> }
>
> Thoughts?
>
>
> We're designing a similar DSL for targeting version-selection rules to a
> particular module or set of modules
> (https://github.com/gradle/gradle/blob/master/design-docs/component-metadata.md#story-build-script-targets-versionselection-rule-to-particular-module).
>
> I'm not sold on that DSL, but we do need a way to say "this block applies to
> _all_ modules".
>
> I guess we can either choose to use a string-based selector for choosing the
> modules that apply:
>
> dependencies {
>    components/versions/some-other-rule-type {
>        modules('*') {
>        }
>        modules('com.google.guava:*') {
>        }
>        modules('com.google.guava:guava') {
>        }
>    }
>
> Or use explicit parameters:
>
> dependencies {
>    components/versions/some-other-rule-type {
>        allModules {
>        }
>        group('com.google.guava') {
>        }
>        group('com.google.guave').name('guava') {
>        }
>    }
>
> I think I prefer the first approach. But we should definitely aim for
> consistency here.
> --
>
> Darrell (Daz) DeBoer
> http://www.gradleware.com
>
>
>
>
> --
> Szczepan Faber
> Core dev@gradle; Founder@mockito
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>
>
>
> --
> Adam Murdoch
> Gradle Co-founder
> http://www.gradle.org
> CTO Gradleware Inc. - Gradle Training, Support, Consulting
> http://www.gradleware.com
>
>
>



--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: improvements to conflict resolution

Adam Murdoch

On 5 Aug 2014, at 7:29 pm, Szczepan Faber <[hidden email]> wrote:

The argument to replacedBy() needs to be a module id, not a module selector
- you can't replace something with 'myorg:*' because we have no idea what
modules there are in 'myorg'. At least, we don't yet.

Ok. Can you elaborate why we can't replace something with 'myorg:*'?
I'm curious, why cannot I say that: please replace module A with any
module found in the graph that matches some spec?

Because we’re stating facts about what happened to the module, not expressing what to do with the graph. You can’t say ‘please replace’, you can only say ‘was replaced by’. So something like replacedBy(‘org:*’) means: this module was replaced by every module with organisation ‘org’.

When A is replaced by multiple modules B and C, then whenever we see A in the graph, we can only replace it by a compatible version of A or a compatible version of both B and C. We can’t choose only B or only C. If we choose one we also have to include the other. So, if we have A and B in the graph, we have to end up with B and C in the result.

Given this, when A is replaced by every module in organisation ‘org’, then if we see A and org:B in the graph, then we have to end up with every the modules on ‘org' in the result. Which is fine, if that’s what you want, except we currently have no good way of knowing what all the modules of org are.

Net result is that you can only call replacedBy() with a module id. If you want to replace something with all the modules in an organisation, you’ll have to list them out in your rule - something has to, and it’s your problem at the moment. We might at some later point add some way to make this more convenient.



I'm definitely not sold on `modules('*') { }` being better than `all { }`

Sure, tell us what's your preference regarding the dsl.

1. I'm curious if there is a plan to align dependency resolve rules
and version selection rules in some way?


Yes. Whatever pattern we use to target things for the component rules, we
should use the same pattern for the other things.

That would be nice. It's getting hard to grasp all kinds of rules
applicable in various parts of the model.

There’s only 2 things really:

- Dependency declarations. These are the edges in the graph. Dependency resolve rules operate on these. These are just criteria to use to choose a particular component, nothing more. Currently this means (group, module, version-selector).

- Component meta-data. The components are the nodes in the resolved graph. Component meta-data rules operate on these. This is just bunch of facts about the component, its dependencies, its history, what it conflicts with, what it is compatible with, and so on.

The goal is to keep these separate, so we only state things about dependencies (the edges) in dependency resolve rules, and only state things about components in component meta-data rules, so that we can reuse the component meta data in places other than just resolving dependency edges.


--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
CTO Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: improvements to conflict resolution

Szczepan Faber
> Because we're stating facts about what happened to the module, not
> expressing what to do with the graph. You can't say 'please replace', you
> can only say 'was replaced by'. So something like replacedBy('org:*') means:
> this module was replaced by every module with organisation 'org'.

What about a fact like this: "Module X was replaced by a bunch of
modules that match certain naming convention".

Anyhow, I'm fine with replacing with specific module id. All real use
cases I have now are happy with specific module replacement.

> When A is replaced by multiple modules B and C, then whenever we see A in
> the graph, we can only replace it by a compatible version of A or a
> compatible version of both B and C. We can't choose only B or only C. If we
> choose one we also have to include the other. So, if we have A and B in the
> graph, we have to end up with B and C in the result.
>
> Given this, when A is replaced by every module in organization 'org', then
> if we see A and org:B in the graph, then we have to end up with every the
> modules on 'org' in the result. Which is fine, if that's what you want,
> except we currently have no good way of knowing what all the modules of org
> are.
>
> Net result is that you can only call replacedBy() with a module id. If you
> want to replace something with all the modules in an organisation, you'll
> have to list them out in your rule - something has to, and it's your problem
> at the moment. We might at some later point add some way to make this more
> convenient.
>
>
>
> I'm definitely not sold on `modules('*') { }` being better than `all { }`
>
>
> Sure, tell us what's your preference regarding the dsl.
>
> 1. I'm curious if there is a plan to align dependency resolve rules
> and version selection rules in some way?
>
>
> Yes. Whatever pattern we use to target things for the component rules, we
> should use the same pattern for the other things.
>
>
> That would be nice. It's getting hard to grasp all kinds of rules
> applicable in various parts of the model.
>
>
> There's only 2 things really:
>
> - Dependency declarations. These are the edges in the graph. Dependency
> resolve rules operate on these. These are just criteria to use to choose a
> particular component, nothing more. Currently this means (group, module,
> version-selector).
>
> - Component meta-data. The components are the nodes in the resolved graph.
> Component meta-data rules operate on these. This is just bunch of facts
> about the component, its dependencies, its history, what it conflicts with,
> what it is compatible with, and so on.
>
> The goal is to keep these separate, so we only state things about
> dependencies (the edges) in dependency resolve rules, and only state things
> about components in component meta-data rules, so that we can reuse the
> component meta data in places other than just resolving dependency edges.
>
>
> --
> Adam Murdoch
> Gradle Co-founder
> http://www.gradle.org
> CTO Gradleware Inc. - Gradle Training, Support, Consulting
> http://www.gradleware.com
>
>
>



--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: improvements to conflict resolution

Daz DeBoer-2
In reply to this post by Adam Murdoch



On Mon, Aug 4, 2014 at 10:16 PM, Adam Murdoch <[hidden email]> wrote:

On 4 Aug 2014, at 7:30 pm, Szczepan Faber <[hidden email]> wrote:

Thanks Daz for input! Is the implementation of version-selection rules
happening soon?

It looks like that we're getting to consensus regarding the API:

dependencies {
   components {
       modules(moduleSelectorNotation) { detailsObject ->
         detailsObject.replacedBy(moduleSelectorNotation)

The argument to replacedBy() needs to be a module id, not a module selector - you can’t replace something with 'myorg:*’ because we have no idea what modules there are in ‘myorg’. At least, we don’t yet.

I’m definitely not sold on `modules(‘*’) { }` being better than `all { }`

I think if we go with a notation-based approach for targeting a rule (which I like), then it would be good to be consistent.
So option 1:

dependencies {
    versionSelection {
        modules {} // For all modules
        modules('*') {} // Same as above
        modules('org:*') {}
        modules('org:my-module') {}
    }
}

And option 2:

dependencies {
     versionSelection {
         all {}
         group('org') {}
         group('org').module('my-module') {}
     }
}

I'm notoriously bad at coming up with nice DSL's so I'd be very happy with alternative suggestions.
Gary will be working on this soon, so it would be good to come up with some sort of consensus.
Daz


       }
   }
}

1. I'm curious if there is a plan to align dependency resolve rules
and version selection rules in some way?

Yes. Whatever pattern we use to target things for the component rules, we should use the same pattern for the other things.

2. Do we want to use an existing imperative API
"dependencies.components.eachComponent" as a way to pass down the
'replacedBy' declarations to the resolution algorithm? E.g.
declarative data 'dependencies.components.modules' would form one of
the 'eachComponent' rules behind the hood. Based on Adam's feedback, I
assume that we _don't_ want to do this.

I think it’s still an open question as to whether we use the component (instance) meta-data rules or introduce a new type of meta data for streams of work and how they map to modules over time.

If we decide to add this to component meta-data then we would use whatever DSL happens to be available for defining the component meta-data, which happens to be `dependencies.components.eachComponent`. We’d later add the modules(_) { … } convenience DSL.


Cheers!

On Sun, Aug 3, 2014 at 11:09 PM, Daz DeBoer
<[hidden email]> wrote:



On Fri, Aug 1, 2014 at 5:18 AM, Szczepan Faber <[hidden email]> wrote:


b) I would add some higher level api that would use a). For example:

dependencies {
  components {

modules("com.google.collections:google-collections").deprecatedBy("com.google.guava:guava")
    //or replacedBy(), supersededBy()
  }
}

Thoughts?


We're designing a similar DSL for targeting version-selection rules to a
particular module or set of modules
(https://github.com/gradle/gradle/blob/master/design-docs/component-metadata.md#story-build-script-targets-versionselection-rule-to-particular-module).

I'm not sold on that DSL, but we do need a way to say "this block applies to
_all_ modules".

I guess we can either choose to use a string-based selector for choosing the
modules that apply:

dependencies {
   components/versions/some-other-rule-type {
       modules('*') {
       }
       modules('com.google.guava:*') {
       }
       modules('com.google.guava:guava') {
       }
   }

Or use explicit parameters:

dependencies {
   components/versions/some-other-rule-type {
       allModules {
       }
       group('com.google.guava') {
       }
       group('com.google.guave').name('guava') {
       }
   }

I think I prefer the first approach. But we should definitely aim for
consistency here.
--

Darrell (Daz) DeBoer
http://www.gradleware.com



--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email




--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
CTO Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com






--
Darrell (Daz) DeBoer
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: improvements to conflict resolution

Adam Murdoch
In reply to this post by Szczepan Faber

On 5 Aug 2014, at 8:26 pm, Szczepan Faber <[hidden email]> wrote:

Because we're stating facts about what happened to the module, not
expressing what to do with the graph. You can't say 'please replace', you
can only say 'was replaced by'. So something like replacedBy('org:*') means:
this module was replaced by every module with organisation 'org'.

What about a fact like this: "Module X was replaced by a bunch of
modules that match certain naming convention”.

This doesn’t really help. How do you know which modules to match against? Some of them will be in the graph, but not necessarily all of them.


--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
CTO Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: improvements to conflict resolution

Adam Murdoch
In reply to this post by Daz DeBoer-2

On 6 Aug 2014, at 9:25 am, Daz DeBoer <[hidden email]> wrote:




On Mon, Aug 4, 2014 at 10:16 PM, Adam Murdoch <[hidden email]> wrote:

On 4 Aug 2014, at 7:30 pm, Szczepan Faber <[hidden email]> wrote:

Thanks Daz for input! Is the implementation of version-selection rules
happening soon?

It looks like that we're getting to consensus regarding the API:

dependencies {
   components {
       modules(moduleSelectorNotation) { detailsObject ->
         detailsObject.replacedBy(moduleSelectorNotation)

The argument to replacedBy() needs to be a module id, not a module selector - you can’t replace something with 'myorg:*’ because we have no idea what modules there are in ‘myorg’. At least, we don’t yet.

I’m definitely not sold on `modules(‘*’) { }` being better than `all { }`

I think if we go with a notation-based approach for targeting a rule (which I like), then it would be good to be consistent.
So option 1:

dependencies {
    versionSelection {
        modules {} // For all modules
        modules('*') {} // Same as above
        modules('org:*') {}
        modules('org:my-module') {}
    }
}

And option 2:

dependencies {
     versionSelection {
         all {}
         group('org') {}
         group('org').module('my-module') {}
     }
}

There’s a slight semantic difference between the two options, not just syntax: modules() { … } would implicitly match any component instance that is-a module - that is something that comes from an ivy or maven repo, whereas all { … } would match any component instance regardless of where it came from.


--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
CTO Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: improvements to conflict resolution

Szczepan Faber
Thanks everyone for feedback. I conclude that the incubating API for now can be:

dependencies {
    components {
        module(moduleSelectorNotation).replacedBy(moduleNotation)
        module(moduleSelectorNotation) {
           replacedBy(... moduleNotation)
           setReplacedBy(... moduleNotation)
        }
    }
}

1. Initially, both moduleSelectorNotation and moduleNotation will
support the same inputs:
 - String like "com.google.collections:google-collections"
 - ModuleIdentifier instance
 - Map with group and name
2. replacedBy() appends new replacement target. There can be multiple
targets because a module can be replaced by a set of modules.
3. replacedBy() accepts var args, e.g: replacedBy("org:foo1", "org:foo2")
4. setReplacedBy() replaces the currently set replacement target

Cheers!

On Wed, Aug 6, 2014 at 3:14 AM, Adam Murdoch
<[hidden email]> wrote:

>
> On 6 Aug 2014, at 9:25 am, Daz DeBoer <[hidden email]> wrote:
>
>
>
>
> On Mon, Aug 4, 2014 at 10:16 PM, Adam Murdoch <[hidden email]>
> wrote:
>>
>>
>> On 4 Aug 2014, at 7:30 pm, Szczepan Faber <[hidden email]> wrote:
>>
>> Thanks Daz for input! Is the implementation of version-selection rules
>> happening soon?
>>
>> It looks like that we're getting to consensus regarding the API:
>>
>> dependencies {
>>    components {
>>        modules(moduleSelectorNotation) { detailsObject ->
>>          detailsObject.replacedBy(moduleSelectorNotation)
>>
>>
>> The argument to replacedBy() needs to be a module id, not a module
>> selector - you can't replace something with 'myorg:*' because we have no
>> idea what modules there are in 'myorg'. At least, we don't yet.
>>
>> I'm definitely not sold on `modules('*') { }` being better than `all { }`
>
>
> I think if we go with a notation-based approach for targeting a rule (which
> I like), then it would be good to be consistent.
> So option 1:
>
> dependencies {
>     versionSelection {
>         modules {} // For all modules
>         modules('*') {} // Same as above
>         modules('org:*') {}
>         modules('org:my-module') {}
>     }
> }
>
> And option 2:
>
> dependencies {
>      versionSelection {
>          all {}
>          group('org') {}
>          group('org').module('my-module') {}
>      }
> }
>
>
> There's a slight semantic difference between the two options, not just
> syntax: modules() { ... } would implicitly match any component instance that
> is-a module - that is something that comes from an ivy or maven repo,
> whereas all { ... } would match any component instance regardless of where it
> came from.
>
>
> --
> Adam Murdoch
> Gradle Co-founder
> http://www.gradle.org
> CTO Gradleware Inc. - Gradle Training, Support, Consulting
> http://www.gradleware.com
>
>
>



--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Loading...