Quantcast

component replacement declarations

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

component replacement declarations

Szczepan Faber
Upcoming Gradle 2.2 contains new incubating component replacement rules:

dependencies {
    components {
        module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
    }
}

See more in http://www.gradle.org/docs/nightly/release-notes. This
declaration is used during conflict resolution, Gradle will prevent
both collections and guava appear in the same dependency tree,
preferring any version of guava over every version of collections.
Most importantly, if the dependency tree _only_ contains collections
it will _not_ be replaced (because there is no conflict). Multiple
modules can have the same target replacement. However, we don't
support (yet) having single module replaced by multiple modules.

The full DSL in the current form, based on our previous discussion on
the mailing list. I'm starting new email thread on purpose (for good
or bad, let's see whether it helps).

dependencies {
    components {
    //declaring replacement:
        module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
        module(someModuleIdentifier).replacedBy("com.google.guava:guava")

        //querying for the replacement target:
        ModuleIdentifier id =
module("com.google.collections:google-collections").getReplacedBy()

        //querying for the replacement source:
        ModuleIdentifier id =
module("com.google.collections:google-collections").getId()
    }
}

Javadoc (for interface names, etc.):
http://www.gradle.org/docs/nightly/javadoc/org/gradle/api/artifacts/dsl/ComponentMetadataHandler.html#module(java.lang.Object)

This DSL can grow to accommodate features like:
a) replacing single module with a set of modules. I'd love to have
this. I also want to make incremental progress.
b) other component module metadata (e.g. releasable units, impl module
consistent with api module). I'd love to have this, too.

Let's confirm this API and/or make changes to it before 2.2 release.
Other related APIs that we should have in mind (for consistency):

1. component selection rules:

configurations.conf.resolutionStrategy {
    componentSelection {
        all { ComponentSelection selection ->
            if (selection.candidate.group == 'org.sample') {
                selection.reject("rejecting experimental")
            }
        }
        module("org.sample:api") { ComponentSelection selection ->
            if (selection.candidate.version == "1.1") {
                selection.reject("known bad version")
            }
        }
    }
}

2. component metadata rules:

dependencies {
    components {
        eachComponent { ComponentMetadataDetails details,
IvyModuleDescriptor ivyModule ->
            if (details.id.group == 'my.org' && ivyModule.branch == 'testing') {
                details.changing = true
            }
        }
    }
}

Cheers!
--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: component replacement declarations

Daz DeBoer-2
Thanks Szczepan!

I've finally found the time to get fully across this feature. I haven't yet looked into the implementation.

Here's some initial feedback on the model.

1. We've named this feature 'component replacements', but the modelling is done at the "module" level (ComponentModuleMetadata).  I don't think this is ideal: I'd like to think about a 'replacement' being modelled at the component level ("org:name:version" instead of "org:name"). In most cases we'll consider all components with the same 'org:name as having the same replacements, and the DSL would make it convenient to define this. But the underlying model should associate 'replacements' with a component, I think.
2. Eventually, we'd like a published component to contain replacement information. For this to be convenient, we'd normally publish a component and say it 'replaces' or 'supercedes' a set of other components. I think it might be helpful to model the relationship in this way, using 'replaces' rather than 'replaced by'.

And on the DSL:
The DSLs for all of our 'rules' that apply to dependency resolution are converging on a pattern that will hopefully allow them to be migrated into the new configuration rule infrastructure at some stage. The pattern is:

model-rule-type {
    eachXXX { SomeMutableType mutable[, OtherType immutableInput]* ->
    }
    module("group:name") { SomeMutableType mutable[, OtherType immutableInput]* ->
    }
}

Exceptions:
- For component selection rules, 'all' is used instead of 'eachComponentSelection'.
     - I'm thinking this works better as a partner method to 'module()'
- For component metadata rules, the module() method doesn't exist yet.
- For dependency resolve rules, the 'model-rule-type' containing element doesn't exist.

If we stick with the same pattern for component replacements, we'd have:

dependencies {
    components {
        module("com.google.guava:guava") { ComponentSubstitutionDetails component ->
            component.replacesModule("com.google.collections:google-collections")
        }
    }
}

That's it for now. Thanks for restarting the discussion!
Daz

On Wed, Sep 24, 2014 at 3:31 AM, Szczepan Faber <[hidden email]> wrote:
Upcoming Gradle 2.2 contains new incubating component replacement rules:

dependencies {
    components {
        module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
    }
}

See more in http://www.gradle.org/docs/nightly/release-notes. This
declaration is used during conflict resolution, Gradle will prevent
both collections and guava appear in the same dependency tree,
preferring any version of guava over every version of collections.
Most importantly, if the dependency tree _only_ contains collections
it will _not_ be replaced (because there is no conflict). Multiple
modules can have the same target replacement. However, we don't
support (yet) having single module replaced by multiple modules.

The full DSL in the current form, based on our previous discussion on
the mailing list. I'm starting new email thread on purpose (for good
or bad, let's see whether it helps).

dependencies {
    components {
    //declaring replacement:
        module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
        module(someModuleIdentifier).replacedBy("com.google.guava:guava")

        //querying for the replacement target:
        ModuleIdentifier id =
module("com.google.collections:google-collections").getReplacedBy()

        //querying for the replacement source:
        ModuleIdentifier id =
module("com.google.collections:google-collections").getId()
    }
}

Javadoc (for interface names, etc.):
http://www.gradle.org/docs/nightly/javadoc/org/gradle/api/artifacts/dsl/ComponentMetadataHandler.html#module(java.lang.Object)

This DSL can grow to accommodate features like:
a) replacing single module with a set of modules. I'd love to have
this. I also want to make incremental progress.
b) other component module metadata (e.g. releasable units, impl module
consistent with api module). I'd love to have this, too.

Let's confirm this API and/or make changes to it before 2.2 release.
Other related APIs that we should have in mind (for consistency):

1. component selection rules:

configurations.conf.resolutionStrategy {
    componentSelection {
        all { ComponentSelection selection ->
            if (selection.candidate.group == 'org.sample') {
                selection.reject("rejecting experimental")
            }
        }
        module("org.sample:api") { ComponentSelection selection ->
            if (selection.candidate.version == "1.1") {
                selection.reject("known bad version")
            }
        }
    }
}

2. component metadata rules:

dependencies {
    components {
        eachComponent { ComponentMetadataDetails details,
IvyModuleDescriptor ivyModule ->
            if (details.id.group == 'my.org' && ivyModule.branch == 'testing') {
                details.changing = true
            }
        }
    }
}

Cheers!
--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email





--
Darrell (Daz) DeBoer
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: component replacement declarations

Szczepan Faber
Awesome feedback! Let's skype on it today.

On Wed, Sep 24, 2014 at 5:50 PM, Daz DeBoer
<[hidden email]> wrote:

> Thanks Szczepan!
>
> I've finally found the time to get fully across this feature. I haven't yet
> looked into the implementation.
>
> Here's some initial feedback on the model.
>
> 1. We've named this feature 'component replacements', but the modelling is
> done at the "module" level (ComponentModuleMetadata).  I don't think this is
> ideal: I'd like to think about a 'replacement' being modelled at the
> component level ("org:name:version" instead of "org:name"). In most cases
> we'll consider all components with the same 'org:name as having the same
> replacements, and the DSL would make it convenient to define this. But the
> underlying model should associate 'replacements' with a component, I think.
> 2. Eventually, we'd like a published component to contain replacement
> information. For this to be convenient, we'd normally publish a component
> and say it 'replaces' or 'supercedes' a set of other components. I think it
> might be helpful to model the relationship in this way, using 'replaces'
> rather than 'replaced by'.
>
> And on the DSL:
> The DSLs for all of our 'rules' that apply to dependency resolution are
> converging on a pattern that will hopefully allow them to be migrated into
> the new configuration rule infrastructure at some stage. The pattern is:
>
> model-rule-type {
>     eachXXX { SomeMutableType mutable[, OtherType immutableInput]* ->
>     }
>     module("group:name") { SomeMutableType mutable[, OtherType
> immutableInput]* ->
>     }
> }
>
> Exceptions:
> - For component selection rules, 'all' is used instead of
> 'eachComponentSelection'.
>      - I'm thinking this works better as a partner method to 'module()'
> - For component metadata rules, the module() method doesn't exist yet.
> - For dependency resolve rules, the 'model-rule-type' containing element
> doesn't exist.
>
> If we stick with the same pattern for component replacements, we'd have:
>
> dependencies {
>     components {
>         module("com.google.guava:guava") { ComponentSubstitutionDetails
> component ->
>
> component.replacesModule("com.google.collections:google-collections")
>         }
>     }
> }
>
> That's it for now. Thanks for restarting the discussion!
> Daz
>
> On Wed, Sep 24, 2014 at 3:31 AM, Szczepan Faber <[hidden email]> wrote:
>>
>> Upcoming Gradle 2.2 contains new incubating component replacement rules:
>>
>> dependencies {
>>     components {
>>
>> module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
>>     }
>> }
>>
>> See more in http://www.gradle.org/docs/nightly/release-notes. This
>> declaration is used during conflict resolution, Gradle will prevent
>> both collections and guava appear in the same dependency tree,
>> preferring any version of guava over every version of collections.
>> Most importantly, if the dependency tree _only_ contains collections
>> it will _not_ be replaced (because there is no conflict). Multiple
>> modules can have the same target replacement. However, we don't
>> support (yet) having single module replaced by multiple modules.
>>
>> The full DSL in the current form, based on our previous discussion on
>> the mailing list. I'm starting new email thread on purpose (for good
>> or bad, let's see whether it helps).
>>
>> dependencies {
>>     components {
>>     //declaring replacement:
>>
>> module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
>>         module(someModuleIdentifier).replacedBy("com.google.guava:guava")
>>
>>         //querying for the replacement target:
>>         ModuleIdentifier id =
>> module("com.google.collections:google-collections").getReplacedBy()
>>
>>         //querying for the replacement source:
>>         ModuleIdentifier id =
>> module("com.google.collections:google-collections").getId()
>>     }
>> }
>>
>> Javadoc (for interface names, etc.):
>>
>> http://www.gradle.org/docs/nightly/javadoc/org/gradle/api/artifacts/dsl/ComponentMetadataHandler.html#module(java.lang.Object)
>>
>> This DSL can grow to accommodate features like:
>> a) replacing single module with a set of modules. I'd love to have
>> this. I also want to make incremental progress.
>> b) other component module metadata (e.g. releasable units, impl module
>> consistent with api module). I'd love to have this, too.
>>
>> Let's confirm this API and/or make changes to it before 2.2 release.
>> Other related APIs that we should have in mind (for consistency):
>>
>> 1. component selection rules:
>>
>> configurations.conf.resolutionStrategy {
>>     componentSelection {
>>         all { ComponentSelection selection ->
>>             if (selection.candidate.group == 'org.sample') {
>>                 selection.reject("rejecting experimental")
>>             }
>>         }
>>         module("org.sample:api") { ComponentSelection selection ->
>>             if (selection.candidate.version == "1.1") {
>>                 selection.reject("known bad version")
>>             }
>>         }
>>     }
>> }
>>
>> 2. component metadata rules:
>>
>> dependencies {
>>     components {
>>         eachComponent { ComponentMetadataDetails details,
>> IvyModuleDescriptor ivyModule ->
>>             if (details.id.group == 'my.org' && ivyModule.branch ==
>> 'testing') {
>>                 details.changing = true
>>             }
>>         }
>>     }
>> }
>>
>> Cheers!
>> --
>> Szczepan Faber
>> Core dev@gradle; Founder@mockito
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>     http://xircles.codehaus.org/manage_email
>>
>>
>
>
>
> --
> Darrell (Daz) DeBoer
> http://www.gradleware.com



--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: component replacement declarations

Daz DeBoer-2
So I'm battling a bit as to whether we should really be modelling these replacements at the component level (using rules as suggested) or should be designing these at a higher level, on some sort of new model concept.

Reviewing some of the old emails on this, I found this from Adam:

I think it’s still an open question as to whether we use the component (instance) meta-data rules or introduce a new type of meta data for streams of work and how they map to modules over time.
If we decide to add this to component meta-data then we would use whatever DSL happens to be available for defining the component meta-data, which happens to be `dependencies.components.eachComponent`. We’d later add the modules(_) { … } convenience DSL.

So the alternative to component metadata rules will be modelling some new data structure, which we've done in the current implementation with "component module" (ComponentModuleMetadata).  This doesn't quite represent a 'stream of work', but perhaps it could evolve in that way.

So the big question in my mind is do we:
a) Implement 'replaces/replacedBy' at the component instance level, and configure it via component metadata rules
OR
b) Do a bit more work to model component replacements in the context of a 'stream of work' and clearly separate the DSL from the metadata rules.


On Wed, Sep 24, 2014 at 10:41 AM, Szczepan Faber <[hidden email]> wrote:
Awesome feedback! Let's skype on it today.

On Wed, Sep 24, 2014 at 5:50 PM, Daz DeBoer
<[hidden email]> wrote:
> Thanks Szczepan!
>
> I've finally found the time to get fully across this feature. I haven't yet
> looked into the implementation.
>
> Here's some initial feedback on the model.
>
> 1. We've named this feature 'component replacements', but the modelling is
> done at the "module" level (ComponentModuleMetadata).  I don't think this is
> ideal: I'd like to think about a 'replacement' being modelled at the
> component level ("org:name:version" instead of "org:name"). In most cases
> we'll consider all components with the same 'org:name as having the same
> replacements, and the DSL would make it convenient to define this. But the
> underlying model should associate 'replacements' with a component, I think.
> 2. Eventually, we'd like a published component to contain replacement
> information. For this to be convenient, we'd normally publish a component
> and say it 'replaces' or 'supercedes' a set of other components. I think it
> might be helpful to model the relationship in this way, using 'replaces'
> rather than 'replaced by'.
>
> And on the DSL:
> The DSLs for all of our 'rules' that apply to dependency resolution are
> converging on a pattern that will hopefully allow them to be migrated into
> the new configuration rule infrastructure at some stage. The pattern is:
>
> model-rule-type {
>     eachXXX { SomeMutableType mutable[, OtherType immutableInput]* ->
>     }
>     module("group:name") { SomeMutableType mutable[, OtherType
> immutableInput]* ->
>     }
> }
>
> Exceptions:
> - For component selection rules, 'all' is used instead of
> 'eachComponentSelection'.
>      - I'm thinking this works better as a partner method to 'module()'
> - For component metadata rules, the module() method doesn't exist yet.
> - For dependency resolve rules, the 'model-rule-type' containing element
> doesn't exist.
>
> If we stick with the same pattern for component replacements, we'd have:
>
> dependencies {
>     components {
>         module("com.google.guava:guava") { ComponentSubstitutionDetails
> component ->
>
> component.replacesModule("com.google.collections:google-collections")
>         }
>     }
> }
>
> That's it for now. Thanks for restarting the discussion!
> Daz
>
> On Wed, Sep 24, 2014 at 3:31 AM, Szczepan Faber <[hidden email]> wrote:
>>
>> Upcoming Gradle 2.2 contains new incubating component replacement rules:
>>
>> dependencies {
>>     components {
>>
>> module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
>>     }
>> }
>>
>> See more in http://www.gradle.org/docs/nightly/release-notes. This
>> declaration is used during conflict resolution, Gradle will prevent
>> both collections and guava appear in the same dependency tree,
>> preferring any version of guava over every version of collections.
>> Most importantly, if the dependency tree _only_ contains collections
>> it will _not_ be replaced (because there is no conflict). Multiple
>> modules can have the same target replacement. However, we don't
>> support (yet) having single module replaced by multiple modules.
>>
>> The full DSL in the current form, based on our previous discussion on
>> the mailing list. I'm starting new email thread on purpose (for good
>> or bad, let's see whether it helps).
>>
>> dependencies {
>>     components {
>>     //declaring replacement:
>>
>> module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
>>         module(someModuleIdentifier).replacedBy("com.google.guava:guava")
>>
>>         //querying for the replacement target:
>>         ModuleIdentifier id =
>> module("com.google.collections:google-collections").getReplacedBy()
>>
>>         //querying for the replacement source:
>>         ModuleIdentifier id =
>> module("com.google.collections:google-collections").getId()
>>     }
>> }
>>
>> Javadoc (for interface names, etc.):
>>
>> http://www.gradle.org/docs/nightly/javadoc/org/gradle/api/artifacts/dsl/ComponentMetadataHandler.html#module(java.lang.Object)
>>
>> This DSL can grow to accommodate features like:
>> a) replacing single module with a set of modules. I'd love to have
>> this. I also want to make incremental progress.
>> b) other component module metadata (e.g. releasable units, impl module
>> consistent with api module). I'd love to have this, too.
>>
>> Let's confirm this API and/or make changes to it before 2.2 release.
>> Other related APIs that we should have in mind (for consistency):
>>
>> 1. component selection rules:
>>
>> configurations.conf.resolutionStrategy {
>>     componentSelection {
>>         all { ComponentSelection selection ->
>>             if (selection.candidate.group == 'org.sample') {
>>                 selection.reject("rejecting experimental")
>>             }
>>         }
>>         module("org.sample:api") { ComponentSelection selection ->
>>             if (selection.candidate.version == "1.1") {
>>                 selection.reject("known bad version")
>>             }
>>         }
>>     }
>> }
>>
>> 2. component metadata rules:
>>
>> dependencies {
>>     components {
>>         eachComponent { ComponentMetadataDetails details,
>> IvyModuleDescriptor ivyModule ->
>>             if (details.id.group == 'my.org' && ivyModule.branch ==
>> 'testing') {
>>                 details.changing = true
>>             }
>>         }
>>     }
>> }
>>
>> Cheers!
>> --
>> Szczepan Faber
>> Core dev@gradle; Founder@mockito
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>     http://xircles.codehaus.org/manage_email
>>
>>
>
>
>
> --
> Darrell (Daz) DeBoer
> http://www.gradleware.com



--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email





--
Darrell (Daz) DeBoer
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: component replacement declarations

Adam Murdoch

On 25 Sep 2014, at 6:12 am, Daz DeBoer <[hidden email]> wrote:

So I'm battling a bit as to whether we should really be modelling these replacements at the component level (using rules as suggested) or should be designing these at a higher level, on some sort of new model concept.

Reviewing some of the old emails on this, I found this from Adam:

I think it’s still an open question as to whether we use the component (instance) meta-data rules or introduce a new type of meta data for streams of work and how they map to modules over time.
If we decide to add this to component meta-data then we would use whatever DSL happens to be available for defining the component meta-data, which happens to be `dependencies.components.eachComponent`. We’d later add the modules(_) { … } convenience DSL.

So the alternative to component metadata rules will be modelling some new data structure, which we've done in the current implementation with "component module" (ComponentModuleMetadata).  This doesn't quite represent a 'stream of work', but perhaps it could evolve in that way.

So the big question in my mind is do we:
a) Implement 'replaces/replacedBy' at the component instance level, and configure it via component metadata rules
OR
b) Do a bit more work to model component replacements in the context of a 'stream of work' and clearly separate the DSL from the metadata rules.

It feels like ultimately this is something we need to know at the component instance level. Given an instance x, we need to know:

1. For each instance y that also appears in the graph, do x and y conflict?
2. When x and y conflict, which one should we select? Do we replace x with y or y with x?
3. When x is replaced by y, which other instances must also be included in the result?
4. When x is replaced by y, which other instances must also be replaced by y?

#3 is for the case where a module is busted up into several others. #4 is for the case where several modules are combined into a single module. This can happen over time (eg groovy 1.x was busted up into pieces from groovy 2.0 and later) and can happen as an aggregation (eg if I choose groovy-all I need to pull all of the groovy modules out of the result). 

So this is the kind of model we want DependencyGraphBuilder to work with. For now, it doesn’t need to be anywhere near this general, as there are performance implications to trying to make this work generally with any arbitrary set of instances.

From the build author’s point of view, we want a different model. We should be able to declare concrete and specific facts in the DSL and use those facts to derive answers to the above questions. We don’t want to expose the above questions in the DSL, at least not yet.

The model we want to aim for is one where we describe the history of a component instance: where did this instance come from?

So, we have a few options for the user model:

1. This is just more stuff you add to the component instance meta-data. Each component instance would describe its full history - all the other component instances that it is newer than (ie replaces). Not literally - you’d use some kind of declarative convenience. You’d use the existing component meta-data rules to attach this history.

2. This is stuff you add to the meta-data about a module/component. We’d use a new module meta-data rules DSL for this.

3. Some combination of the two, where there isn’t any concept of a module/component, but there is a separate convenience DSL to describe the history of all components instances with a particular module id, without requiring the meta-data of each instance.

The big downside of #1 is when there are transitive changes. For example: a was replaced by b and then split into (c,d). Given this, if I end up with a and d in the graph, I need to figure out that a and d conflict and that I need to replace a with c and d.

With #1, we’d need to find some meta-data for an instance of b in order to apply the rule, even though b is never referenced in the graph. So I reckon #1 is out.

From a user’s point of view, there probably isn’t much difference between #2 and #3.




On Wed, Sep 24, 2014 at 10:41 AM, Szczepan Faber <[hidden email]> wrote:
Awesome feedback! Let's skype on it today.

On Wed, Sep 24, 2014 at 5:50 PM, Daz DeBoer
<[hidden email]> wrote:
> Thanks Szczepan!
>
> I've finally found the time to get fully across this feature. I haven't yet
> looked into the implementation.
>
> Here's some initial feedback on the model.
>
> 1. We've named this feature 'component replacements', but the modelling is
> done at the "module" level (ComponentModuleMetadata).  I don't think this is
> ideal: I'd like to think about a 'replacement' being modelled at the
> component level ("org:name:version" instead of "org:name"). In most cases
> we'll consider all components with the same 'org:name as having the same
> replacements, and the DSL would make it convenient to define this. But the
> underlying model should associate 'replacements' with a component, I think.
> 2. Eventually, we'd like a published component to contain replacement
> information. For this to be convenient, we'd normally publish a component
> and say it 'replaces' or 'supercedes' a set of other components. I think it
> might be helpful to model the relationship in this way, using 'replaces'
> rather than 'replaced by'.
>
> And on the DSL:
> The DSLs for all of our 'rules' that apply to dependency resolution are
> converging on a pattern that will hopefully allow them to be migrated into
> the new configuration rule infrastructure at some stage. The pattern is:
>
> model-rule-type {
>     eachXXX { SomeMutableType mutable[, OtherType immutableInput]* ->
>     }
>     module("group:name") { SomeMutableType mutable[, OtherType
> immutableInput]* ->
>     }
> }
>
> Exceptions:
> - For component selection rules, 'all' is used instead of
> 'eachComponentSelection'.
>      - I'm thinking this works better as a partner method to 'module()'
> - For component metadata rules, the module() method doesn't exist yet.
> - For dependency resolve rules, the 'model-rule-type' containing element
> doesn't exist.
>
> If we stick with the same pattern for component replacements, we'd have:
>
> dependencies {
>     components {
>         module("com.google.guava:guava") { ComponentSubstitutionDetails
> component ->
>
> component.replacesModule("com.google.collections:google-collections")
>         }
>     }
> }
>
> That's it for now. Thanks for restarting the discussion!
> Daz
>
> On Wed, Sep 24, 2014 at 3:31 AM, Szczepan Faber <[hidden email]> wrote:
>>
>> Upcoming Gradle 2.2 contains new incubating component replacement rules:
>>
>> dependencies {
>>     components {
>>
>> module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
>>     }
>> }
>>
>> See more in http://www.gradle.org/docs/nightly/release-notes. This
>> declaration is used during conflict resolution, Gradle will prevent
>> both collections and guava appear in the same dependency tree,
>> preferring any version of guava over every version of collections.
>> Most importantly, if the dependency tree _only_ contains collections
>> it will _not_ be replaced (because there is no conflict). Multiple
>> modules can have the same target replacement. However, we don't
>> support (yet) having single module replaced by multiple modules.
>>
>> The full DSL in the current form, based on our previous discussion on
>> the mailing list. I'm starting new email thread on purpose (for good
>> or bad, let's see whether it helps).
>>
>> dependencies {
>>     components {
>>     //declaring replacement:
>>
>> module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
>>         module(someModuleIdentifier).replacedBy("com.google.guava:guava")
>>
>>         //querying for the replacement target:
>>         ModuleIdentifier id =
>> module("com.google.collections:google-collections").getReplacedBy()
>>
>>         //querying for the replacement source:
>>         ModuleIdentifier id =
>> module("com.google.collections:google-collections").getId()
>>     }
>> }
>>
>> Javadoc (for interface names, etc.):
>>
>> http://www.gradle.org/docs/nightly/javadoc/org/gradle/api/artifacts/dsl/ComponentMetadataHandler.html#module(java.lang.Object)
>>
>> This DSL can grow to accommodate features like:
>> a) replacing single module with a set of modules. I'd love to have
>> this. I also want to make incremental progress.
>> b) other component module metadata (e.g. releasable units, impl module
>> consistent with api module). I'd love to have this, too.
>>
>> Let's confirm this API and/or make changes to it before 2.2 release.
>> Other related APIs that we should have in mind (for consistency):
>>
>> 1. component selection rules:
>>
>> configurations.conf.resolutionStrategy {
>>     componentSelection {
>>         all { ComponentSelection selection ->
>>             if (selection.candidate.group == 'org.sample') {
>>                 selection.reject("rejecting experimental")
>>             }
>>         }
>>         module("org.sample:api") { ComponentSelection selection ->
>>             if (selection.candidate.version == "1.1") {
>>                 selection.reject("known bad version")
>>             }
>>         }
>>     }
>> }
>>
>> 2. component metadata rules:
>>
>> dependencies {
>>     components {
>>         eachComponent { ComponentMetadataDetails details,
>> IvyModuleDescriptor ivyModule ->
>>             if (details.id.group == 'my.org' && ivyModule.branch ==
>> 'testing') {
>>                 details.changing = true
>>             }
>>         }
>>     }
>> }
>>
>> Cheers!
>> --
>> Szczepan Faber
>> Core dev@gradle; Founder@mockito
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>     http://xircles.codehaus.org/manage_email
>>
>>
>
>
>
> --
> Darrell (Daz) DeBoer
> http://www.gradleware.com



--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email





--
Darrell (Daz) DeBoer


--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
CTO Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: component replacement declarations

Daz DeBoer-2
On Wed, Sep 24, 2014 at 4:19 PM, Adam Murdoch <[hidden email]> wrote:

On 25 Sep 2014, at 6:12 am, Daz DeBoer <[hidden email]> wrote:

So I'm battling a bit as to whether we should really be modelling these replacements at the component level (using rules as suggested) or should be designing these at a higher level, on some sort of new model concept.

Reviewing some of the old emails on this, I found this from Adam:

I think it’s still an open question as to whether we use the component (instance) meta-data rules or introduce a new type of meta data for streams of work and how they map to modules over time.
If we decide to add this to component meta-data then we would use whatever DSL happens to be available for defining the component meta-data, which happens to be `dependencies.components.eachComponent`. We’d later add the modules(_) { … } convenience DSL.

So the alternative to component metadata rules will be modelling some new data structure, which we've done in the current implementation with "component module" (ComponentModuleMetadata).  This doesn't quite represent a 'stream of work', but perhaps it could evolve in that way.

So the big question in my mind is do we:
a) Implement 'replaces/replacedBy' at the component instance level, and configure it via component metadata rules
OR
b) Do a bit more work to model component replacements in the context of a 'stream of work' and clearly separate the DSL from the metadata rules.

It feels like ultimately this is something we need to know at the component instance level. Given an instance x, we need to know:

1. For each instance y that also appears in the graph, do x and y conflict?
2. When x and y conflict, which one should we select? Do we replace x with y or y with x?
3. When x is replaced by y, which other instances must also be included in the result?
4. When x is replaced by y, which other instances must also be replaced by y?

#3 is for the case where a module is busted up into several others. #4 is for the case where several modules are combined into a single module. This can happen over time (eg groovy 1.x was busted up into pieces from groovy 2.0 and later) and can happen as an aggregation (eg if I choose groovy-all I need to pull all of the groovy modules out of the result). 

So this is the kind of model we want DependencyGraphBuilder to work with. For now, it doesn’t need to be anywhere near this general, as there are performance implications to trying to make this work generally with any arbitrary set of instances.

From the build author’s point of view, we want a different model. We should be able to declare concrete and specific facts in the DSL and use those facts to derive answers to the above questions. We don’t want to expose the above questions in the DSL, at least not yet.

The model we want to aim for is one where we describe the history of a component instance: where did this instance come from?

So, we have a few options for the user model:

1. This is just more stuff you add to the component instance meta-data. Each component instance would describe its full history - all the other component instances that it is newer than (ie replaces). Not literally - you’d use some kind of declarative convenience. You’d use the existing component meta-data rules to attach this history.

2. This is stuff you add to the meta-data about a module/component. We’d use a new module meta-data rules DSL for this.

3. Some combination of the two, where there isn’t any concept of a module/component, but there is a separate convenience DSL to describe the history of all components instances with a particular module id, without requiring the meta-data of each instance.

The big downside of #1 is when there are transitive changes. For example: a was replaced by b and then split into (c,d). Given this, if I end up with a and d in the graph, I need to figure out that a and d conflict and that I need to replace a with c and d.

With #1, we’d need to find some meta-data for an instance of b in order to apply the rule, even though b is never referenced in the graph. So I reckon #1 is out.

That's a fair argument. So in effect it's not enough to know the metadata of the 2 nodes involved in the potential conflict, we'd also need metadata for intervening nodes in a replacement chain. Given this, maybe we should go with something very similar to Szczepan's original work, but using a different enclosing block (not 'components'). This block would represent logic where we deal with groups of components, be that modules, releasable units, etc. 

Something like:

dependencies {
     units { // Need a better name for this
          module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
          releasableUnit("spring2") {
               module("org.springframework:spring-core")
               module("org.springframework:spring-beans")
               // Would need to list the entire set here, for the 'replacedBy' to work.
          }
          module("org.springframework:spring").replacedBy("spring2")
     }
}

Alternatively, we could go with using 'components' block, but ensure that all component-instance-rule methods are clearly named:

dependencies {
    components {
         // component-instance-rules
         eachComponent { .. rule .. }
         eachComponent.forModule("org.google.collections:google-collections") { ... }

         // model configuration as above
         module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
         releasableUnit("spring2") {
         ....
   }
}

Feedback? We need to come up with a reasonable pattern for this DSL for 2.2
Daz


From a user’s point of view, there probably isn’t much difference between #2 and #3.




On Wed, Sep 24, 2014 at 10:41 AM, Szczepan Faber <[hidden email]> wrote:
Awesome feedback! Let's skype on it today.

On Wed, Sep 24, 2014 at 5:50 PM, Daz DeBoer
<[hidden email]> wrote:
> Thanks Szczepan!
>
> I've finally found the time to get fully across this feature. I haven't yet
> looked into the implementation.
>
> Here's some initial feedback on the model.
>
> 1. We've named this feature 'component replacements', but the modelling is
> done at the "module" level (ComponentModuleMetadata).  I don't think this is
> ideal: I'd like to think about a 'replacement' being modelled at the
> component level ("org:name:version" instead of "org:name"). In most cases
> we'll consider all components with the same 'org:name as having the same
> replacements, and the DSL would make it convenient to define this. But the
> underlying model should associate 'replacements' with a component, I think.
> 2. Eventually, we'd like a published component to contain replacement
> information. For this to be convenient, we'd normally publish a component
> and say it 'replaces' or 'supercedes' a set of other components. I think it
> might be helpful to model the relationship in this way, using 'replaces'
> rather than 'replaced by'.
>
> And on the DSL:
> The DSLs for all of our 'rules' that apply to dependency resolution are
> converging on a pattern that will hopefully allow them to be migrated into
> the new configuration rule infrastructure at some stage. The pattern is:
>
> model-rule-type {
>     eachXXX { SomeMutableType mutable[, OtherType immutableInput]* ->
>     }
>     module("group:name") { SomeMutableType mutable[, OtherType
> immutableInput]* ->
>     }
> }
>
> Exceptions:
> - For component selection rules, 'all' is used instead of
> 'eachComponentSelection'.
>      - I'm thinking this works better as a partner method to 'module()'
> - For component metadata rules, the module() method doesn't exist yet.
> - For dependency resolve rules, the 'model-rule-type' containing element
> doesn't exist.
>
> If we stick with the same pattern for component replacements, we'd have:
>
> dependencies {
>     components {
>         module("com.google.guava:guava") { ComponentSubstitutionDetails
> component ->
>
> component.replacesModule("com.google.collections:google-collections")
>         }
>     }
> }
>
> That's it for now. Thanks for restarting the discussion!
> Daz
>
> On Wed, Sep 24, 2014 at 3:31 AM, Szczepan Faber <[hidden email]> wrote:
>>
>> Upcoming Gradle 2.2 contains new incubating component replacement rules:
>>
>> dependencies {
>>     components {
>>
>> module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
>>     }
>> }
>>
>> See more in http://www.gradle.org/docs/nightly/release-notes. This
>> declaration is used during conflict resolution, Gradle will prevent
>> both collections and guava appear in the same dependency tree,
>> preferring any version of guava over every version of collections.
>> Most importantly, if the dependency tree _only_ contains collections
>> it will _not_ be replaced (because there is no conflict). Multiple
>> modules can have the same target replacement. However, we don't
>> support (yet) having single module replaced by multiple modules.
>>
>> The full DSL in the current form, based on our previous discussion on
>> the mailing list. I'm starting new email thread on purpose (for good
>> or bad, let's see whether it helps).
>>
>> dependencies {
>>     components {
>>     //declaring replacement:
>>
>> module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
>>         module(someModuleIdentifier).replacedBy("com.google.guava:guava")
>>
>>         //querying for the replacement target:
>>         ModuleIdentifier id =
>> module("com.google.collections:google-collections").getReplacedBy()
>>
>>         //querying for the replacement source:
>>         ModuleIdentifier id =
>> module("com.google.collections:google-collections").getId()
>>     }
>> }
>>
>> Javadoc (for interface names, etc.):
>>
>> http://www.gradle.org/docs/nightly/javadoc/org/gradle/api/artifacts/dsl/ComponentMetadataHandler.html#module(java.lang.Object)
>>
>> This DSL can grow to accommodate features like:
>> a) replacing single module with a set of modules. I'd love to have
>> this. I also want to make incremental progress.
>> b) other component module metadata (e.g. releasable units, impl module
>> consistent with api module). I'd love to have this, too.
>>
>> Let's confirm this API and/or make changes to it before 2.2 release.
>> Other related APIs that we should have in mind (for consistency):
>>
>> 1. component selection rules:
>>
>> configurations.conf.resolutionStrategy {
>>     componentSelection {
>>         all { ComponentSelection selection ->
>>             if (selection.candidate.group == 'org.sample') {
>>                 selection.reject("rejecting experimental")
>>             }
>>         }
>>         module("org.sample:api") { ComponentSelection selection ->
>>             if (selection.candidate.version == "1.1") {
>>                 selection.reject("known bad version")
>>             }
>>         }
>>     }
>> }
>>
>> 2. component metadata rules:
>>
>> dependencies {
>>     components {
>>         eachComponent { ComponentMetadataDetails details,
>> IvyModuleDescriptor ivyModule ->
>>             if (details.id.group == 'my.org' && ivyModule.branch ==
>> 'testing') {
>>                 details.changing = true
>>             }
>>         }
>>     }
>> }
>>
>> Cheers!
>> --
>> Szczepan Faber
>> Core dev@gradle; Founder@mockito
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>     http://xircles.codehaus.org/manage_email
>>
>>
>
>
>
> --
> Darrell (Daz) DeBoer
> http://www.gradleware.com



--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email





--
Darrell (Daz) DeBoer


--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
CTO Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com






--
Darrell (Daz) DeBoer
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: component replacement declarations

Adam Murdoch

On 26 Sep 2014, at 4:44 am, Daz DeBoer <[hidden email]> wrote:

On Wed, Sep 24, 2014 at 4:19 PM, Adam Murdoch <[hidden email]> wrote:

On 25 Sep 2014, at 6:12 am, Daz DeBoer <[hidden email]> wrote:

So I'm battling a bit as to whether we should really be modelling these replacements at the component level (using rules as suggested) or should be designing these at a higher level, on some sort of new model concept.

Reviewing some of the old emails on this, I found this from Adam:

I think it’s still an open question as to whether we use the component (instance) meta-data rules or introduce a new type of meta data for streams of work and how they map to modules over time.
If we decide to add this to component meta-data then we would use whatever DSL happens to be available for defining the component meta-data, which happens to be `dependencies.components.eachComponent`. We’d later add the modules(_) { … } convenience DSL.

So the alternative to component metadata rules will be modelling some new data structure, which we've done in the current implementation with "component module" (ComponentModuleMetadata).  This doesn't quite represent a 'stream of work', but perhaps it could evolve in that way.

So the big question in my mind is do we:
a) Implement 'replaces/replacedBy' at the component instance level, and configure it via component metadata rules
OR
b) Do a bit more work to model component replacements in the context of a 'stream of work' and clearly separate the DSL from the metadata rules.

It feels like ultimately this is something we need to know at the component instance level. Given an instance x, we need to know:

1. For each instance y that also appears in the graph, do x and y conflict?
2. When x and y conflict, which one should we select? Do we replace x with y or y with x?
3. When x is replaced by y, which other instances must also be included in the result?
4. When x is replaced by y, which other instances must also be replaced by y?

#3 is for the case where a module is busted up into several others. #4 is for the case where several modules are combined into a single module. This can happen over time (eg groovy 1.x was busted up into pieces from groovy 2.0 and later) and can happen as an aggregation (eg if I choose groovy-all I need to pull all of the groovy modules out of the result). 

So this is the kind of model we want DependencyGraphBuilder to work with. For now, it doesn’t need to be anywhere near this general, as there are performance implications to trying to make this work generally with any arbitrary set of instances.

From the build author’s point of view, we want a different model. We should be able to declare concrete and specific facts in the DSL and use those facts to derive answers to the above questions. We don’t want to expose the above questions in the DSL, at least not yet.

The model we want to aim for is one where we describe the history of a component instance: where did this instance come from?

So, we have a few options for the user model:

1. This is just more stuff you add to the component instance meta-data. Each component instance would describe its full history - all the other component instances that it is newer than (ie replaces). Not literally - you’d use some kind of declarative convenience. You’d use the existing component meta-data rules to attach this history.

2. This is stuff you add to the meta-data about a module/component. We’d use a new module meta-data rules DSL for this.

3. Some combination of the two, where there isn’t any concept of a module/component, but there is a separate convenience DSL to describe the history of all components instances with a particular module id, without requiring the meta-data of each instance.

The big downside of #1 is when there are transitive changes. For example: a was replaced by b and then split into (c,d). Given this, if I end up with a and d in the graph, I need to figure out that a and d conflict and that I need to replace a with c and d.

With #1, we’d need to find some meta-data for an instance of b in order to apply the rule, even though b is never referenced in the graph. So I reckon #1 is out.

That's a fair argument. So in effect it's not enough to know the metadata of the 2 nodes involved in the potential conflict, we'd also need metadata for intervening nodes in a replacement chain. Given this, maybe we should go with something very similar to Szczepan's original work, but using a different enclosing block (not 'components'). This block would represent logic where we deal with groups of components, be that modules, releasable units, etc. 

Something like:

dependencies {
     units { // Need a better name for this

I’d just call them ‘modules’ for now. The logic applies to components that happen to be published as modules in an ivy/maven repo.

          module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
          releasableUnit("spring2") {
               module("org.springframework:spring-core")
               module("org.springframework:spring-beans")
               // Would need to list the entire set here, for the 'replacedBy' to work.
          }
          module("org.springframework:spring").replacedBy("spring2”)

I would make these blocks:

module(“…”) { ModuleMetaDataDetails module ->
… some stuff about the module ...
}
eachModule { ModuleMetaDataDetails module -> … }


     }
}

Alternatively, we could go with using 'components' block, but ensure that all component-instance-rule methods are clearly named:

dependencies {
    components {
         // component-instance-rules
         eachComponent { .. rule .. }
         eachComponent.forModule("org.google.collections:google-collections") { ... }

         // model configuration as above
         module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
         releasableUnit("spring2") {
         ....
   }
}

Feedback? We need to come up with a reasonable pattern for this DSL for 2.2
Daz


From a user’s point of view, there probably isn’t much difference between #2 and #3.




On Wed, Sep 24, 2014 at 10:41 AM, Szczepan Faber <[hidden email]> wrote:
Awesome feedback! Let's skype on it today.

On Wed, Sep 24, 2014 at 5:50 PM, Daz DeBoer
<[hidden email]> wrote:

> Thanks Szczepan!
>
> I've finally found the time to get fully across this feature. I haven't yet
> looked into the implementation.
>
> Here's some initial feedback on the model.
>
> 1. We've named this feature 'component replacements', but the modelling is
> done at the "module" level (ComponentModuleMetadata).  I don't think this is
> ideal: I'd like to think about a 'replacement' being modelled at the
> component level ("org:name:version" instead of "org:name"). In most cases
> we'll consider all components with the same 'org:name as having the same
> replacements, and the DSL would make it convenient to define this. But the
> underlying model should associate 'replacements' with a component, I think.
> 2. Eventually, we'd like a published component to contain replacement
> information. For this to be convenient, we'd normally publish a component
> and say it 'replaces' or 'supercedes' a set of other components. I think it
> might be helpful to model the relationship in this way, using 'replaces'
> rather than 'replaced by'.
>
> And on the DSL:
> The DSLs for all of our 'rules' that apply to dependency resolution are
> converging on a pattern that will hopefully allow them to be migrated into
> the new configuration rule infrastructure at some stage. The pattern is:
>
> model-rule-type {
>     eachXXX { SomeMutableType mutable[, OtherType immutableInput]* ->
>     }
>     module("group:name") { SomeMutableType mutable[, OtherType
> immutableInput]* ->
>     }
> }
>
> Exceptions:
> - For component selection rules, 'all' is used instead of
> 'eachComponentSelection'.
>      - I'm thinking this works better as a partner method to 'module()'
> - For component metadata rules, the module() method doesn't exist yet.
> - For dependency resolve rules, the 'model-rule-type' containing element
> doesn't exist.
>
> If we stick with the same pattern for component replacements, we'd have:
>
> dependencies {
>     components {
>         module("com.google.guava:guava") { ComponentSubstitutionDetails
> component ->
>
> component.replacesModule("com.google.collections:google-collections")
>         }
>     }
> }
>
> That's it for now. Thanks for restarting the discussion!
> Daz
>
> On Wed, Sep 24, 2014 at 3:31 AM, Szczepan Faber <[hidden email]> wrote:
>>
>> Upcoming Gradle 2.2 contains new incubating component replacement rules:
>>
>> dependencies {
>>     components {
>>
>> module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
>>     }
>> }
>>
>> See more in http://www.gradle.org/docs/nightly/release-notes. This
>> declaration is used during conflict resolution, Gradle will prevent
>> both collections and guava appear in the same dependency tree,
>> preferring any version of guava over every version of collections.
>> Most importantly, if the dependency tree _only_ contains collections
>> it will _not_ be replaced (because there is no conflict). Multiple
>> modules can have the same target replacement. However, we don't
>> support (yet) having single module replaced by multiple modules.
>>
>> The full DSL in the current form, based on our previous discussion on
>> the mailing list. I'm starting new email thread on purpose (for good
>> or bad, let's see whether it helps).
>>
>> dependencies {
>>     components {
>>     //declaring replacement:
>>
>> module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
>>         module(someModuleIdentifier).replacedBy("com.google.guava:guava")
>>
>>         //querying for the replacement target:
>>         ModuleIdentifier id =
>> module("com.google.collections:google-collections").getReplacedBy()
>>
>>         //querying for the replacement source:
>>         ModuleIdentifier id =
>> module("com.google.collections:google-collections").getId()
>>     }
>> }
>>
>> Javadoc (for interface names, etc.):
>>
>>http://www.gradle.org/docs/nightly/javadoc/org/gradle/api/artifacts/dsl/ComponentMetadataHandler.html#module(java.lang.Object)
>>
>> This DSL can grow to accommodate features like:
>> a) replacing single module with a set of modules. I'd love to have
>> this. I also want to make incremental progress.
>> b) other component module metadata (e.g. releasable units, impl module
>> consistent with api module). I'd love to have this, too.
>>
>> Let's confirm this API and/or make changes to it before 2.2 release.
>> Other related APIs that we should have in mind (for consistency):
>>
>> 1. component selection rules:
>>
>> configurations.conf.resolutionStrategy {
>>     componentSelection {
>>         all { ComponentSelection selection ->
>>             if (selection.candidate.group == 'org.sample') {
>>                 selection.reject("rejecting experimental")
>>             }
>>         }
>>         module("org.sample:api") { ComponentSelection selection ->
>>             if (selection.candidate.version == "1.1") {
>>                 selection.reject("known bad version")
>>             }
>>         }
>>     }
>> }
>>
>> 2. component metadata rules:
>>
>> dependencies {
>>     components {
>>         eachComponent { ComponentMetadataDetails details,
>> IvyModuleDescriptor ivyModule ->
>>             if (details.id.group == 'my.org' && ivyModule.branch ==
>> 'testing') {
>>                 details.changing = true
>>             }
>>         }
>>     }
>> }
>>
>> Cheers!
>> --
>> Szczepan Faber
>> Core dev@gradle; Founder@mockito
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>     http://xircles.codehaus.org/manage_email
>>
>>
>
>
>
> --
> Darrell (Daz) DeBoer
> http://www.gradleware.com



--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email





-- 
Darrell (Daz) DeBoer


--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
CTO Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com






-- 
Darrell (Daz) DeBoer


--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
CTO Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: component replacement declarations

Kon Soulianidis
Just a quick note, this is going to the the old dev list on codehaus instead of the new mailing list
Kon Soulianidis
Ph: +61 430 015 377

On 25 September 2014 at 1:31:58 pm, Adam Murdoch ([hidden email]) wrote:


On 26 Sep 2014, at 4:44 am, Daz DeBoer <[hidden email]> wrote:

On Wed, Sep 24, 2014 at 4:19 PM, Adam Murdoch <[hidden email]> wrote:

On 25 Sep 2014, at 6:12 am, Daz DeBoer <[hidden email]> wrote:

So I'm battling a bit as to whether we should really be modelling these replacements at the component level (using rules as suggested) or should be designing these at a higher level, on some sort of new model concept.

Reviewing some of the old emails on this, I found this from Adam:

I think it’s still an open question as to whether we use the component (instance) meta-data rules or introduce a new type of meta data for streams of work and how they map to modules over time.
If we decide to add this to component meta-data then we would use whatever DSL happens to be available for defining the component meta-data, which happens to be `dependencies.components.eachComponent`. We’d later add the modules(_) { … } convenience DSL.

So the alternative to component metadata rules will be modelling some new data structure, which we've done in the current implementation with "component module" (ComponentModuleMetadata).  This doesn't quite represent a 'stream of work', but perhaps it could evolve in that way.

So the big question in my mind is do we:
a) Implement 'replaces/replacedBy' at the component instance level, and configure it via component metadata rules
OR
b) Do a bit more work to model component replacements in the context of a 'stream of work' and clearly separate the DSL from the metadata rules.

It feels like ultimately this is something we need to know at the component instance level. Given an instance x, we need to know:

1. For each instance y that also appears in the graph, do x and y conflict?
2. When x and y conflict, which one should we select? Do we replace x with y or y with x?
3. When x is replaced by y, which other instances must also be included in the result?
4. When x is replaced by y, which other instances must also be replaced by y?

#3 is for the case where a module is busted up into several others. #4 is for the case where several modules are combined into a single module. This can happen over time (eg groovy 1.x was busted up into pieces from groovy 2.0 and later) and can happen as an aggregation (eg if I choose groovy-all I need to pull all of the groovy modules out of the result). 

So this is the kind of model we want DependencyGraphBuilder to work with. For now, it doesn’t need to be anywhere near this general, as there are performance implications to trying to make this work generally with any arbitrary set of instances.

From the build author’s point of view, we want a different model. We should be able to declare concrete and specific facts in the DSL and use those facts to derive answers to the above questions. We don’t want to expose the above questions in the DSL, at least not yet.

The model we want to aim for is one where we describe the history of a component instance: where did this instance come from?

So, we have a few options for the user model:

1. This is just more stuff you add to the component instance meta-data. Each component instance would describe its full history - all the other component instances that it is newer than (ie replaces). Not literally - you’d use some kind of declarative convenience. You’d use the existing component meta-data rules to attach this history.

2. This is stuff you add to the meta-data about a module/component. We’d use a new module meta-data rules DSL for this.

3. Some combination of the two, where there isn’t any concept of a module/component, but there is a separate convenience DSL to describe the history of all components instances with a particular module id, without requiring the meta-data of each instance.

The big downside of #1 is when there are transitive changes. For example: a was replaced by b and then split into (c,d). Given this, if I end up with a and d in the graph, I need to figure out that a and d conflict and that I need to replace a with c and d.

With #1, we’d need to find some meta-data for an instance of b in order to apply the rule, even though b is never referenced in the graph. So I reckon #1 is out.

That's a fair argument. So in effect it's not enough to know the metadata of the 2 nodes involved in the potential conflict, we'd also need metadata for intervening nodes in a replacement chain. Given this, maybe we should go with something very similar to Szczepan's original work, but using a different enclosing block (not 'components'). This block would represent logic where we deal with groups of components, be that modules, releasable units, etc. 

Something like:

dependencies {
     units { // Need a better name for this

I’d just call them ‘modules’ for now. The logic applies to components that happen to be published as modules in an ivy/maven repo.

          module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
          releasableUnit("spring2") {
               module("org.springframework:spring-core")
               module("org.springframework:spring-beans")
               // Would need to list the entire set here, for the 'replacedBy' to work.
          }
          module("org.springframework:spring").replacedBy("spring2”)

I would make these blocks:

module(“…”) { ModuleMetaDataDetails module ->
… some stuff about the module ...
}
eachModule { ModuleMetaDataDetails module -> … }


     }
}

Alternatively, we could go with using 'components' block, but ensure that all component-instance-rule methods are clearly named:

dependencies {
    components {
         // component-instance-rules
         eachComponent { .. rule .. }
         eachComponent.forModule("org.google.collections:google-collections") { ... }

         // model configuration as above
         module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
         releasableUnit("spring2") {
         ....
   }
}

Feedback? We need to come up with a reasonable pattern for this DSL for 2.2
Daz


From a user’s point of view, there probably isn’t much difference between #2 and #3.




On Wed, Sep 24, 2014 at 10:41 AM, Szczepan Faber <[hidden email]> wrote:
Awesome feedback! Let's skype on it today.

On Wed, Sep 24, 2014 at 5:50 PM, Daz DeBoer
<[hidden email]> wrote:
> Thanks Szczepan!
>
> I've finally found the time to get fully across this feature. I haven't yet
> looked into the implementation.
>
> Here's some initial feedback on the model.
>
> 1. We've named this feature 'component replacements', but the modelling is
> done at the "module" level (ComponentModuleMetadata).  I don't think this is
> ideal: I'd like to think about a 'replacement' being modelled at the
> component level ("org:name:version" instead of "org:name"). In most cases
> we'll consider all components with the same 'org:name as having the same
> replacements, and the DSL would make it convenient to define this. But the
> underlying model should associate 'replacements' with a component, I think.
> 2. Eventually, we'd like a published component to contain replacement
> information. For this to be convenient, we'd normally publish a component
> and say it 'replaces' or 'supercedes' a set of other components. I think it
> might be helpful to model the relationship in this way, using 'replaces'
> rather than 'replaced by'.
>
> And on the DSL:
> The DSLs for all of our 'rules' that apply to dependency resolution are
> converging on a pattern that will hopefully allow them to be migrated into
> the new configuration rule infrastructure at some stage. The pattern is:
>
> model-rule-type {
>     eachXXX { SomeMutableType mutable[, OtherType immutableInput]* ->
>     }
>     module("group:name") { SomeMutableType mutable[, OtherType
> immutableInput]* ->
>     }
> }
>
> Exceptions:
> - For component selection rules, 'all' is used instead of
> 'eachComponentSelection'.
>      - I'm thinking this works better as a partner method to 'module()'
> - For component metadata rules, the module() method doesn't exist yet.
> - For dependency resolve rules, the 'model-rule-type' containing element
> doesn't exist.
>
> If we stick with the same pattern for component replacements, we'd have:
>
> dependencies {
>     components {
>         module("com.google.guava:guava") { ComponentSubstitutionDetails
> component ->
>
> component.replacesModule("com.google.collections:google-collections")
>         }
>     }
> }
>
> That's it for now. Thanks for restarting the discussion!
> Daz
>
> On Wed, Sep 24, 2014 at 3:31 AM, Szczepan Faber <[hidden email]> wrote:
>>
>> Upcoming Gradle 2.2 contains new incubating component replacement rules:
>>
>> dependencies {
>>     components {
>>
>> module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
>>     }
>> }
>>
>> See more in http://www.gradle.org/docs/nightly/release-notes. This
>> declaration is used during conflict resolution, Gradle will prevent
>> both collections and guava appear in the same dependency tree,
>> preferring any version of guava over every version of collections.
>> Most importantly, if the dependency tree _only_ contains collections
>> it will _not_ be replaced (because there is no conflict). Multiple
>> modules can have the same target replacement. However, we don't
>> support (yet) having single module replaced by multiple modules.
>>
>> The full DSL in the current form, based on our previous discussion on
>> the mailing list. I'm starting new email thread on purpose (for good
>> or bad, let's see whether it helps).
>>
>> dependencies {
>>     components {
>>     //declaring replacement:
>>
>> module("com.google.collections:google-collections").replacedBy("com.google.guava:guava")
>>         module(someModuleIdentifier).replacedBy("com.google.guava:guava")
>>
>>         //querying for the replacement target:
>>         ModuleIdentifier id =
>> module("com.google.collections:google-collections").getReplacedBy()
>>
>>         //querying for the replacement source:
>>         ModuleIdentifier id =
>> module("com.google.collections:google-collections").getId()
>>     }
>> }
>>
>> Javadoc (for interface names, etc.):
>>
>>http://www.gradle.org/docs/nightly/javadoc/org/gradle/api/artifacts/dsl/ComponentMetadataHandler.html#module(java.lang.Object)
>>
>> This DSL can grow to accommodate features like:
>> a) replacing single module with a set of modules. I'd love to have
>> this. I also want to make incremental progress.
>> b) other component module metadata (e.g. releasable units, impl module
>> consistent with api module). I'd love to have this, too.
>>
>> Let's confirm this API and/or make changes to it before 2.2 release.
>> Other related APIs that we should have in mind (for consistency):
>>
>> 1. component selection rules:
>>
>> configurations.conf.resolutionStrategy {
>>     componentSelection {
>>         all { ComponentSelection selection ->
>>             if (selection.candidate.group == 'org.sample') {
>>                 selection.reject("rejecting experimental")
>>             }
>>         }
>>         module("org.sample:api") { ComponentSelection selection ->
>>             if (selection.candidate.version == "1.1") {
>>                 selection.reject("known bad version")
>>             }
>>         }
>>     }
>> }
>>
>> 2. component metadata rules:
>>
>> dependencies {
>>     components {
>>         eachComponent { ComponentMetadataDetails details,
>> IvyModuleDescriptor ivyModule ->
>>             if (details.id.group == 'my.org' && ivyModule.branch ==
>> 'testing') {
>>                 details.changing = true
>>             }
>>         }
>>     }
>> }
>>
>> Cheers!
>> --
>> Szczepan Faber
>> Core dev@gradle; Founder@mockito
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>     http://xircles.codehaus.org/manage_email
>>
>>
>
>
>
> --
> Darrell (Daz) DeBoer
> http://www.gradleware.com



--
Szczepan Faber
Core dev@gradle; Founder@mockito

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email





-- 
Darrell (Daz) DeBoer


--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
CTO Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com






-- 
Darrell (Daz) DeBoer


--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
CTO Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com



Loading...