Re: Structuring the plugin portal code.

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: Structuring the plugin portal code.

Luke Daley-2


On 9 May 2014 at 3:07:05 pm, Luke Daley ([hidden email]) wrote:

Hi,

I’m trying to bust the new plugin portal code out of core-impl and parts out of core. I’m unsure about how to do this and avoid a circular dependency.

There are 3 parts:

1. core - integrates plugin resolvers into the build lifecycle, provides dependency management services needed for plugin resolution
2. general plugin infrastructure for plugin resolvers to use (e.g. interfaces to implement)
3. specific plugin portal plugin resolver implementation

My plan was to create a new ‘plugin-portal’ project for #3, and put the interfaces and support (#2) in ‘core’. Thus, circular dependency.

The way that core-impl solves this is through some service loader indirection. I can see how we could do a similar thing in this case. The ‘plugin-portal’ project could advertise a PluginResolver implementation that ‘core’ loads at runtime. Therefore, ‘core’ would have no compile dependency on ‘plugin-portal’. Is this the way to go?

One question then is how to order the plugin resolvers in core. If it just loads them arbitrarily off the classpath, what orders them? One option would be for the plugin portal resolver to be called out especially (e.g. it’s loaded via some kind of marker interface that lives in core).

Any ideas?

Another way to do this would be to move RepositoryHandler and DependencyHandler and their dependencies out to a separate project like dependency-resolution-api. That way plugin-portal can depend on this instead of core to break the cycle. We’d also have to move the base plugin resolver types out of core as well.

— 

Luke Daley
Gradleware
Join us for Gradle Summit 2014, June 12th and 13th in Santa Clara, CA: http://www.gradlesummit.com
Reply | Threaded
Open this post in threaded view
|

Re: Structuring the plugin portal code.

Adam Murdoch

On 9 May 2014, at 3:06 pm, Luke Daley <[hidden email]> wrote:

Hi,

I’m trying to bust the new plugin portal code out of core-impl and parts out of core. I’m unsure about how to do this and avoid a circular dependency.

There are 3 parts:

1. core - integrates plugin resolvers into the build lifecycle, provides dependency management services needed for plugin resolution
2. general plugin infrastructure for plugin resolvers to use (e.g. interfaces to implement)
3. specific plugin portal plugin resolver implementation

My plan was to create a new ‘plugin-portal’ project for #3, and put the interfaces and support (#2) in ‘core’. Thus, circular dependency.

The way that core-impl solves this is through some service loader indirection. I can see how we could do a similar thing in this case. The ‘plugin-portal’ project could advertise a PluginResolver implementation that ‘core’ loads at runtime. Therefore, ‘core’ would have no compile dependency on ‘plugin-portal’. Is this the way to go?

One question then is how to order the plugin resolvers in core. If it just loads them arbitrarily off the classpath, what orders them? One option would be for the plugin portal resolver to be called out especially (e.g. it’s loaded via some kind of marker interface that lives in core).

Any ideas?

The relationship between core and the dependency management stuff is actually a little different to this. You could think of the dependency management and plugin resolution capabilities being structured this way:

1. Public DSL and APIs through which you use the capability - defining repositories, dependencies, plugins, whatever.
2. These are backed by some internal services that do the work. These internal services are decoupled from the public types. They use their own request and response types. The services are injected into the public stuff.
3. The internal services have an implementation that provides some resolution algorithm.
4. The algorithm is backed by some internal resolver API that represent some source for the things being resolved.
5. There are a bunch of resolver implementations.

#2 is the contract between the DSL and the implementation, so this needs to be visible to everything. Ideally, #3 - #5 would not be visible from #1, and #1 not visible from #3 - #5.

Currently, for dependency management, here’s what we do:

- #1 lives in core, because the dependency management DSL has references to core (eg via Configuration -> Buildable -> Task -> Project -> ConfigurationContainer and back).
- #2 lives in core, because the internal service API is not entirely decoupled from the DSL.
- #3, #4, and #5 live together in core-impl.

We use discovery to find #3 from #1, but not to find the resolver implementations - #3 takes care of that.

I would do something similar for the plugins: move the algorithm, the resolver API and resolver implementations out of core, inject some entry point service into core, and let the implementation of the entry point service take care of everything else.

The resolution algorithm and the resolver implementations don’t necessarily need to be separated from each other. I would keep them together for now.


--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com

Join us for Gradle Summit 2014, June 12th and 13th in Santa Clara, CA: http://www.gradlesummit.com

Reply | Threaded
Open this post in threaded view
|

Re: Structuring the plugin portal code.

Adam Murdoch
In reply to this post by Luke Daley-2

On 9 May 2014, at 9:24 pm, Luke Daley <[hidden email]> wrote:



On 9 May 2014 at 3:07:05 pm, Luke Daley ([hidden email]) wrote:

Hi,

I’m trying to bust the new plugin portal code out of core-impl and parts out of core. I’m unsure about how to do this and avoid a circular dependency.

There are 3 parts:

1. core - integrates plugin resolvers into the build lifecycle, provides dependency management services needed for plugin resolution
2. general plugin infrastructure for plugin resolvers to use (e.g. interfaces to implement)
3. specific plugin portal plugin resolver implementation

My plan was to create a new ‘plugin-portal’ project for #3, and put the interfaces and support (#2) in ‘core’. Thus, circular dependency.

The way that core-impl solves this is through some service loader indirection. I can see how we could do a similar thing in this case. The ‘plugin-portal’ project could advertise a PluginResolver implementation that ‘core’ loads at runtime. Therefore, ‘core’ would have no compile dependency on ‘plugin-portal’. Is this the way to go?

One question then is how to order the plugin resolvers in core. If it just loads them arbitrarily off the classpath, what orders them? One option would be for the plugin portal resolver to be called out especially (e.g. it’s loaded via some kind of marker interface that lives in core).

Any ideas?

Another way to do this would be to move RepositoryHandler and DependencyHandler and their dependencies out to a separate project like dependency-resolution-api. That way plugin-portal can depend on this instead of core to break the cycle. We’d also have to move the base plugin resolver types out of core as well.


I would avoid this for 2 reasons:

- This is only part of the dependency management DSL. The other parts have (cyclic) dependencies on the other parts of core.
- The plugin resolution implementation should not be using the dependency management DSL. Instead it should be using dependency management services. I don’t want our infrastructure to interact via the DSL types.


--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com

Join us for Gradle Summit 2014, June 12th and 13th in Santa Clara, CA: http://www.gradlesummit.com

Reply | Threaded
Open this post in threaded view
|

Re: Structuring the plugin portal code.

Luke Daley-2
In reply to this post by Adam Murdoch


On 10 May 2014 at 7:40:14 am, Adam Murdoch ([hidden email]) wrote:


On 9 May 2014, at 3:06 pm, Luke Daley <[hidden email]> wrote:

Hi,

I’m trying to bust the new plugin portal code out of core-impl and parts out of core. I’m unsure about how to do this and avoid a circular dependency.

There are 3 parts:

1. core - integrates plugin resolvers into the build lifecycle, provides dependency management services needed for plugin resolution
2. general plugin infrastructure for plugin resolvers to use (e.g. interfaces to implement)
3. specific plugin portal plugin resolver implementation

My plan was to create a new ‘plugin-portal’ project for #3, and put the interfaces and support (#2) in ‘core’. Thus, circular dependency.

The way that core-impl solves this is through some service loader indirection. I can see how we could do a similar thing in this case. The ‘plugin-portal’ project could advertise a PluginResolver implementation that ‘core’ loads at runtime. Therefore, ‘core’ would have no compile dependency on ‘plugin-portal’. Is this the way to go?

One question then is how to order the plugin resolvers in core. If it just loads them arbitrarily off the classpath, what orders them? One option would be for the plugin portal resolver to be called out especially (e.g. it’s loaded via some kind of marker interface that lives in core).

Any ideas?

The relationship between core and the dependency management stuff is actually a little different to this. You could think of the dependency management and plugin resolution capabilities being structured this way:

1. Public DSL and APIs through which you use the capability - defining repositories, dependencies, plugins, whatever.
2. These are backed by some internal services that do the work. These internal services are decoupled from the public types. They use their own request and response types. The services are injected into the public stuff.
3. The internal services have an implementation that provides some resolution algorithm.
4. The algorithm is backed by some internal resolver API that represent some source for the things being resolved.
5. There are a bunch of resolver implementations.

#2 is the contract between the DSL and the implementation, so this needs to be visible to everything. Ideally, #3 - #5 would not be visible from #1, and #1 not visible from #3 - #5.

Currently, for dependency management, here’s what we do:

- #1 lives in core, because the dependency management DSL has references to core (eg via Configuration -> Buildable -> Task -> Project -> ConfigurationContainer and back).
- #2 lives in core, because the internal service API is not entirely decoupled from the DSL.
- #3, #4, and #5 live together in core-impl.

We use discovery to find #3 from #1, but not to find the resolver implementations - #3 takes care of that.

I would do something similar for the plugins: move the algorithm, the resolver API and resolver implementations out of core, inject some entry point service into core, and let the implementation of the entry point service take care of everything else.

The resolution algorithm and the resolver implementations don’t necessarily need to be separated from each other. I would keep them together for now.

So, in concrete terms, …

1. Leave the PluginResolver (and dependencies) interface in core

2. Create plugin-resolution project that provides a single PluginResolver implementation that core resolves via lookup

- depends on core (PluginResolver etc.)

- depends on coreImpl (for dependency resolution infrastructure) 


If so,

- How do I get at the dependency management infrastructure in plugin-resolution? service lookup like in core?

- Do I depend on the concrete types in coreImpl instead of core interfaces? (e.g. DefaultDependencyFactory instead of DependencyFactory) in plugin-resolution?

- Do I create a PluginResolutionServices in core that plugin-resolution provides the impl for (like DependencyManagementServices)? 

— 

Luke Daley
Gradleware
Join us for Gradle Summit 2014, June 12th and 13th in Santa Clara, CA: http://www.gradlesummit.com
Reply | Threaded
Open this post in threaded view
|

Re: Structuring the plugin portal code.

Adam Murdoch

On 10 May 2014, at 11:52 am, Luke Daley <[hidden email]> wrote:



On 10 May 2014 at 7:40:14 am, Adam Murdoch ([hidden email]) wrote:


On 9 May 2014, at 3:06 pm, Luke Daley <[hidden email]> wrote:

Hi,

I’m trying to bust the new plugin portal code out of core-impl and parts out of core. I’m unsure about how to do this and avoid a circular dependency.

There are 3 parts:

1. core - integrates plugin resolvers into the build lifecycle, provides dependency management services needed for plugin resolution
2. general plugin infrastructure for plugin resolvers to use (e.g. interfaces to implement)
3. specific plugin portal plugin resolver implementation

My plan was to create a new ‘plugin-portal’ project for #3, and put the interfaces and support (#2) in ‘core’. Thus, circular dependency.

The way that core-impl solves this is through some service loader indirection. I can see how we could do a similar thing in this case. The ‘plugin-portal’ project could advertise a PluginResolver implementation that ‘core’ loads at runtime. Therefore, ‘core’ would have no compile dependency on ‘plugin-portal’. Is this the way to go?

One question then is how to order the plugin resolvers in core. If it just loads them arbitrarily off the classpath, what orders them? One option would be for the plugin portal resolver to be called out especially (e.g. it’s loaded via some kind of marker interface that lives in core).

Any ideas?

The relationship between core and the dependency management stuff is actually a little different to this. You could think of the dependency management and plugin resolution capabilities being structured this way:

1. Public DSL and APIs through which you use the capability - defining repositories, dependencies, plugins, whatever.
2. These are backed by some internal services that do the work. These internal services are decoupled from the public types. They use their own request and response types. The services are injected into the public stuff.
3. The internal services have an implementation that provides some resolution algorithm.
4. The algorithm is backed by some internal resolver API that represent some source for the things being resolved.
5. There are a bunch of resolver implementations.

#2 is the contract between the DSL and the implementation, so this needs to be visible to everything. Ideally, #3 - #5 would not be visible from #1, and #1 not visible from #3 - #5.

Currently, for dependency management, here’s what we do:

- #1 lives in core, because the dependency management DSL has references to core (eg via Configuration -> Buildable -> Task -> Project -> ConfigurationContainer and back).
- #2 lives in core, because the internal service API is not entirely decoupled from the DSL.
- #3, #4, and #5 live together in core-impl.

We use discovery to find #3 from #1, but not to find the resolver implementations - #3 takes care of that.

I would do something similar for the plugins: move the algorithm, the resolver API and resolver implementations out of core, inject some entry point service into core, and let the implementation of the entry point service take care of everything else.

The resolution algorithm and the resolver implementations don’t necessarily need to be separated from each other. I would keep them together for now.

So, in concrete terms, …

1. Leave the PluginResolver (and dependencies) interface in core

2. Create plugin-resolution project that provides a single PluginResolver implementation that core resolves via lookup

- depends on core (PluginResolver etc.)

- depends on coreImpl (for dependency resolution infrastructure) 



Yeah, I think so. I think you’ll also need to extract an interface out of PluginResolverFactory and inject that instead of a PluginResolver.

If so,

- How do I get at the dependency management infrastructure in plugin-resolution? service lookup like in core?


Yes.

- Do I depend on the concrete types in coreImpl instead of core interfaces? (e.g. DefaultDependencyFactory instead of DependencyFactory) in plugin-resolution?



Interfaces, not implementations.

- Do I create a PluginResolutionServices in core that plugin-resolution provides the impl for (like DependencyManagementServices)? 



I don’t think you need to. Just make the factory available.


--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com

Join us for Gradle Summit 2014, June 12th and 13th in Santa Clara, CA: http://www.gradlesummit.com