design question

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

design question

Ittay Dror
Hi,

I want to create a plugin that will allow using Gradle to build C/C++ artifacts. The issue I want to tackle is dependency management.

Basically, a project can create 4 types of artifacts: a library (static / shared) or an executable and header files. To compile, one needs the headers, to link, one needs the library. So, in Ivy, if I have the configuraitons 'executable', 'shared', 'static', 'devel', then dependencies map to 'devel->devel' and 'static'->'static' and executable/shared are mapped to 'shared' or 'static' according to the type of library that is used. The different artifacts map to different configurations (via publications).

In Gradle, I want to have 'shared' and 'static' dependencies of the 'compile' configuration and create from them the above Ivy dependency. So if i have 'compile-shared "some.org:amodule:1.0:shared"', it will create an Ivy module descriptor with a 'devel' configuration (for compiling) and 'shared' configuration (for linking) mapped to 'shared' configuration of 'amodule'.

So far, sounds ok?

My problem is then how to change the way Gradle maps the dependencies in the gradle file to Ivy modules. My digging shows this (for 0.1.4):
1. Ivy module descriptors are created by ModuleDescriptorConverter. DefaultDependencyManger has an instance of it passed through its ctor.
2. The DependencyDescriptor is created by the Dependency instances, created by a DependencyFactory instance, also a member of DefaultDependencyManger, through its ctor
3. DefaultDependencyManager is created by DefaultDependencyManagerFactory, the instance of which is used by DefaultSettings and ProjectFactory, both are created statically through Build.newInstanceFactory (through SettingsProcessor).
4. newInstanceFactory is called by Main and returns a closure, so it is not easily manipulated.
5. 'dependencies' used in the project file is a method in DefaultProject which calls DependencyManager methods. DependencyManager is an interface,

I think all this means that I can't do what I want (extend the module converter to create a special kind of Ivy module and create a new Dependency class so it create the right Ivy dependency construct).

I think this can be solved by using IOC conventions so that every object has 'set' methods to set the services it uses. Then, even if Main/Build create default instances, a plugin can redefine them.

What do you think?

Ittay
Reply | Threaded
Open this post in threaded view
|

Re: design question

hans_d
Administrator

On Jun 15, 2008, at 11:45 AM, Ittay Dror wrote:

>
> Hi,
>
> I want to create a plugin that will allow using Gradle to build C/C++
> artifacts.

That would be very cool.

It might be also a good use case for implementing rules or synthetic  
dependencies as offered by Rake. http://docs.rubyrake.org/user_guide/ 
chapter03.html

What do you think?

> The issue I want to tackle is dependency management.
>
> Basically, a project can create 4 types of artifacts: a library  
> (static /
> shared) or an executable and header files. To compile, one needs the
> headers, to link, one needs the library. So, in Ivy, if I have the
> configuraitons 'executable', 'shared', 'static', 'devel', then  
> dependencies
> map to 'devel->devel' and 'static'->'static' and executable/shared are
> mapped to 'shared' or 'static' according to the type of library  
> that is
> used. The different artifacts map to different configurations (via
> publications).
>
> In Gradle, I want to have 'shared' and 'static' dependencies of the
> 'compile' configuration and create from them the above Ivy  
> dependency. So if
> i have 'compile-shared "some.org:amodule:1.0:shared"', it will  
> create an Ivy
> module descriptor with a 'devel' configuration (for compiling) and  
> 'shared'
> configuration (for linking) mapped to 'shared' configuration of  
> 'amodule'.
>
> So far, sounds ok?

Yep.

In Gradle 0.2, which is planned to be released today, we have  
introduced classifiers. The notation looks like: "org.testng:testng:
5.7:jdk14" which is similar to your one. On the other hand, this is  
not relevant for C/C++ projects and as you provide an own instance of  
the ModuleDescriptorConverter that should be fine.

>
> My problem is then how to change the way Gradle maps the  
> dependencies in the
> gradle file to Ivy modules. My digging shows this (for 0.1.4):
> 1. Ivy module descriptors are created by ModuleDescriptorConverter.
> DefaultDependencyManger has an instance of it passed through its ctor.
> 2. The DependencyDescriptor is created by the Dependency instances,  
> created
> by a DependencyFactory instance, also a member of  
> DefaultDependencyManger,
> through its ctor
> 3. DefaultDependencyManager is created by  
> DefaultDependencyManagerFactory,
> the instance of which is used by DefaultSettings and  
> ProjectFactory, both
> are created statically through Build.newInstanceFactory (through
> SettingsProcessor).
> 4. newInstanceFactory is called by Main and returns a closure, so  
> it is not
> easily manipulated.
> 5. 'dependencies' used in the project file is a method in  
> DefaultProject
> which calls DependencyManager methods. DependencyManager is an  
> interface,
>
> I think all this means that I can't do what I want (extend the module
> converter to create a special kind of Ivy module and create a new  
> Dependency
> class so it create the right Ivy dependency construct).
>
> I think this can be solved by using IOC conventions so that every  
> object has
> 'set' methods to set the services it uses. Then, even if Main/Build  
> create
> default instances, a plugin can redefine them.
>
> What do you think?

If I had Gradle written in Java there would be interfaces for all the  
relevant classes and ctor or setter injection. For me Gradle has been  
also an experiment how to possibly do things slightly different than  
in Java. But I'm not really happy with the current situation.

First of all. Groovy has dynamic setters and getters. You can say in  
Gradle 0.1.4:

dependencies. moduleDescriptorConverter = myNewConverterInstance

The big question is of course the type. In DefaultDependencyManager  
we have declared: ModuleDescriptorConverter moduleDescriptorConverter

An alternative would be: def moduleDescriptorConverter  (In Ruby for  
example that would be the only choice). Than myNewConverterInstance  
could be of any type (duck typing). And now we are in the current  
dynamic vs static typing debate. I'm not sold on giving up static  
typing. I love my IDE support enabled by this. I also think it makes  
the code more readable.

Groovy, although an dynamic language, offers a kind of static typing  
which Gradle usually uses. So how to do IoC then? What you can do is  
to say:

dependencies. moduleDescriptorConverter = [convert:  
{DefaultDependencyManager dependencyManager ->
             myLogic
             myReturnValue
} as ModuleDescriptorConverter

This assigns an instance of ModuleDescriptorConverter with an  
overwritten convert method (I'm not saying that this is fantastic ;)).

Why not introducing interfaces like we would do in Java? We might  
have to do this. Although interfaces in Groovy are somewhat a weak  
construct. For example although the dependencies instance of Project  
is of type DependencyManager, we can say dependencies.  
moduleDescriptorConverter = something. This is not declared in the  
interface but there is no runtime check for such stuff. After all  
Groovy is a dynamic language. The only reason why I have introduced  
the current interfaces is to avoid cyclic dependencies between packages.

My current favorite design would be to use a static language for the  
core of Gradle which offers all the syntactic goodies of Groovy. For  
the parts where we really need the dynamicity we would of course  
still use Groovy (e.g. Project, Convention, Plugins, ...) On the  
other hand, I have no plans to change this at the moment. I don't  
feel its is that painful right now.

- Hans

>
> Ittay
> --
> View this message in context: http://www.nabble.com/design-question- 
> tp17848015p17848015.html
> Sent from the gradle-dev mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>

--
Hans Dockter
Gradle Project lead
http://www.gradle.org





---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: design question

Ittay Dror
Hi Hans,
hdockter wrote
It might be also a good use case for implementing rules or synthetic  
dependencies as offered by Rake. http://docs.rubyrake.org/user_guide/ 
chapter03.html

What do you think?
I think it is tricky for C++. There are several reasons:
1. it is hard to code such rules without being dependent on the OS. e.g., invoking gcc and cl (visual studio's compiler) are very different. I want a plugin that will be able to compile on all platforms, so the plugin will need to provide the compilation command, not the user. so the user can only supply the dependency.
2. modern compilers can usually create dependencies for you. what there is left is mapping of source paths to output paths, which is better done through using the convention object.

The only selling point here (for me) is that creating such rules allows to incorporate file generation, but this can easily be replaced with a 'gen-sources' target.

hdockter wrote
Groovy, although an dynamic language, offers a kind of static typing  
which Gradle usually uses. So how to do IoC then? What you can do is  
to say:

dependencies. moduleDescriptorConverter = [convert:  
{DefaultDependencyManager dependencyManager ->
             myLogic
             myReturnValue
} as ModuleDescriptorConverter
Thank you for the explanation. I agree that with the examples you gave IOC is not required.

I think I can make some progress now. Thanks for the help.

Ittay
Reply | Threaded
Open this post in threaded view
|

Re: design question

Russel Winder-2
On Mon, 2008-06-16 at 01:13 -0700, Ittay Dror wrote:

> I think it is tricky for C++. There are several reasons:
> 1. it is hard to code such rules without being dependent on the OS. e.g.,
> invoking gcc and cl (visual studio's compiler) are very different. I want a
> plugin that will be able to compile on all platforms, so the plugin will
> need to provide the compilation command, not the user. so the user can only
> supply the dependency.

To be honest your best bet is to use SCons for C, C++, Fortran and
LaTeX, it already has this infrastructure in place.  It will take a lot
of effort to replicate all the infrastructure in Gradle.  Another
alternative is Waf, but I have been forbidden by its developer from
mentioning it and was forcibly ejected from the mailing list for
complaining that there was no way of setting bindir for installation.

The SCons developers are looking to improve their support for Java --
which is currently very minimal.  So an alternative to further
developing Gradle is actually to put effort into SCons.  Or take all the
material from SCons and dump it in Gradle of course.  However the DAG
processing is almost certainly very different.

> 2. modern compilers can usually create dependencies for you. what there is
> left is mapping of source paths to output paths, which is better done
> through using the convention object.

See SCons.

SCons does parsing for dependency calculation, and MD5 sums to avoid
using timestamps.

--
Russel.
====================================================
Dr Russel Winder                 Partner

Concertant LLP                   t: +44 20 7585 2200, +44 20 7193 9203
41 Buckmaster Road,              f: +44 8700 516 084
London SW11 1EN, UK.             m: +44 7770 465 077

signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: design question

hans_d
Administrator

On Jun 16, 2008, at 10:29 AM, Russel Winder wrote:

> On Mon, 2008-06-16 at 01:13 -0700, Ittay Dror wrote:
>
>> I think it is tricky for C++. There are several reasons:
>> 1. it is hard to code such rules without being dependent on the  
>> OS. e.g.,
>> invoking gcc and cl (visual studio's compiler) are very different.  
>> I want a
>> plugin that will be able to compile on all platforms, so the  
>> plugin will
>> need to provide the compilation command, not the user. so the user  
>> can only
>> supply the dependency.
>
> To be honest your best bet is to use SCons for C, C++, Fortran and
> LaTeX, it already has this infrastructure in place.  It will take a  
> lot
> of effort to replicate all the infrastructure in Gradle.  Another
> alternative is Waf, but I have been forbidden by its developer from
> mentioning it and was forcibly ejected from the mailing list for
> complaining that there was no way of setting bindir for installation.
>
> The SCons developers are looking to improve their support for Java --
> which is currently very minimal.  So an alternative to further
> developing Gradle is actually to put effort into SCons.

That might be the way for Gant to go.

Gradle has enough unique selling points and its own vision, that  
there is no point at all to join SCons.

- Hans

> Or take all the
> material from SCons and dump it in Gradle of course.  However the DAG
> processing is almost certainly very different.
>
>> 2. modern compilers can usually create dependencies for you. what  
>> there is
>> left is mapping of source paths to output paths, which is better done
>> through using the convention object.
>
> See SCons.
>
> SCons does parsing for dependency calculation, and MD5 sums to avoid
> using timestamps.
>
> --
> Russel.
> ====================================================
> Dr Russel Winder                 Partner
>
> Concertant LLP                   t: +44 20 7585 2200, +44 20 7193 9203
> 41 Buckmaster Road,              f: +44 8700 516 084
> London SW11 1EN, UK.             m: +44 7770 465 077

--
Hans Dockter
Gradle Project lead
http://www.gradle.org





---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: design question

Russel Winder-2
Hans,

On Mon, 2008-06-16 at 10:59 +0200, Hans Dockter wrote:

> > The SCons developers are looking to improve their support for Java --
> > which is currently very minimal.  So an alternative to further
> > developing Gradle is actually to put effort into SCons.
>
> That might be the way for Gant to go.

I think you miss the point of Gant by this comment.

> Gradle has enough unique selling points and its own vision, that  
> there is no point at all to join SCons.

In the Java space this is undoubtedly true.  In the C/C++/Fortran/LaTeX
space, SCons is currently the market leader.  The problem is in the grey
areas between.

--
Russel.
====================================================
Dr Russel Winder                 Partner

Concertant LLP                   t: +44 20 7585 2200, +44 20 7193 9203
41 Buckmaster Road,              f: +44 8700 516 084
London SW11 1EN, UK.             m: +44 7770 465 077

signature.asc (196 bytes) Download Attachment