performance of addFlatDirResolver

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

performance of addFlatDirResolver

jaco.uys
Hi,

Finding jars in a local lib dir seems to take really long.

I have a module that does not need any jars from the local lib, but I included some artificial deps just to the test performance issues.

Compiling without any deps defined in the local libs takes 4-5 secs. With some artificial deps introduced from the lib dir takes 71 secs.

Do you know of anything that could cause this?

Regards Jaco



Reply | Threaded
Open this post in threaded view
|

Re: performance of addFlatDirResolver

jaco.uys
Hi,

Ok, after some more playing around it seems that the issue is with the amount of resolvers.

I have the following resolvers configured:

        addFlatDirResolver('lib', new File(rootDir, 'lib2'))
        addMavenRepo()
        classpathResolvers.add(name: 'jboss', url:'http://repository.jboss.org/maven2')
        classpathResolvers.add(name: 'java.net', url:'http://download.java.net/maven/2')
        classpathResolvers.add(name: 'jibx', url:'http://jibx.sf.net/maven2')

As more resolvers are added, so the performance of finding a jar in the local lib decreases quite dramatically. I guess there is some check that is done against all resolvers regardless.

Regards Jaco
Reply | Threaded
Open this post in threaded view
|

Re: performance of addFlatDirResolver

hans_d
Administrator
Hi Jaco,

On Jun 7, 2008, at 8:02 PM, jaco.uys wrote:

>
> Hi,
>
> Ok, after some more playing around it seems that the issue is with the
> amount of resolvers.
>
> I have the following resolvers configured:
>
>         addFlatDirResolver('lib', new File(rootDir, 'lib2'))
>         addMavenRepo()
>         classpathResolvers.add(name: 'jboss',
> url:'http://repository.jboss.org/maven2')
>         classpathResolvers.add(name: 'java.net',
> url:'http://download.java.net/maven/2')
>         classpathResolvers.add(name: 'jibx',
> url:'http://jibx.sf.net/maven2')
>
> As more resolvers are added, so the performance of finding a jar in  
> the
> local lib decreases quite dramatically. I guess there is some check  
> that is
> done against all resolvers regardless.

Thanks for the hint. The flatDirResolver is not using a cache, maybe  
this is the reason why there is a always a complete lookup. I have to  
investigate. This is an important issue: http://jira.codehaus.org/ 
browse/GRADLE-109

- Hans

>
> Regards Jaco
> --
> View this message in context: http://www.nabble.com/performance-of- 
> addFlatDirResolver-tp17709833p17711478.html
> Sent from the gradle-user mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>

--
Hans Dockter
Gradle Project lead
http://www.gradle.org





---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: performance of addFlatDirResolver

hans_d
Administrator
In reply to this post by jaco.uys
Hi Jaco,

On Jun 7, 2008, at 8:02 PM, jaco.uys wrote:

>
> Hi,
>
> Ok, after some more playing around it seems that the issue is with the
> amount of resolvers.
>
> I have the following resolvers configured:
>
>         addFlatDirResolver('lib', new File(rootDir, 'lib2'))
>         addMavenRepo()
>         classpathResolvers.add(name: 'jboss',
> url:'http://repository.jboss.org/maven2')
>         classpathResolvers.add(name: 'java.net',
> url:'http://download.java.net/maven/2')
>         classpathResolvers.add(name: 'jibx',
> url:'http://jibx.sf.net/maven2')
>
> As more resolvers are added, so the performance of finding a jar in  
> the
> local lib decreases quite dramatically. I guess there is some check  
> that is
> done against all resolvers regardless.

BTW: If you start Gradle with -j, the debug output of Ivy is printed  
to the console.

- Hans

>
> Regards Jaco
> --
> View this message in context: http://www.nabble.com/performance-of- 
> addFlatDirResolver-tp17709833p17711478.html
> Sent from the gradle-user mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>

--
Hans Dockter
Gradle Project lead
http://www.gradle.org





---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: performance of addFlatDirResolver

hans_d
Administrator
In reply to this post by hans_d

On Jun 9, 2008, at 9:16 AM, Hans Dockter wrote:

> Hi Jaco,
>
> On Jun 7, 2008, at 8:02 PM, jaco.uys wrote:
>
>>
>> Hi,
>>
>> Ok, after some more playing around it seems that the issue is with  
>> the
>> amount of resolvers.
>>
>> I have the following resolvers configured:
>>
>>         addFlatDirResolver('lib', new File(rootDir, 'lib2'))
>>         addMavenRepo()
>>         classpathResolvers.add(name: 'jboss',
>> url:'http://repository.jboss.org/maven2')
>>         classpathResolvers.add(name: 'java.net',
>> url:'http://download.java.net/maven/2')
>>         classpathResolvers.add(name: 'jibx',
>> url:'http://jibx.sf.net/maven2')
>>
>> As more resolvers are added, so the performance of finding a jar  
>> in the
>> local lib decreases quite dramatically. I guess there is some  
>> check that is
>> done against all resolvers regardless.
>
> Thanks for the hint. The flatDirResolver is not using a cache,  
> maybe this is the reason why there is a always a complete lookup. I  
> have to investigate. This is an important issue: http://
> jira.codehaus.org/browse/GRADLE-109

The reason for this behavior is that the internal chain resolver we  
are using is not set to returnFirst = true. I haven't submitted any  
fixes yet as I would like to make the internal chain resolver options  
configurable first: http://jira.codehaus.org/browse/GRADLE-111

- Hans

>
> - Hans
>
>>
>> Regards Jaco
>> --
>> View this message in context: http://www.nabble.com/performance-of- 
>> addFlatDirResolver-tp17709833p17711478.html
>> Sent from the gradle-user mailing list archive at Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>     http://xircles.codehaus.org/manage_email
>>
>>
>
> --
> Hans Dockter
> Gradle Project lead
> http://www.gradle.org
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>

--
Hans Dockter
Gradle Project lead
http://www.gradle.org





---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: performance of addFlatDirResolver

hans_d
Administrator

On Jun 10, 2008, at 1:57 PM, Hans Dockter wrote:

>
> On Jun 9, 2008, at 9:16 AM, Hans Dockter wrote:
>
>> Hi Jaco,
>>
>> On Jun 7, 2008, at 8:02 PM, jaco.uys wrote:
>>
>>>
>>> Hi,
>>>
>>> Ok, after some more playing around it seems that the issue is  
>>> with the
>>> amount of resolvers.
>>>
>>> I have the following resolvers configured:
>>>
>>>         addFlatDirResolver('lib', new File(rootDir, 'lib2'))
>>>         addMavenRepo()
>>>         classpathResolvers.add(name: 'jboss',
>>> url:'http://repository.jboss.org/maven2')
>>>         classpathResolvers.add(name: 'java.net',
>>> url:'http://download.java.net/maven/2')
>>>         classpathResolvers.add(name: 'jibx',
>>> url:'http://jibx.sf.net/maven2')
>>>
>>> As more resolvers are added, so the performance of finding a jar  
>>> in the
>>> local lib decreases quite dramatically. I guess there is some  
>>> check that is
>>> done against all resolvers regardless.
>>
>> Thanks for the hint. The flatDirResolver is not using a cache,  
>> maybe this is the reason why there is a always a complete lookup.  
>> I have to investigate. This is an important issue: http://
>> jira.codehaus.org/browse/GRADLE-109
>
> The reason for this behavior is that the internal chain resolver we  
> are using is not set to returnFirst = true. I haven't submitted any  
> fixes yet as I would like to make the internal chain resolver  
> options configurable first: http://jira.codehaus.org/browse/GRADLE-111

Both issues are fixed in svn.

- Hans

>
> - Hans
>
>>
>> - Hans
>>
>>>
>>> Regards Jaco
>>> --
>>> View this message in context: http://www.nabble.com/performance- 
>>> of-addFlatDirResolver-tp17709833p17711478.html
>>> Sent from the gradle-user mailing list archive at Nabble.com.
>>>
>>>
>>> --------------------------------------------------------------------
>>> -
>>> To unsubscribe from this list, please visit:
>>>
>>>     http://xircles.codehaus.org/manage_email
>>>
>>>
>>
>> --
>> Hans Dockter
>> Gradle Project lead
>> http://www.gradle.org
>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>    http://xircles.codehaus.org/manage_email
>>
>>
>
> --
> Hans Dockter
> Gradle Project lead
> http://www.gradle.org
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>

--
Hans Dockter
Gradle Project lead
http://www.gradle.org





---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email


Reply | Threaded
Open this post in threaded view
|

Re: performance of addFlatDirResolver

jaco.uys
Thanks!.

hdockter wrote
On Jun 10, 2008, at 1:57 PM, Hans Dockter wrote:

>
> On Jun 9, 2008, at 9:16 AM, Hans Dockter wrote:
>
>> Hi Jaco,
>>
>> On Jun 7, 2008, at 8:02 PM, jaco.uys wrote:
>>
>>>
>>> Hi,
>>>
>>> Ok, after some more playing around it seems that the issue is  
>>> with the
>>> amount of resolvers.
>>>
>>> I have the following resolvers configured:
>>>
>>>         addFlatDirResolver('lib', new File(rootDir, 'lib2'))
>>>         addMavenRepo()
>>>         classpathResolvers.add(name: 'jboss',
>>> url:'http://repository.jboss.org/maven2')
>>>         classpathResolvers.add(name: 'java.net',
>>> url:'http://download.java.net/maven/2')
>>>         classpathResolvers.add(name: 'jibx',
>>> url:'http://jibx.sf.net/maven2')
>>>
>>> As more resolvers are added, so the performance of finding a jar  
>>> in the
>>> local lib decreases quite dramatically. I guess there is some  
>>> check that is
>>> done against all resolvers regardless.
>>
>> Thanks for the hint. The flatDirResolver is not using a cache,  
>> maybe this is the reason why there is a always a complete lookup.  
>> I have to investigate. This is an important issue: http://
>> jira.codehaus.org/browse/GRADLE-109
>
> The reason for this behavior is that the internal chain resolver we  
> are using is not set to returnFirst = true. I haven't submitted any  
> fixes yet as I would like to make the internal chain resolver  
> options configurable first: http://jira.codehaus.org/browse/GRADLE-111

Both issues are fixed in svn.

- Hans

>
> - Hans
>
>>
>> - Hans
>>
>>>
>>> Regards Jaco
>>> --
>>> View this message in context: http://www.nabble.com/performance- 
>>> of-addFlatDirResolver-tp17709833p17711478.html
>>> Sent from the gradle-user mailing list archive at Nabble.com.
>>>
>>>
>>> --------------------------------------------------------------------
>>> -
>>> To unsubscribe from this list, please visit:
>>>
>>>     http://xircles.codehaus.org/manage_email
>>>
>>>
>>
>> --
>> Hans Dockter
>> Gradle Project lead
>> http://www.gradle.org
>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>    http://xircles.codehaus.org/manage_email
>>
>>
>
> --
> Hans Dockter
> Gradle Project lead
> http://www.gradle.org
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>

--
Hans Dockter
Gradle Project lead
http://www.gradle.org





---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Reply | Threaded
Open this post in threaded view
|

Re: performance of addFlatDirResolver

hans_d
Administrator

On Jun 16, 2008, at 10:30 PM, jaco.uys wrote:

>
> Thanks!.

Orginally I wanted to release Gradle 0.2 today. But while updating  
the user's guide I've discovered a couple of minor issues. I'm  
confident to release tomorrow.

- Hans

>
>
> hdockter wrote:
>>
>>
>> On Jun 10, 2008, at 1:57 PM, Hans Dockter wrote:
>>
>>>
>>> On Jun 9, 2008, at 9:16 AM, Hans Dockter wrote:
>>>
>>>> Hi Jaco,
>>>>
>>>> On Jun 7, 2008, at 8:02 PM, jaco.uys wrote:
>>>>
>>>>>
>>>>> Hi,
>>>>>
>>>>> Ok, after some more playing around it seems that the issue is
>>>>> with the
>>>>> amount of resolvers.
>>>>>
>>>>> I have the following resolvers configured:
>>>>>
>>>>>         addFlatDirResolver('lib', new File(rootDir, 'lib2'))
>>>>>         addMavenRepo()
>>>>>         classpathResolvers.add(name: 'jboss',
>>>>> url:'http://repository.jboss.org/maven2')
>>>>>         classpathResolvers.add(name: 'java.net',
>>>>> url:'http://download.java.net/maven/2')
>>>>>         classpathResolvers.add(name: 'jibx',
>>>>> url:'http://jibx.sf.net/maven2')
>>>>>
>>>>> As more resolvers are added, so the performance of finding a jar
>>>>> in the
>>>>> local lib decreases quite dramatically. I guess there is some
>>>>> check that is
>>>>> done against all resolvers regardless.
>>>>
>>>> Thanks for the hint. The flatDirResolver is not using a cache,
>>>> maybe this is the reason why there is a always a complete lookup.
>>>> I have to investigate. This is an important issue: http://
>>>> jira.codehaus.org/browse/GRADLE-109
>>>
>>> The reason for this behavior is that the internal chain resolver we
>>> are using is not set to returnFirst = true. I haven't submitted any
>>> fixes yet as I would like to make the internal chain resolver
>>> options configurable first: http://jira.codehaus.org/browse/ 
>>> GRADLE-111
>>
>> Both issues are fixed in svn.
>>
>> - Hans
>>
>>>
>>> - Hans
>>>
>>>>
>>>> - Hans
>>>>
>>>>>
>>>>> Regards Jaco
>>>>> --
>>>>> View this message in context: http://www.nabble.com/performance-
>>>>> of-addFlatDirResolver-tp17709833p17711478.html
>>>>> Sent from the gradle-user mailing list archive at Nabble.com.
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------
>>>>> --
>>>>> -
>>>>> To unsubscribe from this list, please visit:
>>>>>
>>>>>     http://xircles.codehaus.org/manage_email
>>>>>
>>>>>
>>>>
>>>> --
>>>> Hans Dockter
>>>> Gradle Project lead
>>>> http://www.gradle.org
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> -------------------------------------------------------------------
>>>> --
>>>> To unsubscribe from this list, please visit:
>>>>
>>>>    http://xircles.codehaus.org/manage_email
>>>>
>>>>
>>>
>>> --
>>> Hans Dockter
>>> Gradle Project lead
>>> http://www.gradle.org
>>>
>>>
>>>
>>>
>>>
>>> --------------------------------------------------------------------
>>> -
>>> To unsubscribe from this list, please visit:
>>>
>>>    http://xircles.codehaus.org/manage_email
>>>
>>>
>>
>> --
>> Hans Dockter
>> Gradle Project lead
>> http://www.gradle.org
>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe from this list, please visit:
>>
>>     http://xircles.codehaus.org/manage_email
>>
>>
>>
>>
>
> --
> View this message in context: http://www.nabble.com/performance-of- 
> addFlatDirResolver-tp17709833p17872709.html
> Sent from the gradle-user mailing list archive at Nabble.com.
>
>
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>

--
Hans Dockter
Gradle Project lead
http://www.gradle.org





---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email