Quantcast

Translating Ivy dep version syntax to Maven in POM generation

classic Classic list List threaded Threaded
25 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Translating Ivy dep version syntax to Maven in POM generation

WonderCsabo
I found out that the gradle maven plugins can generate invalid POMs due to invalid version syntax. This is becuase most gradle users use the Ivy syntax, but that is illegal in the maven POM. Since Gradle supports both Ivy and Maven dependency version syntax, we could translate the version to maven when generating the pom.

I would be glad to help implementing this, because i use lot of projects with both maven and gradle users. I created this thread on the advice of Peter in the initial forums topic. But i am really new to Gradle development, actually i am also not deeply experienced as a user, and i could not even get IntelliJ to import the Gradle codebase correctly. :S
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Translating Ivy dep version syntax to Maven in POM generation

Daz DeBoer-2
Hi
Yes it would be good if Gradle produced POM files that were valid to use in Maven! There are a few current limitations noted in https://github.com/gradle/gradle/blob/master/design-docs/publication-model.md#fix-pom-generation-issues, one of which is dynamic versions. It would be ok if any fix only applied to the 'maven-publish' plugin, but if it worked for the 'maven' plugin as well, that would be cool.

If you want to contribute this change, that would be great. A few pointers to start with:
1. Read https://github.com/gradle/gradle/blob/master/README.md to get started. Do NOT import the project into IntelliJ (this doesn't yet work), but instead use `.gradlew idea` to generate the idea project file that can be opened directly.
2. Take a look at this recent pull request for an idea of the tests and production code that might be involved: https://github.com/gradle/gradle/pull/274. It's important that any contribution include unit tests and integration tests.
3. To test that your changes are good and the tests are passing, run '.gradlew :maven:check'.
3. Any questions regarding getting Gradle to build or design considerations, just post them to this list.

cheers
Daz



On Sun, Jun 29, 2014 at 12:20 PM, WonderCsabo <[hidden email]> wrote:
I found out that the gradle maven plugins can generate invalid POMs due to
invalid version syntax. This is becuase most gradle users use the Ivy
syntax, but that is illegal in the maven POM. Since Gradle supports both Ivy
and Maven dependency version syntax, we could translate the version to maven
when generating the pom.

I would be glad to help implementing this, because i use lot of projects
with both maven and gradle users. I created this thread on the advice of
Peter in the  initial forums topic
<http://forums.gradle.org/gradle/topics/maven_publish_creates_invalid_dependency_version_numbers>
. But i am really new to Gradle development, actually i am also not deeply
experienced as a user, and i could not even get IntelliJ to import the
Gradle codebase correctly. :S



--
View this message in context: http://gradle.1045684.n5.nabble.com/Translating-Ivy-dep-version-syntax-to-Maven-in-POM-generation-tp5712755.html
Sent from the gradle-dev mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email





--
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Translating Ivy dep version syntax to Maven in POM generation

WonderCsabo
I think we should specify the feature first. This is the syntax for Ivy dependencies, and here is for Maven.

- Dynamic versions are not know to Maven at all. How we could translate them? Maybe go to maven central and find out the latest version?
- The plus notation can be easily converted to a Maven version range.
- Finally the mathematical ranges not use the same syntax as Maven. For example (,2.0[ in Maven should read (,2.0). The Ivy syntax is not too consistent...
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Translating Ivy dep version syntax to Maven in POM generation

Adam Murdoch

On 30 Jun 2014, at 7:18 am, WonderCsabo <[hidden email]> wrote:

I think we should specify the feature first.  This
<http://ant.apache.org/ivy/history/2.1.0/ivyfile/dependency.html>   is the
syntax for Ivy dependencies, and  here
<http://docs.codehaus.org/display/MAVEN/Dependency+Mediation+and+Conflict+Resolution#DependencyMediationandConflictResolution-DependencyVersionRanges>  
is for Maven.

- Dynamic versions are not know to Maven at all. How we could translate
them? Maybe go to maven central and find out the latest version?

Assuming we’re talking about ‘latest.<status>’ selectors, I can see a few options:

1. Fail the conversion for anything that cannot be translated.
2. Just copy across anything that cannot be translated.
3. Replace with the version that currently matches the selector. This is lossy.
4. As for #3 but also retain the selector somewhere else in the pom, similar to rev and revConstraint in Ivy.
5. As for #3 but also generate some additional meta-data file that Gradle would use.

The ‘real’ solution here is #5, but for now I would go with #2, as this is no worse than where we are now.

We might also combine #1 or #2 with better options for tweaking the outgoing dependencies before the pom.xml or ivy.xml is generated.


--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
CTO Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Translating Ivy dep version syntax to Maven in POM generation

Justin Ryan
Just to be clear, you're trying to maintain the dynamic constraint and NOT replace them with the resolved version? I point this out since it's conventional to publish a resolved POM and not the range query. I do see the use-case for publishing with a range, but in general it's not what people what.


On Sun, Jun 29, 2014 at 6:37 PM, Adam Murdoch <[hidden email]> wrote:

On 30 Jun 2014, at 7:18 am, WonderCsabo <[hidden email]> wrote:

I think we should specify the feature first.  This
<http://ant.apache.org/ivy/history/2.1.0/ivyfile/dependency.html>   is the
syntax for Ivy dependencies, and  here
<http://docs.codehaus.org/display/MAVEN/Dependency+Mediation+and+Conflict+Resolution#DependencyMediationandConflictResolution-DependencyVersionRanges>  
is for Maven.

- Dynamic versions are not know to Maven at all. How we could translate
them? Maybe go to maven central and find out the latest version?

Assuming we’re talking about ‘latest.<status>’ selectors, I can see a few options:

1. Fail the conversion for anything that cannot be translated.
2. Just copy across anything that cannot be translated.
3. Replace with the version that currently matches the selector. This is lossy.
4. As for #3 but also retain the selector somewhere else in the pom, similar to rev and revConstraint in Ivy.
5. As for #3 but also generate some additional meta-data file that Gradle would use.

The ‘real’ solution here is #5, but for now I would go with #2, as this is no worse than where we are now.

We might also combine #1 or #2 with better options for tweaking the outgoing dependencies before the pom.xml or ivy.xml is generated.


--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
CTO Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com




Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Translating Ivy dep version syntax to Maven in POM generation

WonderCsabo
By the way the dynamic version feature is very rarely used. Actually i never seen it in a real-life gradle project. So option two may suffice for now. Also i was wondering maybe we can replace the `latest` keyword with a half-open range, but that can have culprits.


Regarding the technical side, i ran `gradlew idea`, then opened the ipr file. I have to import the unlinked Gradle project, then setup a project JDK. I got a weird error now in the editor for org.gradle.api.publication.maven.internal.ant.RepositoryFactory does not implement GroovyObject.invokeMethod, but making the project from the IDE or from terminal works like a charm. :S
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Translating Ivy dep version syntax to Maven in POM generation

Radim Kubacki
On Mon, Jun 30, 2014 at 10:54 AM, WonderCsabo <[hidden email]> wrote:
By the way the dynamic version feature is very rarely used. Actually i never
seen it in a real-life gradle project. So option two may suffice for now.
Also i was wondering maybe we can replace the `latest` keyword with a
half-open range, but that can have culprits.


Regarding the technical side, i ran `gradlew idea`, then opened the ipr
file. I have to import the unlinked Gradle project, then setup a project
JDK. I got a weird error now in the editor for
org.gradle.api.publication.maven.internal.ant.RepositoryFactory does not
implement GroovyObject.invokeMethod, but making the project from the IDE or
from terminal works like a charm. :S

I see the same error in my IJ and yet the project can be built in IJ as well as using command line Gradle. My guess is that there is some bug in IJ's parsing code for Groovy files and I'd expect that future updates will fix it.
 
-Radim


--
View this message in context: http://gradle.1045684.n5.nabble.com/Translating-Ivy-dep-version-syntax-to-Maven-in-POM-generation-tp5712755p5712769.html
Sent from the gradle-dev mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Translating Ivy dep version syntax to Maven in POM generation

Daz DeBoer-2
In reply to this post by WonderCsabo
On Mon, Jun 30, 2014 at 1:54 AM, WonderCsabo <[hidden email]> wrote:
By the way the dynamic version feature is very rarely used. Actually i never
seen it in a real-life gradle project. So option two may suffice for now.
Also i was wondering maybe we can replace the `latest` keyword with a
half-open range, but that can have culprits.

Note that when we refer to 'dynamic versions', we mean both version ranges like '1.+' and ivy-specific latest selectors like 'latest.integration'. While you may not have seen the latter used, I'm aware of several very large organisations for which this is vital to their development process. So you might want to be careful when assuming something is "very rarely used" :).

For now, let's map the version range selectors to a Maven-compatible syntax, and simply copy anything else as-is. This will fix many cases where Maven cannot handle a Gradle-generated POM, while keeping the solution simple.
--
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Translating Ivy dep version syntax to Maven in POM generation

Daz DeBoer-2
In reply to this post by Justin Ryan
On Sun, Jun 29, 2014 at 9:54 PM, Justin Ryan <[hidden email]> wrote:
Just to be clear, you're trying to maintain the dynamic constraint and NOT replace them with the resolved version? I point this out since it's conventional to publish a resolved POM and not the range query. I do see the use-case for publishing with a range, but in general it's not what people what.

Yes, we're talking about http://issues.gradle.org/browse/GRADLE-1578: when a dependency contains a version range Gradle doesn't produce a valid POM for consumption by Maven. 

Replacing the 'version selector' with the actual resolved version used would be another feature, which would apply to publishing in both Ivy and Maven metadata formats.



On Sun, Jun 29, 2014 at 6:37 PM, Adam Murdoch <[hidden email]> wrote:

On 30 Jun 2014, at 7:18 am, WonderCsabo <[hidden email]> wrote:

I think we should specify the feature first.  This
<http://ant.apache.org/ivy/history/2.1.0/ivyfile/dependency.html>   is the
syntax for Ivy dependencies, and  here
<http://docs.codehaus.org/display/MAVEN/Dependency+Mediation+and+Conflict+Resolution#DependencyMediationandConflictResolution-DependencyVersionRanges>  
is for Maven.

- Dynamic versions are not know to Maven at all. How we could translate
them? Maybe go to maven central and find out the latest version?

Assuming we’re talking about ‘latest.<status>’ selectors, I can see a few options:

1. Fail the conversion for anything that cannot be translated.
2. Just copy across anything that cannot be translated.
3. Replace with the version that currently matches the selector. This is lossy.
4. As for #3 but also retain the selector somewhere else in the pom, similar to rev and revConstraint in Ivy.
5. As for #3 but also generate some additional meta-data file that Gradle would use.

The ‘real’ solution here is #5, but for now I would go with #2, as this is no worse than where we are now.

We might also combine #1 or #2 with better options for tweaking the outgoing dependencies before the pom.xml or ivy.xml is generated.


--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
CTO Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com







--
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Translating Ivy dep version syntax to Maven in POM generation

WonderCsabo
Then, i propose a module which takes the version declared in the dependency in the .gradle file.

1. If the version cannot be translated to Maven format, because the feature is non-existent in Maven, we should just leave it as is (for example: dynamic versions).
2. If version format is not compatible but can be translated, we transform it so we produce a valid POM (this is for the + notation and the different range syntax).
3. If it is already a Maven format we leave it as is.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Translating Ivy dep version syntax to Maven in POM generation

Daz DeBoer-2

On Wed, Jul 2, 2014 at 7:15 AM, WonderCsabo <[hidden email]> wrote:
Then, i propose a module which takes the version declared in the dependency
in the .gradle file.

1. If the version cannot be translated to Maven format, because the feature
is non-existent in Maven, we should just leave it as is (for example:
dynamic versions).
2. If version format is not compatible but can be translated, we transform
it so we produce a valid POM (this is for the + notation and the different
range syntax).
3. If it is already a Maven format we leave it as is.

This sounds good. But for clarity, I reiterate that in our terminology a "dynamic version" refers to any version criteria that doesn't specify an exact version. So  "1.+", "[1.0,2.0)", "latest.integration" are all "dynamic versions" in our terminology.




--
View this message in context: http://gradle.1045684.n5.nabble.com/Translating-Ivy-dep-version-syntax-to-Maven-in-POM-generation-tp5712755p5712788.html
Sent from the gradle-dev mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email





--
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Translating Ivy dep version syntax to Maven in POM generation

Hans Dockter-2
In reply to this post by WonderCsabo
The Nebula publishing plugin from Netflix provides the functionality to replace in a POM a dynamic version with a resolved version.


Hans


On Wed, Jul 2, 2014 at 4:15 PM, WonderCsabo <[hidden email]> wrote:
Then, i propose a module which takes the version declared in the dependency
in the .gradle file.

1. If the version cannot be translated to Maven format, because the feature
is non-existent in Maven, we should just leave it as is (for example:
dynamic versions).
2. If version format is not compatible but can be translated, we transform
it so we produce a valid POM (this is for the + notation and the different
range syntax).
3. If it is already a Maven format we leave it as is.



--
View this message in context: http://gradle.1045684.n5.nabble.com/Translating-Ivy-dep-version-syntax-to-Maven-in-POM-generation-tp5712755p5712788.html
Sent from the gradle-dev mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Translating Ivy dep version syntax to Maven in POM generation

WonderCsabo
In reply to this post by Daz DeBoer-2
That is right. Actually i misinterpreted the Ivy version specification a little, all things which are not single fixed number (like 1.4.2) are dynamic revisions. But we will translate them except the `latest` selector, which we will leave as is. Sorry for the confusion.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Translating Ivy dep version syntax to Maven in POM generation

WonderCsabo
In reply to this post by Hans Dockter-2
Sorry guys for being silent for almost two months, but i am back to work now.

I have two questions:

* i figured out that Maven actually has a counterpart of the Ivy `latest` selector: `LATEST`. Should we use that?
* i started to think about implementation. This conversion is in `org.gradle.api.publication.maven.internal.ant.DefaultPomDependenciesConverter`, am i right? Here we just simple write out the version String we get from org.gradle.api.artifacts.Dependency#getVersion(). Do we have a class somewhere which interprets an Ivy version string to some Java structure?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Translating Ivy dep version syntax to Maven in POM generation

WonderCsabo
In reply to this post by WonderCsabo
Sorry guys for being silent for almost two months, but i am back to work now.

I have two questions:

* i figured out that Maven actually has a counterpart of the Ivy `latest` selector: `LATEST`. Should we use that?
* i started to think about implementation. This conversion is in `org.gradle.api.publication.maven.internal.ant.DefaultPomDependenciesConverter`, am i right? Here we just simple write out the version String we get from org.gradle.api.artifacts.Dependency#getVersion(). Do we have a class somewhere which interprets an Ivy version string to some Java structure?
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Translating Ivy dep version syntax to Maven in POM generation

Adam Murdoch

On 27 Aug 2014, at 7:48 pm, WonderCsabo <[hidden email]> wrote:

Sorry guys for being silent for almost two months, but i am back to work now.

I have two questions:

* i figured out that Maven actually has a counterpart of the Ivy `latest`
selector: `LATEST`. Should we use that?

Possibly. It’s not entirely clear if this is still supported or not. Perhaps you could try it out with a recent maven version?

* i started to think about implementation. This conversion is in
`org.gradle.api.publication.maven.internal.ant.DefaultPomDependenciesConverter`,
am i right?

Yes.

Here we just simple write out the version String we get from
org.gradle.api.artifacts.Dependency#getVersion(). Do we have a class
somewhere which interprets an Ivy version string to some Java structure?

No, but I’d really like to move to this approach. If you like, you might add a method to VersionMatcher that can parse a string to some new type VersionSelector (say) that represents the version criteria and we can ask questions of. You can then make use of this in DefaultPomDependenciesConverter.

Later, we can then roll this out more widely, so that all version selectors are parsed early and passed around as a VersionSelector instead of a String.


--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
CTO Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Translating Ivy dep version syntax to Maven in POM generation

WonderCsabo
On LATEST: I tried LATEST with Maven 3.2.1 (almost latest stable), and it works currently. But it is strange that i did not found any official documentation on it. Maybe i just cannot use Google... I stumbled upon this old blog entry from Sonatype. According to this, two selectors exist, the LATEST, which is equal to latest.integration in Ivy, and RELEASE which is same as latest.release in Ivy. At least in my interpretation.


On parsing: I hardly believe you do all do dependency tree calculation without such a class. :S But i clearly miss something. But i would gladly create such a class if it's really needed.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Translating Ivy dep version syntax to Maven in POM generation

Adam Murdoch

On 28 Aug 2014, at 9:09 am, WonderCsabo <[hidden email]> wrote:

On LATEST: I tried LATEST with Maven 3.2.1 (almost latest stable), and it
works currently. But it is strange that i did not found any official
documentation on it. Maybe i just cannot use Google... I stumbled upon  this
<http://blog.sonatype.com/2009/12/maven-dependency-resolution-a-repository-perspective/#.U_5hZGM571M>  
old blog entry from Sonatype. According to this, two selectors exist, the
LATEST, which is equal to latest.integration in Ivy, and RELEASE which is
same as latest.release in Ivy. At least in my interpretation.

Given that ‘latest.integration’ and ‘latest.release’ almost certainly won’t work, we may as well map to ‘LATEST’ and ‘RELEASE’ when we can. We should also change the pom parsing code to understand these, so we can use a generated pom in another Gradle build.



On parsing: I hardly believe you do all do dependency tree calculation
without such a class. :S But i clearly miss something. But i would gladly
create such a class if it's really needed.

There is a class, it’s called VersionMatcher. However, you have to parse the string every time you want to ask a question about a selector string, instead of parsing it once and then passing around some representation of the parsed string.


--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
CTO Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com



Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Translating Ivy dep version syntax to Maven in POM generation

WonderCsabo
On Thu, Aug 28, 2014 at 1:23 AM, Adam Murdoch [via Gradle] <[hidden email]> wrote:

On 28 Aug 2014, at 9:09 am, WonderCsabo <[hidden email]> wrote:

On LATEST: I tried LATEST with Maven 3.2.1 (almost latest stable), and it
works currently. But it is strange that i did not found any official
documentation on it. Maybe i just cannot use Google... I stumbled upon  this
<http://blog.sonatype.com/2009/12/maven-dependency-resolution-a-repository-perspective/#.U_5hZGM571M>  

old blog entry from Sonatype. According to this, two selectors exist, the
LATEST, which is equal to latest.integration in Ivy, and RELEASE which is
same as latest.release in Ivy. At least in my interpretation.

Given that ‘latest.integration’ and ‘latest.release’ almost certainly won’t work, we may as well map to ‘LATEST’ and ‘RELEASE’ when we can. We should also change the pom parsing code to understand these, so we can use a generated pom in another Gradle build.

I am getting a little bit confused here. I thought Gradle uses the Ivy version syntax, but understands Maven syntax, too. :S If latest.something does not work at all in Gradle, i think that is another issue, and in that case this version converter does not have to handle latest.something, since it cannot get such an input currently.
 



On parsing: I hardly believe you do all do dependency tree calculation
without such a class. :S But i clearly miss something. But i would gladly
create such a class if it's really needed.

There is a class, it’s called VersionMatcher. However, you have to parse the string every time you want to ask a question about a selector string, instead of parsing it once and then passing around some representation of the parsed string.


OK.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Translating Ivy dep version syntax to Maven in POM generation

Adam Murdoch

On 28 Aug 2014, at 9:32 am, WonderCsabo <[hidden email]> wrote:

On Thu, Aug 28, 2014 at 1:23 AM, Adam Murdoch [via Gradle] <<a href="x-msg://14/user/SendEmail.jtp?type=node&amp;node=5713069&amp;i=0" target="_top" rel="nofollow" link="external">[hidden email]> wrote:

On 28 Aug 2014, at 9:09 am, WonderCsabo <[hidden email]> wrote:

On LATEST: I tried LATEST with Maven 3.2.1 (almost latest stable), and it
works currently. But it is strange that i did not found any official
documentation on it. Maybe i just cannot use Google... I stumbled upon  this
<http://blog.sonatype.com/2009/12/maven-dependency-resolution-a-repository-perspective/#.U_5hZGM571M>  

old blog entry from Sonatype. According to this, two selectors exist, the
LATEST, which is equal to latest.integration in Ivy, and RELEASE which is
same as latest.release in Ivy. At least in my interpretation.

Given that ‘latest.integration’ and ‘latest.release’ almost certainly won’t work, we may as well map to ‘LATEST’ and ‘RELEASE’ when we can. We should also change the pom parsing code to understand these, so we can use a generated pom in another Gradle build.

I am getting a little bit confused here. I thought Gradle uses the Ivy version syntax, but understands Maven syntax, too. :S If latest.something does not work at all in Gradle, i think that is another issue, and in that case this version converter does not have to handle latest.something, since it cannot get such an input currently.

Gradle uses Ivy syntax internally, and in the DSL. For poms, it translates from Maven to Ivy syntax when it parses the pom and passes Ivy version strings around from there.

We don’t want to start supporting ‘RELEASE’ and ‘LATEST’ for Ivy or in the Gradle DSL. So, we want to add the understanding of ‘RELEASE’ and ‘LATEST’ to the pom parsing only.


 



On parsing: I hardly believe you do all do dependency tree calculation
without such a class. :S But i clearly miss something. But i would gladly
create such a class if it's really needed.

There is a class, it’s called VersionMatcher. However, you have to parse the string every time you want to ask a question about a selector string, instead of parsing it once and then passing around some representation of the parsed string.


OK.


View this message in context: Re: Translating Ivy dep version syntax to Maven in POM generation
Sent from the gradle-dev mailing list archive at Nabble.com.


--
Adam Murdoch
Gradle Co-founder
http://www.gradle.org
CTO Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com



12
Loading...