Proposal: way to set an encoding of a Gradle build script

[Yu Nobuoka]

Hi,

I have a proposal to improve the portability of Gradle build scripts.

BACKGROUND:

Currently, Gradle reads a build script file using a system default file
encoding.
(`return IOUtils.toString(inputStream)`)

- Code:
https://github.com/nobuoka/gradle/blob/2d826b213293995f60ca80891094ca201acd8cf7/subprojects/core/src/main/groovy/org/gradle/api/internal/resource/UriResource.java#L60
- Document of `IOUtils.toString`:
http://commons.apache.org/proper/commons-io/javadocs/api-2.4/org/apache/commons/io/IOUtils.html#toString%28java.io.InputStream%29

The fact that a build script file is read using system default file
encoding decreases
portability of a Gradle build script. For example, in case that there is
a project that
has Gradle build script written in UTF-8 encoding, the project will be
built successfully
on a Linux system (default file encoding is UTF-8), but will not be
built successfully
on a Windows system (default file encoding is not UTF-8).

In the past, this issue was topic on Gradle Forums.

-
http://forums.gradle.org/gradle/topics/is_there_a_way_to_tell_gradle_to_read_gradle_build_scripts_using_a_specified_encoding

Written on the topic, the workaround to avoid this problem is set
`-Dfile.encoding=UTF-8`
in the `GRADLE_OPTS` env var. However, setting `-Dfile.encoding=UTF-8' has
a side effect (for instance, the encoding of console output may be chenged),
which won't be wanted.
Therefore, the other way to set an encoding of a Gradle build script is
needed.

PROPOSAL:

I propose that we add new property `buildFileEncoding` to
`ProjectDescriptor` class, and
let Gradle read a build script file with an encoding which is set on
that property.

I wrote such a code, please see:

-
https://github.com/nobuoka/gradle/compare/feature;build_file_encoding;proposal
(it has no test.)

If this change is applyed, we will be able to set an encoding of a
Gradle build script
in settings.gradle file:

=== example
// settings.gradle
rootProject.buildFileName = "otherName.gradle" // we can set a name of a
build file
rootProject.buildFileEncoding = "UTF-8" // we will be able to set a
encoding of a build file
===

Thanks,

--
NOBUOKA Yu,
https://github.com/nobuoka


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Adam Murdoch]

I think we should simply have all Gradle scripts use utf-8 encoding, rather than make this configurable.

On 12 Jan 2014, at 3:29 am, nobuoka <[hidden email]> wrote:

Hi,

I have a proposal to improve the portability of Gradle build scripts.

BACKGROUND:

Currently, Gradle reads a build script file using a system default file
encoding.
(`return IOUtils.toString(inputStream)`)

- Code:
https://github.com/nobuoka/gradle/blob/2d826b213293995f60ca80891094ca201acd8cf7/subprojects/core/src/main/groovy/org/gradle/api/internal/resource/UriResource.java#L60
- Document of `IOUtils.toString`:
http://commons.apache.org/proper/commons-io/javadocs/api-2.4/org/apache/commons/io/IOUtils.html#toString%28java.io.InputStream%29

The fact that a build script file is read using system default file
encoding decreases
portability of a Gradle build script. For example, in case that there is
a project that
has Gradle build script written in UTF-8 encoding, the project will be
built successfully
on a Linux system (default file encoding is UTF-8), but will not be
built successfully
on a Windows system (default file encoding is not UTF-8).

In the past, this issue was topic on Gradle Forums.

-
http://forums.gradle.org/gradle/topics/is_there_a_way_to_tell_gradle_to_read_gradle_build_scripts_using_a_specified_encoding

Written on the topic, the workaround to avoid this problem is set
`-Dfile.encoding=UTF-8`
in the `GRADLE_OPTS` env var. However, setting `-Dfile.encoding=UTF-8' has
a side effect (for instance, the encoding of console output may be chenged),
which won't be wanted.
Therefore, the other way to set an encoding of a Gradle build script is
needed.

PROPOSAL:

I propose that we add new property `buildFileEncoding` to
`ProjectDescriptor` class, and
let Gradle read a build script file with an encoding which is set on
that property.

I wrote such a code, please see:

-
https://github.com/nobuoka/gradle/compare/feature;build_file_encoding;proposal
(it has no test.)

If this change is applyed, we will be able to set an encoding of a
Gradle build script
in settings.gradle file:

=== example
// settings.gradle
rootProject.buildFileName = "otherName.gradle" // we can set a name of a
build file
rootProject.buildFileEncoding = "UTF-8" // we will be able to set a
encoding of a build file
===

Thanks,

--
NOBUOKA Yu,
https://github.com/nobuoka


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email

--

Adam Murdoch
Gradle Co-founder
http://www.gradle.org
VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com

[Yu Nobuoka]

Adam Murdoch wrote

I think we should simply have all Gradle scripts use utf-8 encoding, rather than make this configurable.

Indeed it is one of the ways to avoid problems with encoding of Gradle scripts, but there remains the problem that current Gradle system doesn't always read Gradle scripts using UTF-8 encoding. (Current Gradle system read Gradle scripts using default file encoding.)

Supposing Gradle system supports only Gradle scripts with UTF-8 encoding, I propose changing the statement `IOUtils.toString(inputStream)` on UriResource#getText method into `IOUtils.toString(inputStream, "UTF-8")`.

(I am troubled by this problem when using Gradle on Windows (because its default file encoding is not UTF-8). I know that there is a workaround (setting
`-Dfile.encoding=UTF-8`
in the `GRADLE_OPTS` env var), but its side effect is not wanted.)

[Luke Daley-2]

On 13 Jan 2014, at 4:33 pm, Yu Nobuoka <[hidden email]> wrote:

> Adam Murdoch wrote
>> I think we should simply have all Gradle scripts use utf-8 encoding,
>> rather than make this configurable.
>
> Indeed it is one of the ways to avoid problems with encoding of Gradle
> scripts, but there remains the problem that current Gradle system doesn't
> always read Gradle scripts using UTF-8 encoding. (Current Gradle system read
> Gradle scripts using default file encoding.)
>
> Supposing Gradle system supports only Gradle scripts with UTF-8 encoding, I
> propose changing  the statement `IOUtils.toString(inputStream)` on
> UriResource#getText method
> <https://github.com/nobuoka/gradle/blob/2d826b213293995f60ca80891094ca201acd8cf7/subprojects/core/src/main/groovy/org/gradle/api/internal/resource/UriResource.java#L60>  
> into `IOUtils.toString(inputStream, "UTF-8")`.
>
> (I am troubled by this problem when using Gradle on Windows (because its
> default file encoding is not UTF-8). I know that there is a workaround
> (setting
> `-Dfile.encoding=UTF-8`
> in the `GRADLE_OPTS` env var), but its side effect is not wanted.)

Regardless of what happens with this particular issues, running any JVM in any context without explicitly setting 'file.encoding' is a sure way to madness.

--
Luke Daley
Principal Engineer, Gradleware
http://gradleware.com


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

[Adam Murdoch]

On 14 Jan 2014, at 3:33 am, Yu Nobuoka <[hidden email]> wrote:

Adam Murdoch wrote

I think we should simply have all Gradle scripts use utf-8 encoding,
rather than make this configurable.

Indeed it is one of the ways to avoid problems with encoding of Gradle
scripts, but there remains the problem that current Gradle system doesn't
always read Gradle scripts using UTF-8 encoding. (Current Gradle system read
Gradle scripts using default file encoding.)

We’d change the code, of course, and quite possibly exactly how you’ve suggested below.

The question is more whether this is too big a breaking change to make in a Gradle 1.x release, or whether it should wait for a 2.0 release.

Supposing Gradle system supports only Gradle scripts with UTF-8 encoding, I
propose changing  the statement `IOUtils.toString(inputStream)` on
UriResource#getText method
<https://github.com/nobuoka/gradle/blob/2d826b213293995f60ca80891094ca201acd8cf7/subprojects/core/src/main/groovy/org/gradle/api/internal/resource/UriResource.java#L60>  
into `IOUtils.toString(inputStream, "UTF-8")`.

(I am troubled by this problem when using Gradle on Windows (because its
default file encoding is not UTF-8). I know that there is a workaround
(setting
`-Dfile.encoding=UTF-8`
in the `GRADLE_OPTS` env var), but its side effect is not wanted.)



--
View this message in context: http://gradle.1045684.n5.nabble.com/Proposal-way-to-set-an-encoding-of-a-Gradle-build-script-tp5712199p5712205.html
Sent from the gradle-dev mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email

--

Adam Murdoch
Gradle Co-founder
http://www.gradle.org
VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com

[Luke Daley-2]

On 14 Jan 2014, at 6:42, Adam Murdoch <[hidden email]> wrote:

On 14 Jan 2014, at 3:33 am, Yu Nobuoka <[hidden email]> wrote:

Adam Murdoch wrote

I think we should simply have all Gradle scripts use utf-8 encoding,
rather than make this configurable.

Indeed it is one of the ways to avoid problems with encoding of Gradle
scripts, but there remains the problem that current Gradle system doesn't
always read Gradle scripts using UTF-8 encoding. (Current Gradle system read
Gradle scripts using default file encoding.)

We’d change the code, of course, and quite possibly exactly how you’ve suggested below.

The question is more whether this is too big a breaking change to make in a Gradle 1.x release, or whether it should wait for a 2.0 release.

2.0 in my opinion. It's a breaking change for almost all windows users.

There is a reasonable workaround in the meantime.

Supposing Gradle system supports only Gradle scripts with UTF-8 encoding, I
propose changing  the statement `IOUtils.toString(inputStream)` on
UriResource#getText method
<https://github.com/nobuoka/gradle/blob/2d826b213293995f60ca80891094ca201acd8cf7/subprojects/core/src/main/groovy/org/gradle/api/internal/resource/UriResource.java#L60>  
into `IOUtils.toString(inputStream, "UTF-8")`.

(I am troubled by this problem when using Gradle on Windows (because its
default file encoding is not UTF-8). I know that there is a workaround
(setting
`-Dfile.encoding=UTF-8`
in the `GRADLE_OPTS` env var), but its side effect is not wanted.)



--
View this message in context: http://gradle.1045684.n5.nabble.com/Proposal-way-to-set-an-encoding-of-a-Gradle-build-script-tp5712199p5712205.html
Sent from the gradle-dev mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

   http://xircles.codehaus.org/manage_email

--

Adam Murdoch
Gradle Co-founder
http://www.gradle.org
VP of Engineering, Gradleware Inc. - Gradle Training, Support, Consulting
http://www.gradleware.com