tests: use consistent environment variables for setting charset

The character set in POSIX is set by the locale defined by (in
decreasing order of precedence) the LC_ALL, LC_CTYPE and LANG
environment variables (CHARSET was used by libidn but not libidn2).
LC_ALL is cleared to ensure that LC_CTYPE takes effect, but LC_ALL is
not used to set the locale to ensure that other parts of the locale
aren't overridden.  Since there doesn't seem to be a cross-platform way
of specifying a UTF-8 locale, and not all systems may support UTF-8, a
<precheck> is used to skip the test if UTF-8 can't be verified to be
available.  Test 1035 was also converted to UTF-8 for consistency, as
the actual character set used there is irrelevant to the test.

This patch uses a different UTF-8 locale than the last attempt, namely
en_US.UTF-8. This one has been verified on 7 different Linux and BSD
distributions and is more complete and usable than the locale UTF-8 (on
at least some systems).

5 files changed, 28 insertions, 12 deletions

diff --git a/tests/data/test1034 b/tests/data/test1034
index c47574051..063f52b32 100644
--- a/tests/data/test1034
+++ b/tests/data/test1034
@@ -32,8 +32,12 @@ http
 idn
 </features>
 <setenv>
-CHARSET=UTF-8
+LC_ALL=
+LC_CTYPE=en_US.UTF-8
 </setenv>
+<precheck>
+perl -MI18N::Langinfo=langinfo,CODESET -e 'die "Needs a UTF-8 locale" if (lc(langinfo(CODESET())) ne "utf-8");'
+</precheck>
  <name>
 HTTP over proxy with malformatted IDN host name
  </name>
diff --git a/tests/data/test1035 b/tests/data/test1035
index 05172ec9a..033a48a72 100644
--- a/tests/data/test1035
+++ b/tests/data/test1035
@@ -32,14 +32,17 @@ http
 idn
 </features>
 <setenv>
-CHARSET=UTF-8
-LANG=en_US.UTF-8
+LC_ALL=
+LC_CTYPE=en_US.UTF-8
 </setenv>
+<precheck>
+perl -MI18N::Langinfo=langinfo,CODESET -e 'die "Needs a UTF-8 locale" if (lc(langinfo(CODESET())) ne "utf-8");'
+</precheck>
  <name>
 HTTP over proxy with too long IDN host name
  </name>
  <command>
-http://too-long-IDN-name-cürl-rüles-la-la-la-dee-da-flooby-nooby.local/page/1035 -x %HOSTIP:%HTTPPORT
+http://too-long-IDN-name-cürl-rüles-la-la-la-dee-da-flooby-nooby.local/page/1035 -x %HOSTIP:%HTTPPORT
 </command>
 </client>
 
@@ -50,8 +53,8 @@ http://too-long-IDN-name-cürl-rüles-la-la-la-dee-da-flooby-nooby.local/page/1035
 ^User-Agent:.*
 </strip>
 <protocol>
-GET http://too-long-IDN-name-cürl-rüles-la-la-la-dee-da-flooby-nooby.local/page/1035 HTTP/1.1

-Host: too-long-IDN-name-cürl-rüles-la-la-la-dee-da-flooby-nooby.local

+GET http://too-long-IDN-name-cürl-rüles-la-la-la-dee-da-flooby-nooby.local/page/1035 HTTP/1.1

+Host: too-long-IDN-name-cürl-rüles-la-la-la-dee-da-flooby-nooby.local

 Accept: */*

 Proxy-Connection: Keep-Alive

 

diff --git a/tests/data/test165 b/tests/data/test165
index 6fc0ad2d5..b9a1ed786 100644
--- a/tests/data/test165
+++ b/tests/data/test165
@@ -31,9 +31,12 @@ http
 idn
 </features>
 <setenv>
-CHARSET=UTF-8
-LANG=en_US.UTF-8
+LC_ALL=
+LC_CTYPE=en_US.UTF-8
 </setenv>
+<precheck>
+perl -MI18N::Langinfo=langinfo,CODESET -e 'die "Needs a UTF-8 locale" if (lc(langinfo(CODESET())) ne "utf-8");'
+</precheck>
  <name>
 HTTP over proxy with IDN host name
  </name>
diff --git a/tests/data/test2046 b/tests/data/test2046
index 0f0eb52c6..dcd202f04 100644
--- a/tests/data/test2046
+++ b/tests/data/test2046
@@ -41,9 +41,12 @@ http
 idn
 </features>
 <setenv>
-CHARSET=UTF-8
-LANG=en_US.UTF-8
+LC_ALL=
+LC_CTYPE=en_US.UTF-8
 </setenv>
+<precheck>
+perl -MI18N::Langinfo=langinfo,CODESET -e 'die "Needs a UTF-8 locale" if (lc(langinfo(CODESET())) ne "utf-8");'
+</precheck>
  <name>
 Connection re-use with IDN host name
  </name>
diff --git a/tests/data/test2047 b/tests/data/test2047
index f54a30af6..fc140486b 100644
--- a/tests/data/test2047
+++ b/tests/data/test2047
@@ -42,9 +42,12 @@ http
 idn
 </features>
 <setenv>
-CHARSET=UTF-8
-LANG=en_US.UTF-8
+LC_ALL=
+LC_CTYPE=en_US.UTF-8
 </setenv>
+<precheck>
+perl -MI18N::Langinfo=langinfo,CODESET -e 'die "Needs a UTF-8 locale" if (lc(langinfo(CODESET())) ne "utf-8");'
+</precheck>
  <name>
 Connection re-use with IDN host name over HTTP proxy
  </name>