Unicode bug

[jesus2099]

Hello,

In the statistic screens where we can see the pages' <title>, the list is not displaying correctly when one uses international characters or even accentuated latin characters.

When we have a look at the source code of thise page (keyword.php) we see this:

<META HTTP-EQUIV="content-type" CONTENT="text/html; charset=ISO-8859-1">

For correct display, it should be:

<meta http-equiv="Content-Type" content="text/html; charset=utf-8">

This would not cause any trouble to non unicode browser and this would display correct characters in unicode browsers.

You can have a look at this thread for some more information.

[jesus2099]

Will this easy issue ever be solved?
I would thank you very much for that.
Well maybe it's not that easy?
On my pages, I just had to specify the correct charset [utf-8] and that was all.

[webado]

Stickied this thread.

[webado]

I am wondering if the database would also need to be modified in order to alllow UTF-8 characters to be stored and retrieved corectly. And how this will affect everything else. I expect it would require a conversion of currently contained data.

[jesus2099]

Oh yes maybe!
I can help you testing if you want.
Someday, we allready try to put this unicode in place but it didn't work so you rolled back. I don't remember if it was you who tried or someone esle at statcounter.

[webado]

That wasn't me, I only just got to be a mod recently.

[moishier]

Any updates on fixing this bug?

[moishier]

Still waiting...

[jesus2099]

Hello again!

May I come bach with some new ideas on this subject?
Here I go.

Last time we tried just to put those stats pages in utf-8.
That was not working. I think tht's maybe because it's just not enough!

Here are some examples (visitorpath1, visitorpath2 and recentkeywords)

In visitorpath1 we can see :
blog-notes: %u30DE%u30F3%u30AC%u30FB%u30CD%u30C3%u30C8%u9928
instead of (I hope this forum is unicode, I don't remember) :
blog-notes: マンガ・ネット館

In visitorpath2 we can see :
blog-notes: Ng%u1ECDc Lan
instead of :
blog-notes: Ngọc Lan

In recentkeywords, we can see / instead of :
¥Þ¥ó¥¬¡¦¥Í¥Ã¥È´Û
マンガ・ネット館
ngỌc lan
Ngọc Lan
réveil automatique
réveil automatique

Then I think myself that maybe we would have to change the counter.js and the server side code that is called by sc_tracking_url. We maybe have to change also the way the sc_tracking_url is constructed.

For information, I have made a sample page demonstrating what is possible to do with javascript and serverside code concerning unicode texts.

In fact I can see that the sc_tracking_url components are already escaped by javascript.

So there may be only the serverside code to modify now.
I don't know if you are PHP or what but, in my samples, I used the utf8_encode function to tell PHP to accept Unicode before putting my texts in the display page. This way I can manipulate either unicode and simple texts (simple texts are not affected by the function)..

So if your database supports utf-8, alright, if not you can use a base64_encode or equivalent to store in database.

So when you want to display stats for the users, you extract from DB ; then base64_decode ; then utf8_encode ; then use charset utf-8 in the stats display page. Maybe I am not very sure, tests should be driven.
I you want I can be of some help, I can test all this if you have php on your serverside...?

Here is a test page that can help for this kind of situation : Test en ligne d'encodages en PHP et javascript

Thanks,
jesus2099

PS: I can't upload the 3 PNG attachments (they are very small). I tried Opera then FireFox in vain.

[webado]

You cannot upload anything to this forum.

I suspect converting the Statcounter database to a different encoding method would be difficult and would also take long.

Personally I wasn't aware that url's including query strings can be expressed in a non-latin based character set either.