Http download .csv UTF-8

[Merlot2309]

Hello,

Used the HTTPUtils example to download a .csv file.
It functions eventhough I only need 1 file download ......

But I can´t get it encoded as UTF-8 by code.
I´m also not sure if the encoding has to be done in the Service part or in Main.

Thanks in advance.

Helen.

 

[Erel]

The default encoding is UTF8. It is possible that the other web site uses a different encoding. However it should also work with a different encoding correctly that is unless the web site doesn't return the correct encoding header.

 

[Merlot2309]

Hi,

The website source shows meta with charset UTF-8

The csv file has 2 diamonds with ? as the first 2 characters.
Tried to skip them with substring(2), which has no effect what so ever.

Thanks in advance.

Helen.

 

[agraham]

Perhaps the UTF-8 encoded data starts with a Byte Order Mark which case Asc(data.CharAt(0)) will be code 65279. I don't see why you would see two diamonds though.

 

[robotop]

Hello Helen, may be you can find something useful in this php help page:
PHP: utf8_encode - Manual
BTW, the first 4 bytes of the file you're downloading, seems to be 0xFF 0xFE 0x22 0x00.
ciao

 

[agraham]

"FF FE" is a little-endian UTF-16 BOM! "22 00" is the UTF-16 quote character which might be expected to be the first character in a csv file.

 

[Merlot2309]

Hi,

Thank you both for the info and links, I have to do some reading and it has to sink in.

FF FE indeed is the BOM and the signs ÿþ do appear, depending of the reader that I use on my device.
On the PC they appear as 2 beautiful little diamonds, when opened in Notepad.

Am I thinking in the right direction that I have to use a replace?? as mentioned in Robotops link.

I use ES File Explorer from the Market, you can choose from 4 Editors/Readers and I like the design, ha, ha.
Some readers show UTF-8 as the standard setting.
Honoustly I can´t stand it because I know it must be possible to load the file into a database.

Ciao (stolen from Robotop),

Helen.

Ps. BOM is Dutch for bomb. I promise I will explode when I get this runnning

 

[Erel]

You will need to skip the first two bytes. You can use Substring to truncate the first two characters.

 

[Merlot2309]

Hi,

This is the code part that does what it´s suppose to do.

Sub JobDone (Job As String)

   HandleCsv
   If HttpUtils.IsSuccess(csvUrl) Then
      ceesv = HttpUtils.GetString(csvUrl)
   End If
   HttpUtils.Complete = False 
   ceesv = ceesv.SubString(412)
   File.WriteString(File.DirRootExternal, "vogeltjes.csv", ceesv)

End Sub

I skipped the column names completely (double column names, names with spaces ..........).

That's it for now. Have to start with some new sql commands, so maybe, maybe I have to alter the code sample above.

Thanks for the help!

Helen.