HTTP Post fails if Umlauts sent and UTF-8 is used

Discussion in 'Questions (Windows Mobile)' started by TWELVE, Apr 28, 2008.

  1. TWELVE

    TWELVE Active Member Licensed User

    Hello,

    i got another one:

    The code below works fine, but as soon as "mystring" contains Umlauts, an error message complaining about length mismatch occurs.


    Error description:
    Bytes to be written to the stream exceed the Content-Length size specified.



    stream is a binary file object.
    Request is a webrequest.


    Accordingly to the Helpfile, the binaryfile.new1( , false or true) is set
    to true for ASCII and to false for UTF-8:


    If i set this to true, no error message occurs ( because there's no mismatch between stream length and content-length anymore), but the Umlauts are coverted to something like "?".So ASCII does not help here.



    regards

    TWELVE
     
  2. Erel

    Erel Administrator Staff Member Licensed User

    Can you post a simple text you are trying to send?
     
  3. agraham

    agraham Expert Licensed User

    This is because the umlauts are converted to 2 byte characters in UTF8 so lengthening the string. Try converting to a buffer then using the buffer size like this -

    Code:
    Sub Globals
          
    Dim buffer(0As Byte
    End Sub


    Sub App_Start

      buffer() = stream.StringToBytes(
    "mystring")
      Request.ContentLength = ArrayLen(buffer()) 
      ...
      stream.New1(Request.GetStream,
    false)
      ...
      stream.WriteBytes(buffer()) 

    End Sub
     
  4. TWELVE

    TWELVE Active Member Licensed User

    Sure...Umlauts are Ä,Ö and Ü ( lower case ä,ö,ü).

    "Ich fahre nach Österreich"

    kind regards

    TWELVE
     
  5. Erel

    Erel Administrator Staff Member Licensed User

    As Agraham wrote you should use the buffer size and not the string size.
    You will need to add a Bitwise object:
    Code:
    request.New1(...)
        request.Method = 
    "POST"
        request.ContentType = 
    "application/x-www-form-urlencoded"
        bitwise.New1
        s = 
    "Ich fahre nach Österreich"
        buffer() = bitwise.StringToBytes(s,
    0,StrLength(s))
        request.ContentLength = ArrayLen(buffer())
        stream.New1(request.GetStream,
    true)
        stream.WriteBytes(buffer())
        response.New1
        response.Value = request.GetResponse
        textbox1.Text = response.GetString
     
  6. agraham

    agraham Expert Licensed User

    Silly me, I used the stream before it was opened :sign0161: But out of interest do you actually need the Bitwise object? Would this work?
    Code:
    request.New1(...)
        request.Method = 
    "POST"
        request.ContentType = 
    "application/x-www-form-urlencoded"
        s = 
    "Ich fahre nach Österreich"
        stream.New1(request.GetStream,
    false' UTF8 encoding
        buffer() = stream.StringToBytes(s) ' use the stream to convert
        request.ContentLength = ArrayLen(buffer())
        stream.WriteBytes(buffer())
        response.New1
        response.Value = request.GetResponse
        textbox1.Text = response.GetString
    There are some subtleties here that I don't understand :confused: Why does Erel's example open the stream as ASCII? And does the Bitwise StringToBytes method return UTF8 formatted bytes/characters
     
  7. Erel

    Erel Administrator Staff Member Licensed User

    Bitwise.New1 uses the default UTF8 encoding.
    Bitwise.New2 allows you to choose other encodings.

    As we are using the stream just to write bytes and not strings it doesn't matter whether we open it as ASCII or UTF8. The encoding only matters when reading or writing strings.

    request.ContentLength must be set before request.GetStream. Otherwise you will get an error.
     
  8. TWELVE

    TWELVE Active Member Licensed User

    Hello,

    i solved this issue now by using:

    instead of using:

    After some research and trials i found the code page 1252 to be appropriate for the german umlauts.


    I do understand the difference between the count of characters and the count of bytes they allocate.I didn't know how to get that calculated properly and clean...thx to agraham and Erel for your code examples.

    Usually Unicode or UTF-8 is they better choice for supporting different languages / characters.For the moment i will stay with the code page, since this is the quicker fix for me ( the webserver also need an additional module for UTF-8 support..)

    @Erel: is it possible to implement something like "STRByteLength(string)" to have the bytes of the string counted instead of the characters..? Or maybe as additional parameter in STRLength(string, mode)...?


    kind regards

    TWELVE
     
    Last edited: Apr 28, 2008
  9. Erel

    Erel Administrator Staff Member Licensed User

    The length depends on the encoding used.
    As agraham and I wrote in previous posts you can measure it by first converting the string to bytes (using a specific encoder/code page).
     
  10. agraham

    agraham Expert Licensed User

    Hmm! Thanks Erel, it is times like this when you realise how little you really know about some things - like character codings :confused:

    A case for the Door library Erel?
     
  11. Erel

    Erel Administrator Staff Member Licensed User

    I don't think that it will be simpler than using a Bitwise object:
    Code:
    Sub App_Start
        bitwise.New2(RequiredCodePage)
        
    Msgbox(STRByteLength("Some string"))
    End Sub

    Sub STRByteLength(str)
        
    Return ArrayLen(bitwise.StringToBytes(str,0,StrLength(str)))
    End Sub
     
  12. TWELVE

    TWELVE Active Member Licensed User

    Thanks again Gentlemen for your rapid response in that matter...i really appreciate...

    kind regards

    TWELVE
     
Loading...
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice