Android Question XmlSax is crashing when using Hebrew

[Shay]

Using this code:

Getting the XML file from server:

        Select Job.JobName
            Case "Post1"

                Dim OutStream As OutputStream
                OutStream = File.OpenOutput(File.DirInternal, "Global.xml", False) ' Job.Tag is read to set the Original Filename we specify earlier in the creation of the Job
                File.Copy2(Job.GetInputStream,OutStream) ' save the file
                OutStream.Close
                in = File.OpenInput(File.DirInternal, "Global.xml")
                parser.Parse(in, "Parser")
                in.Close

XML pharsing

Sub Parser_EndElement (Uri As String, Name As String, Text As StringBuilder)
    If parser.Parents.IndexOf("item") > -1 Then
        If Name = "title" Then
            Title = Text.ToString
        Else If Name = "link" Then
            Link = Text.ToString
        Else if Name = "color" Then
            TitleColor = Text.ToString   
        End If
    End If
    If Name = "item" Then
        clv1.AddTextItem(Title, Link) 'add the title as the text and the link as the value   
    End If
End Sub

XML file:

<?xml version="1.0" encoding="windows-1255"?>

<rss version="2.0">
    <channel>
       
        <item>
            <title>ששש</title>
            <color>5</color>
            <link>1</link>
            <description>none</description>
        </item>

        <item>
            <title>bbb</title>
            <color>6</color>
            <link>2</link>
            <description>none</description>
        </item>

    </channel>
</rss>

logs:

MainMenu, Response from server: <?xml version="1.0" encoding="windows-1255"?>
<rss version="2.0">
    <channel>
      
        <item>
            <title>���</title> <--------- should be Hebrew
            <color>5</color>
            <link>1</link>
            <description>none</description>
        </item>
        <item>
            <title>bbb</title>
            <color>6</color>
            <link>2</link>
            <description>none</description>
        </item>
    </channel>
</rss>
Error occurred on line: 103 (MainMenu)
org.apache.harmony.xml.ExpatParser$ParseException: At line 7, column 10: not well-formed (invalid token)
    at org.apache.harmony.xml.ExpatParser.parseFragment(ExpatParser.java:515)
    at org.apache.harmony.xml.ExpatParser.parseDocument(ExpatParser.java:474)
    at org.apache.harmony.xml.ExpatReader.parse(ExpatReader.java:321)
    at org.apache.harmony.xml.ExpatReader.parse(ExpatReader.java:279)
    at anywheresoftware.b4a.objects.SaxParser.parse(SaxParser.java:80)
    at anywheresoftware.b4a.objects.SaxParser.Parse(SaxParser.java:73)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:511)
    at anywheresoftware.b4a.shell.Shell.runMethod(Shell.java:702)
    at anywheresoftware.b4a.shell.Shell.raiseEventImpl(Shell.java:339)
    at anywheresoftware.b4a.shell.Shell.raiseEvent(Shell.java:246)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:511)
    at anywheresoftware.b4a.ShellBA.raiseEvent2(ShellBA.java:134)
    at anywheresoftware.b4a.BA$2.run(BA.java:328)
    at android.os.Handler.handleCallback(Handler.java:615)
    at android.os.Handler.dispatchMessage(Handler.java:92)
    at android.os.Looper.loop(Looper.java:137)
    at android.app.ActivityThread.main(ActivityThread.java:4921)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:511)
    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1027)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:794)
    at dalvik.system.NativeStart.main(Native Method)
** Service (starter) Destroy **
** Activity (mainmenu) Pause, UserClosed = true **
** Service (widget) Start **

 

[Erel]

The file is not encoded in UTF8. You need to use TextReader to load it with the correct encoding:

Dim tr As TextReader
tr.Initialize2(in, "Windows-1255")
parser.Parse2(tr, ...)

 

[moster67]

You should be able to use my ICUB4A-library to help you to determine the character-encoding.
https://www.b4x.com/android/forum/threads/icub4a-detecting-character-encoding-formats.65411/

 

[Shay]

parser.Parse (this must be inputSteam - I am getting compile error

Compiling generated Java code.    Error
B4A line: 140
parser.Parse(tr, \
javac 1.8.0_73
src\Av\co\il\mainmenu.java:536: error: incompatible types: BufferedReader cannot be converted to InputStream
mostCurrent._parser.Parse((java.io.InputStream)(_tr.getObject()),"Parser");

    Dim OutStream As OutputStream
                OutStream = File.OpenOutput(File.DirInternal, "Global.xml", False) ' Job.Tag is read to set the Original Filename we specify earlier in the creation of the Job
                File.Copy2(Job.GetInputStream,OutStream) ' save the file
                OutStream.Close
                in = File.OpenInput(File.DirInternal, "Global.xml")
                in.Close
                Dim tr As TextReader
                tr.Initialize2(in, "Windows-1255")
                parser.Parse(tr, "Parser")
                tr.Close

I also tried in = tr

 

[DonManfred]

i guess tr should be a inputstream? The XX in parser.Parse(XX,yyy)
Why not using Job.GetInputstream at this point then? In fact you dont nee to save the stream to a xml first.

Forget it. i think i am wrong.

 

[Shay]

I bypass the above issue using this, so now this is not hebrew issue, still it is crashing with different stacks, if title is hebrew or title is english
I think I will move to DB instead of XML files, this library is not stable

code I used:

ub ReadTextReader (FileName As String ,NewFileName As String)
    Dim TextReader1 As TextReader
    Dim TextWriter1 As TextWriter
    TextReader1.Initialize2(File.OpenInput(File.DirInternal, FileName ),"Windows-1255")
    TextWriter1.Initialize2(File.OpenOutput(File.DirInternal, NewFileName , False),"Windows-1255")
    Dim line As String
    line = TextReader1.ReadLine   
    Do While line <> Null
        Log(line) 'write the line to LogCat
        line = TextReader1.ReadLine
        TextWriter1.WriteLine(line)
    Loop
    TextReader1.Close
    TextWriter1.Close
End Sub

and then I am using
ReadTextReader("Global.xml", "Global1.xml")
in = File.OpenInput(File.DirInternal, "Global1.xml")




stack while using hebrew:

MainMenu, Response from server: <?xml version="1.0" encoding="windows-1255"?>
<rss version="2.0">
    <channel>
       
        <item>
            <title>���</title>
            <color>5</color>
            <link>1</link>
            <description>none</description>
        </item>
        <item>
            <title>bbb</title>
            <color>6</color>
            <link>2</link>
            <description>none</description>
        </item>
    </channel>
</rss>
<?xml version="1.0" encoding="windows-1255"?>
<rss version="2.0">
    <channel>
       
        <item>
            <title>ששש</title>
            <color>5</color>
            <link>1</link>
            <description>none</description>
        </item>
        <item>
            <title>bbb</title>
            <color>6</color>
            <link>2</link>
            <description>none</description>
        </item>
    </channel>
</rss>
Error occurred on line: 104 (MainMenu)
org.apache.harmony.xml.ExpatParser$ParseException: At line 6, column 10: not well-formed (invalid token)
    at org.apache.harmony.xml.ExpatParser.parseFragment(ExpatParser.java:515)
    at org.apache.harmony.xml.ExpatParser.parseDocument(ExpatParser.java:474)
    at org.apache.harmony.xml.ExpatReader.parse(ExpatReader.java:321)
    at org.apache.harmony.xml.ExpatReader.parse(ExpatReader.java:279)
    at anywheresoftware.b4a.objects.SaxParser.parse(SaxParser.java:80)
    at anywheresoftware.b4a.objects.SaxParser.Parse(SaxParser.java:73)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:511)
    at anywheresoftware.b4a.shell.Shell.runMethod(Shell.java:702)
    at anywheresoftware.b4a.shell.Shell.raiseEventImpl(Shell.java:339)
    at anywheresoftware.b4a.shell.Shell.raiseEvent(Shell.java:246)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:511)
    at anywheresoftware.b4a.ShellBA.raiseEvent2(ShellBA.java:134)
    at anywheresoftware.b4a.BA$2.run(BA.java:328)
    at android.os.Handler.handleCallback(Handler.java:615)
    at android.os.Handler.dispatchMessage(Handler.java:92)
    at android.os.Looper.loop(Looper.java:137)
    at android.app.ActivityThread.main(ActivityThread.java:4921)
    at java.lang.reflect.Method.invokeNative(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:511)
    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1027)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:794)
    at dalvik.system.NativeStart.main(Native Method)

 

[Erel]

The library is stable. Please create a small project that demonstrates the problem and upload it (with the XML file).

 

[Shay]

Erel just use the demo project that is attached to the library and use the XML I posted above, change first title to Hebrew
The XML need to be downloaded from some web server

 

[Erel]

The error you encounter is related to text encoding. The only way to test it is to either download the XML directly from the web server or if you upload the downloaded file.

 

[Shay]

What do you mean direct download, this is what I did, first downloaded the file

 

[Erel]

If you want me to help you then you need to upload a project that demonstrates the problem. Either include the code that download the XML file or include the XML file itself and load it from File.DirAssets.

 

[ilan]

parser.Parse (this must be inputSteam - I am getting compile error

Dim tr As TextReader
tr.Initialize2(in, "Windows-1255")
parser.Parse2(tr, ...)

you can see that erel is using parser.Parse2(...,...) "2"!!!