B4J Question Speed of JSON vs. usage of Substring

[kostefar]

Dear All,

On my old i7 quadcore, the below loop made for testing takes 44 seconds to run through 100 mio. times:

Log (DateTime.Time(DateTime.Now))
    For i = 0 To 100000000
    Dim JSON As JSONParser
    Dim str2 As String = $"{"a":"1","b":"2","c":"3"}"$
    JSON.Initialize(str2)

    Dim M As Map = JSON.NextObject

    Dim a As String = (M.Get ("a"))
    Dim b As String = (M.Get ("b"))
    Dim c As String = (M.Get ("c"))
    Next
    Log (DateTime.Time(DateTime.Now))

Whereas the more "manual" approach below takes only 9 seconds:

Log (DateTime.Time(DateTime.Now))
    For i = 0 To 100000000
    
        Dim str2 As String = $"{"a":"1","b":"2","c":"3"}"$
        Dim a As String = str2.SubString2 (str2.IndexOf ($""a""$) + 5,str2.IndexOf2 ($"","$, str2.IndexOf ($","a""$)))
        Dim b As String = str2.SubString2 (str2.IndexOf ($""b""$) + 5,str2.IndexOf2 ($"","$, str2.IndexOf ($","b""$)))
        Dim c As String = str2.SubString2 (str2.IndexOf ($""c""$) + 5,str2.IndexOf2 ($""}"$, str2.IndexOf ($","c""$)))
        
    Next
    Log (DateTime.Time(DateTime.Now))

Can I conclude, based on the above, that JSON is not very efficient on parsing (at least short) inputs in comparison to using the Substring function, or is there a more efficient way of doing it with JSON than this?
This matters to me when 100´s of strings need to be parsed every second.
My JSON library version is 1.21.

 

[William Lancee]

44 seconds for 100,000,000 iterations is really fast for what the JSON parser does.
It takes ANY valid JSON string which can include lists and maps as well as scalars.
All variations of the string in terms of line breaks and blank spaces are valid.

Ultimately, to access them you need to lookup an item in a map.

So yes, if your input strings are as simple as the above and have a consistent format you can do it faster with a highly specific
parser that won't be flexible if the format changes.

Is worth the effort to reduce the 100,000,000 iterations job from 44 to 9 seconds?

"This matters to me when 100´s of strings need to be parsed every second."

Does this mean the data comes into the app in a dynamic manner, or are you reading a file.

If it comes in a dynamic manner, what is the time between reads? Probably way longer than time to process the string.
If the file is fairly static, you can preprocess the file and transform it into a simpler format that can be read even faster.

 

[kostefar]

44 seconds for 100,000,000 iterations is really fast for what the JSON parser does.
It takes ANY valid JSON string which can include lists and maps as well as scalars.
All variations of the string in terms of line breaks and blank spaces are valid.

Ultimately, to access them you need to lookup an item in a map.

So yes, if your input strings are as simple as the above and have a consistent format you can do it faster with a highly specific
parser that won't be flexible if the format changes.

Is worth the effort to reduce the 100,000,000 iterations job from 44 to 9 seconds?

"This matters to me when 100´s of strings need to be parsed every second."

Does this mean the data comes into the app in a dynamic manner, or are you reading a file.

If it comes in a dynamic manner, what is the time between reads? Probably way longer than time to process the string.
If the file is fairly static, you can preprocess the file and transform it into a simpler format that can be read even faster.

Thanks William,

These are data coming in via websockets that are inserted in a MySQL table which needs to be reacted on in as little time as possible. So I will then stick to my own parser code, but good to know that I chose the most efficient approach. I do understand of course that in many other situations JSON would be preferable.

 

[OliverA]

Please note that JSON will be more robust of for some reason the sequence of the objects change within the JSON string. So instead of the string being in a neat order of abc, if you receive cba, your string parsing will collect the wrong data. Is the extra time saved worth accidentally saving the wrong data or are you 100% guaranteed that the order will always be correct?

 

[kostefar]

Please note that JSON will be more robust of for some reason the sequence of the objects change within the JSON string. So instead of the string being in a neat order of abc, if you receive cba, your string parsing will collect the wrong data. Is the extra time saved worth accidentally saving the wrong data or are you 100% guaranteed that the order will always be correct?

I understand the risks indeed and have not found a situation so far, after several months, where things changed. But if they do, then yes: I´ll indeed need to have a fresh look at the delimiters used.

 

[teddybear]

Can I conclude, based on the above, that JSON is not very efficient on parsing (at least short) inputs in comparison to using the Substring function, or is there a more efficient way of doing it with JSON than this?
This matters to me when 100´s of strings need to be parsed every second.
My JSON library version is 1.21.

The conclusion is incorrect, you can parse the str2 using substring is based on the elements of the json string are restricted on in order and length. this is hard-coding for parsing the special json case.

 

[kostefar]

The conclusion is incorrect, you can parse the str2 using substring is based on the elements of the json string are restricted on in order and length. this is hard-coding for parsing the special json case.

In fact the actual json string that I´m parsing has two object levels and more things to parse. I also tested it on that and the conclusion was the same.
Anyway, I did not want to start a discussion about if there actually are cases where JSON is faster and easier - there´s a reason why there´s a JSON library, and often it will be the better choice.
I only wanted to know if I could optimize my JSON code to beat the manual parsing, but that seems not to be the case.