B4J Question HTTP-requests batch

peacemaker · Nov 13, 2021

HI, All

A server has HTTP REST API.
We need to make 3 requests one by one with control of execution, for each request, say, batch:

1) HTTP GET (search db), if any fail during network operations - stop with False result.
2) If found - HTTP DELETE request, if any fail during network operations - stop with False result.
3) If successfully deleted - HTTP POST request, to insert data, if any fail during network operations - stop with False result.

How to implement such synchronous batch universally ? Maybe even cross-platform.

Andrew (Digitwell) · Nov 13, 2021

Why not have a sequence of calls

B4X:

wait for (DoSearchdb(params...)) complete (result as map)
if (result.getdefault("success",false)) then
  wait for (doDelete(params...)) complete (delresult as map)
  if (delresult.getdefault("success",false)) then
    wait for (doInsertdata(params...)) complete (insresult as map)
       if (insresult.getdefault("success",false)) then
          do success stuff....
          return true
       else
          dofailure or rollback stuff...
       end if
   else
       do failure or rollback  stuff...
   end if
else
   dofailure or rollback stuff...
end if

return false

private sub DoSearchDb(params...) as resumablesub
    dim result as map = createmap()

   dim job as httpjob
   job.initalize(me,"")
  job.download(downloadurl)
  if (job.success) then
    result.put("success",true)
    do other stuff....
  else
    result.put("success",false)
    do other stuff...
  end if

  job.release

 return result
end sub

peacemaker · Nov 13, 2021

Indeed:

Real B4J test code:

#Region Project Attributes
    #MainFormWidth: 600
    #MainFormHeight: 600
#End Region

Sub Process_Globals
    Private fx As JFX
    Private MainForm As Form
    Private xui As XUI
    Private Button1 As B4XView
End Sub

Sub AppStart (Form1 As Form, Args() As String)
    MainForm = Form1
    MainForm.RootPane.LoadLayout("Layout1")
    MainForm.Show
End Sub

Sub Button1_Click
    wait for (Batch) complete (done As Boolean)
    Log("Batch = " & done)
End Sub

Sub Batch As ResumableSub
    wait for (DoReq1) complete (result As Map)
    If (result.getdefault("success", False)) Then
        Log("DoReq1 finished")
        wait for (DoReq2) complete (delresult As Map)
          If (delresult.getdefault("success", False)) Then
            Log("DoReq2 finished")
            wait for (DoReq3) complete (insresult As Map)
               If (insresult.getdefault("success",False)) Then
                Log("DoReq3 finished")
                  Return True
            Else
                Log("DoReq3 failed")
               End If
        Else
            Log("DoReq2 failed")
        End If
    Else
        Log("DoReq1 failed")
    End If
    Return False
End Sub

private Sub DoReq1 As ResumableSub
    Dim result As Map = CreateMap()

    Dim j As HttpJob
    j.Initialize("1", Me)
    j.Download("https://www.google.com")
    Wait For (j) JobDone(j As HttpJob)
    Log("DoReq1 =" & j.Success)
    result.put("success", j.Success)
    j.Release
    Return result
End Sub

private Sub DoReq2 As ResumableSub
    Dim result As Map = CreateMap()

    Dim j As HttpJob
    j.Initialize("2", Me)
    j.Download("https://www.google.com")
    Wait For (j) JobDone(j As HttpJob)
    Log("DoReq2 =" & j.Success)
    result.put("success", j.Success)
    j.Release
    Return result
End Sub

private Sub DoReq3 As ResumableSub
    Dim result As Map = CreateMap()

    Dim j As HttpJob
    j.Initialize("3", Me)
    j.Download("https://www.google.com")
    Wait For (j) JobDone(j As HttpJob)
    Log("DoReq3 =" & j.Success)
    result.put("success", j.Success)
    j.Release
    Return result
End Sub

Waiting for debugger to connect...
Program started.
DoReq1 =true
DoReq1 finished
DoReq2 =true
DoReq2 finished
DoReq3 =true
DoReq3 finished
Batch = true

Waiting for debugger to connect...
Program started.
java.net.UnknownHostException: Unknown host (www.g3oogle.com)
at java.base/java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method)
.....
at java.base/java.lang.Thread.run(Thread.java:834)
ResponseError. Reason: java.net.UnknownHostException: unknown host (www.g3oogle.com), Response:
DoReq1 =false
DoReq1 failed
Batch = false

DoReq1 =true
DoReq1 finished
ResponseError. Reason: Forbidden, Response: <html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>openresty</center>
</body>
</html>
DoReq2 =false
DoReq2 failed
Batch = false

DoReq1 =true
DoReq1 finished
DoReq2 =true
DoReq2 finished
ResponseError. Reason: Forbidden, Response: <html>
<head><title>403 Forbidden</title></head>
<body>
<center><h1>403 Forbidden</h1></center>
<hr><center>openresty</center>
</body>
</html>
DoReq3 =false
DoReq3 failed
Batch = false

peacemaker · Nov 13, 2021

But it's not a universal way, not for any batch.

Andrew (Digitwell) · Nov 13, 2021

peacemaker said:
But it's not a universal way, not for any batch.

Not sure what you mean.

peacemaker · Nov 13, 2021

I mean that it would be good to "invent" a way to make any qty of synchronous sequence of HTTP-requests.

Andrew (Digitwell) · Nov 13, 2021

I thought this should work, but it doesn't.

B4X:

Private Sub Button1_Click
'    xui.MsgboxAsync("Hello world!", "B4X")
    
'    Private req1 As Object = DoReq1
'    Private req2 As Object = DoReq2
'    Private req3 As Object = DoReq3
    
'    Private reqs As List = Array As Object(DoReq1,DoReq2,DoReq3)
    
    callwithhalt(Array(DoReq1,DoReq2,DoReq3))
End Sub


private Sub callwithhalt(reqs As List) As ResumableSub
    
    Private i As Int
    For i = 0 To reqs.Size -1
        Log($"Calling ${(i+1)}"$)
        wait for (reqs.Get(i)) complete (result As Map)
        Log($"Done ${(i+1)}"$)
        If Not(result.getdefault("success",False)) Then
          Return False
        End If    
    Next
    Return True
End Sub

The resumable subs seem to get confused.

Logger connected to: Google Pixel 2 XL
--------- beginning of main
*** Service (starter) Create ***
** Service (starter) Start **
** Activity (main) Create, isFirst = true **
Call B4XPages.GetManager.LogEvents = True to enable logging B4XPages events.
** Activity (main) Resume **
Calling 1
*** Service (httputils2service) Create ***
** Service (httputils2service) Start **
** Service (httputils2service) Start **
DoReq3 =true
DoReq2 =true
DoReq1 =true
Done 1
Calling 2
Error occurred on line: 43 (B4XMainPage)
java.lang.RuntimeException: Resumable sub already completed
at anywheresoftware.b4a.keywords.Common.WaitFor(Common.java:1734)
at uk.co.digitwell.testhttpbatch.b4xmainpage$ResumableSub_callwithhalt.resume(b4xmainpage.java:143)
at anywheresoftware.b4a.shell.DebugResumableSub$DelegatableResumableSub.resumeAsUserSub(DebugResumableSub.java:48)
at java.lang.reflect.Method.invoke(Native Method)
at anywheresoftware.b4a.shell.Shell.runMethod(Shell.java:732)
at anywheresoftware.b4a.shell.Shell.raiseEventImpl(Shell.java:348)
at anywheresoftware.b4a.shell.Shell.raiseEvent(Shell.java:255)
at java.lang.reflect.Method.invoke(Native Method)
at anywheresoftware.b4a.ShellBA.raiseEvent2(ShellBA.java:144)
at anywheresoftware.b4a.BA.raiseEvent(BA.java:193)
at anywheresoftware.b4a.shell.DebugResumableSub$DelegatableResumableSub.resume(DebugResumableSub.java:43)
at anywheresoftware.b4a.BA.checkAndRunWaitForEvent(BA.java:267)
at anywheresoftware.b4a.ShellBA.raiseEvent2(ShellBA.java:137)
at anywheresoftware.b4a.BA.raiseEvent(BA.java:193)
at anywheresoftware.b4a.keywords.Common$14.run(Common.java:1770)
at android.os.Handler.handleCallback(Handler.java:938)
at android.os.Handler.dispatchMessage(Handler.java:99)
at android.os.Looper.loop(Looper.java:223)
at android.app.ActivityThread.main(ActivityThread.java:7656)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:592)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:947)

Andrew (Digitwell) · Nov 13, 2021

Ok, got it working. Needs to use CallSub in the same way Jobdone works.

B4X:

Private Sub Button1_Click
'    xui.MsgboxAsync("Hello world!", "B4X")
    
'    Private req1 As Object = DoReq1
'    Private req2 As Object = DoReq2
'    Private req3 As Object = DoReq3
    
    Private reqs As List = Array As Object(DoReq1,DoReq2,DoReq3)
    
    callwithhalt(reqs)
End Sub


private Sub callwithhalt(reqs As List) As ResumableSub
    
    Private i As Int
    For i = 0 To reqs.Size -1
        Log($"Calling ${(i+1)}"$)
        wait for (reqs.Get(i)) done(result As Map)
        Log($"Done ${(i+1)}"$)
        If Not(result.getdefault("success",False)) Then
          Return False
        End If   
    Next
    Return True
End Sub


private Sub DoReq1
    Dim result As Map = CreateMap()

    Dim j As HttpJob
    j.Initialize("1", Me)
    j.Download("https://www.google.com")
    Wait For (j) JobDone(j As HttpJob)
    Log("DoReq1 =" & j.Success)
    result.put("success", j.Success)
    j.Release
    CallSub2(Me,"done", result)
End Sub

private Sub DoReq2
    Dim result As Map = CreateMap()

    Dim j As HttpJob
    j.Initialize("2", Me)
    j.Download("https://www.google.com")
    Wait For (j) JobDone(j As HttpJob)
    Log("DoReq2 =" & j.Success)
    result.put("success", j.Success)
    j.Release
    CallSub2(Me,"done", result)
End Sub

private Sub DoReq3
    Dim result As Map = CreateMap()

    Dim j As HttpJob
    j.Initialize("3", Me)
    j.Download("https://www.google.com")
    Wait For (j) JobDone(j As HttpJob)
    Log("DoReq3 =" & j.Success)
    result.put("success", j.Success)
    j.Release
    CallSub2(Me,"done", result)
End Sub

*** Service (starter) Create ***
** Service (starter) Start **
** Activity (main) Create, isFirst = true **
Call B4XPages.GetManager.LogEvents = True to enable logging B4XPages events.
** Activity (main) Resume **
Calling 1
*** Service (httputils2service) Create ***
** Service (httputils2service) Start **
** Service (httputils2service) Start **
DoReq2 =true
Done 1
Calling 2
DoReq3 =true
Done 2
Calling 3
DoReq1 =true
Done 3

Andrew (Digitwell) · Nov 13, 2021

hmm, in my excitement. I didn't notice that the order is not right.

Calling 1
DoReq2 =true <--- This should be req1
Done 1
Calling 2
DoReq3 =true <--- This should be req2
Done 2
Calling 3
DoReq1 =true <--- This should be req3
Done 3

Andrew (Digitwell) · Nov 13, 2021

With the use of a lot of logging, I think I can see what is going on.

B4X:

Private Sub Button1_Click

    Private reqs As List
    reqs.Initialize
    Log("Creating object 1")
    Private mp1 As Map = CreateMap("item":1,"name":"req1","object":DoReq1)
    reqs.Add(mp1)
    
    Log("Creating object 2")
    Private mp2 As Map = CreateMap("item":2,"name":"req2","object":DoReq2)
    reqs.Add(mp2)
    
    Log("Creating object 3")
    Private mp3 As Map = CreateMap("item":3,"name":"req3","object":DoReq3)
    reqs.Add(mp3)
    
    Log("======Starting=====")   
    wait for (callwithhalt(reqs)) complete (retn As Boolean)
    Log($"=====Finishing=====  success is ${retn}"$)
End Sub


private Sub callwithhalt(reqs As List) As ResumableSub
    
    Private i As Int
    For i = 0 To reqs.Size -1
        Private mp As Map = reqs.Get(i)
        Log($"Calling ${mp.Get("name")}"$)
        
        wait for (mp.Get("object")) done(result As Map)
        Log($"Done ${mp.Get("name")}"$)
        If Not(result.getdefault("success",False)) Then
          Return False
        End If   
    Next
    Return True
End Sub


private Sub DoReq1
    Log("starting Req1")
    Dim result As Map = CreateMap()

    Dim j As HttpJob
    j.Initialize("1", Me)
    j.Download("https://www.google.com")
    Wait For (j) JobDone(j As HttpJob)
    Log("------DoReq1 =" & j.Success)
    result.put("success", j.Success)
    j.Release
    CallSub2(Me,"done", result)
End Sub

private Sub DoReq2
    Log("starting Req2")
    Dim result As Map = CreateMap()

    Dim j As HttpJob
    j.Initialize("2", Me)
    j.Download("https://www.google.com")
    Wait For (j) JobDone(j As HttpJob)
    Log("------DoReq2 =" & j.Success)
    result.put("success", j.Success)
    j.Release
    CallSub2(Me,"done", result)
End Sub

private Sub DoReq3
    Log("starting Req3")
    Dim result As Map = CreateMap()

    Dim j As HttpJob
    j.Initialize("3", Me)
    j.Download("https://www.google.com")
    Wait For (j) JobDone(j As HttpJob)
    Log("------DoReq3 =" & j.Success)
    result.put("success", j.Success)
    j.Release
    CallSub2(Me,"done", result)
End Sub

** Service (starter) Start **
** Activity (main) Create, isFirst = true **
Call B4XPages.GetManager.LogEvents = True to enable logging B4XPages events.
** Activity (main) Resume **
Creating object 1
starting Req1
Creating object 2
starting Req2
Creating object 3
starting Req3
======STarting=====
Calling req1
*** Service (httputils2service) Create ***
** Service (httputils2service) Start **
** Service (httputils2service) Start **
------DoReq1 =true
Done req1
Calling req2
------DoReq3 =true
Done req2
Calling req3
------DoReq2 =true
Done req3
=====Finishing===== success is true

I would have expected the resumable sub to be called when the wait for is executed not when it is assigned.

peacemaker · Nov 13, 2021

Andrew (Digitwell) said:
Private mp1 As Map = CreateMap("item":1,"name":"req1","object"oReq1)

B4J line: 30
Private mp1 As Map = CreateMap(\
src\b4j\example\b4xmainpage.java:102: error: incompatible types: void cannot be converted to Object
_mp1 = parent.__c.createMap(new Object[] {(Object)("item"),(Object)(1),(Object)("name"),(Object)("req1"),(Object)("object"),(Object)(__ref._doreq1 /*void*/ (null))});
^
1 error

I think, in the first variant it needs to call second only if the previous Success is = true

Andrew (Digitwell) · Nov 13, 2021

That's odd. I get that error now, if I do a clean compile.

But in principle it seems that the function gets called as soon as it is assigned to the object not the wait for.

tchart · Nov 14, 2021

@peacemaker I have a library that does real synchronous HTTP calls.

I use it similar to what you are doing - I have a back end application that does a number of HTTP calls that are required to be done in a certain order.

If you'd like it let me know.

tchart · Nov 14, 2021

@peacemaker Ive uploaded the library

Synchronous HTTP Request Library

IMPORTANT NOTE There are many reasons you shouldnt use this library e.g. it will hold up your main thread. This is especially important for UI applications. *** Please dont turn this thread into a sermon regarding synchronous vs asynchronous calls. It is for specific use cases *** This is a...

www.b4x.com

B4J Question HTTP-requests batch

peacemaker

Expert

Andrew (Digitwell)

Well-Known Member

peacemaker

Expert

peacemaker

Expert

Andrew (Digitwell)

Well-Known Member

peacemaker

Expert

Andrew (Digitwell)

Well-Known Member

Attachments

Andrew (Digitwell)

Well-Known Member

Attachments

Andrew (Digitwell)

Well-Known Member

Andrew (Digitwell)

Well-Known Member

Attachments

peacemaker

Expert

Andrew (Digitwell)

Well-Known Member

tchart

Well-Known Member

tchart

Well-Known Member

Synchronous HTTP Request Library

Similar Threads