Android Question Accessibility clic problem

[Alain75]

Hi,

I developed a package manager which lists all the packages installed on the smartphone. I integrated the B4A accessibility library v0.11 and I am able to perform click on the "Force stop" button shown on the application details settings with PerformNodeActionOnViewWithArgs.

To go further, I tried to display the Storage sub screen of the application details settings in order to empty its cache. As the PerformNodeActionOnViewWithArgs method doesn't perform the click, I wrote a sub which identifies the nodeinfo and performs a click action on it. Unfortunately, with no any more success... Below is the code :

Sub evt_OnAccessibilityEvent (event As Object, node As Object)
...
        ' todo is action to perform
        Select todo.ToLowerCase()
            Case "back":    Ok = svc.PerformGlobalAction(kst.GLOBAL_ACTION_BACK)
            Case "down":    Ok = svc.PerformNodeActionOnViewWithArgs(True,"android:id/title",kst.ACTION_SCROLL_FORWARD,Null)
            Case "clic":    Ok = ClicText(node,"Stockage")
            Case Else:      Ok = svc.PerformNodeActionOnViewWithArgs(todo.contains(":id/"),todo,kst.ACTION_CLICK,Null)
        End Select
...
end sub

Sub ClicText(node As Object,text As String) As Boolean
    Dim ln As List = node
    ln = ln.Get(0)
    For i=0 To ln.Size-1
        If GetString(ln.Get(i),"getText")=text Then
            Log("Clic on "&text)
            Try
                'ln.Get(i).As(JavaObject).RunMethod("performAction",Array(kst.ACTION_FOCUS)) 'unusefull
                'ln.Get(i).As(JavaObject).RunMethod("performAction",Array(kst.ACTION_SELECT)) 'unusefull
                Return ln.Get(i).As(JavaObject).RunMethod("performAction",Array(kst.ACTION_CLICK))
            Catch
                Log(LastException)
                Return False
            End Try
            Exit
        End If
    Next
    Return False
End Sub
Sub GetString (jo As JavaObject,mt As String) As String
    Try
        Dim txt As String = jo.RunMethod(mt,Null)
    Catch
        txt = "error"
    End Try
    Return txt
End Sub

Any Idea ?

 

[Alain75]

Hurray,

sorry for the spam but after hours of search and trials, I solved my problem I added a dispatchGesture method as mentionned in other posts and wrote a sub which identifies the rectangle in which to click.

In order to share, below is a partial extract of the accessibility service module : ClicText clicks on a view identified by text or ID using the DispatchGesture, GetString gets the text or ID of a node and GetRectangle gets the screen coordinates of the view to click

Sub ClicText(node As Object,text As String) As Boolean
    Dim lnode As List = node, method As String = IIf(text.contains(":id/"),"getViewIdResourceName","getText")
    lnode = lnode.Get(0)
    For i=0 To lnode.Size-1
        If GetString(lnode.Get(i),method)=text Then
            Try
                Dim r As Rect = GetRect(lnode.Get(i))
                Return DispatchGesture(r.CenterX,r.CenterY)
            Catch
                Log(LastException)
                Return False
            End Try
            Exit
        End If
    Next
    Return False
End Sub

Sub GetString (jo As JavaObject, method As String) As String
    Try
        Dim txt As String = jo.RunMethod(method,Null)
    Catch
        txt = "error"
    End Try
    Return txt
End Sub

Sub GetRect(jo As JavaObject) As Rect
    Dim res As Rect
    res.Initialize(0,0,0,0)
    Try
        jo.As(JavaObject).RunMethod("getBoundsInScreen",Array(res))
    Catch
        Log(LastException)
    End Try
    Return res
End Sub

public Sub DispatchGesture(x As Int,y As Int) As Boolean
    Dim jo As JavaObject
    jo.InitializeContext
    Return jo.RunMethod("DispatchGesture", Array(x,y))
End Sub

#if Java
import android.accessibilityservice.AccessibilityService;
import android.content.ComponentName;
import android.view.accessibility.AccessibilityEvent;
import android.graphics.Path;
import android.accessibilityservice.GestureDescription;
import anywheresoftware.b4a.BA;

public boolean DispatchGesture(int x,int y) {
    GestureDescription.Builder gestureBuilder = new GestureDescription.Builder();
    Path path= new Path();
    path.moveTo(x, y);
    gestureBuilder.addStroke(new GestureDescription.StrokeDescription(path, 0, 50));
    return super.dispatchGesture(gestureBuilder.build(), null, null) ;
}
#End If