Spanish Problema con accesibilidad

Jannete

Member
Hola a todos. Estoy queriendo desarrollar una aplicacion para uso personal, la cual publique una foto de Instagram de forma automatica.

Me base en el ejemplo de whatsapp (el cual modifique para poder enviar imagenes), todo esta perfecto excepto que me atore en los ultimos eventos.

A dar click en un boton de la aplicacion, se inicia el intent para la carga de la imagen, abre la primer pantalla de instagram (recorte de fotografia), alli inserto esta cadena:

acs.PerformNodeActionOnViewWithArgs (True, "com.instagram.android:id/save",AccServConstants.ACTION_CLICK, Null)

Se abre la siguiente pantalla (filtros), alli es donde me atoro, ya que el citado "com.instagram.id:/save" no funciona (ni siquiera aparece en los logs esta vista de evento), he revisado las demas vistas, pero ninguna funciona, en el log solo aparecen estas vistas (o eventos):

com.instagram.android:id/divider
com.instagram.android:id/filter_view

También aparecen vistas null, in vestigando un poco, he visto que hay vistas hijas (child_views), las cuales no son accesibles directamente.

¿Alguien sabe que evento tendria que llamar para hacer click en la siguiente flecha?

Él probado con todo esto y nada:

'acs.PerformNodeActionOnViewWithArgs (True, "com.instagram.android:id/save",AccServConstants.ACTION_CLICK, Null)
'acs.PerformNodeActionOnViewWithArgs (True, "com.instagram.android:id/next",AccServConstants.ACTION_CLICK, Null)
'acs.PerformNodeActionOnViewWithArgs (True, "com.instagram.android:id/process",AccServConstants.ACTION_CLICK, Null)
'acs.PerformNodeActionOnViewWithArgs (True, "com.instagram.android:id/send",AccServConstants.ACTION_CLICK, Null)
'acs.PerformNodeActionOnViewWithArgs (True, "com.instagram.android:id/finish",AccServConstants.ACTION_CLICK, Null)
'acs.PerformNodeActionOnViewWithArgs (True, "com.instagram.android:id/upload",AccServConstants.ACTION_CLICK, Null)

'acs.PerformNodeActionOnViewWithArgs (True, "com.instagram.android:id/button_next",AccServConstants.ACTION_CLICK, Null)
'acs.PerformNodeActionOnViewWithArgs (True, "com.instagram.android:id/button_save",AccServConstants.ACTION_CLICK, Null)
'acs.PerformNodeActionOnViewWithArgs (True, "com.instagram.android:id/button_finish",AccServConstants.ACTION_CLICK, Null)
'acs.PerformNodeActionOnViewWithArgs (True, "com.instagram.android:id/button_send",AccServConstants.ACTION_CLICK, Null)
'acs.PerformNodeActionOnViewWithArgs (True, "com.instagram.android:id/button_process",AccServConstants.ACTION_CLICK, Null)
'acs.PerformNodeActionOnViewWithArgs (True, "com.instagram.android:id/button_upload",AccServConstants.ACTION_CLICK, Null)
'acs.PerformNodeActionOnViewWithArgs (True, "com.instagram.android:id/right_button",AccServConstants.ACTION_CLICK, Null)

De no ser posible encontrar el evento correspondiente, habra alguna forma de emular el click en el boton correspondiente? (algo asi como la biblioteca que ya existe -no recuerdo el nombre-. El problema es que esa biblioteca no sirve desde un servicio, con la accesibilidad me imagino que si es posible).

Muchas gracias como siempre!
 

José J. Aguilar

Expert
Licensed User
Hola Jannete:

Como siempre que te vemos por aquí, es para una pregunta bastante compleja, jeje.
No tengo ni idea sobre lo que comentas, pero, ¿has probado a basarte en este ejemplo que es específico para instagram en vez del de whatsapp?

Entiendo que probablemente funcione igual y tengas el problema un paso más adelante (al usar los filtros), pero no sé, por si acaso.

Y no sé si le has echado algún ojo a este hilo, a ver si pudiera estar relacionado...

Si no, podrías probar a hacer un pequeño proyecto en el que reproduzcas el problema y lo subas al foro. En caso de que en el de español no consigas ayuda, podemos echarte una mano a hacerlo en el de inglés.

saludos,
 

Jannete

Member
Efectivamente, la vista esta bajo un arbol (aparece en el log "ContentChangeTypes: [CONTENT_CHANGE_TYPE_SUBTREE]"), por eso no me aparecen descripciones ni otros eventos (sale null).

Me explico, en la primer pantalla (en la que puedo hacer click), me sale en una linea:

className: android.widget.ImageView; (efectivamente la flecha siguiente es un ImageView)
contentDescription: Siguiente; (clara la descripcion sobre para que sirve ese boton)
viewIdResName: com.instagram.android:id/save; (este es mi comando para poder realizar la accion)

En la siguiente pantalla no me sale nada de esa informacion, y que como la vista esta bajo un subtree, solo me muestra el arbol, pero no me da eventos para este. Alguien tendra alguna idea?, gracias.
 

Jannete

Member
Hola Jannete:

Como siempre que te vemos por aquí, es para una pregunta bastante compleja, jeje.
No tengo ni idea sobre lo que comentas, pero, ¿has probado a basarte en este ejemplo que es específico para instagram en vez del de whatsapp?

Entiendo que probablemente funcione igual y tengas el problema un paso más adelante (al usar los filtros), pero no sé, por si acaso.

Y no sé si le has echado algún ojo a este hilo, a ver si pudiera estar relacionado...

Si no, podrías probar a hacer un pequeño proyecto en el que reproduzcas el problema y lo subas al foro. En caso de que en el de español no consigas ayuda, podemos echarte una mano a hacerlo en el de inglés.

saludos,

Gracias como siempre, en si tengo todo resuelto (el intent, el servicio de accesibilidad, etc), solo me faltan los comandos para lograr hacer los clicks con el servicio de accesibilidad (en si tanto asi como resuelto no, ya que me base en gran parte del ejemplo de accesibilidad ;) )

Adjunto el proyecto con el que estoy trabajando.

Muchas gracias como siempre.
 

Attachments

  • Instagram Post.zip
    13.2 KB · Views: 22

Jannete

Member
Sigo intentando y una semana después nada.

@José J. Aguilar , ¿habrá alguna forma de generar un click en cierta área de la pantalla (como en la biblioteca SendInput), usando la biblioteca de accesibilidad?

Muchas gracias y saludos!
 

José J. Aguilar

Expert
Licensed User
Hola Jannete:

He ejecutado el ejemplo que subes, pero no tengo instagram, lo siento...

¿Has intentado con este ejemplo?


En cuanto a simular un tap, parece que es posible, pero complejo, según sfsameer dice en este hilo:

"2- Difficult to implement (It took us 2 weeks to implement it in our project Teamviewer Alternative) because you will need to exact X,Y of the screen to simulate the touches"

Puedes echar un ojo a su ejemplo a ver si te sirve de ayuda.

saludos,
 
Last edited:

Jannete

Member
Muchas gracias como siempre!!!. El intent ya está (no carga la imagen con texto, pero algo es algo), el servicio ya está a medias... En el segundo anlace que me dejaste se menciona que hay un evento click, creo es lo que busco (yo no tendría problemas en encontrar la coordenada que busco, ya que quiero la aplicación para uso personal y en un solo dispositivo, por lo que espero no batallar tanto).

Si consigo la línea para simular el click más lo que llevo armado, creo que tendré solucionado el asunto. Trabajaré en el ejemplo y te informo. Muchas gracias nuevamente!!!!!
 

moster67

Expert
Licensed User
This answer is based upon my AccessibiltyServices library.

Never tried Instagram but if you don't have an ID / name of the view, you should be able to use its coordinates. It should show up if you log the contents in the event
OnAccessibilityEvent (Event As Object, node As Object)
Check Event and Node for information.
If you don't see the full logs, then you can use my tool B4XIdeViewer

Then, you need to implement the DispatchGesture method of AccessibilityServices which you can use to simulate a tap on a view.
It is not exposed in my library but only in the paid version. But it seems someone else might have done it (never tried their example).
See here:

There is also some automation example here on the forum by @sfsameer but I have never tried it.
 

Jannete

Member
This answer is based upon my AccessibiltyServices library.

Never tried Instagram but if you don't have an ID / name of the view, you should be able to use its coordinates. It should show up if you log the contents in the event

Check Event and Node for information.
If you don't see the full logs, then you can use my tool B4XIdeViewer

Then, you need to implement the DispatchGesture method of AccessibilityServices which you can use to simulate a tap on a view.
It is not exposed in my library but only in the paid version. But it seems someone else might have done it (never tried their example).
See here:

There is also some automation example here on the forum by @sfsameer but I have never tried it.
First of all, thank you very much.

I have filtered the events, and there are certain actions or views that appear in the log (like the one I use to give my first click on what I have advanced), in your whatsapp example the following button is pressed with the event "com .whatsapp : id / send ", on instagram to go from the snip screen use" com.instagram.android:id/save ".

The problem is in the second screen (filters), there is no event for a button, since apparently everything is nested under a tree:

B4X:
my event: EventType: TYPE_WINDOW_STATE_CHANGED; EventTime: 96026695; PackageName: com.android.systemui; MovementGranularity: 0; Action: 0; ContentChangeTypes: []; WindowChangeTypes: [] [ ClassName: android.view.View; Text: []; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; Enabled: true; Password: false; Checked: false; FullScreen: false; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0
my node: [[android.view.accessibility.AccessibilityNodeInfo@4a4f3; boundsInParent: Rect(0, 0 - 720, 720); boundsInScreen: Rect(0, 174 - 720, 894); packageName: com.instagram.android; className: android.view.View; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: com.instagram.android:id/filter_view; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: false; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: false; visible: true; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@4a8b4; boundsInParent: Rect(0, 0 - 720, 720); boundsInScreen: Rect(0, 174 - 720, 894); packageName: com.instagram.android; className: android.widget.FrameLayout; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: com.instagram.android:id/grids_container; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: false; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: false; visible: true; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@54a1f; boundsInParent: Rect(0, 0 - 204, 490); boundsInScreen: Rect(26, 894 - 230, 1384); packageName: com.instagram.android; className: android.widget.RadioButton; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: null; checkable: true; checked: true; focusable: true; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: true; visible: true; actions: [AccessibilityAction: ACTION_FOCUS - null, AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@551a1; boundsInParent: Rect(0, 0 - 204, 490); boundsInScreen: Rect(230, 894 - 434, 1384); packageName: com.instagram.android; className: android.widget.RadioButton; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: null; checkable: true; checked: false; focusable: true; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: true; visible: true; actions: [AccessibilityAction: ACTION_FOCUS - null, AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@55923; boundsInParent: Rect(0, 0 - 204, 490); boundsInScreen: Rect(434, 894 - 638, 1384); packageName: com.instagram.android; className: android.widget.RadioButton; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: null; checkable: true; checked: false; focusable: true; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: true; visible: true; actions: [AccessibilityAction: ACTION_FOCUS - null, AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION
Message longer than Log limit (4000). Message was truncated.
1
my event: EventType: TYPE_WINDOW_CONTENT_CHANGED; EventTime: 96026822; PackageName: com.instagram.android; MovementGranularity: 0; Action: 0; ContentChangeTypes: [CONTENT_CHANGE_TYPE_SUBTREE]; WindowChangeTypes: [] [ ClassName: android.widget.FrameLayout; Text: []; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; Enabled: true; Password: false; Checked: false; FullScreen: false; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0
my node: [[android.view.accessibility.AccessibilityNodeInfo@4a4f3; boundsInParent: Rect(0, 0 - 720, 720); boundsInScreen: Rect(0, 174 - 720, 894); packageName: com.instagram.android; className: android.view.View; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: com.instagram.android:id/filter_view; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: false; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: false; visible: true; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@4a8b4; boundsInParent: Rect(0, 0 - 720, 720); boundsInScreen: Rect(0, 174 - 720, 894); packageName: com.instagram.android; className: android.widget.FrameLayout; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: com.instagram.android:id/grids_container; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: false; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: false; visible: true; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@54a1f; boundsInParent: Rect(0, 0 - 204, 490); boundsInScreen: Rect(26, 894 - 230, 1384); packageName: com.instagram.android; className: android.widget.RadioButton; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: null; checkable: true; checked: true; focusable: true; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: true; visible: true; actions: [AccessibilityAction: ACTION_FOCUS - null, AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@551a1; boundsInParent: Rect(0, 0 - 204, 490); boundsInScreen: Rect(230, 894 - 434, 1384); packageName: com.instagram.android; className: android.widget.RadioButton; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: null; checkable: true; checked: false; focusable: true; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: true; visible: true; actions: [AccessibilityAction: ACTION_FOCUS - null, AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@55923; boundsInParent: Rect(0, 0 - 204, 490); boundsInScreen: Rect(434, 894 - 638, 1384); packageName: com.instagram.android; className: android.widget.RadioButton; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: null; checkable: true; checked: false; focusable: true; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: true; visible: true; actions: [AccessibilityAction: ACTION_FOCUS - null, AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION
Message longer than Log limit (4000). Message was truncated.
1
my event: EventType: TYPE_WINDOW_CONTENT_CHANGED; EventTime: 96027063; PackageName: com.instagram.android; MovementGranularity: 0; Action: 0; ContentChangeTypes: [CONTENT_CHANGE_TYPE_SUBTREE]; WindowChangeTypes: [] [ ClassName: android.widget.FrameLayout; Text: []; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; Enabled: true; Password: false; Checked: false; FullScreen: false; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0
my node: [[android.view.accessibility.AccessibilityNodeInfo@4a4f3; boundsInParent: Rect(0, 0 - 720, 720); boundsInScreen: Rect(0, 174 - 720, 894); packageName: com.instagram.android; className: android.view.View; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: com.instagram.android:id/filter_view; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: false; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: false; visible: true; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@4a8b4; boundsInParent: Rect(0, 0 - 720, 720); boundsInScreen: Rect(0, 174 - 720, 894); packageName: com.instagram.android; className: android.widget.FrameLayout; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: com.instagram.android:id/grids_container; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: false; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: false; visible: true; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@54a1f; boundsInParent: Rect(0, 0 - 204, 490); boundsInScreen: Rect(26, 894 - 230, 1384); packageName: com.instagram.android; className: android.widget.RadioButton; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: null; checkable: true; checked: true; focusable: true; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: true; visible: true; actions: [AccessibilityAction: ACTION_FOCUS - null, AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@551a1; boundsInParent: Rect(0, 0 - 204, 490); boundsInScreen: Rect(230, 894 - 434, 1384); packageName: com.instagram.android; className: android.widget.RadioButton; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: null; checkable: true; checked: false; focusable: true; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: true; visible: true; actions: [AccessibilityAction: ACTION_FOCUS - null, AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@55923; boundsInParent: Rect(0, 0 - 204, 490); boundsInScreen: Rect(434, 894 - 638, 1384); packageName: com.instagram.android; className: android.widget.RadioButton; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: null; checkable: true; checked: false; focusable: true; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: true; visible: true; actions: [AccessibilityAction: ACTION_FOCUS - null, AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION
Message longer than Log limit (4000). Message was truncated.
1
my event: EventType: TYPE_WINDOW_CONTENT_CHANGED; EventTime: 96027164; PackageName: com.instagram.android; MovementGranularity: 0; Action: 0; ContentChangeTypes: [CONTENT_CHANGE_TYPE_SUBTREE]; WindowChangeTypes: [] [ ClassName: android.widget.FrameLayout; Text: []; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; Enabled: true; Password: false; Checked: false; FullScreen: false; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0
my node: [[android.view.accessibility.AccessibilityNodeInfo@4a4f3; boundsInParent: Rect(0, 0 - 720, 720); boundsInScreen: Rect(0, 174 - 720, 894); packageName: com.instagram.android; className: android.view.View; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: com.instagram.android:id/filter_view; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: false; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: false; visible: true; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@4a8b4; boundsInParent: Rect(0, 0 - 720, 720); boundsInScreen: Rect(0, 174 - 720, 894); packageName: com.instagram.android; className: android.widget.FrameLayout; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: com.instagram.android:id/grids_container; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: false; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: false; visible: true; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@54a1f; boundsInParent: Rect(0, 0 - 204, 490); boundsInScreen: Rect(26, 894 - 230, 1384); packageName: com.instagram.android; className: android.widget.RadioButton; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: null; checkable: true; checked: true; focusable: true; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: true; visible: true; actions: [AccessibilityAction: ACTION_FOCUS - null, AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@551a1; boundsInParent: Rect(0, 0 - 204, 490); boundsInScreen: Rect(230, 894 - 434, 1384); packageName: com.instagram.android; className: android.widget.RadioButton; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: null; checkable: true; checked: false; focusable: true; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: true; visible: true; actions: [AccessibilityAction: ACTION_FOCUS - null, AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@55923; boundsInParent: Rect(0, 0 - 204, 490); boundsInScreen: Rect(434, 894 - 638, 1384); packageName: com.instagram.android; className: android.widget.RadioButton; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: null; checkable: true; checked: false; focusable: true; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: true; visible: true; actions: [AccessibilityAction: ACTION_FOCUS - null, AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION
Message longer than Log limit (4000). Message was truncated.
1
my event: EventType: TYPE_WINDOW_CONTENT_CHANGED; EventTime: 96027372; PackageName: com.instagram.android; MovementGranularity: 0; Action: 0; ContentChangeTypes: [CONTENT_CHANGE_TYPE_SUBTREE]; WindowChangeTypes: [] [ ClassName: android.widget.FrameLayout; Text: []; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; Enabled: true; Password: false; Checked: false; FullScreen: false; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0
my node: [[android.view.accessibility.AccessibilityNodeInfo@4a4f3; boundsInParent: Rect(0, 0 - 720, 720); boundsInScreen: Rect(0, 174 - 720, 894); packageName: com.instagram.android; className: android.view.View; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: com.instagram.android:id/filter_view; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: false; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: false; visible: true; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@4a8b4; boundsInParent: Rect(0, 0 - 720, 720); boundsInScreen: Rect(0, 174 - 720, 894); packageName: com.instagram.android; className: android.widget.FrameLayout; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: com.instagram.android:id/grids_container; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: false; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: false; visible: true; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@54a1f; boundsInParent: Rect(0, 0 - 204, 490); boundsInScreen: Rect(26, 894 - 230, 1384); packageName: com.instagram.android; className: android.widget.RadioButton; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: null; checkable: true; checked: true; focusable: true; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: true; visible: true; actions: [AccessibilityAction: ACTION_FOCUS - null, AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@551a1; boundsInParent: Rect(0, 0 - 204, 490); boundsInScreen: Rect(230, 894 - 434, 1384); packageName: com.instagram.android; className: android.widget.RadioButton; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: null; checkable: true; checked: false; focusable: true; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: true; visible: true; actions: [AccessibilityAction: ACTION_FOCUS - null, AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@55923; boundsInParent: Rect(0, 0 - 204, 490); boundsInScreen: Rect(434, 894 - 638, 1384); packageName: com.instagram.android; className: android.widget.RadioButton; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: null; checkable: true; checked: false; focusable: true; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: true; visible: true; actions: [AccessibilityAction: ACTION_FOCUS - null, AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION
Message longer than Log limit (4000). Message was truncated.
1
my event: EventType: TYPE_WINDOW_CONTENT_CHANGED; EventTime: 96027473; PackageName: com.instagram.android; MovementGranularity: 0; Action: 0; ContentChangeTypes: [CONTENT_CHANGE_TYPE_SUBTREE]; WindowChangeTypes: [] [ ClassName: android.widget.FrameLayout; Text: []; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; Enabled: true; Password: false; Checked: false; FullScreen: false; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0
my node: [[android.view.accessibility.AccessibilityNodeInfo@4a4f3; boundsInParent: Rect(0, 0 - 720, 720); boundsInScreen: Rect(0, 174 - 720, 894); packageName: com.instagram.android; className: android.view.View; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: com.instagram.android:id/filter_view; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: false; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: false; visible: true; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@4a8b4; boundsInParent: Rect(0, 0 - 720, 720); boundsInScreen: Rect(0, 174 - 720, 894); packageName: com.instagram.android; className: android.widget.FrameLayout; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: com.instagram.android:id/grids_container; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: false; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: false; visible: true; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@54a1f; boundsInParent: Rect(0, 0 - 204, 490); boundsInScreen: Rect(26, 894 - 230, 1384); packageName: com.instagram.android; className: android.widget.RadioButton; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: null; checkable: true; checked: true; focusable: true; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: true; visible: true; actions: [AccessibilityAction: ACTION_FOCUS - null, AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@551a1; boundsInParent: Rect(0, 0 - 204, 490); boundsInScreen: Rect(230, 894 - 434, 1384); packageName: com.instagram.android; className: android.widget.RadioButton; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: null; checkable: true; checked: false; focusable: true; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: true; visible: true; actions: [AccessibilityAction: ACTION_FOCUS - null, AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@55923; boundsInParent: Rect(0, 0 - 204, 490); boundsInScreen: Rect(434, 894 - 638, 1384); packageName: com.instagram.android; className: android.widget.RadioButton; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: null; checkable: true; checked: false; focusable: true; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: true; visible: true; actions: [AccessibilityAction: ACTION_FOCUS - null, AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION
Message longer than Log limit (4000). Message was truncated.
1
my event: EventType: TYPE_WINDOW_CONTENT_CHANGED; EventTime: 96027574; PackageName: com.instagram.android; MovementGranularity: 0; Action: 0; ContentChangeTypes: [CONTENT_CHANGE_TYPE_SUBTREE]; WindowChangeTypes: [] [ ClassName: android.widget.FrameLayout; Text: []; ContentDescription: null; ItemCount: -1; CurrentItemIndex: -1; Enabled: true; Password: false; Checked: false; FullScreen: false; Scrollable: false; BeforeText: null; FromIndex: -1; ToIndex: -1; ScrollX: -1; ScrollY: -1; MaxScrollX: -1; MaxScrollY: -1; AddedCount: -1; RemovedCount: -1; ParcelableData: null ]; recordCount: 0
my node: [[android.view.accessibility.AccessibilityNodeInfo@4a4f3; boundsInParent: Rect(0, 0 - 720, 720); boundsInScreen: Rect(0, 174 - 720, 894); packageName: com.instagram.android; className: android.view.View; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: com.instagram.android:id/filter_view; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: false; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: false; visible: true; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@4a8b4; boundsInParent: Rect(0, 0 - 720, 720); boundsInScreen: Rect(0, 174 - 720, 894); packageName: com.instagram.android; className: android.widget.FrameLayout; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: com.instagram.android:id/grids_container; checkable: false; checked: false; focusable: false; focused: false; selected: false; clickable: false; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: false; visible: true; actions: [AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@54a1f; boundsInParent: Rect(0, 0 - 204, 490); boundsInScreen: Rect(26, 894 - 230, 1384); packageName: com.instagram.android; className: android.widget.RadioButton; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: null; checkable: true; checked: true; focusable: true; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: true; visible: true; actions: [AccessibilityAction: ACTION_FOCUS - null, AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@551a1; boundsInParent: Rect(0, 0 - 204, 490); boundsInScreen: Rect(230, 894 - 434, 1384); packageName: com.instagram.android; className: android.widget.RadioButton; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: null; checkable: true; checked: false; focusable: true; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: true; visible: true; actions: [AccessibilityAction: ACTION_FOCUS - null, AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION_CLEAR_SELECTION - null, AccessibilityAction: ACTION_CLICK - null, AccessibilityAction: ACTION_ACCESSIBILITY_FOCUS - null, AccessibilityAction: ACTION_SHOW_ON_SCREEN - null], android.view.accessibility.AccessibilityNodeInfo@55923; boundsInParent: Rect(0, 0 - 204, 490); boundsInScreen: Rect(434, 894 - 638, 1384); packageName: com.instagram.android; className: android.widget.RadioButton; text: null; error: null; maxTextLength: -1; contentDescription: null; tooltipText: null; viewIdResName: null; checkable: true; checked: false; focusable: true; focused: false; selected: false; clickable: true; longClickable: false; contextClickable: false; enabled: true; password: false; scrollable: false; importantForAccessibility: true; visible: true; actions: [AccessibilityAction: ACTION_FOCUS - null, AccessibilityAction: ACTION_SELECT - null, AccessibilityAction: ACTION
Message longer than Log limit (4000). Message was truncated.
1

Certainly the only thing I have to do is to be able to generate a Click event at certain coordinates, I have been seeing the DispatchGesture for a long time, but without success to implement it. I will review the code that they shared from the WhatsApp answering machine.

Just as a question, how much would the library cost me with the click action involved?
 

Jannete

Member
@José J. Aguilar , he visto los ejemplos (el del Whatsapp automatizado y el de accesibilidad en el #extend).

En la biblioteca con el #Extend añadido, aparece este sub:

B4X:
public Sub RunDispatchGesture
    Dim nativeMe As JavaObject
    nativeMe.InitializeContext
    return dispatchGesture(clickBuilder.build(), null, null);
End Sub

Aqui espera parametros, en lugar de null trate de darle coordenadas y no me funciono. (lei que habia que cambiar unas lineas en el xml para que pudiese activarse el gesture, y asi lo hice, mas no logro hacer el evento click).

En el ejemplo de Whatsapp, esta esta funcion:

B4X:
public Sub RunDispatchGesture(mp11 As Map)

    Dim nativeMe As JavaObject
    nativeMe.InitializeContext
    nativeMe.RunMethod("DispatchGesture", Array(mp11.Get("GestX"),mp11.Get("GestY"),mp11.Get("EndPointGestX"),mp11.Get("EndPointGestY")))
    
'    ClickSendButton
    
End Sub

No entiendo algo, se espera una coordenada X y Y iniciales, y las mismas finales, si por ejemplo, pongo 300x, 300y, 300x, 300y; ¿no me genera un evento tipo scroll en lugar de click?

Apenas y medio la llevo con el b4a, ahora de Java no tengo nada de nociones, ¿alguna sugerencia?

Muchas gracias, y disculpa las molestias.

Saludos!
 

Jannete

Member
Ok, algo cansada de pelear con este tema, abri tambien un hilo en el foro de ingles:


Dejo esto aqui por si sale la respuesta en ese hilo y alguien a futuro tiene la misma duda, no pierda la continuidad o no este indagndo por todo el foro.
 

Jannete

Member
Al fin!!!!! Tuve que dejar de usar la biblioteca de Accesibilidad, y con ayuda del automatizador de Whatssapp al fin lo logre. Fue tan simple como:

B4X:
Sub Inicio
    Sleep(3000)
    '
    RunDispatchGesture(670, 116, 670, 116)
    
    Sleep(3000)
    '
    RunDispatchGesture(670, 116, 670, 116)
    
    Sleep(3000)
    '
    RunDispatchGesture(670, 116, 670, 116)
    
    Sleep(2000)
    RunPerformGlobalAction(1) 'Tecla Atras
    Sleep(2000)
    RunPerformGlobalAction(1) 'Tecla Atras   
    
    StopService(Me)
End Sub

Abro con el intent la imagen en Instagram, y con el codigo dado arriba, lanzo los clicks para oprimir las flechas siguiente (las coordenadas dadas por el log no ayudaron, por alguna razon estaban desviadas, pero en opciones de desarrollador, marque el mostrar el puntero y saque las coordenadas de alli).

Me hubiese gustado poder usar la libreria de accesibilidad para agregar texto (ya que con el intent no se carga), el detalle es que si bien puedo acceder a la vista (el edittext), me pide los parametros que le quiera agregar en forma de Mapa (por decir asi, puedo poner algo como "EditText, SET_TEXT, y aqui pide un mapa como parametro", no me deja agregar un string que seria lo que terminaria mi trabajo a la perfeccion. Se me ocurrio que puedo dar click en la caja de texto (como hice anteriormente), y escribir en ella, pero mi pregunta es, ¿mediante la accesibilidad es posible escribir en una vista?.

Una ultima duda, el evento RunPerformGlobalAction permite tomar una screenshoot, oprimir la tecla atras, ¿permitira oprimir la tecla power y la home?.

De nuevo mil gracias por todo, sobretodo a @José J. Aguilar que siempre me ayuda tan amablemente. Muchas gracias!!
 
Top