B4A Library Accessibility Services (assisting users with disabilities, automation etc)

Discussion in 'Additional libraries, classes and official updates' started by moster67, Jul 17, 2018.

  1. moster67

    moster67 Expert Licensed User

    This is a partial wrap/library for Android's Accessibility Services which you can read about here:
    https://developer.android.com/reference/android/accessibilityservice/AccessibilityService

    Android's Accessibility Services is meant to assisting users with disabilities in using Android devices and apps but can also be used for automation.
    Some automation tools, such as Tasker, are using Accessibility Services. In this post, I am attaching a sample app which permits you to automate WhatsApp so messages can be sent to contacts and groups without any user-interaction.

    It should be noted that an app such as the attached WhatsApp-sample might not be permitted on Google Play since it could be used as a bot unless you are able to motivate that it is purely used for users with disabilities.

    I started to write this wrapper for a personal app I am using when riding my motorbike. I have seen many requests for a wrap of the Accessibility Services and initially, I meant to write a complete and generic wrapper but I have realized that it is a huge task and I simply don't have enough time to do so. It is much easier to write a customized wrapper which covers your requirements. I am, therefore, also attaching the java source-code for the wrapper which you can extend and/or customize according to your needs. If you would like a customized library and you want me to help you, you can contact me by PM.

    That said, the attached library contains the most important functionality and should be usable for many projects you might have in mind. Open the sample-app to better understand how it works. You also need to add some things in the Manifest and there are also some configuration-files which need to be set up properly. You can find these files in the Resource-library which is attached.

    Read also the docs. Unfortunately, there isn't much documentation to be found on the internet except for the official documentation provided by Google (see the link earlier in this post) although you can find some information/examples on StackOverflow.

    Here is a YouTube video to see the WhatsApp sample app in action:



    Please remember that creating libraries and maintaining them takes time and so does supporting them. Please consider a donation if you use my free libraries as this will surely help keeping me motivated. Thank you!

    Good luck!
     

    Attached Files:

    Last edited: Jul 17, 2018
  2. Star-Dust

    Star-Dust Expert Licensed User

    I do not know if I understand correctly, with this app can I make sure that the messages that you receive on WhatsApp and that I read with ListenNotifier, Can send to on a another number :p ?

    Or anyway can I send messages from my app through whatsapp?


    Congratulations.
     
    Last edited: Jul 18, 2018
  3. JordiCP

    JordiCP Well-Known Member Licensed User

    Not an easy one, congratulations! :):)

    I started with it and took another approach since I thought it was going to be a big monster (not moster, sorry for the bad joke ;))
    Will dive into the source when I can, to learn from it
     
  4. scsjc

    scsjc Active Member Licensed User

    Awesome ;)
     
  5. Star-Dust

    Star-Dust Expert Licensed User

    On the Samsung S6 and Honor 9 they work properly.
    On Huawei mediapd M3 does not send the message but simply open whatsApp and write the message in the contact but does not send. What the Web API already does
     
  6. moster67

    moster67 Expert Licensed User

    You can probably approach it in different ways. My intention was to make a generic and complete wrapper but I simply don't have the time.
     
    Peter Simpson and JordiCP like this.
  7. moster67

    moster67 Expert Licensed User

    The purpose of the library is not exclusively to WhatsApp messages - I just thought it was a good sample app to include.
    Anyway, it could be that the mediapad is slower (or WhatsApp was executing slower) and when the send-button was going to be "clicked", the screen content had not yet been loaded and the button was not available. In the sample app, there are some delay methods - try to increase the one set at 100 ms to 500ms and see if it works.
    I tried the sample app on numerous devices, also slow ones, and it worked just fine.
     
  8. Star-Dust

    Star-Dust Expert Licensed User

    I know that the library has a wider use. ;)
    Reading the difficulties of DonManfred I wanted to try the example. you attached WhatsApp example and I try it.
    I have not thought of creating another example myself, also because I do not know the library well enough to be able to develop my example. :)

    Huawey MediaPad M3 is not a slow device, it is a fairly powerful tablet. I suppose the problem is something else, but it was just a curiosity.
     
  9. Lello1964

    Lello1964 Active Member Licensed User

    :):) wOOOOOOwwwwww
     
  10. MarcoRome

    MarcoRome Expert Licensed User

    Well done Mike
     
  11. Inman

    Inman Well-Known Member Licensed User

    Thank you for the library. This was one of my requests in the Wish section.

    I was trying your example and once I enable the accessibility permission, I get the following error. I did put the additionalres folder correctly.
    Code:
    *** Service (accservice) Create ***

    service was created
    ** 
    Service (accservice) Start **
    service was started
    *** 
    Service (starter) Create ***
    ** 
    Service (starter) Start **
    ** 
    Activity (main) Create, isFirst = true **
    ** 
    Activity (main) Resume **
    ** 
    Service (accservice) Start **
    service was started
    ** 
    Activity (main) Resume **
    java.lang.NullPointerException: Attempt 
    to invoke interface method 'java.lang.String java.lang.CharSequence.toString()' on a null object reference
        at com.tillekesoft.accessibilityservices.AccessibilityEventsListenerWrapper.onAccessibilityEvent(AccessibilityEventsListenerWrapper.java:161)
        at android.accessibilityservice.AccessibilityService$
    2.onAccessibilityEvent(AccessibilityService.java:1524)
        at android.accessibilityservice.AccessibilityService$IAccessibilityServiceClientWrapper.executeMessage(AccessibilityService.java:
    1710)
        at com.android.internal.os.HandlerCaller$MyHandler.handleMessage(HandlerCaller.java:
    37)
        at android.os.Handler.dispatchMessage(Handler.java:
    109)
        at android.os.Looper.loop(Looper.java:
    166)
        at android.app.ActivityThread.main(ActivityThread.java:
    7377)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:
    469)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:
    963)
    There is no B4A line number mentioned as it looks like the error is raised from library. When I checked the library source code, the line in bold is the one raising error (line 161).
    I could see 3-4 log entries for "my event" and "my node". I then commented out the entire code inside acs_OnAccessibilityEvent but is still getting the above error.

    I tried on Galaxy S8+ running Android 8.0 and Huawei P20 Pro running Android 8.1 and got the same error. Any idea?
     
  12. moster67

    moster67 Expert Licensed User

    It seems like it is the same error @DonManfred mentioned in this thread/post:
    https://www.b4x.com/android/forum/threads/accessibility-services-lib-running-the-sample.95221/

    As I replied in this thread, I am unsure why this is happening since I tested it on numerous devices and was not able to reproduce it.
    Try to uninstall the app and then run it again.

    I'll now be away from the forum for a while but when I'll be back maybe I will look into it (since there are some methods to modify a bit to make them more generic). Otherwise, the java sources are there if anyone else would like to have a look.
     
  13. Inman

    Inman Well-Known Member Licensed User

    Thank you replying. I somehow missed Don's thread. We will continue the discussion there.

    Uninstall and reinstall did not seem to work in my case. Please check Don's thread for more details.
     
  14. Scotter

    Scotter Active Member Licensed User

    This is sooo useful!
    I'm trying to incorporate it into my simple card game B4A app.
    Copied all the libraries, etc.
    Now I see the AccService.bas has many references to WhatsApp.
    Of course, I want to remove or change them.
    But I'm not sure which ones are safe to remove and what to change.

    Also, in that same AccService.bas module I see this:
    Code:
    Sub CheckIfServiceIsEnabled
        
    If acs.IsAccessibilityServiceEnabled("com.mytest/.accservice")  = False Then 'insert package-name and name of your service
            CallSub2(Main,"IsTheAccessibilityServiceEnabled"False)
            
    CallSub2(Main,"SetCheckbox"False)
        
    Else
            
    CallSub2(Main,"IsTheAccessibilityServiceEnabled"True)
            
    CallSub2(Main,"SetCheckbox"True)
        
    End If
    End Sub
    I'm not sure what to replace "com.mytest/.accservice" with since I don't see where "com.mytest" is in your sample project so I would figure out - based on the ?name? of my project, how to replace this string.

    Any tips?

    Thanks!
     
    Last edited: Dec 17, 2018
  15. moster67

    moster67 Expert Licensed User

    You can see it in the Build Configuration -> Package Name (Ctrl+B). Study the WhatsApp sample app, look at the Manifest-file and also in the Additional Resources and you should be able to see how it all fits together.
    After getting the user to activate the Service, most of your coding will take place in the OnAccessibilityEvent (see AccService module).
     
    Last edited: Dec 17, 2018
    Scotter likes this.
  16. Scotter

    Scotter Active Member Licensed User

    Thanks!
    And... just checking... I noticed there already is an accessibility library built into B4A.
    For a simple card game where my goal is to add the ability for screenreaders to have efficient input (like speaking "Shuffle" instead of "Button Shuffle", is your component overkill because I don't need any kind of automation?
     
  17. moster67

    moster67 Expert Licensed User

    That is news for me. I must have missed that.....that is good to know.
    Hard to tell. Automation is not the main-reason for writing this lib - it is just some extra functionality you get for free.
    If you think ContentDescription is enough, then I guess you are fine. Check out the Screener app I mentioned in the other thread and see if you are fine. Check if TalkBack works fine and decide. Do you need to add gestures or other features to help out people with disabilities? Let a blind user try your app and let you know if you can make any improvements...it is really up to you.
     
    Scotter likes this.
  18. Scotter

    Scotter Active Member Licensed User

    I wonder what Erel's take on this is? How does one tag a person in this forum?
     
  19. DonManfred

    DonManfred Expert Licensed User

    Begin the name qith @ followed by the username
    @Scotter wrote:......
     
    Rockefeller Goldman and Scotter like this.
  20. Scotter

    Scotter Active Member Licensed User

    @Erel - May I hear your thoughts on the "accessibility" library built into B4A?
    Mainly I'm wondering if it includes enough functionality to do the basics I need for a simple card game.
    Like what I'm imagining is the blind person taps on an object and hears what I want them to hear for that object because I added a bit of code to every object that has some text that will get auto-converted for the user into speech.
    Does the default accessibility library enable that? Are there examples I can see specifically for B4A?
    Thanks! Sorry if I'm asking dumb questions!
     
    Rockefeller Goldman likes this.
Loading...
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice