B4J Question Authentication and button click on website

[kostefar]

Dear All,

I have not used b4j for the last 3 years (nor any other programming language) so my apologies if my question seems a bit stupid. I did try to find answers on the forum, but did not succeed.
This is my situation: I want to login on a website with login and password fields + a "Sign in" button and do various stuff after that.
At first I thought it´d be a piece of cake to do it all with httpjob, so I loaded Fiddler to have a look at the packets going back and forth. Unfortunately there are some hash strings that need to be sent to the server, which I guess are created on the client side using code received from the server, so it´s a bit more complicated than I thought at first.
So what I´m thinking about is using a browser library where the authentication is done by automatically filling in the authentication fields and clicking on the Sign in button and then hopefully at that point I´ll have been able to grab all necessary hashes and tokens in an event that captures the responses, and can proceed with httpjob.

The parts of the website with the authentication fields and the button looks like this:

 <div class="form-group username">
            <label class="control-label" for="Email">Email</label>
            <input class="form-control" placeholder="Email Address" autocomplete="off" tabindex="1" spellcheck="false" type="text" data-val="true" data-val-required="The Email field is required." id="Email" name="Email" value="" />
            <span class="text-danger" id="unError" style="display:none;">Please enter a valid email address</span>
        </div>
            <div class="form-group password">
                <label class="control-label" for="Password">Password</label>
                <input type="password" class="form-control" placeholder="Password" autofocus="autofocus" autocomplete="off" tabindex="2" id="Password" name="Password" />
            </div>

<div class="form-group">
            <button type="submit" id="SubmitLogin" tabindex="3">
                Sign in
            </button>
        </div>

I remember having done something like this many years ago, but I think it was in vb6, where I believe that the key to being successful was using the tabindex values.

I hope somebody can help me here, thanks in advance!

 

[EnriqueGonzalez]

You need something like this:

Selenium Library - Pnd_Selenium

Selenium automates browsers. That's it! What you do with that power is entirely up to you. Primarily it is for automating web applications for testing purposes, but is certainly not limited to just that. Boring web-based administration tasks can (and should) also be automated as well. Library...

www.b4x.com

Doing so with httpjob will be way harder as you need to parse the html, if you want that route

jSoup HTML Parser

This is my first attempt at a wrapper so it's a work in progress. Consider it a Beta although it isn't feature complete, I'm adding features as I require them :) Not all functions and documentation implemented or tested fully yet. Library is compatible with B4A and B4J. 1. Download jsoup...

www.b4x.com

Or any other html parser on the forum

 

[kostefar]

You need something like this:

Selenium Library - Pnd_Selenium

Selenium automates browsers. That's it! What you do with that power is entirely up to you. Primarily it is for automating web applications for testing purposes, but is certainly not limited to just that. Boring web-based administration tasks can (and should) also be automated as well. Library...

www.b4x.com

Doing so with httpjob will be way harder as you need to parse the html, if you want that route

jSoup HTML Parser

This is my first attempt at a wrapper so it's a work in progress. Consider it a Beta although it isn't feature complete, I'm adding features as I require them :) Not all functions and documentation implemented or tested fully yet. Library is compatible with B4A and B4J. 1. Download jsoup...

www.b4x.com

Or any other html parser on the forum

Thanks a lot EnriqueGonzalez!

Parsing is something I can easily do by myself, but like I said, the problem is the hash (sort of token) strings that are being generated on client side, which I have no clue how are generated. So the first suggestion sounds indeed like an interesting solution although a bit overkill-ish for me, since I just wanted to handle the authentication fields and the button with a few lines of code, but if this can do the job - why not!!!

 

[kostefar]

So I´ve installed Selenium, but fail to get it working. I´m using the latest chromedriver, which is 111.0.5563.65, whereas the chromedriver is for 111.0.5563.64, so not sure if that´s what´s causing the below:

Waiting for debugger to connect...
Program started.
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Starting ChromeDriver 111.0.5563.64 (c710e93d5b63b7095afe8c2c17df34408078439d-refs/branch-heads/5563@{#995}) on port 29511
Only local connections are allowed.
Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe.
ChromeDriver was started successfully.
mar. 12, 2023 12:12:05 AM org.openqa.selenium.remote.http.WebSocket$Listener onError
WARNING: Invalid Status code=403 text=Forbidden
java.io.IOException: Invalid Status code=403 text=Forbidden
    at org.asynchttpclient.netty.handler.WebSocketHandler.abort(WebSocketHandler.java:92)
    at org.asynchttpclient.netty.handler.WebSocketHandler.handleRead(WebSocketHandler.java:118)
    at org.asynchttpclient.netty.handler.AsyncHttpClientHandler.channelRead(AsyncHttpClientHandler.java:78)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
    at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:336)
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323)
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:444)
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:280)
    at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
    at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
    at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
    at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
    at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.lang.Thread.run(Thread.java:748)
Error occurred on line: 24 (Main)
org.openqa.selenium.remote.http.ConnectionFailedException: Unable to establish websocket connection to http://localhost:58364/devtools/browser/d540e957-71e3-49f2-b29c-6ace1f6ea046
Build info: version: '4.7.2', revision: '4d4020c3b7'
System info: os.name: 'Windows 10', os.arch: 'amd64', os.version: '10.0', java.version: '1.8.0_162'
Driver info: driver.version: ChromeDriver
    at org.openqa.selenium.remote.http.netty.NettyWebSocket.<init>(NettyWebSocket.java:102)
    at org.openqa.selenium.remote.http.netty.NettyWebSocket.lambda$create$3(NettyWebSocket.java:128)
    at org.openqa.selenium.remote.http.netty.NettyClient.openSocket(NettyClient.java:107)
    at org.openqa.selenium.devtools.Connection.<init>(Connection.java:78)
    at org.openqa.selenium.chromium.ChromiumDriver.lambda$new$2(ChromiumDriver.java:116)
    at java.util.Optional.map(Optional.java:215)
    at org.openqa.selenium.chromium.ChromiumDriver.<init>(ChromiumDriver.java:114)
    at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:82)
    at org.openqa.selenium.chrome.ChromeDriver.<init>(ChromeDriver.java:71)
    at com.pendrush.selenium.kotlin.SeleniumLib.initialize(SeleniumLib.kt:32)
    at com.pendrush.selenium.Wrapper.initialize2(Wrapper.java:39)
    at com.pendrush.selenium.Wrapper.Initialize(Wrapper.java:35)
    at b4j.example.main._appstart(main.java:78)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at anywheresoftware.b4a.shell.Shell.runMethod(Shell.java:629)
    at anywheresoftware.b4a.shell.Shell.raiseEventImpl(Shell.java:237)
    at anywheresoftware.b4a.shell.Shell.raiseEvent(Shell.java:167)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at anywheresoftware.b4a.BA.raiseEvent2(BA.java:111)
    at anywheresoftware.b4a.shell.ShellBA.raiseEvent2(ShellBA.java:100)
    at anywheresoftware.b4a.BA.raiseEvent(BA.java:98)
    at b4j.example.main.start(main.java:38)
    at com.sun.javafx.application.LauncherImpl.lambda$launchApplication1$161(LauncherImpl.java:863)
    at com.sun.javafx.application.PlatformImpl.lambda$runAndWait$174(PlatformImpl.java:326)
    at com.sun.javafx.application.PlatformImpl.lambda$null$172(PlatformImpl.java:295)
    at java.security.AccessController.doPrivileged(Native Method)
    at com.sun.javafx.application.PlatformImpl.lambda$runLater$173(PlatformImpl.java:294)
    at com.sun.glass.ui.InvokeLaterDispatcher$Future.run(InvokeLaterDispatcher.java:95)
    at com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
    at com.sun.glass.ui.win.WinApplication.lambda$null$147(WinApplication.java:177)
    at java.lang.Thread.run(Thread.java:748)

Not sure if I should start a new thread about this?

 

[MicroDrie]

I remember having done something like this many years ago

I don't think that statement can better describe what the challenge is for you. Logging in today has become many times more complex because it has become a constant competition between the good and the bad. So it is no longer a simple postal assignment and a specific solution found with great difficulty is no longer long-lived for that found solution.

The observation that a solution finally found today may no longer work after tomorrow, limits the applicability of the solution found to a large extent. To find that solution, you also have a challenge in the fact that your test attempts can be noticed as a hacking attempt, which can block your user account and/or give rise to tightening the security measures when logging in.

Perhaps it is therefore more convenient and robust to let the user log in, save the received HTML page and extract the desired information with Jsoup, for example.

 

[kostefar]

I don't think that statement can better describe what the challenge is for you. Logging in today has become many times more complex because it has become a constant competition between the good and the bad. So it is no longer a simple postal assignment and a specific solution found with great difficulty is no longer long-lived for that found solution.

The observation that a solution finally found today may no longer work after tomorrow, limits the applicability of the solution found to a large extent. To find that solution, you also have a challenge in the fact that your test attempts can be noticed as a hacking attempt, which can block your user account and/or give rise to tightening the security measures when logging in.

Perhaps it is therefore more convenient and robust to let the user log in, save the received HTML page and extract the desired information with Jsoup, for example.

Thanks for your feedback MicroDrie! I actually think this could be achieved with the Selenium library, I just don´t have a clue on how I should do it.