B4J Question jElasticSearch Authentication

cjpryor

Active Member
Licensed User
Forgive me if I missed this but I did try to find it in the forums. I would like to develop an application using jElasticSearch (https://www.b4x.com/android/forum/threads/server-jelasticsearch-search-and-text-analytics.73335/) but I cannot figure out how to provide login credentials to the server. For now I would like to start with basic auth. Looking at the documentation (https://www.b4x.com/b4j/help/jelasticsearch.html), I find the following which seems to imply that I need to add my credentials to the QueryParameters but I do not know how to do that.

Method_636.png
PerformRawRequest (Method As String, Endpoint As String, QueryParameters As Map, Payload As String) As ESResponse​

Performs a raw request.

Method - Request method (GET, POST, ...)
EndPoint - Request end point.
QueryParameters - Map of query parameters. Pass Null if not required.
Payload - Body payload. Pass an empty string if not required.

Or, I suppose that it is more likely that it would be in the initialization ...

Initialize (EventName As String, Hosts As List)​

Initializes the client and sets the list of hosts.

EventName - Currently there are no events.
Hosts - A list or array with one or more hosts.
Example:
esclient.Initialize("", Array("127.0.0.1:9200"))


Any suggestions would be greatly appreciated.

Thanks!
 

cjpryor

Active Member
Licensed User
Is it possible to get the source code for jElasticsearch.jar? I'm running into issues with its use and I think it would be helpful to see the source code.

In particular, when I execute the following statement:

B4X:
Log(esclient.PerformRawRequest("GET", "_cluster/health", Null, "").ResponseAsString)

I get the following stack trace:

B4X:
OnFailure: http://127.0.0.1:9200
Error occurred on line: 36 (Main)
org.apache.http.ConnectionClosedException: Connection closed
    at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.endOfInput(HttpAsyncRequestExecutor.java:344)
    at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:261)
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
    at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
    at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
    at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
    at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588)
    at java.base/java.lang.Thread.run(Thread.java:832)

Thanks!

Clay
 
Upvote 0

cjpryor

Active Member
Licensed User
Thanks Erel

Before trying to figure out what is happening with the jar, I decided to try running ElasticSearch as a service. The stack trace went away but I can't seem to get the BasicAuthenticationFilter to pass the credential into ElasticSearch.

For now I decided to find the latest version of ElasticSearch where the example works. As it turns out, it still works with 7.17.8 and stops working with 8.0.x. As it turns out, 8.0.x started automatically configuring security with the ElasticSearch install.

I will revisit that later. For now I am going to just run development with localhost and no security.

Thanks again,

Clay
 
Upvote 0

cjpryor

Active Member
Licensed User
So I finally got back to this. I can't seem to get the BasicAuthenticationFilter to work with this wrapper. I think we may need to add the authentication to the wrapper itself - probably in the initialization. Perhaps something like this: https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/8.5/_basic_authentication.html

Anyway, here are the results of my attempts.

In general, The whole BasicAuthenticationFilter is bypassed in the Indexer portion of the ElasticSearch example. The BasicAuthenticationFilter is triggered in the Search portion of the example. However, it still fails to authenticate to the ElasticSearch server.

Details:

1. when run from browser (http://localhost:51042/search?query=B4X) I get a login prompt and then results
2. when I Get and Post in postman with Basic auth results are returned with no login prompt
3. when I run the Elastic search example in B4J
a. Check of cluster health fails. See attached error_cluster_health.txt
b. Index does not go through authorization filter and fails authorization at check to see if document exists in meta collection (If Main.esclient.Exists("meta", "dates", directory) Then). See attached error_meta_exists.txt
b. Query successfully goes through filter where it verifies the credentials and returns true (at both the Filter and CheckCredentials) but fails when issuing the call to search. See attached error_search.txt

Code snippets (for reference):

From RestClientWrapper:

Initialize:
/**
     * Initializes the client and sets the list of hosts.
     *EventName - Currently there are no events.
     *Hosts - A list or array with one or more hosts.
     *Example:<code>
     *esclient.Initialize("", Array("127.0.0.1:9200"))</code>
     */
    public void Initialize(BA ba, String EventName, List Hosts) {
        HttpHost[] hh = new HttpHost[Hosts.getSize()];
        for (int i = 0;i < Hosts.getSize();i++) {
            String r = (String) Hosts.Get(i);
            int c = r.indexOf(':');
            hh[i] = new HttpHost(r.substring(0, c), Integer.parseInt(r.substring(c + 1)));
        }
        client = RestClient.builder(hh).setFailureListener(new FailureListener() {
            public void onFailure(HttpHost host) {
                System.err.println("OnFailure: " + host);
            }
        }).build();
    }

In BaxicAuthenticationFilter:

BasicAuthenticationFilter:
'Return True to allow the request to proceed.
Public Sub Filter(req As ServletRequest, resp As ServletResponse) As Boolean
    If req.GetSession.GetAttribute2("logged in", False) = True Then Return True
    Dim auths As List = req.GetHeaders("Authorization")
    If auths.Size = 0 Then
        resp.SetHeader("WWW-Authenticate", $"Basic realm="Realm""$)
        resp.SendError(401, "authentication required")
        Return False
    Else
        If CheckCredentials(auths.Get(0)) Then
            req.GetSession.SetAttribute("logged in", True)
            Return True
        Else
            resp.SendError(401, "authentication required")
            Return False
        End If
    End If
End Sub

Private Sub CheckCredentials (auth As String) As Boolean
    Dim success As Boolean = False
    If auth.StartsWith("Basic") = True Then
        Dim b64 As String = auth.SubString("Basic ".Length)
        Dim su As StringUtils
        Dim b() As Byte = su.DecodeBase64(b64)
        Dim raw As String = BytesToString(b, 0, b.Length, "utf8")
        Dim UsernameAndPassword() As String = Regex.Split(":", raw)
        If UsernameAndPassword.Length = 2 Then
            'up to you to decide which credentials are allowed <---------------------------
            If UsernameAndPassword(0) = userid And UsernameAndPassword(1) = password Then
                success = True
            End If
        End If
    End If
    Return success
End Sub

In AppStart:

AppStart:
srvr.Initialize("")
    srvr.Port = 51042
    srvr.AddFilter("/*", "BasicAuthenticationFilter", False)
    srvr.AddBackgroundWorker("Indexer")
    srvr.AddHandler("/search", "search", False)
    srvr.Start
 
    esclient.Initialize("", Array("127.0.0.1:9200"))

    'test connection
    Try
        Dim respString As String = esclient.PerformRawRequest("GET", "_cluster/health", Null, "").ResponseAsString
   
        Log("AppStart esclient.PerformRawResuest response = :" & respString)

    Catch
        Log("AppStart Error = :" & LastException)
    End Try
 
    StartMessageLoop

in Indexer

IndexFolder:
    Dim versions As Map
 
    Try
        If Main.esclient.Exists("meta", "dates", directory) Then
            versions = Main.esclient.Get("meta", "dates", directory)
            Log($"Current indexed Docs (${directory})"$)
            For Each lib As String In versions.Keys
                Log($"${lib}: $DateTime{versions.Get(lib)}"$)
            Next
        Else
            versions.Initialize
        End If

    Catch
        Log("IndexFolder Error = :" & LastException)
    End Try

In Search

B4X:
Sub Handle(req As ServletRequest, resp As ServletResponse)
    Dim QueryString As String = req.GetParameter("query")
    If QueryString = "" Then
        resp.SendError(500, "Missing query.")
        Return
    End If
    resp.ContentType = "text/html"
    Dim Query As Map = CreateMap("simple_query_string": _
        CreateMap( _
            "fields": Array("text"), _
            "query": QueryString))
 
    Dim Highlighter As Map = CreateMap( _
        "pre_tags": Array("<b>"), "post_tags": Array("</b>"), _
        "fields": CreateMap("text": CreateMap()))
    'count the number of total results grouped by directory.
    'https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html
    '(we need to use directory.keyword instead of directory because we are using the default mapping)
    Dim aggs As Map = CreateMap("directorys": CreateMap("terms": CreateMap("field": "directory.keyword")))
    Dim esres As ESResponse = Main.esclient.Search("main", "library", _
        CreateMap("query": Query, _
         "highlight": Highlighter, _
         "aggs": aggs, _
         "_source": Array("directory")))
 

Attachments

  • error_cluster_health.txt
    2.2 KB · Views: 86
  • error_meta_exists.txt
    2.2 KB · Views: 90
  • error_search.txt
    1.8 KB · Views: 93
Last edited:
Upvote 0
Top