B4J Question jElasticSearch Authentication

cjpryor

Active Member
Licensed User
Forgive me if I missed this but I did try to find it in the forums. I would like to develop an application using jElasticSearch (https://www.b4x.com/android/forum/threads/server-jelasticsearch-search-and-text-analytics.73335/) but I cannot figure out how to provide login credentials to the server. For now I would like to start with basic auth. Looking at the documentation (https://www.b4x.com/b4j/help/jelasticsearch.html), I find the following which seems to imply that I need to add my credentials to the QueryParameters but I do not know how to do that.

​

Method_636.png
PerformRawRequest (Method As String, Endpoint As String, QueryParameters As Map, Payload As String) As ESResponse​

Performs a raw request.

Method - Request method (GET, POST, ...)
EndPoint - Request end point.
QueryParameters - Map of query parameters. Pass Null if not required.
Payload - Body payload. Pass an empty string if not required.

Or, I suppose that it is more likely that it would be in the initialization ...

Initialize (EventName As String, Hosts As List)​

Initializes the client and sets the list of hosts.

EventName - Currently there are no events.
Hosts - A list or array with one or more hosts.
Example:
esclient.Initialize("", Array("127.0.0.1:9200"))


Any suggestions would be greatly appreciated.

Thanks!
 
Is it possible to get the source code for jElasticsearch.jar? I'm running into issues with its use and I think it would be helpful to see the source code.

In particular, when I execute the following statement:

Code:
Log(esclient.PerformRawRequest("GET", "_cluster/health", Null, "").ResponseAsString)

I get the following stack trace:

Code:
OnFailure: http://127.0.0.1:9200
Error occurred on line: 36 (Main)
org.apache.http.ConnectionClosedException: Connection closed
    at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.endOfInput(HttpAsyncRequestExecutor.java:344)
    at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:261)
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
    at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
    at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
    at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
    at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588)
    at java.base/java.lang.Thread.run(Thread.java:832)

Thanks!

Clay
 
Upvote 0
Thanks Erel

Before trying to figure out what is happening with the jar, I decided to try running ElasticSearch as a service. The stack trace went away but I can't seem to get the BasicAuthenticationFilter to pass the credential into ElasticSearch.

For now I decided to find the latest version of ElasticSearch where the example works. As it turns out, it still works with 7.17.8 and stops working with 8.0.x. As it turns out, 8.0.x started automatically configuring security with the ElasticSearch install.

I will revisit that later. For now I am going to just run development with localhost and no security.

Thanks again,

Clay
 
Upvote 0
So I finally got back to this. I can't seem to get the BasicAuthenticationFilter to work with this wrapper. I think we may need to add the authentication to the wrapper itself - probably in the initialization. Perhaps something like this: https://www.elastic.co/guide/en/elasticsearch/client/java-api-client/8.5/_basic_authentication.html

Anyway, here are the results of my attempts.

In general, The whole BasicAuthenticationFilter is bypassed in the Indexer portion of the ElasticSearch example. The BasicAuthenticationFilter is triggered in the Search portion of the example. However, it still fails to authenticate to the ElasticSearch server.

Details:

1. when run from browser (http://localhost:51042/search?query=B4X) I get a login prompt and then results
2. when I Get and Post in postman with Basic auth results are returned with no login prompt
3. when I run the Elastic search example in B4J
a. Check of cluster health fails. See attached error_cluster_health.txt
b. Index does not go through authorization filter and fails authorization at check to see if document exists in meta collection (If Main.esclient.Exists("meta", "dates", directory) Then). See attached error_meta_exists.txt
b. Query successfully goes through filter where it verifies the credentials and returns true (at both the Filter and CheckCredentials) but fails when issuing the call to search. See attached error_search.txt

Code snippets (for reference):

From RestClientWrapper:

[CODE lang="java" title="Initialize"]/**
* Initializes the client and sets the list of hosts.
*EventName - Currently there are no events.
*Hosts - A list or array with one or more hosts.
*Example:<code>
*esclient.Initialize("", Array("127.0.0.1:9200"))</code>
*/
public void Initialize(BA ba, String EventName, List Hosts) {
HttpHost[] hh = new HttpHost[Hosts.getSize()];
for (int i = 0;i < Hosts.getSize();i++) {
String r = (String) Hosts.Get(i);
int c = r.indexOf(':');
hh = new HttpHost(r.substring(0, c), Integer.parseInt(r.substring(c + 1)));
}
client = RestClient.builder(hh).setFailureListener(new FailureListener() {
public void onFailure(HttpHost host) {
System.err.println("OnFailure: " + host);
}
}).build();
}[/CODE]

In BaxicAuthenticationFilter:

[CODE lang="b4x" title="BasicAuthenticationFilter"]'Return True to allow the request to proceed.
Public Sub Filter(req As ServletRequest, resp As ServletResponse) As Boolean
If req.GetSession.GetAttribute2("logged in", False) = True Then Return True
Dim auths As List = req.GetHeaders("Authorization")
If auths.Size = 0 Then
resp.SetHeader("WWW-Authenticate", $"Basic realm="Realm""$)
resp.SendError(401, "authentication required")
Return False
Else
If CheckCredentials(auths.Get(0)) Then
req.GetSession.SetAttribute("logged in", True)
Return True
Else
resp.SendError(401, "authentication required")
Return False
End If
End If
End Sub

Private Sub CheckCredentials (auth As String) As Boolean
Dim success As Boolean = False
If auth.StartsWith("Basic") = True Then
Dim b64 As String = auth.SubString("Basic ".Length)
Dim su As StringUtils
Dim b() As Byte = su.DecodeBase64(b64)
Dim raw As String = BytesToString(b, 0, b.Length, "utf8")
Dim UsernameAndPassword() As String = Regex.Split(":", raw)
If UsernameAndPassword.Length = 2 Then
'up to you to decide which credentials are allowed <---------------------------
If UsernameAndPassword(0) = userid And UsernameAndPassword(1) = password Then
success = True
End If
End If
End If
Return success
End Sub[/CODE]

In AppStart:

[CODE lang="b4x" title="AppStart"]srvr.Initialize("")
srvr.Port = 51042
srvr.AddFilter("/*", "BasicAuthenticationFilter", False)
srvr.AddBackgroundWorker("Indexer")
srvr.AddHandler("/search", "search", False)
srvr.Start

esclient.Initialize("", Array("127.0.0.1:9200"))

'test connection
Try
Dim respString As String = esclient.PerformRawRequest("GET", "_cluster/health", Null, "").ResponseAsString

Log("AppStart esclient.PerformRawResuest response = :" & respString)

Catch
Log("AppStart Error = :" & LastException)
End Try

StartMessageLoop[/CODE]

in Indexer

[CODE lang="b4x" title="IndexFolder"] Dim versions As Map

Try
If Main.esclient.Exists("meta", "dates", directory) Then
versions = Main.esclient.Get("meta", "dates", directory)
Log($"Current indexed Docs (${directory})"$)
For Each lib As String In versions.Keys
Log($"${lib}: $DateTime{versions.Get(lib)}"$)
Next
Else
versions.Initialize
End If

Catch
Log("IndexFolder Error = :" & LastException)
End Try[/CODE]

In Search

B4X:
Sub Handle(req As ServletRequest, resp As ServletResponse)
    Dim QueryString As String = req.GetParameter("query")
    If QueryString = "" Then
        resp.SendError(500, "Missing query.")
        Return
    End If
    resp.ContentType = "text/html"
    Dim Query As Map = CreateMap("simple_query_string": _
        CreateMap( _
            "fields": Array("text"), _
            "query": QueryString))
 
    Dim Highlighter As Map = CreateMap( _
        "pre_tags": Array("<b>"), "post_tags": Array("</b>"), _
        "fields": CreateMap("text": CreateMap()))
    'count the number of total results grouped by directory.
    'https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-bucket-terms-aggregation.html
    '(we need to use directory.keyword instead of directory because we are using the default mapping)
    Dim aggs As Map = CreateMap("directorys": CreateMap("terms": CreateMap("field": "directory.keyword")))
    Dim esres As ESResponse = Main.esclient.Search("main", "library", _
        CreateMap("query": Query, _
         "highlight": Highlighter, _
         "aggs": aggs, _
         "_source": Array("directory")))
 

Attachments

Last edited:
Upvote 0
Back
Top