Android Question Net library openSSL and parsing of headers

[William Hunter]

I recently acquired a Motorola Moto G phone running KitKat 4.4.4. The first problem I encountered was that connections to mail servers using openSSL were blocked. This prompted me to think of doing a rewrite of one of my apps. In looking at the Net library I saw that openSSL was still used for POP3 connections. I have no doubt that a new release of the Net library is in the works to update this mode.

Some time ago, prior to the Net library supporting STAT and TOP, member Wim Lambrecht developed his wPOP library, which not only included STAT and TOP, but also did some basic HEADER PARSING. There are times when I do not care to download the complete message content of a mail server, just to get at the headers. (Yes, I am well aware of the use of the mailparser module.)

My wish is that the header parsing elements of Wim’s excellent wPOP library be incorporated into the next version of the Net library. That would make my day.

Thank you, and best regards

 

[William Hunter]

As I understand it all Nexus branded devices running KitKat 4.4.4 now block connections using openSSL, as described in my post above. It would seem that Google intends this to be the norm for future releases of Andriod. In view of this I would appreciate some commentary from Erel, as this does have impact on B4A.

1. Is there an update to the Net library forthcoming, to deal with Google's response to the Heartbleed vulnerability?

2. Do you see merit in adding wPOP’s basic header parsing to the Net library, and would you consider doing this?

Best regards

 

[Erel]

How do you see that openSSL is used by the Net library?

 

[William Hunter]

How do you see that openSSL is used by the Net library?

Thank you for your reply Erel. I may have made an erroneous assumption here. I know the Net Library is based on Apache Commons Net, and I believe that Apache Commons Net has historically used openSSL for POP3 connections, as openSSL has been widely used as the defacto industry standard for a number of years.

In looking at the documentation for the POP3 section of the Net library, I was expecting to see some reference to the updating of SSL, and the support of TLS. (This is a subject I don’t fully understand.)

I take it from your comment that the Net library does not use openSSL. I, like many others, am not a professional developer and find it quite daunting at times to fully understand certain changes that affect our work. If the Net library does not use openSSL, for POP3 connections, what exactly does it use?

Also, while the Net library is a very useful tool, would you consider adding some basic header parsing in a future release? From my point of view this would be a very welcome addition.

Best regards

 

[Erel]

The Net library uses the OS native APIs to create a SSL socket.

You can use MailParser module to parse the headers.

 

[William Hunter]

The Net library uses the OS native APIs to create a SSL socket.

You can use MailParser module to parse the headers.

Thank you for your response Erel. I have tried working with the MailParser a number of times. Unfortunately I cannot seem to have things work according to my needs. I need to parse the Subject and From headers into two arrays, with the intention of populating a ScrollView from these two arrays.

I have included my test code below, and have highlighted the area where am having difficulty. Would you please take a look at this, and if there is a way to accomplish what I wish to do, show me how it is done.

Sub Process_Globals
      Dim pop As POP3
      Dim SetTextSize As Boolean
      Dim M As Message
      Dim aIndex As Int
      Dim messageCount As Int
      Dim arraySubject() As String
      Dim arrayFrom() As String
      Dim Subject As String
      Dim From As String
End Sub
Sub Globals
  
End Sub
Sub Activity_Create(FirstTime As Boolean)
   If FirstTime Then
      pop.Initialize("pop.mailserver.net", 110, "username", "password", "pop")
      'pop.UseSSL = True ' uncomment if SSL required
      pop.Status
   End If
   Activity.LoadLayout("Start")
   pop.ListMessages
End Sub

Sub POP_StatusCompleted (Success As Boolean, NumberOfMessages As Int, TotalSize As Int)
    messageCount =     NumberOfMessages
    Log("Success=" & Success & ", NumberOfMessages=" & NumberOfMessages & ", TotalSize=" & TotalSize)
End Sub

Sub POP_ListCompleted (Success As Boolean, Messages As Map)
    If Success = False Then
        Log(LastException.Message)
    Else
        'download all messages
        'change last parameter to True if you want to delete the messages from the server
        Log("Messages.Size = " & Messages.Size)
        For i = 0 To Messages.Size - 1
            pop.DownloadMessage(Messages.GetKeyAt(i), False)
        Next      
    End If
    pop.Close
End Sub

Sub POP_DownloadCompleted (Success As Boolean, MessageId As Int, MessageText As String)
   If Success = False Then
      Log("LastException = " & LastException.Message)
   Else
      Log("MessageID = " & MessageId)
      'Parse the mail
      M = MailParser.ParseMail(MessageText, File.DirRootExternal)
      Log("M = " & M)
        
     ' ############# put Subject and From headers into two arrays here #####################
     Dim arraySubject(messageCount) As String '( ReDim Arrays for messageCount here
     Dim arrayFrom(messageCount) As String    '(
     aIndex = 0
      For i = 0 To messageCount - 1
        arraySubject(aIndex) = M.Subject '( how do I get the headers into
        arrayFrom(aIndex) = M.FromField  '( a correctly indexed array???
        aIndex = aIndex + 1
     Next
     ' #####################################################################################
    
     CreateScrollView
   End If
End Sub

Sub Activity_Pause(UserClosed As Boolean)
  
End Sub

Sub Activity_Resume

End Sub

Sub CreateScrollView
    Dim ScrollView1 As ScrollView
    Dim lstChecks As List
    Dim Height As Int
    Dim FontSize As Int
    Dim FontColor1 As Int : FontColor1 = Colors.White
    Dim FontColor2 As Int : FontColor2 = Colors.Black
    Dim Panel1 As Panel
    Dim yAxis As Int
    SetTextSize = True ' this inserted just for testing - normally set in settings
    If SetTextSize = True Then
        Height = 60dip
        FontSize = 16
    Else
        Height = 45dip
        FontSize = 12
    End If
    ScrollView1.Initialize(500)
    Panel1 = ScrollView1.Panel
    'If Label5.Visible = False Then ' adjusts ScrollView height depending on number buttons
        yAxis = 100%y - 105dip
    'Else If Label5.Visible = True Then
        'yAxis = 100%y - 150dip
    'End If
    Activity.AddView(ScrollView1, 0, 50dip, 100%x, yAxis)
    lstChecks.Initialize
    aIndex = 0
    For i = 1  To messageCount
        Dim chk As CheckBox
        chk.Initialize("chkBoxSelection")
        Subject = arraySubject(aIndex) 'get the item from the array
        chk.Text = Subject
        chk.textSize = FontSize
        If i Mod 2 = 0 Then
            chk.TextColor = FontColor1
            chk.Color =  Colors.DarkGray
        Else
            chk.TextColor = FontColor2
            chk.Color =  Colors.White
        End If
        chk.tag = i
        lstChecks.Add(chk)
        Dim lbl1 As Label
        lbl1.Initialize("")
        From = arrayFrom(aIndex) 'get the item from the array
        lbl1.Text = From
        lbl1.textSize = FontSize
        lbl1.Gravity = Gravity.CENTER_VERTICAL
        If i Mod 2 = 0 Then
            lbl1.TextColor = FontColor1
            lbl1.Color = Colors.DarkGray
        Else
            lbl1.TextColor = FontColor2
            lbl1.Color = Colors.White
        End If
        Panel1.AddView(chk, 0, Height * (i - 1), 50%x, Height)
        Panel1.AddView(lbl1, 50%x, Height * (i - 1), 50%x, Height)
        aIndex = aIndex + 1
    Next
    Panel1.Height = lstChecks.Size * Height
End Sub

Thank you again, and best regards

 

[Erel]

Moved to the questions forum.

DownloadCompleted is called once for each downloaded message.

Use a List instead of an array and add the last message to the global list each time.

 

[William Hunter]

Moved to the questions forum.

DownloadCompleted is called once for each downloaded message.

Use a List instead of an array and add the last message to the global list each time.

Thank you for your response Erel. I have tried your suggestion without success. If, as you say, DownloadCompleted is called once for each downloaded message, then the code below should produce a correctly indexed List. Instead only the header for the first message is added to the list.

Sub POP_ListCompleted (Success As Boolean, Messages As Map)
    If Success = False Then
        Log(LastException.Message)
    Else
        'download all messages
        'change last parameter to True if you want to delete the messages from the server
        Log("Messages.Size = " & Messages.Size)
        For i = 0 To Messages.Size - 1
            pop.DownloadMessage(Messages.GetKeyAt(i), False)
        Next      
    End If
    pop.Close
End Sub

Sub POP_DownloadCompleted (Success As Boolean, MessageId As Int, MessageText As String)
   If Success = False Then
      Log("LastException = " & LastException.Message)
   Else
       Log("MessageID = " & MessageId)
      'Parse the mail
      M = MailParser.ParseMail(MessageText, File.DirRootExternal)
      Log("M = " & M)
     ' ############# put Subject and From headers into two lists here ######################
     'For i = 0  To messageCount - 1 ' using this loop will put the first message header into i numder of indexes
        listSubject.Add(M.Subject) '( how do I get the headers into
        listFrom.Add(M.FromField)  '( a correctly indexed list???
     'Next
     ' #####################################################################################
     Log("listFrom.Size = " & listFrom.Size)
     Log("MessageCount = " & messageCount)
     If listFrom.Size = messageCount Then
        CreateScrollView
     End If
   End If
End Sub

I may be in error with the way I have set things up. Would you please give me further guidance, so that I can get this working?

Best regards

 

[DonManfred]

Instead only the header for the first message is added to the list.

Are you sure it is the FIRST message? If i should give a tip on this i would say it´s the LAST message, not the first.

Where did you DIM M which you use in

M =

?

Try to put the dim into the sub

'Parse the mail
Dim M As Message
M = MailParser.ParseMail(MessageText,File.DirRootExternal)
[..]

You are reusing the same object. Putting M (or some content of it) into a list will only put a REFERENCE to the ONE-TIME-DIMmed Variable. Not it´s content. So i guess the list holds 100 items; but it´s 100times the same subject...

You need to DIM a new message each time.

 

[William Hunter]

You need to DIM a new message each time.

Thank you DonManfred - mystery is solved. You were correct, I had declared M as a Global - foolish me

Best regards

 

[DonManfred]

You can honor my answer with a click on the like-button