B4J Question OpenAI file error for Speech to Text [SOLVED]

[Tim Chapman]

The OpenAI code below works in that it will respond with text, respond with a picture and it creates a voice file when requested.
The error occurs in playing the file and has to do with MediaPlayer and MediaURI.

The MediaURI seems to be unhappy with \ characters in the file path in line 313 of the code below.
This is the error I am getting:

java.lang.RuntimeException: java.lang.RuntimeException: java.lang.IllegalArgumentException: java.net.URISyntaxException: Illegal character in opaque part at index 2: C:\Users\Owner\AppData\Roaming

I would like to be able to specify the folder where the tts file from the speech response myself rather than use the C:\Users\Owner\AppData\Roaming folder which is the xui.DefaultFolder.

Also, the code won't run in the debug mode.

My code below comes from these two:

[B4X] OpenAI - A.I. Text & Image generation

Only tested in B4A so far This library is designed to facilitate communication with the multimodal OpenAI API, enabling your B4X applications to leverage the capabilities of OpenAI's models for text generation (GPT-3.5 Turbo and GPT-4), image generation (DALL-E 3), text-to-speech...

www.b4x.com

[B4X] OpenAI Chat Bot, Image Generator Using B4J

[B4X] OpenAI Chat Bot, Image Generator Using B4J This B4J OpenAI chat bot and image generator sample is based on Blueforce's [B4X] OpenAI - A.I. Text & Image generation library. This uses OpenAI and needs to be subscribed to their services to have the API key and paid account to generate images...

www.b4x.com

In the library code for OpenAI, I modified the original line to make the calling of the speech to text via the API to work:
request = $"{"model":"tts-1","input":"Sing to me.","voice":"alloy"}"$

This is the original line which does not work:
'request = $"{"model":"tts-1","input":"${Prompt}","voice":"${getVoice}"}"$
I don't have enough understanding to correct the problem in those two lines if someone could also help with that.

I would like to solve all four problems. The error, selecting the file location, being able to run it in debug mode and the change of the OpenAI library line above.

'Ctrl + click to export as zip: ide://run?File=%B4X%\Zipper.jar&Args=OpenAIChatImage.zip

Sub Class_Globals
    Private Root As B4XView 'ignore
    Private xui As XUI 'ignore
   
    Private oai As OpenAI
    Private chatMsg As messageText
    Private maximumSize As Int = 0
   
    'CHAT
    Private clvMessages As CustomListView
    #If B4J
    Private edEdit As TextArea
    Private pEdit As Pane
    #End if
    #If B4A
    Private edEdit As EditText
    Private pEdit As Panel
    #End if

    'CLV TEXT ASSISTANT
    Private lbTextAssistant As Label
    #If B4J
    Private pBackgroundSpeech As Pane
    #End if
    #If B4A
    Private pBackgroundSpeech As Panel
    #End if
   
    'CLV USER TEXT
    Private lbUserText As Label
    #If B4J
    Private pUserBackgroundSpeech As Pane
    #End if
    #If B4A
    Private pUserBackgroundSpeech As Panel
    #End if
   
    Private lblRunOpenAIChat As B4XView
    Private lblRunOpenAIImgGen As B4XView
   
End Sub

'You can add more parameters here.
Public Sub Initialize
 #If B4J
xui.setdatafolder("")
#End If
End Sub

'This event will be called once, before the page becomes visible.
Private Sub B4XPage_Created (Root1 As B4XView)
    Root = Root1
    B4XPages.SetTitle(Me, "OpenAI Chat and Image Generator")
    'load the layout to Root
    Root.LoadLayout("ChatPage")
    AddKeyPressedListener(B4XPages.GetNativeParent(Me))
    InitAI
End Sub

Private Sub B4XPage_Appear

End Sub

#If B4J
Private Sub AddKeyPressedListener(Form1 As Form)
#End If
#If B4A
Private Sub AddKeyPressedListener(Form1 As Activity)
#End If
    Dim r As Reflector
    #If B4J
    r.Target = Form1.RootPane
    r.AddEventHandler("Main_KeyPressed", "javafx.scene.input.KeyEvent.KEY_PRESSED")
    r.AddEventHandler("Main_KeyPressed", "javafx.scene.input.KeyEvent.KEY_RELEASED")
    #End If
    #If B4A
   
    #End If

End Sub

#If B4J
Private Sub Main_KeyPressed_Event(e As Event)
    Dim kp As Reflector
    kp.Target = e 'e is a KeyEvent instance
    Dim KeyCode As String = kp.RunMethod("getCode")
'    Log(KeyCode)
    If KeyCode = "ENTER" Then B4XPages.MainPage.RunOpenAIChat
    e.Consume
End Sub

Private Sub SelectedUIView_MouseClicked (EventData As MouseEvent)
    Dim selectedView As B4XView = Sender
    Try
        Select selectedView.Tag
            Case "lblRunOpenAIChat"
                RunOpenAIChat
            Case "lblRunOpenAIImgGen"
                RunOpenAIImgGen
        End Select
    Catch
        Log(LastException)
    End Try
End Sub
#End If

Private Sub InitAI
   
    lblRunOpenAIChat.Font = xui.CreateFontAwesome(25)
    lblRunOpenAIChat.Text = Chr(0xF27A)  'Chr(0xF1D8)
    lblRunOpenAIImgGen.Font = xui.CreateFontAwesome(25)
    lblRunOpenAIImgGen.Text = Chr(0xF1FC)
   
    oai.Initialize(Me, "OpenAI", "MyAPIKey") ' Provide your OpenAI secret key here
   
    'In conjunction with ChatGPT, the library will buffer all messages from GPT
    'and the user to maintain the context of the chat history. Use .ResetChat to start over.
    oai.ChatModel = oai.MODEL_GPT35_TURBO 'or MODEL_GPT4

    Edit_Bot("Hello, to facilitate our conversation please avoid short questions."&CRLF&"Tell me how can I help you today?")
   
End Sub

Private Sub edEdit_TextChanged (Old As String, New As String)
'    Dim i As Int =  su.MeasureMultilineTextHeight(edEdit, New)
    Dim i As Int =  edEdit.Height
    If i > maximumSize Then Return 'REACHED THE SIZE LIMIT
   
    If i > 0.07*clvMessages.AsView.Height Then 'SMALL, INCREASE IT TO THE LIMIT
        #If B4J
        pEdit.PrefHeight = i
        edEdit.PrefHeight = i
        #End If
        #If B4A
        pEdit.Height = i
        edEdit.Height = i
        #End If
        AdjustSize_Clv
    End If
End Sub

Private Sub AdjustSize_Clv
    clvMessages.Base_Resize(clvMessages.AsView.Width, clvMessages.AsView.Height)
    Sleep(0) 'TO MAKE SURE YOU ADJUSTED THE SIZE, BEFORE SCROLLING DOWN
    If clvMessages.Size > 0 Then clvMessages.JumpToItem(clvMessages.Size - 1)
End Sub

Private Sub Edit_Bot (messages As String) 'LEFT SIDE (BLUE)
    Dim msgBot As messageText
    msgBot.Initialize
    msgBot.messages = messages
    msgBot.assistant = True
    #If B4J
    Dim pBot As Pane
    #End If
    #If B4A
    Dim pBot As Panel
    #End If
    Try
        pBot.Initialize("pBot")
        pBot.Tag = messages
        pBot.SetLayoutAnimated(0, 0, 0, clvMessages.AsView.Width, 0.15*clvMessages.AsView.Height)
        clvMessages.Add(pBot, msgBot)
        AdjustSize_Clv
    Catch
        Log(LastException)
    End Try

'    If Not (edEdit.IsInitialized) Then edEdit.Initialize("")
'    edEdit.Text = pBot.Tag
'    Log(pBot.Tag)

End Sub

Private Sub clvMessages_VisibleRangeChanged (FirstIndex As Int, LastIndex As Int)
    Dim ExtraSize As Int = 2
    For i = 0 To clvMessages.Size - 1
    #If B4J
    Dim pn As Pane
    #End If
    #If B4A
    Dim pn As Panel
    #End If
        pn = clvMessages.GetPanel(i)
        If i > FirstIndex - ExtraSize And i < LastIndex + ExtraSize Then
            #If B4J
            If pn.NumberOfNodes = 0 Then
            #End If
            #If B4A
            If pn.NumberOfViews = 0 Then
            #End If
                Dim m As messageText = clvMessages.GetValue(i)
                If m.assistant Then
                    pn.LoadLayout("clvTextAssistant")
                    lbTextAssistant.Text = m.messages
                Else
                    pn.LoadLayout("clvUserText")
                    lbUserText.Text = m.messages
                End If
            End If
        Else
            #If B4J
            If pn.NumberOfNodes > 0 Then
            #End If
            #If B4A
            If pn.NumberOfViews > 0 Then
            #End If
                #If B4J
                pn.RemoveAllNodes
                #End If
                #If B4A
                pn.RemoveAllViews
                #End If
            End If
        End If
    Next
End Sub

Public Sub RunOpenAIChat
    Private sText As String = edEdit.Text.Trim
    User_Edit(sText)
    OpenAI_Chat_Bot
End Sub

Private Sub User_Edit (messages As String) 'RIGHT SIDE (GRAY)
    If Not(chatMsg.IsInitialized) Then chatMsg.Initialize
    chatMsg.messages = messages
    chatMsg.assistant = False
    #If B4J
    Dim pn As Pane
    #End If
    #If B4A
    Dim pn As Panel
    #End If
    Try
        pn.Initialize("pn")
        pn.Tag = messages
        pn.SetLayoutAnimated(0, 0, 0, clvMessages.AsView.Width, 0.15*clvMessages.AsView.Height)
        clvMessages.Add(pn, chatMsg)
        AdjustSize_Clv
    Catch
        Log(LastException)
    End Try
End Sub

Private Sub OpenAI_Chat_Bot
'    Dim systemMsg As String
'    systemMsg = "Act like a math teacher called Tom, write witty but informative."
    Dim chatText As String
    chatText = edEdit.Text
   
    edEdit.Enabled=False
    Sleep(10)
    edEdit.Enabled=True
    edEdit.Text =""

'    oai.SystemMessage(systemMsg) 'optional, tell the bot once how to interpret the chat /act
    oai.ChatMessage(chatText)
    OpenAI_TextToSpeech_Bot
End Sub

' Implement corresponding event handlers in your activity or class
Private Sub OpenAI_ChatResponse(response As String)
    Edit_Bot(response)
    Log("OpenAI_ChatResponse Log: " & response)
End Sub

Private Sub OpenAI_ImageResponse(image As B4XBitmap)
    Dim img As ImageView
    img.Initialize("img")
    #If B4J
    img.SetImage(image)
    #End If
    #If B4A
    img.SetBackgroundImage(image)
    #End If
    #If B4J
    Dim pn As Pane
    #End If
    #If B4A
    Dim pn As Panel
    #End If
    Try
        pn.Initialize("pn")
        #If B4J
        pn.AddNode(img, 0, 0, clvMessages.AsView.Width, clvMessages.AsView.Height)
        #End If
        #If B4A
        pn.AddView(img, 0, 0, clvMessages.AsView.Width, clvMessages.AsView.Height)
        #End If
        clvMessages.Add(pn, img)
        AdjustSize_Clv
    Catch
        Log(LastException)
    End Try
End Sub

Private Sub OpenAI_TextToSpeech_Bot
    Dim speechtext As String
    speechtext = edEdit.Text
   
    ' Interact withTTS
    oai.TTSVoice = oai.TTS_FABLE 'optional, TTS_ALLOY,TTS_ECHO,TTS_FABLE,TTS_ONYX,TTS_NOVA,TTS_SHIMMER
    oai.TextToSpeech(speechtext)
End Sub

Private Sub OpenAI_TTSResponse(folder As String, filename As String)
    Dim MediaPlayer As MediaPlayer
   
    #If B4J
    MediaPlayer.Initialize("mp", folder)
    'MediaPlayer.Initialize("mp", File.DirApp & "\tts.mp3")
    #End If
   
    #If B4A
    MediaPlayer.Initialize
    #End If
   
'    MediaPlayer.Load(folder, filename)
    MediaPlayer.Play
End Sub

Private Sub OpenAI_ImageGeneration_Bot
    Dim imagetext As String
'    imagetext = "A blue elephant in a Greenhouse"
    imagetext = edEdit.Text
    Try
        ' Interact with DALL-E 3
'            oai.ImageAspectRatio = oai.IMAGE_16_9
        oai.generateImage(imagetext)
    Catch
        Log(LastException)
    End Try
End Sub

Private Sub RunOpenAIImgGen
    OpenAI_ImageGeneration_Bot
End Sub

Sub OpenAI_Error(message As String)
      Edit_Bot("Error encountered: " & message)
  Log("Error: " & message)
End Sub

 

[Tim Chapman]

I used Perplexity AI to help me solve the problems with the code. Now debug mode works as well.
I had to modify the OpenAI library in order to be able to specify where the mp3 file from the text to speech is saved.
I am still missing something. The OpenAI_TTSResponse sub in my code causes an exception because the file is not present to be copied. Then it goes again and finds it. I don't understand this and don't know how to fix it. It seems that when the OpenAI text to speech returns, it is not waiting long enough before renaming the file path and trying to copy the file. Please help with this.
I have made the OpenAI library into a class in my code to make it easier to troubleshoot until this is all resolved.

 

[Tim Chapman]

If you want to see how I used Perplexity AI to fix problems with the code, see these:

The following is the OpenAI library for the B4J programming...

The OpenAI library for the B4J programming language you provided is designed to interact with OpenAI's API for generating chat responses, images, and...

www.perplexity.ai

This is the OpenAI library for the B4J programming...

The error you are encountering is due to an issue with the file path used in the JavaFX Media class for playing the text-to-speech (TTS) audio file. The error...

www.perplexity.ai

 

[Tim Chapman]

I have fought with this for hours with no success. I created a new post here because I was not getting any responses here with 73 views.

OpenAI Text To Speech Problems

I have converted the OpenAI library to a Class module for easier troubleshooting. I have modified the OpenAI Class to allow me to specify the location I want the mp3 file saved. I run the code and put in a text. I click the far right label which I have made to give a speech response. I get an...

www.b4x.com

 

[Tim Chapman]

See here for the solution.

OpenAI Text To Speech Problems [SOLVED]

I have supplied my code with my OpenAI API key so that it will be simple for anyone to run the code for troubleshooting purposes. I will be removing that key from my OpenAI profile very soon since it can't remain exposed to the public. Hopefully this will make it easier for anyone to assist me...

www.b4x.com