B4J Question Problem with Resumable Sub that recursively call itself.

[max123]

Hi all,

I've this code in my library (at bottom of post), the entry point here is the MinifyAllFilesOnFolderAsync sub,
the sub should recursively minimize all JS and CSS files in the specified folder SourceDir and save them in DestDir
mantaing the folder structure.

I call it from Main this way:

Dim Compressor As YUICompressor
Compressor.Initialize(Me, "Compressor")
Compressor.MinifyAllFilesOnFolderAsync (File.DirTemp, File.DirApp, True)
Wait For Compressor_FolderComplete (Success As Boolean, Item As MinifiedFolderItem)

The MinifyAllFilesOnFolderAsync sub is called, then inside it, the sub MinifyAllFilesOnFolderInternalAsync is called,
a Wait For wait for all files are processed recursively and then the result return should return back to the calling sub (a Wait For in the Main)

It is called with:

    MinifyAllFilesOnFolderInternalAsync (SourceDir, DestDir, UseMinPrefix)
    Wait For Compressor_FolderComplete (Success As Boolean, FolderItem As MinifiedFolderItem)

At this point all files are listed in the SourceDir folder, it process some files and recursively search files inside subfolfers and process them if they are JS or CSS, but then at some time the execution flow is not what I expect.

The sub MinifyAllFilesOnFolderInternalAsync call itself recursively if folder is found, but it should only terminate one time and call back the MinifyAllFilesOnFolderAsync to process it's return and then return it back to the Main with CallSubDelayed3 .

Some files are processed, but I never see the COMPLETED log, the sub never return back to calling sub.

Note that MinifyFileToFileAsync called to process a file when found, is also asyncronous sub, and it works well if called from the main.

In the MinifyAllFilesOnFolderInternalAsync I even use File.ListFilesAsync instead of File.ListFiles.

I followed it for 4 hours line by line in debug mode, but no luck.
The For Each f As String In Files should only terminate one time and here the sub should return back the non-Null value, that is a MinifiedFolderItem Type.
Instead it is called more times, even if the sub itself do not return back but continue it's execution, but at the end do not fires the event on the calling sub.

Please someone know what is wrong here ?
Many thanks.

#Event: FolderComplete (Success As Boolean, Item As MinifiedFolderItem)

Type MinifiedFolderItem (SourceDir As String, DestDir As String, OriginalSize As Int, MinifiedSize As Int, Compression As Float)
 
Public Sub MinifyAllFilesOnFolderAsync (SourceDir As String, DestDir As String, UseMinPrefix As Boolean) As ResumableSub
    mFolderFilesOriginalSize = 0
    mFolderFilesMinifiedSize = 0
   
'''    Wait For (MinifyAllFilesOnFolderInternalAsync (SourceDir, DestDir, UseMinPrefix)) Complete (FolderItem As MinifiedFolderItem)
    MinifyAllFilesOnFolderInternalAsync (SourceDir, DestDir, UseMinPrefix)
    Wait For Compressor_FolderComplete (Success As Boolean, FolderItem As MinifiedFolderItem)
   
    Log("======================== COMPLETED ========================")

    If Success Then
        FolderItem.SourceDir = SourceDir
        FolderItem.DestDir = DestDir
        FolderItem.OriginalSize = mFolderFilesOriginalSize
        FolderItem.MinifiedSize = mFolderFilesMinifiedSize
        FolderItem.Compression = MapFloat(mFolderFilesMinifiedSize, 0, mFolderFilesOriginalSize, 100, 0)
        If SubExists(mModule, mEventName & "_FolderComplete") Then
            CallSubDelayed3(mModule, mEventName & "_FolderComplete", True, FolderItem)
        End If
    Else
        If SubExists(mModule, mEventName & "_FolderComplete") Then
            CallSubDelayed3(mModule, mEventName & "_FolderComplete", False, Null)
        End If
    End If
   
    Return True
End Sub

Private Sub MinifyAllFilesOnFolderInternalAsync (SourceDir As String, DestDir As String, UseMinPrefix As Boolean) As ResumableSub
   
    ' Add error handling for file existence check
    If File.Exists(SourceDir, "") = False Or File.IsDirectory(SourceDir, "") = False Then
        Log("The SourceDir folder does not exist: " & SourceDir)
        CallSubDelayed3(Me, mEventName & "_FolderComplete", False, Null)
        Return True ' Ensure proper termination
    End If
   
    If DestDir = "" Then DestDir = SourceDir
   
    Log(" ")
    Log("CURRENT SOURCE FOLDER: " & SourceDir)
    Log("  CURRENT DEST FOLDER: " & DestDir)
    Log(" ")
   
'    Dim Files As List = File.ListFiles(SourceDir)

    ' Handle asynchronous file listing
    Wait For (File.ListFilesAsync(SourceDir)) Complete (Success As Boolean, Files As List)
   
    If Success Then
        For Each f As String In Files
       
            If File.IsDirectory(SourceDir, f) = False Then
                Log("FOUND FILE: " & File.Combine(SourceDir, f))
           
                Dim fName As String
           
                ' Touch .js and .css files only, ignore all other extentions
           
                If f.EndsWith(".js") Then
                    If UseMinPrefix Then
                        If f.ToLowerCase.Contains(".min") = False Then
                            fName = f.SubString2(0, f.IndexOf(".js")) & ".min.js"
                        Else
                            Log("JS File already minified. Skip it: " & File.Combine(SourceDir, f))
                            Continue
                        End If
                    Else
                        fName = f
                    End If
                    Log(" ") : Log(">>>>>>>>> PROCESS JS FILE: " & File.Combine(SourceDir, f))
                    Log("Minified JS file name: " & fName)
                    mInternalOperation = True
                    '''                    Wait For (MinifyFileToFileAsync (SourceDir, f, DestDir, fName)) Complete (Success As Boolean)
                    MinifyFileToFileAsync (SourceDir, f, DestDir, fName)
                    Wait For Compressor_FileComplete (Success As Boolean, jsItem As MinifiedFileItem)
                    Log("JS Returned from MinifyFileToFileAsync")
                    If Success Then
                        mFolderFilesOriginalSize = mFolderFilesOriginalSize + jsItem.OriginalSize
                        mFolderFilesMinifiedSize = mFolderFilesMinifiedSize + jsItem.MinifiedSize
                    Else
                        CallSubDelayed3(Me, mEventName & "_FolderComplete", False, Null)
                    End If
                Else If f.EndsWith(".css") Then
                    If UseMinPrefix Then
                        If f.ToLowerCase.Contains(".min") = False Then
                            fName = f.SubString2(0, f.IndexOf(".css")) & ".min.css"
                        Else
                            Log("CSS File already minified. Skip it: " & File.Combine(SourceDir, f))
                            Continue
                        End If
                    Else
                        fName = f
                    End If
                    Log(" ") : Log(">>>>>>>>> PROCESS CSS FILE: " & File.Combine(SourceDir, f))
                    Log("Minified CSS file name: " & fName)
                    mInternalOperation = True
                    '''                    Wait For (MinifyFileToFileAsync (SourceDir, f, DestDir, fName)) Complete (Success As Boolean)
                    MinifyFileToFileAsync (SourceDir, f, DestDir, fName)
                    Wait For Compressor_FileComplete (Success As Boolean, cssItem As MinifiedFileItem)
                    Log("CSS Returned from MinifyFileToFileAsync")
                    If Success Then
                        mFolderFilesOriginalSize = mFolderFilesOriginalSize + cssItem.OriginalSize
                        mFolderFilesMinifiedSize = mFolderFilesMinifiedSize + cssItem.MinifiedSize
                    Else
'                        If SubExists(Me, mEventName & "_FolderComplete") Then
                        CallSubDelayed3(Me, mEventName & "_FolderComplete", False, Null)
                        Return True
'                        End If                       
                    End If
                End If
            Else
                Log("FOUND FOLDER: " & File.Combine(SourceDir,f))
               
                ' Recursive call to process subfolders
                ''' Wait For (MinifyAllFilesOnFolderInternalAsync (File.Combine(SourceDir, f), File.Combine(DestDir, f), UseMinPrefix)) Complete (FolderItem As MinifiedFolderItem) ' << RECURSIVELY CALL ITSELF
                MinifyAllFilesOnFolderInternalAsync (File.Combine(SourceDir, f), File.Combine(DestDir, f), UseMinPrefix)
                Wait For Compressor_FolderComplete (Success As Boolean, FolderItem As MinifiedFolderItem) ' << RECURSIVELY CALL ITSELF
                Log("Returned from MinifyAllFilesOnFolderInternalAsync")
                                   
                If Success = False Then
                    CallSubDelayed3(Me, mEventName & "_FolderComplete", False, Null)
                    Return True ' Ensure proper termination
                End If
                                           
'                Continue  ' Should this be used to skip next iteration ???
            End If
        Next
    Else
        Log("ERROR. CANNOT LIST FILES")
        '        Return Null
        CallSubDelayed3(Me, mEventName & "_FolderComplete", False, Null)
        Return True ' Ensure proper termination
    End If
   
    ' Call FolderComplete event only once after processing all files and subfolders
    Dim FolderItem As MinifiedFolderItem
    FolderItem.Initialize
   
    '    Sleep(1)
    Log(" ") : Log ("FOLDER MINIFICATION END. SHOULD ONLY CALLED AT END WHEN PROCESSED RECURSIVELY FOLDERS, SUBFOLDERS AND FILES ONE BY ONE WITH For Each")  ' <<<<< THIS IS CALLED MORE TIMES BUT SHOULD ONLY CALLED ONE TIME
    CallSubDelayed3(Me, mEventName & "_FolderComplete", True, FolderItem)
   
    Return True ' Ensure proper termination
End Sub

 

[max123]

Please, any help is appreciated.....

Is something that have to do with this part of code that call the sub itself recursively, because if I comment it, all the files are successfully processed in the main folder root and here the COMPLETE log magically appears in the log. So how to manage it ?

            Else
                Log("FOUND FOLDER: " & f)
         
                ''' Wait For (MinifyAllFilesOnFolderInternalAsync (File.Combine(SourceDir, f), File.Combine(DestDir, f), UseMinPrefix)) Complete (FolderItem As MinifiedFolderItem) ' << RECURSIVELY CALL ITSELF
                MinifyAllFilesOnFolderInternalAsync (File.Combine(SourceDir, f), File.Combine(DestDir, f), UseMinPrefix) ' << RECURSIVELY CALL ITSELF
                Wait For Compressor_FolderComplete (Success As Boolean, FolderItem As MinifiedFolderItem)
         
'                If Success = False Then
                ''    If SubExists(Me, mEventName & "_FolderComplete") Then
'                       CallSubDelayed3(Me, mEventName & "_FolderComplete", False, FolderItem)
                ''    End If
'                End If
                         
                Log("Returned from MinifyAllFilesOnFolderInternalAsync")
'                Continue  ' Should this be used to skip next iteration ???
            End If

And this is the log:

Call B4XPages.GetManager.LogEvents = True to enable logging B4XPages events.

CURRENT SOURCE FOLDER: C:\Users\Massimo\AppData\Local\Temp\
CURRENT DEST FOLDER: C:\Users\Massimo\DOCUME~1\PROGET~2\ES23DE~1\YUICOM~1\B4J\Objects

FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\.ses
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\0845f16a-f66c-413a-b422-f1745c232c01.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\0d0a3963-a7e1-4991-874f-7333d957b681.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\1585c8b7-2fe4-4b50-8f11-53798c28b798.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\19b4b1ef-b389-47d6-84ee-64ddbde2a9c6.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\1afd37a5-1ebc-484b-af9b-29d10b60a60e.tmp.node
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\1e34b307-0dbf-41d5-9893-b42c6ceb0b96.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\21c38fc8-8ff0-47d7-80f4-0761057fa3fc.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\21dd2b2e-e96e-43e4-bcb4-9358b955f195.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\2cf3490a-1295-44cf-b9fb-3f21b916486e.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\3140643c-4c84-468d-8922-ea2dbddb5230.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\31cb0dd8-d1e4-44ee-8997-2162621239c8.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\32168852-367f-405b-90f0-c6126eeed033.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\329b82ad-a64e-4fa7-9aa5-c43c77c97705.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\3f7aa744-eb33-4e2f-8d45-d8ffb8812cde.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\48c41dc2-cff8-4a25-a986-2025a3058cfe.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\514b10f7-2c8f-4e0a-a25b-afbe4d719580.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\5172dc63-3df0-4dd2-8294-b6118b4ada16.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\5fdf4da9-bdda-4616-9bc2-55fdb575b741.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\60a57993-875a-4188-9fe1-2ff6b1173443.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\638701a3-51f2-415c-98ea-2777aacd1737.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\734a9f63-aadd-4ba5-a54a-36f7bb7e1515.tmp.node
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\7bce62ad-99b4-4fe6-95b3-6e4cf71370ef.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\7f2e758d-088b-49d1-ae28-c233f8fc2d72.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\7f35bf82-6a80-4c92-a4da-24c44c0f5952.tmp
FOUND FOLDER. FOR NOW WE DO NOT PROCESS IT RECURSIVELY: C:\Users\Massimo\AppData\Local\Temp\7zO02220FF3
FOUND FOLDER. FOR NOW WE DO NOT PROCESS IT RECURSIVELY: C:\Users\Massimo\AppData\Local\Temp\7zO0E6AEA8F
FOUND FOLDER. FOR NOW WE DO NOT PROCESS IT RECURSIVELY: C:\Users\Massimo\AppData\Local\Temp\7zO85F544FB
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\8003b2e0-ac8a-4470-9167-bdc4eed8bce0.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\874ffc32-1e62-4af6-bda1-b715b2fca594.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\935c8da8-b1b4-4c09-ab6b-13f951f7050e.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\97166447-17a1-4c79-acff-bd5c45bcbcdb.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\9dd557db-693a-4aa6-b336-e574c995265c.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\a18dcc72-0011-4e81-9120-3bf06edf4448.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\a8a4c5ea-fbcb-4b5a-9aeb-8e8acfa2a4e6.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\APPX.1gkxy0gi5bvkeipbro7o2r5lc.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\APPX.8xvwovemnc9ym9butkwcqdbcc.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\APPX.dyc8hqsiw8mqky2zqdpvi6j5f.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\APPX.eriowxceruf94cy_7l53y6zf.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\APPX.rj8qtr2x939auf4n6icuqc1tc.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\APPX.ttoax1ft9oi6j65hr0b1w1kfb.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\b2b921d2-2bf6-46a0-b8d7-f70755a186a8.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\b3a08b0b-e6c7-40f2-b1ed-870c0fb574ba.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\b6f58915-95fe-45c5-9b7c-9a1db97d7ab3.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\codemirror.css

>>>>>>>>> PROCESS CSS FILE: C:\Users\Massimo\AppData\Local\Temp\codemirror.css
Minified CSS file name: codemirror.min.css
-------------------------------------------------------------------------------------------
Executable: "C:\Users\Massimo\AppData\Local\Temp\yuicompressor-2.4.8.jar"
Arg [1]: -jar
Arg [2]: "C:\Users\Massimo\AppData\Local\Temp\yuicompressor-2.4.8.jar"
Arg [3]: --line-break
Arg [4]: 5000
Arg [5]: -o
Arg [6]: "codemirror.min.css"
Arg [7]: "C:\Users\Massimo\AppData\Local\Temp\codemirror.css"
Final Command: java -jar "C:\Users\Massimo\AppData\Local\Temp\yuicompressor-2.4.8.jar" --line-break 5000 -o "codemirror.min.css" "C:\Users\Massimo\AppData\Local\Temp\codemirror.css"

Original file: C:\Users\Massimo\AppData\Local\Temp\codemirror.css
Minified file: C:\Users\Massimo\DOCUME~1\PROGET~2\ES23DE~1\YUICOM~1\B4J\Objects\codemirror.min.css
Input: 8720 Bytes = 8.52 kB
Output: 6068 Bytes = 5.93 kB
Compression: 30.41 %
Saving: 2652 Bytes = 2.59 kB
Minimized file is 69.59 % than original file.
-------------------------------------------------------------------------------------------
CSS Returned from MinifyFileToFileAsync
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\codemirror.js

>>>>>>>>> PROCESS JS FILE: C:\Users\Massimo\AppData\Local\Temp\codemirror.js
Minified JS file name: codemirror.min.js
-------------------------------------------------------------------------------------------
Executable: "C:\Users\Massimo\AppData\Local\Temp\yuicompressor-2.4.8.jar"
Arg [1]: -jar
Arg [2]: "C:\Users\Massimo\AppData\Local\Temp\yuicompressor-2.4.8.jar"
Arg [3]: --line-break
Arg [4]: 5000
Arg [5]: -o
Arg [6]: "codemirror.min.js"
Arg [7]: "C:\Users\Massimo\AppData\Local\Temp\codemirror.js"
Final Command: java -jar "C:\Users\Massimo\AppData\Local\Temp\yuicompressor-2.4.8.jar" --line-break 5000 -o "codemirror.min.js" "C:\Users\Massimo\AppData\Local\Temp\codemirror.js"

Original file: C:\Users\Massimo\AppData\Local\Temp\codemirror.js
Minified file: C:\Users\Massimo\DOCUME~1\PROGET~2\ES23DE~1\YUICOM~1\B4J\Objects\codemirror.min.js
Input: 402012 Bytes = 392.59 kB
Output: 195703 Bytes = 191.12 kB
Compression: 51.32 %
Saving: 206309 Bytes = 201.47 kB
Minimized file is 48.68 % than original file.
-------------------------------------------------------------------------------------------
JS Returned from MinifyFileToFileAsync
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\codemirror.min.css
CSS File already minimized. Skip it: C:\Users\Massimo\AppData\Local\Temp\codemirror.min.css
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\codemirror.min.js
JS File already minimized. Skip it: C:\Users\Massimo\AppData\Local\Temp\codemirror.min.js
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\cv_debug.log
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\d74f3061-d905-4246-aac1-0d36874bb16a.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\d90c6b79-5fef-4a62-84a7-ab483cedd6af.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\d9b7d3c6-189f-4fb3-855f-f4f512ba2921.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\d9c94dd2-5f12-4e81-9dbc-33831e9726e1.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\dc2e8911-dc55-404a-a023-0c361a0c9428.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\e6199337-a84f-464c-9400-84ecfcf01cd1.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\e90e7a1c-f6e3-4278-80ab-f93770ff66a9.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\ee362878-9604-4662-a6ce-26f3baf9e6f8.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\events.log
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\f1fbd4ea-2145-4c36-ac56-4595b1279e94.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\f2a6fb1f-59d5-4bb9-81a2-c044ee42923c.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\f2b6677c-f09d-4e94-9506-5e7c182103cd.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\f397d128-43e4-4e89-a659-7f28346f7e1a.tmp
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\f5bcef5f-702c-4311-b207-feb9f95e3482.tmp
FOUND FOLDER. FOR NOW WE DO NOT PROCESS IT RECURSIVELY: C:\Users\Massimo\AppData\Local\Temp\hsperfdata_Massimo
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\jusched.log
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\mat-debug-15856.log
FOUND FOLDER. FOR NOW WE DO NOT PROCESS IT RECURSIVELY: C:\Users\Massimo\AppData\Local\Temp\MicrosoftEdgeDownloads
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\MicrosoftEdgeUpdate.log
FOUND FOLDER. FOR NOW WE DO NOT PROCESS IT RECURSIVELY: C:\Users\Massimo\AppData\Local\Temp\msdtadmin
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\NotifyIconGeneratedAumid_6095903436633358930.png
FOUND FOLDER. FOR NOW WE DO NOT PROCESS IT RECURSIVELY: C:\Users\Massimo\AppData\Local\Temp\ogs697819384
FOUND FOLDER. FOR NOW WE DO NOT PROCESS IT RECURSIVELY: C:\Users\Massimo\AppData\Local\Temp\ogs916013928
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\prof.log
FOUND FOLDER. FOR NOW WE DO NOT PROCESS IT RECURSIVELY: C:\Users\Massimo\AppData\Local\Temp\proteinrun
FOUND FOLDER. FOR NOW WE DO NOT PROCESS IT RECURSIVELY: C:\Users\Massimo\AppData\Local\Temp\PSPad
FOUND FOLDER. FOR NOW WE DO NOT PROCESS IT RECURSIVELY: C:\Users\Massimo\AppData\Local\Temp\SmartScreen
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\temp.js

>>>>>>>>> PROCESS JS FILE: C:\Users\Massimo\AppData\Local\Temp\temp.js
Minified JS file name: temp.min.js
-------------------------------------------------------------------------------------------
Executable: "C:\Users\Massimo\AppData\Local\Temp\yuicompressor-2.4.8.jar"
Arg [1]: -jar
Arg [2]: "C:\Users\Massimo\AppData\Local\Temp\yuicompressor-2.4.8.jar"
Arg [3]: --line-break
Arg [4]: 5000
Arg [5]: -o
Arg [6]: "temp.min.js"
Arg [7]: "C:\Users\Massimo\AppData\Local\Temp\temp.js"
Final Command: java -jar "C:\Users\Massimo\AppData\Local\Temp\yuicompressor-2.4.8.jar" --line-break 5000 -o "temp.min.js" "C:\Users\Massimo\AppData\Local\Temp\temp.js"

Original file: C:\Users\Massimo\AppData\Local\Temp\temp.js
Minified file: C:\Users\Massimo\DOCUME~1\PROGET~2\ES23DE~1\YUICOM~1\B4J\Objects\temp.min.js
Input: 402012 Bytes = 392.59 kB
Output: 195703 Bytes = 191.12 kB
Compression: 51.32 %
Saving: 206309 Bytes = 201.47 kB
Minimized file is 48.68 % than original file.
-------------------------------------------------------------------------------------------
JS Returned from MinifyFileToFileAsync
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\temp.min.js
JS File already minimized. Skip it: C:\Users\Massimo\AppData\Local\Temp\temp.min.js
FOUND FOLDER. FOR NOW WE DO NOT PROCESS IT RECURSIVELY: C:\Users\Massimo\AppData\Local\Temp\Wondershare
FOUND FOLDER. FOR NOW WE DO NOT PROCESS IT RECURSIVELY: C:\Users\Massimo\AppData\Local\Temp\WPF
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\ws_Crypto_20240414_0.log
FOUND FILE: C:\Users\Massimo\AppData\Local\Temp\yuicompressor-2.4.8.jar

FOLDER MINIFICATION END. SHOULD ONLY CALLED AT END WHEN PROCESSED RECURSIVELY FOLDERS, SUBFOLDERS AND FILES ONE BY ONE WITH For Each
======================== COMPLETED ========================

MINIFIED FOLDER REPORT:

Set 'ShowReport' to automatically show these infos on the log after minimizing any JS or CSS file.

Source: C:\Users\Massimo\AppData\Local\Temp\
Dest: C:\Users\Massimo\DOCUME~1\PROGET~2\ES23DE~1\YUICOM~1\B4J\Objects
Input: 812744 Bytes = 793.70 kB
Output: 397474 Bytes = 388.16 kB
Compression: 51.09 %
Saving: 415270 Bytes = 405.54 kB
Minimized files are 48.91 % than original file.

Required 8509 milliseconds to complete

 

[max123]

Ok. this forum seem dead, noone replies...

 

[Daestrum]

It's probably that members (including myself) are finding it hard to understand the problem.

Maybe make a small example app, which doesn't read files and use minify etc, but simply adds a value to a global variable. That way we wouldnt have to make an elaborate setup just to try and help.

 

[OliverA]

One thing I noticed is that in MinifyAllFilesOnFolderAsync you have

    MinifyAllFilesOnFolderInternalAsync (SourceDir, DestDir, UseMinPrefix)
    Wait For Compressor_FolderComplete (Success As Boolean, FolderItem As MinifiedFolderItem)

How do you know that the callback is prefixed with Compressor? Since the callback code in MinifyAllFilesOnFolderInternalAsync is

CallSubDelayed3(Me, mEventName & "_FolderComplete", False, Null)

?

MinifyAllFilesOnFolderAsync also uses mEventName for the callback therefore something is wrong with using Compressor_ as a prefix for the Wait For. If MinifyAllFilesOnFoldersInternalAsync is just a private function/method of the class, then use

CallSubDelayed3(Me, "Compressor_FolderComplete", False, Null)

In MinifyAllFilesOnFolderInternalAsync

Note:
I'm only focusing on the 2nd parameter of the CallSubDelayed3 method here.

 

[max123]

Thanks both @Daestrum and @OliverA, I now changed my code and I have a full recursion on ResumableSubs.
Actually I processed 120 files inside main folder and nested sub-folders, but something is already wrong in my code.

@OliverA this on my actual code works well, and call back the calling sub MinifyAllFilesOnFolderAsync:

CallSubDelayed3(Me, mEventName & "_AllFolderComplete", True, mFolderItem)

But you are absolutely right, this works for me just because in the Main I initialize the class with Compressor event.
Many thanks for the tip, I will change it in my code. ? ? ?

Actually I have 2 problems.

The first (big) problem is that Resumable Sub call itself recursively, but never reach the end of a sub to return back to a calling sub that then returs to the Main.
To solve this I used a global list, then add all folder names in the root, then any folder I process, if it is not a sub-folder I remove this from the List.
After that I check if List size reached zero size, if not I will continue to next files and folders, by calling a _FolderComplete inside the same sub, if size reached I will return back to a calling sub that now I changed to this:

    MinifyAllFilesOnFolderInternalAsync (SourceDir, DestDir, UseMinPrefix)
    Wait For Compressor_AllFolderComplete (Success As Boolean, FolderItem As MinifiedFolderItem)

This works but I'm sure it is not the right way to do it. At the end when there are no folders and files to process, it should go to the end of a sub and finally call the calling sub.

The second problem I have is that all recursively folders and sub-folders are scanned, every JS and CSS files are regulary processed, but this not happen in just one folder when I have an sub-folder with inside a file that don't need to be processed. Here when returning back, this line do not more call the _FolderComplete event and just blocks here, no errors at all but it just blocks the process I think in an infinite loop.

CallSubDelayed3(Me, mEventName & "_FolderComplete", True, Null)

As Daestrum says, may I have to create a small project to reproduce it, by just replacing the sub that process files to always return true.

 

[max123]

So, I reproduced it in a separate small project, it works same that my original project.

Here I show both my problems:

- First problem is the end of a MinifyAllFilesOnFolderInternalAsync sub that is never reached, as I explained I use a global list to know when there are no more folders and files to process, but this is not good, it should reach the end of a sub without use a list to know that threre are no more folders and files to process.

- Second problem, I placed in the InputFolder (to unzip and place on Objects folder) a rpm folder that block the process, by removing it from InputFolder, all other folders and files are processed and the sub MinifyAllFilesOnFolderInternalAsync return back a value to calling sub MinifyAllFilesOnFolderAsync, that successfully return back to the Main and all seem works fine here.
I've tried to process a lot of folders and sub-folders and return back right values to the main, only this rpm folder blocks, it contains a changes folder with just inside a file that have not to be processed. In other contests folder recursion seem to happen without problems.

Because is difficult to me working with folders and subfolders and manage files directly from DirAsset without copy folder by folder and file by file, I've attached the InputFolder zip, just unzip it in the B4J project Objects folder, it contains 4 folders with inside some js and css files and some html files, just to test files that need and not need to be processed, even not to be copied.

When you execute, the \Objects\OutputFolder will be created and all subfolders and files are saved here mantaining the same folder structure, but just renaming files by adding .min extension.

This do not reach the end, because rpm folder block the process. Try to remove rpm folder from \Objects\InputFolder and run the project again, this time it works and all subs returns back to the main with the MinifiedFolderItem filled with values.

 

[OliverA]

Here I show both my problems:

Using Method/WaitFor/CallSubDelayed way of handling things jams up after so many recursive calls. I'm attaching a project with 3 classes. The first class, TraverseFolders uses the same process as yours. It jams up after processing so many directories/files. TraverseFolders2 rewrites everything to use Subs defined As ResumableSub without using CallSubDelayed. TraverseFolder3 uses Method/WaitFor/CallSubDelayed for the public-facing method (allowing for the #Event definition) and declares all private Subs As ResumableSub. Both TraverseFolder2 and TraverseFolder3 have no issues processing folders with large counts of sub-folders/files.

 

[max123]

Many Thanks @OliverA I will try it. ?

EDIT: I tried it just now, placing my four folders in c:\temp, only TraverseFolder2 seem to work as expected, and it is the only one that (when all folders and files are processed) return back to the Main. The TraverseFolder1 blocks as my code in rpm sub, TraverseFolder3 seem to process all files, but at the end do not return to the Main. I will study your code better in debug mode using breakpoints. Thanks

 

[OliverA]

TraverseFolder3 seem to process all files, but at the end do not return to the Main

I discovered a bug in TraverseFolder3. Try it with this version.

 

[OliverA]

That did not fix the issue. Interestingly, it looks like it gets stuck in the last recursive call of TraverseAsyncInternal, without returning to TraverseAsync, and therefore the CallSubDelayed3 to Main is never executed.

 

[max123]

Mmmm, Thanks. I will try it. So seem there is no way to just use CallSubDelayed as I do in my code ?
Now I will sudy better the code flow.

 

[OliverA]

So seem there is no way to just use CallSubDelayed as I do in my code ?

The final return to Main can be a CallSubDelayed. I fixed the bug (it was a silly one). TraverseFolders3 works correctly now.

 

[max123]

Now even FolderTraverse3 seem to works to me, it returns to the main, here a screenshoot.

 

[max123]

It looked too good, unfortunately my sub rpm still blocks and doesn't allow me to return to the Main.
There is still something wrong in FolderTraverse.
I will check better the code flow.

 

[OliverA]

There is still something wrong in FolderTraverse1

As expected. Using CallSubDelayed recursively does not seem to work. Only FolderTraverse2 and FolderTraverse3 work. 2 defines all involved methods As Resumable Sub. 3 uses CallSubDelayed to take advantage of the #Event attribute that can be used to define events for a class. I may make a separate post later in the forum to inquire why recursively using CallSubDelayed does not work.

 

[max123]

If you want to know because this happen, you will have to place this:

Public Sub TraverseAsync(SourceDir As String)As ResumableSub
    cProcessFile = 0
    dProcessFile = 0

so the counts reset any time and all three traverse can be compared in the log

I will try to compare Traverse and Traverse2 and if I succcess I will post a screenshot.

 

[max123]

Here the log difference between FolderTraverse, FolderTraverse2 and FolderTraverse3 classes. The first do nort return back, but this strangely seem to happen only with this rpm folder, it is the only one of 3 folders that have a subfolder with inside a file.

 

[OliverA]

Here the log difference between FolderTraverse2 and FolderTraverse, the first completes and return to the Main, the second do not.

Correct. There seems to be an issue with recursion and CallSubDelayed. The solution, for now, is to use ResumableSubs (as used in TraverseFolders2 and TraverseFolders3).

 

[max123]

@OliverA I want to use your TraverseFolder3 method in my library, but I've difficult to adapt my Subs to even return a List back,
because the main sub called (the one that process files) is a Public sub to just process one file, and it cannot return a List, already return Success and MinifiedFileItem.

Can I just use a global List without return it back from the ProcessFile and then from TraverseAsyncInternal ?

May I can just fill and clear it when necessary without return back from the Subs.

Now I will try to creare TraverseFolders4 with global list.