B4J Library TensorFlow - an experimental machine/deep learning wrapper for B4J

Discussion in 'B4J Libraries & Classes' started by moster67, Sep 1, 2018.

  1. moster67

    moster67 Expert Licensed User

    TensorFlow - an experimental machine/deep learning wrapper for B4J

    After writing a TensorFlowLite library for Android, I was asked to do a similar one for B4J. My library wraps TensorFlow (v. r.1.10) and is based on some Java examples found in TensorFlow's Github pages. I also added some functionality. Please note that this is not a complete wrapper of TensorFlow but a rather small one permitting you to test out some of the features although still very much usable. Since the Java APIs are available, one could probably make a complete wrapper but in this moment I don't have the time. In addition, it seems like TensorFlow version 2.0 will be released later this year so it might be better to wait until then.

    First some background (from the TensorFlow website):

    What is TensorFlow?
    TensorFlow™ is an open source software library for high performance numerical computation. Its flexible architecture allows easy deployment of computation across a variety of platforms (CPUs, GPUs, TPUs), and from desktops to clusters of servers to mobile and edge devices. Originally developed by researchers and engineers from the Google Brain team within Google’s AI organization, it comes with strong support for machine learning and deep learning and the flexible numerical computation core is used across many other scientific domains.

    You can read more here:
    https://www.tensorflow.org/

    The user scenarios can be numerous. This wrapper (and the demo-app) provided by me lets you load some pictures from your computer which TensorFlow will then analyze and try to figure out what objects are shown in the pictures. The objects suggested can be more than one and therefore they are sorted per a confidence-score. This can be done because TensorFlow is analyzing the image against a predefined model (a sort of classifier, graph), which has been trained to recognize certain objects. In the demo, I am using a generic sample-model (Inception5h) created by Google and which recognizes various objects.

    More importantly, you can create and train your own models, specifically trained to perhaps recognize animals, cars etc and let you get far better results in terms of accuracy. I have also created my own model which recognizes my guitars. I set up what needed on my Mac and created a model in about 30 minutes. Models can be of different types, depending on how they were created and/or (re)trained. Some common properties of a model can be named differently. Therefore when you are initializing the library, you need to pass on various parameters. See IDE-help for suggestions. You can find instructions for creating models on TensorFlow's web-site and of course on YouTube. I also recommend the following Codelab (tutorial) by Google: tensorflow-for-poets.

    Screenshot from Demo-app:

    Capture1.jpg

    How to use the wrapper in your app?
    1) The official TensorFlow library by Google is being developed continuously and perhaps future libraries may not work with my current implementation. Therefore, for this wrapper you will need to download the following version (1.10) from here (please note that also other necessary libraries (= dependencies) are included:
    https://www.dropbox.com/s/8haz4xa6a06l7jb/Tesnorflowlibs.zip?dl=0
    and copy them to your additional library folder.
    2) Download and extract the attached TensorFlow library wrapper and its XML-file and copy them to your additional library folder.
    3) Finally, you will of course need a model (classifier/graph). The demo app is using an Inception5h-model created by Google which you can download from here:
    https://storage.googleapis.com/download.tensorflow.org/models/inception5h.zip
    Unzip to a folder of your choice and point to it in your app..
    4) A demo-app is attached as well. Look at the code in the demo-app and you can see how quick and easy it is to implement this wrapper in your own apps.

    I have added the Java-sources too if someone would like to add/change functionality or maybe keep the wrapper updated and in line with newer future releases of TensorFlow.

    Note: There are different models available which you can use as a base for creating and training your own. The differences are normally trade-offs between speed and accuracy. Another difference is that a model can be more focused for image-recognition (like my demo) while others can be optimized for object-detection.

    Library-information (thanks to @Informatix LibDoc-generator) :
    B4JTensorflow

    Author: moster67/Mikael Osterhed
    Version: 0.3
    • TensorFlow
      • Events:
        • OnTensorFlowLoaded
      • Functions:
        • Initialize (EventName As String, ModelInputName As String, ModelOutputName As String, ModelDir As String, ModelName As String, LabelsName As String, ModelScaledPixelSize As Int)
          Initializes tensorflow. Pass EventName for using events in B4A. Other parameters are:
          ModelInputName = entrypoint in graph, ModelOutputName=exitpoint in graph,
          ModelDir=directory where model and label are stored, ModelName=name of model-file,
          LabelsName=name of label-file, ModelScaledPixelSize=Size used in model
          Note: Above parameters depend on the model in use. Check model specification and/or (re)training script.
        • RecognizeImageBitmap (image As javafx.scene.image.Image) As String
          Method for recognizing an image-object in memory. Similar to Bitmap in B4A.
          Returns a String with a possible result.
          Example: Dim bmp as Image
          bmp = some image in memory
          Dim result as String = ten.RecognizeImageBitmap(bmp)
          Log(result)
        • RecognizeImages (ImagePathAndFileNames As java.util.List) As java.util.List
          Method for recognizing images. Returns a List with possible results
          Usage: Pass a List with path and filename(s) of the images to recognize
          Example: Dim PicList As List
          PicList.Initialize
          PicList.add("C://MachineLearning//testimages//porcupine.jpg")
          Dim result As String = ten.RecognizeImages(PicList) 'better to iterate list
          Log(result

    Tested only on PC (Windows). It will probably work on Mac and Linux too. If not, please let me know.

    Please remember that creating libraries and maintaining them takes time and so does supporting them. Please consider a donation if you use my free libraries as this will surely help keeping me motivated. Thank you!

    Enjoy!
     

    Attached Files:

    Last edited: Sep 2, 2018
  2. Don Oso

    Don Oso Member Licensed User

    wow , Thanks !
     
  3. Johan Hormaza

    Johan Hormaza Active Member Licensed User

    Excellent! great job!!
     
  4. rbghongade

    rbghongade Active Member Licensed User

    Tested with great results. Can you please expose the training methods? That would be a break through
     
  5. moster67

    moster67 Expert Licensed User

    As you may have read on TensorFlow's website, the API is currently experimental and is not covered by TensorFlow API stability guarantees. In addition, as mentioned in the first post, the APIs are numerous and would require quite a lot of work and unfortunately I don't have time in this period to do that.

    I found that with Python, it is pretty straight forward to create your own models. By following the "Tensorflow for poets" tutorial, which I linked to in my first post, it is not too difficult to make your own models. There are also many tutorials on YouTube covering this.
     
    rbghongade likes this.
Loading...
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.
    Dismiss Notice