TensorFlow - an experimental machine/deep learning wrapper for B4J
After writing a TensorFlowLite library for Android, I was asked to do a similar one for B4J. My library wraps TensorFlow (v. r.1.10) and is based on some Java examples found in TensorFlow's Github pages. I also added some functionality. Please note that this is not a complete wrapper of TensorFlow but a rather small one permitting you to test out some of the features although still very much usable. Since the Java APIs are available, one could probably make a complete wrapper but in this moment I don't have the time. In addition, it seems like TensorFlow version 2.0 will be released later this year so it might be better to wait until then.
First some background (from the TensorFlow website):
What is TensorFlow?
TensorFlow™ is an open source software library for high performance numerical computation. Its flexible architecture allows easy deployment of computation across a variety of platforms (CPUs, GPUs, TPUs), and from desktops to clusters of servers to mobile and edge devices. Originally developed by researchers and engineers from the Google Brain team within Google’s AI organization, it comes with strong support for machine learning and deep learning and the flexible numerical computation core is used across many other scientific domains.
You can read more here:
https://www.tensorflow.org/
The user scenarios can be numerous. This wrapper (and the demo-app) provided by me lets you load some pictures from your computer which TensorFlow will then analyze and try to figure out what objects are shown in the pictures. The objects suggested can be more than one and therefore they are sorted per a confidence-score. This can be done because TensorFlow is analyzing the image against a predefined model (a sort of classifier, graph), which has been trained to recognize certain objects. In the demo, I am using a generic sample-model (Inception5h) created by Google and which recognizes various objects.
More importantly, you can create and train your own models, specifically trained to perhaps recognize animals, cars etc and let you get far better results in terms of accuracy. I have also created my own model which recognizes my guitars. I set up what needed on my Mac and created a model in about 30 minutes. Models can be of different types, depending on how they were created and/or (re)trained. Some common properties of a model can be named differently. Therefore when you are initializing the library, you need to pass on various parameters. See IDE-help for suggestions. You can find instructions for creating models on TensorFlow's web-site and of course on YouTube. I also recommend the following Codelab (tutorial) by Google: tensorflow-for-poets.
Screenshot from Demo-app:
How to use the wrapper in your app?
1) The official TensorFlow library by Google is being developed continuously and perhaps future libraries may not work with my current implementation. Therefore, for this wrapper you will need to download the following version (1.10) from here (please note that also other necessary libraries (= dependencies) are included:
https://www.dropbox.com/s/8haz4xa6a06l7jb/Tesnorflowlibs.zip?dl=0
and copy them to your additional library folder.
2) Download and extract the attached TensorFlow library wrapper and its XML-file and copy them to your additional library folder.
3) Finally, you will of course need a model (classifier/graph). The demo app is using an Inception5h-model created by Google which you can download from here:
https://storage.googleapis.com/download.tensorflow.org/models/inception5h.zip
Unzip to a folder of your choice and point to it in your app..
4) A demo-app is attached as well. Look at the code in the demo-app and you can see how quick and easy it is to implement this wrapper in your own apps.
I have added the Java-sources too if someone would like to add/change functionality or maybe keep the wrapper updated and in line with newer future releases of TensorFlow.
Note: There are different models available which you can use as a base for creating and training your own. The differences are normally trade-offs between speed and accuracy. Another difference is that a model can be more focused for image-recognition (like my demo) while others can be optimized for object-detection.
Library-information (thanks to @Informatix LibDoc-generator) :
B4JTensorflow
Author: moster67/Mikael Osterhed
Version: 0.3
Tested only on PC (Windows). It will probably work on Mac and Linux too. If not, please let me know.
Please remember that creating libraries and maintaining them takes time and so does supporting them. Please consider a donation if you use my free libraries as this will surely help keeping me motivated. Thank you!
Enjoy!
After writing a TensorFlowLite library for Android, I was asked to do a similar one for B4J. My library wraps TensorFlow (v. r.1.10) and is based on some Java examples found in TensorFlow's Github pages. I also added some functionality. Please note that this is not a complete wrapper of TensorFlow but a rather small one permitting you to test out some of the features although still very much usable. Since the Java APIs are available, one could probably make a complete wrapper but in this moment I don't have the time. In addition, it seems like TensorFlow version 2.0 will be released later this year so it might be better to wait until then.
First some background (from the TensorFlow website):
What is TensorFlow?
TensorFlow™ is an open source software library for high performance numerical computation. Its flexible architecture allows easy deployment of computation across a variety of platforms (CPUs, GPUs, TPUs), and from desktops to clusters of servers to mobile and edge devices. Originally developed by researchers and engineers from the Google Brain team within Google’s AI organization, it comes with strong support for machine learning and deep learning and the flexible numerical computation core is used across many other scientific domains.
You can read more here:
https://www.tensorflow.org/
The user scenarios can be numerous. This wrapper (and the demo-app) provided by me lets you load some pictures from your computer which TensorFlow will then analyze and try to figure out what objects are shown in the pictures. The objects suggested can be more than one and therefore they are sorted per a confidence-score. This can be done because TensorFlow is analyzing the image against a predefined model (a sort of classifier, graph), which has been trained to recognize certain objects. In the demo, I am using a generic sample-model (Inception5h) created by Google and which recognizes various objects.
More importantly, you can create and train your own models, specifically trained to perhaps recognize animals, cars etc and let you get far better results in terms of accuracy. I have also created my own model which recognizes my guitars. I set up what needed on my Mac and created a model in about 30 minutes. Models can be of different types, depending on how they were created and/or (re)trained. Some common properties of a model can be named differently. Therefore when you are initializing the library, you need to pass on various parameters. See IDE-help for suggestions. You can find instructions for creating models on TensorFlow's web-site and of course on YouTube. I also recommend the following Codelab (tutorial) by Google: tensorflow-for-poets.
Screenshot from Demo-app:
How to use the wrapper in your app?
1) The official TensorFlow library by Google is being developed continuously and perhaps future libraries may not work with my current implementation. Therefore, for this wrapper you will need to download the following version (1.10) from here (please note that also other necessary libraries (= dependencies) are included:
https://www.dropbox.com/s/8haz4xa6a06l7jb/Tesnorflowlibs.zip?dl=0
and copy them to your additional library folder.
2) Download and extract the attached TensorFlow library wrapper and its XML-file and copy them to your additional library folder.
3) Finally, you will of course need a model (classifier/graph). The demo app is using an Inception5h-model created by Google which you can download from here:
https://storage.googleapis.com/download.tensorflow.org/models/inception5h.zip
Unzip to a folder of your choice and point to it in your app..
4) A demo-app is attached as well. Look at the code in the demo-app and you can see how quick and easy it is to implement this wrapper in your own apps.
I have added the Java-sources too if someone would like to add/change functionality or maybe keep the wrapper updated and in line with newer future releases of TensorFlow.
Note: There are different models available which you can use as a base for creating and training your own. The differences are normally trade-offs between speed and accuracy. Another difference is that a model can be more focused for image-recognition (like my demo) while others can be optimized for object-detection.
Library-information (thanks to @Informatix LibDoc-generator) :
B4JTensorflow
Author: moster67/Mikael Osterhed
Version: 0.3
- TensorFlow
- Events:
- OnTensorFlowLoaded
- Functions:
- Initialize (EventName As String, ModelInputName As String, ModelOutputName As String, ModelDir As String, ModelName As String, LabelsName As String, ModelScaledPixelSize As Int)
Initializes tensorflow. Pass EventName for using events in B4A. Other parameters are:
ModelInputName = entrypoint in graph, ModelOutputName=exitpoint in graph,
ModelDir=directory where model and label are stored, ModelName=name of model-file,
LabelsName=name of label-file, ModelScaledPixelSize=Size used in model
Note: Above parameters depend on the model in use. Check model specification and/or (re)training script. - RecognizeImageBitmap (image As javafx.scene.image.Image) As String
Method for recognizing an image-object in memory. Similar to Bitmap in B4A.
Returns a String with a possible result.
Example: Dim bmp as Image
bmp = some image in memory
Dim result as String = ten.RecognizeImageBitmap(bmp)
Log(result) - RecognizeImages (ImagePathAndFileNames As java.util.List) As java.util.List
Method for recognizing images. Returns a List with possible results
Usage: Pass a List with path and filename(s) of the images to recognize
Example: Dim PicList As List
PicList.Initialize
PicList.add("C://MachineLearning//testimages//porcupine.jpg")
Dim result As String = ten.RecognizeImages(PicList) 'better to iterate list
Log(result
- Initialize (EventName As String, ModelInputName As String, ModelOutputName As String, ModelDir As String, ModelName As String, LabelsName As String, ModelScaledPixelSize As Int)
- Events:
Tested only on PC (Windows). It will probably work on Mac and Linux too. If not, please let me know.
Please remember that creating libraries and maintaining them takes time and so does supporting them. Please consider a donation if you use my free libraries as this will surely help keeping me motivated. Thank you!
Enjoy!
Last edited:
