[B4X] Faces, Assemblies, Traces, and Points. Standard Classes for creating a story.

Since retiring some time ago, I have been thinking about creating stories that involve
computer operated robots which carry out actions and speech - like actors in a drama.

It would have at least two mobile robotic platforms that can move on a stage - about the size of a dining table.
Each platform would have a Android tablet on-board that receives instructions from a desktop or laptop.
Those instructions would determine movements on the stage.
Each tablet would also show a human-like face on the screen.
This face would display emotions and lipsync to sound tracks.

I have been working intermittently on pieces of this project.
For me, the journey is definitely more fun than getting there.

Here I present some of my experiences.  I will concentrate on things one can do with B4X
on the Desktop and Android tablet. Most of the code should also run on B4i. 

A short sample of the end point of this tutorial can be seen in this animated .gif

____Session F. animated gif

For now, I will discuss the software building blocks of this ambitious project.
There are other discussions I can have about hardware and inter-device communication.
I'll leave that for another time.

Let's create a set of structures - each will be implemented as one or more standard classes.
   Story: a set of ordered scenes		'to be done
   Scene: a collection of Faces
   Face: a collection of Assemblies
   Assembly: a collection of Traces
   Trace: a set of ordered Points
   Point: a pair of ordered Numbers

_____________________________________________Tutorial A._______________________________

Let's start from the bottom.
One could define a "Point" as a custom type. I have done that in the past, and that works well.
But as a class, it can have time-saving methods such as adding points and finding the distance between points.

One note about terminology.  It is very cumbersome to always distinguish between the instances of a class and the class itself.
My convention here is to use lowercase and mostly plural for instances and ProperCase and singular for classes.
For example points are instances of the class Point.

In B4X, an instance of a class can spawn instances of itself and other classes.  This is very useful.
If we start with a generator instance, we can use it to generate new instances, without cluttering up modules with extra methods. 

Copy the following in a new project or download the attached.zip

B4XMainPage
_____________________________________
Sub Class_Globals
	Private Root As B4XView
	Private xui As XUI
	Private CV As B4XCanvas
	Private Pnt As Point			'generator instance
End Sub

Public Sub Initialize
	Pnt.Initialize
End Sub

Private Sub B4XPage_Created (Root1 As B4XView)
	Root = Root1
	CV.Initialize(Root)
	
	Dim center As Point = Pnt.New(Root.Width / 2, Root.Height / 2)
	Log(center.X & TAB & center.Y)	'300    300
	
	CV.drawCircle(center.X, center.Y, 50dip, xui.Color_Blue, False, 3)

	Log(2 * center.DistanceTo(Pnt.New(0, 0)))	'length of screen diagonal
'Or
	Log(2 * Pnt.New(0, 0).DistanceTo(center))	'length of screeen diagonal
	
'You could also define center as halfway down the diagonal
	Dim TopLeft As Point = Pnt.New(0, 0)
	Dim BottomRight As Point = Pnt.New(Root.Width, Root.Height)
	center = TopLeft.halfwayTo(BottomRight)
	Log(center.X & TAB & center.Y)	'300    300

'Or as the centroid
	center = TopLeft.Add(BottomRight).MultBy(1 / 2)
	Log(center.X & TAB & center.Y)	'300    300
End Sub

Point
___________________________________
Sub Class_Globals
	Public X, Y As Float
End Sub

Public Sub Initialize
End Sub

'Spawns a new instance of Point
Public Sub New(X_ As Float, Y_ As Float) As Point
	Dim P As Point
	P.Initialize
	P.X = X_
	P.Y = Y_
	Return P
End Sub

'Copies a Point
Public Sub Copy As Point
	Return New(X, Y)
End Sub

'Subtract second Point from first
Public Sub Minus(p As Point) As Point
	Return New(X - p.X, Y - p.Y)
End Sub

'Multiplies a Point by a constant
Public Sub MultBy(factor As Float) As Point
	Return New(factor * X, factor * Y)
End Sub

'Adds two Points
Public Sub Add(p As Point) As Point
	Return New(X + p.X, Y + p.Y)
End Sub

'Finds distance between two Points
Public Sub distanceTo(p As Point) As Float
	Dim dx As Float = p.X - X
	Dim dy As Float = p.Y - Y
	Return Sqrt(dx * dx + dy * dy)
End Sub

'Finds the mid-Point between two Points
Public Sub halfwayTo(p As Point) As Point
	Return New((X + p.X) / 2, (Y + p.Y) / 2)
End Sub

'Returns a point that is partway (fraction) along the line between two points
Public Sub partwayTo(p As Point, fraction As Float) As Point
	Return New(X + fraction * (p.X - X), Y + fraction * (p.Y - Y))
End Sub


_____________________________________________Tutorial B._______________________________
A "Trace" is a geometric shape containing the geometric information which remains when location, scale, orientation and reflection are removed.
I am coining this term because traces are more than just regular shapes like a circles, rectangles, polygons, or enclosed spaces.
They can also be dots, lines and un-closed curves. The term refers to tracing a drawing, or tracing a numbered set of dots.

I also did not want to confuse these geometrical structures with the View objects shaped by the excellent B4X Shape library created by @Steve05.

Traces are rendered as a series of line segments, if they are filled they are rendered with B4XPath.
They do not have physical dimensions until they are rendered on a given screen at a specified position with a tilt angle and a scaling factor. 
In addition, to make an instance of Trace visible you have to provide a color scheme and line width.

1. A trace's form is an ordered set of Points.  This is implemented as a List.  A trace can be a set of daisy-chained curves.

2. It should be possible to move and rotate traces. A trace's position itself is a Point. Traces also have a TiltAngle. 

3. Traces should be screen-independent. This is implemented by standardizing all dimensional units.
A convenient unit is Min(Screen width, Screen height) / 1000.  This makes the drawings fit in any orientation.
and 10 standard units are equivalent to 1% of the maximum sized square that fits the screen.

4. It also should have color and texture information so that it can be properly rendered.

_______  Insert screen shot of traces

The drawing above was made with the following code.
Because of extensive use of Point, the Trace class is compact (about 450 lines) and is included in the attached .zip
___________________________________

'In Sub Class_Globals
'	Private Traces(11) As Trace			'they can also be named individually, in this demo there are 11
'	Public SU As Float

Private Sub B4XPage_Created (Root1 As B4XView)
	Root = Root1
	centerX = Root.Width / 2
	centerY = Root.Height / 2
	SU = Min(Root.Width, Root.Height) / 1000
	backgroundClr = Root.Color
	
	CV.Initialize(Root)
	For i = 0 To Traces.Length - 1
		Traces(i).Initialize		'a un-formed Trace at the screen center
	Next

	'When Traces are defined, the specified dimensions are given relative to the Root, they are automatically changed to standard units.
	Traces(0).Line(150)						'A 150 pixel wide line which when drawn will be at the center of current screen
	Traces(0).MoveTo(centerX - 200, 75)		'Move the line into position, left of center, near the top, you'll see this when Trace is drawn
	Traces(0).Rotate(-135)					'Rotate the line around its own center, this will not be visible until the Trace is drawn
	Traces(0).Specify(CreateMap("outline": black, "thickness": 3))
	
	Traces(1).Triangle(50, 100, 100)
	Traces(1).MoveTo(centerX, centerY - 200)
	Traces(1).Rotate(90)
	Traces(1).Specify(CreateMap("fill":paleGreen, "outline":green, "thickness":3))

	Traces(2).Rectangle(200, 100)
	Traces(2).Rotate(90)
	Traces(2).Specify(CreateMap("fill":paleBlue, "outline":blue, "thickness":3))
	
	Traces(3).Polygon(50, 5)
	Traces(3).MoveTo(centerX - 100, centerY + 200)
	Traces(3).Rotate(180)
	Traces(3).MoveLeft(5)		'An alternative way to move Traces is as a percentage in a Left, Right, Up, Down direction
	Traces(3).Specify(CreateMap("fill":paleRed, "outline":blue, "thickness":3))

	'Four pairs of x,y co-ordinates representing the points defining a Bezier curve, the curve will be mirrored to form a symmetric Trace
	Traces(4).SymCurve(centerX, centerY - 13, centerX - 100, centerY - 75, centerX - 55, centerY + 53, centerX, centerY + 70, 25)
	Traces(4).MoveRight(30)		
	Traces(4).MoveDown(30)
	Traces(4).Specify(CreateMap("fill":crimson, "thickness":3))
	
	Traces(5) = Traces(4).copy	'Clones the previous Trace (heart) and moves and rotates it, it will be drawn in a different color scheme
	Traces(5).MoveLeft(25)
	Traces(5).Rotate(15)
	Traces(5).Specify(CreateMap("outline":crimson, "thickness":3))

	'Freeform Trace defined by a sequence of points - this is just a amorphous blob 
	Traces(6).AddPoint(100,100)
	Traces(6).AddPoint(150,200)
	Traces(6).AddPoint(200,100)
	Traces(6).AddPoint(250,400)
	Traces(6).AddPoint(300,100)
	Traces(6).AddPoint(350,400)
	Traces(6).AddPoint(100,400)
	Traces(6).AddPoint(100,100)
	Traces(6).Shrink(50)			'Scales the blob to half its size (50%) - when drawn it occupied too much space for this demo 
	Traces(6).MoveRight(30)
	Traces(6).MoveUp(10)
	Traces(6).Smooth(5, 5)			'A very useful smoothing function
	Traces(6).Rotate(135)
	Traces(6).Specify(CreateMap("fill":paleRed, "outline":crimson, "thickness":3))
	
	Traces(7).Oval(200, 100)		'This is an oval, if the width and height are the same it is a circle
	Traces(7).MoveLeft(30)
	Traces(7).Specify(CreateMap("fill":paleRed, "outline":crimson, "thickness":3))

	Traces(8).Arrow(250, 15, 60, 15)	'The Arrow class is embedded: https://www.b4x.com/android/forum/threads/b4x-a-class-to-draw-on-canvas-many-types-of-arrows-at-any-angle.142539/
	Traces(8).MoveRight(19)
	Traces(8).MoveDown(13)
	Traces(8).Rotate(-45)
	Traces(8).Specify(CreateMap("outline":blue, "thickness":1, "dashed":True))
	
	Traces(9).Dot
	Traces(9).Specify(CreateMap("outline":black, "thickness":3))
	
	'This simple Bezier curve will become the base line for the curved text example - it can be drawn or remain un-rendered
	Traces(10).Curve(centerX + 50, centerY - 253, centerX - 100, centerY - 75, centerX - 55, centerY + 53, centerX, centerY + 70, 1000)
	Traces(10).MoveLeft(10)
	Traces(10).Rotate(90)
	Traces(10).MoveTo(centerX, 50)
	Traces(10).Specify(CreateMap("outline":red, "thickness":1, "closed":False))
	
	renderTraces
End Sub

Private Sub renderTraces
	Dim r As B4XRect
	r.Initialize(0, 0, Root.Width, Root.Height)
	CV.ClearRect(r)
	For Each dude As Trace In Traces
		dude.Render
	Next

'Fonts sizes will be automatically adjusted to screen sizes and resizing - this text is drawn in the center of a Trace (-45 degree arrow)
	Traces(8).addText("Testing A String", xui.CreateDefaultBoldFont(11), blue)

'The text follows the curve that is Trace(10)
	Traces(10).addCurvedText("Standard Units Work Well Here", xui.CreateDefaultBoldFont(15), black)
End Sub

___________________________________

Download the project, and open it in B4J. You can see the effect of changing screen sizes by resizing the form.

_____________________________________________Tutorial C._______________________________

One of the methods of Trace is drawing smooth curved amorphous forms. How this is done is explained here
https://www.b4x.com/android/forum/threads/b4x-fitting-a-smooth-curved-line-to-a-sequence-of-points.143178/

The text on a curve method is explained here
https://www.b4x.com/android/forum/threads/b4x-text-along-any-curve.138639/

The Trace class can render Bezier curves. I will explain now how these types of curves are created.
See a good example of the power of bezier curves https://www.b4x.com/android/forum/threads/b4x-b4xpages-happy-hearts-day-with-fun-tasia-flowers.145886/

_______  Insert screen shot of bezier heart

For those not familar with Bezier curves, they are smooth complex curves that are specified by 4 points.
An simple example is shown here: https://www.b4x.com/android/forum/threads/b4x-text-along-any-curve.138639/

There are start and end points that are the terminals of the curve. In between are two control points, conceptualized as pulling-forces on the curve.
Since the whole curve is defined by these 4 points, it can be animated by simply changing the points and re-drawing the curve.
There is no need for bitmap/image processing or movie frames. Just B4XCanvas and B4XPath objects.

Attached is an editor for the curve seen in the image above.
(For this demo, I have made the right side a mirror image of the left.)

Download the project, and open it in B4J. You can see the effect on the curve by dragging the control points.
You might be interested to see the unusual way that touch events are implemented.

Also try re-sizing the form, you'll see that all of the dragging and editing work seamlessly on the resized screen.
When the mouse is released, the Log will show the new standardized control points for the curve. They will be un-affected by resizing.
These points can be exported to any code that includes the Point and Trace classes, and shown on any sized screen.

_____________________________________________Tutorial D._______________________________

Back to my goal of showing expressive human-like faces on flat screens.
A 2D face can be seen as a collection of about 20 traces: 
  head (skull, jaw, neck)
  hair (hair top, hair bottom)
  nose (top-part, nostrils)
  eyebrow
  ear (top-part, lobe)
  eye (upper/lower lid - eyeball and pupil are separate)
  mouth (upper/lower lip, upper/lower teeth, tongue)
  
  _______  Insert screen shot of curves of face

_____________________________________________Tutorial E._______________________________

The Assembly class is an ordered collections of traces.  This is also implemented as a List

The traces in a Assembly should move in a co-ordinated way. 
Traces already are capable of rendering themselves.
The assembly can use its traces for animation.

_______  Insert screen shot of one face

In the attached project, you'll see that a lot of complexity can be reduced by this abstraction.
A convincing human-like face can be rendered with about 320 lines of code, including all bezier curve control points.

'A sample of the code is shown below

	mouthArea.Initialize
	mouthArea.addTrace(throat)
	mouthArea.addTrace(tongue)
	mouthArea.addTrace(upperTeeth)
	mouthArea.addTrace(lowerTeeth)
	For Each tr As Trace In makeToothLines
		mouthArea.addTrace(tr)
	Next
	
	mouthArea.Render

_____________________________________________Tutorial F._______________________________

This is the final session in the tutorial. It contains all the techniques discussed in the earlier sessions.
Some of the classes in the attached .zip file have matured from the earlier versions.  
So if you plan a similar project use the ones here.

It also has:

1. Multiple faces
2. Lip synching
3. Emotional expression

It is challenging, but the biggest lesson of all is that if you break things down into manageable parts,
many seeminly impossible tasks become possible.

It also helps if you have tools like B4X.



