Sparks Description

From Fiona Docs
Jump to: navigation, search

Contents

ASRSpark

Description

Its name is due to the acronym ASR (Automatic Speech Recognition), and it's designed to convert an audio input into a text that represents it.

Use

Spark Connections
Asking Interfaces Answering Interfaces

This spark provides IFlow<AudioWrap*> functions and it needs to use the ones from the IFlow<char*>. This means that, in order to include this feature in your avatar, you need to connect it with a spark which calls ASR to send audio data through IFlow<AudioWrap*>. At the same time, in order to use the generated output, it should be connected with another spark which can be called from ASR making use of the IFlow<char*> functions.

Figure 1. Configuration example for ASRSpark

AVInputSpark

Description

This spark allows to capture audio and video from the user's webcam, and send info to those sparks who may need it.

Use

Spark Connections
Asking Interfaces Answering Interfaces

In order to use this spark into your configuration, you must connect it with sparks which provide IFlow<AudioWrap*> and IFlow<Image*> functions. Since it implements IConcurrent features (needed by the framework to start/stop threads), it uses the ThreadSpark to create a thread to process the audio/video input stream. It is not mandatory to connect IAsyncFatalErrorHandler interface, FIONA's framework will do this for you.

A configuration example for this spark is shown below. To see examples of the other components that appear in Figure 2, you should go through the spark's section along this documentation.

Figure 2. Configuration example for AVInputSpark

CameraControlSpark

Description

It lets you change the camera position.

Use

Spark Connections
Asking Interfaces Answering Interfaces


This component must be connected with sparks that offer ICamera. Since this spark modifies the camera position and affects the character's video output, it uses IFrameEventPublisher to subscribe to character's render refresh. This way, this spark will be notified everytime a new video frame is rendered and will have the opportunity to set a new camera position. It is not mandatory to connect this interface, FIONA's framework will do it for you.


Figure 3. Configuration example for CameraControlSpark

Properties

  • Scene_Camera_Position_X:
  • Scene_Camera_Rotation_X:
  • Scene_Camera_Position_Y:
  • Scene_Camera_Rotation_Y:
  • Scene_Camera_Position_Z:
  • Scene_Camera_Rotation_Z:
  • Scene_Camera_Dynamic: Sets if the camera will move.
  • Scene_Camera_Circle: It indicates whether the camera path will be like a circle or a diamond (true or false value).
  • Scene_Camera_CameraParameters_VisionAngle:
  • Scene_Camera_Speed: This property sets the camera speed (between 0 and 1).

CharacterEmbodiment3DSpark

Description

This spark is the heart of the avatar. It gives a body to your character, renders it and sends the stream to the SparkRender player.
Soon, you will be able to change between your favourite character appearance.

Use

Spark Connections
Asking Interfaces Answering Interfaces


This component needs to be connected to sparks which offer IAsyncFatalErrorHandler functions, in order to manage possible failures, and also needs to be related to those that provide IControlVoice and IAudioQueue characteristics. It can also connect with modules that need to call IAnimation, IFaceExpression, IEyes, INeck, IConcurrent, IApplication, IAsyncFatalErrorHandler, IEventQueue, ICamera and IFrameEventPublisher functions, because it implements them.

Figure 4. Configuration example for CharacterEmbodiment3D

ChatSpark

Description

This spark allows communication between the user and the avatar by connecting with a dialogue manager, through IFlow<char*> interface.

Use

Spark Connections
Asking Interfaces Answering Interfaces

To give your avatar the ability to establish a dialogue with the user, it is necessary to connect this spark with a dialogue manager which offers and use the interface IFlow<char*>.
Since it implements IConcurrent features (needed by the framework to start/stop threads), it uses the ThreadSpark to create a thread to process the messages sent by the user. It is not mandatory to connect IAsyncFatalErrorHandler interface, FIONA's framework will do this for you.

Figure 5. Configuration example for ChatSpark

EyeContactSpark

Description

It sets a new position for the eyes and head, based on the face coordinates received through IDetectedFacePositionConsumer interface.

Use

Spark Connections
Asking Interfaces Answering Interfaces

Since this spark modifies the eyes and head position and affects the character's video output, it uses IFrameEventPublisher to subscribe to character's render refresh. This way, this spark will be notified everytime a new video frame is rendered and will have the opportunity to set a new position. It is not mandatory to connect this interface, FIONA's framework will do it for you.

To have this spark as part of the avatar's configuration it is needed to connect it with sparks which offer IEyes, INeck and IFrameEventPublisher (remember that this connection could be omitted by the user when building the avatar) functions. Furthermore, this spark offers IDetectedFacePositionConsumer features, which means, it should be connected with any spark which can pass face position info.

Figure 6 shows how you could connect this spark with other components.

Figure 6. Configuration example for EyeContactSpark

FaceTrackerSpark

Description

It gives your avatar the ability of detecting your face through your webcam.

Use

Spark Connections
Asking Interfaces Answering Interfaces

In order to add this spark to a configuration, it is necessary to remember this spark calls IDetectedFacePositionConsumer method to pass face position info, which means it must be connected with any spark which offers an implementation of that interface. On the other hand, it can be connected with sparks that use IFlow<Image*> interface to pass video stream.

Since it implements IConcurrent features (needed by the framework to start/stop threads), this means it uses the ThreadSpark to create a thread to process the audio/video input stream. It is not mandatory to connect IAsyncFatalErrorHandler interface, FIONA's framework will do this for you.

You can watch this Tutorial for more information: How to teach your avatar to follow your gaze in Fiona

Figure 7. Configuration example for FaceTrackerSpark

RebeccaAIMLSpark

Description

This spark is a dialogue manager, which gets a user text message, and generates an answer, so it can establish a conversation between the user and the avatar.

Use

Spark Connections
Asking Interfaces Answering Interfaces

If you want to include this spark into a configuration, you need to connect it having regard to the fact that it need to call another spark which offers IFlow<char*> interface functions. It is also necessary to remember that it will be called by sparks which require functions from the same interface. For example, a basic configuration might connect RebecaAIMLSpark, with ChatSpark as shown below:

Figure 8. Configuration example for RebeccaAIMLSpark

FliteVoiceSpark

Description

This spark represents a TTS (text-to-speech) module. It receives a text, and changes it to spoken words.

Use

Spark Connections
Asking Interfaces Answering Interfaces

Since it implements IConcurrent features (needed by the framework to start/stop threads), this means it uses the ThreadSpark to create a thread to synthesize the input text into an audio wave. It is not mandatory to connect IAsyncFatalErrorHandler interface, FIONA's framework will do this for you.

This spark receives text through IFlow<char*> interface and storages audio buffers into an audio queue which can be accesed through IAudioQueue interface methods.

Figure 9. Configuration example for FliteVoiceSpark

VoiceStartSpark

Description

This spark allows to launch an initial speech and manages the lips movement.

Use

Spark Connections
Asking Interfaces Answering Interfaces

This component must be connected with sparks which offer IFlow<char*>(more specifically with TTS's) and IFaceExpression functions. This spark provides IControlVoice features.

Since this spark modifies the avatar appearance and affects the character's video output, it uses IFrameEventPublisher to subscribe to character's render refresh. This way, this spark will be notified everytime a new video frame is rendered and will have the opportunity to change the avatar's face expression. It is not mandatory to connect this interface, FIONA's framework will do it for you.

You can see a configuration example for this kind of component at Figure 9.


Properties

  • Tts_Text: Using this property you can change the words your avatar will say.
Personal tools
Namespaces

Variants
Actions
Main
Documents
Management
Toolbox