jNabServer & Nabaztag/tag documentation

Author: Juha-Pekka Rajaniemi <juha-pekka.rajaniemi (at) cs.uta.fi>, July 2007

Click here to see this page in printable form

• General information about jNabServer
• Nabaztag/tag
  • Hardware
  • Protocol
  • Request
  • Packets
  • Blocks
    • Ping interval block
    • Ambient block
    • Reboot block
    • Message block
  • Choreographies
• jNabServer
  • Configuring nabaztag
  • Architecture
  • Plugin system
  • Class descriptions and building simple custom jNabServer
  • Creating packets in jNabServer
  • Using and building choreography files
  • Example plugin
  • Build in XML api
  • Logger
• References

General information about jNabServer

JNabServer is an open-source server software which handles communication with Nabaztag/tags, Wi-Fi enabled robotic rabbits that can be used as physical user-agents in human-computer interaction ( Nabaztag in Wikipedia).

JNabServer is written in j2se 6.0 (Java Standard edition) so it is platform independent. Main idea is that people can develop their applications and ideas more freely on jNabServer platform by removing the need for connection to Violet's services as these services has no possibility of operating fast enough from all possible locations on earth. This can be seen in improvement from tens of minutes on Violet servers to milliseconds on local JNabServer at best cases. Also jNabServer opens a lot of possibilities as there is no need to have connection to the internet all the time.

Nabaztag/tag device can handle various forms of interaction, from voice to touch (button press), and from RFID 'sniffing' to ear movements. It can response by moving its ears, by displaying or changing the color of its four LED lights. It can also play sounds which can be music, synthesized speech or other voices.

jNabServer has been produced in University of Tampere at the department of Computer Science and in speech-based and pervasive interaction group. From this point onwards in this manual I refer Nabaztag/tag as Nabaztag.

Nabaztag/tag

Nabaztag recieves code on its every boot. We call it the byte code and it is used to operate all internal actions what Nabaztag does. At the time of writing there is no tools released to modify these operations. We are explaining in detail how byte code operates inside Nabaztag. All this information is based on testing and there is still a lot of unknown issues. We are mostly targetting this documentation on communications in byte level.

If you are not interested about developing jNabServer forward you can skip straight to jNabServer part of documentation.

Hardware

Nabaztag/tag has five leds, four in front and one in bottom. Two moving ears which positions are tracked with optical sensors, a microphone, speaker and rfid-reader.

RFID-reader reads tags of type ISO 14443 B. These tags operate on 13,56mhz and it has 4096-bit EEPROM. Nabaztag can read ID's of these tags.

Sounds recorded from microphone are encoded with ADPCM before they are sent to server.

Byte code

When nabaztag connects to server for the first time after it has been powered on or rebooted it calls server to have send byte code to it. Byte code contains everything what nabaztag needs to operate. Logic what to do when user input comes to device and how to operate data incoming from the server.

Protocol

On basic network configuration Nabaztag is working as a HTTP network client which automatically calls the server with fixed interval also called ping call. Interval is variable and it can be changed by sending PingIntervalBlock in the packet. Nabaztag can request following five files from server: bc.jsp, locate.jsp, p4.jsp, record.jsp and rfid.jsp. These calls are hardcoded in to the byte code.

Requests

Nabaztag calls following files from server spontaneously or after received user input.

In these requests we are using server IP 192.168.0.2 as an example.

GET http://192.168.0.2/vl/bc.jsp

After booting Nabaztag starts calling bc.jsp until it receives and processes the correct byte code from server that has been set-up in configuration.

GET http://192.168.0.2/vl/locate.jsp

When Nabaztag has successfully loaded the byte code it first calls locate.jsp which has to tell two IP's for Nabaztag. First IP is pingserver from where p4.jsp is called.

After newline next value is broadcast server. Nabaztag will replace all "broadcast" strings from the Message Block with this value when Nabaztag recieves message.

GET http://192.168.0.2/vl/p4.jsp

p4.jsp is basic ping call from server. Ping purpose is to tell server if any input has come from user or tell if last message sent from server is successfully played. Information about these events are sent in

Parameter sd can be numeric value in range from 0 to 8 with following meanings

POST http://192.168.0.2/vl/record.jsp

When user keeps pressing the button on Nabaztag's head it starts voice recording. After user has released button it sends record.jsp call to the server with encoded audio data in its POST part of message.

GET http://192.168.0.2/vl/rfid.jsp

When Nabaztag has read the correct type of RFID tag it sends rfid.jsp call to server. It passes serial of sending Nabaztag, version of bytecode and ID of the RFID. If Nabaztag reads ID wrong or ID is kept in front of Nabaztag so that it repeatetly reads tag then Nabaztag sends zero ID (sixteen zeros in a row).

Packets

Server responses to Nabaztag on demand when it request a ping command. The answer packets are custom byte arrays. These byte arrays start with byte 7F and end with FF and 0A bytes. In between there can be any amount of blocks. Blocks consists of three parts. First there is a byte telling the type of the block. Known block types are 03 , 04 , 09 and 0A. Next three bytes are reserved for telling the length of data in the blocks. In the length bytes last byte is the most significant byte and bytes are unsigned. After the length bytes there is the data for the block.

Example of a packet containing two blocks. First block is a type 03 which is coloured orange, has length of one and the value of its only byte is five. Next block is a type 09 which is coloured in green and it has a length of zero.

Blocks

Ping interval block (03)

Ping interval block length is always one. When this block is sent to Nabaztag it changes the ping interval to the given time in seconds.

Ambient block (04)

Ambient block sets light flashing patterns when there is no other messages playing. These lights can be interrupted by making the Nabaztag to do some other operation like playing a sound. Ambient block is between 22 to 24 bytes long always. Lenght of block tells how nose led of Nabaztag is blinked. When length is 22 bytes Nabaztag will not blink nose led, with 23 bytes it blinks once every second and with length of 24 bytes Nabaztag double blinks its nose led every second. Byte 21 has right ear value what to change to when block start playing and byte 22 has left ear value.

Data part of block always starts with bytes 7F FF FF FF. Values of bytes from 5 to 20 are unknown. These can be filled with zeroes.

Reboot block (09)

Reboot block is always built from following four bytes: 09 00 00 00. This reboots Nabaztag so it will call bc.jsp again for the byte code.

Message block (0A)

Message block is encrypted ASCII string message sent to Nabaztag. The encryption algorithm is following: C[i] = (B[i] - 0x2F) * (1 + 2 * C[i-1]) where C is encrypted byte array, B is decrypted byte array and i is range from zero to length of byte data. As this algorithm gives encrypted value of next character depending on byte before current there is need for byte C[-1] which has constant value 35.

Message block contains commands that are separated from each other with newline character. Known commands are:

ID ID of the message. Format is ID x. This way messages can have id's. Purpose is unknown.

CL set color command. Format is CL 0xAABBCCDDh where AA is number of led from 00 to 04, BB, CC and DD are color values for red, green and blue.

PL set palette command. Format is PL 0X where X is from 0 to 7. This sets palette color for choreography files.

CH choreography command. Format is CH <url>. This command gets choreography file from given url and plays it.

MU play sound command. Format is MU <url>. Plays sound from given url.

MC synonym for MU command.

ST play stream command. Format is ST <url>. Plays shoutcast streaming radio from given URL.

MW wait command. Doesn't take any parameters. This command forces Nabaztag to finish given commands (e.g. playing sound) before going to the next one.

Choreographies

Choreographies are files that can be played by calling them from message blocks with CH command. Choreographies contains led control and ear control. Each of the five leds can be controlled to have RGB values from 0-255. Ears can be moved to certain positions from 0x0 to 0x12 forwards or backwards. Also there is a possibility to move an ear forwards a certain step count. These files are made from three parts. First there is a header which contains the length of bytes and in middle there is the data and at the end four zero bytes. Here we have example of simple choreography file where the data part is in orange color. This data contains one command that tells the center led to change into red.

Data is filled with smaller command byte arrays. Choreography files are timed and this reflects also to the command bytes. First byte of command bytes tells how long to wait before playing a new command from the beginning of last command. Second byte tells the type of command. After these two bytes there can be any amount of bytes relative to the command. In this example we have one command where the first byte on white background tells that the command is played instantly when the file is started to play. Next on blue background we have the type of command byte and in the end with orange color we have the data of command.

List of possible commands:

Tempo command sets how fast commands are played. Type byte is 1. Data part is build from one byte that tells tempo. At default this value is 10. Below is example that sets tempo to 15.

Led command sets color of given led to RGB color. Type byte is 7 and data is build following way: first byte tells led which color to change from 0 to 4, next led is red value followed by green and blue bytes. Lastly there is two zero bytes that are being ignored. Here in below we have example which sets center led red value to 255, green to 255 and blue to zero.

Ear command type byte is 8 and it consist of three data bytes. First one is the ear that can be 0 (right) or 1 (left). Next is the ear position between 0x0 and 0x12 (0-18). And third byte is the direction of rotation, 0 (forward) or 1 (backward). In the example, right ear is rotated 'forward up' position by rotating the ear backwards.

Play random MIDI command is type 0x10 command. Plays random mini note. No data part.

Ear step command is type 0x11 command. Data is made from two bytes: first byte selects the ear 0 (right) or 1 (left) and next byte tells how many steps to go forwards. In the example we move left ear ten steps forwards.

Addition to these commands there are "ifne" command that is some sort of loop command and "attend" command that synchronizes choreography to sound. There two commands has not been tested and their functionality isn't sure yet.

jNabServer

JNabServer is Java based server software that is designed to handle operations which are introduced in the previous chapter.

jNabServer is build on top of very light weight HTTP-server. This means that all functions that jNabServer does are communicated through HTTP-protocol. These functions include communication with Nabaztag and XML API. HTTP-protocol was chosen because Nabaztag uses it. There was no reason to start building our own communication layer. This means that all data coming into jNabServer can be either in parameters of requested URL or in POST data of message. XML API uses slightly inconvenient way of using .xml ending files where you have to pass POST data.

What this means is that jNabServer is a framework. Actual functionality wanted for Nabaztag is done in form of plugins. XML API provides tools for making raw data request straight to the Nabaztag without interfering with the plugin. At all time a Nabaztag can have only one active plugin, and all commands coming from Nabaztag are passed to this plugin.

Usually instance of plugin's parent, the Nabaztag -class is passed to the plugin as a parameter in constructor. This gives the plugin an ability to change itself to another plugin inside the Nabaztag -class. This method is called state architecture.

Supported requests

jNabServer can handle any kind of request, but mostly requests are five type of calls: bc.jsp, locate.jsp, p4.jsp, record.jsp and rfid.jsp. These requests must have string "/vl/" prepeding to make server understand them. Also server can pass local files by starting request with string "/local/". For example if you would like to call c:\mysound.mp3 file from server you request "/local/c:\mysound.mp3" file from server.

Architecture

Cycle of request

As Nabaztag uses HTTP-protocol to communicate with jNabServer it means that network architecture is relative simple. Cycle of one HTTP request starts from Nabaztag. Nabaztag asks for one of the possible files from jNabServer. These request are more detailed explained in requests part of documentation. In jNabServer MicroServer socket server receives this call and passes it to new Worker class. Worker class parses request. If request is bootcode or locate request it creates response and sends it to Nabaztag. After sending data connection is closed and Worker terminated. If request was any other type than bootcode or locate it is passed to Nabaztag class. If Nabaztag has any pending forced packets it sends one of them and Worker is terminated. If Nabaztag class doesn't have any pending forced packets it parses request and passes call to correct method in plugin. After correct method has been called Nabaztag tells plugin to send data.

Class descriptions and building simple custom jNabServer

MicroServer(fi.uta.cs.nabaztag.server), purpose of microserver is to receive incoming socket connections from Nabaztag and pass them to Worker threads. To setup custom server one needs to create instance of MicroServer and start it. MicroServer takes pingServer and broadcastServer parameters which usually are local ip address of server where jNabServer is running. In below is example of most basic implementation of jNabServer. Only problem in it is that it doesn't do anything except that it sends bootcode to nabaztag.

import fi.uta.cs.nabaztag.server.MicroServer;

class myNabaztagServer {
	public static void main (String [] args) {
		MicroServer server = new MicroServer("192.168.1.100", "192.168.1.100");
		server.start();
	}
}

Burrows(fi.uta.cs.nabaztag) is container for all Nabaztags that jNabServer knows. Nabaztags can be added to server specifically by their serial or just let them connect server when they are added automatically to known Nabaztags. Serial of Nabaztag can be found from bottom of device.

Nabaztag(fi.uta.cs.nabaztag). For every Nabaztag server knows there is class Nabaztag. Class contains most basic information about Nabaztag: its serial and custom name. Nabaztag class takes care of passing requests to plugin that is set for it or if there is forced packets this class will send those packets and ignore incoming request.

NabaztagPlugin(fi.uta.cs.nabaztag.plugins) is abstract class from which all plugins should be inherited. These classes can be passed to Nabaztag class to make Nabaztag operate some custom actions.

Now with these three more classes we can expand myNabaztagServer example. In this example server will be loading example echo plugin that is in here in documentation. Now myNabaztagServer has functionality for Nabaztag with serial 000000000001.

import fi.uta.cs.nabaztag.Burrows;
import fi.uta.cs.nabaztag.Nabaztag;
import fi.uta.cs.nabaztag.server.MicroServer;

class myNabaztagServer {
	public static void main (String [] args) {

		Nabaztag myNab = new Nabaztag("000000000001");
		myNab.setPlugin(new EchoPlugin(myNab));
		Burrows.addNabaztag(myNab);

		MicroServer server = new MicroServer("192.168.1.100", "192.168.1.100");
		server.start();
	}
}

ChoreographyLibrary(fi.uta.cs.nabaztag) is container for choreography files. In message block when one calls CH broadcast/chorlibrary/chorname this call will be passed to ChoreographyLibrary to find "chorname" from it and then to output to HTTP response.

Choreography(fi.uta.cs.nabaztag) classes are used to create choreographies. Once a choreography has been build with this class it can be saved to the ChoreographyLibrary.

Packet(fi.uta.cs.nabaztag.response) is a container for blocks that are sent to Nabaztag. Packet has two uses. First it can change given blocks to binary array with generatePacket method. Another use for Packet is to parse binary packet into message blocks. This can be done by feeding binary array in the constructor.

Block(fi.uta.cs.nabaztag.response) is the parent class of classes AmbientBlock, MessageBlock, PingIntervalBlock and RebootBlock. Block contains basic methods to get the type and size of a block.

AmbientBlock(fi.uta.cs.nabaztag.response) can set behaviour for Nabaztag when it is not receiving any commands. Currently it can change blinking speed of "nose"-led.

MessageBlock(fi.uta.cs.nabaztag.response) contains string data. Detailed information about the string data is in here. MessageBlock can encode and decode message block data to readable format. Class has encode() method that takes string as parameter which will be encoded. getContent() method in other hand is used to decode given data.

PingIntervalBlock(fi.uta.cs.nabaztag.response) is block that can change ping interval time of Nabaztag. Pass time in seconds to the constructor.

RebootBlock(fi.uta.cs.nabaztag.response). Sending reboot block to Nabaztag causes it to reboot and ask bootcode again.

Request(fi.uta.cs.nabaztag.request) is a class that parses incoming requests from Nabaztag. It also receives possible POST data when it is sent in record.jsp request.

Creating packets in jNabServer

All that is needed for classes to make packets are in fi.uta.cs.nabaztag.server.response package. Class Packet is a container for all blocks that are inherited from class Block. To build a packet you first need to create an instance of it and then use addBlock method to add blocks to the packet. After all blocks has been added to the packet you can either send it to any nabaztag with nabaztag.forceNextPacket(Packet) or create byte array with Packet.generatePacket(). In following example we create a packet with few blocks. This example can be used only in plugins. In other places you can replace the last line with line nabaztag.forceNextPacket(p);.

//new packet instance
Packet p = new Packet();
//setting pinginterval to five seconds
p.addBlock(new PingIntervalBlock(5));
//generate byte array of packet
int [] byteArray = p.generatePacket();

Using and building choreography files

There is two possible ways of using choreography files. You can either directly call a local .chor file to play or create choreography classes that can be requested from choreogoraphyLibrary. To request file named "myChor" from choreographyLibrary add following command to messageblock: CH broadcast/chorlibrary/myChor\n. If you are using SimpleMessageBlock you can call method playChoreographyFromLibraray(String name).

Plugin system

Nabaztag has only one plugin active and all commands coming from Nabaztag is passed to this plugin. If plugin knows which Nabaztag it belongs to (e.g. passing Nabaztag as parameter to plugin) then it can change the plugin to any other plugin. This method gives possibility to build a state machine.

Example plugin

In this example plugin the purpose is to create echo. There are three things it does. At first it records echo.wav file in onRecord method. At second it plays echo.wav on the next call that goes to Nabaztag with forceSendPacket method and the third one is that pressing the button changes the ping interval into two seconds. Changing ping time and playing sounds differ so that in playing a sound, Nabaztag is told to force play sound when next ping comes from it. This prevents Nabaztag to call the plugin at all and it just plays the sound. In onClick method in other hand returns the response to Nabaztag on the same ping call.

package fi.uta.cs.nabaztag.plugin;

import java.io.FileOutputStream;
import java.io.IOException;
import java.util.Arrays;

import fi.uta.cs.nabaztag.Nabaztag;
import fi.uta.cs.nabaztag.server.request.Request;
import fi.uta.cs.nabaztag.server.response.Packet;
import fi.uta.cs.nabaztag.server.response.PingIntervalBlock;
import fi.uta.cs.nabaztag.server.response.SimpleMessageBlock;


/**
 * Purpose of this plugin is to make Nabaztag to say last
 * thing it has recorded. Plugins need to be inherited from
 * NabaztagPlugin class.
 *
 * @author Juha-Pekka Rajaniemi
 */
public class EchoPlugin extends NabaztagPlugin {

	 //is the echo send?
	 private boolean sendEcho;

	 //give current nabaztag in constructor. This
	 //give possibility to change plugins on the fly
	 //optional.
	 public EchoPlugin(Nabaztag n) {
			 super(n);
			 sendEcho = false;
	 }

	 
		/**
		 * When data has been sent to Nabaztag
		 */
	 @Override
	 public void dataSent() {
			 //is the echo sent yet?
			 if (sendEcho) {
					 //if not then we create messageblock with id 600
					 SimpleMessageBlock smb = new SimpleMessageBlock(600);
					 //tell jNabServer to play local file named echo.wav
					 smb.playLocalSound("echo.wav");
					 //tell jNabServer to wait for echo.wav to play
					 smb.waitForCommands();
					 //required method for SimpleMessageBlock to generate block
					 smb.finish();

					 //then create packet which will be sent to Nabaztag
					 Packet p = new Packet();
					 //add created simple message block to packet
					 p.addBlock(smb);
					 //add another block which tells nabaztag to ping
					 //jNabServer every second
					 p.addBlock(new PingIntervalBlock(1));

					 //tell plugin owner add packet to forced queue
					 bunny.forceNextPacket(p);

					 //echo has been sent
					 sendEcho = false;
			 }


			 dataToSend = null;
	 }

	 /** What to do when user has double clicked */
	 @Override
	 public void onDoubleClick() {}

	 /** When ears has been moved */
	 @Override
	 public void onEarsMove(int rightEar, int leftEar) {}

	 /** When succesfully all messages has been sent */
	 @Override
	 public void onEndOfMessage() {}

	 //variable to save last income data. This is because
	 //Nabaztag keeps sending data for several times...
	 private int[] lastData;

	 
		/**
		 * When audio has been recorded parameter data has WAV raw
		 * data in array.
		 */
	 @Override
	 public void onRecord(int[] data) {
			 //don't do anything if data has been recieved already
			 if (lastData != null && Arrays.equals(lastData, data)) {
					 return;
			 }

			 //write data to echo.wav
			 try {
					 FileOutputStream fos = new FileOutputStream("echo.wav");
					 for (int i = 0; i < data.length; i++) {
							 fos.write(data[i]);
					 }

					 fos.close();

					 lastData = data;

					 sendEcho = true;
			 }
			 catch (IOException ioe) {}
	 }

	 
		/**
		 * When Nabaztag rfid this method is called
		 */
	 @Override
	 public void onRfid(String rfid) {}

	 
		/**
		 * When button on Nabaztag head is clicked. Example
		 * how to send message message to nabaztag during same
		 * call.
		 */
	 @Override
	 public void onSingleClick() {
			 //create ping interval block
			 PingIntervalBlock pib = new PingIntervalBlock(2);
			 //add it to packet
			 Packet p = new Packet();
			 //add block to packet
			 p.addBlock(pib);

			 //nabaztag send dataToSend byte array to Nabaztag
			 //when this method has been processed. Packet has
			 //method generatePacket() which returns byte array
			 //version of packet.
			 dataToSend = p.generatePacket();
	 }

	 @Override
	 public void onSingleClickWhilePlayingSound() {}

	 @Override
	 public void onTimeOut() {}

	 
		/**
		 * When nothing matches then this method is run. Nabaztag's request
		 * is given as parameter
		 */
	 @Override
	 public void unknownRequest(Request r) {}

}

Build in XML API

In XML API client sends data in POST format to server. Server processes and responses in XML. Here is a list of possible calls to jNabServer

Requests

/xml/rawrequest.xml By passing data rawrequest.xml one can send packets and blocks straight from a client to Nabaztag. Nabaztag then processes given packets as forced packets and sends them in next call.

<!DOCTYPE RAWREQUST [

<!ELEMENT rawrequest (packet+) >
<!ELEMENT packet (ambientblock, pingintervalblock, messageblock, rebootblock) >
<!ELEMENT ambientblock ()>
<!ELEMENT pingintervalblock (#PCDATA)>
<!ELEMENT messageblock (#PCDATA)>
<!ELEMENT rebootblock (#PCDATA)>

<!ATTLIST packet serial CDATA #REQUIRED>
]>

/xml/jnabcommand.xml. This command can change the plugin of certain a Nabaztag or run methods on some plugin. This call can set the given Nabaztag's plugin to the given class. Currently setPlugin passes always the plugin's owner Nabaztag -class in the constructor. Another command is a method. It calls given methods from Nabaztag's plugin. No parameters can be given yet.

<!DOCTYPE JNABCOMMAND [

<!ELEMENT jnabcommand (setPlugin+, method+) >
<!ELEMENT setPlugin (#PCDATA) >
<!ELEMENT method (#PCDATA)>

<!ATTLIST jnabcommand serial CDATA #REQUIRED>
<!ATTLIST method name CDATA #REQUIRED>
]>

Responses

Both rawrequest and jnabcommand reponses are same kind of messages. In a case of success the response is following:

<response>
	<success>1</success>
</response>

and when there is an error this kind of XML is given:

<response>
	<error>
		<message>Error message from API</message>
	</error>
</response>

/xml/jnabserverinfo.xml is slightly different case compared to rawrequest and jnabcommand. This XML does not require client to send any data in POST format. It only responses with information about the server.

<!DOCTYPE JNABSERVERINFO [

<!ELEMENT jNabServerInfo (registeredNabaztags+) >
<!ELEMENT registeredNabaztags (nabaztag*) >
<!ELEMENT nabaztag (plugin+, serial+)>
<!ELEMENT plugin (#PCDATA)>
<!ELEMENT serial (#PCDATA)>
]>

Logger

Logger is a helper class built specially for the jNabServer. By default it just prints to the console all messages that are sent to it. To prevent the console messages to appear, Logger needs to have at least one listener set to it. This is done by using addListener() method to add a class as a listener. Listening class must implement fi.uta.cs.nabaztag.utils.LoggerListener interface.

Logger has one important method: addLog(). As a first parameter it takes the log message and as a second parameter an optional message level integer. By default all exceptions are at level 1 and all messages are at level 3.

References

No references