mupuf.org // we are octopimupuf.org

The Gaze Tracking Project (GTP)

De­scrip­tion

This pro­ject has been writ­ten dur­ing my sec­ond year’s train­ing pe­riod in the Uni­ver­sity of Cen­tral Lan­cashire(UCLAN) (UK), work­ing for the re­search group called Child Com­puter In­ter­ac­tion (ChiCI). I was there for 3 months as an Eras­mus/ex­change stu­dent. Af­ter my work at the lab, I was foll­wo­ing two mod­ules called “Mi­cro­proces­sor-based sys­tem” and “Im­age and speech pro­cess­ing”.

The goal of the pro­ject was to de­velop a tool to analyse vir­tual-key­board lay­outs us­ing an al­ready-ex­ist­ing gaze track­ing de­vice. The vir­tual key­board lay­outs were al­ready writ­ten in Java.

With this goal in mind, I de­vel­oped a Qt-based C++ soft­ware that would re­ceive data from the gaze track­ing de­vice and from key­board/mouse in or­der to log this data into csv files. This was al­ready great but we needed some­thing more er­gonom­i­cal. This is why I added the screen­shot mode that would draw the data gath­ered into a sin­gle im­age. Here is the re­sult:

The green and the blue lines re­spec­tively rep­re­sent where the path taken by the gaze and the mouse. Some ad­di­tional in­for­ma­tion are also over­laid such as time­stamps and mouse events.

But, what if we want to analyse mov­ing con­tent like videos or web pages (scrolling) ? To ad­dress this prob­lem, I added a video-cap­ture mode. This is the only fea­ture that is not portable and that is lim­ited to Win­dows. The way it is im­ple­mented is by pe­ri­od­i­cally mak­ing a snap­shot of the screen at a de­fined rate, draw the data on top of it and save it in a video en­coded in divx or mjpeg. Here is how the re­sult looks like:

As I didn’t want every re­search ap­pli­ca­tion to be writ­ten in C++/Qt, I had to cre­ate bind­ings for other lan­guages. I de­cided to use sock­ets to com­mu­ni­cate with the gtp server and then wrote bind­ings in C/C++, Vi­sual Basic.​net, C# and … Java. Us­ing these bind­ings, It was very easy for every pro­gram­mer to con­trol the gtp server but also to get gaze track­ing in­for­ma­tion. The added ben­e­fit of this sys­tem is that mul­ti­ple ap­pli­ca­tions can ac­cess the eye track­ing in­for­ma­tion at the same time.

My first demon­stra­tion for these bind­ings was what I was ac­tu­ally asked to do, eg. analysing vir­tual key­boards in Java. Here are the re­sults:

My sec­ond demon­stra­tion is a game I made the night be­fore the pre­sen­ta­tion of my pro­ject. This game uses the gaze to aim at fly­ing su­per heros be­fore shoot­ing them down by press­ing the space key.

The above video is a screen­cast of the game made by gtp. Thus, the player doesn’t see the green line, he only sees the red dot. The game worked quite well and I was able to aim at any­thing quickly. How­ever, in this video, the com­puter had too many things to com­pute at the same time so it be­came laggy and un­re­spon­sive. This ex­plains why I was miss­ing so many su­per heros. Just to clar­ify things, I was us­ing a very cheap lap­top I had bought 1 year be­fore.

Comments

Comments