Tag Archives: virtual world

Cornell Does It Again: Sonar+AI for eye-tracking

Tech, , , , , , ,

If you remember:

Now, Cornell released another paper on GazeTrak, which uses sonar acoustics with AI to track eye movements.

Our system only needs one speaker and four microphones attached to each side of the glasses. These acoustic sensors capture the formations of the eyeballs and the surrounding areas by emitting encoded inaudible sound towards eyeballs and receiving the reflected signals. These reflected signals are further processed to calculate the echo profiles, which are fed to a customized deep learning pipeline to continuously infer the gaze position. In a user study with 20 participants, GazeTrak achieves an accuracy of 3.6° within the same remounting session and 4.9° across different sessions with a refreshing rate of 83.3 Hz and a power signature of 287.9 mW.

Major drawback, however, as summarized by Mixed News:

Because the shape of the eyeball differs from person to person, the AI model used by GazeTrak has to be trained separately for each user. To commercialize the eye-tracking sonar, enough data would have to be collected to create a universal model.

But still though, Cornell has now come out with research touting sonar+AI as a replacement for camera sensors (visible and infrared) for body, face and now eye tracking. This increases the possibilities of VR and AR which is smaller in size, more efficient in energy and more responsive to privacy. I’m excited for this work.

Video of GazeTrak (eye-tracking)

Video of PoseSonic (upper-body tracking)

Video of EchoSpeech (face-tracking)

Sentience + ARToolKit = AR done right

Uncategorized, , , , ,

sentience is a software library that allows for robotic stereo vision using stereo webcams (like Minoru 3D, which is of British origin despite its Japanese name), and is written in C#. Meanwhile, ARToolKit is one of the most widely-employed augmented reality software frameworks (and is also, like sentience, FOSS). 

Continue reading Sentience + ARToolKit = AR done right →

On “Summer Wars”

Uncategorized, ,

I can say that Summer Wars, Mamoru Hosoda’s 2009 film, was decent. I can’t say that it presented anything startlingly new (other than the family angle and the consistently-amazing animation style that I can expect from few other directors besides Mr. Hosoda) because, after finishing the film, I (and a few others: here, here and here) realized that the plot for this film was a latter day revisitation (not necessarily a rehash, but a timely revision) of his 2000 Digimon Adventure film, Children’s War Game.

I can name a few differences in the technological/industrial aspect between Children’s War Game and Summer Wars:

  • Children’s is much more replete with A.I. vs. A.I. (in the form of the Digimon characters), while a form of A.I. is mostly posited as a world-eating antagonist vs. the whole of humanity.
  • Unlike the less-likely scenario presented in Children’s at the time of its release, Summer was more reflective of the very-likely integration of pervasive Internet-based social network accounts with more superfluous user avatars and persistent virtual environments (like, say, integrating Facebook or Myspace with IMVU or Second Life)
  • Because of the lack of a large A.I. population in Summer‘s virtual world (known as "OZ"), the story line is driven more by the contributions, conflicts, hopes and fears of the human participants than in Children’s.

But if Summer Wars is a more timely revision of Children’s War Game, then it is (IMO) logical that a sequel to Summer would be a more timely revision of his last Digimon Adventure film, Diaboromon Strikes Back. This time, the fight (and whatever such a fight would be over) would take place in the real world and after a few years post-Summer Wars, just as in Diaboromon; the key ingredient of such a sequel would be augmented reality (and AR/VR glasses), just as in the 2006 series Dennou Coil.

I just hope that the plot for Summer Wars II isn’t as lightweight terribly composed as Diaboromon. It might need more A.I. than Summer Wars, but I hope that it doesn’t rely as much on its predecessor in order to wow and entrance theater-goers.

3D windows

Uncategorized, ,

I like how Open Cobalt, another virtual world project, allows for the linking of two in-world spaces using 3D window frames, which can swivel from side-to-side, provide live views of whatever’s going on in the opposite end of the window and allow for one to walk an avatar through the window into the other in-world space. It reminds me of what I once told someone when I was logged into SL (I think it had more to do with streaming video than with being able to walk between virtual world areas, though). Example of this method begins below at 0:38.

I wonder, however, if this method of hyperlinking can be embedded into the virtual world’s persistent objects, such as buildings? In fact, just like how originally non-electronic text was eventually imported to the Web and enhanced with context-specific links, the same could be done with 3D copies of real world (and virtual) objects into the virtual world, namely appending links into the objects which link them to the views of other virtual objects; a wiki approach could be used in this process, whereby "redlinks" (or in this case, null spaces) can be linked within these virtual objects in order to allow users to create the new objects in the null spaces.

The problem of appending links to 3D objects is the way that these links should look embedded within the objects without any visually disturbances. If frames are going to be one of the main presentational means of linking one object to another, then it would have to maintain a color or design scheme that is distinguishable from the rest of the object but not too distinguishable. It would also need a means of navigation that makes frames easily responsive to the mouse pointer, no matter how visually skewed the linked portion looks from the rest of the object.

"Hyperframes" could also be a major boost to the usage of "tabbed traveling" or "tabbed teleportation" in virtual worlds, since they would be middle-clickable to allow for a new tab of a new space to be visited non-linearly.

A better explanation of “tabbed browsing in Second Life”

Uncategorized, ,

 OK, so SL has three key features that I want to utilize in this proposal: SLURLs, landmarks and teleportation. 

My idea is that an SL user can embed a SLURL in a specific location in-world, perhaps as a script, and the next user can left-click on the script to select one of two pertinent actions: "teleport" or "add as landmark":

  • If "add as landmark" is selected, then it will open either as a separate HUD window menu which displays the landmark inventory and various means of landmark management, or, if the option is middle-clicked, as a whole new tab containing the same.
  • If "teleport" is middle-clicked (or the script itself can be middleclicked), then the teleportation load can take place in another new tab while the user is still in the previously-open location.
  • The kicker: once the new tab is finished loading the view of its own location, the user can then select the new tab and immediately move the avatar’s presence to the new tab’s location, and then move freely back to the older tab with the avatar’s presence following the user to the older tab’s location. You move in, the avatar moves in; you move out, the avatar moves out, and so on.
  • The avatar’s behavior across tabs can be optionally switched off if the user simply wants to view another location’s goings-on without the user’s avatar getting involved or, worse, stuck in the middle of such events.

I think this navigation method can work for the following reasons:

  • Instead of saying "I’ll be going now, gotta head to Luskwood", it can instead be "gimme a sec, I’m looking in Luskwood as we speak….oh, nevermind, I’m back here at Badwolf’s…nah, its dead there, now I’m up in this space station filled with alien avatars".
  • Flying from one place to another can be quicker, farther, multitasked and much more balanced.
  • Going from one grid to another and back (or maybe three grids or more in separate tabs) can also be a matter of seconds.
  • You wouldn’t have to be automatically logged out of Second Life if one grid is closing for repairs.
  • Avoiding griefer attacks while watching (peeking?) the disasters unfold in another tab will either be hilarious or sobering from a distance.

So yes, it is a proposal to make virtual world clients more like modern web browsers without necessarily entailing the embedding of the web inside the virtual world (although current web-browsing capabilities of the SL client may benefit from the above-described virtual world tabbed browsing); it captures the non-linearity of the tabbed navigational experience and fits it upon the virtual world that is much too linear (think "IE6") to navigate fully and smoothly within a multitasking framework.

Tabbed browsing in Second Life

Uncategorized, , ,

I think that tabbed browsing, which has worked wonders for web browsing (and may work similar wonders in file navigation), can be a feasible, necessary addition to the desktop clients of virtual worlds like Second Life. Rather than the single-tasking that forces a user to view only one persistent virtual world location at a time within the client, tabbed navigation of the virtual world can allow the user, through the user’s avatar, to glide effortlessly between the tabs which contain open locations; the indication of presence can also follow the user between the tabs, allowing other users to know when the user’s avatar is in the tab or not.

This sort of browsing, which can also be applied to virtual earth clients such as Google Earth, can also allow the user to open other locations with a simple middle-click of a teleport link, which will then open in a separate, adjacent tab. This can also be deemed as opening separate instances within the client.

Tabbed browsing of virtual worlds may result in greater capabilities for user multitasking, and can allow for users to WYSIWYGially drag copies of virtual objects from one location in a tab to another location in a separate tab, alot like text links in Firefox.

Ning

Uncategorized, ,

Has anyone tried Ning yet?

I started using it around Monday/Tuesday night. It’s a “social network hosting service” (SNHS), where one can create his or her own little MySpace for free. It’s not that flexible in its social network management tools (feels alot like creating your first Geocities or Tripod website), but I haven’t heard of any competitors to Ning yet, which may be why Ning doesn’t feel all that professional.

However, while it is a bit late in the social networking boom to make a prediction, I think that Ning, as probably the first SNHS, could start a trend will redefine how we’ve long conceived of social networks as only being stand-alone services that can only appeal to the most financially-lucrative demographic.

Also, since the most generalized social networks tend to provide the ability to create interest-specfifc e-groups for free from within the network, what does this mean for the e-group if the social networks can now be created for free and for specfifc interests within the SNHS? Does this mean that the e-group become irrelevant, or does this mean that e-groups are now free to explore more specfic, more exclusive interests than those which would be allowed in a general SN like MySpace or Facebook?

Finally, what if Ning could be replicated in the 3d virtual world business? What if virtual world social networks can be created for free from within a virtual world network hosting service (VWNHS)? Virtual worlds like Second Life and IMVU already possess a few (but not most) of the trappings of a social network like MySpace and Facebook, namely, in-world e-groups, chat and messages.

Unfortunately, such may not occur for as long as development of virtual world social networks and models (a la Second Life or Kaneva) remains as prohibitevly expensive as it is right now. Lack of adoption of cross-platform standardization also results in unnecessary reinvention of wheels in this regard.

Otherwise, this one furry-oriented virtual world that was started in 2004 may have enjoyed some of the benefits and usage that has gone to the larger virtual worlds rather than shut down due to lack of attention and input.