Could this be the future of smartphones/household computers? HoloDesk is a novel interactive system combining an optical see through display and Kinect camera to create the illusion that users are directly interacting with 3D graphics. A virtual image of a 3D scene is rendered through a half silvered mirror and spatially aligned with the real-world for the viewer. Users easily reach into an interaction volume displaying the virtual image. This allows the user to literally get their hands into the virtual display. A novel real-time algorithm for representing hands and other physical objects, which are sensed by the Kinect inside this volume, allows physically realistic interaction between real and virtual 3D objects.