Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Introducing Tangible Holograms for Data Physicalisation and Big Data Exploration

1 865 vues

Publié le

Seminar talk delivered in the Global Information Systems lab at ETH Zurich.

Publié dans : Sciences
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

Introducing Tangible Holograms for Data Physicalisation and Big Data Exploration

  1. 1. 2 December 2005 Introducing Tangible Holograms for Data Physicalisation and Big Data Exploration Prof. Beat Signer http://www.beatsigner.com Web & Information Systems Engineering Lab Department of Computer Science Vrije Universiteit Brussel WEB & INFORMATION SYSTEMS ENGINEERING
  2. 2. 01001000100 0101001100010010100010010 0100100010010001000100001001000 100010001100001001000110010101010110 1100010010100010010010010100010101011001 101001100001100 101010100 00101000100 110101010101 00100 01010101000 1001 1111111111 111 0000000010 0 11111111 0101001011 010101010 111010101 001010010110 Fluid Cross-Media Information Spaces
  3. 3. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 3September 23, 2017 Prof. Dr. Beat Signer Interactive Paper, Cross-Media Information Architectures Audrey Sanctorum User-defined Cross-Device and Cross-Media Interaction Reinout Roels MindXpres: Extensible Content- driven Presentation Tool CISA Human-Machine & Human-Information Interaction Information Systems & Management Information Visualisation & Navigation WEB & INFORMATION SYSTEMS ENGINEERING CROSS-MEDIA INFORMATION SPACES AND ARCHITECTURES (CISA) Payam Ebrahimi Data Physicalisation
  4. 4. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 4September 23, 2017 Dr. Ahmed A.O. Tayeh Open Cross-Media Authoring, Fluid Document Formats WEB & INFORMATION SYSTEMS ENGINEERING CISA Human-Machine & Human-Information Interaction Information Systems & Management Information Visualisation & Navigation CROSS-MEDIA INFORMATION SPACES AND ARCHITECTURES (CISA) Kushal Soni Smart Environments, Implicit Human-Computer Interaction Jan Maushagen Learning Analytics, Adaptive Persuasive ICT Tools Sandra Trullemans Personal Cross-Media Information Management
  5. 5. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 5September 23, 2017 WEB & INFORMATION SYSTEMS ENGINEERING CISA Human-Machine & Human-Information Interaction Information Systems & Management Information Visualisation & Navigation CROSS-MEDIA INFORMATION SPACES AND ARCHITECTURES (CISA) Lars Van Holsbeeke Smart Environments, Implicit Human-Computer Interaction
  6. 6. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 6September 23, 2017 Cross-Media Information Spaces and Architectures Information Systems & Management Information Visualisation & Navigation Information Visualisation nd NavigationHuman-Machine & Human-Information Interaction CISA RSL [4] MindXpres [7] OC2 [5] associate file system [2] data-driven storytelling [10] XIMA [12] cross-device interaction [8] iServer/iPaper [3] ViDaX [15] data physicalisation [30] map-based interaction [17] non-linear storytelling [25] PaperProof [27]iGesture [21] Query by Sketch [28] PaperSketch [29] Context Modelling Toolkit [9] Midas [19] SpeeG2 [22] PaperPoint [24] Print-n-Link [16] digital libraries [11] Mudra [20] mixed reality [26] ArtVis [14] open cross-media linking [1] PimVis [13] DocTr [6] EdFest [23] source code visualisation [18]
  7. 7. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 7September 23, 2017 ArtVis: TUI-based Information Exploration  Advanced visualisation techniques in combination with a TUI  explore Web Gallery of Art  faceted browsing  phidgets-based TUI  RFID-tagged physical objects  Three main components to explore, analyse and browse the information Bram Moerman
  8. 8. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 8September 23, 2017 ArtVis Setup Bram Moerman
  9. 9. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 9September 23, 2017 ArtVis Visualisation Bram Moerman
  10. 10. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 10September 23, 2017 Tangible Bits (Ishii & Ullmer, 1997)  Beyond GUIs ("painted bits")  Tangible User Interfaces (TUIs) augment the physical space by coupling digital information to everyday objects and environments  physical instantiation of GUI elements in TUI  Key concepts  interactive surfaces  unidirectional coupling of atoms and bits  ambient media (inspired by Life Wire)  Tangible bits mainly act as input controls for digital information and services (output on computer screens) Hiroshi Ishii Brygg Ullmer
  11. 11. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 11September 23, 2017 Radical Atoms (Ishii et al., 2012)  Vision taking a leap beyond Tangible Bits  assuming a hypothetical generation of materials that can dynamically change their form and appearance  vision-driven design research  Radical Atoms is  a computationally transformable and reconfigurable material that is bidirectionally coupled with an underlying digital model (bits) - changes to digital model are reflected in physical representation and vice versa  a future material that can transform its shape, conform to constraints and inform the users of its affordances  a vision for the future of human-material interaction and Material User Interfaces (MUIs) in which all digital information has a physical manifestation so that we can directly interact with it
  12. 12. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 12September 23, 2017 Radical Atoms …  Transform  interface can transform its shape to modify the model and reflect changes in the computational model  Conform  interface has to conform to some physical laws and user constraints (e.g. for safety)  Inform  user has to be informed about changing interface affordances
  13. 13. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 13September 23, 2017 Interactions with Radical Atoms  Direct touch and gestural interaction  gestures coupled with direct touch create an interaction appropriate for Radical Atoms since users are able to rapidly reform dynamic materials at all scales  Context-aware transformation  context-aware transformations of the tool/interface - e.g. screwdriver adapting to the type of screw it is operating on  Shape-memory clay: Perfect Red  Perfect Red is a fictional material that can be sculpted like and responds according to rules inspired by CAD operations - e.g. if we split a piece in two even halves, then the operations performed on one part are mirrored on the other part
  14. 14. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 14September 23, 2017 TRANSFORM TRANSFORM, MIT  Dynamic shape display that can physically render 3D content  tangible interaction with digital content - geospatial data - 3D modelling - …  TRANSFORM display can also interact with the physical world around it  remote users can be displayed physically  Step towards MIT’s vision of Radical Atoms
  15. 15. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 15September 23, 2017 Video: TRANSFORM
  16. 16. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 16September 23, 2017 Data Physicalisation (Jansen et al., 2015)  Physical objects used for output  often static encoding of digital information  physical visualisations can outperform their virtual counterparts  How can we achieve dynamic data physicalisation with dynamic affordances  e.g. 3D visualisation with additional dimensions represented by physical variables such as temperature, texture, weight or plasticity  TRANSFORM is one possible approach; tangible holograms are another experimental mobile solution - can be used for multidimensional collaborative exploration and analysis of big data sets
  17. 17. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 17September 23, 2017 Mixed Reality "The perceived congruence concerns both visual as well as tactile perception. A perfect hologram will appear embodied but trying to touch it will destroy the illusion." Microsoft HoloLens Jansen, 2014 [34]
  18. 18. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 18September 23, 2017 Tangible Hologram (TangHo) Setup Timothy J. Curtin
  19. 19. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 19September 23, 2017 Tangible Graph Hologram
  20. 20. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 20September 23, 2017 Collaborative TangHo Interaction
  21. 21. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 21September 23, 2017 Robotic Arm Prototype  Lego Mindstorms-based 6DOF arm prototype  motors can be locked or set to float mode - useful for bidirectional I/O  replaceable sphere for different types of non-visual feedback  Challenges  inverse kinematics (final system will be body mounted) - use robotics toolbox (MATLAB)  limited HoloLens hand tracking  3D printing of final arm Timothy J. Curtin
  22. 22. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 22September 23, 2017 TangHo Prototype Timothy J. Curtin
  23. 23. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 23September 23, 2017 TangHo Interactions Timothy J. Curtin
  24. 24. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 24September 23, 2017 Data Physicalisation Framework  Inspired by research agenda of Jansen et al. [34]  Understanding the design space  how do users map data to physicalisations  Understanding the perceptual effectiveness  e.g. what are the just-noticeable differences that a physical variable can convey?  Framework for dynamic data physicalisation  data processing with device-independent output - TextDIO: text for static or dynamic reports - PhysDIO: data points, data variables and physical variables  text and visualisation drivers for representation on specific devices - e.g. tangible hologram prototype
  25. 25. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 25September 23, 2017 Conclusions  Tangible holograms for data physicalisation and simulation of radical atoms  physical augmentation of highly configurable virtual objects rather than programming of physical material (programmable matter)  mobile solution that is no longer bound to a specific place such as a the TRANSFORM table  modular solution that can be configured for specific domains (e.g. different spheres for various physical variables)  Data physicalisation framework for investigating new opportunities in dynamic data physicalisation  new data physicalisation design guidelines (as in InfoVis)  new evaluation techniques  …
  26. 26. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 26September 23, 2017 References  B. Signer, Fundamental Concepts for Interactive Paper and Cross-Media Information Spaces, ISBN 978-3-8370-2713-6, May 2008  ArtVis Project  http://www.beatsigner.com/flyers/ArtVis.pdf  Tangible Holograms  http://beatsigner.com/tangibleHolograms.html  TRANSFORM  https://www.youtube.com/watch?v=lCARHatJQJA
  27. 27. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 27September 23, 2017 References ... [1] B. Signer and M.C. Norrie, A Model and Architecture for Open Cross-Media Annotation and Link Services, Information Systems 36(3), Elsevier, May 2011  http://beatsigner.com/publications/signer_IS2011.pdf [2] B. Signer, What is Wrong with Digital Documents? A Conceptual Model for Structural Cross-Media Content Composition and Reuse, Proceedings of the 29th International Conference on Conceptual Modeling (ER 2010), Vancouver, Canada, November 2010  http://beatsigner.com/publications/signer_ER2010.pdf
  28. 28. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 28September 23, 2017 References ... [3] B. Signer, Fundamental Concepts for Interactive Paper and Cross-Media Information Spaces, Dissertation ETH No. 16218, Zurich, Switzerland, 2005  http://beatsigner.com/publications/signer2006.pdf [4] B. Signer and M.C. Norrie, As We May Link: A General Metamodel for Hypermedia Systems, Proceedings of the 26th International Conference on Conceptual Modeling (ER 2007), Auckland, New Zealand, November 2007  http://beatsigner.com/publications/signer_ER2010.pdf
  29. 29. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 29September 23, 2017 References ... [5] S. Trullemans and B. Signer, Towards a Con- ceptual Framework and Metamodel for Context-Aware Personal Cross-Media Information Management Systems, Proceedings of ER 2014, 33rd International Conference on Conceptual Modelling, Atlanta, USA, October, 2014  http://beatsigner.com/publications/trullemans_ER2014.pdf [6] S. Trullemans, A. Vercruysse and B. Signer, DocTr: A Unifying Framework for Tracking Physical Docu- ments and Organisational Structures, Proceedings of EICS 2016, Brussels, Belgium, June 2016  http://beatsigner.com/publications/trullemans_EICS2016.pdf
  30. 30. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 30September 23, 2017 References ... [7] R. Roels and B. Signer, MindXpres: An Exten- sible Content-driven Cross-Media Presentation Platform, Proceedings of WISE 2014, 15th Interna- tional Conference on Web Information System Engineering, Thessaloniki, Greece, October, 2014  http://beatsigner.com/publications/roels_WISE2014.pdf [8] A. Sanctorum and B. Signer, Towards User-defined Cross-Device Interaction, Proceedings of DUI 2016, 5th Workshop on Distributed User Interfaces, Lugano, Switzerland, June 2016  http://beatsigner.com/publications/sanctorum_DUI2016.pdf
  31. 31. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 31September 23, 2017 References ... [9] S. Trullemans and B. Signer, A Multi-layered Context Modelling Approach for End Users, Expert Users and Programmers, Proceedings of the Interna- tional Workshop on Smart Ecosystems cReation by Visual dEsign (SERVE 2016), Bari, Italy, June 2016  http://beatsigner.com/publications/trullemans_SERVE2016.pdf [10] R. Roels, Y. Baeten and B. Signer, An Interactive Data Visualisation Approach for Next Generation Presentation Tools: Towards Rich Presentation-based Data Exploration and Storytelling, Proceedings of CSEDU 2016, Rome, Italy, April, 2016  http://beatsigner.com/publications/roels_CSEDU2016.pdf
  32. 32. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 32September 23, 2017 References ... [11] M.C. Norrie, B. Signer and N. Weibel, Interactive Paper as a Reading Medium in Digital Libraries, Pro- ceedings of ECDL 2008, 12th European Conference on Research and Advanced Technology for Digital Libraries, Aarhus, Denmark, September 2008  http://beatsigner.com/publications/norrie_ECDL2008.pdf [12] B. Signer, M.C. Norrie, P. Geissbuehler and D. Heiniger, Telephone Interface for Avalanche War- nings based on Information Server for Adaptable Con- tent Delivery, Proceedings of Pervasive 2002, Zurich, Switzerland, August 2002  http://beatsigner.com/publications/signer_Pervasive2002.pdf
  33. 33. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 33September 23, 2017 References ... [13] S. Trullemans, A. Sanctorum and B. Signer, PimVis: Exploring and Re-finding Documents in Cross- Media Information Spaces, Proceedings of the Inter- national Working Conference on Advanced Visual Interfaces (AVI 2016), Bari, Italy, June 2016  http://beatsigner.com/publications/trullemans_AVI2016.pdf [14] B. Dumas, B. Moerman, S. Trullemans and B. Signer, ArtVis: Combining Advanced Visualisation and Tan- gible Interaction for the Exploration, Analysis and Browsing of Digital Artwork Collections, Proceedings of AVI 2014, Como, Italy, May 2014  http://beatsigner.com/publications/dumas_AVI2014.pdf
  34. 34. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 34September 23, 2017 References ... [15] B. Dumas, T. Broché, L. Hoste and B. Signer, ViDaX: An Interactive Semantic Data Visualisation and Exploration Tool, Proceedings of AVI 2012, Internatio- nal Working Conference on Advanced Visual Inter- faces, Capri Island, Italy, May 2012  http://beatsigner.com/publications/dumas_AVI2012.pdf [16] Moira C. Norrie, Beat Signer and Nadir Weibel, Print-n-Link: Weaving the Paper Web, Proceedings of DocEng 2006, ACM Symposium on Document Engineering, Amsterdam, The Netherlands, October 2006  http://beatsigner.com/publications/norrie_DocEng2006.pdf
  35. 35. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 35September 23, 2017 References ... [17] M. Grossniklaus, M.C. Norrie, B. Signer and N. Weibel ,Putting Location-Based Services on the Map, Proceedings of W2GIS 2006, Hong Kong, China, December 2006  http://beatsigner.com/publications/grossniklaus_W2GIS2006.pdf [18] R. Roels, P. Mestereaga and B. Signer, An Interactive Source Code Visualisation Plug-in for the MindXpres Presentation Platform, Communications in Computer and Information Science (CCIS), 583, 2016  http://beatsigner.com/publications/roels_CCIS2016.pdf
  36. 36. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 36September 23, 2017 References ... [19] C. Scholiers, L. Hoste, B. Signer and W. De Meuter, Midas: A Declarative Multi-Touch Interaction Framework, Proceedings of TEI 2011, 5th Interna- tional Conference on Tangible, Embedded and Em- bodied Interaction, Funchal, Portugal, January 2006  http://beatsigner.com/publications/scholliers_TEI2011.pdf [20] L. Hoste, B. Dumas and B. Signer, Mudra: A Unified Multimodal Interaction Framework, Proceedings of ICMI 2011, 13th International Conference on Multi- modal Interaction (ICMI 2011), Alicante, Spain, November 2011  http://beatsigner.com/publications/scholliers_TEI2011.pdf
  37. 37. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 37September 23, 2017 References ... [21] B. Signer, U. Kurmann and M.C. Norrie, iGesture: A General Gesture Recognition Framework, Proceedings of the 9th International Conference on Document Analysis and Recognition (ICDAR 2007), Curitiba, Brazil, September 2007  http://beatsigner.com/publications/signer_ICDAR2007.pdf [22] L. Hoste and B. Signer, SpeeG2: A Speech- and Gesture-based Interface for Efficient Controller-free Text Entry, Proceedings of ICMI 2013, 15th Inter- national Conference on Multimodal Interaction, Sydney, Australia, December 2013  http://beatsigner.com/publications/hoste_ICMI2013.pdf
  38. 38. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 38September 23, 2017 References ... [23] B. Signer, M. Grossniklaus and M.C. Norrie, Interactive Paper as a Mobile Client for a Multi- Channel Web Information System, World Wide Web Journal (WWW), Vol. 10, No. 4, Springer, December 2007  http://beatsigner.com/publications/signer_WWWJ2007.pdf [24] B. Signer and M.C. Norrie, PaperPoint: A Paper- based Presentation and Interactive Paper Prototyping Tool, Proceedings of TEI 2007, First International Conference on Tangible and Embedded Interaction, Baton Rouge, USA, February 2007  http://beatsigner.com/publications/signer_TEI2007.pdf
  39. 39. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 39September 23, 2017 References ... [25] A. Vogelsang and B. Signer, The Lost Cosmo- naut: An Interactive Narrative Environment on Basis of Digitally Enhanced Paper, Proceedings of the Inter- national Conference on Virtual Storytelling 2005, Strasbourg, France, December 2005  http://beatsigner.com/publications/vogelsang_ICVS2005.pdf [26] L. Hoste and B. Signer, Expressive Control of Indirect Augmented Reality During Live Music Performances, Proceedings of NIME 2013, 13th International Confe- rence on New Interfaces for Musical Expression, Daejeon, Korea Republic, May 2013  http://beatsigner.com/publications/hoste_NIME2013.pdf
  40. 40. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 40September 23, 2017 References ... [27] N. Weibel, A. Ispas, B. Signer and M.C. Norrie, PaperProof: A Paper-Digital Proof-Editing System, Proceedings of CHI 2008, 26th International Confe- rence on Human Factors in Computing Systems (Interactivity Track), Florence, Italy, April 2008  http://beatsigner.com/publications/weibel_CHI2008.pdf [28] A. Ispas, H. Schuldt, M.C. Norrie and B. Signer, Towards Query by Sketch, Michael Springmann, Second DELOS Conference on Digital Libraries, Pisa, Italy, December 2007  http://beatsigner.com/publications/springmann_DELOS2007.pdf
  41. 41. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 41September 23, 2017 References ... [29] N. Weibel, B. Signer, M.C. Norrie, H. Hofstetter, H.-C. Jetter and H. Reiterer, PaperSketch: A Paper- Digital Collaborative Remote Sketching Tool, Proceed- ings of IUI 2011, International Conference on Intelli- gent User Interfaces, Palo Alto, USA, February 2011  http://beatsigner.com/publications/weibel_IUI2011.pdf [30] B. Signer and T.J. Curtin, Tangible Holograms: Towards Mobile Physical Augmentation of Virtual Objects, Technical Report WISE Lab, WISE-2017-01, March 2017  http://beatsigner.com/publications/signer_arXiv2017.pdf
  42. 42. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 42September 23, 2017 References ... [31] H. Ishii and B. Ullmer, Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms, Proceedings of CHI 1997, ACM Conference on Human Factors in Computing Systems, Atlanta, USA, March 1997  http://dx.doi.org/10.1145/258549.258715 [32] H. Ishii, D. Lakatos, L. Bonanni and J.-B. Labrune, Radical Atoms: Beyond Tangible Bits, Toward Transformable Materials, interactions, 19(1), January 2012  http://dx.doi.org/10.1145/2065327.2065337
  43. 43. Beat Signer - Department of Computer Science - bsigner@vub.ac.be 43September 23, 2017 References ... [33] Y. Jansen et al., Opportunities and Challenges for Data Physicalizations, Proceedings of CHI 2015, ACM Conference on Human Factors in Computing Systems, Seoul, Korea, April 2015  https://doi.org/10.1145/2702123.2702180 [34] Y. Jansen, Physical and Tangible Information Visualization, 2014, Ph.D. Dissertation, Université Paris-Sud  https://tel.archives-ouvertes.fr/tel-00981521

×