HoloFlower BCI

social-mental reformation through brain controlled 3d data visualization in XR

The Holo Flower is a real-time data visualization interface displayed in holography on digital devices like smartphone and tablets. The input data of the flower is based on real-time bio-metric data and EEG signals from Emotiv Epoc+ headband. The concept behind the Holo Flower is to unrolling and visualizing vulnerable details of daily brain events.

As passive data sensing become widely accepted for wellness and mental health purpose, many wearable technology products gain rapid market shares. Recently the enthusiasm of wearable technology is clearly booming as we see Apple and Google’s Fitbit deals stay at the top of finance media. The vision of this study is to discuss possibilities of a shifting purpose from individual based product to a further community for intuitive knowing
‍
‍KEYWORDS
AR; Cognitive Enhancement; User Interface; Interaction; Brain Computer Interaction; Experience Design; HCI; Digital Media; Responsive Environment; Data Visualization; Product Design; Visual art; Cognitive Science; User Experience; Communication; Interpersonal Listening; Emotion Regulation
‍
Yiqi Zhao
Note: this research is supported by Object Based Media, MIT Media Lab

2070 Sensory Sharing and Perception Enhancement Friction

1   BACKGROUND

‍1.1 Our future? Heading into BBI(brain to brain interaction)‍

three-stages of data sharing:

●encrypted sharing vessels for data analysis systems

●lateral hierarchy: environmental collections, wearable sensing devices data, invasive brain activities data

●longitudinal hierarchy: passive data for infrastructure functioning, passive data for social-economical functioning, active data for citizen’s personalized needs

1.2 What is the medium between understanding?

From interpersonal communication to intuitive knowing: While we live a peaceful and liberal life until our society went into chaos because of biotechnological applications for mankind and the built environment. The more we blurred the boundary of human, machine, and the environment, the more we strike on our social norm and order, thus our world was concorde more and more by dictators for controlling efficiency.

‍

2 Purpose

2.1 Interpersonal communication? Intuitive knowing?

As passive data sensing become widely accepted for wellness and mental health purpose, many wearable technology products gain rapid market shares. Recently the enthusiasm of wearable technology is clearly booming as we see Apple and Google’s Fitbit deals stay at the top of finance media. The vision of this study is to discuss possibilities of a shifting purpose from individual based product to a further community for intuitive knowing

2.2 Initial Concept

The Holo Flower is a real-time data visualization interface displayed in holography on digital devices. The input data of the flower is based on biometric data and EEG signals from Emotiv Epoc+ headband. The concept behind the Holo Flower is to unrolling and visualizing vulnerable details of daily brain events.

  • self-train focusing (mental command)
  • preserve and share brain activities
  • meditation enhancement

‍

3  CONCEPT DEVELOPMENT

‍

3.1 Overview
  • Mixed Reality/Hologram
  • Generative Machine Learning
  • 3D Data Visualization
  • Brain EEG data - Mental Command!

Reveal a social activity or relationship phenomenon at a reformed quality after technology caused social destruction

Convey how data’s as an information medium could step into daily activities and fulfill social desire

Illustrate the trend of unrolling and visualizing all the vulnerabilities of human natures

‍

3.2 Hardware and Software

Brain Data Collecting: Emotiv Epoc+ (14(+CMS/DRL, references) electrodes, sequential sampling 128 sps, 0.2-42 hz, Lipo battery(12 hrs), embedded gyroscope)

Digital Interface: Unity3D

Server: Cortex Cloud Server supported by Emotiv Company

‍

3.3 Proposed System

4 Design Implementation

‍

4.1 data stream used
  • ‍eeg: The raw EEG data (14 chanels) from the headset.‍
  • mot: The motion data from the headset.‍
  • met: The results of the performance metrics detection.‍
  • com: The results of the mental commands detection.‍
  • fac: The results of the facial expressions detection.

‍

4.2 Visualization | Layer I: EEG Streaming

The real-time EEG data is mapped from 14 individual stream(Fig.2) to the central line of each flower petals (Fig. 1). The initial flower petal mesh(Fig. 3) is designed in Autodesk 3ds Max.

Proposed Flower Behaviors:

  • Shaking
  • Deforming
Fig 2. Electrodes name of collected dataundefined
Fig 1. White square shows central line of each petals
Fig 3. Initial flower design

‍‍

4.3 Visualization | Layer II: Train Command

Flower Behaviors:

  • Rotating
  • Lifting
  • Deforming

The designed deforming behavior based on mesh vertices velocity is tested with real-time user testing.

A method simulating pressure is developed in C# to generate [ray cast] 14 mapped locations onto flower meshes. Each ray is moving from petal end to flower center based on EEG data: ApplyPressureToPoint(Vector3 _point, float _pressure)

Based on several testing, a decent range of pressure force that applied on the mesh vertices was noted. (Fig 4-5)

Fig 5. updated deformation with adjustable force as pressure input
Fig 4. user testing with rotation and deformation based on real-time EEG data

‍

4.4 Tablet Interface
  • Shaking: Wile a new user session token (Fig 6-7)
  • Rotating: Depends on user trained command
  • Lifting: Depends on user trained command
  • Deforming: Based on real-time EEG data
Fig 6. Flower state when disconnected
Fig 7. Petals start shaking while a new user detacted
‍
Fig 8. Tablet display showing connection and training command
‍
Fig 9. Strong signal visualization related to an extreme facial expression during user testing

‍

‍

4 Design Implementation | Flower Materialization with GANS

‍

4.1 Human-Machine Collaboration Data-vis

To discuss possibilities of AI created visualization mixed with human data engineered visualization, this chapter examined generative machine learning tools for creating responsive material finish on the the flower rendering. Conditional GAN is applied to a responsive tile sample in order to create an tile pattern while maintaining its attribution to a particular class.

Based on: Implementation of Creative Adversarial Networks https://arxiv.org/pdf/1706.07068.pdf. Phillip Kravtsov, Phillip Kuznetsov

Tile Testing: CGAN

Original tiles were generated from 27 classes of painting. Later Gaussian noise was added. The tile is then sampled into a consistent style.

Fig 10. Generated Sample
‍

‍

Tile Testing: Imaging toning, hue, and pixel glitch

In order to reach a more deemed color but active visual quality, an adapting dynamic hue is mapped to tile image.(Fig 11) Hue gradient map used: dark red - orange - blue - white

A glitch style(Fig 12) is applied to the tile by taking every pixel of the tile image and shifted by a delta vector

Fig 11. Generated Hue
‍
Fig 12. Generated Hue and Pixel
‍

‍‍

4.2 Prefab of Tile Filter

Tone of the tile and Glitch Speed are applied to the prefab of flower render material as Unity Shader.

var index : int = (Time.time * framesPerSecond) % frames.Length;

renderer.material.mainTexture = frames[index];

Yiqi, designer, technologist, artist.

about me
contact me
⇑ TOPBACK TO WORKS


Have a good day !
HOME
Contacts
WA, USA
vanyiqi@gmail.com
sirrandom
Follow
/Long dream stretches, because of you.
Copyright © 2022 Yiqi Zhao. All rights reserved. 0_0