The Cyberworlds 2018 Program Booklet is available here
The Best Full Paper Awards:
- “User Dependent Template Update for Keystroke Dynamics Recognition”
presented by Abir Mhenni
- “MaeSTrO: A Mobile App for Style Transfer Orchestration using Neural Networks”
presented by Amir Semmo
- “Computational Analysis of Smile Weight Distribution across the Face for Accurate Distinction between Genuine and Posed Smiles”
presented by Hassan Ugail
The Best Short Paper Awards:
- “Predicting Ordinal Level of Sedation from the Spectrogram of Electroencephalography”
presented by Haoqi Sun
- “EEG-based Evaluation of Mental Fatigue Using Machine Learning Algorithms”
presented by Yisi Liu
The Best Poster Award:
- “The Behavior Symptoms of Undergraduates’ Social Anxiety in the Virtual World”
presented by Lin Huang
Fn - full paper with paper ID=n, Sn - short paper with paper ID=n, Pn - poster paper with paper ID=n
GTn - General Track paper session number n;
CHMIn - Cognitive Human-Machine Interaction Track paper session number n;
CSBMn - Cybersecurity and Biometrics Track paper session number n;
CCCMn - Cyber Cities and Cyber Manufacturing Track paper session number n.
Color coding: n plenary meeting; n parallel paper session; n food break; n cultural event; n lecture room not used.
|Coference registration will be done in the lobby of NEC every day from 08:30 to 17:00|
|2-Oct||Pre-conference reception wth food and drinks provided.
"Cosmo" restaurant (2nd level), opposite NEC, 19:00-21:00
|TIME||Lecture Room LR 1||Lecture Room LR 2|
|09:00-09:15||Opening greetings by the conference chairs|
|09:15-10:15||Keynote 1: Kiyoshi Kiyokawa|
|10:30-11:30||Posters fast forward (15'), presentations and demos (45')
P109, P110, P111, P112, P115, P116, P117, P120, P121, P122, P123
|11:30-12:45||Panel: Human-Robot Co-Existence: Blessing or Curse?|
|13:45-15:45||GT1 (Kiyoshi Kiyokawa): F26, F69, F101, F35||CHMI1 (Reinhod Scherer): F33, F100, S105, S46, S61|
|16:00-18:00||GT2 (Andres Iglesias): F40, F50, F76, F64||CHMI2 (Anastasios Bezerianos): F106, F20, S89, S21|
|18:15-22:00||Two way bus downtown and traditional bumboat
river cruise (free for the delegates).
Buses to depart from NEC at 18:15 and the return buses from Marina Bay Sands to NEC at 22:00.
|TIME||Lecture Room LR 1||Lecture Room LR 2|
|09:00-11:00||GT3 (Marius Erdt): F60, F83, F88, S97||CHMI3 (Hassan Ugail): F48, F82, F52, F16|
|11:15-13:15||GT4 (Zheng Jianmin): S67, S51, S8, S31, S34, S30||CHMI4 (Liu Yisi): F19, F41, F78, S99|
|14:15-15:15||Keynote 2: Christophe Rosenberger|
|15:15-16:15||Keynote 3: Chen Chun-Hsien|
|16:15-17:00||Industrial talk: Terry Yin|
|17:15-21:30||Bird show and conference banquette at Jurong
Bird Park. Buses to depart fro NEC at 17:15.
Private bird show: 18:00-19:00. Cocktail reception and dinner: 19:00 - 21:30. Buses depart for NEC at 21:45.
|TIME||Lecture Room LR 1||Lecture Room LR 2|
|09:00-11:00||GT5 (Angelos Amditis): S45, S15, S24, S55, S10, S47||CSB1 (Christophe Charrier): F75, F81, F92, F85|
|11:15-12:15||GT6 (Alexei Sourin): S86, S74, S77||CSB2 (Patrick Bours): S80, S66, S96|
|12:15-13:15||CCCM1 (Henry Johan): F9, F14||CSB2 (Patrick Bours): S42, S73, S36|
|14:15-15:55||CCCM2 (Bodo Urban): F37, S68, S54, S84||CHMI5 (Khng Kiat Hui): S32, S95, S90, S118, S17|
|16:10-17:10||Keynote 4: Bodo Urban|
|17:10-17:30||Best paper awards; CW2019 presenation; Closing|
|The List of Conference
the name of the presenting author is underlined
|Paper ID||General Track: Full Papers|
Structures in Surface Mesh Models for 3D Printing
Jidong Wang, Ruibin Zhao and Mingyong Pang
How to improve strength-to-weight ratio of printed models is an important topic in 3D printing. We in this paper propose a novel structure modeling method based on the implicit function technique and the finite element method (FEM). Our method first obtains a set of sampled points in a given surface mesh model by using a probability-based strategy, and generates an adaptive tetrahedral mesh from the points. FEM is then used to analyze the stress of the tetrahedral mesh and a stress map of the input model is created. The method finally builds a result model composed of a shell and an interior single-gyroid lattice. The lattice is defined by a piecewise 3D implicit function, and has several special structural properties just like the structure of the light but strong butterfly wing. The lattice together with the shell forms a natural 3D structure for 3D printing. Local thickness of lattice rods in the structure adaptively changes with stress distribution for withstanding external loads. Experimental results show that our method can deal with various surface mesh models in rapid way, and the resulted models for 3D printing have high strength-to-weight ratios.
Differences between Head Mounted Displays and PC Screens
Andreas Siess, Matthias Wölfel and Nico Häffner
Recently virtual reality (VR) applications are shifting from professional use cases to more entertainment-centered approaches. Therefore aesthetic aspects in virtual environments gain in relevance. This paper examines the influence of different color determining parameters on user perception habits between head mounted displays (HMD) and computer screens. We conducted an empirical study with 50 persons that were asked to adjust the color temperature, saturation and contrast according to their personal preferences using a HMD as well as a computer screen, respectively. For cross validation we tested a second user group of 36 persons that were asked to adjust the color temperature exclusively. By using a set of five different panorama images—each of them representing an exemplary scenario—we have found that color perception differs significantly. This depends on the used output device as well as gender: i.e. females preferred a significantly colder color scheme in VR compared to their preferences on the computer screen. Furthermore they also chose a significant colder color scheme on the HMD compared to their male counterparts. Our findings demonstrate that content created for conventional screens can not simply be transferred to immersive virtual environments but for optimal results needs reevaluation of its visual aesthetics.
|F40||Facial Expression Editing
in Face Sketch using Shape Space Theory
Chenlei Lv, Zhongke Wu, Xingce Wang, Dan Zhang, Xiangyuan Liu and Mingquan Zhou
Facial expression editing in face sketch is an important and challenging problem in computer vision community as facial animation and modeling. For criminal investigation and portrait drawing, automatic expression editing tools for face sketch improve work efficiency obviously and reduce professional requirements for users. In this paper, we propose a novel method for facial expression editing in face sketch using shape space theory. The new facial expressions in the sketch images can be regenerated automatically. The method includes two components: 1) face sketch modeling; 2) expression editing. The face sketch modeling constructs 3D face sketch data from 3D facial database to match the 2D face sketch. Using facial landmarks, the “shape” of the face sketch is represented in shape space. The shape space is a manifold space which removes the rigid transform group. In shape space, the accurate 3D face sketch model is obtained which is consistent to the original 2D face sketch. For expression editing, we change the parameters of 3D face sketch model in the shape space to obtain new expressions. The expression transfer in 3D face sketch model can be mapped into the 2D face sketch. The advantages of our method are: full-automatic in modeling process; no requirements of drawing skills to user and friendly interaction; robustness to head poses and different scales. In experiments, we use the 3D facial database, FaceWareHouse, to construct the 3D face sketch model and use face sketch images from database: CUHK Face sketch Database (CUFS) to show the performance of expression editing. Experimental results demonstrate that our method can effectively edit facial expressions in face sketch with high consistency and fidelity.
|F50||A Robust and Efficient
Algorithm for Multi-body Continuous Collision Detection
Binbin Qi and Mingyong Pang
Multi-body collision detection is a key and important technology in societies of computer graphics, system simulation, virtual reality, etc, and has been widely used in various applications. To deal with the collision problems in large scale multi-body simulations robustly and efficiently, we in this paper proposed a robust and efficient algorithm of continuous multi-body collision detection based on the kinetic “sweep and prune” (SaP) technique and the event-driven mechanism. Our algorithm first culls redundant detection calculations among very large numbers of moving bodies, and then automatically generates events to predict these collisions, probably taken place in coming time, of the object pairs. All these events are been pushed into a priority queue, which is used to drive our algorithm to run. By introducing a new hybrid bounding box hierarchy in the event processing process, our algorithm can detect positions where the object pairs collide. We discovered the event blocking problem potentially occurred during event processing, and further proposed several methods to alarm or relieve the system from the event blocking state. Experimental results show that our algorithm has good stability and strong robustness, and it can improve the speed and accuracy of the multi-body collision detection effectively.
|F60||Enhancing Sketching and
Sculpting for Shape Modeling
Kai Wang, Jianmin Zheng and Hock Soon Seah
Sketch-based modeling uses freeform strokes as basic modeling metaphor and provides an intuitive way for shape modeling, for instance, for cyberworlds. This paper presents a new method to enhance sketch-based modeling. The core idea of the method is to enhance the sketching process by allowing the user to iteratively sketch to progressively create initial shapes that interpolate the sketched strokes. This process considers all the sketches and the up-to-date constructed 3D shape, which enables the user to be aware of the shape of the sketched model. The key underlying technique that supports this process is a novel surface construction algorithm, which generates 3D triangular mesh models with gradual shape changes during iterative sketching. Experiments demonstrate that the presented method can allow users to intuitively and flexibly create and edit 3D models even with complex topology, which is usually difficult in existing sketch-based modeling systems.
|F64||An Experimental Comparison
of Text Classification Techniques
Suyash Lakhotia and Xavier Bresson
Text classification is the task of labeling text data from a predetermined set of thematic labels. It has become of increasing importance in recent years as we generate large volumes of data and require the ability to search through these vast datasets with flexible queries. However, manually labeling text data is an extremely tedious task that is prone to human error. Thus, text classification has become a key focus of machine learning research, with the goal of producing models that are more efficient and accurate than traditional methods. The objective of this work is to rigorously compare the performance of current text classification techniques, from standard SVM-based, statistical and multilayer perceptron (MLP) models to recently enhanced deep learning models such as convolutional neural networks and their fusion with graph theory. Extensive numerical experiments on three major text classification datasets (Rotten Tomatoes Sentence Polarity, 20 Newsgroups and Reuters Corpus Volume 1) revealed two results. First, graph convolutional neural networks perform with greater or similar test accuracy when compared to standard convolutional neural networks, SVM-based models and statistical baseline models. Second, and more surprisingly, simpler MLP models still outperform recent deep learning techniques despite having fewer parameters. This implies that either benchmark datasets like RCV1 containing more than 420,000 documents from 52 classes are not large enough or the representation of text data as tf-idf document vectors is not expressive enough.
|F69||MaeSTrO: A Mobile App for
Style Transfer Orchestration using Neural Networks
Max Reimann, Mandy Klingbeil, Sebastian Pasewaldt, Amir Semmo, Matthias Trapp and Jürgen Döllner
Mobile expressive rendering gained increasing popularity among users seeking casual creativity by image stylization and supports the development of mobile artists as a new user group. In particular, neural style transfer has advanced as a core technology to emulate characteristics of manifold artistic styles. However, when it comes to creative expression, the technology still faces inherent limitations in providing low-level controls for localized image stylization. This work enhances state-of-the-art neural style transfer techniques by a generalized user interface with interactive tools to facilitate a creative and localized editing process. Thereby, we first propose a problem characterization representing trade-offs between visual quality, run-time performance, and user control. We then present MaeSTrO, a mobile app for orchestration of neural style transfer techniques using iterative, multi-style generative and adaptive neural networks that can be locally controlled by on-screen painting metaphors. At this, first user tests indicate different levels of satisfaction for the implemented techniques and interaction design.
|F76||What User Interface to Use
for Virtual Reality? 2D, 3D or Speech–A User Study
Yannick Weiß, Daniel Hepperle, Andreas Sieß and Matthias Wölfel
In virtual reality different demands on the user interface have to be addressed than on classic screen applications. That’s why established strategies from other digital media cannot be transferred unreflected and at least adaptation is required. So one of the leading questions is: which form of interface is preferable for virtual reality? Are 2D interfaces—that are mostly used in combination with mouse or touch interactions— the means of choice, although they do not use the medium’s full capabilities? What about 3D interfaces that can be naturally integrated into the virtual space? And last but not least: are speech interfaces, the fastest and most natural form of human interaction/communication, which have recently established themselves in other areas (e.g., digital assistants), ready to conquer the world of virtual reality? To answer these question this work compares these three approaches based on a quantitative user study and highlights advantages and disadvantages of the respective interfaces for virtual reality applications.
|F83||An Automatic Method for
Semantic Focal Feature Point Tracking of 3D Human Model in Motion Sequence
Peng Xiaoyu, Tan Xiaohui and Wang Kang
In this paper, a method, for automatically identifying and tracking garment related semantic focal feature points on 3D human model in motion sequence, is proposed. We consider the problem of automatic focal feature point identification and tracking when non-rigid shape deformation is occurred. The main contribution is that a novel method of tracking focal feature points when the human avatar move in front of depth camera. Firstly, we learn a regression analysis model that derives the relationship between sampled and focal feature points. Secondly, we build a model of correspondence maps to calculate the tracking results. The method can track garment-related feature points for different people in different motion and shape. We demonstrate on a wide variety of experiments that our approach leads to a significant identification and tracking result with input depth sequences.
|F88||Self-Training System for
Tennis Shots with Motion Feature Assessment and Visualization
Masaki Oshita, Takumi Inao, Tomohiko Mukai and Shigeru Kuriyama
This paper describes a prototype self-training system for tennis forehand shots that allows trainees to practice their motion forms by themselves. Our system uses a motion capture device to record a trainee’s motion, and visualizes the differences between the features of the trainee’s motion and the correct motion as performed by an expert. This system enables trainees to understand the errors in their motion and how to reduce or eliminate them. In this study, we classified the motion features and corresponding visualization methods using one- dimensional spatial, rotational, and temporal features based on the key sporting poses. We also developed a statistical model for the motion features, allowing the system to assess and prioritize all features of a trainee’s motion. This research focuses on the motion of a tennis forehand shot and evaluates our prototype through several user experiments.
Interactive Agent-based Simulations
Timothy Davison, Faramarz Samavati and Christian Jacob
Building and interacting with 3D agent-based simulations that contain a large number of agents is a significant challenge. What if we want to create an intricate new arrangement of agents, or reconfigure a large number of agents? We present LifeBrush, a cyberworld for interactively painting large and elaborate multi-agent simulations with commodity virtual reality systems that we can then simulate and explore. Our main methodology uses sketch-based discrete element texture synthesis to paint agent arrangments. We define a map to convert agents to elements in this framework when we paint and back to agents when we simulate. Like creating new colors on a paint palette, we create example agent arrangements and configurations in an example palette. We paint new agents into a scene with sketch-based generative brushes. We also use those brushes to reconfigure agents to match examples created in the palette. Then we simulate, pause the simulation and modify the agents with our sketch-based tools. This iteration loop enables new levels of interactivity for the design, simulation, and exploration of agent-based simulations.
|General Track: Short Papers|
|S8||Autonomous Virtual Player
in a Video Game Imitating Human Players: the ORION Framework
Cédric Buche, Cindy Even and Julien Soler
This paper introduces the design of autonomous virtual player based on imitation learning using human behavior observations. The ORION model provides both data mining techniques allowing the extraction of knowledge and behavior models allowing the control of the autonomous behaviors. ORION is also an operational tool allowing the representation, transformation, visualization and prediction of data. We illustrate the use of our model by detailing the implementation of a virtual player for the video game Unreal Tournament 3. Thanks to ORION, data from low level behaviors were collected through three scenarios performed by human players: movement, long range aiming and close combat. Behaviors can then be learned from the obtained data-sets after transformations and application of data mining techniques. ORION allows us to build a complete behavior using an extension of a Behavior Tree integrating ad hoc features in order to manage aspects of behavior that we have not been able to learn automatically.
|S10||Training an FCN with
Synthetic Images for Component Segmentation with Applications in Orientation
Estimation and Image Inpainting. Achim Rehberger, Kai Weber and Yvonne Jung
The detection and segmentation of real objects along with their components is still an advanced topic. More detailed segmentation is needed to solve tasks like position and orientation estimation or for doing inpainting of components. In this paper, we specifically focus on cars and present a segmentation of components, such as rims or lights, which requires detailed and accurate training data. A decomposed 3D model is used to render highly detailed images of the car that fit to corresponding ground truth images. The main challenge is to create high quality synthetic datasets that allow reliable and accurate segmentation of real-world footage. Different camera shots, filters, environment maps, shapes, and neural networks are used and their benefits as well as problems are discussed within this paper. The accuracy and reliability of the segmentation depends on the quality and variability of the rendered images. We started with simple 3D models and a real-time renderer and reached accurate and reliable segmentation with almost photorealistic images that are created with a global illumination renderer. These results are then used for replacing components of the car as well as for deriving the position of special points of interest, like the center of a wheel, which is also necessary for subsequent processing such as correctly aligning the 3D model with the real camera stream for Augmented Reality applications. Here, the quality of replacing a component of an object with its rendered 3D counterpart depends on the accuracy of segmentation. Therefore, segmented components are used to determine the position and orientation of the car along with the size of the inpainting area. Then, a matching rendered image of the component is inpainted only into the segmented area. In this regard, we also compare two different approaches for deriving the center points of the components of an object.
|S15||Glossy Reflections for
Mixed Reality Environments on Mobile Devices
Tobias Schwandt, Christian Kunert and Wolfgang Broll
Glossy reflections of the surroundings play a major role when trying to achieve a seamless fusion of real and virtual objects in Mixed Reality (MR) environments. Traditionally, the necessary information about the ambiance is captured using mirrored balls, HDR cameras, fish-eye lenses, RGB-D cameras or 360-degree cameras. While these approaches allow for pretty good results, they require a rather complex setup. Our approach is based on a single RGB camera capturing the environmental lighting at a certain location within the scene. Therefore, we apply a precomputation step generating a 360-degree environment map and combine it with a camera-based image stitching for a continuous enhancement and update of the lighting information. We show that our approach allows for realistic and high-quality reflections within an AR/MR environment in real time even on mobile devices.
|S24||Text to 3D Model of
Chinese Ancient Architecture
Yan Wang, Pu Ren, Mingquan Zhou, Wuyang Shui and Pengbo Zhou
Three-dimensional (3D) modeling is currently a creative task that requires modelers with strong professional skills and background knowledge, especially in the field of 3D modeling of Chinese ancient architecture (CAA). At present, most of the studies on 3D CAA modeling are based on hard-coded constructive rules, which need completed, complex and formalized descriptions. We present a generative system bridging the gap between the Chinese text and 3D models that allows users to generate 3D models by natural language. First, a Bayesian network is learned from existing CAA data to provide relationships of different structural components. Second, by parsing the Chinese text inputted by the user, key components of the CAA will be determined; and other matched structural components will be calculated by inferencing the trained Bayesian network. Third, the synthesis of all components is achieved by a proposed placement optimizing algorithm. Finally, we evaluate the effectiveness of the trained Bayesian network and demonstrate the application to generate 3D CAA model rapidly from the Chinese text.
Curves with Sharp Features
Jingjie Zhao, Jidong Wang, Ruibin Zhao and Mingyong Pang
Implicit curves play an essential role in the societies of medicine, meteorology, geology, geo-physics, visualization and so on. In this paper, we propose an algorithm to visualize implicit curves and reproduce their sharp features in 2D plane. To access the subdivision cells of a user-defined 2D domain, our algorithm first creates a quadtree by using a top-down and adaptive quad-tree construction technique. In each cell, the method locates exact one feature point of the numerical field defined by the implicit function defining an implicit curve. A discrete optimization technique is employed to calculate the feature points. A dual mesh is subsequently constructed for the quadtree by taking the feature points as its vertices. Our algorithm approximates local part of the implicit curve in each cell of the dual mesh with a modified version of the marching squares method. Collecting all the approximations in the cells, our method finally reproduces the implicit curve with sharp features. Experiments show that our method can efficiently extract the sharp features of implicit curves, and it can work with various implicit curves with or without sharp features robustly.
|S31||On Multiple-view Matrix
Based 3D Reconstruction from Multiple-view Images
Hui-Min Huang, Rui-Bin Zhao and Ming-Yong Pang
In this paper, we propose a multiple-view matrix based 3D reconstruction algorithm for generating a 3D point cloud model for a scene or an object from several sequence images. The algorithm first extracts a group of Scale Invariant Feature Transform (SIFT) feature points from each image, and divides the points into different groups according to the matching degrees among the points. Secondly, a set of 3D point clouds are reconstructed from the feature points with a calculated a multiple-view matrix. Then, a complete result is generated by merging the point clouds with an incremental algorithm and the estimated camera parameters. Furthermore, our result is optimized by employing a Bundle Adjustment (BA) method. Owing to the introduction of the multiple-view matrix and the group-based SIFT matching, our algorithm has the ability to accurately reconstruct a 3D point cloud model only with several images. The performance of our algorithm is evaluated on a group of benchmark datasets, and is compared to two state-of-the-art methods.
|S34||A Benchmark for Distance
Ulrich Krispel, Dieter W. Fellner and Torsten Ullrich
The need to analyze and visualize distances between objects arises in many use cases. Although the problem to calculate the distance between two polygonal objects may sound simple, real-world scenarios with large models will always be challenging, but optimization techniques – such as space partitioning – can reduce the complexity of the average case significantly.
Our contribution to this problem is a publicly available benchmark to compare distance calculation algorithms. Furthermore, we evaluated the two most important techniques (hierarchical tree structures versus grid-based approaches).
Games in the Edo Period Using Interactive Techniques for Museum Exhibits
Asako Soga, Masahito Shiba and Takuzi Suzuki
The purpose of this study is to raise interest in a kind of Japanese board game Sugoroku in the Edo period, and to support exhibits of it at museums. We developed a computer-aided Sugoroku games using modern interactive techniques. In this system, the user rolls a dice-type device equipped with a microcomputer. Since the system detects the values of the dice-type device, the players can simply play by just throwing the die. By projecting the game’s progress on the Sugoroku sheet with a ceiling projector, the system shows the current positions of the players and the candidate destinations. With this guide, they can play Sugoroku games even without knowing the rules. The system was used at a special exhibition of the National Museum of Japanese History for eight weeks. We evaluated our computer-aided Sugoroku games with visitors on three days. Almost half of the visitors marked the best score for all items, indicating that this system was successfully accepted by them.
Video-haptic Interaction in Cyberspace
Guo Song and Alexei Sourin
Video-conferencing and video calls are common nowadays. However, without being able to physically touch or feel each other, we cannot have full immersive communication achieved. Unlike earlier attempts of joining video and haptic communication into one integrated system, we propose to setup an asynchronous method of exchanging haptic interaction data while using traditional ways of video communication. The data packets are exchanged over the Internet cloud server where only participating haptic interface point coordinates, orientation angles of the device handles are transmitted. We also explore more options like using depth-sensing cameras and hand-tracking devices to capture motion of hand, arm or the whole body of one user so that he becomes both visible and tangible to the other party. The proposed way of communication is validated by technical measurements and a user study. A test of remote connections over long distances was carried out to pave way for further studies.
|S51||A Framework for 3D Object
Segmentation and Retrieval using Local Geometric Surface Features
Dimitrios Dimou and Konstantinos Moustakas
Robotic vision and in particular 3D understanding has attracted intense research efforts the last few years due to its wide range of applications, such as robot-human interaction, augmented and virtual reality etc, and the introduction of low-cost 3D sensing devices. In this paper we explore one of the most popular problems encountered in 3D perception applications, namely the segmentation of a 3D scene and the retrieval of similar objects from a model database. We use a geometric approach for both the segmentation and the retrieval modules that enables us to develop a fast, low-memory footprint system without the use of large-scale annotated datasets. The system is based on the fast computation of surface normals and the encoding power of local geometric features. Our experiments demonstrate that such a complete 3D understanding framework is possible and advantages over other approaches as well as weaknesses are discussed.
|S55||A Figurative and
Non-topological Approach to Mathematical Visualization
Atsushi Miyazawa, Masanori Nakayama and Issei Fujishiro
The term figurative refers to any form of mathematical visualization that retains strong references to the geometry found in the real world. This paper explains the figuration process for some basic mathematical functions definable in the n-dimensional complex projective space. In the latter part of this paper, we raise a question that has been neglected thus far: What does the Riemann sphere’s axis stand for? We show that the answer can be obtained only by observing from the inside the sphere by setting the viewpoint of the immersive environment to the origin, which is always undefined in projective geometry. We also draw some basic math functions that are familiar to us on the projective plane and observe the invariant properties that exist among the functions, which were thought to be different from one another.
Assessment: a Novel Protocol & Analysis of Judge Expertise
Cindy Even, Anne-Gwenn Bosser and Cédric Buche
For video game designers, being able to provide both interesting and human-like opponents is a definite benefit to the game’s entertainment value. The development of such believable virtual players also known as Non-Player Characters or bots remains a challenge which has kept the research community busy for many years. However, evaluation methods vary widely which can make systems difficult to compare. The BotPrize competition has provided some highly regarded assessment methods for comparing bots’ believability in a first person shooter game. It involves humans judging virtual agents competing for the most believable bot title. In this paper, we describe a system allowing us to partly automate such a competition, a novel evaluation protocol based on an early version of the BotPrize, and an analysis of the data we collected regarding human judges during a national event. We observed that the best judges were those who play video games the most often, especially games involving combat, and are used to playing against virtual players, strangers and physically present players. This result is a starting point for the design of a new generic and rigorous protocol for the evaluation of bots’ believability in first person shooter games.
|S74||Effects of Electrical Pain
Stimuli on Immersion in Virtual Reality
Matthias Wölfel and Joey Schubert
The ultimate goal of virtual realty is to create a simulated world around us which is indistinguishable from the physical world as we know it. In such an environment our actions could lead to severe effects on our body. What would happen if one gets hit by a bullet, car or lightning? How would the felt pain change our perception of the virtual environment? It turns out that the influence of nociception (pain) on the human perception in virtual environments is not well covered in the scientific literature besides pain control/management. The goal of this publication is to investigate the influence of pain stimuli on immersion as well as decision making and to foster research and discussion in this direction.
Charcoal Cyber Arts
Yee Xin Chiew, Hock Soon Seah and Santiago E. Montesdeoca
In this paper, we present a stylization pipeline in 3D object space to emulate traditional charcoal drawing style in real time for cyber arts. First, we introduce an algorithm to produce a rough, grainy charcoal effect based on the lighting available in a 3D scene and the height map of a paper substrate. Then, to further refine the stylized result, we introduce several methods to reproduce some common techniques used in charcoal drawings, such as mixing, smudging and edge softening. The effects can be art-directed in real time to achieve sophisticated charcoal renders that reflect the aesthetic vision of the artist.
|S86||GPS Trail Visualizer for
Andreas Chrisna Mayong, Vajisha U. Wanniarachchi, Owen Noel Newton Fernando and May Oo Lwin
Due to the rising health issues and changes in people's daily routines, people became more health conscious over the past few decades. As a result, along with the rapid growth in mobile phones and emerging activity tracking devices, lots of mobile applications were introduced to track and encourage people's daily exercise routines. Among these applications, Global Position System (GPS) technology-based applications facilitate users to visualize their jogging/running trail in a 2-dimensional view on a google map and share it with friends. This paper suggests a methodology to enhance the visualization from 2-dimensional to 3-dimensional and more realistic visual interpretation by developing a Hyperlapse video using Google Street View Images.
|S97||Parallel 3D Skeleton
Extraction using Mesh Segmentation
Iason Manolas, Aris Lalos and Konstantinos Moustakas
There are several works performing accurate skeleton extraction, however, their main drawback is the extensive computational requirements and the lack of solutions that can be executed in multi-core computing systems. These challenges, become more demanding when we are dealing with dense 3D models. To cope with this scarcity we propose a novel method that extends a well known contraction-based skeletonization method, enabling its decentralization resulting in significant improvement in skeleton extraction times.
|Track on Cognitive Human-machine Interaction: Full Papers|
Computation Approach for Automatic Skeletal Motion Learning in Human
Francisco Calatayud, Luis de la Vega-Hazas and Andrés Iglesias
The realistic animation of human characters is a hot topic of research in computer graphics, with remarkable applications in computer animation and video games. A current trend is the application of powerful evolutionary computation techniques to meet the needs of increasing sophistication in the field. These techniques are inspired by the principles and mechanisms of biological evolution, such as selection, mutation, recombination, crossover, and so on. A great advantage of such methods is that they do not make any assumption on the problem to be solved. In this paper we present a new evolutionary computation approach for automatic skeletal motion learning in human animation. This approach is intended to generate automatically a sequence of motions on a human skeleton leading to plausible and realistic movements of the human body. This sequence is obtained autonomously (i.e. without human intervention) through an iterative intelligent process where the digital characters learn the optimal values of forces applied to selected bones according to the desired motion routines for proper movement. An illustrative example is discussed in detail to show the performance of this approach. This method can readily be adapted/extended to other skeleton configurations and interesting motions with only minor modifications.
|F19||Stable Feature Selection
for EEG-based Emotion Recognition
Zirui Lan, Olga Sourina, Lipo Wang, Yisi Liu, Reinhold Scherer and Gernot R. Müller-Putz
Affective brain-computer interface (aBCI) introduces personal affective factors into human-computer interactions, which could potentially enrich the user’s experience during the interaction with a computer. However, affective neural patterns are volatile even within the same subject. To maintain satisfactory emotion recognition accuracy, the state-of-the-art aBCIs mainly tailor the classifier to the subject-of-interest and require frequent re-calibrations for the classifier. In this paper, we demonstrate that the recognition accuracy of aBCIs deteriorates when re-calibration is ruled out during the long-term usage for the same subject. Then, we propose a stable feature selection method to choose the most stable affective features, for mitigating the accuracy deterioration to a lesser extent and maximizing the aBCI performance in the long run. We validate our method on a dataset comprising six subjects’ EEG data collected during two sessions per day for each subject for eight consecutive days.
|F20||EEG-based Cadets Training
and Performance Assessment System in Maritime Virtual Simulator
Yisi Liu, Zirui Lan, Olga Sourina, Hui Ping Liew, Gopala Krishnan, Dimitrios Konovessis and Hock Eng Ang
Deep investment in the maritime industries has led to many cutting edge technological advances in shipping navigation and operational safety to ensure safe and efficient logistical transportations. However, even with the best technology equipped onboard, maritime accidents are still occurring with at least three quarters of them attributed to human errors. Due to the rising need to address the human factors in shipping operations, various human factors studies are conducted in maritime domain. In this paper, an Electroencephalogram (EEG)-based cadets training and performance assessment system is proposed and implemented that could be used in the maritime virtual simulator. The system includes an EEG processing and analyses part and an evaluation part. It could recognize the brain states such as mental workload, emotions, and stress from raw EEG signal recorded during the exercises in the simulator and then give an indicative recommendation on “pass”, “retrain”, or “fail” of the cadet based on the EEG recognition results and input of the level of the task difficulty performed.
Activities in Perception of Shape-analogous English Letters Based on EEG
Rohit Bose, Sim Kuan Goh, Kian F Wong, Nitish Thakor, Anastasios Bezerianos and Junhua Li
Brain computer interface (BCI) technique has been demonstrated that human intentions or stimulus perception can be recognized using EEG signal recorded from the human scalp. When an intention is initiated in the brain or an external stimulus is perceived, the underlying relevant processing alters brain activity. This alteration in brain activity can be reflected in EEG signal. The intention or stimulus perception is therefore classified based on the alteration in brain activity. It might be difficult to classify brain activities in the perception of shape-analogous English letters because the similar shape could lead to less difference in brain activity. In order to explore classification feasibility and classification performance of shape-analogous letters using EEG signal, we performed an experiment of shape-analogous letter perception, in where participants perceived four letters (i.e., ‘p’, ‘q’, ‘b’ and ‘d’) while EEG signal was recorded. The F-score method was employed to assess the discriminative power for each feature, and a subgroup of features with high discriminative powers was then selected and fed into classifiers. Five classifiers (i.e., k-Nearest Neighbors (kNN), Support Vector Machine (SVM), Linear Discriminant Analysis (LDA), Random Forest (RF) and AdaBoost (ADA)), which are either pervasive or advanced in the field of machine learning, were utilized to classify brain activities in perception of shape-analogous letters. For each classifier, its parameters and the number of used features were optimized. Based on the performance comparison among the classifiers, Random Forest (RF) classifier achieved a maximal accuracy of 74.1%, but it was not statistically significantly better than the SVM. Our study demonstrated that brain activities in perception of shape-analogous English letters can be classified based on EEG signal and showed that random forest classifier outperformed other classifiers according to the results of comparison.
|F41||Computational Analysis of
Smile Weight Distribution across the Face for Accurate Distinction between
Genuine and Posed Smiles
Ahmad Al-dahoud and Hassan Ugail
In this paper, we report the results of our recent research into the understanding of the exact distribution of a smile across the face, especially the distinction in the weight distribution of a smile between a genuine and a posed smile. To do this, we have developed a computational framework for the analysis of the dynamic motion of various parts of the face during a facial expression, in particular, for the smile expression. The heart of our dynamic smile analysis framework is the use of optical flow intensity variation across the face during a smile. This can be utilised to efficiently map the dynamic motion of individual regions of the face such as the mouth, cheeks and areas around the eyes. Thus, through our computational framework, we infer the exact distribution of weights of the smile across the face. Further, through the utilisation of two publicly available datasets, namely the CK+ dataset with 83 subjects expressing posed smiles and the MUG dataset with 35 subjects expressing genuine smiles, we show there is a far greater activity or weight distribution around the regions of the eyes in the case of a genuine smile.
|F48||Sign Words Annotation
Assistance using Japanese Sign Language Words Recognition
Natsuki Takayama and Hiroki Takahashi
A Japanese sign language corpus is essential to activate analysis and recognition research of Japanese sign language. It requires collecting large scale of video data and annotating information to build a sign language corpus. Generally, building a sign language corpus is tedious work, and assistance is necessary. This paper describes one of the assistance methods for annotation tasks of sign words using Japanese sign language words recognition. The words recognition extracts sign features from a video, segments it into meaningful units, and annotates word labels to them automatically. At this time, the user’s annotation tasks can be reduced from the full-manual work to confirmation and correction of the annotation. The proposed sign words recognition is composed of body-parts tracking, feature extraction, and words classification. The five types of approaches including i) feature fusion and ii) multi- stream HMM to handle the multiple body-parts are applied and compared. We build a video database of Japanese sign language words and a manual annotation interface to evaluate the proposed method. The database includes 92 Japanese sign language words which are signed by ten native signers. The total number of videos is 4,590, and 3,900 videos of 78 words except for recording and sign errors are used for the evaluation. The classification accuracies were 75.88% and 93.35% in the signer and trial opened conditions, respectively, when the parts-based feature fusion and multi-stream HMM using relative weights for body-parts are employed. Moreover, the expected work reduction ratio of annotation tasks using the interface was 38.01%.
|F52||REVAM: a Virtual Reality
Application for Inducing Body Size Perception Modifications
Cédric Buche and Nathalie Le Bigot
In this paper, we present a 3D virtual environment for inducing body ownership illusion. The key idea is to take advantage of virtual reality potential regarding body size perception. The application, called REVAM, links two main aspects of body size perception: the first one focuses on its modification and the second one concerns its assessment. Based on some previous evidence that it is possible to modify body-size perception through an illusion of ownership over a virtual body, the application proposes to couple a tactile stimulation when viewing an avatar from a third person perspective (a condition known to produce this kind of illusion). In addition, the application offers the possibility to choose between avatars of different builds, and to perform morphing to reduce the avatars body. Moreover, the application allows to implicitly measure how people perceive their body size from an affordance estimation task in which people have to appreciate if they can pass through doors of different sizes without twisting their shoulders. To test the application we carried out an experiment on 16 female participants who performed the affordance estimation task five times: the first time before being exposed to their chosen avatar to get a baseline measure, and the four other times after being exposed to their avatar in different situations. These different situations are defined by the crossing of two experimental factors: morphing (presence or absence) and simultaneous visuotactile stimulation (presence or absence). A two-way repeated-measures Anova showed a main effect of the morphing: mean door width through which the participants estimate they can fit was significantly reduced (p < .05) when morphing was present. However, this effect did not interact with the presence of a simultaneous tactile stimulation. This indicates that exposing people to a virtual body reduced in size, as proposed in the present application, could be an effective way to modify body size perception, at least temporarily. The final goal would be to help patients with body image disorders, such as anorexia nervosa.
|F78||Neural Mechanisms of
Social Emotion Perception: An EEG Hyper-scanning Study
Li Zhu, Fabien Lotte, Gaochao Cui, Junhua Li, Changle Zhou and Andrezj Cichocki
EEG-based hyper-scanning refers to two or more subjects engaged in a task together or performing the same action together while neurophysiological signals are simultaneously recorded from them. This is one of the manners for investigating between-subject neural activities involved in social interactions. Emotion perception plays an important role in human social interactions. Interaction and emotional state influence each other. In this study, we aim to investigate how between-subject interaction modulates emotion perception based on event related potentials (ERPs), connectivity analysis and classification analysis. We found that there are distinct differences appearing between paired subjects who performed the task together, which are early ERP components (N250 and N400), late ERP components (P1500 and N1500), and the greater amplitude in N250 for the seconding responding subject compared to the first one. In the exploration of connectivity using phase locking value (PLV), we found that there are significant differences among different frequency bands for each subject under positive and negative stimuli and the significant difference of hyper-connectivity existed in the gamma frequency band between positive and negative stimulus trials. In the classification analysis, we compared the hyper-features for two individual subjects separately, the performance was improved when hyper-features of the PLV was employed compared to the features of power spectrum density.
|F82||Investigation on the
Correlation between Eye Movement and Reaction Time under Mental Fatigue
Vianney Renata, Fan Li, Ching-Hung Lee and Chun-Hsien Chen
With the recent development of eye tracking technology, research in eye movement and pattern has increased due to its potential to be a non-obstructive physiological measure tool. This study attempts to understand to which extent the eye behavior is relatable with human’s mental chronometry in responding to changes subjected to different levels of mental fatigue.
An analysis of the eye movement metrics when interacting with multiple short performance-based tasks under different states of mental fatigue is performed. It is concluded that the eye movement has influence in the resulting reaction time and the mental fatigue state of the individual. Thus, indicating the relationship as a strong potential to predict an individual’s mental fatigue state. Another finding is that the relationship between the eye movement metrics and mental chronometry becomes stronger as the subjective mental fatigue level increases.
|F100||Exactly Periodic Spatial
Filter For SSVEP Based BCIs
Kiran Kumar G.R. and Ramasubba Reddy M.
This study introduces a novel, high accuracy, calibration less spatial filter for reliable steady-state visual evoked potential (SSVEP) extraction from noisy electroencephalogram (EEG) data. The proposed method, exactly periodic subspace decomposition (EPSD), utilises the periodic properties of the SSVEP components to achieve a robust spatial filter for SSVEP extraction. It tries to extract the SSVEP components by projecting the EEG data onto a subspace where only the target signal components are retained. The performance of the method was tested on an SSVEP dataset obtained from ten subjects and compared with common SSVEP spatial filtering and detection techniques. The results obtained from the study shows that EPSD consistently provides a significant improvement in detection performance than other SSVEP spatial filters used in brain-computer interface (BCI) applications.
|F106||Powering Up Attentional
Focus: Validating a school-based deep breathing intervention with mobile
EEG—a pilot exploration
Khng Kiat Hui and Ravikiran Mane
Electrophysiological and neuroimaging data are important sources of information for validating the efficacy or effects of interventions. Many interventions for children are carried out in the schools especially if they are educationally relevant. However, factors such as high costs and physical constraints have typically limited the use of electrophysiological and neuroimaging tools to laboratory settings. Despite their reduced capabilities, the appearance of low-cost, quick-to-set-up mobile equipment in recent years have renewed the possibility of applying such techniques to monitor effects in school-based interventions. The current study explores the utility of a low-cost, mobile electroencephalography (EEG) headset system in detecting neurophysiological effects of a school-based deep breathing intervention, found in a previous behavioral study to be efficacious in reducing self-reported state anxiety and enhancing test performance in children. As part of a larger pilot study, EEG, respiration, and behavioral data were collected from a group of right-handed 11-year-olds as they performed a flanker task of attentional focus twice, once with a deep breathing intervention and once without. Results from power spectral analyses suggest that the low-cost, low- resolution, mobile EEG system is able to detect power spectra differences associated with flanker interference and intervention effects.
|Track on Cognitive Human-machine Interaction: Short Paper|
CSA-Based Approach for Machine-Driven Calculation of Outline Curves of
Akemi Gálvez and Andrés Iglesias
This paper addresses the problem of obtaining automatically a good approximation of the outline curve of skin lesions from dermoscopy images. This problem appears as a critical step in machine-driven segmentation of dermoscopy images for semi-automatic early diagnosis of cutaneous melanoma. Given a set of feature points selected by a specialist, the method applies a powerful nature-inspired metaheuristic optimization method called cuckoo search algorithm (CSA) to obtain the free-form parametric Be ́zier curve that fits the points better in the least-squares sense. Two illustrative examples of a benign and a malignant skin lesions (a naevus and a melanoma, respectively) are used to evaluate the performance of the method. Our experimental results show that the method performs very well and can be applied as a intermediate step of semi-automatic image segmentation for early diagnosis of cutaneous melanoma.
|S21||EEG-based Evaluation of
Mental Fatigue Using Machine Learning Algorithms
Yisi Liu, Zirui Lan, Han Hua Glenn Khoo, King Ho Holden Li, Olga Sourina and Wolfgang Mueller-Wittig
When people are exhausted both physically and mentally from overexertion, they experience fatigue. Fatigue can lead to a decrease in motivation and vigilance which may result in certain accidents or injuries. It is crucial to monitor fatigue in workplace for safety reasons and well-being of the workers. In this paper, Electroencephalogram (EEG)-based evaluation of mental fatigue is investigated using the state-of-the-art machine learning algorithms. An experiment lasted around 2 hours and 30 minutes was designed and carried out to induce four levels of fatigue and collect EEG data from seven subjects. The results show that for subject-dependent 4-level fatigue recognition, the best average accuracy of 93.45% was achieved by using 6 statistical features with a linear SVM classifier. With subject-independent approach, the best average accuracy of 39.80% for 4 levels was achieved by using fractal dimension, 6 statistical features and a linear discriminant analysis classifier. The EEG-based fatigue recognition has the potential to be used in workplace such as cranes to monitor the fatigue of operators who are often subjected to long working hours with heavy workloads.
|S32||Designing a Digital
Fitness Game System for Older Adults in Community Settings
Jinhui Li, Mojisola Erdt, James Chong Boi Lee, Harsha Vijayakumar, Caroline Robert and Yin-Leng Theng
Exergames is one of the new innovative approaches used in primary healthcare programmes. The current study introduces a fitness game system, HOCAMOSE-VETS, which includes digital rehabilitation exercises and exergames designed with a particular focus on older adults. Besides the new exergames, the system also allows care staff to actively schedule, monitor and assess the progress of older adults’ exercise activities. A survey-based study was conducted to investigate the overall user acceptance of the digital fitness game system. We found that users’ perceived ease of use and usefulness have a significant impact on their actual intention of using the game system. However, the output quality of the system is not significantly associated with users’ perception of the usefulness and ease of use of the system. The findings from this research have provided new insights into designing elderly fitness games.
|S46||Cross Dataset Workload
Classification Using Encoded Wavelet Decomposition Features
Wei Lun Lim, Olga Sourina and Lipo Wang
For practical applications, it is desirable for a trained classification system to be independent of task and/or subject. In this study, we show one-way transfer between two independent EEG workload datasets: from a large multitasking dataset with 48 subjects to a second Stroop test dataset with 18 subjects. This was achieved with a classification system trained using sparse encoded representations of the decomposed wavelets in the alpha, beta and theta power bands, which learnt a feature representation that outperformed benchmark power spectral density features by 3.5%. We also explore the possibility of enhancing performance with the utilization of domain adaptation techniques using transfer component analysis (TCA), obtaining 30.0% classification accuracy for a 4-class cross dataset problem.
|S61||A Visual Keyboard System
using Hybrid Dual Frequency SSVEP Based Brain Computer Interface with VOG
Saravanakumar D. and Ramasubba Reddy M.
The focus of this paper is to increase the number of targets and classification rate in the SSVEP-BCI visual keyboard system. The dual frequency steady state visual evoked potential (SSVEP) and video-oculography (VOG) based hybrid system has been developed in this study. The visual stimuli (targets) were designed using dual frequency SSVEP method. This method could create more targets through a suitable combination of limited number of frequencies. The keyboard screen was divided into three sections (left, middle and right), and each section visual stimuli/keys were designed with a unique set of frequencies. The webcam based video-oculography was used to detect the direction of the eye gaze. This selection reduces the issue of misclassification of SSVEP frequencies. Extended multivariate synchronization index (EMSI) method is used for SSVEP frequency recognition. Both online and offline experiments were conducted on 10 subjects and an average online detection accuracy of 94.987% was obtained with the information transfer rate (ITR) of 82.786 bits/minutes.
|S89||A Novel Visual Keyboard
System for Disabled People/Individuals using Hybrid SSVEP Based Brain
Saravanakumar D. and Ramasubba Reddy M.
This paper aims to design a new stimulus paradigm for SSVEP based keyboard system. The proposed paradigm was implemented using black and white checkerboard flickering visual stimuli along with the integration of video-oculography (VOG). The on-screen speller was designed using three frequencies. The goal of this study is how to increase more number of targets using less number of stimulus frequencies. It is achieved by the use of VOG data. The study was carried out using 36 selected characters. A webcam is integrated along with the system to obtain VOG data. The webcam captures the images of the eyes, which in turn is used to detect the eye gaze direction. This additional information from VOG overcomes the limitations of SSVEP based spelling system. The extended multivariate synchronization index (EMSI) method is used for SSVEP frequency recognition. Offline and online analysis of the experiment were conducted and the duration of recognition of each character required by the participant was calculated based on the classification accuracy. Online experiment was conducted on 10 subjects to validate the accuracy and information transfer rate (ITR) of the system. An average online detection accuracy of 90.46 % was obtained with the ITR of 65.98 bits/minutes.
|S90||Promoting Healthy and
Active Ageing Through Exergames: Effects of Exergames on Senior Adults’
Chen Li, Jinhui Li, Tan Pham Phat, Yin-Leng Theng, and Bing Xun Chia
Exercise games (exergames) are defined as the combination of exercise and interactive fitness games. The previous pilot study has found positive influences of exergaming on the psychological well-being of senior adults, and this study aims to further investigate the effects of exergaming playing on the elderly’s motivation and attitude towards exergames, psychosocial well-being (sociability and loneliness), and inter- generational perception. A 2 (pre-test vs. post-test) X 3 (play alone vs. play with elderly vs. play with youths) mixed quasi-experiment was conducted (N=317) in Singapore. Over 6 weeks’ playing time, the elderly’s attitude towards exergames, their perception towards youth, and sociability significantly increased. The elderly’s level of loneliness significantly decreased over 6 weeks, Exergaming could be considered as a way of promoting healthy and active ageing.
|S95||Prediction of Negative
Symptoms of Schizophrenia from Objective Linguistic, Acoustic and Non-verbal
Conversational Cues. Debsubhra
Chakraborty, Shihao Xu, Zixu Yang, Victoria Chua,
Yasir Tahir, Justin Dauwels, Nadia Magnenat Thalmann, Bhing-Leet Tan and
Speech disorders are among the salient characteristics of negative symptoms of schizophrenia. Such impairments are often exhibited through disorganized speech, inappropriate affective prosody, and poverty of speech. The current method of detecting such symptoms requires the expertise of a trained clinician, which may be prohibitive due to cost, stigma or high patient-to-clinician ratio. An objective method to extract non- verbal and verbal speech-related cues can help to automate and simplify the assessment method of severity of speech-related symptoms of schizophrenia. In this paper, a novel automated method is presented which uses speech content from schizophrenic patients to predict the clinician-assigned subjective ratings of their negative symptoms. Specifically, the interviews of 50 schizophrenia patients were recorded and features related to acoustics, linguistics and non-verbal conversation were extracted. The subjective ratings can be accurately predicted from the objective features with an accuracy of 64-82% using machine learning algorithms with leave-one-out cross-validation. Our findings support the utility of automated speech analysis to aid clinician diagnosis, monitoring and understanding of schizophrenia.
|S99||Personas and Emotional
Design for Public Service Robots: A Case Study with Autonomous Vehicles in
Penny Kong, Henriette Cornet and Fritz Frenkler
Emerging technologies for future mobility will drastically change the way humans interact with machines and the environment. The common denominator in technologies such as autonomous vehicles (AVs) and artificial intelligence is the absence of the human, which can be addressed with a service robot designed to appeal to human emotion. As service robots tend to operate in environments where there is a diversity of users and thus user requirements, there lies a gap in the definition of how these interactions should be designed. This paper discusses the use of personas in the development of service robots for multi-stakeholder environments through a case study on AVs for public transportation in Singapore.
|S105||Predicting Ordinal Level
of Sedation from the Spectrogram of Electroencephalography
Haoqi Sun, Sunil B. Nagaraj and M. Brandon Westover
In Intensive Care Unit, the sedation level of patients is usually monitored by periodically assessing the behavioral response to stimuli. However, these clinical assessments are limited due to the disruption with patients’ sleep and the noise of observing behaviors instead of the brain activity directly. Here we train a Gated Recurrent Unit using the spectrogram of electroencephalography (EEG) based on 166 mechanically ventilated patients to predict the Richmond Agitation-Sedation Score, scored as ordinal levels of -5, -4, ... up to 0. The model is able to predict 50% accurate with an error not larger than 1 level; and 80% accurate with an error not larger than 2 levels on hold-out testing patients. We show typical spectrograms in each sedation level and interpret the results based on the visualization of the gradient with respect to the spectrogram. Future improvements include utilizing the EEG waveforms since waveform patterns are clinically thought to be associated with sedation levels, as well as training patient-specific models.
|S118||Improved User Interface
for a Virtual Integrated Therapy for Active Living (VITAL) – Health
Box: An Elderly Perspective
Bing Xun Chia, Chuan Cheng, May Thet Hnin, Zwe Marn Tun Lwin, Tan Phat Pham, Quoc Nam Tran Nguyen and Yin-Leng Theng
Most technology designs are oriented to the younger population with a lack of attention from an elderly perspective. This paper aims to gather user needs and requirements from the elderly and propose a user-friendly interface for a Virtual Integrated Therapy for Active Living (VITAL) – Health Box for the elderly population. Utilising usability evaluation method, a focus group discussion was conducted with 10 participants to gather user needs and preferences about user interface design for VITAL Health Box. Participants were asked questions relating to content, preference and needs based on a discussion guide. A set of improved parameters and user interface was proposed. Major consideration aspects are UI design and functionalities with optimisation for elderly people, such as providing clear and simple instruction information, giving multi-language choices, displaying text and suitable graphic or icon to reduce the confusion or frustration to the elderly users. The findings of this research provide insight for designing an elderly-friendly user interface.
|Track on Cybersecurity and Biometrics: Full Papers|
|F75||A 3D Approach for the
Visualization of Network Intrusion Detection Data
Wei Zong, Yang-Wai Chow and Willy Susilo
With the increasing threat of cyber attacks, machine learning techniques have been researched extensively in the area of network intrusion detection. Such techniques can potentially provide a means for the real-time automated detection of attacks and abnormal traffic patterns. However, misclassification is a common problem in machine learning techniques for intrusion detection, and a lack of insight into why such misclassification occurs impedes the improvement of machine learning models. This paper presents an approach to visualizing network intrusion detection data in 3D. The purpose of this is to facilitate the understanding of network intrusion detection datasets using a visual representation to reflect the geometric relationship between various categories of network traffic. This can potentially provide useful insight to aid the design of machine learning techniques. This paper demonstrates the usefulness of the proposed 3D visualization approach by presenting results of experiments on commonly used network intrusion detection datasets.
|F81||Enhancing the Security of
Transformation Based Biometric Template Protection Schemes
Loubna Ghammam, Morgan Barbier and Christophe Rosenberger
Template protection is a crucial issue in biometrics. Many algorithms have been proposed in the literature among secure computing approaches, crypto-biometric algorithm and feature transformation schemes. The BioHashing algorithm belongs to this last category and has very interesting properties. Among them, we can cite its genericity since it could be applied on any biometric modality, the possible cancelability of the generated BioCode and its efficiency when the secret is not stolen by an impostor. Its main drawback is its weakness face to a combined attack (zero effort with the stolen secret scenario). In this paper, we propose a transformation-based biometric template protection scheme as an improvement of the BioHashing algorithm where the projection matrix is generated by combining the secret and the biometric data. Experimental results on two biometric modalities, namely digital fingerprint and finger knuckle print images, show the benefits of the proposed method face to attacks while keeping a good efficiency.
Patrick Bours and Thilo Denzer
Gait authentication using a mobile phone’s acceleration sensor offers a convenient, user-friendly and subtle procedure of authenticating individuals to their mobile phone. This study analyses the possibility of cross-pocket gait recognition, which means creating the reference and the probe with the accelerometer sensor in different trouser pockets (left and right). The results of our analysis show that there is a significant performance degradation when comparing same-pocket gait recognition with cross-pocket gait recognition. In our analysis we have used a new distance metric that shows to give good (same-pocket) performance results compared to known analysis methods. We have also shown that a multi-reference template can give excellent performance without any performance degradation for cross-pocket gait recognition.
|F92||User Dependent Template
Update for Keystroke Dynamics Recognition
Abir Mhenni, Estelle Cherrier, Christophe Rosenberger and Najoua Essoukri Ben Amara
Regarding the fact that individuals have different interactions with biometric authentication systems, several techniques have been developed in the literature to model different users categories. Doddington Zoo is a concept of categorizing users behaviors into animal groups to reflect their characteristics with respect to biometric systems. This concept was developed for different biometric modalities including keystroke dynamics. The present study extends this biometric classification, by proposing a novel adaptive strategy based on the Doddinghton Zoo, for the recognition of the user’s keystroke dynamics. The obtained results demonstrate competitive performances on significant keystroke dynamics datasets.
|Track on Cybersecurity and Biometrics: Short Papers|
|S36||A New Black Box Evaluation
Protocol for Biometric Systems
Antoine Cabana, Christophe Charrier and Alain Louis
As a trending method for the authentication, biometrics tends to be integrated in various devices, and in particular in smartphones. If the evaluation is performed on operational device, the biometric sample and algorithm are not reachable by the assessors. So, these latter have to perform an evaluation on a system considered as a black box. This kind of evaluation implies numerous manual comparison.
This paper proposes a methodology to perform an evaluation of biometric black boxes. Two preliminary experiments were performed in order to determine an optimized conduct. This paper describes the used methodology to perform evaluation on black boxes systems, and the results obtained on the systems under test.
|S42||Experiments on Deep Face
Recognition using Partial Faces
Ali Elmahmudi and Hassan Ugail
Face recognition is a very current subject of great interest in the area of visual computing. In the past, numerous face recognition and authentication approaches have been proposed, though the great majority of them use full frontal faces both for training machine learning algorithms and for measuring the recognition rates. In this paper, we discuss some novel experiments to test the performance of machine learning, especially the performance of deep learning, using partial faces as training and recognition cues. Thus, this study sharply differs from the common approaches of using the full face for recognition tasks. In particular, we study the rate of recognition subject to the various parts of the face such as the eyes, mouth, nose and the forehead. In this study, we use a convolutional neural network based architecture along with the pre-trained VGG-Face model to extract features for training. We then use two classifiers namely the cosine similarity and the linear support vector machine to test the recognition rates. We ran our experiments on the Brazilian FEI dataset consisting of 200 subjects. Our results show that the cheek of the face has the lowest recognition rate with 15% while the (top, bottom and right) half and the 3/4 of the face have near 100% recognition rates.
|S66||Kinect vs Lytro in RGB-D
Valeria Chiesa and Jean-Luc Dugelay
Light field cameras are becoming increasingly popular thanks to higher capabilities with respect to regular cameras in capturing information of a scene. Even though the principle associated with structured light sensors is quite different from the technology behind light field cameras, data provided by these technologies are similar in terms of depth map. With the aim of comparing the potential of Kinect and Lytro sensors on face recognition, two experiments are conducted on separate but publically available datasets and validated on a database acquired simultaneously with Lytro Illum camera and Kinect V1 sensor. The results obtained on RGB and depth maps are integrated with an experiment based on fusion at score level. The introduction of depth information in the RGB data is found more effective than standard bi dimensional imaging, especially in case of occlusions.
|S73||RHU Keystroke Touchscreen
Mohamad El-Abed, Mostafa Dafer and Christophe Rosenberger
Biometric systems are currently widely used in many applications to control and verify individual’s identity. Keystroke dynamics modality has been shown as a promising solution that would be used in many applications such as e-payment and banking applications. However, such systems suffer from several performance limitations (such as cross-devices problem) that prevent their widespread of use in real applications. The objective of this paper is to provide researchers and developers with a public touchscreen-based benchmark collected using a mobile phone and a tablet (both portrait and landscape orientation each). Such a benchmark can be used to assess keystroke-based matching algorithms. Furthermore, It is mainly developed to measure the robustness of keystroke matching algorithms vis-a`-vis cross-devices and orientation variations. An online visualizer for the database is also given to researchers allowing them to visualize the acquired keystroke signals.
|S80||Analysis of Keystroke
Dynamics For the Generation of Synthetic Datasets
Denis Migdal and Christophe Rosenberger
Biometrics is an emerging technology more and more present in our daily life. However, building biometric systems requires a large amount of data that may be difficult to collect. Collecting such sensitive data is also very time consuming and constrained, s.a. GDPR legislation. In the case of keystroke dynamics, existing databases have less than 200 users. For these reasons, we aim at generating a keystroke dynamics synthetic dataset. This paper presents the generation of keystroke data from known users as a first step towards the generation of synthetic datasets, and could also be used to impersonate users’ identity.
|S96||A Client based Anomaly
Traffic Detection and Blocking Mechanism by Monitoring DNS Name Resolution
With User Alerting Feature
Yong Jin, Kunitaka Kakoi, Nariyoshi Yamai, Naoya Kitagawa and Masahiko Tomoishi
Malware has become one of the most critical targets of network security solutions nowadays. Many types of malware receive further instructions from the C&C servers and the attack targets may be instructed by IP addresses which causes direct attacks without DNS name resolution from the malware-infected computers. In the meanwhile, several programs that are hidden from the users (e.g., malware, virus, etc.) may perform DNS name resolutions for cyber attacks or other communications. In this paper, we propose a client based anomaly traffic detection and blocking mechanism by monitoring DNS name resolution per application program. In the proposed mechanism, by the collaboration of DNS proxy and packet filter, DNS traffic is monitored on the client and the traffic destined to the IP addresses obtained without DNS name resolution or the traffic from unrecognized programs will be detected and blocked. In addition, in order to mitigate false positive detection, an alert-window will be shown to let the users decide whether to allow the traffic or not. We implemented a prototype system on a Windows 7 client and confirmed that the proposed mechanism worked as expected.
|Track on Cyber Cities and Cyber Manufacturing: Full Papers|
|F9||Towards Automatic Optical
Inspection of Soldering Defects
Wenting Dai, Abdul Mujeeb, Marius Erdt and Alexei Sourin
This paper proposes a method for automatic image-based classification of solder joint defects in the context of Automatic Optical Inspection (AOI) of Printed Circuit Boards (PCBs). Machine learning-based approaches are frequently used for image-based inspection. However, a main challenge is to manually create sufficiently large labeled training databases to allow for high accuracy of defect detection. Creating such large training databases is time-consuming, expensive, and often unfeasible in industrial production settings. In order to address this problem, an active learning framework is proposed which starts with only a small labeled subset of training data. The labeled dataset is then enlarged step-by-step by combining K-means clustering with active user input to provide representative samples for the training of an SVM classifier. Evaluations on two databases with insufficient and shifting solder joints samples have shown that the proposed method achieved high accuracy while requiring only minimal user input. The results also demonstrated that the proposed method outperforms random and representative sampling by ~ 3.2% and ~ 2.7%, respectively, and it outperforms the uncertainty sampling method by ~ 0.5%.
Defect Detection Using Deep Autoencoders and Data Augmentation
Abdul Mujeeb, Wenting Dai, Marius Erdt and Alexei Sourin
Surface level defect detection, such as detecting missing components, misalignments and physical damages, is an important step in any manufacturing process. In this paper, similarity matching techniques for manufacturing defect detection are discussed. We are proposing an algorithm which detects surface level defects without relying on the availability of defect samples for training. Furthermore, we are also proposing a method which works when only one or a few reference images are available. It implements a deep autoencoder network and trains input reference image(s) along with various copies automatically generated by data augmentation. The trained network is then able to generate a descriptor—a unique signature of the reference image. After training, a test image of the same product is sent to the trained network to generate a test image descriptor. By matching the reference and test descriptors, a similarity score is generated which indicates if a defect is found. Our experiments show that this approach is more generic than traditional hand-engineered feature extraction methods and it can be applied to detect multiple type of defects.
Method for Converting Polygon Soup Building Models for 3D Geospatial
Kan Chen, Henry Johan and Marius Erdt
Polygon soup building models are fine for visualization purposes such as in games and movies. They, however, are not suitable for 3D geospatial applications which require geometrical analysis, since they lack connectivity information and may contain intersections internally between their parts. In this paper, we propose an appearance-driven method to interactively convert an input polygon soup building model to a two-manifold mesh, which is more suitable for 3D geospatial applications. Since a polygon soup model is not suitable for geometrical analysis, our key idea is to extract and utilize the visual appearance of the input building model for the conversion. We extract the silhouettes and use them to identify the features of the building. We then generate horizontal cross sections based on the locations of the features and then reconstruct the building by connecting two neighbouring cross sections. We propose to integrate various rasterization techniques to facilitate the conversion. Experimental results show the effectiveness of the proposed method.
|Track on Cyber Cities and Cyber Manufacturing: Short Papers|
|S54||Securing Spatial Data
Infrastructures in the Context of Smart Cities
Kanishk Chaturvedi, Andreas Matheus, Son H. Nguyen and Thomas H. Kolbe
Spatial Data Infrastructures play a very important role in linking and integrating various distributed systems in smart city applications. One such concept called Smart District Data Infrastructure (SDDI) is already being implemented in different districts of European cities, which allows managing various actors, stakeholders, sensors, simulation tools and semantic 3D city models within one common operational framework. Such distributed systems involve open data sources belonging to different platforms. On the other side, there are various users and applications who want to access and work on all these systems in convenient ways using single sign-on. If not secured, it may cause a major threat to disclose sensitive information to untrusted or unauthorized entities. This paper presents a novel implementation approach of securing distributed components of the SDDI in the district Queen Elizabeth Olympic Park in London. It establishes proper authorization and authentication to allow privacy, security and controlled access to all stakeholders and the respective components. The implementation combines the use of state-of-the-art concepts such as OAuth2 access tokens, OpenID Connect user claims and Security Assertion Markup Language (SAML) based Single-Sign-On (SSO) authentication.
|S68||Using Mobile Phone Data to
Determine Human Mobility Patterns in Paris
Eric Valega Prawirodidjojo, Rui Jie Quek, Bu–Sung Lee, Vincent Gauthier and Markus Schläpfer
With the rapid expansion of cities around the world, large number of movements are made daily as people commute from their homes to their destinations, including workplaces. From these movements, trends and patterns can be derived which in turn, can provide valuable insights for urban planning. This is particularly relevant in the ‘smart cities’ context. However, such movement data are often difficult to gather and analyse without infringing on privacy rights, especially with the increasing concerns on privacy issues. This paper reports on the use of aggregated mobile phone tracking data together with train network data to analyse movement patterns in, out, and within La Défense (Paris’ business district). The findings can assist city planners by providing a better understanding of people’s travel patterns.
Streaming and Loading of 3D Scene
Budianto Tandianus, Hock Soon Seah, Tuan Dat Vu and Anh Tú Phan
In this paper, we present an approach for out-of-core dynamic streaming of a virtual scene. We use the traditional client/server architecture, where the main responsibility of the client application is visualization and interaction, and the main responsibility of the server is accepting and serving requests from the client application. The client will query geometry in the camera proximity to the server and the server will stream geometry (in CityGML format) to the client application. We implement the client application using Unity game engine. Performance comparison between traditional loading and our dynamic streaming are provided. We also show the scalability advantage of our work.
|P109||The Role of Wearable
Technology in Children’s Creativity
In primary and high school settings across the world, wearable technologies are becoming more and more common. Augmented reality, is another increasingly common technology, which when combined with wearables, has the potential to benefit student learning in different settings. This combination can potentially be used to support problem-solving and creativity for children, a relatively unexplored area. In this work, an initial expiration is described about the contextual variables that can impact the levels of low to high creativity moments. Our goal is to use wearables and sensors to further minimize the gap between the topic of children’s creativity by measuring moments of creative fixation and providing feedback in moments of low creativity to encourage creativity. The primary contributions of this work, are a preliminary pilot study with ten primary-school students (K-6), a presentation of our initial results and finally, an examination and discussion of how creativity in children can be supported or enhanced by combining wearable technology and augmented reality.
|P110||Deep Learning with Long
Short-term Memory Recurrent Neural Network for Daily Container Volumes of
Storage Yard Predictions in Port. Yinping Gao, Daofang Chang, Chun-Hsien Chen and Ting Fang
With the development of China’s Belt and Road Initiative (BRI), the port plays a significant role and its operation management faces some pressure. In this regard, prediction of daily container volumes will provide the manager with data support for better plan of a storage yard. In this work, by deep learning the historical dataset, the long short-term memory (LSTM) recurrent neural network (RNN) is trained and used to predict daily volumes of containers which will enter the storage yard. The raw dataset of a certain port from 2013 to 2016 is chosen as the training set and the dataset of 2017 is used as the test set to evaluate the performance of the proposed prediction model. Then the LSTM model is established with Python and Tensorflow framework. The structure parameters are adjusted to find the optimal LSTM network, so as to improve the prediction accuracy. It appears that the LSTM model with two hidden layers and 30 hidden layer units has less prediction error between the real data and predicted data of 2017. The prediction error of daily container volumes between predicted value and real data of 2017 is about 12.39%, which is less than the people-predicted error. It is promising that the proposed LSTM RNN model can be applied to predict the daily volumes of containers and have higher prediction accuracy.
|P111||The Behavior Symptoms of
Undergraduates’ Social Anxiety in the Virtual World
Yungang Wei, Lin Huang, Wei Wang, Yanqiu Zhang and Zihan Wang
Nowadays, the use of virtual world is increasingly popularized, as coding and analyzing behaviors in the virtual world is more rapid and accurate. The contemporary undergraduates suffer from severe social anxiety. To cope, the paper takes the undergraduates' social anxiety as the subject for research and uses the virtual world to collect the data of specific behavior to predict the social anxiety in the virtual world, finding that the quantity of windows, turn frequency and clothing fitness in the virtual world all can significantly predict the social anxiety yet the measurement of psychological characteristics such as social anxiety in the virtual world needs further study.
|P112||Effects of Sound Volume
Change When Squeezing a Virtual Soft Object with a Bare Hand
Mie Sato, Zentaro Kimura, Yuki Tanaka, Natsumi Motoura, Naoki Hashimoto and Arie E. Kaufman
In order to improve the perception of the core part of a virtual soft object when a user squeezes it with his/her bare hand, we apply multisensory integration of visual and auditory stimuli to our proposed AR system. The visual stimuli are real-time stereoscopic images in which the user’s bare hand is squeezing the virtual soft object in the actual scene. As the auditory stimuli, we focus on the sound volume change linked to the movements of the user’s thumb and index finger. The present paper reports the effects of sound volume change in enhancing the pseudo-softness and the perception of the core part of a virtual soft object. The experimental results of our study statistically show that when a user squeezes a virtual soft object with his/her bare hand, multisensory integration of the visual and auditory stimuli effectively increases the feeling of grasping and facilitates handling of the virtual soft object. In addition, auditory stimuli that have a clearly audible sound volume change at the core part naturally enhance the perception of the core part of the virtual soft object.
|P115||Outliers Removal of Highly
Dense and Unorganized Point Clouds Acquired by Laser Scanners in Urban
Gerasimos Arvanitis, Aris S. Lalos, Konstantinos Moustakas and Nikos Fakotakis
Recently, there is a tremendous interest in the processing of unorganized point clouds, generated using a variety of 3D scanning technologies such as structured light and LIDAR systems. Without a doubt, the most compelling problem in this domain is the removal of outliers. To effectively address the aforementioned issue, we present a novel method, that detects accurately and efficiently the outliers by exploiting the spatial coherence in the object geometry and the sparsity of the outliers in the spatial domain. This is achieved by solving a convenient convex method called Robust PCA (RPCA). To demonstrate the effectiveness of the proposed technique, we evaluate it by using real scanned point clouds which are extremely dense consisting of millions of points.
|P116||Real-time Haptic Rendering
of Double-points Interaction
Xinli Wu, Wenzhen Yang, Minxiong Zhang, Xin Huang, Xuxiao Wu and Zhigeng Pan
Force generation will improve the immersion and authenticity of the virtual environment with human computer interaction. Considering the limitations of single-point interaction with force generation, this paper proposes real-time force generation methods based on double-points interaction. Analyzing the double-points interaction behaviors, we propose different equations to calculate the interactive force under four different interaction states. We establish an experimental environment to test the efficiency of double-points haptic interaction. The evaluation results show that the method can generate interactive force in time, enhance the immersion and authenticity of the virtual environment, and improve the naturalness of human computer interaction.
Cyberworlds for Environmental Monitoring
Maria Krommyda, Evangelos Sdongos, Stefano Tamascelli, Athanasia Tsertou, Geli Latsa and Angelos Amditis
ICT advances in emerging domains such as Internet of Things (IoT), Augmented Reality/ Virtual Reality (AR/VR), big data analytics, cyber-physical systems and cloud computing have revolutionized and boosted the creation of cyberworlds as information spaces that allow us to augment the way we interact with each other and with the physical world. Naturally, other than businesses cyberworlds can benefit modern hyper connected societies at their entirety (transport, mobility, health, smart living, etc.) and further to that the physical world around us can also be part of such process. In the present paper the focus is given on citizen-powered cyberworlds for Environmental Monitoring which are created from crowdsourced observations engaging, through gamification, citizens and communities. The means of engagement include serious gaming, collection of geo-tagged IoT, such as images, video and sensor measurements as well as management and storage of diverse IoT as OGC compliant observations all conveyed into a dedicated information space.
|P120||Are Online Co-Adaptive
Sensorimotor Rhythm Brain-Computer Interface Training Paradigms Effective?
José Diogo Cunha and Reinhod Scherer
Operating a non-invasive electroencephalogram (EEG) based sensorimotor rhythm brain-computer interface (BCI) is a skill that typically requires extensive training. Lately, online co-adaptive feedback training approaches achieved promising results. Does this also mean that users can have meaningful BCI-based interactions after training?
To answer this question an online study was conducted with 10 na ̈ıve (first time) users. The users trained to gain BCI control by playing a Whack-A-Mole game for about 30 minutes. During this time BCI parameters were adapting to the users EEG patterns. The adaptation was then stopped and users continued playing the game with the trained BCI for another 20 minutes. Eight out of the ten users were able to control the BCI and play the game. These preliminary results seem to suggest that online co-adaptation is an effective way to gain BCI control.
Observation of Hidden Physical Quantities in Occupational Therapy. Alberto Fornaser, Mariolino De Cecco,
Paolo Tomasin, Matteo Zanetti, Giovanni Guandalini, Barbara Gasperini, Patrizia Ianes, Francesco Pilla and Rossella Ghensi
The human being is used to the management of those physical quantities that can be sensed by his/her five senses. The technology has grown following the constraint that tries to provide to the user senses interfaces that result natural and intuitive. With the born and the spread of virtual worlds, thanks to the Virtual and Augmented Reality revolution, such paradigm has enlarged comprising: (i) virtual parameters, (ii) directly sensed parameters, and (iii) indirectly sensed ones. The spread of sensing, connectivity, and intelligence in ubiquitous computing can occur using any device, in any location, and in any format and then fed back to the user has opened the problem/opportunity of how to synthesize the great amount of indirect and direct amount of data in a, still, natural way. This paper focuses on the development of an interface for data fruition by a physician or an occupational therapist in the context of the remote monitoring of disabled subjects that try different technological aids in a home scenario. The objective is the fruition of complex datasets in the most natural and intuitive way.
Cyberspace of Scientific Papers Based on WebVR Technology
Michael Charnine, Konstantin Kuznetsov and Oleg Zolotarev
In this paper we describe a cyberspace of scientific papers in which the most cited and significant documents are represented by large spheres. The distance between documents is proportional to their semantic similarity regardless of language. The measure of semantic similarity of multilingual documents is proposed. This measure is determined by the maximum correlation between explicit and implicit connectivity of the documents. The cyberspace is built entirely automatically from collection of texts using methods: t-SNE, SCCI and A-Frame. The proposed cyberspace implemented by WebVR and interactive 3D graphics can be considered as a dynamic learning environment that is convenient for discovering new significant articles, ideas and trends. The cyberspace is a powerful tool of information integration and it allows you to visualize documents of different languages in a single space.
|P123||Fatigue Prediction and
Intervention for Continuous Play in Video Games
Thanat Damrongwatanapokin and Koji Mikami
Recently, there are many video games that keep relatively high difficulty for an extended period of time. However, the game with high level of challenges will induce more frustration and tiredness causing players to take more short breaks than usual. While mental fatigue has been studied widely, there are not many game studies and applications related to mental fatigue. One of the possible applications that we want to explore is artificial insertion of an interval serving as a break for players before they get fatigue and take a break from playing games. We plan to use Electroencephalogram (EEG) for fatigue monitoring and machine learning approaches to help predict the right time for intervention. Currently, we have created a prototype game to be used in the experiment and collect sample data for the machine learning.