基于脑功能网络的虚拟现实晕动症检测
作者:
作者单位:

南京信息工程大学自动化学院

基金项目:

国家自然科学基金(62206130);江苏省自然科技计划(BK20200821);南京信息工程大学科研启动经费(2020r075);江苏高校教育信息化研究课题(2023JSETKT032)


Virtual Reality Sickness Detection Based on Brain Functional Networks
Author:
Affiliation:

School of Automation,Nanjing University of Information Science Technology

  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献
  • | |
  • 引证文献
  • | |
  • 文章评论
    摘要:

    基于对不同眩晕状态下的脑电信号(EEG)解码提出有效的检测方案,有助于研究虚拟现实晕动症的缓解方法。本文采用多元变分模态分解将EEG划分为五个频段,并根据晕动症量表结果将数据划分为不同眩晕状态组,利用PLV方法计算EEG频段内和频段间的功能连接以构建超邻接矩阵,并基于SVM和CNN等模型进行分类识别。研究结果显示,聚类系数、局部效率和加权节点度三种具有显著性差异的拓扑特征融合后,在眩晕vs.非眩晕,高眩晕vs.低眩晕两个任务中的最高平均分类准确率分别为91.70%和96.00%。此外,本文还将超邻接矩阵直接输入CNN模型,在两个任务中得到的平均分类准确率分别达到93.40%和98.50%。结果表明本研究所提方法可用于虚拟现实晕动症的检测,并为进一步研究晕动症对各脑区功能耦合的影响提供了参考。

    Abstract:

    An effective detection scheme based on decoding electroencephalogram (EEG) signals under different states of sickness is proposed, which is helpful for studying methods to alleviate virtual reality motion sickness (VRMS). This article uses multivariate variational mode decomposition (MVMD) to divide EEG into five frequency bands and divides the data into different sickness state groups based on the results of the motion sickness scale. The phase-locking value (PLV) method is used to calculate the functional connections within and between EEG frequency bands to construct a super adjacent matrix, and classification recognition is performed based on models such as support vector machine (SVM) and convolutional neural network (CNN). The research results show that the fusion of three topological features with significant differences in clustering coefficient, local efficiency, and weighted node degree results in sickness vs Non sickness, high sickness vs The highest average classification accuracy in the two tasks of low sickness was 91.70% and 96.00%, respectively. In addition, this article also directly inputs the super adjacency matrix into the CNN model, achieving average classification accuracies of 93.40% and 98.50% in two tasks, respectively. The results indicate that the method proposed in this study can be used for the detection of motion sickness in virtual reality and provide reference for further research on the impact of motion sickness on the functional coupling of various brain regions.

    参考文献
    Malone S, Brünken R. Hazard perception, presence, and simulation sickness—a comparison of desktop and head-mounted display for driving simulation[J]. Frontiers in psychology, 2021, 12: 647723.
    Hua C, Chai L, Yan Y, et al. Assessment of virtual reality motion sickness severity based on EEG via LSTM/BiLSTM[J]. IEEE Sensors Journal, 2023.
    Chang E, Kim H T, Yoo B. Virtual reality sickness: a review of causes and measurements[J]. International Journal of Human–Computer Interaction, 2020, 36(17): 1658-1682.
    Yildirim, Caglar. Don’t make me sick: investigating the incidence of cybersickness in commercial virtual reality headsets[J]. Virtual Reality 2020,24(2): 231-239.
    Vlahovic S, Suznjevic M, Skorin-Kapov L. A survey of challenges and methods for Quality of Experience assessment of interactive VR applications[J]. Journal on Multimodal User Interfaces, 2022, 16(3): 257-291.
    Niso G, Romero E, Moreau J T, et al. Wireless EEG: A survey of systems and studies[J]. NeuroImage, 2023, 269: 119774.
    Singh A K, Krishnan S. Trends in EEG signal feature extraction applications[J]. Frontiers in Artificial Intelligence, 2023, 5: 1072801.
    薛沐辉,徐宝国,李浪,等.基于迁移学习的手部自然动作脑电识别[J].南京信息工程大学学报,2024,1-12.
    XUE Muhui, XU Baoguo, LI Lang, et al. EEG recognition of natural hand movements based on transfer learning[J]. Journal of Nanjing University of Information Science Technology 2024,1-12.
    Nawaz R, Cheah K H, Nisar H, et al. Comparison of different feature extraction methods for EEG-based emotion recognition[J]. Biocybernetics and Biomedical Engineering,2020,40(3): 910-926.
    Yen C, Lin C L, Chiang M C. Exploring the frontiers of neuroimaging: a review of recent advances in understanding brain functioning and disorders[J]. Life, 2023, 13(7): 1472.
    Jiang L, He J, Pan H, et al. Seizure detection algorithm based on improved functional brain network structure feature extraction[J]. Biomedical Signal Processing and Control, 2023, 79: 104053.
    Gu L, Yu Z, Ma T, et al. EEG-based classification of lower limb motor imagery with brain network analysis[J]. Neuroscience, 2020, 436: 93-109.
    常文文,聂文超,袁月婷,等.基于多层脑功能网络特征的动作意图识别[J].电子科技大学学报,2023,52(01):14-22.
    CHANG Wenwen, NIE Wenchao, YUAN Yueting, et al.Action intention recognition based on multi-layer brain functional network features [J] Journal of University of Electronic Science and Technology of China, 2023, 52 (01): 14-22.
    Hua C, Tao J, Zhou Z, et al. EEG classification model for virtual reality motion sickness based on multi-scale CNN feature correlation[J]. Computer Methods and Programs in Biomedicine, 2024, 251: 108218.
    Kennedy R S, Drexler J, Kennedy R C. Research in visually induced motion sickness[J]. Applied ergonomics, 2010,41(4): 494–503.
    ur Rehman N, Aftab H. Multivariate variational mode decomposition[J]. IEEE Transactions on signal processing, 2019, 67(23): 6039-6052.
    Pandey P, Seeja K R. Subject independent emotion recognition from EEG using VMD and deep learning[J]. Journal of King Saud University-Computer and Information Sciences, 2022, 34(5): 1730-1738.
    Liu D, Cao T, Wang Q, et al. Construction and analysis of functional brain network based on emotional electroencephalogram[J]. Medical Biological Engineering Computing, 2023, 61(2): 357-385.
    罗志增,郑文涛.基于多层时变功能脑网络特征的运动想象识别[J].华中科技大学学报(自然科学版),2024,52(05):56-63.
    LUO Zhizeng, ZHENG Wentao. Motor imagery recognition based on multi-layer time-varying functional brain network features[J]. Journal of Huazhong University of Science and Technology (Natural Science Edition), 2024, 52 (05): 56-63.
    Gonuguntla V, Kim J H. EEG-Based functional connectivity representation using phase locking value for brain network-based applications[C]//2020 42nd Annual International Conference of the IEEE Engineering in Medicine Biology Society (EMBC). IEEE, 2020: 2853-2856.
    Wang H, Liu X, Li J, et al. Driving fatigue recognition with functional connectivity based on phase synchronization[J]. IEEE Transactions on Cognitive and Developmental Systems, 2020, 13(3): 668-678.
    di Biase L, Ricci L, Caminiti M L, et al. Quantitative high density EEG brain connectivity evaluation in parkinson’s disease: The phase locking value (PLV)[J]. Journal of Clinical Medicine, 2023, 12(4): 1450.
    Basha S S, Dubey S R, Pulabaigari V, et al. Impact of fully connected layers on performance of convolutional neural networks for image classification[J]. Neurocomputing, 2020, 378, 112-119.
    Mao W L, Fathurrahman H I K, Lee Y, et al. EEG dataset classification using CNN method[J]. In Journal of physics: conference series,2020, 1456(1): 12-17.
    Virtual Reality Sickness Detection Based on Brain Functional Networks
    Yang Wenqing1 Hua Chengcheng*1,2,3 Yin Liping1 Tao Jianlong1 Chen Yuechi1 Dai Zhian1 Liu Jia1,2,3
    (1. School of Automation, Nanjing University of Information Science Technology, Nanjing 210044, China; 2. Jiangsu Intelligent Meteorological Detection Robot Engineering Research Center, Nanjing University of Information Science Technology, Nanjing 210044, China;3. Jiangsu Collaborative Innovation Center for Atmospheric Environment and Equipment Technology, Nanjing University of Information Science Technology, Nanjing 210044, China)
    Abstract:An effective detection scheme based on decoding electroencephalogram (EEG) signals under different states of sickness is proposed, which is helpful for studying methods to alleviate virtual reality motion sickness (VRMS). This article uses multivariate variational mode decomposition (MVMD) to divide EEG into five frequency bands and divides the data into different sickness state groups based on the results of the motion sickness scale. The phase-locking value (PLV) method is used to calculate the functional connections within and between EEG frequency bands to construct a super adjacent matrix, and classification recognition is performed based on models such as support vector machine (SVM) and convolutional neural network (CNN). The research results show that the fusion of three topological features with significant differences in clustering coefficient, local efficiency, and weighted node degree results in sickness vs Non sickness, high sickness vs The highest average classification accuracy in the two tasks of low sickness was 91.70% and 96.00%, respectively. In addition, this article also directly inputs the super adjacency matrix into the CNN model, achieving average classification accuracies of 93.40% and 98.50% in two tasks, respectively. The results indicate that the method proposed in this study can be used for the detection of motion sickness in virtual reality and provide Reference for further research on the impact of motion sickness on the functional coupling of various brain regions.
    Keywords:Virtual Reality Motion Sickness; EEG; Multivariate Variational Mode Decomposition; Brain Functional Connectivity; Network Topological Features
    相似文献
    引证文献
    引证文献 [0]
    网友评论
    网友评论
    分享到微博
    发 布
引用本文

杨文清,化成城,殷利平,陶建龙,陈玥池,戴志安,刘佳.基于脑功能网络的虚拟现实晕动症检测[J].南京信息工程大学学报,,():

复制
分享
文章指标
  • 点击次数:18
  • 下载次数: 0
  • HTML阅读次数: 0
  • 引用次数: 0
历史
  • 收稿日期:2024-11-20
  • 最后修改日期:2025-01-07
  • 录用日期:2025-01-09

地址:江苏省南京市宁六路219号    邮编:210044

联系电话:025-58731025    E-mail:nxdxb@nuist.edu.cn

南京信息工程大学学报 ® 2025 版权所有  技术支持:北京勤云科技发展有限公司