Take a look at the Recent articles

Abstract

In numerous studies neurophysiological mechanisms of information perception, memorization and storage including neurochemical and molecular processes have been disclosed. At the same time the key memory extraction mechanisms are still unknown. Information extraction occurs in the subjective, mental sphere of brain activity, which has been not available for investigation by existing neurophysiological methods.
In recent years, progressive methods of EEG wavelet transform analysis have been developed. Using this method of EEG processing, we have identified the principal possibility for revealing human brain mental activity. Our study showed a possibility of reliable identification of the recall process during mental answer to a question. Significant differences in the continuous EEG wavelet transform were identified in participants who remembered but did not answer the question in different test intervals: while understanding the question and while formulating mental answer. The most significant differences between the brain ability to reproduce correct mental response and none manifested in the beta range of EEG in the occipital lead O1.
The data obtained suggest the possibility of identifying the process of information retrieval from memory on the basis of continuous EEG wavelet analysis. Obviously, the mental activity characterizing memory has a real neurophysiological basis and can be studied by objective methods.Further, larger-scale studies in this area will allow objectivizing the study of such important psychophysiological phenomena as memory, consciousness, and attention,which may also be of importance in clinical work.

Keywords

EEG, continuous wavelet transform, mental activity, memory, information extraction

Introduction

The brain is a unique neuronal organization possessing the ability to mental activity, which manifests itself through consciousness, in thoughts, feelings, emotions, memory, i.e. through the person's subjective perception of oneself and the world around.
The need to study the nature of brain's mental activity was indicated by world famous biologists and neurophysiologists [1-7].
Memory is a unique ability of the brain to memorize and reproduce information. Memory mechanisms include perception, memorization, storage and subsequent reproduction of information. Based on numerous studies, only the neurophysiological mechanisms of perception, memorization and storage of information in the brain have been disclosed, which include neurochemical and molecular processes [8-10].
At the same time, the key memory mechanisms for information extraction are still completely unknown. Information extraction occurs in the subjective, mental sphere of brain activity, which has been not available for investigation by existing neurophysiological methods until the present time [11,12]
In recent years, progressive methods of wavelet transform for electroencephalogram (EEG) analysis have been developed [13]. Using the wavelet transform-based EEG method, we have identified the fundamental possibility for objective recording of the human brain mental activity [14], which opened the prospect for revealing mechanisms of information extraction from memory.
The purpose of the work was to develop a fundamentally new information technology for recognizing the process of information extraction from memory in the mental activity of the brain based on the wavelet transform of electroencephalogram (EEG).

Methods

Study Procedure
To solve this problem, we developed an experimental model and software allowing us to test information extraction from human memory [15].
The experimental model and information software made it possible to identify and compare various states of the brainmental activity on electroencephalographic indicators. One state was registered when the individual remembered and gave the correct mental answer to the question, the other one was registered when he or she did not know or was not able to remember the answer.
We conducted a pilot study. Ten first-year medical students (male aged 17-20 years) volunteered to take part in the current survey conducted at the I.M. Sechenov First Moscow State Medical University. The inclusion criteria for the selection of the subjects were university students, male gender, age 17-20, right-handedness. The exclusion criteria were acute diseases, alcohol or substance abuse, current psychoactive medication, clinically significant neuorological problems.
All participants signed the informed consent to processing of personal data and participation in the study. For the participants aged 17 the Written Informed Consent was obtained from their legally authorized representatives. The study was approved by the Sechenov University Ethics Committee on May 18, 2017 (protocol №4).
The experimental model and the research procedure which we previously described in the article [15], included several stages.
Before testing, electrodes were fixed on the subject's scalp to record EEG.
The experimenter gave an instruction describing the study procedure to the participant. According to the instruction, various images with a question appeared sequentially on the screen in front of the subject. One image was demonstrated for 5 seconds. During this time, the subject was thinking about and understanding the question.
After the image on the screen disappeared, the individualwas expected to give a mental answer. The time allowed for the mental answer was also 5 seconds.
During the tests, the EEG was registered and recorded for further comparative wavelet analysis considering two mental states: extracting information from memory or lack of knowledge.
The experimenter then found out from the subject whether he had responded or not and entered this data into the test results.
Upon study completion, all results with EEG recording were saved in a CSV file for further processing using continuous wavelet transform [12].
The EEG registration was performed in the initial state with open (60 sec.) and closed (60 sec.) eyes, during task completion and after completion with open and closed eyes using the electroencephalograph “Neuron-spectrum” (Neurosoft LLC, Ivanovo, https://neurosoft.com). The EEG was recorded with monopolar channels according to the 10–20 system in the occipital (O2, O1), parietal (P4, P3), central (C4, C3), frontal (F4, F3), and temporal (T4, T3) leads. The combined reference electrodes were placed on the earlobes. The filtration band was 0.5–70.0 Hz, the time constant was 0.32 s, and the rejection filter was 50 Hz. Digitization frequency was 200 Hz.
Data processing and analysis
The work was based on the wavelet transform method permitting to conduct frequency-time analysis of EEG signals registered during demonstration of questions and during answering them. In particular, continuous EEG wavelet transform in the frequency range from 0.5 to 30 Hz was used. The Morlet wavelet was experimentally chosen as a basis function.
Continuous wavelet transform is an inner product of the s(t) signal and the two-parameter wavelet function a,b(t), of the selected type. The EEG signal is used as the s(t) signal in this study. Continuous wavelet transform of the s(t) signal is as follows:

In which “a” is a timescale parameter which is inversely proportional to the frequency and responsible for the wavelet width, “b” is a parameter of the shiftdetermining the position of the wavelet on the time axis.
The wavelet function a,b(t) of the сurrent set is obtained from a single mathematical function by stretching or compressing and further shifting:

The Morlet wavelet function selected in this study is mathematically expressed as follows:

The Morlet Wavelet (Figure 1) is a plane wave modulated by the unit width Gaussian (Schoberg [12]).
To obtain wavelet spectra, we used a two-dimensional representation of the three-dimensional surface of the wavelet transform energy on the frequency-time plane carried out by calculating the logarithm of the of S(a,b) function square for each scale and shift:

Normalization of the wavelet spectrum energy to the total energy at any time was applied to equalize the amplitudes of the wavelet spectra for different subjects:

The quantitative characteristic of the wavelet frequency range was determined as the average energy of the normalized wavelet spectrum within the specified frequency (Δf) and time limits (Δt):
.
Wavelet transform was applied to the 10 secondrecording sections (5 seconds for questiondemonstration, 5 seconds for mental response).
The advantage of the wavelet transform is the ability to analyze the spectral structure of the signal in dynamics. Therefore, the researcher is able to observe changes in various frequencies corresponding to certain states of the brainduring the observation time.
The EEG signal processing was carried out with the following algorithm:

1. In the native recordings, fragments corresponding to the period of question demonstration and the period of mental response to the question were distinguished in each of the 10 specified EEG leads, with their subsequent distribution into two classes: "Yes" - the subject remembered and gave a mental answer to the question, "No" –he could not remember or did not know the answer, and hence did not answer the question.
2. Сontinuous wavelet transform was conducted for all recording sections.
3. The wavelet spectra were averaged for each subject separately by leads (O2, O1, P4, P3, C4, C3, F4, F3, T4, T3) and two classes: "Yes", "No". At the same time, equality of the number of spectra that represented each averaged image was сontrolled to exclude an artificial qualitative change in one of the classes.
4. The frequency range of the averaged wavelet spectrum was divided into generally accepted EEG frequency ranges: delta (0.5-4 Hz), theta (4-8 Hz), alpha (8-14 Hz), beta (14-30 Hz), total (0.5-30 Hz), as well as into time intervals: Q interval – the period of question demonstration (from the 2ndto the 5th second), R interval – the period of expected mental response (from the 6th to the 9th second), QR interval - the entire period of the question-answer cycle (from the 2ndto the 9th second). The first second of the question demonstration was not included in the analysis, since no recall processes are expected for this period.
5. In each current selection of the averaged wavelet spectrum, the average energy of the PΔf,Δt wavelet spectrum for the selected recording periods was calculated.

The obtained values of the average energy of the wavelet spectrum of two classes, "Yes" and "No" were compared with each other using the Wilcoxon test (W) [16]. Statistically significant differences were determined with p-values < 0.05.

Results

Figure 2 demonstrates the differences in EEG signals between "Yes" and "No" classes.The EEG spectra averaged over 10 participants in the lead O1, which showed the most significant difference in the R interval in the beta EEG range (indicated with a rectangle) are presented.
The figure demonstrates the dependence of frequency changes (vertical axis) over time (horizontal axis) in the question-answer cycle. The brightness of the points indicates the relative energy of the wavelet spectrum. It can be noticed that the average brightness of the area in the rectangles is higher for the wavelet spectrum of EEG signals of "No" class (the subject could not remember or did not know the answer) than for signals of "Yes" class.
Analysis of the results revealed several significant differences in the EEG of the participants who gave a mental answer to the question and those who could not remember or did not know the answer.
Table 1 indicates the significance of differences in the Q interval (from the 2nd to the 5th second). The results presented in Table 1 show that significant differences were observed in the delta range in the lead T3 and the alpha range in the leads C3, F4, T3. The most significant differences were registered in the frontal lead (F4) (W = 2, p = 0.006).
In Table 2 the significances of differences in the R interval (6 to 9 seconds) are presented. Significant differences were found in the leads O1 (in the delta and beta ranges) and O2 (in the total frequency range). The most significant differences were revealed forthe O1 lead in the beta range (W = 2, p = 0.002).
The QR interval includes the entire period of one question-answer cycle, so averaging over this period allows to eliminate the influence of all other mental experiences of the participants and to reveal period-specific recall (Q) and mental response (R). Thus, we are able to identify the most general differences in the mental activity of the brain in case of successful and unsuccessful memory reproduction processes. Table 3 shows the significances of differences in the QR interval (from the 2ndto the 9th second).
Table 3 demonstrates that significant differences in the T3 lead in the delta, theta, alpha ranges, in the C3 lead in the alpha range and in the P3 lead in the total frequency range. The most significant differences in the QR interval were revealed for the T3 lead (W = 2, p = 0.0059).
In Table 4 the average energies of the normalized wavelet spectrum averaged across all participants in the given frequency and time limits are shown. All differences in average normalized energy of EEG wavelet spectra between classes “Yes” and “No” were significant (р < 0,037).

Figure 1. Morlet Wavelet

Annotation: Abscissa - report number, ordinate - function amplitudes values. The presented shape was obtained at scale and shift parameters a = b = 2

Figure 2. Generalized wavelet spectrum of lead O1 (A - class "Yes", B - class "No")

Annotation: Abscissa - time (s). Ordinate - frequency (Hz). The scale on the right shows the brightness in grayscale, where the light tone represents the maximum normalized energyEn, and the black tone represents the minimum one. The vertical line marks the end of the question demonstration. The horizontal lines mark the boundaries of the EEG frequency ranges (delta, theta, alpha, beta). The area marked by a rectangle shows the zone with the highest significance of differences between classes: A and B.

Table 1. Assessment of significance in differences between "Yes" and "No" classes in the Q interval

Lead	Significance of differences (p-value)
	delta	theta	alpha	beta	total
O2	0.08	0.11	0.16	0.92	0.49
O1	0.43	0.19	0.63	0.77	0.92
P4	0.63	0.56	0.23	0.49	0.85
P3	0.56	0.85	0.49	0.77	0.38
C4	0.85	0.77	0.13	0.43	0.70
C3	0.38	0.56	0.02*	0.38	0.43
F4	0.63	0.63	0.006*	0.11	0.70
F3	1	0.63	0.32	0.13	0.63
T4	1	0.56	0.19	0.28	0.49
T3	0.05*	0.16	0.02*	0.38	0.32

Annotation: ^*significant differences (Wilcoxon test) between classes "Yes" and "No"

Table 2. Assessment of significance in differences between "Yes" and "No" classes in the R interval

Lead	Significance of differences (p-value)
	delta	theta	alpha	beta	total
O2	0.63	0.38	0.57	0.16	0.04*
O1	0.05*	0.28	0.11	0.002*	0.11
P4	0.77	0.38	0.56	0.19	0.08
P3	0.43	0.56	0.77	0.32	0.16
C4	0.85	0.92	0.38	0.85	0.19
C3	0.28	0.43	0.23	0.38	0.06
F4	0.49	0.23	0.92	0.85	0.11
F3	0.49	0.49	0.63	0.100	0.49
T4	0.85	1	0.43	0.38	0.23
T3	0.19	0.16	0.43	0.92	0.06

Annotation: ^*significant differences (Wilcoxon test) between classes "Yes" and "No"

Table 3. Assessment of significance in differences between "Yes" and "No" classes in the QR interval

Lead	Significance of differences (p-value)
	delta	theta	alpha	beta	total
O2	0.32	0.28	0.16	0.49	0.13
O1	0.16	0.32	0.38	0.08	0.49
P4	0.56	0.43	0.85	0.32	0.38
P3	0.28	0.49	0.63	0.38	0.05*
C4	0.77	1	0.85	0.70	0.32
C3	0.11	0.28	0.05*	0.28	0.13
F4	0.64	0.70	0.32	0.38	0.23
F3	0.57	0.70	0.38	0.77	0.49
T4	0.76	0.64	0.92	0.32	0.28
T3	0.02*	0.03*	0.006	0.49	0.28

Annotation: ^*significant differences (the Wilcoxon test) between classes "Yes" and "No"

Table 4. The values of average normalized energy of EEG wavelet spectra in different leads for “No” and “Yes” classes

Time interval	EEG Lead	EEG Frequency range	AveragenormalizedenergyofEEGwaveletspectra, conventional units (*10-3) (M±SD)		Change of the EEG wavelet spectra for “No” response in comparison with “Yes”response
			“Yes” class	“No” class
Q	O1	alpha	2.693±0.557	2.405±0.289	Decrease (p = 0.037)
Q	O1	beta	1.762±0.246	1.5844±0.317	Decrease (p = 0.027)
Q	O1	alpha + beta	2.321±0.401	2.0768±0.262	Decrease (p = 0.010)
Q	O1	total	3.409±0.001	3.4098±0.001	Increase (p = 0.020)
Q	P3	theta	2.809±0.353	2.525±0.313	Decrease (p = 0.014)
R	O2	theta	2.591±0.181 ·	2.650±0.196	Increase (p = 0.037)
R	O1	total	3.409±0.001	3.409±0.00	Increase (p = 0.020)
TP	O1	total	3.409±0.001)	3.409±0.001	Increase (p = 0.010)
TP	P3	total	3.409±0.001	3.410±0.001-	Increase (p = 0.014)

Discussion

As a result of the conducted study, we can conclude that there is a principal possibility of reliable detection of differences in EEG signals in the case of extracting information from memory and being unable to answer the question.
During continuous EEG wavelet transform, significant differences were identified between the participants who remembered the answer to the question and those who could not answer. This was show for different test intervals - while understanding the question and while formulating the mental answer, in different EEG leads. At the same time, the most significant difference between the brain's ability to recall mental response and inability to do it manifested itself in the beta range of EEG in the occipital lead O1 (p = 0.002). One of the possible explanations for that is a visual form of questions demonstration activating the occipital region of the cerebral cortex.
The obtained data showed that we can identify the process of extracting information from memory on the basis of continuous wavelet EEG analysis. This allowed us to draw an important conclusion about the principal possibility for an objective study of the nature of brain mental activity [17]. We suggest registered phenomena referred to the mental activity that characterizes the memory. Extraction of information from memory has a real neurophysiological basis, and this process can be studied by objective methods.
The empirical results reported in this paper should be considered in the light of some limitations. The power analysis for calculation of the optimal sample size was not conducted in advance. This is a pilot study, and therefore, the sample size is small. The questionnaire used in this study was not validated previously. We are also planning to conduct the comparative analysis of the results with the data of EEG spectral analysis and mapping.

Conclusion

Our research findings demonstrated the fundamental possibility to identify the process of information retrieval from memory using continuous EEG wavelet transformation and calculation of the EEG wavelet energy. Extraction of information from memory has a real neurophysiological basis, which can be studied with modern objective methods. Revealing the mechanisms of brain mental activity, including memory, is undoubtedly an actual fundamental scientific biological problem and its solution opens up wide opportunities for the development of innovative information technologies.Further, larger-scale studies in this area will allow objectivizing the study of such important psychophysiological phenomena as memory, consciousness, and attention,which may also be of importance in clinical work.

Acknowledgement

We would like to the thank all the participants who took part in this study.

Funding

The research was supported by the Russian Foundation for Basic Research (RFBR) grant 19-07-00008/19.

Conflict of interest

The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Author's contributions

Evgeny A. Yumatov – conceptualization, project administration, writing draft;
Nikolay A. Karatygin – investigation;
Elena N. Dudnik – validation, writing draft;
Mikhail Yu. Budnikov – editing original draft;
Anna I. Filipchenko – investigation;
Lyudmila T. Sushkova– supervision;
Roman V. Isakov – data processing;
Waleed Al-Haidri – data processing, visualization;
Oleg S. Glazachev– formal analysis, review and editing

References

1. Pavlov IP (1951) Dvadcatiletnijopytob"ektivnogoizucheniyavysshejnervnojdeyatel'nosti (povedeniya) zhivotnykh. Polnoesobraniesochinenij [Twenty years of experience in the objective study of the higher nervous activity (behavior) of animals. Full composition of writings]. Moscow-Leningrad: Izd.AN SSSR. 1951.
2. Bekhtereva NP, Budzen PV, Gogolicyn YuL (1977) Mozgovyekodypsihicheskojdeyatel'nosti [Brain codes of mental activity]. Leningrad: Nauka. 1977.
3. Ivanitsky AM (1999) Glavnayazagadkaprirody: kaknaosnoveprocessovmozgavoznikayutsub"ektivnyeperezhivaniya [Main mystery of nature: how subjective experiences arise on the basis of brain processes]. PsikhologicheskiiZhurnal 20: 93-104.
4. Nagel T (2001) Myslimost' nevozmozhnogoiproblemaduhaitela [Conceiving the Impossible and the Mind-Body Problem]. Voprosyfilosofii 8: 101-112.
5. Popper K (2008) Znanieipsihofizicheskayaproblema: V zashchituvzaimodejstviya [Knowledge and the Body-Mind Problem: In Defence of Interaction]. Moscow: URSS.
6. Sudakov KV (2010) Sistemnyemekhanizmypsihicheskojdeyatel'nosti [Systemic mechanisms of mental activity]. Zhurnalnevrologiiipsikhiatriiimeni S.S. Korsakova 110: 4-14.
7. Crick F, Koch C (1995) Why neuroscience may be able to explain consiousness. Scientific American 273: 84-85.
8. Andreev OA, Velichkovsky BB, Popov LI (1987) MekhanizmyPamyati [Mechanisms of Memory]. Moscow: Nauka.
9. Zinchenko TP (2002) Pamyat' veksperimental'nojikognitivnojpsihologii [Memory in experimental and cognitive psychology]. Saint Petersburg: Piter.
10. Rose S (1992) The Making of Memory: From Molecules to Mind. New York and London: Anchor Books.
11. Yumatov EA (2014) To knowledge of the origin of the brain mental activity. World Journal of Neuroscience 4: 170-182.
12. Schoberg AG (2014) Sovremennyemetodyobrabotkiizobrazhenij: modificirovannoevejvlet-preobrazovanie [Modern methods of image processing: modified wavelet transform]. Habarovsk: Izd. Tihookeanskogogosudarstvennogouniversiteta.
13. Hramov AE, Koronovskii AA, Makarov VA, Pavlov AN, SitnikovaEYu (2015) Wavelets in Neuroscience. Springer.
14. Yumatov EA, Hramov AE, Grubov VV, Glazachev OS, et al. (2019) Possibility for recognition of psychic brain activity with continuous wavelet analysis of EEG. Journal of Behavioral and Brain Science 9: 67-77.
15. Yumatov EA, PotapovVYu, Dudnik EN, Karatygin NA, Pertsov SS (2020) Methodology, experimental model and software for identifying memory replication processes based on wavelet analysis of electroencephalograms. Trends in Medicine 20: 1-4.
16. Miller Jr, RG (1997) Beyond ANOVA: basics of applied statistics.CRC Press.
17. Yumatov EA (2019) The molecularly fielding psychophysical nature of the brain mental activity. NeuroscienceandMedicine 10: 55-74.