Models of neurofeedback learning.
28 April 2020
Traditional functional neuroimaging establishes correlative relationships between brain activity and behavior. By contrast, neurofeedback that involves functional neuroimaging enables the manipulation of neural activity in circumscribed regions, functional connections and spatiotemporal activity patterns as independent variables, and thus represents a way of investigating the relationship between brain activity and behavior that is comparable to brain stimulation.
In neurofeedback, brain activation is volitionally regulated through learning; as the activation acts as an independent variable, it allows causal inferences to be made between brain activity and behavior. The different behavioral changes that have been observed to result from self-manipulation of neural activation indicate that the physiological consequences of neurofeedback may be considered to be a form of endogenous neural stimulation. Thus, neurofeedback has been used to modulate behaviourally relevant functional networks and to provide self-administered therapy. Concerns have been expressed about how the rapid attempts to use neurofeedback for clinical rehabilitation and therapy have outpaced the development of a proper understanding of the neural mechanisms and neuroplastic changes that underlie neurofeedback.
Here we present extant theories and models that have been proposed to explain neurofeedback learning and its underlying mechanisms. There are overlaps and compatibilities among the theories; for example, the operant (or instrumental) learning theory can be considered to form a part of the dual-process view, and motor learning and skill learning theories may have commonalities, whereas the global workspace theory, which presupposes the conscious awareness of reinforcement (feedback) for learning, seems to be compatible with some aspects of the awareness theory. Future, hypothesis-based experiments should shed new light on the validity of the above-mentioned theories in neurofeedback learning and performance.
Operant (or instrumental) learning.
The operant learning theory, as applied to neurofeedback, states that control of brain activity proceeds when correct or desired brain responses are reinforced by contingent feedback and/or reward. The theory considers three main elements in its description of the procedure, discriminative stimuli, responses, and reinforcers. A large extant literature of operant learning in humans and animals has elucidated the neurophysiological correlates of operant learning and highlighted the selective involvement of prefrontal and striatal synapses. However, experimental instructions and subjective reports of the use of mental strategies in human studies have led some researchers to propose other explanatory mechanisms of neurofeedback learning.
Motor learning.
According to this model, acquiring control over neurophysiological signals is similar to the acquisition of motor learning involving a well-organized sequence of movements and symbolic information. Although there has been much scientific and clinical investigation of this theory in different types of motor learning, there is no specific application of this model to neurofeedback training in recent times.
Dual-process theory.
The dual-process theory attempts to integrate feedforward and feedback learning processes in explaining neurofeedback learning. In this model, the naive learner searches for an effective mental strategy, either on their own or based on the experimental instructions. If the learner does not find the strategy to be effective to control the feedback signal, they may search for a new one until an effective strategy is discovered. Upon successive reinforcement, the strategy that best matches the feedback may become automatic. Alternately, the learner may never learn to find an effective strategy, upon which the brain may rely on the feedback signal alone to guide learning, or the subject may fail to learn at all. An experimental investigation of this hypothesis would involve neurofeedback training in the presence or absence of explicit instructions for attaining control while monitoring participants' reports of mental strategies that are used and their brain and behavioural correlates.
Awareness theory.
The awareness theory competes directly with the instrumental learning model in biofeedback literature. The theory states that the feedback signal provides information about a physiological response (that is, brain activity) to which the subject becomes aware of, and this leads to voluntary control over the response. The model considers three elements, awareness of reinforcers (feedback and reward), the reinforcer response contingency and the response itself. However, theoretical analyses and later tests in animals and humans concluded that awareness of the response is neither necessary nor sufficient to acquire control over the brain activity.
Global workspace theory.
The global workspace theory of neurofeedback learning proposes that learning control of neural activity is enabled by the wide, global distribution of the feedback signal in the brain so that it becomes conscious. A testable prediction of this theory is that a non-conscious or subliminal feedback signal, for example, by backward masking, does not help in acquiring control of the brain activity being trained. However, the hypothesis should also be evaluated in light of the existing evidence for subliminal instrumental learning, unconscious processing of reward stimuli, and the distinctions between conscious and non-conscious representations on the one hand, and automatic and deliberate processing on the other.
Skill learning.
Recently, there have been proposals to view neurofeedback and brain-computer interfaces (BCI) or brain-machine interfaces (BMI) learning within the framework of cognitive skill learning. According to this proposal, neurofeedback learning involves an initial phase of rapid change in performance and a late phase of more gradual improvement as the skill is consolidated and performance asymptotes. Functional and structural changes in the dorsomedial striatum have been shown to be associated with the early phase, whereas such changes in the dorsolateral striatum have been shown to be associated with the late phase. Recently, similar changes have been observed in neurofeedback learning in animals and humans, providing support to this theory.
Sitaram, R., Ros, T., Stoeckel, L., Haller, S., Scharnowski, F., Lewis-Peacock, J., … Sulzer, J. (2016). Closed-loop brain training: the science of neurofeedback. Nature Reviews Neuroscience, 18(2), 86–100. doi:10.1038/nrn.2016.164
In neurofeedback, brain activation is volitionally regulated through learning; as the activation acts as an independent variable, it allows causal inferences to be made between brain activity and behavior. The different behavioral changes that have been observed to result from self-manipulation of neural activation indicate that the physiological consequences of neurofeedback may be considered to be a form of endogenous neural stimulation. Thus, neurofeedback has been used to modulate behaviourally relevant functional networks and to provide self-administered therapy. Concerns have been expressed about how the rapid attempts to use neurofeedback for clinical rehabilitation and therapy have outpaced the development of a proper understanding of the neural mechanisms and neuroplastic changes that underlie neurofeedback.
Here we present extant theories and models that have been proposed to explain neurofeedback learning and its underlying mechanisms. There are overlaps and compatibilities among the theories; for example, the operant (or instrumental) learning theory can be considered to form a part of the dual-process view, and motor learning and skill learning theories may have commonalities, whereas the global workspace theory, which presupposes the conscious awareness of reinforcement (feedback) for learning, seems to be compatible with some aspects of the awareness theory. Future, hypothesis-based experiments should shed new light on the validity of the above-mentioned theories in neurofeedback learning and performance.
Operant (or instrumental) learning.
The operant learning theory, as applied to neurofeedback, states that control of brain activity proceeds when correct or desired brain responses are reinforced by contingent feedback and/or reward. The theory considers three main elements in its description of the procedure, discriminative stimuli, responses, and reinforcers. A large extant literature of operant learning in humans and animals has elucidated the neurophysiological correlates of operant learning and highlighted the selective involvement of prefrontal and striatal synapses. However, experimental instructions and subjective reports of the use of mental strategies in human studies have led some researchers to propose other explanatory mechanisms of neurofeedback learning.
Motor learning.
According to this model, acquiring control over neurophysiological signals is similar to the acquisition of motor learning involving a well-organized sequence of movements and symbolic information. Although there has been much scientific and clinical investigation of this theory in different types of motor learning, there is no specific application of this model to neurofeedback training in recent times.
Dual-process theory.
The dual-process theory attempts to integrate feedforward and feedback learning processes in explaining neurofeedback learning. In this model, the naive learner searches for an effective mental strategy, either on their own or based on the experimental instructions. If the learner does not find the strategy to be effective to control the feedback signal, they may search for a new one until an effective strategy is discovered. Upon successive reinforcement, the strategy that best matches the feedback may become automatic. Alternately, the learner may never learn to find an effective strategy, upon which the brain may rely on the feedback signal alone to guide learning, or the subject may fail to learn at all. An experimental investigation of this hypothesis would involve neurofeedback training in the presence or absence of explicit instructions for attaining control while monitoring participants' reports of mental strategies that are used and their brain and behavioural correlates.
Awareness theory.
The awareness theory competes directly with the instrumental learning model in biofeedback literature. The theory states that the feedback signal provides information about a physiological response (that is, brain activity) to which the subject becomes aware of, and this leads to voluntary control over the response. The model considers three elements, awareness of reinforcers (feedback and reward), the reinforcer response contingency and the response itself. However, theoretical analyses and later tests in animals and humans concluded that awareness of the response is neither necessary nor sufficient to acquire control over the brain activity.
Global workspace theory.
The global workspace theory of neurofeedback learning proposes that learning control of neural activity is enabled by the wide, global distribution of the feedback signal in the brain so that it becomes conscious. A testable prediction of this theory is that a non-conscious or subliminal feedback signal, for example, by backward masking, does not help in acquiring control of the brain activity being trained. However, the hypothesis should also be evaluated in light of the existing evidence for subliminal instrumental learning, unconscious processing of reward stimuli, and the distinctions between conscious and non-conscious representations on the one hand, and automatic and deliberate processing on the other.
Skill learning.
Recently, there have been proposals to view neurofeedback and brain-computer interfaces (BCI) or brain-machine interfaces (BMI) learning within the framework of cognitive skill learning. According to this proposal, neurofeedback learning involves an initial phase of rapid change in performance and a late phase of more gradual improvement as the skill is consolidated and performance asymptotes. Functional and structural changes in the dorsomedial striatum have been shown to be associated with the early phase, whereas such changes in the dorsolateral striatum have been shown to be associated with the late phase. Recently, similar changes have been observed in neurofeedback learning in animals and humans, providing support to this theory.
Sitaram, R., Ros, T., Stoeckel, L., Haller, S., Scharnowski, F., Lewis-Peacock, J., … Sulzer, J. (2016). Closed-loop brain training: the science of neurofeedback. Nature Reviews Neuroscience, 18(2), 86–100. doi:10.1038/nrn.2016.164
