The recognition network is trained on ‘Fantasies’ (programs sampled fromlibrary) and ‘Replays’ (programs found during waking).

The highlighted passage states: "The recognition network is trained on ‘Fantasies’ (programs sampled from library) and ‘Replays’ (programs found during waking)."

In context, this describes a key mechanism in DreamCoder’s "dreaming" sleep phase, where the system improves its procedural skill—the neural network component that guides efficient program search. The recognition network is a neural model trained to predict which programs are likely to solve new tasks given observed data.

Training data for this network comes from two complementary sources:

1. **Replays**: These are real programs discovered during the waking phase when DreamCoder attempts to solve actual training tasks. Replays help ensure the recognition model learns from the actual problems it faces, reinforcing solutions it has found and preventing forgetting.

2. **Fantasies**: These are random programs generated by sampling from the current learned library (the generative model of DreamCoder’s language). Fantasies create imagined or hypothetical tasks and corresponding programs, enabling the recognition network to learn from a much broader, more varied dataset than just the training tasks. This broad and diverse "dream" data is critical because the actual number of training tasks is relatively small (often on the order of 100–200), insufficient alone to train a high-capacity neural network effectively.

Together, replays and fantasies provide a balanced training regimen for the recognition network: replays ground learning in empirical experience, while fantasies expand the network’s generalization capacity by exposing it to a wide variety of synthetic problems and solutions drawn from the generative model.

This dual training approach supports DreamCoder’s goal of efficient, scalable program induction by enabling the neural search guidance to generalize beyond previously seen tasks, helping the system to find promising programs more quickly and effectively when tackling new problems.

Q. [DreamCoder: Dreaming Phase] During the "dreaming" phase of sleep, what is the data that the recognition model is trained on?
A. _‘Fantasies’_ (programs sampled from library) and _‘Replays’_ (programs found during waking).

This prompt is pluckable. It directly targets the exact insight the user highlighted—the dual training sources ('Fantasies' and 'Replays') for DreamCoder's recognition network—and frames it as a single, atomic recall unit. The question is specific and self-contained: it anchors the concept to DreamCoder's 'dreaming' phase and the recognition model, so it remains unambiguous when reviewed months later without the original document. The answer preserves fidelity to the source by including the brief parenthetical definitions from the highlight itself, ensuring the recalled knowledge stays whole rather than decontextualized. It avoids testing trivial fragments or introducing external information beyond the user's engagement, and the distinctive terminology (Fantasies/Replays/dreaming phase) gives it strong long-term stability.


Kevin Ellis, Catherine Wong, Maxwell Nye, Mathias Sablé-Meyer, Luc Cary, Lucas Morales, Luke Hewitt, Armando Solar-Lezama, Joshua B. Tenenbaum

2021 (exact date not specified in the excerpt, but the work is from 2021)

Summary: DreamCoder is a machine learning system that learns to solve problems by writing programs, growing its own domain-specific languages and neural search strategies through a "wake-sleep" cycle inspired by human learning and memory consolidation. It demonstrates the ability to rediscover foundational concepts in programming, mathematics, and physics, and to generalize across diverse domains with data efficiency and interpretability, offering a step toward more human-like, compositional, and scalable AI.

DreamCoder: Growing Languages for Generative World Models

Q. What are the two types of inputs used to train DreamCoder's recognition network?
A. Programs sampled from the library, and programs found during waking.

The prompt directly targets the core insight the user highlighted: the dual training data sources (fantasies from the library and replays from waking) for DreamCoder's recognition network. It is atomic—testing a single, well-defined fact—and faithful to the highlighted concept, capturing the complete knowledge unit rather than a trivial fragment. It is also stable: 'DreamCoder's recognition network' is specific, self-explanatory terminology that remains clear in isolation months later. A minor improvement would be to include the paper's canonical terms ('fantasies' and 'replays') in the answer, but this does not undermine the prompt's clarity, retrievability, or fidelity to what the user found interesting.


Q. What are the two learning phases where the DreamCoder recognition network is trained?
A. Fantasies and Replays

The prompt directly targets the specific detail the user highlighted—the two sources ('Fantasies' and 'Replays') used to train DreamCoder's recognition network—making it meaningful and faithful to the user's interest. It is atomic (tests one well-defined fact), uses precise terminology drawn verbatim from the source, and remains self-explanatory and stably retrievable in isolation months later. One minor imprecision: the prompt labels Fantasies and Replays as 'learning phases' when they are more accurately training-data categories within the dreaming phase, but this does not obscure the recall target or introduce external information, so the prompt remains pluckable.


Duringeach sleep cycle the next library bootstraps off the concepts learned during earlier cycles, growing anincreasingly deep learned library. Simultaneously the generative and recognition models bootstrap eachother: A more specialized library yields richer dreams for the recognition model to learn from, whilea more accurate recognition model solves more tasks during waking which then feed into the nextlibrary. Both sleep phases also serve to mitigate the combinatorial explosion accompanying programsynthesis

The highlighted passage explains a key part of DreamCoder's learning mechanism that occurs during its iterative "wake-sleep" cycles, particularly focusing on how the system progressively improves its internal knowledge representations and problem-solving efficiency over time.

**Core ideas unpacked:**

1. **Library Bootstrapping and Growth Over Sleep Cycles:**
   During each sleep phase (particularly the abstraction phase), DreamCoder updates and extends its internal library of domain-specific programming abstractions—essentially building a richer set of reusable conceptual building blocks. This process is cumulative: the library built in one sleep cycle forms the foundation for learning in the next cycle. In other words, the system "bootstraps" its knowledge, gradually growing a deeper, more powerful set of concepts by abstracting common patterns from the programs it has found so far.

2. **Mutual Bootstrapping Between Generative and Recognition Models:**
   DreamCoder simultaneously maintains and improves two intertwined components:
   - The **generative model**, which encodes the current library, essentially representing prior knowledge about what programs look like in the domain.
   - The **recognition model**, a neural network guiding program search during waking by predicting plausible program components for new tasks.

   These two models enhance each other:
   - A richer, more specialized library (generative model) enables DreamCoder to create more structured and meaningful "dreams" (imagined programs and tasks) to train the recognition model. This makes the recognition model better at predicting programs for new tasks.
   - A more accurate recognition model, in turn, finds better and more programs during waking hours, which supply new data to improve the library in the next sleep cycles.

   This forms a virtuous cycle where the symbolic knowledge and neural procedural skill improve hand-in-hand.

3. **Mitigating the Combinatorial Explosion in Program Synthesis:**
   Program synthesis involves searching through an enormous space of possible programs, growing combinatorially with program size and complexity. The combined learning during the sleep phases curtails this explosion by:
   - Learning **higher-level abstractions** reduces the effective search depth, making solutions more succinct.
   - Improving the recognition model reduces the search breadth, focusing attention on the most promising program candidates.

In sum, the passage highlights how DreamCoder's cyclical learning approach enables it to build increasingly sophisticated domain knowledge and efficient search capabilities—symbolized by progressive library growth and recognition model training—that work together to tame the otherwise intractable search problem of program synthesis. This mutual bootstrapping underlies the system's ability to become a more expert solver as it accumulates experience.

Q. [DreamCoder: Sleep Phase] How does the _abstraction step_ affect the combinatorics of program synthesis?
A. Reduces depth by creating higher-level functions that simplify programs.

The prompt is pluckable. It targets a core mechanism explicitly present in the highlighted text: how DreamCoder's sleep phases mitigate the combinatorial explosion in program synthesis. Specifically, it isolates one atomic, well-defined concept—the abstraction step reduces search depth by learning higher-level functions that compress/simplify programs—which is a faithful and accurate representation of the source material. The question uses precise, source-grounded terminology ('abstraction step', 'combinatorics of program synthesis') and includes contextual anchoring ([DreamCoder: Sleep Phase]), so it remains clear and retrievable when reviewed in isolation months later. Although the highlight also describes the complementary breadth-reduction role of the recognition model and the mutual bootstrapping dynamic, omitting these does not undermine this prompt; each is a distinct knowledge unit better served by its own prompt, and splitting them aligns with the atomicity principle. The tested insight is durable, meaningful, and directly tied to what the user highlighted.


Q. How does DreamCoder mitigate the combinatorial explosion during sleep cycles?
A. It bootstraps on previously learned concepts.

The prompt targets the right topic area but fails on fidelity and stability. Its answer—'It bootstraps on previously learned concepts'—is too vague and actually conflates two distinct ideas from the highlight: cumulative library growth across sleep cycles versus the specific mechanisms that curb combinatorial explosion. Per the source material, the explosion is mitigated by (1) learning higher-level abstractions, which shortens programs and reduces search depth, and (2) training the recognition model, which narrows search breadth toward promising candidates. A reviewer months later retrieving 'bootstraps on previously learned concepts' would not recover this precise, durable causal insight, and the generic phrasing could apply to almost any iterative learning system. A pluckable version would isolate one concrete mechanism, e.g., asking how abstraction learning or recognition-model training specifically reduces the program-search space.


Q. According to DreamCoder authors, how do generative and recognition models interact?
A. They mutually enhance each other to mitigate the combinatorial explosion in program synthesis.

The prompt directly targets the core insight of the user's highlight: the mutual bootstrapping (virtuous cycle) between DreamCoder's generative and recognition models. It is atomic enough—testing one coherent relationship—and uses precise, source-faithful terminology ('generative model', 'recognition model', 'combinatorial explosion', 'program synthesis') that keeps it self-explanatory when reviewed in isolation months later. The mention of DreamCoder provides built-in context anchoring. While the answer compresses the directional details (richer library → richer dreams → better recognition model, and vice versa) and folds in the combinatorial-explosion benefit, both elements come straight from the highlighted passage and together form the complete, meaningful unit the user engaged with.


Q. In DreamCoder's sleep cycles, how does the learned library grow?
A. By building on concepts learned in earlier cycles

The prompt is atomic and faithful: it isolates one coherent idea from the highlighted passage—DreamCoder's library grows cumulatively by bootstrapping off concepts learned in prior sleep cycles—and phrases it with specific, self-explanatory language ('DreamCoder's sleep cycles,' 'learned library') that remains clear months later without the source context. It targets a genuine mechanism the user highlighted rather than a triviality, and the answer is stable and unambiguous. A minor weakness is that the answer closely restates the question and the prompt captures only one of the three intertwined ideas in the highlight (library bootstrapping, mutual generative/recognition bootstrapping, and combinatorial explosion mitigation), but as a single atomic memory it still satisfies the meaningfulness and stability criteria.


Q. [DreamCoder: Sleep Phase] How does the _dreaming step_ affect the combinatorics of program synthesis?
A. Reduces breadth by training a neural model to prioritize promising program paths. Together, they shrink the search space dramatically.

This prompt fails stability and fidelity checks. First, the answer is internally incoherent: it describes only one mechanism ('reduces breadth by training a neural model') yet concludes 'Together, they shrink the search space dramatically,' referring to a second mechanism (e.g., abstractions reducing search depth) that is never stated. Reviewed in isolation months later, the dangling plural 'they' would be confusing and unretrievable. Second, the scope is misaligned with the highlight: the passage states that *both* sleep phases mitigate the combinatorial explosion—the abstraction phase reduces search depth via richer libraries, and the dreaming/wake-sleep phase reduces search breadth via the recognition model—but the prompt isolates only the 'dreaming step,' fragmenting the coherent insight the user engaged with. Third, the phrasing mixes an incomplete causal claim with a vague summary rather than testing one atomic, well-defined fact.


Q. According to DreamCoder's authors, in what direction does mutual enhancement occur between the generative and recognition models in wake-sleep cycles?
A. The generative model influences the recognition model, and the recognition model's output influences the generative model

This prompt fails on meaningfulness and stability. First, it is nearly tautological: the question asks about the 'direction' of 'mutual enhancement,' and since 'mutual' by definition implies bidirectionality, the answer ('generative influences recognition, and recognition influences generative') can be deduced from the question alone without any memory of the source material—so it tests a triviality rather than reinforcing knowledge. Second, it discards the substantive, memorable content of the highlight: the specific causal mechanisms (a richer library yields richer 'dreams' to train the recognition model; a better recognition model solves more tasks during waking, which feed into the next library abstraction cycle). A pluckable version would target that concrete virtuous-cycle mechanism, e.g., asking what the generative model provides to train the recognition model and what the recognition model contributes back to the library. As written, the prompt is vague ('in what direction'), generic, and would not durably anchor the user's actual interest.


Not specified (preprint, likely 2021-2022)

Summary: This review surveys the vast differences in learning rates across animal tasks, from rapid one-shot learning to ultra-slow acquisition requiring thousands of trials. Meister argues that fast learning is enabled by neural representations—often sparse and high-dimensional—either genetically encoded or slowly acquired through experience. Slow learning, especially in laboratory tasks with abstract or unnatural contingencies, may reflect the need to first develop suitable neural codes. The article challenges the field’s focus on slow-learning tasks in model organisms and suggests that understanding the mechanisms of fast, naturalistic learning may be more informative for neuroscience.

Learning Fast and Slow: A Survey of Learning Rates and Neural Mechanisms in Animals

Another case of learning from single exposure is the Bruce effect: A female mouse learns the smellof her mating partner and uses that to recognize the mate in subsequent encounters. If she meets astrange male that threatens infanticide, she will terminate the pregnancy [4, 5] about half the time,whereas if she meets her mate again that does not happen. Learning the mate’s odor changes hersubsequent behavior in response to this particular mouse. Again the complexity of this task is onebit. Note that the physiological pathways underlying the Bruce effect are innate and do not need tobe learned. However, to implement the program the dam must acquire a small amount of olfactoryinformation during the mating experience and retain that through subsequent encounters.

The highlighted passage discusses the Bruce effect as a striking example of very rapid, single-exposure learning in animals—specifically, female mice. Here, the female mouse acquires the odor of her mating partner during a brief encounter and then uses that olfactory memory to distinguish him from strange males in later encounters. This distinction is behaviorally consequential: if she encounters an unfamiliar male that poses a risk of infanticide, she will terminate her pregnancy about half the time; if she meets her familiar mate, the pregnancy continues.

The key idea is that this learning—recognizing and remembering a specific mate’s scent—is accomplished after just one exposure, qualifying as a "one-shot" learning event with a task complexity of one bit (a simple binary distinction: mate versus non-mate). Although the physiological mechanisms (the pathways triggering the pregnancy block) are innate, the implementation of this effect still requires the female to acquire and store sensory information about the mate’s scent. Thus, while the reflexive biological system for pregnancy termination is genetically programmed, the actual olfactory learning—the critical sensory information that guides this innate system in a flexible, experience-dependent way—is acquired through experience and retained for future use.

In context, this example illustrates how certain biologically vital behaviors depend on fast learning of essential environmental cues tied to survival and reproduction. It complements the broader argument that some forms of learning happen extremely quickly, especially when they rely on innate neural circuits that only need to incorporate a small amount of sensory information, versus other types of learning which require slow, repeated exposures. The Bruce effect exemplifies how an innate behavioral program can integrate a minimal but critical sensory memory acquired in a single event to shape future adaptive behavior.

Q. Why is the Bruce effect classified as a one-bit learning task?
A. It produces a binary behavior. Strange Male: terminate pregnancy. Familiar mate: maintain it.

This prompt is well-targeted to what the user highlighted: the passage explicitly notes 'the complexity of this task is one bit,' and the interpretation centers on this one-bit characterization as the core insight. The prompt is atomic—it asks a single, well-defined question—and the answer cleanly captures the binary mate-vs-non-mate discrimination that makes the task 'one bit.' It is also stable: the prompt names the Bruce effect, providing built-in context so it remains self-explanatory months later without the source material. A minor caveat is that the answer slightly oversimplifies by stating pregnancy termination occurs deterministically with a strange male (the source says 'about half the time'), and it frames the bit as a behavioral output rather than an olfactory discrimination, but these are acceptable simplifications that preserve the essential concept faithfully.


Q. Why is the Bruce effect a one-bit, one-shot learning task?
A. It involves a binary decision—terminate or maintain pregnancy—based on a single exposure to a male’s scent.

The prompt targets the core insight of the highlighted passage: the Bruce effect as a canonical example of fast, minimal-complexity learning. The question is anchored by the named phenomenon ('Bruce effect'), giving it built-in context for isolated review months or years later, and the answer cleanly connects both properties the user cared about—the one-bit complexity (binary mate vs. non-mate discrimination) and the one-shot nature (odor learned in a single mating exposure). While it bundles 'one-bit' and 'one-shot,' these are tightly coupled facets of a single coherent idea in the source, not two independent facts, so atomicity is preserved. The answer is concrete and faithful to the text; the slight simplification (termination occurs only ~half the time with strange males) does not undermine the essential binary-classification logic or long-term retrievability.


Q. Why is the Bruce effect classified as a one-bit learning task?
A. It produces a binary behavioral outcome: either terminate pregnancy (strange male) or maintain it (familiar mate).

This prompt is pluckable. It targets a specific, explicit claim from the highlighted passage—that the Bruce effect constitutes a one-bit learning task—and asks for its underlying reason, which is directly supported by the source: the binary discrimination between mate and non-mate leading to two distinct behavioral outcomes (terminate vs. maintain pregnancy). The prompt is atomic, testing a single well-defined concept; it is faithful to the user's evident interest in how fast learning can have minimal informational complexity while driving adaptive behavior; and it is stable, since the question names the Bruce effect and provides enough built-in context to remain self-explanatory during isolated reviews months later. The answer is concrete and precise, mapping cleanly onto the highlighted text.


A good laboratory example of "one-shot" learning occurs during fear conditioning in rodents. Here aformerly innocuous stimulus gets associated with a painful experience, such as electric shock, leadingto subsequent avoidance of the conditioned stimulus [ 3 ]. Rats and mice will form this associationafter a single experience lasting only seconds, and alter their behavior over several hours, withnear-perfect recognition of the conditioned stimulus. This is an adaptive warning system to dealwith life-threatening events, and rapid learning in this case has a clear survival value. Formally, theresulting behavior maps that stimulus onto one of two actions: freeze vs proceed. So the complexityof this task is one bit, representing the maximum amount of information that the animal can transferfrom its experience to future behavior (Section A).

The highlighted passage uses fear conditioning in rodents as a clear example of "one-shot" learning, where an animal acquires a new behavior after just a single, brief experience. Specifically, a stimulus that was previously neutral (like a sound) becomes associated with a painful event (e.g., an electric shock). After this one-time pairing, rats or mice quickly learn to avoid that stimulus, showing a near-perfect behavioral response (freezing to the stimulus) within hours.

This rapid learning is described as an adaptive, survival-related warning system—it allows the animal to react swiftly to potential dangers without needing repeated exposure to harmful stimuli. The passage further formalizes this behavior as a decision between two possible actions—freeze or proceed—thereby quantifying the task's complexity as one bit of information. This means the animal is effectively transferring a full unit of binary information (dangerous/not dangerous or freeze/do not freeze) from a single experience to influence future behavior.

In the broader context of the article, this example illustrates a form of fast learning where the animal efficiently extracts meaningful information from minimal experience, contrasting with other tasks that require extensive repetition to learn even simpler associations. This underscores the point that biological systems can and do learn certain critical, ecologically relevant associations extremely quickly, likely because such fast learning confers significant survival advantages.

Q. What is _'one-shot'_ learning in the context of rodent fear conditioning?
A. Permanent behavioral change after a single exposure to a stimulus paired with pain

The prompt targets the core insight of the highlighted passage: one-shot learning in rodent fear conditioning, where a single brief pairing of a stimulus with pain produces a lasting behavioral change. It is atomic (tests one well-defined concept), stable (self-contained phrasing that remains clear months later without the source context), and meaningful (directly reflects what the user highlighted—the remarkable speed and durability of this adaptive warning system). Minor caveats: the answer's use of 'permanent' slightly overstates the source ('alter their behavior over several hours'), and it omits that the stimulus was originally innocuous, but these do not undermine the prompt's fidelity to the central idea.


Q. What is the information complexity (in bits) of the fear conditioning response in rodents?
A. One bit: freeze or proceed.

The prompt is atomic, targeting a single well-defined fact explicitly stated in the highlighted passage: the fear conditioning task maps onto two actions (freeze vs. proceed), giving it an information complexity of one bit. This quantification is a distinctive, analytically interesting element of Meister's information-theoretic framing that the user deliberately highlighted, so it captures a detail they genuinely care about rather than a triviality. The question is also stable: 'fear conditioning in rodents' and 'information complexity in bits' are precise, self-explanatory terms that remain clear in isolation months or years later, and the answer ('one bit: freeze or proceed') embeds its own justification, reinforcing recall. It faithfully represents the complete coherent unit of knowledge from the highlight without introducing external information.


Q. What is the information complexity (in bits) of the fear conditioning response in rodents, and why?
A. One bit, because the learning results in exactly two possible behavioral outcomes: freeze or proceed.

The prompt is pluckable. It targets a specific, quantitative insight explicitly stated in the highlighted text: that the fear conditioning task maps a stimulus onto exactly two behavioral outcomes (freeze vs. proceed), yielding an information complexity of one bit. This is a precise, memorable formalization that the user clearly engaged with, as it appears verbatim in the highlight. The prompt is atomic (the 'why' directly explains the 'what' as a single coherent concept), stable (the terminology is concrete and self-explanatory in isolation), and faithful to the source material. While the broader context involves one-shot learning and survival value, the one-bit quantification is a distinctive, durable insight that stands alone well on long review intervals.


Q. What is 'one-shot' learning in the context of rodent fear conditioning?
A. A learning process where a single exposure to a stimulus paired with pain leads to permanent behavioral change, occurring within seconds and consolidating over hours.

The prompt targets a relevant concept (one-shot learning via rodent fear conditioning) and its question is specific and self-contained. However, it fails on fidelity and stability: the answer asserts a 'permanent' behavioral change, which is not claimed in the highlighted text—the source only states the animal alters its behavior over several hours with near-perfect recognition. This overgeneralization could mislead on long review intervals. Additionally, the prompt omits the most distinctive element of the highlight: the formalization of the task as a binary choice (freeze vs. proceed) with a complexity of one bit, which the user's interpretation emphasized as the key quantitative insight. As written, the prompt tests a generic definition while drifting from the source's actual claims.


Why use information theory?What recommends these information-theoretic measures of behavior, as compared to any otherstatistical tool that correlates states with actions? An important desirable for any measure of taskcomplexity is additivity: If a task is composed of two independent subtasks, the complexity of thecombined task should be the sum of the individual complexities. The definition for complexity inEqn 1 satisfies this additivity constraint. For example, this allows us to derive the complexity ofthe binary decision tree in Figure 4b: The shortest route from state s1 to the reward requires onecorrect 2-way decision followed by two correct 3-way decisions. If the actions "left", "right" and"back" are equally probable to begin with, the complexity is T = log2 2 + 2 × log2 3 = 4.2 bits. Alarger tree with 64 end nodes, as used in a recent experimental study [8], will have task complexityT = log2 2 + 5 × log2 3 = 8.9 bits.

The author here explains why information theory is particularly well-suited to measuring learning and task complexity in animals, compared to other statistical approaches that might simply correlate stimuli (states) with responses (actions).

The key advantage highlighted is **additivity**: when a task can be broken down into independent subtasks, the overall complexity measure should equal the sum of the complexities of these component subtasks. This makes the measure compositional and scalable—a highly desirable property for characterizing complex behaviors that often involve sequences or combinations of simpler decisions.

Using the information-theoretic measure defined earlier in the text (likely the entropy-based "complexity" from Eqn 1), this additivity naturally holds. The author illustrates this with a binary decision tree example (Figure 4b). For an animal navigating to a reward location through a sequence of decisions, the total task complexity is calculated by summing the log-base-2 of the number of equally probable options at each decision point. For instance, starting at state s1, the animal first makes a 2-way choice (log2 2 = 1 bit) followed by two 3-way choices (each log2 3 ≈ 1.58 bits), summing to about 4.2 bits total complexity.

This approach also generalizes to larger trees, like the 64-end-node structure studied in reference [8], where the total complexity sums up to about 8.9 bits by including more decision points (one 2-way plus five 3-way stages).

In summary, the author argues that information theory provides a principled, quantitative, and compositional metric of task complexity, capturing how much information an animal must learn to perform a given behavior. This property is critical for comparing and understanding diverse learning tasks, especially when tasks can be decomposed into sequences of decisions.

Q. In a decision tree with one 2-way decision followed by two 3-way decisions, what is the total complexity in bits (assuming equal initial probabilities)?
A. 4.2 bits ($log_2 + 2log_3$)

This prompt targets a decorative numerical example rather than the core insight the user highlighted. The user's interest was clearly in WHY information theory suits measuring task complexity—specifically the additivity property: if a task decomposes into independent subtasks, total complexity equals the sum of the subtask complexities. The prompt instead asks for recall of a specific computed value (4.2 bits) from Figure 4b, which is merely an illustration of that principle. Testing the exact number fails the meaningfulness criterion (it tests the trees, not the forest) and is also weak on stability: an arbitrary decimal like '4.2 bits' is hard to retrieve reliably months later in isolation, and the answer formatting ('$log_2 + 2log_3$') is imprecise, omitting the actual operands (log2 2 + 2 × log2 3). A pluckable version would ask about the additivity constraint itself—e.g., 'What desirable property must a measure of task complexity satisfy when a task consists of independent subtasks, and how does the information-theoretic definition meet it?'


According to current understanding, the strength of a synapse gets modified depending to the historyof activity in the pre- and the post-synaptic neurons [ 24 ]. Of particular importance is the degreeof coincident activity, which controls whether the synapse experiences long-term potentiation ordepression. Sometimes the degree of this plasticity is modulated by a third variable, such as a diffuseneuromodulator that may represent the valence of the animal’s experience. In addition there are somenon-associative mechanisms, for example a neuron’s threshold for firing depends on its own recentactivity. Synaptic plasticity may appear at multiple pre-synaptic and post-synaptic sites, and it canact on several different timescales [ 25]. However, all these effects are ultimately driven by threevariables of neural activity: presynaptic firing, the postsynaptic membrane potential, and (optionally)the concentration of a neuromodulator.To appreciate just how constraining these rules are, it helps to compare to the learning algorithmsused in artificial neural networks (ANNs). In a typical optimization algorithm for an ANN, eachsynaptic weight gets updated in a way that may depend on all the other weights and all the activitiesin the network [ 26]. Every synapse "sees" every other synapse and all the neurons. In a network thesize of cortex, the change in one synapse might depend on the state of a billion other variables. In thebiological cortex, it depends on only three. One should expect that such a limited learning rule willimpose strong constraints on what can and cannot be learned in biological networks

The highlighted passage focuses on the biological mechanisms of synaptic plasticity—how synapses change their strength based on neural activity—and contrasts this with learning rules used in artificial neural networks (ANNs). The author explains that in biological brains, synaptic modifications depend primarily on three variables: the firing activity of the presynaptic neuron, the membrane potential of the postsynaptic neuron, and optionally, the concentration of neuromodulators that can signal the valence (positive or negative value) of an experience.

This plasticity is typically governed by coincident activity between pre- and postsynaptic neurons, determining whether synaptic strength increases (long-term potentiation) or decreases (long-term depression). Additional non-associative processes, such as changes in a neuron's firing threshold based on recent activity, also contribute to synaptic changes. These mechanisms operate at multiple synaptic sites and over various timescales.

The passage then highlights a critical contrast: in ANNs, synaptic weights (analogous to biological synapses) can be updated based on the activity and states of virtually all other neurons and synapses in the network. This global interaction means that any synapse's change can depend on myriad factors—potentially billions of variables in a cortex-sized network.

In stark contrast, biological synapses are constrained to adjust based on only three local variables, drastically limiting the complexity and flexibility of learning in biological systems compared to artificial ones. Because of this, the author argues that such a restricted learning rule imposes fundamental constraints on the kind of information biological neural networks can encode and what kinds of learning they can achieve readily.

In essence, the passage underscores the simplicity and locality of biological synaptic plasticity rules against the global, flexible updates possible in artificial networks, framing this as an important constraint shaping biological learning capabilities. This insight supports the broader theme of the article regarding why some learning processes in animals are inherently slow or limited and how the brain’s neural coding must be adapted to these constraints.

Q. What are the three variables that control synaptic plasticity in biological neurons?
A. 1. Presynaptic firing
2. Postsynaptic membrane potential
3. Neuromodulator concentration (optional)

The prompt is pluckable. It isolates a single, well-defined fact that the author explicitly emphasizes as the anchor of the highlighted passage: biological synaptic plasticity is ultimately driven by three local variables (presynaptic firing, postsynaptic membrane potential, and optionally neuromodulator concentration). This is meaningful rather than trivial—it is the factual foundation for the passage's core argument about the locality of biological learning rules versus the global weight updates in ANNs, and it directly reflects what the user highlighted. The prompt is also stable: it uses precise, standard terminology ('synaptic plasticity', 'presynaptic firing', 'postsynaptic membrane potential', 'neuromodulator'), is self-contained without needing the original document context, and has a clear, unambiguous answer faithful to the source (including the 'optional' qualifier on neuromodulators). While a complementary prompt about the contrast with ANN updates (three variables vs. potentially billions) would capture the comparative insight even more fully, this prompt cleanly and durably captures a key, self-contained unit of knowledge from the highlight.


Q. In a cortex-sized network, how many variables can influence a single synaptic change in biological vs artificial networks?
A. Biological: 3 variables; Artificial: up to billions of variables

The prompt targets the central, most striking insight of the highlighted passage: Meister's explicit comparison that a single biological synapse's change depends on only three local variables (presynaptic firing, postsynaptic membrane potential, optional neuromodulator), while an ANN weight update in a cortex-sized network may depend on billions of other variables. This quantitative contrast ('three' vs 'a billion') is precisely what the author uses to illustrate how constraining biological plasticity rules are, and it anchors the broader theme of limits on biological learning. The prompt is meaningful (captures the user's evident interest in the bio-vs-ANN comparison), atomic (tests one well-defined fact), and stable (uses concrete, self-contained language—'cortex-sized network', 'biological vs artificial', specific counts—that remains answerable in isolation months later). Its answer is faithful to the source text, which states the dependence 'on only three' variables versus 'a billion other variables'. A minor imperfection is that recalling the identity of the three variables would deepen retention, but the core numeric contrast is durable and clearly pluckable.


Humans engage in some slow learning tasks as well, particularly in esoteric domains like art, music,and sports. For example, to become a golf pro takes about 5 years of practice, hitting perhaps 100drives a day. Top players can land an approach shot within about 5% of the hole, from varyingdistances [21]. Supposing the player could hit twice as far if she wanted and in all possible directions,that amounts to a precision of 11 bits after several years of training (see Section A.3). Clearly this isanother case of ultra-slow learning (Figure 1).From this survey it emerges that the learning rate spans an enormous range, even within the samespecies. For example, a mouse exploring a labyrinth extracts about 10 bits of information afterreceiving 10 rewards, whereas the same mouse strapped into a visual discrimination box will learn afraction of 1 bit after 5000 rewards. Another observation is that tasks phrased in the animal’s naturalvocabulary – such as mating, escaping, feeding, or navigation – get learned much faster than tasksthat require abstract or arbitrary associations between contexts and actions. The wide gap of 4 logunits across learning rates (Figure 1) suggests that fast and slow learning are served by very differentbiological mechanisms. Pursuing that large discrepancy may well reveal something interesting aboutthe brain

The highlighted passage emphasizes a key finding of the article: learning rates vary dramatically across tasks and contexts, even within the same species, including humans. It contrasts ultra-slow human learning examples—such as becoming a professional golfer, which requires about five years and billions of hits leading to very high precision (quantified as 11 bits of task-relevant information)—with very fast learning observed in animals, such as mice rapidly extracting about 10 bits of information after only 10 reward experiences while navigating a labyrinth. In stark contrast, the same mice learn much less (less than 1 bit) even after thousands of trials on relatively simple but abstract visual discrimination tasks in laboratory conditions.

The passage draws a crucial distinction that tasks involving natural, evolutionarily relevant behaviors—like mating, escaping predators, feeding, or spatial navigation—are learned much faster by animals than tasks requiring abstract or arbitrary associations between sensory cues and actions. This suggests that the brain’s capacity for learning is not uniform but depends heavily on how closely the learning content aligns with the animal's natural "vocabulary" or ecological and evolutionary niche.

Finally, the author highlights that the observed 10,000-fold (4 log units) difference in learning rates strongly implies that fast and slow learning are underpinned by fundamentally different biological mechanisms. Recognizing and pursuing an understanding of this disparity promises to uncover important insights into brain function and learning processes.

In summary, this passage situates the enormous variability in learning speed within a conceptual framework that ties rapid learning to naturalistic, evolutionarily-salient tasks supported by specialized neural coding, while slow learning tends to involve unnatural or artificial associations that the brain must acquire more laboriously, possibly involving different or additional neural mechanisms.

Q. How does a mouse's learning rate differ between exploring a labyrinth vs. visual discrimination? (bits per reward)
A. Labyrinth: 10 bits of information after 10 rewards; Visual discrimination: fraction of 1 bit after 5000 rewards

This prompt is pluckable. It targets the single most striking empirical anchor of the user's highlight: the same mouse extracting ~1 bit per reward while navigating a labyrinth versus gaining less than 1 bit after thousands of rewards in an artificial visual discrimination box. This contrast is not a decorative detail—it is the concrete embodiment of the survey's headline finding that learning rates span orders of magnitude within one species, and it implicitly encodes the natural-vocabulary-versus-abstract-task distinction the user engaged with. The prompt is atomic (one coherent contrast, not two unrelated facts), uses precise source terminology ('bits', 'rewards', 'visual discrimination'), and its extreme numeric asymmetry makes it highly retrievable in isolation months later, satisfying both the meaningfulness and stability criteria. Its main limitation is that it omits the author's mechanistic interpretation (different biological substrates for fast vs. slow learning), but that omission does not undermine the durability of the tested fact itself.


Summary: This paper investigates how transformer-based language models, particularly GPT-2 and its variants, learn and reason over complex, hierarchical structures defined by synthetic context-free grammars (CFGs). By constructing challenging CFGs that require dynamic programming (DP) for parsing, the authors show that generative transformers can not only learn these grammars but also internally represent their hierarchical structure and mimic DP-like reasoning. The study introduces novel probing techniques to reveal that hidden states and attention patterns in transformers encode deep syntactic information, and explores the impact of architectural choices and data corruption on model robustness and interpretability.

How Do Transformers Reason? Interpreting Transformers on Synthetic Context-Free Grammars

A probabilistic context-free grammar (CFG) is a formal system defining a string distribution usingproduction rules. It comprises four components: terminal symbols (T), nonterminal symbols (NT),a root symbol (root ∈ NT), and production rules (R). We represent a CFG as G = (T, NT, R),with L(G) denoting the string distribution generated by G.

The highlighted passage defines what a probabilistic context-free grammar (CFG) is and outlines its fundamental components. Specifically, it explains that a CFG is a formal system used to generate strings according to certain probabilistic rules. The CFG consists of:

- **Terminal symbols (T):** These are the basic symbols or tokens that appear in the final strings generated by the grammar.
- **Nonterminal symbols (NT):** These are abstract symbols used during the generation process to define hierarchical structure and intermediate forms. Among these, one is designated as the root symbol, which is the starting point for generating strings.
- **Production rules (R):** These are the rules that specify how nonterminal symbols can be expanded or rewritten as sequences of terminals and/or other nonterminals, thereby defining the language (set of possible strings).

The CFG is represented as a tuple \( G = (T, NT, R) \), and the notation \( L(G) \) denotes the distribution over strings generated by the grammar \( G \).

Within the broader context of the paper, this definition sets the stage for the authors’ use of CFGs as a controlled, rigorously defined framework to study and interpret how transformer-based language models like GPT learn and reason over hierarchical and recursive language structures. By working with CFGs, which have clear algorithmic parsing procedures (such as dynamic programming), the authors aim to precisely characterize the internal mechanisms and representations that transformers develop when generating and parsing complex sequences. This contrasts with natural language, where such strict formalizations and algorithmic clarity are lacking, and helps them better understand transformer reasoning capabilities in a well-defined setting.

Q. In the notation $L(G)$ for a context-free grammar, what does $L(G)$ represent?
A. The string distribution generated by the grammar $G$

The prompt isolates a single, well-defined fact taken directly from the highlighted text: that L(G) denotes the string distribution generated by grammar G. It is atomic (tests exactly one concept), uses precise terminology from the source, and remains fully self-explanatory when reviewed in isolation months later—no original context is needed to understand or answer it. Although it covers only a slice of the full CFG definition the user highlighted, it is not a decontextualized triviality: L(G) is the central output notation of the formalism and is used throughout the paper, so retaining its meaning genuinely supports long-term engagement with the material. It faithfully reflects what the user highlighted without introducing external information.


Q. What distinguishes terminal symbols ($T$) from nonterminal symbols ($NT$) in a context-free grammar?
A. Terminal symbols ($T$) are final elements in the output string, while nonterminal symbols ($NT$) can be expanded.

This memory prompt is pluckable. It targets the core definitional insight from the user's highlighted text: the distinction between terminal and nonterminal symbols, which is the foundational structural element of the CFG definition they engaged with. The prompt satisfies the key criteria: (1) Meaningful — the user deliberately highlighted the formal definition of CFGs and its components, indicating genuine interest in retaining these fundamentals, which serve as the basis for the paper's framework for interpreting transformer reasoning; (2) Stable — the question is self-contained and uses precise notation ($T$, $NT$) directly tied to the source material, remaining unambiguous even when reviewed in isolation months later; (3) Atomic — it tests exactly one well-defined distinction without compounding multiple concepts; (4) Faithful — the answer accurately reflects the interpretation provided (terminals as final output elements, nonterminals as expandable intermediates). While the answer could be marginally enriched by mentioning that expansion occurs 'via production rules,' this does not undermine the prompt's clarity or durability. The concept is also timeless CS knowledge, making it ideal for long-term spaced repetition.


Summary: Zheng and Meister investigate the striking discrepancy between the high data rates of human sensory input (gigabits per second) and the much lower information throughput of human behavior and cognition (about 10 bits per second). They review evidence from diverse domains—typing, speech, memory sports, and neuroscience—to show that this low throughput is a universal feature of human cognition, not a limitation of motor output or memory. The authors argue that this paradox remains largely unexplained, challenge common misconceptions about photographic memory and subjective experience, and propose that the brain operates in two distinct modes: a high-dimensional, parallel "outer brain" and a low-dimensional, serial "inner brain." They call for new research to understand the neural bottlenecks and architectures underlying this profound information bottleneck.

The Paradoxical Slowness of Human Behavior

Another objection to a very low rate of perception is that, by definition, it only takes into account things that we canconsciously experience and access, so that we can talk about them 33. Surely, this argument goes, the myriad imagesthat flit over our retina every day must carry a great deal of fine-grained information into the brain. Perhaps thisinformation cannot be accessed consciously as photographic memory, but it is still stored in the brain.A specific proposal for unconscious processing involves the development of visual circuits. It is claimed that the visualsystem learns about the statistics of natural scenes from being exposed to millions of those images over time. Throughmechanisms of synaptic plasticity 34, the structure of neural circuits responds to visual experiences in the course ofearly development, which alters the subsequent function of those circuits35. How much information does this processextract from the environment?The famous stripe-rearing experiments on kittens can serve as an example36. The kittens were raised in an environmentpainted with vertical black and white stripes. From the kitten’s perspective, because it still got to shake its head, theorientation of the stripes was restricted to perhaps ±20 degrees from the vertical. This did lead to some reorganizationof brain wiring in the visual cortex. Let us suppose that the kitten brain has completely incorporated the change invisual statistics. Whereas in the natural world, all orientations are about equally represented across 180 degrees, in thenew environment they are restricted to a 40-degree range. The information associated with such a change in probabilitydistributions is log2(180/40) ≈ 2 bits.By perfectly learning the new distribution of angles, the kitten acquired only two bits. The same argument appliesto other aspects of natural image statistics: The distribution of spatial and temporal frequencies differs from theuniform, but the shape of that distribution can be captured by just a few bits37. This is also true of the distribution ofcolor spectra. In each of these cases, processes of activity-dependent development can learn only a few bits over theanimal’s lifetime.

The highlighted passage addresses a common objection to the idea that human perception operates at a very low information rate (around 10 bits/s): the objection is that perception only accounts for what we consciously experience and can articulate, but our sensory systems presumably take in a vastly richer, fine-grained stream of visual information unconsciously.

The authors engage this objection by focusing on unconscious processing through developmental plasticity—specifically, how the visual system adapts to natural scene statistics during early life. They use the example of the classic "stripe-rearing" experiments on kittens, where kittens raised in an environment with only vertical black and white stripes experienced a highly constrained set of orientations (~±20 degrees from vertical). This limited sensory input caused reorganization of the neural circuitry in the primary visual cortex.

However, the authors quantify the information gained through this experience: since natural orientation varies across 180 degrees, but the kittens’ environment only spanned 40 degrees, the difference in visual statistics corresponds to about 2 bits of information (log2(180/40) ≈ 2 bits). That is, the developmental plasticity processes encoded only a few bits about the altered environment.

They extend this logic to other natural image statistics—spatial and temporal frequency distributions and color spectra—arguing that these are also learned or encoded with just a few bits across an animal’s lifetime. This is largely because many natural scene statistics are invariant or have low-dimensional structures, meaning the brain doesn’t have to learn or store huge amounts of detailed information separately at every point in the visual field.

In essence, the passage supports the authors’ broader claim that even unconscious visual learning and development operate on surprisingly low information rates. While the raw sensory input to the brain is enormous, the neural adaptations and learned internal models of the visual environment require encoding only minimal amounts of information. This undercuts the objection that unseen unconscious processes might vastly increase perception’s information throughput, reinforcing the paradox of the brain’s slowness in terms of usable information despite massive sensory input.

Q. How many bits of information did kittens acquire when learning a restricted range of stripe orientations (±20° from vertical) compared to natural environments?
A. 2 bits (calculated as log₂(180/40))

This prompt is pluckable. It is atomic, testing one well-defined quantitative result from the highlighted passage: that the stripe-rearing kittens' perfect adaptation to the restricted orientation environment yielded only ~2 bits (log₂(180/40)). The question is self-contained and stable—it specifies the experimental context (kittens, ±20° stripe orientations), so it remains interpretable months later without the original document, and the answer is precise and verifiable. Importantly, it stays faithful to what the user engaged with: the 2-bit figure is not decorative trivia but the punchline of the authors' argument that even unconscious developmental plasticity extracts only a few bits from the environment, directly undercutting the objection about rich unconscious perception. Recalling this surprising, concrete number naturally re-anchors the broader insight about the brain's low information throughput.


The retina projects directly to the superior colliculus (SC, also called optic tectum), which is thought to act as a furtherbottleneck. Neurons in the upper layers of the SC have fine-grained receptive fields similar to those in the retina. Bycontrast, the lower layers of the SC represent pre-motor signals that reflect the animal’s behavioral decisions. Examplesinclude the saccadic eye movements of monkeys, reorienting of the head in owls, or approach vs avoidance of a novelobject by rodents 103–105. Thus it appears that the SC distills a massively parallel and high-dimensional sensory inputinto a map of suitable targets for action. The circuit mechanisms underlying this sifting are coming into view106–108.Recent work suggests that the SC also plays a role in phenomena of attention that do not involve overt movement109.Perhaps it contributes to a bottom-up “saliency map”110 that ultimately directs the attentional bottleneck in cognition.Second, what are the principles of neural function on the two sides of the interface? Taking the water flow analogyfurther: The design of the Hoover Dam turbines relies on rather different engineering principles from the design of abeer bottle. No one would consider making them of the same materials, given the enormous difference in performance.Yet the brain seems to rely on the same materials throughout: neurons, synapses, and glia. The cerebral cortex is toutedas the substrate for both rapid and parallel sensory processing (e.g. visual cortex) and for slow and serial cognition

The highlighted passage discusses the role of the superior colliculus (SC) as a crucial "bottleneck" in the flow of sensory information from the retina to behavioral actions. The authors explain that neurons in the upper SC layers have receptive fields similar to those in the retina, processing detailed sensory input in a massively parallel and high-dimensional manner. In contrast, the lower SC layers translate this sensory information into premotor signals that direct specific behavioral decisions—such as eye movements in monkeys, head orientation in owls, or approach/avoidance behaviors in rodents. This suggests the SC acts as a key site where the brain distills vast, parallel sensory data into a simplified map of actionable targets.

Additionally, emerging research indicates the SC is involved not only in movement-related functions but also in attention, possibly generating a bottom-up "saliency map" that guides the brain’s attentional bottleneck—i.e., the selective focus on particular pieces of information for cognitive processing.

The passage then uses the analogy of the Hoover Dam's turbines versus a beer bottle to highlight a puzzle: although different computational functions (fast, parallel sensory processing vs. slow, serial cognition) seem to demand very different "engineering" designs, the brain uses the same fundamental biological "materials" (neurons, synapses, glial cells) across these functions. For example, the cerebral cortex supports both the fast, parallel processing in sensory areas (like the visual cortex) and the slow, serial processing in cognitive areas, despite their differing operational requirements.

In context, this underscores one of the article's central conundrums: how the brain reconciles the large disparity between the complexity and speed of sensory input processing versus the much slower, low-dimensional processing seen in cognition and behavior. The SC exemplifies an early major "filtering" or "sifting" stage that reduces vast sensory information into manageable signals for decision-making, yet the principles underlying how neural circuits achieve such drastic compression and how the same neural substrates support vastly different processing modes remain poorly understood. This gap points to key areas for future neuroscience research.

Q. What is the primary transformation function of the superior colliculus in sensory processing?
A. Converts (parallel) retinotopic and auditory spatial maps into a coordinate framework for directing attention and gaze.

While the prompt targets a genuinely interesting concept—the superior colliculus as a processing bottleneck—the provided answer fails fidelity to the highlighted text on two counts. First, it introduces 'auditory spatial maps,' information absent from both the highlight and its interpretation, which focus exclusively on direct retinal input to the SC. Second, it narrows the SC's function to 'directing attention and gaze,' omitting the core insight the user engaged with: the SC distills massively parallel, high-dimensional sensory input into a low-dimensional map of suitable targets for broad behavioral actions (saccades, head reorienting, approach/avoidance). This compression/bottleneck framing is the essential idea anchoring the passage within the paper's larger thesis, and it is lost here. A pluckable version should test that specific distillation principle without importing external details.


Q. What is the primary transformation function of the superior colliculus in sensory processing?
A. It converts massively parallel, high-dimensional sensory input into a map of suitable targets for action.

The prompt isolates the core insight the user highlighted: the superior colliculus acts as a bottleneck that distills massively parallel, high-dimensional sensory input into a map of suitable targets for action. It satisfies atomicity (tests exactly one well-defined concept), fidelity (the answer closely paraphrases the highlighted claim rather than introducing external detail or testing a trivial fragment), and stability (naming the superior colliculus explicitly and using the source's own terminology keeps the question unambiguous and self-explanatory when reviewed months later without the original context).


t is generally thought that memory is stored in the strengths of synaptic connections,and to some extent in the properties of individual neurons. So we can obtain an upper bound on the brain’s capacityby counting the number of synapses (∼ 1014) and multiplying by the dynamic range of synaptic strength, estimated at5 bits44, resulting in 5 × 1014 bits ≈ 50 Terabytes required to specify all synapses. One may want to add a few bitsper neuron to specify its functional parameters, but given the small number of neurons (∼ 1011), that will leave theestimate at the same order of magnitude. This calculation assumes that any combination of connections is possibleand leads to a viable brain, so the upper bound of 50 TB is also a vast overestimate.

The highlighted passage discusses an estimate of the brain's theoretical memory storage capacity based on synaptic connections. The authors explain that memory is conventionally believed to be encoded primarily in the strengths of synapses—the connections between neurons—and partially in the properties of individual neurons themselves.

To estimate an upper bound on how much information the brain can store, they multiply two key factors:

1. The number of synapses in the brain, approximately 10^14 (100 trillion).

2. The dynamic range of synaptic strength, estimated at about 5 bits per synapse. This reflects how many distinguishable strength states a synapse can take, effectively quantifying how much information one synapse can hold.

Multiplying these gives roughly 5 × 10^14 bits, which equates to about 50 terabytes (TB) of data, representing the amount needed to specify all synaptic connections in the brain. Adding some additional bits per neuron to account for individual neuron parameters (there are roughly 10^11 neurons) does not significantly alter this estimate, since neuron count is much smaller than synapse count.

However, the authors emphasize that this is an *upper bound*—an overestimate. It assumes that any arbitrary combination of synaptic connections is possible and would result in a functioning brain, which is unlikely. Real brains are much more constrained; many connection patterns are biologically implausible or nonviable, so the actual information content stored is likely far less.

In summary, the passage conveys that while the brain’s capacity for storing synaptic information is large—on the order of tens of terabytes—this number is a theoretical maximum and does not necessarily represent the actual usable memory storage of the brain. It sets a scale for brain capacity that helps frame other discussions, such as comparing the limited bandwidth of human cognition to its extensive neural hardware.

Q. (Meister, 2024) Why is the 50TB napkin sketch estimate of brain storage (# synapses * dynamic range per synapse) considered a vast overestimate?
A. It assumes all possible synaptic combinations result in a viable brain

This prompt is pluckable. It targets the core insight of the highlighted passage—the reason the ~50 TB synaptic-capacity figure is labeled a 'vast overestimate'—rather than a trivial fragment. The question is atomic, asking only about the viability assumption behind the upper-bound calculation, and the answer ('it assumes all possible synaptic combinations result in a viable brain') faithfully mirrors the source text ('assumes that any combination of connections is possible and leads to a viable brain'). Stability is strong: the prompt cites the author/year for context, uses concrete figures (50TB, synapses × dynamic range) that remain self-explanatory in isolation, and avoids vague or compound framing. The concept is durable and meaningful—a memorable quantitative anchor plus its key caveat—which suits long-term spaced retrieval.


Q. How is the upper bound of brain storage capacity calculated (Meister, 2024)?
A. Multiply the number of synapses (~10^14) by the dynamic range of synaptic strength (5 bits)

The prompt directly targets the core insight of the user's highlight: the method for estimating the brain's theoretical storage upper bound. It is atomic (tests one calculation), faithful to the highlighted concept (synapse count × dynamic range of synaptic strength), and stable—using specific, concrete numbers (~10^14 synapses, 5 bits) and a clear concept ('upper bound of brain storage capacity') that remains self-explanatory in isolation months later. The citation anchor (Meister, 2024) further supports retrieval. While the answer could optionally include the resulting figure (≈50 TB) or the caveat that it is a vast overestimate, these omissions do not undermine the clarity or durability of the tested insight.


Q. Why is the 50 TB estimate of brain storage capacity considered an overestimate?
A. It assumes all possible synaptic combinations would result in a viable brain, which is not true.

The memory prompt is pluckable. It isolates a single, well-defined causal insight that is directly stated in the highlighted passage: the 50 TB figure is an upper bound because the calculation assumes any arbitrary combination of synaptic connections would yield a viable brain. This is precisely the caveat the authors emphasized ('so the upper bound of 50 TB is also a vast overestimate'), so the prompt has strong fidelity to what the user engaged with. The question is atomic—testing one concept—and the question stem itself supplies sufficient context ('the 50 TB estimate of brain storage capacity'), making it self-explanatory and stably retrievable months or years later without access to the original document. The answer is concise and unambiguous, matching the source wording closely.


7.1 Inefficient neural hardwareA frequently cited argument is that neurons are simply inefficient devices for information processing, so nature needsto employ huge numbers of them to complete even simple operations. In particular, individual neurons – unliketransistors – are subject to random biochemical noise, so perhaps one needs to average over many neurons to get areliable signal. Also, perhaps the brain includes a large amount of redundancy, with populations of essentially identicalneurons, to guard against the loss of neurons from aging or injury. Neither of these explanations seems plausible.Regarding “noisy neurons”, the analysis reviewed above (section 4) directly addresses that point and shows that asingle neuron can transmit several bits per spike about its dendritic inputs. Certain irreducible noise sources– such asthermal fluctuations of ion channels or vesicle fusion events – may well limit the rate of information transfer, but theprecision of single neurons remains remarkable. A single optic nerve fiber can reliably signal the arrival of its preferredfeature in the visual stimulus with a precision of 1 millisecond 67. Even deep in the primate cortex, a single neuroncan distinguish between two visual stimuli as precisely as the entire monkey 68. What was once considered irreduciblenoise at the population level may, in fact, be a reliable signal about variables or inputs that the experimenter cannotcontrol69,70, such as an itch on the monkey’s head.On the subject of redundancy, again there is precious little evidence for duplication of essentially identical neurons.For example, in the fruit fly, many neuron types exist in just two copies, one on each side of the brain 71,72. One mightargue that fruit flies are cheap and short-lived, so nature simply duplicates the entire organism to ensure robustness.However, the principle applies even to large animals like ourselves. In the primate retina, every point in the visual fieldis covered by just one neuron of each cell type, with very little overlap 39,73,74. In the visual cortex, a small stroke canabolish vision in a section of the visual field. Apparently, there is no redundancy built in for one of the most commonforms of brain injury.

The highlighted passage (section 7.1 Inefficient neural hardware) addresses a commonly proposed explanation for why the brain uses billions of neurons despite a low overall information throughput: the idea that neurons are inefficient processing units. The passage critically examines two main aspects of this argument.

First, it considers the notion that individual neurons are noisy and unreliable compared to transistors, and thus many neurons are needed to average out this noise and produce stable, reliable signals. However, the authors counter this by citing prior analyses (section 4) showing that single neurons can transmit several bits of information per spike with remarkable precision. For example, optic nerve fibers can time their responses with millisecond precision, and neurons deep in the primate cortex can discriminate visual stimuli as well as the whole animal’s behavior. Moreover, what might previously have been considered irreducible noise at the population level may actually be reliable signaling of other variables unknown to the experimenter. Hence, the evidence suggests that neuronal noise is not so severe that it demands large numbers of redundant neurons simply to overcome unreliability.

Second, the explanation that the brain employs extensive redundancy—populations of essentially identical neurons—to guard against neuron loss from injury or aging is addressed. Here, the authors note that evidence for such redundancy is very limited. Even in small brains like the fruit fly, neuron types exist only in very small numbers (often just one pair, one on each brain side). In primate visual systems, each point in the visual field is represented by a single neuron per cell type with very little overlap, and even small strokes that damage a tiny portion of the visual cortex can lead to permanent vision loss in a corresponding field region. This suggests that there is minimal built-in redundancy to compensate for neuron loss.

Overall, the authors argue that neither neuronal inefficiency due to biochemical noise nor massive redundancy among neurons convincingly explain why the brain needs billions of neurons or why human cognition operates at a slow information throughput. This challenges a popular simplifying assumption and indicates that the paradox of the brain’s enormous capacity but slow throughput must be explained by other factors beyond mere neural inefficiency or redundancy.

Q. How does the primate retina's structure challenge the assumption of neural redundancy?
A. Each point in the visual field is covered by just one neuron of each cell type, with minimal overlap, showing no built-in redundancy.

The prompt is pluckable. It isolates a single, well-defined concept (atomicity): how the primate retina's one-neuron-per-cell-type-per-visual-field-point architecture contradicts the redundancy hypothesis. It is faithful to the highlighted passage—the answer mirrors the source text almost verbatim ('just one neuron of each cell type, with very little overlap')—and it targets a concrete, striking detail that anchors the section's broader argument against neural redundancy, which the user clearly engaged with. The question is also stable for long-term review: its terms ('primate retina', 'neural redundancy') are specific and self-explanatory in isolation, requiring no fading context to interpret, and the factual answer is precise enough to be consistently retrieved even months later.


So the presence of a concrete task leads to a natural binning of the actions, in this case into keystrokes. It also allowsus to separate the signal, namely the entropy that determines task performance, from noise, namely the part that doesnot. The difference between total entropy and noise entropy is the mutual informationI = H(A) − H(A|C) (4)where C stands for the entire context of the measurement, namely the task you are solving and the entire history ofyour actions up to the present point in time. H(A|C) is the conditional entropy of your actions in the next second,given that I know the entire context:H(A|C) = − ∑(a,c)∈A×Cp(a, c) log2 p(a|c) (5)where p(a|c) is the conditional probability that you perform action a given context c, and A × C the joint set of allpossible actions and contexts.

The highlighted passage explains how defining a concrete task enables a meaningful and objective way to measure the information content (entropy) in human behavior by naturally segmenting actions into discrete units relevant for the task—for example, keystrokes in typing.

Without a specific task, it’s difficult to decide what variations in behavior count as distinct "actions" because some differences are trivial or unintentional noise (such as slight variations in timing or force of key presses) that don't affect task performance. By focusing on a task, one can clearly separate:

- **Signal:** the entropy that reflects meaningful, task-relevant variations in actions (e.g., different keys pressed that truly change the message).

- **Noise:** inconsequential variations that do not impact the task outcome (e.g., minor timing differences or unintentional fluctuations).

The difference between total entropy (uncertainty or variety in all observed actions) and noise entropy (uncertainty remaining after controlling for all relevant context) is called **mutual information**, denoted as I = H(A) - H(A|C).

Here:

- **H(A)** is the total entropy of actions A.

- **C** is the full context of the measurement—meaning the task constraints and the history of prior actions that together inform what is expected or planned next.

- **H(A|C)** is the conditional entropy representing the unpredictable or noisy part of the next action given the context C.

- The conditional entropy is calculated as the weighted sum across all actions and contexts of the conditional probability of an action given the context, which quantifies how much uncertainty remains in predicting the action even when the context is known.

In essence, this formalism uses information theory to quantify the "richness" or effective information processing of behavior by focusing on the task-relevant components and discounting irrelevant noise, providing a rigorous way to assess human information throughput.

Q. What does context `(C)` represent in conditional entropy `H(A|C)`?
A. The task being solved and the entire history of actions up to the present time

This prompt is pluckable. It is atomic and precise, testing one well-defined concept drawn directly from the highlighted text: the definition of the conditioning variable C in the mutual information formula I = H(A) − H(A|C). This definition ('the task you are solving and the entire history of your actions') is not a trivial fragment—it is the linchpin of the signal-vs-noise separation the user's interpretation centers on, since H(A|C) quantifies the noise remaining given context. The prompt maintains strict fidelity to both the highlight and its interpretation, uses the exact terminology and notation from the source, and is stable for long-term review: the mathematical notation H(A|C) serves as a durable anchor that keeps the question self-explanatory even months later, when the original reading context has faded.


Q. How does a concrete task help separate signal from noise in behavioral analysis?
A. It provides natural binning of actions (e.g., keystrokes) and distinguishes task-relevant variations from irrelevant ones.

The prompt is pluckable. It directly targets the core insight the user engaged with: a concrete task enables objective analysis of behavior by providing natural binning of actions (e.g., keystrokes) and thereby separating task-relevant signal from inconsequential noise. It is atomic, testing a single well-defined causal relationship rather than a compound set of facts. It remains faithful to the highlighted passage—the answer mirrors the source's own framing ('natural binning,' signal vs. noise)—without introducing external material or reducing the idea to a trivial fragment. The question is also stable: 'How does a concrete task help separate signal from noise in behavioral analysis?' is self-contained and unambiguous when reviewed months later in isolation, with a concrete example (keystrokes) anchoring the recall. While it omits the accompanying mutual information formula, that omission reflects appropriate atomicity—the formula could be its own prompt—and does not distort the tested concept.


Take for example a human typist working from a hand-written manuscript. An advanced typist produces 120 wordsper minute2. If each word is taken as 5 characters, this typing speed corresponds to 10 keystrokes a second. Howmany bits of information does that represent? One is tempted to count the keys on the keyboard and take the logarithmto get the entropy per character, but that is a huge overestimate. Imagine that after reading the start of this paragraphyou are asked what will be the next let...English contains orderly internal structures that make the character stream highly predictable. In fact, the entropy ofEnglish is only ∼ 1 bit per character 3. Expert typists rely on all this redundancy: If forced to type a random charactersequence, their speed drops precipitously.

The highlighted passage uses the example of a skilled typist to illustrate how human information throughput is measured in real-world tasks, and why naive calculations can drastically overestimate it.

The typist produces about 120 words per minute, which converts to roughly 10 keystrokes per second (assuming five characters per word). A first instinct might be to estimate the information content of each keystroke by using the total number of keys on the keyboard (about 50-60) and computing the entropy per character as the logarithm of that number, which would produce a high value of bits per keystroke. However, this approach neglects an important fact: the English language is highly structured and predictable, so not all sequences of characters are equally likely.

The key point is that English text contains considerable redundancy (repetitive and predictable patterns), reducing the true entropy to about 1 bit per character rather than the much higher upper bound implied by the keyboard size. This means the amount of actual "new" information communicated per keystroke is far lower than a raw count of keys might suggest. Expert typists depend on this redundancy to maintain their speed; if they were made to type random, unpredictable characters without these statistical patterns, their typing speed would drop dramatically.

In context, this example supports the authors' argument about the low information rate of human behavior (~10 bits/s). It shows how information theory can tease apart actual information content from superficial measures, and emphasizes that human cognitive and motor performance is constrained by the inherent structure and predictability of sensory inputs like language, rather than by a simple count of raw symbols or possible actions. This example also illustrates the broader theme of the paper: despite enormous neural capacity and sensory input rates, the brain operates at a surprisingly limited throughput for actual behavior and cognition, due in part to the structure and constraints of the information it processes.

Q. What is the entropy (information content) per character in English text?
A. ~1 bit per character, due to language's orderly internal structures and predictability.

The memory prompt targets the central factual anchor of the highlighted passage—the true entropy of English being ~1 bit per character—which is precisely the detail the user engaged with when contrasting naive keyboard-count estimates against the reality of linguistic redundancy. The question is atomic (tests one well-defined concept), specific and self-explanatory for isolated review months later ('entropy per character in English text' remains unambiguous without the original context), and faithful to the highlighted insight. The answer also preserves the essential causal explanation (orderly structure and predictability), keeping the concept whole rather than decontextualized. While one could imagine enriching it with the typist-speed contrast, that would make it compound; as-is, it cleanly isolates the durable, meaningful fact at the heart of the user's interest.


Unfortunately, evidence for photographic or eidetic memory is weakat best. If such people existed, they would sweep the medals in worldwide memory contests, like “Binary Digits”.Instead, the world champions continue to clock in at a measly 10 bits/s.One scientific report stands out here, about a single human subject who could fuse a random dot stereogram after eachmonocular image had been presented on different days24. Each image had 100 × 100 pixels and was presented for 2minutes, which would require a perceptual rate of 100 bits/s, ten times what we propose. Now a ten-fold discrepancyis nothing dramatic, given the million-fold conundrum we pose. Nevertheless, there are good reasons to question thisclaim. For example, neural signals from the two eyes are merged already in the primary visual cortex. So the subject’sbrain would have to somehow inject the memorized image into neurons very early in the visual system, or perhaps evenstore it there. The claim is so unexpected and potentially revolutionary that it should have triggered many follow-upexperiments. No supporting reports have appeared, suggesting that those studies did not go as hoped for.Another popular story of photographic memory involves an artist who can paint an entire cityscape at building-leveldetail after flying over the city in a helicopter 25. His paintings are remarkably intricate, but they do not get the detailsof each building correct26. Even if they did, that would still amount to less than 10 bits/s rate of acquisition. Supposethat after a 45-minute helicopter ride27, the artist drew 1,000 identifiable buildings on a grid, each building with one of1000 possible styles, and all of them were correctly drawn. In this case, he acquired the information for the paintingat a rate of 104 bits/2700 s ≈ 4 bits/s.4

The highlighted passage addresses common claims and popular beliefs about photographic or eidetic memory—the idea that some individuals can perfectly remember vast amounts of visual detail instantly and recall it with high fidelity. The authors argue that the scientific evidence supporting such extraordinary memory abilities is very weak. If people with photographic memory truly existed, they would dominate highly competitive memory sports such as the "Binary Digits" contest, where current world-record holders achieve information processing rates on the order of just 10 bits per second.

They discuss one notable scientific report of a person who could merge two monocular images (random dot stereograms) presented on different days, implying a perceptual information acquisition rate of about 100 bits per second—ten times higher than the general human cognitive throughput (~10 bits/s). Although this would represent a significant deviation from the norm, given the enormous overall discrepancy between sensory input and behavioral output information rates (a factor of about a million), this tenfold difference is relatively minor and not enough to resolve that paradox. Moreover, the authors question the validity of this claim on neurophysiological grounds, noting that visual signals from both eyes merge early in the brain (primary visual cortex), so storing independent monocular images at such an early stage would require unusual neural mechanisms. Importantly, no replication or supporting experiments have followed this initial report, casting doubt on its robustness and suggesting that it may not represent a reliable or generalizable phenomenon.

The authors also mention the well-known case of an artist who paints expansive cityscapes with remarkable detail after brief helicopter flights, often cited as evidence of photographic memory. However, they note that the paintings, while impressive, do not accurately capture fine-grained details of every building. Even if they did, the estimated rate of information acquisition for the artist (about 4 bits per second over 45 minutes) still falls well below the general cognitive throughput limit of 10 bits per second. Thus, these examples do not contradict the notion of a low-rate cognitive bottleneck in human information processing.

In summary, this passage serves to debunk the idea that photographic or eidetic memory undermines the proposed low information throughput of human cognition. The few exceptional cases cited either lack robust scientific confirmation or operate at rates consistent with the general ~10 bits/s limit, reinforcing the article’s broader contention of a fundamental speed limit on human cognitive processing.

Q. During a helicopter ride, a cityscape artist perfectly remembers 50 buildings each with 1000 possible styles – what is the total information processed (in bits)?
A. 50 * Log2(1000) ≈ 500 bits

This prompt is not pluckable. First, it breaks fidelity to the highlighted concept: the source passage describes an artist drawing 1,000 buildings (each with 1,000 possible styles) yielding ~10^4 bits over 45 minutes (≈4 bits/s), but the prompt substitutes '50 buildings' — a fabricated variant that turns the item into a generic arithmetic exercise rather than a recall of the user's insight. Second, it misses the core meaning of the highlight: the user engaged with the argument that even extraordinary 'photographic memory' feats, when quantified, still fall at or below the ~10 bits/s cognitive throughput limit, thereby debunking eidetic memory claims. The prompt tests only the mechanical bit-count computation and drops the crucial rate context (bits/s) and its implication entirely. Third, on stability grounds, reviewing this months later presents a self-contained math problem whose answer (≈500 bits) actually conflicts with the paper's real figures, creating confusion and anchoring to trivia instead of the durable idea.


Q. Calculate: If an artist perfectly remembered 1000 buildings (each with 1000 possible styles) from a 45-minute helicopter ride, what would be their information processing rate?
A. 4 bits/s (10^4 bits / 2700s), still within normal human processing limits

This prompt fails the pluckability test because it targets a surface-level arithmetic detail from a decorative supporting example rather than the core insight the user highlighted. The highlighted passage's central, durable idea is that claims of photographic/eidetic memory are scientifically weak and that even extraordinary feats (the stereogram subject at ~100 bits/s, the helicopter artist at ~4 bits/s) fall within or near the ~10 bits/s cognitive throughput limit—thus failing to resolve the million-fold sensory-behavioral paradox. Instead of prompting recall of this conceptual takeaway, the prompt asks the reviewer to recompute '10^4 bits / 2700 s,' testing trivial calculation mechanics. While the answer's tagline ('still within normal human processing limits') nods toward the real insight, the primary retrieval target is a decontextualized fragment whose significance evaporates once isolated from the surrounding argument about the cognitive bottleneck. A more pluckable version would ask why such exceptional-memory anecdotes do not undermine the ~10 bits/s limit, making the conceptual point—not the arithmetic—the object of long-term retention.


The vast gulf between peripheral and central information rates results in large part from the difference between paralleland serial processing. For example, the peripheral visual system performs image processing in a massively parallelfashion. The retina produces 1 million output signals, each of which is the result of a local computation on the visualimage. Subsequently, the primary visual cortex takes over, using a parallel set of 10,000 modules called “hyper-columns”. Each of these covers about 0.5 mm2 of the cortical surface, contains ∼100,000 neurons, and elaborates acomplete set of feature descriptors for a small patch of the visual field 75,76.8

The highlighted passage explains a key reason behind the enormous difference between the high information processing rates observed in the peripheral sensory systems (like the retina) versus the much lower information throughput in central cognition and behavior. The authors state that this discrepancy largely arises due to the difference in how information is processed: the peripheral sensory systems operate in a massively parallel fashion, whereas central cognition functions largely in a serial manner.

To illustrate this, they describe how the retina processes visual information. The retina generates about 1 million distinct output signals, each corresponding to a localized computation on a small region of the visual image. This represents a highly parallel processing scheme where many independent channels simultaneously encode different features of the visual scene.

Following the retina, the primary visual cortex (V1) continues this parallel processing by organizing neurons into around 10,000 "hypercolumns." Each hypercolumn is a module covering roughly 0.5 mm² of cortex and contains approximately 100,000 neurons. Each hypercolumn specializes in analyzing multiple visual features for a small patch of the visual field, thus maintaining parallel, high-dimensional processing.

This parallel architecture in the peripheral and early cortical stages enables the simultaneous handling of a vast amount of sensory data at high rates. In contrast, later stages of brain function, involved in higher cognition and decision-making, reduce this complex high-dimensional input into a small number of bits to control behavior, often through serial, one-at-a-time processing.

In summary, the authors highlight that the peripheral brain’s ability to process vast amounts of sensory information in parallel underlies its high information rate. Meanwhile, central brain regions significantly downsample and serially process only the essential information needed for behavioral control, thus explaining the paradoxical large gap between peripheral sensory data rates and overall cognitive throughput.

Q. What fundamental difference exists between peripheral and central neural processing that affects information rates?
A. Peripheral processing: parallel
Central processing: serial (ie. CPU metaphor)

The prompt directly targets the core insight anchored by the user's highlight: that the gulf between peripheral (~high-rate) and central (~low-rate) information throughput stems from massively parallel versus serial processing. It is atomic—testing exactly one conceptual distinction—clear, and self-contained, so it remains answerable in isolation months later. The answer faithfully reflects the source material, and the 'CPU metaphor' cue reinforces rather than distorts the idea. While it omits the illustrative numbers (1M retinal outputs, 10,000 hypercolumns), those are supporting examples rather than the durable takeaway the user engaged with, so their exclusion is appropriate. Minor room for improvement exists (e.g., noting this underlies the ~10 bits/s behavioral ceiling), but the essential unit of knowledge is captured durably.


Q. How do peripheral and central neural systems differ in processing mode and resulting information throughput?
A. Peripheral: massively parallel, enabling high throughput.
Central: serial processing, resulting in lower throughput.

The prompt faithfully captures the exact core insight the user highlighted: the gulf between peripheral and central information rates arises from massively parallel versus serial processing. It is atomic—testing a single coherent contrast—and its phrasing ('processing mode' and 'resulting information throughput') mirrors the source's own framing, so it remains self-explanatory when reviewed in isolation months later. It wisely avoids testing decorative details like hypercolumn counts or cortical area sizes, which are less durable and not the point of the highlight. The peripheral/central terminology is standard and stable, making reliable long-term retrieval plausible.


Q. What fundamental difference exists between peripheral and central neural processing that affects information rates?
A. Peripheral processing operates in parallel while central processing is serial, allowing peripheral systems to process more information simultaneously.

This prompt is pluckable. It isolates the single core causal insight from the highlighted passage—that the gulf between peripheral and central information rates stems from parallel versus serial processing—which is exactly the idea the user engaged with. The prompt is atomic (tests one well-defined concept), faithful to the highlighted concept without dragging in decorative specifics like hypercolumn counts or neuron numbers, and remains clear and self-explanatory when reviewed in isolation months later, since the parallel/serial contrast is a durable, memorable distinction tied to the paper's central theme.


This ballpark number of 2 bits/spike allows us to estimate the information rate at different stages of sensory processing.For example, the output of the eye consists of 1 million axons from retinal ganglion cells22. With particularly strongstimuli one can drive them at an average rate of 50 Hz.

The highlighted passage explains that individual neurons, particularly retinal ganglion cells in the eye, communicate information via spikes—discrete electrical impulses. Each spike can carry approximately 2 bits of information. Given this, one can estimate the rate at which these neurons transmit information.

Specifically, the human eye’s output to the brain consists of around 1 million retinal ganglion cell axons. Under conditions of strong stimulation, each of these neurons can fire spikes at an average rate of 50 times per second (50 Hz). Multiplying the bits per spike (2 bits) by the rate of spikes (50 Hz) and the number of neurons (1 million) provides an estimate of the total information transmission capacity of the optic nerve.

In essence, this passage quantifies the impressive data transmission capability of the eye’s neural output: one million neurons sending spikes at 50 times per second, each spike conveying about 2 bits, collectively resulting in a massive potential information rate going from the eye to the brain.

This forms part of the broader context in which the authors highlight a paradox: although the eye and sensory organs capture and relay visual information at very high rates (on the order of millions of bits per second), the overall information processing rate that manifests as human behavior or cognition is drastically slower (around 10 bits per second). The highlighted passage illustrates the "outer brain" side of this contrast—the high-capacity sensory input transmission that is filtered and heavily compressed before it reaches the level where behavior or cognition occurs.

Q. What is the maximum firing rate of retinal ganglion cells when exposed to strong stimuli?
A. 50 Hz (spikes per second)

This prompt fails on both meaningfulness and fidelity. The user's highlight and interpretation center on the coherent, multiplicative insight: ~2 bits/spike × 50 Hz × 1 million retinal ganglion axons yields the optic nerve's enormous bandwidth (~10⁸ bits/s), which contrasts sharply with the ~10 bits/s of human behavioral output. Isolating just '50 Hz' strips away that causal/estimative context, leaving a decontextualized trivia fragment—one input parameter among three—rather than the durable idea the user actually engaged with. Additionally, there is a precision problem that undermines long-term stability: the source states that strong stimuli allow driving the cells at 'an average rate of 50 Hz,' yet the prompt labels this the 'maximum firing rate.' This misstatement could cause confusion or doubt at review time (real RGCs can transiently exceed 50 Hz), making retrieval unreliable months later. A pluckable version would target the full estimation chain or the sensory-bandwidth-vs-behavioral-rate contrast.