Charles Catania, Ph.D

Delay of Reinforcement, the Operant Reserve, and ADHD

The components of attention-deficit hyperactivity disorder in children seem disparate, but each can be interpreted in terms of unusually steep delay-of-reinforcement gradients. The slope of the delay gradient can affect (1) the differential reinforcement of rapid responding, (2) the potency with which attention is maintained by discriminative stimuli that function as conditioned reinforcers, and (3) the balance between impulsive behavior and self-control when consequences that are relatively immediate are pitted against those that are relatively delayed. Different balances among the components (e.g., attention-deficit with little or no hyperactivity) can be understood in terms of differences in delay-gradient parameters.

In Skinner’s Operant Reserve theory, reinforced responses added to a reserve depleted by responding. It couldn’t handle the higher rates maintained by partial reinforcement than by continuous reinforcement but it would have if not just the last but also earlier responses preceding a reinforcer, each weighted by delay, had contributed to growth of the reserve. In that case, partial reinforcement generates steady states in which reserve decrements produced by responding balance increments produced when reinforcers follow responding. In computer simulations of this variation on Skinner’s reserve, cumulative records and quantitative data for extinction, random ratio, random interval and other schedules are consistent with those of real performances. 

An animal model of ADHD is provided by spontaneously hyperactive rats (SHR), which emit higher response rates than Wistar Kyoto (WKY) controls under equivalent schedules of reinforcement. Running the reserve simulation using delay gradients of different steepness generates rate differences consistent with those obtained from SHR and WKY rats.

Dr. Catania began his career in behavior analysis in Fall 1954, when he enrolled in Fred Keller’s course in introductory psychology.  That course included a weekly laboratory on the behavior of rats, and Catania continued working with rats and pigeons and other organisms over subsequent decades.  In Spring 2004, having closed his pigeon laboratory the previous summer, he celebrated his half century of animal lab activity with a rat demonstration in an undergraduate learning course.  During those decades, he had examined the behavior engendered and maintained by a variety of reinforcement schedules, with an abiding interest in relation schedule performances to fundamental behavioral processes such as the delay-of—reinforcement gradient.  He also increasingly impressed by the striking parallels between biological account of evolution in terms of Darwinian natural selection and behavior analytic account of operant behavior in terms of the selection of behavior by its consequence.  He regard the refinement and extension of selectionist accounts as crucial prerequisites for analyses of our own behavior as behavior analysts, including the verbal and nonverbal behavior that enters into our construction of theories and models.