A Conversation between Theory, Methods, and Data

Abstract

The 10 year anniversary of the COGITO Study provides an opportunity to revisit the ideas behind the Cattell data box. Three dimensions of the persons × variables × time data box are discussed in the context of three categories of researchers each wanting to answer their own categorically different question. The example of the well-known speed-accuracy tradeoff is used to illustrate why these are three different categories of statistical question. The 200 persons by 100 variables by 100 occasions of measurement COGITO data cube presents a challenge to integrate theories and methods across the dimensions of the data box. A conceptual model is presented for the speed-accuracy tradeoff example that could account for cross-sectional between persons effects, short-term dynamics, and long-term learning effects. Thus, two fundamental differences between the time axis and the other two axes of the data box include ordering and time scaling. In addition, nonstationarity in human systems is a pervasive problem along the time dimension of the data box. To illustrate, the difference in nonstationarity between dancing and conversation is discussed in the context of the interaction between theory, methods, and data. An information theoretic argument is presented that the theory-methods-data interaction is better understood when viewed as a conversation than as a dance. Entropy changes in the development of a theory-methods-data conversation provide one metric for evaluating scientific progress.