Widespread e-commerce activity on the Internet has led to new opportunities to collect vast amounts of micro-level market and nonmarket data. In this paper we share our experiences in collecting, validating, storing and analyzing large Internet-based data sets in the area of online auctions, music file sharing and online retailer pricing. We demonstrate how such data can advance knowledge by facilitating sharper and more extensive tests of existing theories and by offering observational underpinnings for the development of new theories. Just as experimental economics pushed the frontiers of economic thought by enabling the testing of numerous theories of economic behavior in the environment of a controlled laboratory, we believe that observing, often over extended periods of time, real-world agents participating in market and nonmarket activity on the Internet can lead us to develop and test a variety of new theories. Internet data gathering is not controlled experimentation. We cannot randomly assign participants to treatments or determine event orderings. Internet data gathering does offer potentially large data sets with repeated observation of individual choices and action. In addition, the automated data collection holds promise for greatly reduced cost per observation. Our methods rely on technological advances in automated data collection agents. Significant challenges remain in developing appropriate sampling techniques integrating data from heterogeneous sources in a variety of formats, constructing generalizable processes and understanding legal constraints. Despite these challenges, the early evidence from those who have harvested and analyzed large amounts of e-commerce data points toward a significant leap in our ability to understand the functioning of electronic commerce.

As Internet-based commerce becomes increasingly widespread, large data sets about the demand for and pricing of a wide variety of products become available. These present exciting new opportunities for empirical economic and business research, but also raise new statistical issues and challenges. In this article, we summarize research that aims to assess the optimality of price discrimination in the software industry using a large e-commerce panel data set gathered from Amazon.com. We describe the key parameters that relate to demand and cost that must be reliably estimated to accomplish this research successfully, and we outline our approach to estimating these parameters. This includes a method for “reverse engineering” actual demand levels from the sales ranks reported by Amazon, and approaches to estimating demand elasticity, variable costs and the optimality of pricing choices directly from publicly available e-commerce data. Our analysis raises many new challenges to the reliable statistical analysis of e-commerce data and we conclude with a brief summary of some salient ones.

The growing expanse of e-commerce and the widespread availability of online databases raise many fears regarding loss of privacy and many statistical challenges. Even with encryption and other nominal forms of protection for individual databases, we still need to protect against the violation of privacy through linkages across multiple databases. These issues parallel those that have arisen and received some attention in the context of homeland security. Following the events of September 11, 2001, there has been heightened attention in the United States and elsewhere to the use of multiple government and private databases for the identification of possible perpetrators of future attacks, as well as an unprecedented expansion of federal government data mining activities, many involving databases containing personal information. We present an overview of some proposals that have surfaced for the search of multiple databases which supposedly do not compromise possible pledges of confidentiality to the individuals whose data are included. We also explore their link to the related literature on privacy-preserving data mining. In particular, we focus on the matching problem across databases and the concept of “selective revelation” and their confidentiality implications.

This paper describes opportunities and challenges of using functional data analysis (FDA) for the exploration and analysis of data originating from electronic commerce (eCommerce). We discuss the special data structures that arise in the online environment and why FDA is a natural approach for representing and analyzing such data. The paper reviews several FDA methods and motivates their usefulness in eCommerce research by providing a glimpse into new domain insights that they allow. We argue that the wedding of eCommerce with FDA leads to innovations both in statistical methodology, due to the challenges and complications that arise in eCommerce data, and in online research, by being able to ask (and subsequently answer) new research questions that classical statistical methods are not able to address, and also by expanding on research questions beyond the ones traditionally asked in the offline environment. We describe several applications originating from online transactions which are new to the statistics literature, and point out statistical challenges accompanied by some solutions. We also discuss some promising future directions for joint research efforts between researchers in eCommerce and statistics.

This paper explores the application of functional data analysis (FDA) as a means to study the dynamics of software evolution in the open source context. Several challenges in analyzing the data from software projects are discussed, an approach to overcoming those challenges is described, and preliminary results from the analysis of a sample of open source software (OSS) projects are provided. The results demonstrate the utility of FDA for uncovering and categorizing multiple distinct patterns of evolution in the complexity of OSS projects. These results are promising in that they demonstrate some patterns in which the complexity of software decreased as the software grew in size, a particularly novel result. The paper reports preliminary explorations of factors that may be associated with decreasing complexity patterns in these projects. The paper concludes by describing several next steps for this research project as well as some questions for which more sophisticated analytical techniques may be needed.

In this paper, we examine the price dynamics of on-line art auctions of modern Indian art using functional data analysis. The purpose here is not just to understand what determines the final prices of art objects, but also the price movement during the entire auction. We identify several factors, such as artist characteristics (established or emerging artist; prior sales history), art characteristics (size; painting medium—canvas or paper), competition characteristics (current number of bidders; current number of bids) and auction design characteristics (opening bid; position of the lot in the auction), that explain the dynamics of price movement in an on-line art auction. We find that the effects on price vary over the duration of the auction, with some of these effects being stronger at the beginning of the auction (such as the opening bid and historical prices realized). In some cases, the rate of change in prices (velocity) increases at the end of the auction (for canvas paintings and paintings by established artists). Our analysis suggests that the opening bid is positively related to on-line auction price levels of art at the beginning of the auction, but its effect declines toward the end of the auction. The order in which the lots appear in an art auction is negatively related to the current price level, with this relationship decreasing toward the end of the auction. This implies that lots that appear earlier have higher current prices during the early part of the auction, but that effect diminishes by the end of the auction. Established artists show a positive relationship with the price level at the beginning of the auction. Reputation or popularity of the artists and their investment potential as assessed by previous history of sales are positively related to the price levels at the beginning of the auction. The medium (canvas or paper) of the painting does not show any relationship with art auction price levels, but the size of the painting is negatively related to the current price during the early part of the auction. Important implications for auction design are drawn from the analysis.

Online auctions are fast gaining popularity in today’s electronic commerce. Relative to offline auctions, there is a greater degree of multiple bidding and late bidding in online auctions, an empirical finding by some recent research. These two behaviors (multiple bidding and late bidding) are of “strategic” importance to online auctions and hence important to investigate.

In this article we empirically measure the distribution of bid timings and the extent of multiple bidding in a large set of online auctions, using bidder experience as a mediating variable. We use data from the popular auction site www.eBay.com to investigate more than 10,000 auctions from 15 consumer product categories. We estimate the distribution of late bidding and multiple bidding, which allows us to place these product categories along a continuum of these metrics (the extent of late bidding and the extent of multiple bidding). Interestingly, the results of the analysis distinguish most of the product categories from one another with respect to these metrics, implying that product categories, after controlling for bidder experience, differ in the extent of multiple bidding and late bidding observed in them.

We also find a nonmonotonic impact of bidder experience on the timing of bid placements. Experienced bidders are “more” active either toward the close of auction or toward the start of auction. The impact of experience on the extent of multiple bidding, though, is monotonic across the auction interval; more experienced bidders tend to indulge “less” in multiple bidding.

Propensity score methods were proposed by Rosenbaum and Rubin [Biometrika 70 (1983) 41–55] as central tools to help assess the causal effects of interventions. Since their introduction more than two decades ago, they have found wide application in a variety of areas, including medical research, economics, epidemiology and education, especially in those situations where randomized experiments are either difficult to perform, or raise ethical questions, or would require extensive delays before answers could be obtained. In the past few years, the number of published applications using propensity score methods to evaluate medical and epidemiological interventions has increased dramatically. Nevertheless, thus far, we believe that there have been few applications of propensity score methods to evaluate marketing interventions (e.g., advertising, promotions), where the tradition is to use generally inappropriate techniques, which focus on the prediction of an outcome from background characteristics and an indicator for the intervention using statistical tools such as least-squares regression, data mining, and so on. With these techniques, an estimated parameter in the model is used to estimate some global “causal” effect. This practice can generate grossly incorrect answers that can be self-perpetuating: polishing the Ferraris rather than the Jeeps “causes” them to continue to win more races than the Jeeps ⇔ visiting the high-prescribing doctors rather than the low-prescribing doctors “causes” them to continue to write more prescriptions. This presentation will take “causality” seriously, not just as a casual concept implying some predictive association in a data set, and will illustrate why propensity score methods are generally superior in practice to the standard predictive approaches for estimating causal effects.

This article provides an assessment of the causal effect of customer relationship management (CRM) applications on one-to-one marketing effectiveness. We use a potential outcomes based propensity score approach to assess this causal effect. We find that firms using CRM systems have greater levels of one-to-one marketing effectiveness. We discuss the strengths and challenges of using the propensity score approach to design and execute CRM related observational studies. We also discuss the applicability of the framework in this paper to study typical causal questions in business and electronic commerce research at the firm, individual and economy levels, and to clarify the assumptions that researchers must make to infer causality from observational data.

Modern business is rushing toward e-commerce. If the transition is done properly, it enables better management, new services, lower transaction costs and better customer relations. Success depends on skilled information technologists, among whom are statisticians. This paper focuses on some of the contributions that statisticians are making to help change the business world, especially through the development and application of data mining methods. This is a very large area, and the topics we cover are chosen to avoid overlap with other papers in this special issue, as well as to respect the limitations of our expertise. Inevitably, electronic commerce has raised and is raising fresh research problems in a very wide range of statistical areas, and we try to emphasize those challenges.

This paper makes three contributions: (1) the paper provides a better understanding of online behavior by showing the main drivers of Internet portal choice, (2) the rich data allow for a deeper understanding of brand substitution patterns than previously possible and (3) the paper introduces a wider statistics community to a new data opportunity and a recently developed method.

Network-based marketing refers to a collection of marketing techniques that take advantage of links between consumers to increase sales. We concentrate on the consumer networks formed using direct interactions (e.g., communications) between consumers. We survey the diverse literature on such marketing with an emphasis on the statistical methods used and the data to which these methods have been applied. We also provide a discussion of challenges and opportunities for this burgeoning research topic. Our survey highlights a gap in the literature. Because of inadequate data, prior studies have not been able to provide direct, statistical support for the hypothesis that network linkage can directly affect product/service adoption. Using a new data set that represents the adoption of a new telecommunications service, we show very strong support for the hypothesis. Specifically, we show three main results: (1) “Network neighbors”—those consumers linked to a prior customer—adopt the service at a rate 3–5 times greater than baseline groups selected by the best practices of the firm’s marketing team. In addition, analyzing the network allows the firm to acquire new customers who otherwise would have fallen through the cracks, because they would not have been identified based on traditional attributes. (2) Statistical models, built with a very large amount of geographic, demographic and prior purchase data, are significantly and substantially improved by including network information. (3) More detailed network information allows the ranking of the network neighbors so as to permit the selection of small sets of individuals with very high probabilities of adoption.

The emergence of online communities has enabled firms to monitor consumer-generated online word-of-mouth (WOM) in real-time by mining publicly available information from the Internet. A prerequisite for harnessing this new ability is the development of appropriate WOM metrics and the identification of relationships between such metrics and consumer behavior. Along these lines this paper introduces a metric of a purchasing population’s propensity to rate a product online. Using data from a popular movie website we find that our metric exhibits several relationships that have been previously found to exist between aspects of a product and consumers’ propensity to engage in offline WOM about it. Our study, thus, provides positive evidence for the validity of our metric as a proxy of a population’s propensity to engage in post-purchase online WOM. Our results also suggest that the antecedents of offline and online WOM exhibit important similarities.

Born in Sydney, Australia, on April 20, 1939, Chris Heyde shifted his interest from sport to mathematics thanks to inspiration from a schoolteacher. After earning an M.Sc. degree from the University of Sydney and a Ph.D. from the Australian National University (ANU), he began his academic career in the United States at Michigan State University, and then in the United Kingdom at the University of Sheffield and the University of Manchester. In 1968, Chris moved back to Australia to teach at ANU until 1975, when he joined CSIRO, where he was Acting Chief of the Division of Mathematics and Statistics. From 1983 to 1986, he was a Professor and Chairman of the Department of Statistics at the University of Melbourne. Chris then returned to ANU to become the Head of the Statistics Department, and later the Foundation Dean of the School of Mathematical Sciences (now the Mathematical Sciences Institute). Since 1993, he has also spent one semester each year teaching at the Department of Statistics, Columbia University, and has been the director of the Center for Applied Probability at Columbia University since its creation in 1993.

Chris has been honored worldwide for his contributions in probability, statistics and the history of statistics. He is a Fellow of the International Statistical Institute and the Institute of Mathematical Statistics, and he is one of three people to be a member of both the Australian Academy of Science and the Australian Academy of Social Sciences. In 2003, he received the Order of Australia from the Australian government. He has been awarded the Pitman Medal and the Hannan Medal. Chris was conferred a D.Sc. honoris causa by University of Sydney in 1998.

Chris has been very active in serving the statistical community, including as the Vice President of the International Statistical Institute, President of the Bernoulli Society and Vice President of the Australian Mathematical Society. He has served on numerous editorial boards, most notably as Editor of Stochastic Processes and Their Applications from 1983 to 1989, and as Editor-in-Chief of Journal of Applied Probability and Advances in Applied Probability since 1990.

His research has spanned almost all areas of probability and statistics, ranging from random walks to branching processes, from martingales to quasi-likelihood inference, from genetics to option pricing, from queueing theory to long-range dependence. He has edited twelve books, and authored or co-authored three books, I. J. Bienaymé: Statistical Theory Anticipated (1977), with E. Seneta, Martingale Limit Theory and Its Application (1980), with P. Hall, and Quasi-Likelihood and Its Applications (1977). Chris Heyde has been an outstanding citizen and leader of the probability and statistics research community.

The interview ranges over his education in Australia, moves to the USA and the UK, return to Australia, appointment at Columbia, major research contributions, and professional society and editorial activities. It ends with a look forward in time and some concerned comments about the future for statistics departments.