The limits of mathematical and statistical analysis of big data
From the CFO Journal's Morning Ledger on April 18, 2014

The limits of social engineering
Writing in
 MIT Technology Review, tech reporter Nicholas Carr pulls from a new book by one of MIT’s noted data scientists to explain why he thinks Big Data has its limits, especially when applied to understanding society. Alex ‘Sandy’ Pentland, in his book “Social Physics: How Good Ideas Spread – The Lessons from a New Science,” sees a mathematical modeling of society made possible by new technologies and sensors and Big Data processing power. Once data measurement confirms “the innate tractability of human beings,” scientists may be able to develop models to predict a person’s behavior. Mr. Carr sees overreach on the part of Mr. Pentland. “Politics is messy because society is messy, not the other way around,” Mr. Carr writes, and any statistical model likely to come from such research would ignore the history, politics, class and messy parts associated with humanity. “What big data can’t account for is what’s most unpredictable, and most interesting, about us,” he concludes.

Jensen Comment
The sad state of accountancy and many doctoral programs in the 21st Century is that virtually all of them in North America only teach the methodology and technique of analyzing big data with statistical tools or the analytical modeling of artificial worlds based on dubious assumptions to simplify reality ---
http://www.trinity.edu/rjensen/Theory01.htm#DoctoralPrograms

The Pathways Commission sponsored by the American Accounting Association strongly proposes adding non-quantitative alternatives to doctoral programs but I see zero evidence of any progress in that direction. The main problem is that it's just much easier to avoid having to collect data by beating purchased databases with econometric sticks until something, usually an irrelevant something, falls out of the big data piñata.

"A Scrapbook on What's Wrong with the Past, Present and Future of Accountics Science"
Bob Jensen Jensen
February 19, 2014
SSRN Download:  http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2398296 

 


On May 2, 2012 former TAR Senior Editor Steve Katchelmeier wrote the following:

Is it your position Bob that no "accountics" research is of interest to practitioners? I hope that's an overly extreme characterization, but it seems to be implied in your latest post in this thread. I for one see no inconsistency at all between research of the style you call accountics and research of relevance to practice.

Steve

May 3, 2012 reply from Bob Jensen

Hi Steve,

Of course there are instances when accountics science findings impact practice and/or teaching. My most obvious example is when accountics scientist Mary Barth was on the IASB. Undoubtedly issues under deliberation impacted on her research, and on occasion accountics science research findings had a bearing on her and other members of the board with whom she consulted. Mary has more practice experience than most accountics scientists. She was an Andersen partner before entering Stanford's PhD program. She was on the IASB from the beginning until 2009.

As you once pointed out, there are times when a accountics scientist conducting a behavioral experiment using real world subjects in an accounting firm  impacted the future practice of that firm. You once pointed out to me a paper where this happened in Ernst & Young.

AIS professors have had some impact on practice, most notably Bill McCarthy's seminal REA (Resource-Event-Agent) accounting system.

But what I think we need to do is look what accountics science critics have been saying for decades and
what accountics scientists have been ignoring with a wall of silence.

Even accountics scientists admit that their research methods are limited mostly to things they can model without leaving campus. They do not pretend all things can be modeled. The point out that in a free world other researchers are free to use other methods and publish in a wide array of journals.

But there's one huge problem that evolved with accountics science since the 1960s
In their zeal to bring science into academic accounting research they literally took over all the doctoral programs in North America for the past five decades. Our doctoral graduates developed mathematical and econometric skills --- that's a good thing. But at the same time our doctoral programs for more than five decades have been generating graduates who are not skilled in other methods of research such as history research, case study research, philosophy research, etc.
http://www.trinity.edu/rjensen/Theory01.htm#DoctoralPrograms

As Bob Kaplan, Steve Zeff, and many other critics point out that accountics scientists tend to only focus on problems that can be approached with mathematics and statistics. Accounting teachers and practitioners and even accountics scientists saw great problems in avoiding a much broader set of problems in the education and the profession.

In an effort to motivate accounting faculty to conduct research using methods other than accountics science, many new journals were formed. The journals had high hopes that in a publish or parish environment accounting researchers would branch out from accountics science using other methods and generate non-accountics findings to a much broader set of problems.

This would have been a good thing except for one thing --- our doctoral program graduates over the past five decades really preferred accountics science. In part this is because much of accountics scientists rarely have to leave the campus.

Also most of our doctoral program graduates since the 1960s prefer accountics science methods because that's all they learned.

And so we have the following events that happened in academic accounting research:

  1. New journals like Accounting Horizons, Issues in Accounting Education, and a bunch of others were formed to motivate our doctoral program graduates to tackle problems that are intractable in accountics science.

    Mautz, R. K. 1987. Editorial. Accounting Horizons (September): 109-111.

    Mautz, R. K. 1987. Editorial: Expectations: Reasonable or ridiculous? Accounting Horizons (December): 117-120.


     

  2. But instead of paying attention to the missions of these new journals like Accounting Horizons, accountics scientists hijacked those journals turning them into TAR clones for accountics science publications. Accounting teaching teachers and practitioners ended up ignoring Accounting Horizons as much as they ignored TAR. ---
    http://www.cs.trinity.edu/~rjensen/temp/ZeffCommentOnAccountingHorizons.ppt  

 

"Research on Accounting Should Learn From the Past" by Michael H. Granof and Stephen A. Zeff, Chronicle of Higher Education, March 21, 2008

The unintended consequence has been that interesting and researchable questions in accounting are essentially being ignored. By confining the major thrust in research to phenomena that can be mathematically modeled or derived from electronic databases, academic accountants have failed to advance the profession in ways that are expected of them and of which they are capable.

Academic research has unquestionably broadened the views of standards setters as to the role of accounting information and how it affects the decisions of individual investors as well as the capital markets. Nevertheless, it has had scant influence on the standards themselves.

Continued in article

 

This of course begs the question that if accountics scientists finally listen to their many critics what should they do to "advance the profession in ways that are expected of them and of which they are capable?"

  1. I don't think anybody has an answer to this question and would not put much faith in quickie surveys of accounting firms and corporations.
    The first thing accountics scientists need to do is get off campus and make extensive contacts inside accounting firms and business firms of all sizes. Interactively and over time they would, thereby, identify profession problems that need to be researched that would advance the profession.
     
  2. Accountics scientists have done little to improve the IASB and FASB conceptual frameworks. For example, earnings-per-share (eps) is probably the most important performance index tracked by investors and analysts. To date the conceptual framework for earnings and eps is lousy. I think accountics scientists could bring innovative research to bear on conceptual frameworks for earnings and most other items in the conceptual framework.
     
  3. To date accountics scientists mostly ignore externalities (non-convexities) that are very difficult to bring into their empirical and analytical models. Perhaps if they get beyond their models and purchased databases they can bring other research methods to bear upon these externalities.
     
  4. Most importantly, in my opinion, accountics scientists need to start communicating with teachers and practitioners. For example, their record to date in messaging with accounting teachers and practitioners on the AAA Commons is a big, big, big ZERO!

How Accountics Scientists Should Change: 
"Frankly, Scarlett, after I get a hit for my resume in The Accounting Review I just don't give a damn"
http://www.cs.trinity.edu/~rjensen/temp/AccounticsDamn.htm
One more mission in what's left of my life will be to try to change this
http://www.cs.trinity.edu/~rjensen/temp/AccounticsDamn.htm
 

 

Conclusion

The problem is not that accountics scientists own TAR and will not allow any articles to be published that do not contain equations and/or statistical inference tables. There are numerous other outlets publishing a wider variety of accounting research.

One enormous problem is that, in their zeal to bring science into academic accounting research, accountics scientists literally took over all the doctoral programs in North America for the past five decades. Our doctoral graduates developed mathematical and econometric skills --- that's a good thing. But at the same time our doctoral programs for more than five decades have been generating graduates who are not skilled in other methods of research such as history research, case study research, philosophy research, etc.
http://www.trinity.edu/rjensen/Theory01.htm#DoctoralPrograms

But an even bigger problem is that accountics science is an easier path to research. Most accountics scientists conduct research in Plato's Cave without having to  leave the campus. Over more than five decades accountics scientists have grown soft and avoid the hard problems in research, such as the problem of providing creative and innovative findings for the accountancy profession.

 

How did academic accounting research become a pseudo science?
http://www.trinity.edu/rjensen/theory01.htm#WhatWentWrong



Gasp! How could an accountics scientist question such things? This is sacrilege!
Let me end my remarks with a question: Have Ball and Brown (1968)—and Beaver (1968) for that matter, if I can bring Bill Beaver into it—have we had too much influence on the research agenda to the point where other questions and methods are being overlooked?
Phil Brown of Ball and Brown Fame

"How Can We Do Better?" by Phillip R. Brown (of Ball and Brown Fame), Accounting Horizons (Forum on the State of Accounting Scholarship), December 2013 ---
http://aaajournals.org/doi/full/10.2308/acch-10365
Not Free

Philip R. Brown AM is an Honorary Professor at The University of New South Wales and Senior Honorary Research Fellow at The University of Western Australia.

I acknowledge the thoughtful comments of Sudipta Basu, who arranged and chaired this session at the 2012 American Accounting Association (AAA) Annual Meeting, Washington, DC.

The video presentation can be accessed by clicking the link in Appendix A.

Corresponding author: Philip R. Brown AM. Email:

When Sudipta Basu asked me whether I would join this panel, he was kind enough to share with me the proposal he put to the conference organizers. As background to his proposal, Sudipta had written:

Analytical and empirical researchers generate numerous results about accounting, as do logicians reasoning from conceptual frameworks. However, there are few definitive tests that permit us to negate propositions about good accounting.

This panel aims to identify a few “most wrong” beliefs held by accounting experts—academics, regulators, practitioners—where a “most wrong” belief is one that is widespread and fundamentally misguided about practices and users in any accounting domain.

While Sudipta's proposal resonated with me, I did wonder why he asked me to join the panel, and whether I am seen these days as just another “grumpy old man.” Yes, I am no doubt among the oldest here today, but grumpy? You can make your own mind on that, after you have read what I have to say.

This essay begins with several gripes about editors, reviewers, and authors, along with suggestions for improving the publication process for all concerned. The next section contains observations on financial accounting standard setting. The essay concludes with a discussion of research myopia, namely, the unfortunate tendency of researchers to confine their work to familiar territory, much like the drunk who searches for his keys under the street light because “that is where the light is.”



 
ON EDITORS AND REVIEWERS, AND AUTHORS

I have never been a regular editor, although I have chaired a journal's board of management and been a guest editor, and I appointed Ray Ball to his first editorship (Ray was the inaugural editor of the Australian Journal of Management). I have, however, reviewed many submissions for a whole raft of journals, and written literally hundreds of papers, some of which have been published. As I reflect on my involvement in the publications process over more than 50 years, I do have a few suggestions on how we can do things better. In the spirit of this panel session, I have put my suggestions in the form of gripes about editors, reviewers, and authors.

One-eyed editors—and reviewers—who define the subject matter as outside their journal's interests are my first gripe; and of course I except journals with a mission that is stated clearly and in unequivocal terms for all to see. The best editors and the best reviewers are those who are open-minded who avoid prejudging submissions by reference to some particular set of questions or modes of thinking that have become popular over the last five years or so. Graeme Dean, former editor of Abacus, and Nick Dopuch, former editor of the Journal of Accounting Research, are fine examples, from years gone by, of what it means to be an excellent editor.

Editors who are reluctant to entertain new ways of looking at old questions are a second gripe. Many years ago I was asked to review a paper titled “The Last Word on …” (I will not fill in the dots because the author may still be alive.) But at the time I thought, what a strange title! Can any academic reasonably believe they are about to have the last say on any important accounting issue? We academics thrive on questioning previous works, and editors and their reviewers do well when they nurture this mindset.

My third gripe concerns editors who, perhaps unwittingly, send papers to reviewers with vested interests and the reviewers do not just politely return the paper to the editor and explain their conflict of interest. A fourth concerns editors and reviewers who discourage replications: their actions signal a disciplinary immaturity. I am referring to rejecting a paper that repeats an experiment, perhaps in another country, purely because it has been done before. There can be good reasons for replicating a study, for example if the external validity of the earlier study legitimately can be questioned (perhaps different outcomes are reasonably expected in another institutional setting), or if methodological advances indicate a likely design flaw. Last, there are editors and reviewers who do not entertain papers that fail to reject the null hypothesis. If the alternative is well-reasoned and the study is sound, and they can be big “ifs,” then failure to reject the null can be informative, for it may indicate where our knowledge is deficient and more work can be done.1

It is not only editors and reviewers who test my emotional state. I do get a bit short when I review papers that fail to appreciate that the ideas they are dealing with have long yet uncited histories, sometimes in journals that are not based in North America. I am particularly unimpressed when there is an all-too-transparent and excessive citation of works by editors and potential reviewers, as if the judgments of these folks could possibly be influenced by that behavior. Other papers frustrate me when they are technically correct but demonstrate the trivial or the obvious, and fail to draw out the wider implications of their findings. Then there are authors who rely on unnecessarily coarse “control” variables which, if measured more finely, may well threaten their findings.2 Examples are dummy variables for common law/code law countries, for “high” this and “low” that, for the presence or absence of an audit/nomination/compensation committee, or the use of an industry or sector variable without saying which features of that industry or sector are likely to matter and why a binary representation is best. In a nutshell, I fear there may be altogether too many dummies in financial accounting research!

Finally, there are the International Financial Reporting Standards (IFRS) papers that fit into the category of what I describe as “before and after studies.” They focus on changes following the adoption of IFRS promulgated by the London-based International Accounting Standards Board (IASB). A major concern, and I have been guilty too, is that these papers, by and large, do not deal adequately with the dynamics of what has been for many countries a period of profound change. In particular, there is a trade-off between (1) experimental noise from including too long a “before” and “after” history, and (2) not accommodating the process of change, because the “before” and “after” periods are way too short. Neither do they appear to control convincingly for other time-related changes, such as the introduction of new accounting and auditing standards, amendments to corporations laws and stock exchange listing rules, the adoption of corporate governance codes of conduct, more stringent compliance monitoring and enforcement mechanisms, or changes in, say stock, market liquidity as a result of the introduction of new trading platforms and protocols, amalgamations among market providers, the explosion in algorithmic trading, and the increasing popularity among financial institutions of trading in “dark pools.”



 
ON FINANCIAL ACCOUNTING STANDARD SETTING

I count a number of highly experienced financial accounting standard setters among my friends and professional acquaintances, and I have great regard for the difficulties they face in what they do. Nonetheless, I do wonder


. . .

 
ON RESEARCH MYOPIA

A not uncommon belief among academics is that we have been or can be a help to accounting standard setters. We may believe we can help by saying something important about whether a new financial accounting standard, or set of standards, is an improvement. Perhaps we feel this way because we have chosen some predictive criterion and been able to demonstrate a statistically reliable association between accounting information contained in some database and outcomes that are consistent with that criterion. Ball and Brown (1968, 160) explained the choice of criterion this way: “An empirical evaluation of accounting income numbers requires agreement as to what real-world outcome constitutes an appropriate test of usefulness.” Note their reference to a requirement to agree on the test. They were referring to the choice of criterion being important to the persuasiveness of their tests, which were fundamental and related to the “usefulness” of U.S. GAAP income numbers to stock market investors 50 years ago. As time went by and the financial accounting literature grew accordingly, financial accounting researchers have looked in many directions for capital market outcomes in their quest for publishable results.

Research on IFRS can be used to illustrate my point. Those who have looked at the consequences of IFRS adoption have mostly studied outcomes they believed would interest participants in equity markets and to a less extent parties to debt contracts. Many beneficial outcomes have now been claimed,4 consistent with benefits asserted by advocates of IFRS. Examples are more comparable accounting numbers; earnings that are higher “quality” and less subject to managers' discretion; lower barriers to international capital flows; improved analysts' forecasts; deeper and more liquid equity markets; and a lower cost of capital. But the evidence is typically coarse in nature; and so often the results are inconsistent because of the different outcomes selected as tests of “usefulness,” or differences in the samples studied (time periods, countries, industries, firms, etc.) and in research methods (how models are specified and variables measured, which estimators are used, etc.). The upshot is that it can be difficult if not impossible to reconcile the many inconsistencies, and for standard setters to relate reported findings to the judgments they must make.

Despite the many largely capital market outcomes that have been studied, some observers of our efforts must be disappointed that other potentially beneficial outcomes of adopting IFRS have largely been overlooked. Among them are the wider benefits to an economy that flow from EU membership (IFRS are required),5 or access to funds provided by international agencies such as the World Bank, or less time spent by CFOs of international companies when comparing the financial performance of divisions operating in different countries and on consolidating the financial statements of foreign subsidiaries, or labor market benefits from more flexibility in the supply of professionally qualified accountants, or “better” accounting standards from pooling the skills of standard setters in different jurisdictions, or less costly and more consistent professional advice when accounting firms do not have to deal with as much cross-country variation in standards and can concentrate their high-level technical skills, or more effective compliance monitoring and enforcement as regulators share their knowledge and experience, or the usage of IFRS by “millions (of small and medium enterprises) in more than 80 countries” (Pacter 2012), or in some cases better education of tomorrow's accounting professionals.6 I am sure you could easily add to this list if you wished.

In sum, we can help standard setters, yes, but only in quite limited ways.7 Standard setting is inherently political in nature and will remain that way as long as there are winners and losers when standards change. That is one issue. Another is that the results of capital markets studies are typically too coarse to be definitive when it comes to the detailed issues that standard setters must consider. A third is that accounting standards have ramifications extending far beyond public financial markets and a much more expansive view needs to be taken before we can even hope to understand the full range of benefits (and costs) of adopting IFRS.

Let me end my remarks with a question: Have Ball and Brown (1968)—and Beaver (1968) for that matter, if I can bring Bill Beaver into it—have we had too much influence on the research agenda to the point where other questions and methods are being overlooked?

February 27, 2014 Reply from Paul Williams

Bob,
If you read that last Horizon's section provided by "thought leaders" you realize the old guys are not saying anything they could not have realized 30 years ago. That they didn't realize it then (or did but was not in their interest to say so), which led them to run journals whose singular purpose seemed to be to enable they and their cohorts to create politically correct academic reputations, is not something to ask forgiveness for at the end of your career.

Like the sinner on his deathbed asking for God's forgiveness , now is a hell of a time to suddenly get religion. If you heard these fellows speak when they were young they certainly didn't speak with voices that adumbrated any doubt that what they were doing was rigorous research and anyone doing anything else was the intellectual hoi polloi.

Oops, sorry we created an academy that all of us now regret, but, hey, we got ours. It's our mess, but now we are telling you its a mess you have to clean up. It isn't like no one was saying these things 30 years ago (you were as well as others including yours truly) and we have intimate knowledge of how we were treated by these geniuses

 

 



David Johnstone asked me to write a paper on the following:
"A Scrapbook on What's Wrong with the Past, Present and Future of Accountics Science"
Bob Jensen
February 19, 2014
SSRN Download:  http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2398296 

Abstract

For operational convenience I define accountics science as research that features equations and/or statistical inference. Historically, there was a heated debate in the 1920s as to whether the main research journal of academic accounting, The Accounting Review (TAR) that commenced in 1926, should be an accountics journal with articles that mostly featured equations. Practitioners and teachers of college accounting won that debate.

TAR articles and accountancy doctoral dissertations prior to the 1970s seldom had equations.  For reasons summarized below, doctoral programs and TAR evolved to where in the 1990s there where having equations became virtually a necessary condition for a doctoral dissertation and acceptance of a TAR article. Qualitative normative and case method methodologies disappeared from doctoral programs.

What’s really meant by “featured equations” in doctoral programs is merely symbolic of the fact that North American accounting doctoral programs pushed out most of the accounting to make way for econometrics and statistics that are now keys to the kingdom for promotion and tenure in accounting schools ---
http://www.trinity.edu/rjensen/Theory01.htm#DoctoralPrograms

The purpose of this paper is to make a case that the accountics science monopoly of our doctoral programs and published research is seriously flawed, especially its lack of concern about replication and focus on simplified artificial worlds that differ too much from reality to creatively discover findings of greater relevance to teachers of accounting and practitioners of accounting. Accountics scientists themselves became a Cargo Cult.

Shielding Against Validity Challenges in Plato's Cave ---
http://www.trinity.edu/rjensen/TheoryTAR.htm

Common Accountics Science and Econometric Science Statistical Mistakes ---
http://www.cs.trinity.edu/~rjensen/temp/AccounticsScienceStatisticalMistakes.htm

The Cult of Statistical Significance: How Standard Error Costs Us Jobs, Justice, and Lives ---
http://www.cs.trinity.edu/~rjensen/temp/DeirdreMcCloskey/StatisticalSignificance01.htm

How Accountics Scientists Should Change: 
"Frankly, Scarlett, after I get a hit for my resume in The Accounting Review I just don't give a damn"
http://www.cs.trinity.edu/~rjensen/temp/AccounticsDamn.htm
One more mission in what's left of my life will be to try to change this
http://www.cs.trinity.edu/~rjensen/temp/AccounticsDamn.htm 

What went wrong in accounting/accountics research?  ---
http://www.trinity.edu/rjensen/theory01.htm#WhatWentWrong

The Sad State of Accountancy Doctoral Programs That Do Not Appeal to Most Accountants ---
http://www.trinity.edu/rjensen/theory01.htm#DoctoralPrograms