Bishops Teaching Children: Chapter 6

A Practical Method
By Which Roman Catholic Bishops
Can Personally Direct
The Religious Education
Of the Children of Their Dioceses

John Kelleher
Cambridge, Massachusetts USA
Written in the Year of the Great Jubilee
A.D. 2000

That may seem an odd way to begin a chapter on Science, but it serves to emphasize that no one can use anything that has rotten parts.

When a local church with its local ordinary uses the Bishops Teaching Children method, people are going to make money, or lose it, on the basis of how well their methods enabled the students of the diocese to answer bishops' Questions. Within an implementation of the Bishops Teaching Children method, Science determines how well bishops' Questions were answered, and whose methods were responsible, and therefore Science determines where the money flows. Quite obviously then, for an implementation of the Bishops Teaching Children method not to be rotten, its Science element must be of great integrity.

In some American industries and government agencies, as well as in American primary and secondary education, it is common for 'consultants,' 'developers,' 'sellers,' and 'evaluators' to drift back-and-forth between very porous boundaries. For example, a 'consultant' for a company one week could well serve as a government-funded 'peer evaluator' of similar products the next week.

Fields not in good order (such as American education) have poor evaluative processes to begin with. The rigor of evaluations within such fields can only be decreased further when there is no bright line between potential sellers and potential evaluators.

As will become clear, within an implementation of the Bishops Teaching Children method, the Science actually done 'in house,' by an employee of the local church and the local ordinary, is limited in scope, though of course, not in importance.

The chief Scientist functions first of all as a psychometrician (a specialist in test design and evaluation), then as a specialist in research design (so that Competitors can be evaluated fully and fairly), and then as the scientific editor and principal writer of a yearly publication that is part scientific journal and part Consumer Reports (the independent magazine that evaluates consumer products, published by a non-profit organization which accepts no advertising or other gratuities from manufacturers).

(Although the term 'the' Scientist - one person - will be used here for convenience, in practice a small number of different people may do different parts of the job of 'the' Scientist. For instance, a psychometrician is probably needed only during times when Questions are being written and tests prepared and evaluated.)

This Scientist and a small staff will probably be all that is needed. Moreover, there is no reason why he and his staff could not be funded directly by Competitors, based on a kind of tax Competitors pay proportional to (for example) the number of children using their methods.

However, it is imperative, for many reasons, that the Scientist and his staff have absolutely no other connection with any person or organization who might profit from the evaluations. Therefore it is suggested that

Catholics certainly have the means to understand that observing such niceties can not be a pro forma exercise, easily dismissed as inessential or 'impolite.' Catholics understand that man is fallen, prone to sin. People do sometimes deceive themselves. They lie, they steal, they cheat, they collude, they hurt children for money. These are the facts, and an implementation of the Bishops Teaching Children method will rapidly go rotten if it does not persistently acknowledge them.

Further, the 'transparency' of the Bishops Teaching Children method is essential to the success of the students. Severe harm may result if the 'transparency' of the Bishops Teaching Children method becomes clouded, or suspect; that is, if the full truth is not told consistently, or if people even suspect that the full truth is not being told consistently.

Therefore, just as everyone needs to know that bishops and only bishops wrote every single Question, everyone needs to know that the Scientists working for the local church and the local ordinary have absolutely no financial stake in how the yearly evaluations turn out. Everyone needs to know that each yearly Report was produced without regard for anything but the truth. Everyone needs to know that, within the limits available to current Science, each yearly Report states exactly what happened.

No Scientist - no one - writes Questions for a bishop, nor even 'helps' a bishop write Questions. To say it one more time, Questions are written solely by bishops. However, the Scientist does need to help the bishop assemble Questions into a Scientifically valid yearly set to be given to the children.

So, once Questions have been written, the 'good' ones have to be found by the trial-and-error process outlined in Chapter 5, the bishop has to decide the weight that corresponds to 'adult minimal competence' for each Question, enough Questions have to be selected in each of the four parts of the exam (which correspond to the four parts of the Catechism), and so forth.

Even though only he or another bishop can write Questions, the bishop still needs the assistance of the Scientist even to know what is needed to assemble Questions into a Scientifically valid yearly set, and of course, once the bishop knows that, he still needs the Scientist's help to actually do it.

Beyond assisting in assembling the yearly set of Questions, the principal business of the 'in house' Science of the Bishops Teaching Children method is the production of a yearly Report. The Report will consist of four major parts.

1. One, a summary for lay readers. This section should be similar to the reports on products in the independent consumer magazine, Consumer Reports. It should summarize in an accessible format the success of the various competitions.

However, more informal competitions, involving other adults, are also going on. It is conceivable that answers could be obtained for questions like the following.

3. Three, raw data regarding each Competitor at least sufficient to provide knowledgeable readers, and Competitors, with enough information to be able to check the Scientist's work.

4. Four, the 'Competitors' Forum' section. This is the section in which Competitors get to complain that they were not evaluated fairly, or otherwise suggest objective improvements to the methods of evaluation. Naturally, Competitors also get to challenge the objectivity and scientific validity of any other Competitor's complaints. There will be further discussion of the 'Competitors' Forum' section later on this chapter.

The yearly Report is meant to present the evidence at the diocesan level (and, as mentioned above, perhaps also at the parish level). However, each pastor in every parish should of course also be supplied with a list of the performance of each child and each religious education class in his parish. Similarly, each family should be given information about the religious knowledge of its own children.

In any event, even apart from the yearly Report and the separate localized reports sent to pastors and families, all raw data must be made permanently and conveniently available to all. All should have convenient and permanent access to all information collected by the Bishops Teaching Children method, and at no or modest cost. This 'transparency' is an essential component of the Bishops Teaching Children method, and is fundamental to its expression of the sacramental character of the local church as 'teacher' of 'all.'

Before beginning a discussion of 'in house' Science and its yearly Report, a word should be said about the extraordinary opportunities the Bishops Teaching Children method offers for science done 'out of house'; that is, not done under the auspices of the local ordinary and the local church.

The Bishops Teaching Children method sets up almost a dream situation for scientific research in issues related to schooling. The 'transparency' of the Bishops Teaching Children method is a godsend for any researcher.

In the Bishops Teaching Children method, Competitors use identifiable methods to advance the intellectual knowledge of large numbers of children regarding exactly the same subject matter, with all the information gathered a matter of public record and readily available to anyone who wants it. This combination of circumstances creates an impressively congenial climate for sound scientific investigation of many important matters related to schooling.

However, the steady, unflagging optimism which the Bishops Teaching Children method shows in its humble 'transparency' toward any and all 'out of house' scientific investigation is not founded on optimism about science, or even on optimism about human nature, but solely on the Lord, who gave the Church the mission to "Go therefore and make disciples of all nations, baptizing them in the name of the Father and of the Son and of the Holy Spirit, teaching them to observe all that I have commanded you." [cf. Mt 28:19-20]

A strenuous effort was made in Endnote 1, Chapter 2 to put the Bishops Teaching Children method on a firm sacramental and moral foundation, and to show that 'privacy' can not be the proper foundation for handing on the faith the local church professes in union with the whole Catholic Church, precisely because 'privacy' takes little or no account of the sacramental character of the local church as 'teacher,' and that of the local ordinary as chief teacher, of 'all' the children of the diocese.

The sizable technical and practical advantages of 'transparency' were also briefly noted there. However, to reiterate the point made in that Endnote, the only real issue is not technical or practical, but sacramental and moral. The Bishops Teaching Children method, or any proposal for religious education, stands on a sacramental and moral foundation, or it can not stand.

Further, the transparency intrinsic to the Bishops Teaching Children method enables many others, including both friends and enemies of the local church or the whole Catholic Church, to readily determine the extent of students' knowledge of the faith. This can be a great source of benefit to the local church and even a means of evangelization, but it should be noted that with this knowledge those others also will possess additional power, which, like all human power, can be used both for good and for ill. Such risks can be assumed with a glad heart only if it is clear that their assumption is part of the expression of the sacramental character of the local church and the sacramental and apostolic character of the local ordinary as chief teacher.

Within 'privacy,' everything is ultimately divided into the non-intrusive, and the intrusive. Non-intrusive is always good, and intrusive is always bad. It would seem then that, according to 'privacy,' the radical transparency of the Bishops Teaching Children method is very, very intrusive, and therefore, very, very bad.

However, it is ironic that someone who wanted to defeat the ideas of the Bishops Teaching Children method by using an argument from 'privacy' would be unable to do so. To the contrary, the Bishops Teaching Children method agonizes about its 'intrusiveness' far more than 'privacy' ever could, because in reality the Bishops Teaching Children method is very easily made non-intrusive, at least by 'privacy's' own standards.

To accomplish this seemingly impossible feat, only one thing would be required: that all the participants in the Bishops Teaching Children method make a 'private' decision to be transparent!

To see how this could work, we need to take a closer look the definition of 'intrusive,' according to 'privacy.' In 'privacy's' strongest use of the term, someone has been 'intrusive' if they have obtained information about you that you haven't traded to them for something of benefit to you, or that you haven't deliberately given to them of your own free will.

Thus, the mere fact that a complete stranger possesses information about you, does not offend 'privacy.' According to 'privacy,' this is only 'intrusive' if you did not (implicitly or explicitly) give your permission for them to have it. It is normally not considered 'intrusive' to ask people to deliberately give away 'private' information about themselves of their own free will, or to ask them to trade that 'private' information for something you have which they value.

However, as we will see, Americans can also make 'private' information available to millions of people, not trade it for anything that those people give back, nor even deliberately give it to them, and that also may not be considered unduly intrusive. Even further, some 'private' information is simply public knowledge, with or without a person's permission, and that also is not considered intrusive.

Americans already make all sorts of information about themselves available to complete strangers all the time. For instance, when I went to the supermarket today, the store gave me fifty cents off the grapes I had purchased. I got this discount after I gave the clerk a special card to scan, which enabled the supermarket to keep a record of all my purchases today and associate them with my name and address. Nearly everyone at my supermarket now gives these cards to the clerk, and thus regularly 'trades' their entire purchasing history to the store, in exchange for a small discount on a few items.

American students regularly give colleges the scores they made on various standardized tests. In effect they trade this information to colleges, not even for a few dollars off tuition, but merely in exchange for the possibility that the colleges will admit them as students.

Of course, nearly all Americans have also traded 'private' information about themselves to insurance companies, credit bureaus, and a whole host of other businesses, in exchange for the goods and services that those businesses offer.

These are examples of 'privacy' being exchanged for a perceived benefit, but exchanged only with the entity providing the benefit. Sometimes, however, 'privacy' is traded for a benefit offered by only one of the millions of entities which eventually end up with the information, and that is not considered intrusive, either. The "Publisher's Clearing House Special" is an example of this.

The "Publisher's Clearing House Special" is a famous free raffle, which millions of Americans enter every year. It may be different in its very latest incarnations, but for years, a condition of winning the raffle was that you make your name, address, and image publicly available, solely at the discretion of the company. I have seen advertisements on national television and in national print media in which a person's name, city, and image appeared, together with the fact that he won ten million dollars.

If a stranger came up to that same winner on the street and demanded, "Tell me your name, the city where you live, and how much you have in your bank account," the winner would probably have refused to do so. However, a winner in this raffle in effect gave permission for millions of complete strangers to possess that very information about him - the same information he might have refused to divulge if asked directly. Moreover, none of these millions of strangers were paying or exchanging anything for this information, and some of them might try to use the information to exploit the person who allowed it to be given out. Nonetheless, this was not considered 'intrusive.'

Finally, many 'private' aspects of our lives are simply public knowledge, with or without our permission. In essence we have 'traded' the information as a condition of living in our society. If we belong to a political party and are registered to vote, that is information available to anyone. In most municipalities, if we own a home, anyone at all can look up exactly how much we paid for it. As a condition of voting, or of buying a home, we make 'private' information available to anyone who wants it.

In sum, 'privacy' ironically would have no choice but to allow the radical transparency of the Bishops Teaching Children method, as long as the local church and its local ordinary made it clear that each student's 'privacy' was being freely 'traded' by his family for the 'benefit' of participating in the diocese's religious education. 'Religion' is a strictly 'private' matter in America, to begin with. Making a 'private' decision to be transparent would have to be acceptable, by any of 'privacy's' several standards.

Thus, requiring the signing of a legally sound version of the following would probably be more than sufficient to defeat any argument from 'privacy':

As the parent/legal guardian of the student registering for religious education in this diocese, I hereby acknowledge that regular diocesan examinations are an intrinsic part of this schooling, and agree that any and all scores my child makes on the diocesan examinations, associated with his/her name, age, and parish, are the property of the diocese, and will become a matter of public record, at the sole discretion of the diocese.

However, the Bishops Teaching Children method, possessing a higher standard than 'privacy' by which to judge the conduct of its own affairs, is only beginning its moral reflection at the point at which 'privacy' must fall silent.

For in fact, any actions a local church takes to teach, which do not clearly express its sacramental character, do indeed put human dignity at risk. It would be right to use 'privacy's' worst epithet against those actions: they are 'intrusive.' Outside of the context of its sacramental character, a local church has no business inquiring into the intellectual knowledge of even one person.

Therefore, the local ordinary should reserve the subject matter taught and monitored within an implementation of the Bishops Teaching Children method solely to the intellectual knowledge of the faith the local church professes in union with the whole Catholic Church. As was argued in the previous chapter, this careful restriction could have a worthy practical expression in the goal of bringing 'all' to adult minimal competence in the Catechism of the Catholic Church.

The foregoing discussion was undertaken to drive home the point that the radical transparency fundamental to the Bishops Teaching Children method need not offend 'privacy.' A Sacred Pastor can, without offending 'privacy,' express his true pastoral mission, by teaching 'all' 'as much as possible' about the faith.

In keeping with the legal structures of our time and place, he simply asks families to 'trade' the 'private' information needed to keep the Bishops Teaching Children method radically transparent, for the 'benefit' of religious education. Once all the participants in the Bishops Teaching Children method make a 'private' decision to be transparent, 'privacy' is fully satisfied!

At the same time, the local ordinary carefully guards the true dignity of 'all,' not only by helping them to learn 'as much as possible,' but also by ensuring that what 'all' learn within an implementation of the Bishops Teaching Children method is limited to intellectual knowledge of the Catechism.

This brief digression, in which the radical transparency of the Bishops Teaching Children method is shown to be able to satisfy the requirements of 'privacy,' is now over. However, that in no way ends the matter for Catholics, because the radical transparency of the Bishops Teaching Children method guarantees - engineers in - an intrinsic unpredictability. We should not fear that unpredictability, but why should we not?

There will be unexpected dangers and challenges associated with using the Bishops Teaching Children method. There may also be unexpected large benefits; for instance, as suggested here, there may be a massive increase in the ability of scientists to understand and thus improve the religious education of Catholic children, and it is conceivable that the great bulk of this science will be performed with little direct cost to the Church.

However, and to repeat, as blunt and as coarse as it is, the Bishops Teaching Children method still does not stand or fall on the basis of any purely utilitarian assessment. Although there are many cogent technical and practical reasons to think that the Bishops Teaching Children method will work much better than the current system of religious education, it should not be adopted by any local ordinary and local church purely on the basis of a technical or practical assessment.

The ultimate basis of the Bishops Teaching Children method is not a technique. Indeed, the Bishops Teaching Children approach is not founded on confidence in human cleverness or even human decency. Insofar as possible, the Bishops Teaching Children method tries to remain as humble as dirt, and places its confidence in the Lord alone. It is the Lord who has told the Church to teach 'all.' This mandate the Church can not lay down, whatever the cost.

The 'transparency' essential to the Bishops Teaching Children method of its very nature injects a wild unpredictability into the religious education of the children of a local church. No local ordinary who wants to control the process of religious education (rather than its outcome), will find the Bishops Teaching Children method congenial to his purpose.

Similarly, no local ordinary will have much love for the Bishops Teaching Children method, if he would rather not deal with strong and yet unexpected challenges, even attacks, on the basis of the additional power which its 'transparency' will provide everyone - the faithful, bureaucracies, all friends and all enemies of the local church or the whole Catholic Church, and etc.

Thus, it is of the essence of the 'transparency' of the Bishops Teaching Children method that, for example, the sheer amount of profound science it will unleash is - literally - unpredictable, as is the amount of benefit that science would provide to the religious education of Catholic children and even to the general schooling of all children of the world.

However, in the end, the 'transparency' of the Bishops Teaching Children method is not founded on some benefit which, by definition, is unpredictable, but rather, the 'transparency' exists for one purpose alone: it enables the local church, and all its members, in union with the local ordinary as chief teacher, to express its sacramental character as 'teacher' of 'all.'

It can not be said often enough: in the end, the Bishops Teaching Children method is to be judged not merely by a cost-benefit analysis (which it probably would well satisfy), but also and decisively on whether it truly does express the sacramental character of the local church and the apostolic authority of the local ordinary as its chief teacher better than any current rival.

Almost certainly, at least some 'out of house' science (and perhaps many other forces and powers, as well) will take advantage of the 'transparency' of the Bishops Teaching Children method. There will be costs associated with that. Whatever the costs, these should primarily be balanced, not against equally unpredictable benefits, both present and future, but against whether the Bishops Teaching Children method is or remains the best practical method of expressing the local church's sacramental character and responsibility, under its local ordinary as chief teacher, to teach 'all' its children intellectual knowledge of the faith it professes in union with the whole Catholic Church.

The Bishops Teaching Children approach, humble as dirt, ceaselessly looks forward to the day that it dies, replaced by some much more faithful system of religious education. Until that day, the basis for its unflagging optimism about the ultimate consequences of its 'transparency' is not any human agency or power, but the Lord alone, who is with the Church "always, to the close of this age." [cf. Mt 28:20]

Thus far we have discussed three important initial or foundational issues in the Science element of the Bishops Teaching Children method. One, the necessity for great integrity in that Science element, particularly because money, and perhaps other rewards and inducements, will flow directly from the evaluations performed by that Science.

Two, as has just been shown, regarding the intrinsic and essential transparency of the Scientific data and results that are generated by the Bishops Teaching Children method, the demands of 'privacy' can rather easily be satisfied, and on 'privacy's' own terms.

Three, the necessity, in and out of season, for imperturbable optimism regarding that transparency. Down to the smallest detail, down to every Competitor's entire record of success, down to each and every child's subscores on the yearly Questions, all of it should joyfully be made a matter of public record, conveniently and permanently available to anyone who wants to look at it.

To repeat, this radical optimism is not founded on the supposed benevolence of a pluralist society, or even on the supposed individual sanctity or consummate wisdom of each baptized member of the local church, but only on the Lord, who himself has given the local church the mission to teach 'all,' and therefore, the duty to know what 'all' have learned about the faith the local church professes in union with the whole Catholic Church.

Now that these preliminary points have been made, what does the 'in house' Science of the Bishops Teaching Children method look like?

All discussion of the 'in house' Science done by the Bishops Teaching Children method must begin by observing that, regarding all methods and curriculums for teaching students how to answer bishops' Questions, the Bishops Teaching Children method never heard of the word 'best.'

The Bishops Teaching Children method systematically replaces the word 'best' with the phrase, 'better than current rivals.' Thus, for the Bishops Teaching Children approach, there are no 'tried and true' methods and curriculums. Every single method and curriculum is forever 'experimental,' radically provisional.

Nor is any method or curriculum ever measured against a theoretical ideal. Each method and each curriculum survives or falls, lives or dies, on the basis of head-to-head competition against actual, practical, current rivals, and on that basis alone.

As was noted in Endnote 2, Chapter 5, this competition must not only be yearly, but also longitudinal. The Bishops Teaching Children method defines the sole goal of religious education to be adult minimal competence in the Catechism. Thus, yearly Competition is only preliminary Competition.

Establishing each Competitor's relative ability to effect adult minimal competence in the Catechism is the sole purpose of the 'in house' Science of the Bishops Teaching Children method. That requires studies over a long term period of time, and such longitudinal studies are not optional, for the following sharply practical reason. It is already known that the benefits of some educational programs fade over time, leading to no long-term advance in student knowledge. A short-term success does not necessarily predict long-term effectiveness.

For example, by 1985, it was already known that the federally-sponsored Head Start program was of no long-term academic benefit to the low-income pre-schoolers for whom the program is intended. Children attending the program display some initial scholastic gains, but "in the long run, cognitive and socioemotional test scores of former Head Start students do not remain superior to those of disadvantaged students who did not attend Head Start." 1*

Despite initial apparent success with students, the Head Start program has no effect on how much low-income children learn in the long run - the only thing we're really interested in.

If a longitudinal study had never been conducted, we might easily have believed that Head Start was a 'success.' After all, the short-term gains achieved by students in Head Start programs are demonstrable. The problem is, these gains don't last. Longitudinal studies are important.

(Vigorous competition based on outcomes is also very important. No evidence refuting the government's own findings about the academic significance of Head Start has emerged since 1985. Nonetheless, the Head Start program continues to be touted and funded, and supported by both political parties. In the meantime, French pre-schools (ecoles materneles), which are run employing an educational philosophy anathema to American educational specialists, do demonstrably raise both the long-term academic achievement and the socioemotional competence of racially and ethnically diverse low-income children.) 2*

In the first place, then, the Science of the Bishops Teaching Children method is concerned about long-term efficacy; specifically, how effective each method or curriculum is at getting children to adult minimal competence in the Catechism. As has just been shown, short term success does not necessarily predict long-term effectiveness.

On the other hand, there are strong reasons to think that short-term failure usually does accurately predict long-term ineffectiveness.

The commonsense way to say this is that the Science of the Bishops Teaching Children method is concerned not only with efficacy but also about safety. A method or curriculum that is markedly ineffective compared with its Competitors after one year may not be safe.

There could conceivably be methods that look ineffective in the short term, which are actually superior in the long term. For instance, in the United States every year, a substantial number of children leave the first grade not reading at grade level. Moreover, many American children in this situation at the end of first grade never catch up. The American educational system apparently does not typically correct reading deficits. Instead, these deficits remain. For instance, in a longitudinal study of 54 children from first through fourth grades, Dr. Connie Juel 3* found that, with nearly 90 per cent probability, poor readers in first grade were poor readers in fourth grade.

Suppose that a Competitor, knowing this, decided to spend the entire first grade of 'religious education' not on religious education per se, but on ensuring that every single child in the class read at grade level. (Methods of beginning reading instruction far more effective than those commonly used in American schools may be available.) 4* What might the long term and the short term consequences be?

It is even conceivable that no short term knowledge deficit would be evident in those first graders. After all, as noted in the previous chapter, before beginning Competition, all new methods and curriculums must pass a prima facie evaluation to ensure that at least nothing contrary to the faith will be taught, and that at least some amount of all four parts of the Catechism will be taught (how small an amount is up to the Competitor, not the bishop). Perhaps the children would even practice their reading using material based on elements of the Catechism.

However, suppose that those children were behind their peers in knowledge of the Catechism after having completed the first grade. There is no question that, over the long term, a class composed entirely of strong readers is going to learn the Catechism more efficiently than classes that have many poor readers. The Competitor's approach, which might have looked 'ineffective' in the short term, probably would be measurably more effective than many other methods over the long term.

The Competitor would have taken some time away from direct religious education, in order to teach the children a sub-skill not directly related to teaching the Catechism, but exceedingly beneficial to their learning of the Catechism in the long run.

An enormous amount of pedagogical ineffectiveness measured over the short term is almost certainly the result of inept, clumsy, inane, and otherwise bad teaching. Continuing to reward such pedagogies with additional religious education monies would be stupid, and dangerous for the children. On the other hand, we have just imagined the possibility that not all short term 'ineffectiveness' is due to pedagogical ineptitude. To the contrary, we have imagined a situation in which a short term measurement did not clearly reveal the benefits of a Competitive method that was actually more effective in the long run.

We certainly would not want to eliminate such a possibility a priori. We definitely want to reward, not punish, daring and ambitious Competitors who have a more sophisticated understanding than current rivals.

Nonetheless, a year is a very long time, particularly in the schooling of a young child. In the interests of protecting the children, it is not too much to mandate that any programs that are grossly inferior to their current rivals after one year, should receive no more funding, and that programs which are substantially but more modestly inferior after two years should also be given the ax.

It is simply too dangerous to continue to teach children using methods and curriculums demonstrated to be markedly inferior to current Competitors.

Thus single-year Scientific evaluations are performed not to establish efficacy, but to monitor safety. Yearly evaluations exist solely to weed out any grossly inferior programs as quickly as possible.

Ultimately, long-term efficacy is the only thing the Bishops Teaching Children approach cares about. Long-term efficacy is also the Bishops Teaching Children method's ultimate definition of 'safety.' Nonetheless, the Bishops Teaching Children method also cares about short term safety, and monitoring that is the real purpose of the yearly Scientific evaluations.

The Bishops Teaching Children method is designed to let money flow toward those Competitors whose methods of religious education are more effective at educating children to adult minimal competence in the Catechism, based on the children's answers to bishops' Questions.

Why then is a Science element necessary at all? After all, children are going to receive a definite score on their yearly examinations. Couldn't you just look at their scores, and see which Competitor produced the highest scores? Why would a Science element be needed to 'evaluate' this?

The general answer is found in the expression, 'all else being equal.' All else being equal, you could just look at scores and separate the better methods from the worse. For example, suppose the children schooled using Competitor A's methods produced an average score of 60, while children using Competitor B's methods made an average score of 61.

Suppose we let the two Competitors argue for a moment, to see why a Science element might still be needed.

A. The 'difference' in scores is not real. Suppose you gave a similar examination to the children 100 times. The children might make slightly different scores each time. How do you know that, the next time they took the exam, our children wouldn't have scored 61, and yours 60?

B. Just look at the numbers! 61 is a higher score than 60 - case closed. Diocese, give us the money!

In reality, A is correct. On different days, people do make at least slightly different scores on similar exams. It is indeed possible that, on another day, A's children might have made the higher average score. The 'difference' in the two competitors' scores might not be real. We might unfairly be judging B to be better, when all that may have happened is that more of B's children had a lucky day on the day of the test. It could even be that A was the superior method!

Is the apparent difference between the scores of Competitor A and B a real difference? Science exists that can help resolve this question, but the point is that the question itself is much more complicated than it appears to be at first glance. For instance, the notion that an observed difference in scores might only be an apparent difference, that it might not necessarily represent a real difference, is itself quite an advance over "common sense."

Take another case. This time, A's children make an average score of 91, while the children taught using B's methods still average 61.

A. It is impossible for a thirty-point average difference to be the result of all our children having a good day! This difference is real - our methods are far superior to yours. Diocese, give us the money!

B. You found the ten brightest children in the diocese and trained them using your method. Your 'average' score is the average of only ten children, and the ten brightest you could find, at that. By contrast, 10,000 children were instructed using our method, and took the examination. You are claiming that money should flow toward your company and away from ours, not on the basis of the superiority of your method, but on the basis of the prior abilities of the very small and carefully selected group of children who used your method.

Here it is Competitor B who is making an excellent point. Are the groups of children using the two methods similar enough to say that the difference in scores - even though it was large - was really due to a difference in the methods, rather than to differences in the children? Again, the situation is not as simple as it first appeared.

A score on an examination is a fact. However, that fact by itself does not tell us what it means, nor what we should do about it. Specifically, should we stop the local church's religious education money from flowing to a Competitor because of that fact? Should we reward a Competitor with additional funding because of it? A score on an examination does not tell us this, nor are common sense approaches to such questions necessarily adequate. Indeed, common sense approaches may lead to grossly inaccurate evaluations in some cases.

Considerations like these provide one more reason why the Bishops Teaching Children method has three elements. Competition is locked tight to Scientific evaluation of students' answers to bishops' Questions. All three elements of the Bishops Teaching Children method are essential to its existence.

Another reason that the Science element is an essential component of the Bishops Teaching Children method is that a local church and its ordinary might decide to allow sub-Competitions within the overall Competitive goal of bringing 'all' the Catholic children in the diocese to adult minimal competence in the Catechism.

For instance, 'all' means 'all.' But what if a local church suspects that the baptized children of many inactive Catholics in the diocese are not even attending religious education? What if a local church has, for some parts of the year, responsibility for the religious education of the Catholic children of migrant workers? What if (as given in the example above), many Catholic children in the diocese fail to learn the Catechism not because they don't try but because they are poor readers?

Should a local church simply throw up its hands at these difficulties? Should it rely on the haphazard and (in particular) unevaluated schemes of a small number of idealistic and temporary volunteers? Should it create its own (unevaluated and uncompetitive) paid bureaucracy, give it the appropriate title, and claim that the existence of the bureaucracy is the solution of the problem?

These familiar 'solutions' are not the only ones available when a local church uses the Bishops Teaching Children method.

Rather than resort to despair, a priori and unevaluated techniques and theories, or the formation of a bureaucracy, a diocese could allow rivals to Compete to make such situations better. After all, nowhere in the Catechism does it state that a local ordinary, or a bureaucracy under his control, or even earnest, hardworking volunteers, must necessarily have even good solutions to problems such as these, let alone the best solutions among all that could be devised.

Competition will certainly not solve all problems. Also, Competition, as the unflinchingly crass and coarse Bishops Teaching Children method envisions it, is always at root a Competition for money, and therefore, a local church has to have some money available before it can have such sub-Competitions.

However, suppose a local church and its ordinary were willing and able to try sub-Competitions in particular instances. As long as all of them also remained resolutely part of the Bishops Teaching Children method; that is, as long as they all remained locked exclusively on achieving adult minimal competence in the Catechism for 'all,' as demonstrated by answers to bishops' Questions, and as long as the short term safety measures outlined above were observed, a local church could certainly encourage such sub-Competitions if it wished.

In reality it might be very difficult to satisfy the two simple requirements mentioned in the preceding paragraph, since in the United States at least, programs to 'help' someone or to alleviate some condition hardly ever stay locked on one goal and are hardly ever evaluated only by their success at achieving that goal.

Can Americans really even conceive of brisk and pointed sub-Competitions within the sole goal of the Bishops Teaching Children method, to enable 'all' to reach adult minimal competence in the Catechism? Will they instead inevitably think up 'promising' new goals that are proclaimed to be 'extensions' to that sole goal, goals evaluated by who knows what criteria? Will romantic Americans inevitably find the unyielding humility of the Bishops Teaching Children method, which only tries to do one limited thing well, just too boring, and a little too unremittingly practical?

Even to ask such questions is in part to answer them. It remains to be seen whether true sub-Competitions within the overall Competition of the Bishops Teaching Children method are even possible, at least within American Catholic dioceses. Of course, if they are, evaluating their success without the Science element of the Bishops Teaching Children method would be practically impossible.

One further word about Competitors is needed here. Religious education, like all education, will have methods of organization and administration. These methods may be a pre-packaged part of a particular Competitor's method, or they may be separate. (For instance, a parish may have purchased religious education textbooks, and also have hired someone to administer its religious education program.)

If separable from other religious education costs, the Scientific Report must view these methods of organization and administration as separate 'Competitors', and determine how relevant they are to increasing the children's knowledge of their faith.

So, in the case given just above, it may be possible to compare the scores of children in a parish which had hired a religious education administrator, to the scores of children in a parish which had not, to the scores of children in a parish which had used a Competitor that included administration and organization of the program as a pre-packaged part of its method.

If it is Scientifically not possible to 'unbundle' the different specific Competitors that are used, then the Scientist lists them as a single, bundled Competitor, for example, "Sister Jane plus Textbook X."

The Bishops Teaching Children method does not want the organizational and administrative costs of religious education to be hidden, or simply assumed. As far as possible, organizational and administrative methods also must be subject to inexorable Competition, based not only on cost, but also on whether the children learn more as a result of using those methods.

The four parts of the yearly Report, the first three of which are written by the local church's Scientist and his staff, can now be briefly outlined. As mentioned above, the first part of each year's Report is for lay readers, and summarizes all evaluations using an attractive and accessible format, similar to the consumer magazine, Consumer Reports.

In the first part, the Scientist will describe the results of informal competitions (for example, between both parishes and dioceses, as described above). In addition, he will identify any Competitors that are grossly inferior to the others after one year, and any that are substantially if more modestly inferior to rivals after two years of evaluation. These Competitors are the ones to receive no funding in future years.

After that, unless he can compare Competitors on the basis of their long term efficiency at bringing students to adult minimal competence in the Catechism, he should sit tight and do no further evaluation of Competitors, except regarding any sub-Competitions authorized by the local ordinary.

Specifically, the Scientist should not rate Competitors on their short term results, if these Competitors have met the standards for safety described above. All safe Competitors should receive no further short term evaluation by the Scientist.

The Bishops Teaching Children method is solely interested in long term efficacy. People need to have that message burned into their brains, and the yearly Report is a good place to start.

Moreover, any truly superior Competitors are going to make their rivals look 'unsafe' after a certain amount of time. Truly superior Competitors are going to 'raise the curve.' Competitors rated 'safe' over one or two years may look 'grossly inferior' to truly superior rivals over three, four, or five years, and will be culled then.

Within an implementation of the Bishops Teaching Children method, Competition is inexorable. It never ends. No Competitor ever gets a free pass into next year's Competition. Once short term safety has been established, there is absolutely no need for yearly Reports to make snap judgments about the possible long term efficacy of any Competitive methods and curriculums. The Bishops Teaching Children method will sort things out soon enough.

At some point, of course, some Competitors will have established their relative long term efficacy. It should be noted that even the worst of these 'survivor' Competitors will have established its short term safety, every single year of its existence - an impressive standard in itself.

Eventually, even such 'survivors' could be culled, if some of them were so superior that other 'survivors' looked 'substantially inferior' by comparison. Even if such overwhelming superiority by a particular 'survivor' was never observed, a local church would still know of one or more Competitors who safely and effectively brought students to adult minimal competence in the Catechism.

An important point needs mentioning here. Once a Competitor has established long term efficacy, it can no longer be measured against any Competitor who had not done the same.

Remember, short term evaluations are performed only in the interests of safety, and long term efficacy is the Bishops Teaching Children method's ultimate definition of safety. A Competitor who had established long term efficacy is 'safe' by a standard higher than any standard available over the short term.

Therefore, a 'survivor' can only be deemed 'unsafe' if it is substantially inferior to other 'survivors.'

On the other hand, 'survivors' should form part of the pool of Competitors by which short term safety is measured. New Competitors that are grossly inferior to any rivals, including 'survivors,' should be culled after one year. (Also, of course, new Competitors should be culled after two years if they are not grossly but still substantially inferior.)

One important aspect of the first part of the yearly Report will be the listing of the per-pupil cost of each 'safe' Competitor. (If a parish uses 'mixed' methods that can not readily be separated, the total per-pupil cost is listed.) The radical fiscal transparency shown in Chapter 7 to be a valid aspect of the general radical transparency of the Bishops Teaching Children method will give the Scientist ready access to this information.

In a second and technical part of the yearly Report, the Scientist will explain his methods and give the specific quantitative results of each evaluation. For instance, he will define how he establishes who might be a 'grossly inferior' Competitor compared to current rivals.

Whatever evaluative methods he chooses, the Scientist should make a very large distinction between substantial differences between Competitors, and merely statistically significant differences between them. 'Substantial' differences are statistically significant differences that matter.

Competitors within an implementation of the Bishops Teaching Children method should not gain or lose business on the basis of measurable but trivial differences. To begin with, a local church and its ordinary have absolutely no interest in Competition. A local church could care less if all Competitors survive, or if only one does, as long as 'all' are efficiently brought to adult minimal competence in the Catechism.

What possible difference to the local church could it make if a Competitor achieved this goal (for example) an average of one day sooner than rivals? Competitors might wish to receive monetary rewards on the basis of such measurable but trivial differences, but that kind of Competition is inconsistent with the overall point of the Bishops Teaching Children method, which is to reward Competitors not because they are 'significantly' better than rivals, but solely because they helped the children of the diocese come to adult minimal competence in the Catechism.

Moreover, no one is going to die as a result of having religious education that is a little sub-par. The Scientist's evaluations are not life or death kinds of decisions for the children, but they might be for the Competitors.

Furthermore, as will be discussed a little later, although the Scientist is making evaluations that will affect the fate of businesses, and the religious education of children, he is not infallible. He can make serious mistakes.

For all these reasons it is essential that the Scientist keep his evaluative focus on 'substantial' rather than merely statistically significant differences between Competitors.

In the third part of the yearly Report, the Scientist will include raw data regarding each Competitor at least sufficient to provide knowledgeable readers, and Competitors, with enough information to be able to check the Scientist's work.

The fourth part of the yearly Report is called 'Competitors' Forum.' This is the section in which Competitors get to complain that they were not evaluated fairly, or otherwise suggest objective improvements to the methods of evaluation. Naturally, Competitors also get to challenge the objectivity and scientific validity of any other Competitor's complaints.

This fourth part of each Report is essential. The Bishops Teaching Children method will probably be vastly more effective at bringing 'all' to adult minimal competence in the Catechism than the current system of religious education, but it is still entirely a human project. It is not a 'technique,' a wind-up machine 'out there' that inevitably leads to improvements if we just hop on it. The Bishops Teaching Children method is just one more human project, and that is all it will ever be.

In the best of circumstances, human beings make mistakes. They change their minds about what is 'best.' Their understanding is always only partial.

In particular this means that the Science element of any implementation of the Bishops Teaching Children method will never be a wind-up 'evaluation machine' that we can simply crank to give us an infallible 'right answer.'

The Scientist of the local church in fact will have to be involved in what is not only a crucial task for the Bishops Teaching Children method, Competitors, and all the members of the local church, but is unfortunately also a very messy and uncertain task. That is simply the nature of science.

The Scientist functions like the umpire in a baseball competition. The umpire exists so that competition is 'fair;' which is to say, that evaluations of Competitors are made as far as possible so that 'all else is equal.'

Each year, the Scientist has to choose which existing scientific techniques to use in order to do the evaluations in the yearly Report. He is the umpire. He calls a Competitor Out, or Safe, and, just as in baseball, his decision must be irreversible. If the Scientist calls you Out, you're Out. Everyone knows that arguing with the ump can result in ejection from competition.

The Scientist will consistently apply the evaluative methods he chooses, so that all Competitors will be judged by the same standards. However, even after standards are defined explicitly and applied consistently, the questions remains: are those standards really the best ones?

Because true science is not a 'technique,' a machine you can simply crank to get the 'right answer,' at some point a live human being has to make a decision, in order for science to be applied. Somebody, not some thing, not some 'technique,' a moral agent, a responsible human being, has to be the 'umpire' of the Competition within an implementation of the Bishops Teaching Children method, and his decision has to stand. That is the Scientist's role, and it is a weighty one.

This is an additional important reason why the yearly evaluations are inherently conservative, normally identifying only clearly 'unsafe' Competitors, while remaining mute about all the others.

This is also why the 'Competitors' Forum' is essential to the proper working of any implementation of the Bishops Teaching Children method.

In this fourth part, the 'Competitors' Forum,' the Scientist in effect serves as the editor of a scientific journal written by Competitors, in which the pros and cons of various evaluative methods are laid out and debated by those Competitors.

Since the entire emphasis of each yearly Report is on substantial rather than possibly statistically significant but otherwise marginal differences, the Scientist rejects submissions to the 'Competitor's Forum' which do not show substantial differences in evaluation occurring as the result of the ideas in the submission. Of course, if a Competitor can make any kind of reasonable argument that differences might be 'substantial,' the Scientist should bend over backwards to accept the article. He accepts all submissions that do satisfy this requirement, edits these to achieve a standard format of presentation among all submissions, but does not otherwise interfere.

The gold standard for scientific journals today is peer review. Unfortunately, like everything else in science, 'peer review' is not a 'technique,' some sort of machine that operates independently of limited, fallible human beings.

We should recall that Dr. Semmelweiss's proposals were rejected, both before and after being subjected to 'peer review.' We should also recall that both theory and practice within current American education is so seriously disordered that current American educational 'experts' are, for example, wont to ignore "most of the major scientific results of more than 100 years of linguistics and psycholinguistics." Finding 'peers' within seriously disordered fields is, to put it mildly, problematic.

At any rate, peer review of articles written for the 'Competitors' Forum' is impossible. By explicit definition, the Scientist is the 'umpire' of the Competition, and therefore is by definition not the peer of any Competitor. The Scientist of the local church in a real sense is 'peerless.'

Also, Competitors are not peers. They are direct business rivals, and, however intense any competition among scientists may become, the difference between a scientific 'peer' and a direct business rival remains immense. It is ridiculous, and therefore dangerous, to pretend that these rivals are not 'really' such, and can therefore be 'objective,' and act as 'peers.'

Each year's 'Competitors' Forum' will therefore be a messy, unpredictable, rough-and-tumble grab-bag of argument, counter-argument, theory, and complaint.

Nonetheless, articles in it from Competitors who seriously ignore or outright reject (for example) "most of the major scientific results of more than 100 years of linguistics and psycholinguistics" will be few, for one simple reason: such Competitors will rapidly go out of business, and will therefore have no right to submit any more articles to the yearly 'Competitors' Forum.'

When a local church uses the Bishops Teaching Children method, it is simply not going to make good business sense for Competitors to ignore long-standing consensus scientific results. Therefore, Competitors (at least, the ones who survive) will not ignore those results. Rather, Competitors who survive are apt to value scientific sophistication and rigor.

We can therefore expect the long-term scientific propriety of the 'Competitors' Forum' to be acceptable.

The Scientist of the local church is the 'umpire' of the Competition, and as such his evaluations must in general be irreversible. He calls them as he sees them, and if you're Out, you're Out.

In baseball, an umpire's decisions on balls and strikes for example, are irreversible, not because anyone expects him to be infallible, but because otherwise teams would argue endlessly about every pitch, while certainly always being by definition in a less 'objective' position than the umpire.

The parallel to the Bishops Teaching Children method is exact. The Scientist is 'right' every time, not because he is infallible, but because Competition would otherwise quickly devolve into an endless series of claims and counter-claims by Competitors, who by definition stand to gain more from a particular outcome than the Scientist.

It is important to emphasize that the Scientist must function like the umpire in a baseball game if the Bishops Teaching Children method is to work.

The baseball umpire makes important and irreversible decisions, and is neither an infallible arbiter nor a hired 'expert' guaranteed to come to our own preconceived conclusion. He inhabits a middle ground, but it is a ground with which we are still culturally familiar, and therefore we must keep our eyes on the umpire, if we are to understand the Scientist's role within the Bishops Teaching Children method.

We know in advance that the umpire is experienced, but not infallible. We know that he, and no one else, must make the call. We know that he could be wrong, but is often right. We know that his decision, right or wrong, will stand, and must stand, because otherwise competition would quickly devolve into an endless series of claims and counter-claims by competitors about what happened, kids arguing for endless hours about whether in fact "Tommy was safe by a mile!", and then the game itself would grind to a halt - perhaps even be forgotten entirely.

Either a requirement of total infallibility, or a reversion to total relativism, makes baseball impossible. To play baseball, all competitors need the umpire to make the calls, so that, within the context of the umpire's calls, the game can be played, the competition can continue.

Since these days, especially when money and children are involved, we are so very prone to resort either to claims of infallibility, or to relativism and hired 'experts,' it perhaps can not be emphasized enough: neither of these often-employed cultural models will do, if an implementation of the Bishops Teaching Children method is to survive. The Scientist must be the umpire. He must make his non-infallible evaluations, and they must stand, irreversible and unchallenged. Otherwise the Bishops Teaching Children method itself grinds to a halt and evaporates into the chaos of 'modern' life.

Thus, just as parents, prior to participating in the local church's religious education, must sign a release which has the effect of guaranteeing the Bishops Teaching Children method's transparency, all Competitors, prior to Competing, must agree in writing that the evaluations in each year's Report are final, unchallengeable, and irreversible. As a pre-condition of Competing, all Competitors must agree that the Scientist of the local church is the umpire of the Competition.

However, the methods by which the Scientist arrives at his evaluations, while therefore never subject to direct challenge or to reversal, must always be subject to long term review.

Providing this long term review is the fundamental purpose of the 'Competitors' Forum.' By means of it Competitors force the Scientist, and any other interested reader, to at least read both their objections to his methods, and their proposals for improvements to them.

In spite of their natural tendency to present biased complaints and analyses, Competitors may become aware that arguments which appear 'reasonable' or 'objective' or 'unbiased' have a greater tendency to be adopted in future Reports. They may also become aware that Competitive rivals may well challenge, in the same 'Competitors' Forum,' any arguments submitted which those rivals see as arguing for unfair advantages or as special pleading.

The 'Competitors' Forum' is a rough-and-ready, unpredictable, messy, and uncertain part of the yearly Report, and of the Bishops Teaching Children method. Nonetheless, it is an essential part. It alone provides an explicit forum for ongoing, long-term review of the evaluative methods employed by the Scientist of the local church.

Beyond the production of the yearly Report, and advising the bishop regarding the yearly set of Questions, what remains for the Scientist to do every year is to supervise the allocation of all untried 'experimental' Competitors.

These are the untried Competitive methods and curriculums, tested only for prima facie adherence to the faith, which no more than five percent of parishes try each year. The Scientist must decide how to assign these new Competitors to parishes in the diocese, how many parishes or classes need to be involved to provide results of sufficient technical precision and validity, and so forth.

As always, the Scientist should be interested in substantial improvements over rival Competitors, rather than marginal if statistically significant ones. Thus, in his research designs, he need not risk large numbers of children to establish a marginal effect. On the other hand, he should give experimental Competitors enough margin to demonstrate real potential.

This aspect of the Bishops Teaching Children method is designed explicitly for the Dr. Semmelweisses of the religious education world. We simply don't know when an obscure Competitor with an unfamiliar or 'ridiculous' idea will greatly improve religious education in one fell swoop. We also need to recall that nearly all the best medical minds of Dr. Semmelweiss's day either outright denied the existence of his vastly better results, or just 'knew' that these were not due to the superiority of his ideas, but to some entirely irrelevant and unimportant happenstance.

Further, we need to recall that, at any time, we too might make exactly the same mistake. What we 'know' is not necessarily the truth. What we 'know' can be dangerous to the children. The 'experimental' Competitors are welcomed by the Bishops Teaching Children method, precisely in order to leave the raw possibility of our abject ignorance always open, and to increase the chance that some future Dr. Semmelweiss will help the children, literally better than we 'know.'

On the other hand, it can be predicted with virtual certainty that nearly all experimental Competitors will be no better than current ones, and we can be sure that some will be drastically worse, 'unsafe.'

Nonetheless, all aspects of the Bishops Teaching Children method, including this one, must remain resolutely incurious about methods. Methods are simply not the business of the local church and the local ordinary, and opinions about methods are never expressions either of the sacramental character of the local church, or of the apostolic authority of the local ordinary as chief teacher.

However, by definition, no one can know how effective an untried Competitor will be. Therefore, the local church should of course build in a proper element of risk to the yearly experimental Competition. First of all, untried Competitors should provide all experimental materials and services free of charge. At the very least, the local church should not pay to have itself experimented on. Moreover, this adds an additional level of financial risk to the Competitor.

If too many unsafe experimental Competitors regularly emerge, a local church may decide that experimental Competitors must be culled for safety after only half a year or even sooner, rather than a year. A Scientist could certainly also drastically limit the number of children involved in any one 'experimental' Competitive method or curriculum, such that only large improvements might reliably be detected, but fewer children were exposed to potentially 'unsafe' Competitors.

If a local church wanted to further discourage crackpots, it could, for example, require all 'experimental' Competitors to post a bond, with the funds to be returned to them if their methods proved to be not grossly inferior to rivals.

In other words, a local church can legitimately decide how to strike a balance between rewarding ambition and punishing foolhardiness.

Nonetheless, a local church, the local ordinary, and the Scientist of the local church may not reject any Competitor a priori, once that Competitor has established a prima facie case to teach nothing contrary to the faith. To repeat, such a priori rejections require the local church to 'canonize' some theory about teaching methods, and this never expresses the local church's sacramental character as 'teacher.'

The explicitly 'experimental' part of the Bishops Teaching Children method is the essential Wild Card in its inexorable Competition. It reduces the opportunity for any Competitor, or any group of them, to rest on previous laurels, and allows any (by definition, unexpected) Dr. Semmelweisses a chance to emerge, to Compete, and therefore, to help the children learn.

The Bishops Teaching Children method never heard of the word 'best,' and what we 'know' can harm children, as well as help them.

The Scientist of the local church, while endeavoring to protect the children, should also remember that, and create research designs for untried Competitors accordingly.

1. U.S. Department of Health and Human Services. The Impact of Head Start on Children, Families and Communities: Final Report of the Head Start Evaluation, Synthesis and Utilization Project, Executive Summary. Washington, DC. 1985. p. 1.

2. The French studies documenting the pronounced long-term success of the ecoles maternelles are extensive, and ongoing to the present day, but not much available in English. A report contemporaneous with the Head Start study is given in Centre for Educational Research and Innovation. Immigrants' Children at School. Paris: Organisation for Economic Co-operation and Development. 1987. pp. 178-259.

3. Juel C. Learning to Read and Write: A Longitudinal Study of Fifty-Four Children from First through Fourth Grade. Journal of Educational Psychology, 1988, (80), pp. 437-447.

4. McGuinness C, McGuinness G. Reading Reflex. New York: The Free Press. 1998. A method demonstrated in a small number of experiments to be markedly more effective, both as a tutoring pedagogy for poor readers of all ages, and with beginning readers in a classroom, than typical instructional approaches. It should be noted that these studies were small, and that the specific example remains hypothetical and is given here only as a hypothetical: what if a Competitor found a more reliable method of teaching a sub-skill vital to learning the Catechism.