                                                              916

 1    UNITED STATES DISTRICT COURT
      EASTERN DISTRICT OF NEW YORK
 2    ------------------------------x
      THE CITY OF NEW YORK, et al.,
 3    
                         Plaintiffs,          88 Civ. 3474 (JMcL)
 4    
                 v.                           
 5    
      UNITED STATES DEPARTMENT OF COMMERCE,
 6    et al., 
      
 7                       Defendants.
      ---------------------------------x
 8    CITY OF ATLANTA, et al.,
                     
 9                       Plaintiffs,
      
10                v.                          92 Civ. 1566 (JMcL)
                                              
11    MOSBACHER, et al.,
      
12                       Defendants.
      ---------------------------------x
13    FLORIDA HOUSE OF REPRESENTATIVES, 
      et al.
14    
                         Plaintiffs,
15                v.                          92 Civ. 2037 (JMcL)
      
16    BARBARA H. FRANKLIN, Secretary of
      Commerce,
17    
                         Defendant.
18    --------------------------------x
      
19    
                                              May 18, 1992
20                                            9:30 a.m.
      
21    
      
22    Before:
      
23               HON. JOSEPH M. McLAUGHLIN,
      
24                                            Circuit Judge
      
25
                                                              917

 1               MR. RIFKIND:  Your Honor, following up on the 

 2    notice that I gave the other day, we did file over the 

 3    weekend and served a memorandum in support of the 

 4    proposition that the confidentiality order which governs the 

 5    exhibit we have marked in absentia, which I think is PX 212 

 6    --  it's actually Court Exhibit 1, which is the tape 

 7    generated by the Bureau of the Census with the corrected 

 8    data of every hamlet, village and block in America, the fact 

 9    should no longer be subject to a confidentiality order.  

10               We still believe that, we believe it quite 

11    seriously and we think before this proceeding is over that 

12    issue ought to be resolved in large measure because, given 

13    the public nature of this proceeding, it seems to us quite 

14    inappropriate that anything should be under wraps and is 

15    likely to engender a degree of disquietude publicly if it 

16    is.

17               But attempting to emulate a certain Solomonic 
                                                               

18    propensity that I have been instructed in the last week, I 

19    am advised now by the government that the particular 

20    exhibits that we immediately have in mind using that are 

21    derived from it and which will be used in connection with 

22    the testimony of Professor Keane --  

23               THE COURT:  Will he be the first witness today?  

24               MR. RIFKIND:  No.  Either late today or tomorrow, 

25    probably.
                                                              918

 1               --  that those exhibits they will either waive or 

 2    do not claim they are subject to the confidentiality order 

 3    and so we don't have to clear the courtroom when those are 

 4    put up or seal people off when the witness is testifying 

 5    about them, and so I would liKE to reserve on the broader 

 6    question at this point and get on with business of trying 

 7    the case without undue fuss about the broader question.

 8               I just don't want to waive my larger point.  I 

 9    think it's a substantial and real point, but in the interest 

10    of practicality we will proceed in reliance on the 

11    government's stipulation.  

12               THE COURT:  Fine.

13               Call your next witness.  

14               MR. RIFKIND:  Plaintiffs now call Franklin M.  

15    Fisher.

16    FRANKLIN M. FISHER, 

17             called as a witness by the plaintiff, having

18             first been duly sworn, was examined and

19             testified as follows: 

20    DIRECT EXAMINATION 

21    BY MR. RIFKIND:

22         Q.    Dr. Fisher, what is your occupation?

23         A.    I'm professor of economics at the Massachusetts 

24    Institute of Technology.

25         Q.    As a professor of economics, do you have any 
                                                              919

 1    areas of specialization?

 2         A.    I do.

 3         Q.    And what is that or what are they?

 4         A.    I specialized in three general areas:

 5               The first is econometrics, which is the 

 6    application of statistical methods to phenomena; the second 

 7    is microeconomic theory, analysis of how the price system 

 8    works, how markets behave; the third is industrial 

 9    organizations with particular emphasis on antitrust, and 

10    that's, in my version, it's the empirical study of how 

11    markets behave and how competition does and does not work.

12         Q.    Have you taught courses relating to the 

13    application of sticks to economics?

14         A.    I have.  I was one of the people who introduced 

15    econometrics into the graduate program at MIT more than 30 

16    years ago and with occasional pauses for rest and 

17    refreshment, I have taught some econometric course from then 

18    about --  I haven't taught it in the last two years, but I 

19    have taught it for the years between that, 1960 to 1990.

20         Q.    Does econometrics involve the estimation of 

21    statistical magnitude?

22         A.    It involves the estimation of parameters by the 

23    means of statistics, the test of hypotheses concerning 

24    economic magnitudes and similar fauna.

25         Q.    Have you written in the area of statistics as 
                                                              920

 1    applied to economics?

 2         A.    Yes, I have.

 3               I have, I have published, depending on how you 

 4    count it, at least two books involving statistical theory.  

 5    The reason I say --  

 6               THE COURT:  That brings back memories -- I used 

 7    to be a dean -- depending how you count, I have published 

 8    two books.  

 9               MR. RIFKIND:  That is the essence of this case, 

10    your Honor. 

11               THE WITNESS:  Well, I can explain why I said 

12    that, your Honor.

13               I can count beyond two fairly accurately.

14               The second book I have in mind is the second 

15    volume of my collected works, which contains a great many of 

16    my articles on the subject.  I wasn't sure whether I should 

17    count that as a separate publication.  I don't know whether 

18    the dean will count it as a separate publication.

19               Anyway, we have already counted a great many 

20    articles on the subject and various books and articles also 

21    using statistical methods as opposed to the theory. 

22         Q.    Have you been involved in other professional 

23    activities besides teaching?

24         A.    Yes.  I have spent I think it's fair to say 

25    massive amounts of time as an expert often, but not 
                                                              921

 1    exclusive, in antitrust cases and in other forms of 

 2    litigation.

 3         Q.    Are you a member of any professional 

 4    associations?

 5         A.    I am a member of the American Economics 

 6    Association, I am a member and a Fellow of the Econometrics 

 7    Society.

 8         Q.    What is the Econometrics Society, sir?

 9         A.    The Econometrics Society is the principal 

10    national society which promotes two aims, one is the 

11    application of mathematical methods to economics and the 

12    other is econometrics as I stated before, the application of 

13    statistical methods to economics.

14         Q.    Have you served as an editor?

15         A.    I have.  I was an American editor of the Review 

16    of Economic Studies in the 1960s, I was then an associate 

17    editor of the Journal of the American Statistical 

18    Association for a few years in the 1960s and I was the 

19    editor of the Econometrica, which is the Journal of the 

20    Econometrics Society from the mid-sixties to mid-seventies.

21         Q.    Has your work involved subjects other than 

22    economics?

23         A.    Yes.

24         Q.    Can you give us an example, Dr. Fisher?

25         A.    Sure.
                                                              922

 1               I was involved in two pieces of litigation 

 2    involving employment discrimination, one on behalf of the 

 3    Department of Labor in the Harris Bank case, one on behalf 

 4    of a group of employees of the New Haven Register in a case 

 5    in New Haven, and I have written I guess it's at least one 

 6    article on that subject.

 7         Q.    You testified as an expert in those cases?

 8         A.    I did.

 9               I also did a study many years ago now of the 

10    effect of the removal of railroad firemen on railway 

11    accidents which I did originally for the United 

12    Transportation Union.

13               You know, econometrics tend to consider economics 

14    fairly broad.  I did some studies of television viewing 

15    behavior.  You can consider that partly as economics or not.

16               I have served on two national Academy of Sciences 

17    either panels or as a member of a subgroup of a panel, the 

18    principal one being the one that reviewed the literature on 

19    punishment as a deterrent to crime.

20               Let's see.

21               Long, long ago, I actually wrote a paper called 

22    The Qualitative Analysis of Supreme Court Decisions, which I 

23    immediately wish to point out had the subtitle The Use of 

24    Abuses of Quantitative Methods.

25         Q.    You mentioned the National Academy work and the 
                                                              923

 1    work for the Department of Labor, I think you said.

 2               Have you served as a consultant to any other 

 3    agencies of the federal government?

 4         A.    Yes.  I served for a while as a consultant for 

 5    the Federal Power Commission to review their --  not to 

 6    review, to assist as a consultant to their staff in building 

 7    econometric models in the natural gas industry.

 8               I was on an advisory panel, I no longer remember 

 9    what it was called, for the Federal Reserve Board working on 

10    price statistics.

11               Well, I know there's another one, but I have 

12    forgotten what it is.

13               Oh, yes, I recently served as a reviewer for the 

14    office of technology assessment.

15         Q.    Have you done any work for the General Services 

16    Administration?

17         A.    Yes.  Many years ago I worked for the General 

18    Services Administration in connection with econometric 

19    models of minerals industries.  That was done in connection 

20    with the management of the strategic stockpile.

21         Q.    Have you worked for the military?

22         A.    I worked for, not directly, I worked for the 

23    office of Naval analysis, which was in turn part of the 

24    center for Naval studies or something like this, which was 

25    in turn sponsored by the Navy studying the questions of cost 
                                                              924

 1    and benefits of various incentives to re-enlistment in the 

 2    Navy.

 3         Q.    Did you testify as an expert witness on behalf of 

 4    plaintiffs in the 1980 census litigation?

 5         A.    I did, indeed.

 6         Q.    Let me show you at this time, let me ask you to 

 7    refer at this time to Plaintiff's Exhibit 651, Dr. Fisher.

 8               What is Plaintiff's Exhibit 651?

 9         A.    It's my curriculum vitae and bibliography.

10         Q.    Is that an accurate statement of your 

11    professional qualifications and publications?

12         A.    I sure hope so.

13         Q.    I take it that is a yes, more or less?  

14               THE COURT:  Depending on how you count.

15         A.    Unless this has been reput together just as a 

16    joke since I last looked at it.  

17               MR. RIFKIND:  Your Honor, I offer Plaintiff's 

18    Exhibit 651 in evidence.  

19               MR. SITCOV:  No objection.  

20               THE COURT:  651 is admitted.  

21               MR. RIFKIND:  And at this time I offer Dr. Fisher 

22    as an expert in statistics.  

23               MR. SITCOV:  No objection.  

24               THE COURT:  He is qualified to express an opinion 

25    on matters of statistics.
                                                              925

 1               (Plaintiff's Exhibit 65 marked for identification 

 2    was received in evidence.) 

 3    BY MR. RIFKIND:

 4         Q.    Prior to the commencement of this litigation, 

 5    which goes back to 1988, had you made any particular study 

 6    of the census or of the Census Bureau, Dr. Fisher?

 7         A.    Not except in connection with the 1980 census 

 8    case.

 9         Q.    Are you familiar with the term "loss function 

10    analysis"?

11         A.    Yes, I am.

12         Q.    Is it a tool that is used by statisticians?

13         A.    Yes, among others, it is used by statisticians.

14         Q.    Can you describe for the court what a loss 

15    function analysis is?

16         A.    Yes.

17               A loss function analysis is a systematic way of 

18    describing decision criteria and the consequences of using 

19    them.

20               You have to make a decision.  The decision will 

21    have various consequences, some of them may be positive, 

22    some of them may be negative.

23               In order to decide which, let us say, of several 

24    alternatives to pick, one has to either implicitly or 

25    explicitly lay down the criterion by which you are going to 
                                                              926

 1    say that one is better than another.

 2               The normal way of doing that, the explicit way, 

 3    can be described in terms of loss function.  Let me give an 

 4    example.

 5               You are an agency, you want to know which of two 

 6    drugs should be released or perhaps if neither should.  Both 

 7    drugs are partially efficacious in the treatment of some 

 8    disease, one drug, however, is more efficacious than the 

 9    other, but also more likely than the other to produce a 

10    rather unpleasant side effect.

11               In deciding whether that drug should be released 

12    rather than the drug which is less efficacious, but also 

13    less potentially harmful, you have to implicitly or 

14    explicitly weigh the desirability of relieving the disease 

15    against the undesireability of promoting the side effect.  

16    You have to decide, as it were, the trade-off between having 

17    some number of people continue to have the disease or some 

18    number of people, some other number of people get the side 

19    effect.

20               Expressed explicitly, you can do that in the form 

21    of a loss function; how badly off, as it were, will you be 

22    if people continue to have the disease versus how badly you 

23    think --  not you, but the world would be if they have the 

24    side effect.

25               Once that is done, then analysis can proceed in 
                                                              927

 1    terms of data and in terms of the probabilities that these 

 2    various things will, in fact, happen if the drug is released 

 3    so that one can make a statement as to whether or not drug A 

 4    or drug B scores higher in term of the criterion you set 

 5    forth.  

 6               THE COURT:  Where did the term come from, loss 

 7    function? 

 8               THE WITNESS:  Well, I'm not actually sure.  It is 

 9    common, you could describe this, if you took an optimistic 

10    view of the world, as the net benefit function.  

11               It is often done in terms of loss, nothing is 

12    perfect, this is the pessimistic view of the world, we want 

13    to be at least as badly off as possible.

14               In statistical uses, the term is natural, because 

15    in some sense you want to know how far away you are from the 

16    truth, the least loss.  

17               THE COURT:  Least loss? 

18               THE WITNESS:  In cases like the census, we are 

19    talking about how are you going to measure accuracy.  

20               MR. RIFKIND:  It's a dismissal science, your 

21    Honor, you would expect a negative statement.  

22               THE COURT:  All my preferences against 

23    statisticians have been validated in the last week.  

24               MR. RIFKIND:  I am alarmed to hear it. 

25               THE WITNESS:  Well, that may be because you have 
                                                              928

 1    been exposed to an onvercount of statisticians.

 2         Q.    Are you saying, Professor Fisher, that all 

 3    choices involve a loss function?

 4         A.    All consistent choices, in other words, all 

 5    choices which are made according to a criterion which 

 6    doesn't contradict itself can be explicitly written in a 

 7    loss function, although obviously not everybody who makes 

 8    decisions does that in an explicit fashion.

 9         Q.    But in consistent decisions, it is an implicit 

10    loss function even though it is not explicated?

11         A.    Yes.  Anyone who is operating without a 

12    consistent criterion can work on a loss function.  What that 

13    loss function is depends on what the criterion are.

14         Q.    Are there some general principles that are kept 

15    in mind in articulation of loss function or in the 

16    employment of loss functions?

17         A.    Well, I think so.

18               In any case involving a decision of both 

19    importance and complexity one ought to be careful to 

20    articulate what the criteria are and to articulate them 

21    sufficiently explicitly so that, as it were, the world can 

22    understand how the decision has been made.

23               Secondly, one ought to write down the criteria in 

24    advance.  That's like specifying the loss function in 

25    advance.
                                                              929

 1               If one does not do that, then after the event, if 

 2    you don't like the way the data come out, it will be 

 3    possible almost always to cobble up a loss function which 

 4    justifies the decision you want to come to anyway, but that, 

 5    of course, is not very good practice.

 6               Now, it is also true that it is not always 

 7    obvious what loss function one ought to use or what loss 

 8    function one wants to use and you can't avoid doing it 

 9    because you have to make a decision you are going to 

10    implicitly use some function or another, but in complicated 

11    cases one will typically want to use several alternative 

12    loss functions and then to see whether it makes any 

13    difference, but those also ought to be specified in advance.

14         Q.    How, if you liken this in the context of the 

15    census, is it appropriate to go about choosing loss 

16    functions to use?

17         A.    Well, as I said before, everybody says we want to 

18    make the census as accurate as possible and what we are 

19    really talking about is how are we going to measure 

20    inaccuracy, and there are several decisions that have to be 

21    made about what we are going to mean by inaccuracy and there 

22    are, in fact, several possibilities.

23               Fundamentally, what you would really like would 

24    be to have a loss function that really reflects the cost to 

25    society of making the wrong decision, but that is very hard 
                                                              930

 1    to do, because the census data are used for a number of 

 2    different purposes.

 3         Q.    A number of different purposes?

 4         A.    A number of different purposes, yes, and it's 

 5    hard to reflect all of them in one loss function.

 6               Here are some of the decisions that have to be 

 7    made and that, you know, there are good arguments to do it 

 8    one way or the other:

 9               A, are you really interested in measuring numbers 

10    of people or are you really interested in measuring the 

11    state --  I'm sorry --  the share of a state or some other 

12    political subdivision in the national population or below 

13    the state level the share of some city, let us say, in the 

14    state population and so on?

15         Q.    Share of some larger aggregate, like the national 

16    population?

17         A.    Yes, the share of some larger aggregate.

18               And there are reasons to think of both.

19               Many things, of course, depend on what I will 

20    call distributive accuracy, that is the measurement of 

21    shares; some things may depend on numeric accuracy.

22               To take an example, my understanding is that 

23    there are federal funds programs that apply to cities above 

24    a certain size and that would have to do not with how big 

25    the city is relative to the state or the nation, but how big 
                                                              931

 1    it is in terms of bodies.

 2               Secondly, there is the question of how you are 

 3    going to count errors made in one, let's say, geographic 

 4    area.  

 5               I will talk for a while now, if you don't mind, I 

 6    will talk about states as an example of geographic areas, 

 7    although much of what I have to say applies to other 

 8    definitions as well.

 9               How are you going to weight errors in one state 

10    against errors in another? 

11               If you have two alternatives and one of them 

12    makes a distributive share of, let's say, California more 

13    accurate but the distributive share of some other state less 

14    accurate, how are you going to decide on that trade-off? 

15               And here there is more than one problem to 

16    decide.  Here's an example:

17               Do states count the same?  Is a one percent error 

18    in the share of the big state just as bad but no worse than 

19    a one percentage point error in the share of a small state, 

20    or should you on the other hand be concerned, more concerned 

21    about errors where there are a lot of people than about 

22    errors where there are relatively few people? 

23               Secondly, assuming you have decided that, what 

24    about the following question:

25               Do you care about big errors --  how do you care 
                                                              932

 1    about big errors versus small errors? 

 2               Since I'm working in terms of distributive 

 3    shares, the shares have to add up to one so I can't make an 

 4    error only in one state in this example, so I will make an 

 5    error in two.

 6               Suppose that you have overestimated the share of 

 7    state A by ten percentage points and you have underestimated 

 8    the share --  well, I better take a somewhat smaller 

 9    numbers.  

10               Suppose you overestimated the share of state A by 

11    four percentage points and you have underestimated state B 

12    by four percentage points, you have made two four percentage 

13    point errors and those errors aid up to eight percentage 

14    points.

15               Now suppose another case in which you have 

16    estimated everything correctly, except that for eight states 

17    you have made, for each of them, an error of one percentage 

18    point.  The total amount of percentage point errors you have 

19    made adds up to eight in both cases, but you have to decide 

20    is it worse to have the error concentrated so there are big 

21    errors some places and no errors elsewhere or is it just as 

22    bad to have small errors everywhere else? 

23               One could decide that question either way after 

24    some reason to do it.  If you believed that errors should 

25    count the same no matter --  only the total errors, so to 
                                                              933

 1    speak, should count, then you might use or you probably 

 2    would use what the Census Bureau calls an absolute error 

 3    loss function, take the absolute value of the errors and add 

 4    them up.

 5               If you believe on the other hand that big errors 

 6    count more than a sum of small errors adding to the same 

 7    thing, then you would want to use a loss function which 

 8    reflects that.  

 9               One way to do that, quite a common way, is to 

10    square the errors and add them up.  You square the errors so 

11    that the big ones get emphasized.  If you square an error 

12    which is twice the size of another error it gets four times 

13    the importance.  

14               The Bureau did that, also, in terms of what is 

15    called the squared error loss function.

16         Q.    Is that a common practice, using squared errors?

17         A.    Use of the squared errors is the most common 

18    practice in statistics, but that is because it is, A, known 

19    to be easy to handle; B, there is some theorems about 

20    estimations involved in it; and C, statisticians are 

21    typically, how should I put it, statisticians aren't making 

22    decisions, they are preparing the results and making 

23    parameters and someone else uses the results.  But squared 

24    errors come naturally to statisticians, and by the way to 

25    others, it is used in other contexts.
                                                              934

 1         Q.    Can you think of an area in other fields in which 

 2    squared differences are used that might be familiar to the 

 3    court?

 4         A.    Not specifically offhand.  I know that it is 

 5    quite common in the literature about decisions in economics 

 6    to suppose that deviations from optima are measured in terms 

 7    of squared differences.

 8         Q.    How about the antitrust field?

 9         A.    Oh, I'm sorry, yes, there is a very good one.

10               When concentration in a particular market is 

11    measured, there is more than one way in which one can do it 

12    and you can describe this in some sense not as errors, but 

13    as measurements of loss.  We don't like concentrated 

14    markets.

15               One way to do it would be to say, use what is 

16    called the concentration ratio.  You take the share of the 

17    first four firms or perhaps the first eight firms and add 

18    them up.  That method counts --  that's like the absolute 

19    error loss function, it counts each share as equivalent to 

20    any other shares and if three of the first four firms have 

21    ten percent and the other firm has 40 percent, then the 

22    number you get is 50 percent and it doesn't matter that it's 

23    unequally spread among the first four firms.

24               A somewhat more modern measure, the one that is 

25    now in most common issue, is the Hirschman-Herfindahl index. 
                                                              935

 1               The Hirschman Herfindahl index takes the share of 

 2    all the firms in the industry and squares them, adds them 

 3    up, and then multiplies by 10,000 to make them come out 

 4    without a decimal point.  

 5               And that is like the squared error loss function.  

 6    It has the property that a very uneven distribution of 

 7    shares which adds to the same thing is counted as much more 

 8    concentrated than a relatively even distribution.

 9               One firm with a big share contributes more to 

10    concentration in that measure than two firms which have 

11    shares that add to the big firm's share.

12         Q.    And has that approach, to your knowledge, been 

13    adopted by the Department of Justice?

14         A.    The Department of Justice has merger guidelines 

15    as of 1982 and as revised in 1984 and again this year put 

16    very heavy use on the Hirschman-Herfindahl index.  In fact, 

17    that is the only measure of concentration they use, as I 

18    recall.

19         Q.    Okay.

20               Did the Census Bureau use loss functions in 

21    connection with the question of whether or not to correct 

22    the 1990 census?

23         A.    They did.

24         Q.    Can you describe to us what factors the Bureau 

25    took account in doing their loss functions?
                                                              936

 1         A.    The Bureau calculated basically three different 

 2    loss functions.

 3               In the first place, they worked always in terms 

 4    of distributive shares --  I'm describing this in terms of 

 5    the decisions I said one would have to make. 

 6               They worked in terms of distributive shares, not 

 7    in terms of numeric accuracy.  Within that they counted 

 8    errors in one state as equivalent to errors in any other 

 9    state irregardless of how many people live in one state, 

10    that is, big and small states were counted the same.

11               They calculated a squared error loss function 

12    adding the sum of squared errors, they calculated absolute 

13    error loss function adding the absolute values of the errors 

14    across the states, and --  I should say, I should remind you 

15    again they also did this for other, other definitions of 

16    locality beside state but the principal emphasis was on 

17    state --  and they also made a calculation about house 

18    apportionment.

19         Q.    I take it, Dr. Fisher, you don't and they don't 

20    know the true population of the United States, do they?

21         A.    No.  Nobody knows the true population of the 

22    United States or, indeed, of the shares of the individual 

23    states.

24         Q.    How, then, do you do this analysis without 

25    knowing what you are comparing what with?
                                                              937

 1         A.    Well, first, as I say, you decide what you are 

 2    going to mean by inaccuracy or what you are going to mean by 

 3    loss, and let me talk, I don't think it will make any 

 4    difference, let me talk about the squared error loss 

 5    function.

 6               You do not, however, literally take the 

 7    difference between the truth and the estimate, either the 

 8    census, the original enumeration estimate or the dual system 

 9    estimate, you don't take those differences, square them and 

10    add them up.  You can't do that because you don't know what 

11    the true populations are.

12         Q.    By dual systems estimates, you are referring to 

13    what?

14         A.    I am referring to the proposed adjustment.

15               You can't do that, you don't know what the truth 

16    is.  If you knew what the truth was you would use it and 

17    then, you know, then we would all be free, as it were.

18         Q.    As it is.

19         A.    Then instead what you have to do is to ask what 

20    the expected accuracy is, whether the expected value of the 

21    loss function is.

22               If you can use statistical methods to calculate 

23    how big an error on average you will expect --  I should 

24    perhaps say when statisticians and I use the term "expected" 

25    in a context like that what we mean is on average.
                                                              938

 1               If you took a set of numbers and weighted each by 

 2    the probability of its occurrence, the result would be the 

 3    expected value.  

 4               The simplest example is if I had a coin which had 

 5    a 75 percent probability of coming up heads, the expected 

 6    number of heads that will come up if I toss it a thousand 

 7    times is 750.

 8               In any event, back to lose functions, one has to 

 9    calculate the expected loss from using one estimator or the 

10    other and that it is possible to do without using the truth, 

11    because statistical methods are largely useful because they 

12    do provide measures of their own expected inaccuracy.

13         Q.    Could you give an example of how this feat is 

14    pulled off?

15         A.    I could.

16               Because this is an example that is supposed to 

17    exemplify, which is what examples are usually for, and, 

18    therefore, be easy to understand, this example is a lot 

19    simpler than working with the census.

20               Suppose that for some reason that we will not go 

21    into I have decided that I want to know the average height 

22    of lawyers practicing in New York City and I have two ways 

23    of doing this, and for this purpose my loss function is 

24    simply how well am I measuring that single number.

25               I have two ways of going about this.
                                                              939

 1               Way number one:

 2               I send out a team equipped with a measuring 

 3    device and they measure the height of every lawyer they can 

 4    find.  They measure it accurately, they measure --  they 

 5    actually find every lawyer with one exception, that is, that 

 6    the measuring device will not measure the height of anybody 

 7    over six feet tall.  The result of this is that they come 

 8    back with a complete enumeration of the heights of lawyers 

 9    in New York City except for the ones who happen to be over 

10    six feet tall.

11               That is version one., What I will call the biased 

12    measuring rod version.

13               Now I send out another team.  This other team is 

14    equipped with a better measuring device, they don't have 

15    that problem, but they also can't do a complete enumeration.  

16    What that team does is to take a sample of the sum number of 

17    lawyers, a random sample and reports back to me a statistic, 

18    they report the mean of the height of the lawyers in the 

19    sample and they also report the variance in the sample, that 

20    is a measure of the spread of the data in the sample around 

21    the mean.  High variance means that lawyers' heights are not 

22    tightly clustered around the mean, a low variances means it 

23    is closer to the mean.

24               I now decide I want to choose between these two 

25    ways of doing it.  I don't know why I want to know this, but 
                                                              940

 1    if I wanted to do this problem it probably would be possible 

 2    to combine the two things and that would be optimal, but for 

 3    simplicity suppose we have to choose the two estimators, the 

 4    bias measuring rod estimator and the sample based estimator.

 5         Q.    I take it from the first of those two you got one 

 6    measure and from the survey approach you got another answer?

 7         A.    Yes.  If I didn't get --  if I got the same 

 8    answer, there wouldn't be any point in choosing, so the 

 9    example is only interesting if there is a different answer.

10               As a matter of fact, in this particular example, 

11    the example is only interesting if the survey or the sample 

12    method comes back with a higher number than the biased 

13    measuring rod.

14         Q.    Which you would and anticipate?

15         A.    Yes.

16         Q.    The way you rigged the example?

17         A.    That's right.

18               Now, the sampling method has provided me with a 

19    certain amount of information.  It has provided me, first, 

20    with an estimate of how biased the biased measuring rod 

21    method is.

22               The sampling method is an unbiased measure, the 

23    mean of the sample is an unbiased sample of the mean of the 

24    height of the population of New York lawyers unbiased in the 

25    sense that if I went through that procedure many, many times 
                                                              941

 1    on average what I got would be the correct height.

 2               The difference between that and the biased 

 3    measuring rod version is a measure of the inaccuracy of the 

 4    biased measure rod version, but, of course, it's not a 

 5    perfect measure of that because, as we got into this, 

 6    because I didn't know the truth.

 7               So I know that if I used the biased measuring rod 

 8    version my expected error will be the difference between 

 9    that and the sampling --  sample based measure.

10               On the other hand, if I used the sample based 

11    measure --  now we come to in some sense the crucial point 

12    --  I know I'm uncertain as to how good that is, but I can 

13    make some calculation as to how far off I am likely to be, 

14    because given that I also know the variance of the heights 

15    of the lawyers who were included in the sample, the spread 

16    around the mean, statistical theory tells me that I can 

17    calculate as it were the variance which the sample mean 

18    itself will have, that is, the variance which this procedure 

19    generates if I do lots of different samples how disbursed 

20    they would be around the true mean.  That gives me a measure 

21    on average of the --  that gives me --  there are too many 

22    averages.  That gives me a measure of the expected squared 

23    error that I will make if I use the sample method.

24               If that is bigger than the square of the error 

25    which I have already estimated in using the biased measuring 
                                                              942

 1    rod method, then I use the biased measuring rod method.  If 

 2    it is smaller, then in this example I should use the sample 

 3    based methods.

 4               Now, the point here is that I generated in this 

 5    example without knowing the truth both an estimate, the 

 6    sample based estimate of the things I want to know and an 

 7    estimate of how accurate that method is expected to be, and 

 8    using that I can decide between that and using a wholly 

 9    different method.

10               As it happens, there is no reason to suppose, you 

11    can't tell from this example and you are not intended to be 

12    able to tell from this example whether it will turn out that 

13    I should use the sample based method or the biased measuring 

14    rod method, that depends on the data.  

15               The fact that I have generated the measures of 

16    expected inaccuracy from the sample does not tip the scales 

17    in terms of using the sample method itself.  That result 

18    depends on how diffuse the sample was that I happened to 

19    get.

20         Q.    Let me understand this, though.

21               You are measuring, as I understood it, the bias 

22    in the biased measuring rod example or figure based on the 

23    sample figure?

24         A.    That is true.

25         Q.    And you say that doesn't tip the scales in this 
                                                              943

 1    horse race in favor of the sample approach?

 2         A.    It does not.  It does not, because the sample 

 3    method also generates a measure of the expected inaccuracy 

 4    with which I am measuring the bias.  That is another way to 

 5    put it.  I am going to take that into account.

 6         Q.    Let me ask you, is it important to know in the 

 7    example you give what proportion of New York lawyers were 

 8    included in your sample survey?

 9         A.    No, it's irrelevant.  It's only  --  statistical 

10    theory, to the surprise of most nonstatisticians, tells us 

11    that in examples like that, the accuracy of the measurement 

12    depends not on what proportion of the population you have 

13    sampled, but on the actual size of the population.

14               The answer to that question, if the data are the 

15    same, will be the same if you have a thousand lawyers in the 

16    sample regardless whether there are 10,000 or 100,000 or a 

17    million lawyers in New York.

18         Q.    That is, it depends on the size of the sample?

19         A.    It depends on the size of the sample and not how 

20    big the sample is relative to the universe.

21         Q.    Is that a disputed proposition among 

22    statisticians?

23         A.    No.  

24               THE COURT:  I don't understand it.  Take me 

25    through it again. 
                                                              944

 1               THE WITNESS:  Okay.

 2               It is a remarkable fact --  

 3               THE COURT:  Let's talk 100,000 lawyers in the 

 4    State of New York and you want to conduct your survey, take 

 5    me through it. 

 6               THE WITNESS:  All right.

 7               I go out and I collect a sample using appropriate 

 8    random devices, I collect a sample of, let us say, 200 

 9    lawyers and I take their average height and --  

10               MR. RIFKIND:  Is he clear he is addressing the 

11    thing you don't understand, your Honor?  

12               THE COURT:  Yes.  

13               MR. RIFKIND:  Or is it just the last proposition? 

14               THE WITNESS:  I assume it was the last 

15    proposition.

16               THE COURT:  Yes.  That's what he is doing. 

17               THE WITNESS:  I'm getting to that.  

18               MR. RIFKIND:  I'm sorry.  I guess I didn't 

19    understand. 

20               THE WITNESS:  I collect their average height.  I 

21    also measure the variance in the sample around that height, 

22    and now the question is, how inaccurate will the average 

23    height of the sample be as a measure of, on average, as a 

24    measure of the true average height of the population, and 

25    the quite remarkable fact is that the number you --  you 
                                                              945

 1    need two numbers to know that:

 2               You need to know the variance in the sample 

 3    itself and you also need to know how many people were in the 

 4    sample.

 5               I think, what did I say --  

 6               THE COURT:  200. 

 7               THE WITNESS:  200.  Thank you.

 8               It does not matter how many lawyers there are in 

 9    the State of New York, it does not matter whether there are 

10    100,000 lawyers or a million lawyers that could have been in 

11    the sample, statistical theories, in this case sometimes 

12    called a piece of law of large numbers, tells us that the 

13    variance of the sample mean around the true mean depends 

14    only on the variance of the sample inside the sample itself 

15    and on the actual number of people in the sample, it does 

16    not depend on the fraction of the population being sampled.  

17               THE COURT:  The linchpin of the whole exercise, 

18    then, is calculating the variance? 

19               THE WITNESS:  You need to calculate the variance 

20    inside the sample in that example.  

21               THE COURT:  Can you do that? 

22               THE WITNESS:  Oh, but that is easy, only because 

23    I picked an easy example.  

24               THE COURT:  Shoot 199 lawyers. 

25               THE WITNESS:  Well, that would reduce the 
                                                              946

 1    variance by quite a lot and it might have other undersirable 

 2    statistical properties, although however desirable the 

 3    results might be in other respects.

 4               But the way to think about the variance inside 

 5    the sample --  there are easier calculations --  is you take 

 6    the height of each lawyer, you subtract the mean of the 

 7    sample from that height, you square it and then you average 

 8    that over the 200 lawyers.  It's the average squared 

 9    deviation of the individual lawyers' height from the sample 

10    mean.

11               I'm sure, I'm sure you are not going to care 

12    about this, but actually, you would not divide that sum of 

13    squares by 200, you would divide it by 199.  I hope you 

14    won't ask me why.  

15               THE COURT:  I won't. 

16               I am satisfied.  I'm ready to teach statistics 

17    now.  

18               MR. RIFKIND:  Okay, your Honor.  

19    BY MR. RIFKIND:

20         Q.    Now could you try and convert this example about 

21    lawyers' heights to the subject of the census?

22         A.    I could try.

23               We have two proposed measures of the population.  

24    We have the census which we know to be biased in a sense, we 

25    know it produces undercount.  Of course, if we know exactly 
                                                              947

 1    what the undercounts were with per situation, we would make 

 2    the correction, but then we would know the truth.

 3               We also have a method which is, in part, a 

 4    sampling method, it begins with the PES and combines the PES 

 5    with the results of the census, it just does not go off with 

 6    the example itself, it combines the results with the census 

 7    to create an estimated adjustment.

 8               If this matter were as simple as the height of 

 9    lawyers example, this would be an easy shot, but it isn't.   

10    Partly it isn't because the way in which we measure 

11    inaccuracy isn't simply the question of the distance of one 

12    number from another, this has to do with what loss function 

13    one is using.

14               Secondly, it isn't because the statistical 

15    properties of the adjusted population estimator are not 

16    nearly as simple as the statistical properties of the random 

17    sample of lawyers' heights.  Indeed, I choose the example of 

18    lawyers' heights that would have very easy statistical 

19    province.

20               What the Census Bureau did here was to go through 

21    something of the same exercised.  They developed what is 

22    known as the total error model and did some other things as 

23    well designed so that they could make the calculation of 

24    whether or not --  whether the expected accuracy of the 

25    adjustment, adjusted estimates was better or worse than that 
                                                              948

 1    of the census.

 2         Q.    Just so I understand the analogy here, the biased 

 3    measuring rod enumeration in the lawyers' case in census 

 4    terms --

 5         A.    That's the original enumeration.

 6         Q.    And the random sample is the PES?

 7         A.    Yes, but the analogy doesn't hold quite exactly, 

 8    because the adjusted estimates are not merely the results --  

 9    I mean, the census are not merely the results of random 

10    sample, but combining the random sample with the original 

11    enumeration.

12         Q.    Let me ask you, in the lawyers' heights analogy, 

13    the random error is in the survey but not what you described 

14    as bias in the survey?

15         A.    Yes.

16         Q.    In census terms, is there bias in the PES?

17         A.    Let me put it differently:

18               There is bias in the adjusted estimates and there 

19    is bias in the PES and the census has to make an attempt or 

20    did make an attempt to account for that bias in its loss 

21    function analysis.

22         Q.    And how did they do that?

23         A.    Well, they did both analytic and simulating 

24    calculations.

25         Q.    Is this an area that you have studied?
                                                              949

 1         A.    I have not studied with any specificity how the 

 2    total error model and these estimates of bias are, in fact, 

 3    constructed.  I, as it were, take this over at the point at 

 4    which they have done that and are using the results in the 

 5    loss function analysis.  

 6               MR. RIFKIND:  Your Honor will recall that Dr. 

 7    Wolter and others spoke about the total error model.  I 

 8    gather what I am being told at the time has been past that 

 9    point.

10         Q.    Having done these calculations you described, 

11    what emerged from the Bureau's analysis?

12         A.    From the loss function analysis?

13         Q.    Yes.

14         A.    The Bureau, the Bureau's loss function analysis 

15    showed that for each of the three loss functions that they 

16    did, that is, for each of the measures of inaccuracy 

17    adjustment, one adjustment was less inaccurate than the 

18    census.

19         Q.    Each of the three being?

20         A.    The first two are both in terms of distributive 

21    shares: one of them uses the sum of squared errors, one of 

22    them uses the sum of absolute errors, and the third looked 

23    at house apportionment seats, the apportion of house seats.  

24    Sorry.  

25         Q.    At this time, Professor Fisher, let me call your 
                                                              950

 1    attention to Plaintiff's Exhibit 43.

 2               Are you familiar with that exhibit?

 3         A.    Yes, I am.

 4         Q.    Could you tell us what it is?

 5         A.    This is a memorandum dated June 27, 1991, it's a 

 6    memorandum for the undercount steering committee from John 

 7    H. Thompson, chief statistical support division, subject, 

 8    "Updated loss function analysis for states and places." 

 9               It presents the results of what the Bureau was 

10    calling, presumably still calls, it's undated loss function 

11    analysis as distinguished from two other versions.

12         Q.    Does it give the results of that loss function 

13    analysis as of June 27?

14         A.    It does.

15         Q.    And what does Mr. Thompson's report shows?

16         A.    Well, it showed several things.

17               If you will take a look at table 1, which is 

18    Bates stamp number after the zeros 11293, it's the first 

19    table with lots of numbers, down at the bottom in the lower 

20    right-hand corner you will see a positive number which 

21    begins with several zeros and then 653.  That is, as it 

22    were, the bottom line of this.  

23               That is the difference between census lost and 

24    adjusted lost using total difference, using the squared 

25    error loss function, and the fact that it is positive shows 
                                                              951

 1    that the census loss is greater.

 2               The census loss, indeed, as Mr. Thompson pointed 

 3    out is, in fact, substantially greater at the national 

 4    level, Mr. Thompson says at page 2, he says at the U.S. 

 5    level the ratio of the census squared error loss to the 

 6    average adjusted squared error loss is about ten to one.

 7               That is, in fact, known to be an underestimate of 

 8    what the ratio really is.

 9         Q.    That says that the adjustment is a lot better 

10    than the census?

11         A.    Yes.

12         Q.    Incidentally, I see on top of table 1 appear the 

13    words right at the upper left-hand corner, Dr. Fisher, the 

14    word PRODSE, P R O D S E, in all caps.

15               Can you tell me what that reference to?

16         A.    PRODSE is an acronym.  It refers --  the census 

17    had to make, among other decisions, a decision as to how 

18    certain estimated biases in the undercount estimates should 

19    be distributed over the post-strata.  It used two methods 

20    for that.

21               Basically, it had estimated the shall biases for 

22    what it called I think evaluation post-strata, these are big 

23    aggregates of the 1,392 post-strata, and having gotten those 

24    estimates had to decide how to distribute the bias down over 

25    the post-strata which made up the evaluation post-strata. 
                                                              952

 1               It shows two rather different ways of doing this.  

 2    One way was to say a post-stratum which has a lot of people 

 3    also has a lot of bias.  The bias is proportionate --  the 

 4    bias is proportional to the number of people in the 

 5    post-stratum as estimated by the PES.

 6               That is an acronym which we will see on some of 

 7    these documents known as --  I cannot --  this is an 

 8    unpronounceable word, it's UNWEPRS, U N W E P R S.  

 9    Sometimes the W is left out.

10               The other version is known as PRODSE, and stands 

11    for Production Dual Systems Estimator, and there is assumed 

12    that the bias, the contribution to bias of each post-strata 

13    that make up an an evaluation post-stratum is proportional 

14    to the undercount.  

15               In fact, every undercount within an evaluation 

16    post-stratum has the same percentage bias.

17         Q.    We have looked at a table that is headed 

18    production DSE or PRODSE.

19               Did they also study the unweighted person's 

20    version of this?

21         A.    They did.  I don't know of any case in which they 

22    made a material difference which they used.

23         Q.    Are the results substantially the same on both 

24    approaches?

25         A.    Yes.
                                                              953

 1         Q.    While we are looking at table 1, is it correct to 

 2    observe, Professor Fisher, looking down to the last entry 

 3    before total where it says California, then looking over to 

 4    the right-hand column under --  the column headed 6, that 

 5    given the number of zeros and so on the number for 

 6    California seems rather larger than the rest?  

 7               THE COURT:  The number what?  

 8               MR. RIFKIND:  The number given for California on 

 9    table 1 in the lower right-hand corner just above the total 

10    line is a number that says 000004816617.

11         Q.    That seems larger than --  

12               THE COURT:  Most of the other numbers in that 

13    column?

14         Q.    Most of the other numbers and substantially so.

15         A.    It is substantially larger than all the other 

16    numbers in that column.

17               By the way, can I say something about whether 

18    these numbers are big or small? 

19               There are all these zeros that make them 

20    troublesome to read.

21         Q.    I will come back to that for just a minute, I 

22    want to deal with California first.

23               Does California's --  is California running away 

24    with this?  Should we have deleted California from this list 

25    in order to get a fair assessment of it?
                                                              954

 1         A.    Of course not.

 2               What you are looking at here is a squared error 

 3    loss function which means that you have decided that big 

 4    errors count a lot.  California has a great big error.

 5               Implicitly, explicitly, when you use a squared 

 6    error loss function, say you are going to add the sum of the 

 7    squares, you are saying that that's the way I'm going to 

 8    trade off big errors for small errors and errors in one 

 9    state against errors in another state.

10               When you have done that, to observer that one 

11    state, in this case California, contributes most of the 

12    difference in loss measured this way is a true statement, 

13    but is not a reason for deciding that you don't want to use 

14    these results.

15               It might be a reason to decide, gee, I don't 

16    really like the criterion I chose because it didn't come out 

17    the way I thought it would come out, but that is the 

18    criterion you used.  

19               You could adopt another criteria, which is one 

20    thing you could do is to use the absolute error loss 

21    function which, if you do, you will find something of the 

22    same phenomenon but it won't be nearly so pronounced, but 

23    basically choosing the loss function you are going to 

24    measure already decides, as I said before, how you are going 

25    to weight big errors against small errors, how you are going 
                                                              955

 1    to weight errors in one state and another and you decided 

 2    you are going to put them this way and add them up.

 3               What you do, what you are interested in is the 

 4    total.

 5         Q.    And that is the figure given in the lower 

 6    right-hand corner?

 7         A.    That is the figure in the lower right-hand 

 8    corner.

 9         Q.    You called my attention just a moment ago to the 

10    fact that at least to the lay eye a figure which begins 

11    .000065, et cetera, looks like a very small number, and I 

12    want to ask you, does that mean that after all this sweat 

13    and bother, the corrections is a very small affair?

14         A.    No, it doesn't mean that for two reasons:

15               One reason it doesn't have to do with the way 

16    these numbers are measured is that we know the difference 

17    between the adjusted and the census estimates makes a 

18    material difference.  This is, however, we are talking here 

19    about a measure of a difference in accuracy.

20               Second and more important, the fact that there 

21    are this inconvenient number of zeros, inconvenient because 

22    you and the reporter just discovered it's hard to keep tract 

23    of exactly how many there are, the fact that there are a 

24    large number of zeros is an artifact in the way in which 

25    loss is being measured.
                                                              956

 1               Let me give you an example.

 2               This is being done first in terms of distributive 

 3    accuracy.  That means loss is being measured in terms of 

 4    errors in fractions where the fractions are all less than 

 5    one.  Then the error is being squared.

 6               To take an example, I'm going to ignore the 

 7    District of Columbia in this example.  There are 50 states 

 8    and on average each of the 50 states has two percent of the 

 9    nation's population.  That's.02.

10               Suppose that you made an error in a particular 

11    state's population of four percent.  That would be, in terms 

12    of its distributive share, that would come out to four 

13    percent of .02, and that would be an error of the form of 

14    .00 --  I hope I get this right -- .0008.

15               You are now going to square that.  When you 

16    square .00 --  when you square point three zeros 8, you are 

17    going to get a number whose first digit is a 6 with a lot of 

18    zeros in front of it, I'm really not going to try to do it 

19    on the stand, I'm under oath, it has a lot of zeros, and 

20    even when you add that over 50 states you are going to get a 

21    number on the order of magnitude of what you look at here.

22               To take a different example, if you had chosen to 

23    measure accuracy in terms of numeric accuracy and used the 

24    squared error loss function so that you start by measuring 

25    it in terms of error of numbers of people and adding it up, 
                                                              957

 1    you can quite easily get up to numbers of the hundreds of 

 2    billions right away, but that is also an artifact of the 

 3    units you have chosen to do this.

 4         Q.    Perhaps you could look for a moment at table 2, 

 5    PX 43.

 6               What does that show us?

 7         A.    Table 2 does the same sort of thing for 

 8    geographic areas other than states.

 9               At the top of table 2, at the right, you have the 

10    bottom line answer for counties of size less than 200,000 

11    and for counties of size more than 200,000 and, again, it 

12    turns out that census has a greater loss than adjustment.

13               These are the two figures in column 6, the two 

14    top figures.

15         Q.    Right.

16         A.    That is then done again at various other places 

17    in table 2 for metro areas of various sorts, and it is true 

18    for all of them that the census is less accurate than 

19    adjustment.

20               On the next page it's done for nonmetro areas, 

21    and with one exception, which I guess we will get to, the 

22    census turns out to be less accurate than the adjustment.

23               And then it is done for metro areas, and so on.

24         Q.    Well, I guess we have gotten to the one 

25    exception.
                                                              958

 1               What is that, Doctor?

 2         A.    The one exception is nonmetro areas located in 

 3    places of 50,000 or more.

 4         Q.    Can you tell from this how many people live in 

 5    such nonmetropolitan areas in places of 50,000 or more?

 6         A.    Well, as I told you before, if we knew the truth 

 7    we wouldn't be here, but I know a good approximation.

 8         Q.    A good approximation, yes.

 9         A.    Yes, I can.

10               Approximately 60,000 people.  There were five 

11    such places in the United States.

12         Q.    And their aggregate population?

13         A.    Their aggregate population is 60,000.

14         Q.    It says nonmetropolitan areas in places of 50,000 

15    or more.  You would think if there are five of them there 

16    would be at least 250,000 of them?

17         A.    Well, you might wonder about that.

18               In fact, it's not that there are nonmetro areas 

19    of 50,000 or more in these places.  If these are things 

20    defined as nonmetro areas which happen to be located in 

21    places of 50,000 or more, to take an example which I know is 

22    not a true now but would have been a true one years ago, 

23    farms in the middle of Brooklyn might be included in this 

24    sort of thing.

25         Q.    Okay.  And that is on this Plaintiff's Exhibit 43 
                                                              959

 1    the only exception in the proposition that the adjustment 

 2    comes out ahead?

 3         A.    Yes.  And even that exception gets reversed later 

 4    on.

 5         Q.    In some subsequent study by the Bureau?

 6         A.    Yes.

 7         Q.    Looking also at table 2 on Bates stamp 11294, I 

 8    see an entry at the top half of the page for Los Angeles 

 9    County.

10               Do you see that, Dr. Fisher?

11         A.    I do.

12         Q.    That also seems to be quite large relative to the 

13    other entries in that column, does it not?

14         A.    It is.

15         Q.    And I guess I should put again the question I put 

16    to you with respect to California:

17               Does that suggest that that is sort of pulling 

18    everybody else along and ought to be thrown out?

19         A.    Well, it certainly --

20         Q.    Or skewing the results?

21         A.    Certainly not skewing the results.

22               It's certainly true that loss Los Angeles 

23    contributing, by this particular tabulation, is contributing 

24    by far the biggest piece of the loss.  You have chosen a 

25    loss function in which you said you are going to add the 
                                                              960

 1    losses across county and you are going to measure the 

 2    counties and you are going to measure the losses by squared, 

 3    and what you discovered here is that Los Angeles is being 

 4    counted by the census very, very badly, and that matters.  

 5    That is not a reason to not adjust the census and say we 

 6    won't worry about that; Los Angeles County, the last time I 

 7    looked, was part of the United States.  

 8               MR. RIFKIND:  Your Honor, I offer at this time 

 9    Plaintiff's Exhibit 43.  

10               MR. SITCOV:  No objection.  

11               THE COURT:  43 is admitted.

12               Would this be an appropriate point to take a 

13    break?  

14               MR. RIFKIND:  Fine, your Honor.  

15               THE COURT:  Let's take a recess until 11:00

16               (Recess) 

17    

18               (Continued on the next page.)

19    

20    

21    

22    

23    

24    

25    
                                                              961

 1         Q.    Let me go back just one step, Professor Fisher.  

 2    You were describing to us some moments ago two 

 3    unpronounceable terms called PRODSE and UNWEPRS, unweighted 

 4    persons.  You indicated that the bureau had carried out 

 5    analyses under both of those methods of bias allocation and 

 6    that they came out not sharply different.  

 7               Are there other methods of bias allocation that 

 8    they should be or did use?

 9         A.    The human mind can invent other methods of bias 
                                                                 

10    allocation, and you have to do something about allocating 

11    the bias.  You have to use something.  These are fairly 

12    different ones.  I find it hard to imagine a plausible 

13    method of bias allocation that doesn't lie somewhere between 

14    these extremes, and the fact that it doesn't seem to matter 

15    which you do suggests that it doesn't matter much what 

16    method of bias allocation you use. 

17         Q.    Does that difference indicate robustness of the 

18    data?

19         A.    Yes, at least within the limits represented by 

20    those two methods.  The results are robust for what method 

21    you use.

22         Q.    Let me call your attention to Plaintiff's Exhibit 

23    712 in the second volume before you.  I ask you whether that 

24    is -- is that a fair summary of what you have been saying 

25    about Mr. Thompson's memorandum set forth in Plaintiff's 
                                                              962

 1    Exhibit 43?  

 2               MR. SITCOV:  Your Honor, I am going to object to 

 3    any questions on this chart, for a number of reasons, one 

 4    being this is a chart we did not receive until Friday at 

 5    about three in the afternoon.  It had never been produced to 

 6    us before.  Along with exhibits in this binder that begin at 

 7    707, we had not received any of them in advance. 

 8               In addition, during Dr. Fisher's deposition, I 

 9    asked him if he had prepared anything written.  He said he 

10    had not and he did not expect to.  I asked if any expert for 

11    the plaintiffs intended to prepare anything in writing in 

12    connection with his testimony, and he was instructed not to 

13    respond.  

14               It would have been impossible for us ever to have 

15    known that documents such as this would ever exist.  We have 

16    essentially been precluded from learning about any of this.  

17    We didn't even see these things until Friday.  We certainly 

18    have not had an opportunity to question him about it in 

19    advance, to consider them at all.  

20               I think we have actually been blindsided on 

21    purpose.  

22               THE COURT:  Mr. Rifkind?  

23               MR. RIFKIND:  Your Honor, this chart that is 712 

24    is something, frankly, I had originally supposd the 

25    plaintiff would write on the blackboard as he spoke.  Then 
                                                              963

 1    it seemed to me it would move along faster if it was written 

 2    out in advance.  It is entirely and exclusively a summary of 

 3    what he has already testified to, put forth in form easy to 

 4    refer to for future reference.  

 5               THE COURT:  So it is a reprise of the testimony 

 6    he has already given?  

 7               MR. RIFKIND:  Exactly so.  

 8               THE COURT:  One page?  

 9               MR. RIFKIND:  One page.  

10               THE COURT:  I will overrule the objection, taking 

11    it just for illustration purposes only.  

12               MR. RIFKIND:  Exactly.  I was not planning to put 

13    it in evidence.  It is the blackboard for this question.  

14               THE COURT:  All right.  

15               MR. RIFKIND:  Thank you, sir.  

16    BY MR. RIFKIND:

17         Q.    What does Exhibit 712 show?

18         A.    Exhibit 712, as it were, extracts various of the 

19    conclusions that can be read out of whatever exhibit it was 

20    we were talking about a few minutes ago, before the break.

21         Q.    Plaintiff's Exhibit 43?

22         A.    Yes.

23         Q.    The updated losses?

24         A.    Yes.  It reflects what are the calculations in 

25    Plaintiff's Exhibit 43 for which the loss function gives a 
                                                              964

 1    result.  

 2               The one at the top, for instance, is the main 

 3    one.  That is the one in which the loss function is 

 4    calculated by states.  The next item after, next column 

 5    after that, shows a census count of how many people live in 

 6    the states.  In this case it happens to be the census count 

 7    of how many people live in the United States.  

 8               Then the box marked adjusted is checked, 

 9    reflecting that that loss function leads to a conclusion 

10    that the adjustment is more accurate than the census.  

11               Similar things are done further down.

12         Q.    As you called our attention to earlier, the only 

13    box that is checked on the righthand side that is under the 

14    census is this category 50,000 or more, nonmetropolitan 

15    areas and places of?

16         A.    Places of 50,000 or more, yes.  A lot of the 

17    details, of course, of Plaintiff's Exhibit 43 is not on 

18    here, nor are some of the conclusions.  For example, the 

19    conclusion I referred to before, that at the state level the 

20    inaccuracy of the census is over ten times that of the 

21    adjusted count, is not reflected on this document.

22         Q.    That is, this loss function analysis shows that 

23    the census is ten times more inaccurate?

24         A.    Yes.  In terms of squared error loss function it 

25    does show that.  Actually, as I said before, that is a 
                                                              965

 1    slight underestimate -- I don't know if it is slight.  That 

 2    is an underestimate.

 3         Q.    So it is something more than ten times?

 4         A.    Yes.

 5         Q.    Thank you, Dr. Fisher.  

 6               Did the bureau do any loss function analysis 

 7    subsequent to Mr. Thompson's memorandum of June 27th?

 8         A.    It did.

 9         Q.    Let me call your attention to Plaintiff's Exhibit 

10    42.  Could you tell us what that is?

11         A.    This is a memorandum dated July 11, 1991.  It is 

12    a memorandum for John H. Thompson, chief statistical support 

13    division, from Dave Bateman, the chief of coverage studies 

14    and evaluation staff.  The subject is given as the final 

15    report for 1990 PES evaluation project P16: total error 

16    model - loss function evaluation.  In fact, what this gives 

17    is what is known, I think, in this proceeding as the final 

18    loss function estimates, although in fact there were some 

19    later versions somewhat after that.

20               MR. RIFKIND:  I just note for the record, your 

21    Honor, that the document indicates also that copies of this 

22    were sent to the Undercount Steering Committee, to Director 

23    Bryant, Deputy Director Concannon, and to two people at 

24    least in the Department of Commerce, Mr. Plant and Mr. 

25    Starr, who are in the cc column below that memorandum.
                                                              966

 1         Q.    What are the differences between this loss 

 2    function of July 11th and the updated loss function that we 

 3    discussed when you were looking at Plaintiff's Exhibit 43?

 4         A.    There are a couple of differences.  First, this 

 5    document contains material on the absolute error loss 

 6    function as well as on the squared error loss function.  

 7    Second, this document contains material on House 

 8    apportionment.  

 9               Third, while it also contains material on the 

10    squared error loss function, that differs from the material 

11    in Plaintiff's Exhibit 43 in relatively minor ways because 

12    Exhibit 42 reports computations of certain magnitudes which 

13    were done one way in Exhibit 43 and which had to be done a 

14    different way for the absolute error loss function, and for 

15    consistency the same computation is used in the squared 

16    error loss function.  That really doesn't matter very much.

17         Q.    You said the differences are not of statistical 

18    significance?

19         A.    I don't know whether they are of statistical 

20    significance, but they are not material.

21         Q.    What results from this final loss function as 

22    reflected in PX-42?

23         A.    Take a look --

24         Q.    There are a lot of Greek letters I see.  Can we 

25    pass over those?
                                                              967

 1         A.    I was planning on that.

 2         Q.    Thank you, sir.

 3         A.    Let's start with more or less where we have been 

 4    before.  If you will look back at Bates stamp 6090, you will 

 5    see on what is called table 6, you will see material for 

 6    both the squared error loss function and the absolute error 

 7    loss function.  The squared error loss function, the bottom 

 8    line appears in column 6 all the way at the bottom.  That 

 9    difference is .000007, again indicating that the census is 

10    less accurate than adjusted counts.  

11               The absolute error loss function bottom line 

12    appears in column 9 at the bottom.  The difference there is 

13    .00688, after the decimal point.  That again reflects a 

14    conclusion that the census is less accurate, the original 

15    enumeration is less accurate than the adjusted counts.  

16               Notice, incidentally, that the number of zeroes 

17    is considerable less.  That is because the effect of 

18    squaring has disappeared and you are measuring this in 

19    different units. I think that is basically what one wants to 

20    draw from that.  These also have been done using the PRODSE 

21    method.  

22               You can also find in the document a similar table 

23    using the UNWEPRS method.  That appears at Bates stamp 6087.  

24    The bottom line conclusions are substantially the same, in 

25    fact they are the same, and the numbers are not remarkably 
                                                              968

 1    different.  

 2               There is also an analysis of House error, error 

 3    in the misapportionment of House seats, and it is concluded 

 4    that the expected number of House seats which will be 

 5    misallocated if you use the census is two greater than you 

 6    would expect if you use the adjustment.

 7         Q.    You are looking at table 2 now?

 8         A.    I am looking at table 2, page 6086, or, for that 

 9    matter, table 1, page 6085.  The difference between them has 

10    to do with the UNWEPRS/PRODSE difference.  You can find some 

11    of these conclusions written up in the body of the text just 

12    before the tables.

13         Q.    Can you tell from this report anything about the 

14    magnitude of the difference in error between the corrected 

15    and uncorrected counts?

16         A.    I could try.  For the sake of consistency, I will 

17    continue with the PRODSE version, which is table 6, Bates 

18    stamp 6090, with loss measured as squared error.  The ratio 

19    of census loss to adjusted loss is somewhere between 10 and 

20    20.  I don't remember whether it is in the body of the 

21    document.  I can't figure out exactly what it is as we sit 

22    here, but it is considerably bigger than 10.  In terms of 

23    absolute error loss, the ratio is substantially less than 

24    that.  It is on the order of between 3 and 4.

25         Q.    But on either measure, there is a nontrivial 
                                                              969

 1    difference?

 2         A.    That is correct.

 3         Q.    Indeed, a substantial difference?  Would you 

 4    regard that as a substantial difference, Dr. Fisher?

 5         A.    Yes, I would regard it as a substantial 

 6    difference.  I know it is a statistically significant 

 7    difference, but I know that from other work.

 8         Q.    I guess what I am trying to figure out is, is 

 9    this a close call based on those data?

10         A.    No, I don't think it is a close call.  This is 

11    the way one has chosen to measure inaccuracy.  The census 

12    method is less accurate.  One way to correct for the units 

13    in which you have measured it, to correct for all the 

14    zeroes, is to take the ratios.  The census, in taking that 

15    measure, comes out to be lots more inaccurate.

16         Q.    You say this is the way chosen to measure the 

17    differences.  Is this an appropriate way to messure the 

18    difference?

19         A.    These are two different ways.  These are two 

20    obviously appropriate ways to do it.

21         Q.    Do you know whether this report was available to 

22    the secretary at the time it made its decision?  

23               MR. SITCOV:  Objection.  

24               THE COURT:  If you know, without speculating.

25         A.    The appearance of Bates stamps on the bottom 
                                                              970

 1    suggests it is in the administrative record.  

 2               THE COURT:  Does that answer the question?  

 3               MR. RIFKIND:  Yes, it does.

 4         Q.    Let me ask you this question, Dr. Fisher.  Are 

 5    you aware of any other studies didn't done by the Census 

 6    Bureau on loss function analysis?

 7         A.    Yes, I am.

 8         Q.    Do they point in any direction other than the 

 9    direction of these two we have looked at?

10         A.    They did not.

11         Q.    Have you seen any loss function analysis that did 

12    not favor the superior accuracy of the adjusted figures?

13         A.    I have not.

14         Q.    Does the Census Bureau's loss function analysis 

15    support the recommendation of the Undercount Steering 

16    Committee and the director of the bureau that the adjusted 

17    data are more accurate than the original enumeration?

18         A.    It does.

19         Q.    In your professional opinion, were those loss 

20    function analyses properly carried out by the bureau?

21         A.    They were.  The bureau spent a considerable time 

22    on this, and indeed prepared to do this well in advance.  A 

23    lot of thought and care has obviously gone into it.

24         Q.    You have referred to the fact that they both did 

25    a variety of loss functions and did them with a variety of, 
                                                              971

 1    at least two, modes of bias allocation.  Does that bear on 

 2    your view that this supports the recommendation of the 

 3    Undercount Steering Committee and the director?

 4         A.    Yes.  To count House apportionment, they chose 

 5    three versions.  They decided in advance they were going to 

 6    do those.  It is not, I suppose, impossible to think of 

 7    other versions, but these things were laid down in advance.  

 8    It was not a secret they were going to do it.  These were in 

 9    the category of the things sort of reasonable people would 

10    certainly set out to do.  And they did it and it comes out.  

11    So it doesn't matter which of these criteria you choose.  It 

12    comes out unambiguously one way.

13         Q.    Is there any respect in which the loss function 

14    analysis is biased or weighted in favor of one or the other 

15    outcome?

16         A.    No.

17         Q.    Could you say that it is conservative or radical 

18    in respect of one or the other outcome?

19         A.    Let me put it this way.  Given the loss functions 

20    that were chosen, there is no reason to believe that it is 

21    weighted one way or the other.  However, one could, in 

22    considering the loss functions that were chosen, consider 

23    the question of whether those loss functions really do, in 

24    fact, embrace the criteria through which one would want to 

25    have decisions made.  
                                                              972

 1               I should explain in what I am about to say, I am 

 2    not pretending to say what those criteria should be.  I am 

 3    trying to explicate what the consequences are of describing 

 4    the criteria for inaccuracy in different ways.  

 5               Both the absolute loss function and the squared 

 6    error loss function treat errors which are due to the 

 7    uncertainty of the adjustment process as if they can be 

 8    traded on a one-for-one basis with errors that are due to 

 9    the undercount.  Therefore, had these loss functions come 

10    out with a bottom line at zero, let's say, one would say 

11    that the census was equally accurate to the adjustment.  But 

12    there are reasons for believing that one might not want to 

13    say that, and therefore that these loss functions are 

14    conservative, conservative in favor of the census.

15         Q.    How is that?

16         A.    We know who is undercounted.  We know who has 

17    been undercounted for a considerable period of time 

18    historically over and over again.  It is minorities and, by 

19    extension, the people who live in places where there are 

20    large groups of minorities.  

21               In a way, what you are saying if you count errors 

22    due to that the same as errors due to the uncertainty of the 

23    adjustment, you are adopting a criterion that says someone 

24    is going to be miscounted.  In the one case I know who those 

25    someones are, the same as I said before, the same grounds 
                                                              973

 1    who have been miscounted for decades.  That's bad.  

 2               In the other case, let's say, suppose that there 

 3    was an even trade, someone is going to be miscounted, it is 

 4    uncertain, I want to average the amount of error.  The count 

 5    is going to be the same if I use the adjustment that is for 

 6    the census, but of course if I use the adjustment I will not 

 7    know in advance who is being miscounted.  

 8               It seems to me that one might reasonably say -- 

 9    one doesn't have to say but one might reasonably say -- that 

10    it is rather worse to adopt a method which ensures that 

11    certain definable groups will be undercounted and 

12    presumably, therefore, underrepresented, than it is to adopt 

13    a method which at the same level of miscounting you don't 

14    know who it is.  

15               Of course, if we could reduce the inaccuracy, 

16    everybody would want to do that.  But what I am suggesting 

17    here is that one might prefer to say that one man-one vote 

18    ought to mean that everybody has the same chance of being 

19    represented as opposed to knowing that certain groups will 

20    be underrepresented.  

21               We do, in fact, for some activities in our 

22    society, deal that way.  For some time we had, during the 

23    Vietnam War, a lottery in the military draft.  Some people 

24    would be drafted.  It seemed somehow fairer not to know who, 

25    as it were, in advance, than to pick a particular group.  We 
                                                              974

 1    suggest jurors by lot, and so on.  

 2               So there is at least some argument for saying 

 3    that the choice of these loss functions which do not count 

 4    errors -- which count errors of both types the same is a 

 5    choice conservative in the set of loss functions one might 

 6    wish to adopt, conservative in the favor of the census.  

 7               Of course, it is not true that the adoption of 

 8    those loss functions that the Census Bureau did adopt, it is 

 9    not true that those show that there is a tie.  There isn't a 

10    tie.  As we said before, there is a substantial conclusion 

11    in favor of the adjustment.

12         Q.    Is the essence of what you were saying before 

13    that the loss function assumes a sort of equality between 

14    bias and variance?

15         A.    You can put it that way, but I don't think that 

16    is a terrific way to put it.  I don't let anybody summarize 

17    my testimony, Mr. Rifkind, even --

18         Q.    So I am learning.

19         A.    The reason it is not a terrific way is that bias 

20    is used in more than one sense.  There is no issue about 

21    statistical bias versus variance involved here when you 

22    don't know which way the statistical bias goes.  It is bias 

23    here, but it is bias in the sense of the undercount, biased 

24    against particular groups and localities.  In that sense it 

25    is considered conservative, that that piece of bias is being 
                                                              975

 1    traded off for, shall we say, uncertainty.  Uncertainty was 

 2    measured both by variance and more generally by squared 

 3    error.

 4         Q.    When you were measuring the height of New York 

 5    lawyers earlier this morning, your surveyors using a 

 6    measuring rod that was only six feet long, I take it that 

 7    was biased against lawyers over six feet?

 8         A.    That was, and it is also biased statistically.

 9         Q.    And also biased statistically.  In that sense is 

10    the enumeration biased both in the traditional sense and 

11    biased statistically?

12         A.    It is certainly biased in the extraditional 

13    sense.  It is also biased statistically if you consider that 

14    to be a statistical measurement.  

15               MR. RIFKIND:  Your Honor, I offer PX-42 in 

16    evidence at this time.  

17               MR. SITCOV:  No objection.  

18               THE COURT:  42 is received.  

19               (Plaintiff's Exhibit 42 for identification was 

20    received in evidence)

21         Q.    Dr. Fisher, have you considered whether there is 

22    a relationship between undercount and minority status 

23    reflected in the data that the Census Bureau produced?

24         A.    I have.

25         Q.    Have you done any analysis of that subject?
                                                              976

 1         A.    I have.  Actually, it has been done for me under 

 2    my supervision.

 3         Q.    Let me call your attention to Plaintiff's Exhibit 

 4    693, of which a largish version is at the easel, your Honor.  

 5    The small one can be found in the second volume.  

 6               When you say it was done for you, Dr. Fisher -- 

 7               MR. SITCOV:  Sit your Honor, here I am again 

 8    going to make the same objection.  I would like to read from 

 9    the transcript of Dr. Fisher's deposition, if I could.

10              "Q.    Have you done any independent analysis of 

11    the 1990 census in PES?

12              "A.    Well, I've thought about the loss function, 

13    questioned at some length, and I've come to certain 

14    conclusions about it.  And in that sense it is the case that 

15    I've done independent analyses.  But those are analyses of 

16    the principles involved rather than of the independent 

17    analyses of the data themselves.

18              "Q.    Have you committed any of these analyses to 

19    writing?

20              "A.    No.

21              "Q.    Do you intend to commit any of them to 

22    writing before trial?

23              "A.    I hope not.

24              "Q.    Are you personally aware whether any other 

25    person who has been -- who you believe may be an expert for 
                                                              977

 1    plaintiffs in this case, if any such person has done any 

 2    independent analysis of the census or PES data?"  

 3               He was then directed not to respond to that 

 4    question.  

 5               It would have been impossible for us to know 

 6    whether this chart existed.  When we received plaintiff's 

 7    exhibits, they made a point of mixing them up, as you can 

 8    tell by the order of the numbers, so it would have been 

 9    impossible for us to tell at the time we received this 

10    document that Dr. Fisher would be testifying about it.  

11               MR. RIFKIND:  Your Honor, they have had -- 

12               THE COURT:  Hold on a second.  Is 693 in evidence 

13    already?  

14               MR. RIFKIND:  It is not.  

15               THE COURT:  It looks familiar, but I guess all 

16    these things look the same.  

17               MR. RIFKIND:  It isn't all that different from 

18    some others, but it is not.  

19               THE COURT:  Let me hear you.  

20               MR. RIFKIND:  They have had Plaintiff's Exhibit 

21    693 since the day it was due to be delivered, I think the 

22    1st of May, as well as the underlying sources for it, which 

23    are entirely Census Bureau materials.  

24               THE COURT:  What they didn't have was the 

25    knowledge that Dr. Fisher would be addressing this?  
                                                              978

 1               MR. RIFKIND:  That is exactly true, and they 

 2    didn't know that until the end of last week.  

 3               MR. SITCOV:  We also did not know that he would 

 4    be testifying about any analyses of data.  He said he hadn't 

 5    done any, had planned not to do any.  When I asked if 

 6    anybody had done any, he was directed not to respond.  

 7               MR. RIFKIND:  No.  

 8               THE WITNESS:  Can I say some something?  

 9               THE COURT:  Stay out of this.  

10               THE WITNESS:  That is not a correct 

11    characterization of what I said in the deposition.  

12               THE COURT:  Mr. Rifkind?  

13               MR. RIFKIND:  The witness reminds me it is not a 

14    correct characterization of what he said.  He was asked, and 

15    the reason there was the instruction was what he knew about 

16    what other witnesses who were going to testify might be 

17    saying, and under the rules that they had laid down in other 

18    depositions in which we had tried to acquire the same 

19    information, we said since they won't tell us about that, we 

20    won't tell you about that either.  

21               But this wasn't prepared by any other witness who 

22    is going to testify in this action.  This was prepared after 

23    the time of this deposition, hadn't been prepared when the 

24    deposition was taken.  It was prepared at my request.  And 

25    Dr. Fisher, I will examine him about this in a minute, 
                                                              979

 1    understands how that material was prepared.  

 2               THE COURT:  Who prepared this?

 3         Q.    Dr. Fisher, who prepared these materials?

 4         A.    This was prepared by members of the staff of 

 5    Charles River Associates, a consulting firm with which I 

 6    have a serious and longstanding connection.  

 7               THE COURT:  At Mr. Rifkind's request?  

 8               MR. RIFKIND:  Exactly so.  

 9               THE COURT:  I see.  

10               MR. SITCOV:  Your Honor, two things in response 

11    to that.  They obviously waited until after I had deposed 

12    him, as Mr. Rifkind just said, to have this prepared, so 

13    there would have been no possible way for me to learn about 

14    it.  

15               Secondly, I did not only ask him if he was aware 

16    of things that had been prepared by other witnesses, I said 

17    other experts.  

18               I also asked, after he had been directed not to 

19    respond to the question prepared by other experts, whether 

20    he had seen anything that had been prepared by other 

21    witnesses, and he was directed not to respond to that as 

22    well.  You will see that on page 16.  

23               THE COURT:  In any event, he had not then seen 

24    this?  

25               MR. SITCOV:  That's right.  They had obviously 
                                                              980

 1    orchestrated this so that I could never know about it, and 

 2    then they directed him not to respond to anything that might 

 3    even have led me to think that something like this may have 

 4    come into existence.  They gave us the documents in an order 

 5    that, I suppose, makes sense if you threw them up in the air 

 6    and tagged them as they came down.  I guess that is some 

 7    sort of random sample.  That's what we got.  

 8               There would have been no possible way for me to 

 9    relate this to Dr. Fisher in advance of Friday at three 

10    o'clock as I was racing to the airport.  I don't think that 

11    that has anything whatsoever to do with the rules of civil 

12    procedure that are designed to sort of level the playing 

13    field so that we have some idea of what we are supposed to 

14    be cross-examining this expert about.  

15               MR. RIFKIND:  May I add two little thoughts to 

16    this?  First of all, when all of the sturm and drang was 

17    done at the deposition, the witness actually answered it.  I 

18    don't know what counsel was instructing him at the time.  

19    But he actually said, "I don't know," which, since he was 

20    under oath, I believe to be true.  

21               MR. SITCOV:  He said that subject to the 

22    instruction that he was given that he could not respond 

23    about certain topics.  That was not just a blanket "I don't 

24    know."  

25               THE COURT:  I don't want to waste a great deal of 
                                                              981

 1    time on this.  

 2               MR. RIFKIND:  I might also say one other thing, 

 3    your Honor, because it keeps rearing its head.  We have 

 4    tried to give them the particular volumes of material 

 5    associated with each witness at the earliest probable date.  

 6    But the deal we had with them quite explicitly discussed on 

 7    the phone with your Honor's clerk at the time was that they 

 8    would in all events be available the day before the witness 

 9    testified.  That was the only deal we had with them.  In 

10    fact, we gave them last week this week's stuff.  

11               MR. SITCOV:  Your Honor, I received this at three 

12    o'clock on Friday.  It is true we had gotten a mountain of 

13    things from the plaintiffs.  But, as I said, there would 

14    have been no possible way in advance of Friday for me to 

15    know that this would be coming in through Dr. Fisher, and 

16    certainly on the basis of his testimony there would have 

17    been no way to know that.  

18               Plaintiffs never supplemented his testimony, as 

19    the rules require.  They never seasonably advised us that he 

20    would be testifying about data from the 1990 census.  In 

21    fact, he said he hadn't even reviewed it.  

22               THE COURT:  This is obviously another case where 

23    there is an inherent tension between the parties' duty to 

24    get at the truth in the case on the one hand and 

25    considerations of fairness to the other side on the other.  
                                                              982

 1    A balance has to be struck.  It is difficult for me to know 

 2    where justice lies in this particular case without hearing 

 3    the testimony.  

 4               Let me overrule the objection for the moment, 

 5    permit the examination to continue, in the hope that what 

 6    light is shed by this outweighs its unfairness to the 

 7    government.  I do sympathize with the government's dilemma 

 8    here.  If at the end of the use of this document the 

 9    government feels that they have been sandbagged 

10    prejudicially, I invite them to make a motion to strike, at 

11    which time I will be able to rule more intelligently.  

12               Go ahead, Mr. Rifkind.  

13               MR. RIFKIND:  Thank you, your Honor.  

14    BY MR. RIFKIND:

15         Q.    Dr. Fisher, what does Plaintiff's Exhibit 693 

16    show?

17         A.    We know that minorities, here defined as nonblack 

18    Hispanic and black, tend to be undercounted.  That shows up 

19    in -- Apart from the fact that it shows up over and over 

20    again in the history of the census, it shows up in the 

21    undercount in this census.  

22               What this exhibit does is to turn that into a 

23    relationship involving states and to ask whether it is true 

24    that states with large minorities also tend to be the states 

25    with large undercount.  
                                                              983

 1               On the horizontal axis of this diagram --

 2         Q.    If you would like to approach the chart, you may.  

 3    As you will.  

 4               THE COURT:  I have the chart in front of me.

 5         A.    I will get my exercise later.  

 6               THE COURT:  All right.

 7         A.    On the horizontal axis of this diagram is 

 8    measured the percent minority, minority being defined in the 

 9    way I described before -- the percent minority as counted by 

10    the census by state.  On the vertical access is the percent 

11    undercount as estimated.  Each box here is a state.  The 

12    District of Columbia does not appear on the diagram.  

13               There is a general positive tendency for the 

14    dots, showing that there is a tendency for states with 

15    higher percent minority to have a higher percentage 

16    undercount.  That is summarized by the regression line, 

17    which is the black line in the middle of the diagram and 

18    gives what the average tendency actually is.  

19               In the box to the lower right in the exhibit 

20    appears the equation for the regression line and some 

21    remarks about the test of significance.  Although, I will 

22    say about that that this diagram and this regression does 

23    not test whether minorities are undercounted.  That has to 

24    do with the estimates for the minority poststrata under- 

25    count.  What it tests is the proposition that given the 
                                                              984

 1    estimates of the undercount, that systematic relationship 

 2    between minority and undercount carries over into undercount 

 3    by state, which is obviously does.  

 4               THE COURT:  Why do I need a statistic to tell me 

 5    that?  Isn't that self-evidence?  If minorities are 

 6    undercounted, does it not follow as the night the day that 

 7    the states where most of them are will be undercounted?  

 8               THE WITNESS:  Your Honor, it does almost follow 

 9    as the night the day.  One certainly is not surprised by 

10    this result.  It might not quite follow because it could be 

11    the case that minorities also tend to live in states where 

12    there are groups that have been overcounted.  

13               THE COURT:  A washout.  I see.  

14               THE WITNESS:  That turns out to the to be true.  

15    That is what the diagram says.  

16               THE COURT:  But it could have happened, and that 

17    is why you did the chart?  

18               THE WITNESS:  Yes.  

19               THE COURT:  Thank you.

20         Q.    Does this analysis show in some sort of powerful 

21    and conclusive way that that is not true?  

22               THE COURT:  Sustained.  

23               MR. SITCOV:  Thank you, your Honor.

24         Q.    What can you say from point of view of statistics 

25    about the sort of demonstration that is being made on this 
                                                              985

 1    chart?

 2         A.    As I said, this does not test whether minorities 

 3    are undercounted.  But this does test the proposition that 

 4    given the estimates of the unit undercount, that carries 

 5    over into states, in the geographic areas, and on that basis 

 6    there is no doubt that it does.  It is a very tight fit.  

 7               THE COURT:  Powerful evidence.  

 8               THE WITNESS:  I thought you just sustained that 

 9    one.

10         Q.    You said tight fit.  Does that have some meaning 

11    in statistics?

12         A.    Yes.  It means that the standard error -- hear 

13    here it means in fact that the standard error with which you 

14    estimated the slope of the regression line is rather small 

15    relative to the slope.  The slope estimate is about 4.6 

16    times the standard error.

17         Q.    Is that significant, 4.6?

18         A.    In using statistical significance, it certainly 

19    is.  The probability that that result due to chance, as it 

20    says here, if in fact the true number is zero, the 

21    probability of a number that big, that you would get a 

22    number that big, is less than one in 200,000.  

23         Q.    Thank you.  

24               It says "including D.C."  Do you know whether it 

25    would look different if you included the District of 
                                                              986

 1    Columbia?

 2         A.    If you were to include the District of Columbia, 

 3    you would find essentially the same result.  As I recall, 

 4    you would find that, in fact, the relation was bigger, that 

 5    is, the slope of the line was a little bit steeper.

 6         Q.    Did you do similar studies or have similar 

 7    studies done with respect to minorities defined other than 

 8    as nonblack Hispanics and blacks?

 9         A.    Yes.  

10               MR. SITCOV:  Your Honor, before we move to 

11    another topic, I am going to renew my motion, if he is done 

12    with this, since that would be some topic other than the one 

13    addressed by this with document.  I am going to renew my 

14    motion that the document be excluded and that the testimony 

15    be struck.  

16               It is quite clear, as the witness just said, that 

17    this is powerful evidence in favor of adjustment.  If it is 

18    powerful evidence in favor of adjustment, we never knew 

19    about its existence.  Rule 26(e)(2) says that a party is 

20    under an obligation to seasonably -- and I don't want to get 

21    this incorrect -- to seasonably amend a prior response that 

22    was correct when made if it is no longer true and the 

23    circumstances are such that failure to amend is in substance 

24    a knowing concealment.  

25               That is clearly the case here.  There can't be 
                                                              987

 1    any doubt that Mr. Rifkind knew what the purpose was of 

 2    preparing this.  He knew it would be powerful evidence, and 

 3    he kept it from us without in any way amending Dr. Fisher's 

 4    answer that he didn't know of anything that would be written 

 5    by him.  We were not allowed to determine if anything would 

 6    be written for him, either by an expert retained by the 

 7    plaintiffs or an expert retained for testimony.  There would 

 8    have been no possible way for us to know about the existence 

 9    of this in advance of Friday at three o'clock.  

10               THE COURT:  As my previous comments clearly 

11    indicate, whether this is powerful or not I will take no 

12    position.  But it doesn't add anything that I haven't seen 

13    already.  It seems to me it is explicating what is fairly 

14    obvious, that if minorities are uncounted, then the states 

15    that have minorities are the ones that are getting zapped.  

16               MR. SITCOV:  It is more than that, your Honor.  

17    This shows a particular relationship with particular 

18    coefficients and a particular T statistic.  There would have 

19    been no way for us to determine whether or not the data were 

20    correct, whether or not the analyses were correctly done, so 

21    that the results that Dr. Fisher is testifying about are 

22    correct.  

23               THE COURT:  How would you go about determining 

24    all those things?  

25               MR. SITCOV:  I would have asked him exactly what 
                                                              988

 1    data set he used.  I would have asked him exactly how the 

 2    calculations were done.  I would have asked him how he 

 3    determined the standard errors.  

 4               THE COURT:  Why can't you do that when you cross?  

 5               MR. SITCOV:  I certainly can, but then all I will 

 6    do is know the information.  These are gigantic data sets.  

 7    It would be impossible for me to use that information to 

 8    cross-examine him.  I would be essentially undertaking a 

 9    deposition on the witness stand.  

10               These are not the kind of data sets, your Honor, 

11    that you can just take an plug into a computer, rub them, 

12    and you get an answer.  I am assuming that this was one that 

13    was done with many hundreds of reply cases and so forth in 

14    order to develop standard errors.  

15               MR. RIFKIND:  The witness is shaking his head in 

16    the negative.  

17               THE WITNESS:  Boy, is that wrong.  

18               THE COURT:  Let me again punt.  I will let you 

19    conduct your cross-examination.  If, after you have explored 

20    these things with Dr. Fisher, you still feel aggrieved, 

21    renew your motion at that point.  

22               MR. SITCOV:  Thank you, your Honor.  

23               MR. RIFKIND:  I would like to point out to your 

24    Honor that at the time we produced this chart on the 1st of 

25    May, we also produced with it Plaintiff's Exhibit 695, which 
                                                              989

 1    is the data from which this is calculated.  It is a one-page 

 2    sheet of data.  All of the data on PX 695 is data that we 

 3    had gradually extracted from the defendant's.  That is, it 

 4    is Census Bureau data, not data that we found or we derived.  

 5               Finally, to be perfectly clear, these exhibits 

 6    were prepared as the sunset in the last days of April.  We 

 7    gave them to them within a day or so of receiving it, as I 

 8    recall.  We were running around quite frantically trying to 

 9    get them together.  It just isn't so that we have been 

10    sitting on them secretly.  I wish I were so well organized, 

11    but I am not.  

12               THE COURT:  All right.  Continue.  

13               MR. RIFKIND:  I don't think I have offered 693 in 

14    evidence, and I do so now.  

15               MR. SITCOV:  We object.  

16               THE COURT:  The objection is overruled subject to 

17    the parameters that I indicated a few minutes ago that you 

18    can move to strike after your cross. 

19               (Plaintiff's Exhibit 693 for identification was 

20    received in evidence)

21         Q.    I don't want to repeat, but I think you said, 

22    Professor Fisher, that a similar analysis with respect to an 

23    differently defined set of minorities, a more broadly 

24    defined set of minorities, would include people in addition 

25    to those who are black and nonblack his spanics?
                                                              990

 1         A.    That's true.

 2         Q.    Was there any material difference in the outcome?

 3         A.    No.  It is substantially the same.

 4         Q.    You considered the matter also with the District 

 5    of Columbia included, and that came out?

 6         A.    We did it both with the District of Columbia 

 7    included and excluded, and it does not matter for the 

 8    conclusion.

 9         Q.    For the sake of clarity, you indicated that these 

10    had been prepared by Charles River Associates?

11         A.    That's true.

12         Q.    You reviewed them after they had been produced?

13         A.    I did.

14         Q.    Did Charles River Associates --

15         A.    I'm sorry.  After they had been --

16         Q.    Produced.

17         A.    Yes, I know.  I reviewed them after they had been 

18    created, before they were produced in the litigation sense.

19         Q.    Right.  So far as you know, did Charles River 

20    Associates do any other similar types of work in connection 

21    with this litigation?

22         A.    Yes, they did.

23         Q.    Do you know in general what they did?

24         A.    I do.

25         Q.    What was it?
                                                              991

 1         A.    At the request of Dr. Kane, they prepared -- 

 2               MR. SITCOV:  Objection, your Honor.  This is all 

 3    hear stay that is unrelated to his expert opinion.  He is 

 4    talking about what this other group did at the request of 

 5    yet another of plaintiff's experts.  I think it would be 

 6    best to have that expert testify about exactly what he 

 7    requested Charles River do.  

 8               THE COURT:  I am inclined to agree.  

 9               MR. RIFKIND:  I was trying to simplify rather 

10    than complicate life.  Dr. Kane is quite competent to 

11    testify to his own work.  But since the name Charles River 

12    Associates came up, I was only going to ask Dr. Fisher 

13    whether he reviewed those charts as well.  

14               THE COURT:  All right.  Did you review some other 

15    charts?

16         Q.    Did you review some other charts prepared by 

17    Charles River Associates for Dr. Kane?

18         A.    Yes, I did.

19         Q.    Thank you, sir.  

20               Dr. Fisher, have you read the Secretary's 

21    decision dated July 15, 1991, relating to the correction of 

22    the census?

23         A.    Yes, I have.

24         Q.    Have you read it with particular attention to the 

25    Secretary's discussion of loss function analysis?
                                                              992

 1         A.    Yes, I have.

 2         Q.    Let me call your attention to Plaintiff's Exhibit 

 3    9, which is the Secretary's decision, your Honor.  

 4               THE COURT:  Yes.

 5         Q.    Particularly to page 1-4.  

 6               THE COURT:  Did he read this whole thing on TV?  

 7    Did the Secretary read this whole Exhibit 9 on TV?  

 8               MR. RIFKIND:  That is a very good question.  

 9               MR. SITCOV:  I can answer it for you.  The answer 

10    is no.  

11               THE COURT:  I didn't recall it being that long.  

12               MR. SITCOV:  He had a short prepared statement 

13    that he read.  

14               MR. RIFKIND:  Do you mean did he read it out 

15    loud?  

16               THE COURT:  Yes.  

17               MR. RIFKIND:  I am interested also in the 

18    question but I may never know the answer whether he read it 

19    before he signed it at the end of the lengthy document.  

20    Indeed, I asked at the start whether he had read the 18,000 

21    pages of the administrative record, so-called.  But since we 

22    are not going to be apparently graced with his presence, we 

23    may never know the answer to that mystery.  

24               THE COURT:  We digress.  

25               MR. SITCOV:  Your Honor, I would appreciate 
                                                              993

 1    something less in the way of insulting comments about the 

 2    Secretary.  He is not here.  There has been no demonstration 

 3    of his having done anything untoward, notwithstanding 

 4    plaintiff's repeated attempts to do so.  I really wish Mr. 

 5    Rifkind would refrain from that sort of comment.  

 6               THE COURT:  You have both gotten your shots?  Now 

 7    you both feel better.  Let's move ahead.  

 8    BY MR. RIFKIND:

 9         Q.    Would you look, Professor Fisher, at page 1-4.

10         A.    All right.

11         Q.    Let me call your attention to the bottom 

12    paragraph of that page, where it says, and I read, "Based on 

13    the measurements so far completed, the Census Bureau 

14    estimated that the proportional share of about 29 states 

15    would be made more accurate and about 21 states would be 

16    made less accurate by adjustment."  We will stop there.  

17               Would you comment on that sense?

18         A.    Yes.  That sentence is the product of what I can 

19    only describe as a cascade of errors.  It is not true.

20         Q.    Why do you say that, sir?

21         A.    I am going to have to take you through the 

22    cascade, as it were.  The place to go, I think, is page 2-29 

23    of Exhibit 9, where this is discussed in some greater 

24    length.  

25               If you will look at page 2-29, down in the 
                                                              994

 1    paragraph that begins on that page down ten lines, beginning 

 2    at the end of the line.  "If only the variance measured in 

 3    the total error model is used, then the shares of an 

 4    estimated 21 states are made worse by adjustment (using an 

 5    absolute value loss function)."  Then there is a footnote.  

 6               The first thing to say is that that is not right.  

 7    Even accepting what the Secretary appears to be counting -- 

 8    and it will turn out, as I shall explain, that the Secretary 

 9    is not in fact counting what he appears to be counting -- 

10    but the count of 21 does not come from an absolute value 

11    loss function.  It comes from the squared error loss 

12    function.  The count from the absolute value loss function 

13    is 11.

14         Q.    So he is overstating by a factor of more than two 

15    the -- a factor of two the number that is made worse by 

16    adjustment here?

17         A.    A little less than two.

18         Q.    A little less than two, is that correct?

19         A.    That is correct.  Now put that aside.  Let me see 

20    if I can give you a road map of what I have to say about 

21    this paragraph.  Maybe we can do it in an organized way.  

22               First, the notion that one wants to count the 

23    number of states whose proportions will be made worse 

24    implies a loss function that I do not consider to be a 

25    rational choice of a loss function.  There are at least two 
                                                              995

 1    versions of what that loss function could possibly be, and 

 2    neither of them seems to me the way a rational person would 

 3    go about it.  

 4               Second, it is not true, even putting together -- 

 5    forgetting about the difference between the 21 and the 11, 

 6    looking at the place the 21 comes from -- it is not true 

 7    that 21 is the number of states which you expect have their 

 8    accuracy in terms of distributive shares disimproved by 

 9    adjustment, which is what this purports to be.  It is, in 

10    fact, something else.  

11               Third, it turns out to be the case, the rest of 

12    this paragraph goes on about what happens if you increase 

13    the variance of total error model, and it comes to the 

14    conclusion that the number of states worsen, it goes from 

15    the number of states which appear to be made worse off in 

16    the sense of having their accuracy lessened.  That goes from 

17    21 to 29.  It is not true, and the Secretary was told it was 

18    not true -- 

19               MR. SITCOV:  Objection, your Honor.  

20               THE COURT:  Sustained.  Strike the last remark.

21         A.    It was not true that the correct way to think 

22    about that is to compare that count to 25, half the number 

23    of states.  It is to compare it to a number quite different 

24    from that and rather higher.

25         Q.    Why don't we start with the first of those.  The 
                                                              996

 1    first, as I recall, was you said this is not an appropriate 

 2    loss function, not a rational loss function.

 3         A.    I did.

 4         Q.    Is that a fair statement?

 5         A.    Yes.

 6         Q.    Would you explain that to us.

 7         A.    There are two things you could mean, could be 

 8    doing, by counting the number of states whose proportions 

 9    get worse.  I always mean it is hard to talk about this 

10    without saying worse or better.  What I always mean is not 

11    that the proportions got bigger, but that they became less 

12    accurate.  

13               There are two ways -- and then to base a decision 

14    on, of course.  

15               There are two things you might mean.  The first 

16    is that I am interested in the number of states in which the 

17    probability is greater than a half that adjustment will make 

18    their share -- will disimprove the accuracy with which their 

19    share is measured.  The second is a somewhat different 

20    matter, and that is that I wish to know the expected number 

21    of states which will have their shares disimproved.  

22               I think it would be useful if I illustrated what 

23    the difference is.

24         Q.    Please do.

25         A.    I think I want my exercise, because experience 
                                                              997

 1    suggests I can't keep track of this without writing it where 

 2    you could see it.  

 3               MR. RIFKIND:  May the witness write on the white 

 4    board?  

 5               THE COURT:  Sure.

 6         Q.    Suppose that you had ten states in which the  

 7    probability that adjustment makes things bettr in terms of 

 8    improving the distributed shares, is approximately 1.  You 

 9    are quite sure that adjustment would improve the distributed 

10    shares for 10 states.  But that you had 40 states, I am 

11    going to ignore the District of Columbia in what follows 

12    simply because even numbers are easier to work with -- you 

13    had 40 states in which the probability that adjustment would 

14    make things better in terms of distributive shares was  

15    .49999, lots more 9s.  In that case there are 40 states in 

16    which it is more likely than not that adjustment will make 

17    things worse.  There are only 10 states in which it is more 

18    likely than not that adjustments will make things better.

19         Q.    In this hypothetical?

20         A.    In this hypothetical, yes.  These are not the 

21    facts.  

22               But, in fact, if you ask what will happen, the 

23    answer is on these figures the 10 states which you know are 

24    going to be made better will be made better, and you expect 

25    half of the 40 states for which the probability is very, 
                                                              998

 1    very close to a half, half of those will be made better and 

 2    half will be made worse, you just don't know which ones.  

 3               So the expected number of states made better is, 

 4    in fact, 10 plus 1/2 of 40, which is 10 plus 20, which is 

 5    30.  That is quite different from the number of states in 

 6    which the probability is greater than a half, that is, it is 

 7    more likely than not, that they will be made better off.  

 8               So these two different versions can be quite 

 9    different.  They may, in fact, not be much different in 

10    practice, but in principle they can be quite different.

11         Q.    Thank you.

12         A.    Now let me begin with whether these would be 

13    sensible loss functions.  Let's start with the one that 

14    takes the number of states in which it is more likely than 

15    not that they will be improved by adjustment.  Suppose you 

16    did that and you said, well, what I am really interested in 

17    is whether that is bigger than half the states.  

18               As this example suggests, you would make some 

19    very strange and I think quite irrational decisions.  Faced 

20    with this example, were those the facts, you would decide 

21    not to adjust on the basis that in more than half of the 

22    states it was more likely than not that adjustment would 

23    worsen things, even though in fact you are pretty sure that 

24    in approximately 30 states it will turn out that adjustment 

25    makes things better.  
                                                              999

 1               The problem there is that any probability greater 

 2    than a half, no matter how big, is being offset by any 

 3    probability less than a half, no matter how close to a half 

 4    it is.  I can't imagine that any rational person would want 

 5    to make the decision based on that.  

 6               It is a closer question, but I think it has the 

 7    same answer, whether one wants to make a decision based on 

 8    the expected number of states which will be improved versus 

 9    disimproved by adjustment.  Here is why.  This is actually 

10    my favorite example.  

11               Suppose that for some truly peculiar reason the 

12    census manages to count the population of New York at zero 

13    and it manages to count the population of Texas as its 

14    actual population plus the population of New York, and for 

15    all other states the census counts the population at very 

16    close to the actual population.  

17               Now we have an adjustment method.  The adjustment 

18    method, if applied across all the states, will approximately 

19    correct the count in Texas and New York.  However, for each 

20    of the other states which have been counted approximately 

21    correctly, there is some probability close to a half -- 

22    there isn't much of a correction to be made, you are not 

23    very sure of what the correction is, and with probability 

24    very close to a half for the 48 other states you will make 

25    things worse in terms of distributive share.  You won't make 
                                                              1000

 1    them a lot worse, but you will make them worse.  

 2               In that circumstance, the expected number of 

 3    states which will be made worse by adjustment is, in fact, 

 4    -- 24?

 5         Q.    48?

 6         A.    It is 24.  I'm sorry.  I need to change the 

 7    example a bit.  Suppose it were true that in order to -- I 

 8    want to get a number that comes out to be 25.  

 9               Suppose in order to do this you were virtually 

10    certain that you would make things worse in the other 48 

11    states, virtually certain, but you know you are not going to 

12    make them very much worse.  You are pretty sure that you 

13    have done something less accurate, but the change isn't very 

14    pig.  In that circumstance, looking at the expected number 

15    of states that will be made worse off, that expected number 

16    is 48.  The counts looks like 48 to 2 that you shouldn't 

17    adjust.  

18               But I cannot imagine in that hypothetical why any 

19    rational person wouldn't, in fact, adjust.  The problem with 

20    using the expected number of states which will be made 

21    better off or worse off by adjustment is that it pays no 

22    attention whatever to the question of how much better off or 

23    how much worse off.  It counts any error of totally trivial 

24    magnitude one way as offsetting any error of serious and 

25    substantial consequence the other way.  That doesn't seem to 
                                                              1001

 1    me to be a rational criterion.

 2         Q.    Is what we should learn from this, according to 

 3    you, Professor Fisher, that the Secretary chose a method 

 4    that implicit in the Secretary's statement here at 2-29, is 

 5    that he either didn't care or didn't take account of the 

 6    magnitude of the corrections or the probability of their 

 7    occurring?  

 8               MR. SITCOV:  If that question is finished, I am 

 9    going to object.  It is quite leading perhaps.  

10               MR. RIFKIND:  I don't know where it was leading, 

11    since I couldn't quite figure it out myself.  

12               THE COURT:  It is withdrawn.  

13               MR. RIFKIND:  It is withdrawn.

14         Q.    Having outlined these various possibilities, 

15    would you articulate once again perhaps what it is that the 

16    error, in summary what is the error that you see in the 

17    Secretary's choice of loss function?

18         A.    The loss functions involved in the one case -- 

19    well, in one case, the case in which you look at the number 

20    of states which are more likely than not to be improved, 

21    that ignores how much more likely than not it is and can 

22    lead you to circumstances in which you think there are a 

23    great number of states that will be tipping one way, whereas 

24    the number of states you expect to have something happen to 

25    is in fact greatly the other way.  
                                                              1002

 1               Secondly, both methods, and this is at least as 

 2    important, both versions of that loss function, a second one 

 3    being the expected number of states, ignore how much an 

 4    improvement occurs or does not occur, and, as I said before, 

 5    any disimprovement, no matter how slight, is allowed to 

 6    count the same as any improvement no matter how great.

 7         Q.    You then raised a second problem with the 

 8    Secretary's decision which had something to do with 21 is 

 9    not the number of states expected to be disimproved.

10         A.    21 is not, in fact, either of those things.  21, 

11    the count of 21, turns out to be the answer to a somewhat 

12    peculiar question.  I can state what it is in answer to, but 

13    I can't make it sound nonpeculiar because it is peculiar.

14         Q.    What is it the answer to?

15         A.    It is the answer to the question:  In how many 

16    states is it the case that the squared error committed if 

17    you use the census will be less than the squared error 

18    committed if you use the adjusted version, provided that you 

19    attribute to the adjusted version not the squared error that 

20    will actually be made in that state but the squared error 

21    which you would expect to be made on average.

22         Q.    You say that is not an appropriate or relevant 

23    thing to be asking from a statistical point of view?

24         A.    I can't imagine why any rational person would 

25    want to base a decision on the answer to that question.  I 
                                                              1003

 1    can imagine calculating the answer to that question because 

 2    it gives some hints about what the answer to questions like 

 3    the expected number of states is.  But the answer to that 

 4    question seems to me to be of no possible relevance whatever 

 5    in any direct way.

 6         Q.    Is it possible to calculate the number of 

 7    stateses expected to be made less accurate by adjustment?

 8         A.    It is possible.  As I told Mr. Sitcov at my 

 9    deposition, it was possible to do it, and indeed I thought 

10    quite a lot about it both at and after the deposition.

11         Q.    Have you done so or has it been done?

12         A.    It has.  

13               MR. SITCOV:  I object again, your Honor.  The 

14    witness has just said that he did some calculations after 

15    the deposition as a result of questions that I asked him in 

16    the deposition.  We never received those calculations in any 

17    way that would allow us to check them, in any way that would 

18    allow us to know that he had done them.  The plaintiffs 

19    certainly haven't seasonably amended any responses to 

20    questions that I asked.  

21               This is just another case where we have been 

22    sandbagged.  We can't mount any kind of defense to this.  

23               MR. RIFKIND:  Once again, much of the prior 

24    discussion carries over here.  We provided them with the 

25    material that had been prepared on May 1.  It was prepared 
                                                              1004

 1    within hours before it was given to them.  It was not until 

 2    the deposition that we considered the possibility of doing 

 3    it.  

 4               MR. SITCOV:  Perhaps Mr. Rifkind could tell me 

 5    how before Friday at three o'clock I would have known which 

 6    of these many documents in the dozens of volumes that they 

 7    had provided to us were the ones that included that and were 

 8    going to be used for Dr. Fisher's testimony.  

 9               MR. RIFKIND:  You knew that these were going to 

10    be used this week because you had them since May 1 and you 

11    had every reason to suppose that I was going to use them.  

12               MR. SITCOV:  How would I know that they were 

13    going to be used with Dr. Fisher?  

14               MR. RIFKIND:  What difference does it make 

15    whether they were going to be used with Dr. Fisher or Dr. 

16    Daly?  

17               THE COURT:  The objection is overruled.  

18               Are you going to finish your direct before lunch?  

19               MR. RIFKIND:  I would guess not, actually, sir.  

20               THE WITNESS:  I hope not.  

21               THE COURT:  Why?  

22               THE WITNESS:  Because I'm getting hungry.  

23               THE COURT:  We will break at 12:30, is that good?  

24               THE WITNESS:  That would be fine with me, your 

25    Honor.  
                                                              1005

 1               THE COURT:  He wants to eat at 12:30.  

 2    BY MR. RIFKIND:

 3         Q.    Let me call your attention, then, to Plaintiff's 

 4    Exhibit 686.  Would you tell us what that does?

 5         A.    This exhibit sets forth by state calculations of 

 6    the probability that the adjustment will, in fact, improve 

 7    the measurement of distributive accuracy.  It does so using 

 8    the variance as given in the total error model.  

 9               You will recall that when I was commenting on 

10    Plaintiff's Exhibit 9, I mentioned that the Secretary went 

11    on to discover, I think I did, went on to discover what 

12    happened if you increase the variance.  I have done that 

13    too, but this is the first shot.  

14               It calculates both the number of states in which 

15    the probability of being made better off is by adjustment is 

16    greater than half.  That number is approximately 40.  In 

17    fact, the number is exactly 40.  It also calculates the 

18    expected number of states to be made better off.  That 

19    number, as it turns out, is almost exactly 40 as well, 

20    although it doesn't have to be the same.  

21               The table organizes states in descending order of 

22    the probability that the adjustment will make them better 

23    off.

24         Q.    So this shows the probability that an adjustment 

25    improves distributional accuracy, is that what this is 
                                                              1006

 1    addressed to?

 2         A.    Yes.  Perhaps I ought to say that these 1s at the 

 3    top are, of course, not 1s literally.  It is 1.000, and in 

 4    fact it is 1.000 for California to several more decimal 

 5    places.

 6         Q.    What does that tell you about California, that 

 7    1.000?

 8         A.    It is virtually certain that adjustment will make 

 9    the distributional share of California measured more 

10    accurately.

11         Q.    The same is true of Indiana?

12         A.    Yes.

13         Q.    And so on through Ohio, Wisconsin, Pennsylvania, 

14    and Minnesota and Rhode Island?

15         A.    That's true.

16         Q.    Then what happens?

17         A.    Then the probability begins to decline.  It does 

18    not decline very fast.  In Massachusetts the probability of 

19    being made more accurate is very, very slightly less than 1.  

20    That is, the probability that it won't be made more accurate 

21    is about 1 in 1,000.  Similarly, for New Hampshire.  The 

22    probability of being made less accurate is about 2 in 1,000 

23    for Nebraska, 3 in 1,000 for Kansas and Connecticut, and so 

24    on.  

25               The break point comes between New York and 
                                                              1007

 1    Arkansas towards the bottom of the table.  New York has a 

 2    probability of about 6 out of 10 of being made more accurate 

 3    in distributive terms.  Arkansas has a probability of just 

 4    under .5.  And so on down.  For Maryland it is very unlikely 

 5    that the distributive accuracy will be improved.

 6         Q.    Can the same analysis be made on the basis of 

 7    numeric accuracy as opposed to distributional accuracy?

 8         A.    It can.

 9         Q.    Did you do so?

10         A.    Yes.

11         Q.    I call your attention to 684.  

12               THE COURT:  Before you leave, those numbers 

13    appearing under the probability column, they are not 

14    percents, are they?  

15               THE WITNESS:  You mean in the probability column?  

16               THE COURT:  Yes.  1.00 -- 

17               THE WITNESS:  No, they are not percent.  That 

18    number is 1.  

19               THE COURT:  1 what?  

20               THE WITNESS:  I'm sorry. 

21               MR. RIFKIND:  Thank you.

22               THE WITNESS:  Let's start with a different number 

23    other than the 1s.  Then I will get back to the 1s.  

24               If you look down at, let us say, let's sake New 

25    York.  It is about 12 from the bottom.  
                                                              1008

 1               THE COURT:  I see it.  

 2               THE WITNESS:  The number that appears there is 

 3    0.599.  That means that the chances are 599 out of 1,000 

 4    that in New York distributive accuracy will be improved by 

 5    an adjustment.  As you go up the table, that probability 

 6    goes up.  For Massachusetts, the probability is 999.  

 7               THE COURT:  California the chances are 1,000 out 

 8    of 1,000?  

 9               THE WITNESS:  Yes.  Of course that isn't 

10    literally true.  The literal statement is it is 

11    999999999999, some large number of 9s, out of something that 

12    is one bigger than that.  But within the limits of three 

13    decimal places, it is a sure thing.  The probability that it 

14    will be made less accurate is much less, considerably less, 

15    than 1 in 1,000.

16         Q.    Just calling your attention to Plaintiff's 

17    Exhibit 684, I ask you, does that go through the same sort 

18    of approach on the side of numeric and opposed to 

19    distributional accuracy?

20         A.    Yes, it does.

21         Q.    What is the conclusion you reached on the basis 

22    of 684?

23         A.    684 again lists by states in descending order of 

24    probability the probability that numerical accuracy will be 

25    improved by adjustment in that state.  It does it for, 
                                                              1009

 1    again, the variance of the total error model as given by the 

 2    bureau.  

 3               You can get a count of the number of states in 

 4    which it is more likely than not that numerical accuracy 

 5    will be improved.  This time it is 39 rather than 40.  You 

 6    can also get a count of the expected number of states which 

 7    will have numeric accuracy improved.  That also happens to 

 8    be 39; but, again, that is an accident, it doesn't have to 

 9    come out that way.  

10               I guess there are two things about this table as 

11    compared to the other table that we just looked at that are 

12    worth remarking on.  One is that there are a lot more states 

13    this time in which you are next to certain that accuracy 

14    will be improved.

15         Q.    That is all the states down through Washington, 

16    is it?

17         A.    Yes.  Then there are a bunch of .999s right after 

18    it.  

19               The other is, as I said before, different states, 

20    as you would expect, that will have the numeric accuracy 

21    improved and will have the distributive accuracy improved.  

22    That is a different measure.  Likely too, I should say.  You 

23    don't know which ones will.

24         Q.    What can one say about the states on either of 

25    these versions that come out less than .5?
                                                              1010

 1         A.    It is not hard to see why they come out to be 

 2    less than .5.  The easiest way to see that is to start with 

 3    the numerical accuracy version.  

 4               (Continued on next page) 

 5    

 6    

 7    

 8    

 9    

10    

11    

12    

13    

14    

15    

16    

17    

18    

19    

20    

21    

22    

23    

24    

25    
                                                              1011

 1         Q.    In connection with this discussion, I would like 

 2    to call your attention, if I may, to Plaintiff's Exhibit 

 3    707, which has previously been received in evidence.

 4               Can you see that?

 5         A.    Yes.  I am going to have to come down for this.

 6         Q.    Could you come down and tell us what you have to 

 7    say about the relationship of these two?

 8         A.    Yes.

 9               Could I have some nonbook copies of the exhibits 
                                                                  

10    we have just been looking at.  

11               MR. RIFKIND:  Let me hand the witness, again, 

12    Exhibits 684 and 686.  

13               MR. ZITCOV:  I will just again note for the 

14    record that briefly this is another thing we didn't receive 

15    until Friday at 3:00 o'clock.  

16               MR. ZIMROTH:  Is that 707?  

17               MR. RIFKIND:  It was received in evidence during 

18    Dr. Ericksen's testimony.  

19               MR. ZITCOV:  It was?  Then I take that one back, 

20    your Honor.

21               I am chasten.  

22               MR. RIFKIND:  Focus on what you have, Mike.  

23               MR. ZITCOV:  We will, if you just tell me what I 

24    got.  

25               MR. RIFKIND:  You got this.
                                                              1012

 1         Q.    Go ahead, Dr. Fisher.

 2         A.    Okay.

 3               Let me see if I can do it from this side.

 4               There are some states here which have big 

 5    measured undercounts and some states which have relatively 

 6    small measured undercounts.

 7               In the case of small measured undercounts, the 

 8    adjustment to be made is naturally small.

 9               What you expect to happen is that the uncertainty 

10    of the adjustment connected with states in which the 

11    adjustment is small may very well be as big as the 

12    adjustment itself.

13               So take an example that is not drawn from real 

14    data, if you had a state which you thought the numeric 

15    accuracy, the estimate of the undercount, you undercounted 

16    it by two or three people, it would be very likely that the 

17    uncertainty with which you knew that that was two or three 

18    was sufficient that you would expect that the probability 

19    that an adjustment would make things better would be rather 

20    low.

21               Not surprisingly, then, if you look at on Exhibit 

22    684, which I am holding in my hand and I'm going to point 

23    out the states, if you look at the states in which it is 

24    more likely than not that adjustment will not improve 

25    accuracy, they are the states at the left-hand side of the 
                                                              1013

 1    chart.  They are states like Rhode Island and Connecticut, 

 2    New Hampshire and Pennsylvania.  They are the ones over here 

 3    where adjustment is going to be small anyway and you are not 

 4    going very much and you are not certain that things will 

 5    make things better.

 6               The states in which a big adjustment gets made, 

 7    the probability that adjustment makes things better in terms 

 8    of numeric accuracy, turns out to be greater than a half.  

 9    By the time you get over to, let us say, California or 

10    Arizona over in here, the probability is very high, indeed, 

11    that you are going to make things more accurate by 

12    adjustment.  That's what one ought to expect.

13               In terms of distributive accuracy, there is a 

14    similar phenomenon, except here you have to look at the 

15    different place on Plaintiff's Exhibit 707.

16               States whose distributive accuracy will be 

17    changed by adjustment are states where the estimated 

18    undercount in percent terms differs from the national 

19    average.

20               If all states were undercounted by the same 

21    percent, then distributive accuracy wouldn't be changed by 

22    adjustment at all.

23               A state of the national average estimated 

24    undercount is, as I recall, 2.1 percent.  A state with an 

25    undercount estimated at 2.1 percent will have its relative 
                                                              1014

 1    share unchanged.  So states around the national average in 

 2    terms of percentage undercount will be states to whose 

 3    distributive share adjustment doesn't make much difference 

 4    and just as in the previous case, these are likely to be the 

 5    states in which you are not really sure that adjustment is 

 6    going to improve accuracy.

 7               Now, if you look at Plaintiff's Exhibit 686 and 

 8    ask which are the states in which it is more probable than 

 9    not that adjustments will make things worse, they turn out 

10    to be the states in the middle of the chart, the states 

11    with, by and large, the states with undercount rates roughly 

12    around 2.1.  They are states like Mississippi, which is in 

13    here (indicating), Mississippi, Kentucky, Utah, all with 

14    undercount rates in the center clustering around 2.1.

15               And it's true that some of the states with 

16    undercounts near 2.1, you are relatively certain, but almost 

17    all the ones in which it appears more likely than not that 

18    adjustment will make things worse in terms of distributive 

19    accuracy are the ones in which adjustment isn't doing very 

20    much in terms of distributive accuracy.  

21               MR. RIFKIND:  Your Honor, at this time I would 

22    like to offer Plaintiff's Exhibit 684 and 686 in evidence.  

23               MR. ZITCOV:  I object.  

24               THE COURT:  684 and 686 are both admitted, again, 

25    subject to the comments I made to Mr. Sitcov if at the end 
                                                              1015

 1    of cross he feels that he has been unfairly sandbagged by 

 2    the late production of these, he may renew his argument.  

 3               MR. ZITCOV:  Thank you.  

 4               (Plaintiff's Exhibits 684 and 686, respectively 

 5    marked for identification were received in evidence.)

 6               MR. RIFKIND:  Perhaps this is an appropriate time 

 7    for lunch.  

 8               THE COURT:  I think it would be.

 9               We will resume at quarter of two.

10               (Luncheon recess) 

11    

12    

13    

14    

15    

16    

17    

18    

19    

20    

21    

22    

23    

24    

25    
                                                              1016

 1               A F T E R N O O N     S E S S I O N.

 2                                      1:45 o'clock p.m.

 3    FRANKLIN M. FISHER,            resumed 

 4               MR. RIFKIND:  Good afternoon, your Honor.  

 5               THE COURT:  Mr. Rifkind.  

 6    DIRECT EXAMINATION (Continued) 

 7    BY MR. RIFKIND:

 8         Q.    We were discussing just before lunch the number 

 9    of states expected to be made better by adjustment under the 

10    loss function analyses that the Bureau prepared.

11               Is it possible to say anything --

12         A.    It is true we were talking about the number of 

13    states to be expected to do better, but that is independent 

14    of the loss functions that the Bureau prepared, that is true 

15    --  this is a calculation of probabilities that doesn't 

16    depend on the loss functions.

17         Q.    Fair enough.

18               Based on that analysis, however, can you say 

19    anything about the proportion of the population that lives 

20    in the states expected to be made better?

21         A.    Yes.

22         Q.    What can you say?

23         A.    Well, you can do this two ways.

24               Way number one is to look at what fraction of the 
                                                                   

25    population lives in the states where it is more likely than 
                                                              1017

 1    not that adjustment will make the distributive shares 

 2    better, and the other way is to weight the states by 

 3    population and ask, in effect, what is the expected fraction 

 4    of the population living in areas that would be made better, 

 5    and you don't get much of a difference.

 6               You will recall that it's 40 out of 51 states 

 7    either way that are expected to be made better or which have 

 8    the property that it is more likely than not that it will be 

 9    made better.  That's about 80 percent of the states.  It's 

10    85 percent of the population or 90 percent of the 

11    population, depending on which of the other two measures you 

12    take.

13               So it's, as it were, a bigger fraction of the 

14    population than of states are likely to be made better.

15         Q.    So 85 or 90 percent of the population, depending 

16    on which way you calculate it, are living in states that are 

17    expected to be made better?

18         A.    That's not totally accurate, no.

19         Q.    Correct me.

20         A.    You should have left it alone.

21         Q.    I should have left it alone?  I want to be sure I 

22    got it when I leave it.

23         A.    If you look at --  let me do it as precisely as I 

24    can.

25               If you look at the states in which it is more 
                                                              1018

 1    likely than not that they will be made better by adjustment 

 2    in terms of distributional accuracy, 90 percent of the 

 3    population lives in those states.

 4               If you take the other measure, which is, in 

 5    effect, to say if you got a state in which the probability 

 6    is a half that it will be made better, you take half the 

 7    population as the expected number, expected number of people 

 8    involved.  As you weight that by the population, then you 

 9    add that across states, then you come out with about 85 

10    percent.

11         Q.    It's either 85 percent or 90 percent?

12         A.    Yes.

13         Q.    I think the next item in your cascade, Professor 

14    Fisher, had to do with variance.

15               Perhaps in that regard we should turn to page 229 

16    of the Secretary's --  

17               THE COURT:  2-29?

18         Q.    2-29, the Secretary's decision, which is 

19    Plaintiff's Exhibit 9.

20         A.    Okay.

21         Q.    Does that indicate where you want to attack the 

22    Secretary with respect to variance?

23         A.    Well, it reminds me --

24         Q.    What you had in mind with respect to the 

25    Secretary's decision --
                                                              1019

 1         A.    It reminds me what I had to say about the 

 2    Secretary's decision, I'm not particularly interested in 

 3    attacking him.

 4         Q.    Fair enough.

 5         A.    All right.

 6               The Secretary having made the estimate that it 

 7    will be 21 states that are measured says, "However, this is 

 8    clearly a minimum estimate," and then goes on and quotes 

 9    from the undercount steering committee to say, "As a matter 

10    of judgment, the total understatement of variance of the 

11    estimates from the smoothing model may be in the range of a 

12    factor of 1.7 to 3.0 in terms of variance." 

13               That is the end of the quote from the undercount 

14    steering committee.

15               And then he says that if you allow for a variance 

16    of 2.0, which is the lower end of the undercount steering 

17    committee range, the proportional shares are about 28 or 29 

18    states will be worsened by an adjustment of distributive 

19    accuracy.

20               Now, I have made the calculation of what, in 

21    fact, is true about the expected number of states that will 

22    be made worse off, that is, how that has affected if you 

23    change the variance.

24               You will recall that according to one of the 

25    exhibits we were talking about before lunch, I can't at the 
                                                              1020

 1    moment remember which one but you can probably look it up 

 2    faster than I can, the one that gave the probability of the 

 3    states that would be made better off in terms of 

 4    distributive accuracy --

 5         Q.    Exhibit 686, I am told, PX 686.

 6         A.    Thank you.

 7               That that calculation showed that 40 states is 

 8    the expected number of states that would be made better off.  

 9    That was, as I said at the time, with the variance of the 

10    total error model not multiplied by anything except one.

11               If you multiply that variance by two, which is 

12    what the Secretary is doing in this passage, and make the 

13    same calculation, the number of states that would be made 

14    better off, the expected number of states that would be made 

15    better off by adjustment goes from 40 to 35, which is still 

16    not close to what is, in fact, here the correct cutoff 

17    point, namely, half the states, 25 and a half, without 

18    counting the District of Columbia.

19               I have then asked the question, by what amount, 

20    by what factor would you have to increase the variance in 

21    the total error model in order to drive the expected number 

22    of states that would be made better off for adjustment down 

23    as far as half the number of states? 

24               And the answer to that turns out to be you have 

25    to increase the variance by a factor of between seven and 
                                                              1021

 1    eight before that happens.  That is more than double the top 

 2    end of the undercount steering committee's range.

 3         Q.    Does that suggest that this analysis is what the 

 4    statisticians call robust?

 5         A.    It was certainly robust.

 6               You have to get --  you have to take the extreme 

 7    end of the undercount steering committee's range and more 

 8    than doble that again before you get to the break-even point 

 9    for this one.

10         Q.    You refer to a break-even point of 25/25.

11         A.    25 and a half, actually.

12         Q.    25 and a half, 24 and a half, I suppose.

13         A.    No, 25 and a half, 25 and a half.  It's 51 is the 

14    problem.

15         Q.    Okay.

16               In analyzing whether or not this analysis favors 

17    the PES or the uncorrected counts, is that the appropriate 

18    break-even point?

19         A.    If you are really going to calculate the expected 

20    number of states made better as I have done, the answer is 

21    yes, it is.

22         Q.    The Secretary refers at the bottom of page 2-29 

23    and the top of page 2-30 to a shift of proportional shares 

24    of about 28 or 29 states being worsened, allowing for a 

25    variance factor of 2.0.
                                                              1022

 1               Is that a significant observation?

 2         A.    No, it's not.

 3         Q.    Why is that?

 4         A.    Well, in the first place, as I said before, that 

 5    number, either the 21 or the 28 to 29 is the answer to a 

 6    quite peculiar question.

 7               In the second place, if you are going to ask that 

 8    question, it turns out somewhat counterintuitively that it 

 9    is not true that the break point, the place at which there 

10    is --  at which you say you can't tell is at roughly 25 

11    states.  It isn't the 25 states.  In the sort of central 

12    model everybody would consider and that the undercount 

13    steering committee did consider, that break point occurs 

14    at 34.

15         Q.    That is to say, if 34 states are disimproved and 

16    the rest are improved, you would still favor adjustment?

17         A.    No.

18               That is the point at which, so far as that 

19    particular measure is concerned, you can't tell which one is 

20    being favored.

21         Q.    Tweedle-dum, Tweedle-dee effect?

22         A.    As it were.

23         Q.    As it were.

24               How do you know that it's 34?

25         A.    Well, as I said, it doesn't have to be 34, the 
                                                              1023

 1    central model, the simplest central model which everybody 

 2    would consider first gives 34.

 3               To know that, to know exactly how, you would have 

 4    to know not terribly different but some statistical theory.  

 5    You can do that.  

 6               You can also find in the administrative record 

 7    occasions in which it is pointed out that, in fact, it is 

 8    34.

 9         Q.    One second and I will find it.

10               Let me call your attention to Plaintiff's Exhibit 

11    54, which I believe has been received in evidence as the 

12    report of the undercount steering committee, dated June 21, 

13    1991.

14               Let me call your attention in particular to the 

15    addendum to that, which begins at Bates stamp 911 and then 

16    particularly to the fifth page of that addendum at Bates 

17    stamp 915.

18               What does this say about the break-even point 

19    question?

20         A.    Well, if you will look at Bates stamp page 915, 

21    after the last mathematical expression, the second sentence 

22    in the paragraph begins, "Under assumptions of normality, 

23    the negative sign should appear in an expected 68 percent of 

24    the states or about 34 out of 50.  Intuition that the 

25    break-even point is when half of the states have negative 
                                                              1024

 1    loses and half have positive is not direct."  

 2               "Not" is underlined in the original.

 3               This document, by the way, is referred to on page 

 4    2-29 of the Secretary's report and it is, in fact, part of 

 5    an appendix to the Secretary's report.

 6         Q.    I take it from what you say that the Secretary's 

 7    analysis on page 2-29, however, does not reflect that 

 8    caution that we have just looked at?

 9         A.    It does not.  The Secretary appears to believe 

10    that 28 or 29 --  

11               MR. ZITCOV:  Objection, your Honor, and move to 

12    strike.  How could he know what the Secretary believes or 

13    appears to believe?  

14               He can say what it says, but he can't describe 

15    what it --  

16               THE COURT:  I agree.

17         Q.    Would you confine yourself to what it appears 

18    from the Secretary's written decision.

19         A.    The Secretary's pages --  I got to find it again 

20    now.  Other.  Hang on a second.

21               (Pause)

22               Between page 2-29 and 2-30 is the place in which 

23    it says that if you allow variance of 2.0, the proportional 

24    shares are about 28 or 29 states would be worsened by 

25    adjustment by distributive accuracy.
                                                              1025

 1               I do not know why 28 or 29 can be of any 

 2    importance, unless one things that 28 or 29 moves things to 

 3    the other side of the break point, and, in fact, it does 

 4    not.  

 5               MR. RIFKIND:  Your Honor, I am advised I misspoke 

 6    a minute ago.  Plaintiff's Exhibit 54 has not been received 

 7    in evidence, it was referred to a number of times as the 

 8    report of the undercount steering committee, and it appears 

 9    from these Bates stamp numbers appears to be part of the 

10    so-called administrative record and, indeed, this portion of 

11    it is attached as an appendix to the Secretary's decision, 

12    and as Professor Fisher has just said, referred to in the 

13    Secretary's decision and, therefore, move Plaintiff's 

14    Exhibit 54 in evidence at this time.  

15               MR. ZITCOV:  No objection.  

16               THE COURT:  Plaintiff's Exhibit 54 is received.

17               (Plaintiff's Exhibit 54 marked for identification 

18    was received in evidence.) 

19    BY MR. RIFKIND:

20         Q.    Let me call your attention, Professor Fisher, to 

21    Plaintiff's Exhibit 41 if I may.

22               What is Plaintiff's Exhibit 41?

23         A.    Plaintiff's Exhibit 41 purports to be notes of 

24    Secretary's meeting on census adjustment-technical issues, 

25    July 8, 1991, and are notes of, as the first paragraph 
                                                              1026

 1    suggests, a meeting that the Secretary held with the Census 

 2    Bureau technical experts and department policy officials.

 3         Q.    Looking at what is marked Bates number 3217, the 

 4    third page of those notes, let me call your attention to the 

 5    next to the last paragraph, second sentence.  Would you look 

 6    at that material.

 7               Perhaps you can read it into the record, 

 8    Professor Fisher.

 9         A.    Starting with the second sentence?

10         Q.    Right.

11         A.    "The Secretary asked whether an adjustment was 

12    advisable if accuracy is improved in the majority of states 

13    at the expense of the minority.  Fay responded that if the 

14    harm from an adjustment was equal to the improvement, the 

15    census should be used.  Fay then explained that the lost 

16    function analysis, as designed, produced a result that ran 

17    against intuition.  According to Fay's calculation, the 

18    'break-even' point where adjustment would do as much harm as 

19    good is 34 states harmed rather than the 25 states that one 

20    would expect.  Thus, in Fay's view, the estimated 21 states 

21    whose accuracy would be apparently harmed by an adjustment 

22    is below the break-even point of 34 and, thus, the PES is 

23    not as bad as it may seem."

24         Q.    Is that the break-even point that you were just 

25    making to the court a moment ago?
                                                              1027

 1         A.    It is.

 2         Q.    And your understanding is that Fay is making the 

 3    same point to the Secretary that you just made to us?  

 4               MR. ZITCOV:  Object.  He could not possibly know 

 5    the point that Mr. Fay is making to the Secretary.  He can 

 6    ask him if that's the same point that Dr. Fisher is making, 

 7    but he certainly can't testify to the point that Dr. Fay was 

 8    making.

 9         Q.    Is the point that is made in these notes at page 

10    3217 and attributed to Mr. Fay in a response to a question 

11    from the Secretary the point that you were just making to 

12    the court?  

13               MR. ZITCOV:  I object to the form of the question 

14    again, your Honor.  

15               THE COURT:  That is a different question.

16               I will overrule the objection.

17         A.    And the answer is, of course it is.

18         Q.    Are you aware of any serious statistical 

19    professional dispute about the validity of the point that is 

20    being made here in these notes?

21         A.    No.  One can argue as to whether the true 

22    break-even is 34.  That does depend on the model used.  But 

23    the fact that the break-even certainly doesn't have to be 

24    50/50 and is unlikely to be 50/50 is not a matter of 

25    dispute.
                                                              1028

 1         Q.    Does that conclude your cascade with respect to 

 2    the points first raised with respect to the Secretary's 

 3    discussion of loss function?

 4         A.    Well, let me have a look.

 5               (Pause)

 6               That's not all I have to say about the 

 7    Secretary's discussion of loss functions, but I think I have 

 8    now covered what I have to say about what you originally 

 9    asked me about, which is this business about 21 states.

10               Let's see.

11               It despeaks a loss function which to use is 

12    irrational.  The 21 states is a misquoted item.  If you 

13    take what the Secretary's says he is quoting from, it should 

14    be 11.

15               Putting that aside, the 21 states is not, in 

16    fact, an answer to the question the Secretary appears to be 

17    asking.  That answer is much smaller.  That answer is also, 

18    as it happens, approximately 11.

19               It is considerably robust to variance, and if you 

20    take the question and the answer the Secretary gave, the 

21    right thing to compare it to is not 25 states, which is 

22    apparently done, but something more on the order of 34.  

23               I think that's about it.

24         Q.    Thank you, Dr. Fisher.

25               Let me ask you to look at page 2-27 of the 
                                                              1029

 1    Secretary's decision.

 2               The Secretary there refers to hypothesis tests.

 3               Do you see that, Dr. Fisher?

 4         A.    I don't, actually.  Could you direct me?

 5         Q.    It's the next to the last paragraph on the page.

 6               The sentence beginning, "Unfortunately, the 

 7    Census Bureau did not have the time to conduct the 

 8    hypothesis test required by Guideline One.

 9         A.    I'm sorry.  I have it, yes.

10         Q.    "The method they used instead to make these 

11    comparison is called loss function analysis." 

12               Do you understand the Secretary to be saying that 

13    something would be better or preferable to loss function 

14    analysis?

15         A.    That is certainly the implication.

16         Q.    Do you understand what it is this report says 

17    would be better than a loss function analysis?

18         A.    Well, the Secretary, the implication is that a 

19    hypothesis test of something would be better than a loss 

20    function analysis.

21               Now, if you ask me what that means, then I think 

22    there are several possible interpretations as to what it 

23    could mean.

24         Q.    Would you give us those interpretations?

25         A.    Well, it could mean literally a test of the 
                                                              1030

 1    hypothesis that --  

 2               MR. ZITCOV:  I object, your Honor.  I'm not sure, 

 3    but is the witness speculating about what he thinks the 

 4    Secretary may have been thinking?  

 5               If he is, that certainly would be improper 

 6    testimony on his behalf.  

 7               THE COURT:  We are unclear as to what you are 

 8    doing here. 

 9               THE WITNESS:  I'm saying to the extent this calls 

10    for hypothesis tests, I'm listing what the hypothesis tests 

11    are that might be done to answer the call.  

12               THE COURT:  All right.  

13               MR. ZITCOV:  My only concern is that he not be 

14    testifying that he thinks this is what the Secretary means.  

15               THE COURT:  I don't have the impression that he 

16    is doing that.  

17               MR. ZITCOV:  Okay.  That's fine.

18         A.    One possibility is that one might thing of a 

19    hypothesis test that tests the so-called null hypothesis 

20    that the census is more accurate.  That, as it turns out, 

21    cannot be done.

22               One might ask --  one might say, well, what this 

23    calls for is a hypothesis test that the expected accuracy of 

24    the census is higher.  That can be done and it is done --  

25    it is done, in fact, by extending slightly the Bureau's loss 
                                                              1031

 1    function function analysis.

 2               The third thing that one might do occurs 

 3    elsewhere in the report where the Secretary discusses tests 

 4    of the hypothesis that undercounts in different states are 

 5    different from the national average.  Those tests can, in 

 6    fact, be made.  How useful they are is a different question, 

 7    but one can do that.

 8         Q.    What tests would those be?

 9         A.    Those are tests of the proposition that the 

10    percent undercount in each state, taking it one state at a 

11    time, differs from the national average so that there is a 

12    serious reason for believing that the distributive shares 

13    should be affected.

14         Q.    And have you considered that question?

15         A.    Yes.

16         Q.    And what have you concluded?

17         A.    Apparently you can make those tests.  Those 

18    tests, in fact, are mentioned in the Secretary's report in a 

19    different place.  But that is not the --  but that is not a 

20    sensible thing to base a decision on.

21         Q.    What would a sensible thing to base his decision 

22    be?

23         A.    Well, you want to know whether or not --  you 

24    mean in terms of hypothesis, don't you?

25         Q.    Exactly.
                                                              1032

 1         A.    You want to know whether or not the census or the 

 2    adjusted method has expected accuracy higher.  You have used 

 3    your measured expected accuracy in terms of loss function, 

 4    squared error loss function, absolute loss function.  That's 

 5    what they are for.

 6               One could reasonably ask the following question:

 7               Yes, we observed that the adjustment appears more 

 8    accurate in terms of the loss function, that is, it's 

 9    expected accuracy is bigger, is greater, but our estimate of 

10    the expected accuracy is itself subject to sampling error 

11    and one could test the hypothesis that that result, the 

12    apparent victory for adjustment in terms of the loss 

13    function, could have been produced by chance if, in fact, 

14    if, in fact, the true difference were zero.  You can make 

15    that test.

16               You cannot, as I said, ever make the test 

17    literally that the census is more accurate than the 

18    adjustment.

19         Q.    Do I understand that you say you can test for the 

20    question is the difference that is observed likely to be 

21    attributable to chance or to randomness?

22         A.    You can ask --  unfortunately, I'm sorry about 

23    this, but precision of language really matters for this sort 

24    of thing.

25               The kind of test that you will make is a test 
                                                              1033

 1    that says suppose it were true that the true value of the 

 2    loss function difference is zero, what is the probability of 

 3    observing a difference as favorable to adjustment as I do 

 4    observe through random error.

 5         Q.    And have you done that sort of testing?

 6         A.    Yes.

 7         Q.    And what did it conclude?

 8         A.    Oh, you conclude that the probability that that 

 9    result of observing something as big as you observe through 

10    random error is vanishingly small.

11         Q.    Did you do that with respect to the updated loss 

12    functions you previously discussed?

13         A.    I did.

14         Q.    And was it true of that?

15         A.    It was.

16         Q.    Did you do it with respect to the final loss 

17    functions?

18         A.    I did.

19         Q.    And was it true of that?

20         A.    It was.

21         Q.    Let me call your attention to 696.

22               Let me start with the final loss function.  Let 

23    me start with 698.

24               (Pause)           

25               MR. SITCOV:  Your Honor, I think I can move 
                                                              1034

 1    things along while Mr. Rifkind is looking through the 

 2    documents is to object to any discussion of this.  Again, 

 3    this would require testimony from Dr. Fisher about areas 

 4    that he testified on his deposition he hadn't considered.

 5               We have not been advised at all that he has 

 6    considered this since then and have not amended their 

 7    answers, so we certainly object to any discussion of this or 

 8    the introduction of this document in evidence.  

 9               THE COURT:  Your objection is noted and the same 

10    ruling.  

11               MR. RIFKIND:  I know when to take yes for an 

12    answer, but I want to point out that at the deposition of 

13    Dr. Fisher on pages 33 and 34 he discusses precisely this 

14    point.  

15               They have had more than two weeks with the 

16    exhibits I just referred to and they were offered at the 

17    time we introduced the exhibits would relate to hypothesis 

18    testing in which they were told Dr. Fisher was going to 

19    testify about the underlying data on which the results were 

20    based.  

21    BY MR. RIFKIND:

22         Q.    Do you have, Dr. Fisher, in front of you 698?

23         A.    I do.

24         Q.    Could you tell us what that discloses?

25         A.    The operative one --  the easiest sentence to 
                                                              1035

 1    understand, I think, the easiest piece to understand is the 

 2    piece that is given in the starred footnote.

 3               If it were true that the true loss function 

 4    difference for the squared error loss function is between 

 5    the census and the adjustment, if that true difference were 

 6    really zero --

 7         Q.    Zero would mean that they were indistinguishable?

 8         A.    That's right, zero would be a tie.

 9               The probability of observing a value of the loss 

10    function in favor of adjustment as big as you do observe 

11    just by chance would be less than one in 150 million.

12         Q.    Let me call your attention to 697.

13               Does that do the same thing with respect to 

14    unweighted persons --  I'm sorry, I misspoke.

15               Let me call your attention to 696 instead.  

16               THE COURT:  And not 697?  

17               MR. RIFKIND:  And not 697, your Honor.

18               (Pause)

19         Q.    Do you have that in front of you?

20         A.    Yes.

21         Q.    What does that tell you?

22         A.    This does the same thing for the updated loss 

23    function analysis that 698 does for the final loss function 

24    analysis, and here the chance that if it were really a tie 

25    you would get a result as favorable to adjustment through 
                                                              1036

 1    chance as you do get is less than one in 12 million.

 2         Q.    And then let me call your attention to 

 3    Plaintiff's Exhibit 700.

 4               What does that disclose?

 5         A.    This does it for the absolute error loss 

 6    function.  Here the chance --

 7         Q.    Which is a product of the final loss function, 

 8    right?

 9         A.    Yes, it is.

10               Here the chance if it's really a tie, the chance 

11    of observing something as favorable to adjustment as you do 

12    is --  I mean, these other numbers are really low, I think 

13    this one stands out --  this is less than one in 14 

14    trillion.

15         Q.    What do those sets of numbers, those three sets 

16    of numbers you have just given us, suggest to a 

17    statistician?

18         A.    In order to believe --  

19               THE COURT:  A big fee.  

20               MR. RIFKIND:  Alas for Dr. Fisher it ain't so.

21               THE WITNESS:  Yes, actually, your Honor, I would 

22    be happy to multiply --  Mr. Rifkind will be happy to pay me 

23    14 trillion times what he is paying me now, I assure you.

24         A.    In any event, look, in order to believe that the 

25    loss function analysis answers in favor of adjustment are 
                                                              1037

 1    really due to sampling error, you know, given the estimates 

 2    of sampling error included in this, you must believe that an 

 3    incredibly, incredibly improbable event occurred and no 

 4    rational person, of course, would choose to believe that.

 5         Q.    Am I right you did this for both the so-called 

 6    PRODSE version and the unweighted persons version?

 7         A.    That's correct.

 8         Q.    Any consequential difference between the two sets 

 9    of calculations?

10         A.    Well, the size of the enormous numbers varies, 

11    but they remain enormous.

12         Q.    No matter how you slice them?

13         A.    That's correct.

14         Q.    And those are reflected in 697, is that correct, 

15    Dr. Fisher, the unweighted persons for the updated loss 

16    function?

17         A.    Yes.

18         Q.    And in 701, that's the unweighted persons for the 

19    final loss function analysis, is that right?

20         A.    For the absolute error loss.

21         Q.    Yes.

22         A.    Yes.

23         Q.    Have I covered it all?  And 699 does the same 

24    thing with respect to the unweighted persons for the final 

25    loss function squared error loss, is that right?
                                                              1038

 1         A.    Yes.  I shouldn't have been so --  yes, yes.  

 2    Never mind.

 3         Q.    And all of those come to the same conclusion for 

 4    practical purposes, isn't that right?

 5         A.    They do.  

 6               MR. RIFKIND:  Your Honor, I offer at this time 

 7    Exhibits 696 through 701.  

 8               MR. ZITCOV:  I will restate our previous 

 9    objections.  

10               THE COURT:  Exhibits 696 through 701 are, again, 

11    admitted subject to terms that I outlined earlier today.

12               (Plaintiff's Exhibits 696 through 701, 

13    respectively, marked for identification were received in 

14    evidence.) 

15    BY MR. RIFKIND:

16         Q.    Have you considered, Dr. Fisher, whether these 

17    hypothesis tests that you have just spoken about would come 

18    out differently if one increased the level of variance?

19         A.    I have considered what would happen if you 

20    increased the level of variance in the total error model 

21    much the way as we discussed before in terms of the expected 

22    number of states.

23         Q.    And what did you conclude?

24         A.    I recall the numbers, I think, for the squared 

25    error loss function.
                                                              1039

 1               If you double the variance in the standard --  in 

 2    the total error model, it still remains true that the 

 3    probability of observing a result --  

 4               MR. ZITCOV:  Your Honor, I'm afraid I'm going to 

 5    have to interrupt and object again.

 6               During Dr. Fisher's deposition I asked him 

 7    exactly this line of questions and he was directed not to 

 8    respond to the question of how he calculated these numbers.

 9               I'm thinking now of the testimony in my questions 

10    that appear on page 316 of his deposition.  I asked him a 

11    question, "I believe yesterday he testified that if the 

12    variances of the smooth adjustment factors are multiplied by 

13    4.5, the result of loss function analysis changes from 

14    favoring adjustment to favoring the census, is that right?" 

15               "A.    I think that's correct, or for this 

16    particular loss for the squared error plane.

17               "Q.    And did you do that calculation yourself?

18               "A.    No.

19               "Q.    Who did?" 

20               "MS. GOLDSTEIN.  I'm going to instruct the 

21    witness not to answer."

22               Now, we attempted during his deposition to 

23    inquire into this area.  We were prevented from inquiring 

24    into that area and I understand that to be exactly the area 

25    that he is discussing now.  
                                                              1040

 1               MR. RIFKIND:  I am advised that the witness, in 

 2    fact, told him that he had not made the calculation, that he 

 3    knew the results of the calculation and simply did not 

 4    disclose, was told not to disclose who had done the 

 5    calculating, and if that's true, I don't see what the 

 6    objection is.  The evidence was there, the data obviously 

 7    which could be replicated was also there.  

 8               MR. ZITCOV:  How can we replicated it when he 

 9    said he didn't do it and he won't tell us who did? 

10               "Q.    And did you do this calculation yourself? 

11               "A.    No.  

12               "Q.    Who did?  

13               'MS. GOLDSTEIN:  I am going to instruct thh 

14    witness not to answer."

15               THE COURT:  Wouldn't this problem be left to that 

16    point when you put the question on cross as to who did the 

17    calculations?  

18               MR. ZITCOV:  Well, yes, but we still would have 

19    the same problem, your Honor, and that is I will find out 

20    today as I am cross-examining him for the first time 

21    whatever I find out rather than during discovery when I 

22    could use it in my cross-examination.  

23               THE COURT:  Let's wait.  He may say Mickey Mouse 

24    and the whole thing may become --  

25               MR. ZITCOV:  It sounds to me that that's the 
                                                              1041

 1    likely answer, but --  

 2               THE COURT:  Let's wait and see.  

 3               MR. RIFKIND:  I just want to note for the record 

 4    three things:

 5               One, all the data that were used came from the 

 6    Census Bureau.

 7               Two, the results of these calculations were 

 8    available to them in writing and given to them in writing 

 9    two weeks ago, at which time we specifically asked them, "Do 

10    you want any of the underlying material by which this was 

11    calculated," and I have never been asked for it.  

12               MR. ZITCOV:  That was a request from the 

13    plaintiffs that we never received.  

14               THE COURT:  Are you near to finish, Mr. Rifkind?  

15               MR. RIFKIND:  I would guess about a half hour 

16    more or so.  

17               THE COURT:  Let's take a five minute break at 

18    this point.

19               (Recess) 

20               MR. RIFKIND:  Your Honor, we recessed before PX 

21    41, which were the minutes of the July 8 meeting with the 

22    Secretary, and I would like to move PX 41 into evidence if I 

23    have omitted to do so, and I think I did.  

24               MR. ZITCOV:  We have no objection since it is in 

25    the so-called administrative record.  
                                                              1042

 1               THE COURT:  41 is admitted.

 2               (Plaintiff's Exhibit 41 marked for identification 

 3    was received in evidence.) 

 4    BY MR. RIFKIND:

 5         Q.    Professor Fisher, prior to the date of the 

 6    Secretary's decision, July 15, 1991, did anyone at the 

 7    Bureau conduct a hypothesis test?

 8         A.    Yes, an attempt was made by Dr. Fay to conduct a 

 9    hypothesis test.

10         Q.    Let me call your attention to PX 44 and ask you 

11    what that is.

12               Is that the --

13         A.    It's June 28, 1991 memorandum for the undercount 

14    steering committee from Robert E. Fay, senior mathematical 

15    statistician, "Subject:  Hypothesis test for improvement."

16         Q.    Let me call your attention to the bottom of the 

17    second page, which is Bates number 11192, the top of page 

18    11193 and ask you what does this memorandum report?

19         A.    It reports the results of what Fay did, and in 

20    particular he says, "I report results --  " I am at the page 

21    11192, "I report the results using the variances for the 

22    state estimates as published and 1.5 and 2.0 times the 

23    publication variances." 

24               Skipping a sentence, "Table 1 where those results 

25    are provides evidence that the state distribution is 
                                                              1043

 1    improved by adjustment, although it also shows some 

 2    sensitivity to assumptions about the understatement of the 

 3    variance of the smooth estimates.  This sensitivity must be 

 4    weighted against the observation that in almost all other 

 5    respects tests by design understate the evidence in favor of 

 6    the improvement from adjustment."

 7         Q.    Does the Secretary's decision refer to the 

 8    results Dr. Fay reported?

 9         A.    I do not believe it does.

10         Q.    Let me call your attention to Plaintiff's Exhibit 

11    9 at page 2-31.

12         A.    I'm sorry, it does refer.

13         Q.    You are referring to the footnote on that page?

14         A.    I am.

15         Q.    Footnote 70 on page 2-31?

16         A.    Yes.

17         Q.    What do you conclude about that footnote?

18         A.    I think it is misleading as a discussion of Fay's 

19    results.

20         Q.    In what sense?

21         A.    Well, the footnote says, "Nonetheless, although 

22    the hypothesis tests rejected the superior distributive 

23    accuracy of the census counts if only the measured variance 

24    was changed the adjusted figures, the superior -- " that 

25    should be charged to the adjust figures, I think --  "the 
                                                              1044

 1    superior the accuracy of the census counts was easily 

 2    accepted for a variance factor of 2.0 and appears (by 

 3    interpolation) acceptable to any variance factor in the 

 4    undercount steering committee's plausible range of 1.7 to 

 5    3.0." 

 6               Now, that, in fact, seems to me, let us say, at 

 7    best to put considerable spin on what Fay's results actually 

 8    are.

 9               Taking Fay's results at Fay's value on page 4, 

10    what one discovers is that for states, if you double the 

11    variance 2.0, you fail to reject, not accept, you fail to 

12    reject the null hypothesis that the census is better at the 

13    roughly 11 percent level.  That means that the probability 

14    of Fay observing the results he observed if, in fact, the 

15    true difference was zero, and if these tests are accurate, 

16    was about 11 percent.  That is bigger than the standard 

17    significant levels used by statisticians, but it is not so 

18    big that one could reasonably say that you easily accept the 

19    hypothesis that the census had greater accuracy.

20               That is particularly true if you combine it with 

21    Fay's discussion that I read earlier that said this whole 

22    thing is set up to --  in almost all other respects the 

23    tests by design understates the evidence in favor of the 

24    improvement from adjustment.

25               The Secretary makes no mention of that.
                                                              1045

 1         Q.    I think you said earlier on, Dr. Fisher, that you 

 2    can't test whether one or the other of these two estimates 

 3    is more accurate than the other?  

 4               Did I understand you correctly?

 5         A.    If that's what you are literally asking for, you 

 6    literally cannot do it.

 7               I will be glad to explain.

 8         Q.    Will you explain it?

 9         A.    We have here two measures.  They purport to 

10    measure the same thing.  We want to know which is closer to 

11    the truth.

12               If all we have are the two measures and we don't 

13    know the truth, you can't possibly tell which is closer to 

14    the truth.

15               You can make a statement about which is expected 

16    to be closer to the truth, that's different.  That 

17    statement, as I said before, is made, in effect, with loss 

18    function analysis.

19               To return to my example of this morning of 

20    measuring the heights of lawyers, the biased measuring rod 

21    version gives one answer, let's say five foot ten, the 

22    sampling measure gives another, let's say five foot eleven, 

23    but you don't know --  since you don't know what the truth 

24    is, there is no way you can test the hypothesis that one is 

25    closer to the truth than the other.
                                                              1046

 1               What you can do is to ask, given the estimates of 

 2    what happens if you repeat the sampling procedure, that is, 

 3    given what we know about sampling theory, you can ask which 

 4    of these is likely to be closer, you can ask --  you can 

 5    compare the expected errors from one with the expected 

 6    errors from the other, but you can't literally test whether 

 7    the error from one is bigger than the error from the other 

 8    because to do that really would require the truth.

 9         Q.    Can you ask how likely it is that one is better 

10    than the other?

11         A.    Yes, yes, you can do that.

12         Q.    And I take it that what we have been discussing 

13    up until now deals with exactly that point, does it not, how 

14    much more likely it is that one is better than the other?

15         A.    Yes, measuring better in terms of the loss 

16    functions, which take into account the expected inaccuracies 

17    of the different methods, you can ask the question how 

18    likely is it that the apparent difference in favor of 

19    adjustment is just that, apparent, and that answer turns out 

20    to be very, very low.

21               That's also --  I forget whether we did this 

22    before or not.  That answer --  I'm sorry. 

23               THE WITNESS:  Your Honor, I need to ask --  I'm 

24    not sure where we are on what I was asked just before the 

25    break in which there was an objection.
                                                              1047

 1               If you sustained it, I don't want too ride right 

 2    over that.  

 3               THE COURT:  Does anybody remember?  

 4               MR. ZITCOV:  Or care?  

 5               MR. ZITCOV:  I had objected to the use or to the 

 6    testimony about the multiplication of the rate of variance, 

 7    and I don't know that your Honor had ruled at the time that 

 8    we broke.  

 9               THE COURT:  I don't recall, either.

10         Q.    Why don't you proceed on the assumption that you 

11    can go right forward.  

12               MR. ZITCOV:  Well, because if the judge was going 

13    to sustain my objection, then he shouldn't.  

14               MR. RIFKIND:  I am talking to the witnesses.  

15               THE COURT:  I totally lost it at the moment.  

16               MR. RIFKIND:  That is not very encouraging.

17               Let me drop the question and move on because no 

18    one can quite figure out where it fits in.

19         Q.    Dr. Fisher, let me call your attention to page 

20    2-34 of the Secretary's decision.

21         A.    Yes.

22         Q.    You see the sentence on the bottom, "Only 18 of 

23    the 51 states have an undercount rate that is significantly 

24    different from the national average"?

25         A.    I do.
                                                              1048

 1         Q.    Would you comment on that observation by the 

 2    Secretary?

 3         A.    That reports the results of a hypothesis test 

 4    that I mentioned before, namely, in how many states is it 

 5    the case that you can reject the null hypothesis that the 

 6    states have the same undercount rate as the national 

 7    average, and the answer is in 18 of them you can.

 8               The Secretary then goes on to say that means in 

 9    33 states we don't know whether they are higher or lower or 

10    the same as the national average and appears to put some 

11    weight on that fact.

12               I think that is a really bad misuse of hypothesis 

13    testing in this connection.

14         Q.    And why is that?

15         A.    Well, there are several reasons.

16               Let's take the minor reasons first:

17               If it were the case that the difference in the 

18    undercount rates from the national average were simply due 

19    to chance and that that was typically true, let's ask what 

20    would you expect to find, and the answer is you would expect 

21    then to find that if you did that test at the five percent 

22    level, in five percent of the cases you would find something 

23    that appeared to be significant.  Five percent of 51 states 

24    is a little bit more than two and a half states, so in 

25    between two and three states you would expect to find that 
                                                              1049

 1    result.

 2               You don't find it, you find it in 18 states.  

 3    That suggests that it certainly is not due to chance.

 4               Secondly, it is not the case that doing this test 

 5    state by state is the right way to think about it.  

 6               Let me give you an example.

 7               I'm tempted to pick Mr. Sitcov for this example, 

 8    but he might take it the wrong way and let me simply --  

 9               MR. ZITCOV:  You can use me. 

10               THE WITNESS:  No, no, no, you won't like it.  

11    Trust me.  It's too easy a shot.  You'll see why in a 

12    minute.

13         A.    Let's suppose, Mr. Rifkind, that you offer to bet 

14    on flipping coins with me and that although I can't say that 

15    in real life I have ever known you to do this before I have 

16    some reason to believe that you are playing with unfair 

17    coins, let's say, weighted towards heads, and you say, "All 

18    right, let's test the hypothesis that I play with unfair 

19    coins.  I have a pocket full of coins.  I will allow you to 

20    test it." 

21               We take out the first coin and we flip it and it 

22    comes up heads six times out of ten, we flip it ten times, 

23    and you say, "Oh, well, a fair coin could easily do that 

24    some large fraction of the time, that's not significant at 

25    the five percent level, no evidence." 
                                                              1050

 1               We take out the second coin and it comes up 

 2    heads, I don't know, seven times out of ten, and you say, 

 3    "Oh, well, that's not significant at the five percent 

 4    level," I assume I have the numbers roughly right, "that's 

 5    not evidence, either." 

 6               I take out the third coin and it comes out five 

 7    times out of ten and I take the fourth coin, maybe this one 

 8    comes up four times out of ten, and as I do it coin after 

 9    coin it tends to come up a lot heads and even though no 

10    individual one of them is significant, if you asked what is 

11    the probability that that is happening, if, in fact, you 

12    tend to have fair coins in your pocket, the answer is it is 

13    getting smaller and smaller and smaller that that 

14    probability is not being calculated.

15         Q.    That is, you can accumulate across by a pocket 

16    full of pennies?

17         A.    As it were.

18               Merely the fact that 33 states don't give 

19    something which is significantly different than the national 

20    average does not tell you --  each one separately does not 

21    tell you if all of them together would be significant.

22               More important than that is the following:

23               If one is going to formulate a decision on the 

24    question of how many states have a statistical significant 

25    undercount, then you are implicitly using one of two loss 
                                                              1051

 1    functions, and I think it's quite easy to show that there 

 2    are loss functions that no rational person would use.  This 

 3    is not a good criteria to base a decision on.

 4               Either you are saying I will not adjust --  let's 

 5    take the extreme version --  I will not adjust unless all 

 6    the states show a significant undercount different from the 

 7    national average or you are saying I will not adjust unless 

 8    a majority of states show an undercount significantly 

 9    different from the national average or you might have some 

10    other level in mind, but that is the sort of thing that is 

11    involved.

12               Now, look, that surely cannot be right.

13               Suppose it were the case that --  I'm going to do 

14    this in terms of 50 again --  48 state had an undercount 

15    rate that was right on the national average and with 48 

16    states right on the national average one state had an 

17    undercount rate that was statistical significant and much 

18    bigger than the national average and the other state had an 

19    undercount rate that is statistical significant --  I mean 

20    statistical significantly different from the national 

21    average and in one state that undercount rate is bigger than 

22    the national average and in the other state its necessarily 

23    lower and let's suppose those undercount rates are large.

24               In that circumstance, what will adjustment do? 

25               Adjustment will not change the distributive 
                                                              1052

 1    shares of the 48 states that are right on the national 

 2    average, what adjustment will do is to correct the two 

 3    things that you obvious want to correct.  To insist that you 

 4    won't do that because a lot of states won't be affected is 

 5    an irrational thing to do.

 6               Now, the same thing is true if one takes the view 

 7    that one needs a majority of the states to be --  that 

 8    undercount rate statistical significantly different than the 

 9    national average before adjustment, because it's very easy 

10    to see that you could have a majority of the states with 

11    undercount rates that are right about the national average, 

12    those are the states for which adjustment won't affect, they 

13    won't be statistical significantly different from the 

14    national average and you will fail to correct the states 

15    that you should correct because you do not know that the 

16    states which it doesn't matter but correct are statistical 

17    significantly different than the national average.

18               In terms of Plaintiff's Exhibit I think it's 707, 

19    it's the undercount rates by state charts --

20         Q.    The one that is on the board now?

21         A.    Yes.

22         Q.    Yes, sir. 

23               THE WITNESS:  If I may go up, your Honor.  

24               THE COURT:  Sure.

25         Q.    What I'm saying is, look, there are a lot of 
                                                              1053

 1    states in the middle here that have undercount rates quite 

 2    close to the national average.  Those are states whose 

 3    adjustments, where adjustment won't affect the distributive 

 4    share by very much.  Those are also states in which you are 

 5    not sure that the undercount rate really is different from 

 6    the national average, not surprisingly, it's not very much 

 7    different.

 8               To say you need a majority in which you do get 

 9    statistical significant is to say that I will not correct 

10    the states in which something really serious is happening, 

11    both at the right-hand side and the left-hand side, I won't 

12    do that because there are a lot of states here that may not 

13    need correction when correction will do essentially nothing 

14    to those states.

15               If, in fact, one looks at which are the 18 states 

16    in which there is a statistical significant undercount, it 

17    is not surprising the ones at the two ends of this chart, 

18    the ones up near California and Arizona at the right and the 

19    ones down here, Rhode Island and Connecticut at the left, 

20    those are the ones where adjustment will matter and those 

21    are the ones where you know perfectly well that adjustment 

22    ought to matter.

23         Q.    Is it fair, in essence, to say, Dr. Fisher, that 

24    you disagree with the rationality of the statements at the 

25    bottom of 2-34 and the top of 2-35 of the Secretary's 
                                                              1054

 1    decision?

 2         A.    I do not believe that --  some of the those 

 3    statements are literally true.  I do not believe that those 

 4    statements can form the basis for a rational decision.

 5               MR. RIFKIND:  Before I move on to the next 

 6    subject, your Honor, I would like to now move into evidence 

 7    Plaintiff's Exhibit 44, which was the hypothesis test 

 8    prepared by Robert Fay on June 28, 1991.  

 9               MR. ZITCOV:  No objection.  

10               THE COURT:  44 is admitted.

11               (Plaintiff's Exhibit 44 marked for identification 

12    was received in evidence.) 

13    BY MR. RIFKIND: .

14         Q.    You testified earlier in the day about the loss 

15    function analyses carried out by the Bureau.

16               Do you recall that testimony, Dr. Fisher?

17         A.    Yes.

18         Q.    What is your view, from a professional point of 

19    view, from the way those loss functions were carried out?

20         A.    The Bureau --  it's very high.  The Bureau put a 

21    lot of effort into this, they spent a long time at it, it's 

22    not something they cobbled up at the last moment and the 

23    results seem to me to be entirely convincing.

24         Q.    You say it's not something they cobbled up at the 

25    last minute.
                                                              1055

 1               Do you know whether they had this under 

 2    consideration for some extensive period of time?

 3         A.    I do.

 4         Q.    Let me call your attention to Plaintiff's Exhibit 

 5    2 at 278, which is the National Academy report.

 6         A.    Yes.

 7         Q.    Are you familiar with that report, Dr. Fisher?

 8         A.    I'm familiar with this section of the report, I'm 

 9    certainly not familiar with the whole report.

10         Q.    You are referring to the section beginning at the 

11    bottom of loss functions --  the bottom of the pages of 278 

12    referred to as "loss functions and yardsticks"?

13         A.    Yes.

14         Q.    Is the work that the Census Bureau did in 

15    connection with loss function analysis consistent with the 

16    views expressed by the National Academy panel in this 

17    report?

18         A.    It is consistent with it.  The Bureau did not in 

19    every respect follow the advice of the National Academy.  

20    They didn't use the same loss functions, but they used loss 

21    functions in the same family.

22         Q.    In the same family?

23         A.    Yes.

24         Q.    And they used loss functions?

25         A.    Oh, yes.  They used loss functions.  You can't 
                                                              1056

 1    make decisions without using loss functions.

 2         Q.    Was the National Academy, as you understand it, 

 3    advocating the use of loss functions on an explicit basis?

 4         A.    Yes, they certainly were.

 5         Q.    And that's what the Bureau did?

 6         A.    Yes.

 7         Q.    Let me call your attention, then, to Plaintiff's 

 8    Exhibit 647.

 9               Can you tell me what Exhibit 647 is?

10         A.    Exhibit 647 is a May 23, 1990 memorandum for the 

11    undercount steering committee from John H. Thompson, chief 

12    statistics support division, "Subject:  Proposal for 

13    assessing the accuracy of the adjusted and unadjusted 

14    census."

15         Q.    Looking at the third paragraph of that and trying 

16    to avoid the Greek letters, what is this memorandum urging?

17         A.    This memorandum says, "We can express the 

18    accuracy statistically in terms of a loss function," and 

19    what the memorandum goes on to do is to discuss what loss 

20    functions might be used.

21               There is then a set of references for more 

22    background on the use of loss functions for census numbers. 

23               MR. RIFKIND:  Let me offer Plaintiff's Exhibit 

24    647 into evidence at this time, your Honor.  

25               MR. ZITCOV:  This is a so-called administrative 
                                                              1057

 1    document.  

 2               THE COURT:  647 is admitted.  

 3               (Plaintiff's Exhibit 647 marked for 

 4    identification was received in evidence.)

 5    BY MR. RIFKIND:

 6         Q.    Let me call your attention to Plaintiff's Exhibit 

 7    654.

 8               Do you have that, Dr. Fisher?

 9         A.    I do.

10         Q.    What is Plaintiff's Exhibit 654?

11         A.    Minutes of the undercount steering committee 

12    meeting, June 14, 1990.

13         Q.    Let me ask you to look at Bates 7857.

14               You see Objective Two on that page?

15         A.    I do.

16         Q.    And what do the minutes reflect at that point?

17         A.    The minutes reflect that John Thompson, the 

18    author of the exhibit we were just looking at, whose number 

19    I have forgotten --

20         Q.    Exhibit 647?

21         A.    Just let me make sure.

22               (Pause)

23               Yes, Plaintiff's Exhibit 647.

24               -- gave a walk-through of what is obviously 

25    Plaintiff's Exhibit 647.
                                                              1058

 1               There was then some discussion about how loss 

 2    functions should be used.  Concerns were expressed over 

 3    generating only one number loss function, since important 

 4    information and/or problems can be obscured by a single 

 5    number, and various other things were then discussed.

 6         Q.    Is that consistent with the Bureau using several 

 7    different loss functions?

 8         A.    Yes, it is.

 9         Q.    And that is what they did, is it not?

10         A.    It is.

11         Q.    Let me call your attention to Plaintiff's Exhibit 

12    657.  

13               MR. RIFKIND:  Before I do that, I move the 

14    admission of Plaintiff's Exhibit 654 in evidence.  

15               MR. ZITCOV:  No objection.  

16               THE COURT:  654 is admitted.

17               (Plaintiff's Exhibit 654 marked for 

18    identification was received in evidence.) 

19    BY MR. RIFKIND:

20         Q.    Would you look at 657, then, Professor Fisher.

21         A.    Okay.

22         Q.    What is Plaintiff's Exhibit 657?

23         A.    This is a January 31, 1991 memorandum to John 

24    Thompson, chief statistical support division from David 

25    Bateman, chief coverage studies and evaluation staff, 
                                                              1059

 1    "Subject:  Documentation of structure of errors in the PES."

 2         Q.    Looking at page Bates number 11473, the second 

 3    page of that document --

 4         A.    Yes.

 5         Q.    Towards the bottom of that page, what does that 

 6    tell you about the Bureau's plans on that page?

 7         A.    The last full paragraph on that page clearly 

 8    shows the fact that the Bureau was planning to use loss 

 9    functions.  

10               MR. RIFKIND:  I offer Plaintiff's Exhibit 657 in 

11    evidence.  

12               MR. ZITCOV:  No objection.  

13               THE COURT:  657 is admitted. 

14               (Plaintiff's Exhibit 657 marked for 

15    identification was received in evidence.) 

16    BY MR. RIFKIND:

17         Q.    Let me call your attention to 653, Plaintiff's 

18    Exhibit 653.

19               Do you see that?

20         A.    I do.

21         Q.    And what is Plaintiff's Exhibit 653?

22         A.    That is a May 28, 1991 memorandum for Ccris Mihm, 

23    general accounting office from Jim Dinwoody, chief program 

24    design staff, decennial planning division, DSE audit, and it 

25    says, "Attached is the final study plan for loss function 
                                                              1060

 1    evaluation."

 2         Q.    Turn the page.  Is there a memorand attached 

 3    thereto?

 4         A.    There is.  It is a memorandum dated April 8, 

 5    1991, memorandum for Henry F. Woltman, assistant division, 

 6    chief for census design, statistical support division, from 

 7    Richard A. Griffin, chief census design branch, statistical 

 8    support division, "Subject:  Final study plan for loss 

 9    function evaluation study.  

10               MR. RIFKIND:  I offer Plaintiff's Exhibit 653 in 

11    evidence at this time.  

12               MR. ZITCOV:  I object; improper foundation.  

13               THE COURT:  Overruled.  653 is admitted.

14               (Plaintiff's Exhibit 653 marked for 

15    identification was received in evidence.) 

16    BY MR. RIFKIND:

17         Q.    Let me call your attention to Plaintiff's Exhibit 

18    655 at this time, Dr. Fisher.

19               What is Plaintiff's Exhibit 655?

20         A.    This is a May 3, 1991 memorandum for undercount 

21    steering committee from Paula Schneider, chair, "Subject: 

22    June 1-2 decision meeting."

23               It sets out Ms. Schneider's suggestions for 

24    points to be discussed during the steering committee's 

25    deliberations in the then forthcoming weekend of June 1 
                                                              1061

 1    and 2.

 2         Q.    Let me ask you to look at the second page under 

 3    Guideline Two.

 4         A.    Item 2 under Guideline Two says, "Description of 

 5    results of national and state loss function analysis."

 6               And then after that under number 5 it says, 

 7    "Assessment of local loss function," and that is some more.  

 8               MR. RIFKIND:  Your Honor, I offer Plaintiff's 

 9    Exhibit 655.  

10               MR. ZITCOV:  No objection.  

11               THE COURT:  655 is received.

12               (Plaintiff's Exhibit 655 marked for 

13    identification was received in evidence.)

14               MR. RIFKIND:  I have been belaboring this point 

15    because of a hearsay objection earlier on when I asked 

16    whether any of these were put in the decision-making 

17    process, and since much of this was in the so-called 

18    administrative record, I take it we suppose that the 

19    Secretary was aware of it.           

20    BY MR. RIFKIND:

21         Q.    Let me call your attention now to PX 141.

22               What is Plaintiff's Exhibit 141?

23         A.    This is a July 11, 1991 memorandum for Robert A. 

24    Mosbacher, Secretary of Commerce, through Michael R. Darby, 

25    Under Secretary, Department of Commerce, Barbara Everitt 
                                                              1062

 1    Bryant, director Bureau of the Census from Robert E. Fay, 

 2    senior mathematical statistician, Bureau of the Census, 

 3    "Subject:  Comments regarding July 8 meeting."

 4         Q.    And looking at the last page of that document, 

 5    and in particular paragraph 7, what does that indicate to 

 6    you, Dr. Fisher?

 7         A.    Paragraph 7 is entitled, "Total error model and 

 8    loss function analysis," and goes on to describe material 

 9    related to the loss function analysis.

10               In particular it says, starting in the middle of 

11    the sentence, "Many statisticians, including myself, would 

12    regard the combination of these two analyses --  " I take 

13    that to be total error model and loss function analysis --  

14    "as a reasonable attempt to answer objectively the question 

15    of whether and to what degree adjustment realizes gains in 

16    the accuracy of the estimated population distribution."            

17               MR. RIFKIND:  I offer Plaintiff's Exhibit 141 in 

18    evidence.  

19               MR. ZITCOV:  Your Honor, we have no objection, 

20    but perhaps it would speed things up if the witness is 

21    simply going to read the exhibits, we would be happy to 

22    admit into evidence any of the exhibits with Bates stamp 

23    numbers from the so-called administrative record.  

24               MR. RIFKIND:  Thank you, I will refer to that in 

25    the future, but I have completed my list.  
                                                              1063

 1               THE COURT:  It is admitted.

 2               (Plaintiff's Exhibit 141 marked for 

 3    identification was received in evidence.) 

 4    BY MR. RIFKIND:

 5         Q.    You Professor Fisher, let me call your attention 

 6    to page 2-28 of the Secretary's decision.

 7         A.    Yes.

 8         Q.    In the middle of the page, the second full 

 9    paragraph, the third sentence, do you see the sentence 

10    beginning, "However, when one imperfect estimator is being 

11    compared to another," and what follows there?  Do you see 

12    that, sir?

13         A.    I do.

14         Q.    Perhaps it would be better if you read it into 

15    the record, Dr. Fisher, because I am going to ask you to 

16    comment on it in a minute.

17         A.    Where do you want me to start?

18         Q.    Just where I began.

19         A.    "However, when one imperfect estimator is being 

20    compared to another, it is more difficult to interpret the 

21    loss of one estimate.  The temptation is to call one 

22    estimator the 'truth' and measure loss against it.  But one 

23    is not measuring the loss against the truth.  This is simply 

24    measuring loss of one estimate against another.  There is no 

25    reason to think this analysis tells you anything about the 
                                                              1064

 1    truth.  In loss function analysis, it is critical to 

 2    consider the base being used for comparison -- losses are 

 3    measured relative to that base."

 4               The next paragraph goes on, "The loss function 

 5    analysis run by the Census Bureau asks whether the 

 6    enumeration or the production DSE was closer to the 'ideal' 

 7    DSE."  I am omitting the foot note.

 8               "This did not form a statistical test of whether 

 9    the production DSE are more or less accurate than the census 

10    counts.  It only calculates which set of numbers on average 

11    is closer to another," underlined in the original, "set of 

12    estimates, (the target population.)  These tests were simply 

13    not proper statistical tests to address the critical 

14    hypothesis about the distributive accuracy of the PES and 

15    the census enumeration."

16         Q.    Would you comment on that text?

17         A.    I think it reveals serious misunderstand of what 

18    the Census Bureau did and how you use loss function 

19    analysis.

20               It is perfectly true, of course, that when you 

21    don't know the truth you can't literally calculate the loss 

22    from using what you are using, you can only calculate the 

23    expected loss.  That, of course, is quite right.  But the 

24    notion that what the Census Bureau actually did has nothing 

25    to do with the distributive accuracy of the PES and the 
                                                              1065

 1    census enumeration is quite wrong.  Let me explain.

 2               The Census Bureau, in the course of its loss 

 3    function analysis, calculated a thousand simulations of the 

 4    population which they called target populations.  Target 

 5    populations were, that was an unfortunate choice of words, 

 6    because it makes it sound as though target is what you are 

 7    trying to hit, nevertheless, that is what the Census Bureau 

 8    called it and I guess I am stuck with that name.

 9               Much of the confusion about this in the 

10    Secretary's report comes from the apparent believe that the 

11    way loss function analysis is done is to generate the target 

12    population and then look to see whether the census or the 

13    production DSE is closer to the target.  That is not true.

14               The target populations are generated as it were 

15    intermediate calculating devices with which to assess the 

16    expected accuracy of the two different measures, in effect, 

17    which you needed as an intermediate calculation to calculate 

18    the loss functions.

19               I could give, I guess I could give an example as 

20    to how that sort of thing might work drawn from a different 

21    field or does work drawn from a different field, it's not 

22    totally different.

23         Q.    Go ahead.

24         A.    Well, I think if I may use --  I want to draw 

25    something, whatever it is called, the flip chart.  That's 
                                                              1066

 1    what I want to use.  

 2               THE COURT:  Go right ahead.

 3         Q.    A clean slate.

 4         A.    Well, I'm going to take us back to those by-gone 

 5    days of yesteryear when we were all in perhaps --  I think I 

 6    learned this perhaps in junior high, I'm not sure, where 

 7    almost everybody had that at one time or another.

 8               Here we have, let's say, the Washington Monument 

 9    and we would like to estimate the height of the Washington 

10    Monument and we are not going to do it by standing on top of 

11    the Washington Monument and dropping a tape measure down the 

12    side, we are going to do it by standing some distance away 

13    from the Washington Monument and, using some trigonometry, 

14    using some measurements on the bottom, and to make this 

15    interesting we are also going to estimate the height of the 

16    Lincoln Memorial, which is going to be the other end, and it 

17    has been pointed out to me in this example that the way I am 

18    about to draw it suggests that the observer is standing in 

19    the middle of the reflecting pool.  I am going to ignore 

20    that, it's not necessary.

21               Over here we have the Lincoln Memorial.  I will 

22    not attempt to illustrate this with the statute of Lincoln 

23    inside.  And the observer stands somewhere in between and 

24    the observer sights the top of the Washington Monument and 

25    measures the angle that that sighting takes with the ground 
                                                              1067

 1    and measures along the ground the distance to the foot of 

 2    the Washington Monument.

 3               Similarly, the observer takes a sighting at the 

 4    top of the Lincoln Memorial, measures the distance along the 

 5    ground to the bottom of the Lincoln Memorial and measures 

 6    the sighting that the observer takes from the top of the 

 7    Lincoln Memorial.  

 8               Now, this information is enough to calculate the 

 9    height of the Washington Monument and the height of the 

10    Lincoln Memorial.  If you know this angle, you know the 

11    distance to the base of the monument, then you go look up a 

12    trigonometric table by seeing what --  look up in a 

13    trigonometric table, you look up what the tangent of the 

14    angle is and that will enable you to figure out what the 

15    height is.

16               Now, the point of this example is that in doing 

17    that, you have had to calculate the distance that you are 

18    standing from the foot of the Washington Monument and the 

19    distance you are standing from the foot of the Lincoln 

20    Memorial, but you are not, in fact, going to be comparing 

21    the heights of the two in terms of which base is closer, the 

22    Washington Monument or the Lincoln Memorial, those are 

23    calculations that you need in order to be able to figure out 

24    what the heights are.

25               So in some general sense it is with the target 
                                                              1068

 1    populations.  Those are used as intermediate constructs like 

 2    the distances from the base to enable you to calculate the 

 3    expected distance that the census is from the truth and the 

 4    expected distance that the adjustment is from the truth.

 5         Q.    You bring it all back so well and with such 

 6    clarity.  

 7               THE COURT:  Still an ugly memory. 

 8               THE WITNESS:  Actually, your Honor, I never came 

 9    close to the actual numbers when I was doing it.

10         Q.    Well, the Secretary says, "There is no reason to 

11    think this analysis tells you anything about truth." 

12               Is that a sound proposition?

13         A.    No, it's not.  This analysis tells you precisely 

14    which one of these estimates is expected to be more 

15    accurate, that is, closer to the truth, closer being 

16    measured according to whichever yardstick you have chosen 

17    squared or not squared in terms of the loss function.

18         Q.    But it is telling you something about the truth?

19         A.    It certainly is.

20         Q.    Let me call your attention to page 2-28, the page 

21    we are on.

22               The very bottom of the page, the Secretary says, 

23    "Instead of comparing the production DSE that would be used, 

24    they compared the mean of a thousand draws from a model 

25    refelcting the statistical properties of the DSE.  This 
                                                              1069

 1    effectively eliminates the inaccuracies derived from using 

 2    one particular set of adjustments."  

 3               Would you comment on that statement, Dr. Fisher?

 4         A.    Yes.  It's got two problems with it.

 5               In the first place, it is not true that "this 

 6    effectively eliminates the inaccuracies." 

 7               What the Bureau did was quite properly to use the 

 8    mean as a way of better estimating the things they needed to 

 9    get in the loss function.

10               What they did was correct.

11               What is more, the Bureau also calculated --  what 

12    this appears to say they did not calculate --  namely, 

13    something they didn't use the mean, but used the actually 

14    production DSE, and that was also done before July 15.

15         Q.    Let me call your attention to Plaintiff's Exhibit 

16    641.

17               Can you tell me what 641 is, Dr. Fisher?

18         A.    This is a document dated, although it is a little 

19    hard to read, 7/9/91, "Note for Mark W. Plant, Deputy Under 

20    Secretary and Harry A. Scarr, executive assistant for 

21    statistical affairs, from Richard A. Griffin, chief census 

22    design branch, statistical support division, "Subject:  Loss 

23    functions."

24         Q.    And what does one infer from this document?  

25               MR. ZITCOV:  I object.
                                                              1070

 1         Q.    What does this document advise you?

 2         A.    This document, in the first place, on its face, 

 3    this document says that it's responding to a request about 

 4    using the actual production adjustment factors instead of 

 5    the simulations.

 6               Table 1 gives those results.  Table 1 is the 

 7    second page of the document.

 8               If you look at table 1, if you look at table 1, 

 9    what one sees, if this is done, and this appears to be what 

10    would be done if you didn't want to do what the Secretary 

11    says you shouldn't do, the answer turns out to be favorable 

12    to adjustment.

13               I am now looking at the bottom line of the fourth 

14    column from the right, the column is headed D1 equals and 

15    then there is some algebra.  That is in terms of the squared 

16    error loss function.  It gives a difference in favor of 

17    adjustment that is, in fact, more favorable to adjustment 

18    than is reported by the Bureau when the Bureau did what they 

19    did regularly.

20               A similar statement is true in terms of the 

21    absolute error loss function.  That is reported at the 

22    bottom of the last column.

23               Moreover, in terms of the Secretary's count of 

24    states with minus signs in these columns, if you do it this 

25    way for the squared error loss function, you get --  one, 
                                                              1071

 1    two, three, four, five minus signs instead of the 21 that 

 2    the Secretary emphasizes in his report, considerably more 

 3    favorable toward adjustment, as I recall, and for the 

 4    absolute error loss function you get --  I guess I don't 

 5    recall --  one, two, three, four, also five, which is more 

 6    favorable than the 11 that you would get from the absolute 

 7    error loss function that the Secretary mentions, but does 

 8    not get the count right from his report.

 9         Q.    I take it you are not suggesting that counting 

10    minus signs is the right way to do it?

11         A.    No, I'm certainly not.  But if you are counting 

12    the minus signs and you think it is the right way to do it, 

13    there is no question this is much more favorable toward 

14    adjustment.

15         Q.    Dr. Fisher, is it possible that if you would make 

16    error adjustment, that would make adjustment a bad thing?

17         A.    Of course adjustment is not a bad thing.  No 

18    matter what it is we thought we knew, it's always 

19    conceivable that lerking behind what we cannot know are 

20    errors distributed in such a perverse way as to make what we 

21    know untrue.

22               Butron Russel once said that we cannot tell that 

23    the world was not created ten minutes ago complete with its 

24    fossils and us with our memories, but no rational person 

25    would act on the basis that that is so.
                                                              1072

 1         Q.    So you can't eliminate the possibility that there 

 2    are errors unaccounted for here?

 3         A.    No.  Errors, errors unaccounted for that might 

 4    possibly be there are, in some sense, the things you have 

 5    not eliminated.

 6               It is conceivable that in the wilds of the 

 7    southwest deserts there is a very large unobserved city of 

 8    people who have refused to be counted, they have covered 

 9    themselves with an invisible shield.  That doesn't seem 

10    remarkably likely, but I suppose it could be true if they 

11    are very good at it.

12         Q.    And if that were true, what?

13         A.    If that were true, then if they lived in Arizona, 

14    actually, it would be quite a good --  Arizona or New 

15    Mexico, it would be quite a good idea to make the adjustment 

16    in Arizona and New Mexico, because Arizona and New Mexico 

17    have been undercounted anyway by more than the national 

18    average.

19               However, if that were to be true not in New 

20    Mexico or Arizona but in the middle of the white mountains 

21    of New Hampshire, then adjustment as proposed would make New 

22    Hampshire really totally wrong.

23         Q.    What orders of magnitude would you need to make 

24    this thing significantly wrong?

25         A.    Well, they would have --  let me tell you the two 
                                                              1073

 1    sorts --  let me get at this a slightly different way.

 2               You can ask the question, suppose there are 

 3    errors and they are randomly distributed and underestimated 

 4    the variance of what we are doing, how big would that have 

 5    to be in order to mean that adjustment was wrong when it 

 6    appears to be right, and the answer is the --  the answers 

 7    you get in favor of adjustment are pretty robust to 

 8    increases to the variance of the total error model.

 9               I cited one statistic about expected numbers of 

10    states today.  You have to multiply that variance by seven 

11    before that number turns over, seven to eight, and there are 

12    other similar statistics.

13               So just saying there are errors and they might be 

14    big is unlikely to make it different.

15               Secondly, however, it's possible that the errors 

16    involved are not random, that they are somehow distributed 

17    perversely, that is, that in just those states in which you 

18    think you have undercounted the most, it turns out that you 

19    have done things which are tending to onvercount, and in the 

20    states in which you think you undercounted the least there 

21    are large groups of people which you have missed.

22               It would both --  you would have to believe, in 

23    order to believe that adjustment is wrong, you would have to 

24    believe, one, that those errors are distributed perversely 

25    and, B, that they are big enough, big enough to dominate the 
                                                              1074

 1    movement toward correctness of correcting for the errors in 

 2    the census that we do, in fact, know about.  

 3               MR. RIFKIND:  Can I have just a second, your 

 4    Honor.  

 5    BY MR. RIFKIND:

 6         Q.    Let me call your attention, Dr. Fisher, to 

 7    Plaintiff's Exhibit 643.

 8               Can you tell us what that is?

 9         A.    This is a fax with the fax date June 28, 1991.  

10    It's a fax to Mark Plant, Deputy Under Secretary for 

11    economic affairs, Department of Commerce, from John 

12    Thompson, statistical support division, Bureau of the 

13    Census, Washington, D.C., and in handwriting it says, and it 

14    is true that it is, "memo showing loss functions at 1.5, two 

15    times variance."

16         Q.    What is the conclusion that is reached here?

17         A.    It is the case that at 1.5 and at two times 

18    variance, the loss function, which I believe here is the 

19    squared error loss function, still favors adjustment 

20               MR. RIFKIND:  Your Honor, I offer Plaintiff's 

21    Exhibit 643.  

22               THE COURT:  642 is admitted.  

23               MR. SITCOV:  On that one I would object to.  

24               THE COURT:  Part of the record in the 

25    administrative record.  
                                                              1075

 1               MR. ZITCOV:  That's right.  I'm sorry, your 

 2    Honor.

 3               (Plaintiff's Exhibit 643 marked for 

 4    identification was received in evidence.) 

 5               MR. RIFKIND:  I am told that I omitted to move 

 6    into evidence Plaintiff's Exhibit 641, a memorandum from 

 7    Plant and Scarr to Griffin, which the witness referred 

 8    earlier.  

 9               MR. ZITCOV:  That one I will object to because 

10    there is no foundation.  

11               THE COURT:  641 is admitted.

12               (Plaintiff's Exhibit 641 marked for 

13    identification was received in evidence.) 

14               MR. RIFKIND:  I have no further questions.  

15               THE COURT:  All right.  Let's take ten before we 

16    start cross.

17               (Recess) 

18               MR. RIFKIND:  Your Honor, just for aid in the 

19    future reading of the transcript, it occurred to me it might 

20    be useful to mark this drawing of Professor Fisher as an 

21    exhibit for identification only.  

22               We can make a small copy overnight.  I'm sure Mr. 

23    Sitcov and I could agree that it is a simulacrum of what Dr. 

24    Fisher has drawn.  So I propose we mark that for 

25    identification as PX 725.  
                                                              1076

 1               MR. ZITCOV:  If I knew what that was, I will 

 2    agree, but I have never heard of simulacrum.  

 3               MR. RIFKIND:  A likeness.  

 4               I think this will encumber the record, but I'm 

 5    sure that will do the trick.  

 6               MR. ZITCOV:  I don't know for what it is worth, 

 7    your Honor, but I do know that the Washington Monument is 

 8    555 and a half feet high.  

 9               THE COURT:  You are a statistical goldmine.  

10               MR. ZITCOV:  Actually I run past it every day. 

11    CROSS-EXAMINATION 

12    BY MR. ZITCOV:

13         Q.    Dr. Fisher, can you turn to Plaintiff's Exhibit 
                                                                 

14    645, please.

15               In particular, I would like you to turn to page 

16    7870.

17               The very first line --  do you have it?

18         A.    Yes.

19         Q.    The very first line on that page leads, "L sub CI 

20    equals (P sub CI minus P sub NN) squared" doen't it?

21         A.    Yes.

22         Q.    Do you know if that formula is correct?

23         A.    The formula is a definition of what he is going 

24    to mean by L sub CI, so either the question has no meaning 

25    or of course it's correct.
                                                              1077

 1         Q.    So it can only be correct?

 2         A.    At this stage, yes.

 3         Q.    Could you turn to Plaintiff's Exhibit 653, 

 4    please, and in particular I would like you to look at page 6 

 5    of that exhibit, which has some number on the right margin, 

 6    0536.

 7         A.    053-what?

 8         Q.    0536.

 9               Do you see that?

10         A.    I do.

11         Q.    At the bottom it says, "L sub CI -- "

12         A.    I'm sorry, we are not in the same place.

13         Q.    Do you have Plaintiff's Exhibit 653?

14         A.    653?  Yes, I do.

15         Q.    Can you turn to page 6.

16         A.    Yes.  That does not have in the margin the number 

17    you said it does.

18               Okay.  I'm back with you, yes.

19         Q.    Does it say at the bottom L sub CI equals loss 

20    incurred due to the use of the census numbers for graphic 

21    area?

22         A.    That's what it says.

23         Q.    Turn to the next page.

24               It says L equals (P sub CI minus P sub TI) 

25    squared.
                                                              1078

 1               Do you see that?

 2         A.    I do.

 3         Q.    Is the formula I just read you correct?

 4         A.    I think, again, this defines what the author is 

 5    going to mean by L sub CI and, in fact, he gives two 

 6    alternative versions, of which you read the first.

 7         Q.    So does that mean it is correct or it isn't 

 8    correct?

 9         A.    Again, either the question doesn't mean anything 

10    or is trivally correct.  There is nothing about it that 

11    could possibly be wrong.

12         Q.    Is random sample a technical term in statistics?

13         A.    Yes.

14         Q.    Do you recall you gave an example this morning of 

15    a sample based estimator for lawyers' heights?

16         A.    I do.

17         Q.    And in that sample based estimator was built on 

18    the assumption that your survey team took a random sample in 

19    the technical statistical sense of that term, is that right?

20         A.    Yes.

21               (Continued on the next page) 

22    

23    

24    

25    
                                                              1079

 1         Q.    Your sample based estimator is based on the 

 2    assumption that the sample survey team makes unbiased 

 3    measurements of height, is that right?

 4         A.    Actually, they make accurate measurements of 
                                                              

 5    height.  It doesn't matter.

 6         Q.    Suppose you wanted to know the average height of 

 7    lawyers for every single law firm in New York State with a 

 8    margin of error of one inch.  Are you with me?

 9         A.    I want to know, is this law firm by law firm?

10         Q.    Yes.

11         A.    Okay.

12         Q.    What do you need to sample several lawyers from 

13    each law firm?

14         A.    Yes.  I'm sorry, you need to know within one 

15    inch.  Yes, you would at least need to do that.  

16               THE COURT:  Every law firm?  

17               THE WITNESS:  That is what he wants me to know, 

18    your Honor, is the average height of lawyers in each law 

19    firm.  That is a lot of different statistics.  

20               THE COURT:  And every lawyer in the state?  

21               THE WITNESS:  Right.

22         Q.    Do you know if the PES sampled several people in 

23    every hamlet in the United States?

24         A.    It did not.

25         Q.    Is it your testimony that Dr. Bryant, that is, 
                                                              1080

 1    the census bureau director, had decided to recommend 

 2    adjustment to the Secretary if the PRODSE squared error loss 

 3    function analysis favored adjustment?

 4         A.    No, I don't recall saying that.

 5         Q.    Is it your testimony that Dr. Bryant intended to 

 6    recommend adjustment to the Secretary if the PRODSE absolute 

 7    error loss function analysis favored adjustment?  

 8               MR. RIFKIND:  I object to the question.  I don't 

 9    know what we are talking about intended.  All the witness 

10    testified to is what was or was not in some report.  

11               THE COURT:  I think it is a fair question.  You 

12    may continue.

13         A.    I didn't testify to that effect either.

14         Q.    Is that your belief?

15         A.    It is my belief that Dr. Bryant took into account 

16    the results of the loss function analysis, but I do not know 

17    that she made her decision for recommendation on either one 

18    of those criteria solely.

19         Q.    Is it your understanding that Dr. Bryant had 

20    decided to recommend adjustment to the Secretary if the 

21    UNWEPRS squared error loss function analysis favored 

22    adjustment?

23         A.    I have no reason to believe that she made her 

24    decision based solely on that criterion.  I do have some 

25    reason to believe that that was part of the criterion, as 
                                                              1081

 1    with the others.

 2         Q.    Would the same answer be true for the UNWEPRS 

 3    absolute error loss function?

 4         A.    Yes.

 5         Q.    Could you turn to Plaintiff's Exhibit 43.  Do you 

 6    still have it in front of you?

 7         A.    I have it.  You will have to give me a moment to 

 8    shift books.

 9         Q.    Would you turn to the page that is Bates stamped 

10    11293.

11         A.    Okay.

12         Q.    At the bottom it has the estimated risk 

13    difference in column 6 from adjustment, is that right?

14         A.    It has the value of the squared error loss 

15    function.  You can describe that as a risk difference.

16         Q.    Let's suppose that the estimated risk difference 

17    for California was 15 parts per million.  Okay?  And let's 

18    suppose that the estimated risk difference for the other 51 

19    areas combined was negative 10 parts per million.  In that 

20    situation adjustment would improve the distributive accuracy 

21    for California, is that right?

22         A.    No.  I'm sorry, you can't make that statement 

23    about individual states.  It's probably true, but you can't 

24    make that -- that's like counting the minus signs here.

25         Q.    So you wouldn't know whether adjustment in that 
                                                              1082

 1    circumstance improves the distributive accuracy of 

 2    California?

 3         A.    Not from those facts.  I'm not saying it can't be 

 4    known, but that wouldn't be the way to know it.

 5               MR. SITCOV:  Your Honor, I don't know if you have 

 6    a copy of the administrative record here in the courtroom.  

 7    I am assuming you don't.  

 8               Could I approach the witness stand?  

 9               THE COURT:  Sure.  

10               MR. SITCOV:  For the record, your Honor, I am 

11    showing the witness page 1128 of the administrative record.  

12               THE COURT:  All right. 

13               MR. RIFKIND:  That is Plaintiff's Exhibit 55.

14         Q.    Do you see the statement on that page that says, 

15    "Loss function analysis shows that there will be an accuracy 

16    gain in proportion of population for 29 states offset by 

17    possible inaccuracy in 21"?

18         A.    I see the sentence.

19         Q.    Do you consider that claim to be mistaken?

20         A.    Yes, I believe that's wrong.

21         Q.    Do you find it irrational for the census bureau 

22    director to have considered this claim?

23         A.    No.  That is a little different.  This is the 

24    same 29/21 count that is in the Secretary's report.  It is 

25    almost irresistable, once you have put out all those tables, 
                                                              1083

 1    to count the positive and negative minus signs.  And if you 

 2    are careful about it, it does yield some information.  

 3               But it does not literally mean what it says here 

 4    that it means, and I do consider it irrational to base a 

 5    decision on this if you are going to compare it -- I don't 

 6    think it is the best way to do it anyway.  Let me just leave 

 7    it that way.  I do think it is irrational to base the 

 8    decision on this, for several reasons.

 9         Q.    Is it irrational to consider?

10         A.    No.  How could it be irrational to think about 

11    it?

12         Q.    Could you please turn to page Bates number 903 in 

13    that same volume.  You will see there is a piece of paper 

14    sticking out on that page.

15         A.    Yes.

16         Q.    That is page 6 of the Undercount Steering 

17    Committee's main report.  I think it is about halfway down 

18    there is a statement talking about the loss function 

19    analysis:  "This analysis shows that for 39 of 50 states and 

20    most places of 100,000 or more" --

21         A.    I'm sorry.  I'm not with you.  I see things about 

22    that.  I don't see where you are reading from.  I have it, 

23    yes.

24         Q.    I will start again.  "This analysis shows that 

25    for 39 of 50 states and most places of 100,000 or more, the 
                                                              1084

 1    a adjusted population shares are closer to truth than are 

 2    the census population shares."  

 3               Do you consider this claim to be mistaken?

 4         A.    Yes, I do.

 5         Q.    Do you consider it irrational for the Undercount 

 6    Steering Committee to have considered the claim?

 7         A.    No.  As I said before, how can it be irrational 

 8    to think about it?  But it doesn't literally mean what they 

 9    say it means.

10         Q.    Would you turn to page 960 in the record.

11         A.    Okay.

12         Q.    If you turn to page 957, you will see that this 

13    is the appendix of the Undercount Steering Committee report.  

14    I believe it has been introduced into evidence earlier in 

15    the testimony, but I am not certain I could tell you exactly 

16    what number it is.  

17               About halfway down the page, the Undercount 

18    Steering Committee reports that they have done a new loss 

19    function analysis that changed the estimated number of of 

20    states whose shares were made worse from adjustment on the 

21    basis of loss function analysis.  Do you see that?

22         A.    No, actually I don't.  Which page are we on?

23         Q.    We are on page 960.

24         A.    It doesn't say quite what you said, I think.  It 

25    does say they have done a new analysis, but it doesn't say 
                                                              1085

 1    that there has been a change.  It you says what the new 

 2    analysis shows.

 3         Q.    Do you see the statement that says something to 

 4    the effect that the new analysis shows that 21 states are 

 5    made less accurate for an adjustment?

 6         A.    Yes.

 7         Q.    Do you consider this claim to be mistaken?

 8         A.    I do.

 9         Q.    Do you consider it to be irrational for the 

10    Undercount Steering Committee to have considered the claim?

11         A.    No.  I think there is some mild information in 

12    that number 21, but that is not literally a count of what 

13    they say it is a count of here.

14         Q.    So it was not irrational for Director Bryant to 

15    consider that, and it wasn't irrational for the Undercount 

16    Steering Committee to consider that, but it was irrational 

17    for the Secretary to consider that, is that right?

18         A.    Come, come.  That's not what I said.  I said it's 

19    not irrational to think about it, and I said I believe in 

20    the case of the Secretary that if you looked at it and you 

21    understood what it was, there is some information in it.  I 

22    said it would be irrational to base a decision upon it.  And 

23    it is in fact a mistake to think that that number 21 

24    literally is the number of states that would be made worse 

25    off.  It is not.
                                                              1086

 1         Q.    So it was not irrational for the Secretary to 

 2    consider that?

 3         A.    How can it be irrational for the Secretary to 

 4    consider anything related to the subject?  The question is 

 5    whether it is rational for the Secretary to take it 

 6    seriously to base the decision on.

 7         Q.    Could you turn to Plaintiff's Exhibit 684.

 8         A.    Are we done with the volume you gave me?

 9         Q.    Yes.

10         A.    Would you like to take it back?

11         Q.    Not particularly.

12         A.    I'll store it over here.

13         Q.    I have seen it before.  I know the ending.

14         A.    Did the Secretary?  I'm sorry.

15         Q.    Does Exhibit 684 show that for each state the 

16    error in its population share is likely to be reduced by 

17    adjustment?

18         A.    What?

19         Q.    Does Exhibit 684 show that for each state the 

20    error in its population share is likely to be reduced by 

21    adjustment?

22         A.    No.  It doesn't show anything of the sort.  You 

23    are not even close.  Are you sure you are looking at the 

24    right exhibit?

25         Q.    I think so.
                                                              1087

 1         A.    Then I will answer the question you have asked.  

 2    No, it certainly doesn't show that, and I don't see why 

 3    anyone would suppose that it did.

 4         Q.    Is there any evidence that you know of that shows 

 5    that for each state the error in its population share is 

 6    likely to be reduced by adjustment?

 7         A.    I know some evidence that that is true for some 

 8    of the states, but it doesn't come from Exhibit 684.

 9         Q.    Do you know of any evidence that shows that for 

10    every state --

11         A.    Oh, for every state?

12         Q.    Yes.

13         A.    It is not true that for every state it is likely 

14    to be reduced by adjustment.  There are some states for 

15    which it is quite unlikely.  You can't read any of that 

16    information out of -- I'm sorry.  There may be a problem 

17    here.  I may have a book that is badly marked, and I 

18    apologize.  

19               My 685 and 684 in this book are inadvertently the 

20    same exhibit, and they shouldn't be.  I am looking at one 

21    that says "Numerical Accuracy by State," which is why I keep 

22    wondering why you are asking me about distributive accuracy 

23    by state.  I apologize.  That is what accounts for my 

24    dumfounded answer to the question why should you suppose 

25    that it does.  
                                                              1088

 1               MR. RIFKIND:  684 is numerical accuracy.

 2         A.    684 is numerical accuracy.

 3         Q.    686.

 4         A.    Then I take back my apology.  It was insulting, 

 5    but I was right.  It was 686 that had to do with the 

 6    numerical accuracy.  684 doesn't have to do with it on its 

 7    face.

 8         Q.    Are we agreed that you are aware of no evidence 

 9    which shows it for each state the error in its population 

10    share is likely to be reduced by adjustment?  That is for 

11    each state.

12         A.    You mean every state?

13         Q.    Yes.

14         A.    I do not believe that it is true that for every 

15    state.  I know it is not true that if by "likely" you mean 

16    more probable than not, it is not true that for every state 

17    the distributive share is more likely than not to be 

18    improved by adjustment.  There are around eleven states for 

19    which that is not so.

20         Q.    Would you agree that we should reasonably require 

21    of an adjustment a procedure that each locality's error is 

22    more likely to be reduced than increased, that no locality 

23    will have good reason to believe otherwise, even post facto?

24         A.    No.

25         Q.    Do you consider that to be an irrational 
                                                              1089

 1    requirement?

 2         A.    I consider it to be an impossible requirement.

 3         Q.    Do you consider it to be an irrational 

 4    requirement?

 5         A.    Yes.

 6         Q.    Do you consider it to be an unreasonable 

 7    requirement?

 8         A.    I surely do.

 9         Q.    Could you turn to Plaintiff's Exhibit, I believe 

10    it is, 2.

11         A.    I have the wrong book.  Hang on.  Yes.

12         Q.    Plaintiff's Exhibit 2 is identified as the 

13    "Bicentennial Census, New Directions for Methodology 1990."  

14    I believe you testified about a description of loss function 

15    analysis that is contained on pages 278 through 280?

16         A.    I don't remember -- I certainly testified about 

17    the section beginning on page 278.  I am relatively familiar 

18    with it beyond page 280.

19         Q.    Could you turn to page 282.  Would you tell me if 

20    you believe that the following statement is irrational?

21         A.    Where are you?

22         Q.    I am about in the middle of the page, about 

23    two-thirds of the way down in the first full paragraph, the 

24    sentence that begins "One further point."  "Although each 

25    locality will know whether its count is higher or lower 
                                                              1090

 1    after adjustment, we can reasonably require of an adjustment 

 2    procedure that each locality's error is more likely to be 

 3    reduced than increased and that no locality will have good 

 4    reason to believe otherwise, even post facto."  

 5               Do you believe that that statement is irrational?

 6         A.    Yes.

 7         Q.    Is it unreasonable?

 8         A.    Yes.  I think if that is to be the criteria for 

 9    adjustment, I think it is fairly easy to show that there are 

10    cases in which you would plainly want to adjust and in which 

11    that criterion is not met, and in when I which I think every 

12    rational person would agree to that.

13         Q.    Disagreeing with that would exclude someone from 

14    the universe of rationality?

15         A.    Come, come.  What I am saying is if they thought 

16    about the examples and considered the question, I think they 

17    would have to agree with it.

18         Q.    Would you agree that significance levels of 5 

19    percent and 1 percent are generally used by statisticians in 

20    testing hypotheses?

21         A.    Yes.

22         Q.    And the 5 percent and 1 percent levels are 

23    generally accepted as the point at which the null hypothesis 

24    is rejected?

25         A.    Yes.
                                                              1091

 1         Q.    Do you agree --

 2         A.    It is not invariably true, but it is most often 

 3    true.

 4         Q.    Would you agree that less significant results 

 5    would be suggestive even if not probative?

 6         A.    Yes.

 7         Q.    Do you agree that attempting to use a weaker 

 8    significance level in litigation, where the burden of proof 

 9    is the preponderance of the evidence standard, reflects a 

10    flawed understanding of what significance levels really 

11    mean?

12         A.    Yes, but you would have to interpret that 

13    statement carefully.  I think I wrote that statement, as a 

14    matter of fact.  What I meant when I said it was that if you 

15    think you can get to the preponderance of the evidence 

16    standard by using a lower significance level, then you 

17    haven't understood about significance level.

18         Q.    In fact, the significance level below 50 percent 

19    does not correspond to a preponderance of the evidence 

20    standard, is that right?

21         A.    Yes, that is certainly true.

22         Q.    If I understand that, a significance level above 

23    50 percent does not correspond to the evidence level 

24    standard either, does it?

25         A.    That is true.  You can't do evidence standards 
                                                              1092

 1    with significance levels, or for that matter at all, as far 

 2    as I know.

 3         Q.    The computation of standard errors of coefficient 

 4    or the corresponding T statistics is a matter of 

 5    considerable importance, isn't it? 

 6         A.    It depends on the problem.  In multiple 

 7    regression, certainly.

 8         Q.    It is routinely done by all professionals, isn't 

 9    it?

10         A.    In regressions, yes.  That's what that passage 

11    was about.  In as a matter of fact, that is what that 

12    article you are quoting from was about.

13         Q.    An estimate that is at least two times its 

14    standard error is significant at the 5 percent level, is 

15    that right?

16         A.    As a general rule of thumb, that is true, 

17    although that depends on the sample size and the number of 

18    parameters estimated for that matter.

19         Q.    An estimate that is less than two times the 

20    standard error is not significant at the 5 percent level, is 

21    it?

22         A.    On a two-tail test, that is typically true.  On a 

23    one-tail test, it is not always true.

24         Q.    Do you agree with the statement that 

25    conventionally one gets ahold of something when an estimate 
                                                              1093

 1    is about five times its standard error?

 2         A.    No.  I know that appears in the transcript of my 

 3    testimony from the 1980 trial, and I told you at my 

 4    deposition that that is garbled.  I cannot have said five 

 5    times.  What I think I probably says is twice times, 

 6    misspeaking slightly, and the reporter got it as five times.

 7         Q.    So if it says in the transcript of your testimony 

 8    five times, that was wrong?

 9         A.    That's not what I said.  As I said, I told you 

10    that before.

11         Q.    So it should have read, "Conventionally, one gets 

12    ahold of something when an estimate is about two times its 

13    standard error," is that correct?

14         A.    Yes.

15         Q.    What that convention means is that if you have an 

16    estimate that is more than twice its standard error, the 

17    estimate reflects something that is real as opposed to 

18    something due just to chance, is that right?

19         A.    Speaking loosely, yes.

20         Q.    That is a convention among statisticians, isn't 

21    that right?

22         A.    It is not a convention whether that matches the 5 

23    percent standard.  That is a fact which is either true or 

24    not true in particular cases.  Generally, it is roughly 

25    true.  It is a convention to use the 5 percent standard.
                                                              1094

 1         Q.    According to that convention, then, if you have 

 2    an estimate that is less than twice the standard error, the 

 3    estimate reflects something that is just due to chance as 

 4    opposed to something that is real, is that right?

 5         A.    No, that is not right.

 6         Q.    You are aware, aren't you, that for a majority of 

 7    states, the estimated adjustment to the population share is 

 8    less than two times the standard error?  Aren't you?

 9         A.    You mean the estimated undercount is less than 

10    two times the undercount?

11         Q.    Yes.

12         A.    Yes.  I testified about that.  It is true for 33 

13    of the 51 states, counting the District of Columbia.

14         Q.    You will recall I took your deposition on April 

15    15th of this year?

16         A.    And 16th and 17th, if I recall.

17         Q.    It was a fun time.  Three days I will never 

18    forget.  

19               At your deposition you expressed certain opinions 

20    about adjustment, didn't you?

21         A.    I did.  

22               THE COURT:  And about lawyers, no doubt.  

23               MR. SITCOV:  Actually, he was quite gentle, your 

24    Honor, gentler than I might have been.

25         Q.    Those opinions that you expressed at your 
                                                              1095

 1    deposition were not based on any independent analysis you 

 2    had had performed of the 1990 PES or the census, isn't that 

 3    right?

 4         A.    I had not at that time done any analysis myself 

 5    with the data.  I had, of course, looked at the analysis of 

 6    the data done by the census bureau.

 7         Q.    You hadn't reviewed the bureau's loss function 

 8    analysis in specific detail, had you?

 9         A.    I'm sorry, you are saying had or had not?

10         Q.    Had not.

11         A.    I don't think that's right.  I had reviewed it.

12         Q.    In specific detail?

13         A.    I'm not sure what you mean by "specific detail," 

14    but I had certainly looked at those loss function documents.

15         Q.    Do you recall that I asked you the following 

16    questions and you gave these answers?  

17               MR. RIFKIND:  What page?

18               MR. SITCOV:  5.

19         Q.   "Q.    Have you reviewed the loss function 

20    analysis that the census bureau conducted?

21              "A.    In broad outline, certainly.  In very 

22    specific detail, no."  

23               Does that refresh your recollection that you had 

24    not, at the time of your deposition, reviewed the census 

25    bureau loss function analysis in specific detail?
                                                              1096

 1         A.    No.  It refreshes my recollection that I had not 

 2    reviewed it in very specific detail.

 3         Q.    You had not studied the specific details of the 

 4    calculations, had you?

 5         A.    I'm sorry.  Are you saying I have not or I had 

 6    not?

 7         Q.    You had not.

 8         A.    Well, I had studied some of them.  I certainly 

 9    had not studied all of them.

10         Q.    You hadn't checked any of the bureau's loss 

11    function computations, had you?

12         A.    That's true, I had not.

13         Q.    Prior to the time I took your deposition on April 

14    15, 1992, you had only reviewed a handful of documents 

15    regarding the bureau's loss function analysis, is that 

16    right?

17         A.    If a handful means less than five, I'm not sure.  

18    I had reviewed three principal documents that I can 

19    remember.

20         Q.    You had only spent about a total of one hour 

21    reviewing the principal documents regarding the loss 

22    function analysis, is that right?

23         A.    I think you will find that what I said was I 

24    spent about an hour reading those documents.  That is not 

25    quite the same thing.
                                                              1097

 1         Q.    I thought that is the question I asked, that you 

 2    had only spent a total of about one hour reviewing those 

 3    documents.

 4         A.    You asked what you asked and I answered what I 

 5    answered.  But I recall saying I'm a fast reader in that, 

 6    and I certainly meant I had only spent about an hour reading 

 7    it.  I did not mean to say, and I don't believe I did, that 

 8    that's all the amount of time I spent thinking bit.

 9         Q.    Do you recall testifying at your deposition that 

10    the national undercount was about 5 percent?

11         A.    Yes, I probably said that, and my memory failed.

12         Q.    Loss function analysis depends on the variances 

13    of the smooth adjustment factors, isn't that right?

14         A.    Among other things, yes, in the sense that the 

15    variance of the smooth adjustment factors is one of the 

16    things that eventually goes into the values of the loss 

17    functions, at least indirectly.

18         Q.    Those variances are the outputs from the 1990 

19    production smoothing model, is that right?

20         A.    Yes.

21         Q.    So the census bureau's loss function analysis 

22    depends on outputs from the production smoothing model?

23         A.    Oh, that is certainly right.

24         Q.    The the adjustment depends on those, so it must 

25    be true.
                                                              1098

 1         Q.    I'm sorry?

 2         A.    The adjustment depends on outputs from the 

 3    smoothing model, so surely the loss function does.

 4         Q.    The assumptions in the smoothing model can be 

 5    sufficiently far from the truth so that the results of the 

 6    model have unacceptably large degrees of error, correct?

 7         A.    That is conceivable.

 8         Q.    The variances computed from the smoothing model 

 9    depend on the assumptions of the model, don't they?

10         A.    Yes.

11         Q.    At the time of your deposition, you hadn't 

12    considered the impact of failures of the assumptions in the 

13    smoothing model on the variances computed from the smoothing 

14    model, had you?

15         A.    No.  But I did report to you a statistic as to 

16    whether if, when all was said and done, you multiplied the 

17    variance in the total error model, which is the way this was 

18    entered, when you multiplied that by a fairly large number, 

19    whether you changed the result.  So I had in that sense 

20    considered.

21         Q.    You were not aware of new studies that used the 

22    results of the 1990 production smoothing model to measure 

23    departures from truth of the assumptions in the model, isn't 

24    that right?

25         A.    That is true.  As I told you, I had not studied 
                                                              1099

 1    the smoothing model in any detail.

 2         Q.    You had not made any independent study of the 

 3    reliability of the census bureau's co-variance matrix for 

 4    the smoothing adjustment factors, had you?

 5         A.    That is true.

 6         Q.    You didn't know at your deposition of any P 

 7    studies that considered that question, did you?

 8         A.    I think I told you that I didn't know studies by 

 9    -- I didn't know which studies were catgerized as P studies.  

10    But basically the answer to that is yes, I did not.

11         Q.    The assumptions in the smoothing model are wrong, 

12    the variances computed from the smoothing model could be 

13    biased estimates of the true variances, isn't that right?

14         A.    They could be.

15         Q.    At your deposition you didn't know whether the 

16    variances computed from the 1990 production smoothing model 

17    were too small by as much as a factor of 2, did you?

18         A.    That's right, I did not.  I do know whether the 

19    results of loss function analysis depend on that, but I do 

20    not know that they are too small by a factor of 2.

21         Q.    You didn't even have a view as to how likely that 

22    was, did you?

23         A.    That's true, I didn't, and I don't.

24         Q.    When I took you deposition, you didn't know 

25    whether the bureau's best subsets procedure for variable 
                                                              1100

 1    selection vitiated any claim as to the statistical 

 2    properties of the model, did you?

 3         A.    I think that's true.

 4         Q.    Smoothing models have fundamental assumptions, 

 5    don't they?

 6         A.    As I told you then, Mr. Sitcov, and I tell you 

 7    now, I have just told you, I didn't study the smoothing 

 8    model, I hadn't studied the smoothing model.  If you want to 

 9    keep asking me questions about did I know things about the 

10    smoothing model, the answer is going to be no, I didn't.

11         Q.    Does that mean you didn't know the fundamental 

12    assumptions of the 1990 production smoothing model when I 

13    took your deposition?

14         A.    Not in any detail, I do not.

15         Q.    Do you know them today?

16         A.    No not in any more detail than I did then.

17         Q.    Smoothing models are not widely used in 

18    econometrics, are they?

19         A.    I believe that's true.

20         Q.    When I took your deposition, you were unable to 

21    refer me to a single recent paper in the American Economic 

22    Review or in any other major economic journal that used 

23    smoothing models, were you?

24         A.    Let me cut through this.  I don't recall your 

25    asking that.  But if you had asked it, I would have said no, 
                                                              1101

 1    I couldn't.

 2         Q.    You haven't personally used smoothing in your 

 3    work since you testified in Cuomo v. Baldridge, have you?  

 4               MR. RIFKIND:  I am going to object to this, after 

 5    letting it go for a bit, because it seems it has nothing to 

 6    do with the scope of the direct.  The witness didn't testify 

 7    about smoothing.  We had a witness last week who did.  This 

 8    witness made it quite clear in his opening remarks that he 

 9    was not going to discuss matters except the matters that he 

10    did discuss and didn't hold himself out to be expert in 

11    them.  

12               THE COURT:  I think it is well within the scope.  

13    Go ahead.

14         Q.    You haven't used smoothing in your work since you 

15    testified in Cuomo v. Baldridge, have you?

16         A.    Except for giving a smoothing problem in an 

17    econometrics exam, that's correct.

18         Q.    You first became aware of variance presmoothing 

19    in August of last year, is that right?

20         A.    Yes.

21         Q.    You hve never used variance presmoothing in your 

22    work, have you?

23         A.    That's true.

24         Q.    You have never seen variances presmoothed outside 

25    the context of this litigation, have you?
                                                              1102

 1         A.    I don't think I have ever seen a formal 

 2    discussion of variance presmoothing except for that.

 3         Q.    You haven't made any independent assessment of 

 4    the reliability of presmoothing, have you?

 5         A.    No, I have not.

 6         Q.    You don't know which evaluation study, if any, 

 7    demonstrated the reliability of the presmoothing, do you?

 8         A.    Well, I don't know which one.  I have some memory 

 9    that some of the tests, the pretests, the studies that 

10    talked about the tests of the census that were done on 

11    various localities may have discussed that, but I'm not 

12    sure.

13         Q.    I am talking now about the evaluation studies of 

14    the actual census and post-enumeration survey.

15         A.    No, I do not.  I'm sorry for riding over the end 

16    of the question.  No, I don't know.

17         Q.    The raw variances are presmoothed before they are 

18    fed into the smoothing model and used to smooth the 

19    adjustment factors, is that right?

20         A.    I'm sorry.  I missed the beginning of the 

21    question.

22         Q.    The raw variances are presmoothed before they are 

23    fed into the smoothing model and used to smooth the 

24    adjustment factors, is that right?

25         A.    Yes, I think that is right.
                                                              1103

 1         Q.    You haven't made any independent assessment of 

 2    the reliability of the raw variances, have you?

 3         A.    That's true.

 4         Q.    You don't know which evaluation study, if any, 

 5    demonstrated the reliability of the estimated raw variances, 

 6    do you?

 7         A.    That's true, I do not.  As I keep telling you, I 

 8    haven't studied this subject, and my answer to almost all of 

 9    these questions is going to be no, I don't know.

10         Q.    When I took your deposition on April 15th of this 

11    year, you hadn't reviewed the procedures used in evaluation 

12    follow-up, had you?  

13               THE COURT:  In evaluation of?  

14               MR. SITCOV:  Evaluation follow-up.

15         A.    I'm pretfy sure that's right.

16         Q.    You weren't sure whether your planned testimony 

17    would depend in any way on the reliability of the evaluation 

18    follow-up, is that right as well?

19         A.    I frankly don't remember, but it sounds likely.

20         Q.    Your testimony depends on the reliability of the 

21    bureau's total error model, is that right?

22         A.    In part.  Wait a minute.  I'm sorry.  Go ahead.  

23    I shouldn't produce so long a pause.  In part, colon, or 

24    something other than full stop, I have asked the question of 

25    whether the results that I have reported are sensitive to 
                                                              1104

 1    serious understatement in the variance or, for that matter, 

 2    generally in the squared bias of the bureau's total error 

 3    model.  And I know that the results are robust.  So in that 

 4    sense the testimony does not depend on that.

 5         Q.    I'm sorry.  Didn't hear the last part.

 6         A.    In that sense the testimony does not depend on 

 7    that.

 8         Q.    Do you recall that I asked you on page 327 of 

 9    your deposition, "Will your planned testimony, as far as you 

10    know, depend in any way on the reliability of the total 

11    error model?"  Your answer was, "As a logical matter, yes."

12         A.    But now you say in any way, and the answer is 

13    yes, in some ways it does, but not in every way.  And I know 

14    more about it now than I did then.

15         Q.    As of the time of your deposition, you had not 

16    reviewed the total error model, had you?

17         A.    That's true.

18         Q.    As of the time of you have deposition, you didn't 

19    know which particular sources of error were quantified in 

20    the total error model, did you?

21         A.    Basically, yes.

22         Q.    The total error model quantifies bias for each of 

23    the evaluation poststrata, is that right?

24         A.    I'm not sure I know.

25         Q.    After the biases were estimated for the 
                                                              1105

 1    evaluation poststrata --

 2         A.    I'm sorry.  You say the evaluation poststrata?

 3         Q.    Yes.

 4         A.    I'm sorry.  I misunderstood you.  Yes, it did.

 5         Q.    Do you know how many evaluation poststrata there 

 6    are?

 7         A.    I believe initially there were 13.

 8         Q.    Do you recall testifying at your deposition you 

 9    thought there were 156?

10         A.    There is an intermediate stage in which the 13 

11    are taken to 156.  So there is the use of 156 at one point.

12         Q.    But there aren't 156 evaluation poststrata?

13         A.    I think that is correct.

14         Q.    After the biases were estimated for the 

15    evaluation poststrata, they were shared down mathematically 

16    to the 1392 individual poststrata, is that right?

17         A.    Yes.

18         Q.    No attempt was made to estimate bias for each 

19    individual poststratum on the basis of data unique to that 

20    poststratum, is that right?

21         A.    You mean only on the basis of data?

22         Q.    Yes.

23         A.    Yes, I think that is true.

24         Q.    In state level biases were computed from the 

25    biases in the individual poststrata, is that right?
                                                              1106

 1         A.    Yes.

 2         Q.    The bureau's loss function analysis depends at 

 3    least to some extent on those estimated state level biases, 

 4    is that right?

 5         A.    Yes.

 6         Q.    You weren't prepared at your deposition to give 

 7    an expert opinion about the procedures used to share the 

 8    biases down from the evaluation poststrata to the individual 

 9    poststrata, were you?

10         A.    Probably not. 

11         Q.    In fact, you hadn't even studied the question of 

12    the allocation of the biases, had you?

13         A.    Well, I don't think that is fair.  I recall 

14    telling you what the two procedures that they used were.  I 

15    may have gotten one of them slightly wrong, but I do recall 

16    testimony about that at the deposition.

17         Q.    Do you recall testifying that you hadn't really 

18    studied the question of the allocation?

19         A.    That is certainly true.

20         Q.    Do you recall testifying at your deposition that 

21    you didn't know the assumptions the bureau used to allocate 

22    the biases?

23         A.    As I said, the deposition went on for three days, 

24    and I'm sure that somewhere in it there is definitely a 

25    passage that describes what those assumptions are.
                                                              1107

 1         Q.    It is your opinion, isn't it, that smoothing 

 2    introduces bias into the estimated adjustment factors?

 3         A.    Yes, I think that is likely, although I really 

 4    haven't studied the smoothing model and I am not absolutely 

 5    sure it is true.

 6         Q.    You testified at your deposition that estimates 

 7    were made of the bias?

 8         A.    Probably.

 9         Q.    It is a fact that the bias was not taken into 

10    account in the total error model, isn't it?

11         A.    I don't know.

12         Q.    You don't know if that bias was taken into 

13    account for total error model?

14         A.    I don't know.  The total error model makes an 

15    attempt to estimate the result of all biases in the 

16    adjustment estimates.  I don't know that they specifically 

17    take account of that one.  I don't know that they don't 

18    either.

19         Q.    In your opinion, is it reasonable to assume that 

20    whites who live in neighborhoods that have a big black 

21    population are less likely to be counted than whites who 

22    don't?

23         A.    It seems reasonable.

24         Q.    Do you agree that English literacy is related to 

25    undercounting?
                                                              1108

 1         A.    Yes, that seems quite reasonable.

 2         Q.    Do you agree that people who have trouble reading 

 3    and writing English are harder to count than people who are 

 4    literate?

 5         A.    Isn't that what I just said?

 6         Q.    If you did, then it trailed off at the end of 

 7    your answer.  I didn't hear it.

 8         A.    I don't know why that question is any different 

 9    from the one before.  Yes, I think that statement is likely 

10    to be true.  I don't, on the other hand, hold myself out as 

11    an expert on whether it is true.  It seems it makes sense.

12         Q.    Do you think it likely that Hispanics who live in 

13    areas with few minorities are likely to have considerably 

14    higher English literacy than Hispanics who live in areas 

15    where there are higher levels of minorities?

16         A.    I don't know.  I could see a reason for believing 

17    that, but we have had moved away from what I am prepared to 

18    say is likely.

19         Q.    If adjustment improves the accuracy of the share 

20    of the national population held by blacks but makes worse 

21    the proportional shares of the states, would that, in your 

22    opinion, be a good case for not adjusting the census?

23         A.    Can you tell me what it does with the substate 

24    level, or am I not permitted to know?

25         Q.    That is all you are permitted to know.
                                                              1109

 1         A.    You are stubborn about this.  You wouldn't tell 

 2    me at my deposition either.

 3         Q.    You must have been talking to my wife.

 4         A.    If all you know is that blacks are undercounted, 

 5    and I am not permitted to infer from that that the places 

 6    where they live are also undercounted within states, but if 

 7    all you know is that blacks tend to be undercounted -- as I 

 8    say, without that inference -- and you know that adjustment 

 9    will make the states' distributive shares less accurate, 

10    then I would not, I think, find that a reason to adjust.  

11               As I think I told you at the deposition, you have 

12    prevented me from knowing anything material that depends on 

13    whether blacks are undercounted.  That is, you have 

14    prevented me from knowing whether that means they are 

15    underrepresented or live in areas to which federal funds 

16    tend not to run.  Merely knowing that they are 

17    undercounted,, without any consequence to flow from that, 

18    doesn't seem to me to be a reason to adjust.  We are very 

19    far from what the truth is.

20         Q.    Do you recall testifying at deposition that in 

21    the hypothetical situation I just asked you about, there is 

22    a pretty good case for not adjusting?

23         A.    Isn't that what I just said?

24         Q.    I didn't hear that.  

25               THE COURT:  I did.
                                                              1110

 1         A.    I don't think that is any different from the 

 2    answer I just gave.  

 3               MR. SITCOV:  You are the only one that counts.

 4         Q.    If I understood your testimony earlier, it is 

 5    your opinion based on your expertise as an econometrician, 

 6    that bias in the census, in the sense of an undercount, is 

 7    more serious than variance.  Do I have that right?

 8         A.    With one exception.

 9         Q.    Okay.

10         A.    I mean I want to be as clear as I can about this.  

11    I did discuss that question, and I did testify as to why I 

12    thought that that was a plausible way to view it.  But I 

13    don't think I was testifying when I said that, and I think I 

14    would have made it clear about that, as an econometrician.  

15    I was testifying about what reasonable people might want to 

16    use for a loss function.

17         Q.    You have written no professional papers on this 

18    topic, have you?

19         A.    That's true.

20         Q.    When I deposed you a few weeks ago, you weren't 

21    aware of any papers in the economic literature on this 

22    topics, were you?

23         A.    The topic of census adjustment?

24         Q.    No.  On whether or not not bias in the census is 

25    more serious than variance.
                                                              1111

 1         A.    That, I take it, is -- I see.  Yes, that is true, 

 2    I don't.

 3         Q.    Is bias in the PES more serious than variance in 

 4    the PES?

 5         A.    Not in the same sense.

 6         Q.    Bias and variance are on a different scale, 

 7    aren't they?

 8         A.    One is measured squared and the other one isn't.  

 9    You would have to square one to make it comparable.  But 

10    that is not what we have been talking about.

11         Q.    In order to compare the two, we would have to 

12    compared bias squared with variance?

13         A.    There are several things you might do.  But in 

14    terms, if you are using a squared error loss function, then 

15    what you would do would be to compare bias squared with 

16    variance, yes.  If you were using a different loss function, 

17    you would have to make a different adjustment.

18         Q.    Then we would be talking about squared percentage 

19    points of error, wouldn't we?

20         A.    Yes.

21         Q.    In your opinion, squared percentage points of 

22    error from bias and from variance don't trade off one for 

23    one?

24         A.    Bias in the sense of undercount, yes.  By bias in 

25    the sense of statistical bias, not necessarily.
                                                              1112

 1         Q.    Are you familiar with the sure thing principle 

 2               THE COURT:  The what?

 3         Q.    The sure thing principle.

 4         A.    "Familiar" is a little strong.  I am pretty sure 

 5    I will recognize it if you remind me what it is and I know 

 6    that I have seen it, but I don't know what it means as you 

 7    present it to me here.

 8         Q.    According to the principles of decision theory, 

 9    is it correct that someone who violates the sure thing 

10    principle is behaving irrationally?

11         A.    Would it help if you told me what the sure thing 

12    principle is so I can give a really intelligent answer, or 

13    do you want me to guess?

14         Q.    So you just don't know what the sure thing 

15    principle is?

16         A.    I can probably conjure it up in my memory, and I 

17    believe the answer to the question you have asked me is yes.  

18    But one of the sure things I would like to know before I 

19    violate it is what I am talking about, and you are not 

20    allowing me to know.

21         Q.    I will make sure you know what the sure thing 

22    principle is before I am done.  

23               If adjustment makes an immense gain for accuracy 

24    in California but there is some loss for the rest of the 

25    country spread over a lot of places, is it your opinion that 
                                                              1113

 1    you might not want to adjust depending on how that loss is 

 2    spread?

 3         A.    I'm sorry.  Could you just do that again?

 4         Q.    Sure.  If adjustment makes an immense gain in the 

 5    accuracy for California but there is some loss for the rest 

 6    of the country spread over a lot of places, is it your 

 7    opinion that you might not want to adjust, depending on how 

 8    that loss is spread?

 9         A.    I can create an example in which the decision to 

10    adjust would depend on how the loss is spread, yes.  It 

11    would depend on the magnitudes involved.  Maybe I ought to 

12    just create the example.  

13               If we are measuring accuracy in terms of squared 

14    error loss, it is perfectly possible to produce an example 

15    in which if the loss is concentrated in one or two places, 

16    the squared error loss function turns against adjustment, 

17    whereas if it is spread over a lot of places it remains in 

18    favor of adjustment.

19         Q.    So the patterns of gains and losses among the 

20    states is important in deciding whether to adjust?

21         A.    No.  It is important in that example because that 

22    example suggests only that it affects the results of the 

23    overall loss function, the criterion on which I have decided 

24    to measure how losses and gains matter.  

25               In another sense, the answer to the question is 
                                                              1114

 1    yes, because once I have decided one or another loss 

 2    function -- in the example I am using the squared error loss 

 3    function -- I have implicitly decided on how the pattern is 

 4    going to matter.  If I were to use an absolute error loss 

 5    function, however, the pattern would not matter.

 6         Q.    I believe your testimony today, and I don't want 

 7    to misstate it, is that loss function analysis does not show 

 8    that the shares of all 50 states would be improved by 

 9    adjustment.  Is that right?

10         A.    That is certainly true, it does not.

11         Q.    Loss function analysis doesn't tell you for sure 

12    whether adjustment makes things more accurate, does it?

13         A.    For sure?

14         Q.    Yes.

15         A.    Nobody knows for sure.

16         Q.    Loss function analysis can't identify the 

17    particular states whose shares are made more accurate by 

18    adjustment, can it?

19         A.    Yes, I think that is true.  By the way, I am not 

20    saying you can't know anything about that question.  I am 

21    answering the question in terms of whether loss function 

22    analysis as such does that.

23         Q.    And loss function analysis can't identify the 

24    particular states whose shares are made worse by adjustment, 

25    can they?
                                                              1115

 1         A.    Same answer or answers perhaps I should say.

 2         Q.    Suppose the Court rules in plaintiff's favor and 

 3    orders an adjustment of the 1990 bicentennial census on the 

 4    basis of the 1990 PES.  Suppose further that the adjustment 

 5    causes Utah's proportional share of the population to be 

 6    decreased.  I would also like you to assume that Utah sues 

 7    the United States in district court in Utah to stop the 

 8    adjustment.  

 9               If I were to call you as an expert witness, if I 

10    understand you, you would not be able to testify under oath 

11    that Utah's proportional share of the population was made 

12    more accurate by adjustment?

13         A.    No, but I could make some statements as to 

14    whether it was probable.  To be able to testify under oath 

15    that it was made more accurate requires me to know the truth 

16    of what the share actually is.

17         Q.    I left you a binder on the table top in front of 

18    you.  Do you see it?  Not the huge fat one the plaintiffs 

19    gave you.

20         A.    I understand.  I am a putting that aside.

21         Q.    It is that skinny little one.  

22               MR. SITCOV:  Your Honor has one as well.  

23               MR. RIFKIND:  Could we have one?  

24               MR. SITCOV:  Yes.

25         Q.    Could you turn to Defendant's Exhibit 603.
                                                              1116

 1         A.    Only if you tell me where it is in the binder.  

 2    Never mind, I've got it.

 3         Q.    Could you turn to pages 3 and 4.

 4         A.    All right.

 5         Q.    This document purports to discuss loss function 

 6    analyses in the context of adjustment, does it not?

 7         A.    Yes, I think that's right.

 8         Q.    It is correct, isn't it, that the equations on 

 9    pages 3 and 4 of Defendant's Exhibit 603 are loss functions, 

10    aren't they?

11         A.    Yes.  Although, as I told you at my deposition, 

12    one of them is simply an approximation to one of the others.

13         Q.    There isn't a standard loss function that is 

14    recognized as the one that is best to use in considering the 

15    accuracy of census adjustments, is there?

16         A.    No, and there probably is no single one.  That is 

17    why the bureau and everybody who recommended it have used 

18    several.

19         Q.    So there is no single loss function that is 

20    generally recognized as being the most appropriate for 

21    adjustment, is that right?

22         A.    Didn't I just answer that?

23         Q.    I just wanted to be sure that that was your --

24         A.    I believe the answer to that is no, and that is 

25    why people use more than one.  
                                                              1117

 1               THE COURT:  The answer is yes, yes, there is no 

 2    single standard.  

 3               THE WITNESS:  I beg your pardon.  I see the 

 4    problem.

 5         A.    Your statement is correct, there is no single 

 6    way.

 7         Q.    Loss functions are supposed to tell you the costs 

 8    that one thinks are borne if you make a wrong decision, is 

 9    that right?

10         A.    Ideally.

11         Q.    You think that loss functions are thiigs which 

12    are supposed to describe the consequences of making errors, 

13    is that right?

14         A.    Again, ideally.  Unfortunately, some decisions 

15    involve consequences of considerable complexity, and it may 

16    not be possible to literally represent the cost.  That is 

17    why in that case one uses different loss functions.

18         Q.    The choice among sensible loss functions is not 

19    one that comes within the purview of the statistics, does 

20    it?

21         A.    I think that is true, although statisticians can 

22    assist you in considering the consequences of the choices 

23    you have made.

24         Q.    The choice of a loss function is supposed to 

25    represent the view of the decision-maker or the view of 
                                                              1118

 1    society as to the tradeoffs between different losses, is 

 2    that right?

 3         A.    Yes.

 4         Q.    So the choice among loss functions is in the 

 5    domain of philosophers, among others?

 6         A.    I believe what I said at my disposition is that 

 7    people who have studied this question included economists, 

 8    political scientists, decision theoriests, and philosophers. 

 9         Q.    Neither the squared error nor the absolute error 

10    loss function explicitly incorporates the costs of 

11    misapportioning Congressional seats, is that right?

12         A.    That is true.

13         Q.    To do that explicitly, you would want to consider 

14    the relative size of the population and the number of 

15    Congressional seats per voter, wouldn't you?

16         A.    You might.

17         Q.    Pardon me?

18         A.    You might.

19         Q.    The bureau's loss function analysis on 

20    Congressional seats didn't consider either factor, did it?

21         A.    No.  It merely considered how many Congressional 

22    seats would be misapportioned given the formula for 

23    apportionment that is in place.

24         Q.    There are other problems with the census, in your 

25    opinion, beyond the misapportionment of Congress, isn't that 
                                                              1119

 1    right?

 2         A.    I'm sorry?  I may have misunderstood you.

 3         Q.    There are other problems with the census, in your 

 4    opinion, beyond the misapportionment of Congress?

 5         A.    You mean there are other consequences from the 

 6    errors in the census?

 7         Q.    Okay.

 8         A.    Yes, I think that's right.

 9         Q.    One of those consequences is the misallocation of 

10    federal funds, is that right?

11         A.    Yes.

12         Q.    Neither squared error nor the absolute error loss 

13    function explicitly trades off the cost of misapportioning 

14    Congress against the cost of misallocating federal tax 

15    funds, isn't that right?

16         A.    That is true.

17         Q.    In fact, it may beyond our ability to develop a 

18    loss function which explicitly measures in any very helpful 

19    way the costs of misapportionment of Congressional seats and 

20    misallocation of federal funds, isn't that right?

21         A.    Yes.  That is in part because different federal 

22    programs operate in very different ways, and there isn't a 

23    single formula which is going to tell you the answer for 

24    each one.

25         Q.    You agree, do you not, that the bureau's loss 
                                                              1120

 1    function analysis gives estimated risk differences rather 

 2    than true risk differences?

 3         A.    Yes.

 4         Q.    These estimated risk differences incorporate some 

 5    amount of sampling error?

 6         A.    Yes.  That is why I did the hypothesis test that 

 7    I testified about with regard to the loss functions.

 8         Q.    Do you agree that sampling error in an estimated 

 9    risk difference can be a serious problem?

10         A.    In principle, surely.  In practice, it doesn't 

11    appear to have been in these.

12         Q.    You contest the hypothesis that the true risk 

13    difference is zero and the apparent risk difference is as 

14    large as it is due to sampling error, isn't that right?

15         A.    Yes.

16         Q.    When I took your deposition, you weren't sure 

17    about the appropriate way to address the issue of sampling 

18    error in the estimated risk difference, isn't that right?

19         A.    That's true, at that time I was not, although I 

20    believe I gave you some suggestions as to how to think about 

21    about this.

22         Q.    Would you take a look at Defendant's Exhibit 602 

23    that is in that binder.

24         A.    Under the previous tab perhaps?

25         Q.    Pardon me?
                                                              1121

 1         A.    Where in the binder is that?

 2         Q.    Under TAb 2.

 3         A.    Yes, I have it.  

 4               MR. SITCOV:  Your Honor, this was introduced in 

 5    evidence earlier today.  I can tell your clerk which of the 

 6    previous numbers it is.  

 7               THE COURT:  I have it.  What difference does it 

 8    make?

 9         Q.    Could you look at table 6.  I think you testified 

10    about that earlier today.  That is the prodsy table.

11         A.    Give me a minute to find it.  Yes.

12         Q.    This table reports the bureau's loss function 

13    analyses or at least some of them?

14         A.    Yes.

15         Q.    In your opinion, I think, according to your 

16    testimony today, these data favor adjustment?

17         A.    Yes.

18         Q.    When I took your deposition, you thought the 

19    evidence from this table was very suggestive but not 

20    probative, is that right?

21         A.    I don't remember that.

22         Q.    On page 299 of your deposition transcript, I 

23    asked you the following questions and you gave the following 

24    answers.

25              "Q.    Let's consider table 6 without any 
                                                              1122

 1    hypotheticals.  Is the evidence from table 6, in your 

 2    opinion, probative or suggestive or neither in favor of 

 3    adjustment?

 4              "A.    It's very suggestive in favor of 

 5    adjustment."  

 6               Does that refresh your recollection that you 

 7    thought that this table was suggestive?

 8         A.    Yes.  I undoubtedly said that because that was 

 9    before I had performed the hypothesis test on it.

10         Q.    So you now have changed your opinion based on 

11    tests that you did after I took your deposition?

12         A.    Well, sir, I told you the tests.  I described to 

13    you the tests that we would do.  I have now done the tests, 

14    and I find the suggestion which I found to be very 

15    suggestive is in fact extremely probative.  It's true.

16         Q.    Isn't it a fact that that is based on work you 

17    did after I deposed you?

18         A.    Yes.

19         Q.    In fact, when I deposed you, it was the case that 

20    you couldn't interpret the estimateked risk difference 

21    because you didn't know its standard error, did you?

22         A.    I don't know that I couldn't interpret it, but it 

23    was true that I didn't know the standard error.

24         Q.    But you had to know the standard error to do the 

25    tests you were just talking about, didn't you?
                                                              1123

 1         A.    Yes.

 2         Q.    That is something else you have learned since I 

 3    took your deposition, is that correct?

 4         A.    It is something I have learned since you took my 

 5    deposition.  It is not something else.  We are talking 

 6    basically about the same thing.

 7         Q.    Dr. Fisher, is it your view that everyone who 

 8    makes a decision is using a loss function either explicitly 

 9    or implicitly?

10         A.    If they are making an internally consistent 

11    decision with consistent criteria, the answer is yes.

12         Q.    You had to decide to wear a tie today, didn't 

13    you?

14         A.    Yes.

15         Q.    Did you use an explicit loss function?

16         A.    Actually, I did, you know.  You will be glad to 

17    know about this.  When I put on the tie this morning, I had 

18    two ties that I had picked out.  One of them I thought went 

19    better with this suit and shirt, but it turned out to have a 

20    spot on it, and I wondered -- I took into account the 

21    question of whether it would be worse for me to appear with 

22    a spot or with a tie that didn't quite match.  In that 

23    sense, yes, I did.  

24               THE COURT:  Are you married?  

25               THE WITNESS:  Yes, and my wife is not going to 
                                                              1124

 1    forgive me for coming with a tie with a spot on it, I'm 

 2    telling you.  That was one of the losses I took into 

 3    account.  

 4               THE COURT:  I would love to have heard your 

 5    marriage proposal.  

 6               THE WITNESS:  That, I think, your Honor, is 

 7    outside the scope.

 8         Q.    Dr. Fisher, could you take a look at Plaintiff's 

 9    Exhibit 686.

10         A.    We are back to the big books?

11         Q.    Yes, I think so.

12         A.    Hang on.  Yes.

13         Q.    Did you do these calculations yourself?

14         A.    No.  They were done under my supervision.

15         Q.    When were they done?

16         A.    I believe I saw these results for the first time 

17    in late April.

18         Q.    Did you personally supervise the calculations?

19         A.    It depends what you mean.  I discussed with the 

20    people who did them what calculations were to be done, and 

21    they went through -- I was assured that they had gone 

22    through Charles River Associates' standard data management 

23    procedure.  Then they were reported to me, and I discussed 

24    the results.  I did not myself stand over the computer and 

25    watch them being done, no.
                                                              1125

 1         Q.    Did you select the programs that were used?

 2         A.    Not personally, no.  That is what I meant by the 

 3    data management procedure.  However, there is a procedure 

 4    for doing that.

 5         Q.    Did you check any of the actual data outputs for 

 6    consistency, and so forth?

 7         A.    I think these are the data outputs.  Did I 

 8    personally check them?  No.  As I said, there is an 

 9    established procedure, and I made sure that they went 

10    through that procedure for checking.  I did not myself do 

11    the checking.

12         Q.    Is there an expert which explains how the 

13    probabilities in this chart were calculated?

14         A.    I'm sorry.  Is there a what?

15         Q.    Is there an exhibit which explains how the 

16    probabilities were calculated?

17         A.    A specific exhibit?

18         Q.    Yes.

19         A.    That gives the formulas?  No, but I can direct 

20    your attention -- no, I don't think there is an exhibit in 

21    evidence.  I don't know whether it has been marked for 

22    identification, but I can call your attention to a document 

23    which I know perfectly well you have, which is a member 

24    from, I believe, Bruce Spencer to a Ms. Muhlry, dated 

25    sometime around the beginning of this year or the end of 
                                                              1126

 1    last that describes how that is done. 

 2         Q.    That is the same process that was used to compute 

 3    the probabilities in this table?

 4         A.    If I have the right memorandum, I answer that as 

 5    yes.

 6         Q.    Can you explain to me how that is done?

 7         A.    In general terms, I can.  In specific terms, I 

 8    would like to use the memorandum.

 9         Q.    Tell me in general terms how it is done.

10         A.    The question is what is the probability that the 

11    error from the adjustment is bigger than the error from the 

12    census.  There are two pieces to this.  Let me begin by 

13    ignoring the fact that in the error from the adjustment 

14    there is both bias and sampling error.  

15               If there was just sampling error, then what one 

16    would do would be to ask how big would the error have to be 

17    in order to be bigger than the estimated undercount, and one 

18    could look up that probability from a table of a particular 

19    distribution.  I will come to the question of what 

20    distribution in a minute.  

21               What I have just described, however, is not quite 

22    accurate for how this is done because you must first correct 

23    for the bias in the estimated adjustment.  Having made that 

24    estimate and subtracted it off from both sides of what was 

25    really a double inequality, you then do the calculation I 
                                                              1127

 1    have just described.  

 2               These probabilities were calculated using the 

 3    normal distribution.  Earlier results in a published paper 

 4    by -- Spencer is one of the authors.  I don't remember.  I 

 5    think it may be Spencer and Muhlry, I don't remember at the 

 6    moment, that studied the results of various of the tests 

 7    done before the census, suggested that the exact 

 8    distribution really does not matter very much.

 9         Q.    Would you look at the bottom of the page of 686, 

10    under PRODSE.  It gives a source.  Can you tell me what the 

11    STPS-0.1.DAT file is?

12         A.    No.  That identifies a particular data file on 

13    the census tapes or received from the Bureau of the Census.

14         Q.    Right after that there was another data file 

15    identified.

16         A.    I am not going to know the rest of those either 

17    with any more specificity than that.  They are put there so 

18    somebody who has the tapes can figure out where it came 

19    from.

20         Q.    So you don't know what those data files are?

21         A.    No.

22         Q.    Could you turn to 698, please, Plaintiff's 

23    Exhibit 698.  Do you have it?

24         A.    I do.

25         Q.    Did you do the calculations that are represented 
                                                              1128

 1    on that?

 2         A.    Personally?

 3         Q.    Yes.

 4         A.    No.  They were also done by Charles River 

 5    Associates under my supervision in the same manner.

 6         Q.    When was the first time you saw the chart?

 7         A.    I have a slight problem with answering that.  Let 

 8    me explain what it is.  It is not really material.  One of 

 9    these charts, I can't remember which one, in the first form 

10    in which it was produced to you, had an error on it.  I 

11    don't think it was this one, but I'm not sure.  Mr. Rifkind 

12    is shaking his head no.  If it was not in one, then I have a 

13    leaner answer.  

14               I saw this a couple of weeks ago.  I don't 

15    remember exactly when.

16         Q.    Was it your idea to have this chart made?

17         A.    It was my idea to have the calculation done, and 

18    it was certainly my idea to put on the expressions in terms 

19    of probability so that it would be more understandable.  But 

20    the basic chart had been drawn up in more or less this form 

21    when I saw it first.

22         Q.    Did you personally supervise the calculations 

23    that went into this chart?

24         A.    Only in the same sense as before.

25         Q.    Is there an exhibit which explains how the 
                                                              1129

 1    standard deviation was calculated?

 2         A.    I don't believe there is.

 3         Q.    Could you explain how the standard deviation was 

 4    calculated?

 5         A.    Yes, I can.

 6         Q.    Would you please.

 7               THE COURT:  Mr. Sitcov, is it likely you will 

 8    finish today?  

 9               MR. SITCOV:  It depends when today ends, I 

10    suppose.  I guess I could.  Although, quite frankly, your 

11    Honor, I don't mean to cause the witness any problems, but 

12    we have had some question about a number of the exhibits 

13    that we objected to, and I might want to have an opportunity 

14    to talk to some of our advise sores to see if there are any 

15    further questions I might want to ask him.  

16               THE COURT:  That said, I have the impression that 

17    we ought to adjourn until tomorrow then.  

18               MR. SITCOV:  Okay.  

19               THE COURT:  We will resume tomorrow morning at 

20    9:30.  

21               (Adjourned to 9:30 a.m., May 19, 1992) 

22    

23

24

25
