                                                              386

 1    UNITED STATES DISTRICT COURT
      EASTERN DISTRICT OF NEW YORK
 2    --------------------------------x
      THE CITY OF NEW YORK, et al.,
 3    
                         Plaintiffs,          89 Civ. 3474 (JMcL)
 4    
                 v.                           
 5    
      UNITED STATES DEPARTMENT OF COMMERCE,
 6    et al., 
      
 7                       Defendants.
      ---------------------------------x
 8    CITY OF ATLANTA, et al.,
                     
 9                       Plaintiffs,
      
10                v.                          May 13, 1992
                                              9:30 a.m.
11    MOSBACHER, et al.,
      
12                       Defendants.
      ---------------------------------x
13    FLORIDA HOUSE OF REPRESENTATIVES, 
      et al.
14    
                         Plaintiffs,
15                v.                          __Civ.___(JMcL)
      
16    BARBARA H. FRANKLIN, Secretary of
      Commerce,
17    
                         Defendant.
18    --------------------------------x
      Before:
19    
                 HON. JOSEPH M. McLAUGHLIN,
20    
                                              Circuit Judge
21    

22

23

24

25
                                                              387

 1               (Trial continued)

 2               (In open court)

 3               THE COURT:  Good morning.  Mr. Sitcov, well- 

 4    oiled?  

 5               MR. SITCOV:  Well-greased, your Honor.
                                                       

 6    EUGENE P. ERICKSEN, 
                          

 7         having been previously sworn, resumed the 

 8         stand and continued to testify as follows: 

 9               THE WITNESS:  Good morning.  

10    CROSS-EXAMINATION CONTINUED 

11    BY MR. SITCOV:

12         Q.    Dr. Ericksen, when we were finished yesterday, I 

13    believe we were discussing Appendix G of Plaintiff's Exhibit 

14    195.  That is the loss function analyses that were done by 

15    Kirk Wolter and Beverly Causey.  

16               It is correct that you believe that that document 

17    shows an adjusted count would be superior to the census in 

18    terms of the proportional distribution of the population on 

19    a state-by-state and substate-by-substate basis, isn't that 

20    right?

21         A.    You are talking about -- sorry.  Are you 

22    referring to the year 1990?

23         Q.    Pardon me?

24         A.    Are you referring to the year 1990?

25         Q.    Yes.
                                                              388

 1         A.    What the analysis of Wolter and Causey does, 

 2    coupled with the analysis that went into our joint report, 

 3    that led to the conclusion that were a loss function 

 4    analysis calculated on the basis of 1990 data, which states 

 5    and similar areas to those that Wolter and Causey dealt 

 6    with, that such a loss function analysis would show that the 

 7    original enumeration would have greater error than the 

 8    results of the original enumeration coupled with the results 

 9    of the PES.

10               Since Wolter and Causey dealt with data that were 

11    collected from 1980, it does not provide correct evidence 

12    about 1990.  It simply provides a guide as to what we might 

13    have expected from 1990.

14               And so, essentially, what we were doing in our 

15    report on pages 20 and 21 was prophesizing what the results 

16    of the loss function analysis would show.

17         Q.    I see.  So the Walter and Causey paper didn't 

18    even use 1990 data, did it?

19         A.    No.  The Wolter and Causey paper was written 

20    during the decade of the 1980s, and it was written to 

21    provide guidance as to what we might expect from the 1990 

22    Census.

23         Q.    So this was data from the 1980 census?

24         A.    I believe that their data were from 1980.  I 

25    don't have that perfectly in mind, but I believe they did 
                                                              389

 1    use 1980 census data, yes.

 2         Q.    And neither you nor any of the other plaintiffs' 

 3    nominees to the Special Advisory Panel conducted a loss 

 4    function analysis using 1990 data before you submitted your 

 5    joint report to the Secretary, isn't that right?

 6         A.    That is correct.

 7         Q.    Now, another of the loss function analyses in the 

 8    Wolter Causey paper showed that the population shares of 

 9    each of the 50 states would be improved by adjustment, isn't 

10    that right?

11         A.    Again, I assume that you are talking about in 

12    1990.

13         Q.    That's right.

14         A.    The Wolter and Causey paper was not based on the 

15    1990 data, so the answer to your question is yes.

16         Q.    Well, the loss function analysis that you 

17    referred to on page 20 of the joint report that you 

18    coauthored doesn't show the population shares of each of the 

19    50 states would be improved by adjustment, does it?

20         A.    I could just answer it doesn't demonstrate it. 

21               Our analysis reported in this document, coupled 

22    with the results of Professor Tukey and Appendix H and the 

23    paper of Wolter and Causey Appendix G, predicted very likely 

24    that a loss function analysis conducted for the 50 states 

25    based on 1990 data would show that the use of the PES data 
                                                              390

 1    improved their estimate of the distribution population, and 

 2    it did.

 3         Q.    That's not my question.

 4               My question is:  The loss function analysis that 

 5    you referred to on page 20 of your joint report doesn't show 

 6    that the population shares of each of the fifty states would 

 7    be improved by adjustment, does it?

 8         A.    Our -- we were not discussing an empirical study 

 9    on page 20, so the answer to your question is yes.

10         Q.    In fact, it doesn't tell you how many states' 

11    shares would be made more accurate by adjustment, does it?

12         A.    Without data, how could we do such a thing?

13         Q.    So the answer to my question is yes?

14         A.    The answer to your question is yes.

15         Q.    And it doesn't tell you which state's shares 

16    would be made more accurate by adjustment, does it?

17         A.    No, it does not.

18         Q.    It is the case, isn't it, that loss function 

19    analysis doesn't enable you to identify any particular state 

20    that's made more or less accurate by adjustment, isn't that 

21    right?

22         A.    With certainty?

23         Q.    I'm just asking you, is that right?

24         A.    It does help you identify the states which are 

25    more likely to be made more accurate by adjustment, but loss 
                                                              391

 1    function analysis is an exercise in probabilities, and these 

 2    probabilities would never be a hundred percent, so the 

 3    answer to your question is not with certainty.

 4         Q.    Well, do you recall testifying at your deposition 

 5    in this case on page 303 at line 19:  

 6               "We don't, in fact, know any particular state 

 7    that was made more or less accurate, and loss functions are 

 8    generally intended to apply to the aggregate states, not the 

 9    individual state."

10         A.    I would like to check that passage, and then I'll 

11    answer your question.

12         Q.    Please feel free.  

13               MR. ZIMROTH:  Would you tell us what page that 

14    is?  

15               MR. SITCOV:  303, starting on line 19.

16         A.    Now, if your question is do I recall it, I 

17    certainly recall having read it in my deposition, that I 

18    think what I said in my deposition is about what I just said 

19    to you just now.

20         Q.    So it is correct then that with the loss function 

21    you don't know any particular state that was made more or 

22    less accurate by adjustment?

23         A.    Not with 100 percent certainty.

24         Q.    Okay.

25               THE COURT:  Is that no with a hundred percent 
                                                              392

 1    certainty?

 2               THE WITNESS:  Right.

 3               THE COURT:  Or is that it wasn't made more 

 4    accurate with a hundred percent certainty?

 5               THE WITNESS:  Well, the point is that with the 

 6    results of loss function analysis, you might say, for a 

 7    certain state, it looks very, very likely that it is made 

 8    more accurate.  But this is a sample estimate.  It's never 

 9    going to be a hundred percent.  

10    BY MR. SITCOV:

11         Q.    Now, the Undercount Steering Committee originally 

12    determined that loss function analysis showed that the 

13    population shares of 11 states would be made worse by 

14    adjustments, is that right?  

15               MR. ZIMROTH:  You want to refer him to a 

16    particular page?  

17               MR. SITCOV:  Well, I'm asking him if he knows.  

18    He hasn't said he doesn't know.

19         A.    I don't recall particular numbers for show.  

20    Sure.  I do know that there was a statement there that some 

21    would be more and some would be made less accurate with the 

22    adjustment.  But don't ask me to recall particular numbers.

23         Q.    Well, do you have Plaintiff's Exhibit 55 in front 

24    of you?

25               If you turn to page -- I'll leave off all the 
                                                              393

 1    zeros -- 1146, you'll see that there is a recommendation 

 2    from the Undercount Steering Committee.

 3         A.    Page 1146 has a title "Technical Assessment."

 4         Q.    Right.

 5         A.    Rather than a recommendation.

 6         Q.    Okay.  "Technical Assessment."  Now, could you 

 7    turn to page 6 of that document.

 8               Now, in the third full paragraph down the one 

 9    that begins "loss functions," doesn't that paragraph say 

10    that 39 states would have their population shares made 

11    closer to the truth than 11 states would have their 

12    population shares made worse after adjustment?

13         A.    That is what it says.

14         Q.    And then the Undercount Steering Committee took a 

15    second loss function analysis in connection with the 

16    adjustment decision, isn't that right?

17         A.    I believe that's correct.

18         Q.    And this time the Undercount Steering Committee 

19    concluded that the population shares of 21 states would be 

20    made worse by adjustment, isn't that right?

21         A.    Like I said before, I don't recall the exact 

22    numbers.  I do know that I did at least that one analysis.  

23    But was that number correct, I don't recall.

24         Q.    Does 21 sound about right?

25         A.    It could be.
                                                              394

 1         Q.    And in this scenario as well, that is, the second 

 2    loss function that the Undercount Steering Committee did, 

 3    you couldn't identify which state's proportional shares of 

 4    the population were made better by adjustment and which ones 

 5    were made worse, isn't that right?

 6         A.    Well, again, the difference is that I would say 

 7    that these things should be estimated within terms of 

 8    probability being made better or worse.  And when we are 

 9    talking about probabilities, it isn't then saying that you 

10    know which ones they are.

11         Q.    But you don't know which ones they are?

12         A.    In the way which I defined no knowing, the answer 

13    is no.

14         Q.    The true population of the United States is 

15    unknown, isn't it?

16         A.    We do not know exactly what the true population 

17    of the United States is, that's true, right.

18         Q.    And it's not possible to determine the true 

19    population of any state, is it?

20         A.    We can only estimate it, and the only estimate 

21    that we could come up with would obviously have uncertainty.

22         Q.    So is your answer to my question yes, it is not 

23    possible to determine the true population of any state?

24         A.    With complete certainty?

25         Q.    Yes.
                                                              395

 1         A.    Whether answering yes or no, I will simply say it 

 2    is not possible to know with 100 percent certainty what the 

 3    true population is of any state.

 4         Q.    But you feel almost a hundred percent sure you 

 5    know the true population of any state?

 6         A.    I'm not in a position to quantify it.  We don't 

 7    know for a certainty the population of any state.

 8         Q.    So in order for the Census Bureau to undertake a 

 9    loss function, it had to create a true population, didn't 

10    it?

11         A.    It had to create a model for a true population.

12         Q.    And this so-called true population was created by 

13    using data from the PES and the evaluations of the PES; that 

14    is the total error amount, isn't that right?

15         A.    That is correct.

16         Q.    Now, if the bias in the production dual system 

17    estimate were understated in the total error model, that 

18    could affect the results of the loss function analysis, 

19    couldn't it?

20         A.    It could.

21         Q.    Dr. Ericksen, I want you to consider loss 

22    function analysis for the 50 states and the District of 

23    Columbia. 

24               Now, in order to interpret the results of a loss 

25    function analysis for the 50 states and the District, you 
                                                              396

 1    need to know the pattern of gains and losses and not just 

 2    the total across the 50 states and the District, isn't is 

 3    that right?

 4         A.    There are various different interpretations you 

 5    would make of the results of the loss function analysis.  

 6    For some interpretations, the answer would be yes; for other 

 7    interpretations, the answer would be no.

 8         Q.    To determine whether or not to adjust the 1990 

 9    Decennial Census, you would want to know the pattern across 

10    the 50 states, wouldn't you?

11         A.    I'm not sure exactly what you mean by "the 

12    pattern."  

13               What would be mainly concerned with the results 

14    of the loss function would be to know an aggregate whether 

15    the errors from one set of estimates are larger than the 

16    errors from another set of estimates.

17         Q.    Do you recall that when I took your deposition, I 

18    showed you the Bureau's loss function analysis report, and 

19    we considered changing some of the numbers in the loss 

20    function analysis report, and I asked you what if it came 

21    out that California's number in this loss function report 

22    was something other than it was, and that the total of the 

23    50 states -- the 50 states and the District split the 

24    remaining difference.

25               And I asked you if that would change your opinion 
                                                              397

 1    about loss function.

 2               And you said, on page 332 of your deposition:      

 3               "It would depend on the pattern in the other 

 4    states."    

 5         A.    Yes, I think that what I was trying to explain to 

 6    you was that if you were to change the results of the loss 

 7    function, what I would really want to do is to assess the 

 8    entire pattern of change in the loss function to understand 

 9    why the numbers were changed, what the assumptions were 

10    changed to make this difference occur, what it was about the 

11    data, that because of the pattern in the data that were used 

12    in the loss function, to cause this result.  

13               And it is a sufficiently complicated analysis 

14    that when you asked me about it in my deposition, you said 

15    what particular pattern would make a fair adjustment, that 

16    it was too complex for me to answer just sitting there in 

17    the deposition.

18         Q.    Well, you are an expert in loss functions, aren't 

19    you?

20         A.    Within the field of statistics I do not consider 

21    myself to be a particular expert in loss functions, no.

22         Q.    Well, you wrote a report with Ericksen -- with 

23    Estrada, Tukey and Wolter in which you recommended favored 

24    adjustment in the use of loss functions, didn't you?

25         A.    It was a joint report with contributions by many 
                                                              398

 1    people.  And together this supplemented our recommendation.

 2         Q.    So you were not able to tell me, when I took your 

 3    deposition, what pattern would make a fair adjustment?

 4         A.    That right.

 5         Q.    And you couldn't tell me what pattern would make 

 6    a disfavored adjustment, is that right?

 7         A.    Not sitting there that moment, no.

 8         Q.    Now, in order to decide whether the results from 

 9    loss financial analysis favored adjustment you need to make 

10    a rigorous assessment of how the calculations are done, is 

11    that right?

12         A.    Yes.

13         Q.    And when you submitted your report with Estrada, 

14    Tukey and Wolter to the Secretary on June 21st, 1990, you 

15    didn't have the opportunity to review the results pertaining 

16    to loss functions using data from the 1990 Census and PES, 

17    did you?

18         A.    We had not seen the Census Bureau's results.  

19    Therefore, all we could do was write the prediction that we 

20    did based on -- on pages 20 and 21.

21         Q.    And prior to July 15th, 1991, you had not made a 

22    rigorous assessment of the Bureau's loss function 

23    calculations.

24         A.    That is correct.

25         Q.    Would you take a look at Defendant's Exhibit 107. 
                                                              399

 1               MR. SITCOV:  That has already been entered into 

 2    evidence, your Honor, as Plaintiff's Exhibit 540 A, 16 A.  

 3    It is in the so-called administrative record at bates stamp 

 4    number 0006074.

 5         Q.    Do you have it, Dr. Ericksen?

 6         A.    Yes, I do.

 7         Q.    You are familiar with this document, aren't you?

 8         A.    I'm knowledgeable with the results of this 

 9    document, yes.

10         Q.    And this is the report of loss function analyses 

11    the Census Bureau conducted in connection with the decision 

12    whether it would adjust the 1990 Decennial Census, isn't it?

13         A.    That's what it is.

14         Q.    Dr. Ericksen, I'd like you to take a look at 

15    Table 3, which is in the back of this document, and it's at 

16    bates stamp number -- without the zeros in front -- 6,087.           

17    And it is entitled, "Table of Loss Function Comparisons 

18    (UNWEPRS)" --

19         A.    I don't have any bates numbers on my copy. 

20               THE COURT:  I have the table, if you want it.

21         A.    Yes, the tables aren't clear.  Why don't you come 

22    and show me which one you have in mind.  It doesn't have any 

23    table marks too.

24               (Pause)

25    BY MR. SITCOV:
                                                              400

 1         Q.    Do you recall testifying about this table at your 

 2    deposition?

 3         A.    I recall answering your questions about this 

 4    table at the deposition, yes.

 5         Q.    And you remember that I asked you if in your 

 6    mind, in your opinion as an expert in this case, this table 

 7    favored adjustment?

 8         A.    I do recall that.

 9         Q.    Do you remember that you said that you couldn't 

10    tell me without further consideration?

11         A.    Don't recall that this moment, but if it's in 

12    there, I'm sure that's what I said.  

13               MR. ZIMROTH:  May we have the page reference, Mr. 

14    Sitcov?  

15               MR. SITCOV:  329.  

16    BY MR. SITCOV:

17         Q.    Do you recall that I -- I'm thinking now of my 

18    intent on page 329.  

19               MR. ZIMROTH:  329?  

20               MR. SITCOV:  329.

21               THE WITNESS:  Let me find the place.

22               MR. SITCOV:  Okay.

23         A.    What I actually said in the deposition was that I 

24    couldn't recall whether this was a chart that's the actual 

25    chart --
                                                              401

 1         Q.    I don't think there's a question pending.  

 2               My question is:  Do you recall me asking this 

 3    question and you giving this answer, Line 10:

 4               "Q.    Okay.  Do you have the results of this 

 5    chart in your mind?  In your opinion as an expert, does this 

 6    case favor adjustment?

 7               "A.    The question would require further 

 8    consideration unless you can help me remember the 

 9    conclusions that the Census Bureau derived from this chart 

10    so I can remember if and when I asked you to look at this 

11    chart in the past.  

12               "Q.   Does that refresh your recollection that 

13    you said that it would require further consideration before 

14    you could tell me whether the results of this chart favored 

15    adjustment?"

16               MR. ZIMROTH:  Your Honor, that question and 

17    answer is in the context that I think the witness ought to 

18    be able --

19               THE COURT:  I heard you so far.  

20               MR. ZIMROTH:  -- which I think the witness ought 

21    to be able to answer as he began to answer earlier before 

22    Mr. Sitcov cut him off.

23               THE COURT:  Look, answer as you are comfortable.

24               THE WITNESS:  Look, the point is that tables on 

25    loss function analysis arrived in my office some time after 
                                                              402

 1    I submitted my report.  

 2               There was a series of tables, as I recall, six or 

 3    eight tables, and when you handed me this particular table, 

 4    I wasn't sure that this was the one that I actually based my 

 5    conclusions on.  And why this table was also in there with 

 6    all the others, I could not remember at that time.  

 7               I certainly was aware of the fact that the 

 8    Census Bureau had decided on the basis of their loss 

 9    function analysis as a whole that the adjusted numbers were 

10    more accurate than the unadjusted numbers.

11         Q.    Well, do you recall that after I asked you that 

12    question, you conferred with your attorneys, and then you 

13    were able to tell me whether or not you thought that chart 

14    favored adjustment?

15         A.    I don't have any particular recollection of that.

16         Q.    Well, would you turn to page 330.  Do you see at 

17    line 6 it says, "Witness confers with attorneys," that I 

18    made a statement about that?

19               Then you asked to have the question reread.  And 

20    then you said:  

21               "Looking just at the numbers in this chart, I 

22    have not attempted to relate the numbers in this chart to 

23    any commentary in any document.  The sum of losses due to, 

24    attached to the original enumeration by the squared error 

25    loss function appear to be larger than the sum of losses due 
                                                              403

 1    to this series of adjusted estimates, so that would favor.  

 2               "Similarly, the sum of losses by the absolute 

 3    difference criterion for the original enumeration appear to 

 4    be greater than the sum of losses due to the adjusted series 

 5    of estimates.  So in both instances, just looking at these 

 6    numbers, they appear to support a decision to adjust."  

 7               Does that refresh your recollection that after 

 8    you conferred with your attorneys you were able to tell me 

 9    that this chart favored adjustment?

10         A.    This is the question and answer that occurred.

11         Q.    No, I'm asking, does it refresh your 

12    recollection?

13         A.    Yes.

14         Q.    Now, looking in column 4 in that table, do you 

15    see it?

16         A.    I do.

17         Q.    The total in column 4 is an estimated risk, isn't 

18    it?

19         A.    That's one term you could use for it.

20         Q.    And if we had to double the entries in the 

21    variance/co-variance matrix for the smooth adjustment 

22    factors to make it a more realistic estimate of the sampling 

23    error, you wouldn't have been able to tell me the impact on 

24    the total in column 4, isn't that right?

25         A.    I have not done such analysis, so I would not be 
                                                              404

 1    able to tell you the impact without any numbers in Volume  

 2    4.

 3         Q.    That's because you don't know the formula for 

 4    estimated risk, isn't it?

 5         A.    As I sit here today, I could not give you the 

 6    formula for how to double the variance/co-variance.  Having 

 7    looked at this column, that's true.

 8         Q.    In fact, you don't know whether you ever knew the 

 9    formula, isn't that right?

10         A.    I don't recall whether I had known that or not, 

11    no.

12         Q.    Now, the total in column 5 is an estimated risk, 

13    isn't it?

14         A.    You could refer to it as that.

15         Q.    And the total in column 6 is an estimated risk 

16    difference, isn't it?

17         A.    You could refer to it as that.

18         Q.    And this estimated risk difference is estimating 

19    a real risk difference, isn't it?

20         A.    You could refer to it as that, yes.

21         Q.    And the total in column 6 is affected by sampling 

22    error, isn't it?

23         A.    It's an estimate based on a sample, so it would 

24    be affected by sampling error, yes.

25               (Continued on next page) 
                                                              405

 1         Q.    In fact, when I took your deposition, you didn't 

 2    know how you would calculate a standard error for the total 

 3    in column 6 and you couldn't tell me how to do it, isn't 

 4    that right?

 5         A.    That's right.  This whole report was given to me 

 6    sometime in July, after July 15.  I was not in a position to 

 7    confer with the people at the Census Bureau who had done 

 8    this analysis.  

 9               The calculation of the sampling error for a 

10    complex statistic like this would be highly complex.  So I 

11    don't know how they did it.  It didn't have the information 

12    in this report that would tell me how to do it.  Without 

13    being able to find out what the Census Bureau had done, I 

14    certainly wasn't in a position to make the calculation 

15    myself.

16         Q.    In fact, you the didn't even know whether or not 
                                                                  

17    someone else could have computed that standard error, did 

18    you?

19         A.    Without the information that the Census Bureau 

20    had, I wasn't in any position to make such an assessment.  

21    You recall that I was not able to find out what the Census 

22    Bureau had done after July 15.

23         Q.    Now, the total in column 6 might differ from zero 

24    just due to sampling error, isn't that right?

25         A.    Any estimate based on sample has that possibility 
                                                              406

 1    however low a probability it may be.

 2         Q.    But you hadn't specifically considered the 

 3    question at the time I took your deposition, isn't that 

 4    right?

 5         A.    No, I had not.

 6         Q.    And when I took your deposition, you testified 

 7    you were not in a position to give a practically informed 

 8    answer to the question whether the difference between the 

 9    total in column 6 and zero could be due to sampling error.  

10    Do you remember?

11         A.    A practically informed answer based on data?  No, 

12    because I had not made such a calculation nor seen the 

13    results of such a calculation.

14         Q.    Different choices in smoothing models could 

15    result in different undercount estimates for any particular 

16    place, isn't that right?

17         A.    That is correct.

18         Q.    So the choice of a smoothing model has an impact 

19    on the estimated uncounted rate, isn't that right?

20         A.    Sure.  Any time you could have a difference, the 

21    difference may be very small, but certainly will have some 

22    difference.

23         Q.    And uncertainty about which smoothing model to 

24    use isn't reflected in the standard error, isn't that right?

25         A.    Uncertainty about which smoothing model to use is 
                                                              407

 1    not reflected in the standard error.

 2         Q.    And it is also the case, isn't it, that the 

 3    calculation of the final standard error for the smooth 

 4    adjustment factors depends on the assumptions underlying the 

 5    smoothing model?

 6         A.    Along with many other things it depends on those 

 7    as summations, yes.

 8         Q.    And if the assumptions are wrong, the standard 

 9    errors could be too small, couldn't they?

10         A.    Well, assumptions are never perfectly correct at 

11    any time.  The issue is how far the reality deviates from 

12    those assumptions.  It would perhaps cause the standard 

13    errors to be different, higher or lower, I cannot give you 

14    an opinion at this time.

15         Q.    So you are saying it's not possible that the 

16    standard error could be too small if the assumptions are 

17    wrong?

18         A.    I didn't think that's what you asked me.

19         Q.    I asked you if the assumptions are wrong could 

20    the standard errors be too small?

21         A.    Oh, could?

22         Q.    Yes.

23         A.    They could be.

24         Q.    And that would make the estimated adjustments 

25    look more reliability than they actually are, wouldn't it?
                                                              408

 1         A.    The point is that you say it could be too small, 

 2    but it could be too small by a very small amount, it could 

 3    be too large by a very small amount, it could be different.  

 4    Without doing some kind of an analysis I could not give you 

 5    any specific opinion at this time.

 6         Q.    So you can't tell me as we sit here today whether 

 7    the estimated adjustments would look more reliable than they 

 8    really are if mistakes in the assumptions in the smoothing 

 9    model cause the standard errors to be too small, is that 

10    your testimony?

11         A.    If we really knew that the standard errors were 

12    too small, then the answer to your question would be yes.

13         Q.    The smooth adjustment factors are affected by 

14    sampling error, aren't they?

15         A.    Like any sample estimate, they have sampling 

16    error, yes.

17         Q.    And the variance of the smooth adjustment factors 

18    is obtained by squaring the standard error of the smooth 

19    adjustment factors, right?

20         A.    That's right.

21         Q.    And the variance from the smoothing model is one 

22    of the inputs in the loss function analysis the Census 

23    Bureau conducted, right?

24         A.    That's right.

25         Q.    You are aware, aren't you, that the Undercount 
                                                              409

 1    Steering Committee issued a report to the Census Bureau 

 2    director on adjustments, the one I just showed you, right?

 3         A.    That's right.

 4         Q.    And you are aware that the Undercount Steering 

 5    Committee issued an addendum to that original report, right?

 6         A.    Yes, I am aware that there was an addendum.

 7         Q.    And the addendum to the report states that the 

 8    Undercount Steering Committee had originally understated the 

 9    variance of the estimates from the smoothing model by a 

10    range of 1.7 to 3.0, isn't that right?

11         A.    Let me take a look.  

12               (Pause)

13               Which one his that?

14         Q.    Pardon me? 

15               That is Plaintiff's Exhibit 540 A.16.A and it 

16    starts at Bates number 6074.  

17               MR. SOLOMON:  What is the page number of the 

18    document other than the page number at the top?  

19               MR. ZITCOV:  There are a number of different 

20    pages.

21               It is the eleventh page from the back, which 

22    happens to be numbered page 5.

23               (Pause) 

24               MR. ZITCOV:  Your Honor, I made a mistake, that 

25    is not the right Bates stamp number that I gave you.  It's 
                                                              410

 1    1146 for the Undercount Steering Committee report.  My 

 2    apologies.

 3         A.    You said 540 A?

 4         Q.    No, I made a mistake, Doctor Ericksen.

 5               It is in Plaintiff's Exhibit 540 A and it begins 

 6    at 1146.  

 7               MR. ZITCOV:  Can I approach the witness and I 

 8    will find it for him?  

 9               THE COURT:  Certainly.  

10               MR. ZITCOV:  And I will be happy to find it for 

11    your Honor if you would like.  

12               THE COURT:  I have it.

13               (Pause) 

14               MR. ZITCOV:  Your Honor, it is page 1202.

15         Q.    I would like you to take a look at the paragraph 

16    that begins at the bottom of page 1202 of the so-called 

17    administrative record and it says, "Summary reaction and 

18    judgment.

19               It is likely that 1 A, 1 B and 1 C add 

20    considerably to the total uncertainty.  It is possible the 

21    contribution from 1 D is not trivial as well, although none 

22    of the evidence so far elucidates the effect of 1 D as 

23    separately important." 

24               As a matter of judgment, the total understatement 

25    of variance from the smoothing model may be in the range of 
                                                              411

 1    a factor of 1.7 to 3.0 in terms of variance or 1.3 to 1.7 on 

 2    the standard error."  

 3               MR. ZIMROTH:  Maybe we ought to ride the rest of 

 4    that so it is in context.  

 5               MR. ZITCOV:  All I am asking, your Honor, is 

 6    whether or not that refreshes your recollection that the 

 7    addendum to the Undercount Steering Committee report 

 8    suggested that the estimates from the smoothing model of 

 9    variance might have been underestimated by a factor in the 

10    range of 1.7 to 3?  

11               MR. ZIMROTH:  I think it is only fair if you are 

12    going to read the first half of the paragraph you read the 

13    second half of the paragraph in which the Undercount 

14    Steering Committee says it's hard to believe that smoothing 

15    did not at least produce some gains relative to the 

16    unsmoothed estimates.  

17               MR. ZITCOV:  Your Honor, isn't that what 

18    cross-examination is for?  

19               THE COURT:  I have no idea.

20               Can you answer it?

21               THE WITNESS:  I think he is just asking me if I 

22    remember that this paragraph is in the document, and the 

23    answer to the question would be yes.

24         Q.    And does that refresh your recollection that the 

25    Undercount Steering Committee said the original report had 
                                                              412

 1    understated the variance of the estimates from the smoothing 

 2    model by a factor in the range of 1.7 to 3?

 3         A.    What it actually says it may be in the range of 

 4    1.7 to 3.  

 5               Since this was after the July 15 date when I 

 6    actually had a chance to read this document, I've not had 

 7    the opportunity to discuss this issue with anybody from the 

 8    Census Bureau to find out their reasoning for this, what 

 9    kind of evidence they have for it.  It certainly is not in 

10    this document.  That's an issue that I would be interested 

11    in learning more about.

12         Q.    You considered at your deposition, didn't you 

13    consider it a problem to be considered with regard to the 

14    accuracy of adjustment?

15         A.    Isn't that what I just said now?  Yes.

16         Q.    One of the outputs to the production smoothing 

17    model was the variance co-variance matrix for the raw 

18    adjustment factors, is that right?

19         A.    One of the outputs?

20         Q.    I'm sorry, inputs.

21         A.    Oh.  That was a long question.  Now give it to me 

22    right.

23         Q.    One of the inputs to the production smoothing 

24    model was the variance co-variance matrix for the raw 

25    adjustment factors, is that right?
                                                              413

 1         A.    That's right.

 2         Q.    None of the P studies considered the reliability 

 3    of that study, did they?

 4         A.    None of the P studies considered the reliability 

 5    of that program, that's correct.

 6         Q.    And none of the P studies considered the quality 

 7    control work on that program, did they?

 8         A.    No.  Presumably the Bureau felt it had done its 

 9    research elsewhere, it did not include them in the P 

10    studies.

11         Q.    And none of the P studies addressed the 

12    reliability of variance pre-smoothing, did they?

13         A.    Not in the P studies, no.

14         Q.    And as of the time I took your deposition five 

15    weeks ago, you had not undertaken any studies of the 

16    reliability of the variance co-variance matrix for the 

17    smooth adjustment factors, had you?

18         A.    By reliability I'm talking about the errors in 

19    that due to sampling variability, I had not conducted my own 

20    study, that is correct.

21         Q.    So the answer is as of the time of your 

22    deposition, you had not undertaken any such study?

23         A.    I believe that's just what I just said, yes.

24         Q.    In your opinion as an expert, you would consider 

25    a sample survey that had a fabrication rate of ten percent 
                                                              414

 1    to be unacceptable, isn't that right?

 2         A.    I would be highly concerned about a fabrication 

 3    rate of ten percent.  If I was in a situation where I had to 

 4    make a decision based on the results of such a survey and I 

 5    had no opportunity to go out and collect more data, I would 

 6    perhaps look at certain things like what would happen to the 

 7    results if you exclude the fabricated data.  But I certainly 

 8    would be concerned about a survey with a fabrication rate of 

 9    ten percent, that is true.

10         Q.    Do you recall --

11         A.    Are we through with this?

12         Q.    Pardon me? 

13               Yes.

14               Do you recall testifying at your deposition that 

15    you would consider a sample survey to be unacceptable if it 

16    had a fabrication rate of ten percent?

17         A.    I don't recall that particular passage.  If it's 

18    in there, I'm sure it's in there.  If you show me where it 

19    was, I could explain to you what I meant.  

20               MR. ZIMROTH:  Your Honor, this is a completely 

21    misuse of the deposition.  This witness didn't say anything 

22    inconsistent with that, he didn't say that his recollection 

23    needed refreshing and --  

24               THE COURT:  How can we know if it is inconsistent 

25    unless we allow him to ask the question?  
                                                              415

 1               MR. ZIMROTH:  Because he is reading from the 

 2    deposition before he has asked him the question directly.  

 3    He has asked him a question about the ten percent, he 

 4    answered the question, now he says he needs the deposition 

 5    to refresh his recollection.

 6               Let him look at the deposition.  

 7               THE COURT:  Can you do it that way, Mr. Sitcov?  

 8               MR. ZITCOV:  I thought I was doing it in the 

 9    normal way, I asked the question and he answered it 

10    differently than he answered it in the deposition and I 

11    wanted to know now if I can refresh his recollection, that's 

12    all.

13               I don't think that what I am doing is unfair, 

14    even unique.  It's the kind of thing that happens normally.  

15               THE COURT:  How much longer do you have?  

16               MR. ZITCOV:  For this witness?  

17               THE COURT:  Yes.  

18               MR. ZITCOV:  I would say at least an hour or so, 

19    maybe more.  

20               THE COURT:  All right.  Continue.  

21    BY MR. ZITCOV:

22         Q.    Could you turn to page 491, please.

23         A.    Don't have it.

24         Q.    You don't have it?

25         A.    No.  You only gave me up to 459.
                                                              416

 1               (Handing to the witness)

 2               (Pause)

 3         Q.    Will you turn to page 491, please.

 4               Tell me when you have it.

 5               Are you there?

 6         A.    I'm here.  

 7               MR. ZIMROTH:  Your Honor, this just proves the 

 8    point.  There is nothing inconsistent in this passage --  

 9               THE COURT:  I don't know.  I don't have it in 

10    front of me.  

11               MR. ZIMROTH:  I'm sorry, your Honor.

12         Q.    "Q.    So there is no rate of fabrications that 

13    in your opinion would automatically be unacceptable?

14               "A.    No.

15               "Q.    50 percent?  

16               "A.    Would not be acceptance.

17               "Q.    Okay.  40 percent?

18               "A.    Forty percent would not be acceptable.

19               "Q.    Thirty percent?

20               "A.    Thirty percent would not be acceptable.  

21    And it's also unheard of in any survey I worked on.

22               "Q.    Twenty percent.

23               "A.    The same.

24               "Q.    Ten percent?

25               "A.    The same." 
                                                              417

 1               Does that refresh your recollection that you 

 2    testified without qualification at your deposition that a 

 3    sample survey that had a rate of fabrication of ten percent 

 4    would automatically be unacceptable?  

 5               MR. ZIMROTH:  Your Honor, Mr. Sitcov hasn't read 

 6    the questions immediately before or the questions 

 7    immediately after and that's why what he is doing now is 

 8    entirely unfair.  

 9               MR. ZITCOV:  Your Honor, it seems to me if they 

10    want to cover that on redirect they are entirely able and it 

11    would be appropriate for them to do so.  

12               THE COURT:  The crucial question is whether the 

13    witness understands what is going on and is comfortable 

14    answering the question as being put.

15               THE WITNESS:  Well, look, he is asking me 

16    questions I've never dealt with in my whole career of 

17    working with survey data.  

18               He started asking me a series of questions at 

19    which point it would be unacceptable.  We got down to eight 

20    percent in the deposition, I said I don't know; now it's ten 

21    percent; I don't know.  Ask me next month it might be 12 

22    percent.  

23               These are not hard and fast numbers that you sit 

24    there and make automatic decisions with.  I tried to answer 

25    his questions as best I could at the time of the deposition.  
                                                              418

 1    It's a totally hypothetical question.

 2         Q.    So there are sample surveys that you would 

 3    consider acceptable that had a 12 percent rate of --

 4         A.    It is possible.  Like I just explained to you, 

 5    Mr. Sitcov, that if I had to make a decision and I knew that 

 6    there was a 12 percent fabrication rate and I had to make a 

 7    decision based on the data, I would do some kind of 

 8    sensitivity analysis where I would remove the fabricated 

 9    data to see if the results changed.  If you have to make a 

10    decision based on data, you have to make a decision.  

11               THE COURT:  You play with the cards you are 

12    dealt? 

13               THE WITNESS:  That's right.

14         Q.    Fabrication is a source of bias in the PES, isn't 

15    it?

16         A.    It is.

17         Q.    Fabrications in the P-sample could cause an 

18    overstatement of the net undercount, couldn't they?

19         A.    You need to give me some more information to 

20    completely answer that question, but I can imagine a likely 

21    scenario where fabrications in the P-sample would overstate 

22    the undercount.

23         Q.    You don't know how many fabrications there were 

24    in the PES, do you?

25         A.    I know what the Census Bureau reported.  I do not 
                                                              419

 1    know for a fact what that number was.  Like any other 

 2    estimate, it's based on sample data.  We know it with 

 3    uncertainty.

 4         Q.    So you don't know how many fabrications that were 

 5    in the PES?

 6         A.    Not with 100 percent certainty, no.

 7         Q.    Would it be fair to say based on everything you 

 8    heard, you don't think the number of fabrications in the PES 

 9    is particularly large?

10         A.    Relative to the overall size of the PES sample, I 

11    do not consider the number of fabrications to be overly 

12    large.

13         Q.    And one basis for that opinion is the P5a study, 

14    isn't that right?

15         A.    It's based on the Census Bureau report and P5a, I 

16    believe that is the basis, one basis for that opinion, yes.

17         Q.    Could you take a look at what is before you as 

18    Defendant's Exhibit 109.  It is part of a plaintiff's 

19    exhibits, the exhibit that has all of the P studies in them, 

20    and it is Bates number 4971.  

21               THE COURT:  What is the corresponding plaintiff's 

22    exhibit number?  

23               MR. ZIMROTH:  It should be 540 A 5 A? 

24               Is it P5a, Gene?  

25               THE COURT:  We will fill it in later.  
                                                              420

 1               MR. ZIMROTH:  It would be 540 A 5 A.  

 2               THE COURT:  540 A 5 A.  

 3    BY MR. ZITCOV:

 4         Q.    You have Plaintiff's Exhibit 195 in front of you? 

 5               That is right at your right hand.

 6               Can you turn to tab C.  That is appendix C of the 

 7    report on the 1990 Decennial Census and PES that you 

 8    submitted with Estrada, Tukey and Walter, is that right?

 9         A.    That's right.

10         Q.    Can you turn to page 16 of appendix C, and that 

11    page begins with an underscored heading, "Imputation, 

12    fabrication, census day addressed in this reporting 

13    erroneous enumeration," and you say, "In this section we 

14    report on four additional PES evaluation studies, P3, 

15    imputation reanalysis, P4, census day address misreporting, 

16    P5a, P-sample fabrication, and P5a matching reanalysis.  We 

17    first dismiss project P5a as a Census Bureau reports that 

18    only 13 fabrications with a weighted sum of 64,667 persons 

19    were found.

20               Is it your opinion that there was only 13 

21    fabrications in the PES?

22         A.    It's my opinion based on the fact that the Census 

23    Bureau had a very extensive fabrication control procedure 

24    built into the PES and based on the results of this study 

25    that the number of fabrications was very small, given that 
                                                              421

 1    it's an estimate based on a sample, we don't know that it's 

 2    exactly 13, it's 13 plus or minus some small quantity, so I 

 3    don't know that it's exactly 13, it's probably quite close 

 4    to it, though.

 5         Q.    Okay.

 6               Now, weighted to the entire population those 13 

 7    fabrications added almost 65,000 fabricated people to the 

 8    population, is that right?

 9         A.    That's right.

10         Q.    That is a number 5,000 times as large as the 

11    fabrications, isn't that right?

12         A.    That's right, because of the nature --  that's 

13    because it was a sample in the evaluation followup study and 

14    you apply the weights due to sampling and it comes to 5,000 

15    times 13, 65,000, that's right.

16         Q.    So just a few mistakes in the post-enumeration 

17    survey can lead to a big mistake in the adjustment, isn't 

18    that right?

19         A.    No.  You are confusing apples and oranges here, 

20    Mr. Sitcov.

21               The estimate of 65,000 is based on the evaluation 

22    followup study.  That was a subsample of the PES, and the 

23    evaluation followup study was used to evaluate the quality 

24    of the PES data.  So if there was an error in the evaluation 

25    followup study that P5a is based upon, it would not affect 
                                                              422

 1    the adjustment, it would affect our assessment of the 

 2    quality of the adjustment data.

 3         Q.    I see.

 4               Now, in your report to the Secretary, you didn't 

 5    state anywhere that there were more than 15 fabrications in 

 6    the PES, did you?

 7         A.    I only informed the Secretary of the Census 

 8    Bureau reports that there were 13 fabrications.  That's what 

 9    we did.

10         Q.    Could you turn to page 9 of P5a.

11               Do you have it?

12         A.    I have it.

13         Q.    Okay.

14               Now, the last paragraph on page 9 says, "When 

15    reviewing the results, the limitations of the study should 

16    be kept in mind.  The data for the study were collected in 

17    the evaluation file which was not designed specifically to 

18    detect fabrication.  In the PES, if a case did not fall in 

19    the quality control sample or if a fictitious case remained 

20    undetected in the quality control effort, the case might 

21    have been identified as fictitious by the after followup 

22    matching operation.  However, matchers must establish that a 

23    number of criteria have been met before a case is determined 

24    to be fictitious.  These criteria depend on information 

25    provided by the interviewers.  
                                                              423

 1               The evaluation followup matching used the PES 

 2    following matching procedures.  No additional quality 

 3    control or matching procedures were implemented in the 

 4    evaluation followup to detect fabrication.  Thus, it is 

 5    possible that the evaluation followup did not identify more 

 6    cases as fictitious because there was not enough new and 

 7    additional interviewer information to establish that the 

 8    cases were fictitious in the PES." 

 9               Now, you didn't advise the Secretary in your 

10    report that the data for the P5a study were collected in the 

11    evaluation followup which was not designed specifically to 

12    detect fabrications in the PES, did you?

13         A.    We did not include that in our report, that is 

14    correct.

15         Q.    And you didn't advise the Secretary that the only 

16    cases that were even considered in P5a were those in the 

17    quality control sample drawing from the PES, did you?

18         A.    What I informed the Secretary is the conclusion 

19    that the Census Bureau drew.  The Census Bureau concluded, 

20    keeping this qualification in mind without doubt, that the 

21    best estimate of the number of fabrications was 13 weighted 

22    up to 65,000 and that's what in their judgment they put in 

23    the total error model.  Therefore, relying on the Census 

24    Bureau's judgment in the matter, that's what I reported to 

25    the Secretary, yes.
                                                              424

 1         Q.    You didn't advise the Secretary that the only 

 2    cases that were even considered of P5a were those in the 

 3    quality control sample drawn from the PES, did you?

 4         A.    My report says what it says.

 5         Q.    Does it say that?

 6         A.    No, it does not.

 7         Q.    And you didn't advise the Secretary that no 

 8    additional quality control or matching procedures were 

 9    implemented in evaluation followup to detect fabrications, 

10    did you?  

11               MR. ZIMROTH:  Is there a claim in this case the 

12    Secretary didn't have the Census Bureau's own reports on 

13    this subject?  Is that the claim in the case? 

14               I'm really puzzled by this waste of time, your 

15    Honor.  

16               THE COURT:  The claim is that the witness did not 

17    advise the Secretary and, therefore, the Secretary did not 

18    know it?  

19               MR. ZITCOV:  That's right.  

20               THE COURT:  But if the Census Bureau had told him 

21    he would know it?  

22               MR. ZIMROTH:  Right.  

23               Now I am asking whether there is a claim in this 

24    case whether the Secretary did not have this full massive 

25    volume of reports in front of him.  
                                                              425

 1               Was this witness supposed to, you know, just 

 2    simply repeat everything in the reports?  He was doing a 

 3    summary.  He was reporting on what this bureau itself had 

 4    concluded.  

 5               THE COURT:  Mr. Sitcov?  

 6               MR. ZITCOV:  Your Honor, it is certainly relevant 

 7    that the witness was quite selective in what he included in 

 8    his report to the Secretary about fabrications.

 9               The witness selected some information from this 

10    report and didn't select other of the information.  

11               THE COURT:  And it is relevant because the 

12    Secretary might then choose to ignore the report feeling 

13    that it had been deliberately skewed?  

14               MR. ZITCOV:  Or just to take the --  

15               MR. ZIMROTH:  He didn't audit the report, that's 

16    the point.

17               THE COURT:  He is trying to justify it.

18               Is that the idea?  

19               MR. ZITCOV:  That is one.  

20               THE COURT:  Do I have to make these ideas up for 

21    you?  

22               MR. ZITCOV:  No.  Another one is that the report 

23    for panelists was skewed.  

24               MR. ZIMROTH:  Your Honor, this witness has 

25    already testified that when he made --  when he wrote his 
                                                              426

 1    report on June 21, he did not have the final version of all 

 2    of these reports, he has testified to that effect, he had 

 3    underlying data.  He put into the report what he thought was 

 4    relevant.

 5               The Secretary had in front of him all of these 

 6    reports upon which to make his basis.  The Census Bureau 

 7    came to precisely the conclusion that Mr. Ericksen said it 

 8    came to.  

 9               MR. ZITCOV:  Your Honor, if all we needed was the 

10    reports, then we wouldn't need a trial.  Obviously if this 

11    would be a case to be decided on the record, we wouldn't 

12    need Dr. Ericksen to testify to anything --  

13               MR. ZIMROTH:  It would be nice to have five 

14    minutes worth of cross-examination on what Dr. Ericksen 

15    spend a day and a half testifying about yesterday and the 

16    day before.  Not a word.  

17               THE COURT:  It does seem --  

18               MR. ZITCOV:  He -- 

19               MR. ZIMROTH:  He wasn't called as an expert in 

20    loss function analysis and Mr. Sitcov spends an hour 

21    cross-examining about it, especially when we are going to 

22    have an expert on loss function analysis.  

23               He wasn't here to testify about smoothing.  Mr. 

24    Sitcov knows we have one of the leading experts in the 

25    country here on that subject to come testify and he is 
                                                              427

 1    wasting Dr. Ericksen's time and our time cross-examining on 

 2    that subject as well.  

 3               MR. ZITCOV:  Your Honor, I guess the plaintiff's 

 4    position is that Dr. Ericksen should be able to introduce 

 5    all of the report that he and the other three authored, that 

 6    he should be able to introduce all of the P studies and 

 7    their underlying data, that he should be able to testify 

 8    about adjustment and smoothing and that we should not be 

 9    able to question him about the things that he has introduced 

10    and testified about.  

11               THE COURT:  I think you have carried this 

12    dialogue far enough.

13               If the point you are now trying to make is that 

14    the witness was selective in what he told the Secretary, 

15    that point has been made.  Let's move on to something else.  

16               MR. ZITCOV:  Okay.  

17    BY MR. ZITCOV:

18         Q.    Do you consider it likely that there were --  you 

19    said that there may have been more than 13 fabrications, is 

20    that right?

21         A.    It's possible.

22         Q.    Do you consider it likely there were as many as 

23    30 fabrications in the PES?

24         A.    Look, I have not conducted my own independent 

25    study of fabrications in PES, I only know what the census 
                                                              428

 1    report is.

 2         Q.    I am asking you, do you consider that likely?

 3         A.    It's a possibility, but I don't have any opinion 

 4    as to whether or not it is likely.

 5         Q.    If instead of 13 fabrications in the PES, there 

 6    were a thousand fabrications in the PES, would your opinion 

 7    about adjustment change?

 8         A.    I would have to consider it in the context of 

 9    everything else that we know about the PES and the original 

10    enumeration, yes.

11         Q.    Do you consider it likely there were as many as a 

12    thousand fabrications in the PES?  

13               THE COURT:  Asked and answered.  Next.

14         Q.    What if there were 6,000 fabrications in the PES, 

15    would that cause your opinion about adjustment to change?

16         A.    Look, like everything else about the PES, if you 

17    present me with some new information, I would want to 

18    consider it in context with everything else, I would want to 

19    confer with the Census Bureau's statisticians that developed 

20    these estimates.  I would certainly consider it.  How it 

21    would all come out, I don't know at this time.

22         Q.    Would you take a look at Defendant's Exhibit 102, 

23    please you.  This is one that has not been introduced 

24    before.  

25               THE COURT:  Defendant's Exhibit 102, an original.  
                                                              429

 1               MR. ZITCOV:  An original by Dr. Ericksen, your 

 2    Honor, a double whammy.

 3               (Pause) 

 4    BY MR. ZITCOV:

 5         Q.    This is an article that you wrote about 

 6    estimating the population in the 1990 --  using the 1980 

 7    Decennial Census, is that right?

 8         A.    It is.

 9         Q.    And you advocated an adjustment of the 1980 

10    Decennial Census in this article, didn't you?

11         A.    By the time this article --  I certainly made an 

12    argument that the 1980 census would have been improved by 

13    adjustment, yes.  

14               MR. ZITCOV:  Your Honor, we move for admission of 

15    Defendant's Exhibit 102.  

16               MR. ZIMROTH:  No objection.  

17               THE COURT:  Defendant's Exhibit 102 is admitted.

18               (Defendant's Exhibit 102 marked for 

19    identification was received in evidence.) 

20    BY MR. ZITCOV:

21         Q.    Would you turn to page 109 of Defendant's Exhibit 

22    102, please.

23               That is a comment on your article by Barbara A. 

24    Bailar?  

25               MR. ZIMROTH:  That is another story, your Honor.  
                                                              430

 1    I'm sorry, I didn't look at the whole exhibit, your Honor, 

 2    and 102, which was represented to be an article by Professor 

 3    Ericksen and his colleague Joseph Kadane, has attached to it 

 4    a comment by some, by another person, Barbara Bailar and 

 5    several other comments as well.  

 6               THE COURT:  So?  

 7               MR. ZIMROTH:  So, therefore, I have no objection 

 8    to the admissibility of 102 insofar as it is the article by 

 9    Ericksen and Kadane, and I do have an objection with respect 

10    to the others.  

11               THE COURT:  Mr. Sitcov?  

12    BY MR. ZITCOV:  

13         Q.    Dr. Bailar was commenting on your --  the article 

14    you wrote with Kadane, wasn't she?

15         A.    She was.

16         Q.    And you recognize that to be the comment that was 

17    published under her name in reaction to your article?

18         A.    I do.  

19               THE COURT:  You still object?  

20               MR. ZIMROTH:  Excuse me just one second.

21               (Pause) 

22               MR. ZIMROTH:  Your Honor, I'm just puzzled.  I 

23    mean, I have no objection if the witness wants to --  excuse 

24    me --  if Mr. Sitcov wants to examine Dr. Ericksen about 

25    anything that is in any of these comments.  It's just a 
                                                              431

 1    puzzle to me why that makes it admissible into evidence.  

 2               THE COURT:  It probably doesn't.  

 3               MR. ZIMROTH:  Yes.  

 4               THE COURT:  Are you going to withdraw the 

 5    objection?  

 6               MR. ZIMROTH:  Yes, I am going to withdraw the 

 7    objection.  

 8               THE COURT:  Everything is admitted in 102.  

 9               MR. ZITCOV:  Thank you.           

10               THE COURT:  Maybe this would be a good point to 

11    take a ten minute break.  We are going to be adjourning for 

12    lunch at noon, so let's keep the break to precisely 10 

13    minutes.           

14               (Recess)

15               THE COURT:  Are we ready?  

16               MR. ZITCOV:  Yes.  

17               THE COURT:  Mr. Sitcov, you may resume.  

18               MR. ZITCOV:  And I will speak more slowly.  

19               THE COURT:  Good.  

20    BY MR. ZITCOV:

21         Q.    Do you have Defendant's Exhibit 102 in front of 

22    you?

23         A.    Yes, I do.

24         Q.    Is it opened to page 109?

25         A.    Yes, it is.
                                                              432

 1         Q.    And that is a comment on your article by Barbara 

 2    Bailar?

 3         A.    Yes, it is.

 4         Q.    And Barbara Bailar is one of the experts that 

 5    plaintiffs have included in their list of expert witnesses 

 6    that they intend to call in this case, is that right?

 7         A.    She is.

 8         Q.    Curbstoning is a synonym for fabrications in the 

 9    survey, isn't that right?

10         A.    It's one synonym for fabrications, yes.

11         Q.    Could you turn to page 112 of Dr. Bailar's 

12    article.  

13               In the paragraph in the left column at the top 

14    that carries over from page 111 there is a statement that 

15    says, beginning in the third line on the top, "For the other 

16    system, the CES, there was no acknowledgment for erroneous 

17    conclusions, yet the authors used my own estimates of .6 

18    percent to 4 percent of the CES interviewers curbstone." 

19               Do you consider that to be a reasonable estimate 

20    of curbstoning in the CES?

21         A.    I don't know.  I haven't seen any real study of 

22    curbstoning in the PES.  Absent such a study --  

23               MR. ZIMROTH:  In the CES.

24         A.    You are talking about the current population 

25    survey?
                                                              433

 1         Q.    Yes. ?

 2         A.    Absence seeing such a study, I don't have a real 

 3    opinion of that.

 4         Q.    You consider that to be a reasonable estimate of 

 5    curbstoning in the PES?

 6         A.    Absolutely not.  The PES is totally different 

 7    from the current population survey.  In the post-interview 

 8    surveys there was a procedure included to defect 

 9    fabrication.  Where an interviewer was found to fabricate, 

10    that work was done by another interviewer and that's the 

11    reason why the number of fabrications is so slow in the PES, 

12    why it is reasonable to believe that.  

13               So whatever may be true of the current population 

14    survey has nothing to do with whatever may be true about the 

15    post-enumeration survey.

16         Q.    The 1990 PES was a sample survey of almost 

17    400,000 people, is that right?

18         A.    It was.

19         Q.    You don't of any other sample survey of that size 

20    conducted in the United States, do you?  

21               THE COURT:  Ever?  

22               MR. ZITCOV:  Yes.

23         A.    I'm quite confident that there have been, but I 

24    couldn't tell you what they were.

25         Q.    The CES is a much smaller survey than the 1990 
                                                              434

 1    PES, isn't that right?

 2         A.    It is a smaller survey than the PES, yes.

 3         Q.    How big is the CES?

 4         A     Well, the size of the CES has varied.  The best 

 5    of my recollection every month they interview between 50- 

 6    and 70,000 households.

 7         Q.    And the CES uses professional interviewers, 

 8    doesn't it?

 9         A.    It does.

10         Q.    Now, if a rate of fabrications in the PES was .6 

11    percent, which is Dr. Bailar's low estimate, that would 

12    equal about 2,400 fabrications, is that right?  

13               MR. ZIMROTH:  Excuse me.  Are we talking about 

14    the CES or PES?  

15               MR. ZITCOV:  PES.  

16               MR. ZIMROTH:  Dr. Bailar's estimate is about the 

17    CES.  

18               MR. ZITCOV:  That's correct.  

19               MR. ZIMROTH:  I'm confused.

20         Q.    My question is if the rate of fabrication in the 

21    PES was .6 percent, which is Dr. Bailar's low estimate for 

22    the CES --  

23               MR. ZIMROTH:  Which this witness said he had no 

24    opinion about.  

25               THE COURT:  Let him finish the question.  
                                                              435

 1               MR. ZITCOV:  Thank you, your Honor?

 2         Q.    -- that would equal about 2,400 fabrications in 

 3    the PES?

 4         A.    That's right.

 5         Q.    And if we to take her high estimate of 4 percent 

 6    fabrications in the CES, that would be about 16,000 

 7    fabrications in the PES, wouldn't it?  

 8               THE COURT:  Just doing the math?  

 9               MR. ZITCOV:  Yes.

10         A.    Look, your question doesn't make any sense, but 

11    if you want me to say multiply four percent by 400,000, the 

12    answer would be --  let me make sure I got my zeros right --  

13    then the --

14         Q.    That would be about 16,000, wouldn't it?

15         A.    That is --  four percent times 400,000 is 16,000.

16         Q.    Now I believe you testified that P5a, which was 

17    --  which we talked about a moment ago was not designed 

18    specifically to defect fabrications in the PES, is that 

19    right?

20         A.    The document said what it said.

21         Q.    There was a P study that was designed for the 

22    purpose of detecdting fabrications in the PES, wasn't there?

23         A.    Well, I actually believe that it was the intent 

24    of the Census Bureau to use P5a to get an estimate of 

25    fabrications that went into the total error model.  So if 
                                                              436

 1    that's what they did, that's my understanding of their 

 2    intent and I believe that was always the plan.

 3         Q.    Will you turn to Exhibit 110, please.  

 4               MR. ZIMROTH:  Defendant's Exhibit 110?  

 5               MR. ZITCOV:  Defendant's Exhibit 110, which is 

 6    also in Plaintiff's Exhibit 540 A, I don't know what the 

 7    order is.  This is P6.

 8               This is also, your Honor, a P study and it is one 

 9    of the ones in that --  I guess it is Plaintiff's Exhibit 

10    540.  

11               MR. ZIMROTH:  540 A series, the Census Bureau 

12    studies, your Honor.

13         Q.    Do you see P6 there?

14         A.    I do.

15         Q.    And that says, "Final report for PES evaluation 

16    P6 fabrication in the P-sample interviewer effect," is that 

17    right?  

18               MR. ZIMROTH:  Just for the record I want to say 

19    it is dated July 10, 1991.  

20               MR. SITCOV:  Well, I guess, your Honor, you 

21    better say for the record that P5a is dated July 9, 1991.  

22               MR. ZIMROTH:  That would clarify a lot, actually, 

23    if you said that.

24         Q.    So this is the report, this report you are 

25    looking at, says, "Final report for PES evaluation project 
                                                              437

 1    P6 fabrications in the P-sample interviewer effect," right?  

 2    We are looking at the same one?

 3         A.    That's right.  This was not something that went 

 4    into my June 21 report.

 5         Q.    Could you turn to page 12 of P6.

 6               Do you see in the last paragraph it states, 

 7    "There has been speculation that in data collection such as 

 8    the Census Bureau's current surveys, between one half and 

 9    one and a half of the interviews are fabricated."  It cites 

10    Beamer and Stokes 1989.

11               Do you see that?  

12               MR. ZIMROTH:  I don't have that on that page.

13               On page 6?  

14               MR. ZITCOV:  Page 12 of P6.  

15               MR. ZIMROTH:  Page 12?  

16               MR. ZITCOV:  Yes.  

17               MR. ZIMROTH:  One second.

18               Where are you reading from?  

19               MR. ZITCOV:  The last paragraph.  

20               MR. ZIMROTH:  Would you read it began, please?  

21    I'm sorry, could you read it again.  

22               MR. ZITCOV:  "It has been the speculation that in 

23    data collection such as the Census Bureau's current surveys, 

24    between one half and one and a half percent of the 

25    interviews are fabricated.  Beamer and Stokes, 1989."
                                                              438

 1         Q.    Do you see that?

 2         A.    Yes.

 3         Q.    So we have established already that the PES had 

 4    about 400,000 people, correct?

 5         A.    That's right.

 6         Q.    So using the range from page 12 of P6, there 

 7    would be between 2,000 and 6,000 fabrications in the PES, is 

 8    that right?

 9         A.    No, it's not right, because I explained to you 

10    before in the current population surveys there is no 

11    procedure for detecting fabrications and redoing the 

12    interviewers' work.  In the post-interview work there was a 

13    procedure detecting fabrications and when they were found 

14    the interviewers work was redone.

15         Q.    I am not asking you if you think that's the 

16    range, what I am asking you if we used that range and 

17    applied it to the PES, there would be between 2,000 and 

18    6,000 fabrications in the PES; is that right?  

19               MR. ZIMROTH:  Are you asking him to do the 

20    arithmetic for you?  Is that what you are asking? 

21               MR. SITCOV:  Yes.  

22               MR. ZIMROTH:  Okay.  

23               THE COURT:  Maybe we can take your word on this, 

24    Mr. Sitcov.

25         A.    It comes out.
                                                              439

 1         Q.    Good.

 2               Now, according to page 12, the lowest estimated 

 3    rate of fabricated interviews in the PES in a regional 

 4    office was 2.1 percent, is that right?

 5         A.    This is an estimate of 2.1 percent of the --  

 6    indicates the excess and not match rates that were expected 

 7    by a model the Census Bureau used.  

 8               The author of this report is inferring from that 

 9    that may indicate fabrications, but the evidence in this 

10    report is quite indirect, and the important point is that 

11    this was not the result that the Census Bureau included in 

12    the total error model.

13         Q.    So P6 suggests that the lowest rate of 

14    fabrications in a regional office in the PES was 2.1 

15    percent?  

16               MR. ZIMROTH:  I object.  That is a 

17    mischaracterization of the witness' testimony.  

18               THE COURT:  I think the witness can handle it.

19         A.    This is exceedingly indirect evidence and I 

20    believe that the authors of this report are only telling you 

21    that there are certain part of the nonmatch rates of these 

22    interviewers' workload they can't explain.  It's only a very 

23    indirect suggestion.

24         Q.    But it does suggest that the rate, that the 

25    lowest rate of fabrications in a regional office in the PES 
                                                              440

 1    was 2.1 percent?

 2         A.    Very indirectly it makes that suggestion.

 3         Q.    And that the highest rate of fabrications in a 

 4    regional office in the PES was 8.79 percent, is that right?

 5         A.    It very indirectly makes that suggestion.  The 

 6    suggestion is so indirect that the Census Bureau did not 

 7    include his estimates in the total error model.

 8         Q.    In other words, according to P6, the range of 

 9    fabricated interviews in the PES extends from a rate equal 

10    to the net undercount at the low end to a rate that is four 

11    times the size of the net undercount rate at the high end, 

12    is that right?

13         A.    The Census Bureau did not include P6 as an 

14    estimate of fabrications.  The numbers in here say what they 

15    say.  That's all I've got to say about it.

16         Q.    My question to you is, according to this, does 

17    the range of fabrications that was found by P6 extend from a 

18    low end where the rate is equal to the net undercount to a 

19    range at the high end that is more than four times the net 

20    undercount?

21         A.    That's what these numbers are.

22         Q.    That range is not consistent with 13 fabrications 

23    in the PES, is it?

24         A.    That's why the Census Bureau didn't use it.

25         Q.    No.  My question is, is that range consistent 
                                                              441

 1    with 13 fabrications in the PES?

 2         A.    Certainly not.

 3         Q.    And if the rate of the fabricated interviews in 

 4    the entire P-sample were 8.79 percent, that would have 

 5    equaled about 35,000 fabrications, isn't that right?

 6         A.    Close enough.  I'll take your word for it.

 7         Q.    And if the rate of fabrications for the whole 

 8    P-sample had been 2.1 percent, that would have equaled about 

 9    8,400 people, isn't that right?

10         A.    Fortunately, it probably wasn't.

11         Q.    No, that's not my question.  My question is, if 

12    the rate of fabrication for the whole P-sample had been 2.1 

13    percent, that would have equaled about 8,400 people, right?

14         A.    2.1 percent times 400,000 is equal to 8,400.

15         Q.    And appendix C, I believe, of your report to the 

16    Secretary, that is Plaintiff's Exhibit 195, I believe you 

17    testified makes no mention of P6?

18         A.    P6 was not included in the total error model so 

19    we didn't see fit to measure it.

20               (Continued on the next page.)

21    

22    

23    

24    

25    
                                                              442

 1         Q.    After all the PES field operations were 

 2    completed, including the production follow-up operations, 

 3    there were people in both the E-Sample and the P-Sample 

 4    whose match status was unresolved, is that right?

 5         A.    There were people in the P-Sample and E-Sample 
                                                                

 6    whose matched status was unresolved, that is correct.

 7         Q.    And you have no opinion whether the unresolved 

 8    people in the PES were more like people who were resolved in 

 9    evaluation follow-up or who remained unresolved in the 

10    evaluation follow-up, is that right?

11         A.    I'm afraid I don't understand your question.

12         Q.    Well, were the people in the PES who remained 

13    unresolved more like the people who were resolved in the 

14    evaluation follow-up or who remained unresolved in the 

15    evaluation follow-up?

16         A.    To say that someone was resolved in the 

17    evaluation follow-up, people were resolved and unresolved in 

18    the post-enumeration survey.

19         Q.    Well, let me see if this helps you answer the 

20    question.

21               Would you turn to page 538 of your deposition.  

22               MR. ZIMROTH:  Here's another sample, your Honor.  

23    The witness has not said that he needed his recollection 

24    refreshed.  He just said he didn't understand the question.

25               THE COURT:  I'll permit it.
                                                              443

 1         Q.    On line 17, I asked you the following question:

 2               "Q.    After all the follow-up operations in the 

 3    PES were completed, were there people in both the E-Sample 

 4    and the P-Sample whose match status was unresolved?

 5               "A.    You mean after all the fieldwork was 

 6    complete? 

 7               "Q.    Yes.

 8               "A.    Yes, there were.

 9               "Q.    In your opinion, were those unresolved 

10    people in the PES more like the people who were resolved in 

11    the evaluation follow-up or who were unresolved in the 

12    evaluation follow-up?

13               "Mr. Corwin:  Could you read that back.

14               "The pending question was read.

15               "A.    I don't have an opinion on that question 

16    at this time."

17               Does that passage that I just read to you 

18    correctly state that you don't have an opinion about whether 

19    unresolved people in the PES are more like people resolved 

20    in the evaluation follow-up or who remain unresolved in the 

21    evaluation follow-up?

22         A.    I guess your question was as clear to me then as 

23    it is now, consequently, I didn't have an opinion.

24         Q.    Okay.  Well, you couldn't tell me at your 

25    deposition whether you could even consider this question, 
                                                              444

 1    could you?  

 2               MR. ZIMROTH:  He said he couldn't understand the 

 3    question, so how could he consider it?

 4               THE COURT:  Move on.  

 5               MR. SITCOV:  Well, your Honor, if I just may ask 

 6    your indulgence, I asked him the question:  

 7               "Have you ever considered that question?"  

 8               And he answered:  

 9               "I may have.  At this time I don't remember 

10    considering that specific question."  

11    BY MR. SITCOV:

12         Q.    Does that refresh your recollection that you 

13    don't know whether or not you considered that specific 

14    question?

15         A.    Sure does.

16         Q.    After all the fieldwork was done in the 

17    post-enumeration survey, models were used to impute 

18    probabilities for the unresolved people, is that right?

19         A.    Models were used to impute probabilities to the 

20    unresolved people, that is correct.

21         Q.    And these were mathematical models?

22         A.    We can call them that, yes.

23         Q.    For the unresolved people in the P-Sample, a 

24    probability of match was imputed, is that right?

25         A.    That's right.
                                                              445

 1         Q.    And for the unresolved people in the E-Sample, a 

 2    probability of correct enumeration was imputed, right?

 3         A.    Right.

 4         Q.    Is that right?

 5         A.    That's right.

 6         Q.    In the model used to impute match status for the 

 7    P-Sample unresolved was different from the model that was 

 8    used to impute correct enumeration status for the E-Sample 

 9    that was unresolved, is that right?

10         A.    Would you do that one again?

11         Q.    Sure.  Were they two different models, the 

12    E-Sample model and the P-Sample model?

13         A.    To the best of my recollection, yes.

14         Q.    And neither had been used before the 1990 PES, 

15    had they?

16         A.    The Census Bureau engaged in a decade of testing 

17    those computation models.  They continued to improve and 

18    develop and make changes.  To the best of my recollection, 

19    there were changes in the model between the 1988 dress 

20    rehearsal and the 1990 Census.  To the best of my 

21    recollection, those changes did not have a major impact.

22         Q.    Well, they weren't the same models that had been 

23    used before?

24         A.    No, they were not the same models.

25         Q.    Then were the differences major or minor?  
                                                              446

 1               MR. ZIMROTH:  He just answered that.

 2         A.    The impact of the differences appeared to be 

 3    minor.

 4         Q.    Do you recall that at your deposition you 

 5    couldn't tell me whether or not the dissimilarities were 

 6    fundamental or important?

 7         A.    I don't remember the discussion at this time.

 8         Q.    Okay.  Well, could you turn to page 544 of your 

 9    deposition.  Actually, page 543.

10               Starting on line 24, I asked you the question:

11               "Q.    You testified that there were models used 

12    to determine the probability of match for correct 

13    enumeration and that there was a different model for the 

14    E-Sample and the P-Sample.  I asked you:  Had either -- and 

15    I just want to know, had either model ever been used before?  

16               "Mr. Zimroth:  You mean exactly the same model?  

17    That's what you want to know?  

18               "Mr. Sitcov:  Yes.

19               "The Witness:  You do mean exactly the same 

20    model? 

21               "Q.    Yes.

22               "A.    No, they weren't exactly the same.

23               "Q.    Okay.  Are you thinking of a similar model 

24    that had been used before?

25               "A.    To the best of my recollection, there were 
                                                              447

 1    similarities between the models that had been used in the 

 2    dress rehearsal censuses in the PES.

 3               "Q.    Were the dissimilarities fundamental or 

 4    important or were they just minor?  

 5               "Mr. Zimroth:  Objection.  What do you mean 

 6    fundamental Mr. Sitcov?  Important to whom and for what 

 7    purpose?  

 8               "Mr. Sitcov:  If the witness doesn't understand 

 9    my question, he'll tell me.  

10               "The Witness:  I don't know what you mean by 

11    fundamental."

12               Does that refresh your recollection that you 

13    couldn't tell me whether the differences were fundamental in 

14    the two models?

15         A.    At that time, I could not remember the 

16    differences in the two models. 

17         Q.    You are aware, aren't you, that during evaluation 

18    follow-up, a subsample of E- and P-Sample unresolved cases 

19    were reinterviewed, is that right?

20         A.    That's right.

21         Q.    And that was done by selecting a sample of 

22    unresolved cases and sending interviewers into the field to 

23    gather more information, is that right?

24         A.    Yes.  My understanding is that they went into the 

25    field to find as many of the unresolved cases as they could.  
                                                              448

 1    This was about February 1991, to see if at that time they 

 2    could possibly get some information that would allow them to 

 3    resolve it.

 4         Q.    In the evaluation follow-up, the evaluation 

 5    follow-up interviewers had the original PES forms completed 

 6    for each of the subjects of the study, didn't they?

 7         A.    I believe they did.  I'm not certain about that.

 8         Q.    Those were not independent reinterviews, were 

 9    they?

10         A.    Well, if the intention was to try to assign them 

11    a status other than unresolved, and all they knew was 

12    whatever the characteristics were, then I don't think the 

13    concept of independence and dependence would apply there.

14         Q.    So are you saying that you don't know if it was 

15    an independent reinterview?

16         A.    No, what I'm saying is that it's not like a 

17    matching study or some other kind of study where you want to 

18    do a second test to see if you got it right the first time. 

19               In this case, they knew it had been decided 

20    unresolved and they were going to try to gather some more 

21    information so they could assign it to a resolved or 

22    unresolved status.  

23               So what I'm telling you is that the concept of 

24    independence doesn't really apply in this situation.

25         Q.    If it did apply, would that be an independent 
                                                              449

 1    reinterview?

 2         A.    Doesn't make -- your question makes no sense.  I 

 3    can't answer that question.

 4         Q.    Well, all right.

 5               Could you turn to Defendant's Exhibit 111.

 6               MR. SITCOV:  Now, this is another P study, and 

 7    this is -- so I take it, it is in Plaintiff's Exhibit 540 A, 

 8    and I wouldn't know how to tell you any closer than that, 

 9    your Honor, what it looks like.  

10               MR. ZIMROTH:  Which P study is it?  

11               MR. SITCOV:  P 3.  

12               MR. ZIMROTH:  Then it should be A 3.  And this 

13    one is dated July 10, 1991, your Honor.

14    BY MR. SITCOV:

15         Q.    Do you have it, Dr. Ericksen?

16         A.    I have Exhibit 111.

17         Q.    Okay.  And is that the final report for 1990 PES 

18    evaluation project P 3 evaluation of imputation methodology 

19    for unresolved match status cases?

20         A.    It is.

21         Q.    This report contains the results of the 

22    evaluation rematching study we were just talking about, 

23    doesn't it?

24         A.    It does.

25         Q.    Would you turn to page 3, please.
                                                              450

 1               And I direct your attention to the second full 

 2    paragraph.  And the paragraph reads:  

 3               "The second comparison is between the overall 

 4    probability of match for PES unresolved cases (based on 

 5    production imputation results) and the results from the 

 6    evaluation follow-up.  However, the percent of P-Sample 

 7    cases still resolve after evaluation follow-up" -- 

 8               MR. ZIMROTH:  Excuse me.  I think Mr. Sitcov is 

 9    misreading what's on the page.  

10               MR. SITCOV:  I am?

11               THE COURT:  Yes.  

12    BY MR. SITCOV:

13         Q.    Let me start again.

14               "The second comparison is between the overall 

15    probability of match for PES unresolved cases (based on 

16    production imputation results) and the results from an 

17    evaluation follow-up.  However, the percent of P-Sample 

18    cases still unresolved after evaluation follow-up is 58.55 

19    percent which limits any further inferences.  Even so, the 

20    match probability for PES unresolved cases is 41.82 percent 

21    compared to 37.16 percent overall for these cases in the 

22    evaluation follow-up (when counting 41.82 percent of the 

23    evaluation follow-up unresolved persons as matches)." 

24               Now, you considered this paragraph to be evidence 

25    in favor of the imputation model, don't you?
                                                              451

 1         A.    Within the limits of its applicability giving the 

 2    58.55 percent, the results of the P 3 study in general 

 3    supported the use of the imputation model.

 4         Q.    Now, this study evaluation follow-up data was 

 5    used to study the imputation model, is that right?

 6         A.    That's right.

 7         Q.    and the results of the imputation model were used 

 8    to fill in the missing data in the evaluation follow-up, is 

 9    that right?

10         A.    That's right.

11         Q.    Would you please turn to page 10 of the P 3 

12    report.

13         Q.    The chart or the table that is identified as 

14    Table 3.1 presents the results that were reported in the 

15    paragraph under page 3 that I just asked you about, is that 

16    right?

17         A.    That's right.

18         Q.    And the numbers at the tops of the columns on 

19    this chart represent the imputed match probabilities in 

20    production, correct?

21         A.    That's right.

22         Q.    Okay.  In other words, the first column contains 

23    data for those unresolved P-Sample persons for whom the 

24    model predicted a zero to 25 percent probability of being a 

25    match, right?
                                                              452

 1         A.    That's right.

 2         Q.    Is that right?

 3         A.    Right.

 4         Q.    And the last column contains data for those 

 5    unresolved P-Sample persons for whom the model predicted 

 6    there was a 75 to a hundred percent probability of being a 

 7    match, is that right?

 8         A.    That's right.

 9         Q.    And if the model was working properly, the 

10    percentage of cases matched in evaluation follow-up should 

11    increase as the imputed match probability increases, is that 

12    right?

13         A.    Not completely.  The problem with what your 

14    question is asking is that when they went back out in 

15    February 1991, which was in general about six months after 

16    PES, the unresolved low of the proportion of the cases, they 

17    still couldn't tell whether they should match them or not.  

18    And in one column it was 77.25 percent of the cases they 

19    still couldn't tell.  

20               The only evidence that you can get from this 

21    table would be the ratio of those determined to be matched 

22    and those determined to be unmatched; and as you go from the 

23    zero to 25 percent of the 6.38, and you add that to the 

24    43.82, and that gives you a proportion which is between zero 

25    and 25 percent.  And those percentages go up as the impute 
                                                              453

 1    exceed match probability goes up.

 2         Q.    Well, my question is:  If the model was working 

 3    properly, shouldn't the percentage of cases matched in the 

 4    evaluation follow-up increase as the imputed matched 

 5    probability increases?

 6         A.    The answer is yes, as long as the denominator in 

 7    that calculation is the sum of matched and unmatched cases.  

 8    You can't include the unresolved cases in the evaluation 

 9    follow-up because ten months after the census, they still 

10    couldn't get the information they needed to make a 

11    resolution.

12         Q.    Well, Table 3.1 shows that the match probability 

13    assigned by the imputation model -- that as the matched 

14    probability assigned by the imputation model increases from 

15    50 to 75 percent to 75 to a hundred percent, the match rate 

16    in the evaluation follow-up declined, isn't that right?

17         A.    No, it's not right.

18         Q.    Well, in the 50 to 75 percent column, 16.54 

19    percent of the cases were matches, is that right?

20         A.    That is correct.

21         Q.    And in the 75 to a hundred percent column, 15.21 

22    percent of the cases were matches.

23         A.    Right.  But that's not the relevant statistic.  

24    The relevant statistic is the ratio of the 16.54 to the sum 

25    of 16.54 plus 15.38.  And that ratio is just over 50 percent 
                                                              454

 1    which corresponds to the category 50 to 75 percent.  That's 

 2    what you can tell; the fact that ten months after the census 

 3    65.94 percent of those cases was unresolved, that's all you 

 4    can say.

 5         Q.    The chart also shows that as match probability 

 6    assigned by the model goes up, the percentage unresolved in 

 7    the evaluation follow-up goes up rather than down, isn't 

 8    that right?

 9         A.    The proportion that were still unresolved is 

10    greater at the right side of the table than it is at the 

11    left side of the table.

12         Q.    It's greater than the column where the model 

13    predicted there would be the most matches, isn't that right?

14         A.    That's right.  

15         Q.    You wouldn't use a model if you didn't know its 

16    assumptions, would you?

17         A.    As a practical statistician, am I ever to 

18    understand the assumptions of a model.  I'm sure in real 

19    life there have been some that I have missed as would be 

20    true of any other statistician I know.

21         Q.    My question is:  Would you use a model if you did 

22    not know its assumptions?

23         A.    In working in the real world, I'm sure I have in 

24    the past.  Once the assumptions are pointed out to me that 

25    I've missed, I try to incorporate them in my thinking about 
                                                              455

 1    it.

 2         Q.    Well, can you turn to page 555 of your deposition 

 3    transcript.

 4               Actually, turn to page 554, line 19:

 5               "Q.    Would you use a model before you knew the 

 6    assumption 

 7               "Mr. Zimroth:  Mr. Sitcov, this is an area that 

 8    we went through in some length previously.  I'm not going to 

 9    direct the witness not to answer.  I'm just pointing it out 

10    so perhaps you'll go on to another area.  If you insist on 

11    repeating this area, I'll allow the witness to answer once 

12    again, if you want to continue with the pending line.  Do 

13    you want to have continue with the pending line?           

14               "Mr. Sitcov:  Yes.  

15               "Can you read the pending question, please.

16               "The pending question was read.

17               "A.    Would you use a model before you knew its 

18    assumptions?  

19               "Mr. Sitcov:  Yes.

20               "A.    No.  In general you would want to consider 

21    the assumptions of the model before you would use them.?  

22               Does that refresh your recollection as you 

23    testified in your deposition that you would want to know -- 

24               MR. ZIMROTH:  That's precisely what he testified 

25    to on his cross-examination, your Honor.
                                                              456

 1               THE COURT:  I'll permit it.

 2         Q.    (Continuing)  -- that you would want to know the 

 3    assumptions before you would use the model?

 4         A.    I think that's what I just said.

 5         Q.    Okay.  At your deposition five weeks ago you 

 6    testified you didn't know the assumption in the production 

 7    model imputing match status for P-Sample unresolved or 

 8    correct enumeration status for E-Sample unresolved, isn't 

 9    that right?

10         A.    That's because, as I told you before, I didn't 

11    know exactly the model that the Census Bureau had decided to 

12    use.

13         Q.    And you didn't even know which, if any, of the 

14    P-Studies described those assumptions, did you?

15         A.    The P-Studies didn't describe the assumptions.

16         Q.    I'm sorry?

17         A.    The P-Studies did not describe the assumptions.

18         Q.    And you didn't know those assumptions at the time 

19    you wrote the report to the Secretary, did you?

20         A.    As I told you before, I didn't know exactly which 

21    model the Census Bureau had decided to use.

22         Q.    Parameters in the imputation models and P-Sample 

23    unresolved were estimated using data from the PES and PES 

24    follow-up, is that right?

25         A.    I need that question again.
                                                              457

 1         Q.    Were parameters in the imputation models for the 

 2    E- and P-Sample unresolved estimated using data from the PES 

 3    and PES follow-up?

 4         A.    I believe they were, yes.

 5         Q.    And estimating parameters in the imputation 

 6    models using data from PES and PES follow-up requires an 

 7    assumption that the final unresolved group are the same as 

 8    persons resolved in PES follow-up, is that right?

 9         A.    I don't remember that particular detail at this 

10    time.

11         Q.    Estimating parameters in the imputation models -- 

12    okay, I think I just asked you that.  

13               You are aware, aren't you, that weighted to the 

14    national totals there were about 4 million unresolved cases 

15    in the P-Sample and another 3 million in the E-Sample, is 

16    that right?

17         A.    I've thought of those quantities as percentages.  

18    Those numbers don't sound far off, but I haven't actually 

19    made that calculation.

20         Q.    So if those numbers are right, there were a total 

21    of 7 million unresolved in the E-Sample and the P-Sample?

22         A.    If those numbers are right, 4 million plus 3 

23    million equals 7 million.

24         Q.    And that would be compared to the national 

25    undercount estimate of about 5 million people, is that 
                                                              458

 1    right?

 2         A.    Well, the national undercount was 5 million.

 3         Q.    Okay.  Do you have Plaintiff's Exhibit 502 in 

 4    front of you, or can you find it?  That is some of the stuff 

 5    that was introduced by Mr. Zimroth yesterday.

 6               That's an ethnographic evaluation of the 1990 

 7    Decennial Census report series?

 8         A.    Yes, it is.

 9         Q.    Okay.  I'd like you to consider the following 

10    hypothetical situation.

11               We are in a big city in the southwest, and we 

12    have a community with the following groups, four groups: 

13               Recent immigrants from Mexico and Central 

14    America, native-born Mexican Americans, Blacks and Anglos.

15               In your opinion, which group would have the 

16    highest census omission rate?

17         A.    Give me the groups again, please.

18         Q.    Okay.  Recent immigrants from Mexico and Central 

19    America, native-born Mexican Americans, Blacks and Anglos.  

20               MR. SITCOV:  And blacks what?

21               MR. SITCOV:  Anglos.  

22               MR. ZIMROTH:  Sorry.

23         A.    I don't know anything else about the economic 

24    characteristics or anything else about them?

25         Q.    No.
                                                              459

 1         A.    I don't think that I want to rank order them.  In 

 2    general, I would consider the Blacks and the native-born 

 3    Mexican Americans and the recent immigrants especially to 

 4    the extent they're poor to have a high probability of 

 5    undercount.  

 6               In terms of quantifying the differences, without 

 7    more information, that's not something I can do at this 

 8    time.

 9         Q.    In your opinion, which group would have the 

10    lowest census omission rate?

11         A.    All, depending on the characteristics of the 

12    Anglos, they might have the lower, but you'll have to tell 

13    me more about their characteristics before I can really 

14    answer that question.

15         Q.    Okay.  If I showed you ethnographic data which 

16    demonstrated that the Anglos had the highest census omission 

17    rate, would you change your opinion or would you reject the 

18    data?

19               THE COURT:  Would you change your opinion what?  

20               MR. SITCOV:  Or would you reject the data.

21               THE COURT:  I don't know that his opinion is firm 

22    enough to be changed.  

23               MR. SITCOV:  I thought his opinion was that he 

24    thought that the Anglos would probably have the lowest 

25    undercount rate, omission rate.
                                                              460

 1               THE COURT:  I don't think it was quite that 

 2    clear.  

 3    BY MR. SITCOV:

 4         Q.    Okay.  Did you think that the Anglos would have 

 5    the lowest?

 6         A.    No.  I said I'd need to have more information 

 7    about the other characteristics of the Anglos to really have 

 8    an opinion on that.  

 9               MR. SITCOV:  Could I have just one moment, your 

10    Honor?

11               THE COURT:  Sure.

12               (Pause) 

13               MR. SITCOV:  I don't think I have any further 

14    questions at this time.

15               (Pause)

16               THE COURT:  Mr. Zimroth, any redirect?  
                                                         

17               MR. ZIMROTH:  I do, and I would appreciate very 

18    much that we have a few minutes to organize my thoughts.

19               THE COURT:  If we have to wait until you organize 

20    your thoughts, we will be here until next January.

21               How much time do you want?  

22               MR. ZIMROTH:  15 minutes.

23               THE COURT:  And then we'd come back and go for 15 

24    minutes?  

25               MR. ZIMROTH:  Sure.
                                                              461

 1               THE COURT:  Will you finish in that time?  

 2               MR. ZIMROTH:  No.  Would it be better to just 

 3    adjourn now until this afternoon?

 4               THE COURT:  Why waste 15 minutes?  

 5               MR. ZIMROTH:  Okay, fine.

 6               THE COURT:  Why don't you plough into it and then 

 7    sort it out later.  

 8               MR. ZIMROTH:  Excuse me?

 9               THE COURT:  Why don't you just plough into it and 

10    sort it all out during lunch.  

11               MR. ZIMROTH:  I would really prefer, if you don't 

12    mind.

13               THE COURT:  All right.  Ten minutes.  

14               MR. ZIMROTH:  Thank you, your Honor.

15               THE COURT:  Let's take a ten-minute break.

16               (Recess)

17               (In open court) 

18               MR. MILLET:  Your Honor, if the Court please, may 

19    I raise another housekeeping matter?

20               THE COURT:  Sure.  

21               MR. MILLET:  During the break, we learned from 

22    the plaintiffs that they do intend to change the order in 

23    which they had advised us and the Court that they were 

24    calling their witnesses.  I understand now that Dr. Bailar, 

25    who is due to be following Dr. Wolter, their next witness, 
                                                              462

 1    will now be put back into some time next week, and that they 

 2    will be changing the order of some of their other witnesses 

 3    as well.

 4               I can only repeat the request that I've made many 

 5    times before already, that the plaintiffs please provide us 

 6    with all of their exhibits organized by witness, as we have 

 7    done for them, so that we can promptly review them and have 

 8    the proper people look at them.

 9               Plaintiffs during the break indicated to me that 

10    they don't have copies available to us.  I find that quite 

11    surprising, quite frankly, with all the resources they have 

12    at their disposal, that they are not able to spare us an 

13    extra set and tell us what order they are in.  

14               And I would ask the Court that they be instructed 

15    to do so, especially now that they are changing the order on 

16    us.

17               THE COURT:  Mr. Zimroth, or Mr. Rifkind?  

18               MR. ZIMROTH:  Your Honor, my understanding is 

19    that these exhibits are on their way.

20               THE COURT:  On their way?  

21               MR. ZIMROTH:  My understanding is that these 

22    exhibits are on their way.

23               THE COURT:  On their way to where, Prague?  

24               MR. RIFKIND:  To here from my office. 

25               MR. MILLET:  For each witness?  
                                                              463

 1               MR. RIFKIND:  You have -- 

 2               MR. ZIMROTH:  You have the next witness.  

 3               MR. SITCOV:  And I received Estrada at 2 o'clock 

 4    this morning.  

 5               MR. ZIMROTH:  And you will have Rolph.  That's on 

 6    its way down.  

 7               MR. MILLET:  How about the rest of them?  

 8               MR. ZIMROTH:  The rest of them we don't have.  

 9    You have the next three witnesses.

10               MR. MILLET:  That's what we are asking for.

11               THE COURT:  The next three?  

12               MR. MILLET:  We'd like to have all of them, just 

13    as our people have provided them, so our people can get to 

14    work on it.

15               THE COURT:  Can you accommodate them?  

16               MR. ZIMROTH:  We are giving them as quickly as we 

17    can.  Very shortly they will have three witnesses in 

18    advance.  

19               MR. RIFKIND:  Let me put it this way, your Honor: 

20    By lunch they will have all the witnesses that appear this 

21    week.  And before the end of the week, they will have at 

22    least the beginning of the following week's witnesses.  

23               MR. MILLET:  May we also have a representation 

24    for the record the order of their expected witnesses?  

25               MR. RIFKIND:  The representation of the record, 
                                                              464

 1    which I have just given --

 2               THE COURT:  Why don't you do it at the end of the 

 3    day.  

 4               MR. RIFKIND:  Okay.  

 5               MR. MILLET:  Thank you, your Honor.

 6               THE COURT:  Okay.

 7               Mr. Zimroth, have you organized your thoughts?  

 8               MR. ZIMROTH:  No, but I'm ready to proceed 

 9    anyway.

10    EUGENE P. ERICKSEN, resumed

11    REDIRECT EXAMINATION 
                           

12    BY MR. ZIMROTH:

13         Q.    Professor Ericksen, could you please look at 

14    P 16.  That would be in 540 A.16.

15               (Discussion off the record) 

16               MR. SITCOV:  Your Honor, do you have the one that 

17    I marked as 106?  It might be easier for you to find.

18               THE COURT:  Thank you.

19               (Pause)

20               THE COURT:  I think we are all in sync.  

21               MR. ZIMROTH:  Okay.

22         Q.    Will you please turn to table 14, Professor 

23    Ericksen.

24         A.    Okay, I have it.

25         Q.    On your cross-examination Mr. Sitcov showed you 
                                                              465

 1    that chart, did he not?

 2         A.    Yes, he did.

 3         Q.    And will you take a look at the bottom row, which 

 4    is "national."

 5         A.    I see it.

 6         Q.    And do you see the numbers 2.11 and 0.49?

 7         A.    I do.

 8         Q.    Mr. Sitcov, in his cross-examination, asked you 

 9    to divide one number into the other.

10         A.    That's right.

11         Q.    He asked you to divide 0.49 into 2.11.  Is that 

12    right?

13         A.    It's actually the other way around.

14         Q.    I'm sorry.  To divide 2.11 into 0.49?

15         A.    That's right.

16         Q.    And you accommodated him and you did that?

17         A.    I did that.

18         Q.    In your opinion, does that arithmetical exercise 

19    tell you anything meaningful about the PES?

20         A.    No, I don't believe it does.

21         Q.    Does it tell you anything meaningful about the 

22    differential undercount?

23         A.    No, it does not.

24         Q.    Could you explain to the Judge why that is?

25         A.    Yes.  The objective of the post-enumeration 
                                                              466

 1    survey was to produce adjustments of population in different 

 2    states and localities so that we could correct the problem 

 3    of the distribution of population that's occasioned by the 

 4    undercount of people.  And the Census Bureau designed the 

 5    PES to do that.

 6               And as part of the evaluation of the post- 

 7    enumeration survey, they defined categories which are known 

 8    as evaluation poststrata.  And those are the groups that are 

 9    indicated on this table from 1 to 13.  They group the 1392 

10    poststrata into 13 categories.  For example, evaluation 

11    poststratum 1 is all minority poststrata in central cities 

12    in the northeast.

13               And evaluation poststratum 2 is nonminority 

14    people living in central cities in the northeast.  And you 

15    can see that the estimated undercount for the first 

16    evaluation poststratum is 6.83, and the second was minus 

17    0.75.

18               Now, the rest of the evaluation poststrata is 

19    defined similarly for different areas.  And the question is 

20    whether the difference in the undercount between areas, 

21    particularly between areas where minorities are 

22    concentrated, where the undercounts are high, and areas 

23    where minorities are few and undercounts are low, is 

24    affected by the bias.  So what one would need to do would be 

25    to look at the effect of the bias on the differences in the 
                                                              467

 1    net undercounts between groups and areas.

 2         Q.    Did the Census Bureau do this division exercise 

 3    that Mr. Sitcov asked you to do?  

 4               MR. SITCOV:  Object.  I feel it would be 

 5    speculative, your Honor.

 6               THE COURT:  Overruled.  

 7               You may answer if you know.

 8               THE WITNESS:  No, this was not the -- the Census 

 9    Bureau, I'm sure, made that calculation at some point, but 

10    that was not the major point made in the reporting of PES by 

11    the Census Bureau.

12         Q.    Okay.  Did you do the -- did you look at these 

13    figures in light of what you think -- what you have 

14    testified is the appropriate purpose --

15         A.    Yes, I did.

16         Q.    -- about these evaluations?

17         A.    Yes, I did.

18         Q.    And what did you find?

19         A.    What I found was that the use of the PES to 

20    supplement the original enumeration increased the population 

21    share of areas of minority areas.  And when one incorporated 

22    the estimated bias for those areas from the PES, it made a 

23    very small effect on the correction provided by the 

24    post-enumeration survey.

25         Q.    And did the Census Bureau do a similar analysis?
                                                              468

 1         A.    Yes, it did.

 2         Q.    And what did it find?

 3         A.    That was a loss function analysis that we've 

 4    talked about, and it found that when the bias and the PES 

 5    was taken into account, the effect on the distribution of 

 6    population was such that it deemed the original enumeration 

 7    supplemented by the PES to be more accurate than the 

 8    original enumeration by itself.

 9         Q.    Now, could you look at page -- I'm sorry.  One 

10    other thing.

11               Could you look at PX 9, which is the Secretary's 

12    decision.  Page 2-23.

13               Do you have it?

14         A.    I do.

15         Q.    Do you see the sentence that talks about 1.4 

16    percent and the 2.1 percent?

17         A.    I do.

18         Q.    Is the Secretary there doing this arithmetical 

19    task that Mr. Sitcov asked you to do?  

20               MR. SITCOV:  I object.  He can testify if he 

21    knows what it is.  I don't know that he can say what the 

22    Secretary did.

23         Q.    What is that there?

24         A.    The sentence that reads: 

25               "The model does show that the PES is biased to 
                                                              469

 1    have overestimated the undercount, and that a biased 

 2    estimate of the undercount would be about 1.4 percent rather 

 3    than the production estimate of 2.1 percent."  

 4               That's making a statement that one-third of 2.1 

 5    percent is biased and deducting it from the national 

 6    undercount.

 7         Q.    And is that, in essence, what Mr. Sitcov asked 

 8    you to do in this exercise?

 9         A.    It is.

10         Q.    And, in your opinion, does that say anything 

11    meaningful about the PES or about the net undercount or -- 

12    excuse me -- or about the differential undercount?

13         A.    No, it does not.

14         Q.    Could you look at page 20 of your report, which 

15    is, I believe, PX 195.

16         A.    Okay.

17         Q.    Is that a loss function analysis that you have 

18    there?

19         A.    No.  It's a discussion of what we anticipate a 

20    loss function analysis would show.

21         Q.    And what did you anticipate a loss function 

22    analysis would show?

23         A.    That the distribution of undercount for states 

24    and other areas would be improved by use of the PES.

25         Q.    And at the time you wrote this, did you have 
                                                              470

 1    available to you the results of the Census Bureau's loss 

 2    function analysis?

 3         A.    No.

 4         Q.    But, subsequently, you have seen those results?

 5         A.    That is correct.

 6         Q.    And do they comport with what you predicted that 

 7    they would show?

 8         A.    Yes, they do.

 9         Q.    Now, Professor Ericksen, Mr. Sitcov asked you 

10    about the report, I think it was, P5a.  Is that right?

11               Do you remember that?

12         A.    Yes, I do.

13         Q.    And he accused you of selective incorporation?

14         A.    Yes, he did.

15               MR. SITCOV:  Object, your Honor.

16               THE COURT:  Overruled. 

17         Q.    Did you have the P5a, the final report of P5a 

18    available to you at the time you wrote your report?

19         A.    No, I did not.

20         Q.    When you wrote your report, did you understand 

21    that the Secretary would have available to him information 

22    that was not available to you?

23         A.    Yes, I did.

24               MR. ZIMROTH:  I have nothing further, your Honor.

25               THE COURT:  Any recross?  
                                                              471

 1               MR. SITCOV:  I think I may.  It's 5 of 12.  Could 

 2    we break now?

 3               THE COURT:  Well, I think they are trying to get 

 4    him on an airplane.  

 5               MR. ZIMROTH:  Yes, I'm trying to get him out of 

 6    here, your Honor.  

 7               MR. SITCOV:  Could I have just one second, your 

 8    Honor?

 9               THE COURT:  Yes.

10               (Pause) 

11    RECROSS-EXAMINATION 
                          

12    BY MR. SITCOV: 

13         Q.   Dr. Ericksen, you had the tables for all the 

14    P-Studies available to you before the Secretary's decision, 

15    didn't you?

16         A.    I did have tables.  Whether I had all the tables, 

17    I have not made that check, to answer your question yes or 

18    no.  

19               MR. SITCOV:  No other questions.  

20               MR. ZIMROTH:  That's it, your Honor.

21               THE COURT:  Pleasure, Doctor.

22               THE WITNESS:  I have enjoyed it.  Thank you.

23               (Witness excused)
                                  

24               THE COURT:  All right.  We'll recess and resume 

25    at 2:15.  
                                                              472

 1               MR. RIFKIND:  I'd like to take a minute of 

 2    housekeeping and use these three minutes.

 3               THE COURT:  All right.  

 4               MR. RIFKIND:  I was asked to make a 

 5    representation and I'd like to give a notice, if I may.  

 6               You may step down.  

 7               The order of our witnesses, is Dr. Wolter, Dr. 

 8    Rolph, Professor Estrada, Dr. Fisher, Dr. Cain, Dr. Bailar, 

 9    Professor Tukey, Dr. Fienberg.  

10               The change resulted because we were running a 

11    little behind of what we hoped to be.  And one witness had 

12    an engagement that we were going to smash into, so we moved.

13               Let me give a notice then, if I may.

14               We have a stipulation that I referred to in 

15    opening covering the supposed confidentiality of the 

16    adjusted tapes, the tapes that give you the corrected data 

17    for every block of the country.  

18               For reasons not apparent to us, although the 

19    government was finally persuaded by the Magistrate to 

20    produce those tapes to us, they did so only under a 

21    confidentiality agreement which we reluctantly acceded to in 

22    the interest of getting them.  That confidentiality 

23    agreement provides that on three days notice, we may move to 

24    lift the confidentiality associated with it, and I give 

25    notice now that on Monday morning I will so move.
                                                              473

 1               I think I want to avoid arguments that some of 

 2    the references we may wish to make in next week's testimony 

 3    would breach the confidentiality if uttered in this open 

 4    courtroom.  I'd like to avoid having to seal the courtroom, 

 5    or make other strange arrangements for dealing with that 

 6    sort of evidence, and I will assert that I see no plausible 

 7    basis for keeping the stuff confidential at this point.           

 8    But that can wait until Monday.  I'm required by the 

 9    stipulation to give three days notice.

10               THE COURT:  May I ask that you confer with the 

11    government in the hopes that you can resolve this amicably 

12    among yourselves?  

13               MR. MILLET:  I was just going to suggest that.  

14               MR. RIFKIND:  Okay.  I raise it now -- actually, 

15    we asked them if it could be waived by letter of last week, 

16    and we were told no.  But I'm willing to try again.

17               THE COURT:  Well, I find -- 

18               MR. RIFKIND:  And I adhere their suggestion.

19               THE COURT:  I find you to be a reasonable, 

20    amicable and congenial group.  If they think it will we 

21    done, we'll do it.  

22               MR. RIFKIND:  We'll do our best.

23               (Luncheon recess)  

24    

25    
                                                              474

 1               A F T E R N O O N     S E S S I O N.

 2                                      2:30 o'clock p.m.           

 3               MR. SOLOMON:  Plaintiffs call Dr. Kirk Wolter.

 4    KIRK M. WOLTER, 

 5             called as a witness by the plaintiff, having

 6             first been duly sworn, was examined and

 7             testified as follows: 

 8               MR. SOLOMON:  We have given to your clerk one 

 9    binder of documents that we will be referring to and I would 

10    like to give Dr. Wolter a set as well.  

11               We have given them to the government a couple of 

12    days ago.  

13    DIRECT EXAMINATION 

14    BY MR. SOLOMON:

15         Q.    Dr. Wolter, what is your occupation?
                                                     

16         A.    I'm a statistician and a manager.

17         Q.    Where are you currently employed?

18         A.    I'm employed at the A.C. Nielsen Company in 

19    Northbrook, Illinois.

20         Q.    How long have you been employed at Nielsen?

21         A.    Roughly three and a half years.  It will be four 

22    years this September.

23         Q.    Since about September of 1988, is that right?

24         A.    That's correct.

25         Q.    Prior to September of 1988, where were you 
                                                              475

 1    employed?

 2         A.    I was employed full-time at the U.S. Bureau of 

 3    the Census, I was employed part-time at the George 

 4    Washington University.

 5         Q.    For how long were you employed at the Census 

 6    Bureau?

 7         A.    Fourteen years.

 8         Q.    Dr. Wolter, have you served as a member of the 

 9    special advisory panel that was appointed pursuant to the 

10    stipulation and order entered by Judge McLaughlin?

11         A.    Yes, I did.

12         Q.    What is your position at A.C. Nielsen?

13         A.    I am vice president statistical design worldwide.

14         Q.    And as vice president statistical design 

15    worldwide, what generally are your responsibilities?

16         A.    I am responsible for statistical methodology 

17    worldwide.

18               We operate in 28 countries.  No statistical or 

19    other related quantitative procedures are used in the 

20    company without my approval.  The buck stops with me.

21         Q.    Describe briefly for the court the business of 

22    A.C. Nielsen?

23         A.    Nielsen is in two general areas of business.  One 

24    is media research, the other is marketing research.

25               We are perhaps better known for our media 
                                                              476

 1    research.  These are the well-known Nielsen television 

 2    ratings.

 3               In the marketing research area, we produce 

 4    information on the total sales volume of all consumer 

 5    packaged goods sold in the United States and in 27 other 

 6    countries worldwide.

 7               We also publish considerable information about 

 8    the promotional conditions or causal conditions under which 

 9    consumer sales occur.

10         Q.    What is the chief statistical technique that you 

11    as the person where the buck stops at Nielsen employ for the 

12    business of Nielsen that you just described?

13         A.    The field of survey sampling really is the 

14    bedrock or foundation underlying all of the work done at the 

15    Nielsen Company in both media research and marketing 

16    research.

17         Q.    Dr. Wolter, are you familiar with the 

18    post-enumeration servey or the PES that was conducted by the 

19    Bureau in connection with the 1990 census?

20         A.    Yes, I am.

21         Q.    We will be discussing the PES in great detail.

22               Let me ask you now, is that a sample survey?

23         A.    Yes, it is.

24         Q.    Would you give the court some idea of the 

25    dimension of the use of sample surveying in the media 
                                                              477

 1    research business of Nielsen?

 2         A.    This is a, this is a huge business bringing in 

 3    hundreds of millions of dollars in revenue for the Nielsen 

 4    Company.

 5               For the television industry generally, the 

 6    Nielsen television ratings provide a kind of currency 

 7    whereby business is transacted between advertisers and the 

 8    television networks.

 9               The television industry in the United States 

10    amounts to tens of billions of dollars and it literally 

11    could not function as a business without this kind of 

12    information produced by Nielsen.

13               Worldwide, of course, the impact of Nielsen data 

14    on the television industry is even larger.

15         Q.    And the statistical bedrock of all of that 

16    commerce is what, sir?

17         A.    Well, is, again, a series of sample surveys 

18    conducted by the Nielsen Company.

19         Q.    Can you give the court some idea of the dimension 

20    of commerce generally that depends on survey sampling with 

21    respect to the other part of the Nielsen business?

22         A.    The marketing research area is an even larger 

23    component of the Nielsen business, and without wanting to be 

24    very precise, overall worldwide generates somewhere between 

25    half a billion and a billion dollars in revenue for us.
                                                              478

 1               But more than that, the entire food distribution 

 2    system and distribution system for consumer packaged goods 

 3    in the United States and in other countries is driven by 

 4    Nielsen data.

 5               I once heard it said that during the height of 

 6    the military establishment in the United States, America 

 7    spent about as much on guns as on butter, meaning that 

 8    America spent as much on food as on the military, and that 

 9    entire food industry literally is driven by the Nielsen data 

10    and is made more efficient as a consequence.

11         Q.    Dr. Wolter, where were you educated?

12         A.    I took my undergraduate degree at St. Olaf 

13    College in Minnesota, my masters and Ph.D. are from Iowa 

14    State Universsity in Ames.

15         Q.    What was the subject of your doctoral 

16    dissertation?

17         A.    Statistical models of measurement error.

18         Q.    And your Ph.D. is in statistics?

19         A.    Yes, it is.

20         Q.    At the time that you were working on the subject 

21    of your Ph.D., was the field of sample surveying a new one?

22         A.    No, no, definitely not.

23         Q.    Can you trace the current history of the field as 

24    of the time you were, in fact, looking toward your Ph.D.?

25         A.    The modern field of survey sampling was codified 
                                                              479

 1    in the 1930s and 1940s.  It emerged as a formal discipline 

 2    in statistics at that point in time.  And, of course, there 

 3    has been very lively research and implementation of sample 

 4    surveys ever since, not only in this country, but in all 

 5    countries of the world.

 6               There were informal surveys conducted well before 

 7    the thirties, even into the last century and perhaps before.

 8               One can recall history lessons where in ancient 

 9    times or Biblical times there were censuses and surveys.  

10    But as a modern scientific discipline, codification occurred 

11    roughly from the 1930s onwards.

12         Q.    Do you consider the field of survey sampling to 

13    be a separate subfield within the field of statistics?

14         A.    Yes, it is.

15         Q.    You had mentioned when you were at the Bureau you 

16    did some teaching.  Where did you teach?

17         A.    I taught at the George Washington University, 

18    which is in Washington, D.C.

19         Q.    Describe generally for the court what you --  did 

20    I cut you off?  I'm sorry.

21               Describe generally to the court what you did at 

22    George Washington University when you taught there.

23         A.    I taught course in survey sampling, I directed a 

24    Ph.D. student, I served on dissertation committees of other 

25    Ph.D. students, I drafted a master's degree program in 
                                                              480

 1    survey sampling and carried out other departmental 

 2    activities.

 3         Q.    How many years did you teach at George 

 4    Washington?

 5         A.    It was 13 or 14 years.  

 6               THE COURT:  Full-time faculty? 

 7               THE WITNESS:  No, it was part-time.

 8               I taught one course per semester.

 9         Q.    How many courses per year?

10         A.    Typically, two courses per year, one per 

11    semester.

12         Q.    Over the period that you were at George 

13    Washington University, how many students did you teach 

14    survey sampling to?

15         A.    I would estimate between 180 and 200 students.

16         Q.    Does George Washington --

17         A.    These are, these are Ph.D. level students, I 

18    might add.  This was not --  these were not undergraduate 

19    courses.

20         Q.    Did the George Washington University have a 

21    subspecialty or a specialty in survey sampling?  Was that 

22    one of the disciplines that it specialized in in statistics?

23         A.    Yes, it did, in part because of the very large 

24    federal statistical establishment that was located in the 

25    area.  George Washington was a kind of training ground for 
                                                              481

 1    federal statisticians.

 2         Q.    In the field of survey sampling, do you consider 

 3    Professor Gene Ericksen to be an expert?

 4         A.    Yes, sir, I do.

 5         Q.    To your knowledge, does the University of 

 6    California at Berkeley have a known survey sampling 

 7    department?

 8         A.    I'm, I'm uncertain.

 9         Q.    Do you hold any memberships in statistics related 

10    professional organizations?

11         A.    Yes, I do.

12         Q.    Could you tell the court which ones?

13         A.    I am a member of the American Statistical 

14    Association, I am an elected member of the International 

15    Statistical Institute, I am a member of the Institute of 

16    Mathematical Statistics, I am an executive member of the 

17    American Marketing Association, I am a current scientific 

18    secretary of the International Association of Survey 

19    Statisticians.

20         Q.    What is the job of a scientific secretary of the 

21    International Association of Survey Statisticians?

22         A.    This is an elected post that by and large is 

23    responsible for the intellectual well-being of the 

24    membership.

25         Q.    Are you a Fellow of the American Statistical 
                                                              482

 1    Association?

 2         A.    Yes, I am.

 3         Q.    When were you elected to be a fellow?

 4         A.    I don't recall.  It was 80, 81, 82, somewhere in 

 5    that general time frame.

 6         Q.    I am going to ask you about the work you did at 

 7    the Census Bureau a little more in a moment.

 8               When you were there, did you receive any awards?

 9         A.    At the Census Bureau?  Certainly I received 

10    awards, yes.

11         Q.    Can you describe to the court the awards you 

12    received?

13         A.    I received the Census Bureau's highest award, 

14    which is called the Bronze Medal.

15               I received the --  

16               THE COURT:  The best they can give is bronze? 

17               THE WITNESS:  Unfortunately, yes.  At that point, 

18    their parent agency takes over and reserves all the better 

19    awards.  

20               But I am pleased to report that I also won one of 

21    their awards, their Silver Medal Award.

22               I received other certificates of recognition, 

23    cash awards and the like, and I was awarded one of the 

24    agency's annual EEO awards.

25         Q.    The parent award that you were just talking about 
                                                              483

 1    was an award conferred by the Department of Commerce, is 

 2    that right?

 3         A.    Yes, that's correct.

 4         Q.    Since you received your Ph.D., have you received 

 5    any awards from Iowa State?

 6         A.    Yes, yes.

 7         Q.    Tell the court which ones, please.

 8         A.    Well, at the time of my Ph.D., I received an 

 9    award as the most outstanding Ph.D. recipient that year, and 

10    then later in my career I received an award from Iowa State 

11    as their most outstanding young alumnus.

12         Q.    Have you written any books?

13         A.    Yes, I have.  I have written one book in survey 

14    sampling, specifically, a specialized topic within survey 

15    sampling known as variance estimation.

16         Q.    Do you know whether that work is used in the 

17    field of survey sampling?

18         A.    Oh, yes, indeed.  It's used around the world.

19         Q.    Have you written or published articles on 

20    subjects of survey sampling or more generally in statistics?

21         A.    Yes, I have.  I have written a large number of 

22    articles.

23         Q.    Would you look, please, at the binder that is in 

24    front of you, which I think you will find in tab 1 we have 

25    marked as PX 537 for identification a document that, for the 
                                                              484

 1    record, is the curriculum vitae of Dr. Wolter.

 2               Would you look at that and tell me if you can 

 3    identify that?

 4         A.    Yes.  This is a recent vitae.

 5         Q.    Of yours?

 6         A.    Yes, that's correct.  

 7               MR. SOLOMON:  We would offer Plaintiff's Exhibit 

 8    537.  

 9               MR. MILLET:  No objection, your Honor.  

10               THE COURT:  537 is admitted.  

11               (Plaintiff's Exhibit 537 marked for 

12    identification was received in evidence.)

13    BY MR. SOLOMON:

14         Q.    Dr. Wolter, returning to your career at the 

15    Census Bureau, describe to the court the positions you held 

16    beginning in 1974, just briefly.

17         A.    From 1974 I was designated a mathematical 

18    statistician in one of the Bureau's divisions known as the 

19    research center for measurement.

20               From roughly 1978 I was appointed to a 

21    supervisory post as assistant division chief in the Bureau's 

22    statistical research division.

23               Then somewhere around 1980 I assumed the highest 

24    technical post, staff post within the Bureau, that was known 

25    as the chief or senior mathematical statistician.
                                                              485

 1               And then by 1982, roughly, or 83, I have 

 2    forgotten which, I became chief of the Bureau's statistical 

 3    research division.

 4         Q.    Did you hold the position of chief of the 

 5    statistical research division from 1983 until you left the 

 6    Bureau in 1988?

 7         A.    Yes, I did.

 8         Q.    Describe, please, generally what the statistical 

 9    research division of the Bureau of the Census was during the 

10    time you were its chief?

11         A.    The stat research division was one of the major 

12    research divisions within the Census Bureau.  It was a group 

13    of roughly 100 scientists, most with masters and Ph.D. 

14    degrees in such fields as statistics, mathematics, computer 

15    science, demography, political science and so forth.

16               Our mission, quite simply, was to monitor the 

17    progress of the work in the rest of the Bureau, to evaluate 

18    the quality of work, to work with other census executives in 

19    identifying problems and opportunities in census programs, 

20    to develop new methodologies and transfer them into 

21    production so as to reduce cost or improve quality or 

22    timeliness of Census Bureau data products.

23               We served as a kind of conduit, also, between the 

24    outside world and the Census Bureau and brought developments 

25    outside of the Census Bureau to the attention of Census 
                                                              486

 1    Bureau executives and conversely brought internal problems 

 2    and opportunities to the attention of outside researchers 

 3    and other constituencies.

 4         Q.    During the time that you were at the Bureau, what 

 5    was the reputation of the statistical research division?

 6         A.    I think it was a very high one.

 7               Earlier you asked about the history of survey 

 8    sampling and I spoke about the codification of the field 

 9    beginning in the 1930s and to a very substantial extent the 

10    statistical research division was the place or one of the 

11    preeminent places in the world where this codification 

12    emerged and took place.

13               The division has come to be regarded around the 

14    world as one of the preeminent contributors to this field of 

15    science.

16         Q.    Describe for the court the role that you played, 

17    generally, as the chief of the statistical research division 

18    at the Bureau?

19         A.    That, that is a post of administration with 

20    personnel responsibilities, budget responsibilities, space 

21    responsibilities, union responsibilities; it's a post of 

22    professional leadership, it's a post of policy development, 

23    it's a post of --  it's a major point of interface between 

24    the Bureau and other organizations outside the Bureau.

25         Q.    To what particular projects, if any, did you in 
                                                              487

 1    particular devote yourself while you were the chief of the 

 2    statistical research division?

 3         A.    I devoted my activities to all of the above.  

 4    Certainly one of the technical or program areas that I 

 5    devoted considerable attention to was the problem of 

 6    undercounts and miscounts in decennial censuses and, indeed, 

 7    of other censuses, too.

 8         Q.    Did you, while you were at the Bureau, devote 

 9    yourself to the study of the potential amelioration of 

10    undercounts and miscounts in decennial censuses?

11         A.    Yes.  That was one of our very important 

12    activities during the 1980s.  

13               MR. SOLOMON:  Your Honor, we would proffer Dr. 

14    Wolter as an expert for any questions.  

15               MR. MILLET:  I'm sorry, an expert for any 

16    questions?

17               MR. SOLOMON:  Yes.  

18               MR. MILLET:  Certainly we have no objection to 

19    Dr. Wolter's expertise in statistic, but I don't think I 

20    could quite go as far as Mr. Solomon.  

21               MR. SOLOMON:  The issues that are relevant to 

22    this lawsuit.  

23               MR. MILLET:  We have no objection to that, as 

24    long as that is so.  

25               THE COURT:  Dr. Wolter is qualified and may 
                                                              488

 1    express his opinions.  

 2    BY MR. SOLOMON:

 3         Q.    Dr. Wolter, when you were at the Bureau, did the 

 4    Bureau conduct economic censuses?

 5         A.    It conducts economic censuses each five years.

 6         Q.    Describe very briefly to the court what economic 

 7    censuses are?

 8         A.    The economic censuses cover the economic census 

 9    activities of firms in the United States, firms as opposed 

10    to households.

11               There is a census of retail trade, there is a 

12    census of wholesale trade, there is a census of selected 

13    service industries, there is a census of manufacturers, of 

14    governments, of agricultural, and so forth.  These censuses 

15    attempt to enumerate all firms engaged in those economic 

16    endeavors.

17         Q.    What role, if any, did you play in connection 

18    with the evaluation of any of the economic censuses taken at 

19    the Bureau?

20         A.    From the late 1980s I was assigned responsibility 

21    for a crosscutting task force charged with the 

22    responsibility for evaluating the quality of the economic 

23    censuses.

24               In those years we were completing an evaluation 

25    program of the 1978 round of economic censuses.  
                                                              489

 1               When I say 1978, incidentally, I mean that the 

 2    census actually takes place in 1979 regarding economic 

 3    activity during the calendar year 1978.

 4               Then subsequent to that we began the planning of 

 5    the evaluation of the 1982 economic censuses.  Subsequently, 

 6    we approved those plans, executed the plans, published the 

 7    results and moved on towards the 1987 economic censuses.

 8         Q.    What use, generally, is made of the economic 

 9    census figures published by the Bureau?

10         A.    Massive uses.  First of all --  

11               THE COURT:  Shape up.

12         A.    They are used by the federal government for the 

13    national accounts, they are used throughout the Census 

14    Bureau for sampling purposes for subsequent monthly and 

15    annual surveys, they are used by private industry for sample 

16    surveys, they are used by my company for weekly surveys. 

17               They are a very important component of the 

18    American economy.

19         Q.    Is the gross national product one of the economic 

20    pieces of data that rely on the economic censuses published 

21    by the Bureau?  

22               MR. MILLET:  Objection; relevance.  

23               THE COURT:  Overruled.

24               You may answer.

25         A.    The national accounts are produced by our sister 
                                                              490

 1    agency, the Bureau of Economic Analyses based on inputs from 

 2    the Census Bureau, the economic census being a very key 

 3    preeminent input.

 4         Q.    And the gross national product, how is that 

 5    derived from that data?

 6         A.    That's a very elaborate process.

 7         Q.    All I want to know is if you know whether the 

 8    gross national product is one of the data outputs that rely 

 9    on the Census Bureau inputs taken from the economic 

10    censuses?

11         A.    Yes, it relies on input from the economic 

12    censuses.

13         Q.    Were evaluations carried out with respect to the 

14    1982 economic census?

15         A.    Yes.

16         Q.    Were those evaluations based, in part, on 

17    sampling techniques?

18         A.    They were based extensively on sampling 

19    techniques.

20         Q.    As a result of the evaluations that were carried 

21    out by the Bureau, did the Bureau communicate to I think you 

22    said the department, the Bureau of Economic Affairs, 

23    potential improvements that could be made to the census 

24    counts based on the evaluations that were based on sampling?

25         A.    We published evaluations of the quality of the 
                                                              491

 1    1982 economic censuses.  Those evaluations were made 

 2    available to the Bureau of Economic Analysis.

 3         Q.    Of what use did the Bureau make of those 

 4    evaluations?

 5         A.    The Bureau determined that it was possible to use 

 6    the results of those evaluations to improve or correct 

 7    deficiencies in the 1982 economic censuses, and so the 

 8    economic censuses were, in fact, corrected or adjusted for 

 9    those identified difficulties.

10         Q.    Since the time you left the Bureau in 1988, have 

11    you stayed generally abreast of the work the Bureau has done 

12    in connection with its undercount research and PES?

13         A.    I have stayed generally abreast, yes.

14         Q.    How have you done that?

15         A.    Through a variety of means.

16               First and foremost, I was a member of the special 

17    advisory panel that was created as a result of your Honor's 

18    stipulation and order in 1989.

19               Secondly, I have attended a range of meetings, 

20    annual meetings, local meetings of professional 

21    organizations, such as the American Statistical Association, 

22    whereby discussions of this matter took place.

23               And finally, I've kept generally in touch to the 
      extent that I could with former colleagues and staff members 
24    at the Census Bureau.
                 (Continued on the next page) 
25         Q.    Have you testified before on the subject of 
                                                              492

 1    census accuracy and correction of census figures?
                                                       

 2         A.    Yes.  I testified in a lawsuit in 1984 in New 

 3    York City concerning the completeness of the 1980 Decennial 

 4    Census.

 5         Q.    On that occasion, you were a witness called by 

 6    the Bureau of the Census, is that right?

 7         A.    That's correct.

 8         Q.    And have you testified under oath on deposition 

 9    in this matter?

10         A.    In the instant matter.

11         Q.    Yes.

12         A.    Yes, for five days.

13         Q.    In connection with those depositions or any of 

14    the work that you have done in preparation for your 

15    testimony here, have you received any remuneration?

16         A.    No, I have not.

17         Q.    Have you been promised any remuneration?

18         A.    No, I have not.

19         Q.    As a member of the Special Advisory Panel, can 

20    you estimate how much time you spent devoted to the subject 

21    of the post-enumeration survey in the correction of the 1990 

22    census count?

23         A.    The intensity of the work varies, of course -- it 

24    did vary, of course, from the time to time.  In the early 

25    months of the panel, the panel tended to meet on the order 
                                                              493

 1    of once a month, and I devoted perhaps on the order of two 

 2    or three days per month to my role as a panel member.

 3               Later, once all the census activities and 

 4    evaluation activities were under way, we tended to meet less 

 5    often, and I tended to devote no more than a day per month 

 6    to my activities on this panel.

 7               Then moving beyond 1990 and into the spring of 

 8    1991, my activities intensified considerably, and I devoted 

 9    several days per month to this responsibility.  There were 

10    times in June of that year when I devoted very considerable 

11    time to this responsibility.

12         Q.    As a member of the Special Advisory Panel, did 

13    you make a recommendation to have Secretary Mosbacher on the 

14    question of whether or not the 1990 Census counts should be 

15    statistically adjusted?

16         A.    Yes, I did.

17         Q.    What was your recommendation?

18         A.    I recommended that the corrected counts were more 

19    accurate and absolute in distributional terms than the 

20    original raw census counts.  And thus, on those grounds 

21    principally, I recommended that he certify the corrected 

22    counts as the official 1990 Census figures.

23         Q.    I would like you to turn to the next tab in your 

24    binder, tab 2, where we have marked for identification as PX 

25    538, for the record, a June 21st, 1991 letter from Dr. 
                                                              494

 1    Wolter to Secretary Mosbacher.  

 2               Have you seen that document before, sir?

 3         A.    Yes, I have.

 4         Q.    What is it?

 5         A.    It's the letter that I wrote to Secretary 

 6    Mosbacher conveying my recommendations.  

 7               MR. SOLOMON:  We would offer PX 538.  

 8               MR. MILLET:  Your Honor, I have no objection to 

 9    all but the last page.  I just showed this to Mr. Solomon 

10    just before the examination began.

11               As your Honor can see, Exhibit 530 is bates stamp 

12    510 and I believe it's 525.  And then there's a table at the 

13    end which appears to not have a bates stamp.  This does not 

14    bear the telefax legend that is on the bottom of the other 

15    pages.  I do not believe it is part of the administrative 

16    record.  

17               If Mr. Solomon can lay a foundation for the last 

18    page, we don't object.

19               We don't object to 510 to 525.  The objection is 

20    limited to the last page.  

21               MR. SOLOMON:  May I ask a question of this 

22    witness?

23               THE COURT:  Yes.  

24    BY MR. SOLOMON:

25         Q.    Dr. Wolter, on page 14 of your letter of 
                                                              495

 1    recommendation, in the paragraph beginning on Tuesday, June 

 2    18, on the fourth line, you have a reference there to "this 

 3    table."  And you say "See table 2 attached."

 4         A.    Yes, sir.

 5         Q.    Do you see that?

 6         A.    I see that passage.

 7         Q.    And I want you then to look at the last page of 

 8    this exhibit.

 9         A.    Yes.

10         Q.    PX 538, and tell me whether that is the table 

11    that was attached.

12         A.    Yes, it was.

13         Q.    Identify what that table is, sir.

14         A.    It's an analysis performed by the Census Bureau 

15    regarding the accuracy of the congressional proportion based 

16    on corrected and uncorrected census counts.

17         Q.    Was that attached to the letter that you sent to 

18    Secretary Mosbacher?

19         A.    My secretary inadvertently deleted this table 

20    from the original FAX.  I noted the error on the order of an 

21    hour after the error occurred.  I believe it was immediately 

22    faxed to the Secretary at that point; so, at that point, it 

23    was detached from the letter.  But then we mailed a hard 

24    copy of the letter to the Secretary; and, unless I'm 

25    remembering this incorrectly, the entire package, with the 
                                                              496

 1    table, was mailed at that point in time.  

 2               MR. SOLOMON:  We renew our offer.  

 3               MR. MILLET:  I appreciate the explanation.  I 

 4    withdraw the objection.

 5               THE COURT:  Exhibit -- what was the number?  

 6               MR. SOLOMON:  538.

 7               THE COURT:  -- 538 is admitted.

 8               (Exhibit PX 538 for identification was received 

 9    in evidence) 

10    BY MR. SOLOMON:

11         Q.    Dr. Wolter, did you also participate in the 

12    preparation of the joint report to the Secretary?

13         A.    Yes, I did.

14         Q.    Who were the coauthors on that report?

15         A.    Gene Ericksen, John Tukey and Leo Estrada, 

16    co-panel members.

17         Q.    Can you give the Court a brief sketch of the 

18    reasoning that underlay your recommendation to the 

19    Secretary.

20         A.    Well, generally, there is a fundamental problem 

21    of differential undercount in the decennial censuses.  This 

22    problem was first measured for the 1940 Decennial Censuses.  

23    It's been measured for each and every decennial census since 

24    then.

25               For the 1990 Census, however, for the very first 
                                                              497

 1    time the Census Bureau was well-positioned to do something 

 2    about that differential undercount.  For the very first 

 3    time, it has a very precise measurement of the differential 

 4    undercount.  And for the very first time, it has means of 

 5    adding the missed and miscounted people back into the raw 

 6    undercount -- excuse me -- back into the raw census so as to 

 7    correct that raw census for the missed and miscounted 

 8    people.

 9               It would be a great service, in my judgment, to 

10    customers of census data to have the more accurate data.  

11    That, essentially, was the basis for my recommendation.

12         Q.    What did you rely on in making that recommenda- 

13    tion to the Secretary?

14         A.    Well, I relied on many inputs.

15               First of all, as I just stated, there was a 

16    record of undercount measurements and methodology since the 

17    1940 census.  I am knowledgeable of all of that activity.

18               I have personally been involved in undercount and 

19    correction-like work at the Census Bureau since roughly the 

20    1978-79 time frame.

21               And I am -- I have worked considerably ever since 

22    that time frame on this matter.  So I brought to bear all of 

23    this history and knowledge that the Bureau generally and 

24    that I personal have developed over these many years.

25               Furthermore, there is a record of undercount 
                                                              498

 1    measurement, and in other countries, such as Canada, 

 2    Australia, the U.K., and other countries, and I have 

 3    generally enjoyed exchanges with colleagues in other 

 4    countries.  And so I am aware of their activities and 

 5    experiences in this matter.

 6               Finally, of course, I am very intimately involved 

 7    or knowledgeable about the 1980 Decennial Census.  I took 

 8    part in that undercount measurement process and brought that 

 9    experience to bear on 1990.

10               Then there was an extensive research and 

11    evaluation effort throughout the decade of the 1980s, 

12    planning for the census of 1990.  And that entire experience 

13    base was brought to bear on the decisions in 1990.  

14               And finally, of course, from my perspective as a 

15    member of the Special Advisory Panel in 1990, I had access 

16    to considerable information on the conduct of the original 

17    enumeration, on the conduct of the coverage measurement 

18    processes, and on their quality, and I brought all of that 

19    current information to bear on this decision as well.

20         Q.    At the time that you made the recommendation to 

21    the Secretary, based on the information and the background 

22    that you have just described, did you believe that 

23    reasonable or knowledgeable statisticians could disagree 

24    about whether the PES should be used to adjust the census 

25    counts?
                                                              499

 1         A.    No, there was a clear-cut matter.  It was 

 2    absolutely clear cut that considerable improvement in the 

 3    raw counts could be achieved by utilization of the 

 4    correction or of the corrected data.

 5         Q.    After the time that you made your recommendation, 

 6    -- after the time you made your recommendation, did you 

 7    learn that the Undercount Steering Committee at the Bureau 

 8    and the Director of the Bureau also made either findings or 

 9    recommendations concerning whether the PES should be used?

10         A.    Yes, I learned of their recommendations.

11         Q.    And what were their recommendations that you 

12    learned?

13         A.    They recommended to the Secretary the use of the 

14    corrected counts.

15         Q.    After that time, did you read the Secretary's 

16    decision?

17         A.    Yes, I did.

18         Q.    Do you recall about when you read the Secretary's 

19    decision?

20         A.    On the 15th of July or shortly thereafter of 

21    1991.

22         Q.    In a word, what was his decision?

23         A.    In a word, he decided that the raw counts were 

24    the preferred counts in his judgment.

25         Q.    Do you agree with the conclusion or his 
                                                              500

 1    reasoning?

 2         A.    No, of course not.

 3         Q.    Having read his report, have you changed your 

 4    mind with respect to your position?

 5         A.    No, I have not changed my mind.

 6         Q.    Okay.  I'd like you to look at the third tab of 

 7    your book.  That has been marked as PX 9.  And what we have 

 8    here, just for the convenience of the Court and witness, are 

 9    excerpts of the Secretary's decision that I may be referring 

10    to with Dr. Wolter.  

11               MR. SOLOMON:  If the Court wishes to have the 

12    complete copy, I'll find it.  

13         Q.   But there are some specific places that I'd like 

14    to call your attention to.

15               To begin, I would like you to return to page 266.  

16    That's 2-66.

17               And the discussion of guideline 6 which runs from 

18    266 through 270 I want to call your attention to a couple of 

19    statements made by the Secretary.  

20               At the bottom of 2-66 it says: 

21               "The Census Bureau produced highly technical 

22    research on a very tight production schedule, using tools 

23    that were on the cutting edge of statistical theory and 

24    survey methods."

25               Then over on 2-70, in the conclusion, in the 
                                                              501

 1    second paragraph, he says:  

 2               "To the Census Bureau's great credit, the 

 3    statistical tools used to calculate and evaluate the 

 4    adjusted counts are at the cutting edge of statistical 

 5    research.  But such cutting edge research is not tried and 

 6    true -- it requires more thorough scrutiny before it can be 

 7    used to affect the allocation of political representation 

 8    and Federal funding."

 9               Would you comment on those statements.  In 

10    particular, do you believe them to be fair and accurate?

11         A.    No.  That's -- they are neither fair nor 

12    accurate, and represent a distortion of what I know to be 

13    the facts.

14               If I can take the statements one by one, perhaps 

15    that might be an efficient way of proceeding.

16               First of all, I don't agree that the Bureau 

17    produced work on a production schedule.  I wouldn't say it 

18    was necessarily a very tight production schedule, it was 

19    certainly an appropriate production schedule, certainly of 

20    the kind and similar to the production schedule that the 

21    Census Bureau always operates under in the context of the 

22    decennial census.

23               The Secretary also remarks about tools that were 

24    at the cutting edge of statistical theory and survey 

25    methods, and I have to tell you that this is an incomplete 
                                                              502

 1    statement and tends to mislead as it's worded.  The fact of 

 2    the matter is -- 

 3               MR. MILLET:  I'm going to object to that, your 

 4    Honor.  Now he's characterizing the Secretary's intent and 

 5    not discussing the document.

 6               THE COURT:  Overruled.  

 7               You may continue.

 8         A.    (Continuing)  Well, the fact of the matter is 

 9    that the -- that the tools that are used in this process 

10    were, to my knowledge, first used as early as the 1940s, 

11    perhaps even earlier than that.  They were used then by Ed 

12    Demming and others.  Demming is the patriarch of the U.S. 

13    quality movement.  These tools were used generally for the 

14    first time in connection with the decennial census in 1950.  

15    They have been used in connection with U.S. Decennial 

16    Censuses periodically ever since.  They've been used by 

17    other countries ever since.

18               And so, in a certain sense, what was done in 1990 

19    is an outgrowth, is an evolutionary outcome of work that was 

20    begun in 1940.  And to say that somehow suddenly these 

21    methods are at the cutting edge I believe is inappropriate.

22               At the same time, enormous improvements and 

23    refinements, and so on, were carried out and tested during 

24    the 1980s.  But even there, the improvements and refinements 

25    were tested in real field tests repeatedly during the decade 
                                                              503

 1    of the 1990s.  And so by the time of the 1990 Census, the 

 2    tools were improved, they were an evolutionary outcome of 

 3    earlier work that was done as early as the 1940s, and in 

 4    that sense, I see nothing particularly cutting edge about 

 5    them.

 6         Q.    Dr. Wolter, let interrupt you for a moment.  

 7               You said that statistical tests had been carried 

 8    out in the '90s.  I assume you meant the '80s?

 9         A.    In the 1980s, that's correct.

10         Q.    Okay.  Thank you.

11         A.    The next passage I had from you was on page 2-70?

12         Q.    Yes.

13         A.    He says, "To the Census Bureau's great credit, 

14    the statistical tools" -- and I think the Census Bureau does 

15    deserve great credit, I might add.  That's one point on 

16    which I would agree with the secretary.

17               (Continuing) -- "...are at the cutting edge of 

18    statistical research."  I have just spoken to that issue.  

19    But --

20               THE COURT:  You prefer stone age tools.

21               THE WITNESS:  What's that?

22               THE COURT:  You prefer stone age tools, sharpened 

23    a bit.

24               THE WITNESS:  Only if they are better than 

25    cutting edge tools.  That isn't often the case.
                                                              504

 1         A.    In any event, such cutting edge tools or research 

 2    is not tried and true.  This is nuts.  

 3               The work that was done, as I said, was developed 

 4    as a part of an -- as a four- or five-decade long 

 5    improvement process.  It was a part of a process of 

 6    iterative refinement of testing, problem identification, 

 7    methodological problem-solving, further testing, further 

 8    refinement, and so on and so on.

 9               And even after the gains of the 1980s were 

10    achieved, all of the new improved methods were tested fully 

11    in a number of tests, real field tests throughout the 1980s.

12               So in my judgment, the work was tried and true.

13         Q.    Have you completed your answer?

14         A.    Yes.

15         Q.    Okay.  Tell the Court what role you personally 

16    played in connection with undercount research at the Bureau.  

17    Describe it generally, at first, if you would.

18         A.    From the time that I was appointed Chief of the 

19    Statistical Research Division, and even before then I 

20    assumed considerable responsibility for undercount 

21    measurement within the Census Bureau, and retained that 

22    position and responsibility throughout the decade of the 

23    1980s.

24         Q.    Did you have personal involvement during the 

25    period 1978 prior to 1983?
                                                              505

 1         A.    Yes.  I became involved in planning for the 1980 

 2    Census Coverage Measurement Program.  I had staff involved 

 3    in that, and I was involved and directed their work.

 4               Subsequently, I was asked to lead a cross-cutting 

 5    Census Bureau Task Force charged with evaluating the 1980 

 6    census and evaluating the quality of the 1980 census 

 7    undercount estimates.

 8         Q.    What was the problem that the Bureau was trying 

 9    to address in conducting the research and the tests that you 

10    just generally made reference to?

11         A.    Well, the basic problem here is differential 

12    undercount.  Differential undercount causes some groups in 

13    the country to be undercounted more than other groups in the 

14    country.

15               Census data is inaccurate as a result.  Customers 

16    of census data, as a result, are making business decisions 

17    in light of flawed census data.

18               It is incumbent on the Census Bureau to try to 

19    improve that process, if at all possible.  That's the basic 

20    problem we were trying to address.

21         Q.    Why does that problem arise?

22         A.    It arises as a result of the fundamental 

23    interaction between census-taking methodology and the 

24    American social fabric.

25               As early as 1940, as I said, it was learned that 
                                                              506

 1    there is a differential undercount between blacks, for 

 2    example, and non-blacks; between men and women; and between 

 3    various age groups.  That differential undercount remarkably 

 4    has remained constant or very nearly constant in every 

 5    decennial census since then.  In spite of the Census 

 6    Bureau's good efforts.  And they are very good and they are 

 7    very enormous efforts to try to eliminate the differential 

 8    undercount by enumeration techniques, that is, by way of the 

 9    original enumeration.  

10               And yet, in spite of all the Bureau's good 

11    efforts, and in spite of decades of work, in spite of 

12    billions of dollars over the many decades, the differential 

13    undercount has remained persistent, constant, almost 

14    intractable, at least intractable for the kinds of 

15    conventional enumeration methods the Bureau has utilized.

16         Q.    Would you give the Court a thumbnail sketch of 

17    the design of the PES as used in the 1990 Census, because I 

18    would like to use that as a backdrop to ask you to retell 

19    the history of the projects at the Bureau leading up to the 

20    PES of 1990.

21         A.    Okay.  PES, of course, stands for Post- 

22    Enumeration Survey.  And from that name, as you might 

23    imagine, this is a sample survey and it's a sample survey 

24    that follows the original enumeration.  It's am large sample 

25    survey of households and people.  For each of the 
                                                              507

 1    individuals in this sample survey, a determination is made 

 2    as to whether the person was counted in the original 

 3    enumeration or not counted in the original enumeration.

 4               That determination of and enumeration status is 

 5    made by way of what the Bureau calls matching.  It matches 

 6    the person based on name, address, and other characteristics 

 7    of the individual, to the census returns, and thereby 

 8    determines whether the person was enumerated or not in that 

 9    original enumeration.

10               Then the Bureau takes the original enumeration, 

11    together with this post-censal survey, and the information 

12    on which of those individuals were counted and which were 

13    not, and develops an estimate of the size of the total 

14    population of the United States; not only nationally, but by 

15    various demographic groups and for various geographic areas 

16    of the United States.

17               The difference between these PES-based total 

18    population estimates and the original enumeration based 

19    population estimates is commonly referred to as the 

20    decennial census undercount.

21         Q.    You mentioned that the process by which the PES 

22    arrives at the determination of whether someone was counted 

23    or not was done by matching.  Do you have personal knowledge 

24    of the matching rules that were used as part of the 1990 

25    census?
                                                              508

 1         A.    Yes, generally.

 2         Q.    Can you describe to the Court how the matching 

 3    went forward.

 4         A.    Well, the matching went forward in two or three 

 5    basic layers.

 6               First of all, there was developed during the 

 7    1980s what's called a computer expert system that implements 

 8    matching automatically.  This is matching based on the 

 9    names, the addresses, the race, the marital status, age, 

10    et cetera, of the individuals enumerated in the post-census 

11    survey.  

12               The automated matcher, if I can use that term, 

13    matches about 75 percent of the persons counted in the post- 

14    censal survey.

15               All other individuals are referred to human 

16    beings for matching, and there is a multi-stage human 

17    matching process that starts with a staff of well-trained 

18    clerks in census processing offices who compare the 

19    post-censal survey people versus the original enumeration 

20    returns to determine who was -- who is in the original 

21    enumeration and who is not.

22               Then the results of that matching are referred to 

23    further teams of better clerks, better in the sense that 

24    they exhibit greater skill, better in the sense that they 

25    are trained more, better in the sense that they are under 
                                                              509

 1    tighter control, administratively.

 2               And really to cut a long story short, there are 

 3    several additional layers of human matching whereby better 

 4    and better clerks and technical personnel are applied to 

 5    this matching operation.  There are further field follow-ups 

 6    to gather additional information, and then there is further 

 7    layering of additional human matching effort which 

 8    ultimately culminates in determining which of the PES people 

 9    are counted and which are not counted in the original 

10    enumeration.

11         Q.    Did the 1990 PES use imputation?

12         A.    Yes, it did.

13         Q.    Describe briefly for the Court what imputation 

14    is?

15         A.    In all sample surveys of households and people, 

16    there is inevitably some missing data.

17               There is inevitably some segment of the 

18    population that doesn't cooperate or which supplies faulty 

19    data.

20               This isn't true just in the United States.  It's 

21    true around the world.  And it isn't just true in PES-like 

22    surveys, but it's true in every sample survey known to me.  

23    Inevitably, there's always some -- some missing data.  And 

24    always, survey statisticians must decide how to handle this 

25    missing data.  There's really no issue.  It's not a question 
                                                              510

 1    of should we handle the missing data or shouldn't we; it 

 2    always must be handled in any sample survey.  It's often 

 3    handled by estimating what the missing data would have been 

 4    had it been observed.  Statisticians and others in the 

 5    survey profession often refer to this process as imputation.

 6               And the Census Bureau imputes missing data in the 

 7    original enumeration as well as in the PES as well as in 

 8    every other census and survey that the Census Bureau 

 9    conducts.

10         Q.    Do you have an opinion whether with respect to 

11    the 1990 PES there was a relatively large, or relatively 

12    small amount of imputation that was necessary by reason of 

13    missing data?

14         A.    There was an unprecedentedly small level of 

15    missing data in the 1990 Post-Enumeration Survey requiring 

16    the imputation process, as I have just described it.

17         Q.    Would you just briefly finish off the description 

18    of how the PES gets to the adjusted numbers, and now I 

19    understand we've had matching and we've had any imputation 

20    that might be necessary.

21         A.    Okay.  Well, this can be a long story or a short 

22    story.

23         Q.    Take the short story.

24         A.    Take the short story.  Okay.

25         Q.    Leaving nothing out, sacrificing no accuracy when 
                                                              511

 1    you are speaking, what he (indicating) used to tell me.

 2         A.    Okay.  In broad general terms, we at this point 

 3    in the explanation, we have the PES data as reported, and 

 4    then we have the imputed data in the case where reported 

 5    data was missing.  And in that sense our data is now 

 6    complete, it's either reported or it's imputed, but we can 

 7    now say that the data set is complete.

 8               We now have the original enumeration and we now 

 9    have the complete PES data, and the Census Bureau then 

10    initiates its estimation process.  The Bureau estimates two 

11    proportions from these various data.  First of all, it 

12    estimates the proportion of people enumerated in the 

13    original enumeration correctly, and, of course, the 

14    complimentary proportion is the proportion of the original 

15    enumeration that was, in fact, counted erroneously.

16               I don't know if you are familiar with the term 

17    "erroneous enumerations" or not, but some individuals 

18    regrettably are counted erroneously because they were born 

19    after census day but were counted nonetheless, or they died 

20    before census day and were counted nonetheless.  Or they 

21    were counted in the wrong geographic location, or census 

22    returns were duplicated or triplicated for an individual, 

23    and so on and so on.

24               So that's the first piece of the PES estimation 

25    process, is to estimate the proportion of the original 
                                                              512

 1    census enumeration that was counted correctly versus 

 2    erroneously.

 3               Then the Census Bureau also estimates a second 

 4    proportion.  This is the proportion of the U.S. population 

 5    that was counted in the original enumeration.

 6               Then the original enumeration itself, together 

 7    with these two proportions, as I have described them, allow 

 8    the Census Bureau to estimate the total size of the 

 9    population.  As I said, not only at the U.S. level, but also 

10    for various demographic segments of the United States, and 

11    also for various geographic areas, like states and cities 

12    within the U.S.  

13               Once these direct estimates of total population 

14    are produced, then the Census Bureau goes through another 

15    refinement in the estimation process, and removes what's 

16    called sampling error, or removes as much sampling error as 

17    it can from these direct estimates the size of the 

18    population.

19               At that point, the Bureau has its best estimates, 

20    the size of the population, at, shall we say, aggregate 

21    levels of geography, and for aggregate demographic groups 

22    within the U.S., and it then has the means of carrying the 

23    missed people down to lower levels of geography, and 

24    allocating those missed people to the various small areas -- 

25    blocks, counties, enumeration districts, and the like -- 
                                                              513

 1    within the country.

 2         Q.    Did the Bureau use something called 

 3    stratification in calculating the undercount estimates as 

 4    part of the 1990 PES?

 5         A.    Yes, they do.

 6         Q.    Just describe generally what the stratification 

 7    scheme was?

 8         A.    Are you referring to the post-stratification 

 9    scheme?

10         Q.    Yes.

11         A.    Well, that's a fancy word that statisticians use.  

12    All it means, that we are going to chop up the country into 

13    mutually exclusive and exhaustive pieces, and these pieces 

14    are called poststrata.  

15               In this particular case, there were 1392 pieces 

16    that the overall United States was divided into.  These 

17    poststrata or these pieces were in part defined by 

18    geography.  And in part, they were defined by demographic 

19    areas, such as age, sex, race, ethnicity, housing tenure, 

20    type of place.  Whether it's a central city or not.  

21               And so you wind up with a piece of the country 

22    called or referring to, for example, all black males in New 

23    York City that rent their housing unit and that are age, 

24    say, 20 to 29.  That is an example of one of the 1392 

25    pieces.  
                                                              514

 1               And together these pieces or poststrata 

 2    completely span the entire resident population of the United 

 3    States.

 4         Q.    You spend your life doing survey sampling or 

 5    inspecting survey sampling.  Is stratification common in --

 6         A.    Stratification of this kind is pervasive 

 7    throughout all the work at the Census Bureau.

 8         Q.    Why did the Bureau stratify the 1990 PES?

 9         A.    It does so to reduce sampling variability, which 

10    is one of the limitations on any sample survey, and it also 

11    does so to produce the very best possible corrected census 

12    product, a product that meets the needs of census customers.

13         Q.    You made reference in your description before to 

14    the fact that the Bureau after arriving at the estimate 

15    removed sampling variability.  Is that also called 

16    smoothing?

17         A.    Yes.  Smoothing, and a variety of other names are 

18    used in technical publications to refer to this process.  

19    But cutting through all that jargon, it's really just a 

20    matter of attempting to strip out this one or a portion of 

21    this one limitation on survey results called sampling error.

22         Q.    Is that a common technique in survey sampling?

23         A.    Is what a common technique?

24         Q.    Smoothing, or the reduction of sampling 

25    variability?
                                                              515

 1         A.    It's used in a number of surveys, yes, but it 

 2    isn't as pervasive as post-stratification itself is, but 

 3    nonetheless, it's been used time and again.  

 4               MR. SOLOMON:  I don't know if your Honor wants to 

 5    take an afternoon break.  I'm happy to move on at this time.

 6               THE COURT:  All right.  Let's take a 15-minute 

 7    break.

 8               (Recess)

 9               (Continued on next page)

10    

11    

12    

13    

14    

15    

16    

17    

18    

19    

20    

21    

22    

23    

24    

25               THE COURT:  All right, you may resume.  
                                                              516

 1               MR. SOLOMON:  Thank you, your Honor.  

 2    BY MR. SOLOMON:

 3         Q.    Dr. Wolter, will you please describe to the court 

 4    in overview how the Census Bureau approached the problem of 

 5    the differential undercount and the design of the PES?

 6         A.    In the 1980s or before?

 7         Q.    Yes, in the 1980s.

 8         A.    In the 1980s.

 9               Surely.

10               Starting in roughly in 1984, 85 time frame, there 
                                                                   

11    was, first of all, a period wherein I gained support from 

12    the very highest executive levels within the agency that 

13    this problem, the problem of differential undercount, would 

14    be one of the two or three preeminent objectives for the 

15    agency in the decade of the 1980s.  That agreement from the 

16    top was reached in roughly 1984-85 time frame.

17               Subsequent to that, a staff was developed, 

18    budgets were acquired and real work got underway.

19               There was a period of new methods development, 

20    then there was a period of testing, then there was a period 

21    of further refinement of methods based on the testing, and 

22    it went back and forth between testing and methodological 

23    development.

24               And eventually decision processes were put in 

25    place and finally the 1990 enumeration and PES itself took 
                                                              517

 1    place.

 2         Q.    As part of that effort, were special or 

 3    particular organizations within the Bureau formed to address 

 4    the question of a differential undercount and the design of 

 5    the 1990 PES?

 6         A.    Yes.  There were two principal organizations 

 7    formed within the Census Bureau to address this issue which 

 8    I have now said is one of the two or three preeminent 

 9    executive objectives for the decade.  

10               One of the organizational units came to be known 

11    as the undercount research staff.  This was a staff of 

12    professional statisticians, demographers and the like that 

13    reported to me and the statistical research division.

14               I proposed creation for this staff, I hired this 

15    staff, I trained this staff, I directed the staff and that 

16    went on until the time that I left the Census Bureau.

17               The second principal organization within the 

18    Census Bureau came to be known as the undercount steering 

19    committee, and this was a kind of board of directors, if you 

20    will, that oversaw policy development and the work of the 

21    undercount research staff.

22               The undercount steering committee was composed 

23    of, oh, roughly half a dozen top census executives and took 

24    part in every critical decision throughout the decade, from 

25    1984 until, I suppose, current points in time.
                                                              518

 1         Q.    Who headed the undercount research staff directly 

 2    under you?

 3         A.    A gentleman who I hired by the name of Howard 

 4    Hogan, Dr. Howard Hogan.

 5         Q.    Is Dr. Hogan still at the Bureau?

 6         A.    Yes, he is, to my knowledge.

 7         Q.    What's role, if any, did you play as a conduit or 

 8    liaison in between the work of the undercount research staff 

 9    and the undercount steering committee?

10         A.    Well, again, the undercount steering committee, 

11    as I said, was and is like a board of directors setting 

12    policy, reviewing developments and the like.  I have served 

13    on that board, on that committee.  

14               At the same time, staff work and coordination 

15    work, detailed work, planning work, implementation work is 

16    being accomplished by the undercount research staff which 

17    reported administratively to me.

18               So obviously I was in the middle.  I was the 

19    conduit.  I was reporting findings from the staff to the 

20    board, I was communicating policy from the board back to the 

21    staff, and there was a continual feedback loop.

22         Q.    Were there particular areas within the undercount 

23    research staff that got a particular attention or focus?

24         A.    Every aspect of this issue got exceptional focus 

25    throughout the decade.  Some of the areas that we focused on 
                                                              519

 1    especially in early years were activities like the matching 

 2    process that I mentioned before, such as limiting the amount 

 3    of missing data which I mentioned before, and such as the 

 4    process of removing sampling error, what you called 

 5    smoothing earlier, all of that, among many other areas, 

 6    received considerable attention by this staff and by various 

 7    other staffs that the undercount research staff coordinated.

 8         Q.    Were there other areas of the Census Bureau that 

 9    were involved in the research and design efforts besides the 

10    undercount research staff and the undercount steering 

11    committee?

12         A.    Yes.  There were many other offices within the 

13    Bureau that participated in this overall enterprise.

14               Let me start with the population division.  The 

15    population division is a group of demographers and other 

16    social scientists that study the population of the United 

17    States.  They have been instrumental in producing undercount 

18    estimates for every census since 1940.  Their method of 

19    undercount measurement is known as the method of demographic 

20    analysis.  They participated with us.

21               The field division within the Census Bureau 

22    participated with us.  That is the group charged with 

23    responsibility for actually carrying out the interviewing 

24    process on all surveys and censuses within the agency, PES 

25    and decennial census included.
                                                              520

 1               We participated with an organizational group 

 2    known as the decennial operational division that has 

 3    responsibility for all of the computer work and all of the 

 4    office processing associated with a large enterprise like 

 5    the decennial census and PES.

 6               There was a crosscutting funding group within the 

 7    Census Bureau known as the decennial planning division, and 

 8    we certainly worked extensively with that group.

 9               We worked extensively with colleagues in the 

10    statistical methods division, which is a division charged 

11    with operational responsibility for surveys of households 

12    and people within the Census Bureau.  And, of course, 

13    various other executives, too.  This was an activity that 

14    drew in many people from almost every corner of the Census 

15    Bureau.

16         Q.    What level of resources did the Bureau as a whole 

17    commit to the project of ameliorating the differential 

18    undercounting by sample survey?

19         A.    An extraordinary high level of effort and 

20    commitment was put into this enterprise.

21               As I said, census executives, early in the 

22    decade, decided that this was to be the primary focus or one 

23    of the primary focuses for the 1990 census, reduce the 

24    differential undercount, and it put people, it put staff and 

25    it put commitment behind this.
                                                              521

 1               I would estimate on the order of perhaps a 

 2    hundred person years of headquarter's time was devoted to 

 3    this activity throughout the period from 1985 through 1990, 

 4    and at least $10 million effort was put behind this.  And 

 5    that's not counting the time of all the interviewers and 

 6    matching personnel and other people that took part in this 

 7    overall process.

 8               This was a large endeavor that occupied a major 

 9    initiative by this agency and by many other groups that 

10    participated with this agency.

11         Q.    Did outside professional organizations or 

12    associations play any role in connection with the study of 

13    the adjustment issue during the 1980s?

14         A.    Outside organizations played a key role by design 

15    and that was only right and proper.

16               I'm thinking of such organizations as the 

17    Standing Advisory Committees from the American Statistical 

18    Association, the American Economics Association, the 

19    American Marketing Association, the Pop Association, I'm 

20    thinking of a special advisory panel of, a blue ribbon panel 

21    of experts appointed by the National Academy of Sciences to 

22    review this matter and to offer advice and counsel to the 

23    Census Bureau; I'm thinking of colleagues at Stat Canada in 

24    Ottawa, Canada and other organizations, too, that 

25    participated with us either in a formal or informal role 
                                                              522

 1    throughout the entire decade.

 2         Q.    I think you mentioned that it was important.

 3               Why was it important to have outside advisors or 

 4    experts reviewing this work or commenting on it or 

 5    communicating with the Bureau?

 6         A.    As a matter of science, as a matter of corporate 

 7    culture, as a matter of what's right and proper.

 8               The Census Bureau makes every effort on all 

 9    critical matters to expose its methods, its procedures, its 

10    theories to open public comment.  

11               We wanted the very best advice, we wanted the 

12    very best counsel, we wanted to use the very best possible 

13    procedures and methodologies to solve this very critical 

14    important national problem of decennial census differential 

15    undercount.

16               We felt it was important that we establish some 

17    kind of a consensus of support among professional groups and 

18    other communities that used decennial census data.  We felt 

19    that was important as we move into the 1990 census period 

20    and as we make or made the very important decisions that 

21    were, in fact, made regarding that census.

22         Q.    Dr. Wolter, John Rolf will be testifying here on 

23    behalf of the plaintiffs in this case.

24               Was he on the panel of the National Academy of 

25    Sciences, the blue ribbon panel that you made reference to?
                                                              523

 1         A.    Yes, he was.

 2         Q.    And Dr. Feinberg, Steve Feinberg will also be 

 3    appearing on behalf of the plaintiffs in this case.

 4               Do you know whether he had a position at the 

 5    National Academy of Sciences?

 6         A.    I, I, I have forgotten the exact years of his 

 7    tenure, but for many years Steve was the chair of a parent 

 8    committee within the National Academy of Sciences.

 9               In other words, there was a parent committee 

10    which is commonly known as the committee on national 

11    statistics.  For a number of years early in the decade Steve 

12    chaired that parent committee.  It was that parent committee 

13    that formed and organized the blue ribbon panel on decennial 

14    censuses that I referred to earlier.

15         Q.    How did the Bureau communicate with these outside 

16    agencies the thoughts and design concepts and the testing 

17    procedures that it was coming up with?

18         A.    It communicated by a variety of means.

19               There were frequent meetings with these various 

20    organizations where verbal or oral presentations took place, 

21    where testimony was given, where there was oral exchange 

22    between the committee and the Census Bureau.  Often written 

23    documents were presented to these various committees by the 

24    Census Bureau.

25               Of course, there was widespread one-on-one 
                                                              524

 1    interaction, not only with the individual members of these 

 2    committees, but with the professional community at large.

 3               And then finally, professional organizations like 

 4    the American Statistical Association, like the Population 

 5    Association of America, I presume like the American Bar 

 6    Association, hold periodic annual national meetings and 

 7    regional meetings and the Census Bureau staff presented 

 8    dozens of papers and other kinds of presentations at these 

 9    various national and regional meetings, and that drew in a 

10    variety of participants from these organizations from around 

11    the country and, indeed, other countries.

12         Q.    As a result of the Bureau's internal analysis and 

13    its communications with the outside agencies, the 

14    presentations of papers that you just made reference to and 

15    the one-on-one communications, do you believe that over the 

16    course of the decade of the 1980s, consensus was formed 

17    concerning whether or not the PES could be used to 

18    ameliorate the problem of differential undercounting in the 

19    decennial census?  

20               MR. MILLET:  I object to that.  It calls for 

21    speculation.  It is outside the area of hisexpertise.  

22               MR. SOLOMON:  The the witness testified that the 

23    purpose for communicating with the outside agencies and 

24    experts was to form a consensus.  

25               THE COURT:  I will permit the question.  The 
                                                              525

 1    objection is overruled.

 2         A.    One of my most important responsibilities at the 

 3    Census Bureau was to interact with these outside groups, was 

 4    to stimulate, if you will, the exchange between the Bureau 

 5    and the outside community on this very issue.  That was my 

 6    responsibility, among others.

 7               I stimulated that exchange.  I believe that 

 8    consensus was reached on this matter, that PES based 

 9    procedures could ameliorate the differential undercount.

10               I'm not saying that there was unanimity of 

11    opinion.  There were a few views that I would regard as 

12    extreme, quite frankly.  

13               Most of the advisors, most of the panels, the 

14    committees, the groups that stayed with this issue for a 

15    period of time, became familiar with it, took time to learn 

16    the issues and offer serious advice and counsel were part of 

17    the consensus that I just described.  That was quite clear.

18         Q.    Why if you are talking about adjusting the 1990 

19    census did work begin I think you mentioned in the 1982, 83, 

20    84 time frame?

21         A.    There are probably two dimensions to that issue.

22               First of all, in government there are enormous 

23    lead times.  The government just doesn't move suddenly, it 

24    moves.  

25               THE COURT:  The record won't reflect the meaning, 
                                                              526

 1    but I have it.

 2         A.    It takes time to acquire a budget, it takes time 

 3    to receive approval to acquire computer equipment, it takes 

 4    time to acquire temporary space and staff needed to execute 

 5    a decennial census, it takes years to put in place all of 

 6    the apparatus needed to handle an operation this large.

 7               That's one dimension.

 8               But an even more important dimension of this is 

 9    the fact that the Census Bureau, again, by its scientific 

10    orientation, by its public service orientation, but its 

11    corporate culture seeks to constantly improve, refine, test, 

12    further improve, refine, further test almost everything the 

13    Bureau does.  That's been a pervasive characteristic of the 

14    agency since the 1940s.  And it was that corporate culture 

15    that led us to want to begin this work as early as possible 

16    in the 1980s, starting with the experience in 1980 itself, 

17    learning from that experience, learning what we could about 

18    how to improve that experience, putting in place 

19    improvements, improving, testing, improving and further 

20    testing.

21               All of that takes years and that's why we started 

22    early in the 1980s.

23         Q.    In the 1984 time period, after the undercount 

24    steering committee got together and formed, did it issue a 

25    general plan to the relevant scientific community letting 
                                                              527

 1    them know of this plan of attack?

 2         A.    Yes.  The undercount steering committee, the 

 3    board, if you will, as one of its very earliest initiatives 

 4    made extensive efforts to set policy for the entire 

 5    undercount measurement enterprise throughout the 1980s.  

 6    This policy was drafted and it was presented not only to 

 7    professional communities, but to other political and other 

 8    interested parties who have an interest or a stake in the 

 9    decennial census.

10         Q.    Let me ask you, please, to turn to the next tab 

11    in your notebook, tab 4, which we have marked for 

12    identification PX 541.  

13               MR. SOLOMON:  For the record, it is an October 

14    1984 document entitled, "Requisite planning and research 

15    relating to a decision on census adjustment for 1990," by 

16    KIRK M. Wolter, Friday, October 12, 1984.

17         Q.    Can you identify PX 541?

18         A.    Yes.  It is an article I wrote setting forth the 

19    policy that was developed by the undercount steering 

20    committee.  

21               MR. SOLOMON:  We would offer Plaintiff's Exhibit 

22    541, your Honor.  

23               MR. MILLET:  No objection.  

24               THE COURT:  541 is received.  

25               (Plaintiff's Exhibit 541 marked for 
                                                              528

 1    identification was received in evidence.)

 2    BY MR. SOLOMON:

 3         Q.    This document outlines what is said on page 2 of 

 4    the document to be five major planning goals.

 5               Can you go through those for the court and 

 6    identify them and tell the court whether those planning 

 7    goals were achieved?

 8         A.    Yes, sir.  

 9               MR. MILLET:  Can you identify the page again?  

10               MR. SOLOMON:  I am on page 2 of the document that 

11    bears the production number 16211.  

12               MR. MILLET:  Thank you.

13         A.    There is a paragraph headed number 1, "Complete 

14    the 1980 coverage evaluation studies."

15               This was the first of the goals.

16               There were two major coverage evaluation studies 

17    of the 1980 decennial census.  One was a study based on the 

18    method of demographic analysis.  I mentioned that in passing 

19    earlier.  And secondly there was a study based on a PES 

20    conducted in 1980.  That was also known in 1980 as the 

21    post-enumeration program.

22               Both of those undercount measurement activities 

23    were, in fact, completed, and by the 1985-1986 time frame 

24    the Bureau published a formal report summarizing the 

25    information and findings about the coverage of the 1980 
                                                              529

 1    census.  That goal was, therefore, satisfied.

 2               I'm now on page 3, top of the page, goal 2, which 

 3    reads, "Determine criteria for deciding when census adjusted 

 4    estimates are more accurate than basic census estimates and 

 5    whether adjustment would have any practical effect." 

 6               That goal was achieved by the, by the late 86-87 

 7    time frame.  It was at that point in time that the Bureau 

 8    issued a first draft of its criteria, that is, the criteria, 

 9    the scientific criteria and other criteria that it would 

10    utilize in deciding whether to correct or not to correct the 

11    1990 decennial census undercount.

12               Subsequently, that draft was pushed to a back 

13    burner, quite frankly, in lieu of or as a result of the 

14    Commerce Department's decisions in October of 1980 not to 

15    implement a PES and correction process in 1990.

16         Q.    That was in October of 1987?

17         A.    Yes, sir.

18               Later, of course, when your Honor reinstated the 

19    PES in the stipulation and order of 1989, the Commerce 

20    Department was under obligation to resurrect these criteria 

21    which, indeed, it did, and which, indeed, it published as 

22    its guidelines for census correction, and I would gather or 

23    guess that that date was somewhere in the 89-90 time frame.  

24               MR. MILLET:  Objection and move the witness' 

25    characterization of the Commerce Department's obligations 
                                                              530

 1    under the stipulation be striken.

 2               THE COURT:  The document speaks for itself, so 

 3    avoid characterizing the obligations contained in the 

 4    document.  Okay? 

 5               THE WITNESS:  Sorry.

 6         Q.    Please look at the third of the goals on that 

 7    page.  It says, "Develop direct measurement technique to be 

 8    used for measuring undercount for 1990." 

 9               Was that goal achieved by the Bureau?

10         A.    Yes.  The Bureau considered an array of 

11    undercount measurement methods, tested each one of them 

12    thoroughly, determined which were good, which weren't so 

13    good, narrowed the field, tested the better ones further and 

14    ultimately narrowed down to one principal method that would 

15    be used as the preeminent measure survey based measurement 

16    technique for undercount in 1990.  That was the so-called 

17    PES or post-enumeration survey method.  That was goal 3.

18         Q.    Goal 4 is on page 6.

19         A.    Okay.  Goal 4 is, and I'm quoting again, "Decide 

20    on adjustment methodology for 1990 if decision is made to 

21    adjust." 

22               And this issue has a decade long history.

23               Early in the decade, and, again, I'm thinking of 

24    the 1984 time frame, I assigned this issue to one of the 

25    staffs within the statistical research division, and that 
                                                              531

 1    staff remained in place throughout the decade and its 

 2    principal responsibility was this very goal; it was to, it 

 3    was to develop methods for correction or adjustment if the 

 4    decision was made to adjust.  Its goal was to test those 

 5    methods, its goal was to evaluate those methods under a 

 6    variety of criteria and to recommend to census executives 

 7    the very best possible methodology for use in 1990.  That 

 8    goal was certainly achieved.

 9         Q.    The last goal.

10         A.    The last goal on page 9, this is number 5, and, 

11    again, I quote, "Decision on adjustment." 

12               The decision process that we envisioned was that 

13    the decision about whether to actually correct or not to 

14    correct the decennial census was to be data driven.  In 

15    other words, this decision wasn't to be based on assertion, 

16    this wasn't to be based on speculation, this wasn't to be 

17    based on what somebody thought, this was a decision that was 

18    to be data driven based on real experiences with the 1980 --  

19    excuse me --  1990 enumeration and 1990 coverage measurement 

20    process, and the decision was to be made by the Census 

21    Bureau's director in late 1990 or early 1991 in light of the 

22    data and in light of the decision criteria that I discussed 

23    earlier under goal number 2.

24         Q.    I notice, actually, that you have in item 5, 

25    decision 5, that decision would be made by the director of 
                                                              532

 1    the Census Bureau.

 2               In 1984 was it your assumption that this data 

 3    driven decision, in your words, would be made by the 

 4    director of the Bureau of the Census and not the Secretary 

 5    of Commerce?  

 6               MR. MILLET:  Objection, relevance.  Plaintiffs 

 7    have stipulated this decision was to be made by the 

 8    Secretary in the stipulation and order.  

 9               MR. SOLOMON:  Your Honor, the defendants, in the 

10    stipulation and order, said what they were going to do and 

11    the plaintiffs said they are going to do at least that much, 

12    and in the last paragraph of your order it says we are 

13    reserving all of our rights.

14               Here I simply want the witness to tell me --  Mr. 

15    Millet and I can argue about that interesting question some 

16    other time.  Here I would like to know what the witness' 

17    assumption was, because it obviously bears on whether the 

18    Census Bureau was proper in making the recommendations that 

19    it made and whether the Secretary of Commerce acted 

20    improperly in overruling them.  

21               THE COURT:  For those limited two purposes, I 

22    will overrule the objection and admit the testimony.

23         A.    It was certainly my view, the view of the 

24    undercount steering committee that the decision would be 

25    made by the census director.  This decision was made by the 
                                                              533

 1    census director in 1980.  

 2               In all census history that I'm aware of decisions 

 3    about the conduct of decennial censuses was made at the 

 4    Census Bureau by the Census Bureau staff and it was my 

 5    assumption, it was our assumption at that point in time that 

 6    once again the census director would be making this decision 

 7    among many others.

 8         Q.    In light of the fact that you believed the 

 9    decision was going to be a data driven one, were you aware 

10    of anyone at the Department of Commerce level who had the 

11    requisite knowledge to make a data driven decision?  

12               MR. MILLET:  I will object.  That calls for 

13    speculation.  

14               THE COURT:  If you know you may answer.

15         A.    There was, to my knowledge, there was no one at 

16    the Commerce Department with, with, with adequate knowledge 

17    of the theories, methods and operations of survey sampling 

18    and of evaluation and measurement studies.

19               That isn't to say there weren't individuals who 

20    were knowledgeable about other aspects of statistical 

21    methodology, but not this particular methodology, no.

22         Q.    Did the undercount steering committee set for 

23    itself a schedule by which it would attack each of the five 

24    planning items that we just made reference to and achieve 

25    them?
                                                              534

 1         A.    Yes.  A schedule of broad milestones was 

 2    established covering all of the work to be done on this 

 3    matter from the time of this document through the 1990 

 4    period itself.

 5         Q.    Did the Bureau call to the attention of the 

 6    outside experts and the relevant scientific community the 

 7    schedule so that anyone who wanted to participate in this 

 8    decision would?

 9         A.    Absolutely.  The schedule, the milestones were 

10    widely brought to the attention of all interested 

11    communities.

12         Q.    Would you please turn to the next tab in the 

13    book, tab 5, where we have marked for identification PX 545 

14    of October 1986, a document entitled, "Introduction to the 

15    decennial census undercount and correction." 

16               Can you identify that document?

17         A.    Yes.  This is another document that I wrote and 

18    presented to the Census Bureau standing advisory committees.  

19    That was in October 1986 time frame.

20         Q.    In particular, looking at page 5, is this one of 

21    the places where the Bureau identified through the relevant 

22    statistical community what the schedule was of the key 

23    milestones that it was following?

24         A.    Well, yes.  In section 3.3 on pages 5 and 6 there 

25    is a discussion of decision points.  So the answer is yes, 
                                                              535

 1    this is one of the documents.

 2         Q.    What were the major decision points, for the 

 3    court's benefit?

 4         A.    Well, aside from narrow decisions regarding 

 5    detail and aside from decisions of implimentation of field 

 6    tests and the like, there were two major decision points 

 7    concerning this matter:

 8               One was to be a decision in the spring of 1987, 

 9    the other a decision to be made in late 1990 itself or early 

10    1991.

11               First the 87 decision point.  That was to be a 

12    key decision as to whether the Bureau felt confident that 

13    the PES correction process was likely to be successful if 

14    implemented in 1990.

15               Further, this was also a decision as to whether 

16    to seek funding for a PES in 1990.

17               These two aspects of that decision, of course, go 

18    hand and hand.  If you don't think you are going to be 

19    successful, then you don't ask for the money.

20               So the Bureau was to make, in that time frame, a 

21    decision about feasibility of the undercount measurement 

22    program and the correction program and along with that a 

23    decision as to whether to seek funding to support the 

24    undercount measurement program.

25         Q.    Let me turn your attention to the first paragraph 
                                                              536

 1    of this paper, in particular a sentence that reads, "Just as 

 2    we could not conduct the 1990 enumeration without widespread 

 3    support for our plans and goals, neither can we contemplate 

 4    adjustment without establishing a consensus of support for 

 5    our techniques and standards." 

 6               Is this one of the places where you advised the 

 7    outside scientific world that what the Bureau was seeking to 

 8    obtain was a consensus of the relevant scientific opinion?

 9         A.    Yes, sir.  But, I mean, moreover, all of the 

10    presentations that took place in these meetings themselves 

11    formed the basis for establishing a consensus of support or 

12    indicated our general intent to seek such support.  

13               MR. SOLOMON:  Plaintiffs offer Exhibit 545.  

14               MR. MILLET:  No objection, your Honor.  

15               THE COURT:  Exhibit 545 is admitted.

16               (Plaintiff's Exhibit 545 marked for 

17    identification was received in evidence.) 

18    BY MR. SOLOMON:

19         Q.    Describe for the court how the Bureau received 

20    feedback from the outside agencies, outside expert 

21    professional panels that it was communicating with.

22         A.    There was a variety of feedback.  There was 

23    certainly one-on-one feedback from individual members of the 

24    committees that I have described.  

25               There was one-on-one feedback from other 
                                                              537

 1    scientists around the country and in other countries.

 2               There was feedback from the committees as a 

 3    committee.

 4               There was feedback in the form of oral and 

 5    written recommendations from the American Statistical 

 6    Association, from the POP, that is the population committee, 

 7    from the National Academy of Sciences panel, and so forth.

 8               Of course, there were exchanges of views and 

 9    feedback from the professional community at large by way of 

10    the national and regional meetings that I described earlier 

11    and by way of our exchange with our sister agency in Ottawa, 

12    Stat Canada.

13         Q.    You make mention in your last answer to 

14    particular recommendation, written recommendations by the 

15    outside agencies.

16               Would you turn, please, to the next tab in your 

17    notebook, tab 6, where we have marked for identification PX 

18    550, which is entitled, "Report of the ASA technical panel 

19    on the census undercount." 

20               Can you identify that document?

21         A.    Yes, sir.  This document is known to me.

22               This is, this is a final report produced by yet 

23    another panel that I haven't described yet.

24               There are myriad panels and committees, there was 

25    really a massive, really massive oversight and examination 
                                                              538

 1    of this issue and this is yet another one of the panels.

 2               This panel existed in the roughly 82-83-84 time 

 3    frame, I don't recall exactly when.  It was constituted to 

 4    review what was done in 1980.  It was constituted to review 

 5    the 1980 census, the 1980 undercount measurement program, to 

 6    comment on the quality of a potential correction at that 

 7    point in time, and they issued these final recommendations 

 8    to us, again, in the 82-83-84 time frame, whenever.

 9         Q.    Did the Bureau read and consider if not adopt the 

10    recommendations of the ASA technical panel?  

11               MR. MILLET:  That is a compound question, your 

12    Honor.

13         Q.    Did the Bureau read and review the 

14    recommendations of the ASA technical panel?

15         A.    Well, many of us read it and reviewed it and 

16    certainly relevant executives did, yes.  

17               MR. SOLOMON:  We would offer PX 550.  

18               MR. MILLET:  Your Honor, I'm sure this is 

19    probably an oversight on Mr. Solomon's part.  My PX 550 is 

20    two documents, it's not merely one.  

21               MR. SOLOMON:  Let me see it.  

22               MR. MILLET:  You may.

23               (Pause) 

24               MR. MILLET:  My objection is that both parts of 

25    these have not been authenticated, first of all, and 
                                                              539

 1    secondly, at least to the part that has been authenticated 

 2    and I would object to that first half that is hearsay.  

 3               THE COURT:  The first half is the part you just 

 4    discussed with the witness, right?  

 5               MR. SOLOMON:  I thought, in fact, I was 

 6    discussing the entire document, but I will lay the 

 7    foundation for the second half.  

 8               THE COURT:  Would you, please?  

 9               MR. SOLOMON:  Sure.  

10    BY MR. SOLOMON:

11         Q.    Was it the practice from time to time at the 

12    Bureau to respond in writing to the recommendations made by 

13    the outside expert agencies?

14         A.    Yes.  We often responded in writing.  There may 

15    have been cases where we didn't, but certainly as a matter 

16    of course we always responded in writing formally to the 

17    standing advisory committees.

18         Q.    Look, please, at this document further in.  I 

19    believe it is published in the same place in the journal of 

20    the American Statistical Association.

21               I cannot tell what page it is on, but it begins 

22    with a comment from the Census Bureau by Barbara A. Bailar.

23               Do you see that?

24         A.    Yes.

25         Q.    Can you identify that?
                                                              540

 1         A.    Sure.

 2               Barbara Bailar way my supervisor at the Census 

 3    Bureau.  She wrote this article.

 4               In this article she comments on the final report 

 5    of the ASA technical panel, that is, the report marked 550 

 6    that we just looked at.  

 7               THE COURT:  To whom did she write this? 

 8               THE WITNESS:  Well, I, I suppose to the technical 

 9    panel itself and to all interested parties that may read the 

10    technical panel's report and may wish to know what the 

11    Bureau's views are regarding the technical panel's report.  

12               THE COURT:  Mr. Millet, you don't object, or do 

13    you, to the first half?  

14               MR. MILLET:  I don't object to the authentication 

15    of the first half, your Honor.  I don't think we still have 

16    sufficient authentication for the second half, and I do 

17    maintain my hearsay objection to both.  

18               THE COURT:  The hearsay problem stops me a bit, 

19    Mr. Solomon.  Can you help me there?  

20               MR. SOLOMON:  Sure.

21               Of course, with respect to the authentication, it 

22    is a periodicaL and it is self-authenticating.  

23               MR. MILLET:  I don't think it has been 

24    established that it is a --  

25               THE COURT:  Let's leave that on the back burner.  
                                                              541

 1    Talk about hearsay.  

 2               MR. SOLOMON:  Your Honor, this document is being 

 3    offered to show the growing consensus that occurred within 

 4    the relevant statistical community during the 1980s and it 

 5    would be sufficient if, as I've already established through 

 6    this witness, that the Bureau, in fact, heard and read and 

 7    took account of, whether or not it agreed with, the 

 8    particular recommendations that were made.  The witness has 

 9    already agreed to that.

10               Second, this is, in fact, part of a plan that 

11    leads up to the 1990 census as my next question will, in 

12    fact, demonstrate.  

13               MR. MILLET:  I hear Mr. Solomon saying he is 

14    offering it for the truth of the matter contained therein 

15    and, therefore, I maintain my objection.  

16               THE COURT:  I hear him saying it's a nonhearsay 

17    purpose, state of mind, a relevant effect on the state of 

18    mind on anybody that would read it.  

19               MR. SOLOMON:  That's correct, your Honor.  

20               THE COURT:  All right.  You said it better.  

21               THE COURT:  For that purpose, I shall admit it, 

22    both documents, under the one rubric Plaintiff's Exhibit 

23    550.

24               The authentication problem, I think, is minor.

25               I take this is some kind of --  something is 
                                                              542

 1    blotted out on the bottom, but it looks to be some sort of 

 2    periodical, the American-something.  

 3               MR. SOLOMON:  It is.  I may have misspoke.  Mine 

 4    is legible enough and we will get a more legible copy for 

 5    the court.  It stays the American Statistician, November 

 6    1984.  

 7               THE COURT:  Is that a learned publication of some 

 8    sort? 

 9               THE WITNESS:  It's one of the principal 

10    publications of the American Statistical Association.  

11               THE COURT:  I'm satisfied.  I will admit it.

12               (Plaintiff's Exhibit 550 marked for 

13    identification was received in evidence.) 

14    BY MR. SOLOMON:

15         Q.    Dr. Wolter, look, please, at recommendation 

16    number 12 of the ASA technical panel, which says, the second 

17    to the last page of the first of the parts, "That the Bureau 

18    of the Census sponsor an outside technical advisory group on 

19    undercount estimation and related problems."

20         A.    Yes, sir, I see that.

21         Q.    Did the Bureau follow that recommendation?

22         A.    Yes, it did.

23         Q.    What was the outside technical advisory group 

24    that was impaneled?

25         A.    I described earlier a committee of the National 
                                                              543

 1    Academy of Sciences.

 2               We sought out the National Academy, we asked for 

 3    their help.  Under their ausppices, a panel was constituted 

 4    and reviewed all census methodology, including especially 

 5    the issue of undercounts and miscount.

 6         Q.    As compared to the time that was spent by other 

 7    outside advisory groups, can you describe for the court what 

 8    kind of time the NAS panel put in the project of studying 

 9    the census undercount?  

10               MR. MILLET:  No foundation for that.  

11               THE COURT:  If you know, you may answer.  If you 

12    don't, don't guess.

13         A.    I can't speak to any one individual, but I can 

14    speak to time frame.

15               The panel existed over a several year period.  I 

16    don't remember exactly when their work got underway, but it 

17    was probably in late 84, early 85, and their work continued 

18    throughout the next several years.

19               I recall meeting with them just before I left the 

20    Census Bureau in May of 1988.  So it was a, it was a 

21    prolonged period of time where, I don't know for sure, but 

22    my impression was that most panel members spent a 

23    considerable amount of time immersed in this issue, learning 

24    the details of this issue, asking questions about this 

25    issue, commenting on this issue and so forth.
                                                              544

 1         Q.    Please turn to the next tab in the book we have 

 2    marked as PX 2, a document entitled, "The bicentennial 

 3    census, new directions for methodology in 1990." 

 4               You have here just an excerpt of a very long book 

 5    I supplied you with and the court obviously has copies of 

 6    the entire book.

 7               Can you identify that?

 8         A.    Yes, I can.  It's probably the most important or 

 9    one of the most important work products of the National 

10    Academy panel.  It was a book and a kind of interim report 

11    that they issued regarding 1990 decennial census 

12    methodology.  I believe this was published late in 85 or 

13    early 86, somewhere in there.

14         Q.    Does the book contain recommendations to the 

15    Bureau?

16         A.    Yes, it does.

17         Q.    Do you know whether the recommendations of the 

18    panel of the NAS, the National Academy of Science panel 

19    reflected in the book, reflect the input made to the panel 

20    by Census Bureau?

21         A.    Certainly.  

22               MR. MILLET:  I think that question assumes a fact 

23    not established.  

24               MR. SOLOMON:  The witness testified at length 

25    about discussions the Bureau had with the NAS panel.  
                                                              545

 1               THE COURT:  I will permit it.  The objection is 

 2    overruled.

 3         A.    Let me try to put it this way:

 4               We met repeatedly with the panel, we presented 

 5    documents to the panel, written documents, we presented 

 6    data, information to the panel, and the panel digested, 

 7    deliberated, worked with the data and worked with all of the 

 8    information that we gave to them.

 9               Of course, the panel brought their own 

10    experiences and skills to bear on these critical national 

11    issues as well and ultimately, based on their own 

12    experiences and skills, together with the various 

13    information that we presented, they arrived at this, at this 

14    report and recommendations.

15         Q.    Did the Bureau take into account the 

16    recommendations made by the NAS panel in determining how to 

17    proceed through the 1980s with respect to the design of the 

18    PES?

19         A.    Oh, very definitely yes.  This became some kind 

20    of guiding force for us.  There were a number of very useful 

21    recommendations in this document.

22               This isn't to say that every last recommendation 

23    was adopted.  They were all subjected to tests of 

24    feasibility and so forth and so on, but in philosophy, in 

25    spirit if not to the letter, we followed many of the 
                                                              546

 1    recommendations.  

 2               MR. SOLOMON:  Your Honor, the plaintiffs would 

 3    offer PX 2.  

 4               MR. MILLET:  Assuming this is for state of mind 

 5    as well, I won't object.  

 6               THE COURT:  All right.  Plaintiff's Exhibit 2 is 

 7    admitted on the same rationale, that is, state of mind.  

 8               MR. SOLOMON:  That is both the Bureau's state of 

 9    mind and the Commerce Department's state of mind, is that 

10    correct?  

11               MR. MILLET:  I don't think there is a Commerce 

12    Department state of mind.  

13               MR. SOLOMON:  It is a perfectly obvious the 

14    Commerce Department state of mind was in the administrative 

15    record and we are going to argue to your Honor that all of 

16    this relevant history should have been in the administrative 

17    record and wasn't.  

18               MR. MILLET:  Then, your Honor, I do object.  Are 

19    we are showing the absent of state of mind by showing the 

20    document exists?  

21               There is no foundation that someone looked at it 

22    or not looked at it.  I think we are getting pretty far 

23    afield.  

24               MR. SOLOMON:  We were promised the so-called 

25    administrative record that had all the relevant documents to 
                                                              547

 1    it.  The witness testified that this is one of the key 

 2    documents and it is nowhere to be found.  

 3               MR. MILLET:  The administrative record contains 

 4    the documents that were relevant to the Secretary's 

 5    decision.  This particular document may have been relevant 

 6    to Dr. Wolter's work as he has testified.  That has nothing 

 7    to do with state of mind of the Commerce Department or any 

 8    decision makers within the Commerce Department.  So 

 9    producing documents that show the absence of state of mind, 

10    particularly when Dr. Wolter left the government in 1988, I 

11    think we are getting pretty far afield here.  

12               THE COURT:  Well, it may be a bit unusual, but 

13    just as you can admit a business record to show that there 

14    is not an entry there when you expect it to be there, it 

15    seems to me that when you are dealing with nonhearsay 

16    admitted to show a state of mind, you can admit a document 

17    to show that this document was out there, it should have 

18    been taken into account and it apparently was not. 

19               Therefore, on the state of mind of the Commerce 

20    Department, arguably evincing a decision to stay in 

21    invincible irnorance, on that rationale I will admit it, the 

22    goal of invincible ignorance.  

23               (Plaintiff's Exhibig 2 marked for identification 

24    was received in evidence.)

25    BY MR. SOLOMON:
                                                              548

 1         Q.    Will you look at page 2 of the book, the second 

 2    to the last bullet on the page says, "As in every census 

 3    since 1940, statistical sampling methods will be used to 

 4    obtain responses to many census items," and the paragraph 

 5    goes on from there, I need not read all of it.

 6               At the time that you received this from the NAS 

 7    panel, did you agree that statistical sampling methods had, 

 8    in fact, been used since 1940 in the decennial censuses?

 9         A.    Well, sure, they had been used.

10         Q.    And was statistical sampling methods also used in 

11    the 1990 decennial census, putting aside the PES for the 

12    moment?

13         A.    Yes, surely.

14         Q.    On the top of page 3 in the first bullet it says, 

15    "As has been true since 1960, large computers will be used 

16    to process the census returns in a relatively short span of 

17    time." 

18               Do you see that, sir?

19         A.    Yes, sir.

20         Q.    Are you familiar with the use that the Census 

21    Bureau has made of computers?

22         A.    Yes, sir.

23         Q.    Are computers foreign to the census-taking 

24    process?

25         A.    No.  In fact, in the early 1950s, Univak 1, 
                                                              549

 1    serial number 1 was delivered to the Census Bureau.  This is 

 2    the world's first electronic computer used for commercial 

 3    purposes.  

 4               This is one of the most important facts in the 

 5    history of the Census Bureau.  The Census Bureau was the 

 6    first organization in the world to use computers.

 7               So this statement was true insofar as it goes, 

 8    but actually the Univak 1, serial number 1 processed parts 

 9    of the 1950 census, not merely 1960.

10         Q.    Do you recall seeing in the Secretary's decision 

11    a reference to computers?

12         A.    Yes, I recall a passage early in his report.

13         Q.    What was the thrust of the Secretary's statement?

14         A.    The thrust of his statement was something to the 

15    effect that a computerized process should not overturn a 

16    tried and true process that has been used for 200 years.

17         Q.    In light of what you knew at the Bureau, did you 

18    agree with that statement?

19         A.    No, of course not.  It misses the mark on two 

20    grounds:

21               First of all, computers have been used since 

22    1950, as I just said.  

23               In fact, one of the other principal goals of the 

24    Census Bureau during the 1980s was to automate the 1990 

25    census to the extent practicable.  Automation means 
                                                              550

 1    computing equipment.  So he was just off the mark altogether 

 2    regarding computing.

 3               And secondly, he seemed to, at least in my view, 

 4    I don't know if this was in his state of mind or not, but in 

 5    my view he seemed to disparage computers as somehow being 

 6    less than the, less than the other techniques, presumably 

 7    manual techniques, used or that he believed were used in 

 8    prior censuses, and that just isn't so, either.

 9         Q.    Last stop on this book, will you please turn to 

10    page 17, the last several --

11         A.    Tab 17?

12         Q.    Page 17 of that book.

13               Toward the bottom of the page there is a sentence 

14    that begins, "However, the panel," speaking of the NAS 

15    panel, "also believes that the ultimate goal of the census 

16    should be the accuracy of the census figures.  The evidence 

17    is overwhelming no counting process, however diligent, will, 

18    in fact, enumerate everyone.  Hence, the panel recommends 

19    that the Census Bureau carry out the vigorous program of 

20    research on coverage evaluation and adjustments methods 

21    that, if successful, would permit adjustment of census 

22    figures as part of the methodology of the 1990 census." 

23               Do you see that, sir?

24         A.    Yes, sir.

25         Q.    Do you agree with the statement that the ultimate 
                                                              551

 1    goal of the census should be the accuracy of the census 

 2    figures?

 3         A.    In the final analysis at the end of the day, it 

 4    almost has to be.

 5               There are thousands of uses of census data, but 

 6    at the end of the day, it seems to me the Constitutional 

 7    requirement for the census is the one that must prevail.

 8         Q.    Did you agree that the evidence was overwhelming 

 9    that no counting device, however diligent, will, in fact, 

10    enumerate everyone?

11         A.    Absolutely.  The differential undercount as I 

12    described earlier is just somehow fundamental, fundamental 

13    to the way in which conventional census methods interact 

14    with the U.S. population.  

15               If you can eliminate that differential, you got 

16    to use different technique.  Trying to do the wrong 

17    technique better isn't going to make it, and 40, 50 years of 

18    experience have taught us that.

19         Q.    Do any histories exist that describe the sampling 

20    used by the Bureau in the 1940s and fifties and sixties?

21         A.    Yes, surely.  There are dozens of papers in the 

22    statistical literature authored by Census Bureau staffers, 

23    some of them pioneering papers in this area.  There are some 

24    overviews, some books written, some interviews published by 

25    eminent statisticians from the Census Bureau from that 
                                                              552

 1    period documenting the work.

 2               Some of the names that come to my mind include 

 3    Ross Eckler, Morris Hansen, Bill Hurwitz and others.  These 

 4    are historical names in this field.  These are people who 

 5    worked at the Census Bureau, these are people who preceded 

 6    me in my post at the Census Bureau.

 7         Q.    In particular, who is Dr. Eckler?

 8         A.    Eckler was a statistician at the Census Bureau, 

 9    later he rose to assume the post of director.

10         Q.    I would like you to turn to the next tab in the 

11    book --  

12               THE COURT:  Maybe we can break at this point and 

13    resume tomorrow.  

14               MR. SOLOMON:  Whatever is convenient for your 

15    Honor.  

16               THE COURT:  My attention span has just about run 

17    its limits.  

18               MR. SOLOMON:  Sure.  

19               THE COURT:  All right.  We will recess for the 

20    day.

21               We will start again tomorrow at 9:30.

22               (Adjournment taken to Thursday, May 14, 1992, at 

23    9:30 o'clock a.m.) 

24    

25
