                                                              771

 1    UNITED STATES DISTRICT COURT
      EASTERN DISTRICT OF NEW YORK
 2    ------------------------------x
      THE CITY OF NEW YORK, et al.,
 3    
                         Plaintiffs,          88 Civ. 3474 (JMcL)
 4    
                 v.                           
 5    
      UNITED STATES DEPARTMENT OF COMMERCE,
 6    et al., 
      
 7                       Defendants.
      ---------------------------------x
 8    CITY OF ATLANTA, et al.,
                     
 9                       Plaintiffs,
      
10                v.                          92 Civ. 1566 (JMcL)
                                              
11    MOSBACHER, et al.,
      
12                       Defendants.
      ---------------------------------x
13    FLORIDA HOUSE OF REPRESENTATIVES, 
      et al.
14    
                         Plaintiffs,
15                v.                          92 Civ. 2037 (JMcL)
      
16    BARBARA H. FRANKLIN, Secretary of
      Commerce,
17    
                         Defendant.
18    --------------------------------x
      
19    
                                              May 15, 1992
20                                            9:00 a.m.
      
21    
      
22    Before:
      
23               HON. JOSEPH M. McLAUGHLIN,
      
24                                            Circuit Judge
      
25
                                                              772

 1    JOHN E. ROLPH,                  resumed.           

 2               THE COURT:  Good morning, ladies and gentlemen.  

 3               MR. SHERMAN:  I recall Dr. Rolph to the stand.  

 4    DIRECT EXAMINATION 

 5    BY MR. SHERMAN:

 6         Q.    Dr. Rolph, I would like to turn your attention to 

 7    PX 2 in the binder.

 8         A.    Okay.

 9         Q.    Would you again identify for the record what that 

10    document is?

11         A.    PX 2 is entitled "The bicentennial census, new 
                                                                

12    directions for methodology in 1990."

13               It's the final report of the first National 

14    Academy panel we discussed yesterday.

15         Q.    Was that a collaborative effort of the National 

16    Academy panel?

17         A.    Yes.  It was a report that was produced by the 

18    whole committee.  Various members of the panel wrote 

19    different sections and then the two staff people pulled it 

20    altogether and did a fair bit of additional writing.

21         Q.    And did you take part in the drafting of that 

22    document?

23         A.    Yes, I did.

24         Q.    Does that represent the National Academy panel's 

25    final report?
                                                              773

 1         A.    Yes, it is the final report.

 2         Q.    Dr. Rolph, does that report discuss smoothing 

 3    techniques at all?

 4         A.    Yes.  It has a discussion of smoothing 

 5    techniques.  Specifically, it has an appendix which I 

 6    believe has the title hierarchy goal base models and 

 7    discussed some of the technical aspects of smoothing in that 

 8    appendix, as well as a general discussion of smoothing.

 9         Q.    So was the National Academy panel discussing 

10    smoothing techniques in 1985?

11         A.    Yes.  There were several members of the panel 

12    that were particularly interested in that area and there 

13    were active discussions of the subject.

14         Q.    Dr. Rolph, I would like to draw your attention to 

15    the bottom of page 10 and the top of page 11 in PX 2.

16         A.    Okay.  I have it opened.

17         Q.    I wonder if you could --  well, the penultimate 

18    sentence is rather long, but I wonder if you can tell the 

19    court what the panel said about the census in the --  

20               THE COURT:  What?  

21               MR. SHERMAN:  I'm sorry.

22         Q.    --  said about the census in the paragraph that 

23    carries over from page 10 to 11.

24         A.    The paragraph basically makes two points.

25               The first is the degree of emphasis on counting 
                                                              774

 1    versus estimation and it makes the point that I made 

 2    yesterday, which is that with current census methodology, 

 3    not every record corresponds to a person actually named in a 

 4    questionnaire, and secondly --  so strictly speaking, the 

 5    census provides an estimate of the population as opposed to 

 6    an actual count of every person in the population.

 7         Q.    Thank you.

 8               Dr. Rolph, do you remember testifying about the 

 9    second National Academy panel yesterday?

10         A.    Yes, I do.

11         Q.    And would you tell us how the second second 

12    National Academy panel came to an end?

13         A.    Yes.  We had our --  reconstruct for a second.

14         Q.    Perhaps I can help you.

15               Can you tell us first when the second National 

16    Academy panel came to an end?

17         A.    Yes.  We had our last meeting in late 1988.

18         Q.    And had it been the intention of the second 

19    National Academy panel to continue its work with the Census 

20    Bureau through the 1990 census?

21         A.    Yes.  As I described yesterday, when the panel 

22    was set up it was envisaged that we would give advice to the 

23    Census Bureau through the test of adjustment related 

24    operations in 1986, the dress rehearsals that occurred in 

25    1988 and finally up to and including the 1990 census.
                                                              775

 1         Q.    And up until the Commerce Department's 1987 

 2    decision not to adjust the census, was it the expectation of 

 3    the second National Academy panel that it would be involved 

 4    in planning for and implementing an adjustment if the Bureau 

 5    determined that such an adjustment was warranted?

 6         A.    Yes.  Indeed, we had active discussions amongst 

 7    ourselves and with Bureau staff on even the logistics of how 

 8    we might accomplish exactly that.

 9         Q.    Dr. Rolph, did the second National Academy panel 

10    write a letter explaining why it had ceased meeting?

11         A.    Yes, we did.  The chair of our panel wrote a 

12    letter to the chair of the parent committee of that panel, 

13    namely, the chair of the committee on national statistics.

14         Q.    I would like to draw your attention to PX 7, 

15    please.

16               Do you have it?

17         A.    Yes, I have it.

18         Q.    Is this the letter that you are referring to?

19         A.    Yes, it is.

20         Q.    Did the National Academy panel discuss the 

21    substance of this letter with the chairman?

22         A.    Yes, this letter was the outgrowth of the last 

23    panel meeting which occurred in December of 1988, the 

24    previous month from when this letter is dated, and we had a 

25    quite extensive discussion amongst ourselves and with some 
                                                              776

 1    of the Bureau staff that were at a portion of that meeting, 

 2    and the result of those discussions was the panel 

 3    instructing the chair to write a letter that contained the 

 4    information that is in this particular letter.

 5         Q.    And you were part of this discussion?

 6         A.    Yes, I was.

 7         Q.    And you said the panel authorized this letter?

 8         A.    Yes.  Indeed, the panel directed the chair to 

 9    write the letter.  

10               MR. SHERMAN:  At this time I would offer PX 7.  

11    Bar bar no objection.  

12               THE COURT:  Number 7 is admitted.

13               (Plaintiff's Exhibit 7 marked for identification 

14    was received in evidence.) 

15    BY MR. SHERMAN:

16         Q.    Dr. Rolph, you mentioned the final panel meeting 

17    in December of 1988.

18               Were staff members of the Census Bureau at that 

19    meeting?

20         A.    Yes.  There were portions of the meeting that 

21    were executive session, as was our custom, and there were 

22    also portions of the meetings that were open and had various 

23    Census Bureau staff attending.

24         Q.    And did you learn why the Bureau staff were 

25    prohibited from further full communications with the 
                                                              777

 1    National Academy panel come from the discussions at that 

 2    meeting?

 3         A.    Yes, I would say that's the main basis for my 

 4    information, correct.

 5         Q.    And what were you told at that time about that 

 6    subject?

 7         A.    We were told at the meeting that because a 

 8    lawsuit had been filed the previous month, I believe in 

 9    November, that I think the expression was the Commerce 

10    Department and quote the lawyers unquote had told the Bureau 

11    to cease communicating with us in the full and open way in 

12    which they had before.

13         Q.    Thank you.

14               Dr. Rolph, are you familiar with what are called 

15    the P studies?

16         A.    Yes, I am.

17         Q.    Can you tell us what those are?

18         A.    Basically, they are a set of, if you will, 

19    quality control checks on the post-enumeration survey.

20         Q.    When did you first see the Bureau's plans for the 

21    P studies?

22         A.    I may have seen some very preliminary versions of 

23    them as part of the tale end of our panel meeting, but the 

24    first time I can really focus on the relationship between 

25    the P studies as they actually were carried out was when I 
                                                              778

 1    saw what were called the technical operation plans which 

 2    were given to me by the special advisor committee as part of 

 3    my consulting assignment for them.

 4         Q.    Would you describe that consulting assignment to 

 5    the court, please?

 6         A.    Yes.

 7               I was approached by the special advisory panel 

 8    and asked to give them my evaluation of the technical 

 9    operation plans, which was this summary of the P studies 

10    that I just mentioned.

11               I was also asked to comment on several other 

12    aspects of the post-enumeration survey, the census and the 

13    process of producing the dual system estimates.

14         Q.    Was smoothing one of the areas you were asked to 

15    comment upon?

16         A.    Yes.  I believe there is a section in the report 

17    I prepared explicitly on smoothing.

18         Q.    What was the date of your report to the special 

19    advisory panel?

20         A.    I believe my final report was dated I think it's 

21    August 15, 1990.

22         Q.    Are there any other aspects of the report that 

23    you remember that you haven't yet discussed?

24         A.    Yes.  In addition to commenting on specific P 

25    studies as the summaries were presented in these technical 
                                                              779

 1    operation plans, I also included a discussion of loss 

 2    functions which was in response to one or more of the 

 3    questions that was put to me as part of this assignment.  

 4    That was the other one that comes to mind.  There may be 

 5    others that I am not recalling at this time.

 6         Q.    Loss functions are one of the things you 

 7    described to the court yesterday?

 8         A.    Yes.  They are basically the yardsticks that are 

 9    used to measure the accuracy, if you will, of the corrected 

10    counts and of the uncorrected counts.

11         Q.    Dr. Rolph, what was the recommendation in the 

12    report to the special advisory panel with respect to loss 

13    function also?

14         A.    I believe I recommended, my recommendation were 

15    consistent and along similar lines as the recommendation of 

16    the National Academy panel, namely, I recommended that, 

17    first of all, that several loss functions be used; secondly, 

18    I recommended that the loss functions that be used include 

19    measure of the distributive accuracy, that is, the shares of 

20    the population by various --  the shares of the population 

21    by various groups in the country; thirdly, I recommended 

22    that the loss functions which were looking in a particular 

23    geographic area be done on a per population basis, I think 

24    as I said yesterday, one man one vote or I should say one 

25    person one vote; and finally a consequence of that was to 
                                                              780

 1    specifically --  a specific corollary of that last 

 2    recommendation was specifically to recommend against not 

 3    simply counting up political jurisdictions.

 4               I don't recall as I sit here whether I had that 

 5    exact language in my report.

 6         Q.    And was the Secretary of Commerce's use of loss 

 7    function analysis at odds with your recommendation?

 8         A.    Yes, it was.

 9         Q.    What was the date of your report, Dr. Rolph?

10         A.    August 15, 1990, I believe.

11         Q.    Dr. Rolph, when you were on the National Academy 

12    panel, did that body have a recommendation about 

13    quantitative thresholds to be used in making an adjustment 

14    decision?

15         A.    Yes, we did discuss the subject.

16         Q.    What was the recommendation of the National 

17    Academy panel?

18         A.    I believe this was a subject that came up during 

19    the second panel, and we were specifically discussed the 

20    question of whether to use qualitative thresholds as opposed 

21    to quantitative or numeric thresholds, and the panel 

22    recommended that the Census Bureau used qualitative 

23    thresholds so that they could basically anticipate or did 

24    not need to anticipate in detail every possible situation 

25    that might occur in the 1990 census.
                                                              781

 1         Q.    And did you make a recommendation about the use 

 2    of quantitative thresholds in your report to the special 

 3    advisory panel?

 4         A.    Yes, I did.

 5               In my report to the special advisory panel I 

 6    recommended that the guidelines and associated technical 

 7    operation plans include specific numeric thresholds, that 

 8    is, quantitative thresholds, in order to be used as a 

 9    decision criteria for whether or not to report the corrected 

10    counts as opposed to the uncorrected counts.

11         Q.    Is that at odds with the position you took as a 

12    member of the National Academy panel?

13         A.    Well, it's a different recommendation.  I see the 

14    two as being really quite consistent.

15               The situation when I was on the panel was one in 

16    which the Census Bureau was the body that was moving forward 

17    and carrying out planning for an adjustment and was planning 

18    to implement the adjustment and finally to make an 

19    assessment as to whether the adjustment produced more 

20    accurate counts.

21               At the time I wrote my report for the special 

22    advisory panel, the situated changed.  At that time, by that 

23    time, I should say, the UnderSecretary of Commerce had 

24    announced that there would not be an adjustment of the 1990 

25    census, there had then been a stipulation and order entered 
                                                              782

 1    in this case, there had been a set of preliminary guidelines 

 2    and then final guidelines that had been issued which 

 3    essentially to me indicated a, shall we say, a propensity 

 4    against adjustment, and finally the decision was to be made 

 5    by --  not by the Census Bureau, who would be the most 

 6    informed people in the world, if you would, about the 

 7    subject, but, rather, by a political appointee, namely, the 

 8    Secretary of Commerce, and it was my view at the time I 

 9    wrote the report that given a nontechnical person was making 

10    the decision which, unfortunately, it seemed to me that that 

11    person should have a specific criteria as possible about 

12    these various technical issues, so I, therefore, recommended 

13    specifically numeric criteria.

14         Q.    Is that still your view?

15         A.    That is still my view, yes.  

16               MR. SHERMAN:  At this time, your Honor, I would 

17    like to offer Plaintiff's Exhibit 64.  

18               MR. BARON:  We have no objection.  

19               THE COURT:  Exhibit 64 is admitted.

20               (Plaintiff's Exhibit 64 marked for identification 

21    was received in evidence.) 

22    BY MR. SHERMAN:

23         Q.    Dr. Rolph, since you wrote your report, have you 

24    had occasion to review the results of the P studies?

25         A.    Yes, I have.
                                                              783

 1         Q.    Do you have an opinion about the results of the P 

 2    studies?

 3         A.    Yes.  I believe taken as a body the P studies 

 4    were a successful quality control check on the 

 5    post-enumeration survey and I felt that they were carried 

 6    out in a competent and professional way and the results 

 7    supported the decision of the undercount steering committee 

 8    and the director of the Bureau of the Census to recommend a 

 9    correction --  recommend reporting the corrected counts as 

10    opposed to the uncorrected counts.

11         Q.    Do you have an opinion about what the results of 

12    the P studies indicate with regard to the quality of the 

13    PES?

14         A.    Yes.  My view is that the P studies taken as a 

15    whole indicate that the PES was carried out in a quite high 

16    quality way and the results were really very, very good, I 

17    thought.

18         Q.    Dr. Rolph, you said yesterday that you had 

19    written and, in fact, you were admitted as an expert in 

20    empirical Bayes methodology.

21               Did you testify yesterday that those are the 

22    techniques known as smoothing operation?

23         A.    Yes, that was my testimony yesterday.

24         Q.    Are you familiar with the Bureau's smoothing 

25    operations in the 1990 PES?
                                                              784

 1         A.    Yes, I am.

 2         Q.    Dr. Rolph, in general terms, can you describe to 

 3    the court what smoothing is?

 4         A.    Yes.

 5               Smoothing is a statistical method for reducing 

 6    the effect of sampling error.  It's used to improve the 

 7    accuracy of a collection of estimates, particularly in the 

 8    presence of sampling error.  It improves the accuracy by 

 9    aggregating information across this collection.  It 

10    particularly is useful in the presence of sampling error as 

11    in the case when you have, for example, small sample sizes.

12         Q.    Is smoothing a generally accepted method in the 

13    scientific community?

14         A.    Yes, it is, with the proviso that it goes under 

15    several different names.

16         Q.    What are some of the other names?

17         A.    As I mentioned, it goes under the name empirical 

18    Bayes methods --  

19               THE COURT:  Is that B A Y E S? 

20               THE WITNESS:  The Reverand Thomas, actually.  

21         A.    The other perhaps less colorful names are various 

22    components models, random effects models, and there may be a 

23    few other miscellaneous ones that come up, but those are the 

24    ones that are most commonly used.

25         Q.    Is smoothing used in a wide variety of 
                                                              785

 1    statistical applications?

 2         A.    Yes, it is.

 3         Q.    Can you give us some examples?

 4         A.    Well, probably the most common classic problem 

 5    it's used is in small area estimation, that is, when you are 

 6    trying to estimate, for example, cancer incidents rates 

 7    across, let's say, states; its being used in estimating crop 

 8    yields; it's been used generally in epidemiology for 

 9    estimating mortality morbidity rates; it's been used in 

10    estimating the performance of first year law students as a 

11    function of their LSAT scores and their undergraduates EPAs; 

12    it's been used in estimating highway accident rates.

13               That's are the ones that come immediately to 

14    mind.

15         Q.    Have you used it in your own work?

16         A.    Yes, I have.

17         Q.    Can you give us a few examples of that?

18         A.    Yes.  As I mentioned yesterday, one of the first 

19    projects I undertook when I came to Rand was a study of New 

20    York City fire alarms, and the problem there was, again, a 

21    small area estimation problem, trying to estimate the 

22    probability that a fire alarm that was reported from a 

23    particular street box actually signalled a serious fire as 

24    opposed to a false alarm or rubbish fire, and basically I 

25    used data in this case from the Borough of the Bronx to 
                                                              786

 1    estimate these probabilities and the statistical method I 

 2    used was smoothing.

 3               Other applications I made on my own work include 

 4    estimating or predicting who will be high rate serious 

 5    offenders, which comes out of statistical modeling of 

 6    criminal careers.

 7               I have used it in the current medical malpractice 

 8    work I'm doing in estimating claims rates for individual 

 9    physicians.

10               I can give you more, but that gives you a flavor 

11    of things.

12         Q.    Dr. Rolph, were the results of any of your uses 

13    of smoothing methodology tested in practice?

14         A.    Yes.  Probably the fire alarm example comes 

15    immediately to mind.

16         Q.    How was that tested?

17         A.    Well, the way we carried out the work there is we 

18    had three years of data on fire alarms in the Bronx.  We 

19    developed estimates of the probability that an alarm 

20    reported from each street box in the Bronx actually 

21    signalled a serious fire or did not based on that three 

22    years of data.

23               We then took those estimates and went to the next 

24    year and asked the question, did the smoothed estimates give 

25    better predictions for the next year of the probability that 
                                                              787

 1    the alarms reported a serious fire or did the estimates 

 2    without using smoothing give better estimates, and the 

 3    result of that comparison was that the smoothed estimates 

 4    performed better.

 5         Q.    And did you testify yesterday that a Census 

 6    Bureau study later relied upon the work that you had done in 

 7    the fire alarm case?

 8         A.    Yes.  There was the study I was referring to was 

 9    by two Census Bureau statisticians, Robert Fay and Roger 

10    Herriot, in which they were estimating income for the 39,000 

11    revenue sharing areas in the United States, and part of the 

12    statistical methods they used was a particular estimation 

13    method that appeared in our fire alarm paper.

14         Q.    Do you know if smoothing is used by the Census 

15    Bureau in other areas?

16         A.    Yes, I believe it is.  It's, I think, used in 

17    intercensal income estimation and there are a couple of 

18    others that don't immediately come to mind.

19         Q.    And do you know whether smoothing has been used 

20    by other government agencies in connection with official 

21    work?

22         A.    Yes.  I believe it's used by the Labor Department 

23    in estimating prices.

24         Q.    Was the method of the Bureau referred to as 

25    smoothing tested prior to the 1990 PES?
                                                              788

 1         A.    Yes, it was.  It was tested first in the 1986 

 2    test of adjustment related operations in Los Angeles and, 

 3    secondly, was used in the 1988 dress rehearsals.  Smoothing 

 4    models were applied and analysis of the data was carried out 

 5    using smoothing.

 6         Q.    And have you reviewed the actual procedures used 

 7    by the Bureau in smoothing the 1990 PES?

 8         A.    Yes, I have.

 9         Q.    Do you have an opinion about the quality of those 

10    procedures?

11         A.    Yes.  I thought that they, or I should say I 

12    think that they are fairly standard statistical procedures.  

13    They were carried out in a workmanlike way.  They walked 

14    through and went from one step to the next each in a, I 

15    thought, quite valid and statistically acceptable way and 

16    the results that came out I felt were quite valid and 

17    acceptable.  They were up to the usual statistical 

18    standards.

19         Q.    In general terms, Dr. Rolph, why is smoothing 

20    done?

21         A.    In general terms, smoothing is done when, in the 

22    presence of sampling error, as I mentioned, to reduce its 

23    effect.

24         Q.    And how does it accomplish that?

25         A.    Well, it's somewhat like an averaging procedure.
                                                              789

 1               The idea is if you have a set, a collection of 

 2    estimates each of which has sampling error in it, you 

 3    essentially aggregate information across those estimates in 

 4    order to improve --  in order to produce an improved set of 

 5    estimates, that is, a more accurate set of estimates with 

 6    less sampling error and the statisticians say smaller 

 7    variance.

 8         Q.    Can you tell us what variance is?

 9         A.    Variance is a measure that is used in statistics 

10    to measure sampling error.

11               There is another related term called standard 

12    error.  

13               Both of them are terms of art, if you will, that 

14    statisticians use for this purpose.

15         Q.    So are the terms reduction of variance and 

16    reduction of standard error essentially equivalent?

17         A.    Generally speaking, if you reduce the variance, 

18    you also reduce the standard error, yes.

19         Q.    Are standard error and variance related to sample 

20    size?

21         A.    Yes.  As a general rule the smaller the sample 

22    size, the larger the variance the larger the standard error, 

23    the smaller the sample size --  I'm sorry, I just said that 

24    --  the larger the sample size --  

25               THE COURT:  I thought I heard that before.
                                                              790

 1         A.    I don't have a excuse that it's late in the 

 2    afternoon.

 3               The larger the sample size the smaller the 

 4    variance, that is, the more accurate the estimate.  

 5               THE COURT:  It varies inversely? 

 6               THE WITNESS:  That's correct.

 7         Q.    Dr. Rolph, would the variance of the raw 

 8    adjustment factors in the PES have been different if the 

 9    Bureau had used a larger sample size?

10         A.    Yes.  The variance would have tended to be 

11    smaller.

12         Q.    How much less would the variance have been if the 

13    sample size in the PES had been 300,000 households?

14         A.    Well, given that the actual sample size was 

15    somewhat above 150,000 households and talking about roughly 

16    a doubling the sample size.

17               While it's difficult to give an exact number 

18    without going through the details of the design, a general 

19    rule of thumb is if you double the sample size, the variance 

20    will tend to be decreased by a factor of two, so it would 

21    roughly half the variance as a rough rule of thumb.

22         Q.    Dr. Rolph, do you know when the Bureau determined 

23    the number of post-strata that were to be used in the 1990 

24    PES?

25         A.    Yes.  I believe, according to my best knowledge, 
                                                              791

 1    there was some discussion in 1987, when Dr. Wolter presented 

 2    the results of the Bureau's adjustment work to one of our 

 3    meetings in which they talked about having something on the 

 4    order of about 1,300 post-strata.

 5               Subsequent to that I recall reading a document 

 6    dated in late 1980, I think perhaps August of 1980, which 

 7    laid out a post-stratification scheme which had 1,300-some 

 8    odd post-strata in it, and I can recall seeing another 

 9    document in 1989 that also had 1,300-some odd, I think the 

10    some-odd was slightly different in each of the two, but in 

11    both cases it was over 1,300.

12         Q.    I'm sorry, Dr. Rolph, did you say the subsequent 

13    presentation, him paper by Kirk Wolter was in what year?

14         A.    1987.  I was referring to a presentation that was 

15    made to a National Academy panel.

16         Q.    Do you remember seeing a subsequent paper which 

17    indicated the prospective sample size?

18         A.    Yes.  I think the subsequent paper I was 

19    referring to was the one I referred to as being in late 

20    1988.  I think it was August, August or so of 1988.

21               MR. SHERMAN:  I would like to draw the witness' 

22    and the court's attention to Plaintiff's Exhibit 566.

23               I am not sure that it is in our binders for this 

24    witness, but it has been admitted into evidence yesterday.

25               May I approach the witness, your Honor?  
                                                              792

 1               THE COURT:  Yes.

 2               (Handing to the witness)

 3               (Pause) 

 4               MR. BARON:  Can I have just a minute, your Honor?  

 5    I haven't been present during the entire trial.  

 6               THE COURT:  All right.

 7               (Pause) are 

 8               MR. BARON:  Your Honor, we are ready.  

 9               THE COURT:  All right.  

10    BY MR. SHERMAN:

11         Q.    Dr. Rolph, is this the document you were 

12    referring to?

13         A.    Yes, it is.

14         Q.    And directing your attention to the Bates page 

15    8965, does it say anything there about the prospective 

16    number of the post-strata for 1990?

17         A.    Yes.  This page described the post-strata and 

18    there is a sentence in about the middle of the page which 

19    reads, "The total number of post-strata is 1,332." 

20               That was the 1,300 and some odd I was referring 

21    to.

22         Q.    Thank you.

23               Dr. Rolph, I would like to draw your attention 

24    now to PX 66.

25         A.    Okay.
                                                              793

 1         Q.    Is this one of the documents that you reviewed in 

 2    following the Bureau's smoothing procedure?

 3         A.    Yes, it is.

 4         Q.    What is this document?

 5         A.    This is a document that is a memo from Cary 

 6    Isaki, who is the Bureau statistician who carried out the 

 7    bulk of the smoothing work, to Nash Monsour, dated June 1, 

 8    1989.  The title of the document is, "Explanatory variables 

 9    for regression modeling of coverage factors for the 1988 

10    dress rehearsal and 1990 census."

11         Q.    Is there an attachment to this document?

12         A.    Yes, there is.  The attachment is dated 4/21/89 

13    by C. Isaki and it's titled, "Models for 1990 coverage 

14    evaluation factors."

15         Q.    Drawing your attention to the bottom of the first 

16    page of that attachment, does it say anything there about 

17    the number of expected post-strata for the 1990 PES?

18         A.    Yes.  On the first page of the attachment, the 

19    next to the last line and the last line reads, "This section 

20    defines 110 general categories each broken down by age, sex 

21    cells (12) for a total of 1,320 post-strata."  

22               MR. SHERMAN:  At this time, your Honor, I offer 

23    Plaintiff's Exhibit 66.  

24               MR. BARON:  Your Honor, we have no objection 

25    since this apparently bears the Secretary's Bates stamp and 
                                                              794

 1    will be part of the documents that we will be introduce in 

 2    the administrative record in our case.  

 3               THE COURT:  Plaintiff's Exhibit 66 is admitted. 

 4               (Plaintiff's Exhibit 66 marked for identification 

 5    was received in evidence.) 

 6    BY MR. SHERMAN:

 7         Q.    Dr. Rolph, prior to July 1989, was the Bureau 

 8    planning to use somewhere around 1,300 post-strata in the 

 9    1990 PES?

10         A.    Yes, that is my understanding.

11         Q.    In actually conducting the PES, Dr. Rolph, how 

12    did the Bureau break up the country for smoothing purposes?

13         A.    Well, the Bureau broke up the population of the 

14    country into 1,392 post-strata.  Those 1,392 post-strata 

15    were divided into five regions.  One region consisted of 12 

16    American Indians --  I beg your pardon --  12 post-strata 

17    for American Indians, the remaining 1.380 post-strata were 

18    divided among the four regions of the country, namely, the 

19    northeast, the south, the midwest and the west.

20               Each region had between 300 and 350 or so 

21    post-strata and the definitions of the post-strata were 

22    basically along the lines of geography as well as 

23    characteristics of the population.

24         Q.    Dr. Rolph, would all post-strata in a region be 

25    equally affected by smoothing?
                                                              795

 1         A.    No.  What will tend to happen is those 

 2    post-strata that have large amounts of sampling error, for 

 3    example, ones with a small sample size in the 

 4    post-enumeration survey, they will be affected most by 

 5    smoothing, whereas those post-strata which have small 

 6    sampling error, there is a small variance, will tend to be 

 7    less effected, they will tend to stand on their own, if you 

 8    will.

 9         Q.    I believe you testified before that smoothing is 

10    like an averaging; is that correct?

11         A.    Yes.  I would describe it as a weighted averaging 

12    which also includes information on the characteristics of 

13    the post-strata in order to improve its accuracy.

14         Q.    I'm sorry, it's like a weighted averaging?

15         A.    It is like a weighted average with the addition 

16    it includes information about the characteristics of the 

17    post-strata to improve the accuracy.

18         Q.    Dr. Rolph, I wonder if you would, in layman's 

19    terms, take us through the steps that the Bureau actually 

20    used in the smoothing procedure.

21               What was the first step?

22         A.    Okay.

23               To set the stage, let's recall we are starting 

24    out now with the 1,392 raw adjustment factors and raw 

25    variances for those 1,392 adjustment factors.
                                                              796

 1               The first step is basically a preliminary step 

 2    which is called modeling the variance or also goes under the 

 3    name pre-smoothing, and that was carried out by the Bureau 

 4    to improve the accuracy of the estimates of the variances of 

 5    the raw adjustment factors.  And that's important, because 

 6    those variances figure into the way the smoothing operation 

 7    is carried out.

 8         Q.    Has the procedure of modeling variance ever been 

 9    used before?

10         A.    Yes.  It's a quite widely used technique in 

11    statistics.

12         Q.    Have you used it in your own work?

13         A.    Yes.  I believe there is a section in one chapter 

14    of a statistics text which I wrote with Carl Morris on the 

15    subject.  I believe I used it as part of the example I 

16    mentioned yesterday in analyzing casino winnings.  I believe 

17    I also used it in a regression study I did for a school 

18    finance case.

19               Those are the ones that come immediately to mind.

20         Q.    And have other statisticians used it as well?

21         A.    Yes.  I would describe it as a reasonably 

22    commonly used procedure.

23         Q.    Did the Bureau test the procedure of modeling 

24    variances for smoothing before the 1990 PES?

25         A.    Yes.  In the documentation I reviewed, they 
                                                              797

 1    tested --  they used modeling variances in analyzing the 

 2    1988 dress rehearsal data.

 3         Q.    Each adjustment factor has a model variance.

 4               What is the next step?

 5         A.    The next step is to, given you have these model 

 6    variances, is to carry out a regression.  

 7               MR. SHERMAN:  Your Honor, can I put up a chart 

 8    here?  

 9               THE COURT:  Yes.  

10               MR. SHERMAN:  This has been marked for 

11    identification as PX 719.  

12    BY MR. SHERMAN:

13         Q.    Does this chart represent your regression line, 

14    Dr. Rolph?

15         A.    Yes.

16               If I could get up and explain it.  

17               THE COURT:  Oh, sure.  

18               MR. BARON:  If I may, your Honor, I just want to 

19    state that this document was given to us in a package on May 

20    13.  It does not appear to me to have as its source any 

21    document that was in the administrative record.

22               I would wish counsel to indicate that.  So it has 

23    come very late.

24               I don't object to its use, however.  I think this 

25    and other documents counsel have just given us we might need 
                                                              798

 1    some additional time.  

 2               THE COURT:  If you need it, you got it.  

 3               MR. SHERMAN:  As I stated, as I informed 

 4    defendants' counsel when we gave them this chart, it is to 

 5    be used for illustrative persons with Dr. Rolph's testimony.  

 6               THE COURT:  Go ahead.

 7         A.    What the dots on this chart depict are raw 

 8    adjustment factors.

 9               Along as the vertical axis it runs from about .6 

10    to 1.4, which is roughly the kind of range you see for raw 

11    adjustment factors, and on the horizontal axis we are giving 

12    the mail return rate, which ranges from about one half or 

13    .45 up to .95 or so.

14               So a point on the chart, let's just say we will 

15    take this top point up here, that would correspondence to a 

16    post-strata that has a raw adjustment factor of about, oh, 

17    1.3, indicating about a 30 percent adjustment upward from 

18    the undercount, and it would be a post-strata which has a 

19    mail return rate of, oh, roughly .85 to .9.

20               So each of these points corresponds to a 

21    different post-strata.

22               Now, the idea behind regression, this line, as 

23    you can see, that goes through here is called a regression 

24    line, and basically a regression line is the line that goes 

25    through the points that is closest to the points in terms of 
                                                              799

 1    the vertical distances.  It's fit by a mathematical method 

 2    called least squares, but you should think of it simply as 

 3    being the line that is closest to the points, and so the way 

 4    to interpret it is if you think about a post-strata, say, 

 5    that has a mail return rate of, say, only one half, say only 

 6    about .5.

 7               What the regression would tell you is for 

 8    post-strata with a mail return rate of about 50 percent, you 

 9    would expect the raw adjustment factor to be about roughly 

10    1.1.  That is, you would be adjusting, making about a ten 

11    percent adjustment for the undercount.

12               And that is basically the regression method and 

13    that is the first step of the smoothing method.

14               And I should also add the mail return rate here 

15    is an example of what is called a carrier variable, which I 

16    guess comes up fairly regularly in this case.  

17               MR. BARON:  Your Honor, I would object and move 

18    to strike this testimony.

19               I understood this to be a hypothetical example of 

20    a regression line with an X and a Y axis.  I did not 

21    understand it that Dr. Rolph was going to stand up and 

22    testify.  He has no foundation to establish that the actual 

23    adjustment factors that are in the administrative record are 

24    represented by this chart.  The chart might not, in fact, do 

25    that and he has not testified in advance of standing up that 
                                                              800

 1    this is other than a hypothetical example.  

 2               MR. SHERMAN:  In fact, your Honor, I think, in 

 3    fact, it is taken from data, but I don't think Dr. Rolph has 

 4    attempted to represent that he is giving any more than an 

 5    explanation of the chart.

 6               He certainly isn't attempting to represent what 

 7    the actual correlation of the variables on this chart is.  

 8               MR. BARON:  I heard Dr. Rolph to say that somehow 

 9    that it was the actual raw adjustment factors that are part 

10    of what the Secretary did, in fact, do as opposed to some 

11    hypothetical.  

12               THE COURT:  Is that what you meant to say? 

13               THE WITNESS:  I was trying to explain how to 

14    interpret the regression line for this particular example.  

15    That was all I was meaning by what I was saying.  

16               THE COURT:  Those numbers are not, to your 

17    knowledge, the numbers involved, the precise numbers 

18    involved in the census case? 

19               THE WITNESS:  I was not saying as part of my 

20    testimony that they were.

21               If you are asking me that question, I can respond 

22    to it if you like.  

23               THE COURT:  I am not going to ask that.  You 

24    haven't said that it is yet.  I'm not asking.  

25               MR. BARON:  I still thing there is no foundation.  
                                                              801

 1               THE COURT:  There isn't, but he didn't say it.  

 2               MR. BARON:  All right.  Okay.  

 3    BY MR. SHERMAN:

 4         Q.    Dr. Rolph, what happens to each of the points on 

 5    this chart after the regression line has been drawn through 

 6    them?

 7         A.    Well, the points themselves remain as they are.  

 8    The fitted value on the regression line as I indicated for 

 9    each point is the value on the line that is directly below 

10    the point.

11               So if I can just go and point.  

12               THE COURT:  Yes.

13         Q.    There is a pointer on the table there.

14         A.    The answer to your question is the fitted value 

15    or the regression value for a given adjustment factor is 

16    gotten by taking the point and just going up, in this case 

17    the lines, for example, in this case the fitted value for 

18    this particular point would be approximately, I am going to 

19    eyeball it and say about .7, .8, something like that, or 

20    for, say, an extreme point over here this would correspond 

21    to a mail return rate of about .52 and its fitted value 

22    would be about 1.09 or 1.1.  That would correspondence to a 

23    fitted value in the regression.

24         Q.    If I could let's go to the next chart to we can 

25    go to the next step. 
                                                              802

 1               So there is no confusion, this is purely a 

 2    hypothetical chart.

 3               Dr. Rolph, what happens to each of the adjustment 

 4    factors in the next step?

 5         A.    Okay, if I can approach the chart again.  

 6               THE COURT:  Yes.

 7         A.    In this chart, to reduce the clutter, I have 

 8    simply represented one hypothetical raw adjustment factor.

 9               This would correspondence to a raw adjustment 

10    factor of about 1. oh, say, 2 or so, and a carrier variables 

11    value of about .85.  In the last example it was mail return 

12    rate, but it could be any of a variety of characteristics in 

13    the post-strata.

14               This line here is meant to represent the 

15    regression line from all the points.

16               The fitted value, then, for this raw adjustment 

17    factor is point on the line directly below it now, what 

18    smoothing does is to pull the raw adjustment factor towards 

19    this typical value, this fitted value, and it pulls it by an 

20    amount which depends on the sampling error.

21               So if this raw adjustment factor has a small 

22    sampling error, it will be moved just a little bit, small 

23    variance.  If this raw adjustment factor is based, say, on a 

24    very small sample size, has a large variance, it will be 

25    moved a lot.
                                                              803

 1               But smoothing basically is, as I indicated, like 

 2    an averaging, it's moving these raw adjustment factors 

 3    towards a typical value by an amount depending on sampling 

 4    error.

 5         Q.    Thank you, Dr. Rolph.  

 6               MR. SHERMAN:  Just for identification, that has 

 7    been marked as PX 720 for identification.

 8         Q.    Dr. Rolph, why don't you pull the raw adjustment 

 9    factor all the way down to the line?

10         A.    Well, the --  

11               THE COURT:  I don't understand that question.  

12    Try it again.

13         Q.    You said you moved the raw adjustment factor in 

14    smoothing, you moved it part way toward the line?

15         A.    Yes.

16               If I could go up.

17               The idea behind smoothing is to make the raw 

18    adjustment factor more accurate by reducing the effect of 

19    sampling error.

20               You don't want to move it more than the amount 

21    that you can account for by the sampling error itself.  So 

22    the idea behind this is you move it by an amount that is 

23    consistent with the sampling error, but no more, and for 

24    many of these adjustment factors, they will be further from 

25    the line than what you can account for by sampling error so 
                                                              804

 1    you, therefore, move them only part way.

 2               If you had an adjustment factor, as I indicated, 

 3    which had substantial sampling error, you would move it down 

 4    very close to the line.  

 5               THE COURT:  The line is the regression line? 

 6               THE WITNESS:  The regression line.  

 7               So you should think of the regression line as 

 8    being a typical value or average value, that's the way to 

 9    think about it, and you are trying to essentially, if you 

10    have lots of sampling error, you are trying to move it 

11    towards, if you will, an overall, speaking a little loosely, 

12    an overall average.

13         Q.    And how do you know how far towards the line to 

14    move it?

15         A.    Well, there is a mathematical formula that gives 

16    it to you which I won't attempt to explain, but generally 

17    speaking, you know how far to move it towards the line, it's 

18    going to depend on the variance of this raw adjustment 

19    factor, which you will recall is the quantities that 

20    statisticians or many statisticians, most statisticians can 

21    be used to measure sampling error.

22         Q.    And so the modeling of the variances that you 

23    described in the first step comes into play in this step?

24         A.    Yes.

25               Basically, the model variances in the first step 
                                                              805

 1    are the factors that are used or the quantities that are 

 2    used to determine how far you are going to move this raw 

 3    adjustment factor towards its typical value.

 4         Q.    Thank you.

 5               Dr. Rolph, are all the points moved the same 

 6    distance toward the line?

 7         A.    No, they are not.

 8         Q.    I want to go to Plaintiff's Exhibit 721, again, 

 9    purely hypothetical chart, not based on any actual data in 

10    the case.

11               Can you tell us what this chart shows about 

12    moving the points towards the regression line?

13         A.    Certainly.

14               Yes, I can.  If I could, the question is, can I 

15    explain the difference in the amounts points are moved 

16    towards the line.

17         Q.    If you would explain what the chart shows.

18         A.    Okay, thank you.

19         Q.    Since I am not being very successful in getting 

20    the information through my questions.

21         A.    Okay.  Thank you.

22               The chart, again, on the vertical axis gives the 

23    adjustment factors and on the horizontal axis give these 

24    carrier variables.  This is a characteristic of the 

25    post-strata.  Okay? 
                                                              806

 1               The circles, this chart has about a dozen 

 2    different post-strata depicted on it.

 3               The circles indicate the raw adjustment factors.  

 4    So, for example, this raw adjustment factor corresponds or 

 5    this raw adjustment factor is a post-strata that has a 

 6    carrier variable value of about .92 and has a raw adjustment 

 7    factor of, oh, about 1.5 or so and this point corresponds to 

 8    a post-strata of a raw adjustment factor of 1.2 and a 

 9    carrier variable fact for of about .53.

10               Now can I answer your question? 

11               The question was, as I understand it, can I 

12    explain why and how the different post-strata get affected 

13    differently by smoothing.  And let me take two examples.

14               The first example, let's again take this point 

15    here.

16               The point hear you notice the raw adjustment 

17    factor in smoothing moves about, oh, two-thirds, 

18    three-quarters of the way towards the line.  This would 

19    correspond to a post-stratum where the raw adjustment factor 

20    had substantial sampling error and, therefore, we would move 

21    it towards this typical value or this sort of averaging 

22    process.

23               Conversely, here's a post-strata over here that 

24    has a small variance, small sampling error, perhaps a PES 

25    sample size that was fairly large.  This post-strata will 
                                                              807

 1    get moved just a little bit by the smoothing operation, the 

 2    reason being that it has small sampling error so we really 

 3    don't have to try to reduce the effect of sampling error 

 4    since it's pretty small.

 5               And depending on the estimated sampling error, 

 6    namely, the variance of these points, they will be moved by 

 7    different amounts by the smoothing process.

 8         Q.    So is it the case that the raw adjustment factors 

 9    that you have the most confidence in get moved the least?

10         A.    Yes, that's exactly the case.

11         Q.    Dr. Rolph, you mentioned a couple of times the 

12    carrier variables.

13               Can you explain to the court briefly what the 

14    carrier variables are?

15         A.    Yes.

16               In order to produce this typical value, the 

17    Census Bureau didn't use a single carrier variable, like the 

18    mailback rate, but it used a number of the characteristics 

19    of the post-strata, which would include age, sex, race, mail 

20    return rate was an example I mentioned, owner/renter and so 

21    forth.

22               Basically, you used the characteristics that 

23    gives you the best estimate of this typical value, in 

24    effect.

25         Q.    Can you mention what some of the carrier 
                                                              808

 1    variables were?

 2         A.    Yes.  I think I mentioned a number of them.  In 

 3    addition to the ones I just mentioned --  actually, at the 

 4    moment I'm drawing a blank beyond the list that I just 

 5    rattled off.  

 6               There are a number of others, but they are 

 7    basically the characteristics of the post-strata that are 

 8    reflective of the undercount.

 9         Q.    How did the Bureau select the carrier variables?

10         A.    What the Bureau did was first they prepared a 

11    list of carrier variables they called the must variables, 

12    meaning that those carrier variables will be included in all 

13    of the --  these carrier variables will be included in all 

14    the regressions that are carried out.  Those consist of age, 

15    sex, race, owner/rental status.

16               In addition the Bureau prepared a list of what 

17    they called candidate carrier variables.  Those consisted of 

18    characteristics which past research, both in past censuses, 

19    the dress rehearsal, Bureau scholars and outside scholars 

20    had found to be related to the undercount at one time or 

21    another.

22               The Bureau then used a statistical technique to 

23    select the set of carrier variables that produced the best 

24    fitting typical value, that is, they included all these 

25    variables I designated as must plus the candidate variables 
                                                              809

 1    that produced the best fit.

 2         Q.    I would like to call your attention to PX 76, Dr. 

 3    Rolph.

 4         A.    Okay, I have it in front of me.

 5         Q.    Dr. Rolph, is this one of the documents that you 

 6    reviewed in studying the Bureau's smoothing procedures?

 7         A.    Yes, it is.

 8         Q.    Could you tell the court what this document is?

 9         A.    Yes.  It's a memo from Cary Isaki, the 

10    statistician I mentioned before, to Paula Schneider.  The 

11    subject of the memo is 1990 post-enumeration survey 

12    smoothing factors.

13         Q.    And does this provide any information on the 

14    Bureau's choice of carrier variables?

15         A.    Yes.  This specifically refers to what I referred 

16    to as the must variables, and the idea behind this memo is 

17    to make sure to include age and sex in order to preserve the 

18    demographic information in these smooth adjustment factors, 

19    and that's basically what this memo is talking about.

20         Q.    Thank you.

21               Dr. Rolph, did the Bureau get comments or 

22    suggestions from experts outside the Bureau on which carrier 

23    variables to include?  

24               MR. BARON:  Objection; no foundation.  

25               MR. SHERMAN:  All right.
                                                              810

 1         Q.    If you know.  

 2               THE COURT:  Please restate the question.  

 3               MR. SHERMAN:  All right.

 4         Q.    Let me call your attention to Plaintiff's Exhibit 

 5    162, which was admitted into evidence yesterday.

 6               (Handing to the witness)

 7               (Pause)

 8         A.    I now have it.

 9         Q.    What is 162?

10         A.    162 is a letter on Nielsen Corporation's 

11    stationery.  It's addressed, "Dear Howard," and it's signed, 

12    "Kirk." 

13               I would infer from the content of it and the 

14    context of it it's addressed to Howard Hogan and that the 

15    signatory is Kirk Wolter.

16         Q.    And what is the subject of this letter?

17         A.    It is a letter in which Dr. Wolter is commenting 

18    on the suggested list of carrier variables or candidate 

19    carrier variables that the Bureau had been circulating prior 

20    to finalizing, if you will, their list of carrier variables.

21         Q.    Do you know if Dr. Wolter was a member of the 

22    special advisory panel at this time?

23         A.    Yes, he was.

24         Q.    So let me go back to my question.

25               Did the Bureau receive comments from experts 
                                                              811

 1    outside the Bureau in deciding which variables to use in 

 2    smoothing?

 3         A.    Yes, they did.

 4         Q.    Do you have an opinion about whether the Bureau 

 5    used proper variables in the smoothing operation?

 6         A.    Yes, I do.

 7         Q.    What is that opinion?

 8         A.    Well, I feel, I believe that based on my review 

 9    of the documents that we have discussed already that the 

10    Bureau went through a quite careful process in laying out 

11    both the must variables and the reasons behind them as well 

12    as in assembling a list of candidate variables that was 

13    fairly comprehensive and was vetted, if you will, by a 

14    number of people who are knowledgeable in the field, so I 

15    felt that the process that was used was really very sound.

16         Q.    Dr. Rolph, you have given us the layman's 

17    explanation of smoothing.

18               What is the result as the end of the whole 

19    process?  What do you have?

20         A.    Well, the result is you have, of course --  you 

21    go from having a set of 1,392 raw adjustment factors to 

22    1,392 smoothed adjustment factors which have increased 

23    accuracy as compared to the original raw adjustment factors.

24         Q.    Dr. Rolph, is there a document which describes in 

25    more detail the smoothing process that you have just 
                                                              812

 1    described?

 2         A.    Yes.  I'm aware of at least one such document.

 3         Q.    I would like to call your attention to PX 145.

 4         A.    I have it.

 5         Q.    Have you reviewed this document in the course of 

 6    examining the Bureau's work on smoothing?

 7         A.    Yes, I have.

 8         Q.    Does this document describe the smoothing 

 9    process?

10         A.    Yes.  It describes it in a fair bit of detail.  

11               MR. SHERMAN:  At this time I would like to move 

12    Plaintiff's Exhibit 145.  

13               MR. BARON:  Your Honor, this does not appear to 

14    have a government Bates stamp which would indicate that it 

15    would not be part of what the Secretary considered to be in 

16    the administrative record and, therefore, we object on 

17    authenticity grounds.  

18               THE COURT:  Mr. Sherman?  

19               MR. SHERMAN:  Well, your Honor, this is a 

20    paper --  

21               MR. BARON:  Also, excuse me, there is no date on 

22    this.  

23               MR. SHERMAN:  Your Honor, this is an undated 

24    document.  It is authored by the --  if you wish I can 

25    establish it through Dr. Rolph --  it is authored by the 
                                                              813

 1    staff at the Bureau responsible for the smoothing operation.  

 2    In fact, it was presented as the final version of the 

 3    smoothing process at a conference of the American 

 4    Statistical Association, and we are offering it for the 

 5    proposition that it is not in the administrative record and 

 6    should be.  

 7               MR. BARON:  Your Honor, I thought Dr. Rolph was 

 8    testifying here --  there is also a hearsay problem with the 

 9    document.  

10               THE COURT:  The authenticity and hearsay 

11    objections are overruled.  145 is admitted.

12               (Plaintiff's Exhibit 145 marked for 

13    identification was received in evidence.) 

14    BY MR. SHERMAN:

15         Q.    Dr. Rolph, grain, what was the purpose of the 

16    smoothing procedure?

17         A.    It was to improve the accuracy by reducing the 

18    effect of sampling error of the estimates of the adjustment 

19    factors.

20         Q.    Did it succeed in doing so?

21         A.    Yes, I believe it did.

22         Q.    Did it reduce the standard errors for some groups 

23    of the population more than others?

24         A.    Yes, it did.  As I indicated earlier from the 

25    chart here, it tends to reduce, have a larger effect on 
                                                              814

 1    those post-strata which have large variance, that is, large 

 2    sampling error, and a smaller effect, of course, on 

 3    post-strata that are more accurately estimated by the raw 

 4    adjustment factors.

 5         Q.    If I might go to Plaintiff's Exhibit 722.

 6               This is not a hypothetical.  I believe the 

 7    defendants have been provided with the source.  

 8               Does this chart indicate what you just testified 

 9    to with regard to the reduction of the standard error?

10         A.    Yes, it does.  It indicates --  it breaks down 

11    the population of the country by racial group.

12               The dark bars on the chart indicate the standard 

13    error for the smoothed estimates for those racial groups, 

14    the, if you will, diagonally slashed bar indicated the 

15    standard errors for the unsmoothed portions of the 

16    population.

17               You will note that, for example, with American 

18    Indians, you see a substantial difference in the standard 

19    error, which is the measure of reduction of sampling error.

20               For the smoothed and the unsmoothed, it goes, for 

21    the unsmoothed, it's about 1.3 and for smoothed it goes down 

22    to about 1.0, which is approximately a one quarter 

23    reduction.  Similarly for the Asian group, the Hispanic 

24    group and the black group.

25         Q.    Dr. Rolph, in reducing the standard error, did 
                                                              815

 1    the smoothing operation have an effect on the estimated 

 2    undercount?

 3         A.    Yes, it did.

 4         Q.    Did it have a different effect at different 

 5    geographic levels?

 6         A.    Yes.  As I indicated --  where smoothing is 

 7    really necessary and, hence, where it has the most effect 

 8    it's small groups, where we have large sampling error or 

 9    larger sampling error than when you have for larger groups.

10               Therefore, the effect of smoothing is going to be 

11    greatest on small geographic areas and when you average up 

12    to get larger areas the effect of smoothing, if you have a 

13    large enough area, would be really rather small.

14         Q.    Did smoothing have any substantial effect at the 

15    state level?

16         A.    At the state level, if you look at the percentage 

17    of the population for each state that you get by the 

18    smoothing method as compared to not by the smoothing method, 

19    those shares, those state shares, tend to be almost the same 

20    whether you smooth or don't smooth because, as I indicated, 

21    we are talking about a relatively large area.

22         Q.    Dr. Rolph, I would like to show you what has been 

23    marked as Plaintiff's Exhibit 723 and ask you if that 

24    demonstrates the point you just made?

25         A.    Yes.  This chart, let me just explain it so it is 
                                                              816

 1    clear, this chart, the red bars on the chart indicate the 

 2    estimates of the state share produced by the smooth 

 3    adjustment factors.

 4               So we see, for example, for the largest state, 

 5    California, the estimate with the smoothed adjustment 

 6    factors is slightly above 12 percent of the nation's 

 7    population.

 8               The blue --  I beg your pardon, I misspoke, the 

 9    red is the raw adjustment factors, the blue is the smoothed 

10    adjustment factors.  

11               THE COURT:  Are there two colors in there? 

12               THE WITNESS:  There are two colors.  

13               THE COURT:  I see.  Yes. 

14               THE WITNESS:  Okay, the raw and the smooth.

15         A.    And what you see is the tops of those two are 

16    very close to one another, which is simply depicting 

17    graphically the fact that the estimated share of the 

18    population for California and, indeed, for any of the 

19    states, is about the same whether you use the smoothed 

20    adjustment factors or the raw adjustment factors.

21               This is the effect you get when you start looking 

22    at larger areas of smoothing as opposed to some of the 

23    smaller post-strata I was talking about in my explanation.

24         Q.    Thank you.  

25               MR. SHERMAN:  Your Honor, at this time plaintiffs 
                                                              817

 1    would move PX 722 and 723, which are the two charts that we 

 2    have shown which are based on actual census data.  

 3               MR. BARON:  Objection.

 4               We have a number of concerns about these charts.

 5               First of all, they appear to be --  although they 

 6    are listed as the source documents in the administrative 

 7    record, they do not appear anywhere in the administrative 

 8    record.

 9               Secondly, during discovery in this case, Dr. 

10    Rolph was asked a question whether he intended to submit any 

11    --  do any analysis that would lead to a written work 

12    product in any form, and apparently he has testified to 

13    charts which do represent such analysis.  

14               THE COURT:  Did you prepare these charts? 

15               THE WITNESS:  No, I did not.  

16               THE COURT:  The objections are overruled.  The 

17    two exhibits are admitted.

18               (Plaintiff's Exhibit 722 and 723, respectfully, 

19    marked for identification were received in evidence.) 

20    BY MR. SHERMAN:

21         Q.    Dr. Rolph, I believe you testified that you read 

22    Secretary Mosbacher's decision, is that correct?

23         A.    Yes, I have.

24         Q.    I would like to read you a quote from the 

25    Secretary's decision this is at pages 2-54 to 2-55.  
                                                              818

 1               THE COURT:  Of what exhibit?  

 2               MR. SHERMAN:  I don't believe it is an exhibit, 

 3    your Honor.  

 4               THE COURT:  Yes.  It is in there somewhere.  

 5               MR. BARON:  I think it is PX 9. 

 6               MR. ZIMROTH:  I think this was introduced through 

 7    Professor Ericksen, your Honor.  

 8               MR. SHERMAN:  PX 9.

 9               (Pause) 

10               THE COURT:  And you are reading from?  

11               MR. SHERMAN:  Pages 2-54 to 2-55, carryover 

12    sentence.

13         A.    I don't have PX 9 in my book.  

14               THE COURT:  Here, look on with me, let's not 

15    waste any more time.

16         Q.    "One of the most problematic parts of the 

17    adjusted process was the bundle of statistical 

18    techniques -- " 

19               THE COURT:  You said 2-54?  

20               MR. SHERMAN:  2-54, the bottom, the carryover 

21    sentence to 2-55.  

22               THE COURT:  What is the top of page 2-54, 

23    "Estimates, all of which are"?  

24               MR. BARON:  Yes, that's right.  It is the Bates 

25    stamp with 70 on the bottom.  
                                                              819

 1               MR. SHERMAN:  I'm sorry, at the top of 2-55.  

 2               THE COURT:  Okay.  

 3               MR. SHERMAN:  My apologies.

 4         Q.    "One of the most problematic parts of the 

 5    adjusted process was the bundle of statistical techniques 

 6    contained in the smoothing process." 

 7               Do you see that?

 8         A.    Yes, I do.

 9         Q.    Do you agree with that?

10         A.    I definitely agree that smoothing is a bundle of 

11    statistical techniques, indeed, virtually everything or much 

12    of what the Census Bureau does is, in fact, statistical 

13    techniques.  They carry out surveys, they take censuses, so 

14    smoothing certainly is a bundle of statistical techniques as 

15    I have attempted to explain.

16               What I strongly disagree with is whether there is 

17    anything problematic with it, so I very strongly disagree 

18    with that statement.

19         Q.    Dr. Rolph, in general do you find the reasons 

20    that Secretary Mosbacher gives for overturning the Bureau's 

21    recommendation on adjustment to be cogent?

22         A.    No, I definitely do not.

23         Q.    Dr. Rolph, could you elaborate?

24         A.    I found that sentence as an example of the reason 

25    which I find particularly uncogent, if that is a proper 
                                                              820

 1    word, namely, there is --  I read it as being a general 

 2    attack on the statistical methods that were used to carry 

 3    out the adjustment, and I feel that the Census Bureau 

 4    really, as they generally do, did a superb job in carrying 

 5    out really a rather extensive operation.  

 6               It's probably the most, to my knowledge, 

 7    extensively documented statistical operation in the history 

 8    of the world, this particular census and PES and set of dual 

 9    system estimates.  They did it really in a open setting with 

10    everybody looking at it, including themselves, special 

11    advisory panels, et cetera, et cetera, et cetera, and I 

12    really feel that they did a superb job, and the attacks on 

13    the bundle of statistical techniques being problematic, and 

14    I think there is somewhat stronger language in the 

15    Secretary's decision, is being totally inappropriate.

16               Secondly, I feel that some of the criteria that 

17    the Secretary used were not the best criteria, specifically 

18    with respect to lose functions.  The Secretary tended to 

19    count up political jurisdictions, and as I mentioned 

20    earlier, our panel, and I subsequently in the report I did 

21    for the special advisory panel, explicitly argued that that 

22    was not the way to make such a decision.

23         Q.    Dr. Rolph, you have been qualified as an expert 

24    here and you testified about your work with the Bureau as an 

25    outside expert and your study of the 1990 procedures.
                                                              821

 1               Do you have an opinion about whether the 1990 

 2    original enumeration or the corrected counts are more 

 3    accurate?

 4         A.    Yes, I do.

 5         Q.    What is your opinion?

 6         A.    I believe that the corrected counts are the more 

 7    accurate of the two and I believe that quite conclusively.  

 8               MR. SHERMAN:  No further questions at this time, 

 9    your Honor.  

10               THE COURT:  All right.  Let's take a 20 minute 

11    break before we start the cross.

12               (Recess) 

13    

14    

15               (Continued on the next page.)

16    

17    

18    

19    

20    

21    

22    

23    

24    

25    
                                                              822

 1               THE COURT:  Are you ready, Mr. Baron?  
                                                        

 2               MR. BARON:  Thank you, your Honor.

 3    CROSS-EXAMINATION

 4    BY MR. BARON:     

 5         Q.    Good morning, Dr. Rolph.

 6         A.    Good morning, Mr. Baron.

 7         Q.    Dr. Rolph, you took a graduate-level course once 

 8    from Professor David Freedman, didn't you?

 9         A.    Yes, I remember it well.

10         Q.    In your opinion Professor Freedman is an 

11    excellent teacher of probability at the graduate level; 

12    isn't that correct?

13         A.    Yes, I believe I passed the course, and I believe 

14    I remember something afterwards.

15         Q.    Indeed, you believe Professor Freedman a 

16    high-quality scholar and researcher; isn't that correct?

17         A.    I think that's accurate, yes.

18         Q.    According to your testimony of yesterday, Dr. 

19    Rolph, one purpose of a loss function is to give a yardstick 

20    for measuring how close census counts are to the truth; 

21    isn't that right?

22         A.    Yes.

23         Q.    And another purpose is to give a yardstick for 

24    measuring how close adjusted counts are to the truth; isn't 

25    that correct?
                                                              823

 1         A.    Yes.

 2         Q.    Dr. Rolph, do you know the truth?

 3         A.    No, if I knew it, I doubt that we would be here.

 4         Q.    Dr. Rolph, you testified yesterday, did you not, 

 5    that the Bureau, unlike the Secretary, followed the National 

 6    Academy Decennial Census Panel's recommendations on loss 

 7    functions; isn't that right?

 8         A.    On general terms, I believe I did,.

 9         Q.    The Bureau's technical assessment of the accuracy 

10    of unadjusted versus adjusted census counts is in the report 

11    of the Undercount Steering Committee; is it not?

12         A.    To the best of my recollection.

13         Q.    According to your testimony, the Decennial Census 

14    Panel explicitly recommended against loss function analyses 

15    which count up political jurisdictions; did they not?

16         A.    Yes, I believe I testified to that effect.

17         Q.    Didn't the Undercount Steering Committee present 

18    a loss function analysis for states which counts up states 

19    made better and worse?

20         A.    I don't recall precisely, but that sounds about 

21    right.

22         Q.    Did the Undercount Steering Committee present any 

23    other quantitative results from loss function analysis for 

24    states?

25         A.    I can't recall the specific loss functions they 
                                                              824

 1    used.  I do recall they used several of them, though.

 2         Q.    You testified, did you not, that the Decennial 

 3    Census Panel recommended against loss function analyses that 

 4    were not weighted for population size; isn't that right?

 5         A.    That's correct, yes.

 6         Q.    Are the loss functions of the Undercount Steering 

 7    Committee weighted for population size?

 8         A.    Which loss functions are you referring to?

 9         Q.    The ones in the report, if you know.

10         A.    I don't recall the report in enough detail to 

11    answer that yes.

12         Q.    Does the Secretary's main decision document 

13    present information about the population in areas supposed 

14    to be made better by adjustment?

15         A.    I believe, if I understand your question, which 

16    I'm not sure I do, the answer is yes.

17         Q.    So you don't know whether the Undercount Steering 

18    Committee did, but you believe the Secretary did, isn't that 

19    your testimony?

20         A.    I'm sorry.  I've lost the question.  It did what?

21         Q.    Present loss functions which include information 

22    about the population in areas supposed to be made better by 

23    adjustment?

24         A.    I'm happy to agree that each of those two 

25    documents says what it says.  I can't recall in detail the 
                                                              825

 1    contents of each, but if you represent that to be the case, 

 2    I certainly agree.

 3         Q.    Dr. Rolph, you've been retained as an expert 

 4    witness since June or July of 1991; correct?

 5         A.    Yes, that's correct -- no, I was the retained as 

 6    a consultant, I believe, in June or July of 1991.  I was 

 7    only designated as an expert witness, if I recall correctly, 

 8    a couple of months ago.

 9         Q.    When I took your deposition last month, you 

10    hadn't performed any independent analyses of the 1990 

11    production smoothing model, had you?

12         A.    No, I had not.

13         Q.    At the time of that deposition last month you 

14    hadn't performed any independent calculations regarding how 

15    much uncertainty there was in the variances of the smoothed 

16    adjustment factors, had you?

17         A.    That's correct.

18         Q.    As of the time of your deposition, you had not 

19    performed any statistical analyses on any census-adjustment 

20    related issue since the time you have been a consultant or 

21    retained as an expert in the case, isn't that correct?

22         A.    Yes, that's also correct.

23         Q.    Dr. Rolph, you testified both yesterday and today 

24    about your participation on two panels on census 

25    methodology; correct?
                                                              826

 1         A.    Yes, that's correct.  Two National academy 

 2    Panels.

 3         Q.    And Plaintiff's Exhibit 2 is the final report of 

 4    the first census panel you served on, correct?

 5         A.    Yes, that is Plaintiff's Exhibit 2.

 6         Q.    That report makes no unconditional recommendation 

 7    that the 1990 census should be adjusted, does it?

 8         A.    It does not make an unconditional recommendation, 

 9    that's correct.

10         Q.    Would it be fair, Dr. Rolph, to characterize the 

11    report as recommending more research to solve problems 

12    related to adjustment?

13         A.    That's certainly one component of the report, 

14    yes.  That's one component of it.

15         Q.    The panel did recommend that the Bureau not 

16    concentrate its research efforts on the Dual System 

17    Estimator as an adjustment mechanism, isn't that correct?

18         A.    If you can be more specific, I'm not sure what 

19    you're asking me.

20         Q.    Well, didn't the panel recommend that the Bureau 

21    consider other possible adjustment methodologies?

22         A.    If I recall correctly, there is some language to 

23    that effect.  I'd have to reread the report.  My best 

24    recollection is that we anticipated that the dual system 

25    estimation method would be the one that would be used, 
                                                              827

 1    though.

 2               MR. BARON:  May I approach, your Honor?  

 3               THE COURT:  Yes.  I'm going to hand the witness 

 4    what has been marked as Defendant's Exhibit 505 in your 

 5    Honor's Defendant's Exhibit book.

 6         Q.    What I've handed you is an excerpt from the 

 7    Secretary's decision which is Bates stamped starting at 

 8    Administrative Report 62, 2-46.  It represents a portion of 

 9    what is in Plaintiff's Exhibit No. 9, and the copy that I've 

10    handed you, Dr. Rolph, the only difference between PX-9 and 

11    this is that it also has an Exhibit No. 3 from the Rolph 

12    deposition.

13               Do you recognize this excerpt?

14         A.    Yes, I believe I do.

15         Q.    The Secretary discusses the issues of smoothing 

16    presmoothing and the treatment of variance outliers in this 

17    discussion of Guideline Three; correct?

18         A.    Are you referring my attention to the bottom of 

19    the page?

20         Q.    Throughout the document.

21         A.    I'd have to reread the document to respond.  I 

22    believe there is some discussion along those lines, but I 

23    can't recall the exact wording.

24         Q.    Let me refer to you to page 2-49 in the document 

25    which is Bates stamped Administrative Record 65.  
                                                              828

 1               Do you see the bottom paragraph that starts, 

 2    "Smoothing involves three major judgmental decisions"?

 3         A.    Yes, I do.

 4         Q.    Do you see the rest of the sentence which says 

 5    that those major judgmental decisions are the treatment of 

 6    outliers, the variance presmoothing, and the choice of 

 7    so-called carrier variables?

 8         A.    Yes, I'm reading that sentence.

 9         Q.    So you would agree with me, Dr. Rolph, that the 

10    Secretary is discussing treatment of outliers, variance 

11    presmoothing and choice of so-called carrier variables in 

12    this Guideline Three.  

13               Isn't that right?

14         A.    I believe that's exactly what the sentence says, 

15    yes.

16         Q.    Now, with respect to the Decennial Census Panel, 

17    the first one that is represented in Plaintiff's Exhibit 2, 

18    did that panel make a formal recommendation on the issue of 

19    smoothing, Dr. Rolph?

20         A.    To the best of my recollection, we did not make a 

21    formal recommendation on the issue of smoothing.

22         Q.    Did the panel report make any formal 

23    recommendation on the issue of presmoothing?

24         A.    The answer is the same to that.

25         Q.    Did the panel report make a formal recommendation 
                                                              829

 1    on the treatment of variance outliers?

 2         A.    I'm certain that we did not do that, no.

 3         Q.    So the formal recommendations in that report, 

 4    which is Plaintiff's Exhibit 2, do not address any of the 

 5    following three issues:  Smoothing, presmoothing and 

 6    variance outliers; isn't that correct?

 7         A.    The first part of your question was the formal 

 8    recommendation?

 9         Q.    The formal recommendations in the Decennial 

10    Census Panel Report, which is Plaintiff's Exhibit 2.

11         A.    That's correct, yes.

12         Q.    Thank you.

13               Do you have Plaintiff's Exhibit No. 64.  

14               MR. BARON:  That is Defendant's Exhibit 503, your 

15    Honor, in that book before you.  

16         Q.    Do you recognize this report, Dr. Rolph?

17         A.    Yes, I do.  It is the report I prepared for the 

18    Special Advisory Panel in the summer of 1990.

19         Q.    This report was critical of the Bureau's plans 

20    for the P studies; was it not?

21         A.    This report was critical of the summary document 

22    entitled "Technical Operation Plans," which was the summary 

23    then of the P studies that were planned.

24         Q.    Do you have Plaintiff's Exhibit 71, which is the 

25    the Technical Operation Plans exhibit.  
                                                              830

 1               MR. BARON:  This is, your Honor, in 504 of my 

 2    book before you.

 3         A.    Yes, I have Plaintiff's Exhibit 71 in front of me 

 4    now.

 5         Q.    What's the title of this document, Dr. Rolph?

 6         A.    It's a mouthful.  It reads, "Technical Operation 

 7    Plans for Coverage Measurement and Other Adjustment Related 

 8    Activities of the Bureau of the Census for the 1990 

 9    Decennial Census of Population and Housing."

10         Q.    Your August 15, 1990 report, which is Plaintiff's 

11    Exhibit 64, was mainly commenting on this document, was it 

12    not?

13         A.    That's the document I'm referring to when I say 

14    "technical operation plans", yes.

15         Q.    And you refer to it in your August 15 report as 

16    the TOP report; correct?

17         A.    That's correct.

18         Q.    And the TOP report was dated April 9, 1990 

19    correct?

20         A.    That's what it says on the front of the document.

21         Q.    You weren't aware of any later draft of the TOP 

22    Report when you wrote your August 1990 report, were you?

23         A.    No, I was not.

24         Q.    The TOP Report consisted of a set of fairly short 

25    descriptions ever the P projects and the D projects, 
                                                              831

 1    correct?

 2         A.    Yes.

 3         Q.    Now, at the time you prepared your August 15, 

 4    1990 report, it was your opinion that the technical 

 5    operation plans were incomplete when you examined them 

 6    because they did not contain an explicit and detailed 

 7    description of exactly how the adjusted counts should be 

 8    calculated; isn't that right?

 9         A.    Are you quoting for me now?

10         Q.    Well, is that your opinion, Dr. Rolph?

11               MR. SHERMAN:  Your Honor, the witness has asked 

12    whether counsel is quoting for him, and I think he's 

13    entitled to know.  

14               THE COURT:  I will overrule the objection.  

15               Do you have the question in hand? ?

16               MR. BARON:  Shall I repeat it?

17         A.    Can you repeat just the tail end of the question?  

18    That would be sufficient.  Just the last three or four words 

19    you stated.

20         Q.    Let me read it for the record.  

21               At the time you prepared your report, it was your 

22    opinion that the technical operation plans were incomplete 

23    when you examined them because they did not contain an 

24    explicit and detailed description of exactly how the 

25    adjusted counts should be calculated?
                                                              832

 1         A.    Yes, that was one of my comments in my report or 

 2    words to that effect.

 3         Q.    In fact, you believed at the time of your August 

 4    15 report that the TOP Report had not achieved its purpose 

 5    in articulating the relevant technical and nontechnical 

 6    statistical and policy grounds for the decision on whether 

 7    or not to adjust the 1990 census, isn't that right?

 8         A.    That sounds fairly close to a quote from my 

 9    report.

10         Q.    When you prepared your August 15, 1990 report, 

11    Dr. Rolph, you believe that much more material on choice of 

12    loss functions was needed in the Bureau's TOP Report; did 

13    you not?

14         A.    If that's what my report said, I'm sure I did.  

15    That sounds about right to me, yes.

16         Q.    You would agree, would you not, that since census 

17    counts are used for a variety of purposes, it is impossible 

18    to come up with a loss function that will be optimal for all 

19    uses of the census?

20         A.    That sounds very much like a direct quote.  Yes, 

21    I would agree with that.

22         Q.    There's no loss function, is there, that is 

23    generally recognized in the statistical community as the 

24    correct loss function for use in evaluating adjustment, is 

25    there?
                                                              833

 1         A.    That's correct, yes.

 2         Q.    Indeed, the same statistician could use different 

 3    loss functions to evaluate adjustment of the 1990 census, 

 4    isn't that true?

 5         A.    I certainly could use different loss functions.  

 6    I think you'll have to specify in your question.  I'm not 

 7    sure what you're asking me, quite frankly.  

 8               There are certainly many different loss 

 9    functions.  You could use them to evaluate an adjustment, I 

10    agree with that.  

11               Is there something more to the question?

12         Q.    No.  But let me ask, do you recall at the time 

13    that I took your deposition the following question and 

14    answer:

15               "Q.    Different statisticians could choose 

16    different loss functions to evaluate the results of the 

17    adjustment?

18               "A.    Indeed, the same statistician could use 

19    different loss functions also."

20               Does that refresh your recollection?

21         A.    If you're asking me whether I remember that 

22    question and answer, the answer to your question is no.  

23               If you're asking me whether that sounds about 

24    right, the answer to your question is yes.

25         Q.    Okay.  Just to re-emphasize something you may 
                                                              834

 1    have touched on in your earlier testimony today, loss 

 2    functions which focus on distributive accuracy are important 

 3    to correspond in evaluating adjustment of the 1990 census; 

 4    isn't that your opinion?

 5         A.    Yes, that is my opinion.

 6         Q.    Now, when you wrote your August 15, 1990 report, 

 7    Dr. Rolph, you believed it was important that the standard 

 8    errors of the adjustment factors be estimated accurately so 

 9    that these factors are not shrunk too much or too little 

10    towards the regression surface, isn't that correct? 

11         A.    Would I be permitted to ask whether that's a 

12    quote from my report?

13         Q.    Well, let me ask whether that's what your belief 

14    was at the time that you wrote your report.

15         A.    That certainly in general terms is.  I'm not sure 

16    if I would have phrased it exactly that way.  That's why I 

17    asked whether it was a quote.

18         Q.    You think it's important that the standard errors 

19    of the adjustment factors should not be underestimated, 

20    isn't that correct?

21         A.    I think it's important that they should be 

22    estimated as accurately as possible.

23         Q.    And that includes that they should not be 

24    underestimated, Dr. Rolph; isn't that correct?

25         A.    Neither underestimated nor overestimated, yes.
                                                              835

 1         Q.    In terms of whether the standard errors of the 

 2    adjustment factors were, in fact, underestimated or 

 3    overestimated, it was your view at the time of your 

 4    deposition last month that the standard errors in the model 

 5    were probably underestimated, isn't that correct?

 6         A.    My best recollection was, is, I should say, that 

 7    at that time there was some evidence on the table suggesting 

 8    that there was some underestimation.  I don't believe --

 9         Q.    Thank you.

10               Isn't it also -- 

11               MR. SHERMAN:  Your Honor, he was in the middle of 

12    the answering the question.  

13               THE COURT:  We're happy.  

14               MR. SHERMAN:  Okay.  I'm happy.  

15         Q.    Isn't it also the case, Dr. Rolph, that at the 

16    time of your deposition last month you believed that the 

17    extent to which the variances in the 1990 PES were 

18    understated was still not known with any certainty?  

19               Yes or no.

20         A.    To my knowledge your statement is correct.

21         Q.    You had not performed any independent studies on 

22    the uncertainty of the variances, including the uncertainty 

23    in the underestimation of the variances in the smoothing 

24    model as of the time of your deposition; isn't that right?

25         A.    I had not carried it out any independent research 
                                                              836

 1    of any kind, so that's correct, yes.

 2         Q.    Thank you.  One of the questions the Special 

 3    Advisory Panel asked you to address in your August 15, 1990 

 4    report was are there projects which you feel should be added 

 5    to provide important data for the decision on adjustment, 

 6    isn't that correct?

 7         A.    That sounds right.

 8               Could you tell me what number my report is?  

 9               That might help my refresh my memory.

10         Q.    Well, I believe your report is -- it's our 

11    Exhibit 503 before the court.   I believe it's 664. 

12         A.    Thank you.

13         Q.    If I could refer you to the appendix page that 

14    appears after page 28 in that report.

15         A.    Thank you very much.

16               I'm sorry.  Is there a question pending?

17         Q.    Yes, I asked you, Dr. Rolph, whether, in fact, 

18    there was a question that went something like, "Are there 

19    projects which you feel should be added to provide important 

20    data for the decision on adjustment?" 

21               Was that is one of the questions on this page?

22         A.    Yes, Item 3 says, "What information, if any, is 

23    missing or unclearly stated in the descriptions of those 

24    projects."

25         Q.    How about Item 7?
                                                              837

 1         A.    In Item 7 is -- yes.  You read it exactly.  Yes, 

 2    it is.

 3         Q.    Thank you.  

 4               Now, Dr. Rolph, you testified earlier that the 

 5    Decennial Census Panel report did not make a formal 

 6    recommendation on the issue of presmoothing or the treatment 

 7    of various outliers; isn't that correct?

 8         A.    That is correct.

 9         Q.    And your report of August 15, 1990, itself does 

10    not mention presmoothing, does it?

11         A.    I believe that's the case, yes.

12         Q.    You didn't discuss in your August 1990 report to 

13    the Special Advisory Panel how the Bureau might treat 

14    variance outliers, did you?

15         A.    No, I don't believe it was part of my charge.

16         Q.    In fact, at the time I took your deposition, you 

17    couldn't recall when you first heard of the issue of 

18    presmoothing variances in the context of the 1990 PES; isn't 

19    that correct?

20         A.    I don't recall saying that, but given my memory 

21    that's sounds probably correct.

22         Q.    In fact, you couldn't tell me at the time of your 

23    deposition whether you first heard of variance smoothing in 

24    the context of the 1990 PES before or after you had 

25    submitted your report to the Special Advisory Panel; isn't 
                                                              838

 1    that correct?

 2         A.    I don't recall saying that, but if you 

 3    represented it to be so, I would certainly agree with you.

 4         Q.    Dr. Rolph, there was nonsampling error in the 

 5    PES, was there not?

 6         A.    I believe that there was some evidence of 

 7    nonsampling error, yes.

 8         Q.    And nonsampling error creates bias in the 

 9    production of dual system estimates, does it not?

10         A.    Yes, that's correct.

11         Q.    Smoothing itself can create bias in the 

12    production of dual system estimates, can it not?

13         A.    If the question is is it possible, yes, it's 

14    certainly possible.

15         Q.    The variance presmoothing could also introduce 

16    bias, could it not?

17         A.    Is this hypothetical, or whether it did on these 

18    trials?  

19               THE COURT:  Hypothetical.

20         A.    Hypothetically certainly it could, yes.

21         Q.    I want to know whether it could have introduced 

22    bias.

23         A.    Okay.  I'll answer the question yes.

24         Q.    To estimate the bias due to presmoothing, that 

25    would be a formidable analytic task; would it not?
                                                              839

 1         A.    It sounds like a direct quote, yes.

 2         Q.    A direct quote from you, Dr. Rolph?

 3         A.    Precisely.

 4         Q.    Thank you.  

 5               Could you calculate the smoothed adjustment 

 6    factor for the poststratum just from the data from that 

 7    poststratum?

 8         A.    No, of course not.

 9         Q.    The smooth adjustment factor for one poststratum 

10    then depends on the raw adjustment factors for all the other 

11    poststrata in the region in the model; correct?

12         A.    Yes.  That was the subject of my direct 

13    testimony.

14         Q.    And as between two poststrata -- let's call them 

15    poststratum 1 and poststratum 2.

16         A.    Okay.

17         Q.    A change in the adjustment factor for poststratum 

18    2 might have a larger effect on the value of the smooth 

19    adjustment factor for poststratum 1 than a change in the 

20    adjustment factor for poststratum 1 itself; isn't that 

21    correct?

22         A.    Oh, yes, certainly.

23         Q.    Well, Dr. Rolph, do you know whether this result 

24    happens a lot in the model for the 1990 PES?

25         A.    I haven't gone through and looked at the 
                                                              840

 1    calculations, but if it didn't happen, I would be surprised.

 2         Q.    But you haven't performed any calculations 

 3    yourself using the Bureau's data to come up with an 

 4    independent guess as to how likely this is, correct?

 5         A.    No, but on general principles you would expect it 

 6    to happen just from what I explained this morning.

 7         Q.    It doesn't trouble you?

 8         A.    No, not at all.

 9         Q.    Dr. Rolph, there were assumptions underlying the 

10    Census Bureau's smoothing model for the 1990 production PES; 

11    isn't that correct?

12         A.    That's correct, yes.

13         Q.    In fact, you testified earlier that is a bundle 

14    of assumptions, correct?

15         A.    Is that from my deposition or my direct 

16    testimony?

17         Q.    No, earlier today when you were quoting from the 

18    Secretary's decision, Doctor?

19         A.    I think that misstates my testimony.  

20               I believe I testified that it was a bundle of 

21    statistical techniques.  I think that was the quote from the 

22    Secretary if I'm recalling correctly.

23         Q.    I will accept that.  

24               One assumption underlying the model is that the 

25    covariance matrix for the sampling error in the raw factors 
                                                              841

 1    can be taken as known; isn't that correct?

 2         A.    I think a more accurate statement would be to say 

 3    that the estimates of the variance of the smoothed 

 4    adjustment factors treated the covariance matrix as known.  

 5    I'm not -- I am just trying to sharpen what you said a 

 6    little bit to be technically correct.

 7         Q.    Thank you.  

 8               Another assumption is that the adjustment factors 

 9    can be modeled as a linear combination of carrier variables 

10    plus an unmeasured error; isn't that correct?

11         A.    Are we are talking about the true adjustment 

12    factors in your question?

13         Q.    Yes.

14         A.    If they provide true adjustment factors, that's 

15    correct, yes.

16         Q.    And another assumption is that the unmeasured 

17    errors on the true adjustment factor around the regression 

18    surface have mean zero, are uncorrelated, and have the same 

19    variables; isn't that correct? 

20         A.    That's one of the assumptions in the Bureau's 

21    underlying model.  That's correct.

22               MR. BARON:  With the court's indulgence, I would 

23    like to look at a chart.  

24               MR. BARON:  I have to get the right chart with 

25    the right technical terms, your Honor.  
                                                              842

 1               This is Defendant's Exhibit 66, a blow-up. 

 2               The page from this document I will identify in a 

 3    moment, your Honor.  

 4               The page from this document is listed in 

 5    Defendant's Exhibit 507, which is Exhibit 5 to the Rolph 

 6    deposition at page 7.

 7               I will be happy to give Mr. Sherman a copy.

 8         Q.    Dr. Rolph, can you see that chart?

 9         A.    Yes, I can.

10         Q.    Do the equations on this chart state the 

11    assumptions of the 1990 production smoothing model, and when 

12    I say the equations, I am pointing to the two equations 

13    which are the the third and fourth line up on the blow-up on 

14    Defendant's Exhibit 66?

15         A.    Yes, that's what they do.

16         Q.    Thank you.  

17               You've stated that you're familiar with the 

18    Undercount Steering Committee report; isn't that correct?

19         A.    Yes, that's correct, I have read it.

20         Q.    The Undercount Steering Committee stated that 

21    there had not been sufficient time to evaluate the 

22    sensitivity of the smoothing model to alternative reasonable 

23    assumptions, which would in general allow for positive 

24    covariances of the elements of W in the model among 

25    poststrata in the same poststratum groups; isn't that 
                                                              843

 1    correct, Dr. Rolph?

 2         A.    If you're asking me whether that's a quote, I 

 3    couldn't tell you whether that's an accurate quote or not.

 4         Q.    But you would agree that the Undercount Steering 

 5    Committee did not have sufficient time to evaluate the 

 6    sensitivity of the smoothing model to these alternative 

 7    reasonable assumptions; isn't that right?

 8         A.    You're asking my recollection of that document.  

 9    My best recollection is twofold.  First, they express 

10    concern along the lines that you suggested, and, secondly, 

11    in their conclusions they said something along the lines but 

12    those concerns were not so serious to cause us to make an 

13    adjustment to use the original enumeration as opposed the 

14    corrected counts.

15         Q.    You would agree, you would not, Dr. Rolphm that 

16    within the population of models that have positive 

17    covariances there are certainly some which you would view as 

18    alternative reasonable assumptions to the Bureau's model 

19    that they in fact used; isn't that correct?

20         A.    Certainly, yes, absolutely.

21         Q.    But at the time of your deposition you hadn't 

22    performed any independent research on whether any of these 

23    -- on the reasonableness of any alternative assumption in 

24    the Bureau's model; isn't that right?

25         A.    Yes, and I still have not performed any research 
                                                              844

 1    on that subject.

 2         Q.    Would you agree with the Undercount Steering 

 3    Committee where it has stated that, "Indeed, a more 

 4    reasonable assumption would be that errors are correlated 

 5    among most age groups within each poststratum group?"

 6         A.    That certainly seems quite plausible to me. 

 7               Indeed, my recollection is that in the 

 8    documentation leading up to this decision there was some 

 9    discussion of that subject in some of the underlying 

10    documents.

11         Q.    In your view, Dr. Rolph, variance presmoothing 

12    was prespecified; is that correct?

13         A.    Yes, I believe it was.

14         Q.    So there came a point in time when it was 

15    prespecified; isn't that right?

16               I would like to know when that was, Dr. Rolph. 

17               When was variance presmoothing prespecified by 

18    the Bureau?

19         A.    I need to ask a clarifying question.  

20               Are you asking whether the idea of modeling 

21    variances as an idea, when that was specified, or are you 

22    asking when the particular functional form that the Bureau 

23    used to model the variances to presmooth was specified?  The 

24    terms you're using --

25         Q.    I want to know the particular functional form 
                                                              845

 1    which the Bureau, in fact, used for the 1990 PES, when was 

 2    that prespecified.

 3               Do you know?

 4         A.    I don't know off the top of my head the answer to 

 5    that question, no.  Indeed, let me give you a fuller answer.  

 6    I don't believe that the particular functional form was 

 7    prespecified.  

 8               I'm not going to go any further because my memory 

 9    is not my strongest point.

10         Q.    Let's turn to Defendant's Exhibit 507, the court 

11    has that and I did a blowup of the chart but I don't believe 

12    I handed the witness this document.  

13               MR. BARON:  May I approach, your Honor.  

14               THE COURT:  Yes.

15         Q.    Would you take a look at page 6 of this report, 

16    of this document?

17         A.    Okay, I have it open to page 6.

18         Q.    And the court has it as Defendant's Exhibit 507, 

19    your Honor.  

20               At the top of page 6 do you see the equation 

21    there, Dr. Rolph?

22         A.    Yes, I do.

23         Q.    This is the equation that the Census Bureau used 

24    to presmooth variances; is that correct?

25         A.    No, that's not correct.
                                                              846

 1         Q.    Oh, this isn't the equation the Census Bureau 

 2    used to presmooth variances?

 3         A.    That is correct.

 4         Q.    You mean they used some other equation; correct?

 5         A.    Yes, that's correct.

 6         Q.    Okay.  Do you see the next paragraph starting 

 7    with, "We intend to fit this model, or one similar to it, 

 8    examine the fit and refit the model if necessary"?

 9         A.    Yes, I see that sentence.

10         Q.    In other words, Dr. Rolph, if the Bureau's first 

11    choice of models didn't work, they would reject it and try 

12    April try again; isn't that right?

13         A.    That's not my characterization, no.

14         Q.    Refitting a model is rejecting the model; isn't 

15    that right?

16         A.    I don't think those are synonyms, no.

17         Q.    Well, let me ask you this.  

18               This document is undated, however, at the time 

19    this document was written, wouldn't you say that the 

20    variance presmoothing model was not prespecified?

21         A.    As I indicated earlier, my best recollection is 

22    that it was prespecified, that they were going to model or 

23    presmooth the variances.  The exact functional form was not 

24    prespecified.  That was my quote.  

25               I think that's responsive to your question, at 
                                                              847

 1    least I hope it is.

 2         Q.    Dr. Rolph, you know what a jackknife is, don't 

 3    you?

 4         A.    For camping trips or in statistical jargon.

 5         Q.    Statistical purposes.

 6         A.    Yes, I'm familiar with the term.

 7         Q.    You're aware, aren't you, that the jackknife is 

 8    often used to estimate variances from survey data?

 9         A.    Yes, I am aware of that fact.

10         Q.    You are aware, are you not, that the Bureau used 

11    the jackknife to estimate the raw variance-covariance matrix 

12    for the raw adjustment factors?

13         A.    That's my understanding, yes.

14         Q.    You're not aware of any other examples where 

15    variances have been estimated from survey data by the 

16    jackknife and then through the estimates were presmoothed by 

17    a regression, are you?

18         A.    No example comes to mind one way or the other.

19         Q.    Thank you.  With respect to the treatment of 

20    outliers which is mentioned in the Secretary's discussion of 

21    Guideline Three, is it your view that the treatment of 

22    outliers was prespecified?

23         A.    Again, I believe it was prespecified that the 

24    Bureau would attempt to detect any outliers that occurred 

25    and then to treat them appropriately.  It was not 
                                                              848

 1    prespecified exactly what the details of that treatment 

 2    would be.

 3         Q.    The exact details were not prespecified?

 4         A.    That's my understanding.

 5         Q.    In fact, the exact method for treating outliers 

 6    was not prespecified; is that correct?

 7         A.    That's the substance of what I was just trying to 

 8    say.

 9         Q.    You do believe that it entailed a judgmental 

10    decision on the part of the Bureau; correct?

11         A.    As to specifically the exact method, yes.

12         Q.    There's no standard definition in statistics of 

13    what constitutes the exact line or threshold which divides 

14    points considered outliers from points considered 

15    nonoutliers; isn't that right?

16         A.    I believe that would be a slight overstatement.  

17    I would accept that there is not a precise numerical 

18    threshold that everybody agrees to.  I would characterize 

19    the situation -- there are general principles that one uses 

20    to define outliers, and there's slight variations in how 

21    people adopt those principles.

22         Q.    In fact, the definition of an outlier depends in 

23    part on the statistical model being used, isn't is that so?

24         A.    Yes, indeed in the census case there were two 

25    different definitions in the variance modeling and in the 
                                                              849

 1    adjustment factors.  Because they were different models and 

 2    different definitions, outliers were appropriate in the two 

 3    different models.

 4         Q.    Do you agree that with respect to the Census 

 5    Bureau's presmoothing methodology that there was no 

 6    prespecified procedure for exactly how to modify the outlier 

 7    points since there is no uniformly accepted practice within 

 8    the statistical community?

 9         A.    I accept the first part of your statement.  I'm 

10    not sure I can independently verify that it follows directly 

11    from the second part. 

12         Q.    Well, Dr. Rolph, let me read you some questions 

13    and answers from our deposition of less than a month ago.  

14    Page 342:

15               "Q.    No.  I think that would be fine.  We can 

16    move on.

17               "Let me turn your attention to Exhibit 7, which 

18    is the Undercount Steering Committee report.  If you would 

19    turn to page 9 -- we did not mark --

20               THE COURT:  Are we still in English?  

21               MR. BARON:  Yes.  These are just some 

22    introductory remarks, your Honor.  

23               Deposition Exhibit No. 11 was marked.

24               "Q.    The particular sentence that I would ask 

25    that you focus on is in the paragraph -- 
                                                              850

 1         A.    Mr. Baron, could you slow down a little.  I am 

 2    having trouble following you.

 3         Q.    I'll say it again.

 4               "Q.    The particular sentence that I would ask 

 5    that you focus on is in the paragraph which is labeled 

 6    'prespecification' underlined.  The fourth sentence of which 

 7    states, 'There was no prespecified procedure for exactly how 

 8    to modify the outlier points since there is no uniformly 

 9    accepted practice within the statistical community,'

10                   "Do you see that sentence?

11               "A.    Yes, I do.  I am reading it the complete 

12    paragraph.

13               "Q.    Please do.  Please feel free to read 

14    whatever you wish.

15               "A.    Okay I've read it.

16               "Q.    Let me ask you a preliminary question.  Do 

17    you believe the members of the Undercount Steering Committee 

18    exercised their best professional judgment in compiling 

19    their recommendation to the Secretary of 

20    Commerce --"

21         A.    Could you slow down, please.  I'm having 

22    difficulty following you.

23               MR. SHERMAN:  Your Honor.

24               MR. BARON:  I'm getting to the punch line, your 

25    Honor.  Let me do it again.
                                                              851

 1               "Q.    Let me ask you a preliminary question.  Do 

 2    you believe the members of the Undercount Steering Committee 

 3    exercised their best professional judgment in compiling 

 4    their recommendation to the Secretary of Commerce?

 5               "A.    My reading of the record would be that 

 6    they did, as best I can tell.

 7               "Q.    With respect to the sentence we are 

 8    focusing on, do you agree or disagree with the statement 

 9    expressed in the sentence?

10               "A.    I would agree with it."  

11               MR. SHERMAN:  Your Honor, I'm not sure what 

12    counsel has just shown, but -- 

13               THE COURT:  What is the question "Did you say 

14    that"?   

15               MR. BARON:  I'm asking the witness whether he 

16    agrees or disagrees with the statement that there was no 

17    prespecified procedure for exactly how to modify the outlier 

18    points since there is no uniformly accepted practice within 

19    the statistical community.  

20               THE COURT:  The question is does he agree with 

21    that statement?  

22               MR. BARON:  Yes.  That's why I read it.

23         A.    I will say generally I agree with it.  I was a 

24    little shaky as to the logic of the sentence, but, yes, I 

25    generally agree with that statement.
                                                              852

 1         Q.    Thank you.  Now, Dr. Rolph, you do agree, do you 

 2    not, that omitting 28 outlier variances from the variance 

 3    presmoothing model results in a different apportionment of 

 4    the United States House of Representatives; isn't that 

 5    correct?

 6         A.    From my independent knowledge?

 7         Q.    Do you know whether that does.

 8         A.    I recollect something, I think, in the 

 9    Secretary's report if I'm recalling correctly that bore on 

10    that point.  I can't recall the details that you've just 

11    enunciated, but if that's what it said, I would certainly 

12    agree with it.

13         Q.    Would it help your recollection if I showed you 

14    the Secretary's --

15         A.    I'm not doubting you.  I'm just saying I can't 

16    recall those exact words.  

17         Q.    You don't have any reason to dispute the 

18    Secretary's statement in the Secretary's document that the 

19    -- that omitting 28 outliers variances results in a shift of 

20    the house seat?

21         A.    I have no reason to dispute it, no.  I have not 

22    done any independent analysis.

23         Q.    You are aware, are you not, that the adjustment 

24    based on the production Dual System Estimator would shift 

25    two seats in the House of Representatives?
                                                              853

 1         A.    That's from -- as compared to using the 

 2    uncorrected counts; is that your question?

 3         Q.    No, that's with the adjustment methodology that 

 4    was the proposed alternative.

 5         A.    The two seats, the differences between the 

 6    adjustment and the nonadjustment, is that your question?

 7         Q.    Yes. 

 8         A.    That's my understanding, yes.

 9         Q.    So, in other words, the small changes in 

10    methodology involving how many variance outliers to include 

11    or exclude from the smoothing model results in moving seats 

12    in the U.S. House of Representatives; isn't that right?

13         A.    I'm sorry.  Your last question was comparing 

14    adjustment and nonadjustment.  Your question right now is 

15    about details of the adjustment model.  So I am not sure --

16         Q.    I thought you did not dispute, Dr. Rolph, that 

17    omitting 28 variance outliers would shift one seat, and that 

18    the proposed adjustment methodology shifts two seats, so 

19    there's a difference in one seat between -- including or 

20    excluding 28 variance outliers in the smoothing model; isn't 

21    that right?

22         A.    That's correct.  I agree with you there.  Your 

23    question, could you read -- could somebody read it back so I 

24    can get it straight in my mind?

25         Q.    I will let the reporter do so.  
                                                              854

 1               (Record read)

 2         A.    I was referring to, I thought the earlier 

 3    question that's pending has to do with the characterization 

 4    of omitting 28 variance outliers being a -- I forget your 

 5    term -- it was some term of art, statistical detail or 

 6    something like that.

 7               Is that the substance of your question of your 

 8    question?

 9         Q.    I'm willing to go on, Doctor?

10         A.    Okay.

11         Q.    Let me ask you this.  

12               The Undercount Steering Committee stated that, I 

13    quote, this is page 11 of the Undercount Steering Committee 

14    report, "When considering the outlier variance problem, the 

15    Committee noticed major differences in estimating smooth 

16    variances between eliminating four outliers and eliminating 

17    seven outliers.  The extent of differences for making a 

18    relatively minor change in assumptions is disturbing.?

19               Are you disturbed by the minor change in 

20    assumptions, Dr. Rolph, in the same way as the Undercount 

21    Steering Committee apparently was?

22         A.    I think that has to be put in a broader context.  

23    There were, in addition to the two possibilities you 

24    referred to just now, there was an additional analysis 

25    carried out by the Bureau looking at yet other ways of 
                                                              855

 1    handle those outliers; specifically, using the method called 

 2    "robust regression," which, instead of eliminating the 

 3    outliers, basically downweights them.  

 4               If I'm recalling correctly, I think Bill Bell, 

 5    his name comes to mind as having carried that out.  So as I 

 6    understand the situation, the Undercount Steering Committee 

 7    had on the table in front of it several different treatments 

 8    of outliers.  They had an affirmative analysis of those 

 9    different treatments, and they then exercised what seemed to 

10    me sound reasonable and sound statistical judgment in coming 

11    um with a sensible way of doing it.

12               Was that responsive to your question?

13         Q.    Sure.

14               Dr. Rolph, is it true that outlier points 

15    sometimes can be the result of simple errors in calculation 

16    or transcription?

17         A.    Yes.

18         Q.    Isn't it also true that outlier points can be 

19    valid points in need of further investigation?

20         A.    That's also correct.

21         Q.    In fact, isn't it the case that outliers are 

22    sometimes the most important cases for a study or analysis?

23         A.    Sounds like a quote of Professor Tukey, yes.

24         Q.    Would you agree as a general proposition that the 

25    very fact that outliers are by definition unlike other 
                                                              856

 1    points sometimes signal something unusual is going on that 

 2    is worthy of further investigation?

 3         A.    Yes, I would.

 4         Q.    Let me give you a hypothetical example, Dr. 

 5    Rolph.

 6               Suppose you have 17 observations in a particular 

 7    study of the relationship between two variables and one of 

 8    the observations -- let me call it Observation 12.

 9         A.    Okay.

10         Q.    Constitutes an outlier case.  

11               Let us further suppose that Observation 12 is the 

12    only outlier.  

13               Let us further suppose that including or 

14    excluding Observation 12 significantly changes the standard 

15    deviation of the residuals, of the points, of the 17 points.

16         A.    I have that in mind.

17         Q.    You can further suppose that the standard 

18    deviation with Observation 12 is about three times as large 

19    as the standard deviation without Observation 12.  

20               Are you with me?

21         A.    I'm with you.

22         Q.    Now, Dr. Rolph, let us assume one more thing:  

23    That due to time constraints or budget resources or some 

24    other reason, you are not in a position to determine whether 

25    the outlier is an erroneous point or whether it is a valid 
                                                              857

 1    point which is signaling that something is going on worthy 

 2    of further investigation.  

 3               Then let me ask you, wouldn't you agree that if 

 4    you were not in a position to investigate the validity of 

 5    Observation 12, you would probably want to report the 

 6    results both with and without including Observation 12; 

 7    isn't is that correct, Dr. Rolph?  

 8               Yes or no.

 9               THE COURT:  And be brief.

10               MR. BARON:   It took me a long time to think of 

11    this hypothetical, your Honor. 

12         A.    I find it difficult to give a yes or no answer to 

13    that question.  

14               THE COURT:  Do you understand the question?  

15               THE WITNESS:  I understand the question 

16    perfectly. 

17               THE COURT:  Answer it any way you can.

18         A.    Okay.  In that situation I would report, I would 

19    report the regression without Observation 12.  I would also 

20    report -- I would consider reporting a regression that was 

21    fit with Observation 12 using a robust fitting technique, 

22    and I might as a sensitivity analysis also report a 

23    regression with Observation 12 with a clear flag noting that 

24    it increased the standard deviation by a factor of three.  

25    So in all likelihood was not part of the population being 
                                                              858

 1    fit, and hence violated the assumptions for a standard 

 2    regression model.  

 3               I apologize for the longer answer.

 4         Q.    Let me hand you a document, Dr. Rolph.  

 5               It is marked as Defendant's Exhibit 516 in your 

 6    book, your Honor.  

 7               MR. BARON:  May I approach.  

 8               THE COURT:  Yes.

 9         Q.    This copy is unstapled.

10         A.    Thank you.

11         Q.    Do you recognize this document, Dr. Rolph?

12         A.    Yes, I do.

13         Q.    What is it?

14         A.    It is a Rand note authored by me entitled "Using 

15    Statistical Tools," dated April 1984.  

16               THE COURT:  For fun and profit.

17         Q.    Let me ask you whether you recall the example in 

18    this document of data collected in the 1840s and 1850s by a 

19    Scot named James Forbes?

20         A.    Yes, I do recall that as an example that I used 

21    in this document which was for teaching a short course at 

22    Rand.

23         Q.    Didn't you identify as a hypothetical example of 

24    an outlier a certain case twelve out of 17 observations?

25         A.    Could you point me to the page we're referring 
                                                              859

 1    to?

 2         Q.    Sure.  If you would look at page 26 in the 

 3    document.

 4         A.    Okay.  I'm with you.

 5         Q.    You will see there are 17 observations in that 

 6    diagram, and there is a case 12 that looks like an outlier 

 7    point; isn't that right?

 8         A.    It wasn't so entirely hypothetical, yes.

 9         Q.    What I should have said is that it took me a long 

10    time researching your CV to come up with this hypothetical.

11               Could you look at page 29 of the document.

12               In the paragraph that starts, "The real question 

13    about case 12," would you look at the last sentence of that 

14    paragraph, and tell me, or read to the court what you said 

15    in that sentence starting with "since"?

16         A.    Yes, I wrote, "Since we are not in a position to 

17    investigate the validity of Case 12, we would probably want 

18    to report both of these equations."

19         Q.    Thank you.

20               Dr. Rolph, would you also look at the preface, 

21    the acknowledgements page of this document which is vii.  

22         A.    Yes, I see that.

23         Q.    Do you credit Friedman, Pasani and Pervis in the 

24    second paragraph of this document?

25         A.    Yes, I do.  In fact, virtually all the data in 
                                                              860

 1    this document are drawn from published examples as stated 

 2    there.

 3         Q.    Let me ask you this, Dr. Rolph.

 4               Do you believe that statisticians of solid 

 5    reputation can reasonably disagree over how to interpret the 

 6    details of the results of the 1990 PES?

 7         A.    How to interpret the details was the question?

 8         Q.    Yes.

 9         A.    As a general question that sounds acceptable to 

10    me.

11         Q.    Excuse me.  I didn't hear that.

12         A.    I said as a general question that sounds 

13    acceptable to me.

14         Q.    One more question, Dr. Rolph, you agree, do you 

15    not, that the ultimate issue as to whether there should be 

16    an adjustment of the 1990 census is for policymakers, not 

17    statisticians to decide; isn't that correct?

18         A.    By law as I understand it, the Secretary of 

19    Commerce is to make that decision, and he is a policymaker, 

20    not a statistician the last time I looked.

21         Q.    So you would agree?

22         A.    Yes.

23               MR. BARON:  Thank you, Dr. Rolph.  

24               THE COURT:  Thank you, Mr. Baron.  

25               Any redirect Mr. Sherman?  
                                                              861

 1               MR. SHERMAN:  I think it's limited to one 

 2    question, your Honor 

 3    REDIRECT EXAMINATION
                          

 4    BY MR. SHERMAN:  

 5         Q.   Dr. Rolph, in your answer to Mr. Baron's 

 6    hypothetical you suggested that you would, in addition to 

 7    reporting the two observations, do something called a robust 

 8    technique; is that correct?

 9         A.    Robust regression, that's correct, yes.

10         Q.    Do you know whether anyone at the Census Bureau 

11    did any of that robust sort of technique in testing the 

12    smoothing model used in the 1990 PES?

13         A.    Yes.  I referred, I think, in one of my answers 

14    to Mr. Baron to some work done by Dr. Bell, Bill Bell, at 

15    the Census Bureau which was fitting robust regressions to 

16    the variances when considering this issue of outliers, yes. 

17               So yes, I did.  

18               MR. SHERMAN:  That is all I have, your Honor.  

19               THE COURT:  Thank you Dr. Rolph.  We've been most 

20    enlightened.  

21               THE WITNESS:  It's been a pleasure.   

22               (The witness was excused) 

23               MR. ZIMROTH:  Your Honor, we have originally 

24    believed that Mr. Baron was going to be cross-examining this 

25    witness through the lunch break, so we do have another 
                                                              862

 1    witness but he is in another part of the city.

 2               Is it possible for us to take an early break now 

 3    and come back a little early?  

 4               THE COURT:  How early?  

 5               MR. ZIMROTH:  I would say one o'clock.  

 6               THE COURT:  All right.  

 7               (Luncheon recess)

 8    

 9    

10    

11    

12    

13    

14    

15    

16    

17    

18    

19    

20    

21    

22    

23    

24    

25    
                                                              863

 1               A F T E R N O O N     S E S S I O N.

 2                                      1:00 o'clock p.m. 

 3               THE COURT:  Good afternoon.

 4               Mr. Zimroth?  

 5               MR. ZIMROTH:  The plaintiffs' next witness is 

 6    Leobardo Estrada.

 7               With your Honor's permission, my colleague, Paula 

 8    Desel, will examine the witness.  (D E S E L)

 9    LEOBARDO F. ESTRADA,  

10             called as a witness by the plaintiff, having

11             first been duly sworn, was examined and

12             testified as follows: 

13    DIRECT EXAMINATION 

14    BY MS. DESEL:

15         Q.    Dr. Estrada, what do you do for a living?

16         A.    I'm an associate professor in the Graduate School 

17    of Architecture and Urban Planning at the University of 

18    California at Los Angeles.

19         Q.    How long have you been at the University of 

20    California in Los Angeles?

21         A.    Since 1977; aboot 15 years.

22         Q.    Could you tell the court your educational 

23    background, please?

24         A.    I have a bachelor's degree in sociology from 

25    Baylor University and my masters and doctorate are from the 
                                                              864

 1    sociology department at the Florida State University, 

 2    Tallahassee, Florida.

 3         Q.    What did you do after you received your doctorate 

 4    from Florida State?

 5         A.    My first faculty appointment was at North Texas 

 6    State University in Denton, Texas.

 7         Q.    And how long were you there?

 8         A.    Five years.

 9         Q.    Where did you go after you left North Texas 

10    State?

11         A.    From 1975 to 1977 I was at the Census Bureau.

12         Q.    What were you doing at the Census Bureau?

13         A.    I served as a special assistant to the chief of 

14    the population division, his name was Meyer Zitter.

15         Q.    Can you tell the court what the population 

16    division of the Census Bureau is?

17         A.    The population division at the Bureau of the 

18    Census is the part of it that is responsible for most of the 

19    reports and analysis of characteristics of the population, 

20    such as income statistic, family statistics, and among the 

21    other things they do, they also have the estimates and 

22    projections branch as well as the group that does the 

23    demographic analysis.

24         Q.    What did you do as special assistant to the 

25    population division chief?
                                                              865

 1         A.    I arrived at the Census Bureau during the time 

 2    that they were undertaking their pretesting for the 1980 

 3    census.

 4               I was assigned to oversee the public information 

 5    program that was directed to the Spanish language population 

 6    and was involved with some outreach activities early in the 

 7    time I was there.

 8               I'm sorry, I confused something here.

 9               The first time I was at the Census Bureau I was 

10    there during the pretesting period and I spent my time 

11    looking at the race and ethnicity items that we were 

12    proposing to use in the 1980 census, and as part of that 

13    process my job was to suggest some different studies that 

14    should be undertaken.  

15               We analyzed the results and we tried to come to 

16    some decision about what we should do for the 1980 census in 

17    terms of asking the appropriate questions.

18         Q.    You were at the Census Bureau a second time 

19    besides the job you just described?

20         A.    Yes, I was.

21         Q.    How did that come about?

22         A.    In 1977 I went to my position at UCLA, and after 

23    being there about a year and a half I received a call from 

24    the then director of the Bureau of the Census, Vin Barraba, 

25    asking if I would return to the Census Bureau and assist in 
                                                              866

 1    the management of the 1980 census.

 2         Q.    And did you return?

 3         A.    Yes, I did.  I served as a staff assistant to the 

 4    deputy director of the Bureau of the Census.

 5         Q.    Can you tell the court what you did during your 

 6    second stint at the Census Bureau?

 7         A.    I was there during the period of time when the 

 8    1980 census was undertaken, slightly before and during the 

 9    period.

10               My duties sort of sifted in the early part in 

11    terms of assisting the Bureau in monitoring the outreach 

12    program for the Bureau, particularly the part oriented 

13    toward the Hispanic population.

14               As we moved into the actual implementation of 

15    taking the census, I served more of a role of 

16    troubleshooting many of the little brush fires that arose in 

17    taking the 1980 census.

18         Q.    Can you explain what you mean by the outreach 

19    program during the 1980 census?

20         A.    The Census Bureau has a strong emphasis upon 

21    trying to encourage, encourage participation in the census 

22    and the outreach program is one of the more important 

23    elements of trying to reach out to hard to enumerate 

24    populations.  In this case their concern was how to reach 

25    their Spanish speaking population and I was asked to monitor 
                                                              867

 1    the process and coordinate it.

 2         Q.    How did you go about identifying and finding 

 3    these hard to enumerate populations?

 4         A.    The primary mechanism that was used to identify 

 5    these areas was to look at mailback rates from the prior 

 6    census of 1970, identify geographic areas in which mailback 

 7    rates had been low.  That overlaid on top of Hispanic 

 8    population gave us our original target area that we were 

 9    going to work with.

10               In addition to that, as we began to move forward, 

11    of course, we dealt with other issues as well.

12         Q.    Once you identified these geographic areas, what 

13    kind of problems did you learn were occurring regarding the 

14    outreach and the 1980 census?

15         A.    After we identified target areas, we typically 

16    attempted to visit them to get a sense about what we might 

17    be able to accomplish and do.  

18               We had a series of activities that we could try 

19    to bring forth.  And I think just in general when we would 

20    identify a language area, we also, of course, ran into 

21    issues about immigration, because they were very closely 

22    related, immigration and Spanish language use, which led us 

23    into having to deal with the enumeration of hard to 

24    enumerate groups, like the undocumented aliens.

25               We ran cross issues that had to do with irregular 
                                                              868

 1    housing patterns and other things which we knew would be 

 2    problematic.

 3               Our job was to do what we could to reach out and 

 4    assist people and encourage them to take part in the census 

 5    regardless of all of these flaws that we saw.

 6         Q.    The second aspect of your job that you mentioned 

 7    was troubleshooting during the taking of the 1980 census.

 8               Can you give the court some examples of the kind 

 9    of troubleshooting that you did during that time?

10         A.    The census is a very complicated process and 

11    obviously lots of issues arise in the process of taking the 

12    census.

13               Because of my background, I was asked to monitor 

14    events going on in the five southwestern states, and in 

15    addition to that keep an eye on the offices in Chicago, New 

16    York and Miami.

17               In the process, issues would arise.  Illustration 

18    might be something like the fact that we would discover that 

19    one of our offices in Hollywood was ill prepared to deal 

20    with bilingual enumerators.  We had not allowed for that 

21    area to have any, and yet we discovered a large pocket of 

22    Spanish speaking population in that area that had risen over 

23    the decade.

24               In San Antonio we had problems that had to do 

25    with day laborers and the need to think about what to do 
                                                              869

 1    with enumerating people that might or might not be connected 

 2    to addresses.

 3               In New York City we dealt with the issue of 

 4    duplicate forms.  We had groups who wanted to be counted and 

 5    were asking for forms and claimed not to have received them.  

 6    We had to deal with those kinds of issues as well.  And 

 7    there are a multitude of others that I could list.

 8         Q.    When did you leave the Census Bureau this second 

 9    time?

10         A.    I remained at the Census Bureau until December, 

11    the end of December 1980.

12         Q.    And have you continued to advise the Census 

13    Bureau since that time on census related issues?

14         A.    Yes, I have.

15         Q.    Can you give us some examples of the kind of 

16    advice or the kind of capacity that you served for the 

17    Census Bureau?

18         A.    I've been advising the Census Bureau for about 20 

19    years, at first in an informal way, before I went to work 

20    there in meetings that I had at the Census Bureau with 

21    various individuals.  Eventually I was appointed to the 

22    populations statistics advisory committee and I served in 

23    that capacity for two years.

24               And then prior to the 1980 census, there was a 

25    Hispanic advisory committee to the Census Bureau which I 
                                                              870

 1    served and also chaired.

 2               I would leave the committee when I worked for the 

 3    census and when I left the census I went back on to the 

 4    committee.

 5               I now serve on the Hispanic advisory committee to 

 6    the 1990 census, and I also chaired that committee and now 

 7    just serve as a member.

 8               There is a lot of contact between the Census 

 9    Bureau and myself.  Most recently a week ago I was asked 

10    could they use some table outlines for some new reports that 

11    is coming out, and this is very typical of the type of 

12    activities I carried out for the last 20 years.

13         Q.    Besides the work that you described, just 

14    described at the Census Bureau, what kinds of work have you 

15    done in the areas of your expertise? 

16               Perhaps you could tell the court what those areas 

17    are.

18         A.    I teach --  the courses that I teach describe it 

19    to some extent.  I teach courses on social policy analysis, 

20    I teach courses on demographic methods and I also teach 

21    courses on planning for minority communities.

22               The ideas behind my work are to apply my craft of 

23    demography to different issues in urban areas, in 

24    particular, in terms of things such as intercity 

25    redevelopment issues, land use, transportation, economic 
                                                              871

 1    development, issues that arise in urban areas.

 2         Q.    Has that work led you into any other fields 

 3    besides strictly urban redevelopment?

 4         A.    Well, the basis of my expertise is in the area of 

 5    urban studies, but I've been asked often, because of that 

 6    knowledge, to serve on various advisory boards and 

 7    committees.

 8               Recently I served as a member of the independent 

 9    panel to investigate the Los Angeles police department.

10         Q.    Can you explain to the court what that panel did?

11         A.    It's a panel often referred to as the Christopher 

12    Commission, and our job was to investigate the Los Angeles 

13    police department in terms of the Rodney King incident and 

14    investigate the policies related to use of force and other 

15    issues related to the misuse, excessive use of force by the 

16    police department.

17         Q.    Is the work of that commission completed?

18         A.    The initial report was submitted in July of 1991.  

19    We have since remained together.  We decided to stay 

20    together as a group for a year and to submit reports every 

21    six months to the public to indicate what --  how our 

22    recommendations had been adopted and to what extent they 

23    have been implemented by the the police department.

24         Q.    And, Dr. Estrada, beginning in 1989, you also 

25    served as a member of the special advisory panel to 
                                                              872

 1    Secretary Mosbacher on the issue of the census adjustment 

 2    for 1990, is that right?

 3         A.    That's correct.

 4         Q.    Dr. Estrada, could you turn to Plaintiff's 

 5    Exhibit 194, which I believe is in volume three of your 

 6    exhibits.

 7               Can you tell the court what this document is?

 8         A.    This is the most current version I have in my 

 9    curriculum vitae.

10         Q.    Does this describe in greater detail than you 

11    have given here your work, your qualifications, your 

12    memberships on other committees and commissions?

13         A.    Yes, it does.  

14               MS. DESEL:  At this time I would like to offer 

15    Plaintiff's Exhibit 194 in evidence in this case.  

16               MR. BARON:  No objection.  

17               THE COURT:  194 is received.

18               (Plaintiff's Exhibit 194 marked for 

19    identification was received in evidence.) 

20               MS. DESEL:  I also at this time would like to 

21    proffer Dr. Estrada as an expert.  

22               MR. BARON:  Your Honor, it is not crystal clear 

23    what Professor Estrada is claiming to have expertise in.

24               Would counsel like to summarize in a word?  

25               THE COURT:  Demography, planning for --  
                                                              873

 1               MR. BARON:  But not statistics?  

 2               MS. DESEL:  Not specifically for statistics, 

 3    except he was on the advisory panel so we do anticipate to 

 4    introduce his work regarding census adjustment.  

 5               MR. BARON:  May I be permitted to ask the witness 

 6    a question?  

 7               THE COURT:  On?  

 8               MR. BARON:  Ask one question whether he considers 

 9    himself to be a statistician.  

10               MS. DESEL:  Your Honor, the stipulation 

11    appointing the special advisory panel included a provision 

12    that all members of it were considered experts, but if Mr. 

13    Baron --  

14               MR. BARON:  I have no objection to Dr. Estrada 

15    being qualified as an expert in fields other than 

16    statistics.  

17               THE COURT:  How do you feel about statistics? 

18               THE WITNESS:  Well, I have a general knowledge of 

19    statistics.  Like most people who work in demography, we 

20    taught courses on statistics.

21               There is different branches of it, mathematical 

22    statistics, informal statistics and so forth.  I considered 

23    myself to be applied in my work, I have basic general 

24    knowledge of statistical principles and would not regard 

25    myself as a mathematical statistician, but --  
                                                              874

 1               THE COURT:  Have you ever taught statistics? 

 2               THE WITNESS:  Yes, I have 

 3               THE COURT:  Is basic or what? 

 4               THE WITNESS:  Basic and research methods.  

 5               THE COURT:  Do you have any advanced degrees in 

 6    statistics? 

 7               THE WITNESS:  No, I do not.  

 8               THE COURT:  I will take his testimony on 

 9    demography, social planning and planning for minorities and 

10    to a limited extent on statistics, depending on what is 

11    being asked.  

12               MR. BARON:  Thank you.  

13    BY MS. DESEL:

14         Q.    Dr. Estrada, are you being paid by the plaintiffs 

15    in this case for your work?

16         A.    No, I'm not.

17         Q.    Do you have any expectation or understanding that 

18    you will be paid for your work on this case?

19         A.    No, I do not.

20         Q.    I want to turn now to the special advisory panel.

21               What was your recommendation to the Secretary 

22    Mosbacher?

23         A.    My recommendation to Secretary Mosbacher was that 

24    the dual system estimators that were derived from the 

25    post-enumeration survey be used to compliment the original 
                                                              875

 1    enumeration for the purposes of having more accurate 1990 

 2    census.

 3         Q.    Dr. Estrada, could you please turn to Plaintiff's 

 4    Exhibit 193, I believe it is in the same binder, and 

 5    identify that document for the court?

 6         A.    The document before me is the recommendation that 

 7    I wrote to Secretary Mosbacher.

 8         Q.    Could you explain to the court the structure of 

 9    that report or recommendation?

10         A.    We had each received a letter asking --  from 

11    Secretary Mosbacher asking that we make a recommendation to 

12    him and to respond to each of the guidelines that had been 

13    published in the Federal Register.

14               I prefaced my remarks in regard to those 

15    individual guidelines where the rationale which I felt 

16    described the way --  the foundation for my recommendation.

17               I began my recommendation, first of all, with 

18    this preface indicating that I believe that the 1990 census 

19    was flawed and required adjustment.  I drew that conclusion 

20    by comparing the results of the 1990 census to the 

21    demographic analysis in terms of historical patterns that 

22    appeared in undercount as well as differential undercounts 

23    between groups.

24               I went on to note that I knew that the census was 

25    a difficult undertaking, I was drawing upon the experience 
                                                              876

 1    that I had had in 1980, and yet there were many things that 

 2    I had seen problematic then which continued to be problems.

 3               I began to list some of those issues beginning 

 4    point A on page 4 of my introduction and I go on with two or 

 5    three pages of what I regard as being serious problems that 

 6    had occurred in 1990, and I finish on page 6 listing some 

 7    other things that I did not detail but which add on to the 

 8    many problems.

 9               I concluded in the section that the census as a 

10    method has probably reached its limitations of what it can 

11    do, and I drew that because I know that the effort in 1990 

12    was extensive and yet the results indicate that there was 

13    little improvement in differential undercount.

14               The second part of my introduction is oriented 

15    towards my conclusions regarding the process of the 

16    post-enumeration survey, and I indicate that my feeling, my 

17    conclusions from having observed this process, and we 

18    obtained weekly reports and monthly reports and we met 

19    monthly to receive information regarding its progress, was 

20    that it was a very high quality undertaking, that the 

21    results of it are something that are reliable and that we 

22    can utilize.

23               I have reasons for it that are listed that begin 

24    on page 7 of that preface and continue over to page 8.

25               The third part of my rationale is based on the 
                                                              877

 1    idea that the estimates that are derived from the PES are, 

 2    in fact, consistent with historical patterns from the 

 3    demographic analysis since we have been able to monitor 

 4    census performance and that the patterns that I see in race, 

 5    sex and age differential are consistent, sufficiently 

 6    consistent to draw the conclusion that the PES can be used 

 7    and does, in fact, measure the kinds of issues and problems 

 8    of undercounting that have plagued the census for many 

 9    years.

10               The fourth part of the rationale is built upon 

11    work that was conducted for the panel by panel consultants 

12    related to the robustness of the dual system estimators.

13               I conclude that the evidence that has been put 

14    before me and the information that I received allows me to 

15    conclude that the PES estimates can be used reliably and in 

16    terms of their abilities to give us reliable estimates of 

17    undercount.

18               The final part of my rationale is simply based on 

19    the conclusion which I also drew upon other materials to 

20    arrive at that the dual system estimators can prove the 

21    accuracy of the census at lower levels of geography.  I 

22    refer to John Tukey's work and other work that is performed 

23    for us by Jay Kadane and others in support of that 

24    conclusion.

25               At that point on I go on to discuss each and 
                                                              878

 1    every one of the guidelines in detail, but I felt it was 

 2    important to lay out for Secretary Mosbacher the underlying 

 3    rationale that was built into and that is interwoven in my 

 4    response to the individual guideline items.  

 5               MS. DESEL:  Your Honor, at this time I would 

 6    offer Plaintiff's Exhibit 193, Dr. Estrada's recommendations 

 7    to Secretary Mosbacher.  

 8               MR. BARON:  No objection.  

 9               THE COURT:  Exhibit 193 is admitted.  

10               (Plaintiff's Exhibit 193 marked for 

11    identification was received in evidence.)

12    BY MS. DESEL:

13         Q.    Dr. Estrada, did you also participate in the 

14    creation of a joint report filed by a group of members of 

15    the special advisory panel?

16         A.    Yes, I did.

17         Q.    And who was in that group?

18         A.    It was Gene Ericksen, Kirk Wolter, John Tukey and 

19    myself.

20         Q.    Did you contribute any portions of that report?

21         A.    I contributed a great deal when we discussed the 

22    original outline and I wrote sections for the discussion 

23    regarding demographic analysis.

24         Q.    I want to turn now to the demographic analysis, 

25    Dr. Estrada.
                                                              879

 1               Could you please tell the court what the 

 2    demographic method of estimating the population is?

 3         A.    Demographic analysis is a method which originates 

 4    in the 1950s which is an independent measure of the total 

 5    population of the United States.  It's based on the use of 

 6    administrative records and its intent is to provide us with 

 7    an independent source of information about the size of the 

 8    population of the United States and also provides for some 

 9    information by age, sex and race.

10         Q.    Who was it that developed this method?

11         A.    Credit is usually given to Answy Coale.

12         Q.    Dr. Estrada, can you now turn to Plaintiff's 

13    Exhibit 188 I believe also in that same binder.

14               Could you tell the court what that document is?

15         A.    It's an article in which Professor Coale of 

16    Princeton University presents the methodology for the 

17    demographic analysis method.  It was published in the 

18    Journal of the American Statistical Association in 1955.

19         Q.    Why did Dr. Coale feel the necessity of 

20    developing this alternative method of population estimation?

21         A.    In the introduction to the article, Professor 

22    Coale notes that censuses are not perfect and that there are 

23    obviously going to be errors of misreporting, double 

24    counting and omissions, and he indicates that there was a 

25    need to understand the degree to which these are, in fact, 
                                                              880

 1    problems and he devices the balancing equation and the 

 2    demographic analysis method to evaluate the performance of 

 3    the 1950 census.

 4         Q.    What is the balancing equation that you just 

 5    referred to?

 6         A.    The balancing equation is one in which the 

 7    differences in population between two points in time, such 

 8    as between two censuses, like 1980 to 1990, is dependent 

 9    upon the number of births that have occurred during that 

10    period of time, which would be added to the population to 

11    that equation, minus deaths that have occurred during that 

12    decade and plus or minus the net migration that would have 

13    occurred in terms of the differences between the people 

14    entering the country and those exiting the country.  

15               So it's though three components, births, deaths 

16    and net migration which allows us to estimate what the 

17    population is and what changes occurred in it over two 

18    periods of time.

19         Q.    When Dr. Coale compared his demographic estimate 

20    with the estimate from the 1950 census, what conclusions did 

21    he reach?  

22               MR. BARON:  Objection; speculation.  

23               THE COURT:  They are in the article?

24         Q.    Based on the article, Dr. Estrada, what 

25    conclusions did Answy Coale reach?  
                                                              881

 1               THE COURT:  Overruled.

 2               Tell me what is in the article.

 3         A.    The article, first of all, establishes at least 

 4    empirically for the first time the degree to which there was 

 5    undercount in the 1950 census.  He goes back and also 

 6    attempts to do it for the 1940 and 30 census.  But the key 

 7    is to establish, first of all, the degree to which there is 

 8    undercount.

 9               Then secondly he establishes a pattern of 

10    differential undercount between racial groups, in his case 

11    he referred to them as white and black, and between sex 

12    groups, males and females.

13               He concludes that the census has an undercount, 

14    that it's much worse for the black population than the white 

15    population and also more severe for males than it is for 

16    females.

17         Q.    Does Dr. Coale draw any conclusions about the 

18    geographic distribution of the undercounted population?

19         A.    His conclusions lead him to the --  well, he 

20    infers from the analysis the conclusion that if you find 

21    areas that have high proportions of nonwhite population, you 

22    will, in fact, locate areas of the highest undercount and 

23    also the people most likely to be omitted in the census.

24         Q.    And who does he find those people are, what kinds 

25    of people most likely to be omitted?
                                                              882

 1         A.    Putting those elements together, it would be 

 2    black males would be at the highest level of undercount and 

 3    then blacks as a whole and males as a whole, but the key is 

 4    to look for the areas of concentration of those populations 

 5    in order to associate with undercount.

 6         Q.    Was Dr. Coale's method adopted by the Census 

 7    Bureau?

 8         A.    Yes, it was.

 9         Q.    Do you know when that happened?

10         A.    I don't know the exact date.  I know that by the 

11    1960 census the use of the method of demographic analysis to 

12    evaluate the census was also in place.  Jay Siegel carried 

13    that analysis out and it has been carried out in all 

14    censuses since that time.

15         Q.    So do we have demographic estimates for censuses 

16    before 1960?

17         A.    Yes.  The work that Answy Coale did in this 

18    article was updated by Jay Siegel and they went back in time 

19    all the way to the 1940s, and so we have demographic 

20    analysis and census comparisons since 1940, 50 years.

21         Q.    Has the method changed at all since Coale's 

22    article in 1955?

23         A.    The balancing equation that I referred to earlier 

24    remains the same.

25               What's happened is that there has been changes in 
                                                              883

 1    the types of --  an improvement in the records, in the 

 2    administrative record that are available in part because 

 3    they are better computerized, they are more up to date, 

 4    there is more of them.

 5               In addition to that, there has been enough work 

 6    done on this method now to also recognize that it has 

 7    serious limitations.

 8         Q.    Dr. Estrada, what are the appropriate and best 

 9    uses for the demographic analysis?

10         A.    The appropriate uses of the demographic analysis 

11    are really, really come from the fact that we have had this 

12    method in place for some time and it serves as a historical 

13    yardstick to analyze the performance and evaluate the 

14    performance of the census enumeration.  Its historical 

15    significance is one of the most important in that we have 

16    evidence that we can draw upon from prior efforts.

17               Second use for which it's very appropriate is in 

18    the study of differentials.

19         Q.    What do you mean by differential?

20         A.    In the different degrees of undercount that occur 

21    between blacks and nonblacks, for example.  The use of this 

22    method allows us to focus on that differential and also to 

23    investigate it over time.

24         Q.    Dr. Estrada, was should the demographic analysis 

25    not be used for?
                                                              884

 1         A.    The least appropriate use of the demographic 

 2    analysis is for specific percentages of undercount for 

 3    specific groups.  That's because there is sufficient 

 4    knowledge today about the uncertainty of the components that 

 5    comprise it that make it suspect to use it in that way.

 6         Q.    Dr. Estrada, has the Census Bureau acknowledged 

 7    these strengths and weaknesses in the demographic estimates?

 8         A.    Yes, it has.

 9         Q.    I would like to turn your attention now to 

10    Plaintiff's Exhibit 197 I believe in the same binder.

11               Can you tell the court what that document is?

12         A.    This is a report by Greg Robinson from the 

13    population division at the Bureau of the Census in which he 

14    discusses the plans for the use of the demographic analysis 

15    to evaluate the 1990 census.

16         Q.    Is the population division the same division you 

17    worked at?

18         A.    Yes, it is.

19         Q.    I would like to direct your attention to page 24 

20    of that article.

21         A.    I have located that page.

22         Q.    Can you tell the court what Greg Robinson says 

23    about the areas of the demographic analysis where we have 

24    least confidence?

25         A.    He points out that given the uncertainty of the 
                                                              885

 1    components, we should avoid --  we should have some degree 

 2    of caution regarding the level of coverage, as he refers to 

 3    it here, meaning the percent undercount in terms of its 

 4    absolute level.

 5         Q.    And what does Robinson say about the areas in 

 6    which the demographic analysis is strong?

 7         A.    He provides here the idea that despite that 

 8    limitation, the demographic analysis is, in fact, very 

 9    reliable in identifying differences, meaning differential, 

10    between groups, such as between blacks and whites, as well 

11    as looking at changes over time.  

12               MS. DESEL:  At this time, your Honor, I offer 

13    Plaintiff's Exhibit 197, the Robinson article on plans for 

14    the 1990 census using demographic analysis.  

15               MR. BARON:  No objection.  

16               THE COURT:  Exhibit 197 is admitted.  

17               MS. DESEL:  I will note for the record that this 

18    document is a part of what defendants have called the 

19    administrative record.  

20               MR. BARON:  That's why we don't object, your 

21    Honor.

22               (Plaintiff's Exhibit 197 marked for 

23    identification was received in evidence.) 

24    BY MS. DESEL:

25         Q.    Dr. Estrada, I would now like to turn your 
                                                              886

 1    attention to Plaintiff's Exhibit 184.

 2               (Pause)

 3               What is this document?

 4         A.    There is a cover memo in the front from Paula 

 5    Schneider, who is chief of the population division, to the 

 6    undercount steering committee to indicate the pages that 

 7    follow are a summary of the demographic analysis evaluation 

 8    projects, what we often refer to as the D projects, and that 

 9    memo is dated June 7, 1991.

10               Beneath that is the substance of this piece, 

11    which is a memo from Greg Robinson and Prithwis Das Gupta. 

12    They prepared this memo for Paula Schneider, and what they 

13    do here is assess the levels of error that are found in the 

14    preliminary demographic analysis.

15         Q.    What is Greg Robinson's position at that 

16    population division?

17         A.    Greg Robinson is chief of the population analysis 

18    and evaluation staff.

19         Q.    Was he at the Census Bureau when you were there 

20    back in 1980?

21         A.    Yes, he was.

22         Q.    I turn your attention to page 6 of that exhibit, 

23    of the Robinson memo.

24               What is the report saying about the areas in 

25    which we have certainty for the demographic analysis?
                                                              887

 1         A.    At the bottom of this paragraph, he is 

 2    reiterating the point just discussed in which he says we 

 3    have a lot more confidence, we should have a lot more 

 4    confidence in assessing changes over time than in the level 

 5    of net coverage, or put another way, we can have more 

 6    confidence in our abilities to understand the patterns and 

 7    differentials than we can in the percent undercount figures.

 8         Q.    Look at the very top of that paragraph, Dr. 

 9    Estrada.  What is Robinson saying about the ability to 

10    pinpoint undercount level using the DA?

11         A.    Perhaps it would be easier if I just read the 

12    first couple of sentences, the first couple of lines.

13               It says, "In sum, despite the inability of the 

14    demographic method to estimate undercount levels with 

15    precision, the demographic coverage estimates allows us to 

16    make statements about differences in patterns of coverage." 

17               That's not the end of the sentence, but I will 

18    stop at that point.

19         Q.    Does he go on to give some examples of the kinds 

20    of patterns he is talking about?

21         A.    Yes.  He says, "Such as: the net undercount of 

22    males is greater than for females, the net undercount of 

23    blacks is greater than for nonblacks, the net undercount is 

24    especially high for certain age groups, for example, black 

25    males of ages 25 to 54 in 1990."
                                                              888

 1         Q.    Dr. Estrada, can you summarize, then, the areas 

 2    where we have confidence in the strength of the demographic 

 3    analysis?

 4         A.    The guidelines that are being presented here tell 

 5    us that we should continue to use the demographic analysis 

 6    as a historical yardstick, as a way of understanding change 

 7    over time and improvements or lack thereof of census 

 8    performance over time.

 9               It also indicates that the emphasis upon 

10    differentials is an appropriate one to make, that we can 

11    look at these differences both in terms of whether they 

12    persist or changes over time as a significant way of 

13    utilizing this method as an appropriate way to use it.

14         Q.    And based on these documents, what can you 

15    conclude about the weaknesses of the demographic analysis?

16         A.    Well, the weakness comes when, when we try to use 

17    it for too fine grain an analysis.  It was intended to be a 

18    gross way of looking and making comparison used to specify a 

19    specific group, such as nonblack males aged 10 to 19, and to 

20    give precision to a number with regard to it is not 

21    appropriate.  There is too much uncertainty to use that kind 

22    of an analysis or to approach it that way.  

23               MS. DESEL:  Your Honor, at this time I would like 

24    to offer Plaintiff's Exhibit 184 into evidence, and I will 

25    note, again, for the record that this is a part of what 
                                                              889

 1    defendants call the administrative record.  

 2               THE COURT:  Therefore, no objection, it is 

 3    admitted.  

 4               MR. BARON:  Correct.  

 5               (Plaintiff's Exhibit 184 marked for 

 6    identification was received in evidence.)

 7    BY MS. DESEL:

 8         Q.    Dr. Estrada, have you read Secretary Mosbacher's 

 9    report?

10         A.    Yes, I have.

11         Q.    Are you familiar with the areas where he uses the 

12    demographic analysis as a basis for rejecting the PES?

13         A.    Yes, I have.

14         Q.    Given the strengths and the limitations that you 

15    were just telling us about, is Secretary Mosbacher's use of 

16    the demographic analysis a proper one?

17         A.    Absolutely not.

18               The guidelines that are being given in these 

19    documents indicate that the proper use us in the patterns 

20    and changes over time.  Scant attention was given by 

21    Secretary Mosbacher to exactly those issues from which we 

22    have the most confidence, and almost all of the discussion 

23    of demographic analysis is focused upon these individual 

24    isolated undercoverage percentages for very specific groups.

25         Q.    In your opinion, does the Secretary make 
                                                              890

 1    sufficient use of the demographic analysis as an assessment 

 2    of the 1990 census?

 3         A.    No, I do not believe that he does.  In fact, his 

 4    approach is mostly to use it as a way of rejecting the 

 5    post-enumeration survey results than it is for its intended 

 6    purpose, which is to evaluate the performance of the 1990 

 7    census and to look for differentials or changes in 

 8    differentials that may have occurred as a result of the 1990 

 9    process.

10         Q.    What does the 1990 demographic analysis tell us 

11    about the census, the 1990 census?

12         A.    The demographic analysis in comparison to the 

13    census indicates to us, first of all, that there is a very 

14    severe undercount approximating five million people and it 

15    also tells us that that undercount, the differentials 

16    between blacks and nonblacks which was originally found in 

17    Answy Coale's article in 1955 and which we have seen 

18    historically over this 50 year period continues to be there 

19    and, in fact, became even worse in the last census.

20         Q.    If Secretary Mosbacher had used the demographic 

21    analysis properly, what would its role have been in 

22    assessing the PES?

23         A.    Well, I think that one of the things that he 

24    would have had to give attention to is the fact that there 

25    is a great deal of consistency in the results of the 
                                                              891

 1    post-enumeration survey and the demographic analysis.

 2               You have to look at this as two very distinct 

 3    methods.  One is based upon administrative records and some 

 4    estimates that are made to improve upon those components, 

 5    the other is based on a followup, follow backup survey 

 6    design, and what we have are two different methods and yet 

 7    they come to very close agreement about the degree of 

 8    undercount in the census.

 9               The differentials that are found in the 

10    demographic analysis methods that historically have been 

11    there are also found in the post-enumeration survey results.  

12    And so attention to the overall patterns, the context of the 

13    results and its historical persistence of things such as 

14    black/white undercount were ignored by the Secretary in his 

15    analysis.

16         Q.    Can you explain to the court why the analysis 

17    that the Secretary does engage in is inappropriate?  Give an 

18    example of that inappropriate analysis.

19         A.    Typical of several pieces of his discussion is to 

20    compare the numerical accuracy and percent undercount that 

21    is found for a particular group such as males, nonblack 

22    males of a particular age and compared to the results of the 

23    PES and say that he finds the differences disturbing.

24               In doing so he completely ignores, first of all, 

25    the uncertainty that exists in the demographic analysis 
                                                              892

 1    which makes one suspect these differences, these numerical 

 2    differences, and perhaps most importantly really ignores the 

 3    fact that you have to look at it in the context of despite 

 4    the fact that the magnitude may be different, does it follow 

 5    a pattern as we expect that this is an age group that has 

 6    typically been found to have high undercount, higher than a 

 7    particular other group.

 8         Q.    Can you give an example of what you are talking 

 9    about in the Secretary's decision?

10         A.    In my mind I am referring to a distinction he 

11    makes between the number of white males aged 10 to 19 

12    indicating that he finds a difference between that and the 

13    PES results.

14               This distinction is used in order to reject the 

15    PES when, in fact, the differences could equally be due to 

16    the uncertainty around the PES estimate --  I mean around 

17    the demographic analysis estimate.

18         Q.    Why is the demographic estimate uncertain as to 

19    that particular group, nonblack males 10 to 19?

20         A.    One of the major components that is in the 

21    demographic analysis is a component that has to be derived 

22    from the undocumented population.

23               Unlike --  the Census Bureau has for some time 

24    been aware that there is a need to include this part or this 

25    population into the component as part of the component of 
                                                              893

 1    the migration, net migration item.

 2               We have no records on this group.  There is no 

 3    birth records and unless they happen to die here there are 

 4    no death records, and so this estimate has to be created 

 5    using indirect information.

 6               There is a number of assumptions that go into the 

 7    development of this and I think most importantly in terms of 

 8    the discussion we were having is that it's concentrated in 

 9    undocumented immigration in youthful ages.  It's a finding 

10    that we know from many other studies that have been 

11    conducted at local areas and different places of the 

12    country.

13               But what happens is we have to come up with a 

14    number, we don't have any direct records, and so we begin 

15    working with a set of information that dates back with 

16    assumptions made about the undocumented in the census in 

17    1980, work our way forward, combine some direct and indirect 

18    ways of trying to assess this to develop a number.

19               There really is no independent proof or evidence 

20    about how accurate that number is, it's the best guess that 

21    the census has at the time.

22         Q.    So in your opinion, is this a valid use of the 

23    demographic analysis that the Secretary engages in?

24         A.    The problem is that when he focuses upon these 

25    particular groups and tries to make those kinds of 
                                                              894

 1    comparison of percent undercount, it appears to him that 

 2    there is a difference, but it could be it's as much an 

 3    artifact of a underestimation of undocumented as anything, 

 4    and I think he has to realistically assess the fact that the 

 5    use of it in this way is inappropriate.

 6         Q.    Well, does he realistically assess that fact in 

 7    the document, in the decision document?  

 8               THE COURT:  What fact?

 9         Q.    The fact that there is an uncertainty in the 

10    undocumented estimate?

11         A.    No.  In fact, he treats the numbers as if they 

12    were precise and gives little attention to the fact that 

13    there is a great deal of impercision about them.

14         Q.    Had the Census Bureau reported on that 

15    uncertainty in the undocumented immigration example that you 

16    just gave?

17         A.    Yes, they have.

18         Q.    Do you know where they made that report?

19         A.    It's actually found in a number of different 

20    places, but the D2 report, which is the demographic 

21    evaluation analysis report number 2 by Karen Woodrow is a 

22    good example, because it details --  it provides a lot of 

23    detail as to the assumptions that have to be utilized and 

24    the manner in which these different kinds of ideas are 

25    pulled together.
                                                              895

 1         Q.    Dr. Estrada, I would like you to turn to 

 2    Plaintiff's Exhibit 173.  I'm not sure it's in the binder in 

 3    front of you, though, it may be on the table.  

 4               THE COURT:  It's not in the binder.  

 5               MS. DESEL:  I believe it's in binder 2.

 6               (Handing to the witness)

 7               (Pause)

 8         A.    I have it before me.

 9         Q.    Dr. Estrada, is that the report you were just 

10    referring to?

11         A.    Yes, it is.  

12               MS. DESEL:  Your Honor, at this time I would like 

13    to offer Plaintiff's Exhibit 173, projects on estimates on 

14    undocumented immigration.  

15               MR. BARON:  No objection.  

16               THE COURT:  173 is admitted.  

17               MS. DESEL:  I will note once again for the record 

18    that this is a part of what defendants have characterized 

19    the administrative record.  

20               (Plaintiff's Exhibit 173 marked for 

21    identification was received in evidence.)

22    BY MS. DESEL:

23         Q.    Dr. Estrada, you have read the Secretary's 

24    opinion in total, is that right?

25         A.    Yes, I have.
                                                              896

 1         Q.    Is there anything in that opinion that has 

 2    changed your view about the use of the post-enumeration 

 3    survey as a supplement to the 1990 original enumeration?

 4         A.    No, there is not.

 5               If I may add, I have been reviewing these 

 6    materials for over two years and I find it still --  I'm 

 7    still astonded by the decision by the Secretary of Commerce 

 8    Mosbacher.  I think it's --  I think the information that I 

 9    have seen just overwhelmingly supports the idea of 

10    adjustment, and I would refer to the joint report that part 

11    of the panel members did jointly, our individual reports and 

12    the evaluation studies which we have had an opportunity to 

13    review.

14               I have seen nothing since that time that would 

15    change my mind or my recommendation to the Secretary of 

16    Commerce.  

17               MS. DESEL:  That's all the questions I have for 

18    Dr. Estrada at this time.  

19               THE COURT:  Thank you.

20               We will take a five minute break.  

21               MR. BARON:  Yes, your Honor.  Thank you.

22               (Recess)           

23               THE COURT:  Mr. Baron.  

24    CROSS-EXAMINATION 

25    BY MR. BARON:
                                                              897

 1         Q.    Dr. Estrada, good afternoon.

 2               You haven't ever published an article on census 

 3    adjustment, isn't that correct?

 4         A.    No, I have not.

 5         Q.    You mentioned John Tukey in your direct 

 6    testimony.

 7               Dr. Estrada, you don't consider yourself to be an 

 8    expert in small area estimation, is that right?

 9         A.    That's correct.

10         Q.    You were retained as an expert witness for 

11    plaintiffs in this case in January of 1992, is that a fact?

12         A.    That's correct.

13         Q.    Let me ask you this:

14               You just testified that the Secretary, in order 

15    to use the demographic analysis properly, should have noted 

16    that the differentials found in the demographic analysis are 

17    also there in the PES, is that right?

18         A.    Among the many things that needed to be done was 

19    to, of course, indicate the context of the differentials, 

20    and among them is to indicate the close agreement between 

21    the PES and demographic results.

22         Q.    So that is a yes?

23         A.    That's a yes.

24         Q.    Are the differential in undercounts between black 

25    males and black females found in the 1990 demographic 
                                                              898

 1    analysis also there in the PES?

 2         A.    The differentials between males and females and 

 3    blacks and nonblacks are also found in the post-enumeration 

 4    survey.

 5               The post-enumeration survey is not limited like 

 6    the demographic analysis to a simple dichotomy of black and 

 7    nonblack, the added feature that comes from the 

 8    post-enumeration survey is you also have a information on 

 9    Hispanic undercount and Asian undercount in the PES, and 

10    that means that you have additional information as well.

11         Q.    Can I ask you again the question that I did ask:

12               Are the differential in undercount between black 

13    males and black females found in the 1990 demographic 

14    analysis also there in the PES?  Yes or no?  

15               THE COURT:  Are they mentioned in both?

16         A.    Are they mentioned? 

17               There are undercount estimates for black males 

18    and black females in the post-enumeration survey, yes.

19         Q.    Do you have a copy of the plaintiffs' exhibit 

20    number 9, which is the Secretary's report up there with you? 

21               If not, I will be happy to give it to you.  

22               MS. DESEL:  I believe it is in volume 1, Dr. 

23    Estrada.

24               (Pause)

25         Q.    I ask you to turn to the page that is 2-12 in the 
                                                              899

 1    Secretary's decision, which is Bates stamped on the bottom 

 2    28, page 28.

 3         A.    I have located that page.

 4         Q.    Do you see the four bullets on the page?

 5         A.    Yes, I do.

 6         Q.    You accept, don't you, that these are 

 7    illustrations where the demographic analysis and the PES are 

 8    pointing in opposite directions, do you not?

 9         A.    This is the point that I have just been trying to 

10    make, is that the use of the demographic analysis method for 

11    this kind of results are inappropriate.

12         Q.    But my question was, are the numbers that are 

13    listed here pointing in the opposite direction?

14         A.    I will respond to your question, but I will 

15    indicate once again that this is not the appropriate use --

16         Q.    I am not asking you for why, just are they and 

17    then we will go on.

18         A.    They are in opposite directions.

19         Q.    Okay.  Thank you.

20               Isn't it true that you take the view, Dr. 

21    Estrada, that the differences that occurred as between the 

22    PES and the DA estimates are attributed in the Secretary's 

23    report to error in the PES, isn't that correct?

24         A.    No, that is not correct.  To the error?

25         Q.    To the error in the PES.  
                                                              900

 1         A.    I think we have to distinguish between different 

 2    kinds of groupings that we may be talking about.

 3               There are some errors in the PES --  there are 

 4    some issues relating to correlation bias in the 

 5    post-enumeration survey, but there is also uncertainty in 

 6    the estimates for the demographic analysis.

 7         Q.    Let me ask you whether you recall the following 

 8    sequence at your deposition that I took in March of this 

 9    year.  There was an answer that you gave on page 47.  

10               MS. DESEL:  Thank you, Mr. Baron.

11         Q.    The part of the question on 47, line 18:

12               "Q.    Go through again what other concerns you 

13    have with respect to the Secretary's decision as to where 

14    the decision fails to use demographic analysis for the 

15    purposes for which it is suited.

16               "A.    --  

17               MS. DESEL:  Wait a minute, Mr. Baron.  Is there 

18    any reason you picked up in the middle of the question?  

19               MR. BARON:  I will read the whole question.

20         Q.    "The questions I posed was, what is it in the 

21    Secretary's decision which fails to use demographic analysis 

22    for the purposes for which it is suited, and you mentioned 

23    your concern about numerical accuracy.  Go through again 

24    what other concerns you have with respect to the Secretary's 

25    decision as to where the decision fails to use demographic 
                                                              901

 1    analysis for the purposes for which it is suited.

 2               "A.    First of all, I think there is an evenness 

 3    in the discussion -- " I believe it is corrected as 

 4    unevenness --  "in the discussion of the DA result as 

 5    vis-a-vis the census.  It could point out the numerical 

 6    differences as he has bulleted here for the DA in censuses 

 7    which are not listed.

 8               In addition, I think that differences that occur 

 9    between the PES and DA are attributed to error in the PES." 

10               Do you recall saying that?  

11               MS. DESEL:  Why don't you read the last sentence 

12    as well.  

13               MR. BARON:  I intend to get to it.

14         Q.    Does that refresh your recollection?  

15               MS. DESEL:  Mr. Baron, are you going to read Dr. 

16    Estrada the last sentence?  

17               MR. BARON:  I will be happy to.

18         Q.    Dr. Estrada goes on to say, "And I am suggesting 

19    that those interpretations could be questioned."  

20               THE COURT:  Now what is the question?

21         Q.    The question is, isn't it true that you took the 

22    view at the time of your deposition that differences that 

23    occur as between the PES and DA estimates are attributed in 

24    the Secretary's report to error in the PES?

25         A.    I'm trying to recollect the context of this 
                                                              902

 1    discussion, because we covered a lot of ground in the 

 2    deposition.

 3               I recognize the problem of correlation bias and 

 4    this is related to that section.  It could be that that was 

 5    my intent.

 6               But in general I find the uncertainty around the 

 7    demographic analysis estimates to be a greater problem than 

 8    any of those for the PES.

 9         Q.    You did say in the sentence I just read, you 

10    said, "And I am suggesting that those interpretations could 

11    be questioned," and that is still your view, isn't that 

12    right?

13         A.    I certainly thing that the interpretations can be 

14    questioned, but I'd have to understand the context better, 

15    because in my view is that the DA has more uncertainty 

16    around it than do the estimates of the PES.

17         Q.    I don't want to linger long over this, but at the 

18    time of your deposition --  

19               THE COURT:  Longer would be more accurate.  

20               MR. BARON:  Yes, your Honor.

21         Q.    At the time of your deposition, Dr. Estrada, you 

22    had conducted no research or analysis yourself to determine 

23    what the correct interpretation of these differences would 

24    be, isn't that correct?

25         A.    I guess you are asking if there was additional 
                                                              903

 1    work.

 2               I'd like to keep --  make it clear that the 

 3    information about the demographic analysis was available to 

 4    us from the moment that the first press release on that 

 5    topic became available.  The differentials were presented 

 6    for us, it wasn't necessary to do additional work just to 

 7    obtain those gross numbers.

 8               It's at that level that you can understand the 

 9    use of the demographic analysis and I utilized the 

10    information that both the census had made available to us 

11    regarding that as well as the historical information that is 

12    available in Census Bureau reports going back to the 1940s.

13         Q.    Well, just to make this point clear and this is 

14    the last point in this series, between the time you were 

15    retained as an expert in the litigation and the time that I 

16    took your deposition, you conducted some independent 

17    research or analysis at all in adjusting the 1990 census, is 

18    that correct?

19         A.    The question, as I recall, I did in my deposition 

20    note that I had over the summer after --

21         Q.    No.  Excuse me, Dr. Estrada.  I asked the 

22    question between the time you were retained as an expert in 

23    the litigation in January of 1992 and the time that you and 

24    I were at the deposition together.  

25               THE COURT:  In March.
                                                              904

 1         Q.    In March.

 2         A.    Yes, but I don't want you to ignore the almost 

 3    two years of work that I had done on the panel as a panel 

 4    member.

 5         Q.    Dr. Estrada, you stated in your recommendation 

 6    that you reviewed the P projects before you submitted your 

 7    recommendations to the Secretary, correct?

 8         A.    Could you repeat that?

 9         Q.    You stated in your recommendation that you 

10    reviewed the P projects before you submitted your 

11    recommendation to the Secretary?

12         A.    Yes, I did.

13         Q.    It's the case, is it not, that demographic 

14    analysis as used by the Census Bureau is limited to 

15    providing only national estimates of undercount; isn't that 

16    right?

17         A.    Yes, it is.

18         Q.    So as used by the Census Bureau, one cannot use 

19    demographic analysis to come up with accurate estimates of 

20    subnational areas, such as states, is that correct?

21         A.    If we had information about migration, we might 

22    be able to do that, but we do not have that in the United 

23    States and so at this time that is a limitation, it has to 

24    be used only at the national level.

25         Q.    And demographic analysis doesn't provide direct 
                                                              905

 1    estimates of the Hispanic population of the United States 

 2    even at the national level, correct?

 3         A.    That is correct.

 4         Q.    You did cite national level estimates from 

 5    demographic analysis in your recommendation to the 

 6    Secretary, did you not?

 7         A.    Yes, I did.

 8         Q.    Dr. Estrada, referring you to your individual 

 9    recommendation that I believe you have before you in 

10    Plaintiff's Exhibit 193 at page 2 --

11         A.    I located that page.

12         Q.    You see the points labeled or the subsections A, 

13    B, C, D on page 2?

14         A.    Yes, I do.

15         Q.    Each of the descriptions of percent in A, B, C, D 

16    in your recommendation represents a point estimate for an 

17    undercount or a differential in undercount, isn't that 

18    correct?

19         A.    Yes, it does for the aggregate population.

20         Q.    At the national level, the point estimates you 

21    give in points A, B, C, D would not be characterized as 

22    biased as that term is used in statistics, is that correct?

23         A.    Well, there is uncertainty around this number as 

24    well, but the fact that we are working with the total 

25    aggregate gives us a little bit more confidence in the 
                                                              906

 1    stability of it.

 2         Q.    I wasn't speaking as to uncertainty, I was 

 3    speaking as to bias.

 4               Would you characterize any of these estimates as 

 5    biased?  You wouldn't characterized them as biased?

 6         A.    No, I would not.

 7         Q.    Thank you.

 8               But you say there is uncertainty in the estimates 

 9    for the elements of demographic analysis that are contained 

10    in A, B, C, D here, correct?

11         A.    That is correct.

12         Q.    You didn't mention the uncertainties in the point 

13    estimates in A, B, C, D in your recommendations to the 

14    Secretary, correct?

15         A.    No, I did not in this recomendation.

16               I would like to point out that that is in our 

17    joint report.

18         Q.    Excuse me?

19         A.    I believe that we discussed this in two places in 

20    our joint report.  The first is the addendum that we have 

21    from Jeff Passell that speaks to the issues of uncertainty 

22    and the second is in the discussion that we have in the 

23    joint report together.

24         Q.    But the point --  you didn't point out 

25    uncertainties in the point estimates here?
                                                              907

 1         A.    No, I did not.

 2         Q.    Thank you.

 3               You mentioned a major component of demographic 

 4    analysis as births, deaths and net migrations, correct?

 5         A.    Correct.

 6         Q.    You regard one of those components, net 

 7    migration, as undocumented immigration, correct?

 8         A.    Yes.

 9         Q.    And don't you believe, Dr. Estrada, that the 

10    estimates for undocumented immigration are somewhat fuzzy?

11         A.    Yes, I do.

12         Q.    In an attempt to quantify the uncertainty in the 

13    demographic estimates, the Census Bureau developed ranges of 

14    uncertainty around each of its demographic point estimates 

15    for various subcomponents of the population, did they not?

16         A.    Yes, they did.

17         Q.    And the 1990 decennial census program was the 

18    first time this had been done in a decennial census, 

19    correct?

20         A.    Yes, it's the first time.

21         Q.    And the length of the uncertainty intervals 

22    around the demographic point estimates is important 

23    information to consider in connection with the issue of 

24    adjustment, isn't that correct?

25         A.    Its information that can be taken into account.  
                                                              908

 1    I believe you can draw conclusions whether you utilize that 

 2    information or not.

 3         Q.    Is it important information?

 4         A.    It's hard to assess because it's the first time 

 5    it's ever been done.  We don't have a way of understanding 

 6    its relative goodness or badness as occurs with information 

 7    over time.  It's something that we are looking at for the 

 8    first time and so it is difficult to assess in a very 

 9    straightforward way.

10         Q.    Would you say that the issue of the degree of 

11    skewing in the error intervals or uncertainty intervals is 

12    an important issue?

13         A.    It is an issue for which considerable attention 

14    was given.

15         Q.    And not just the total length of the uncertainty 

16    intervals, but how much uncertainty on the low side of a 

17    point estimate versus uncertainty on the high side of a 

18    point estimate, isn't that also an important issue to 

19    consider?

20         A.    It was, once again, a matter of great discussion.

21         Q.    Between the time of your attendence as an expert 

22    in the case and my deposition, you didn't do research on 

23    your own concerning the length or the balance of uncertainty 

24    in the uncertainty intervals themselves, did you?

25         A.    No, I did not.
                                                              909

 1         Q.    The Secretary uses the term judgmental 95 percent 

 2    confidence interval to refer to these ranges of uncertainty, 

 3    didn't he?

 4         A.    Yes, he did.

 5    

 6    

 7    

 8               (Continued on the next page) 

 9    

10    

11    

12    

13    

14    

15    

16    

17    

18    

19    

20    

21    

22    

23    

24    

25    
                                                              910

 1         Q.    A judgmental 95 percent confidence interval is 
                                                                

 2    not to be understood as a confidence interval as that term 

 3    is typically used in statistical analysis; isn't that right?

 4         A.    No, it is not.

 5         Q.    That is not right?

 6         A.    I guess --

 7         Q.    Or is it that it's different?

 8         A.    Yeah, it's different than --

 9         Q.    Thank you.

10         A.    -- the statistical concept of confidence 

11    intervals.

12         Q.    The basis for developing the judgmental 95 

13    percent confidence intervals used in demographic analyses 

14    was derived by a consensus among the Census Bureau staff; 

15    isn't that correct?

16         A.    Yes, that's correct.

17         Q.    In fact, didn't the Bureau use a so-called Delphi 

18    method for developing this consensus, Dr. Estrada?  

19               THE COURT:  Delphi?  

20               MR. BARON:  Delphi as in Delphic oracle.  

21               THE COURT:  Just go with the oracle.

22         Q.    Isn't that correct?

23         A.    My understanding of the methodology used is that 

24    typically there was a lead person who made a presentation to 

25    his colleagues, and his colleagues sort of discussed it, and 
                                                              911

 1    together they tried to arrive at a consensus for these 

 2    judgmental confidence intervals.

 3         Q.    Demographic analysis produces estimates of the 

 4    ratio of males to females by age and race group; correct?

 5         A.    That is correct.

 6         Q.    And these ratios are called sex ratios; correct?

 7         A.    That's correct.

 8         Q.    And these sex ratios were used in the bureaus P 

 9    projects, especially P-17; correct?

10         A.    That is correct.

11         Q.    Was it your view at the time that I took your 

12    deposition that the information in P-17 had always been 

13    problematic to you?

14         A.    The P-17 figures have been problematic to me, 

15    yes.

16         Q.    This is still your view?

17         A.    It's still my view.

18         Q.    As a member of the Special Advisory Panel, you 

19    are well aware of the extraordinarily technical issues 

20    involved in a question of adjusting the 1990 census, isn't 

21    that correct, Dr. Estrada?

22         A.    That is correct.

23               MR. BARON:  I have no further questions.  

24               THE COURT:  Thank you.  

25               THE COURT: Ms. Desel, do you have any redirect 
                                                              912

 1               MS. DESEL:  I don't, your Honor.  

 2               THE COURT:  Thank you very much.  

 3               Mr. Estrada, it's been a pleasure.  

 4               THE WITNESS:  Thank you  

 5               (The witness was excused) 

 6               MR. HEINZ:  If it's all right with you, we'd like 

 7    to declare an early weekend.  

 8               THE COURT:  No let's work until six.  

 9               MR. MILLET:  Your Honor, at the risk of perhaps 

10    ending the week on less than a cordial note, we still have 

11    not received from the plaintiffs all the exhibits for their 

12    witnesses.  

13               They were to promised to us earlier this week.  I 

14    raised this question our first day of trial.  It's five days 

15    later.  I, frankly, can't understand why with all the 

16    resources and lawyers we've seen on that side of the 

17    courtroom they can't provide us with their exhibits.  For 

18    example, one of their witnesses, Dr. Bailar, was supposed to 

19    have testified this week as their third witness, and we 

20    still do not have all of their exhibits.  

21               THE COURT:  Who is going to respond to that 

22               MR. RIFKIND:  My understanding is they either now 

23    have or will have within the next 15 minutes all of the 

24    exhibits for all of the witnesses they're going to have next 

25    week with the exception of one, and it is one whose 
                                                              913

 1    scheduling we're having a little problem with because he has 

 2    been summoned to another hearing when we thought he was 

 3    committed to being here.  

 4               We will straighten that out in the course of the 

 5    evening and advise them about that witness as fast as we 

 6    can.  

 7               MR. MILLET:  Your Honor, I still see no reason 

 8    why those documents cannot be provided to us.  

 9               THE COURT:  He just said he's going to give them 

10    to you in 15 minutes.

11               MR. RIFKIND:  With the exception of one witness.

12               THE COURT:  That's the one you're referring to?  

13               MR. MILLET:  Yes, your Honor.  

14               MR. RIFKIND:  As to that witness they have been 

15    told orally some days ago which exhibits are peculiar or 

16    particular to that witness, and we will get them those 

17    exhibits just as soon as -- 

18               MR. MILLET:  If we can have an enumeration of 

19    those exhibits without even a separate copy, if he can just 

20    give us a list of the exhibits by number, that will be 

21    enough.  

22               MR. RIFKIND:  I will try to procure that before 

23    the end of the day.  

24               MR. MILLET:  Thank you very much.  

25               THE COURT:  Thank you.  Have a good weekend.  
                                                              914

 1    We'll see you Monday at 9:30.   

 2               (Adjourned to Monday, May 18, 1992 at 9:30)

 3     

 4    

 5

 6

 7

 8

 9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25
